A Document of The World Bank Report No: ICR00003649 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-97009) ON A GRANT IN THE AMOUNT OF US$22.0 MILLION TO THE REPUBLIC OF HAITI FOR AN EDUCATION FOR ALL FAST-TRACK INITIATIVE CATALYTIC FUND PROJECT May 31, 2016 Education Global Practice Latin America and Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective April 1, 2016) Currency Unit = Haitian Gourde (HTG) HTG 1.00 = US$0.01619 US$1.00 = HTG 61.763 FISCAL YEAR October 1 – September 30 ABBREVIATIONS AND ACRONYMS APNRD Action Plan for National Recovery and Development of Haiti—Immediate Key Initiatives for the Future CF Catalytic Fund (Fond catalytique) DDE Département régional d'éducation (Regional Education Department) ECD Early Childhood Development EFA Education for All EFA-1 First Education for All EFA-2 Second Education for All EFA-FTI Education for All Fast-Track Initiative FM Financial Management FTI Fast-Track Initiative GA Grant Agreement GoH Government of Haiti GPE Global Partnership for Education IMOA-EPT Initiative de Mise en Oeuvre Acceleree-Education pour Tous (Accelerated Implementation Initiative – Education for All) ISR Implementation Status Report LEG Local Education Group M&E Monitoring and Evaluation MDTF Multi-Donor Trust Fund MENFP Ministère de l’Education Nationale et de la Formation Professionnelle (National Ministry of Education and Vocational Training) MTR Mid-Term Review NGO Non-Governmental Organization NPV Net Present Value PCU Project Coordination Unit PNCS Programme Nationale de Cantine Scolaire (National School Feeding Program) PWP Policy Working Paper RVP Regional Vice President SE Supervising Entity SNA/EPT Strategie Nationale d’Education pour Tous (National Education for All Strategy) TF Transition Fund TFP Trust Fund Proposal TWP Tuition Waiver Program UNESCO United Nations Educational, Scientific, and Cultural Organization UNICEF United Nations Children’s Fund Senior Global Practice Director: Claudia Maria Costin Practice Manager: Reema Nayar Project Team Leader: Axelle Latortue / Juan Baron ICR Team Leader: Axelle Latortue HAITI Education for All Fast-Track Initiative Catalytic Fund Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ........................................................... 1 2. Key Factors Affecting Implementation and Outcomes ........................................................... 5 3. Assessment of Outcomes ....................................................................................................... 11 4. Assessment of Risk to Development Outcome ..................................................................... 19 5. Assessment of Bank and Borrower Performance .................................................................. 19 6. Lessons Learned .................................................................................................................... 21 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors ........................... 22 Annex 1. Project Costs and Financing .......................................................................................... 23 Annex 2. Outputs by Component.................................................................................................. 24 Annex 3. Economic and Financial Analysis ................................................................................. 25 Annex 4. Grant Preparation and Implementation Support/Supervision Processes ....................... 28 Annex 5. Beneficiary Survey Results ........................................................................................... 29 Annex 6. Stakeholder Workshop Report and Results................................................................... 30 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR ........................................ 31 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................................... 48 Annex 9. List of Supporting Documents ...................................................................................... 49 Annex 10. Origins of the Haiti EFA-FTI CF Project.................................................................... 50 Annex 11: Personal Testimonies of Gratitude by Beneficiaries of the TWP ............................... 52 MAP .............................................................................................................................................. 53 A. Basic Information Haiti Education for All Country: Haiti Project Name: Fast-Track Initiative Catalytic Fund Project ID: P114174 L/C/TF Number(s): TF-97009 ICR Date: 10/25/2015 ICR Type: Core ICR Lending Instrument: SIL Grantee: REPUBLIC OF HAITI Original Total US$22.00 million Disbursed Amount: US$21.99 million Commitment: Revised Amount: US$22.00 million Environmental Category: C Implementing Agencies: Ministry of National Education and Vocational Training Cofinanciers and Other External Partners: None B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 12/11/2009 Effectiveness: 07/15/2010 Appraisal: 12/11/2009 Restructuring(s): Approval: 05/19/2010 Midterm Review: 12/02/2013 06/03/2013 Closing: 10/31/2015 10/31/2015 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Substantial Bank Performance: Moderately Unsatisfactory Grantee Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Moderately Satisfactory Unsatisfactory Moderately Implementing Moderately Quality of Supervision: Unsatisfactory Agency/Agencies: Unsatisfactory Overall Bank Moderately Overall Borrower Moderately Satisfactory Performance: Unsatisfactory Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments (if Indicators Rating Performance any) Potential Problem Project Quality at Entry Yes None at any time (Yes/No): (QEA): Problem Project at any Quality of Supervision Yes None time (Yes/No): (QSA): DO rating before Moderately Closing/Inactive status: Unsatisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Health 14 20 Primary education 78 78 Public administration - Education 8 2 Theme Code (as % of total Bank financing) Child health 9 20 Education for all 82 78 Other human development 9 2 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Pamela Cox Country Director: Mary Barton-Dock (Special Envoy) Yvonne M. Tsikata Practice Manager/Manager: Reema Nayar Chingboon Lee Project Team Leader: Axelle Latortue/Juan Baron Peter Holland/Michael Drabble ICR Team Leader: Axelle Latortue — ICR Primary Author: Richard Carroll — F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) This operation did not have an approved project appraisal document (PAD). The PDO is from the Multi- donor Trust Fund (MDTF) Grant Agreement (GA): “To assist the Recipient in implementing its National Education for All Strategy (SNA/EPT) through the improvement of: (a) access to primary education for poor children of age 6-12; and (b) equity, quality and governance in the Recipient’s education sector.” ii Note: Initially, this operation did not have a Results Framework. The PDO indicators/values are from the final archived Implementation Status Report (ISR). For the purposes of evaluation, the ICR analyzes indicators added at the ICR stage (Section 3). Revised Project Development Objectives (as approved by original approving authority) No revision. (a) PDO Indicator(s) Original Target Actual Value Values (from Formally Revised Achieved at Indicator Baseline Value approval Target Values Completion or Target documents) Years Indicator 1: Schools participating in the teacher compensation program (Number, Custom) Value (Quantitative or n.a. n.a. 3,000 2,824 Qualitative) Date achieved — — 09/30/2011 06/22/2014 Comments Target substantially achieved. This is not a ‘formal’ revision. Borrower ICR quotes a target (including % of 3,600. The target reported in the ISR is a function of the amount of funding divided by achievement) the per school amount of the grants. Number of children participating in the school health and nutrition program (Number, Indicator 2: Custom) Value (Quantitative or n.a. n.a. 50,000 62,000 Qualitative) Date achieved — — 09/30/2011 06/22/2014 Comments Target exceeded. The actual achievement of 62,000 was achieved over two years (17,000 (including % for 2010/11 and 45,000 for 2011/12.). This is not a ‘formal’ revision. achievement) Indicator 3: Children benefiting from primary education enrollment subsidies (Number, Custom) Value (Quantitative or n.a. n.a. 83,000 93,624 Qualitative) Date achieved — — 09/30/2011 11/30/2015 Target exceeded. The target simply reflected the number of students who could be funded Comments given the estimated reallocation (US$7.5 million) from the teacher compensation (including % component (divided by US$90 per student equals 83,000 student-years). Other released achievement) funds covered an additional 20,833 students (2014/15), and the target was not revised in ISRs. This is not a ‘formal’ revision. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Achieved Values (from Formally Revised Indicator Baseline Value at Completion or Target approval Target Values Years documents) Contract signed with United Nations Educational, Scientific, and Cultural Organization Indicator 1: (UNESCO) for curricular and pedagogical support to the Ministère de l’Education iii Original Target Actual Value Achieved Values (from Formally Revised Indicator Baseline Value at Completion or Target approval Target Values Years documents) Nationale et de la Formation Professionnelle (National Ministry of Education and Vocational Training, MENFP) (Yes/No, custom) Value (Quantitative or n.a. n.a. Y Y Qualitative) Date achieved — — 09/30/2011 06/22/2014 Comments Target achieved. Contract was signed in September 2013 between the MENFP and (including % UNESCO. This is not a ‘formal’ revision. achievement) Indicator 2: Implementation of the early childhood development (ECD) action plan (percentage, custom) Value (Quantitative or 0 n.a. 100 40 Qualitative) Date achieved 03/11/2010 — 10/31/2015 06/22/2014 Comments Target not achieved. The action plan was developed, but most has yet to be implemented. (including % This is not a ‘formal’ revision achievement) Indicator 3: Implementation of institutional strengthening action plan (percentage, custom) Value (Quantitative or 0 n.a. 100 80 Qualitative) Date achieved 03/11/2010 — 10/31/2015 06/22/2014 Comments (including % Target not achieved. This is not a ‘formal’ revision. achievement) G. Ratings of Project Performance in ISRs Date ISR Actual Disbursements No. DO IP Archived (US$, millions) 1 02/23/2011 Satisfactory Moderately Satisfactory 11.78 2 08/08/2011 Satisfactory Moderately Satisfactory 13.54 3 02/21/2012 Satisfactory Moderately Satisfactory 8.93 4 09/03/2012 Satisfactory Moderately Satisfactory 19.46 5 04/08/2013 Moderately Satisfactory Moderately Satisfactory 19.74 6 11/30/2013 Moderately Unsatisfactory Moderately Satisfactory 20.54 7 07/06/2014 Moderately Unsatisfactory Moderately Satisfactory 20.54 8 01/05/2015 Moderately Unsatisfactory Moderately Satisfactory 20.54 9 08/18/2015 Moderately Unsatisfactory Moderately Satisfactory 20.54 Note: The reason for the decrease in cumulative disbursements for ISR no. 3 was the refund of unused, direct payment funds (US$4,675,000 - January 5, 2012), from the teacher compensation component of the Project. An initial amount of US$11,735,000 had been disbursed to have funds quickly available after the earthquake. The unused funds (US$4,675,000) were later used for the Tuition Waiver Program. iv H. Restructuring (if any) Not applicable. I. Disbursement Profile v 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. At the time of the initial appraisal of the Education for All Fast-Track Initiative Catalytic Fund Project (the EFA-FTI CF Project, or the Project), Haiti was classified as a ‘fragile state’ with most Haitians living on less than US$2 per day. Haiti ranked 146th out of 177 countries in the United Nations 2007/08 Human Development Index. The high poverty rate, combined with the fact that 81 percent of Haiti’s primary enrolled students were in private schools that required school fees, contributed to the fact that more than 400,000 children were not attending school (CF survey 2008). Up to 30 percent of Haitian children under the age of 5 were malnourished (UNICEF 2010) leading to diminished cognitive abilities. Haitian education infrastructure had also been recently damaged by cyclones, including in particular four successive cyclones in 2008. 2. After the original project was prepared and appraised, the country conditions became far more desperate when Haiti was struck by a massive earthquake (7.3 on the Richter scale) on January 12, 2010, which resulted in hundreds of thousands of fatalities and widespread destruction around the capital, Port-au-Prince, and other cities in 3 of the 10 departments (Nippes, Sud-Est, and Ouest). Estimates of the total number of deaths range from 220,000 to more than 300,000 people (3.5 percent of the population), with another 1.3 million displaced (16 percent of the population). In the education sector in particular, an estimated 38,000 students were killed, and 4,228 schools (representing 85 percent of the schools in the affected areas) and other educational buildings were destroyed (UNICEF). Up to 2.9 million children experienced an interruption to their studies or continued to lack access to basic education because of school closings (UNICEF). The total damage was estimated at US$7.9 billion (120 percent of gross domestic product). 1 In the aftermath of this disaster, the Government of Haiti (GoH) established in March 2010 an interim action plan focused on restoring services in the country, including access to education (Action Plan for National Recovery and Development of Haiti—Immediate Key Initiatives for the Future - APNRD). The already prepared, but not yet approved, project had to be redesigned, in line with the APNRD, at the request of the GoH’s Ministère de l’Education Nationale et de la Formation Professionnelle (National Ministry of Education and Vocational Training, MENFP), to address the new emergency situation (table 1 and section 1.5). 3. Rationale for Bank involvement. The World Bank had been supporting the Education for All Adaptable Program Grant-1 (EFA APG-1) Project that was dealing with pressing needs in the education sector. This project was implementing similar activities as those planned for the EFA- FTI CF Project (in particular, school enrollment and school nutrition and health activities) and had been successful as indicated by exceeding its targets for tuition waiver and school health and nutrition beneficiaries. The GoH’s request for an expansion of these activities was also supported by the sector diagnostic, which was carried out in connection with the national strategy for the education sector—Strategie Nationale d’Education pour Tous (National Education for All 1 Data from Action Plan for National Recovery and Development of Haiti—Immediate Key Initiatives for the Future, March 2010. 1 Strategy, SNA/EPT—July 2009 Project Appraisal Document [PAD], p. 11). 2 The Project was to be financed by the CF (renamed the Global Partnership for Education [GPE] in 2011), and as per CF requirements, its preparation and any revisions would have followed a consultative process requiring approvals by the Local Education Group (LEG) in Haiti as well as the CF. The Bank was selected as the Supervising Entity (SE) for the Project, and therefore, the Project was required to follow Bank procedures, and the Bank assumed full fiduciary oversight and implementation support for the Project. The United Nations Educational, Scientific, and Cultural Organization (UNESCO) was the coordinating agency for the Project. As such, UNESCO served as the primary communication link among the MENFP, LEG, and the GPE. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 4. This Project did not have a PAD that was formally reviewed by the Bank. 3 The Project was governed by the Haiti-Education for All (EFA) Multi-donor Trust Fund (MDTF) Grant Agreement (GA), June 10, 2010, through which the EFA-FTI CF Project was funded. The stated objective in the GA for the MDTF was: “To assist the Recipient in implementing its National Education for All Strategy (SNA/EPT) through the improvement of: (a) access to primary education for poor children of age 6-12; and (b) equity, quality and governance in the Recipient’s education sector.” 5. This GA objective intended to cover additional activities that were to be funded through the MDTF, 4 including school reconstruction and teacher training, which were to contribute to ‘quality’ and ‘governance’ objectives. These activities were to be funded with resources from other donors, but these funds never materialized and, thus, quality and governance objectives could not be as strongly pursued. For the purpose of PDO assessment, this ICR focuses on the major intent of the CF project, improved access, and equity 5 in line with the post-earthquake APDRD, but recognizes that three relatively small subcomponents (less than 15 percent of total planned project costs) also aimed at improved education quality and governance, 6 and should also be assessed proportionately to their importance in the Project as per OPCS guidelines (section on achievement of project/program development objectives). The Project did not have a Results Framework 2 Note that this PAD reflected the pre-earthquake preparation for the CF project, but was not formally approved by the Bank. This PAD is used only for informational purposes by this ICR and not as an official representation of the CF project. 3 Annex 10 of this ICR provides a description of the preparation origins of the Project. 4 The stated objective in the trust fund proposal (TFP) was: “To support the effective implementation of the Government of Haiti’s (GoH) “Strategie Nationale d’Education pour Tous (National Education for All Strategy, SNA-EPT)”National Education for All Strategic Plan - SNA/EPT).” Specifically, the HEFA MDTF will improve access to primary education for poor children of age 6–12 years in a sustainable manner, while improving equity, quality, and governance of the education sector. This PDO more closely matches the GA PDO because it refers to the entire MDTF, rather than just the EFA-FTI CF Project. 5 Following the earthquake, the GoH prepared an interim strategy, (APNRD, March 2010), which included restoring education services in the country. The strategy focused on short-term (six months) and medium-term (two years) horizons. The cornerstones of this interim strategy were the Tuition Waiver Program (TWP) and school feeding programs to maintain and restore school enrollment. In the immediate term, the GoH sought to help as many children as possible to return to school, that is, access. 6 The three subcomponents are (a) Early Childhood Development (ECD) strategy (developed but not implemented) and services for pre-school children (also not implemented) (US$1 million); (b) development of pedagogical tools for a post-disaster situation (US$1 million)—to be implemented by UNESCO; and (c) institutional strengthening for MENFP (US$1 million). 2 initially, but indicators were selected during implementation once project activities were clearly defined post-earthquake by the MENFP. 7 These PDO indicators were essentially output indicators that related to access to education. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 6. There was no formal revision to the PDO. The PDO indicators and targets that were established during the first supervision mission were (a) schools participating in the teacher compensation program (3,000); (b) number of children participating in school health and nutrition program (50,000); and (c) children benefiting from primary education enrollment subsidies (83,000). The indicator on children benefiting from primary education enrollment subsidies was added in 2011 to reflect the reallocation of funds to the TWP. 7. The intermediate indicators were (a) contract signed with UNESCO for curricular and pedagogical support to the MENFP; (b) Implementation of the ECD action plan; and (c) implementation of institutional strengthening action plan (for the MENFP). 1.4 Main Beneficiaries 8. There were five main groups of beneficiaries: (a) primary school students (and, by extension, their parents) who have their school fees paid for them; (b) primary school students who participate in the school feeding program; (c) teachers who are paid from the teacher compensation, or school grant, program (Kout Pous Pou Tounen Lekol—Giving a push to return to school, or Kout Pous); (d) the primary schools and their students who receive extra funding through the teacher compensation program and, later in the Project, payment of tuition waivers; and (e) MENFP staff and clients through the Project’s support for the restoration of MENFP services by providing equipment and training. 1.5 Original Components (as approved) 9. Table 1 shows the original components from the Note Technique of March 2010 and the revisions documented in the second Note Technique of September 2011, 8 both submitted by the 7 The original project design was documented in the “Initiative de Mise en Oeuvre Acceleree-Education pour Tous (Accelerated Implementation Initiative – Education for All, IMOA-EPT) Financement du Fonds Catalytique pour Haiti—Fonds Fiduciaire Multi-bailleurs (FFMB)—Proposition de réallocation—11 Mars 2010” (Accelerated Implementation Initiative-Education for All Financing by the Haiti Catalytic Fund-MDTF). The ICR refers to this document as the ‘Note Technique’ or ‘Technical Note.’ It was submitted by the MENFP and was presented to the Fast-Track Initiative (FTI) Secretariat March 18, 2010; see memo from Robert Prouty - Head of FTI Secretariat to the FTI CF Strategy Committee for no objection by March 22, 2010. The Bank acknowledged the approval April 9, 2010; see memo from Elizabeth King - Director, Education, Human Development Network. The memo formally acknowledged the Bank’s role: “In our role as Supervising Entity, we will ensure that the allocated resources reach their intended beneficiaries. Similar to the original proposal submitted to the CF Committee, the mechanisms which will be in place will include a balance of Government inspection structures and independent (third-party) verification.” 8 Following the emergency period during which the Project implemented fast-disbursing activities on the basis of the March 2010 Note Technique, a second technical note (Note Technique of September 2011) was developed and approved by the LEG and CF, outlining revised activities, within the scope of the GA, to be implemented. 3 MENFP. Column 1 also includes the reference to the GA Schedule 1 section. The GA Schedule 1, Parts I (b) and (c) and Part II were not part of the Project. Table 1. Project Activities Originally Approved and Revised Post-Earthquake Proposal Approved in March 2010 Revised Activities - September 2011 Component 1: Support for the restarting of educational services Component 1: Improvement of access and in disaster-affected areas - US$16 million equity for primary education - US$16 million Component 1.A: Support to nonpublic schools - US$15 million Component 1.A : (a) Support to nonpublic schools - US$7.5 million Technical Note: Transfer through the EFA Technical Unit to support the Directorate of Administration Activity not revised Providing compensation to teachers who will be assigned to (b) Tuition waiver for students - US$7.5 temporary schools covering the three cycles of education. The million (TWP) provision of compensation for teachers will be a temporary The support of the CF is expected to measure. Target areas will be within the three departments affected complement the funding already obtained for by the earthquake: West, South-East, and Nippes. free access to education for the benefit of GA Part I (a) 235,000 children for the 2011/12 academic year in the eight departments targeted by the EFA Project, namely, Grande Anse, West, Artibonite, North, North West, North East, Central, and Nippes. Component 1.B: Establishment of early childhood development Component 1.B: Establishment of early services (pre-school) - US$1 million childhood development services (pre- school) - US$1 million Technical Note: This subcomponent will finance early childhood centers, mainly pre-school activities and nutrition and health Not revised. interventions. GA Part I (d). Component 2: Improved school performance in primary Component 2: Improved school education - US$5 million performance in primary education - US$5 million Component 2.A: School Nutrition and Health Program - US$4 Component 2.A: School Nutrition and million Health Program - US$4 million Technical Note: Implementing Agency: Programme Nationale de Not revised. Cantine Scolaire (National School Feeding Program, PNCS), with management of the EFA Technical Unit On the basis of existing contracts with nongovernmental organizations (NGOs) on school feeding under the EFA project financing, the PNCS will increase the number of beneficiaries of school canteens, provision of Vitamin A, and de-worming. GA Part I (a) (ii) Component 2.B: Support for the development of educational Component 2.B: Support for the tools for teaching in post-disaster situations - US$1 million development of educational tools for teaching in post-disaster situations - US$1 Technical Note: Implementing Agency: UNESCO. million In-service education of teachers and the establishment of an Technical Note: Implementing Agency: academic program tailored to the natural post-disaster situation UNESCO GA Part I (d) (ii) (iv) 4 Post-Earthquake Proposal Approved in March 2010 Revised Activities - September 2011 Not revised. Component 3: Rebuilding the capacity of the Ministry of Component 3: Rebuilding the capacity of Education - US$1 million the Ministry of Education - US$1 million Technical Note: Implementing Agency: EFA Technical Unit Technical Note: Implementing Agency: EFA Technical Unit Will allow emergency equipment purchases (vehicles, office equipment, and generator), as well as technical assistance. This Not revised. will also involve the decentralized structures of the ministry. These funds will also allow the financing of the EFA Technical Unit, which supports the MENFP in the implementation of the Project. Part I (d) Source: Initiative de Mise en Oeuvre Acceleree (IMOA)-EPT Financement du Fonds Catalytique pour Haiti-Fonds Fiduciaire Multi-bailleurs (FFMB) Proposition de réallocation—Septembre 2011. 1.6 Revised Components 10. The Project design as presented in the March 2010 Note Technique was revised and defined in the September 2011 Note Technique to reflect the need to reallocate project funds (as shown in table 1). The main change was the reallocation from the temporary, one-year, teacher compensation program to the TWP. Only about half of the teacher compensation component funds were used owing to an overestimate of eligible schools that would have applied for the grants. The remaining funds for that activity, approximately US$7.8 million, were reallocated to the TWP for the 2011/12 school year. An additional US$2.2 million was allocated to TWP for the 2014/15 school year for a total of US$10 million for the TWP. 1.7 Other significant changes 11. No other significant changes were made. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 12. Project origins. The PAD for the original, pre-earthquake project was prepared in September 2008 for EFA-FTI CF Project funding, but approval of the EFA-FTI CF Project was delayed. The MDTF, through which EFA-FTI CF Project was to be funded, was not approved by the Bank until December 11, 2009. This delay of more than one year in moving the Project forward resulted from a contention by the United Nations Children’s Fund (UNICEF), a member of the LEG in Haiti, that it (UNICEF) was the appropriate agency to implement education projects in a fragile state such as Haiti. The High-Level Committee of the FTI agreed with UNICEF and allowed UNICEF time to organize its ‘Transition Fund (TF)’ to receive funding for education in Haiti and to take steps necessary to be the Managing Entity for the Project. However, these steps were not completed and the TF was not constituted. As a result, the High-Level Committee agreed, as originally planned, that the Bank will be the SE for the operation, with the MENFP as the implementing agency, and funding will go through the MDTF. The FTI High-Level Committee accepted the PAD in December 2009. The multi-donor nature of the process to obtain CF financing 5 could have offered benefits through donor alignment and coordination, but it is unlikely that the Project would have faced this delay were it not prepared using procedures requiring ample consultation and that development partners reach a consensus with the Government on the modality for and use of the funds. See annex 10 for a fuller narrative of this process. The PAD was never formally reviewed by the Bank. 13. The earthquake crippled the MENFP and posed enormous challenges to adapting the Project: headquarters were destroyed, with a catastrophic loss of lives and talent, creating a situation where counterparts were in survival mode and facing huge challenges to accomplishing any task. A UNICEF report 9 estimated that 1,347 teachers were killed, along with tens of thousands of students. A quote from the report gives a sense of the initial post-earthquake conditions: “The single biggest obstacles to schools reopening now is rubble. It’s everywhere. It blocks roads, spills into parks and dangerously tumbles down hillsides…it is simply too much to handle.” Under these conditions the GoH, the Bank, and other stakeholders had to determine how best to refashion the EFA-FTI CF Project. 14. The Bank worked with the MENFP to determine how best to adapt an already prepared project to the new emergency needs. At the request of the MENFP, key changes were agreed (table 1 first column) and endorsed by the LEG and approved by the EFA-FTI CF Project, ensuring a participatory, coordinated approach. After the earthquake, it was clear to the GoH and the Bank that providing access to primary education had an even greater importance. However, the formal Bank procedures for trust fund-financed projects were not fully followed as there was no formal PAD review or funding approval for this specific project at the regional vice president level. 15. The teacher compensation component or Kout Pous (Giving a push to return to school) was designed to respond quickly to the urgent need to provide resources to schools to keep them operating in the aftermath of the January 2010 earthquake. The program was advertised to schools in the three departments affected by the earthquake, which will then fill out a form indicating that the school was (a) located in one of the departments affected by the earthquake and (b) was operating with a principal and attending students. The main challenge for this component was to reach schools quickly, but doing so with very little information (even accurate contact information) about the schools themselves. If the school qualified after being checked by the third-party agent hired by the Project, then the school would have received a US$2,500 grant. Initially, it was estimated that more than 6,000 schools would be eligible, which translated to a total component allocation of US$15 million. In a sense, the teacher compensation grants were like a block of tuition waivers because schools received an amount of funds that compensated for the shortfall in tuition payments by students. A US$2,500 grant was equivalent to providing one-time tuition funding for about 28 students (at US$90) for one year. (Tuition waivers typically funded two 45-student cohorts per participating school—two tranches of US$4,050 each). So, the school grants were on a smaller scale at the school level, but were 9 “Children of Haiti-Three Months After the Earthquake-Process, Gaps and Plans in Humanitarian Action Supporting a Transformative Agenda for Children,” UNICEF, April 2010. 6 delivered more quickly relative to the two-tranche tuition subsidies. The teacher compensation component was therefore a successful feature of the Project’s design. 16. The Project benefited from the experience of implementing similar activities in Haiti. No lessons could have adequately prepared the Project for the devastation from the earthquake. Yet, as the country began to emerge from the disaster, the perennial challenge of improving access was an even greater priority, which would have required increased resources and speed of delivery. Two main features of the Project, tuition waiver and school nutrition and health, had already been implemented for several years under the first Education for All (EFA-1) project. There was no ICR yet to document the lessons because this project still had several more years of implementation. It was advantageous that the same Project Coordination Unit (PCU) was used for both projects. The PCU was already aware of the challenges in implementing the Project’s two main programs, in particular, finding the right NGOs to deliver food for the school canteen program 10 and their timely recruitment, as well as the delivery of the first and second tranches of funding for tuition waivers, and reliable monitoring and evaluation (M&E). The PCU had to take on the lead managerial role for the Project because the MENFP needed time to restore its ability to function after the earthquake. 17. The expansion of the TWP incorporated the lesson that allowing schools to nominate children for the tuition waiver is an effective way of tapping into the excess demand for primary education. The EFA-FTI CF Project was also an example of upscaling activities with a new source of funding, a lesson later documented in the EFA-1 ICR. Another lesson followed by the EFA-FTI CF Project was the use of local capacity to compensate for the MENFP’s lack of capacity. For example, the Project employed a local microfinance organization, Fonkoze, to implement payments of the teacher compensation component. As with EFA-1, the Project also hired local firms to verify student attendance. 18. Risks and mitigation. The pre-earthquake preparation identified several key risks. One risk was social instability that came from a number of factors outside the control of the Project, namely, unemployment and poverty. This risk was to be managed by the achievement of quick wins of tuition waiver and school feeding, as well as job programs supported by the GoH in impoverished areas. Another risk was the misuse of tuition funds and discrimination against nonbeneficiary students. The control against this risk was financial management (FM) training for school management committees by NGOs and public criteria for use of subsidy funds. This risk would also have been managed by a professional PCU with close Bank supervision from the country office. The risks associated with weak capacity at the MENFP could slow project execution. These risks also were to be mitigated by intensive supervision by the PCU and the Bank, in addition to other capacity-building efforts to help offset the MENFP’s weak capacity. The FTI, which became the GPE in 2011, also participated in project supervision to ensure that funds were used for their intended purposes. 10 The school canteen, school nutrition and health, and school feeding programs are used interchangeably to denote the same program. 7 2.2 Implementation 19. Teacher compensation, or Kout Pous, helped keep schools open, but was difficult to administer. The teacher compensation component reached 2,824 schools, providing desperately needed resources to pay teachers for the school year 2010/11. The component disbursed about half of what was originally envisioned, because only half of the originally estimated number of schools applied for the funding. The original estimate was inaccurate mainly because there had not been a school census since 2003 (the next one was 2011), which contributed to the fact that the MENFP did not have an accurate, consolidated list of schools. In addition, the list of schools that constituted the basis for the estimate did not differentiate between schools’ income levels; certain better- funded schools on the list may have decided not to participate because of the relatively low compensation grant amount in comparison to the fees they received from families. The MENFP capacity to monitor these schools had also been destroyed by the earthquake, and so, the grants were managed by a third party, Fonkoze. This agent was responsible to make payments to schools and to monitor and record component activities, which it did satisfactorily. For remote schools, it was still difficult to advertise the program and some schools found it too difficult to get to payment points to receive the funds. 20. The school canteen/student nutrition and health program was generally well implemented. The Project funded this activity for the 2010/11 and 2011/12 school years delivering a total of 62,000 school years of lunches and snacks (the number of school children receiving lunches and snacks for one academic year). The main NGO for implementing the school nutrition and health program, the Bureau de Nutrition et de Developpement (Office for Nutrition and Development), had been implementing such programs for many years and performed satisfactorily during the Project. For the school canteen program as a whole, implementation continued to improve beyond the Project: the Bank established a call center during the 2014/15 school year, which obtained real-time information about the school canteen program through phone calls to schools. The call center helps identify and remedy issues associated with the program and has been successful in ensuring quality of food services and improving governance of the program. 21. Once funds were redirected to the TWP, it was implemented rapidly and all funds were utilized. The Project supported tuition waivers for school year 2011/12. Thanks in large part to the experience that the GoH and the PCU already had with implementing tuition waivers along with the high demand, all funds were reallocated quickly. For the TWP as a whole, schools sometimes experienced significant delays in receiving the second tranche of the tuition waiver. This issue was caused by difficulties in verifying student attendance at recipient schools. To improve the transparency and timeliness of verification, the Bank worked with the MENFP to require the publication of the list of participating schools at the departmental level, an electronic list of students, as well as public presentation of the findings by the verification firms. At the end of the Project, there were remaining funds that were used to fund one tuition waiver tranche for the 2014/15 school year. The second tranche was funded by another development partner. 22. The modest ECD subcomponent was only partially implemented. The ECD action plan was developed under the Project, but less than half of the activities were implemented. Activities implemented included the development of an ECD strategy, a workshop on the implementation of the ECD strategy, and the purchase of equipment to support the ECD unit at MENFP. The action 8 plan also called for the building of ECD classrooms. However, the ECD proposal for these classrooms did not comply with the Grant Category 5 requirement that the implementation of this activity be carried out by a third party, and the classrooms were not funded by the Project. The action plan also called for an inventory of local educational materials to determine the need for pre-school materials, and this inventory was not conducted under the Project. 23. The curriculum development subcomponent was only minimally implemented because of contract delays and lack of policy guidance. UNESCO was hired to provide training and technical assistance for the first phase of national curriculum reform, but, by project close, was only able to provide some initial training. The main reason was the delay in contract agreement between the Bank and UNESCO relating to the Bank’s contract template for contracting with U.N. agencies. By providing additional visibility and momentum, the subcomponent supported the establishment by the GoH of a national curriculum reform committee to guide the overall reform and help overcome political obstacles. However, this committee was not established until the UNESCO contract had expired. The delay in its establishment was linked to the high level of turnover at the MENFP during the time of implementation, including several changes of ministers. Unused funds were reallocated to tuition waivers. UNESCO had to reimburse the Project’s designated account US$376,900 of funds that had been paid for services that would have not been provided before project close. 24. Institutional capacity activities for the MENFP (Component 3) were mostly successfully implemented. This US$1 million component financed the EFA Technical Unit, which supported the MENFP in implementing the Project. This component funded equipment purchases (including vehicles; computers, peripherals, and other office equipment; and generators) and technical assistance and was aimed at the regional offices of the ministry. Staff were surveyed on the use of the equipment and vehicles, and most reported that the resources were important to carry out their functions (section 3.2 has further details.). 25. There was a trade-off in calibrating the size of tuition waiver and teacher compensation grants. One of the issues in implementation of both the teacher compensation and tuition waiver components was the small size of the grants/subsidies. Many schools that might have qualified for the tuition waiver and the teacher compensation were not interested because the amounts were too small. ICR field interviews revealed concerns about the adequacy of the individual tuition subsidy payments, noting that the US$90 per student per year amount had not been adjusted since 2007. The concern on the other side was that raising the tuition waiver subsidy would make the program less financially sustainable as the GoH has been unable thus far to take over the program, even at the US$90 per student level. The modest US$90 amount also serves as a self-targeting mechanism, attracting mainly the poorer schools. 26. There was no midterm review (MTR) for the Project. Despite the lack of an MTR, project fund reallocations were made in a reasonable amount of time and in an appropriate manner in the post-disaster environment. According to the legal advice at the time of reallocation, the GA language was sufficiently general to accommodate the shift of resources into tuition waivers and, therefore, a formal restructuring was obviated. An MTR/restructuring still could have benefited the Project by upgrading the Results Framework to include better indicators of access, equity, and quality. The Project was never designated at risk by any of the Implementation Status Reports (ISRs). 9 2.3 Monitoring and Evaluation (M&E) Design, Implementation, and Utilization 27. Design. The Project’s Results Framework was developed after project effectiveness in the aftermath of the devastating earthquake. The framework was limited to output measures for both the PDO and intermediate indicators. The PDO indicators measured the numbers of beneficiaries, that is, schools participating in the teacher compensation program, primary students receiving tuition waivers, and the number of children benefiting from the school health and nutrition program. Indicator targets mainly reflected the amount of resources available for the activity divided by the benefit amount per school, or per student. For example, the TWP had a target of 83,000 student- years, which was derived by dividing the estimated amount of funds available (US$7.5 million from the initial reallocation) by the tuition subsidy per student (US$90) and not taking into account the cost of the program. The Results Framework contained indicators that indicated but will need to be supplemented with additional evidence to prove whether the school grants and TWP actually improved access and improved education equity and quality for students. For teacher compensation, for example, the conclusion was that if schools did not receive Kout Pous resources, then some teachers could not be paid, and they would have to abandon the schools, thus jeopardizing a full academic year for primary school children. M&E weaknesses of the Results Framework were significantly offset by employing third parties to verify the legitimacy of applications for teacher compensation, implementation of the school health and nutrition program, and, later, the TWP. 28. Implementation and utilization. M&E data tracked the implementation of the three main project components: teacher compensation, the TWP, and school nutrition and health programs. The PCU relied on M&E to determine component progress and reallocations of project funding. The disbursement of the second tranche of the tuition waiver was contingent on on-site verification of every school that had a cohort of students receiving tuition waivers. The institution of the call center further improved M&E with real-time data. The teacher compensation component also required strict M&E by a third party to verify school eligibility, which helped weed out applications from those that did not represent actual schools. The M&E data was weak in making it possible to assess higher-level impacts. For example, cohorts of students from first to sixth grades were subsidized through tuition waivers, but there was no tracking system put in place for these cohorts to determine whether or not the beneficiaries succeeded at higher learning. This limitation makes it difficult for a broader analysis of returns to project investments. However, the ICR provides additional data to measure access, equity, quality, and governance objectives. 2.4 Safeguard and Fiduciary Compliance 29. Safeguards. No environmental or social safeguard policies were triggered. There was no construction in the Project, nor were any plans developed that would have potentially negative environmental or social impacts. 30. Financial Management. The Project mainly financed teacher compensation, school feeding, and TWPs (about 95 percent of the Project). The teacher compensation component (Kout Pous) was carried out through a payment agency, Fonkoze, as the MENFP’s implementation capacity was severely affected by the earthquake. Fonkoze satisfactorily carried out its function making payments to schools (US$2,500 school grants), though it cited a number of problems with the program, in particular, determining school eligibility. The majority of project payments were 10 made using direct payments method. The selection process for schools worked well and uncovered many schools that were not legitimate schools, and they were rejected for funding. The PCU’s FM (and procurement) functions suffered a setback when the two lead specialists resigned in September 2015 thus complicating the effort to reach a final accounting. The Bank supported the PCU to rectify a number of issues (including a category 5 disbursement condition that rejected the MENFP proposal for ECD classroom construction, the UNESCO refund, and the review of the 2014 unqualified audit despite the lack of access to accounting records) raised in the final ISR to attempt to improve the Unsatisfactory rating for FM in the final archived ISR. At project closing, the FM rating remained Unsatisfactory because of the following main shortcomings: (a) the absence of an FM specialist; (b) lack of bookkeeping during the 2014–2015 period; and (c) at the PCU level more generally, the unavailability of information on disbursements by source of funds and category expenditure. At the time of the ICR, substantial progress had been made on these shortcomings with several of the shortcomings resolved, including the hiring of a new FM specialist. 31. Procurement. Procurement included the school canteen contracts for school years 2011/12 and 2012/13, a contract with UNESCO for providing curricular and pedagogical support to the MENFP, and provision of equipment and vehicles to the MENFP. Regarding the school canteen contracts, the selection processes were slow but were completed in time for the start of the school year. The PCU/MENFP needs to implement measures proposed by the Bank to better plan and streamline the selection process and the delivery of services. The UNESCO contract was a single- source selection; yet, the completion of the negotiations took more than 18 months of discussions about description of services, some contract provisions, and issues with the contract template the Bank uses for the United Nations agencies. 2.5 Post-completion Operation/Next Phase 32. Key activities carried out under the Project were also supported by the second Education for All (EFA-2) project (2011–2017), which continues to fund tuition waiver and school health and nutrition programs. EFA-2 is funded by IDA, a second GPE grant, the Canadian Government, and separate parallel financing by the Caribbean Development Bank. A new IDA operation, Providing an Education of Quality in Haiti, is under preparation and will follow up on a number of activities of the CF project including school grants to waive tuition, improvements to educational quality and governance, and building capacity at the MENFP. 3. Assessment of Outcomes 33. This ICR recognizes that the final ISR rated PDO achievement as Moderately Unsatisfactory; however, there is a substantial body of additional evidence that was not considered in the ISR rating that supports substantial ratings for efficacy and efficiency. 3.1 Relevance of Objectives, Design and Implementation Rating: Modest 34. Objectives - Modest. Because the Project did not have a PAD, the PDOs, improved access, equity, quality, and governance, are taken from the MDTF GA, which covered not only the Project, but also other anticipated activities to be financed with additional sources of financing. The main 11 intent of the Project, however, was improved access, which is indicated by its main activities of teacher compensation, TWP, and school nutrition and health activities and its PDO indicators. These activities accounted for almost all of the Project’s resources and were and remained highly aligned to the GoH priorities as outlined in the post-earthquake APNRD; the longer-term SNA/EPT; the sector plan under implementation at the time of writing of the ICR 11; and the Bank’s Country Partnership Framework for Haiti for the period FY2016–FY2019. The relevance of access and equity is evident also because of the high poverty rate and the still large number of children who are out of school, as well as a continued risk of losing gains achieved in access given a lack of a sustainable financing mechanism. Continued demand for the TWP is demonstrated by the fact that the funds for tuition waiver were exhausted ahead of schedule with another donor supporting the continuation of the cohorts through primary school. The relevance of access and equity is evident also when considering the full cost of sending a child to a nonpublic school is about US$131 (tuition at US$80–90 plus the costs of transportation, nutrition, and uniforms) compared with a gross national income per capita of US$400. 12 35. In contrast, quality and governance objectives—though they remained GoH development priorities—were a lesser focus of the Government over the life of the Project (as supported by the APNRD, which emphasized access) and of development partners in alignment with the GoH (as supported by the fact that expected financing for these objectives did not materialize). This lesser emphasis on quality and governance was the case especially during the short-term post-earthquake period, when the utmost priority was accorded to the short-term restoring of access to schools for students most in need. As the emergency period subsided and in the final months before project closing, quality and governance had begun to have a more prominent role in the dialogue and priorities, as evidenced in their inclusion in the Country Partnership Framework for Haiti for the period FY2016–FY2019. However, on balance, even though the Project made some contributions in these areas, quality and governance were only of modest relevance given the prominence of access and equity as the GoH and development partner priorities over the Project’s life. The Project would have benefited from a narrower scope of its objectives during preparation itself or through an early restructuring to focus on access and equity, thereby keeping its development objectives fully relevant. 36. Design - Substantial. The Project’s design is substantially relevant because it funded the types of activities that are important to improve access by overcoming the main impediment to attending school, which is cost. Initially, the teacher compensation grants were appropriate because in the immediate aftermath of the earthquake, schools needed funds quickly to remain open. The Kout Pous was a fast method and payable upon request from a qualifying school. It was also rightly envisioned as a temporary activity that would have served its purpose to deliver funds quickly to schools to maintain access to primary education. Once the emergency subsided, remaining funds could be transferred to the TWP, a program with proven success, and which was consistent with normal school operations. Student nutrition and health were also relevant, because with many children reportedly having nutrition deficits and in need of de-worming, the school canteen 11 Operational Plan 2010–2015 (Plan Opérationnel 2010–2015: Vers la refondation du Système Educatif Haïtien, April 2012). 12 Policy Working Paper (PWP) 7175. “Increasing Access by Waiving Tuition-Evidence from Haiti”. p. 3. January 2015. 12 program was needed to ensure not only school attendance (access), but also poorer students’ readiness to learn (equity and quality). The small curriculum development and ECD components were not implemented successfully, but only a small amount of funds were risked to pursue these activities to the end of the Project, and they potentially might have had a positive impact on quality and governance in Haiti’s education sector. 37. Implementation - Modest. Implementation was relevant in that the right decision was made when the teacher compensation program ended after project year one, with only about half of the US$15 million having been used, and the Project appropriately reallocated the substantial remaining funds (US$10.0 million) to the TWP. This decision promoted improved access by providing additional resources to primary schools and supported school attendance for 93,624 student-years equivalent of tuition benefits. However, there were two important shortcomings in implementation. First, there was the lack of an outcome-oriented Results Framework that will allow better assessment of project achievements. Second, the Project should have been restructured once it was confirmed that financing from other development partners will not materialize, not only to remedy the weak Results Framework, but also to better align the PDO with the main priority of the Project, which was improved access, rather than, for example, improved governance. Implementation relevance was Modest also because of the weak implementation of the ECD and curriculum development subcomponents. 3.2 Achievement of Project Development Objectives Rating: Substantial 38. The objective of the EFA-FTI CF Project (from the MDTF GA) was “to assist the recipient in implementing its SNA/EPT through the improvement of (a) access to primary education for poor children of age 6–12 years and (b) equity, quality, and governance in the recipient’s education sector.” This section assesses access, equity, quality, and governance separately. The ICR takes into account that, given the emergency context of the Project and the focus of project resources, the main aims of the Project were improving access to and equity of primary education, with quality and governance improvements having less emphasis. The ICR does not provide a split evaluation because the Project was not restructured. This section summarizes the Project’s Results Framework (see also annex 2) and provides additional evidence to support the outcome rating. 39. Improved access - High. The Project’s main focus was improving access of primary school children (of age 6–12 years) in a post-disaster and generally deprived educational environment relative to access without the Project. The PDO indicators include schools participating in the teacher compensation program, which reached 2,824 out of a target of 3,000 schools (equivalent to funding another 78,444 students for one year), and, therefore, was substantially achieved. The target assumed that every dollar spent on the component would have reached the schools. In fact, almost all of the component funding went directly to the eligible schools with some resources spent on third-party administration of the component. The grants to the schools, according to ICR field interviews, allowed schools to retain teachers and make other eligible expenditures, which they otherwise could not have in the aftermath of the earthquake. Schools in earthquake zones had insufficient means to pay their teachers because there was reduced funding coming from families, as a result of the major reduction in economic activity immediately following the earthquake. In the absence of project funds, most beneficiary schools would not have 13 been able to conduct any, or as many, classes, thus depriving students of education for the school year. 40. Improved access is also indicated by the number of children benefiting from the TWP. With reallocated funds, it was possible to finance 93,624 student-years of primary education. The target was 83,000 student-years, but that was based on available funding of US$7.5 million left over from the teacher compensation program. The target should have been adjusted when additional funding for the 2014/15 school year became available (ultimately, US$10 million was spent including the costs of administering the program). As stated earlier, the target was determined by the amount of funds that were initially left over from the teacher compensation program. The important question is whether all 93,624 students, or at least a large proportion of them, would have gone to school anyway in the absence of the tuition waiver. In discussions with school administrators during the ICR mission, a large majority of the students would not have been able to pay their tuition. A much smaller percentage, around 15 percent, might have been allowed in school anyway, but not over the long-term. Even if some of these students were permitted to attend school in the short-term without tuition waiver funding or other payment, the school would not have had the proportionate resources for the additional, nonpaying students, which could also reduce educational quality by contributing to the hiring of lower-cost, less-qualified teachers. Annex 11 provides beneficiary testimonial evidence underscoring the value of the TWP. 41. There is additional evidence for improved access and other outcomes beyond the Project’s Results Framework. PWP 7175 Increasing Access by Waiving Tuition-Evidence from Haiti (January 2015) provides key additional evidence to support the position that tuition waivers improve access. The study found that in the case of Haiti, tuition waivers contributed to more students being enrolled, both male and female. These results are reflected in the Beta coefficients presented in table 2 (excerpted from the PWP 7175, p. 11). Table 2. Estimated Impacts of Treatment (TWP) on Access to Primary Education Impacts of TWP Observed in 2011/12 School Year Beta Hat Standard Error Grades 1–4: total number of students 88* 11.66 Grades 1–4: number of female students 41* 6.14 Grades 1–4: number of male students 47* 5.92 Source: Bank staff estimates using Haiti national school census 2011/12. Note: * Indicates the number of degrees of significance; TWP was financing grades 1–4 for this sample of 354 schools. 42. The study exploits the random selection of schools into the program from a group of schools that applied and qualified, meaning that before the program, the schools were very similar and any later differences can be causally attributed to participation in the program. The study measures the results from the 2011/12 school year, the main year in which the CF project funded tuition waivers. A key finding was that treatment schools (those whose students received tuition waivers) had an average of 88 more students in grades 1–4 (the grades financed by the program in 2011/12) than schools that applied to the program but were not selected, with a roughly equal gender split. While treatment schools may have drawn away some students from control schools, there is an overall gain in enrollment, providing evidence of improved access. 43. Overall primary school enrollment increased significantly in Haiti during 2001–2012. The World Bank Haiti Poverty Assessment, ‘Investing in People to Fight Poverty in Haiti’ (2014), also provides evidence of improved enrollment through 2012. The report shows that there has been 14 substantial progress in primary enrollment during 2001–2012 (pp. 86–87). Primary enrollment in 2001 was 78 percent, which increased to 90 percent (average for all primary grade years) by 2012. This increase in enrollment was achieved despite the 2010 earthquake and can be at least partially attributed to interventions such as the TWP, school nutrition and health, 13 and the teacher compensation programs. 44. As pointed out in section 2.1, the US$2,500 school grants in some respects can be viewed as a proxy for tuition waivers, only they could be delivered faster and in a single tranche, which was needed for the post-earthquake emergency period. Both types of funding allowed schools a range of eligible expenditures, the largest of which is teacher salaries. Therefore, the evidence that the PWP provides and that the TWP promotes access, likewise, suggests that the teacher compensation school grants promoted access because they also compensated for the lack of student tuition payments (that spiked post-earthquake). 45. Improved equity - Substantial. Because of the widespread poverty in Haiti (78 percent poverty rate pre-earthquake) and the radically worsened conditions post-earthquake, focusing project resources on poor and vulnerable students did not present a major challenge. In addition to focusing the teacher compensation program on students in earthquake zones, the Project was also designed with a built-in self-targeting mechanism to concentrate the allocation of resources to the poor. The TWP strategy was to make tuition subsidies sufficiently low so as not to interest the better-off schools, yet provide a significant amount of resources to ensure continued school operations. A comparison of schools that applied to the program with nonapplicant schools in the participating departments showed that the pool of schools that applied to the program (as well as the subset that was selected for participation) had on average the same low rates of access to running water, electricity, improved infrastructure, and other characteristics as nonapplicant schools. 14 In some dimensions, notably student-teacher ratios and repetition rates, applicant schools appear slightly worse on average. These results confirm that the TWP benefited average or slightly worse-than-average schools with regard to resources. Given widespread poverty, the average school will indeed be serving poor children. 46. The TWP was also designed to be geographically dispersed to increase equity. Specifically, the majority of participating schools are located outside of the Ouest department (where Port-au- Prince is located and where poverty is substantially lower than in the rest of the country). At least one school from each commune (subregion) of each department was selected for participation, to maximize the geographic distribution of the program and reach rural areas. Access to services has historically been much lower and poverty rates much higher in rural areas, which is reflected in worse educational indicators. The teacher compensation grant (US$2,500) was likewise calibrated to help schools directly affected by the earthquake to pay teacher salaries (and other expenses) and, by being sufficiently modest, did not attract the interest of more well-off schools. 47. Improved quality - Modest. While primarily intended to improve access to education, certain design features of the TWP had likely positive impacts on education quality. As part of the 13 A video providing a compelling account of the importance of school feeding in Haiti (including to achieving school attendance) can be found at: https://www.youtube.com/watch?v=5DGpeIN9Kjc&feature=youtu.be 14 Using data from the 2003 School Census. 15 TWP, schools were required to (among others) hire qualified teachers, provide each student with a minimum of three textbooks, accept students between the ages of 6 and 8 into the first grade and support them through the sixth grade—inputs critical in contributing to student achievement. In particular, encouraging children to start schooling at the right age is likely to have contributed to learning and decreased the likelihood of repetition and dropouts. There were no indicators of quality in the Project’s Results Framework; however, analysis in PWP 7175 links the TWP with quality benefits. The quality improvement under the Project is indicated by a reduction in students repeating grades 1–6 and in the percentage of overage students in grades 1–4 (PWP) as table 3 shows. The negative coefficient indicates the reduction. The analysis found that there is a 10 percent reduction in repetition rates for grades 1–6, and a 9 percent reduction in the share of students over the average age for their grade in grades 1–4, with an equivalent reduction for boys and girls. Although—as is the norm in fragile contexts—there are no nationally representative data on learning outcomes, these proxy indicators are relevant for the context and show comparatively significant improvements. Though a reduction in the families’ financial burden likely contributed to the reduced rates of repetition and overaged students, both of these effects are important indicators of quality improvements. 48. The Project also aimed at longer-term quality improvement through support to the ECD action plan and curriculum development. There are output measures, including a signed contract with UNESCO for curriculum development and the development of an ECD plan (partial implementation), but the evidence of improved quality from these activities is negligible. It is plausible to attribute some improvement in educational quality to the implementation of the school nutrition and health program. In line with existing evidence, field interviews confirmed that children participating in the program not only are more likely to attend school, but also have a higher readiness to learn, and therefore a higher quality educational experience. 15 The fact that children are more likely to attend school if they benefit from the nutrition program also suggests that the snacks and lunches that they receive at school meet nutritional needs that would have not otherwise been met. While there is credible evidence of quality improvements from the Project, the ICR recognizes that evidence such as improved learning through improved test scores is lacking. This lack of evidence could have been remedied by tracking TWP beneficiary performance over the course of their support from the Project (through the sixth grade). Therefore, the ICR rates the achievement of quality improvements as Modest. Table 3. Estimated Impacts of Treatment (TWP) on Quality of Primary Education Impacts of TWP Observed in 2011/12 School Year Beta Hat Standard Error Percent repeaters grades 1–6 -0.10* 0.02 Percent overage grades 1–4 -0.09* 0.03 Note: * Indicates the number of degrees of significance; TWP was financing grades 1–4 for this sample of 354 schools. 49. Improved governance - Modest. Though the MDTF PDO for the GA includes improved governance, this was a minor and indirect objective to the CF project in particular, and there were 15 See The multiple impacts of school feeding: a new approach for reaching sustainability, Luca Molinas and Marc Regnault de la Mothe. Also see Rethinking School Feeding: Social Safety Nets, Child Development, and the Education Sector, Donald Bundy, Carmen Burbano, Margaret Grosh, Aulo Gelli, Matthew Jukes, and Lesley Drake (2009). 16 no outcome indicators of governance. Still, there were some improvements in governance in the implementation of the Project and in targeted activities. The project employed third-party verification of the three main components, namely, teacher compensation, TWP, and school health and nutrition. The selection of schools for teacher compensation was carried out by a third party which screened out illegitimate schools. Independent verification firms verified school enrollment of TWP participants and school health and nutrition beneficiaries. The Project also helped the MENFP build capacity and governance through the purchase of vehicles and equipment to help the institution carry out the monitoring required for the Project. At the ICR stage, a survey was conducted of 9 of the 10 directors of regional education departments (DDEs) who responded to questions about the significance of the resources provided by the Project in carrying out their responsibilities. Eight of the nine respondents stated that they would not have been able to carry out their main responsibilities without transportation and computers provided by the Project. Transportation was needed, in particular, for school inspectors to visit schools, collect information, and provide reports, which contributes to better governance (annex 5). While these governance measures were important, the achievement of this aspect of the PDO is rated Modest. 50. Owing to the successful use of project resources to achieve the main aims of the Project to improve access to and equity of primary education and to improve educational quality and governance, the overall efficacy of the PDO is rated Substantial. 3.3 Efficiency Rating: Substantial 51. An economic rate of return is calculated based on education projects in Haiti. The analysis rests on the estimated value of an additional year of education and the likelihood that project interventions made the additional year possible (annex 3). An internal rate of return of 11.8 percent is estimated for the three main components, namely, teacher compensation, TWP, and school canteen. The net present value (NPV) at 8 percent is US$16.6 million. Value for money was therefore sufficiently achieved. See annex 3 for details. 52. Implementation efficiency was also substantial as the Project met the per pupil cost standard set by EFA-1 and EFA-2, which also funded school nutrition and health and tuition waivers, which cost US$0.66 per meal and US$90 per student per year, respectively, with reasonable cost for administering the programs (for example, US$6.30 per student per year for the TWP). Nearly all of the funds were disbursed and there were no extensions of the Project’s closing date. The Project also contributed to significant improvements in internal efficiency through reduced repetition and potentially reduced dropouts by overage children, which reduces overall costs in the education system. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory 53. Table 4 summarizes the components of the outcome rating. The relevance rating is Modest, while efficacy and efficiency are rated Substantial. The Project disbursed nearly all of its funds in achieving its main goals. While it is correct to hold the Project accountable for the weak performance of a few activities, it is also necessary to weigh their lesser importance to the overall project. The main thrust of the Project, to keep poor children in school after the earthquake, was 17 pursued with the right activities and was achieved. The Project achieved the PDO at a reasonable cost for school enrollment and nutrition and health activities that have proven ‘tried and true’ in previous and ongoing operations. For these reasons, the overall rating is Moderately Satisfactory. Table 4. Summary of Outcome Ratings Rating Category ICR Ratings Relevance Modest Objective Modest Design Substantial Implementation Modest Efficacy Substantial Improved Access High Improved Equity Substantial Improved Quality Modest Improved Governance Modest Efficiency Substantial Overall Outcome Moderately Satisfactory 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 54. Project benefits were calibrated to be self-targeting toward poorer beneficiaries. That is, the project design provided benefits that were significant to poorer schools, but which were sufficiently modest so as not to interest the better-off schools. Poverty was also addressed through the school canteen program, which, as reported to the ICR field team, often provided the only substantial meal to children during the day. With regard to gender, the project benefited boys and girls about equally with both genders having similar reductions in repetition rates and incidents of overage students. (b) Institutional Change/Strengthening 55. Targeted institutional strengthening was to be supported mainly through provision of vehicles and equipment to the MENFP, which intended to help the ministry get up and running after the earthquake. With the ministry buildings destroyed and many of its staff killed, the institution was essentially starting from zero. While the Project did not rebuild education infrastructure, it did add resources to those of EFA-1 by providing support to the school system through the PCU at a time when the MENFP could not function. With regard to institutional strengthening beyond the initial emergency support, the Project supported the development of an ECD action plan, although this plan has not yet been implemented. (c) Other Unintended Outcomes and Impacts (positive or negative) 56. One of the negative results is that the subsidy programs sometimes attracted applications from fictitious schools. The PCU and the implementing agent, Fonkoze, reported that hundreds of these applications were made to receive the teacher compensation grant. In a number of cases, the addresses of the schools were changed slightly so that the school principal could, illegitimately, get additional grant funding. These applications were rejected. 18 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 57. A beneficiary survey was conducted of 9 of 10 DDEs and is summarized in annex 5. As stated in section 3.2, the project provision of equipment and vehicles was key to regional inspectors being able to visit schools, monitor programs, and report findings. 4. Assessment of Risk to Development Outcome Rating: Substantial 58. This rating considers that the outcomes have already been banked through (a) keeping schools running after the earthquake; (b) improved attendance and readiness to learn of primary school students; and (c) improved access to schools through tuition waivers. However, the tuition waiver and school feeding programs themselves require continued external funding and, as such, are not sustainable. There have been efforts by the GoH to sustain these programs, including the National Education Fund, but the programs themselves are not yet sustainable. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Moderately Unsatisfactory 59. The Bank (management and preparation team), as well as the GPE and the LEG, contributed to an appropriate project design that resulted in accelerating funding to schools in the immediate aftermath of the January 2010 earthquake. It was also appropriate to continue the school canteen program for primary school students, which was even more urgent after the earthquake. At the same time, small allocations were made for the ECD program, building capacity at the MENFP, and curriculum development. This recognized the additional need to improve quality of primary education, but did not risk a large amount of funds. 60. The main shortcoming was that the Bank did not follow its own established rules for project approval. 16 As stated in section 2, there was no formally reviewed PAD after the earthquake, nor was there a designation of the Project as an ‘emergency’ project. Such a designation would have allowed a shortened timetable for project approval and a comparably quick provision of resources to Haiti while also complying with Bank procedures. Instead, the Bank approved the Project as a TFP as part of the EFA MDTF. In addition, the Project was defined by a concise, six-page Note Technique that did not appear to be a part either of the MDTF negotiations package or of the TFP. While it was understandable that scarce human resources could not be expended to redraft the PAD, delays of critical resources to Haiti in the aftermath of a major disaster still could have been minimized while also complying with Bank approval procedures. In retrospect, another shortcoming was the decision to channel project funding through an MDTF, when the option of 16 OP/BP 14.40 - Trust Funds, issued July 2008, provides that “all grants financed by Recipient-Executed Trust Funds for which the TF proposals have been submitted for approval on or after July 1, 2008, are subject to the Bank’s operational policies and procedures that apply to IBRD and IDA financing, including the Bank’s framework on governance and anticorruption.” 19 additional financing for EFA-2 was still possible. This decision complicated preparation, but did not produce any significant benefits. (b) Quality of Supervision (including of fiduciary and safeguards policies) Rating: Moderately Unsatisfactory 61. The Bank supported the PCU and MENFP to implement the vast majority of planned activities and disburse the grant. Despite an extremely challenging context, the Bank provided close implementation support, such as by providing advice and supporting decision making to improve the implementation of the TWP (publication of school lists, presentations of verification results by third-party firms, and so on). When the teacher compensation component was completed but only disbursed half its funds, the Bank appropriately switched the funds back to the TWP for which there was high demand. These funds were fully utilized, on schedule because of the substantial need for the program. A significant shortcoming was the missed opportunity to restructure the Project once the emergency subsided after the earthquake. Despite advice from the Bank’s legal team that the reallocations could be made within the existing parameters of the GA (that is, without restructuring the Project), a formal restructuring could have remedied the weak Results Framework and improved the relevance of objectives by better aligning the PDO with the main project priorities. (c) Justification of Rating for Overall Bank Performance Rating: Moderately Unsatisfactory 62. With the quality at entry and at supervision rated Moderately Unsatisfactory, the overall Bank performance rating is Moderately Unsatisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 63. The GoH/MENFP worked with the LEG and the Bank to develop the original project proposal, which was relevant to country needs having benefited from the experience of the successful implementation of EFA-1. The GoH was also quick to react to the earthquake with the post-earthquake APNRD and the Note Technique developed by the MENFP to reconfigure the project design to more rapidly assist the education sector. There were no issues of compliance with covenants, and the grant was fully disbursed. (b) Implementing Agency or Agencies Performance Rating: Moderately Unsatisfactory 64. The PCU administered a project with full disbursement, but there were a number of significant problems. There was a weak M&E function with persistent difficulties in interpreting the importance of the M&E indicators. There were also significant issues with FM, which resulted in an unsatisfactory FM rating at the end of the Project. The MENFP did not effectively use resources available for curriculum development, and most of the funds for this project component had to be reallocated to the TWP (2014/15). Part of the reason for the inability to use these funds 20 was the many (five) changes of ministers at the MENFP during the Project. The MENFP did benefit from the provision of vehicles and equipment, which were essential to restarting operations after the earthquake. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 65. With the Government rating Moderately Satisfactory and Moderately Unsatisfactory rating for implementing agencies, the outcome rating of Satisfactory dictates an overall borrower performance rating of Moderately Satisfactory. 6. Lessons Learned 66. Lesson 1: Projects sometimes need to be substantially redesigned to respond to sudden disasters, but it is still feasible and necessary to adhere to Bank procedures for Bank- administered trust funds. Although the EFA-FTI CF Project had already been fully prepared in late 2009, changes affecting three-quarters of project resources needed to be made rapidly. The Bank’s procedures offer streamlined processes to accelerate emergency assistance that will also allow for regional vice president approval. 67. Lesson 2: Once emergencies subside, formal restructuring may be needed to establish/improve the Project’s Results Framework. The EFA-FTI CF Project did not formally create/revise its Results Framework to establish appropriate indicators to measure project performance. Failure to do this created problems in assessing project impacts during implementation and at the completion stage. 68. Lesson 3: Projects that implement school grant (teacher compensation) or similar programs may need to screen out bogus applications from nonexistent schools. The individual school grant size of US$2,500, while too small to interest many schools (approximately enough to retain two teachers earning a low to midrange salary), did elicit applications that were not legitimate. Effective monitoring and application of screening criteria by a third-party agent led to rejection of these applications. 69. Lesson 4: Setting the level of tuition waiver amounts should also try to balance schools’ needs for resources on one hand, while not encouraging better-off schools to participate on the other (that is, a built-in self-targeting mechanism). Although higher tuition waiver amounts could help schools improve the quality of instruction, it might also funnel scarce education resources to less poor schools. Some schools were able to make some quality improvements with the subsidies, but for most schools, the US$90 allotment per student is just sufficient to meet per student school costs and, therefore, is not enticing to the more established schools. 70. Lesson 5: Projects funded through an MDTF requiring consultation, while often beneficial to overall donor coordination, may also lead to delayed delivery of assistance. The CF project could have begun implementation a full year earlier were it not for differences among development partners regarding who should be the SE for the Project. In the end, the original SE, the Bank, was selected and rather than benefiting the client, assistance was delayed by one year. 21 71. Lesson 6: Projects that intend to fund essentially the same activities as other ongoing projects should consider being implemented as additional financing rather than as standalone projects. The CF project was essentially adding funds to what EFA-2 was already doing with regard to tuition waiver and school nutrition and health. Other than to create an MDTF, it is not really clear what the advantages were to creating a new CF project. EFA-2 could have absorbed the CF funds, and CF activities could have piggy-backed on the EFA-2 Results Framework. 72. Lesson 7: To ensure good governance of tuition subsidy programs, it is important to provide information publicly as to which schools are benefiting. In the case of TWP, the publication of the list of participating schools at the departmental level, as well as the school obligation to provide the list of students in electronic file before signing the annual contract, and the public presentation of the findings by the verification firms in 2011 were important to ensuring that the subsidies reached the intended beneficiaries. 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors (a) Grantee/Implementing Agencies 73. The Bank is in basic agreement on most points of the borrower’s ICR. There are few clarifications. The report refers to a restructuring (Section 2.4 of the Annex 7), while these were indeed changes to the project, but were not formal restructuring. The report also uses original targets from the 2009 PAD, but, because the PAD was not approved, the Bank ICR uses the original targets established at supervision. There is also a discrepancy with respect to the number of schools targeted for the teacher compensation component (3,600 - GoH and 3,000 - Bank). This number changed a number of times mainly because there was not an accurate list of schools and was more than 6,000 schools originally. On the subject of an ‘overly interventionist approach’ by the Bank (section 4.1 of the Annex 7), the Bank was active in supervision and intervened to attempt to reduce delays in project activities. The Bank also believes that the problems with FM are somewhat understated by the borrower’s ICR (as laid out in section 2.4 of the Annex 7). The Bank concurs with the borrower’s lessons learned (section 6 of the Annex 7) and they are valuable complements to the lessons learned in the Bank ICR. (b) Cofinanciers/Donors 74. Not applicable. (c) Other Partners and Stakeholders (for example, NGOs/private sector/civil society) 75. Not applicable. 22 Annex 1. Project Costs and Financing (a) Project Cost by Component (in US$ Million Equivalent) Appraisal Estimate Actual/Latest Estimate Percentage of Components (US$, millions) (US$, millions) Appraisal (%) Component 1: Improvement of access 16.00 17.26 107.9 and equity for primary education Component 1.A(a): Support to nonpublic 15.00 7.06 47.1 schools Component 1.A(b): Tuition waiver for 1.00 0.18 18.0 students Component 1.B: Establishment of early childhood development services (pre- 0.00 10.02 – school) Component 2: Improved school 5.00 3.66 73.3 performance in primary education Component 2.A: School nutrition and 4.00 3.62 90.5 health program Component 2.B: Support for the development of educational tools for 1.00 0.04 4 teaching in post-disaster situations Component 3: Rebuilding the capacity 1.00 1.07 107.0 of Ministry of Education Financing Total Baseline Cost 22.00 21.99 99.9 Physical contingencies 0.00 0.00 0.00 Price contingencies 0.00 0.00 0.00 Total Project Costs 22.00 21.99 99.9 Project preparation fund 0.00 0.00 0.00 Total Financing Required 22.00 21.99 99.9 Source: Appraisal amounts based on the MENFP Note Technique, March 11, 2010. Note: Amounts are still being finalized, but disbursement was reported at near 100 percent. (b) Financing Appraisal Actual/Latest Type of Percentage of Source of Funds Estimate Estimate Cofinancing Appraisal (%) (US$, millions) (US$, millions) Trust Funds – 0.00 0.00 EFA-FTI – 22.00 21.99 99.5 23 Annex 2. Outputs by Component Actual Components Implemented and Their Outputs and Outcomes Project Activities Outputs Outcomes Component 1. Improvement of Provision of one-time Improved access for the equivalent access and equity for primary compensation grants to private of 28 students per school (total of education schools to allow them to restart to 78,000) function following the 2010 Resources for one year to schools Improved access for 93,624 earthquake—2,824 primary schools affected by the earthquake to pay beneficiaries teachers and other expenses Provision of primary education Reduction in overage students Subsidies for students to attend enrollment subsidies (tuition (10%) - Quality nonpublic primary schools waivers to nonpublic schools)— Reduction in repeating students 93,624 primary school students for Support to MENFP to develop and (9%) - Quality one year implement an ECD action plan Targeted subsidies reaching including early childhood centers School selection according to students who are at the economic poverty map average or below (therefore poor) – Equity Component 2. Improved school Financing of school health and Contribution to readiness to learn performance in primary nutrition program for primary and incentive to attend school education school children—62,000 primary No measurable outcome school students for a 110-day School feeding program under school year PNCS to increase the number of beneficiaries of school canteens and Initial training conducted provide Vitamin A supplements and (UNESCO) de-worming. Support to curricular reform Component 3: Rebuilding the Provision of vehicles and Improved ability to perform core capacity of the Ministry of equipment to MENFP responsibilities, school inspections, Education and so on—confirmed by eight of Support for the establishment of an nine departmental directors. Provision of emergency equipment ECD policy purchases (vehicles, office No measurable outcome from the equipment, and generator), as well ECD subcomponent Funding to the PCU to implement as technical assistance to departmental levels of the ministry the CF project Funding for the EFA Technical Unit, which supports the MENFP in the implementation of the Project 24 Annex 3. Economic and Financial Analysis 1. The economic analysis is based on the project costs for three components, namely, teacher compensation, TWP, and school nutrition and health, which account for 95 percent of project funding. The measured benefit is additional wages from additional years of schooling. Assumptions and parameters are presented in table 3.1. Table 3.1. Assumptions of Economic Analysis—Estimated Gains to Educational Attainment by Program Component Estimated Gains to Educational Attainment by Program Component Additional Educational Attainment per Year of Participation in Program Estimate is from original EFA Phase I economic analysis - See Tuition Waivers 0.19 calculation below School Feeding and 0.25 Estimates are from original EFA Phase I economic analysis Deworming School Feeding 0.1 Deworming 0.15 Calculation of Estimated Gains to Educational Attainment from Tuition Waivers Fraction of Subsidized Additional Educational Attainment per Year of Students Participation in Program Extensive Students 0.1 1 Intensive Students 0.9 0.1 Overall (Weighted average) 0.19 Note: Extensive students are students who would have not been in school without subsidies; Intensive students are students who would have been in school without subsidies and will be less likely to drop out. Other Inputs to Calculation Coefficients from Mincer equation (from 2006 CEM) Years of education 0.162 Age 0.127 Age squared -0.0014 Female -0.217 Constant 4.532 Gross domestic product per capita in 2012 (from World US$771 Development Indicators) Average years of education (approximately) 6.000 Life expectancy at age 5 (2011 World Health 62.000 Organization life table) Discount rate 8% 2. Table 3.2 shows the combined net benefit streams from the teacher compensation, school canteen, and TWP programs. The costs are based on total project costs for each of the three programs. The benefits are based on the number of beneficiaries in each of the three programs multiplied by a factor for gains from educational attainment from the TWP/teacher compensation (0.19) and school nutrition and health (0.25) programs. This product yields the total gains from the Project (not yet expressed in monetary terms), which is then multiplied by the years of education 25 coefficient from the Mincer equation (0.162) and then multiplied by the current per capita income in Haiti. The analysis distinguishes between extensive (subsidized students who would have not otherwise been in school) and intensive students (subsidized students who would have been in school and are less likely to drop out) and makes a conservative assumption that only 10 percent of students are extensive and 90 percent are intensive. Table 3.2. Simulated Lifetime Income Stream for Individual with One Additional Year of Education Income With One Year Extra Education Wage Gain (NPV) Year Age Current US$ 0 6 0 -7,277,500.00 -7,277,500.00 1 7 0 -11,705,097.00 -10,838,052.78 2 8 0 0.00 0.00 3 9 0 0.00 0.00 4 10 0 0.00 0.00 5 11 0 -2,000,000.00 -1,361,166.39 6 12 0 0.00 0.00 7 13 0 0.00 0.00 8 14 0 0.00 0.00 9 15 771 5,485,867.67 2,744,299.64 10 16 771 5,485,867.67 2,541,018.18 11 17 771 5,485,867.67 2,352,794.61 12 18 771 5,485,867.67 2,178,513.53 13 19 771 5,485,867.67 2,017,142.16 14 20 771 5,485,867.67 1,867,724.22 15 21 771 5,485,867.67 1,729,374.28 16 22 771 5,485,867.67 1,601,272.48 17 23 771 5,485,867.67 1,482,659.70 18 24 771 5,485,867.67 1,372,833.06 19 25 771 5,485,867.67 1,271,141.72 20 26 771 5,485,867.67 1,176,983.08 21 27 771 5,485,867.67 1,089,799.14 22 28 771 5,485,867.67 1,009,073.28 23 29 771 5,485,867.67 934,327.11 24 30 771 5,485,867.67 865,117.70 25 31 771 5,485,867.67 801,034.90 26 32 771 5,485,867.67 741,698.99 27 33 771 5,485,867.67 686,758.32 28 34 771 5,485,867.67 635,887.33 29 35 771 5,485,867.67 588,784.57 30 36 771 5,485,867.67 545,170.90 31 37 771 5,485,867.67 504,787.87 32 38 771 5,485,867.67 467,396.17 26 Income With One Year Extra Education Wage Gain (NPV) Year Age Current US$ 33 39 771 5,485,867.67 432,774.23 34 40 771 5,485,867.67 400,716.88 35 41 771 5,485,867.67 371,034.15 36 42 771 5,485,867.67 343,550.14 37 43 771 5,485,867.67 318,101.98 38 44 771 5,485,867.67 294,538.87 39 45 771 5,485,867.67 272,721.18 40 46 771 5,485,867.67 252,519.61 41 47 771 5,485,867.67 233,814.45 42 48 771 5,485,867.67 216,494.86 43 49 771 5,485,867.67 200,458.21 44 50 771 5,485,867.67 185,609.45 45 51 771 5,485,867.67 171,860.60 46 52 771 5,485,867.67 159,130.19 47 53 771 5,485,867.67 147,342.77 48 54 771 5,485,867.67 136,428.49 49 55 771 5,485,867.67 126,322.67 50 56 771 5,485,867.67 116,965.44 51 57 771 5,485,867.67 108,301.33 52 58 771 5,485,867.67 100,279.01 53 59 771 5,485,867.67 92,850.94 54 60 771 5,485,867.67 85,973.09 55 61 771 5,485,867.67 79,604.71 56 62 771 5,485,867.67 73,708.07 NPV at 8% 16,649,975.11 Internal Rate of Return 11.8% 27 Annex 4. Grant Preparation and Implementation Support/Supervision Processes (a) Task Team Members Responsibility/ Name Title Unit Specialty Lending/Grant Preparation Peter Holland Senior Education Specialist GEDDR TTL Michael Drabble Senior Education Specialist GEDDR Co-TTL Valandie Kimberly Dorceus Program Assistant LCC8C – Marsha Michel E T Consultant LCSHD - HIS – Dominique Puthod Operations Officer LCSHE - HIS TTL Fily Sissoko Practice Manager GGODR – Yao Wottor Senior Procurement Specialist LCSPT - HIS Procurement Supervision/ICR Patrick Ramanantoanina Senior Education Specialist – TTL Axelle Latortue Education Specialist – TTL Juan Baron Education Specialist – TTL Fabienne Mrozcka FM Specialist – FM Richard Carroll Senior Economist - Consultant – Evaluation Mary Dowling Program Assistant GEDDR – Note: TTL = Task Team Leader. (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle US$, Thousands (including Travel Number of Staff Weeks and Consultant Costs) Lending FY10 0.00 2,939 FY11 0.00 0 Total: 0.00 2,939 Supervision/ICR FY11 0.00 2,380 FY12 0.00 0 FY13 10.26 77,192 FY14 4.67 30,374 FY15 0.00 0 FY16 0.00 0 Total: 14.93 109,946 Note: Most preparation and supervision costs were borne by non-Bank Budget trust funds. 28 Annex 5. Beneficiary Survey Results 1. A survey was conducted by DDEs regarding equipment provided under Component 3 of the Project. This equipment was intended to improve governance of these departmental units by increasing their capacity to perform their basic functions including school inspections and collecting and analyzing data on schools. Almost all DDEs reportedly received the full range of equipment from vehicles and motorcycles to computers, laptops, printers, battery inverters, and flash drives. Table 5.1 summarizes the results of the survey. Nine of the ten departments responded. Table 5.1. Responses to Survey Concerning Equipment Provided to DDEs Question Yes No Have you personally used the vehicles/equipment? 4 5 Would you have been able to perform the same tasks without the equipment? 1 8 Is this vehicle/equipment still in use? 8 1 Have you maintained the vehicle/equipment? 9 0 Is there a replacement plan? 2 7 2. There were also questions on how the vehicles/equipment were used and what additional advantages they provided to the unit. The responses included the following: • Improved ability of school inspectors to get to schools, collect data, and analyze back at the office - three similar responses • Timely visits by inspectors and timely reports – one response • Overall support to the mission of the unit (which is to supervise schools and conduct MENFP activities at the departmental level - four similar responses • Continued operation during power failures (battery inverters) – one response 29 Annex 6. Stakeholder Workshop Report and Results Not applicable. 30 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR EDUCATION FOR ALL PROJECT (EFA) CATALYTIC FUND DONOR AGREEMENT TF097009 PROJECT MANAGER : GOVERNMENT OF HAITI – MENFP IMPLEMENTATION COMPLETION REPORT JULY 2010 – OCTOBER 2015 Contact Project Coordinator: Norbert STIMPHIL Telephone: (509) ……………. / ……………. Email: norbertstimphil@yahoo.fr Website: ept-menfp.ht Date of Project Completion: January 31, 2016 31 Table of Contents Page INTRODUCTION…………………………….…………………………………………………4 EXECUTIVE SUMMARY…………………..………………………………………….….…..5 1. 1. CONTEXT OF THE PROJECT LAUNCH……………………………………………………………………………………..….6 1.1 Economic and social conditions ……………………………………….………………….....6 1.2 Priorities in the Education sector..…………………………………………………………………………………………....6 1.3 Activities of World Bank and other donors in this sector…………………………..……..…7 2. IMPLEMENTATION OF THE PROJECT.…………………………………………………………….…………….…………..8 2.1 Implementation arrangements - Division of labor ……………………………….……….…8 2.2 Quality of project preparation ………….……………………………………………………8 2.3 Monitoring and Evaluation and impact of these processes …………………………..……...9 2.4 Re-structuring and Changes of Objective ………………………………………….…….….9 3. OUTCOMES AND EVALUATIONS……………………………………………………....10 3.1 Relevance of project design and objectives within the context of national priorities ………10 3.2 Outcome indicators within the context of the project’s development objectives ….……......10 4. PERFORMANCE OF THE WORLD BANK AND THE HAITIAN GOVERNMENT..13 4.1 Performance of the World Bank - Preparation and supervision …………………….………13 4.2 Performance of the Haitian Government.…………….……. ……………………….………14 5. DURABILITY AND SUSTAINABILITY……………………………………..…………..15 5.1 Durable and sustainable measures.………………………………………………………… 15 5.2 Non-durable and non-sustainable measures …………………….………………………… 15 5.3 Measures proposed to improve durability ………...………………………………………..15 6. LESSONS EARNED….………………………………………………………….................16 7. ANNEXES………………………………………………………….……………….……….17 32 List of Acronyms CIDA Canadian international Development Agency AFD Agence Francaise de Development (French Agency for Development) CDB Caribbean Development Bank IDB Inter-American Development Bank PAD Public Administration Directorate DSPEP Directorate for Support to the Private Education Sector and Partnerships DED Departmental Education Directorate SNDGPV Strategic National Document for Growth and Poverty Reduction EFA Education for All ESRP Emergency School Reconstruction Project ABE Accelerated Basic Education NSF National Sponsorship Fund IDA International Development Association FTII Fast-Track implementation initiative MENFP Ministry of Education and Vocational Training NCEC Noel Cabinet d’Experts-Comptables (Noel Audit Consulting Firm) NSFP National School Feeding Program NAS/EFA National Action Strategy/Education For All TPU Technical Project Unit UNESCO United Nations Organization for Education, Science and Culture USAID United States Agency for international Development INTRODUCTION 33 In 2007, the Haitian Government undertook the commitment to meet the required objectives for a high standard of basic education for all by the year 2015, following the recommendations of the “Dakar Framework for Action”. To this end, a National Action Strategy to provide Education for All (NAS/EFA) was drawn up and the Education for All project was established with the aim of improving access to basic education for children between the ages of 6 and 12, promoting equity, and improving the standard of education and the governance of the Haitian education system. In September 2009, an additional fund of US$22 million called the “Catalytic Fund” was approved by a committee representing several donors supporting the EFA program, with the aim of expediting, expanding, and acting as a catalyst for the implementation of EFA, supplementing the multi-donor trust fund already in place. The grant agreement between the IDA and the Haitian Government had not yet been signed when, on the January 12, 2010, the country was hit by an earthquake, as a result of which the entire education system was thrown into disarray. In order to meet the demands of the new situation, an extensively amended proposal was drawn up and validated by the MENFP and the funds were re-allocated in April 2010. The components of the fund were further revised in September 2011 in the context of the re-programming of the available balance at the end of the “Kout Pous Pou Tounen Lekòl” (“Helping Hand to Get Back to School”) scheme, which itself had been the beneficiary of the first re-allocation. This document is the final report on the Catalytic Fund and covers the implementation period from July 15, 2010 to October 31, 2015. It gives an account of what the fund has accomplished, providing information on the achievement of targeted outcomes and the success of the various activities. At the same time, it aims to generate a constructive discussion for all actors, whether directly or indirectly involved in the project, specifically by highlighting the lessons learned about issues related to the development of the education system in Haiti. 34 EXECUTIVE SUMMARY The Catalytic Fund was launched during a particularly difficult post-earthquake context characterized, inter alia, by the disruption of the education system throughout the country. The MENFP had to adopt emergency measures to ensure that children did not lose the entire 2010 school year and that the 2010-2011 school year could begin as normal. It is against this background that the Catalytic Fund was revised in April 2010 to enable the “Kout Pous Pou Tounen Lekòl” component to be launched, thus allowing 68 percent of the total amount of US$22 million to be allocated to subsidies for non-State schools affected by the earthquake. The Catalytic Fund was initially planned to extend from 2009-2012, but it was revised to run from 2010-2015. The accomplishments as measured against the targets set are as follows: • Funding of 2,824 non-State schools out of a planned total of 3,600 for a total of US$7,060,000 in 2010-2011 paid as compensation to teachers and to ensure resumption of educational activities in these establishments; • Funding of the school subsidy scheme in the amount of US$10,021,758 in 2011-2012 and 2014-2015 from which 93,624 needy children benefited; • Funding of school feeding programs in the amount of US$3,619,209.24 which benefited 62,000 students in 2010-2011 and 2011-2012; • Drafting and validation of the National Policy on Early Childhood and the establishment of the National Commission on Curriculum Reform; • Provision of vehicles and motorbikes, generators and computer equipment to the MENFP and its decentralized facilities. The sub-component entitled “Support for the development of teaching materials”, whose aim was to lay the institutional foundation for curriculum reform, did not achieve its intended goals. The same is true for the sub-component on the “Establishment of Services for Early Childhood Development”, the Action Plan for which had to be cut by 50 percent because certain planned activities were ineligible. The Catalytic Fund was closed on October 31, 2015, and according to information provided by the TPU/EFA and the final audit, the entire sum of US$21,999,969 obtained from the World Bank for this project was disbursed. Overall, the results can be considered to be moderately satisfactory despite the difficulties encountered in implementation and the various restructuring operations. The World Bank’s performance is assessed as moderately satisfactory both as regards the level of preparation and the supervision of the Catalytic Fund. The performance of the Haitian Government is considered satisfactory while that of its partners is judged to be moderately satisfactory. Several lessons have emerged from the implementation of the Catalytic Fund, most notably regarding the efforts necessary to plan and supervise subsidy operations and the emphasis to be placed on the management of partnerships in order to achieve sustainable outcomes. 35 1. CONTEXT OF THE PROJECT LAUNCH 1.1 Economic conditions After the food crisis of April 2008 which led to hunger riots and developed into a political crisis, Haiti was struck, in less than a month - between August 16 and September 7, 2008 - by four hurricanes and tropical storms: Fay, Gustav, Hanna and Ike, which left a death toll of 200 and huge economic losses totaling more than US$23 million in dead cattle and devastated crops and agricultural infrastructure. With a 2.9 percent growth rate in 2009, the country was pulling itself painfully out of this difficult economic situation when it was struck by an earthquake on January 12, 2010, which exacted a particularly heavy human, economic and financial toll. It is estimated that 250,000 people died and that financial losses amounted to US$10 billion, i.e. 120 percent of the country’s GDP. The Catalytic Fund had been devised for the 2009 economic situation but was finally launched in the post-earthquake context at a time when Haiti was still reeling from a tremendous shock to its economy and society. The education system was severely affected, not only in those departments directly hit by the earthquake, viz West, South-west and Nippes, but also in others affected by the mass movement of people displaced due to the earthquake. 1.2 Priorities in the Education Sector When the Catalytic Fund was designed, the priorities for the Education Sector were de-fined in the National Strategy Document for Growth and Poverty Reduction (NSDGPR) drafted in November 2007 and covering the period 2008-2010 and in the National Action Strategy for Education For All (NAS/EFA). The 2010-2015 Operational Plan for Education and Training was drafted by the MENFP in June 2010 and issued in May 2011. The NSDGPR, the successor to the Interim Cooperation Framework (ICF), reflected the Haitian Government’s determination to find the appropriate responses to the major challenges of economic growth and poverty reduction. In Pillar 2, which focused on human development and access to basic social services, the Education and Training segment called for:  A Policy to reorganize the provision of education to benefit poor children  A Policy to upgrade the standard of Teachers and School Principals  A Policy providing a proper supervisory framework for Schools, Students and Parents  A Policy to improve coordination between the various protagonists in the education sector  A Policy to increase public funding for the education sector The NAS/EFA, drawn up in September 2007 with the aim of achieving Education for All by 2015, identified the following major strategic options to be promoted: 36  Greater equity in early childhood development and protection;  Greater equity in formal and non-formal education;  Greater internal effectiveness within the basic education system;  Greater external effectiveness of the system;  Effective, high-caliber management. The 2010-2015 Operational Plan of the MENFP, prepared by the education and training Working Group, consisted of eight strategic pillars and included governance, educational programs and curriculum and the early childhood. These pillars, among others, were the focus of actions undertaken under the Catalytic Fund. 1.3 Activities of the World Bank and other donors in this sector The World Bank has a strong presence in the Haitian education system, particularly in the implementation of the NAS/EFA operational plan. It not only provides funding for EFA and ABE projects within the Emergency School Reconstruction Project (ESRP) but is also the supervising entity responsible for the coordination of funds allocated by several donors. Its main area of focus is early childhood and basic education. Among the other donors, the IDB also plays a preeminent role by contributing to the funding of the EFA project and financing the construction of school buildings. Its main focus is on improving standards of education, teacher training, and increasing school enrolment. The Canadian International Development Agency (CIDA) concentrates mainly on primary education, and the European Union (EU) funds the professional development of teachers, the renovation of classrooms and a pilot project for over-age students in several regions. USAID, for its part, having funded the first ToTAL reading project, has just approved more than US$25 million to fund the Ann ALE reading and writing project. The other donors are: the French Agency for Development (AFD) which supports the management of human resources in the MENFP, the Spanish Agency for International Cooperation and Development (AECID), the Swiss Cooperation agency, the World Food Program (WFP), and UN-specialized agencies (UNESCO, UNICEF). UNICEF gives priority to supporting the State education system in order to expand the provision of education to the most needy and vulnerable and supports the MENFP in its efforts to guarantee access and the universal right to education for all Haitian children. 2. IMPLEMENTATION OF THE PROJECT 2.1 Implementation arrangements - Division of labor The Catalytic Fund was designed to be implemented using structures already in place within the framework of the EFA project. It is therefore the EFA Coordinator, supported by the officials in the Technical Project Unit (TPU) which had already been set up, and by the Steering Committee 37 for the EFA project (consisting of the technical directorates involved in the implementation of the components of the project under the guidance of the Minister) who was responsible for implementation. Specifically, in the case of the Catalytic Fund, the TPU/EFA played a pivotal role in the execution of the project, especially after the 2010 earthquake. Monitoring and evaluation were the responsibility of a TPU technician, and a consultant - under the direct supervision of the EFA Coordinator - was hired as the focal point of “Kout Pous Pou Tounen Lekòl”. The Steering Committee for the project, of which the above-mentioned focal point was also a member, ensured that the project stayed on track, that the timetable was respected and that the resources made available by the MENFP were properly utilized. In addition, mention must be made on the one hand of the important role played by the DEDs, particularly in supervising the subsidy exercise and the school feeding program, and on the other, the audit and oversight performed by two firms, FPN and Noel Audit Consultancy (NCEC) on payments received and expenditures made by the subsidized schools. The financial management arrangement for the Catalytic Fund was largely the joint responsibility of the EFA project and the World Bank. The World Bank, as manager of the multi- donor trust funds for the establishment of the EFA project, disbursed monies to the EFA accounts which in their turn were directly managed by a financial team consisting of a Finance Chief, an Assistant and two Accountants. The EFA financial reports, also covering the Catalytic Fund, were produced on a quarterly basis by the EFA and submitted to the World Bank. An audit of the financial management of the Catalytic Fund was performed annually. Public procurement was handled, as far as the Catalytic Fund was concerned, according to World Bank rules as applied to the EFA project. A local financial institution was empowered to act as financial intermediary on behalf of the MENFP for the purposes of “Kout Pous Pou Tounen Lekòl”. 2.2 Quality of Project Preparation The preparation process for the Catalytic Fund took place in a difficult context since the building which housed the MENFP had collapsed, which meant that the Ministry’s administrative capacity was significantly reduced. However, overall, the preparation was satisfactory. The World Bank worked very closely with the Haitian government to facilitate re-allocation of budget resources and find the best re-allocation options. As had been the case for the EFA project, the government wished to resort to the Catalytic Fund to strengthen the MENFP, and the World Bank was in agreement with that approach. The TPU, which was already in place, was used to facilitate the implementation of activities financed by the Fund. The budget re-allocation in the wake of the earthquake entailed certain changes in the preparation process, specifically for the implementation of “Kout Pous Pou Tounen Lekòl”. The team working on the preparation of the Catalytic Fund took account of lessons learned from activities already completed within the framework of EFA. Specifically, it had been observed that 38 public-private partnerships represented an under-exploited potential for improving access to social services, allowing the government to play a regulatory, financial and supervisory role with respect to the school feeding program while leaving the actual provision of the service to the non-State sector, and that there was a need to de-politicize hiring procedures, particularly as regards the post of Coordinator, in order to ensure stability in the management of the Fund. 2.3 Monitoring and Evaluation and Impact of these processes A framework for monitoring and evaluation was designed, together with indicators enabling progress on the various components of the Catalytic Fund to be assessed. The framework with the outcomes is to be found in Annex 2. The monitoring process was facilitated by semi-annual progress reports on the implementation of the components submitted to the IDA and the CDB. These semi-annual reports, prepared by a specialist in monitoring and evaluation within the TPU/EFA, are in fact EFA reports which include information on the Catalytic Fund. Each semi- annual report reviews progress on the various components and sub-components of the project, on the implementation of the Annual Operational Plan and the Annual Public Procurement Plan. The outcome indicators relating specifically to the Catalytic Fund were amended during the first re-structuring process after the earthquake in 2010 and during the second re-structuring process on the completion of “Kout Pous Pou Tounen Lekòl”. All the indicators referred to in this final report are those directly related to the final re-allocation of September 2011. 2.4 Restructuring and Changes of Objective The restructuring process which took place in 2010 significantly modified the entire thrust of the Fund, in that the school subsidies program was replaced by an emergency support scheme for non- State schools. However, the budget re-allocation in September 2011, which transferred a balance of US$7,490,000 from “Kout Pous” to the school subsidies program, to some extent realigned the Fund’s activities with its original objectives. Moreover, the fact that the “Early Childhood” sub- component of the Action Plan was curtailed also implied a change in objective as regards the level of implementation compared with initial funding. Revised Target Revised Target Indicator Original Target Apr. 2010 Sep. 2011 et seq Number of schools benefitting - 3,600 NA from Kout Pous scheme Number of students benefitting 135,000 0 83,000 from school subsidy program % implementation of early childhood development activities 100 100 46.65 (compared with initial funding) 39 3. OUTCOMES AND EVALUATIONS 3.1 Relevance of project design and objectives, in the context of national objectives The Catalytic Fund was designed and implemented to support activities already underway within the framework of EFA, which is the implementation platform of NAS/EFA. The Fund’s various components were intended to improve access to basic education for poor children between the ages of 6 and 12 in the interests of equity, and to improve standards and strengthen governance within the education sector in Haiti. These components were defined taking account of the experience of the EFA project and the education strategy defined by the government in the NAS/EFA based on the objectives set out in the Dakar Declaration. The Fund was a direct response to the Dakar Declaration’s objectives of early childhood protection and education and compulsory free education, which also form part of the NAS/EFA. The Catalytic Fund’s design was highly relevant to national priorities both before and after the re- structuring of its components. The amendments made to the components after the earthquake did not shift the overall thrust of the Fund away from NAS/EFA objectives. The support given to non- State schools was also relevant, given the specific context and the risks, and the need to facilitate resumption of school activities. The reallocation in September 2012 of half the amount earmarked for “Kout Pous” to the school subsidy scheme had the effect of reinstating the objective of improving access to and promoting equity in primary education, and thus of increasing the Fund’s own relevance in the wake of the first re-allocation exercise. 3.2 Outcome indicators in the context of the project’s development objectives 3.1.1 Component 1: Improving access to and promoting equity in primary education • Sub-component 1.1: Kout Pous - Support to non-State schools in areas affected by the earthquake The scheme to provide support to non-State schools in areas affected by the earthquake, known as “Kout Pous”, lasted for one year, from August 2010 to July 2011, and benefited 2,824 schools. It was not easy to quantify the number of students benefitting, since the scheme focused on schools, and therefore information collection efforts also concentrated on the number of beneficiary schools, which was considered the key indicator for this sub-component. The initial target of “Kout Pous” was to reach 3,600 schools, but the lack of reliable data made it difficult to be certain as to the exact number of schools that there were to be subsidized. The target of 3,600 was retained nevertheless, but checks revealed that the number of schools eligible for the scheme was only 2,824. In the final analysis, the amount disbursed under this sub-component was US$7,060,000 covering the cost of the subsidies as well as fees paid to the financial institution (Fonkoze) and to 40 independent firms which collected data on schools which were potential beneficiaries under the scheme. Commentary on Indicator Target Real Performance Performance Satisfactory. Comparison Number of schools with target does not receiving subsidies to translate to performance 3,600 2,824 78.5% pay teachers since target is estimate compensation which must be verified after evaluation. • Sub-component 1.2: Subsidy for school fees per student At the outset, the aim of this sub-component was to provide free basic education for three years to 135,000 students, with an initial outlay of US$12.5 million. After the earthquake, this money was allocated to the “Kout Pous” scheme, which however only absorbed half of that amount. The remainder - US$7,040,000 - was therefore re-allocated to the school subsidy program, the aim being that 83,000 students should benefit during the 2011-2012 school year. The unspent balances from other components (curriculum reform, early childhood services etc.) were also re-allocated to this sub-component. In 2014-2015, for example, US$1,998,891 million was transferred, from which approximately 20,500 children benefited. Performan Commentary on Indicator Target Real ce performance Satisfactory. Account taken Number of children of re-allocation in Sep 2011 benefitting from school 83,000 104,095 125% and of additional funding in subsidy 2011-2012 2014-2015. • Sub-component 1.3: Establishment of early childhood development services The aim of this sub-component was to fund pre-schools and provide health and nutrition services. There were delays in programming, because the MENFP was late in approving the proposals made by the Office for the Management of Pre-school Education (BUGEP) and by UNICEF. Moreover, the building and operation of community preschools, which represented 53.35 percent of the budget for this sub-component, were not eligible activities under this project. When the Catalytic Fund was closed, only US$180,214.61 had been spent. The National Early Childhood Policy had been developed and validated by the MENFP and public awareness campaigns had taken place. The following indicators take account of the cuts made to the action plan for this sub-component due the non-eligibility of certain activities. 41 Performan Commentary on Indicator Target Real ce performance Document on National Early Childhood Policy 1 1 100% Satisfactory drafted and validated Public awareness campaign on early 1 1 100% Satisfactory childhood issues Implementation of action plan after curtailment of 100 47 39% Moderately unsatisfactory activities 3.1.2 Component 2: Food security for children attending school • Sub-component 2.1: School nutrition and health program Under this sub-component, the NSFP was intended to increase the number of children receiving school meals, based on existing contracts with companies providing this service. This US$4 million program, therefore, was to reinforce the existing program under EFA by providing meals for an additional 50,000 students, and pay for a deworming campaign during the 2009-2010 school year. Despite the fact that this school health and nutrition program financed by the Catalytic Fund was launched one year late, 62,000 students benefited from it over a 2-year period, 17,000 in 2010- 2011, and 45,000 in 2011-2012. Performan Commentary on Indicator Target Real ce performance Number of students receiving school meals in 50,000 62,000 124% Satisfactory 2009-2010 • Sub-component 2.2: Support for the development of teaching materials appropriate for post-disaster situations The implementation of this US$1 million sub-component, for which UNESCO and the MENFP had joint responsibility, and whose aim was both to devise a curriculum which would be appropriate for the aftermath of a natural disaster and to provide continuous training for teachers, was considerably delayed, and did not achieve the expected outcomes. The decision to focus on curriculum reform, the preparation of the contract between the two parties and the consultations with the World Bank led to delays of an administrative and technical nature which substantially slowed the whole process. The contract between the MENFP and UNESCO was finally signed in September 2013, but was not renewed in July 2014. At the end of the day, 12 MENFP officials were trained to take charge of monitoring the reform process, the Directorate for a High-Quality Curriculum (DHC) was able to purchase some materials, and the National Commission on 42 Curriculum Reform was established in December 2014. When the Catalytic Fund was closed, only US$47,996.12 had been spent, once UNESCO had returned US$377,487.88 of the US$425,484 received. Performan Commentary on Indicator Target Real ce performance National Commission on Yes Yes 100% Satisfactory Curriculum Reform Laying of institutional foundation for 100 4.8 4.8% Unsatisfactory curriculum reform 3.1.3 Component 3: Rebuilding capacity at the Ministry of Education Under the Catalytic Fund, an amount of US$1 million was earmarked for the MENFP for institutional reinforcement, which is a central objective of the EFA project, as an additional contribution to funding already allocated by other donors to strengthen the governance of the education system. Within the context of the April 2010 reallocation of funds, and given the emergency situations to be addressed as a result of the earthquake, the various activities initially planned under this sub-component were revised, and the money transferred to the purchase of equipment and vehicles. A sum of US$756,658 was spent on acquiring 100 motorbikes, 13 vehicles, as well as computers and generators for the MENFP and its various decentralized offices. A breakdown of the distribution of these items is provided in Annex 5. The remaining funds were spent on activities in support of the establishment of the National Office for Education Partnerships (ONAPE). Performan Commentary on Indicator Target Real ce performance Implementation of plan to rebuild MENFP 100 107 107% Satisfactory capacity post- earthquake (in %) 4. PERFORMANCE OF THE WORLD BANK AND THE HAITIAN GOVERNMENT 4.1 Performance of the World Bank – Preparation and Supervision The performance of the World Bank in the preparation and supervision of the Catalytic Fund has been deemed satisfactory. 43 Through the Catalytic Fund, the IDA supported a project that was very closely identified with the priorities of the NAS/EFA. Thanks to the project design, reallocation after January 12, 2010 was made easier following the change in circumstances, making it necessary to earmark a significant share of the funds for financing non-State schools. The Bank played a central role in the restructuring process and in reviewing the actions and targets as well as the “no-objection’ procedure. There were regular supervision missions, which led to important recommendations for improving the way the Fund was run. The World Bank provided invaluable support and the recommendations made served to enhance the procurement process as well as the financial management of the project. However, responses to requests for no-objections were sometimes too long in coming. Furthermore, the MENFP was often of the opinion that the Bank was too closely involved in the direct management of the project, an overly interventionist approach that led to misunderstandings between the donor and the project manager. 4.2 Performance of the Haitian Government The performance of the Haitian Government in the implementation of the Catalytic Fund was deemed satisfactory, while that of its implementation partners was considered to be moderately satisfactory. The Catalytic Fund drew from the experiences acquired in the implementation of the EFA, already underway in the Ministry. Furthermore, the launch of the project was enhanced by input from a technical team that had already gained some experience. Between September 2010 and September 2015, ten semi-annual technical reports were drafted as part of the EFA, including information from the Catalytic Fund. The procurement process in the management of the Catalytic Fund was conducted satisfactorily. The norms and standards of the Bank were adhered to in the issuance of purchase contracts as well as consulting contracts. A Procurement Plan (PP) was developed each year for the implementation of the EFA project activities, including activities pertaining to the Catalytic Fund. Financial management was adequate and despite shortcomings in internal controls, the semi-annual external audits have yielded substantively satisfactory results. The Auditors have unreservedly attested to the fact that the resources were used for their intended purposes, as per the Donor Agreement. The Government of Haiti provided effective support during project implementation and displayed flexibility whenever difficulties arose. The necessary measures were taken to streamline administrative procedures and expedite the implementation of project activities. A number of entities were contracted and the services of others utilized to facilitate the implementation of key, substantive activities. These entities included the TPU, the Consultants, service providers, 44 UNESCO, the NSFP, etc. While some of these groups provided satisfactory services, others did not. 5. DURABILITY AND SUSTAINABILITY 5.1 Durable and sustainable measures There is no doubt that the establishment of the institutional groundwork for curriculum reform has been a vital sub-component of the Catalytic Fund and has been in keeping with a long term vision. The same principle applies to the strategy for the implementation of an early childhood policy and its action plan. The training of leaders in curriculum reform, the strengthening of the relevant directorates, the establishment of the national reform commission and the launch of the Early Childhood Action Plan are all activities that have been developed with a view to their being continued over the long term by the Ministry, once they have been initiated. Furthermore, more thought should be given to completing those activities that are still pending completion. 5.2 Non-durable and non-sustainable measures With respect to the Catalytic Fund, the subsidy granted to non-State schools for the compensation of teachers could only be a temporary arrangement arising from an exceptional crisis situation. This arrangement could not be sustained over time, since this is not one of the roles or functions of the Ministry. The tuition waiver program and the school feeding program have had a very positive impact on school enrollment. However, the government of Haiti is unable to continue this program without financial support from the donors. 5.3 Measures proposed to improve durability As part of the institutional strengthening of the Ministry, World Bank commitment to providing administrative support should be extended over a longer period, to enable the MENFP to better incorporate and develop techniques of financial management and procurement, in order to ensure sustainability over time. Direct external technical assistance should be provided to effect the smooth and efficient handover of the management of administrative affairs. Similarly, the Ministry, in consultation with the donors, should undertake the necessary measures for the institutionalized reform of the curriculum, for developing a curriculum framework, drafting the corresponding public policy document and establishing a common base of knowledge and skills. 45 6. LESSONS LEARNED  Actions undertaken under the Catalytic Fund and which are more likely to be sustainable over time have been the most difficult activities to implement. Special care must be taken to ensure that such actions will be sustainable in the long term, in view of the many challenges involved in coordinating the relevant stakeholders during the initial implementation stages. These are often institutional strengthening activities that require the long term commitment of the partners.  Experience in implementing the Catalytic Fund has shown that planning and supervision are indispensable when providing funding for non-State schools, as this may lead to the emergence of new schools of mediocre quality, established with the sole aim of attracting funding.  The Catalytic Fund was not created to function in an emergency context. However, it has been revised, largely in response to an emergency situation and is intended for use in the context of an emergency. Even before the emergency in Haiti, there was a lack of reliable information on the situation of schools within the country and this led to problems in planning and implementing necessary corrective actions. Rigorous and reliable databases for the educational sector should be developed in order to ensure swift and effective intervention in the event of an emergency.  With the predominance of private schools in Haiti, the tuition waiver program and the school nutrition and health program have contributed significantly to lowering the supply-demand ratio in primary school education, especially for the poorest children.  The establishment of the Technical Project Unit (TPU) was a successful strategy that enabled the Catalytic Fund to undertake major activities in the post-disaster situation of 2010. The management of the fund was restricted to a small unit of qualified and highly placed professionals, working under the aegis of the Ministry. This proved to be a successful strategy for implementing the activities of the project.  A strengthened partnership for service delivery is an effective means of ensuring improved access to education in Haiti, as the Ministry by itself would have been unable to effectively provide such services. The partnership between non-State schools, NGOs providing school meals, institutions for managing finances and assessing subsidies, etc., has been highly effective for the implementation of the Catalytic Fund.  Whenever a partnership is established to undertake fundamental structuring activities, time and energy devoted to seeking consensus among all interested stakeholders is never wasted. Seeking and obtaining consensus among stakeholders on the best approaches to be taken will increase the chances of success of the activities undertaken. 46 The management of the sub-components for curriculum reform and the early childhood action plan under the Catalytic Fund is an indication that this lesson has not been learned.  Despite institutional shortcomings within the public sector, projects should be managed by structures that are already in place, even though such entities may need to be strengthened. This is preferable to systematically creating provisional structures that are disbanded once the project is completed. 7. ANNEXES Annex 1. Costs and sources of funding of the Catalytic Fund Annex 2. Results framework of the Catalytic Fund Annex 3. Output by component – Catalytic Fund Annex 4. Outline of technical and budgetary re-allocations of the Catalytic Fund Annex 5. List of materials provided to the Ministry and decentralized services Annex 6. Mid-term evaluation report of the Catalytic Fund 47 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders No Comments were received. 48 Annex 9. List of Supporting Documents 1. Action Plan for National Recovery and Development of Haiti, March 2010. 2. Adelman, Melissa A. and Holland, Peter A. January 2015. “Increasing Access by Waiving Tuition-Evidence from Haiti.” Research Working Paper 7175, Education Global Practice Group, World Bank, Washington DC. 3. Catalytic Fund Program Document on a Proposed Grant from the Education for All—Fast Track Initiative Catalytic Fund in the Amount of US$22 Million Equivalent to the Republic of Haiti for a Haiti Education for All Fast-Track Initiative Catalytic Fund Project, July 10, 2009. 4. “Children of Haiti-Three Months After the Earthquake-Process, Gaps and Plans in Humanitarian Action Supporting a Transformative Agenda for Children,” UNICEF, April 2010. 5. Haiti Education for All Multi-donor Trust Fund Grant Agreement, June 10, 2010. 6. Implementation Completion and Results Report (IDA-H2860 IDA-H5880) on a Grant in the Amount of SDR 24.8 million (Including Additional Financing) (US$37.0 million equivalent) to the Republic of Haiti for an Education for All Project in Support of the First Phase of the Education for All Program, November 30, 2012. 7. “Investing in People to Fight Poverty in Haiti, Reflections for Evidence-Based Policy- Making,” World Bank Poverty Assessment, 2014. 8. ISRs and other documents in the Project file 9. National Action Plan for National Recovery and Development of Haiti—Immediate Key Initiatives for the Future, March 2010. 10. National Education Strategy/Education for All (SNA/EPT) 49 Annex 10. Origins of the Haiti EFA-FTI CF Project 1. In April 2008, the global EFA-FTI endorsed Haiti as a member state, which opened Haiti’s access to EFA-FTI funds to support implementation of its sector strategy (SNA/EPT), which had been adopted the previous year (May 2007). 2. During May to September 2008, the MENFP worked with the LEG to produce a US$22.0 million grant proposal to the EFA-FTI CF Project. UNESCO was the coordinating agency, and the MENFP and the LEG requested that the Bank be the SE. The US$22 million was to be channeled from the EFA-FTI CF Project through a planned Haiti EFA MDTF. This MDTF was to be created with the expectation that other donors will contribute and funding will reach a target of US$50 million (and will fund other activities in the education sector as well). 3. After the proposal to the EFA-FTI CF Project was produced (conforming to the guidelines issued by the Bank for processing requests of the EFA-FTI CF Project), UNICEF intervened, maintaining that it was the appropriate institution to be the Managing Entity for this type of proposal, arguing that it should implement education interventions in fragile states. A UNICEF- managed global TF, rather than the MDTF, would have received the financial support from the EFA-FTI CF Project. A CF High-Level Committee meeting decided (in September 2008) to comply with UNICEF’s request and that Haiti should instead seek funding from this TF. 4. The TF, however, was never established, nor was UNICEF able to take the necessary steps to become the Managing Entity for the Project. As a result, the Bank and UNICEF agreed that grant preparation would have continued under the original CF preparation arrangements. The Bank and MENFP produced the CF Program Document for a grant from the EFA-FTI CF Project (dated July 10, 2009). The High-Level Committee of the EFA-FTI CF Project approved the grant in September 2009. Thus, the Project incurred a delay of one year since the TF option was put forward by UNICEF and was accepted by the CF High-Level Committee. The Bank Trust Fund proposal to establish the MDTF was approved by the Bank on December 11, 2009. 5. On January 12, 2010, a massive earthquake struck Haiti, which led the MENFP to request that EFA-FTI funds (which had not yet been negotiated with the GoH) be reallocated toward post- earthquake activities. In this reallocation, funding was shifted from tuition waiver to, among others, a short-term, one-time school grant for teacher compensation. This was an emergency measure to encourage teachers not to abandon their schools so that students will be able to continue their classes (the initiative called for was a US$15 million grant with the Creole name of Kout Pous Pou Tounen Lekol and consisted of individual school grants of US$2,500). This change was requested by the MENFP through the “IMOA-EPT Financement du Fonds Catalytique pour Haiti—Fonds Fiduciare Multi-bailleurs (FFMB)—Proposition de réallocation—11 Mars 2010” (referred to as the Technical Note). These changes were approved by the EFA-FTI CF Project (March 18, 2010) and accepted by the Bank (April 9, 2010—see page 2 footnote 5). The CFPTP gave clearance to the task team to amend the MDTF TFP accordingly. 6. The grant package, which contained a draft GA, did not contain the July 10, 2009 PAD, or the Technical Note, but did include the TFP. The package was cleared by Bank legal department and negotiated between the Bank and the GoH. The GA has a development objective and an activities description, which are broader than the activities financed by the US$22 million project 50 because additional funds were expected to be provided to the MDTF to fund the other activities. 17 The trust fund GA is dated June 10, 2010, and the Project was declared effective on July 15, 2010. 17 The GA did not refer either to teacher compensation or to tuition waiver. Instead, it used the more general term ‘Student enrollment program,’ which encompassed both interventions. 51 Annex 11: Personal Testimonies of Gratitude by Beneficiaries of the TWP Témoignage 1-A Trou-du-Nord, Mme Gaspard Généus a confié: Regarde mon visage, ça rayonne. J’estime que je profite plus du programme que mes enfants eux-mêmes. Ma condition de vie s’est considérablement améliorée depuis que l’EPT me permet d’éduquer mes enfants gratuitement. Je ne sais comment les remercier pour tout cela. Dieu seul saura les payer en retour Témoignage 2 : Le témoignage de M. Auguste Mélidor, un parent vivant dans le Nord-Est, est symptomatique des sentiments des parents à l’égard du programme: Je suis un ex-membre des Forces Armées d’Haïti. Quand le corps a été dissout, je me suis retrouvé au chômage du jour au lendemain. Je n’arrivais plus à payer l’écolage de mes deux garçons. Un jour, j’étais au champ quand je les ai vus arriver aux heures de classe. Ils m’ont dit qu’ils ont été renvoyés pour défaut de paiement. Je me suis alors précipité à la direction de l’école et j’ai promis au directeur de régler les comptes dans deux semaines. Le délai étant passé, je n’arrivais toujours pas à payer la somme dû et les enfants ont de nouveau été renvoyés. J’avais le cœur brisé quand l’aîné m’a dit ceci: Papa, perdre quelques jours de classes ne me perturbe pas. C’est le fait de laisser la classe la tête baissée sous le regard des copains, c’est cela qui me terrasse. Je me sens humilié à chaque fois. Je me suis depuis lors résolu à faire n’importe quel boulot pour éviter que mes enfants ne subissent à nouveau de telles humiliations. Aujourd’hui, j’ai encore deux enfants à l’école fondamentale et je n’ai toujours pas un emploi stable. Mais grâce au programme de subvention, je suis heureux de n’avoir plus à faire des boulots à priori humiliant pour sauver l’honneur de mes enfants. Merci ! 52 MAP 53