Document of The World Bank Report No: ICR2592 IMPLEMENTATION COMPLETION AND RESULTS REPORT (TF-92755) ON A EDUCATION FOR ALL FAST TRACK INITIATIVE GRANT IN THE AMOUNT OF US$13.9 MILLION TO THE REPUBLIC OF SIERRA LEONE FOR AN EDUCATION SECTOR DEVELOPMENT PROJECT March 22, 2013 Africa Education Sector Unit for West and Central Africa (AFTEW) West Africa Department 1 Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective January 17, 2013) Currency Unit = Leone (SLL) US$ 1.00 = SLL 4,300 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS CAS Country Assistance Strategy CSO Civil Society Organizations CSR Country Status Report CCTT Child Centered Teaching Techniques DP Development Partners EFA-FTI Education for All-Fast Track Initiative EPDF Education Program Development Fund ESMF Environmental and Social Management Framework ESP Education Sector Plan ESSF Education Sector Support Fund FM Financial Management GDP Gross Domestic Product GER Gross Enrollment Rate GIR Gross Intake Rate GPE Global Partnership for Education IFR Interim Financial Report IPRSP Interim Poverty Reduction Strategy Paper ISDS Integrated Safeguards Data Sheet JSS Junior Secondary School LC Local Councils MDG Millennium Development Goals MEST Ministry of Education, Science and Technology MoU Memorandum of Understanding PRC Primary Completion Rate PDO Project Development Objectives PIU Project Implementation Unit PP Project Paper PRSP Poverty Reduction Strategy Paper REBEP Rehabilitation of Basic Education Project WHO World Health Organization Vice President: Makhtar Diop Country Director: Yusupha B. Crookes Sector Manager: Peter N. Materu Project Team Leader: Susan E. Hirshberg ICR Team Leader: Susan E. Hirshberg SIERRA LEONE Education Sector Development Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ........................................................... 8 2. Key Factors Affecting Implementation and Outcomes ......................................................... 12 3. Assessment of Outcomes ...................................................................................................... 19 4. Assessment of Risk to Development Outcome ..................................................................... 24 5. Assessment of Bank and Borrower Performance.................................................................. 25 6. Lessons Learned.................................................................................................................... 27 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors ........................... 28 Annex 1. Project Costs and Financing ...................................................................................... 29 Annex 2. Outputs by Component .............................................................................................. 30 Annex 3. Economic and Financial Analysis ............................................................................. 35 Annex 4. Grant Preparation and Implementation Support/Supervision Processes ................... 43 Annex 5. Beneficiary Survey Results ....................................................................................... 45 Annex 6. Stakeholder Workshop Report and Results ............................................................... 46 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR .................................... 47 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ................................... 57 Annex 9. List of Supporting Documents................................................................................... 58 Map IBRD 33478 ...................................................................................................................... 59 1 A. Basic Information Sierra Leone EFA FTI Country: Sierra Leone Project Name: Program Project ID: P115782 L/C/TF Number(s): TF-92755 ICR Date: 3/28/2013 ICR Type: Core ICR REPUBLIC OF Lending Instrument: SIL Grantee: SIERRA LEONE Original Total USD 13.90M Disbursed Amount: USD 11.7M Commitment: Revised Amount: USD 13.90M Environmental Category: B Implementing Agencies: Ministry of Education, Science and Technology (MEST) Co-financiers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 06/27/2008 Effectiveness: 09/15/2008 06/30/2010 & Appraisal: Restructuring(s): 12/29/2011 Approval: 09/02/2008 Mid-term Review: 05/17/2010 11/01/2010 Closing: 06/30/2010 09/30/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: High Bank Performance: Moderately Satisfactory Grantee Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Moderately Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Moderately Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): 2 DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Primary education 50 Public administration- Education 25 Secondary education 25 Theme Code (as % of total Bank financing) Education for all 100 100 E. Bank Staff Positions At ICR At Approval Vice President: Makhtar Diop Obiageli Katryn Ezekwesili Country Director: Yusupha Crookes Ishac Diwan Sector Manager: Peter N. Materu Eva Jarawan Project Team Leader: Susan E. Hirshberg Eunice Dapaah ICR Team Leader: Susan E. Hirshberg ICR Primary Author: Kirsten Majgaard, Sandra Beemer F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objective of the project was to expand access to and improve the quality of basic education in Sierra Leone. Revised Project Development Objectives (as approved by original approving authority) The project development objective was not revised during the implementation period. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : Primary School Completion Increased Value 76% quantitative or 61.8% 70% Qualitative) Date achieved 2008 2010/11 2010/11 Comments The target was achieved. Data source is the “2010/11 School Census Report�. (incl. % The project supported the data analysis and publication of the school census 3 achievement) reports. Indicator 2 : Percentage of pupils who pass the NPSE examination Value quantitative or 72% 72% 75% Qualitative) Date achieved 2008 2010/11 2010/11 Target was achieved. In an environment with rapidly rising enrollments, and Comments where the new students brought into the system on average have lower socio- (incl. % economic characteristics than those already enrolled, maintaining learning achievement) outcomes and quality at the same level is an achievement. Indicator 3 : Gross Enrollment Rate increases Value quantitative or 104% 106% 122% Qualitative) Date achieved 2008 2010/11 2010/11 Indicator dropped at 12/2011 restructuring because the baseline was faulty: Comments school survey data collected before 2010 were later found to be highly inflated (incl. % and population estimates were unreliable, with no agreement on how to revise the achievement) baseline figures. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Percentage of girls enrolled in junior secondary school Value quantitative or 40.9% 43.9% 45% 45% Qualitative) Date achieved 2008 2010/11 2011/12 2010/11 Comments Target was achieved. The project supported the government’s girls’ incentive (incl. % program that covers the cost of school fees for girls which encourages parents to achievement) send girls to school as well as keep them enrolled. Number of additional classrooms built or rehabilitated at the primary level Indicator 2 : resulting from project interventions (Core Indicator) Value quantitative or 0 318 285 255 Qualitative) Date achieved 2011 2012 2012 Target missed, however 89% of classrooms were constructed. Upon completion Comments of the project, the MOF provided the Bank with written assurances that the (incl. % remaining 11% would be completed and have already provided the required achievement) funding to MEST for completion. Number of primary teachers who receive in-service training under the Indicator 3 : project. Value 0 1000 1000 1000 4 quantitative or Qualitative) Date achieved 2008 2011 2012 2012 Comments (incl. % Target was achieved with all teachers trained as agreed in the project design achievement) Indicator 4 : Number of junior secondary teachers who receive in-service training Value quantitative or 0 260 260 260 Qualitative) Date achieved 2008 2011 2012 2012 Comments (incl. % Target was achieved with all teachers trained as agreed in the project design achievement) Number of additional qualified teachers resulting from project intervention Indicator 5 : (core indicator) Value quantitative or 0 300 1000 1000 Qualitative) Date achieved 2008 2011 2012 2012 Comments (incl. % Target was achieved with all teachers trained as agreed in the project design achievement) Indicator 6 : Education Sector Annual Performance Review organized Value quantitative or 0 3 3 3 Qualitative) Date achieved 2008 2011 2012 Comments Target achieved. Education Sector Annual Performance Reviews organized for (incl. % Year 1, 2 and 3. achievement) Indicator 7: Learning Assessment System Framework developed Value quantitative or Yes Yes Qualitative) Date achieved 2008 2011 2012 Comments Target achieved. The project supported the development of a learning assessment (incl. % framework and assessment instruments that will be further developed in the next achievement) Global Partnership for Education project. Number of LCs which conduct fee-free awareness campaigns based on Indicator 8: findings of out-of-school study. Value quantitative or Qualitative) Date achieved Comments Indicator was dropped during 12/2011 restructuring because the activity was (incl. % carried out by UNICEF in 2008 during the delay in effectiveness. achievement) 5 Indicator 9: Number of pupils enrolled at primary level Value quantitative or 1,194,503 Qualitative) Date achieved 2010/11 Indicator dropped at 12/2011 restructuring because baseline was later found to be Comments highly inflated. HHS data show 956,000 students in 2007 (CWIQ), 980,000 in (incl. % 2008 (SLDHS). This would indicate a growth in primary enrollments over the achievement) project period. Indicator 10: Number of new entrants into primary 1 Value quantitative or 218,000 Qualitative) Date achieved 2010/11 Indicator dropped at 12/2011 restructuring because baseline was later found to be Comments highly inflated. New entrants in grade 1 has likely dropped, reflecting a (incl. % stabilization of the system. HHS data show improving cohort access over the achievement) project period. Indicator 11: Number of teaching and learning materials that are distributed to primary schools Value quantitative or Qualitative) Date achieved Indicator dropped at 12/2011 restructuring because it was not seen as measuring Comments improved learning. Development of a student assessment framework was added (incl. % as an indicator instead. A validation determined that materials purchased were achievement) delivered and in use. Indicator 12: Teacher payroll and salary verification exercise completed. Value quantitative or Qualitative) Date achieved Comments Indicator dropped during 12/2011 restructuring because the activity was (incl. % undertaken with the support of other donors: in 2008 DFID began the work and it achievement) was continued by the African Development Bank through the present. 6 G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 06/26/2009 Satisfactory Satisfactory 3.00 2 12/30/2009 Moderately Satisfactory Moderately Satisfactory 3.00 3 06/30/2010 Moderately Satisfactory Moderately Satisfactory 3.00 Moderately Moderately 4 04/12/2011 5.32 Unsatisfactory Unsatisfactory Moderately Moderately 5 12/26/2011 8.94 Unsatisfactory Unsatisfactory Moderately Moderately 6 04/23/2012 11.40 Unsatisfactory Unsatisfactory 7 07/31/2012 Moderately Satisfactory Moderately Satisfactory 11.40 H. Restructuring (if any) ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions 06/30/2010 No change MS MS 3.00 Extension by 18 months Minor revisions to components, implementation arrangements 12/29/2011 No change MU MU 8.94 and results framework, and extension by 9 months I. Disbursement Profile 7 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1. Country Context. In 2003, Sierra Leone emerged from eleven years (1991-2002) of devastating civil war and was ranked lowest on the UNDP Human Development Index out of 177 countries. The country was a post-conflict, fragile state with the need to rebuild institutions and restart the economy. It was estimated in the 2011 WDR “Conflict, Security and Development� that “civil conflict costs the average developing country roughly 30 years of Gross Domestic Product (GDP) growth, and countries in protracted crisis can fall over 20 percentage points behind in overcoming poverty�. In 2003/04, 70 percent of the population fell below the poverty line and poverty was heavily concentrated in the rural areas where nearly 80 percent of people were poor compared to 54 percent in the urban areas. The country had one of the lowest levels of life expectancy in the world at 39 years for men and 42 years for women. This was about 10 years shorter than the average for countries in Africa and 25 years shorter than the world. Child mortality rates were among the highest in the world: 17 percent of children died before their first birthday and 28 percent died before they reached the age of 5 years. The maternal mortality rate was one of the highest in the world at 2,000 maternal deaths per 100,000 live births. At the end of the civil war, GDP returned to moderate growth of around 4 percent per year between 2002 and 2005. By August 2007, Sierra Leone undertook its first successful presidential and parliamentary elections which handed over power to the All People’s Congress from the Sierra Leone People’s Party. 2. In June 2001, the government formulated an Interim Poverty Reduction Strategy Paper (IPRSP) that focused on good governance, re-launching the economy, and providing basic social services to the most vulnerable groups. The second phase of the IPRSP focused on good governance, revival of the economy and social sector development. The government developed a full Poverty Reduction Strategy Paper (PRSP) for 2005-2007 that focused on promoting: (i) good governance, security and peace, (ii) pro-poor sustainable growth for food security and job creation; and (iii) human development. The third pillar of the PRSP focused specifically on the attainment of the Millennium Development Goals (MDGs) based on the understanding that poverty reduction must be underpinned by investments in human resources. The successive PRSPs and World Bank Country Assistance Strategies (CAS) have identified weak governance and accountability structures as some of the main risks to their successful implementation. Lack of capacity was identified as a major constraint and a cause of governance weaknesses. At the time of appraisal of the FTI project (2008), Sierra Leone was ranked 158 out of 180 countries on the Transparency International Corruption Perceptions Index (it has since moved up to a rank of 134 by 2011). 3. Sector Context. When Sierra Leone emerged from the civil war most of the country’s social, economic and physical infrastructure had been destroyed. The education sector was devastated with the destruction of school infrastructure, severe shortages of teaching materials, overcrowding in many classrooms in safer areas, displacement of teachers and delay in paying their salaries, frequent disruptions of schooling, disorientation and psychological trauma among children, poor learning outcomes, weakened institutional capacity to manage the system and a serious lack of information and data to place service provision. At the time the project 8 was appraised, primary school enrollment had increased considerably since the end of the war. The best available data at the time showed that the primary school gross enrollment rate had increased from approximately 131 percent in 2002/03 to 162 percent in 2004/05, reflecting that the system was in a phase of catch-up with many overage children and youth enrolling in school after the war. Entry into grade 1 in primary school increased from approximately 200,000 to 300,000 between 2002 and 2005. Despite these increases, it was estimated that 25- 30 percent of primary school-aged children (more than 240,000) were out of school and the main reason cited was economic difficulties. Girls in the rural areas and the Northern region were less likely to attend or complete school. Most schools in Sierra Leone had poor quality classrooms, a lack of textbooks and teaching materials, and about 40 percent of the teachers were unqualified. Given these conditions and the post-conflict fragile state status of the country, it was highly unlikely that Sierra Leone would meet the education MDG targets by 2015. 4. To address the education challenges, the government took a number of initial steps: (i) introduced a fee free policy in 2002/03; (ii) established the Development Partnership Committee in 2003 to provide opportunities for sector engagement, dialogue and sector reviews, co-chaired by the Government of Sierra Leone (GoSL), World Bank and UNDP; (iii) passed the Local Government Act in 2004 to re-establish Local Councils and local governments to begin a decentralization process for the management of basic services with the intention of transferring full responsibility for pre-primary, primary, and junior secondary to the local level by 2008; (iv) passed the Education Act in 2004 that established key legislation to guide the education sector and; (v) developed the Education Sector Plan “A Road Map to a Better Future� (ESP 2007-2015) that outlined the goals and aspirations for education over a nine year period. The ESP emphasized primary education and skills training and provided the foundation for the achievement of universal basic education and expansion of post-basic secondary education. In 2007, the development partners (DPs) appraised the ESP and on April 27, 2007 Sierra Leone was welcomed into full EFA-FTI partnership (currently the Global Fund for Education-GPE) with the approval of an allocation of US$13.9 million. Project Context. At the time of the development of the EFA FTI project, the responsibility for managing and supervising basic education (pre-primary through junior secondary) was in the process of being decentralized to the newly formed Local Councils (LCs). The objective of the decentralization was to improve service delivery and school performance, thereby improving the efficiency and effectiveness of public education spending. The new functions of Local Councils included the recruitment and payment of teachers, provision of textbooks and teaching materials, and the rehabilitation and construction of schools. At the time, however, the new roles and lines of authority and accountability had yet to be clarified. At the local level, for example, it was not yet clear how the Ministry’s District Education Offices and the Local Councils would work together in order for the decentralization and devolution to be effective. Local Councils were also not fully staffed; while they had some technical staff in 2008 (monitoring and evaluation officers), they were not staffed with engineers until 2010. Capacity was also weak in the central government. At the time of project appraisal, the Ministry of Education, Science and Technology had 218 staff vacancies, including many positions at director level, and this situation did not improve until around 2010, when some directors were appointed. This is not unusual for fragile states. As indicated in the 2011 WDR, it can take a 9 generation to reestablish and create legitimate institutions after years of conflict—even in the fastest transforming countries it can take 15 to 30 years to raise institutional performance from that of a fragile state. 1.2 Original Project Development Objectives (PDO) and Key Indicators 5. The objective of the Project was to expand access to and improve the quality of basic education in the Recipient's territory. The key indicators selected to measure progress toward the PDO were: Indicator Baseline (2007/08) Target (2011) Primary Gross enrollment rate increases 104% 106%% Percentage of pupils who pass the 72.5% 72.5% National Primary School Exam (NPSE) 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 6. The PDO was not revised during project implementation. The key indicators were revised to be more aligned with the PDO and with core indicators for the education sector. The gross enrollment rate PDO indicator was dropped because the data were not comparable over time given the weaknesses in the underlying enrollment and/or population census data.1 The primary completion rate was added to replace the GER, and the percentage of girls in JSS indicator was moved from being an intermediate indicator to PDO level since there was an increased focus on girls in the project restructuring. Indicator Baseline 20081 Target 2010 Target 2012 Gross enrollment rate (dropped) 104% 106% Percentage of pupils who pass the NPSE 72.5% 72.5% 72% examination (continued) Primary completion rate (new) 61.8% 70% Percentage of girls enrolled in JSS 40.9% 43.9% 45% (moved from intermediate indicator to PDO level indicator) 1 Source: Sierra Leone Country Status Report (CSR), 2012 draft. 1.4 Main Beneficiaries, 7. The targeted population in the project consisted of children of primary school age, particularly those who did not currently attend school, and those who were at risk of repetition or drop out. Junior secondary school (JSS) girls, in particular, were to be targeted, given their 1 The numbers were somewhat controversial as they were reported differently by various entities and had wide variations ranging from 104% to 152%. While this is common in a post-conflict situation, there was no agreement so the team decided that completion rates would be more accurate. 10 lower enrollment and completion rates. The other beneficiaries were: (i) teachers that received pre-service and in-service training; (ii) school staff, parents and communities where new classrooms were constructed; (iii) staff at the Ministry of Education, Science and Technology (MEST-formerly the Ministry of Education, Sports and Youth-MEYS); and (iv) Local Councils. 1.5 Original Components The original project design comprised three components: 8. Expand Access to Basic Education (US$5.95 million). This component was to carry out activities to enhance the awareness of government fee-free and other education policies, activities aimed at increasing enrollment, especially in primary one, and construction, rehabilitation and furnishing of basic school infrastructure. An out-of-school study was to be carried out as a basis for campaigns to promote these activities. The component was targeted at increasing the number of pupils enrolled at primary level, the number of new entrants into primary one, and the percentage of girls enrolled. 9. Improve Quality of Teaching and Increase Access to Learning Materials (US$7.69 million). The component was to support primary and junior secondary teacher in-service and distance education training, and providing teaching and learning materials to primary and junior secondary schools. The component aimed to provide in-service training to primary teachers and junior secondary teachers and to upgrade the qualifications of unqualified teachers through an on-going distance learning program, as well as distribute sets of primary and adult learners teaching and learning materials to pupils and learners. 10. Enhance Coordination and Management of the Education Sector (US$0.26 million). This component was to strengthening sector capacity to coordinate and monitor education interventions, and improve education sector management. The component also aimed at institutionalizing the review of education sector performance through the annual education sector performance review. 1.6 Revised Components 11. The original project document listed activities that the project was to support. However, some activities in the original design were moved out of the project and supported by other development partners (DPs) because of delays in project effectiveness. The DPs agreed during their regular meetings on which activities they would support based on their own program portfolios. At the time of restructuring, the Bank team clarified which project activities were dropped and which activities were shifted out of the original project design and ultimately supported by the various DPs. The restructuring also added new activities, based on the government’s request, which shifted the focus of the project away from an input based approach to working on incremental, systemic improvements in service delivery. (Annex 2 to the ICR provides a table that details the modifications to each project component.) A summary of the main modifications to the project components were as follows: (i) revision of the classroom construction target from 318 to 285 due to slow implementation; (ii) inclusion of two years of financing for a de-worming program for school-age children; and (iii) inclusion of 11 one year of support for the government’s JSS girls’ incentive program to improve its governance and efficiency. The project first validated the number of girls in school and then made a one-time transfer to the MEST to provide support for one academic year. The restructuring also supported: (i) the procurement and distribution of writing instruments (pens, pencils, erasers, rulers), notebooks, supplementary reading materials, and Braille textbooks for primary and JSS schools; (ii) implementation of a learning assessment framework; (iii) training on test development in primary mathematics and language; (iv) review and revision of ten additional curricula and the printing of the revised curricula and developing a JSS curriculum framework which will lead to a full revision of the curriculum in the next GPE project; and (v) training of 700 school management committees which was a new one week training activity. 1.7 Other significant changes 12. On June 29, 2010, the project was restructured (level one) to extend the closing date of the project from June 30, 2010 to December 31, 2011. The purpose of the extension was to allow the government time to complete: (i) school construction, (ii) project training including teacher training, and (iii) distribution of teaching and learning materials. Implementation of project activities had been delayed due to delays in effectiveness and capacity constraints within MEST and local councils, but the project’s development objectives remained the same and attainable. 13. On December 29, 2011, the second level one restructuring: (i) modified selected indicators in the results framework; (ii) added a disbursement category and reallocated US$1.4 million to the new disbursement category to support the JSS girls incentive program; (iii) reduced the procurement of civil works from 318 classrooms to 285; (iv) introduced additional capacity building and support for the MEST and (v) extended the closing date for an additional nine months from December 31, 2011 to September 30, 2012. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 14. Project Preparation. In May 2007, the Government of Sierra Leone, the Bank and development partners (DP) began project preparation after the EFA-FTI Catalytic Fund (CF) Strategy Committee approved an EFA-FTI grant in the amount of US$13.9 million for activities toward achieving universal basic education in Sierra Leone. This approval was based on the ESP that had been appraised and endorsed by the donor community and the government. Although the EFA-FTI approved the grant in May 2007, the EFA-FTI guidelines were modified to require more rigorous project documentation. This modification led to a longer preparation period that delayed the signing of the grant agreement until September 15, 2008. The parliament of Sierra Leone ratified the project on December 16, 2008 which constituted the government’s formal agreement with the project plan. The Bank project preparation and appraisal teams consisted of technical experts that were appropriate for the development of the project as well as representatives from the local education sector group. On the government side, the preparation of the project included staff from the MEST and was supported by the 12 minister, since MEST had been active participants in the development of the ESP. The project preparation process was collaborative, aligned with the ESP and to the extent possible, based on lessons learned from the emergency project, Rehabilitation of Basic Education Project (REBEP-IDA-H0200). 15. Original Project Design. The original project design was relevant in that it fit squarely into the ESP that was informed by the 2006 Country Status Report (CSR), CAS, and PRSP all of which had the objectives of supporting growth, rapid poverty reduction, and human development. All recognized the strong links between universal basic education and economic growth and poverty reduction. The project design took into consideration comments by the peer reviewers which were responsible for providing the quality guidance on the project. 16. The project design included activities outlined in the ESP that were to assist the government in reaching their goal of increased access and improved quality of basic education through: (i) construction, rehabilitation and furnishing of basic school infrastructure (ii) improving the quality of teaching; (iii) increased access to learning materials, and (iv) enhanced coordination and management of the education sector. The results framework was developed based on the ESP indicators and they were the correct selection to measure progress toward the goal of increased access and improved quality. However, as was discovered after preparation, the 2004 baseline was faulty due to the difficulty in collecting data in a post-conflict situation. The problematic baseline ultimately led to some indicators being dropped during the restructuring. To support capacity building within the ministry, decisions were made to mainstream implementation within the MEST and include implementation activities into the day-to-day work of the ministry staff. This was also the first time, since the end of the civil conflict, that the MEST had been given the responsibility for implementation of projects.2 In addition, the project was designed to be part of an education sector support fund (ESSF-pooled fund) to which the DPs had agreed.3 The results framework was derived from the ESP and indicators were selected to measure access and quality. The project included one disbursement category which, in retrospect, was a good design choice because it provided appropriate flexibility for the simplified restructured design in 2011. The preparation team did identify capacity weakness at all levels as one of the main risks for project implementation. Although the design included some capacity development activities, in the end, they were insufficient to deal with capacity constraints within the system. However, this is not unexpected considering the EFA-FTI project was for a three year period and, as stated previously, it takes between 15 to 30 years to rebuild institutions and capacity after years of civil conflict. It was also evident that the government implementation team was not initially aware of the flexibility intrinsic in having one disbursement category. This lack of knowledge prevented the ESP secretariat from capitalizing on this flexibility and hiring need staff to support implementation. The project was 2 The Rehabilitation of Basic Education (REBEP) was implemented using a separate project implementation unit that was independent of the MEST. 3 The donors originally agreed that the ESSF would be primary modality used to support the ESP. While the ESSF still exists, it has not been used as the primary modality for supporting the ESP. 13 also designed as a three year project according the EFA-FTI guidelines. Although this is the agreed timeframe for EFA –FTI projects, this is too short a period of time in which to implement large scale new school construction particularly in a post-conflict fragile state environment. 17. Restructuring Design. By 2010 it was evident that capacity constraints and lack of coordination between the central ministry and LCs were impacting the implementation of the project. Early in 2011, the project submitted a revised annual plan which shifted resources to ensure that the ESP Secretariat was fully financed and people hired to address issues of day-to- day management, monitoring and evaluation, and financial and procurement staffing. After a full review of the issues and realization that the program’s original design could not be implemented, it was decided to restructure the project in the fall of 2011. The restructuring: (i) reduced the scope of the civil works, (ii) increased resource allocations to support quality, girls and school health and (iii) modified the results framework to more accurately reflect project progress and PDO achievement. This was reflected by the: (i) reduction in the number of classrooms to be constructed from 318 to 285; (ii) inclusion of support for a JSS girl’s incentive program; (iii) review of the JSS curriculum and development of a new JSS curriculum framework; and (iv) support for de-worming of school aged children. (See Annex 2-Table 1 for a detailed list of project activities dropped and added during the restructuring.) The restructuring continued to fit squarely within the ESP (2007-20015) and the Bank’s Country Assistance Strategy. The project PDO remained unchanged and the implementation unit stayed within the ministry structure. In addition, the flow of funds was adjusted to avoid delays and redundancy, especially between the LCs and MEST, and relevant reporting procedures were simplified. Specifically the funds were transferred to the ESSF Leone account and paid directly to the LC education accounts instead of flowing through various levels of the Local Government Finance Department (LGFD). The restructuring was appropriate particularly with the: (i) re-focus on quality through the support of curriculum improvement and assessment, (ii) reallocation of US$1.4 million for girls and (iii) more effective financial management arrangements. The support for girls is judged to be important considering girls are underrepresented in secondary school and given the wide gap in youth literacy rates, which are 48 percent for girls and 68 percent for boy. There remains a critical need to ensure that development progress accrues to women as well as men. Moreover, the adjustment to the flow of funds and the increased resources for the implementation team was important because of the limited capacity related to the post-conflict environment in Sierra Leone. 2.2 Implementation 18. The project was designed to mainstream project implementation within the MEST and to devolve some responsibilities to the Local Councils in keeping with the decentralization efforts in the education sector. The project was also the first post-conflict project to be implemented solely by MEST staff (the first emergency project established a separate Project Implementation Unit for implementation). The MEST was to be responsible for overall implementation, while the local councils were to be responsible for the procurement and supervision of school construction and school furniture. 14 19. Implementation of the EFA-FTI project was mixed. The project was signed in September 2008 and became effective January 7, 2009 after it was ratified by the parliament and the legal opinion was issued by the Attorney General. The first withdrawal application was processed in June 2009 in the amount of US$3 million. These early processing delays reduced the overall implementation period allowing less than one and a half years for actual implementation under the original design. 20. The implementation started with the MEST implementation project coordinator mainstreamed into an Education Sector Plan Secretariat and the MEST’s procurement officer working on: (i) the review of the design of school structures; (ii) bid evaluation process with local councils (LCs) for civil works and furniture and with MEST for teaching and learning materials, and (iii) the education sector annual performance review. On the Bank and DP side, there was: (i) preparation of an Education Program Development Fund (EPDF) application in the amount of US$769,000 to support MEST capacity strengthening; (ii) establishing of the ESSF committee and preparation of the ESSF operational manual, and (iii) planning for the teacher training activities with the MEST and UNICEF developing the child centered teaching techniques (CCTT) modules that would be incorporated into the project distance education activity. 21. As early as December 2009, MEST and Local Councils began to exhibit signs of implementation capacity weakness with problems related to the civil works. They were having difficulties with the bid evaluation process and the MEST procurement officer left the ministry. Although the procurement officer was replaced, the new procurement officer and the chief accountant needed substantial training related to Bank procurement and financial management requirements. Moreover, they were responsible for all MEST procurement and financial management which made it difficult for them to provide adequate support for the implementation of the project. The project design provided for hiring FM and procurement consultants to work with the ESP coordinator and serve as liaisons to the ministry finance officer and procurement unit but they had not been hired. This left the ESP coordinator as the only person to manage all the project-related activities. It was correct to mainstream the implementation within the MEST. However, the Bank preparation team should have identified all key team members and begun the capacity training during preparation to ensure project readiness and smoother implementation. 22. Civil works contracts were signed in late 2009 and early 2010 and the first installment of the contracts were released to the contractors. However, it took until late 2010 and early 2011 for the second installment to be released. Much of this delay was due to lack of knowledge and experience with Bank procurement and financial management (FM) procedures. In addition, some LCs discovered that they had few qualified bidders and had to re-launch the process. This continued to slow the construction of schools. By June 29, 2010 the project had only disbursed 28 percent of the grant and was about to close. The FTI, Bank, DPs, and Government, realizing that Sierra Leone’s capacity constraints had delayed the project implementation, agreed to extend the closing date to December 2011. This would give time for the project to complete ongoing activities and meet its PDO. 23. In late-2010, the Bank shifted its project support strategy. A new project TTL identified specific implementation bottlenecks, began intensive supervision, and worked with the MEST 15 to ensure the ESP Secretariat was properly staffed. However, the project had to be downgraded to moderately unsatisfactory due to slow disbursements, lack of financial management monitoring and reporting – including no Interim Financial Reports since the project’s start or the annual audit for 2009 – and lack of results on the ground. After the shift in focus to implementation bottlenecks and with more attention to procurement issues, implementation improved substantially with: (i) disbursements improving from 22 percent in June 2010 to 64 percent by December 2011, (ii) submission of all Interim Financial Management Reports (IFRs), (iii) submission of the 2009 and 2010 annual audits, (iv) progress in the supply and delivery of teaching and learning materials, (v) continued teacher training, and (vi) fully staffing the ESP Secretariat. In September 2011, the Bank was able to hire an Extended Term Consultant (ETC) based in the Freetown office to provide day-to-day implementation support in the country. While there was progress, the implementation team still needed time to complete the project activities and in the fall of 2011, the MEST requested a second extension. The Bank team asserted that the extension could only be granted if the project was restructured to shift the emphasis on goods and works toward key activities that would begin to move toward a programmatic approach, which would eventually have more long term impact on strengthening the system. The MEST agreed and the project restructuring was completed in December 2011. Between December 2010 and project closing, the Bank also conducted 10 supervision missions to Sierra Leone to work with the MEST and other government groups to improve project implementation. 24. The restructuring did refocus the project design on quality and system-building activities which facilitated implementation progress. However, challenges remained with the construction of schools, the main issues being: (i) delays in MEST receiving reports from LCs which slowed payments; (ii) non-performance of contractors; (iii) delayed payments or over- payments from the LCs to contractors; and (iv) weak LC construction supervision leading to insufficient quality in some cases. These issues persisted until the project closing. However, by project closing, 255 classrooms out of 285, 42 sets of latrines out of 57 and 40 wells out of 57 were completed. The MOF also provided the Bank task team with a letter guaranteeing the completion of the remaining works by the original contractors and supervised by the LCs and have already allocated the resources to the MEST budget as a specific line-item. Despite the civil works challenges, the project experienced substantial progress related to quality and system-building. This can be seen in the: (i) one year support of approximately 75,000 girls through the JSS tuition incentive program, which helped to ensure the retention of girls, and development of an operations manual to guide improved, more transparent implementation of the program in future years; (ii) evaluation of the 1-year distance training which indicated that classroom observation showed that teachers were performing at a higher level; (iii) completion of a policy framework for learning assessment that included a draft instrument for piloting and training of 50 participants on test item writing; (iv) development of a new JSS framework to guide a complete revision of the curriculum over the next few years; (v) de-worming of 1.2 million primary school-aged children in 12 districts; and (vi) school-based management training (see Annex 2 for additional project achievements). These are all substantial achievements of the project. In addition to these successes, the benefits of providing resources directly to the MEST, given post-conflict governance and capacity constraints, cannot be underestimated. The project gave MEST the first opportunity to make expenditure choices and to gain experience in the implementation of projects post-conflict. Again, this is a substantial achievement. By project closing, the project was upgraded to MS with all the quality and 16 system building activities satisfactorily implemented and approximately 89 percent of classrooms, 70 percent of wells and 74 percent of latrines completed. The final project disbursements were 84.17 percent. While 100 percent of the funds were not disbursed, this was partly due to savings related to US$/Le exchange rate gains. 25. Development Partner Coordination. The project preparation began with strong development partner (DP) support and involvement. DPs agreed on an ESSF pooled fund mechanism for donor funds, they developed an Operations Manual and set up a Coordination Committee, signed a Memorandum of Understanding (MoU), and deposits were made into a Leone account and a US$ account for the EFA-FTI funds. Support for the pooled fund mechanism diminished during implementation with most donors implementing their own programs. However, after the restructuring, the project utilized the Leone account to simplify the financial arrangements for the JSS girls’ incentive program and transfer funds directly to schools. The EFA-FTI project was also very helpful in serving as a coordination mechanism for regular meetings between DPs, non-government organizations and Government to discuss project and ESP implementation progress. The DPs remain engaged in the updating of the Education Sector Development Plan and the preparation of the new GPE project with UNICEF as the lead donor and the World Bank as the supervising entity. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 26. Design. The results framework for the project was derived directly from the ESP4 and was appropriate considering the data availability on sector performance at the time. While the indicators themselves were appropriate, the slow implementation progress and some baseline data issues warranted the 2011 restructuring of the results framework. During the restructuring, the gross enrollment rate (GER) and gross intake rate (GIR) were specifically dropped because the data were not comparable over time given the weaknesses in the underlying enrollment and/or population census data. In addition, because the restructuring eliminated activities and introduced others, some of the intermediate indicators were modified. The ICR team judges that these were all appropriate modifications and measures for achievement of the PDO. 27. The project also provided financing for the Education Management Information System (EMIS) within MEST, thereby supporting MEST capacity to monitor sector performance. The previous project had also supported EMIS, and the EFA FTI project provided funding for school census data collection and analysis for 2010/11 and 2011/12. As per the project design, the ESP Coordinator—in addition to being responsible for day-to-day management of the project—had the responsibility to: (i) coordinate development partner interventions/activities, and (ii) lead the process of organizing the annual sector performance reviews which was an appropriate design. 28. Implementation and Utilization of Monitoring and Evaluation . Implementation of the EMIS activities was successful. Support was provided for training related to MEST data entry 4 ESP data was derived from the CSR. 17 and analysis with EPDF funds, while the EFA FTI project funded data collection, entry, and analysis. This resulted in the production of the 2010/11 School Census Report which was the first school census report since 2005. This report established a new baseline for education statistics and going forward will result in more accurate and timely education statistics necessary for well-informed policy and budget decisions. Education data from 2010/11 were also used in the final results framework. The project also provided funding for the 2011/12 school census, for which analysis is on-going, and for pre-census training activities related to the 2012/13 school census. MEST provided regular implementation status reports, and the ESSF Committee met regularly to discuss the status of project implementation. Annual sector performance reviews have also been held. The project conducted several validation exercises during the last year: (i) MEST worked with Civil Society Organizations (CSOs) to validate that the learning materials and furniture reached schools and are in use; (ii) an independent engineering assessment of the civil works was carried out; and (iii) a CSO validation of girls JSS enrollment figures. The ICR team views the validation exercises as a valuable part of the project M&E activities and a best practice. 2.4 Safeguard and Fiduciary Compliance 29. Safeguards. The project was classified as a Category B because it included school construction. An Integrated Safeguard Data Sheet (ISDS) was prepared and disclosed in the InfoShop in September 2008. An Environmental and Social Management Framework (ESMF) was prepared, approved and disclosed in Sierra Leone and the InfoShop on September 10, 2008. There was a post-project environmental audit that showed that the project was in compliance with all safeguards as outlined in the ESMF. 30. Fiduciary and Procurement Compliance. As mentioned previously, the MEST was responsible for implementing the project and the fiduciary management capacity was deemed satisfactory for the implementation of the original project according to OP/BP 10.02. However, the systems were judged weak and therefore the risks associated with FM were substantial. Most financial arrangements of the restructured project remained largely the same with the notable exception of the funds flow from the MEST to the LCs which were streamlined to avoid delays. The responsibilities for project FM were carried out by the finance officer consultant in the ESP Secretariat in the MEST. The project FM arrangements met the minimum requirements for financial management systems with its internal controls and staffing, and financial reporting compliance with the submission of audit reports and Interim Financial Reports (IFRs). 31. At project closing, financial management of the project was rated MS. Other than the audit waiver granted for the first year’s audit, there were no documented deviations, nor were there waivers from the Bank’s fiduciary policies and procedures throughout the project’s four- year implementation. Overall, the project met the financial reporting requirements as agreed in the Financing Agreement. There were initial report submission delays in the first few years owing to some implementation challenges especially the prolonged delay in recruiting FM personnel on the project; but since these were resolved, submission of the financial reports (Audited Financial Statements and Interim Financial Reports) have been consistent and in acceptable formats. For the same reasons, disbursement activities were delayed until early 2011. With the hiring of experienced personnel by the ESP Secretariat, frequent supervision 18 missions, and FM implementation support, disbursement activities improved and stood at 84.17 percent at closing. The 2011 financial audit was unqualified and the 2012 audit is due to two months following the Grace Period, or March 31, 2013. 32. Although the procurement process slowed project implementation, overall procurement was carried out in accordance with the World Bank’s “Guidelines: Procurement Under IBRD Loans and IDA Credits� dated May 2004 and revised in October 2006; and “Guidelines: Selection and Employment of Consultants by the World Bank Borrowers� dated May 2004 and revised in October 2006 and the provisions stipulated in the Legal Agreement. The issues related to procurement at project closing were payments to contractors for incomplete work and non-performing contracts. While the implementation team was not able to resolve these issues prior to closing, the Ministry of Finance provided assurance to the Bank that these issues would be resolved by March 31, 2013. 2.5 Post-completion Operation/Next Phase 33. The Bank is currently working with the government of Sierra Leone to prepare a new Global Partnership for Education Fund (GPEF) operation. The new project is expected to be delivered in FY14 and will focus on improving the quality of basic education services. The project is likely to include: (i) learning assessment activities related to the work started under this project; (ii) curriculum reform based on the curriculum framework already developed, (iii) grants to the local councils to implement annual work plans; and (iv) capacity building activities to further the progress made during this project. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 34. Relevance of Objectives. The project development objectives were highly relevant to the country’s sectoral needs when the project was developed. The project objectives fit squarely into the Bank’s 2005 CAS, the PRSP, and the government and DPs 2007 ESP that aimed to help Sierra Leone make progress toward achievement of the MDGs. Like the 2005 CAS, the new CAS prepared in 2010 also included a strong focus on human development, including basic education, so the project is relevant to the current CAS as well. The objectives were designed to be broad which was appropriate because it allowed for the project to make relevant adjustments when necessary during implementation. 35. Relevance of Design. The mainstreaming of project implementation, the dual focus on access and quality of the original project design, and the results framework were appropriate. However, large scale civil works and procurement activities were less appropriate considering the capacity constraints identified as a risk in the original project paper (PP). In addition, the proposed capacity risk mitigation activities to ameliorate this weakness were under-prepared and unfunded. During the restructuring, the Bank and government teams recognized the continued capacity constraints and refocused the project design on system building and expanded the quality aspects of the project. The restructuring also appropriately modified the results framework to reflect the changes in the project and improved the flow of funds by 19 simplifying the processes to eliminate the bottlenecks that were inhibiting implementation. These were all important modifications and highly appropriate. 36. Relevance of Implementation. The relevance of early stages of project implementation is judged to be modest. This is mainly based on the slow implementation of large scale civil works activities and procurement of goods and materials. As indicated, this was due to the limited experience of the MEST with implementation of Bank supported projects and the capacity constraints of the MEST staff regarding Bank procurement and FM procedures. Moreover, the Bank did not have an adequate support system in country to assist in the amelioration of these constraints. As indicated above, beginning in late-2010, the Bank provided more intensive supervision and took measures to deal with the support system, restructure the project, simplified the flow of funds, and provided procurement assistance and helped standardize the FM reporting. With these measures in place, the government substantially improved their implementation of project activities. Based on these actions the overall rating for relevance of implementation is judged to be substantial. 3.2 Achievement of Project Development Objectives 37. The project development objective was to expand access to and improve the quality of basic education in Sierra Leone. The original results framework PDO indicators were being monitored and data collected as well as reported on a regular basis to the DPs. However, the GER was dropped as an indicator during the restructuring. This was because there were wide variations between the GER figures being reported from different sources—e.g., the 2007 Country Status Report and Statistics Sierra Leone—and because it was later determined that school survey data collected before 2010 had been highly inflated. In addition, population data were based on projections of older census data and found to be uncertain. Based on this information, the Bank team decided to drop the GER indicator and add the primary completion rate (PCR) as a PDO level indicator to measure improvements in access and student flow efficiency within the system. The PCR has several advantages out the GER, since the PCR is net of repeaters as well as a more direct measure of progress toward universal primary completion which is the key MDG goal for education. It should be noted, that while the indicator was dropped and the baseline considered faulty, GER has improved and is now at 122 percent compared to the 104 percent baseline established at the time of project appraisal. 5 At this time, the percentage of girls enrolled in JSS was moved from the intermediate level to the PDO level because the focus on girls had been increased in the project activities. This section evaluates the outcomes against the results framework modified during the 2011 restructuring and other indicators of system performance. 5 Further, according to the 2012 draft education Country Status Report, household survey data show that primary school enrollments were about 956,000 in 2007 using the CWIQ survey, and 980,000 in 2008 using the SLDHS. This is compared with primary enrollments of 1,194,503 in 2010/11, based on the 2010/11 school census, that was validated and broadly considered to be reliable. Thus, enrollments have likely increased in absolute terms over the project period. 20 38. Improved quality in basic education was to be measured using the percentage of pupils who pass the NPSE examination. The target of 72 percent was surpassed by 3 percent. In an environment with rapidly rising enrollments, and where the new students brought into the system on average, have lower socio-economic characteristics than those already enrolled, maintaining learning outcomes and quality at the same level is an achievement.. The intermediate indicators used to measure improved access were: (i) number of additional qualified teachers resulting from project interventions (core indicator); (ii) number of primary teachers who receive in-service training under the project; and (iii) number of junior secondary teachers who receive in-service training. The number of unqualified teachers who were trained and upgraded through distance learning was 1,000 reaching the target of 1,000. The number of primary and JSS teachers that received in-service was 1,000 and 260 respectively, both reaching the target.6 The project supported an assessment of the in-service distance education training program and reported, for the classroom observations, that the training had a positive impact on teacher performance. In addition to the selected indicators, the project also provided: (i) approximately 1.7 million supplementary readers for primary schools that were confirmed in an independent validation to be in schools and used; (ii) writing instruments (pens, pencils, erasers, rulers), notebooks, and Braille textbooks for children with disabilities; and (iii) funds for two years of de-worming for a total of approximately 1.8 million children. An independent monitoring of the de-worming program revealed 87 percent coverage of all primary school- aged children covered which is significantly over the World Health Organization (WHO) target of 80 percent. 39. Increased accessed was to be measured using the primary completion rate and percentage of girls enrolled in junior secondary school. The primary completion rate increased from 61.8 percent in 2008 to 76.1 percent in 2011 exceeding the target of 70.0 percent. The percentage of girls enrolled in JSS increased from 40.9 percent in 2008 to 45 percent 2011 meeting the target. These are substantial achievement for any system over such a short period of time. The intermediate level indicator selected to measure increased access was number of new classroom constructed under the project (core indicator). The number of additional classrooms constructed under the project was 255 missing the revised target of 285. 3.3 Efficiency 40. There is evidence of substantial efficiency in the use of project funds, with unit costs for project activities and inputs all falling close to or below average costs from Sierra Leone or other countries in the region. The fact that independent validations found that inputs, on the whole, have reached the intended beneficiaries and are being used is also viewed very positively from an efficiency or value-for-money standpoint. 41. At the time of preparation of this ICR (February 2013), the financial loss sustained from the non-completion of all civil works stands at around 3 percent of total project disbursements 6 SIDA (Sweden) also financed training for 1000 primary and 260 JSS teachers, bringing to totals to 2000 and 520 respectively. 21 (or around US$ 340,000). The Government of Sierra Leone has guaranteed the completion of all outstanding civil works at their expense. 42. From an economic standpoint, project funds have generally been invested in activities or inputs that are known to produce high returns (such as de-worming, teaching and learning materials) and were needed in the country (classrooms, school furniture). Annex 3 provides some estimates of the cost-effectiveness of the JSS girls’ tuition subsidies and the de-worming of school-age children. The project also scores high in terms of providing employment and income opportunities within Sierra Leone, given that school construction was done by national companies and school furniture were built by local craftsmen. 43. There is potentially a fiscal impact of the project, since it finances system expansion (new schools) and teacher upgrading (which, in principle, qualifies teachers for higher salaries), but these higher costs are within the parameters set out in the Education Sector Plan and costed out in the associated financial simulation model. Thus, the fiscal impact is not additional to projected costs for the sub-sector. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 44. An overall outcome rating of the project is moderately satisfactory based on the following: (i) all project PDO level indicators were achieved; (ii) all but one of the intermediate indicators were achieved; (iii) capacity within MEST was improved although there were difficulties because Sierra Leone had just emerged from many years of conflict, and (iv) the project design correctly mainstreamed implementation within the Ministry, but there were implementation delays since this was the first project the Ministry ever implemented (the previous project had a PIU with dedicated expertise). This delay was, again, to be expected considering the post-conflict status of the country which left the Ministry devastated at the time. Although the project experienced early implementation delays, improvements began earlier than the actual submission of the 2011 restructuring paper and upgrading of the rating from MU to MS. This is evidenced by an increase in disbursements from 20 to 64 percent between December 2010 and December 2011 - which was the result of hiring the appropriate implementation staff. The discussions and work on restructuring began as early as June 2011, but it takes time to conduct and conclude a dialogue with governments on eliminating civil works and procurement of books in favor of system improvements, particularly when the government was facing an election year. Ultimately; the restructured project did shift to working on processes and system development through: (i) a curriculum review which involved extensive consultations across the country and analysis with international and national technical specialists; (ii) developing a student assessment framework, including training of MEST and other professionals in test and item development, sampling, etc. - that created actual test instruments that were piloted; (iii) cleaning up the systems behind the data collection and disbursements of the Girl-Child Support Program which had significant governance issues; and (iv) extensive monitoring and validations, including with the support of CSOs, on the distribution and use of materials in schools, levels of school construction, and promotion of the Girl-Child Support Program. 22 45. The project was downgraded to MU in the April 2011 ISR due to fiduciary issues and lack of robust evidence of results on the ground. The fiduciary problems were addressed efficiently, and implementation began to improve with the hiring of key staff in the ESP Secretariat. Although project activities appeared to be moving, the team was reluctant to raise the ratings for the two subsequent ISRs until there was hard evidence of project progress. Based on implementation reports there seemed to be positive progress and therefore, the team assumed the risk to move forward with the restructuring based on an understanding that the MEST would (i) reduce the amount of procurement in the restructured project, and (ii) conduct validations of all previous and ongoing activities. These validations by MEST, DecSec, CSOs and the Bank were conducted between November 2011 and April 2012 and found that implementation and results had in fact started improving in early 2011. The validations showed that classrooms that were completed were being used and that all goods that had been procured by the project were in schools and being used. Based on the new evidence from the validations, the team raised the rating in the final ISR to MS. If evidence from the validations had been available sooner, the team would have been able to raise the rating from MU to MS before the restructuring was completed. ISRs cannot be retroactively upgraded, but the improvement in fiduciary issues and the results of the validations were discussed in the final ISR of June 2012. 46. In summary, due to the extensive support to the program, MEST was able to build capacity, prepare for a curriculum renewal (in the next project), develop systems to monitor learning outcomes and use more transparent processes for distributing funds to schools. The validations also provided the new data that showed project inputs were not only completed but were being fully utilized. Given the status and capacity at the beginning of the project, and combined with the highly efficient use of project funds, these were significant achievements.. In a post-conflict and fragile state environment the project achievements are judged to be is moderately satisfactory. Original Project – Phase 1 (1/7/2009-12/30/2010) – 25.64 percent disbursement-net grant1 Project Relevance Achievement of PDO (Efficacy) Efficiency Overall Rating Modest Moderately Unsatisfactory Substantial Moderately Unsatisfactory Project Restructuring - Phase 2 (1/2011-9/30/2012) – 74.36 percent disbursement-net grant Project Relevance Achievement of PDO (Efficacy) Efficiency Overall Rating Substantial Moderately Satisfactory Substantial Moderately Satisfactory Overall Project Ratings – 84.17 percent disbursement of total grant Project Relevance Achievement of PDO (Efficacy) Efficiency Overall Rating Substantial Moderately Satisfactory Substantial Moderately Satisfactory2 1 The net grant amount for the project is US$11.7 million. 2 Overall rating = (3)(.2564) +(4)(.7436) =3.74 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development Not applicable (b) Institutional Change/Strengthening 23 47. In addition to providing funds to support the JSS girls’ incentive program, the project was able to strengthen its delivery and transparency. The findings of the JSS validation process, conducted by CSOs, led MEST to improve in the processes of the JSS girls’ incentive program. These changes were: (i) independent validation of girls’ enrollment using NGOs and also triangulated the enrollment data with the EMIS; (ii) improved payment scheme that ensures funds go directly to school bank accounts and parents receive reimbursement of fees paid; and (iii) procedures for faster disbursement of funds for the incentive program. These processes are now standardized throughout the country. (c) Other Unintended Outcomes and Impacts (positive or negative) 48. An unintended outcome of validation exercises added during the restructuring was that it provided an opportunity for linkage between the government and the CSOs. This experience improved government/CSO interaction, strengthened social accountability in the sector, and was beneficial for the government in terms of providing feedback to MEST on the situation on the ground. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable 4. Assessment of Risk to Development Outcome Rating: High 49. The risks identified during project preparation were mostly appropriate: five were rated substantial and two were modest. The risks identified were appropriate, however, in retrospect; the ratings should have been higher for all the risks particularly given the post-conflict status of the country and the decision to mainstream implementation activities within MEST. The modest ratings related to local council administration being changed and inadequate capacity at the local council level should have been rated substantial and high respectively. The five substantial rating related to overall MEST capacity, submission of audit reports, inadequate capacity for Bank procurement and FM and monitoring of education outcomes should have all been rated high. The mitigations measures identified were appropriate however; additional technical support was ultimately needed to improve implementation. The period from 2010 to project completion saw an increase in technical support which led to an improvement in implementation. Going forward the risks remain high related to capacity within the MEST, particularly related to financial management, procurement, and management of Bank projects. While the risks remain high related to capacity within MEST and LCs, the mainstreaming of project implementation remains appropriate for Sierra Leone because MEST as an institution will be strengthened along with their ability to delivery education services. The MEST remains understaffed but continues to add staff and to fill the large number of vacancies. This process is likely to take several years. The risks related to maintaining the achievements of the development outcomes are modest. The objectives of improving access and quality of basic education remain valid and going forward it is expected that there will be ongoing improvements in the primary completion rates and the NPSE examination. The MEST will continue to improve the quality of teaching with support of the next GPE project as well as 24 continue to implement the new curriculum and learning assessment frameworks to improve overall quality in the system. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry 50. Bank Performance. The Bank preparation team included the appropriate technical experts to appraise the project. The project was developed based on the ESP and in accordance with EFA-FTI guidelines. The Bank, working with the government, incorporated mainstreaming of project implementation into the design. This also gave MEST the opportunity to manage a project for the first time after emerging from the civil crisis. The project design appropriately provided flexibility by having one disbursement category, which allowed for added flexibility in the use of the funds. However, the MEST was not properly coached on the benefits and therefore could not take advantage of them. The project was appropriately aligned with the ESP, PRSP and CAS but given the post-conflict status of the country, the project readiness was questionable. The preparation team should have begun capacity training during preparation to ensure project readiness and smoother implementation. Capacity constraints were identified in the project document and resources were provided to support capacity building; however, it will take more years of support to re-establish the capacity within the MEST. The project design also included large scale school construction activities that ultimately had to be scaled back because of limited capacity and a short three year implementation period. The project construction activities were largely responsible for many of the implementation problems. Midway through project implementation the Bank recognized the implementation bottlenecks and took steps to work with government to restructure the project. The restructuring eliminated flow of funds constraints, reduced the civil works and redirected funds to quality inputs and system building which was welcomed by government and DPs. These measures substantially improved implementation. Based on this, Bank performance at entry is judged to be moderately satisfactory, (b) Quality of Supervision Rating: Moderately Satisfactory 51. The Bank conducted many supervision missions during project implementation and reported on the bottlenecks related to implementation civil works and slow disbursements. There were two task team leaders during the implementation period. Post-2010 there was a higher frequency of Bank supervision with 11 supervision missions in two years. In addition, there was an in-country education specialist dedicated to supporting the Bank education program and a country procurement officer that could work directly with MEST staff for the last year of the project. The post-2010 Bank team recognized the need to restructure the project and to narrow the scope of the interventions. The team composition was at that point comprehensive and included education, procurement and financial management, and a governance specialist. This increased effort helped accelerate implementation. The extensions of the closing date were aimed at addressing issues encountered during implementation as well as those that would enhance the development impact of the project. However, the Bank could have done a better job of systematically providing technical advice and guidance on Bank 25 processes and procedures through the early stages of the project which could have begun with a project launch workshop and training. (c) Justification of Rating for Overall Bank Performance 52. Based on the Bank’s quality at entry, the supervision discussion, and project outcomes above, overall Bank performance is rated moderately satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 53. At the time of preparation, the country of Sierra Leone and the government were emerging from civil strife and were deemed a post-conflict, fragile state. They were in the process of rebuilding their institutions and there were substantial capacity constraints. Also, a process of decentralization was ongoing since 2004. Recognizing these issues, the government along with the development partners undertook the challenge of developing a full 9-year education sector development plan to guide them in rebuilding the education system. Once the ESP was developed, the government was an active participant in the presentation of the ESP to the EFA FTI Catalytic Fund Strategy Committee in Bonn, Germany, in 2007. Based on the presentation, the Committee approved a US$13.9 million allocation to Sierra Leone. When EFA FTI modified their guidelines, the Government engaged with the EFA FTI Secretariat to try and accommodate the new procedures which delayed signing of the grant agreement until September 15, 2008. Parliament ratified the grant within an appropriate time period, on December 16, 2008. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 54. The government began mainstreaming the implementation of the project at the same time as they were trying to re-staff their institutions, both at central and local levels. This led to slow implementation progress. The delay in project effectiveness to January 2009 coupled with the delay in the first withdrawal application in June 2009, contributed to the government’s ability to fully staff the ESP secretariat. The secretariat did not become a fully functional unit until early-2011 when it finally: (i) received an orientation on Bank procedures; (ii) understood the flexibility of the disbursement category; and (iii) understood procurement and financial management requirements. Once the secretariat staff understood the processes and the procedures, implementation improved and 81.99 percent of the project was disbursed and most of the project activities were completed by project closing. The ESP secretariat complied with the requirements to submit both IFRs and FMRs but they were often delayed. The audit reports were received and unqualified. However, there continued to be slow implementation of the project civil works activities related to inadequate capacity at the local council level and procurement experience at the national level. This prevented the project from finishing the civil works activities. The government did provide written guarantees that all remaining classrooms, latrines and wells would be completed with budget already allocated and provided to the MEST. 26 (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 55. In light of the government performance and the discussion above, the overall performance of the Borrower is rated moderately satisfactory. 6. Lessons Learned 56. Lesson 1. Projects designed for post-conflict and fragile states need to anticipate that implementation and disbursements will be slow initially. GPE projects are given three years for full implementation. When countries have weak capacity this is not enough time to build the necessary capacity and implement projects, in particular as it relates to civil work activities. The GPE three year implementation requirement should be reconsidered, particularly in post- conflict countries. Bank International Development Association (IDA) projects are allowed five years for implementation. 57. Lesson 2. A strong government project implementation team must be in place prior to project effectiveness and remain in place throughout project implementation. Key secretariat appointments were delayed resulting in slow implementation progress. These delays along with the short implementation period, contributed to the project implementation problems. 58. Lesson 3. Strong Bank supervision with knowledge of Bank procedures as well as Bank staff based in post-conflict countries with low capacity is essential for ensuring the implementation of Bank-supported projects. In the case of this project, the implementation unit was unfamiliar with Bank procedures and would have benefited from earlier support from the Bank on various procedures. The presence of a Bank-supported educator in country proved invaluable for daily support and monitoring of project implementation. Experience with other Bank-supported projects in post-conflict countries has shown that having the TTL in-country is critical for ensuring timely project implementation. A project launch workshop would have been helpful at the beginning of the project to provide an understanding of Bank procedures. 59. Lesson 4. The EFA-FTI project included a school construction component that was difficult to implement due to weak capacity, lack of understanding of Bank procurement methods, and inadequate arrangements for contract management and supervision. Future school construction should include more capacity building and support in Bank procurement methods, but should also be managed closer to the community level with more appropriate contract management and supervision arrangements, and with funds provided directly to local governments or communities to pay contractors. 60. Lesson 5. The project was developed and implemented within the context of structural and institutional decentralization. Because the final decentralization mechanisms could not be fully understood until the implementation process itself started, the implementation of the Sierra Leone project was inherently bound to be slow. Projects designed within a context of on- going decentralization should include an initial period of adjustment, as capacity cannot be built before but it is actually supported and built during implementation. This is especially important for post-conflict countries. 27 7. Comments on Issues Raised by Grantee/Implementing Agencies/Donors (a) Grantee/Implementing agencies See Annex 7 for the borrower portion of the ICR. (b) Cofinanciers/Donors (c) Other partners and stakeholders 28 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) Total Baseline Cost 13.90 11.69 84.10 Physical Contingencies 0.00 0.00 0.00 Price Contingencies 0.00 0.00 0.00 Total Project Costs 13.90 11.69 84.10 Total Financing Required 13.90 11.69 84.10 (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (USD (USD Appraisal millions) millions) Education for All - Fast Track 13.90 11.69 84.10 Initiative 29 Annex 2. Outputs by Component Component 1. Expand Access to Basic Education 61. Construction of new primary and JSS schools (cost~US$3.90 million). The original design planned the construction or rehabilitation of 318 basic school classrooms (corresponding to 52 schools), including head teacher’s offices (52), water wells (52) and 6- compartment pit latrines (52). The schools were also to be provided a special classroom for integrating children with special needs as well as stores for textbooks and exercise books. The restructuring reduced the classroom target from 318 to 285 classrooms and reallocated funds to other activities. The 285 classrooms were allocated across the territory as 3 new schools (two 6-classroom primary and one 3 classroom JSS school) for each of the 19 Local Councils (for a total of 57 schools). The new schools built under the project include an office and storage, water well and VIP latrines. As described under Component 2, the schools were also supplied with furniture and learning materials. An external assessment conducted towards the end of the project found that some of the works had not been completed by the contractors. At project closing, 255 of the 285 classrooms had been completed, along with 40 of the 57 water wells and 42 of the 57 sets of latrines. 62. De-worming (cost~US$ 0.41 million). In spring 2011 the Bank agreed to support a once yearly de-worming of school-aged children under Component 1 which was formalized under the restructuring. This activity was implemented by the Helen Keller Institute and financed the de-worming of children in seven districts in June 2011 (of 680,000 children) and 12 districts in May 2012 (covering 1.3 million children). 63. Girls’ incentive program for JSS (cost~US$ 1.05 million). During restructuring, one year of funding for this ongoing government program was included in Component 1. It consists of subsidies to tuition for girls in JSS schools to off-set tuition reductions for girls. The program finances the fees for all three terms in grade 7, two out of three terms in grade 8, and one out of three terms in grade 9. The program is meant to increase access and retention of girls in JSS. Funds are paid directly to schools. The project also supported activities to make the program more efficient and accurate through the more effective collection and validation of information on girls’ enrollments. This includes the production of an operational manual designed to improve the functioning of the program. 64. School Census/Support to the Education Management Information System (EMIS). The project, together with UNICEF, supported the 2010/11 school census, which was the first complete survey since the civil conflict, and the 2011/12 school census. For those two years, funding was provided for the development of questionnaires, training, data collection and yearbook preparation. Further, the project financed the training for the 2012/13 school census and the establishment of a computer/data room in the MEST for large-scale data analysis. 30 Component 2. Improve Quality of Teaching and Increase Access to Learning Materials 65. Training of teachers (US$~0.67 million). As per the original design, the project funded the in-service training on teaching methodology and classroom management of 1,000 primary and 260 JSS teachers (with another 1,260 teachers trained with funding from SIDA). Further, the project funded the participation of 1000 untrained teachers in a 1-year distance teacher education program, by the end of which teachers become certified to teach in grades 1-3 of primary school. The target for the distance training was initially 300, but revised up to 1000 during restructuring. Two studies were also initially planned under this sub-component: (a) an evaluation of on-going in-service teacher training, and (b) a study on overall teacher training; but the restructuring paper revised this to an evaluation of the 1-year distance teacher training program. This evaluation was completed in late 2012. 66. Furniture for new schools and needy schools (cost~US$ 1.09 million). New primary and JSS schools built under this project as well as other needy schools were supplied with pupil seaters, teacher tables and chairs, office tables and chairs and office cupboards. The furniture were fabricated within the local council areas by artisans, and distributed by the local councils to schools. All the 19 local councils received equal amounts of new school furniture. A total of 10,260 desks and chairs were purchased for primary school and 8,075 desks for JSS students. The project provided 684 sets of primary school teachers’ desks and chairs and 171 for JSS teachers. In addition to desks, the project provided 114 sets of office furniture and 114 cupboards to primary schools as well as 57 sets of office furniture for JSS and 57 cupboards. Local councils made the distribution decisions based on need. External validation of the delivery of the furniture to schools was conducted and found that all the furniture had been delivered. 67. Teaching and learning materials for primary and JSS schools (cost~US$ 2.68 million). As per the original design, the project financed the supply of teaching and learning materials for primary and JSS schools. Some revisions to the types of materials were made during restructuring, as per Table 1. The types of materials procured were: (i) pen, pencils, rulers, erasers; (ii) exercise books; and (iii) supplementary reading materials. The project purchased 850,000 sets of pens, pencils, erasers and rulers, exercise books and 1,700,000 supplementary readers. All the 19 local councils received equal amounts of teaching and learning materials. The project also purchased braille books that were distributed to the six schools for the blind in the Bo, Kenema, Western Area, Kono, Bombali and Koinadugu districts. The first set included 162 primary English textbooks grades 1-6; structural English (108 copies-grades 1-3); social studies (157 copies-grades 1-6); primary science (93 copies- grades3-6): The second set included 350 sets of braille primary school supplementary readers and 100 copies of braille social studies textbooks for blind JSS pupils; The third set is made up of 100 sets of braille English and integrated science textbooks for blind JSS pupils. External validation of the delivery of the T&L materials to schools was conducted. 68. SMC training (cost~US$ 0.15 million). The project supported the training of school- management committees (SMCs) in 50 schools per district in 2012. MEST has been 31 monitoring the activities of SMCs and there are strong indications that schools have benefited from the training with more activities going on than before the training. 69. Learning assessment and curriculum frameworks. The project funded: (i) a learning assessment framework for primary school, field testing of test items, and training of people to write test items; and (ii) the participatory development of a national junior secondary school (JSS) curriculum framework that also addresses student assessment in JSS. Component 3. Enhance Coordination and Management of the Education Sector 70. Education Sector Plan Secretariat Coordination and Operational Costs . The project has funded the staffing and operational costs of the ESP Secretariat (four positions). UNICEF has also supported the ESP Secretariat through the pooled fund. 71. Ministry of Education staff training (cost~US$ 0.19 million). Computer training of professional and administrative staff of the Ministry of Education, Science and Technology. This training was part of a capacity building plan and responded to a need for greater computer literacy as it was difficult for staff to operate computers. After the training, there is a greater use of computers, including of email for communication. Table 1: Revised Project Activities Activities dropped Original activities New activities as Activities updated Activities with no at restructuring of restructuring at restructuring change Component 1. Expand Access to Basic Education Out of School Out of school Incentives for girls Classroom Study study was funded at the JSS level rehabilitation and by UNICEF. based on agreed construction. Awareness raising criteria Reduced target campaign Awareness raising classrooms from campaigns funded De-worming of 318 to 285 Classroom by UNICEF and school children. rehabilitation and other partners Extend coverage to Conduct school construction (318 national. census. New classrooms). questionnaire and process. School Census 32 Component 2. Improve Quality of Teaching and Increase Access to Learning Materials In-service teacher Procurement of Development of In-service teacher training (1,260 850,000 geometry new curriculum training for 1,260 primary and JSS sets framework for JSS primary and JSS teachers) teachers Establish Printing and Distance integrated learning distribution of Teacher Certificate Education training complex in 3 LCs syllabus for 14 JSS training using subjects distance delivery Establish Procurement of for 1000 primary integrated learning televisions and Printing and teachers complex in 3 LCs VCD players. distribution of Braille texts for Evaluation of 1- Evaluate in-service Equip 6 schools in primary and JSS year distance teacher training each local council education training (distance with solar panels. Preparation of course for teachers education) policy framework for learning Procurement and Review and revise assessment distribution of 4 basic course JSS teaching and syllabi Training for test learning materials development on Procurement and language and Procurement and distribution of mathematics at the distribution of teaching and primary level furniture learning materials. Training of school Procurement and management distribution of committees furniture. Procurement of 850,000 geometry sets. Procurement of televisions and VCD players Equip 6 schools in each local council with solar panels 33 Component 3. Enhance Coordination and Management of the Education Sector Strengthen sector Teacher payroll ESP Secretariat capacity to verification funded Coordination and coordinate and by DFID in 2008 Operational Costs monitor education and the EU funded interventions activities that ESP Secretariat attempted to Staff Training Teacher payroll reconcile the verification payroll. There was Financial and a second study Procurement ESP Secretariat done by the Audits Coordination and African Operational Costs Development Bank in 2011/12. There ESP Secretariat was a draft report Staff Training provided by AfDB and the GOSL now Financial and needs to take some Procurement action on the Audits. recommendations. Source: Amended table from Annex 5 of Restructuring Paper, dated 12/28/2011. 34 Annex 3. Economic and Financial Analysis 72. A mission to Sierra Leone took place in January 2013 to collect data for an ex-post economic and financial analysis of the EFA FTI Project. The results of this analysis are presented in this Annex. 73. In summary, there is evidence of substantial efficiency in the use of project funds, with unit costs for project activities and inputs all falling close to or below average costs from Sierra Leone or other countries in the region. The fact that independent validations found that inputs, on the whole, have reached the intended beneficiaries and are being used is also viewed very positively from an efficiency or value-for-money standpoint. 74. At the time of preparation of this ICR (February 2013), the financial loss sustained from the non-completion of all civil works stands at around 3 percent of total project disbursements (or around US$ 340,000). The Government of Sierra Leone has guaranteed the completion of all outstanding civil works at their expense. 75. From an economic standpoint, project funds have generally been invested in activities or inputs that are known to produce high returns (such as de-worming, teaching and learning materials) and were needed in the country (classrooms, school furniture). This annex provides some estimates of the cost-effectiveness of the JSS girls’ tuition subsidies and the de-worming of school-age children. The project also scores highly in terms of providing employment and income opportunities within Sierra Leone, given that school construction was done by national companies and school furniture were built by local craftsmen. 76. There is potentially a fiscal impact of the project, since it finances system expansion (new schools) and teacher upgrading (which, in principle, qualifies teachers for higher salaries), but these higher costs are within the parameters set out in the Education Sector Plan and costed out in the associated financial simulation model. Thus, the fiscal impact is not additional to projected costs for the sub-sector. Financial Analysis A. Value for Money of School Infrastructure 77. Unit costs for construction of primary and JSS schools fall close to unit costs of comparable projects in and outside of Sierra Leone (see Table A3.1). Given Sierra Leones post-conflict situation at the time, this is quite an achievement, since post-conflict states often have less developed private sectors and under-developed road networks. The cost per classroom of FTI-funded schools was US$13,225 (or approximately US$11,425 excluding well and latrines). These are a little higher than the unit costs of the 2009 UNICEF funded schools in Sierra Leone, but the difference can be explained by a simpler design of the UNICEF schools. The gross cost of construction per square meter was about US$183 for the FTI schools; this compared quite well with average figures for school construction for Sub-Saharan African countries as shows in Table A3.1. 35 78. There is no evidence of very low bids for school construction . Too low bids were cited by the MEST as one of the reasons for some contractors failing to complete construction. But a review of all contract amounts showed significant consistency in bid amounts across all contracts. 79. The loss sustained from non-completion of some civil works amounts to about 3 percent of total project disbursements, and the Ministry of Finance has made a commitment to ensure their completion. This estimated loss can be calculated as the cost of completing the 30 classrooms (50% completed), 17 water wells (0% completed) and 15 sets of latrines (0% completed), that were reported as incomplete during the January 2013 ICR mission. The cost of completion these can be estimated at US$340,000 (assuming unit costs of US$11,425 per classroom, US$3,000 per water well, and US$ 7,800 for a full set of latrines (2x3)). Table A3.1: Unit costs of construction of primary and JSS schools, Sierra Leone EFA FTI Project, approximately 2010 Primary schools JSS schools Description of works 6-classroom primary school 3-classroom JSS school All schools include an office, a store, a well with hand pump, and 2 3-compartment latrines. Classrooms measure 20x28 feet ~ 52 square meters. Schools were built by small and medium-sized contractors, procured through NCB, and contracted by Local Councils. Construction materials were available locally, and communities would even help provide certain materials. Average cost per classroom, incl. 13,225 15,439 office, store, well and latrines (US$) Average cost per classroom, incl. 11,425 11,839 office and store (US$) Average cost per gross m2, excl. well 183 - and latrines (US$, 2010 prices) Comparator costs Sub-Saharan ~200 (all projects) - Africa (US$ per gross m2, 2006 ~175 (local gov. managed) prices) ~110 (community managed) Comparator costs from UNICEF Sierra Leone (NGOs construction) --Cost per classroom, incl. office, 11,754 - store, well and latrines (US$ 2009 prices) --Cost per classroom, incl. office 10,144 - and store (US$, 2009 prices) Note: Exchange rate used for EFA FTI project data is 4,150 Le/USD as of end-2010. Source: Project data. Comparator data are from Theunynck (2009): ‘School Construction Strategies for Universal Primary Education in Africa’, The World Bank, which is based on the review of over 200 projects in Sub-Saharan Africa. 80. Unit costs paid for furniture for the FTI primary schools are below average unit costs from 10 Sub-Saharan African countries. In the FTI project, the cost of furnishing schools was US$930 per classroom in primary schools, which is well below the 2006 average of US$1,605 for 10 other countries in the region. The cost of furnishing the JSS schools is 36 higher, at US$2,641 per classroom. No comparator figure has been found for JSS school furniture. The significant difference in the cost between primary and JSS school furniture may be partly explained by the fact that JSS school furniture is larger. Table A3.2: Unit costs of furnishing primary and JSS schools, Sierra Leone EFA FTI Project, approximately 2010 Primary schools JSS schools Description of furniture 90 3-seaters for students, 60 2-seaters for students, provided to each school 6 teacher tables and chairs, 3 teacher tables and chairs, 1 office table and chair, 1 office table and chair, 1 office cupboard 1 office cupboard (all produced by local craftsmen (all produced by local craftsmen using locally available using locally available materials) materials) Number of schools in each 6 3 Local Council, for which (2 new schools+ (1 new school+ furniture were provided 4 schools ‘in dire need’) 2 schools ‘in dire need’) Average cost per school (US$)1 5,581 7,922 Average cost per classroom 930 2,641 (US$, 2010 prices) Comparator: Average cost of 1,605 No reference furnishing a classroom, based (range 1,110 in Uganda to figures available on data from 10 SSA countries 2,200 in Mauritania) (US$, 2006 prices) Note: Exchange rate used is 4,150 Le/USD as of end-2010. Source: Project data. Comparator data are from Theunynck (2009): ‘School Construction Strategies for Universal Primary Education in Africa’, The World Bank. 81. Other unit costs also show evidence of good value for money for such activities as subsidies to girls’ JSS tuition, teacher training, deworming and teaching and learning materials (Table XX). The unit cost of deworming, for example, is the same as internationally cited costs of US$0.50 per child per year for two rounds of deworming. The fact that the project had independent validations carried out confirming that teaching & learning materials, furniture, and deworming had reached schools/students, further support the case made for substantial project efficiency. Table A3.3: Other unit costs (actuals), Sierra Leone EFA FTI Project, approximately 2010 Unit cost (SLL) Unit cost (US$) Explanatory notes Tuition subsidy to girls in US$14 per child Covers about half of full JSS JSS tuition Teacher training, in- US$169 per teacher service Teacher training, 1-year US$462 per teacher Cost includes distance (Training subsistence during upgraded unqualified college-based training teachers to qualified for sessions lower primary) Deworming US$0.25 per child per Includes est. cost of year medication (US$0.04 per child), although the project did not finance it 37 Pens, pencils, rulers Pencils 90-110 Pencils 0.02-0.03 NCB procurement erasers Pens 160 Pens 0.04 Rulers 320 Rulers 0.08 Erasers 240 Erasers 0.06 Exercise books 800-1,900 per book 0.19-0.46 per book NCB procurement, prices vary based on number of pages Supplementary reading, primary grades 1-3 Lot 4&5 US$1.36 per book NCB procurement Lot 6 US$0.41-0.47 per book ICB procurement Supplementary reading, primary grades 4-6 Lot 4&5 US$1.53 per book NCB procurement Lot 6 0.32-0.41 ICB procurement Note: Exchange rates used are 4,150 Le/USD as of end-2010, and 0.645 GBP/USD. 82. Exchange rate gains led to undisbursed funds at the end of the project. Compared with the unit costs shown above, the project achieved some savings resulting from delays in implementation leading to exchange rate gains (the EFA FTI grant is in US$, whilst some contracts were made out in Leones). This largely explains why not all project funds were disbursed. B. Evolution in Public Spending on Education 83. The GPE grant did not replace government spending on education. The level of public spending on education in Sierra Leone remained stable from the time of project approval (2008) to 2010, while data on public spending in 2011 and 2012 are not yet available. Public education spending measured in constant prices grew by 14 percent between the two years, and from 2.6 to 2.8 percent of the country’s GDP. Table A3.4: Evolution in government spending on education, 2008-10 2008 2009 2010 Total education spending (million Leone, 2010 prices) 242,979 276,560 277,682 Education spending as % of GDP 2.6% 2.9% 2.8% Source: Chapter 1 of 2013 draft Country Status Report on Education for Sierra Leone, based on MoFED data. 84. There is potentially a fiscal impact of the project, since it finances new schools that will require maintenance and teacher upgrading which, in principle, qualifies the newly qualified teachers for higher salaries (although these higher salaries have not yet been awarded, according to information collected during the ICR mission). In any case, the increase in the salary envelope may be quite modest, given information provided in the Country Status Report, showing that the difference in salaries between unqualified and (starting) qualified teachers may only be around US$ 50-100 a year 7 . In any case, these higher costs are within the 7 The difference between salary grades 3 and 4 is about US$ 50 a year, while the difference between salary grades 2 and 4 is about US$ 100 a year. 38 parameters set out in the Education Sector Plan and costed out in the associated financial simulation model. Economic Analysis 85. The economic analysis provides an estimation of the cost-effectiveness of two project interventions that aimed to keep children and youth in school: (i) the incentives to girls’ JSS participation, and (ii) the de-worming of school-age children. Also, a discussion is included of the distribution of schools and other inputs provided by the project. A. Incentives for girls’ JSS participation 86. The background for providing girls with tuition subsidies for JSS is the persistent gap between girls’ and boys’ rates of participation in JSS. Figure A3.1 shows the size of the gender gap at the beginning and end of the JSS cycle in 2010: while there is only a little gap at entry to JSS (grade 7), there is a sizable gap by grade 9. Only 35 percent of girls attain grade 9 compared with 51 percent of boys. Figure A3.1: School attainment by grade and gender in Sierra Leone, 2010 100 86 Cohort access rate (%) 80 68 86 60 60 51 65 Boys 57 35 32 40 Girls 20 35 20 0 14 Grade 1 Grade 6 Grade 7 Grade 9 Grade 10 Grade 12 Note: The cohort access rates are the likelihoods that children will ever attain the grades shown; it is not exactly the same as current enrollments in those grades. Source: Sierra Leone Country Status Report 2012 (draft) based on data from MICS IV, 2010. 87. Government subsidy of girls’ JSS tuition has existed since 2002, and the project funded the subsidies of the current 2012/13 school year. The subsidy per girl is US$14 per year and there are now around 75,000 girls enrolled in JSS. Many of the girls enrolled would likely have enrolled in JSS even without these subsidies, so the impact of the tuition subsidy would depend on how many additional girls enroll and stay in school due to the subsidy. When the program was first introduced in 2002, there was a large jump in girls’ JSS enrollments (from 30,000 to 50,000 girls), but no increase in the share of girls (39 percent of all enrollment), as boys’ enrollments were also increasing rapidly at the time (Figure A3.2). Data from that time period, however, may not have been that reliable, as the collection of education statistics had just resumed and did not follow the protocols in place today. In the absence of more reliable data, we cannot definitively assess if there was an increase in girls’ enrollment that can be attributed to the subsidies. The share of girls in total JSS enrollments did increase later on, in particular 39 between 2008 and 2010/11, when it grew from 40.9 percent to 45 percent, as reported in the project’s result framework. Figure A3.2: Trend in JSS enrollments by gender, 2000/01-2004/05 100 Enrollments (thousands) 80 60 JSS M 40 JSS F 20 0 2000/01 2001/02 2002/03 2003/04 2004/05 Source: Appendix D in World Bank (2007). Education in Sierra Leone: Present Challenges, Future Opportunities. 88. Given that we cannot reliably assess the increase in girls’ enrollment as a result of the subsidy, for illustrative purposes, we assume the increase is 2 percentage-points, which seems reasonable given the data shown above. Under this brave assumption, an additional 5,860 girls attend JSS, bringing the cost for one additional girl in JSS for one year at US$179, or US$538 for the whole JSS cycle. This calculation is merely an example to illustrate how to get at the cost-effectiveness of such an intervention. Table A3.5: Illustrative example: Impact of JSS tuition subsidy on girls’ enrollment under certain assumptions Girls Boys All % girls GPI Subsidy per additional girl in JSS Without 69,140 91,667 160,807 43% 0.75 subsidy With subsidy 75,000 91,667 166,667 45% 0.82 Change 5,860 0 5,860 2%-point 0.07 US$ 179 89. How long is there a need to sustain this subsidy scheme? Once girls enroll and complete JSS at the same rate of boys (and this may only be a few years away), the tuition subsidy may be better targeted to rural or needy children, since there are very wide gaps in JSS enrollment between rural and urban households and between those from the richest and poorest income quintiles. B. De-worming of school-age children 90. The project funded the de-worming treatment of school-age children once yearly in 2011 and 2012, complementing another yearly treatment regimen funded by another program. Teachers and communities were also provided with health education related to personal and community hygiene and water and sanitation, as these are important for worm control. The WHO recommends the twice yearly deworming of school-age children in Sierra Leone in order to: (i) reduce anemia, underweight, stunting, absenteeism and poor cognition; (ii) improve 40 school attendance, performance and achievement; and (iii) improve future wage earning potential and gross domestic product (GDP). Table A3.6 presents the number of children treated and the cost of treatment (and unit cost) in both years. Table A3.6: Data on the de-worming activity of the Sierra Leone EFA FTI project, 2011 and 2012 Number Number of children reached Project Financing % coverage (assessed of financing per child through independent Enrolled Non- Total districts 1 (US$) (US$)2 validation within a enrolled covered week of distribution) 2011 8 628,733 23,097 651,830 160,000 0.25 87% 2012 12 1,053,865 119,095 1,172,960 250,000 0.21 85% 1 On the pre-advertised national de-worming day, non-enrolled school-age children come to the school to receive treatment along with enrolled children. 2 To this should be added the cost of the drugs, which was donated in this case; the cost is about 3 cents per child plus some transportation costs. 91. Sierra Leone previously had some of the highest rates of infection in the world, but levels are now under control due to regular de-worming. Due to poor access to water and sanitation, regular de-worming is still necessary, and infection rates would return to previous levels, if treatment was discontinued. The impact of de-worming on schooling outcomes have not been measured in Sierra Leone, but evidence exists from other countries. Miguel and Kremer8 found that a school-based mass de-worming program in Kenya increased schooling by 0.15 years per pupil treated9. Given that each pupil needs twice-yearly treatment at US$0.25 for each, the cost of 0.15 additional school years is US$0.50, which brings the cost for a whole additional school year for one child to roughly US$3.50 , an often cited figure in international publications. This cost is comparatively low compared with the cost of many other interventions to support school participation, such as school feeding or other demand incentives. 92. Despite the inexpensive treatment available, parents typically do not give their children the de-worming treatment due to lack of awareness. Therefore, and because of the public health externalities (treated children reduce the transmission to untreated individuals in their communities, also documented by Miguel and Kremer), government intervention in the de- worming of school-age children is justified. C. School inputs and their distribution 93. The project funded school construction (including wells, latrines and furniture) and teaching and learning materials. Due to the over-crowding of classrooms documented in the first Country Status Report for Sierra Leone, school construction was justified. A survey of schools in 2008 found the average student-classroom ratio to be 76, but with considerable 8 Miguel and Kremer (2004). Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities. Econometrica, Vol. 72, No.1. 9 Glewwe and Kremer (2005). Schools, Teachers and Education Outcomes in Developing Countries. 41 variation across regions.10 Similarly, it made sense to fund teaching and learning materials due to the shortage of such materials in classrooms. 94. The distribution of school inputs to schools, however, did not happen in the most equitable or economically most efficient way, since they were simply distributed in equal shares to every Local Councils, despite large differences in the enrollment/population number. This was apparently due to shortage of data, or lack of trust in available data, and this method of distribution was perceived as fair and equitable in the Sierra Leone context at the time. It was therefore appropriate at the time given the country’s fragility status, and the importance of not contributing to any additional conflict over resources. 95. Looking ahead, however, given that Local Councils have widely different school-age population and enrollment numbers, school inputs should either be distributed in a spatially more equitable way, or to where the greatest needs are (and therefore the highest marginal returns). 10 Baseline survey of schools done as part of a World Bank textbook impact evaluation. 42 Annex 4. Grant Preparation and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending/Grant Preparation Eunice Yaa Brimfah Ackwerh Education Specialist AFTEW Task Team Leader Ferdinand Tsri Apronti, Procurement Specialist AFTPW Procurement Mercy Miyang Tembon Education Specialist HDNED Education Robert Wallace Degraft Financial Financial Management Specialist AFTMW Hanson Management Financial Samuel Bruce-Smith Financial Management Consultant AFTEW Management Manush Hristov Sr. Legal Counsel LEGEN Lawyer Beatrix Allah Memsah Social Development Specialist AFTCS Governance Fatu Karim-Turay Team Assistant AFMSL Administrative Ernestina Aboah-Ndow Program Assistant AFCW1 Administrative Supervision/ICR Eunice Yaa Brimfah Ackwerh Senior Education Specialist AFTEW Task Team Leader Susan Hirshberg Senior Education Specialist AFTEE Task Team Leader Bidemi Carrol Consultant AFTEW Implementation Ernestina Aboah-Ndow Program Assistant AFCA1 Administrative Financial Joyce Olubukola Agunbiade Financial Management Specialist AFTMW Management Ferdinand Tsri Apronti Procurement Specialist AFTA1 Procurement Samuel Bruce-Smith Consultant AFTDE Education Peter Darvas Sr Education Economist AFTEW General Education Donald B. Hamilton Consultant AFTEW General Education Manush A. Hristov Senior Counsel LEGEN Lawyer Anders Jensen Senior Monitoring & Evaluation AFTDE M&E Fatu Karim-Turay Team Assistant AFMSL Administrataive Adama Ouedraogo Senior Education Specialist AFTEW Education Financial Oluwole Pratt Financial Management Analyst AFTME Management Kristine Schwebach Operations Analyst AFTCS Operations Serge Theunynck Consultant AFTEE Education Xiao Ye Economist AFRCE Economics Sandra Beemer Consultant AFTEW ICR co-author Kirsten Majgaard Economist, Consultant AFTEW ICR co-author Rose-Claire Pakabomba Program Assistant AFTEW Administrative Christopher Gabelle Sr. Governance Specialist AFTP3 Governance Mose Duphey Consultant AFTN3 Education Viorel Vela Consultant AFTEW Education Adama Ginorlei Consultant AFMSL Administrative 43 (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending Total: N/A1 0.00 Supervision/ICR Total: 45.0 252,290.00 1 The preparation costs for this project were provided by the EFA-FTI under the EPDF grant. 44 Annex 5. Beneficiary Survey Results Not Applicable 45 Annex 6. Stakeholder Workshop Report and Results Not Applicable 46 Annex 7. Summary of Grantee's ICR and/or Comments on Draft ICR 96. Background. The EFA FTI Education Sector Project (the Project) to expand access and to improve quality of basic education in Sierra Leone was approved on September 2, 2008 with the funding of 13,900,000 US Dollars. The Project was declared effective on September 15, 2008. The original Closing Date was June 30, 2010, with first extension to December 31, 2011 and later to September 30, 2012. Implementation has been uneven, and the project required a moderate restructuring and a second extension of nine months in order to complete some critical activities and substantially achieve its development objective of “expanding access to and improve the quality of basic education in Sierra Leone�. The Project Restructuring provided the time and appropriate support to the MEST to complete ongoing activities related to learning materials and civil works, and better mainstreamed the project across the MEST and other relevant ministries to ensure the project investments were sustained. 97. Implementation Progress (an overview). The Sierra Leone EFA-FTI Project witnessed a difficult start-up due to underestimated, yet common capacity constraints found in a Post Conflict country, a focus on inputs which caused procurement delays, as well as other design issues including premature mainstreaming of project implementation into the MEST without adequate support, as MEST had 218 Vacancies in the professional wing as at commencement of the Project. The Closing Date was extended in June 2010 to address the delay in effectiveness from when the grant was first awarded by the EFA-FTI Board in May 2007 until effectiveness. However, the Project only gained momentum in December 2010. As a result, several key project activities were not fully implemented or able to help reach the development objectives. Furthermore, project indicators have proved difficult to measure because baselines have been found to be faulty and a few of the indicators were not influenced by project interventions. The project development objective (PDO) is aligned with Sierra Leone Education Sector Plan (ESP 2007-2015) and was not modified during the restructuring programme. However, several changes were made as in Table 1 below: (i) the PDO indicators and the intermediate outcome indicators were modified to make them realistic and measurable; (ii) some of the project component activities were also modified to ensure increased impact and to wrap-up the project in an orderly fashion; (iii) flows of funds were adjusted to avoid delays and redundancy along with relevant improvements reporting requirements; Table 1 Revisions to the Results Framework Indicators Comments/ Rationale for Change PDO Current (PAD) Proposed change* Comments/ Rationale for Change To expand access to No change and improve the quality of basic education in Sierra Leone PDO Level Results Indicators* 47 Indicator One: Gross Dropped The baseline (2004) and mid-term indicators have Enrollment Rate (GER) been found to be faulty, therefore measuring the changes in GER over the life of the project would be unreliable. GER is also influenced by ebbs and flows common in a Post Conflict situation so that trends are difficult to fully analyze and understand. Indicator Two: Percentage of pupils who pass the NPSE examination Continued Indicator Three: New This is a core indicator for the Education Sector and Primary School a better measure of efficiency and quality. Recent Completion analysis found this to be reliable for two points in time: 2004 and 2010. It will be measured again in 2011 for the final year of the project. Indicator Four: This will be reinforced by the Girls Incentive Intermediate Result Program. indicator Four: Percentage of girls Continued: moved from enrolled in junior Intermediate Indicator to secondary level PDO level Indicator Current (PAD) Proposed change* Comments/ Rationale for Change INTERMEDIATE RESULTS Intermediate Result (Component One): Intermediate Result Dropped The delay in project start up resulted in the activity indicator One: Number being funded by another EDP. A small amount was of LCs which conduct allocated to this activity, but not across all LCs. fee-free awareness campaigns based on findings of out-of- school study Intermediate Result Dropped Like the GER, baseline and mid-term data were indicator Two: Number faulty and not an accurate measure of the system in of pupils enrolled at a Post Conflict situation. primary level Intermediate Result indicator Three: Number of new entrants into primary 1 Dropped Same as above. Intermediate Result Revised Revised downward from 318 to 285. Procurement indicator Four: and monitoring of civil works problematic and Number of additional addressed in the future through community-based classrooms built or initiatives. rehabilitated at the primary level resulting from project intervention Intermediate Result (Component Two): 48 Intermediate Result Continued indicator Five: Number of primary teachers who receive in-service training under the project Intermediate Result Continued indicator Six: Number of junior secondary teachers who receive in- service training Intermediate Result This was the wording of the core indicators indicator Seven: Number of teachers who enroll for the distance Reworded and Revised education program from: Number of new resulting in additional teachers who enroll for the qualification distance education program Intermediate Result Dropped This aggregated number was not helpful in indicator Eight: measuring improved learning. A Framework for Number of teaching and developing a Learning Assessment for future use learning materials that will be developed. are distributed to primary schools Intermediate Result (Component Three): Intermediate Result Continued indicator Nine: Education Sector Annual Performance Review organized Intermediate Result Dropped indicator Ten: Teacher payroll and salary verification exercise completed. 98. The progress achieved during the project implementation based on the revised indicators is highlighted in the Table below: 99. Baseline data was collected in 2004-05 while drafting the Country Status Report (CSR), which is currently being updated. All the project development indicators have been met or surpassed. Project Development Baseline Target Actual (Current) Comments Indicators Percentage of Students 72% 72% 75% Target achieved. who pass NPSE exams Data to be updated for 2011/12 Primary Completion 61.8% 70% 76% Target exceeded. Rate Data to be updated for 2011/12 (if possible) Percentage of girls in 40.9% 45% 45% Target achieved. junior secondary school Data to be updated for 2011/12. Intermediate Results Indicators Baseline Target Actual Comments (Current) Number of new classrooms 285 255* (presently) Further update from 49 constructed under the project Ministry of Finance that have returned from the Monitoring Visit. MEST awaiting report. MOFED agreed to fund completion of remaining classrooms. Number of primary teachers who 1000 1000 Target achieved receive in-service training under the project Number of Junior Secondary teachers 260 260 Target achieved who receive in-service training Number of new teachers who enroll 300 1000 Target surpassed for the distance education program Education Sector Annual Performance 3 3 Target achieved Review organized 100. Project Components. The project components which were to: (i) expand access to basic education, (ii) improve the quality of teaching and increase access to learning materials, and (iii) enhance coordination and management of the education sector were not changed in the restructuring. All investments continued to support progress in the areas of improving access, quality and program management. However, some activities were dropped because they were financed and undertaken with support from another agency during the delay in project effectiveness from May 2007 until September 15, 2008 (e.g.: the Out of School Study and Awareness Campaigns). New activities were added to shift away from an input based-approach to working on incremental, systemic improvements in service delivery (e.g.: Girls’ Incentive Program). The Grant Agreement was therefore revised to be less specific, and to broaden the scope of activities which are financed and undertaken to support achieving the development objective of expanding access to and improving the quality of basic education. Activities which are not in the purview of the MEST or feasible in the context of a project of this size and scope (e.g.: teacher payroll system) were dropped. The revised components as in the signed Amendment Letter of December 29, 2011 are as follows: (a) Implementation of activities aimed at expanding access to basic education which may include, inter alia: (i) construction and rehabilitation of various school facilities including ancillary facilities such as latrines and water supply; (ii) incentives for girls at the Junior Secondary School level to attend and stay in school; and (iii) Local Council and Community-based awareness activities to promote Basic Education. (b) Implementation of activities aimed at improving teaching and learning and increasing the presence and use of learning materials in the classroom which may include, inter alia: (i) provision of learning materials, teacher training and furniture; (ii) assessment of training provided through the project for primary and JSS teachers; and (iii) other activities which may improve the ability of children to learn (e.g.: deworming or other interventions). (c) Provision of support for the coordination, day-to-day management, monitoring and evaluation, and financial and procurement audits of the Program and the Project. A. Improving Access 101. Outstanding construction of Classrooms, Water Wells and 2X3 Ventilated Improved Pit (VIP) Latrines: Most components were completed; Even though 255 of the 285 Classrooms (89%), 40 of the 57 Water Wells with hand pump (70.2%) and 42 sets of 57 sets of VIP Latrines (74%) were completed, civil works still remains a challenge with several issues to be addressed. Several meetings 50 between the ESP Secretariat and the defaulting Contractors have been held to chart the way forward. All have indicated that they would commence the works by pre-financing them whilst awaiting the funds to be provided by the Government of Sierra Leone. Following these meetings, MEST and MOFED had a meeting on January 7, 2013 and the following decisions were reached:  The meeting agreed that the Government of Sierra Leone will finance the completion of the remaining Classrooms, Water Wells and VIP Latrines as indicated in the attached Letter of Commitment from the Ministry of Finance and Economic Development on behalf of the Government of Sierra Leone (See Annex 1).  It was agreed that if funds disbursed to any contractor exceeds work done, such contractor will be requested to complete work in line with funds disbursed within a time frame or else the contract will be terminated and the contract re-awarded and ultimately, such a contractor will be demanded to refund any money received in excess of work accomplished, then black-listed, and in severe cases face the Anti Corruption Commission.  In similar vein, the meeting agreed that if work done is in excess of funds disbursed, the contractor will be required to submit a valid performance bond to facilitate the disbursement of funds and completion of works. 102. Girl child Support Initiative. The 1st disbursement with respect to the payment of tuition fees for 50,648 JSS girls has been completed. A 2nd disbursement became necessary due to the variance between the enrolment figures of girls at JSS submitted by some Principals and that of the Census data. Having addressed this anomaly, the 2nd request for No Objection has been received from the World Bank for payment for an additional 24,213 girls in these affected schools. MEST is now awaiting the disbursement of the funds from the Loans Disbursement Group of the World Bank based at Chennai, India. There was extensive validation of the girls’ enrolment data by the three Civil Society Organizations, namely Every Child Matters, Centre for Local Government, Decentralisation and the Environment (CLOGADE), and Education For All (EFA) Coalition Sierra Leone. The parents and the rest of the Community were sensitized on the process of payment using Bill Boards, Fliers, Handbills, Radio Discussions and Jingles. For ease of implementation of subsequent Girl Child support activities, the process will be documented in an Operational Manual. B. Improving Quality of Teaching and Learning 103. The intermediate indicator’s target was achieved when In-service training was provided for 1000 primary and 260 JSS teachers on teaching methodology and classroom management nationwide. In the case of training using the Distance mode the target of 300 teachers were surpassed as 1,000 untrained and unqualified teachers gained their teaching certificate. Following this, a comprehensive evaluation of the one-year distance education teacher training program was recently completed. The evaluation found, inter alia, that the untrained and unqualified (UU) Teacher Training Programme was an appropriate teacher supply strategy given the post-conflict circumstances at the time of its inception. Over five thousand untrained and unqualified primary school teachers in the rural areas were trained between 2008 and 2012. Although the results of classroom observation of the trained teachers and anecdotal evidence indicated satisfactory performance, the examination results showed that majority of the teachers did not perform well. The evaluation recommended, inter alia, re-design of the programme if it should be adopted as a normal mode of training primary school teachers in the rural areas. Open and Distance Learning (ODL) features and principles should characterize its delivery. The yet-to-be established National Institute of Education, Training and Research (NIETAR) should assume responsibility for in-service training of teachers. 51 (i) International consultant facilitated the development of a Learning Assessment Framework, and trained 50 participants on test item writing. Participants’ developed an item question bank that will be used for upcoming national assessment. School and teacher questionnaires were also developed and piloted for future use. (ii) An international and local consultant developed a framework for the JSS curriculum after extensive consultations across the country with a wide group of people. (iii) School Management Committees (SMCs) Training: Training was conducted for 50 SMCs per district, a total of 7,000 members in 14 educational districts. The objectives of the training were to improve their governance and managerial skills with reference to:  increasing the knowledge of SMCs on school management/functions  ensuring effective and efficient functioning of SMCs countrywide and more especially to  capacitate them as school heads will be signing a Performance Contract with the Ministry of Education, Science and Technology. In the Contract, SMCs have been given prominent roles which are critical to the effective functioning of the schools and to the evaluation of the school heads. (iv) School-based de-worming. MEST in collaboration with the National School and Adolescence Health Program, National Neglected Tropical Disease Control Program, of Ministry Of Health and Sanitation and Helen Keller International conducted the school-based de-worming exercise on 30 May 2012 in 12 districts. This was an increase of four districts over 2011, and the number of children covered nearly doubled from nearly 650,000 to over 1.2 million. The report of the de-worming exercise, which was launched September 18, 2012, revealed that 85 percent of primary school children in the districts were de-wormed, which is well above the WHO expectation of 75 percent. This activity was aimed at providing access and improving school attendance Ministry noted that it was also essential to ensure that children attending primary schools are in good health, well nourished and mentally alert in order for them to be receptive to education during these critical years. (v) Procurement and Distribution of Teaching and Learning Materials. Several textbooks for Primary and JSS pupils were printed in Braille by the Education Centre for the Blind and Visually Impaired and distributed to the 6 schools for the blind in the following districts-Bo, Kenema, Western Area, Kono, Bombali and Koinadugu. The first set includes 162 Primary English text Bks 1-6; Structural English (108 copies-Bks 1-3); Social Studies (157 copies-bks 1-6); Primary Science (93 copies- Bks3-6): The second set consists of 350 sets of brailled Primary School Supplementary Readers and 100 copies of brailled Social Studies textbooks for blind JSS pupils; The third set is made up of 100 sets of brailled English and Integrated Science textbooks for blind JSS pupils; Also for the normal school children was 850,000 sets of pens, pencils, erasers and rulers, exercise books and 1, 700,000 Supplementary readers. (vi) Procurement and Distribution of Furniture. A total of (10,260) 3-seater pupil desk and chairs + 684 sets of teachers’ desks and chairs+ 114 sets of Office Tables and Chairs+ 114 Cupboards for primary Schools and 8,075 sets of 2-seater desks and chairs for JSS pupils + 171 sets of teachers desks and chairs + 57 sets of Office Tables and Chairs+ 57 Cupboards provided at JSS level. This furniture was for the EFA-FTI schools and those in dire need. The Local Councils were permitted to identify and distribute to the schools they deemed to be ‘dire need schools’. 52 C. Enhancing Coordination and Management of the Sector 104. School Census. The printed 2010/2011 School Census Report (Vol. 1) has been launched. Funding for the data analysis, report writing and launching ceremony by Hi.s Excellency the President on the 12th June 2012 was provided by the EFA-FTI Project. This report is divided into two main parts/sections – (i) the schools and (ii) the students. Each section is further subdivided into three main sub-sections-(i) National and regional (ii) local councils and (iii) changes. There are two additional sub- sections under ‘the students’ section-(i) other enrolment issues and (ii) comparison with JSS girls’ education support programme numbers. The Vol 2-‘Teacher Section’ of the Report is outstanding. Full funding support for the 2011.2012 census is from the EFA-FTI funds. Data for 2011/2012 school census has been collected, undergone consistency checks and are presently been analysed. The training of enumerators for school census to be conducted in 2013 has also been done with EFA-FTI funds. 105. Capacity Building of MEST Staff. Computer Training for 250 professional and administrative staff of MEST (186 staff in the North and West and 64 staff in South and East Regions): As part of its staff development strategy, the Ministry with funds provided by the Global Partnership for Education (GPE) Secretariat, undertook a computer training exercise as part of its in-service training programmes to enhance the computing skills of its staff located at Headquarters in the Western Area and in Districts in the North (Bombali, Port Loko, Kambia, Koinadugu and Tonkolili), East (Kenema, Kono, Kailahun ) and the South (Bo, Moyamba, Pujehun, Bonthe ) regions of the Country. The training was on Microsoft packages and focussed specifically on the following five basic modules:  Introduction to MS-Windows  Word Processing with Microsoft Word  Spreadsheet with Microsoft Excel  Microsoft PowerPoint  Internet In order to be able to achieve this task, it was necessary for the Ministry of Education, Science and Technology to have the necessary support in terms of technical assistance. In this regard, MEST with No Objection from the World Bank hired 2 (two) national consulting firms with the requisite experience and technical expertise to undertake this activity. (i) 10 MEST Staff trained in South Africa, 7 on M&E, and 3 staff form the ESP Secretariat in Financial Management, Project Management and Procurement methods. (ii) Improvement in communication amongst MEST Officials and between MEST and Partners through the Installation of Local Area Network (LAN) Connectivity and PABX Intercom system. (iii) ESP Secretariat established with the recruitment of 3 (Three) additional staff, namely, the Finance Officer, Programme Assistant and Finance Assistant to join the Education Sector Plan Coordinator. The aim of this recruitment was to give support to the ESP Coordinator in ensuring that the ESP is implemented well. 106. Challenges. The EFA-FTI Project implementation delays have been largely caused both by MEST internal institutional issues as well as slow, unclear communication and poor coordination between the Bank and MEST. 107. Firstly, the Project’s lifespan was for 3 years in accordance with the implementation plan for the basic education activities. Sierra Leone was received into full EFA-FTI partnership by the EFA FTI Expanded Catalytic Fund Strategy Committee meeting in Bonn (Germany) in May, 2007 with the approval of US$13.9 million for activities towards achieving universal basic education. Immediately 53 following thereafter, was the introduction of new EFA-FTI guidelines that should govern the project’s implementation. Several appeals from the Government of Sierra Leone and the in-country Education Development Partners to the EFA-FTI Secretariat to allow us to operate under the old guidelines at the time the grant was being allocated was to no avail. Prompt response from the Bank would have avoided delays in MEST fulfilling the newly introduced requisite conditions before the grant agreement could be finally signed a year later on the 15th September 2008. Added to the initial delay, it must be considered that for the grant to be effective, it had to be further ratified by the Parliament of the Republic of Sierra Leone, which was done on December 16, 2008. Thus, it should be noted that though the grant was allocated in 2007, the first tranche of US$3,000,000 (Three Million United Sates Dollars) was only received on 3 June 2009, two years later as a result of the reasons stated above. 108. Secondly, it was agreed that the Ministry of Education, Science and Technology will mainstream and implement project activities without a Project Coordinating/Implementation Unit (PCU/PIU) as was the case in the previous education projects. The Officials in the Ministry were to be fully responsible for the project’s implementation with support from the Decentralisation Secretariat (DECSEC) and the Local Government Finance Department (LGFD) working on devolved basic education activities. MEST had to therefore grapple with and resolved a number of issues as follows:  The ESP Secretariat in the Ministry was understaffed. It was the responsibility of one coordinator and the job was too big.  The former head of Procurement Unit died and getting a replacement took some time  The Principal Accountant and the present Procurement Manager were new and had to be oriented on their duties, especially with reference to World Bank procedures.  The new Procurement Manager being an administrator and not a staff in the professional wing of MEST was transferred to another Ministry bringing all ongoing procurement activities to a temporary halt.  A new Administrative Head (Permanent Secretary) took up position in the Ministry in April 2010 in the middle of the project implementation and had to be acquainted with the EFA-FTI Project and implementation issues as he was a Principal Signatory to the designated account held at the local bank.  Delays by line Ministry (MOFED) in processing Withdrawal Applications.  The Procurement and Finance officers of the 19 Local councils also had some capacity issues on relevant procurement and financial management practices  The Memorandum of Understanding for contributing partners with the regards to the operation of the pooled fund was only signed in August 2009.  The mainstreaming of the project has also clearly brought out the challenge of key personnel in MEST (Procurement Manager, Principal Accountant), members of LGFD and DECSEC handling additional duties outside of their normal schedules.  Finally, the Loans Disbursement Group in Chennai should always and promptly indicate shortfalls or gaps in Withdrawal Applications received for the Government to avoid delays in the processing. 109. Additionally, some of the above constraints stemmed from the following issues:  Insufficient orientation to MEST on changes in World Bank procedures during the initial and follow-up process  Unclear guidelines on the level of flexibility deemed acceptable in the implementation of the work plan and on the reprogramming of the funds  Unclear guidelines on which level of activities in the plan would require no-objection from the Bank  Delays in processing no-objection requests 54  Slow follow-up from Bank to rectify procedures where MEST may have not fully satisfied World Bank requirements  Inconsistencies in Chennai and local World Bank Office responses to MEST’ queries in the case of unclear procedures  Lack of clarity related to the position of Sierra Leone with regards to the EPDF fund and of the objectives of the EPDF- and communication from the Bank to the EDP group regarding the fact that the Catalytic Grant (all for capacity building around Procurement, Finance and EMIS for reporting) could be used to cover these activities was only received in early November 2010, after the initial submission for the EPDF monies was submitted to WB Accra in Feb 2009, and after numerous edits and appraisals of the application by the EDP Group and resubmissions to WB Accra.  Although the fund follows IDA procurement rules, processes also have to satisfy National Procurement Procedures. Delays have been caused by these National Procedures even when IDA no objection has been granted.  TTL was not based in Freetown – need more support from the Country Office. This led to Communication issues with the Bank.  Monitoring of Civil Works was a challenge for the Education Sector Plan (ESP) Secretariat as the staff were instructed by the Bank to concentrate on coordinating the implementation of the Project and not being involved in monitoring and tracking Civil Works for which the Bank had indicated that the LC Engineers and DECSEC were more than competent to undertake the monitoring. The result was that the ESP Secretariat acting on the submission of the reports and requests for payment, including valuation certificates from LCs through LGFD of the Ministry of Finance, before the funding modality was modified, effected payments on the supposition of the competence and reliability stressed by the Bank. This was proved negative when subsequent monitoring by the ESP Secretariat and the independent engineering company contracted by the Bank observed that the reports and payment requests did not match with what was actually on the ground. 110. Revised Institutional arrangements. As a result of the above challenges leading to the restructuring of the project, the following points provided evidence of the Government’s and World Bank’s effort to improve the project implementation: (a) the original design of the project and the resulting work plan, which focused mainly on inputs that led to procurement delays, has been redesigned to focus on activities that would impact on system change and quality, thereby improving the implementation profile; (b) other relevant ministries (Finance, Local Government) and units (Decentralization Secretariat), Local Government Finance Department (LGFD) working under the Integrated Project Administrative Unit (IPAU) are now represented in the planning and implementation meetings; (c) the planning and implementation of the various activities is being mainstreamed into the broader MEST rather than the Education Sector Plan (ESP Secretariat); (d) there will be more joint planning with the Decentralization Secretariat (DECSEC) and the local councils as the project implementation progresses; (e) the flow of funds to contractors and local councils will be streamlined to involve the Local Government Finance Department as a monitor, but not a conduit for payments; (f) the project will now utilize a Rapid Results Approach (RRA) for implementation at the community level; (g) the intervention of the Bank to push for the recruitment of additional staff to support the ESP Coordinator in the ESP Secretariat has been a plus to the project, thus the presence of a newly recruited finance officer has impacted significantly on performance in the submission of the Quarterly 55 Financial Management Reports, the execution of the outstanding Audit activity for 2009 and 2010, the conclusion of the 2011 Audit and the increase in the disbursement rate from 2 percent in two years (June 2009-June 2011) to 64 percent and in 2012 to 82 percent. As indicated earlier, the recruitment of three (3) additional staff in the ESP Secretariat has improved its capacity to implement effectively and efficiently the newly designed detailed work plans, including the detailed time frame; and (h) by the insistence of the bank, the project was put on the prioritized list of activities of the Government of Sierra Leone, and was monitored by the Education Task Force set up by the Office of the President. (i) As the TTL was not based in Freetown, which was an early challenge, the Bank being significantly pro-active supported the ESP Secretariat by introducing more intensive and more frequent missions, regular communication via emails and phone calls, and even video and audio conferences with the TTL and the Loans Disbursement Group in Chennai, which helped greatly in salvaging the project and producing the positive turn around in 2010 as critical issues were now speedily addressed, such as the Bank issuing No objection Letters in six (6) minutes after the submission of the request. (j) The ESP Secretariat having played a pivotal role toward a successful implementation of the EFA-FTI project should be provided with additional capacity training opportunities for effective and efficient service delivery in future. 1. LESSONS LEARNED 111. Project implementation readiness is key to achieving project objectives. Project preparation should include the team that would implement/coordinate project activities so that they can contribute to preparation and design and have roles clarified prior to implementation. 112. A thorough analysis of the capacity of participating LCs, contractors and other implementing partners including the Ministry of Education and line Ministries prior to implementation would have curtailed implementation time. 113. Projects are subject to implementation difficulties when clear cut guidelines are not established. 114. World Bank policy of awarding contracts to the lowest bidder has created problems in the implementation of some activities. However, contractors/suppliers should be evaluated using accurately estimated costs indicated as a key criterion for the award of contracts, to avoid under quoted prices. 115. Goods –Teaching and Learning Materials should be delivered by Suppliers directly to the Local Councils and not to MEST Headquarters Warehouse. 116. Local Councils should be informed in time of their roles to distribute the necessary TLM to enable them budget for this activity. 117. The community should be fully involved in the project to create a sense of ownership and confidence in the implementation. 118. The Project Implementing/Coordinating Unit (PIU/PCU/ESP Secretariat) should be allowed by the Bank to undertake periodic monitoring and tracking of interventions, as in this case Civil Works and not depend entirely on reports submitted by the Local Councils and other intermediaries, who were considered to be competent in those areas. This would have prevented overpayments to contractors for works not actually done. 56 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not Applicable 57 Annex 9. List of Supporting Documents Aides-memoire and Implementation Status Reports 2007 – 2012 World Development Report 2011: Conflict, Security, and Development, World Bank, 2011. Country Assistance Strategy, World Bank, 2005. Sierra Leone Education Sector Plan: A Road Map to a Better Future, 2007-2015, MEST, 2007. Sierra Leone 2010/11 School Census Report. Draft Sierra Leone Education Status Report, 2012 Implementation Completion and Results report ICR No. ICR00001528, Rehabilitation of Basic Education Project (REBEP-IDA-H0200). Portfolio Restructuring in a Post-Crisis Environment in Côte d’Ivoire, Project Paper, World Bank, 2008. Project Paper for a proposed restructuring, World Bank, 2010 & 2011. School Construction Strategies for Universal Primary Education in Africa - Should Communities Be Empowered to Build Their Schools? Serge Theunynck, World Bank, 2009. 58 Map IBRD 33478 IBRD 33478 Sl ERRA LEONE SELECTED CITIES AND TOWNS - - - MAIN ROADS ® DISTRICT CAPITALS - - RAILROADS ® NATIONALCAPITAL - - DISTRICT BOUNDARIES ~ RIVERS - ·- INTERNATIONAL BOUNDARIES 120W I I OW GUINEA IO'N ·-·- ·- ·- - ·- · --... IO'N GUINEA 9' N Bcnono Islands 8' N ATLANTIC OCEAN 7' N 0 20 40 60 Kilometers 0 20 40 50 Miles 120W l l 'W NOVEMBER 2004 59