ICRR 13909 Report Number : ICRR13909 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 01/29/2013 Country : Eritrea Project ID : P070272 Appraisal Actual Project Name : Eritrea Education US$M ): Project Costs (US$M): 59.5 88.8 Sector Investment Project L/C Number : C3794 Loan/ US$M): Loan /Credit (US$M): 45.0 47.6 Sector Board : Education Cofinancing (US$M): US$M ): 41.2 Cofinanciers : European Commission Board Approval Date : 06/19/2003 Closing Date : 02/28/2009 08/31/2011 Sector (s): Primary education (62%); Secondary education (30%); General education sector (8%) Theme (s): Education for all (100% - P) Prepared by : Reviewed by : ICR Review Group : Coordinator : Susan Ann Caceres Judyth L. Twigg Navin Girishankar IEGPS2 2. Project Objectives and Components: a. Objectives: According to the Project Appraisal Document (PAD, p. 2, p. 34), the objectives were "increased and more equitable enrollment in basic education and enhanced quality in basic and secondary education ." The objective noted in the Financing Agreement (p. 13) was "to assist the Borrower's effort to reform and develop its education sector to attain increased and more equitable enrollment in basic education and to enhance the quality of its basic and secondary education ." While the wording is slightly different, each version states the same objectives . b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: While the project contained five components, it was to be implemented in two phases (ICR, p. 2; PAD, p. 8). Activities supported during the first phase were those with complete implementation plans at the time of project preparation. Activities included: (1) expansion of access to basic education, where plans and designs were developed and sites identified; (2) teacher in-service training and support for teachers and temporary financing of expatriate teachers; (3) curriculum reform; and (4) implementing the national assessment for Grades 3 and 5, and support for the development of plans for components requiring more preparation (ICR, p.2 and PAD, p. 8). The second phase was to continue the previous activities and begin the remaining ones (ICR, p. 2). The second phase was to start as implementation plans were established (ICR, p. 8). 1. Increasing Equitable Enrollment in Basic Education (appraisal US$18.06 million; actual US$5.40 million) was to support the improvement of access to education through the construction of low cost but durable classrooms and ancillary facilities and support . It was expected to construct 620 elementary and 593 middle level classrooms. 2. Improving Quality of Basic and Secondary Education (appraisal US$22.91 million; actual US$31.90 million ) was to contain seven sub -components: Teacher Training, Development and Professional Support was to provide immediate support to improve the quality of teachers in -service while developing a master plan for teacher recruitment, training, deployment and utilization . Fast-track teacher education programs were to be developed and implemented to orient elementary school teachers who would be transferred to middle schools in September 2003, as well as high school graduates who would be recruited for elementary school positions. A new distance learning program was also to be established to train middle school teachers . Improvements to the physical facilities at teacher training institutes were also to be made . Support for increasing the supply of Eritrean secondary school teachers and the consequent phasing out of expatriates from the teaching force were to make an important contribution to quality at this level . Replacement of Expatriate Teachers by Eritrean Teachers at Secondary Level was to finance expatriate teachers on a short -term basis until local capacity was built up, and develop alternative strategies for training Eritrean nationals in shortaged subjects . Curricula and Pedagogical Reform was to enhance the capacity of the Curriculum Development Division to plan for and implement ongoing curriculum reforms and the capacity of curriculum implementers to deliver the new curriculum . Textbook Printing, Distribution and Utilization was to support the production, copying, and distribution of materials, as well as training for teachers in the effective utilization of textbooks and materials . Expansion and Strengthening of National Assessment and Examinations was to provide technical assistance for design and support for carrying out national assessments at three levels during the first, third, and final year of the project . Reducing Overcrowding in Secondary Schools was to support construction planning and management for 390 secondary classrooms and facilities to reduce current overcrowding and provide an environment more conducive to learning . Computer Classrooms for Secondary Schools was to further refine a Government plan to provide computer classrooms in each of the 47 secondary schools. During the second phase of the project, 20 schools were to be provided with computers and teacher training . 3. Enhancing Capacity for Service Delivery and Sector -wide Planning and Monitoring (appraisal US$1.80 million; actual US$4.40 million ) was to provide a Planning Unit to coordinate policy development, planning and management activities to attain readiness for a sector -wide approach and its subsequent implementation. This component was also to strengthen the delivery capacity of the Ministry of Education, Zobas, and sub-Zobas through provision of equipment, work space, scholarships for the training of vocational educators, training of lecturers, and implementation of a comprehensive capacity development program. 4. Supporting Equitable Provision of Education (appraisal US$1.13 million; actual US$2.20 million) was to reinforce the knowledge base in the education sector through action -oriented research aimed at improving education for girls and other disadvantaged children, as well as interventions to provide incentives to reduce the opportunity cost of schooling and sensitize communities to raise awareness of the benefits of sending children to school. 5. Project Management (appraisal US$1.10 million; actual US$1.10 million) was to support the management of the project. The European Commission Trust Fund provided support for the construction of additional primary and middle schools, two education support facilities, radio extension, three Technical Vocational Education and Training centers, and capacity building for the Ministry (ICR, p. 5). The project's components were revised in June 2007 to account for the additional funds provided by the European Commission (EC), with Bank resources focusing on the following changes : Component 2 provided more secondary school places; extended the utilization of expatriate teachers; provided technical assistance for curriculum reform, including e -content provision; and provided additional learning materials for textbook printing and distribution . Component 3 intensified capacity building activities and provided more training and study opportunities for better education sector management and service delivery . Component 4 piloted a scholarship program for girls and disadvantaged children, and supported early childhood development centers in rural villages to improve the possibilities for rural children's school access and retention. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: Project Cost : Total project costs (including those from IDA and the European Commission ) were US$88.8 million. As a result of the additional funds from the European Commission Trust Fund, IDA resources were reallocated from civil works to other areas (see discussion above). Component One was therefore reduced from the anticipated amount of US$18.0 million to US$5.4 million (ICR, p. 5). The project team reported that the project registered an approximately US$ 5 million increase due to exchange rate gains, most of which were absorbed by expanding project activities . Financing : IDA provided US$47.6 million. Midway through the project in 2007, money from the European Commission Trust Fund was added, doubling the financial size of the project . The European Commission planned to contribute US$ 44.96 million, but by the close of the project it provided US$ 41.2 million to assist with construction to improve access in primary and secondary schools and coverage in the Technical Vocational Education and Training Centers (ICR p. 24). Borrower Contribution : The Borrower was to contribute US$ 14.50 million, but no information is available about its actual contribution (ICR p. 24). Dates The project became effective on September 23, 2003 and closed on August 31, 2011. This was two and Dates: a half years later than originally planned . On June 28, 2007 the additional resources from the European Commission were added to the project and the components were revised . The project was initially extended from February 28, 2009 to February 28, 2011, with a second extension to August 31, 2011, when the project closed. The extensions were granted to accommodate implementation delays and reallocate funds to cover the escalated costs of civil works and equipment, as well as to accommodate actions related to the European Commission Trust Fund and Administrative Agreement (ICR, p. 6). 3. Relevance of Objectives & Design: a. Relevance of Objectives: Objective : Substantial The objectives had substantial relevance . The education sector in Eritrea at appraisal faced a number of challenges: shortage of qualified personnel for the schools and system, weak student learning achievements, overcrowding in schools, low enrollment rates (particularly for girls), and educational inefficiencies . There were concerns of inequity, since there were some groups such as girls, nomadic children, and other minorities that had lower access to schooling, as well regional disparities (PAD, p. 5 and 6). The objectives addressed all of these issues and were consistent with the Bank's Interim Support Note (2005-2007), which emphasized the need for an education development program to increase access to quality education for all school children (p. 14). After project effectiveness, the Government produced a series of reports that described its main priorities for the education sector: (1) improve access and equity in basic education; (2) improve quality of education by consolidating teacher training programs; (3) promote science and technology by intensifying vocational education and training; and (4) expand provision of education by involving the private sector and by promoting strengthening of the formal and non -formal, distance, and out-of-school learning and education programs (ICR, p. 1). b. Relevance of Design: Design : Modest The project used a comprehensive approach to address access to basic education through school construction, teacher development/recruitment, incentives to attract girls and reduce opportunity costs, and strengthening of Ministry capacity. However, this was not the case with the objective to improve education quality . The interventions designed to improve education quality focused on providing the basic level of necessary inputs (rapid increase in the supply of trained teachers, development of curriculum /textbooks) that were lacking in the system, but these interventions were not sufficient to attain the objective of education quality improvement . In other words, in this part of the results chain, the planned activities could not reasonably have been expected to achieve the objective. The project was designed to be implemented in a two -phased approach, since the Bank did not believe the Government had the capacity to implement a sector -wide program (ICR, p. 2). This was done to provide more time for the Government to develop plans for areas needing further analysis . However, this phased approach put more pressure on the Ministry of Education and the project management unit to implement the first phase activities while simultaneously preparing the wider sector program (ICR, p. 7). This created implementation difficulties throughout the life of the project . 4. Achievement of Objectives (Efficacy): The ICR notes that other donors were supporting Eritrea's education sector during the project's lifetime . The interventions from the other donors (African Development Bank, European Commission, Danish International Development Agency, and United Nations Children's Fund ) supported the Government's training of teachers in the new curriculum and developing, printing, and distributing textbooks for all grades (ICR, p. 9). The African Development Bank also provided computer knowledge and equipment to 69 secondary schools (ICR, p. 9). These activities were conducted in parallel with the Bank's activities, and therefore the extent of spillover effects and precise attribution cannot be determined . As previously noted, some targets were revised upward during the course of the project, when additional financing was provided for school construction . This review presents the attainment in relation to both original and revised targets, since the revised targets were not based on reliable data and were set unrealistically high (ICR, p. 8) and so it is more accurate to assess progress in relation to the original targets . Increased enrollment in basic education : Substantial Outputs: 12,907 additional elementary school students were enrolled by the end of the project, not reaching the original target (39,000 additional students) or the revised target (165,309 additional students). The scope of the proposed enrollment increase was high, considering that the 2002 Demographic and Health Survey found a gross enrollment rate of 100% in elementary schools (ICR, p. 13-14). 78,034 additional middle school students were enrolled by the end of the project, meeting the original target (49,000 additional students) but not meeting the revised target (112,436 additional students). A cost-effective school design was prepared and used in civil works and 1,638 classrooms were built or rehabilitated, exceeding both the original (1603) and revised (1400) target. 60 teachers' quarters were constructed . An extension was built to the Asmara Teacher Education Institute, increasing its capacity and resulting in the training of 240 additional teachers. Outcomes: The primary school enrollment rate slightly declined from 67.5% in 2008-2009 to 66.1% in 2009-2010, which did not meet the original target of a 5% increase (72.5%). (This target was not revised during restructuring.) The ICR (p. viii) questions the accuracy of the rate, since it was based on flawed population data. The primary school retention rate increased from 68% in 2008-2009 to 79% in 2010-2011. The original target (78%) was met. (This target was not revised during restructuring .) The primary school promotion rate for boys increased from 78% in 2008-2009 to 81% in 2009-2010. The original target (88%) was not met. (This target was not revised during restructuring .) The transition rate between elementary and middle school slightly increased from 89% in 2001 to 90% in 2009. The elementary pupil-classroom ratio improved from 63:1 in 2001 to 53:1 in 2010 The middle school pupil-classroom ratio improved from 83:1 in 2001 to 49:1 in 2010. More equitable enrollment in basic education : Negligible Outputs: The share of girls in elementary school enrollment remained the same (45%) from the beginning to the end of project, not meeting the original (increased from current level) or revised (50%) target. The share of girls in middle school enrollment remained the same (45%) from the beginning to the end of the project, meeting the revised target (45%) but not meeting the original target (an increase). The share of girls in secondary school enrollment increased from 37% to 43%, meeting both the original target (an increase) and the revised target (40%). The ICR indicated that the share of girls is now similar across all three cycles (basic, middle, and secondary), suggesting that once girls enroll, they remain in school and progress through the system at almost the same rate as boys . A study on interventions targeting girls and other disadvantaged children was conducted . 3,000 girls or disadvantaged children received grants /incentives to enroll in school for two years, exceeding the target of 2,300 (this target was not revised). While the PAD noted inequitable access for girls, nomadic children, other minorities, as well as regional disparities, the ICR presented no data to show that enrollment among nomadic children and other minorities had increased. The ICR (p. 26) reported "disparities amongst student enroll [ment] rates at the elementary level improved, with the Southern Red Sea Zoba having closed the greatest gap (jumping from 9.8% in 2000/01 to 47.4 in 2009/10) but the Zobas Debub and Maekel continue to have the highest gross enrollment rate . However, the ICR did not present other regional information to be able gauge the extent to which regional disparities in enrollment had decreased. The ICR did not report enrollment across socio -economic groups. Outcomes: Primary school promotion rate for girls increased from 82% in 2008-2009 to 84% in 2009-2010. The original target (91%) was not met. (This target was not revised during restructuring .) The ICR did not discuss the extent to which Government promotion policies were a contributing factor to changes in this indicator . The ICR did not report school promotion across socio -economic groups. Enhanced quality in basic and secondary education : Modest Outputs related to education quality : A new curriculum was developed for grades 1-12. A Capacity Assessment of the Ministry of Education was conducted, one of 15 Policy Studies that were completed. 5.2 million new textbooks were developed, printed and distributed, exceeding the target of 4.95 million (this target was not revised). 44 computer laboratories were established and equipped, exceeding the target of 20. 782 community-based Early Childhood Development (ECD) Centers received grants to finance recurrent costs. The ICR (p. vii) indicates that no data were available on enrollment in these Centers; however, it appears that the number of grants disbursed would not be sufficient to reach the original enrollment target of 80,000 pre-school age children. Outputs related to training: 220 Ministry of Education management staff were trained, exceeding the target of 150 (this target was not revised). 102 Technical Vocational Education and Training instructors received training . The share of qualified teachers improved from 35% to 61%. 1,015 uncertified female teachers and 254 uncertified male teachers were upgraded . 4,522 teachers participated in a summer training program, exceeding the target of 2,500. 1,600 teachers successfully completed a distance learning program, out of 2,080 who entered the program. This did not meet the target of 2,080 completing the program. 10 Pedagogical Resource Centers were constructed . The ICR provides no evidence of the effectiveness of the training . Outcomes: The share of schools that use double -shifting declined from 56% in 2001 to 48% in 2010. The primary school completion rate improved from 45% in 2001 to 51% in 2010. Student repetition rate at the elementary level decreased from 23% in 2001 to 13% in 2011. Original target (10%) was met. (No revised target, since this indicator was dropped during restructuring .) Student repetition rate at the middle level decreased from 23% in 2001 to 12% in 2011. Original target (10%) was met. (No revised target, but this indicator was dropped during restructuring .) Student repetition rate at the secondary level decreased from 29% in 2001 to 4% in 2011. Original target (10%) was met. (No revised target, but this indicator was dropped during restructuring .) Student dropout rate at the elementary level remained at 6%. Original target (5%) was not met. (Target was not revised, but this indicator was dropped during restructuring .) Student dropout rate at the middle level decreased from 10% in 2001 to 6% in 2011. Original target (5%) was not met. (Target was not revised, but this indicator was dropped during restructuring .) Student dropout rate at the secondary level remained at 9%. Original target (5%) was not met. (Target was not revised, but this indicator was dropped during restructuring .) The elementary pupil-teacher ratio improved from 45:1 in 2001 to 41:1 in 2010. The middle school pupil-teacher ratio improved from 56:1 in 2001 to 42:1 in 2010. 40% of Grade 3 learners attained the minimum mastery level, which was a decrease from the 2001 baseline (58%) and did not meet the target of 62%. 57% of Grade 5 learners attained the minimum mastery level, increasing from 32% in 2001 and exceeding the target of 37%. The ICR (p. iv and p.16) indicates that the testing instruments were not the same over time, raising issues of comparability of results . The Grade 8 national examination pass rate increased from 71% in 2005/2006 to 74.8% in 2011, not meeting the target of 76% (this target was not revised). The ICR did not discuss the degree to which Government promotion policies contributed to the increases in the completion rate and decreases in the repetition rate . 5. Efficiency: Efficiency is rated Modest. The ICR reports some attainment of efficiency in school construction through the use of cost -effective designs, but construction costs remain higher than in neighboring countries because of exchange rate distortions, centralized price-setting of imported materials, limited competition in the construction industry, and the Red Sea Trading Company's monopoly on imported building materials (ICR, p. 37). For example, the elementary per-classroom costs ranged from US$ 26,891- US$29,430, depending on procurement method . Costs were lower for schools built under the community participation model than under National Competitive Bidding, but the ICR did not report cost comparisons with neighboring countries . The Ministry of Education was able to achieve some efficiency through the large -scale printing of textbooks, with a reduction in the unit cost from appraisal (less than US$1) to US$0.69 per book (ICR, p. 37). ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal No ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: There was substantial relevance of objectives, while relevance of design was modest . One objective was rated substantial for efficacy, with the others were rated modest and negligible . Efficiency was modest. Less progress was made in improving equitable access of girls to primary education . While there were some improvements in education quality (e.g. reduction in pupil-teacher ratio), there was limited progress on dropout rates and examination scores. The extent to which Government promotion policies contributed to increases in completion and promotion rates and decreases in the repetition rate is not clear . a. Outcome Rating : Moderately Unsatisfactory 7. Rationale for Risk to Development Outcome Rating: While the government remains committed to the Education Reform Program and its capacity to implement projects has improved, there are two significant risks to sustaining progress : resources and donor commitment. Some of the development partners are no longer active in the education sector, which eliminates much needed external resources. The government lacks its own resources to sustain development outcomes . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: A large preparation team was deployed by the Bank to respond to the government's desire to prepare and implement quickly. Project preparation was rushed in an effort to respond to the Government's request, but this led to incomplete and uneven preparation of the components, which in the case of civil works led to major delays in construction and project implementation (ICR, p. 6). The project team later reported that the Government cancelled the licenses of all private engineering and construction firms shortly after the project became effective, a factor essentially outside the Bank's control that led to significant delays . The Bank team recognized the low capacity of the government to implement a project focused on strengthening and improving every sub-sector of the system and so proposed to implement the project with a focus first on specific areas, while phasing in others to allow for more analysis where the program needed more development. However, this approach was still too ambitious given the capacity of the Ministry, which was pointed out by a Quality of Supervision Assessment conducted in September 2006. As noted by the ICR (p.7), "the lack of project readiness and limited capacity within the Ministry for the phased project implementation present[ed] problems for project implementation throughout its life ." There were weaknesses with the original Results Framework. Indicators related to improved learning were not defined and probably not achievable, given the project's focus /activities. at -Entry Rating : Quality -at- Moderately Unsatisfactory b. Quality of supervision: Project restructuring addressed issues such as the continued need to finance salaries for caregivers in early childhood development centers and expatriate teachers . The midterm review resulted in the introduction of activities that enhanced project implementation (a Rapid Results Approach (RRA) to facilitate community participation construction activities, decentralized project implementation to accommodate the RRA and incentive schemes, and more monitoring and evaluation ). Close supervision by the Bank's procurement staff helped the Project Management Unit to develop capacity . The Bank raised procurement thresholds, which gave more flexibility to communities to construct elementary schools with up to eight classrooms . Bank support to the Rapid Results Approach resulted in all but US$ 216,000 of the Trust Fund being disbursed, suggesting that the revised approach resulted in substantial increases in disbursements and construction . Implementation progress occurred during the final eighteen months of the project . The Bank team worked well with the Ministry of Education and Implementing Agency, despite deterioration in the relationship between higher levels of the Government and Bank management . However, there were several major shortcomings in Bank supervision . Despite the fact that only 2% of the allocation for construction had been disbursed and project supervisory ratings were in the unsatisfactory range, the Bank team accepted a grant from the European Commission Trust Fund that increased the scope of construction and doubled project financing (ICR, p. 8). There were five TTLs over the course of the project, which led to a slow response from the Bank, particularly related to no objection letters, responses to Government requests, and provision of technical support, which slowed implementation . While it was appropriate to increase the targets to reflect the additional construction outputs from the Trust Fund, the revision of the other project development indicators was not based on sufficient analysis or reliable data, resulting in many indicators that were neither achievable nor good measures of project activities/achievements (ICR, pp. 8, 12). The Bank did not provide adequate supervision of environmental safeguards in relation to project construction activities, since safeguards specialists only supported supervisory missions during the last eighteen months of the project, and the Bank -approved designs did not address sanitation needs in teachers' quarters (ICR, p. 11). "During the last two years of implementation, the Bank made an effort to rectify this situation, but there were still teachers' quarters that did not receive water and latrines and few school fences " (ICR, p. 21). As a result, the project fell short of compliance with environmental safeguards. Quality of Supervision Rating : Moderately Unsatisfactory Overall Bank Performance Rating : Moderately Unsatisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government produced an Education Sector Development Program that proposed areas for sector improvement. The Government was committed to implementing a sector -wide program, but it did not have the plans or institutional capacity to fully implement the program . The Government of Eritrea cancelled the licenses of all private engineering firms and construction companies, which negatively impacted project implementation. Conflict of interest was found between an architectural consultant and construction firm, and the Government suspended licenses of private contractors in order to review and modify regulations, which slowed implementation. All construction materials were imported through the Red Sea Corporation, which led to significant stress on the Bank's procurement rules . The Government went into arrears two times, which impacted project implementation during the last seven months, since the Bank was not able to process payments and Letters of Credit for contracts and calls for funds from the EU Delegation (ICR p.9), Government Performance Rating Moderately Unsatisfactory b. Implementing Agency Performance: A Project Management Unit (PMU) implemented the project, while at the same time working with the Ministry to prepare a wider sector program and plans for phase two of the project . The Project Management Unit worked well with different donor procedures . The implementing team was organized, available to Bank mission members, and engaged in open and frank discussion . The ICR reports that implementation skills steadily improved over the project's lifetime . Positive aspects of performance included institutionalization of a process of curriculum and textbook development, with ownership of the copyrights to all the materials . However, there were shortcomings . There was not full compliance with environmental safeguards . The Project Management Unit coordinated its activities with various Directorates within the Ministry of Education, but there was not timely sharing of documentation and data . The Project Management Unit was not able to provide updates on project activities; instead, other units within the Ministry had to be contacted . No schools were constructed during the first three years of the project, since it took the unit 24 months to hire a consultant and develop school designs . The architectural consulting firm developed school designs that were inconsistent with the project's goals of cost reduction and working with communities and district authorities (ICR, p. 7). This situation was resolved after the midterm review by pursuing more community participation in the construction . Implementation was very slow throughout most of the life of the project . Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: There was a logical alignment between the objective and outcome measures in the original Results Framework . However, the indicators related to improved learning were not defined and probably not achievable, given the project's focus/activities. The targets appear high. Many of the indicators were outputs of specific project activities (e.g. plan, document, etc.) and do not provide evidence of the impact or effectiveness of the interventions (e.g. teacher training, curriculum reform). Enrollment data was only disaggregated by gender, not allowing the tracking of improvements among other groups /regions with inequitable access . Household survey data were not used to assess enrollment by socio -economic groups. b. M&E Implementation: During restructuring, the Results Framework was revised . Targets were increased, but these increases were not based on reliable data and did not utilize the Demographic and Household Survey (2002), which showed a declining school-age population. The new targets were therefore inappropriate, unachievable, and were also not based on project interventions . For example, to reach the new target of elementary school level gender parity, enrollment of an additional 15,000 girls was needed, while project interventions were designed to benefit only 3,000 girls. Thus, the revised Results Framework was weakly associated with project activities (ICR p. 12). Moreover, middle school enrollment targets were not based on analysis (ICR p. 14). Intermediate outcome measures of quality such as completion rate at elementary school and repetition rates were dropped . The ICR did not discuss the Government's promotion policies and so it is unclear whether dropping these indicators was advisable or not, given that these policies may contribute to changes in retention, promotion, and completion rates. The PMU collected data on some project outcomes and output indicators, while other directorates in the Ministry also collected other data. There was no central repository for data . Data were appropriately shared with the Bank. Geographic Information System (GIS) technology was purchased and assisted with school mapping and school-level data. GIS data were shared in a statistical yearbook . A database with information about all schools in the country was created and allowed the government to make decisions about school construction . A separate Education Management Information System was developed for the entire education sector and was used to compile annual education statistics . However, this system was not linked with the other one (e.g. GIS). The integration of the two systems would have made tracking easier and improved the effectiveness of M&E (ICR, p. 11). c. M&E Utilization: Two times a year a progress report was prepared to provide information on the status of project implementation and data on outcomes and indicators . These reports were then used by the government, Bank, and EC . For example, the project team reported that these reports were used to monitor implementation progress, identify bottlenecks, and make project -related decisions. M&E Quality Rating : Modest 11. Other Issues a. Safeguards: According to the Project Appraisal Document (p. 30), two safeguard policies were triggered by the project : Environmental Assessment (OP 4.01) and Involuntary Resettlement (OP 4.12). The project was classified Category B for Environmental Assessment because of the classroom and school construction . The PAD (p. 28) noted that an Environmental and Social Management Framework (ESMF) and Resettlement Policy Framework were prepared by the Government in compliance with Bank policies . The Bank team did not adequately supervise environmental compliance . A safeguard specialist supported the team only during the last eighteen months of the project, and the Bank -approved designs did not take into consideration sanitation needs in teachers' quarters in an effort to build as many classrooms as possible (ICR, p. 11). "At the closing of the project and with Bank support, some modest improvements in compliance with environmental safeguards were noted, particularly regarding water and sanitation and providing fences for most schools . However, it fell short of overall compliance" (ICR, p. 11). b. Fiduciary Compliance: The Government imported all construction materials through the Red Sea Corporation, leading to stress on Bank procurement rules. Quarterly financial reports were provided in a timely manner and agreed format . The final financial report was being reviewed by the Bank at the time of the writing of the ICR, but annual audit reports were previously judged to be satisfactory and unqualified (ICR, p. 12). The PMU conducted financial transactions according to project implementation arrangements and had a monitoring system in place . There was adequate staff in the accounting unit . Given the decentralized flow of money for school construction, decentralized accounting functions were reinforced and manuals were established for each Zoba office . During the first three years of the project, there was limited experience with handling procurement issues, which caused delays. After training and hiring more staff, the procurement unit was able to handle the process and increased volume efficiently. Procurement reviews showed that procurement was conducted based on guidelines and revealed no major discrepancies (ICR, p. 11). c. Unintended Impacts (positive or negative): d. Other: None 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Moderately Moderately Unsatisfactory Unsatisfactory Risk to Development Significant Significant Outcome : Bank Performance : Moderately Moderately Unsatisfactory Unsatisfactory Borrower Performance : Moderately Moderately Unsatisfactory Unsatisfactory Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: The ICR (p. 22-23) reports several lessons: A sectorwide approach in post conflict countries may not be appropriate, when there is lack of government commitment to donor coordination and weak government capacity . Safeguard specialists should be included on project supervision missions on a regular basis to ensure compliance and should not be confined to initial project document reviews . This can help avoid making decisions that may negatively impact the project . Coordination mechanisms within a decentralized system are essential for effective implementation . The project supported a culture of working together effectively from the minister to the local Zobas, which facilitated implementation. This was supported by capacity building during implementation and the introduction of the CP-RRA process, which helped to increase construction . Projects containing civil works need to have designs developed and evaluated during appraisal . Plans and designs should be completed during project preparation . Sufficient time to review and comment on design is crucial. Civil works should be completed at least one year prior to project closing to be able to measure their impact. When official data are not reliable, Bank teams need to exercise caution in preparing Results Frameworks and selecting indicators /targets. When there are severe limitation to the baseline data, these should be specified in the PAD . 14. Assessment Recommended? Yes No 15. Comments on Quality of ICR: The ICR provides a concise description of project implementation and an honest assessment of the shortcomings of the Bank's performance, as well as weaknesses in the Results Framework . It provides substantiation to validate its assertion that the targets in the Results Framework were selected without sufficient analysis and reliable data. a.Quality of ICR Rating : Satisfactory