Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) Report Number : ICRR0021194 1. Project Data Project ID Project Name P094917 3A-WAPP APL 1 (CTB Phase 2) Project Country Practice Area(Lead) Western Africa Energy & Extractives L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-42130,IDA-42140 31-Dec-2010 75,000,000.00 Bank Approval Date Closing Date (Actual) 29-Jun-2006 30-Jun-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 60,000,000.00 0.00 Revised Commitment 57,422,409.39 0.00 Actual 59,838,005.23 0.00 Prepared by Reviewed by ICR Review Coordinator Group Nestor Ntungwanayo Victoria Alexeeva Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives The Adaptable Program Loan (APL) was designed and implemented in two phases, and each phase project had its own Project Appraisal Document (PAD), and its own objectives. Project objectives by phase According to the 2005 Development Credit Agreement (DCA) of the first phase of the project (with Ghana), "the objectives are to extend the lifetime and improve the quality of the bulk power transmission system by providing investment support to replace and expand facilities and by providing technical assistance" (DCA, p.15). The project development objective in the PAD for phase 1 (page 9) was different, i.e., “to increase access of Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) WAPP “Zone A” coastal states to more stable and reliable electricity as a means to alleviate power supply deficits and/or to reduce their collective vulnerability to drought-induced power supply disruptions". As per the 2007 Financing Agreement (FA) of the second phase of the project (with Ghana and Benin), "the objective of the Project is to assist the Recipient in developing a more stable and reliable exchange of electricity between the national power systems of the West Africa Power Pool (WAPP) 'Zone A' Coastal States" (FA, p.5 for Ghana, and p.6 for Benin). 'Zone A' Coastal States are Benin, Côte d'Ivoire, Ghana, Nigeria and Togo. The project objective in the PAD for phase 2 (page 12) was stated as follows: "to assure more stable and reliable exchange of electricity between WAPP “Zone A” Coastal States (Côte d’Ivoire, Ghana, Benin, Togo and Nigeria) as a means to alleviate and/or reduce their collective vulnerability to drought-induced power supply disruptions". Following the IEG/OPCS harmonized guidelines, the ICR validation will be based on the assessment of the project performance on the basis of the objectives as stated in the legal agreements. Program's objective: The WAPP APL program comprised three mutually reinforcing sub-regional infrastructure development projects: (a) WAPP APL 1: the Coastal Transmission Backbone Project; (b) WAPP APL 2: the Western Zone Power Pool mechanism; and (c) WAPP APL 3: the Sahel Zone Power Pool mechanism. As per the PADs of two phases of the project (respectively on p.2 and p.10), "the objective of the West Africa Power Pool (WAPP) program is the development of a robust platform for WAPP comprising three distinct but mutually reinforcing sub-regional power system infrastructure development sub-programs that are fully aligned with the WAPP Organization’s Road Map". b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project has four components, which were planned to be implemented in two phases as detailed below: Component 1: Infrastructure Development- Coastal Transmission Backbone: US$80.46 million at appraisal; US$105.79 million at completion. The component aimed to extend the operational lifetime of key transmission facilities, and improve the quality and reliability of bulk power transfers along the Aboadze-Tema segment. • Phase 1 subcomponent. Key Activities were to: • Construct (1) approximately 215 Km of 330 kV single circuit transmission line from Aboadze to Volta; and (2) a 330 kV switchyard at Aboadze, equipped with phase shifting transformer; • (1) Construct a 330 kV sub-station at Volta adjacent to the Volta Operational Control Center; (2) Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) supply and install terminal equipment at the Aboadze and Volta substations; and (3) supply and install switchgear equipment at the Akosombo and the Volta substations; and • Supply and install a 161 kV phase-shifting transformer, two 330/161 kV autotransformers, and spare transformers. • Phase 2 subcomponent: Key Activities were to: • Co-finance the remaining segments of the Coastal Transmission Backbone (CTB): Prestea- Aboadze (Ghana); Tema (Ghana)-Mome Hagou (Togo); and Momé Hagou-Sakété (Benin). • Upgrade the Akosombo and Volta sub-stations through the supply and installation of: (i) a 330 kV power equipment at the Volta sub-station; (ii) a 161 kV power equipment at the Akosombo sub- station; (iii) control, relaying and metering systems; (iv) cabling systems, grounding, bus works and connections; and (v) steel structures; • Build a third bulk supply point for the Accra/Tema load centers through the design, manufacture, testing, delivery and installation of a 161kV/33kV high voltage substation; and • Upgrade the switchgear at the Kpong generation station through the design, manufacture, testing and erection of substation equipment. Component 2: Upgrade of Transmission Control and Communication System: US$35.09 million at appraisal; US$23.75 million at completion • Phase 1 subcomponent: Upgrade of the VRA System Control Center in Tema, Ghana through: • the supply and installation of equipment for reinforcement and extension of the Supervisory Control and Data Acquisition (SCADA) and communication system; and • the upgrading of the Volta System Control Center to coordinate implementation of the Operational Mitigation and Security Plan for the 330 kV Coastal Transmission Backbone. • Phase 2 subcomponent. Upgrade of Communauté économique du Benin (CEB) System Control Center in Lomé, Togo: • The key activity was to upgrade the supervisory control and data acquisition, telecommunication systems and metering equipment of substations in the 161 kV transmission network within the Recipient’s territory. Component 3: Upgrade of Strategic Power Generation Stations: US$13.90 million at appraisal; US$17.89 million at completion. Activities under each phase were as follows: • Phase 1 subcomponent: • Upgrade of switchyard at Akosombo Generation Station and Tema Substation (Ghana). • Phase 2 subcomponent: • The relocation of CEB gas turbines to Maria Gleta Terminal of the WAGP in Benin; • The development and implementation of an emergency preparedness plan for the Akosombo and Kpong generation stations; and Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) • The rehabilitation of cranes and penstocks at the Akosombo generation station. Component 4: Advisory Services: US$8.97 million at appraisal; US$2.75 million at closing date. Phase 1 subcomponent A series of technical assistance activities were planned to be funded as follows: • Provision of technical assistance to the Authority, including the following activities: (i) improve procurement procedures for electricity imports, (ii) develop and put in place an “Operational Mitigation and Security Plan” for the 330 kV Coastal Transmission Backbone, (iii) develop a strategy to optimize power generation from the Akosombo and Kpong hydropower facilities, including regional peaking supply, and (iv) procurement activities and project implementation, • Provision of technical assistance to the Ministry of energy (MOE) and other regulatory bodies for licensing, regulations, tariff formulation, bid documents and contracts to establish an autonomous transmission utility operation, Phase 2 subcomponent Under this sub-component, planned technical assistance included the following: • To upgrade the Project Implementing Entity’s Akuse training center in support of the WAPP Organization through: (i) acquisition and deployment of a modular power system simulator; (ii) installation of a 330 kV transmission line training field; (iii) provision of capacity building support for the training center staff; (iv) development of specific training transfer modules; and (v) upgrading of the training center’s learning materials and resources, and • To prepare (i) a study on the options for transmission network development for the northern sector of the Recipient’s territory; (ii) environmental and social impact assessments, line route surveys and related engineering design and bid documentation for the construction of the Aboadze-Prestea-Kumasi and Tumu-Han-Wa transmission lines; and (iii) a study on the options for the future development of a modern system control center in the Recipient’s territory. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: Actual project cost (phases 1 and 2) amounted to US$164.74 million, as compared to the appraisal estimate of US$158.5 million for both phases. Financing: The project was financed by three IDA credits in the amounts of US$40 million equivalent for Ghana (phase 1), US$45 million equivalent for the Republic of Ghana, and US$15 million equivalent for the Republic of Benin (phase 2). Phase 1 was co-financed by the Kuwait Fund in the amount of US$12.4 million, as compared to the commitment of US$17.5 million. The European Investment Bank (EIB) committed US$12 million at appraisal that did not disburse due to unresolved legal issues. No cofinancing was planned under phase 2, but subsequently the African Development Bank (AfDB) provided US$23 million to support the financing of various components of the project. The Kreditanstalt für Wiederaufbau Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) (KFW) also co-financed phase 2 for unreported amounts. Borrower Contribution: The Volta River Authority (VRA) contributed US$41.5 million (Ghana-first phase), significantly higher than its commitment of US$14.0 million. GRIDCo contributed US$10.28 million (Ghana-second phase), as compared to its commitment of US$15.0 million. During project implementation, the energy sector was reformed, including the unbundling and creation of a separate transmission enterprise- Ghana Grid Company (GRIDCo), which assumed the VRA's transmission assets and became responsible for project implementation; the restructuring formally transferred the corresponding project obligations from VRA to GRIDCo. Dates: The first phase of the project was approved on 06/30/2005, and became effective on 11/01/2005. The second phase of the project was approved on 06/29/2006, and became effective on 01/30/2008 for Ghana, and on 11/29/2007 for Benin, with a closing date set on 12/30/2010. The second phase of the project was restructured on 12/20/2010 for the Ghana component, and on 12/21/2010 for the Benin component. Key changes during this restructuring were as follows: (i) for the Ghana component, project implementation was transferred from VRA to GRIDCO, and for the Benin component, there was a change in disbursement category to include expenses related to works; (ii) there was also a revision of the project's results framework; and (iii) the project's closing date was extended until June 30, 2013. There were two additional restructurings for the Benin component on 06/24/2013 to extend the closing date for Benin until 06/30/2015, and on 06/12/2015, to extend it until 06/30/2016. 3. Relevance of Objectives & Design a. Relevance of Objectives Despite West Africa's large energy endowments, the region's combined per capita energy consumption in 2003 was about 40,000 Giga Watt hours (GWh). This was as compared to the peak power demand estimated at 6,500 Megawatt (MW). With electricity demand projected to grow by over 7% annually until 2020, electricity demand was expected to reach 140,000 GWh and peak power demand expected to exceed 22,000 MW. Given the differences in resource endowment among ECOWAS countries with significant electricity potential in countries with modest demand (such as Guinea), the goal of establishing a West Africa Power Pool (WAPP) mechanism aimed at integrating national power system operations was relevant in the regional context (PAD, page 6). The PDO was highly relevant in the regional context. ECOWAS member states signed the "ECOWAS Energy Sector Protocol" in 2003, aimed at setting up a unified regional (legal and regulatory) umbrella for a regional energy sector development and in 2006 adopted the "Articles of Agreement" to setting up a semi- autonomous regional, collaborative, power utility-led WAPP to take over WAPP activities from the ECOWAS Member States (PAD, page 7). The PDO was fully aligned with the goals of the New Partnership for Africa's Development (NEPAD) (NEPAD was established to implement an integrated socio-economic development framework for Africa and was formally adopted at the 37rd Summit of the Organization for African Unity in 2001) (PAD, page 10). The PDO was fully aligned with the Bank’s strategies for the two countries and the Bank's regional Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) strategy. The PDs was consistent with the energy sector interventions highlighted in the Country Assistance Strategies (CASs) endorsed by the Bank in 2004 for Ghana and for Benin in 2003 (PAD, page 9). The Bank's Regional Integration Assistance Strategy (RIAS) for West Africa that was updated in 2008 highlighted the need for supporting the ECOWAS initiative to establish interconnected electricity markets and the 2011 RIAS progress report reiterated the need for continued emphasis on cross-border connections (ICR, page 15). The Bank's Country Partnership Strategy (CPS) for Ghana for 2013-2016 identified the power sector as a priority infrastructure sector and highlighted the need for expanding power supply. The first pillar of the CPS for Benin for 2013-2017 highlighted the need for increasing sustainable growth competitiveness and generating employment, through among other things, energy sector investments (ICR, page 15). Rating High b. Relevance of Design An Adaptable Program Loan (APL) was used as the lending instrument to enable the participation of countries which had fulfilled the effectiveness conditions. The PDO was clear and the causal links between the project activities, their outputs and outcomes were logical. The intended outcomes were measurable, in principle. Energy Infrastructure activities such as construction of transmission lines, substations and control centers in Ghana and Benin were aimed at getting interconnection benefits and the outputs were aimed at developing a more stable and reliable exchange of electricity between the national power systems of the WAPP "Zone A" coastal states. There were several shortcomings in the design: • The project design which entailed simultaneous implementation of activities in two countries and concurrent financing of activities by other donors was ambitious. This was exacerbated by the lack of a WAPP group for coordinating project activities. These factors contributed to procurement and implementation delays and eventually to the non-completion of the Togo/ Benin component of the project (ICR, page 16). • There were shortcomings in technical design. The lack of project-specific engineering background studies and bidding documents at preparation contributed to implementation delays. Failure to address issues pertaining to frequency synchronization between the Ghanaian and Nigerian power systems meant that actions by the Nigerian power system are still required for making the interconnection system operational once the Togo/Benin component is complete. (This activity as described in section 4 is not yet complete). There were implementation delays due to the Ghana transmission line (adding about 10 kilometer (Km) to the overall line length, when it was determined that maintaining the original route would result in unacceptably large compensation expenses (ICR, page 10). In the case of the Togo/ Benin project component, the original contract needed modification and switch from the existing communication system to a fiber optic system (ICR, page 11). • There were deficiencies in the Results Framework (RF). The original RF was designed to capture the development outcomes of collective investments (financed by the Bank and other donors) and were not ring-fenced to solely measure the impact of Bank's investments (ICR, page 5). This in conjunction with the fact that the project benefits could only be assessed when all project works were complete, necessitated the incorporation of additional indicators (discussed in section 10b). • Lack of a lead coordinating agency to provide supervision support undermined execution (ICR, page 8). Unlike in subsequent WAPP projects, the WAPP had not designated a lead coordinating agency. This Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) hampered the bank to identify funding gaps early and play its role of donor of last resort more proactively and ultimately contributed to the non-completion of Togo-Benin component of the project (funding for this component from other donors was secured late in the project) (ICR, page 8). Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To assist the recipients in developing a more stable and reliable exchange of electricity between the national power systems of the WAPP "Zone A" Coastal states. Rationale Outputs (ICR, Data Sheet pages iv - vi, pages 16-17, 24-25 and 38-40). Ghana components. • These activities were completed as targeted. The 330 KV Aboadze-Tema transmission line, terminal substations were constructed. Switchgear and electromechanical equipment at the Aboadze switchyard and Volta switching system (including circuit breakers, disconnect switches and protection and control facilities) were upgraded. Equipment for the Third bulk supply point telecommunication system at the Accra East and Achimota substations were installed. The Supervisory Control and Data Acquisition (SCADA) and the Energy Management Systems (EMS) were upgraded. The Environmental Management Plan (EMP) and the Resettlement Action Plans (RAP) were completed. Technical and institutional support, capacity building and logistics support was provided for the installation of contracts for both the phase 1 and phase 2 projects. Expert Advisory Panel was formed to review West African Power Pool's (WAPPs) "Operational Mitigation and Security Plan" and provide advice on the management and operation of the WAPP interconnection system. Technical assistance was provided to the relevant stakeholders (Ministry of Energy, Public Utilities Regulatory Commission and the Energy Commission) to develop and implement an action plan for an autonomous and commercially viable Transmission System Operator (TSO) within WAPP. Evaluation and assessment of dam safety was completed, and an Emergency Preparedness Plan was prepared as targeted. • The VRA control center study was not completed and the VRA training center was not set up as targeted. Benin Components. • Construction works along the 330 kv line were not completed at project closure as the Bank no longer had the available resources for financing this activity. The World Bank closed the financing of this activity on June 30, 2016. Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) • The new dispatching center at the Togo/Benin Bi-national Electricity Company's (CEBs) headquarters and the backup dispatching center of Cotonou were completed and operational at project closure as targeted. The studies pertaining to modernizing the dispatch system, the rehabilitation study and the emergency plan were completed as targeted and the draft of the procedures manual for the West Africa Power Pool network was completed as targeted. Outcomes. • Indicators for both Ghana and Benin were output-oriented. • The key outcome targets associated with the quantity (Mega Watts) of electricity traded between the West Africa Power Pool (WAPP) "Zone A" Coastal states, reduction in power losses (percent) along the principal transmission interconnection links and the percent of peak power demand in WAPP "Zone A" Coastal states met by economy power exchanges on the Coastal Transmission Backbone, were not realized as the project was not fully implemented at project closure Rating Modest PHREVDELTBL PHREVISEDTBL 5. Efficiency Economic Analysis. An economic analysis was conducted at appraisal for transmission infrastructure investments (such as upgrading key sub-stations for cross-border electricity exchanges, upgrading the system control centers of the Togo/Benin Bi-national Electricity Company and VRA and upgrading strategic power generation stations). These components accounted 85% and 87% of the appraisal estimate for Ghana and Benin respectively. The global Net Present Value (NPV) at 10% discount rate was US$64 million (with a NPV of US$20.00 million and US$38.00 million for Ghana and Benin respectively). The Economic Internal Rate of Return (EIRR) for Ghana and Benin was 16% and 35% respectively and the global NPV was 23% (PAD, page 21). An economic analysis was conducted for the Ghana component of the project at project closure (given that the Benin/Togo component was not completed at project closure. The analysis showed that under the current loading of the transmission line, the project yields unsatisfactory economic indices (with a Benefit-Cost ratio below one and EIRR below 12%. Administrative and Operational Issues. Project effectiveness was delayed in both Ghana and Benin. Ghana. Although the project was approved in June 2005 (phase 1) and June 2006 (Phase 2), effectiveness for the latter was delayed 1/30/2018, due to noncompliance with the VRA's solvency requirements as per the legal covenants (ICR, page 9). There were implementation delays due to a combination of factors including, technical design issues necessitating the rerouting of the transmission line, unbundling of the sector which was not anticipated at design, procurement delays associated with the Supervisory Control and Data Acquisition (SCADA) and Energy Management Systems (EMS), exacerbated by factors such as Ghana's new procurement statues (ICR, pages 9 and 10). The activities in Ghana were however completed, albeit with a Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) time overrun of 24 months. Benin. Although the Benin component was approved in June 2006, effectiveness was delayed until 11/29/2007 due to a combination of factors including, insufficient staff resources for the project, delays on the part of the company to comply with the conditions of the other donors (although the African Development Bank approved the project in 2007, it was only initiated five years later in January 2012 and the German Development Agency (Kfw) only agreed to participate in the project in 2011) and contractual disputes between the Togo/Benin Bi-National Electricity Company (CEB) and the contractor. These factors contributed to implementation delays during implementation and the Benin component was not complete at project closure, despite the repeated extensions to the closing date. Regional dimension. It is not clear if the project could be evaluated as a regional project given that it is not complete and that there is no regional power trade (ICR, page 17). Further, even if the Benin component is eventually completed, actions within the Nigerian power system will still be needed for regional interconnections. Efficiency Rating Negligible a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 23.00 Appraisal  86.00 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of objective for the countries, the Bank strategies for the respective countries and the Bank’s regional strategy for West Africa is rated as High. Relevance of design is Modest in view of the technical drawbacks at design and inadequate implementation arrangements. Efficacy of the single objective - To assist Ghana and Benin in developing a more stable and reliable exchange of electricity between the national power systems of the WAPP "Zone A" Coastal states - is Modest. The indicators were output-oriented and the intended key outcomes were not realized as the project was not fully implemented at project closure. Efficiency is rated as Negligible in view of the significant administrative and operational inefficiencies which contributed to implementation delays and eventual non-completion of activities in Benin, despite the extensions to the project closing date and complete disbursement of funds. Taking these ratings into effect, the outcome is rated as Unsatisfactory, reflecting significant shortcomings in design and efficiency. Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) a. Outcome Rating Unsatisfactory 7. Rationale for Risk to Development Outcome Rating Technical risk is High, given that the Benin component was not yet complete at closure (owing to the differences between the Benin/Togo bi National Electricity Company (CEB) and its contractor). Given that the funding from the African Development Bank (AfDB) for this activity closed in 2017, it is not clear if CEB finds alternative financing to replace AfDB. Even if this component is completed in the near future, actions would still be needed to address the issues associated with maintaining a constant frequency between the Ghana and Nigeria Power Systems (through investments in Nigeria), for the operationalization of the project when the CEB system is completed. Financial risk is High. Given that the transmission works (including substations and lines) need sustained sources of funding for maintenance, it is not clear whether the two electricity companies in Ghana and Benin would be able to provide funding for maintenance. Given that resources needed for maintenance are dependent on the income collected by the distribution companies, which are affected by high commercial losses (due among other things to nonpayment of public sector electricity consumption) in the countries, it is not clear if there would be adequate funding for the sustainable maintenance of transmission works (ICR, page 15). Institutional and governmental commitment risk is Substantial. Even if the Benin component is complete and the issue of frequency synchronization between the Ghana and the Nigeria power systems are resolved, significant regional power trade is still contingent upon sufficient generation of surpluses and implementation of WAPP protocols among the participating utilizes, particularly with respect to timely payment for power sales. a. Risk to Development Outcome Rating High 8. Assessment of Bank Performance a. Quality-at-Entry The project was prepared based on the experience from the five-decade evolution of the best known regional power market (the Nordic Power Market) and from the lessons from comparable Bank-financed regional adaptable program loans for the Southern Africa Power Pool (SAPP) and the Energy Community of South East Europe. Lessons incorporated included establishing an institutional structure consisting of the National Transmission Systems Operators (TSOs) and greater autonomy for TSOs through their unbundling, tailoring activities to suit member countries, combined implementation arrangements at the national and regional levels and anchoring to the extent possible on the strongest performing of the beneficiary member Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) country covered by the regional project (in Ghana in this project) (PAD, pages 13-14). The arrangements made at appraisal for fiduciary compliance were appropriate (discussed in section 11). • As discussed in section 3b, there were several shortcomings at design including, technical design shortcomings, inadequate coordination arrangements for activities in two countries and for activities that were concurrently funded by other donors, weakness in the results framework and non-specification of a lead coordinating agency to provide supervision support. These factors contributed to implementation delays and eventually non-completion of activities. • The risks associated with donor coordination and a design that required sequencing of investments of several donors was underestimated. At design, the preparation team did not anticipate the risks associated with funding a component activity (providing engineering services support) for an activity that was entirely linked to an activity funded by other donors (construction of the Benin/Togo bi National Electricity Company (CEB) transmission line). Given that the construction of the line was delayed and eventual non-completion of this activity by other donors, contributed to the delays in the bank funded activities (ICR, page 9). • There were significant shortcomings in M&E design (discussed in section 10a). Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision Missions were conducted regularly (with seventeen Implementation Status Reports (ISRs) filed over a period of eleven years). The supervision team included energy specialists, financial, procurement and social specialists and a power engineer. The continuity of leadership was maintained with few Task Team Leaders (TTLs) over an eleven-year period. The supervision team provided support following the transition from VRA to GRIDC0 in Ghana and this aided in financial management compliance (discussed in Section 10). The team appropriately restructured the project in the wake of challenges that rose during implementation. Although the supervision of activities associated with the Benin component was hampered by limited Bank leverage, the support provided by the team aided in the activity associated with the CEB control center. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Unsatisfactory 9. Assessment of Borrower Performance a. Government Performance The project was prepared within the Economic Community of West African States (ECOWAS) Framework and the commitment of Ghana and Benin as ECOWAS member states was demonstrated by their Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) ratification of the ECOWAS Energy Protocol (EEP) aimed at setting up a unified regional umbrella to facilitate harmonization of legal, regulatory and institutional frameworks for the WAPP. Ghana. The government commitment to the project was demonstrated by their contribution to the project by way of counterpart funding (which was more than planned for the first phase but slightly lower than planned for the second phase). The government also facilitated the unbundling of the power sector and addressed issues associated with the allocation of transmission responsibilities to the GRIDC0. Although there were procurement delays in the wake of the government’s new procurement statues, these were rectified during implementation. Benin. The government commitment to the project was limited during implementation and the government did not take measures to assist the Togo/Benin Bi-National Electricity Company with respect to nonpayment of its bills. Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance The two power utilities - the Togo/Benin Bi-National Electricity Company (CEB) for Benin and the VRA were in charge of implementing the project. Both utilities had set up Project Implementation Units (PIUs) at appraisal. At the regional level, the WAPP Steering Committee consisting of ECOWAS Energy Ministers were responsible for policy oversight. (PAD, page 16). Ghana. The VRA and the GRIDCo followed the safeguard guidelines and although there were minor shortcomings (such as initiating works without compensation to the Project Affected Persons and lack of proactivity in dealing with the frequency synchronization issue), these issues were rectified during implementation. The implementing agency ensured fiduciary compliance (discussed in Section 10) and generally made an effort to enable the project's success. Rating: Moderately Satisfactory. Benin. The performance of CEB with respect to the oversight of contractors and payments was poor. CEB was unable to reach a satisfactory agreement on project completion with respect to the construction of the transmission line. The lack of timely decisions on the part of CEB contributed to the non-completion of activity associated with the transmission line. Rating: Unsatisfactory. Given the two different ratings for agency performance, the combination leads to an overall rating for implementing agency performance of Moderately Unsatisfactory. Implementing Agency Performance Rating Moderately Unsatisfactory Overall Borrower Performance Rating Moderately Unsatisfactory 10. M&E Design, Implementation, & Utilization Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) a. M&E Design The three key outcome indicators disaggregated by power utility – the quantity of MegaWatts of electricity traded between the West Africa Power Pool (WAPP) “Zone A” Coastal countries, the level of power losses along the principal transmission links and the percent of peak power demand met by electricity exchanges- were appropriate for monitoring project performance. These indicators however could only be measured when the entire regional project was completed. An integrated WAPP Program M&E system building upon the country-specific M&E Systems and developed by the United States Agency for International Development (USAID) and the Bank, was to be used by the WAPP General Secretariat for monitoring project performance. The intermediate indicators for the control center component and the generation station improvement implicitly assumed that the system was already operational and would have better been classified as outcome indicators. b. M&E Implementation Given that project benefits could not be assessed until the cross-border interconnections had been completed, it is not clear as to what extent the input, output and outcome indicators anticipated at design could be monitored. Three new indicators were added to the Results Framework during implementation. These indicators included the energy flows through the completed transmission lines and substations. However, given that the components could not operate independently, and every transmission link needed to be operational to monitor performance, the incorporation of these indicators did not improve project monitoring. The ICR provides no details on whether the agency responsible for M&E functioned effectively, whether the M&E was owned by the various stakeholders and the extent to which beneficiaries were involved in defining target indicators and assessing their achievements. It is also not clear whether the system that was designed and implemented was sustainable. c. M&E Utilization Given that the transmission line was not complete, even the three added intermediate indicators were of little practical application in tracking project performance. M&E Quality Rating Negligible 11. Other Issues Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) a. Safeguards The project was classified as a Category B project. Two safeguard policies were triggered: OP/BP 4.01 Environmental Assessment and Involuntary Resettlement (OP/BP 4.12). The PAD (page 29) notes that environmental effects of the project were expected to be limited to the measures required to avoid adverse impacts of construction at existing substations. Environmental Management Plans (EMPs) were prepared by the implementing agencies for Ghana and Benin. The PAD also notes that as part of the project, the implementing agencies were expected to prepare emergency preparedness plans in lieu of the safeguards associated with Safety of Dams (PAD, page 29). Ghana. There was compliance with environmental and social safeguards and the implementing agency prepared the emergency preparedness plans. The ICR (page 13) notes that about 1,700 persons were affected due to project activities. During project construction, project affected persons were compensated according to the VRA's valuation of crops, land and buildings. The ICR (page 13) notes that initially the project was not in compliance with Bank policies because construction activities had started prior payment of compensation. However, by 2011, the VRA had fully settled outstanding compensation to all affected parties (including landowners and stone crackers). Benin. There was compliance with safeguards and the implementing agency prepared the emergency preparedness plans. The project activities did not raise any social safeguard issues as implementation did not require land acquisition or resettlement of people (ICR, page 13). b. Fiduciary Compliance Financial Management. An assessment was conducted at appraisal to judge the financial management capacity of the VRA and CEB. The assessment concluded that the financial management arrangements of the implementing arrangements were deemed to be satisfactory. The financial management risk for Benin was rated High and for Ghana Substantial (PAD 56-57). Ghana. The ICR (page 13) notes that the principal financial management challenge for the Ghana component was the transition from VRA to the GRIDCo in 2009-2010. However since GRIDCo did not have sufficient capacity for financial management, the VRA was entrusted with the financial management responsibility of the project. The Bank supervision team aided in the arrangements associated with finance and accounting, budgeting and financial reporting. The ICR (page 13 notes that the financial management was satisfactory during implementation. The ICR however provides no details on the quality of audits. Benin. The ICR (page 14) reports that financial managements of CEB were deemed to be satisfactory during implementation and the CEB complied with the Bank’s financial management requirements. Procurement Management. An assessment was conducted at appraisal to judge the procurement management capacity of the two implementing agencies. Both VRA and CEB had managed prior Bank- financed projects and were familiar with the Bank's procurement policies. The procurement risk for both Ghana and Benin were rated as Low (PAD, pages 68 and 71). Ghana. Although procurement was delayed initially due to a new procurement law in 2005 and 2006, these were resolved and that was compliance with procurement management (ICR, page 13). Benin. There was compliance with procurement issues (ICR, page 14). Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) c. Unintended impacts (Positive or Negative) The ICR did not indicate any unintended impacts. d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Unsatisfactory Unsatisfactory --- Risk to Development High High --- Outcome Moderately Moderately Bank Performance --- Unsatisfactory Unsatisfactory Moderately Moderately Borrower Performance --- Unsatisfactory Unsatisfactory Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The ICR draws the following main lessons from the experience of implementing this project (with some modification of language) (ICR, pages 20-21). 1 . Risks associated with having two implementing agencies need to be considered at design. Although this project was implemented independently by two implementing agencies, it was in essence an integrated endeavor that connected several countries. The experience of this project showed that establishing a single company to build, operate and maintain the facilities and thereby avoiding the disruptions that originate from piecemeal construction, may be more effective in performing these types of projects. 2 . The risks associated with participation in a multi-donor project need to be taken into account during preparation and the full range of the Bank's instruments must be used to address unanticipated investment needs during implementation. In this project, the Bank contribution to the Benin project component was restricted to financing the CEB control center and providing overall supervision of the transmission line construction funded by other donors. An operational response to addressing gaps in funding could have been pivotal to proactively addressing implementation issues. Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review 3A-WAPP APL 1 (CTB Phase 2) Project (P094917) 3 . Government commitment needs to be secured through involvement of a high-level steering committee for regional projects. Such a committee could help in coordination of the different players. 4 . A lead funding agency needs to be designated at design. In this project, the West Africa Power Pool (WAPP) had not assigned the Bank as the lead coordinating agency and consequently the Bank was unable to proactively address issues associated with funding gaps that arose in the Benin project component. 5 . Better design of results indicators is required at preparation to monitor performance. One shortcoming of this project was the lack of indicators for monitoring the Bank’s contribution as distinct from the whole operation. 14. Assessment Recommended? Yes Please explain The ICR indicates (para. 55) that achievement of the PDO is possible within the next 1–2 years, as the obstacles to completion are resolved through negotiations or arbitration. It therefore would be useful that the project achievements and outcomes be revisited in a couple of years from now to ascertain the completion of planned activities and assess the progress made toward expected initial outcomes and objectives of the projects and the program. 15. Comments on Quality of ICR The ICR is concise and well-written. It candidly discusses the issues associated with a regional project with two countries and concurrent financing by other development partners. It is also candid in discussing the limited leverage the bank had with respect to the Benin component of the project. The quality of evidence provided in the ICR is adequate and the ratings are consistent with the guidelines and the ICR draws pertinent lessons from the experience of implementing this project. The details provided in the ICR about the frequency synchronization issues is rather sparse and it would have helped to provide more discussion of this issue, considering the project's regional implications. a. Quality of ICR Rating Substantial Page 16 of 16