Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) Report Number: ICRR0021695 1. Project Data Project ID Project Name P144305 DTF:EG-Energy/Social Sectors Reform TF Country Practice Area(Lead) Egypt, Arab Republic of Energy & Extractives L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-16152 30-Dec-2016 6,483,425.96 Bank Approval Date Closing Date (Actual) 26-Nov-2013 31-Jul-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 6,500,000.00 6,500,000.00 Revised Commitment 6,483,425.96 6,483,425.96 Actual 6,483,425.96 6,483,425.96 Prepared by Reviewed by ICR Review Coordinator Group Fernando Manibog Kavita Mathur Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The three project development objectives (PDOs) are to strengthen the Government of Egypt's capacity to: (1) design a comprehensive fuel subsidy reform strategy; (2) establish concrete measures for improved financial viability of key energy sector actors; and Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) (3) identify households that would be most vulnerable to the impacts of the fuel subsidy reform. (Schedule 1, page 5 of the Grant Agreement, Middle East and North Africa Transition Fund, dated December 4, 2013) These objectives are assessed respectively as PDO1, PDO2 and PDO3 in the efficacy section below. b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components The project had three main components: Component 1: Power Sector Institutional Development and Financial Viability (appraisal estimate, US$2.7 million; actual cost, US$3.4 million). This component included technical assistance in the following areas: (i) financial management and governance for power utilities; (ii) EETC strengthening to operate in a new market structure, which replaced the Fuel for Power Study when the project was restructured; and (iii) the establishment of an Energy Efficiency Unit. Component 2: Energy Pricing and Fuel Switching Reform (appraisal estimate, US$1.8 million; actual cost, US$1.3 million). This component was intended to produce an energy pricing and fuel switching reform strategy, and to develop a communication strategy for fuel reform. Component 3: Strengthening Social Safety Nets Technical Assistance (appraisal estimate, US$2.0 million; actual cost, US$1.8 million). This component included the following activities: (i) support for the establishment of the data base for the poor and vulnerable; (ii) a baseline survey for beneficiary families; and (iii) the establishment of a Technical Working Unit to support Social Safety Net reform. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost. The total cost estimate at appraisal was US$6.5 million. The actual disbursement at the project’s closing date was US$6.483 million, or 0.3 percent less than the appraised cost. The disbursement Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) on Component 1 was 127 percent of the appraisal estimate, while the disbursements on Components 2 and 3 were 89 percent and 99.7 percent, respectively, of their appraisal estimates. Financing. The project costs were financed entirely on a grant basis from the Middle East and North Africa Transition Fund (TF-16152). Borrower Contribution. There was no Borrower contribution. Dates. The project was approved on November 13, 2013 and declared effective on January 8, 2014. It was restructured once on October 27, 2016 to adjust the components and their respective costs, modify the results framework accordingly, reallocate among disbursement categories as needed, adjust the implementation schedule, and extend the closing date. The project was extended by about 19 months from the original closing date of December 30, 2016 to the final closing date of July 31, 2018. 3. Relevance of Objectives Rationale Country Context The security situation deteriorated in Egypt after the January 2011 revolution, leading to social instability and a slowdown of the economy. Unemployment had risen to 13 percent by December 2011 and poverty rates increased from 21 percent before the revolution to 25 percent by December 2012. More than US$30 billion of foreign reserves was lost post-revolution, thus worsening the already poor balance of payments position. International reserves declined to US$14 billion in June 2013. The overall budget deficit in FY2012 reached 10.8 percent of gross domestic product (GDP) in FY2012 and was projected at 12 percent in FY2013. Weak growth and inadequate tax revenues resulted in higher deficits. At the same time, spending increased on salaries, pensions, and the persistent legacy of huge energy and fuel subsidies. At project appraisal, the country was confronting an increase in chronic poverty as well as transitory poverty from the economic shocks. Sector Context Egypt was burdened with serious power shortages at the time of project appraisal. The country suffered from rolling blackouts since the peak power demand could not be met. The Government was providing very large budget subsidies for energy supplies, given the power sector’s poor financial condition. The prices for electricity and petroleum products were below their costs. Energy subsidies were not targeted to the poor or vulnerable. Yet the high subsidies claimed about 7 percent of GDP and were larger than the Government’s expenditures on both the health and education sectors. The Egyptian Electric Holding Company (EEFC) was heavily indebted and in need of substantial Government support to finance its capital investment program. The Government-owned integrated power company was an inefficient quasi-monopoly that needed to be restructured. Government Strategy Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) The project’s objectives are highly relevant to the Government’s strategy at appraisal. In 2012, the Government aimed to reform electricity tariffs and fuel subsidies, and took actions to increase heavy fuel oil prices, natural gas prices for households, and electricity tariffs. The subsidy for 95 octane gasoline was also eliminated while more than doubling its price. LPG prices for domestic use were also increased. The project’s objectives remain highly relevant to the current Government strategy. Egypt’s 2030 vision document aims to maximize the energy sector’s ability to sustainably develop national traditional energy resources and exploit renewable energy resources, with the goals of enhancing energy security and maintaining growth rates. Egypt’s Energy Sector’s Strategy for 2035, which was aligned with the Egypt 2030 vision document, seeks to maximize renewable energy in the energy mix to reach about 40% share by 2035, restructure and reform the gas sector, promote energy efficiency, and combat global warming by decreasing greenhouse gas emissions. To this end, the Government has implemented energy subsidy reform to enhance its efficiency and ensure sustainability and is implementing programs aimed at improving the performance efficiency of the petroleum sector (Source: Ministry of Petroleum). Bank Strategy The project’s objectives are highly relevant to the World Bank’s Country Partnership Framework (CPF) for Egypt 2015–2019, which organizes the World Bank Group’s support under these three focal areas: (i) improving governance; (ii) private sector job creation; and (iii) social inclusion.” To improve governance, the CPF cites specifically the need for “strengthening governance in the energy sector through energy policy and subsidy reforms.” To promote private sector job creation, the CPF indicates that: “The World Bank Group may support investments in institutions and capacities that contribute to competitiveness …ensuring Egypt’s energy security and diversification through enhanced energy efficiency and supporting financial sustainability of the energy sector.” To enhance social inclusion, the CPF states that “improving social inclusion will be addressed by strengthening the targeting of the social safety net system (SSN),” which the project supports by identifying poor and vulnerable households that are likely to be most impacted by the reform of fuel subsidies. Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To design a comprehensive fuel subsidy reform strategy. Rationale Theory of Change. The project's activities--including technical assistance, studies, consulting services, communication campaigns, and other institutional strengthening efforts--have direct and causal links to the Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) three PDOs, such that achievement of the PDOs could be credibly attributed to the project's interventions. The project's analytic activities are comprehensive in covering both energy pricing and fuel switching reforms. In turn, the analyses serve as the basis for preparing a road map toward the reduction and/or elimination of energy subsidies. In order to implement that road map, promote adoption of reforms, and achieve behavioral changes, the project went further along the causal chain toward achieving the three PDOs by developing a communication strategy for implementing the reforms, including actions to: (i) establish a Core Communication Unit and (ii) launch of the communication framework for the Modernization Strategy of the Ministry of Petroleum (MOP). The concrete results and outcomes are delineated below. Outputs  The Ordena Model for energy pricing was developed, judged as valid and robust, and accepted by the implementing agency and key project stakeholders, although it was also found that its requirement for large amounts of data proved to be a disadvantage.  A road map for subsidy reduction and/or elimination was prepared, using the Ordena Model. The road map included the socioeconomic impact of subsidy reform.  Training in using the Model was provided over 5 days to the Ministry of Electricity and Renewable Energy (MOERE), the Ministry of Petroleum (MOP), the Egyptian Electric Holding Company (EEHC), the Egyptian Electric Utility and Consumer Protection Regulatory Agency, the Egyptian Natural Gas Holding Company (EGAS), and the Egyptian General Petroleum Company (EGPC).  Two study tours in Spain were conducted for 15 participants to observe a liberalized market in operation.  Further training was provided on communications, and also for two other models, the Computed General Equilibrium (CGE) model for energy pricing, and the Subsidy Simulation (SUBSIM) package, which focused primarily on issues related to poverty and inequality. Outcomes The two main PDO 1 indicators were achieved: (1) A comprehensive energy pricing and fuel switching strategy, including detailed action plans for compensatory measures to mitigate the impact of subsidy removal, was developed. (2) A communication strategy for fuel subsidy reform including public consultation was prepared. The achievement of these PDO indicators, which were output-oriented in their formulation, is evident from the following:  Financial and economic costs of natural gas, petroleum products, and electricity were estimated to 2030 using the Ordena Model. An energy pricing strategy was prepared together with mitigation measures for subsidy reform, as well as a monitoring and evaluation framework for energy pricing. Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305)  The Ordena Model was in use at project closure, and is still currently in use, by MOERE and MOP. However, like almost all forecasting models, a major effort is required soon to update the Ordena Model, since it would soon be outdated and would no longer be useful.  As targeted, the Communication Unit was set up and the communication framework for MOP’s Modernization Strategy was launched.  The purpose of the reduction in energy subsidies and the intention to replace them over time by Cash Transfers—to provide more assistance to the poor and vulnerable—was explained to the broad public by the Communications Unit. Rating Substantial OBJECTIVE 2 Objective To establish concrete measures for improved financial viability of key energy sector actors. Rationale The achievement of this PDO depended on the effective delivery of four areas of technical assistance: (1) strengthening of financial management and governance in electric utilities; (2) restructuring of the Egyptian Electricity Transmission Company (EETC) to reflect the changed priorities in the reform program and the opening of the electricity market; (3) establishment of an Energy Efficiency Unit (EEU) in MOERE; and (4) setting-up of a Project Management Team (PMT) in MOERE to implement the entire project. Outputs  A major accounting firm, as project consultant, prepared for Egyptian Electric Holding Company (EEHC) a corporate governance code, a Chief Executive Officer contract, and audit committee charter for the EEHC Board, and other draft documents intended to improve EEHC’s governance. However, despite several tries, a financial forecasting model that was satisfactory to EEHC was not achieved. Training was provided over 9 sessions to 452 individuals.  EETC received extensive technical assistance for its restructuring, including an internal gap analysis, provision of market rules and a power supply code, design of its market operating system, functional specification of its require IT and billing systems, assistance in the unbundling of its accounts, recommendations on its financial restructuring, building of a financial model for EETC operations and planning, support for EETC’s financial reporting systems, and capacity building involving training sessions internally and overseas. Training was provided over 21 sessions to 425 individuals. Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305)  The Energy Efficiency Unit (EEU) was established with the mandate of helping reduce energy usage. Training was provided over 4 workshops to 94 individuals. Outcomes Although with some delay, the two main PDO 2 indicators were achieved: (1) Reform options to improve the financial viability and governance of EEHC were prepared; and (2) An action plan was prepared for (a) the establishment of EEU in the MOERE and (b) at least two of the energy efficiency projects in the NEEAP. The achievement of these PDO indicators is evident from the following:  Internal audit departments were established within EEHC and each of its subsidiaries.  EEHC has prepared bidding documents for an Enterprise Resource Planning program and will issue the request for bids once the valuation and separation of its assets is completed.  EETC is now quasi-independent from EEHC. EETC now has its own General Assembly and is owned by the Government, not EEHC. EETC has been enabled to sign contracts to offtake power from private companies. EETC can also buy and sell electricity at prices set by the regulatory authority and approved by the government. However, although the consultant had prepared much of the basis for the separation of EETC and EEHC, EETC’s assets (especially distribution) remain embedded within EEHC. The separation process is delayed pending completion of the Asset Evaluation Study mentioned above.  The EEU now has a business plan and is working on various programs, one of which is the distribution of LED bulbs for public lighting and public housing, respectively 2.6 million and 11 million bulbs.  The EEU was the main author of the National Energy Efficiency Action Plan (NEEAP), which has been approved by Cabinet. An Energy Efficiency Fund is expected to help implement the Plan.  The PMT satisfactorily fulfilled its expected functions, which involved significant procurement and financial management activities, difficult negotiations with four other ministries, supervision and monitoring of 47 consultants and consulting firms, and delivery of a large number of training sessions. Rating Substantial OBJECTIVE 3 Objective To identify households that would be most vulnerable to the impacts of the fuel subsidy reform. Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) Rationale The specific activities intended to achieve PDO3 include the development of a database on the poor and vulnerable, the preparation of a baseline survey of beneficiary families, and the establishment of a Technical Working Unit to support Social Safety Net (SSN) reforms. Upon completion, the technical assistance activities were expected to help strengthen Egypt’s SSN delivery by improving the targeting of its currently fragmented SSN programs. Moreover, the baseline survey would help to gauge the efficiency of social service delivery and evaluate the medium- to long-term impacts of the reforms. Outputs  The targeted number of databases to be linked was exceeded. Although 9 databases were to be linked initially, the Administrative Control Authority (ACA) has been able to link 39 databases to date, including the Family Smart Card system, which contains information on 80 percent of Egyptian citizens.  The Takaful and Karama Program (TKP) has been established as Egypt’s largest cash transfer program for the poor and vulnerable, with 41 consultants and staff from the Ministry of Social Solidarity (MOSS). The TKP cash transfer program is managing review of household applications, enrollment and rejections. It is also responsible for grievance resolutions, communications and community outreach, management information systems, use of the UNR, monitoring and evaluation. Outcomes  The PDO indicator has been exceeded. The original indicator was to develop a database of the poor and vulnerable with 10 million households registered in the new poverty database. The actual result was that 13 million households were registered in the food subsidy program that was connected to the Unified National Registry (UNR).  The UNR is refined regularly to weed out ineligible households and improve the targeting of social assistance programs.  The ACA has taken over the development of the national database for all government program beneficiaries and the UNR.  As of April 2018, the TKP was operating in all 27 Governorates of Egypt, covering 345 districts and 5,630 villages. Over 2.27 million beneficiary households have been enrolled, exceeding the initial target of 1.5 million households. Cumulative cash payments totaling EGP 18.38 billion have been made, of which around 9.5 million family members have benefitted. Rating Substantial Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) OVERALL EFF TBL OLD Rationale The three PDOs were substantially achieved. The achievement of PDO 1 was substantial: an energy pricing strategy and a corresponding monitoring and evaluation framework was prepared, the energy pricing and subsidy model is still in current use, and a communication strategy was implemented that explained the reduction of energy subsidies and its Cash Transfers replacement to the broad public as well as the poor and vulnerable. The achievement of PDO 2 was substantial: an Energy Efficiency Unit was established and at least two projects were included in the National Energy Efficiency Action Plan. The achievement of PDO 3 was also substantial, with the PDO indicator having been exceeded: 13 million households were registered in the food subsidy program (compared to the target of 10 million). The program is connected to the Unified National Registry, which is refined regularly to improve the targeting of social assistance programs. Overall Efficacy Rating Substantial 5. Efficiency As a technical assistance project, conventional measures of project worth (net present value, economic rates of return) were not calculated in the Project Appraisal Document (PAD) or the Implementation Completion and Results Report (ICR). No comparator technical assistance projects were discussed, hence there was also no cost-effectiveness analysis that would benchmark this project against similar interventions. Benefits At a sectoral level, the ICR emphasizes the point that “the gradual shift from energy subsidies to targeted Cash Transfers undoubtedly increased efficiency in terms of benefits delivered to the poor” (ICR, paragraph 61, page 26). However, the ICR did not provide any quantitative figures to support this assertion, albeit intuitively credible, given the ICR’s explanation that energy subsidies can end up helping the rich, while cash transfers could be much better targeted to the poor population who need help as energy subsidies are being reduced. At a more general macroeconomic level, the ICR cites unquantified benefits from the PAD (page 19) including greater fiscal stability, improved transparency and management of the electricity sector, reduced fiscal burden from subsidies, and mitigation of the adverse impacts of subsidy reform by strengthening the social safety network. Of these claims, the reduced fiscal burden through subsidy reform would have been possible to quantify but was not. Costs The overall project costs remained almost unchanged from the PAD stage, through restructuring, and up to final disbursements and the project’s closing date. The delay in the closing date by 19 months may have increased costs since the PMT staff stayed longer, but this may have been counterbalanced by the longer-term reduction in institutional learning time and costs, as well as the efficiency gains from the location and quicker response of the Bank’s Task Team Leader in Cairo. Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) Overall, given the many possible and largely credible benefits from the highly targeted technical assistance that was provided without cost increases for this relatively small project, the project’s efficiency is rated substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of project objectives is high. The extent to which each of the three PDOs was achieved is substantial, leading to an overall efficacy rating of substantial. Efficiency is substantial. The project’s overall outcome is rated satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Regarding outputs, there are no risks since the policy documents, analytical reports, operational and procedural manuals, training sessions, workshops, and databases have been delivered. The Ordena Model, however, would need serious updating to maintain its utility for forecasting, which may present a significant risk since the model is highly data-intensive and data collection is very time-consuming. Regarding outcomes, the risks are mainly political. The sustainability of outcomes depends ultimately on the Government’s determination and consistency in implementing its plan to reduce energy subsidies partly through the Cash Transfers program, the restructuring of the energy sector to encourage private investment, and improvements in the governance and efficiency of the state-owned companies. Reform performance is unpredictable. If the reform process stalls or reverses, the project’s outcomes are at significant risk. However, the ICR (paragraph 92, page 35), indicates that “so far, the Government is continuing the reform program and all indications are that it will continue with the reform program.” Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) 8. Assessment of Bank Performance a. Quality-at-Entry The project was well designed to provide a plan and a road map for helping to resolve—through specifically tailored technical assistance and capacity-building—a series of major problems in the energy sector of Egypt, including high energy subsidies, poor targeting of the poor or vulnerable, energy prices that were below economic costs, and an inefficient Government-owned integrated power company that was a quasi-monopoly and needed restructuring. The subsidy reform plan was designed to first identify the poor and vulnerable, target them more effectively, and minimize the impacts of subsidy reforms on them. The project’s social safety net aspects benefited from the Bank’s long-term policy dialogue in Egypt on safety nets, as well as its experience and lessons learned from several smaller Government projects that were poorly targeted. The lessons learned from the shortcomings of ongoing programs helped the Government design, target and implement the national Cash Transfers program. Risks were also identified appropriately. For example, a high risk at entry was the need for close coordination among the participating ministries, as well as the Government’s sustained ownership and commitment to support reform policies and programs. Risks were mitigated by the fact that the participating ministries themselves proposed and fully owned the project's technical assistance package; moreover, the establishment of a multi-ministerial Project Steering Committee helped maintain stakeholder ownership and commitment, which involved the provision of adequate resources and coordination mechanisms for implementation (PAD, para 56). In sum, the design of the project, and the monitoring and evaluation (M&E) system, were straightforward and robust. Quality-at-Entry Rating Satisfactory b. Quality of supervision During implementation, there was strong collaboration between the teams from the World Bank's energy and social protection sectors. Two Task Team Leaders supervised the project regularly, one for the two energy reform components, and the other for the social safety net component. One was country-based, which facilitated supervision. The team produced regular Implementation Status and Results Reports (ISRs) and resolved issues efficiently when they emerged. (ICR, Paragraph 88, page 35) For example, the team was responsive when the Government passed an Electricity Law in July 2015 that established the Egyptian Electricity Transmission Company (EETC) as an independent transmission system operator and a market operator. The team responded to these changed priorities by arranging for the project’s restructuring, which occurred in October 2016. Given its enhanced new role and the Government's expressed prioritization of strengthening EETC, the study on fuel for power generation was replaced with technical assistance tailored to EETC’s strengthening needs. For the social safety net component, the ICR (paragraph 90, page 35) indicates that “the World Bank provided more assistance than was initially planned including (a) the development of the Unified National Registry that links 39 databases, including the Takaful and Karama Programs (TKP) and the Family Smart Card (FSC) database, with information on 80 percent of Egyptian citizens, including the poor and non-poor; (b) development of the World Bank’s Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) Strengthening Social Safety Net Project (P145699) for Egypt in 2015 for US$400 million; and (c) capacity development of the Project Management Unit at the Ministry of Social Solidarity, which is implementing the TKP.” Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The PDO indicators were selected appropriately and adequate in number. They were concrete and readily monitorable. The diagram on the project’s theory of change (ICR, Figure 1, page 12) was also clearly presented and easily understood. However, the project’s contributing (and not decisive) role could have been more clearly represented; moreover, the constraining factors could have also been indicated (ICR, paragraph 78, page 33). b. M&E Implementation The project’s implementation status was reviewed regularly by the Bank team based on information provided by the Project Management Team (PMT) from the monitoring indicators. ISRs were produced regularly. In lieu of a mid-term Review, a large mission was held in May 2017 to conduct a detailed review of the project (ICR, paragraph 79, page 33). c. M&E Utilization M&E information was reported regularly to Bank management through ISRs, Aide Memoires, and emails. M&E data formed the basis for discussing the project’s implementation progress with the key project agencies, including EEHC, EETC, MOP, MOSS, the Ministry of Planning, Monitoring, and Administrative Reform, and other stakeholders. For example, when M&E information showed continuing project delays, project ratings were adjusted downward and discussions were held between the Bank team and the Government to identify measures that could be taken to accelerate project implementation (ICR, paragraph 80, page 33). M&E Quality Rating Substantial Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) 10. Other Issues a. Safeguards In line with the World Bank’s Safeguard Policy OP 4.01 on Environmental Assessment, the project was classified as Environmental Category C, indicating that the project would not have any adverse environmental impacts. No social safeguard policies were triggered as the project did not involve any significant social issues. On the contrary, the project provided technical assistance to strengthen Egypt’s social safety net, and also to develop a communications strategy for consultation, participation, and dissemination of information on the energy sector and subsidy reforms. According to the ICR (paragraph 82, page 33), the project’s environmental and social risk was first rated as Moderate in June 2015, but it was changed to a Low risk rating in June 2017 and has remained the same. b. Fiduciary Compliance Financial Management. There were no significant fiduciary issues. The ICR (paragraph 83, page 33) cites this finding from the final project supervision mission Aide Memoire: “For accounting, recording and reporting purposes, the project management utilized a manual accounting system in collaboration with Excel spreadsheets for reporting. The system is capable of generating the quarterly Interim Financial Reports and annual Financial Statements required under the Grant Agreement. As per the project Grant Agreement, the project management contracted with an Independent External Auditor who is responsible for reviewing the project Interim Financial Reports and auditing annual financial statements. All previous reports were received, reviewed, and found acceptable. The project management maintains acceptable records and adheres to the arrangements stated in the project’s Grant Agreement hence, the project’s FM arrangements are found to be Satisfactory.” The ICR also indicates that in the ISRs, the fiduciary risk was first rated as Moderate in June 2015 and remained as Moderate throughout the project implementation period. Procurement. The Project Management Team (PMT) had no prior experience with the World Bank’s procurement procedures. Training was provided at the start of project implementation. However, the various ministries involved in the project used different procurement rules and procedures. Given the large number of consulting contracts that needed to be processed, and PMT’s lack of hands-on experience, procurement proceeded slower than planned. As more experience was gained and more senior staff were brought in, procurement accelerated. The ICR (paragraph 84, page 34) indicates that: “Given their lack of experience, the PMT staff did an acceptable job. Although some minor mistakes were made, there were no serious issues.” c. Unintended impacts (Positive or Negative) --- d. Other --- Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The following lessons were derived from the project’s implementation experience (with some paraphrasing by IEG): Ways need to be identified—partly through the Bank’s convening power—to simplify the coordination of complex projects. The project’s complex set of activities and its institutional management were daunting but might have been simplified. Five ministries, several state-owned companies, six consulting firms, and 41 separate consultancies were involved. Coordinating and ensuring the cooperation of each of these entities was a major task. The World Bank’s convening power was helpful. Complexity was hard to avoid since the various ministries and state-owned companies needed to be kept aware of what was going on and what they needed to do. However, it might have been possible to hire an Egyptian consulting firm to handle the 41 individual consultancies. The Project Management Team (PMT) could have selected them and then turned them over to the apex consulting firm to arrange the terms of service and manage the separate consulting contracts. This approach might have been easier compared to the PMT having handled all the consultants itself. Flexibility to accommodate a dynamic environment is essential. Although the PAD identified some of the implementing agencies at the outset, all the responsible agencies changed several times. The World Bank worked flexibly and engaged proactively with the required agencies in a dynamic environment without requiring project restructuring to take place. This prevented further delays and built effective partnerships with the relevant ministries, which enabled the achievement of better results that initially expected. The coordination required by a project could help break silos both in the Government and in the World Bank, while facilitating reforms. To reform energy sector subsidies and to free up funds that could be used for more targeted social protection cash transfers, teams from the World Bank’s energy and social protection sectors collaborated on the project, as did energy and social protection agencies in the Government and the Administrative Control Authority. Breaking of silos and cross-collaboration enabled the success of the project. Partnerships with a similar approach are being replicated in other World Bank-financed projects in Tunisia, Jordan, and Algeria. Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review DTF:EG-Energy/Social Sectors Reform TF (P144305) 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR was well prepared and thorough. It was evaluative and candid in its analytical approach. A large emphasis was placed on providing concrete evidence behind the intermediate and final outcomes. The figure and tables provided were useful in helping understand this complex technical assistance project. Albeit long, the ICR put a lot of effort in delineating the project’s implementation record and how specific issues evolved and were resolved. The ICR’s format and internal substance within each section complied generally with the Bank’s guidelines on ICR preparation. The lessons, particularly those related to the Bank’s convening role and the importance of cross-sectoral collaboration (i.e., between the Bank’s energy and social protection sectoral staff) were useful and had broad replicability. The ICR had some minor shortcomings, however. For example, it discussed the relevance of objectives only at the appraisal stage, and not their current relevance, as required. Although understandably hard to avoid, the ICR also made use of too many confusing acronyms in the text. It was also very long at 36 pages, or around double the length of ICRs indicated by the guidelines. This is partly due to a large amount of repetition of text and over-explaining, which in some sections gave the impression of “straining to say something because there was little to say.” This could have been resolved with some editorial work. a. Quality of ICR Rating Substantial Page 15 of 15