Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) Report Number : ICRR0021115 1. Project Data Project ID Project Name ENERGY EFF - INDUST P118737 ENTERPRISES Country Practice Area(Lead) Additional Financing Uzbekistan Energy & Extractives P133633,P165054 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-47450,IDA-52410,IDA_- 31-Jan-2016 34,600,000.00 47450,IDA_-52410 Bank Approval Date Closing Date (Actual) 17-Jun-2010 31-Jan-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 25,000,000.00 0.00 Revised Commitment 124,197,527.45 0.00 Actual 108,693,236.04 0.00 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Victoria Alexeeva Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives According to the Project Appraisal Document (PAD) (p. 4) and the Financing Agreement of August 18, 2011 (p. 5), the objective of the project was "to improve energy efficiency in industrial enterprises by designing and establishing a financing mechanism for energy saving investments". b. Were the project objectives/key associated outcome targets revised during implementation? Page 1 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 26-Apr-2013 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components The project included two components: Component A: Development of Energy Efficiency Capacity (appraisal estimate US$1 million, additional financing US$1 million, actual US$1.5 million): This component was to finance: i) developing an Energy Efficiency (EE) strategy for Industrial Enterprises (IEs); ii) increasing the knowledge, experience and expertise in identifying, preparing and implementing EE projects in the industrial sector through targeted training; iii) developing an EE communication strategy for the industrial sector; and v) creating and maintaining the Project Coordination Unit (PCU) to implement the project. At the time of Additional Financing in 2013, the activities were scaled up to further enhance the EE capacity of selected industries, banks, industry associations and energy professionals as well as to strength project management, coordination, and monitoring and evaluation capacity of the MoE and the PCU and improving Uzbekistan’s statistical reporting capacity in the areas of energy consumption and EE. Component B: Credit Line to Participating Banks (PBs) (appraisal estimate US$24 million, additional financing US$99 million, actual US$121.6 million): This component was to finance the provision of loans, through the Ministry of Finance (MoF), to two state-owned banks (Asaka and Uzpromstroy), and a private bank (Hamkor Bank), which were to sign sub-agreements with an allocation of US$8 million to on-lend to IEs to invest in EE sub-projects. During implementation, the MoF was to reserve the right to reallocate these amounts, subject to approval by the World Bank, depending on the actual disbursement progress and demonstrated project pipeline of each participating bank (PB). In addition, PBs were to co-finance project activities and follow their existing loan approval processes for IEs to be reviewed by the World Bank team and considered adequate to verify and monitor long-term investments by IEs. The PBs were to be responsible for ensuring that sub-loan applications and approvals under the project meet all Uzbek and World Bank requirements. The credit line of this component did not change through the additional financing but the total amount of the credit line was increased. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$34.6 million. Actual cost was US$178.3 million due to scale up of the activities. Financing: The project was financed by a US$25 million credit of the International Development Agency (IDA) supplemented by an additional financing (AF) of US$100 million. Borrower Contribution: There were no planned contributions by the Borrower. However, the Borrower’s financial intermediaries (participating banks) and sub-borrowers (industrial enterprises) were to contribute Page 2 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) US$9.6 million. Actual contribution was US$83.1 million. Dates: On April 26, 2013 the project received an additional financing in the amount of US$100 million. The AF included the following changes: i) increase of the current limit for individual sub-borrowers from US$3 million up to US$10 million while the aggregate amount of sub-loans to the group of affiliated companies out of the proceeds of the Additional Financing was not to exceed US$30 million; ii) revision of the eligibility criteria for IEs; iii) increase the financing of Component A to US$2 million to further enhance the EE capacity of selected industries, banks, industry associations and energy professionals through the provision of training, to strengthen project management, coordination, and monitoring and evaluation capacity of the Ministry of Energy and the Project Coordination Unit, and to improve Uzbekistan’s statistical reporting capacity in the areas of energy consumption and EE; iv) increase the financing of Component B to US$123 million; v) the project closing date was extended by two years from January 31, 2016 to January 31, 2018. 3. Relevance of Objectives Rationale In 2010, at the time of project appraisal, Uzbekistan had one of the most energy intensive economies in the world. The country’s industrial sector was highly inefficient and accounted for 38.3 percent of the total energy use. Uzbekistan used three times more energy than the average Eastern European and Central Asian country, and six times more than Germany to produce one unit of its Gross Domestic Product (GDP). In addition, Uzbekistan was also the 35th largest carbon dioxide emitter worldwide. Furthermore, the country’s banking sector mainly consisted of state-owned commercial banks. These banks did not provide loans to improve energy efficiency and the country had little capacity to implement any investments related to energy efficiency. Therefore, the Government of Uzbekistan identified the saving of energy in industrial enterprises as one of its key economic policy priorities and passed several resolutions and decrees. The project supports the government’s law on efficient usage of energy which was passed in 1997 and the 2014 president’s decree which aims to speed up the reform of the energy sector. In 2015, the government set the target to reduce the country’s energy intensity by at least 50 percent by 2030. The project’s objective is in line with the Development Strategy (2017-2021) which also aims to reduce energy consumption and resource intensity. The objective of the project supported the Bank’s Country Assistance Strategy (FY08-11) which highlighted the importance of reducing greenhouse gas emissions and increasing energy efficiency of the Uzbek’s economy. Also, the objective of the project supported the Bank’s Country Partnership Strategy (FY12-15) which emphasized as a key objective improving energy efficiency. The objective is also consistent with the Bank’s most recent Country Partnership Framework (FY16-20) which includes as one of its objectives “to promote energy security and efficiency, and reduce the economy’s energy intensity.” Rating High 4. Achievement of Objectives (Efficacy) Page 3 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) PHEFFICACYTBL Objective 1 Objective To improve energy efficiency in industrial enterprises (IEs) by designing and establishing a financing mechanism for energy saving investments: Rationale A split rating assessment is not conducted as the outcome targets were revised upwards to reflect a scale- up of project activities through AF. The project’s theory of change linked the development of energy efficiency (EE) strategies for IE and EE communications strategy, the enhancement of EE capacity, the establishment of a fully functional PCU and the disbursement of EE funds to improvement in EE in industrial enterprises such as energy savings, CO2 emission reductions and leveraged EE investments. The project design foresaw that the sub-borrower of each sub-project would contribute 20 percent of the total investment cost and the remaining 80 percent would be financed through a sub-loan from a participating bank. Outputs: • A EE Strategy for IEs was developed and findings were disseminated among industrial enterprises, achieving the target. According to the ICR (p. 36), more than 90 enterprises had introduced EE strategies by the time the ICR was written. However, the ICR (p. 19) also states that the project’s direct contribution to EE policy was limited since the government did not fully use the technical assistance resources that were available. • Credit lines were given to 32 industrial enterprises, achieving the original target of 15 enterprises but not achieving the revised target of 37 enterprises. Most sub-loans were given to enterprises in the oil and gas, chemicals and cement industries. The ICR (p.18) states that during implementation the scope of sub- projects shifted from small and medium sized enterprises to state owned enterprises since they were of significant importance to the Uzbek economy. • A financing mechanism for energy saving investments was fully designed and implemented in 76 sub- projects, resulting in a sub-portfolio of US$143.881 million. A pipeline of additional 33 subprojects (total cost US$323.4 million) was identified as of September 30, 2017. • An EE communication strategy was developed by the PCU and endorsed by the Prime Minister in October 2014. The strategy identified communication activities to increase awareness about EE among industrial enterprises, equipment providers, local authorities, commercial banks, mass media, and educational and research establishments The ICR reports that project activities were mentioned 70 times on TV and/or radio during project implementation, 72 articles in the national press covered project activities and EE was discussed in more than 350 articles nationwide; and two four-day media tours were organized during which media representatives were invited to six industrial enterprises to cover their EE programs. • Trainings were carried out to help the IEs to improve their statistical reporting capacity to comply with the regulations pertaining to energy efficiency. Over 150 government officials and professionals from different industries were trained. • The project conducted several trainings targeting the Ministry of Economy (MoE), PCU, participating banks, and project beneficiaries. Areas of trainings included project management, coordination, procurement and monitoring and evaluation. Page 4 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) Outcomes: • The leveraged amount of EE investments disbursed was US$69.58 million, surpassing the original target of US$35.3 million but not achieving the revised target of US$83.12; US$9.82 was lost due to the strengthening of the US dollar against the SDR. Also, US$7.16 million was not committed by the time of the ICR. The equivalent co-financing represented 93 percent of the PDO indicator target. 23 percent of co- financing provided by the participating Banks (PBs), exceeded the 20 percent required by the project legal agreements. • The cumulative annual energy saved increased from 27,00 MWh/year in 2012 to 358,587 MWh/year in 2017, surpassing the original target of 47,000 MWh/year and the revised target of 227,000 MWh/year. • The cumulative CO2 emission reductions increased from 70,000 metric tons in 2012 to 583,227 tons in 2017, surpassing the original target of 110,000 metric tons and the revised target of 470,000 tons. Rating Substantial PHREVDELTBL PHOVRLEFFRATTBL Rationale The achievement of the objective was substantial. Two outcome targets on energy savings and CO2 emission reduction were overachieved, and the third outcome related to the leveraged amount of EE investments was below the target. Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Economic Efficiency: The PAD (p. 80) conducted an economic and financial analysis on two typical EE sub-projects in the brick and textile industries. The first sub-project replaced an outdated circular kiln with a more efficient one which could save up to 30 percent on energy. The economic analysis assumed a US$1.5 million investment over a 20-year period at a 12 percent discount rate. The Economic Rate of Return (ERR) of the sub-project was estimated at 46.7 percent, including energy savings and C02 reductions at US$10/ton. Without carbon emission benefits, the ERR on energy savings was 44.8 percent. The simple payback period for this project was 2.5 years. The Financial Rate of Return was estimated at 12.3 percent with a simple payback period of 5.5 years. The estimation used the same methodology as the ERR but did not include CO2 costs. The second sub-project replaced an old 2x2.5 t/h capacity gas-fired steam boiler and AC electric motors (100 units totaling 1,500 kWe capacity) with converters. The ERR was estimated at 39.5 percent including energy and C02 savings and 36.8 percent with energy savings only. The FRR was estimated at 11.9 percent and the Page 5 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) payback period was about five years. The ERR was lower and the FRR was higher for this investment, compared to replacing kilns in the brick industry, because of the higher subsidies for gas than for electricity. The analysis showed that these two sub-projects were economically and financially worthwhile investments. The ICR (p.42) estimated the ERR and FRR for six sub-projects. However, the scope of the sub-projects changed during implementation and moved from small and medium sized enterprises to state owned enterprises (SOEs), making the comparison more challenging. The ERRs for similar sub-projects ranged from 15.5 to 35 percent and the FRRs ranged from 11.5 to 29.1 percent. The FRRs included in the ICR were lower than the FRRs in the PAD due to the devaluation of the local currency. Operational Efficiency: The project experienced additional costs related due to approval delays for currency conversion. Due to these delays, borrowers were late in repaying sub-loans and were charged commission fees to cover repayment and currency risks by the participating banks resulting in an increase of borrowing costs for project beneficiaries. This had a negative impact on borrowers wanting to use the EE credit lines offered by the participating banks. The project also experienced delays in the implementation of sub-projects due a slow establishment of the evaluation group for international competitive bidding within the inter-ministerial tender committee. Also, the project experienced an 18 months delay in effectiveness due to an extended dialogue with the government related to the focus on state owned entities. There were also procurement delays, and uncommitted funds at project closure. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the objective is rated high given the country’s high energy use and inefficient industrial sector. The project achieved its objective to improve energy efficiency in industrial enterprises by designing and establishing a financing mechanism for energy saving investments to a substantial extent. Efficiency is rated substantial for satisfactory economic and financial rates of return, albeit minor issues in operational efficiency. The overall project’s outcome rating is satisfactory. Page 6 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) a. Outcome Rating Satisfactory 7. Risk to Development Outcome The ICR (p. 31) states that the government continued to implement market-oriented structural reforms in some areas of the economy such as the liberalization of its currency regulations, enabling small companies to access foreign exchange. This might have a positive impact on companies implementing EE investments. Furthermore, energy prices were realigned closer to cost-recovery levels. Also, energy prices for industries which do not introduce energy-saving technologies were increased. However, the ICR (p. 32) states that there is still a significant financing gap for achieving the government’s short- to medium-term industrial energy savings target. Participating banks will need to be encouraged to use their own capital for EE landing and to develop an ongoing EE improvement mechanism within industrial enterprises. 8. Assessment of Bank Performance a. Quality-at-Entry The project built on the Bank’s experience and included lessons learned from implementing similar projects. The Bank identified relevant risk factors. These risks included rigid and hierarchical government procedures, systematic corruption and poor financial management and M&E system, misunderstanding in the industrial sector about the nature of energy efficiency projects, weak procurement capacity, and non- compliance with Bank’s safeguard policies since the participating banks have limited experience in this regard. The Bank’s mitigation efforts were adequate. However, the risk of a slow start-up period due to lengthy clarification processes regarding the eligibility criteria and negotiations of sub-projects with the Ministry of Energy, was underestimated resulting in initial delays of the project. The Results Framework was appropriate and captured how project activities contributed to the achievement of the PDO (see section 9a for more details). Quality-at-Entry Rating Satisfactory b. Quality of supervision According to the ICR (p. 31) initially the Bank conducted annual supervision missions. After the additional financing was approved supervision missions took place on a bi-annual basis to provide technical and operational support on a continuous basis. The ICR also states that the supervision focused on development impact. No safeguard or financial management issues came up during project implementation. The Bank addressed procurement related delays by providing training and adjusted model bidding documents. However, Page 7 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) procurement delays continued throughout project implementation. Implementation progress reporting was adequate and of good quality. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project’s objective was clearly defined. The theory of change was sound and well reflected in the results framework. The selected indicators were appropriate, had a baseline and were measurable and time-bound. The intermediate results indicators were also adequate to capture the contribution of the project’s activities toward achieving the objective. The Bank team stated that the M&E design and arrangements were well embedded institutionally and that the participating Banks heavily relied on the project’s M&E data to assess the financial soundness of their investments. At the time of AF, the outcome targets were revised upwards to reflect a scale-up in project activities. A number of intermediate indicators under Component B were dropped, including the portfolio quality and financial sustainability of PBs (PAD, Annex 3-Results Framework and Monitoring). b. M&E Implementation The PCU was responsible for the coordination of the project’s M&E activities. The PCU collected project performance data from the Asaka, Uzpromstroy and Hamkor Banks. The PCU developed, in cooperation with the PIU’s of the participating Banks (PBs), a progress report template as part of the operational manual. The PBs were responsible for monitoring and supervising the implementation of investments made by the Industrial Enterprises (IEs). Each PB was responsible for preparing a project folder including information on application, appraisal, safeguards, procurement, monitoring and supervision on each approved sub-loan. The Bank team reviewed these folders and conducted random site visits during its supervision missions. During project implementation minor modifications were made to the Results Framework by adding or removing indicators or intermediate indicators to better align them with the PDO, and indicate progress from the original loan to the additional finance. The Bank team stated that M&E data was reliable and of good quality. The team also spot-checked some of the data using basic operating principles of the investments. For instance, if the investment replaced boilers, the team would check that the new boiler’s savings were in line with the improvement in boiler efficiency. Page 8 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) c. M&E Utilization M&E data was used to inform decision making such as the request for additional financing. Also, M&E data was used to monitor implementation progress towards achieving the PDO and identify implementation bottlenecks such as the allocation of funds and eligibility criteria. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project was classified as Category FI and triggered the Bank’s safeguard policy OP/BP 4.01 (Environmental Assessment). The project developed an Environmental Management Framework (EMF) and the Bank trained project staff in implementing the EMF during project preparation, loan effectiveness, and application phase for the initial sub-projects. The project complied with the government of Uzbekistan’s and the Bank’s regulation, policies and procedures for environmental assessment. The project did not trigger any of the Bank’s social safeguard policies. The Bank trained the PCU and PBs in safeguard policies to ensure that sub-projects, which involve land acquisition or imposition on use of resources for any group would not be eligible for any financing. b. Fiduciary Compliance Financial Management: Both, the PCU and the PBs had sufficient financial management capacity. Also, the accounting, reporting, planning and budgeting functions of all entities were adequate throughout project implementation. The Bank assessed the PCU and PBs’ internal control system and found them satisfactory. The PCU collected and consolidated data from the PBs in financial reports. The interim financial reports as well as the audited project financial statements were submitted in a timely manner. The last Bank supervision mission found that the project had appropriate financial controls in place and that the 80 to 20 financing percentage was maintained for each sub-loan agreement. According to the Bank team, the external auditor’s opinion dated June 2017 (most recent available) was not qualified. Procurement: According to the ICR, the project complied with the Bank’s procurement guidelines. In April 2015, the procurement rating was downgraded to Moderately Satisfactory since some contracts took too long to be completed which resulted in implementation delays of sub-projects. The Bank team stated that there were a couple of cases when the bid awards were delayed due to the lack of competitive bids. In some cases, bids were not well advertised, and bidders asked for the extension of bid closing dates to prepare compliant bids. Page 9 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) The Bank provided adjusted model biding documents to the specifics of the project for national and international competitive bidding. Also, the Bank provided training on how to use these documents. However, these documents continued to present challenges for the sub-borrowers. c. Unintended impacts (Positive or Negative) NA d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory --- Bank Performance Satisfactory Satisfactory --- Quality of M&E Substantial Substantial --- Quality of ICR Substantial --- 12. Lessons IEG selected three lessons from the ICR, with some adaptation of the language: • Government support in the investment area which lacks awareness of potential gains is critical for attracting commercial financing. In this project, participating banks faced difficulties in identifying customers and preparing a project pipeline due to limited familiarity with EE investments. When the government passed relevant presidential decrees and regulations to provide clearer guidance for IEs to implement EE measures, technical barriers were removed and interest in EE investments increased. • In order to ensure long-term sustainability of developed EE financing mechanisms, participating banks need to strengthen their marketing and pipeline development capacity. In Uzbekistan, the financing gap for achieving the government’s short- and medium-term industrial energy saving target persists. Participating banks need to be motivated to use their own capital for EE investments. It will be important for participating banks to strengthen their capacity in conducting feasibility studies, developing EE loan appraisal skills and specific loan products for EE. • It is critical for the Bank to provide extensive procurement training at the beginning of project implementation to avoid any delays. In this project, participating banks and IEs were trained at the beginning of project implementation and investments could be made without delays. Also, the project hired consultants to support sub-borrowers in preparing their bidding documents. However, the support was insufficient and sub- borrowers continued to face challenges with national and international competitive bidding documents, resulting Page 10 of 11 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review ENERGY EFF - INDUST ENTERPRISES (P118737) in delays. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is internally consistent and concise. It is results- oriented and provides adequate detail into the economic and financial analysis of the project. Lessons are based on the project implementation experience. The ICR would have benefited from more detailed information in areas such as M&E and procurement. For example, the ICR does not mention that several intermediate indicators were dropped under Component B at the time of AF. Overall, the quality of the ICR is substantial. a. Quality of ICR Rating Substantial Page 11 of 11