Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) Report Number : ICRR0021403 1. Project Data Project ID Project Name P118561 CG Support to Economic Diversification Country Practice Area(Lead) Congo, Republic of Finance, Competitiveness and Innovation L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-48460 28-Feb-2016 7,607,204.56 Bank Approval Date Closing Date (Actual) 16-Dec-2010 31-Dec-2017 IBRD/IDA (USD) Grants (USD) Original Commitment 10,000,000.00 0.00 Revised Commitment 10,000,000.00 0.00 Actual 7,607,204.56 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan J. W. van Holst Christopher David Nelson IEGFP (Unit 3) Krishnamani Pellekaan 2. Project Objectives and Components a. Objectives The Project Development Objective (PDO) as stated in the Financing Agreement (Schedule 1, page 6) and in the Project Appraisal Document (PAD, page 8) was: " To promote private sector growth and investment in the non-oil sectors in the Republic of Congo (ROC) " Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) The PAD states that the project was to focus in particular on Small-Medium Size Enterprises (SMEs) on the Pointe Noire- Brazzaville growth corridor, identified as the most important and immediate source of economic growth and diversification. The PDO was revised through a Level 1 restructuring on February 24, 2014. The revised PDO as stated in the Implementation Completion Report (ICR, page 10) was: "To promote private investment in select non-oil value chain and to support SME development". The assessment of the project’s outcome before restructuring in this Review is based on the extent to which the following two sub-objectives were achieved: (1) To promote private sector growth in the non-oil sectors. (2) To promote private sector investment. The assessment of the project’s outcome after restructuring in this Review is based on the extent to which two revised objectives were achieved, namely: (1) To promote private investment in select non-oil value chains: (2) To support SME development. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 24-Feb-2014 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components There were four components (PAD, pages 8-15). The actual cost of the components is as recorded in the ICR (pages 43-44). One. Support to Public Private Dialogue (PPD) and Investment Climate Reforms. Appraisal estimate US$4.30 million. Actual cost US$1.80 million. This component aimed at supporting the Government's 2009 Action Plan for investment climate reforms. There were two sub-components: (i) financing for staffing the High Level Council for Public-Private dialogue (HCDPP) - a high level Public-Private Platform aimed at adopting an action plan for improving the investment climate and diversification of the economy into non- oil growth sectors: and (ii) Technical assistance for implementing the key policy reforms identified by the HCDPP including, review and improvement of legal, regulatory and institutional frameworks for enterprise creation, rationalization of the licensing and tax regime and support for preparing the legal framework for Special Economic Zones (SEMs). Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) Two. Support to Enterprise Development and Investment Promotion. Appraisal estimate US$8.40 million. Actual cost US$6.30 million. This component aimed at improving the institutional framework for supporting private investments. There were two sub-components: (i) support for creating the House of Enterprise Congo (MEC), responsible for organizing key investment promotion activities targeting potential investors, financing key market studies and providing technical assistance for implementing the main recommendations pertaining to improving the selected value chains: (ii) supporting private sector access to non-financial services through a matching grant program for promoting entrepreneurship in diversified enterprises. Three. Support to the reform of the railway company. Appraisal estimate US$2.70 million. Actual cost US$0.07 million. This component aimed at assisting the Government to implement a strategy for reforming the Congolese railway company (CFCO) through Public Private Partnership (PPP) arrangements. There were two sub-components: (i) Technical assistance to the CFCO for implementing the recommendations of the environmental and social audit: and, (ii) Technical assistance for implementing the PPP strategy. Four. Project Coordination and Implementation. Appraisal estimate US$3.60 million. Actual cost US$4.30 million. This component provided implementation support to the Project Implementation Unit (PIU) in the Ministry of Planning. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. Appraisal estimate US$19.00 million. Revised estimate after restructuring US$15.40 million. Actual cost US$12.47 million. The actual cost of component four activities was about 20% higher than the appraisal estimates and 8% higher than the revised estimate, while the cost of all other components were lower than estimated. The increase in cost of component four activities was met through reallocation of spending between project categories. The actual project cost was lower than the appraisal estimate due to a combination of factors including, exchange rate changes and reduced scope of project activities that were completed at project closure. Project financing. The project was financed by an IDA Credit of US$10.00 million. The ICR (page 17) notes that there were exchange rate fluctuations and the effective amount of the IDA credit was US$9.00 million. 84% of the credit (US$7.64 million) was disbursed at project closure. The team clarified that the balance of the credit was cancelled. There was parallel financing for private sector development activities in by the African Development Bank (AfDB), the European Union (EU) and the French Development Agency (AFD). Borrower contribution. Appraisal estimate US$10.00 million. Revised estimate after restructuring US$0.00 million. In the wake of the 2014-2016 global collapse of oil prices, the Government faced a serious fiscal crisis and as a result its contribution to the project was revised to zero after the Level 2 Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) restructuring of June 13,2017. Actual counterpart contribution by the Government at closure US$4.85 million. Dates. The project was approved on December 16, 2010, became effective on September 13, 2011 and originally scheduled to close on February 28, 2016. Restructuring. These changes were made through a Level 1 restructuring on February 24, 2014, when US$19.2 million or 25% of the loan IDA credit was disbursed. (1) The PDO and the results framework were revised: (2) Activities associated with Special Economic Zone and reform of the Congolese Railways were dropped. The former activity was completed by funding from other resources and the latter was dropped, due to changes in government priorities. Funds allocated to these activities were reallocated to other activities: (3) Two activities, supporting entrepreneurship within the six target value chains that the project focused on, were added: A Business Plan Competition (BPC) and developing two value chains for cassava, furniture, produce and others: (4) The safeguards on Forests was triggered (discussed in section 10a). and, (4) The Ministry of Industrial Development and Private Sector Promotion was made in charge of project implementation in place of the Ministry of Planning as initially set up. The following changes were made through a Level 2 restructuring on October 24, 2016. (1) The closing date was extended by a year for completing ongoing activities associated with the BPC and developing value chains, which were delayed due to the Government's delays in signing the amended Financial Agreement: (2) A revised set of “Doing Business” indicators was added for monitoring project performance. These changes were made through the second Level 2 restructuring on June 13, 2017. (1) This restructuring removed counterpart funding from the financial structure of the project: (2) Activities relating to the BPC were significantly scaled down and matching grants were no longer to be provided to eligible entrepreneurs for business sub-projects related to the BPC: (3) The results framework was modified to reflect the scaling down of project activities relating to BPC: and, (4). The project closing date was extended by about 10 months. The project closed on December 31, 2017. Split rating: Given that the PDOs were revised through a Level 1 restructuring on February 24, 2014, this review is based on a split rating of objectives, 25% (US$1.92 million) before restructuring and d the balance (US$7.64 million) after restructuring. 3. Relevance of Objectives Rationale Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) Original PDO. The Republic of Congo (RoC), with a population of 3.8 million, is a post conflict middle- income country, located in the central-western part of Sub-Saharan Africa. About 70% of RoC’s population lives in Brazzaville, the capital in the center of the country and Pointe Noire on the west coast of Central Africa, where most of the economic activities are concentrated. The RoC’s economic growth is driven by the oil sector, with the sector accounting for 65%, 85% and 92% of the Gross Domestic Product (GDP), government revenue and exports, respectively. The spill overs from the oil sector to the rest of the economy were however limited and 51% of the population were classified as poor in 2008 (PAD, page 1). The extent of the private sector in RoC was limited, with private sector investments in non-oil sector accounting for just about five percent of GDP (as compared to 14% on average in Sub-Saharan Africa and 25% of GDP in Asia (PAD, page 2). The Government's Poverty Reduction Strategy Paper (PRSP) of 2008 articulated the Government's vision of diversification of the economy and reducing poverty by half by 2015. The PRSP 11 for the 2012-2016 period also highlighted the need for RoC to exploit its comparative advantages in natural resource endowments for promoting non-oil growth. The PRSP for the 2012-2016 period reiterated the goal of stimulating inclusive economic growth and economic diversification. The Government's National Development Plan (PND) for the 2012-2016 period included a plan to boost public and private investment. The PDO was well-aligned with the following two pillars of the Bank's Country Partnership Strategy (CPS) for the 2010-2012 period: "promoting growth and macroeconomic stability" and "improving the social environment and integration of vulnerable groups". The Country Partnership Framework (CPF) for the 2013-2016 period defined the country's principal challenge as "how to use its large oil revenues to stimulate broad-based non-oil growth that would generate employment and reduce poverty". The most recent Systematic Country Diagnostic (SCD) completed in May 2017, considered the lack of diversification as a major challenge for sustainable growth and the SCD identified reducing the cost of doing business as the most import factor for achieving sustainable growth. The ICR (page 13) notes that the CPF for the 2018- 2022 period (still in concept note stage), focuses World Bank support in three strategic areas, the first of which was economic diversification for equitable growth. Revised Objective. The revised PDO focused on selected value chains in the non-oil sector and explicitly highlighted the objective of supporting SMEs for fostering entrepreneurship and creating jobs through SMEs. Therefore, the revised PDO were also relevant to Government and Bank strategies for the ROC. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) Objective Original Objective One. To promote private sector growth in the non-oil sectors in the ROC: Rationale Theory of Change. The project supported three clusters of activities for promoting private sector growth in non-oil sectors. The first included, technical assistance activities aimed at addressing the necessary conditions for investment climate reforms through promoting public-private dialogue and implementing key reforms for improving the institutional framework for enterprise creation, licensing and tax regimes, and preparing legal frameworks for the Special Economic Zone. The second cluster aimed at providing concrete firm-level support to SMEs through the creation of the House of Enterprise Congo (MEC) which would provide support mostly to SMEs and through setting up a matching grant scheme for enterprise development. The third cluster was aimed at supporting the reform of the railway company. The sequence and combination of these activities were expected to provide incentives for private sector investment and growth in the non-oil sectors. Outputs (ICR, pages 59-64). • The High Level Public Private Dialogue platform was adopted through the Decree Number 2017-42 on March 28, 2017 for improving the business climate. • 14 Doing Business reforms were adopted and implemented at project closure. This exceeded the target of 10. • Eight Doing Business Indicators aimed at improving the RoC's business environment were adopted compared to the revised target of seven (there were no original targets for this indicator). These reforms were in the areas of business creation, cross border trade, taxation, access to electricity, building permits and access to land. • 81 overall procedures measured by five Doing Business indicator surveys for which the RoC was ranked the worst were reduced during the project’s implementation. This exceeded the target of 80. • The ICR (page 35) notes that these reforms did not, however, improve RoC's overall score in the Bank's Doing Business Report of 2018. RoC lost two places compared to their 2017 standing in overall rating (from 177 to 179) and there was decline in the distance to the frontier score (DTF) score (from 40.09 to 39.57). The distance to frontier helps in assessing the absolute level of regulatory performance over time. Specifically, it seeks to measure the best performance observed on each of the indicators across all economies in the Doing Business sample since 2005. It helps to see the gap between an economy’s performance and the best performance at any point in time and assess the absolute change in the economy's regulatory environment over time. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) • The diagnostic of the Congolese Railway Company was completed. On the other hand, the activities associated with approval of the five-year business plan by the management of Congolese Railways and implementation of the PPP agreement for Congolese Railways was dropped. • The activity associated with Special Economic Zone was dropped, although completed through support from external donors. • About 150 investors in non-oil sectors received support from the Business Council for International Understanding (BCIU) and the French International Business Association (MEDEF). There were no targets for this indicator. • 1,078 jobs (450 in hotels and tourism, 600 in transport and logistics and artisanal crafts sector) were created during the lifetime of the project. This total was short of the target of 2,200 jobs. Outcomes. • The activities such as the creation of the High Level Public Private Dialogue platform and the creation of the House of Enterprise Congo, were output-oriented. Given that there were very few outcomes associated with the extent to which private sector contributed to growth in the non-oil sectors, this review concludes that the project made only a Modest contribution to realizing this outcome. Rating Modest PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective To promote private investment in selected non-oil value chain. Revised Rationale Theory of Change. The revised PDO was narrower in scope and focused on private investment in selected non-oil value chains. Technical assistance activities such as, organizing promotion activities targeting potential investors in sectors other than oil, implementing the recommendations pertaining to selected value chains (in cassava flour, furniture and produce), in conjunction with providing private sector access to non-financial services through matching grants, were aimed at promoting private investment in selected non-oil value chains. These activities could be expected to contribute to the long-term development objective of promoting diversification from an oil-dominated economy. Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) Outputs.(ICR, pages 60-61). In addition to those described above in connection with the first original objective, the following outputs were relevant to this objective. • Various studies were completed at project closure on value chain and marketing issues (a study on priority value chains, a study identifying opportunities in the fruit, market gardening and cocoa sectors, a study identifying a strategy for operationalizing PPP arrangements, and a study on improving logistics performance). • A development strategy was identified for the Pointe Noire - Brazzaville by the Economic Interest Group (GIE). The GIE was made up of the Farmers' Union, with 400 members and a biological fertilizer production Unit. 50 hectares (ha) of land was acquired along with seeds and a contract to sell paprika was awarded to the GIE. The project activities supported the GIE value chains pertaining to fruit juice, furniture (wood) and Cassava flour. Outcomes. • The aggregate private investment in selected value chains at project closure was US$143 million. This exceeded the target of US$50 million. The bulk of this investment was primarily in the hotel sector which had received a Matching Grant. • The number of days to register property went down from 116 days at project appraisal to 55 days at project closure. This improved on the target of 60 days. The number of procedures required for registering a business remained unchanged at ten. The number of tax payments was reduced from 61 at appraisal to 50. This was short of the target of 30. While it is difficult to determine the extent to which the capacity building activities contributed to realizing the PDO, given that the matching grants were restricted to providing non-financial services, it is reasonable to assume that activities pertaining to promoting private investment in conjunction with doing business reforms, substantially contributed to realizing the PDO. Revised Rating Substantial PHEFFICACYTBL Objective 2 Objective Original Objective Two. To promote private sector investment in the ROC. Rationale Theory of Change. Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) The project activities such as addressing the necessary investment climate reforms aimed at promoting private sector investment along the Pointe Noire- Brazzaville corridor identified as the most important and immediate source of economic growth and diversification. Outputs. • The outputs described above for the original objective one were also relevant to this objective. Outcomes. • The indicators aimed at investment climate reforms (such as promoting public-private dialogue and providing concrete firm-level support to SMEs through the House of Enterprise Congo), were output-oriented. In the absence of evidence pertaining to trends in private sector investment through time along the Pointe Noire-Brazzaville Corridor during project implementation, the efficacy of the objective of promoting private sector investment is rated as Modest. Rating Modest PHREVDELTBL PHINNERREVISEDTBL Objective 2 Revision 1 Revised Objective To support SME development Revised Rationale Theory of Change. Activities aimed at supporting SMEs through matching grants to SMEs in selected non- oil value chains. These activities can be expected to support the development of SMEs and thereby to the long term development objective of diversification away from an oil-dominated economy. Outputs. • In addition to the outputs described above, the following outputs were relevant to this objective. • 481 SMEs were supported by the matching grant program at project closure. This exceeded the target of 300. • The aggregate incremental sales of SMEs at project closure was US$17.2 million. This was well short of the target of US$50.00 million. • 28 women were trained through the matching grant in the artisanal sector. Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) • Strategies and action plans were developed for 77 Micro, Small and Medium Enterprises (MSME) at project closure. • Studies were completed by project closure on a range of issues related to SME development (a study on the implementation of pilot incubators for Micro, Small and Medium Enterprises (MSMEs): and a study on reform of the Agency for Development of SMEs). Outcomes. • 85.65% of the SMEs supported by the project through the matching grants achieved the agreed upon target performance indicators (ICR, page 55-56). The indicators included improvement in annual sales, job creation and credit amongst others (ICR, page 16). The matching grants were provided in sectors pertaining to agro-food, wood transformation, agriculture, transport and logistics, microfinance, agro pastoral, industrial welding and boilers and tourism and the hotel sector). This exceeded the target of 75% of SMEs which were supported through the project. Revised Rating Substantial PHOVRLEFFRATTBL Rationale Overall Efficacy The indicators before restructuring were output-oriented and there were no outcome indicators aimed at monitoring project performance with respect to the PDO of promoting private sector growth and investment in the non-oil sector. Efficacy before restructuring is, therefore, rated as Modest. Given that the revised outcomes pertaining to selected value chains and SMEs were largely realized after project restructuring when most of the Bank disbursements took place, efficacy after restructuring and for the project overall is rated Substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic Analysis. Ex ante analysis was conducted in the PAD for activities associated with matching grants. These grants accounted for 42% of the appraisal estimate of the project cost. The economic benefits were assumed to come from the number of jobs created by investors who received matching grants (PAD, page 93). The Net Present Value (NPV) at 12 percent discount rate was estimated at US$18.30 million and Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) the ex-ante Economic Internal Rate of Return (EIRR) was 53.6 percent. An ex post analysis was not carried out due to the lack of time series data (ICR, page 17). Administrative and Operational Issues. The ICR (page 17) notes that the project coordination and implementation cost which represented more than a third (34%) of the actual cost, was about 20% higher than the appraisal estimate. Important reasons for the high cost of project implementation was that the project was subject to implementation delays following effectiveness due to a combination of factors. These factors included lack of clarity as to the ministry would be responsible for project implementation, inadequate government commitment reflected through lack of intersectoral coordination, duplication of programs within different ministries, and high turnover of project coordinators (with a total of four people having held the position by project end). The delays were further exacerbated during implementation by the temporary suspension of disbursements due to issues associated with counterpart funding and delayed government approval of the Level 1 restructuring. Some activities (like the reform of the railways) were cancelled and the project scope was therefore reduced during implementation. There were delays associated with the matching grant activity due to the lack of clear guidelines about the procedural arrangements for providing the grants. Even though US$1.21 million of the matching grants were disbursed by February 2014, this activity did not start to generate any outcomes until October 2016. Given that many activities were completed only in the last year of the project, there was not sufficient time for monitoring some activities (like the impact of training programs), undertaken under the auspices of the project. There was no ex post estimate of the project’s economic rate of return. Other information provided in the ICR and summarized above indicate that project’s efficiency during implementation had significant shortcomings. The project’s efficiency is therefore rated modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Point value (%) *Coverage/Scope (%) Available? 42.00 Appraisal  54.00 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) 6. Outcome Original PDO. Although the original PDOs - to promote private sector growth and investment in the non-oil sectors in the ROC - were highly relevant to the Bank and Government strategies, the original objectives were broad, and the intended outcomes were not measurable. Efficacy of the two objectives was modest, as the indicators were output-oriented and there is no evidence that the expected outcomes were realized. Efficiency, is rated as modest, given that there was no ex post economic analysis in the ICR and there were significant administrative and operational inefficiencies, both at preparation and during implementation. Outcome before restructuring is therefore assessed as Moderately Unsatisfactory. Revised PDO. The revised PDO - to promote private investment in select non-oil value chain and to support SME development - was focused on selected value chains and SME development and reflected a more robust theory of change. Efficacy of the revised objectives - to promote private investment in select non-oil value chain and support SME development - is rated as Substantial, given that the most of the intended outcomes were realized. With modest efficiency, the overall outcome is rated as Moderately Satisfactory after restructuring. Taking into account the ratings discussed above and weighting them by the shares of Bank disbursement before and after restructuring (0.25*3 +0.75*4= 3.75), the overall rating of the project’s outcome is Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Political risk. There is a risk to ongoing benefits from the project activities, assuming that the political situation remains unstable after the 2016 Presidential election. Macroeconomic risk. There is macroeconomic risk associated with the vulnerability of SME performance to adverse external shocks associated with volatility in the global price of oil, given that the economy is still highly oil-dependent. Institutional risk. Based on the information in the ICR (page 30) it is not clear whether the government or the stakeholders have adequate implementation capacity for pursuing the diversification agenda. Overall, therefore, the risk to the project’s development outcome is assessed to be substantial. Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) 8. Assessment of Bank Performance a. Quality-at-Entry Preparation of the project was based on similar private sector development projects with investment climate, investment promotion and SME support in Africa and other regions and lessons from projects implemented by the Bank in post conflict countries (PAD, page 16). Lessons incorporated in the design of this project included, focusing mainly on technical assistance and capacity building activities as opposed to investment activities given that it was a fragile country and providing adequate resources for financing experienced staff for ensuring that the High Level Technical Council for Public-Private Dialogue (HCDPP) and the Congo Enterprise Resource Center (MEC) could quickly become operational. Several risks were identified at preparation including High risks associated with the macroeconomic context and governance risk and High project risk due to the weak institutional capacity. Several risk mitigation measures were incorporated in the project’s design including staffing the Project Coordination Unit (PCU) with experienced consultants. Even with the mitigation measures, the overall project risk was rated as High at appraisal. Appropriate arrangements were made for safeguards and fiduciary compliance (discussed in section 10). There were shortcomings in design. The project which included matching grant activities which required complicated implementation arrangements, was complex and ambitious, given the resources and the weak implementation capacity (especially given that the Project Implementation Unit was also in charge of implementing a project financed by the African Development Bank). This contributed to delays in during implementation. The project’s preparation overestimated the government commitment/ownership to the project at design. Project implementation was undermined by the lack of intersectoral coordination and duplication of programs within the different ministries. The Steering Committee meeting took place well after project approval. This issue was however resolved following the organizational changes made during implementation. It is not clear if there was adequate understanding of the political economy considerations facing the project, particularly with respect to activities pertaining to the reform of the railway company. There was virtually no disbursement of funds linked to this activity and it was dropped following the Level 1 restructuring. There was no clarity at preparation as to which ministry or government institution would be responsible for implementing the project. The Ministry of Planning was initially envisioned to be in charge of implementing the project. Eventually this was resolved with the Ministry of Industrial Development and Private Sector Promotion made in charge of project implementation. The project was prepared primarily by team members from Washington Headquarters. Project preparation by team members in country could have helped in providing a better understanding of the political economy considerations. Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) There were shortcomings in M&E design (discussed in section 9). Quality-at-Entry Rating Moderately Unsatisfactory b. Quality of supervision 14 Implementation Status Results Reports (ISRRs) were filed over a seven-year period (implying an average of two supervision missions every year). The team was headed by a Task Team Leader (TTL) in the initial years by a staff who was new to the Bank. The supervision missions were irregular, with no supervision missions between project approval in December 2010 and project effectiveness in September 2011. The ICR (page 28) notes that there was a distinct improvement in supervision following the Level 1 restructuring and committed effort by the supervision team after the final restructuring helped in identifying the existing activities that could feasibly be completed by project closure. Given that issues pertaining to the Government fiscal crisis following the fall in the price of oil was evident between 2014-2016, it is not clear why the decision to address the issue of lack of counterpart funding was not taken until June 2017. Timely identification of the issue could have helped in exploring other opportunities (discussed in later in Section 12). Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The original PDOs, promoting private sector growth and investment in the non-oil sectors, were to be monitored using two M&E outcome indicators: (a) the total private investments in non-oil sectors along the Pointe Noire- Brazzaville corridor; and (b) the total number of direct jobs created in non-oil sectors in enterprises supported by the project. There were attribution issues associated with these indicators. Given that the activities supported by the project were mainly capacity building activities, it is unclear whether these outcomes could have been achieved without complementary activities such as access to credit. The surveys for establishing baseline data and track performance were envisioned to be launched early during project implementation. Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) b. M&E Implementation The revised M&E design was more narrowly focused on private investment in selected value chains and on the SMEs supported by the project. The revised indicators were more realistic with respect to the activities supported by the project. The revised indicators appropriately included indicators pertaining to Doing Business. There was no dedicated M&E capacity for the project within the Project Coordination Unit (PCU) which was responsible for monitoring performance, also responsible for M&E of an African Development Bank funded project. The ICR (page 26) notes that there were no specialized training arrangements for undertaking M&E activities. There were drawbacks associated with the process of data gathering, with many companies failing to submit the data collection sheets to the PIU. The ICR (page 27) notes there were delays in starting M&E and during implementation, the lack of funding within the PIU undermined the ability of the PCU to undertake field visits. Given that the project focused mainly on capacity building activities, there was no beneficiary survey aimed at monitoring or evaluation to follow up with beneficiaries the extent to which they gained from the capacity building activities. c. M&E Utilization The M&E system was utilized for the limited purpose of informing project management and policy making and for assessing the efficiency of the project at closure (ICR, page 27). M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as a Category B project for environmental assessment purposes. Only the Environmental Assessment safeguard (OP/BP 4.01) was triggered (PAD, page 30). According to the PAD (page 30), no direct negative environmental impacts were anticipated at appraisal, as the project supported technical assistance activities. However, there was the possibility that future investments in key industries and locations, especially investments by the railway company (CFCO), could have adverse Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) environmental impacts. An Environmental and Social audit of CFCO was prepared and publicly disclosed at appraisal (PAD, page 97) and hence compliance with this safeguard was achieved. Although still classified as a Category B project, in addition to the environmental assessment safeguard, safeguards on Forests (OP 4.36) was triggered with the Level 1 restructuring. The restructuring paper (page 9) states that although the project was not expected to have direct impact as such on forests, some activities could have an impact in the timber supply chain. The ICR (page 49) notes that there were no negative impacts on the forestry environment during implementation and hence this project was in compliance with the forestry safeguard. The ICR (page 27) reports that there were no environmental issues during implementation. b. Fiduciary Compliance The Project Coordination Unit (PCU) in the Ministry of Planning was responsible for the fiduciary management. Financial Management. An assessment of the financial management capacity conducted at appraisal, concluded that the financial management risk was Substantial (PAD, page 76). The assessment identified issues associated with staffing arrangements, weak management capacity and issues associated with accounting systems. Mitigation measures adopted at preparation included, staffing the PCU with qualified experts (PAD, page 71). The ICR (page 26) reports that specialized training was provided during implementation to the PIU on financial management. The ICR provides no information on financial management compliance. The Task Team Leader confirmed, however, that there were no financial management issues during implementation. Procurement. An assessment of the procurement management capacity was conducted at appraisal (PAD, page 86). The procurement risk at appraisal was rated as High, mainly on account of the weak environment for governance and low level of capacity and overall lack of transparency. Mitigation measures incorporated at design included, hiring qualified project staff and providing adequate training. With the mitigation measures, the procurement risk was rated as Moderate at appraisal (PAD, page 87). The ICR (page 26) notes that PIU staff were trained during implementation. The ICR provides no information on procurement compliance. Nevertheless, the Task Team Leader advised that there were no procurement issues and no reported case of mis procurement during implementation. c. Unintended impacts (Positive or Negative) --- Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Moderately Outcome --- Satisfactory Satisfactory Moderately Moderately Bank Performance --- Satisfactory Satisfactory Quality of M&E Modest Modest --- Quality of ICR Substantial --- 12. Lessons The ICR (pages 30-32) draws six lessons from the experience of implementing this project. Of these, the four most important were as follows (with some adaptation of language). (1) Careful consideration of the political economy and macroeconomic conditions is required at project appraisal, particularly in the context of a fragile state. In this project, lack of Government Commitment and lack of clarity regarding which of the ministries would be responsible for implementing the project, contributed to the implementation delays in the initial years of the project. (2) A robust theory of change requires a clear causal chain between project activities and expected outcomes, as well as indicators which enable progress in meeting the objectives to be measured. No description of the theory of change was included in the original PAD. Addressing this issue when the project was restructured through a Level 1 restructuring would have helped in providing a clearer understanding of project objectives. (3) The modalities for the management of a matching grant operation needs to be carefully considered at design. The number of steps that needed to be taken for administering matching grants need to be carefully articulated. Further, mechanisms such as communication with stakeholders and/or using an independent/autonomous firm to manage this activity may be needed, particularly in the context of a fragile country with likely weaknesses in public administration. (4) The inclusion of a large percentage of total project cost from counterpart funding can pose significant risks to the achievement of project outcomes. Given RoC's high oil revenues, the project included a large counterpart funding element which was aimed at ensuring government commitment. A significant time elapsed between the collapse in the global oil price during 2014-2016 and Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review CG Support to Economic Diversification (P118561) actions removing counterpart funding for the financing of this project through the Level 2 restructuring in 2017. Earlier identification of the risk of a decline in oil price could have provided more room for exploring other financing options such as applying for additional IDA funding before project closure. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is reasonably clear and candidly discusses the delays that arose during the initial years of the project and the shortcomings in the implementation of M&E. It is also candid in acknowledging that, although the fall in the world price of oil was apparent between 2014- 2016, action was taken to address the issue of lack of counterpart funding through a Level 2 restructuring in June 2017, just six months before project closure on December 2017. The ICR is largely consistent with the guidelines and appropriately conducts a split rating of objectives. The ICR provides no information on issues associated with fiduciary compliance. This information was later clarified by the Task Team Leader. The ICR is unduly long (32 text pages, more than double the recommended length) and would have benefitted from stronger editing. a. Quality of ICR Rating Substantial Page 18 of 18