Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) Report Number: ICRR0021762 1. Project Data Project ID Project Name P118916 Indonesia Infrastructure Guarantee Fund Country Practice Area(Lead) Indonesia Finance, Competitiveness and Innovation L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-81920 31-Mar-2018 4,114,071.75 Bank Approval Date Closing Date (Actual) 11-Sep-2012 31-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 29,600,000.00 0.00 Revised Commitment 4,114,071.75 0.00 Actual 4,114,071.75 0.00 Prepared by Reviewed by ICR Review Coordinator Group Antonio M. Ollero Fernando Manibog Christopher David Nelson IEGFP (Unit 3) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project development objective of the Indonesia Infrastructure Guarantee Fund (IIGF) Project was "to strengthen the Indonesia Infrastructure Guarantee Fund as a single window institution to appraise infrastructure public-private partnership (PPP) projects requiring government guarantees" (Project Appraisal Document, page 8, and Loan Agreement, page 5). Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components Focused on institutional development, the project had two components: Technical Assistance (US$4.6 million estimated at appraisal, US$4.1 million disbursed) aimed to: (a) help develop the IIGF's institutional capacity to: screen, appraise, and supervise infrastructure PPP projects; manage its operations and strengthen its internal controls; build capacity among project sponsors and government contracting agencies; and, develop standardized documents and procedures to enable PPP projects to obtain IIGF guarantees; and, (b) support PPP project preparation activities, including the conduct of feasibility studies and the provision of transaction advisory services. Bank-Supported IIGF Guarantees (US$25 million estimated at appraisal, US$0 disbursed) aimed to support the IIGF in issuing its own guarantees. The component provides Bank financing to back IIGF guarantees to qualifying projects. According to the PAD (pages viii-x) and the ICR (page 11), it was entirely possible that the IIGF would not draw upon this financing --- disbursements would only be made if IIGF guarantees backed by the Bank were called for payment by a guaranteed party. Nonetheless, whether eventually disbursed or not, Bank financing to back IIGF guarantees was deemed useful to the institutional development of the IIGF because the financing "signaled" Bank support for the IIGF's guarantee operations, giving the IIGF greater credibility in the PPP market. The PAD and the Loan Agreement (pages 6-7, 11-12, and 15-21) defined the terms under which the loan proceeds would be utilized and disbursed for this purpose: (a) qualifying IIGF projects utilizing a PPP structure would need to be appraised by the IIGF using the operations manual developed under the project and issued an IIGF guarantee; (b) from among qualifying projects submitted by the IIGF, the Bank would chose to back, in whole or in part, an IIGF guarantee in favor of a Bank-eligible guaranteed party for Bank- eligible risks, namely: breach of contract --- failure of the contracting agency to comply with the financial payments agreed with the project sponsor or investor; failure by a contracting agency to adjust in a timely manner contractually agreed service tariffs; a unilateral change in tariffs by the government or the contracting agency that were not contractually specified; a change or repeal by the government of the agreed framework --- or changes in laws and regulations that adversely affect the financial equilibrium of the project sponsor or investor; (c) for this purpose, the Bank would issue a no-objection letter for the use of the project funds to back an IIGF guarantee, which would have been separately appraised by the Bank; (d) the IIGF would be solely and exclusively responsible for the payment of any claim by a guaranteed party, whether a disbursement of the project funds was made or not by the Bank (Bank-supported guarantees would not be Bank guarantees but rather IIGF guarantees backed by financial support provided by the Bank through the project); and, (d) the loan proceeds from this component would be disbursed at the time a guaranteed party made a claim for a Bank-supported IIGF guarantee. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The project was estimated to cost US$79.6 million at appraisal. Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) Project Financing: The Bank provided a US$29.6 loan to the Government of Indonesia for the project. The loan proceeds were to be on-lent by the government to the IIGF. Only US$4.1 million of the loan was disbursed, for the Technical Assistance component. The US$25 million Bank-Supported IIF Guarantees component was not drawn upon by the IIGF. Borrower Contribution: The contribution of the Government of Indonesia to the project was US$50 million, used to capitalize the IIGF which was created in 2009, according to the Project Appraisal Document (pages x and 10). Dates: The project was approved on September 11, 2012 and became effective on June 12, 2013. The project was restructured on January 20, 2018, with US$3.43 million (11.6 percent) of the original US$29.6 million Bank financing disbursed, to extend the project closing date to allow all pending technical assistance activities to be completed. The project closed on December 31, 2018, nine months after the original closing date of March 31, 2018. 3. Relevance of Objectives Rationale The project was relevant to the strategy adopted by Indonesia for infrastructure development and investment at the time of project appraisal in 2011. The National Medium-Term Development Plan for 2010-2014, the second-phase implementation of the country's National Long-Term Development Plan for 2005-2025, advocated for a strong role by PPPs in infrastructure investment and in national economic development. The document argued that "the ability of private companies to reduce costs, shorten the procurement period, and manage construction and facilities more efficiently, have made PPPs capable of offering value for money compared to the construction of the same facilities by the government." The plan advanced four strategies: (a) the development and harmonization of policies to facilitate the formation of the PPPs; (b) the development of laws and regulations to expand PPP priority sectors; (c) the compilation of the PPP book, or list of government projects that could be implemented each year through cooperation with private companies; and, (d) the strengthening the country's PPP institutions. With the IIGF tasked as the single window institution to appraise to appraise PPP projects requiring government guarantees, the project development objective to strengthen the institutional capacity IIGF was highly relevant to the country's national infrastructure plan. The project remained relevant to the strategy of Indonesia with infrastructure development and investment at the time of project closing in 2018. The National Medium-Term Development Plan for 2015-2019, the third phase implementation of long-term plan, continued to emphasize the role of PPP's in the country's infrastructure and economic plan. The third medium-term plan continues to place a strong emphasis on infrastructure development and financing, including on access to affordable housing, integrated tourism facilities, improved water supply and irrigation, delivered through innovations in infrastructure financing with PPPs. The project was aligned with the Bank Group strategy in Indonesia at project appraisal in 2011. The "Country Partnership Strategy (CPS) for Indonesia FY2009-2012: Investing in Indonesia's Institutions for Inclusive and Sustainable Development" committed Bank Group support to Indonesia's development programs organized around five "core engagements": (a) private sector development; (b) infrastructure; (c) Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) community development and social protection; (d) education; and, (e) environmental sustainability and disaster mitigation. The project development objective was aligned with the CPS strategy for the second core engagement (infrastructure), "to support institutions, both public and private, in finding more effective solutions to bridge the financing gap in infrastructure in a sustainable way, and help strengthen the accountability and capacity of institutions to deliver better infrastructure outcomes" and for the intersection of the first (private sector development) and second core engagements, "to support the establishment of an innovative infrastructure financing institution that will help to accelerate private investment in infrastructure." The project remained aligned with the Bank Group strategy in Indonesia at project closing in 2018. The "Country Partnership Framework (CPF) for the Republic of Indonesia for the Period FY16-FY20" committed Bank Group support to Indonesia's development programs organized around six "engagements": (a) infrastructure platforms at the national level; (b) sustainable energy and universal access; (c) maritime economy and connectivity; (d) delivery of local services and infrastructure; (e) sustainable landscape management; and, (f) "collecting more and spending better." The project development objective was aligned with the "infrastructure development features" of the CPF across all six engagements. The CPF asserts that "reducing the infrastructure gap would help alleviate the constraints that infrastructure bottlenecks impose on competitiveness, growth and creation of jobs; support the delivery of services including health and education; and, provide inclusion and connectivity across the country’s vast archipelago." Investment of over US$100 billion a year would be required over fiscal years 2016-2020 to close the infrastructure gap. The state budget, including foreign borrowing, was expected to finance about one-third of the requirements, with the remaining drawn from state-owned enterprises and the private sector. The Bank Group would "through a package of loans, equity and targeted technical assistance, aim to increase privately-financed infrastructure, including those with partial or full private sector financing or benefiting from a government guarantee." Rating Relevance TBL Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To strengthen the Indonesia Infrastructure Guarantee Fund as a single window institution to appraise infrastructure public-private partnership projects requiring government guarantees. Rationale Theory of Change The Project Appraisal Document did not provide a theory of change (results chain) for the project. The ICR (pages 6-9) provides one: technical assistance to develop the institutional capacity of the IIGF and contingent financing to back IIGF guarantees would deliver as outputs --- through higher standards of project appraisal and environmental and social safeguards performance, an increase in the number of bankable infrastructure Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) PPP projects, a reduced risk perception of PPP projects in the presence of IIGF guarantees, and an increase in private financing of PPP projects, and as an outcome --- a stronger IIGF, acting as the country's single window institution to appraise infrastructure PPP projects requiring government guarantees. Over the long- term, the outcomes would be: sustainable financial institutions that catalyze infrastructure PPPs, and more private capital for Indonesia's infrastructure. The degree of achievement of the project objective is assessed as substantial.  The project exceeded all the quantitative outcome targets defined at appraisal. The IIGF: (a) provided guidance to 42 projects, exceeding the target to provide guidance to six projects (guidance meant the provision by the IIGF of eligibility criteria, safeguards requirements, sample concession agreements, and guarantee application checklists to a contracting agency for a project under PPP preparation); (b) screened 51 projects, exceeding the target to screen four projects (screening involved the determination by the IIGF of: the eligibility of a project for guarantee appraisal; the nature of the risks to be covered; and, whether the guarantee would be an IIGF guarantee using the agency's own capital resources, an IIGF guarantee backed by Bank financing, or a Ministry of Finance guarantee); and, (c) appraised 25 projects, exceeding the target to appraise two projects (appraisal included: the submission by a consulting firm to the IIGF underwriting committee and the IIGF board of directors of a recommendation to issue a guarantee; the determination by the IIGF of the structure of a guarantee package; and the offer by the IIGF of a guarantee term sheet in the request for proposals issued to bidders). The IIGF also: (d) maintained an "unqualified opinion" status for its financial statements for each year during the life of the project (the financial statements were audited by a reputable accounting firm), meeting the sole output target; obtaining an "unqualified audit opinion" was deemed a signal of prudent financial management by the IIGF.  Moreover, according to the ICR (pages 16-17), the project helped strengthen the institutional capacity of the IIGF in ways beyond those defined by the results indicators: (a) Bank support for the IIGF reportedly enhanced investor confidence in IIGF guarantees, according to feedback obtained by Bank staff from private investors and lenders; (b) using its own capital resources, the IIGF issued guarantees valued at US$2.7 billion to 18 projects that mobilized US$13 billion in financing by the end of 2018; (c) the US$13 billion of IIGF guarantees represented a private investment multiplier ratio of 4.9X, which compares well with the MIGA's 3.4X, and a guarantee leverage ratio of 3.7X, compared to the Korea Infrastructure Guarantee Fund's 8.2X; (d) the operations manual prepared under the project set project preparation standards that will be useful to future infrastructure PPP projects in Indonesia; (d) learning from the inability of government contracting agencies to meet environmental and social safeguards requirements that could have otherwise qualified some projects for IIGF guarantees backed by Bank financing, the IIGF is reportedly working to improve compliance by its projects to international safeguards standards; and, (e) technical assistance under the project, including the assistance provided by the Government of Singapore, helped the IIGF prepare corporate governance guidelines, in areas including transparency and disclosure, operational independence of the IIGF, the ring-fencing of IIGF assets from government interference, and the ring-fencing of government contingent liabilities relative to guarantees issued for PPP projects.  No project was issued an IIGF guarantee backed by the project funds. That the financing from the Bank would not been drawn upon was recognized as a possibility during project design: only if a risk covered by the Bank backing an IIGF guarantee occurred --- a breach of contract or changes in laws and regulations --- and a claim for payment by a guaranteed party were made would the financing be drawn upon (see Section 2.D) But the intent was that the project funds would be committed to back IIGF guarantees, whether the financing would eventually be drawn upon or not. However, the Bank Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) financing was not committed to back any IIGF guarantee: two projects that could have utilized the guarantees --- the Government Multifunction Satellite project and the Dharmais Cancer Hospital project --- were submitted to the Bank in May 2018 but withdrawn in October 2018, because the government contracting agencies could not complete the environmental and social safeguards requirements necessary for the Bank to clear the use of the project funds to back IIGF guarantees.  According to the ICR (pages 13-15), there were other reasons why the IIGF failed to tap the Bank financing to back IIGF guarantees: (a) the pipeline of projects was thin, with a slow buildup of projects between 2013 and 2016, leaving little time to apply for and obtain the Bank financing; (b) the project funds allocated to back guarantees was small at US$25 million relative to the potential requirements of some eligible projects --- the Palpa Ring project would have required a guarantee in the amount US$550 million; and, (c) the IIGF had considerable headroom with its own capacity to issue IIGF guarantees, with a Ministry of Finance plan to raise the leveraging capacity of the IIGF to 10 times equity giving the IIGF even greater capacity to issue guarantees using its own capital. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale The project exceeded all quantitative outcome targets defined at appraisal, with the IIGF providing guidance to 42 infrastructure PPP project proposals, screening 51, and appraising 25. Importantly, the project helped strengthen the institutional capacity of the IIGF, with the IIGF issuing guarantees for 18 projects valued at US$2.7 billion issuing its own capital resources, mobilizing some US$13 billion in private financing in the process, setting an operations manual that will be used for future infrastructure PPP projects in Indonesia, working to improve compliance by its projects to international environmental and social safeguards standards, and crafting, with the aid of external technical assistance, its corporate governance guidelines. Overall Efficacy Rating Substantial 5. Efficiency The efficiency of the project is rated as modest. Economic Efficiency: The Project Appraisal Document did not calculate an economic rate of return for the project. Instead, the document (page 18) stated that the economic benefits from the project would be equal to the sum of the economic benefits accruing to each project receiving an IIGF guarantee that was backed by the Bank financing. Because the operation did not deliver any project for guarantee backing, the ICR (page Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) 14) concluded that the project-level economic benefits listed at appraisal were not realized. The Project Appraisal Document also argued that , in general, the guarantee operations of the IIGF would result in several macroeconomic benefits accruing to Indonesia: (a) a decrease in the cost of capital from improved project financing terms, including from a reduction in interest rate spreads and an extension of loan maturities; (b) lower tariffs and government contributions to projects because of the improved terms for project debt; (c) higher private investor participation in PPP projects owing to the better functioning of the IIGF, greater government commitment to the country's PPP program, and the perception by investors of lower risks in the PPP market; and (d) the development of the capital market for long-term finance in Indonesia. The document explained that these macroeconomic benefits could not be quantified ex-ante. The ICR did not attempt to quantify these benefits either. Operational Efficiency: Only US$4.1 million of the original US$4.6 million of the project funds for technical assistance were disbursed. The government had inadvertently left out the budgetary appropriation for some of the project's technical assistance activities form the State Budget for 2019 . The project also closed nine months later than originally scheduled. The government had requested an extension of the project closing date to allow the IIGF to complete pending technical assistance activities. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The outcome of the project is rated as moderately satisfactory. The project objective to strengthen the institutional capacity of the IIGF as the single window institution to appraise infrastructure PPP projects that require government guarantees was highly relevant to the development objectives of Indonesia to raise infrastructure investment, draw private sector financing through PPP projects, and strengthen the country's infrastructure development, financing, and guarantee institutions. The objective was also aligned with the Bank Group strategies in Indonesia, both at appraisal and at project closing. The degree of achievement of the project objective was substantial. The IIGF exceeded the targets for the number of projects provided guidance, screened, and appraised during the guarantee appraisal process. Although there were no targets set for the number or value of guarantees issued, the IIGF extended guarantees with a nominal value of US$2.7 billion to Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) 18 infrastructure PPP projects, mobilizing US$13 billion in financing for these projects by the end of 2018. The IIGF did not draw on any of the US$25 million financing provided by the project to back guarantees, a possibility explicitly recognized at appraisal. But the IIGF should have at least secured commitment of the project funds to back eligible guarantees, except that government contracting agencies failed to meet environmental and social safeguards required by the project standards. The efficiency of the project is rated as modest. No evidence was available to judge the economic efficiency of the project. As to operational efficiency, about US$0.5 million of the US$4.6 million of the project funds allocated for technical assistance was not disbursed because the government had neglected to cover some activities with a budgetary appropriation, and the project closing date had to be extended by nine months to complete some pending technical assistance activities. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome There are several risks to the sustainability of the development outcome achieved by the project. Project pipeline risk. Although there was some progress with the development of infrastructure PPP projects especially during 2016-18, the pipeline of PPP projects in Indonesia remains thin. The slow development of the country's PPP market --- the relative shortfall of projects for which contracts have been awarded to private developers (projects that achieve "commercial close"), or for which financing contracts have been signed (projects that achieve "financial close") --- represents a lost opportunity for the IIGF's guarantee operations. The probability of this risk occurring appears high --- there has been no recent significant acceleration in the pace of project development; and the costs are high --- the profitability of the IIGF could be negatively affected if the project pipeline were to tail off. Sector risk. The viability of many infrastructure PPP projects depends on the integrity of pricing dynamics in the infrastructure sectors. If the government was unable to implement an agreed-upon framework for setting service tariffs, a PPP project would likely call on an IIGF guarantee for a breach of contract. The probability of this risk occurring appears moderate --- structural reforms are underway in many infrastructure sectors, including with cost-reflective tariffs; but the costs, should the risk occur, are high --- the liabilities of the IIGF could rise more than expected. Legal framework risks. Indonesia is still perceived as a risky destination for foreign investment, in part because experience shows that Indonesia’s legal framework can be unpredictable. For example, in 2015, the Indonesian Constitutional Court invalidated the Water Resources Law (Law No. 7/2004), negating many contractual commitments previously made under the law, including those related to water resource commercialization, privatization, and project cost recovery. The probability of this risk recurring is moderate --- legal certainty still looks far from assured in the country; the cost, if this risk occurred, is high --- a major adverse legal decision as was handed down in 2015 could negatively affect both PPP projects and the IIGF which covers project risks related to changes in laws and regulations. Financial risks. The IIGF underwrites the risks related to the payment obligations of government contracting agencies. If a government contracting agency were poorly managed, it could default on its payment obligation to a PPP project investor. The probability of this risk occurring is moderate --- the Ministry of Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) Finance is committed to enforcing fiscal discipline in government contracting agencies; the cost, if this risk occurred, would be high --- the IIGF would need to assume the contractual obligations of a defaulting government contracting agency for a project which the IIGF would have guaranteed. IIGF institutional capacity risk. Created in 2009, the IIGF remains a young institution, with the bulk of its experience confined to ex-ante guarantee appraisals, the structuring of contractual arrangements, and limited monitoring of projects. However, IIGF guarantees are typically issued for 12 to 15 years, which means that the IIGF has not had the full-cycle experience of a guarantee operation. This inexperience could be challenging to the IIGF if a mature project with an IIGF guarantee were to present problems. The probability of this risk occurring is moderate --- the IIGF would have gained some more years of experience in the near term; the cost, if this risk occurred, is moderate --- the IIGF could seek advice from, or learn from the experience of, other guarantee institutions elsewhere. IIGF portfolio risk. Many of the projects for which the IIGF issued guarantees in 2016-18 are still under construction. A portfolio that is concentrated on "early stage" projects suffers many of the adverse financial consequences of an undiversified portfolio --- higher volatility and lower return on a risk-adjusted basis. The probability of this risk occurring is moderate --- the projects which are currently under construction will become operational in 2019-21, although they will still be in the "early gestation" phase; and the cost of this risk is high --- higher volatility for the IIGF's finances. 8. Assessment of Bank Performance a. Quality-at-Entry The quality at entry was moderately satisfactory. The Bank incorporated into the project design the lessons learned from its guarantee operations globally, Indonesia's own experience with guarantee operations, and the Bank's engagement with Indonesia on infrastructure, including in the First, Second, Third and Fourth Infrastructure Development Policy Loans, and the Indonesia Infrastructure Finance Facility (IIFF) Project. Among vital design elements, the project: (a) required the adherence by the IIGF to Presidential Decree No. 78/2010 and Minister of Finance Regulation No. 2018/2010 on the issuance of guarantees for infrastructure PPP projects --- to strengthen government ownership of the project (PAD, page 10); (b) mandated the use by the IIGF of appraisal guidelines laid out in the operations manual, including for environmental and social safeguards --- to set rigorous standards for project appraisal across all projects regardless of funding source (PAD, page 10); (c) introduced a recourse mechanism, allowing the IIGF to recover funds from contracting agencies --- to establish accountability, provide incentive for agencies to honor contractual obligations, and limit risks to the state balance sheet (PAD, page 11); and, (d) provided technical assistance for project preparation --- to build a pipeline of infrastructure PPP projects. The Bank thoroughly considered alternative instruments for the operation. On one hand: (a) the volume of market-ready PPP transactions was judged too thin to be suitable for a Partial Risk Guarantee Facility (PRGF), which would have required the Bank to act as co-guarantor of PPP projects with the IIGF; (b) Deferred Drawdown Option (DDO) Loans were typically used for catastrophe risk management; and, (c) a Development Policy Loan (DPL) would not have provided a platform for the preparation and Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) appraisal of infrastructure PPP subprojects. On the other hand, the Specific Investment Loan offered the flexibility of combining technical assistance for an institutional development initiative with a contingent financing to back IIGF guarantees, enhancing the IIGF's institutional standing in the infrastructure PPP market. The Bank collaborated with other donors to strengthen IIGF's institutional capacity. The Bank secured a technical assistance grant from the Government of Singapore and the Temasek Foundation (also of Singapore) which helped develop a corporate governance framework for the IIGF. The governance framework aimed to set standards of transparency and disclosure at the organization, ensure its operational independence, and ring-fence its assets from potential interference by the government. The Bank considered the operational risks to the project to be substantial, particularly those related to the implementation capacity of the IIGF, which was a new organization and had no experience with Bank operations. The Bank set up mitigation measures to address these risks: (a) the project would follow the financial management arrangements laid out in the operations manual, which would be reviewed by the Bank; (b) assisted by an international consulting firm, the IIGF would assess the capacity and review the procedures of contracting agencies on procurement, to meet Bank procurement standards; and, (c) to prevent the mingling of loan funds with the capital resources of the IIGF, the loan proceeds from the Bank-Supported IIGF Guarantees component would be disbursed only at the time a guaranteed party made a claim for a Bank-supported IIGF guarantee. The Bank adequately vetted many of the operational details of the project: (a) the IIGF would use a single operations manual to appraise all projects applying for government guarantees, whether the guarantees were funded by IIGF or by the Ministry of Finance, and whether the IIGF guarantees were backed by the Bank or not; (b) the Bank would back IIGF guarantees for only a subset of the risks (three of the eight) covered by IIGF and Ministry of Finance guarantees --- breach of contract, and changes in laws and regulations, but not delays or failures with land acquisition, with approvals of licenses and permits, with financial closure, with enforcement against illegal activity, and with project termination risk; and, (c) attention to environmental and social safeguards would be required with the inclusion of the environmental and social management framework, resettlement policy framework, and indigenous peoples framework in the operations manual. The Bank performance was deficient with respect to the following: The size of the Bank-Supported IIGF Guarantees component may have been sub-optimal. According to the ICR (page 22), the median value of guarantees issued by the IIGF was US$120 million, much larger than the total US$25 million in financing available from the project. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision The quality of supervision was moderately satisfactory. Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) The Bank fielded 12 supervision missions during the six-year life of the project, matching the average twice-a-year mission for most investment project financing operations. The supervision mission filed 12 Implementation Results and Status Reports (ISRs). According to the ICR (page 22), the reports were useful, focusing on major implementation issues faced by the project. The supervision missions were proactive in providing advice concerning the preparation of infrastructure PPP projects, strengthening the IIGF’s environmental and social safeguards review function, and capacity building at government contracting agencies for environment and social safeguards compliance. For example: (a) the supervision mission of July 2016 identified the actions that the developers of the Palapa Ring project needed to implement to comply with the operations manual --- the project quickly achieved financial close soon after; (b) the mission of December 2016 prepared an action plan to help the IIGF improve the project application process; (c) the mission of May 2017 facilitated discussions with the Bank's local safeguards team on the review of project documents, the conduct of site visits, and requests for additional technical assistance; (d) the mission of December 2017 commented on revisions to the operations manual proposed by the IIGF; and, (e) the supervision mission of June 2018 continued discussions with the IIGF on strengthening the agency's environmental and social safeguards review function. The mid-term review of June 2015 encouraged the IIGF to exert additional effort in having government contracting agencies adhere to the procedures and standards of the operations manual. It should be noted that the government had agreed (Loan Agreement, page 6) for the IIGF to use the operations manual for all its IIGF guarantee operations, whether an IIGF guarantee was backed by Bank financing or not. There were moderate deficiencies with the quality of supervision: (a) there was a high turnover rate at the task team leader position --- four task team leaders over a six-year period; (b) the Bank could have improved the M&E design during project implementation, including by raising the low targets for project guidance, screening and appraisal; and, (c) according to the ICR (page 26), the Bank could have requested early for a larger Bank-Supported IIGF Guarantees component to cover projects requiring larger guarantee amounts. Quality of Supervision Rating Moderately Satisfactory Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The M&E plan defined one output indicator and three outcome indicators for the project. The output indicator --- that the IIGF received from a reputable audit firm an unqualified opinion on its financial statements for each year during the project's life --- was intended to be a measure of the quality of the firm's financial management. According to the ICR (page 23), this was a suitable metric, although it could have been supplemented with quantitative targets, including for capital adequacy, asset quality, and return Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) on equity. The three outcome indicators -- the number of projects provided guidance, the number of projects screened, and the number of projects appraised by the IIGF --- were to provide evidence of the ability of the IIGF to act as the institution to appraise infrastructure PPP projects requiring government guarantees. According to the ICR (page 23), these were measurable indicators and reflective of the ability of the IIGF to fulfill its mandate as a guarantee institution. However, considering the length of the project (six years from 2012 to 2018), the targets appear to have been set too low --- six guidance issuances, four screenings, and two appraisals --- such that they were bound to be readily exceeded by the IIGF. A valid criticism against the M&E design is that it lacked results indicators for other aspects of institution building envisioned by the project. The technical assistance also aimed to build capacity among project sponsors and contracting agencies in addition to the IIGF. Yet, there were no measures to evidence achievement of these goals. Two projects that could have qualified for Bank backup financing of an IIGF- issued guarantee were withdrawn because the government contracting agencies did not ensure that the projects met environmental and social safeguards requirements. A metric reflecting compliance by eligible projects to safeguards standards would have served to indicate progress with efforts to have contracting agencies assume responsibility for adherence to safeguards standards. There were also no results indicators to measure progress with the support given for PPP project preparation, including the conduct of feasibility studies and the provision of transaction advisory services (consultants hired by the project helped the IIGF structure guarantees for three water projects and ten toll road projects, that were financed by the IIGF, although not backed by the Bank). The M&E design did not define indicators for the provision by the Bank of financing to back IIGF guarantees. It is understandable that neither the Bank nor the government could determine ex-ante whether the financing would be drawn upon, or how much would be disbursed if it were utilized. But the objective was to commit the funds to back IIGF guarantees for eligible projects, hence there should have been at least an indicator to measure the ability of the IIGF to secure commitment from the Bank to back the financing of some IIGF guarantees. b. M&E Implementation M&E data collection and analysis was systematic and regular. The IIGF submitted quarterly progress reports, including: (a) the report on procurement and disbursement; (b) the interim financial review report; (c) the portfolio-level environmental and social safeguards report; (d) the IIGFP quarterly operations report; and, (e) audited and un-audited financial statements. External auditors performed annual audits of the IIGF institution-wide and provided professional opinions on the conduct of the agency's business. c. M&E Utilization According to the ICR, the M&E indicators were useful for gauging the project's progress with the basic institution-building activities at the IIGF that were supported with technical assistance. However, the absence of results indicators measuring progress with other aspects of institution building, including those at project sponsors and at government contracting agencies, make it difficult to conclude that the M&E was fully utilized for strategic decision-making or for project course-correction purposes. The same Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) criticism applies to the lack of results indicators for the Bank-Supported IIGF Guarantees component of the project, which ultimately was not tapped to back any IIGF guarantee. M&E Quality Rating Modest 10. Other Issues a. Safeguards Environmental Safeguards. Infrastructure PPP projects that are issued IIGF guarantees backed by Bank financing would likely have moderate to significant environmental and social impacts, hence the project was assessed as an Environmental Category "A" project at appraisal, a classification requiring a full and comprehensive Environmental Impact Assessment (EIA) following the Bank's safeguard policy OP/BP/GP 4.01 (Environmental Assessment). The project also triggered policies OP/BP 4.04 (Natural Habitats), OP 4.09 (Pest Management), OP/BP 4.11 (Cultural Resources), OP/BP 4.12 (Involuntary Resettlement), OP/BP 4.10 (Indigenous Peoples), and OP/BP 4.37 (Safety of Dams). In response, the IIGF, with the assistance of the Bank, prepared an Environmental and Social Management Framework (ESMF) to: (a) identify the Bank safeguard policies, Indonesian laws and regulations, and international occupational health and safety standards that applied to projects issued IIGF guarantees backed by Bank financing; and, (b) set the arrangements to ensure that the policies, laws and regulations, and standards were implemented and that projects met all safeguards requirements. The ESMF, disclosed in-country at the IIGF's website and at the Bank's InfoShop in October 2010, also included a Resettlement Policy Framework and an Indigenous Peoples Planning Framework. The detailed procedures for the environmental and social review of projects were included in the operations manual. The IIGF hired a an environmental specialist and a social development specialist, who would, with the assistance of other consulting experts, oversee compliance with the procedures by the IIGF, project sponsors, and government contracting agencies. According to the ICR (pages 49-50), the IIGF reported challenges implementing social safeguards for three reasons: (a) weak capacity at government contracting agencies; (b) a general reluctance by contracting agencies to spend time and resources to undertake safeguards analyses and to devise safeguards measures prior to a PPP tender; and, (c) the evolving capacity of the IIGF itself. The project's overall safeguards ratings was never below moderately satisfactory during project implementation (the ratings applied to the Technical Assistance component of the project, since no disbursements were made for the Bank-Supported IIGF Guarantees component). b. Fiduciary Compliance Financial Management. The Bank examined at appraisal whether: the IIGF had the capacity to produce timely, relevant and reliable financial information on the project activities; the IIGF's accounting system and internal controls met the Bank's fiduciary objectives and standards; and the IIGF had a financial management system that allowed the Bank to monitor compliance with agreed procedures. The Bank recognized that the principal financial management risk was that the IIGF did not have any experience managing Bank or other donor-financed projects. To mitigate the risk, Bank required the IIGF to follow financial management arrangements and procedures detailed in the operations manual, which was being Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) prepared with the Bank's active participation. The principles governing financial management --- covering the allocation of the loan proceeds, funds flows, accounting and reporting, internal control and internal audit, disbursement, and external audit --- were outlined in the appraisal document (PAD, pages 40- 46). The project performance with financial management was rated satisfactory throughout project implementation, including in the last ISR of December 2018. Procurement. The Bank assessed at appraisal the capacity of the IIGF to implement procurement. It did not expect the procurement function to be difficult since there were no goods and services to be procured under the Bank-Supported IIGF Guarantees component of the project and there was only one selection of a consultant firm for the Technical Assistance component of the project. The Bank cited the lack of experience of the IIGF with procurement for a Bank-financed project but added that the IIGF had a qualified procurement officers who could be trained by the Bank. The Bank identified three general risks to procurement in the country --- corruption and collusive practices, procurement delays, and lack of capacity at procuring units, but added (PAD, pages 46-50) that mitigation measures agreed with the government were adequate: (a) the inclusion of corporate governance provisions in the operations manual; (b) the creation of a procurement committee; (c) the training of procurement staff; (d) the drafting of a procurement plan; (e) the formal issuance of a request for expressions of interest for consultant services; (f) the inclusion in the operations manual of a framework for assessing PPP procurement practices; and (g) the assessment by the IIGF of projects proposed to be covered by guarantees using the terms of the operations manual. The project performance with procurement was rated satisfactory throughout project implementation, including in the last ISR of December 2018. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory Satisfactory Moderately Bank Performance Moderately Satisfactory Satisfactory Quality of M&E Modest Modest Quality of ICR --- Substantial 12. Lessons Three lessons are drawn from this operation. Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) Governments should work to strengthen the institutional capacities of government contracting agencies and project sponsors to prepare, screen, and appraise PPP projects. In this project for Indonesia, it was not enough to strengthen the institutional capacity of the IIGF to advance the development and financing of infrastructure PPP projects in Indonesia. The project exposed gaps in the institutional capacities of government contracting agencies, as well as project sponsors, to adhere to international standards for environmental and social safeguards for infrastructure projects. The inability of the contracting agencies to meet the safeguards requirements defined in the operations manual forced the IIGF to withdraw the application of two projects that would have otherwise qualified for Bank support of IIGF guarantees. Governments will need to better define the roles of various institutions in the PPP market to help build a more robust pipeline of infrastructure PPP projects. In this project for Indonesia, the government may want to start reviewing the mandate of, and the services offered by, the IIGF going forward. The ICR (page 30) reports that the IIGF has not been allowed to issue guarantees to PPP projects that have already been awarded to project developers. This not only limits the base from which the IIGF may consider projects eligible for guarantees, it also constraints the ability of some project developers to obtain risk products offered by the IIGF. The government may also want to start re-examining the role of the IIGF relative to state-owned enterprises and state-sponsored infrastructure finance companies operating in Indonesia. The Bank will need to strategically and systematically involve the other Bank Group institutions to help country clients better develop their infrastructure PPP market. In this project for Indonesia, the IFC should be able to help spur sustained private sector interest, both domestic and foreign, in Indonesia's infrastructure PPP program. The Palpa Ring project would have required a guarantee of US$500 million, an amount which was well beyond the size of the Bank's financing. The IIGF is reportedly exploring the participation of MIGA in the guarantee for this project. The Bank itself may want to consider acting as a co-guarantor, together with the IIGF, of infrastructure PPP projects; the arrangement should give the Bank a greater role in helping identify, prepare and appraise projects at the outset of project development rather than at the end stage of guarantee appraisal, hence imparting the IIGF with greater knowledge and experience with international standards for infrastructure PPP projects. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provides an informative record of the history of the project and the rationale for the project design: (a) Box 1 (pages 10-11) traces the evolution of the project, from a partial risk guarantee proposal to an technical assistance and standby credit operation focused on the institutional development of the IIGF; (b) Figure 1 (page 7) articulates the theory of change for the project, which was not presented in the Project Appraisal Document; and, (c) the ICR (page 10) explains the nature of the financing to back IIGF guarantees, including why it was vital to include this component in the operation even if the financing were not drawn upon. Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Indonesia Infrastructure Guarantee Fund (P118916) The assessment of the program's results is evidence-based. The ICR: (a) uses the results indicators defined in the M&E plan to rate the degree of achievement of the project objective (pages 12-13); (b) argues that the number and value of guarantees issued by the IIGF in 2012-18 provide evidence of the institutional capacity of the agency to act as a guarantee institution (pages 11-12), although these were not previously defined as indicators of institutional capacity at the IIGF; and, (c) also cites the three water supply projects and the ten toll road projects that were issued guarantees by the IIGF, all of which were prepared and appraised with the assistance of consultants hired under the project, as evidence of the favorable outcome of technical assistance to the IIGF (page 13), although these were not defined as indicators of the efficacy of technical assistance to the IIGF. The analysis of the project results is candid: (a) the ICR (page 12) concedes that the quantitative targets for the results indicators may have been set too low; (b) the ICR (page 20) cites binding constraints to the implementation of safeguards for projects --- the lack of awareness of project sponsors of safeguards standards, the lack of willingness by government contracting agencies to address safeguards issues prior to a PPP tender, and the limited experience of the IIGF for safeguards assessment; and, (c) Box 3 (pages 17-18) explains that, apart from the inability of the contracting agencies to meet safeguards requirements for two projects submitted by the IIGF to the Bank, there were other reasons for the inability of the IIGF to use the Bank-Supported IIGF Guarantees component of the project, including the thin pipeline of infrastructure PPP projects in the country. a. Quality of ICR Rating Substantial Page 16 of 16