Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004856 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF‐12678, H7420, D1050 ON A GRANT IN THE AMOUNT OF SDR 25.64 MILLION (US$ 35.64 MILLION EQUIVALENT) TO THE The Republic of Kiribati FOR THE Pacific Aviation Investment Program – Kiribati Aviation Investment Project December 19, 2019 Transport Global Practice East Asia And Pacific Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 28, 2019) Currency Unit = AU$ AU$ 0.7013 = US$ 1 US$ 0.7193 = SDR 1 FISCAL YEAR January 1 ‐ December 31 Regional Vice President: Victoria Kwakwa Country Director: Michel Kerf Regional Director: Ranjit J. Lamech Practice Manager: Almud Weitz Task Team Leader(s): Pierre Graftieaux ICR Main Contributor: Kristin Panier ABBREVIATIONS AND ACRONYMS ADS‐B Automatic Dependent Surveillance‐Broadcast AF Additional Financing ARFF Aircraft Rescue and Fire Fighting ASK Airport Services Kiribati AWOS Automatic Weather Observation Station CAA Civil Aviation Authority CAAK Civil Aviation Authority Kiribati CXI Cassidy International Airport in Kiritimati CAS Country Assistance Strategy CPF Country Partnership Framework EIRR Economic Internal Rate of Return EMP Environmental Management Plan ESMP Environmental and Social Management Plan ETOPS Extended Twin Engine Operations GoA Government of Australia GoK Government of Kiribati GRM Grievance Redress Mechanism IATA International Air Transport Association ICAO International Civil Aviation Organization ICR Implementation Completion and Results Report IDA International Development Association ISR Implementation Status and Results Report KAIP Kiribati Aviation Investment Project KAP II Kiribati Adaptation Program Phase II KAP III Kiribati Adaptation Program Phase III KDP Kiribati Development Plan KFSU Kiribati Fiduciary Services Unit KV20 Kiribati 20‐year Vision (2016‐2036) M&E Monitoring and Evaluation MICTTD Ministry of Information, Communications, Transport and Tourism Development MFED Ministry of Finance and Economic Development MOU Memorandum of Understanding NZ‐MFAT New Zealand Ministry of Foreign Affairs and Trade NGO Non‐Governmental Organization NPV Net Present Value OHS Occupational Health and Safety PAIP Pacific Aviation Investment Program PDO Project Development Objective PICs Pacific Island Countries PMU Project Management Unit PPIAF Public‐Private Infrastructure Advisory Facility PST Project Support Team PRIF‐TF Pacific Region Infrastructure Facility Multi‐donor Trust Fund RAP Resettlement Action Plan RPF Resettlement Policy Framework TFSU PAIP Technical and Fiduciary Services Unit TRW Bonriki International Airport in Tarawa WB World Bank WB ‐ RPF World Bank Regional Partnership Framework TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) ..................................... 13 II. OUTCOME .................................................................................................................... 14 A. RELEVANCE OF PDOs ............................................................................................................ 14 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 15 C. EFFICIENCY ........................................................................................................................... 20 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 21 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 21 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 23 A. KEY FACTORS DURING PREPARATION ................................................................................... 23 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 24 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 25 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 25 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 26 C. BANK PERFORMANCE ........................................................................................................... 27 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 28 V. LESSONS AND RECOMMENDATIONS ............................................................................. 28 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 31 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 39 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 41 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 42 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 44 ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P128938 Pacific Aviation Investment ‐ Kiribati Country Financing Instrument Kiribati Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Ministry of Information, Communications, Transport, and The Republic of Kiribati Tourism Development, Ministry of Information, Communications, Transport and Tourism Development Project Development Objective (PDO) Original PDO The project development objective is to improve operational safety and oversight of international air transport infrastructure. Revised PDO The project development objective is to improve operational safety and oversight of international air transport and associatedinfrastructure. Page 1 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 22,910,000 22,910,000 20,611,260 IDA‐H7420 5,630,000 5,630,000 5,475,044 TF‐12678 7,100,000 7,100,000 7,237,305 IDA‐D1050 Total 35,640,000 35,640,000 33,323,609 Non-World Bank Financing 0 0 0 Borrower/Recipient 3,540,000 6,260,000 6,260,000 Total 3,540,000 6,260,000 6,260,000 Total Project Cost 39,180,000 41,900,000 39,583,609 KEY DATES FIN_TABLE_DAT Approval Effectiveness MTR Review Original Closing Actual Closing 13‐Dec‐2011 15‐Feb‐2012 06‐Nov‐2014 31‐Dec‐2016 30‐Jun‐2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 16‐Feb‐2016 6.12 Additional Financing Change in Project Development Objectives Change in Loan Closing Date(s) 14‐Sep‐2018 22.48 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Satisfactory Modest Page 2 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 17‐Mar‐2012 Satisfactory Satisfactory 0 02 06‐Dec‐2012 Moderately Satisfactory Moderately Satisfactory .74 03 23‐May‐2013 Moderately Satisfactory Moderately Satisfactory 1.00 04 30‐Dec‐2013 Moderately Satisfactory Moderately Unsatisfactory 2.42 05 29‐Jun‐2014 Moderately Satisfactory Moderately Satisfactory 2.76 06 22‐Dec‐2014 Moderately Satisfactory Moderately Satisfactory 3.77 07 22‐Jun‐2015 Moderately Satisfactory Moderately Satisfactory 5.11 08 23‐Dec‐2015 Moderately Satisfactory Moderately Satisfactory 6.12 09 28‐Jun‐2016 Moderately Satisfactory Moderately Satisfactory 7.20 10 22‐Dec‐2016 Satisfactory Satisfactory 9.85 11 18‐Jun‐2017 Satisfactory Moderately Satisfactory 10.48 12 30‐Nov‐2017 Satisfactory Moderately Satisfactory 16.32 13 11‐Jun‐2018 Satisfactory Moderately Satisfactory 19.81 14 19‐Sep‐2018 Satisfactory Moderately Satisfactory 22.48 15 20‐Mar‐2019 Satisfactory Moderately Satisfactory 28.44 16 01‐Jun‐2019 Moderately Satisfactory Moderately Satisfactory 29.94 SECTORS AND THEMES Sectors Major Sector/Sector (%) Transportation 100 Public Administration ‐ Transportation 15 Aviation 85 Page 3 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Economic Policy 20 Trade 20 Trade Facilitation 20 Private Sector Development 20 Regional Integration 20 Finance 5 Finance for Development 5 Disaster Risk Finance 5 Public Sector Management 20 Public Administration 20 Administrative and Civil Service Reform 20 Social Development and Protection 100 Fragility, Conflict and Violence 100 Urban and Rural Development 15 Disaster Risk Management 15 Disaster Response and Recovery 5 Disaster Risk Reduction 5 Disaster Preparedness 5 Environment and Natural Resource Management 20 Climate change 20 Mitigation 20 ADM STAFF Role At Approval At ICR Regional Vice President: James W. Adams Victoria Kwakwa Country Director: Ferid Belhaj Michel Kerf Director: John A. Roome Ranjit J. Lamech Page 4 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) Practice Manager: Charles M. Feinstein Almud Weitz Task Team Leader(s): Christopher R. Bennett Pierre Graftieaux ICR Contributing Author: Kristin Panier Page 5 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. Country context. The Republic of Kiribati (population approximately 116,000) is one of the smallest, most remote, geographically dispersed, and climate change‐vulnerable countries in the world. Kiribati comprises 32 low‐ lying atoll islands and one raised limestone island with a total land area of 726 km 2, located in three main island groups (Gilbert, Line, and Phoenix) scattered over 3.5 million km 2 of the central and western Pacific Ocean. 2. Kiribati’s remoteness and small islands separated by vast oceans causes significant economic challenges. Its geography raises the cost of public service delivery ‐‐ leading to an infrastructure gap ‐‐ and limits opportunities for private sector development and diversification (with principal exports limited to coconut products and marine products). At appraisal, the country had one of the lowest per capita income in the region (gross national income per capita of US$2,090) and one of the highest rates of poverty with 22 percent of the population estimated to be in extreme poverty and a further 66 percent of the population estimated at risk of falling into extreme poverty. 1 3. Sector context at appraisal. The aviation sector provides vital national, regional and international connectivity. Air services are essential for the import and export of goods, business travel, and a prerequisite for tourism development. Shipping plays an important role in goods transport, but given long distances, is often not a viable alternative for air passenger transport, the export of high value, time sensitive commodities, or importing of emergency supplies. A reliable network of air links, within and between island countries and to major hubs such as Australia, Fiji and New Zealand is essential to the viability of the Pacific Island Countries (PICs) from social, humanitarian, political and economic perspectives. At the time of approval, the project was highly relevant to and consistent with the objectives laid out in the prevailing Country Assistance Strategy (CAS) for Kiribati.2 The project aligned with the Government of Kiribati’s (GoK) strategic objective of achieving sustainable growth through tourism development and the World Bank’s objective of enhancing the provision of public services through improved safe and secure air transport. 4. At appraisal, International Civil Aviation Organization (ICAO) surveys had identified many safety and security deficiencies at Tarawa’s Bonriki (TRW) and Kiritimati Island’s Cassidy (CXI) international airports, the two key airports of the Kiribati territory serving domestic and international flights. The runways were in increasingly poor condition and navigational aids were basic or missing. Due to a lack of appropriate safety and security protocols, international airlines incurred considerable risk and raised serious safety concerns over the condition of runways and aeronautical assets. In some cases, flights were suspended causing significant economic losses, political and social costs. For example, from September 2008 to June 2010 jet services to CXI ceased due to the state of the runway. The weekly Honolulu‐CXI service by a B737 was replaced by a 19‐seat 1960s Gulfstream G1 turboprop. In 2009 air traffic services 1 Pacific Regional Partnership Framework Pacific Islands ‐ Regional partnership framework: FY17‐FY21. Washington, D.C.: World Bank Group. 2017. 2 World Bank. 2011. Kiribati ‐ Country assistance strategy for the period FY2011‐2014 (English). Washington, DC: World Bank. Page 6 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) were suspended for two weeks at TRW, effectively isolating the country. The consequence was a loss of connectivity and a complete halt to the development of tourism. 5. Higher-level objectives to which the Project contributed. Figure 1 Bonriki Airport at Appraisal The Kiribati Aviation Investment Project (KAIP) was guided by GoK’s priorities articulated in the Kiribati Development Plan 2008–11, and the Bank’s broader Pacific Regional Strategy (Regional Engagement Framework FY2006‐2009 for Pacific Islands) and the Kiribati Country Assistance Strategy (CAS) FY2011‐2014. These documents identified the needs and priorities for Kiribati to “mitigate geographic isolation and expand economic opportunities, including core infrastructure – airports and telecoms – to help connect Kiribati to the world.”3 Bonriki – International Airport 6. KAIP is part of the Pacific Aviation Investment Program (PAIP)4, a regional, horizontal Adaptable Program Loan that consists of a series of projects designed to ensure that critical aviation infrastructure meets operational safety requirements, as well as to strengthen regulatory compliance of international air transport of the participating Pacific Island Countries. PAIP supports the focus of the World Bank‘s Regional Engagement Framework Terminal and runway for Pacific Islands on mitigating the effects of economic isolation through, among others, reduced barriers to trade and investment, promotion of tourism, human resource Source: World Bank task team development and mobility and improved environmental management. 3CAS 2011‐14, p. 5. 4Phase I of PAIP included projects in the Republic of Kiribati, the Kingdom of Tonga and Tuvalu in 2011. Samoa entered the program in March 2014 as Phase 2, the Republic of Vanuatu in May 2015 as Phase 3, and Solomon Islands in March 2018 as Phase 4. Page 7 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) Figure 2 Pacific Aviation Investment Program (PAIP) 7. Most Pacific island countries are small with limited human capacity and no experience in implementing major infrastructure projects. As these countries faced similar challenges in their aviation sectors, a regional approach was adopted for PAIP to provide the impetus for harmonization of aviation policy and standards, as well as safety and security infrastructure with clear benefits to participating countries and the region as a whole. As a regional program, PAIP was able to leverage significant regional IDA. Through PAIP, the World Bank, other donor partners, and participating PICs invest over US$300 million between 2012 and 2022 to improve the infrastructure at twelve airports in six states across the region (see Figure 2). For the successful delivery of the program, the PAIP Technical and Fiduciary Services Unit (TFSU) was established to act as the central team leading all aspects of procurement and overall PAIP implementation supported by Project Support Teams (PST). The overall structure of the management of PAIP is in figure 3 below. Page 8 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) Figure 3 PAIP Implementation Organigram Page 9 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) Theory of Change (Results Chain) 8. As shown below, the underlying logic of the proposed project was clear. The key underlying assumptions for Component A were: (i) timely availability of technical studies, (ii) timely deployment of contractors and materials, and (iii) high quality of works. For Component B, the key assumption was strong political commitment along with the capacity to enforce and adopt proposed recommendations, aviation regulations and manuals. For Component C, the key assumption was the availability of staff to train for key aviation functions. For Component D, the key assumption was that the PST with the support of TFSU would have appropriate capacity to complete the functions required of such an aviation project (e.g. procurement, financial management, administration, decision making, etc.). Figure 4 Causal Chain Project Activities Outputs Outcome Impact  Civil works contract  Physical completion  Improved security,  Safer and more for air transport of infrastructure safety and efficient air infrastructure works (runway, regulatory oversight transport services  Technical assistance terminals, fence)  Strengthened  Increased access to to strengthen and installation of capacity of sector critical services institutional equipment in line institutions  Increased economic capacity of air with ICAO safety opportunities transport requirements  Increased regional stakeholders  Completion of integration technical studies covering pertinent sector reforms and implementation of recommendations  Completion of training activities of operators and regulators Project Development Objectives (PDOs) 9. The original PDO as stated in both the Project Appraisal Document (PAD) and the Financing Agreement was to improve operational safety and oversight of the international air transport infrastructure. Key Expected Outcomes and Outcome Indicators 10. The PDO was measured through four indicators: (i) regulatory certification of safety and security at project airports; (ii) state requirements for safety and security reaches global ICAO average; (iii) modernization of air traffic management; and, (iv) implementation of a regional safety and security levy for departing international passengers. Page 10 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) PDO indicator 1 was revised for clarification from “regulatory certification of safety and security at project airports” to “ICAO certification of safety and security at project airports”. Please refer to annex 1 (Results Framework) for more details on outcomes and indicators. Components 11. The project had four components. At approval, the total project cost was estimated at US$ 26.72 million including contingencies, of which US$22.91 million would be financed by IDA, US$0.28 million would be financed by the Government of Australia through the Pacific Region Infrastructure Facility and approximately US$3.54 million by the Government of Kiribati. See annex 3 for detailed (estimated and actual) project costs by component. No components were revised during project implementation. 12. Component A: International Airport Infrastructure Investments (US$17.16 million including contingencies). This component invested in the aviation infrastructure at CXI and TRW international airports to enable them to meet the ICAO safety and security standards while preserving and extending the service life of existing airport assets. It contributed to regional safety and security through the introduction of improved regional navigation and communication technologies. The focus was on improving safety and security as well as operating efficiency. The project investments complemented parallel financing provided by NZ‐MFAT grant (NZ$ 17 million) for the resurfacing of the CXI runway and by a Taiwan, China loan (AU$20.2 million [US$18.1 million at loan signing]) for resurfacing the TRW runway. The original activities included:  Navigation Aid Improvements: The implementation of: (i) ADS‐B at CXI and TRW; and, (ii) Non‐directional Beacon (NDB) at CXI.5  Weather Monitoring: Automatic Weather Observation Stations for CXI and TRW.  Terminal Building Improvements: A new pre‐fabricated terminal building for CXI and major upgrade to the existing TRW terminal.  Security Fence: A security fence for TRW.  Fire Safety: Provision of two fire tenders for TRW and two fire tenders for CXI, and equipment for firemen at CXI and TRW (jackets, oxygen, etc.).  Airfield Maintenance Equipment: Equipment for grass cutting and other basic maintenance at CXI and TRW.  Security Improvements: Improved security screening equipment for CXI and TRW.  Upgrading of Runway Lighting: Approach light improvements at TRW. The existing CXI lighting and cabling was replaced with low power consumption LED fixtures, reducing the energy demand by 90%. 5 ADS‐B is a modern technology which is part of ICAO‘s strategic plan for the Pacific. ADS‐B is a satellite‐based technology for aircraft monitoring. It enhances safety by making an aircraft visible, in real time, to air traffic control and to other appropriately equipped ADS‐B aircraft with position and velocity data transmitted every second. ADS‐B is a cost‐efficient technology: PAIP has estimated that ADS‐B is about 10% the cost of installing radar. It allows for more efficient aircraft operations, as well as providing continuous location data which is of value in case of emergency. A non‐directional beacon (NDB) is a ground‐based, low frequency radio transmitter used as an instrument approach for airports and offshore platforms. Page 11 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938)  Air Traffic Control Equipment: Provision of ceilometers (a device that determines the height of a cloud ceiling or cloud base), signal lamps, radios, backup generator, along with the necessary communication and surveillance equipment for CXI and TRW.  Obstacle Limitation Survey: A WGS84 survey at CXI to identify obstacles.  Fire Tender Vehicle Shelter: A simple shelter at CXI for housing the fire tenders.  Maintenance Equipment Building: A simple shelter at CXI for housing maintenance equipment.  Improved Power Supply: Enhanced power supply for CXI and the surrounding village.  VSAT: Provision of the a very small aperture terminal (VSAT) secure regional satellite communications system to CXI and TRW.6  Seawall: Completion of the seawall at TRW that was partially constructed under the Kiribati Second Climate Adaptation Project (KAP II).  Resurfacing of the CXI‐London road: Provision of a new surface and minor safety improvements to the main road on Kiritimati from the airport to London.  Consulting Services: The Design and Supervision Consultants for: (i) the CXI London road; and, (ii) the aviation investments. The latter included the resurfacing of the TRW runway to be financed by Taiwan, China. 13. Component B: Aviation Sector Reform and Training (US$1.57 million including contingencies). This component supported sector reform both regionally and within each participating country. At appraisal technical assistance activities to the Ministry of Information, Communications, Transport and Tourism Development (MICTTD), included: (i) support to MICTTD to strengthen its capabilities with aviation sector management, policy, safety and security oversight; (ii) assistance to GoK with implementing aviation regulations, including development of civil aviation technical regulations and manuals; (iii) preparation and adoption of an Air Transport Master Plan (including a policy road map for separating the roles of regulator and airport operator) that addresses measures for airport security strengthening (through penal code amendments and public awareness campaign), projections for air traffic demand, including over‐flights, identifying sources of revenue, public service obligations and gap financing, as well as identification and prioritization of domestic airstrip needs; (iv) a training program, including seconding of staff for industry experience; (v) a baseline audit of the project airport safety and security and review progress in the implementation of the ICAO Corrective Action Plan by the Recipient; and, (vi) ongoing safety and security oversight by Pacific Aviation Safety Office (PASO)7 financed by the GoA through PRIF. 14. Component C: Strengthening airport operations and management capacity (US$2.59 million including contingencies). This component assisted GoK to manage and operate their airports, given domestic capacity constraints. At appraisal, activities included: (i) support to MICTTD to establish Kiribati Airports Management and Operations (KAMO) for the management and operation of Kiribati‘s airports, separately from the regulator; (ii) technical assistance to operate KAMO in a way that would allow airport operations to be run on a more commercial and efficient basis, with regular performance monitoring; and, (iii) due to the acute lack of local specialized capacity to operate international airports capable of meeting ICAO safety and security standards, KAMO would initially contract management expertise and onsite training capacity through a three‐year management contract covering 6 VSAT is a regional aviation safety and security network to enable regional civil aviation authorities and air transport organizations to communicate essential safety and security communications in a secure and timely manner. Satellite ground stations were installed at the airports beside the control tower. The ground stations are linked by satellite in a full mesh closed network, with ground equipment housed within each control tower. The network hub is managed in Auckland, New Zealand. VSAT installations allowed for this service delivery, which is a regional public good since it connects all Pacific airports that have this capability. 7 The Pacific Aviation Safety Office (PASO) is an international organization providing regulatory aviation safety and security service across the Pacific Region. Page 12 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) the operations of CXI and TRW. The contract included local staff development and preparation of a transition plan in the last year of the contract. 15. Component D: Project Support (US$1.92 million including contingencies). This component financed the support required by the various parties involved. At appraisal, activities included: (i) incremental operating costs to implement the project; (ii) the costs of the PST; (iii) financial audits; (iv) the annual subscription costs of the VSAT communications system; and, (v) the costs of PAIP TFSU for procurement, financial management, contract management, safeguards and overall technical support.8 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) Revised PDOs and Outcome Targets 16. The PDO was amended through level 1 restructuring in February 2016. The change did not affect the substance of the objective. The restructuring included specific reference to “associated infrastructure” for consistency with other PAIP projects and in order to accommodate the road works between London and CXI on Kiritimati Island. The amended PDO was to “to improve operational safety and oversight of international air transport and associated infrastructure”. Revised PDO Indicators 17. PDO indicators were not revised and targets for both PDO level and intermediate results indicators were extended to align with the new project closing date.9 Revised Components 18. There were no changes in components. Other Changes 19. The original IDA Grant of US$22.91 million for KAIP was approved by the Bank on December 13, 2011 and became effective on February 15, 2012. An additional grant and level 1 restructuring were approved in February 2016, which included an IDA Grant of US$1.2 million from the national IDA‐17 allocation and US$5.9 million from the regional IDA allocation. Counterpart financing of approximately US$1.88 million was provided by GoK in the form of foregone taxes. The Additional Financing addressed a number of funding shortfalls in the parent project and scaled up project activities such as the improved design of the seawall, improved terminal designs, three additional Aircraft Rescue Fire Fighting Vehicles. The same restructuring extended the closing date by two years, until December 31, 2018 to allow a two‐year airport management contract to improve the operational efficiency and financial sustainability of Kiribati’s two international airports. A level 2 restructuring was approved in September 2018 to extend the closing date to June 30, 2019, to allow for the full completion of all activities financed by the project and to ensure the achievement of the PDO. 8 The actual TFSU costs were apportioned to the individual countries in the PAIP program on a pro‐rated basis in accordance with the size of the project financing. This meant that the costs to the initial countries decreased as additional countries entered the program. 9 First extension to December 2018, and another to June 2019. Page 13 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) Rationale for Changes and Their Implication on the Original Theory of Change 20. During the first 3 years of implementation the project experienced a number of shortfalls in its activities due to (i) increased regulatory and engineering requirements, (ii) rapid deterioration of existing equipment, (iii) increased project implementation cost, (iv) limited institutional capacity, and (v) location premium due to country’s remoteness. While there were no changes to the components, the costs for all but one component increased. There were no implications to the Theory of Change. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 21. Aviation plays a vital economic and social role in Kiribati’s development. The two international airports, TRW and CXI, provide critical access to health care, education and import and export markets in the region. In addition, the country’s susceptibility to extreme weather events, in particular storm surges, make airports a vital entry point for relief goods. The project is therefore aligned with the World Bank Group’s strategic twin goals of reducing extreme poverty and boosting shared prosperity. The rehabilitation of airports has fostered regional integration to expand economic opportunities and help connect Kiribati to the world. The World Bank Group brought significant policy and technical expertise in the air transport sector and emphasized institutional strengthening to complement infrastructure investments to help ensure the sustainability of the sector. Improved connective infrastructure continues to be highly relevant for achieving the objectives of the subsequent and current World Bank Regional Partnership Framework (WB‐RPF) FY17‐FY21 – as reflected in objective 4.2. “increased access to basic services and improved connective infrastructure”. This strategic program for nine PICs, including Kiribati, is structured around four focus areas, namely: (i) fully exploiting the available economic opportunities; (ii) enhancing access to employment opportunities; (iii) protecting incomes and livelihoods; and (iv) strengthening the enablers of growth and opportunities (macro‐economic management, infrastructure and addressing knowledge gaps). The WB‐RPF highlights that lack of investments in public infrastructure continue to be one of the key impediments to growth. Moreover, the economic geography of PICs, characterized by internal dispersion and remoteness, is recognized as major constraint to ending absolute poverty and increasing shared prosperity. 22. Rating. The overall relevance of the project is rated High. Air transport and tourism have been and continue to be identified as priority areas in GoK’s development plan and strategy. Kiribati relies on aviation for its high value fresh exports (such as tuna, clams), medical evacuations and emergency relief in case of natural disasters. Kiribati also has a large diaspora living in NZ and Australia that frequently travels back and forth. Alternative options by sea are often unreliable or non‐existent. The PDO is aligned with the objectives of the RPF (improved connective infrastructure). KAIP substantially improved transport connectivity. Moreover, it promoted donor coordination and contributed to institutional capacity building. By improving aviation safety and reducing constraints on economic activity, KAIP is an enabler of economic growth. Page 14 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 23. Although the PDO was amended, a ‘split evaluation’ was deemed unnecessary, as the revision of the PDO statement was not substantive and did not change the key outcomes. At the time of additional financing and level 1 restructuring in 2016, US$5.83 million of the original IDA grant and US$179,000 of the PRIF grant had been disbursed, amounting to a disbursement rate of 26%. The evaluation of efficacy is undertaken according to the revised PDO and Results Framework. As described in the section above, the PDO was to improve operational safety and oversight of the international air transport and associated infrastructure. It was measured through four indicators: (i) ICAO certification of safety and security at project airports; (ii) state requirements for safety and security reaches global ICAO average; (iii) modernization of air traffic management; and, (iv) implementation of a regional safety and security levy for departing international passengers. 24. The project’s objective to improve operational safety and oversight of the international air transport and associated infrastructure was substantially achieved. Despite initial setbacks, the overall implementation of the project components that contributed to the objective were completed before the revised project closing date, but with a two‐year delay when compared to the appraisal project completion date. Through the efforts of the project, Kiribati has improved the safety and security of critical airport infrastructure and built capacity within MICTTD to provide more effective regulatory oversight. 25. Air traffic growth over the project cycle indicates improved confidence of air operators in the safety and security of the airports. Without the project, the country was at risk of seeing air services to the two airports suspended within five years due to safety concerns. At appraisal, TRW had two international flights per week. International flights increased to 4‐5 per week during construction and to 6 per week at project completion. Both airports have cumulated a total of 167,910 passengers for the year 2018, which represents a total growth of 38% compared to 2017. Both airports show the ongoing trend of domestic travel surge which represents a 50% market growth, compared to 20% for international travel. TRW has accommodated a growth in passenger figures of 37% with a total of 148,617 passengers for the year 2018 from 108,430 over 2017. TRW has a larger ratio of domestic travelers representing 2 times the international travelers. CXI has accommodated a growth in passenger figures of 30% with a total of 17,074 passengers for the year 2018 from 13,132 over 2017. CXI has a larger ratio of international travelers representing 2.5 times the domestic travelers.10 Annual passenger data for TRW and CXI were unavailable from the time of appraisal, however, an analysis of 10 months data from 2009 suggests annual passenger numbers of about 42,000 for TRW and 2,500 for CXI. Hence, passenger numbers show an overall percentage increase of 254% for TRW and 583% for CXI confirming the importance of the project to Kiribati’s aviation sector. Page 15 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) (i) ICAO certification of safety and security at project airports 26. Safety and security have been improved through a combination of investments and capacity building such as training of regulators and operators on meeting the necessary safety standards. The project achieved its goal to address and correct safety and security deficiencies at CXI and TRW that were identified through ICAO survey at appraisal. With regard to infrastructure, the project procured security screening and supporting equipment. Terminal Figure 5 Bonriki Airport at Completion Figure 5 Cassidy Airport at Completion Bonriki – International Airport (TRW) Cassidy – International Airport (CXI) Waiting Check in area area Security check New fire trucks Source: World Bank ICR mission, August 2019 Source: World Bank ICR mission, April 2019 improvements support greater separation and controls of arriving and departing passengers, as well as greater security controls for terminal access and management (figure 5). The improved facilities also better protect the 10 Egis Completion Report 2018. Page 16 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) technical equipment from the harsh climate. Terminal and backup power supply upgrades help with the continuity of security services. Both international terminals at TRW and CXI have been upgraded in line with ICAO standards. 11 At TRW, the seawall protecting the runway was completed in June 2019. Navigation and safety aids are fully operational. The target to achieve Fire Safety Standards (Category 6) at CXI and TRW was fully achieved. Four Aircraft Rescue Fire Fighting Vehicles for TRW and CXI were supplied and commissioned in July and Sept 2017. TRW’s airfield perimeter fencing and runway works were completed, and a runway end safety area (RESA) constructed. The rehabilitation of the road CXI‐London was included in the project components during project appraisal of the parent project without a separate indicator. While the PDO was amended to reflect this activity in “associated infrastructure”, a separate indicator was not included in the results framework during restructuring. The road rehabilitation was eventually cancelled due to funding shortfall and change in government priorities and support. 27. Separation of airport operations from civil aviation regulation has been achieved. ICAO requires regulators to have a level of independence from operators, to ensure that safety decisions are not compromised by economic or other interests. The project financed an Air Transport Master Plan (for infrastructure) and an Aviation Sector Strategy (for operations) as well as aviation advisors and airport management consultants. The Air Transport Master Plan was adopted in 2014 and GoK implemented key recommendations. The project contributed to implement key sector reforms, such as the separation of operational and regulatory function within the aviation sector. The cabinet proposal for the separation of regulatory and operational functions was approved in 2015. Civil Aviation is now separated into Airport Services and the regulatory side (Directorate of Civil Aviation) as two Divisions under MICTTD to eliminate a conflict of interest in self‐regulating. A strong focus on removing bias and conflict of interest is now key to CAAK. 28. To help strengthen the capacity of CAAK, the project successfully financed technical experts and training activities on functions required to become an effective and competent regulator. Airport certification is undertaken by the national civil aviation authority (CAA) in accordance with ICAO guidelines, with the support of PASO. These include the airport certification handbook, as well as all relevant annexes. Airport certification manuals were prepared by the airport management consultant and submitted to CAAK for review and comments. The project financed a training needs assessment with a focus on both operational and regulatory roles within the sector. The assessment has helped identify the gaps in the current training. Draft certification policies and procedures were developed, and the airport management consultants have carried out trainings to departmental staff and key management personnel on site for certification preparation and improved airport management competency.12 The project financed a legal specialist to assist the ministry in drafting the new Airport Act. Despite substantial progress with regards to the regulatory framework, policies and procedures, CXI and TRW have not yet received certification of the aerodrome (Part 139) or security (Part 140). (ii) State requirements for safety and security reaches global ICAO average 29. ICAO will conduct the first full Universal Safety Oversight Audit Program (USOAP) audit of Kiribati in early 2020 and will give Kiribati an ‘effective implementation’ percentage score. This is Kiribati’s first full audit. The USOAP audit measures the effectiveness of a country’s aviation oversight against the applicable requirements set forth in the country’s civil aviation manuals and procedures in comparison with ICAO Standards and Recommended Practices (SARPs). It measures the ability of a state to administer and oversight its aviation sector (see figure 7). 11 The CXI upgrade was of particular importance as US security regulations were such that failure to improve passenger segregation and security at CXI could have jeopardized their ability for international flights to Honolulu—a key lifeline and source of tourists. 12 The training plan included ACI‐DNA Aerodrome Certification Seminar (June 2017); AVSEC Training, NZ (Sept 2017); Airport In‐country training (Dec 2017), continuous in‐house training, mentoring and upskilling of airport staff from February 2017 until February 2019. Page 17 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) Figure 6 Effective Implementation Score in Pacific Island Countries 30. The PDO indicator was designed to measure the “lack of effective implementation”. The lower the score the better. The baseline value was estimated to be 95% indicating extremely poor compliance. The target was set to be the average world score. Currently the global average effective implementation score is at 68%, hence the lack of effective implementation is at 32% (100 minus 68%, which is the lack of effective implementation). Most Pacific Island Countries (except Fiji and PNG) sit between 5% and 49% of effective implementation. 13 31. The project played a significant role in the development of proper policies, regulatory framework and safety oversight compliant with ICAO SARPs through its institutional strengthening activities, such as the development of Air Transport Master Plan, Sector Strategy and the engagement of a full‐time Civil Aviation Advisor, who was appointed in July 2016. His role was to provide technical support and advice to CAAK. The Ministry re‐engaged the consultant for an additional term until November 2018 to continue to provide required support and advice to CAAK with funding from Government budget. 32. Despite substantial progress, the project has experienced delays in advancing institutional reforms due to the complexities at play. ICAO is looking for the full picture of safety and security oversight within the Kiribati government system. The delays in the procurement of the institutional strengthening activities, such as sector studies, hiring of the Civil Aviation Advisor and airport management contract, which was reduced from originally three to two years, resulted in delays of reforms and trainings and consequently readiness for the USOAP audit. A recent NZ‐funded study analyzed the gaps in the existing Kiribati aviation regulatory oversight and action plan that Kiribati would need to address before being able to respond to the ICAO Protocol Questions as satisfactory. Forty‐two percent Page 18 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) of the ICAO USOAP Protocol Questions had been answered at the time of the ICR mission (August 2019). (iii) Modernization of air traffic management 33. The project fully achieved the modernization of air traffic management. Navigational aids and communications equipment installation commenced in early 2016. At the time of project closing the VSAT communications system and the ADS‐B are operational at both airports and key staff have been trained on its effective use. Aircraft on the domestic registry were equipped with ADS‐B transmitters. Both airports were also provided with VSAT so the regional CAA and air transport organizations can now communicate essential safety and security communications in a secure and timely manner. Satellite ground stations were installed at the airports beside the control tower. The ground stations are linked by satellite in a full mesh closed network, with ground equipment housed within each control tower. (iv) Implementation of a regional safety and security levy for departing international passengers 34. The project successfully achieved the implementation of a regional safety and security levy to fund the maintenance and upgrading of safety and security equipment. A key requirement of participating in the PAIP regional program was the implementation of a passenger levy to provide income to sustain the investments and regulatory oversight. Since 1 July 2014, Kiribati collects AUD5 (USD3.39) from all departing international passengers. The Air Transport Strategic Plan for Kiribati, which was funded under the project, further assessed the development of new revenue sources. The revenue feeds into the Civil Aviation Special Fund with two separate accounts, one for Civil Aviation and the other for Airport Services. The fund is administered by MICTTD. Table 1: Kiribati Civil Aviation Special Fund Revenue Source Allocation to Allocation to CAAK Airport Services Kiribati (ASK) PAIP Safety and Security Levy 60% 40% CAAK Oversight Fees 100% Nil Flight Information Region 20% 80% (FIR) Revenue Navigational Aids Charge 40% 60% Justification of Overall Efficacy Rating 35. The overall efficacy is rated as Substantial. As described above, the project largely met its objectives, with some minor shortcomings. Two of the four PDO indicators have been fully achieved while the remaining two PDO indicators are delayed but are very likely to be achieved. This is an excellent outcome given the challenges of operating in a remote and fragile PIC such as Kiribati. 13Fiji – 63.95, PNG – 63.32%, Solomon Islands ‐ 34.38%, Vanuatu ‐ 49.24%, Tonga ‐ 34.23%, Samoa ‐ 47.24%, Nauru ‐ 22.08%, Cooks ‐ 5.63%, New Zealand ‐ 85.63%. Page 19 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) C. EFFICIENCY Assessment of Efficiency and Rating 36. An ex post economic analysis was conducted on the project, replicating the analysis undertaken during preparation to the extent possible. Estimated costs at appraisal, additional costs at Additional Financing, and actual costs by component are compared in Table 2 below. Component A, B, and D demonstrated higher final costs than the estimated costs at appraisal. Table 2: Estimated vs Actual Costs, by Component PAD AF Total Estimated Estimated Estimated Cost Cost Cost (US$m) (US$m) (US$m) Component A: International Airport Infrastructure Investments 15.89 8.72 24.62 Component B: Aviation Sector Reform and Training 1.45 1.29 2.74 Component C: Strengthening airport operations and management 2.40 0.00 2.40 capacity Component D: Project support 1.78 1.09 2.87 Contingencies 1.72 1.33 3.05 Total 23.24 12.43 32.63 37. The efficiency analysis and impact quantification exercise for the project is conducted for Component A based on a cost‐benefit analysis (CBA) approach, which follows the same methodology as at appraisal stage. The project financed aviation infrastructure at CXI and TRW international airports to ensure that ICAO safety and security standards are met, and that jet services continue without disruption. Without the project, it was expected that the jet services may not be able to continue, and a similar investment would be needed around 2020. Without such disruption, the project achieved the expected efficiency from (i) avoided jet services disruption which would limit the number of incoming passengers and tourists (who contribute to country’s GDP), and (ii) avoided runway repair costs due to annual flooding. The economic internal rate of return (EIRR) at completion is estimated at 20.1% with net present value (NPV) of US$33.96 million. It should be noted that, while the EIRR indicates sufficient economic return on infrastructure investment, the actual return is lower than the originally expected EIRR of 34.0% calculated at appraisal. 38. The project had several aspects in its design and implementation which impacted its efficiency. The project experienced several shortfalls within its activities due to increased regulatory and engineering requirements, rapid deterioration of existing equipment, increased project implementation cost, limited institutional capacity, and location premium due to country’s remoteness. The costs for all but one component (Component C) increased. The project was extended twice to allow for project activities to be completed, leading to additional implementation costs. Additional costs arose from procurement delays and design changes of the TRW and CXI terminals. In addition, the contribution of travel and tourism to GDP estimated by World Travel and Tourism Council has been updated. The figure as well as the growth rate are lower than those observed and estimated during appraisal, resulting in lower estimated benefits. Page 20 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) 39. The Efficiency Rating is Modest. While the project was able to achieve most of its planned outputs, the above factors highlight that the efficiency was mixed, with delays and other factors bringing down the EIRR and increasing costs of several components. D. JUSTIFICATION OF OVERALL OUTCOME RATING 40. The overall outcome is rated Moderately Satisfactory. The project achieved substantial efficacy against the PDO. Its objectives remain highly relevant to the current development priorities of GoK and the World Bank. Efficiency is rated Modest given the results of the ex post economic analysis and cost overruns incurred for the infrastructure activities and delays in implementation. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 41. The project did not explicitly incorporate a gender dimension into its design or implementation, and thus did not include any actions or indicators to specifically target the needs of women. Institutional Strengthening 42. As discussed above, Components B and C were intended to develop the capacity within GoK for aviation sector management, policy, safety and security oversight. The most significant achievements were: (i) the development and adoption of the Air Transport Master Plan and Sector Strategy; (ii) the separation of operations and regulatory functions; (iii) the development of CAAK as a competent civil aviation authority critical for air transport safety oversight; and, (iv) the management contract for the international airports. A summary of the institutional strengthening activities is shown in table 3 below. Table 3: Institutional Strengthening Activities Regional Kiribati Institutional strengthening activities Airport Management Contract * Airport Master Plan/Business Plan * Aviation Sector Strategies * Flight Information Region Harmonization * Negotiating Air Service Agreements * Page 21 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) PASO Safety Oversight * Regional Air Navigation Service Provision * Regional Aviation Infrastructure Sustainability * Regional Aviation Supply * Support Civil Aviation Authorities and Line Ministries * Training * Mobilizing Private Sector Financing None. Poverty Reduction and Shared Prosperity 43. While the project was not explicitly designed to have a distributional impact on the poor, the investments made under the project support GoK’s strategic objective of achieving sustainable GDP growth. The CAS cited the close link between the airports’ rehabilitation, regional integration, and economic opportunities. Air services are essential for the critical access to health care, education and import and export markets in the region, and a prerequisite for tourism development. Improved air transport safety is crucial to maintain connection to the rest of the world. PAIP Program Benefits 44. KAIP was part of the PAIP regional program, which is the first regional transport program in the PICs. The experiences from PAIP provided lessons for other projects which replicated similar regional approaches in the transport post‐disaster and fisheries sectors. When the World Bank released its STEP procurement processing and management system, it was decided to use the PAIP program for pilot testing due to the technical capacity of TFSU. The KAIP project was part of the initial pilot testing which provided key feedback for the final implementation. Starting in 2014, through TFSU the PAIP program implemented ‘Codes of Conduct’ for protecting children, and these were later expanded to include gender‐based violence. KAIP benefitted from these efforts, with codes being signed by individual consultants and contractor employees. This work formed the basis for key elements of the 2018 Good Practice Note on addressing Gender Based Violence. Other Unintended Outcomes and Impacts 45. At baseline, Kiribati relied heavily on the support of New Zealand’s air‐force for search and rescue operations. With newly installed runway lighting and improved airport facilities, the search and rescue operations for missing fishermen can now be undertaken from CXI or TRW at any time in case of emergencies. Page 22 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 46. The background analyses undertaken for the project were sound. Assessments of the aviation sector in the PICs found that many of the countries lack proper policy, regulatory and infrastructure to comply with ICAO requirements, which justified the need for safety and security improvements. The project design reflected the lessons learned in Kiribati. This was the second major infrastructure investment project for Kiribati financed by the World Bank, after the 2010 Kiribati Road Rehabilitation Project (KRRP ‐ P122151). Lessons from KRRP directly influenced several aspects of project design.  Thin Technical Capacity. Kiribati does not have very many professionals to support project implementation—engineers, accountants, project managers, etc. and as new projects were introduced, staff were often ‘poached’ from other projects, creating a cycle of salary inflation and challenges for implementation. To address this: o KAIP led to the establishment of the first central PMU in the Pacific: the Kiribati Fiduciary Services Unit (KFSU), based in the Ministry of Finance. The KFSU model was to focus on the fiduciary aspects of the project, with the accountant supporting multiple projects, and the KFSU manager addressing reporting. o The PAIP TFSU was established to provide a centralized team who would support the local PST with implementation. For example, there were only two technical staff in the Civil Aviation Division, so they were supported by TFSU with specific technical issues. TFSU also handled most procurement with the local PST participating in setting specifications, reviewing documents, and evaluations; and supported KAIP with managing contractual issues with a dedicated international contracts specialist.  Limited Market Interest. Due to its remoteness, companies were often reticent to provide goods to Kiribati. The PAIP model of procuring the same goods for multiple countries, managed by the TFSU, increased the interest in bidding for projects in Kiribati. With the same equipment supplied to multiple airports in several countries, the availability of spares was also improved.  Higher Contingency Factored into Design. To address the remoteness and risk premium of working in Kiribati, KAIP had higher contingency cost factored into project design. KAIP had contingencies of over 14% compared to 9‐10% for other PAIP countries.  Securing Continued Aviation Financing. Due to the budget approach, small but important equipment— for example binoculars for the control tower, anemometers and weather equipment—could not be readily replaced. PAIP thus introduced a ‘Safety and Security Levy’ with hypothecated funding specifically for this purpose, which is now part of the ticket price. 47. Project preparation was swift and was facilitated by GoK’s commitment and continuity of World Bank staff. The Project Concept Review took place on December 15, 2010 and Board approval followed 11 months later in November 2011. The rapid preparation timeline is notable given the scale and novelty of the regional program and the complexities of the sector in each of the participating countries. Replication of project design across the program facilitated the identification of investment needs. Page 23 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) B. KEY FACTORS DURING IMPLEMENTATION 48. Government-requested changes in the scope of activities. The change in Government in 2016 had led to several changes. These included changes in the TRW and CXI previously approved terminal designs. The new Government instructed new changes to the final terminal designs for TRW and CXI with a view to improve the functionality of the two terminals. The Government’s intervention to the terminal works substantially delayed the project with costs relating to additional works and time extension. Additional changes in the scope of activities included the cancellation of the resurfacing of the London‐CXI road on Kiritimati, and changes in the design of the seawall for the coastal protection for the Tarawa runway. 49. Capacity constraints within MICTDD and PST. During the first three years of project implementation, Kiribati required more support than other participating PAIP countries due to slow recruitment of the PST and changes in ministry positions. There had been no key personnel until September 2014. Consequently, the project suffered from implementation delays. Due to the lack of a PST manager and key focal point within MICTDD, there was delayed feedback from the recipient on designs, bidding documents and evaluation reports, which resulted in procurement delays. 50. Lack of continuity of implementation arrangements. Under the project, the KFSU was created in the Ministry of Finance (MOF) to handle financial management, with the support of the TFSU. Due to the severe shortage of trained personnel in Kiribati, it was considered more efficient to have a single team in the MOF working on multiple projects, rather than each project try to do its own financial management. Due to personnel and management issues, the KFSU was not as effective as had been anticipated. The TFSU provided procurement and contract management support, with inputs from MICTTD. At the end of 2014, GoK questioned the quality and cost of TFSU services and requested an evaluation of the implementation arrangements. The Government allowed the TFSU Service Agreement to lapse by holding off signing the required deed of variation of the Service Agreement. This effectively stopped TFSU services for Kiribati and jeopardized the project. In July 2015, a workshop chaired by the World Bank was organized between TFSU and GoK. Through consultations, GoK acknowledged the preliminary conclusions of the review, and accepted that TFSU offered the most efficient implementation arrangement for this project and agreed to continue using their services. A revised version of the Service Agreement was signed by GoK in October and the re‐engagement came into effect on November 1st, 2015. The de‐facto cessation of TFSU services until November 2015 delayed the procurement of major civil works activities and affected management of ongoing contracts. 51. Construction management around continuing flight operations. The terminal building at TRW was constructed in four stages to allow for flight operations. This involved sectioning off areas and providing temporary hoardings and processing areas to facilitate passengers and security. Flight schedules were maintained with no significant disruption. Flight operations increased significantly during the period of construction with the introduction of additional flights (originally two international flights per week, increased to 4‐5 per week after construction had started). Construction activity worked within curfew times and non‐ operational periods at selected interfaces to avoid noise and disruption. The impact of flight service increase and shipping delays stretched the construction to 24 months. Tightly controlled scheduling was required to ensure flight operations did not impact significantly on day‐to‐day construction and that construction activities and the available building areas were kept safe for public use at each stage of construction. Weekly coordination meetings were held with airport management and airlines to forecast and plan for flights, construction activities and review if measures were effective and modified if required. Page 24 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) 52. Substantial remoteness premium. The remoteness of Kiribati, challenging operating environment, and the logistical complexities resulted in a significant remoteness premium. The extent was difficult to assess during project preparation as there were only a limited number of comparable contracts awarded prior to KAIP. It costs about three times as much to build a seawall in Kiribati or Tuvalu (where material such as basalt rocks is not available) compared to Fiji. Shipping and ancillary transport and packing cost is the key driver. Another important difference is the cost to operate plant and equipment in Kiribati along with expatriate supervision. The cost to have an expatriate mobilized in Kiribati is in the order of US$1,000/day including wages, insurance, flights and accommodation. Similarly plant and equipment cost is very high compared to less remote locations – there is effectively no plant or equipment available to hire and so all equipment utilized on remote projects must be purchased and shipped from Australia or NZ. Plants and equipment also have reduced service life and increased maintenance costs as a result of a very hot climate and high salinity close to the ocean in low‐lying atoll islands such as in Kiribati. For example, an excavator in Kiribati has half the service life and at least double the ongoing maintenance cost of the same item in Australia. One ship had an accident and a container with PAIP equipment was lost, resulting in a delay of over six months to replace the equipment; while contractors reported losing two to three months if any incorrect equipment parts were shipped that needed to be returned/exchanged. 53. Implementation support was adequate. The project implementation support was carried out on a regular basis, and the project Task Team closely monitored project progress and proactively addressed problems. The World Bank team provided extended in‐country implementation support during the initial phases of the project. Opportunities for improvement were identified and acted upon, as reflected in the Additional Financing and Level 1 restructuring. Team members participated in regional steering committees on occasions. Video‐ conferencing was used to support continued dialogue. The Country Management Unit facilitated client dialogue and smoothing of challenging situations. There was a low turn‐over of TTLs (two), throughout the project lifetime. Although addressing implementation challenges with project management and procurement delayed actualization of targets in the Results Framework, overall reporting in ISRs and Aide Memoires was candid throughout. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 54. KAIP was part of the regional PAIP program and as such M&E design was mainstreamed within the program. The PDO indicators were established to track progress against the PDO, and all components were associated with at least one intermediate indicator. However, two of the PDO indicators (airport certification and ICAO USOAP audits) were dependent on external agencies outside of the project’s influence. They are not yet achieved at project closing and, hence, do not fully reflect the actual achievements of the project. The ICAO USOAP audit is conducted by ICAO and measures the effectiveness of the aviation oversight against the applicable requirements set forth in the country’s civil aviation manuals and procedures in comparison with ICAO SARPs. A country can be compliant with all SARPs without obtaining a score of 100%. Hence, the project can have achieved everything that is needed in terms of investments and training to ensure the sector’s compliance, but if ICAO does not carry out the audit at the expected date, or if the manuals are not in place or incomplete, then the rating will be low. Furthermore, while the PDO was amended during restructuring to Page 25 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) include “associated infrastructure” for the rehabilitation of the CXI‐London road, no indicator to measure the progress of works was included in the results framework. Later, the CXI‐London road was dropped from the project activities due to a funding gap, shift of priorities, and lack of support from the new government. M&E Implementation 55. MICTTD, with assistance from TFSU, was responsible for reporting on the Results Framework to IDA; monitored and reviewed the technical performance and the oversight of the implementation of the components and sub‐components; and was responsible for collecting and reporting required data for monitoring progress. Inputs from MICTTD experienced frequent delays. M&E Utilization 56. The project’s results framework was used to monitor implementation progress, allowing the project team, TFSU and PST a way to formally discuss progress of outputs. In February 2016, a new intermediate results indicator was added in relation to the seawall to be built at the eastern end of the runway in TRW: length of coastal protection constructed. Justification of Overall Rating of Quality of M&E 57. Based on the assessment above, overall quality of M&E is rated as Modest. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 58. Environmental and social safeguards. The project’s environmental and social safeguards management was satisfactory. The project had an effective Environmental and Social Management Framework (ESMF) and Resettlement Policy Framework (RPF). The project was rated with an Environmental Assessment category of B. All sub‐projects under Components A (those including major civil works) were subjected to the Environmental Management Plan (EMP), dated July 11, 2011, which were generally adhered to during project implementation. Deficiencies detected during implementation were addressed. Some of the safeguards issues that arose included compensation of trees that were cut down, a dwelling that was obstructing the obstacle limitation survey, asbestos to be removed and disposed of appropriately at TRW; and a land claim associated with the TRW terminal. All of these were resolved at the completion of the project. 59. Occupational health and safety aspects were monitored closely by supervision engineers. HIV/AIDS awareness and prevention measures were put in place. All civil works contracts were required to undertake the HIV Awareness Program in accordance with the World Bank’s toolkit based on the “Road to Good Health”. As noted earlier, contractors and consultants signed codes of conduct for preventing gender‐based violence and child abuse. Page 26 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) 60. Financial management. The project maintained sound financial management (FM) arrangements and complied with reporting and accounting requirements. FM reviews highlighted some issues related to the completeness and accuracy of the commitments register as well as inclusion of purpose of payment in description on the payment voucher. These issues have been addressed during implementation. There have been frequent coordination challenges between the KAIP PST and the accountant in the KFSU, which resulted in delays in the preparation of the financing plan. FM has been rated Satisfactory and Moderately Satisfactory in the Implementation Status and Results Reports (ISR) throughout the project cycle. 61. Procurement. PAIP adopted a regional procurement approach for all participating countries to mitigate the high risk of procurement issues in capacity‐constrained countries. PAIP projects used ‘national’ procurement rules for goods, works and services being delivered to an individual country (e.g., the TRW and CXI terminals), and ‘regional’ procurement rules for goods, works and services being delivered to multiple countries, therefore, benefitting from economy of scale (e.g., fire engines delivered to each of the airports in every countries participating in the PAIP program). For the regional procurements, the TFSU helped all participating countries to agree on harmonized specifications, undertook the procurement with separate contracts for each individual country, and lead a Regional Procurement Evaluation Committee with members from each country receiving equipment to undertake joint evaluations and awards. There were no instances of misprocurement in the project and the Procurement Plan was kept up‐to‐date. Procurement ratings varied between Satisfactory, Moderately Satisfactory and Moderately Unsatisfactory reflecting performance adequately. It was rated Satisfactory in the last ISR. C. BANK PERFORMANCE Quality at Entry 62. The project’s strategic relevance, environmental and social development issues, fiduciary aspects, implementation arrangements and risks were satisfactorily addressed at entry. The project’s preparation was overseen on the World Bank side by an experienced team who had worked on projects in similar challenging contexts and managed the preparation of the overall PAIP program. The project design was consistent across the program, allowing for scaling to national contexts. There was uniformity in needs across most of the program. The aviation modernization was consistent with the industry outlook and direction. Quality of Supervision 63. The World Bank team remained very responsive to changes in circumstances during implementation. On average, two supervision missions were conducted per year. Interim supervision missions (four during the last six months) were conducted toward the latter stages of implementation to minimize delay. Additional guidance was also provided during implementation, when needed, through the organization of workshops and additional support missions of project staff and procurement specialists for example. Over the course of the project, the team was able to maintain a candid dialogue with the PST and GoK to address various issues such as the suspension of the TFSU service agreement. In 2016, the World Bank team conducted a project restructuring Page 27 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) and additional financing. The restructuring supported GoK in adjusting the project design within the original scope. Justification of Overall Rating of Bank Performance 64. For the reasons outlined in the previous paragraphs, the overall rating for World Bank performance is Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 65. At project completion, the risk to development outcome is considered Moderate, particularly in view of the lack of clear commitment to sustain the project beyond its implementation period. Ensuring long‐term sustainability requires local capacity, commitment and financial sustainability. The introduction of the safety and security levy, as well as the efforts to improve the efficiency of flight information region (FIR) revenue allocations, provided a good entry point to increase sector revenues. Through a study commissioned by the Public‐Private Infrastructure Advisory Facility (PPIAF) in support of PAIP, long‐term options have been explored for the delivery of Air Traffic Management (ATM) services in the upper airspace over the south and central Pacific which will pave the way for an improved allocation of upper airspace revenues that will promote greater efficiency, better management, increased safety and a more equitable income distribution between countries.14 However, at the time of completion, it is uncertain whether these revenues will be sufficient and used as intended. Under PAIP, an Airport Asset Management Program for participating PICs has been established, including the roll out of an Airport Pavement Management System, now operational in all PAIP countries. Similarly, with the support of PPIAF, a multi‐year, multi‐airport mechanical and electrical assets maintenance contract has been designed, which for now has not secured funding but has raised the interest of other donors in the region. Future support to maintenance is likely required and could be provided again with a regional approach if there is enough demand from at least three Pacific Island Countries around a common approach. V. LESSONS AND RECOMMENDATIONS 66. The project provides a number of lessons which can be categorized under three headings: (i) the project’s regional approach and its impacts (ii) remoteness constraints, and (iii) PDO level indicator choice. i) Lessons from the project’s regional approach and its impacts 67. Regional approaches are useful for addressing major infrastructure developments in small island and fragile states. There are three main benefits from a regional project: (i) They provide additional financial resources over the national IDA allocations (more than 60% of financing from regional IDA for phase 1 countries at appraisal). 14World Bank/ PPIAF 2015: Developing Options For Upper Air Space Management Towards A Regional Air Traffic Management Facility For Pacific Island Countries https://ppiaf.org/documents/3680/download. Page 28 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) (ii) They provide an opportunity to overcome the capacity constraints for implementing a project by working with other countries. PAIP facilitated a regional learning and coordination mechanism. Each participating country attended regional Program Steering Committee (PSC) meetings comprising of at least one high‐level representative from each country and PASO. Member states discussed the status, challenges and lessons from their respective projects including safety and security compliance. This approach facilitated regional learning and coordination among all participating countries. PAIP succeeded in establishing a framework for regional procurement harmonization and technical expertise. Given the technically complex and sizable civil works investments in a capacity‐ constrained country like Kiribati, the regional approach to procurement has proved to be effective. Procurement delays and complications were minimized through the central procurement team. Procurement of the same goods for several countries in one single tender helped to reduce cost and to facilitate future support and maintenance of harmonized equipment. Moreover, TFSU staff has provided hands‐on training to inexperienced staff in real time. Harmonized procedures such as the use of a single Project Operations Manual in all participating countries simplified fiduciary oversight. Developing regional networks between airport operators, including major airlines as stakeholders, is an approach that could be replicated in other parts of the world with small individual states facing similar constraints in terms of limited capacities and resources to sustain aviation safety and security investments. (iii) They allow for accelerated preparation of subsequent projects joining the program as approaches have already been tested. As the first regional project in the PICs, PAIP has been successful in showing that it is an efficient and effective model. Initially, PAIP was only to include Kiribati and Tonga. However, after Tuvalu joined the World Bank they requested support for aviation, and it was very straightforward to integrate them into PAIP. There were no major delays to preparation by doing this. PAIP subsequently grew with the inclusion of PASO, Samoa, Vanuatu and the Solomon Islands, all of which benefitted from the groundwork that was done for earlier projects. 68. Perception challenges with ‘outsourced’ support. While the central approach towards procurement and technical expertise provided substantial benefits, it has been challenging to implement. Government officials of Kiribati perceived TFSU services as foreign, and felt their interests were not properly defended by TFSU. GoK complained about the cost of TFSU services and believed they were unfairly treated compared to other PAIP countries. The perceived disadvantage has hampered project implementation. Continuous consultation and engagement is needed when new implementation arrangements are introduced. 69. Need for support on the ground. When the above‐mentioned perception challenges were most prevalent, the NZ High Commission in Tarawa played a key role in following up with GoK between WB missions to ease tensions and ensure that the dialogue between development partners and MICTTD was uninterrupted and fluid, clearing any misunderstanding at that critical moment for the project. A Tarawa‐based liaison officer, hired in 2016 and initially shared with the Asian Development Bank, then played that important role of close follow up and continuous consultations, which proved key for this project in particular and the WB portfolio in general. Page 29 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) (ii) Lessons from remote and capacity constrained environments 70. Despite the project’s focus and major financing towards institutional strengthening and capacity development, the complexity of the subject has shown that building capacity in isolated states requires more time than anticipated. That includes project and financial management, procurement as well as technical aviation and regulatory expertise. Given these capacity constraints, the five‐year project implementation timeline at appraisal was unrealistic. More realism is required when preparing projects in remote, capacity‐constrained countries, even more so if it is in a new sector. 71. Increased implementation support needed. The number of missions – including longer in‐country support by World Bank consultants – show that for complex projects such as KAIP increased implementation support is critical. World Bank teams need to be adequately resourced and prepared to provide more support during preparation and implementation to meet the needs in countries like Kiribati. 72. Remoteness premiums need to be factored into project financing. At the time of additional financing the project experienced a significant funding shortfall. As detailed above, cost of shipping, equipment, maintenance and expert staff are two to three times higher than in less remote and isolated countries such as Fiji. In isolated small island states, such as Kiribati, higher cost contingencies should be factored into project design at appraisal, and updated continuously based on latest available data. (iii) PDO level indicator selection 73. The PDO indicator of airport certification and ICAO USOAP audit do not fully reflect the actual achievements of the project. The two indicators are difficult to understand and are dependent on external agencies: primarily the CAAK and ICAO, hence the project had no full control over their achievements. As discussed earlier, a country can be compliant with all SARPs without obtaining a score of 100%. The project can have achieved everything that is needed in terms of investments and training to ensure the sector’s compliance, if ICAO does not carry out the audit at the expected date, or if the manuals are not in place or incomplete, then the indicator will not be achieved. Indicators should be chosen that are within a project’s control and thus are able to reflect its full achievements. . Page 30 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Improve operational safety & oversight of int'l air transport and associated infrastructure Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 1. ICAO certification of safety Yes/No N Y N and security at project airports 13‐Dec‐2011 30‐Jun‐2019 30‐Jun‐2019 Comments (achievements against targets): The Civil Aviation Authority of Kiribati (CAAK) is responsible for airport certifications. The likelihood for this PDO indicator to be achieved by early 2020 is very high. CAAK is committed to certify the airports as soon as possible (ICAO target is to certify all international airports by 2020). The aerodrome manuals were developed with the support of the airport management consultant contract, which was funded by the project. The manuals have been updated. Discussions between CAAK and the operator were under way at the time of the second ICR mission in late August 2019. CAAK was awaiting resubmission of the updated manuals from the airport operator. Page 31 of 46 The World Bank Pacific Aviation Investment - Kiribati (P128938) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 2. State requirement for Percentage 95.00 32.00 95.00 safety and security reaches global ICAO average 13‐Dec‐2011 30‐Jun‐2019 30‐Jun‐2019 Comments (achievements against targets): This indicator is measured in percentage of "lack of effective implementation". The ICAO USOAP (Universal Safety Oversight Audit Program) audit measures the effectiveness of a country’s aviation oversight against the applicable requirements set forth in a country’s civil aviation manuals and procedures in comparison with ICAO Standards and Recommended Practices. It measures the ability of a state to administer and oversight its aviation sector. The target was set to be the average world score, currently the global average effective implementation score is at 68%, the inverse is 32%, which is the lack of effective implementation. ICAO will conduct the first full USOAP Audit of Kiribati in early 2020 and will give Kiribati it's first score. NZ is currently funding technical assistance to Kiribati to support CAAK’s readiness for the USOAP audit. An action plan has been established early 2019 and is currently being implemented. As there have not been any previous ICAO audits for Kiribati, the actual achieved score at completion is unknown. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 3. Modernization of air traffic Text no PASNet or ADS‐B PasNet and ADS‐B PASNet and ADS‐B management operational fully operational 13‐Dec‐2011 30‐Jun‐2019 30‐Jun‐2019 Comments (achievements against targets): Target fully achieved. Page 32 of 46 The World Bank Pacific Aviation Investment - Kiribati (P128938) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion 4. Implementation of a Yes/No N Y Y regional safety and security levy for international 13‐Dec‐2011 26‐Feb‐2019 01‐Jul‐2014 passengers Comments (achievements against targets): The project successfully achieved the implementation of a regional safety and security levy to fund the maintenance and upgrading of safety and security equipment. Since 1 July 2014, Kiribati collects AUD5 (USD3.39) from all departing international passengers. The revenue feeds into the Civil Aviation Special Fund with two separate accounts, one for Civil Aviation and the other for Airport Services. A.2 Intermediate Results Indicators Component: International Airport Infrastructure Investments Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Navigation and safety aids Percentage 0.00 80.00 100.00 fully operational 13‐Dec‐2011 26‐Feb‐2019 30‐Jun‐2019 Comments (achievements against targets): Page 33 of 46 The World Bank Pacific Aviation Investment - Kiribati (P128938) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Achievement of Fire Safety Text Category 3 Category 6 Category 6 Standards at Bonriki and Cassidy airports 13‐Dec‐2011 26‐Feb‐2019 26‐Feb‐2019 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Bonriki and Cassidy terminals Percentage 0.00 75.00 100.00 upgraded 13‐Dec‐2011 26‐Feb‐2019 30‐Jun‐2019 Comments (achievements against targets): Component: Aviation Sector Reform and Training Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Successful implementation of Text No training plan There is current plan Training plan agreed training plan under implemented implementation. Page 34 of 46 The World Bank Pacific Aviation Investment - Kiribati (P128938) 13‐Dec‐2011 26‐Feb‐2019 30‐Jun‐2019 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Separation of civil aviation Yes/No N Y Y regulatory functions from operations 13‐Dec‐2011 26‐Feb‐2019 26‐Feb‐2019 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Adoption of an Air Transport Yes/No N Y Y Master Plan 13‐Dec‐2011 26‐Feb‐2019 26‐Feb‐2019 Comments (achievements against targets): Component: Strengthening Airport Operations and Management Capacity Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 35 of 46 The World Bank Pacific Aviation Investment - Kiribati (P128938) Length of coastal protection Meter(m) 0.00 180.00 180.00 constructed 15‐Nov‐2015 30‐Dec‐2018 30‐Jun‐2019 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Award of contract for airport Yes/No N Y Y   management and operations support 13‐Dec‐2011 29‐May‐2018 29‐May‐2018 Comments (achievements against targets): Page 36 of 46 The World Bank Pacific Aviation Investment - Kiribati (P128938) B. KEY OUTPUTS BY COMPONENT Objective/Outcome: improved operational safety and oversight of international air transport infrastructure 1. ICAO certification of safety and security at project airports 2. State requirements for safety and security reaches global ICAO average Outcome Indicators 3. Modernization of air traffic management 4. implementation of a regional safety and security levy for departing international passengers 1. Navigation and safety aids fully operational 2. Achievement of Fire Safety Standards at Bonriki and Cassidy Intermediate Results Indicators Component A ‐ International Airport airports Infrastructure Investments 3. Bonriki and Cassidy terminals upgraded 4. Length of coastal protection constructed 1. Installation of all navigation and safety aids equipment 2. Civil works contracts for terminal buildings Key Outputs Component A 3. Procurement of 4 fire trucks and fire safety trainings 4. Civil works contract for seawall 1. Successful implementation of agreed training plan Intermediate Results Indicators Component B ‐ Aviation Sector 2. Separation of civil aviation regulatory functions from operations Reform and Training 3. Adoption of an Air Transport Master Plan 1. Training needs assessment Key Outputs Component B 2. Continuous training activities according to training plan 3. Master Plan and Sector Strategy 4. Draft policies, manuals and legal documents Page 37 of 46 The World Bank Pacific Aviation Investment - Kiribati (P128938) Intermediate Results Indicators Component C ‐ Strengthening Airport 1. Award of contract for airport management and operations support Operations and Management Capacity 1. Draft airport certification manuals Key Outputs Component C 2. Continuous airport staff mentoring and training 3. Airport management and operations completion report Page 38 of 46 The World Bank Pacific Aviation Investment - Kiribati (P128938) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Christopher R. Bennett Task Team Leader(s) Jinan Shi Procurement Specialist(s) Stephen Paul Hartung Financial Management Specialist Marta Elena Molares‐Halberg Counsel Anil H. Somani Social Specialist Maria Fernanda Miralles Gasparini Social Specialist Supervision/ICR Pierre Graftieaux Task Team Leader(s) Cristiano Costa e Silva Nunes, Zhentu Liu Procurement Specialist(s) Evaron Doris Masih Financial Management Specialist Christopher R. Bennett Team Member Duangrat Laohapakakul Counsel Nicholas John Valentine Environmental Specialist Sam William Johnson Team Member Craig Andrew Clark Social Specialist Joyce Onguglo Social Specialist Caroline Ruth Holo Team Member B. STAFF TIME AND COST Page 39 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 14.719 103,436.99 Total 14.72 103,436.99 Supervision/ICR FY12 9.035 44,180.37 FY13 22.265 123,193.66 FY14 13.710 107,367.32 FY15 22.550 87,661.33 FY16 19.164 79,332.25 FY17 23.092 105,395.19 FY18 13.288 85,516.21 FY19 11.279 90,169.19 FY20 9.923 66,097.04 Total 144.31 788,912.56 Page 40 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) ANNEX 3. PROJECT COST BY COMPONENT Actual Cost at Estimated Cost at Approval Actual Cost as Percentage Components Project Closing (US$M) of Estimated Cost (%) (US$M) International Airport 15.89 24.62 155 Infrastructure Investments Aviation Sector Reform and 1.45 2.74 189 Training Strengthening Airport Operations and Management 2.40 2.40 100 Capacity Project Support 1.78 2.87 161 Contingencies 1.72 Recipient funds 3.54 6.26 Total 26.78 38.89 145 Page 41 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) ANNEX 4. EFFICIENCY ANALYSIS 1. The efficiency analysis and impact quantification exercise for the ICR of the project is based on a cost‐benefit analysis (CBA), which follows the same methodology at appraisal stage. The CBA focuses on Component A – physical infrastructure investment. Because the project implementation took 7 years, the analysis covers the time horizon of 20 years (2014‐2033) to allow time for the alternative scenario (without‐project), assuming the discount rate of 5%. 15 2. The project financed aviation infrastructure at CXI and TRW international airports to ensure ICAO safety and security standards are met, and that jet services continue without disruption. Without the project, it was expected that the jet services would not be able to continue, and similar investment would be needed, by 2020. Without such disruption, the project achieved the expected efficiency from (i) avoided jet services disruption which would limit the number of incoming tourists (which contribute to the country’s GDP), and (ii) avoided runway repair costs due to annual flooding. 3. Key assumptions include: (a) the estimated contribution of travel and tourism to GDP by the World Travel and Tourism Council (WTTC), which is about US$39.2 million or 18% of GDP in 2018, growing at 2.1% per annum; (b) 50% reduction of tourists due to disruption of jet services in 2020; (c) 0.8% impact to tourism due to annual flooding; and, (d) US$50,000 cost of annual repairs due to flooding. 4. The economic internal rate of return (EIRR) at completion is estimated at 20.1% with a net present value (NPV) of US$33.96 million. It should be noted that, while the EIRR indicates sufficient economic return on infrastructure investment, the actual return is lower than expected (EIRR of 34.0% at appraisal). Because the comparison involves counterfactual scenarios, a sensitivity analysis was also conducted for different timing of the potential disruption, as well as the size of disruption (i.e. percentage of demand loss), to test the robustness of the result (see Figure 7 below). Figure 7. Sensitivity analysis conducted for different timing of the potential disruption 25.0% Year of delayed investment 20.0% 15.0% 2020 2021 IRR 10.0% 2022 2023 5.0% 2024 0.0% 0% 10% 20% 30% 40% 50% 60% % of Tourism reduced as the jet service stops 15 Following World Bank’s Guidance Note on Discounting Costs and Benefits in Economic Analysis of World Bank Projects (May 2016) Page 42 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) 5. The project had several aspects in its design and implementation which impacted its efficiency. The project experienced several shortfalls within its activities due to increased regulatory and engineering requirements, rapid deterioration of existing equipment, increased project implementation cost, limited institutional capacity, and location premium due to country’s remoteness. The costs for all but one component (Component C) increased. The project was extended twice to allow for project activities to be completed, leading to additional implementation costs. Additional costs arose from procurement delays and design changes of the TRW terminal. In addition, the contribution of travel and tourism to GDP estimated by World Travel and Tourism Council has been updated. The figure as well as the growth rate are lower than those observed and estimated during appraisal, resulting in lower estimated benefits. 6. Considering the project was able to achieve most of its planned outputs but encountered delays and increased costs which bring down the EIRR, the overall efficiency for this project is rated Modest. Page 43 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS No comments received as of December 18, 2019. Page 44 of 45 The World Bank Pacific Aviation Investment - Kiribati (P128938) ANNEX 6. SUPPORTING DOCUMENTS (IF ANY) IDA, International Finance Corporation (IFC), Country Assistance Strategy for the Republic of Kiribati FY 2011 – 2014 (Report No. 59355 ‐ KI), 2011 IBRD/IDA/IFC/MIGA Regional Partnership Framework for Kiribati, Nauru, the Republic of Marshall Islands, Federated States of Micronesia, Palau, Independent State of Samoa, Kingdom of Tonga, Tuvalu and Vanuatu for the period FY17‐FY21 (Report No. 100997‐EAP), 2017 Government of Kiribati, Kiribati Development Plan 2016–2019, 2016 Government of Kiribati, KIRIBATI 20‐year vision 2016‐2036, 2016 Kiribati Air Transport Master Plan 2014 Kiribati Air Transport Sector Strategy 2014 TFSU Pacific Aviation Investment Program Quarterly Project Report World Bank, Aide Memoires World Bank, Implementation Supervision and Results Reports World Bank, Systematic Country Diagnostic Report No: 102803‐EAP, 2016 Page 45 of 45