Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) Report Number : ICRR0021635 1. Project Data Project ID Project Name P122151 Kiribati Road Rehabilitation Project Country Practice Area(Lead) Additional Financing Kiribati Transport P144099,P154012 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-D0350,IDA-H6450,TF- 30-Jun-2016 43,716,859.23 99624,TF-A2674 Bank Approval Date Closing Date (Actual) 01-Mar-2011 30-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 20,000,000.00 18,856,811.00 Revised Commitment 44,849,014.96 18,856,811.00 Actual 43,716,859.23 18,703,918.19 Prepared by Reviewed by ICR Review Coordinator Group Victoria Alexeeva Kavita Mathur Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives The project development objective was "to improve the condition of South Tarawa’s main road network, and help strengthen road financing and maintenance capacity." (Financing Agreement, page 4; Project Appraisal Document (PAD), page 4). b. Were the project objectives/key associated outcome targets revised during implementation? Page 1 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 24-Feb-2015 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components Component 1: Infrastructure Improvement (cost at appraisal US$36.5 million; revised cost under additional financing (AF) US$54.5 million; actual cost US$71.0 million) consisted of civil works on the South Tarawa road that included: (i) Reconstruction and Rehabilitation of Paved Roads on South Tarawa; (ii) Rehabilitation of Betio Causeway; (iii) Rehabilitation of Paved Roads in Betio; (iv) Sealing of Feeder Roads; (v) Road Safety Improvements; and (vi) Consulting Services. Component 2: Road Sector Reform (cost at appraisal US$1.3 million; revised cost under AF US$0.68 million; actual cost US$0.3 million) component aimed to strengthen the road sector through (i) Land Transport Institutional Review Study; (ii) Micro-Enterprises for Routine Road Maintenance; (iii) Road Safety Action Plan; and (iv) Road Emergency Response Plan. Component 3: Project Support (cost at appraisal US$1.1 million; revised cost under AF US$2.27 million; actual cost US$3.9 million) included activities in support of the Government of Kiribati (GoK) for implementation of the project: (i) Establishment of a Project Management Unit (PMU); (ii) Project- associated incremental operating costs; (iii) A valuation specialist to identify the appropriate compensation rates for trees and other assets affected by the project; (iv) An NGO to monitor implementation of the Resettlement Policy Framework (RPF); (v) Audit of project accounts. Revisions under components (2015) • Under Component 1: (i) Approximately 11 km of water mains were to be replaced during the construction and rehabilitation of the South Tarawa road infrastructure, as after the commencement of works, the contractor began to have widespread issues with unexpected underground services that were not identified on project drawings; (ii) rehabilitation works were removed due to the failure of the armoring of the Betio Causeway, and the need for the Causeway to undergo a major rehabilitation; (iii) changes in specifications for: the Temaiku road to replace an asphaltic concrete pavement for a lower cost surface dressing; the reconstruction treatment of the sealing of unsealed urban feeder roads, the type of surfacing of the Buota road, and the coastal protection investments. Page 2 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) • Under Component 2: an additional technical assistance activity was included - “Updating Road Law and Drafting of Road Safety Legislation”, while two of the appraisal activities were removed from the project, namely: (i) the “Road Emergency Response Plan” which was no longer considered as necessary, and (ii) the “Land Transport Institutional Review Study” which was financed under the Kiribati Aviation Investment Project (KAIP) as part of a larger transport sector study. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Total Project Cost: The total project cost was US$75.14 million that increased from the appraised cost of US$ 38.84 million mainly due to the underestimation of civil works costs (see details below). Financing: The project was originally financed with an IDA grant of US$ 20 million and co-financed through a US$12 million concessional loan of the Asian Development Bank (ADB) and a grant of US$5.79 million provided by the Government of Australia (GOA) through the Pacific Region Infrastructure Facility Multi-Donor Trust Fund (PRIF), managed by the World Bank. An additional IDA grant of US$6.0 million and a PRIF grant of US$7 million were approved in 2015 to primarily cover a financing gap due to the underestimation of civil works costs. As explained by the AF Project Paper (2015), the financing gap between the estimate and award contract arose primarily because: (i) a remoteness cost premium was not adequately factored into the original cost estimates; (ii) there was significant cost inflation in materials between appraisal in January 2011 and bidding in mid- 2012; and (iii) a higher standard of road than originally appraised was found to be necessary to improve road safety, strengthen environmental protection, and mitigate the risk of maintenance delays in future. Currency fluctuations, increased oil prices and transportation costs were further contributors to the increased cost. The ADB provided a total of US$11.4 million of additional financing through two grants. Borrower contribution: The Government of Kiribati (GoK) committed US$ 1.05 million that was supplemented by AF of US$4.64 million and subsequently another US$1.14 million. Dates: The project was approved on March 1, 2011 and became effective on August 25, 2011. The project closing date was extended once by two years at the time of AF from June 30, 2016 to June 30, 2018. In addition to revisions into the components (see above), the Level 2 restructuring also included: (i) revisions of the PDO indicators, and corresponding outcome targets, and (ii) changes to the implementation arrangements to include a central ‘Kiribati Fiduciary Services Unit’ (KFSU) to manage procurement and financial management. 3. Relevance of Objectives Page 3 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) Rationale The Republic of Kiribati is one of the most remote and geographically dispersed countries in the world. It comprises of 33 atolls and reef islands and has a total land area of 726 sq km scattered over an ocean area of 3.5 million sq km. Kiribati is on the harmonized list of fragile states due to its economic geography and high exposure to adverse effects of climate change, such as of sea level rise, storm surge, and coastal erosion. At appraisal, the main road network on South Tarawa, the country’s capital, had suffered extensive damage and was in need of urgent repairs. Excessive pot holes, cracks, and inadequate drainage had been caused primarily by wet weather and exacerbated by a lack of maintenance. Road maintenance capacity was low, funding was constrained, and there was a narrow revenue base from road user charges. The project PDO was relevant to the GoK development priorities. The GoK consistently identified the national road network as an essential element to economic development. The PDO was aligned with the Kiribati Development Plan 2016–2019, which had as one of its objectives to “Improve access to quality climate change resilient infrastructure in urban and rural areas”. On the longer term, the 2016-2036 Kiribati 20-year Vision (KV20) recognizes the significant cross cutting role of infrastructure in providing basic services and creating an enabling environment for the development of the fisheries and tourism sectors. The objectives remained aligned with the WB Regional Partnership Framework (WB – RPF) FY17-FY21 that, among four areas identified for the nine Pacific Island Countries, focused on strengthening the enablers of growth and opportunities (macro-economic management, infrastructure and addressing knowledge gaps). The WB - RPF highlights that lack of investments in public infrastructure continue to be one of the key impediments to growth. Similarly, climate change continues to pose an existential threat, and investments in coastal protection are critical for the continued survival of several of the nine countries, including Kiribati. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective To improve the condition of South Tarawa's main road network. Rationale * A ‘split evaluation’ was deemed unnecessary, as the revisions to the main PDO targets were minor, and as for the rural roads, these were due to the originally erroneous classification. Page 4 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) Theory of change: The project envisioned to improve the condition of the main road network through investments in rehabilitation and upgrade of non-rural and rural roads. Outputs •5.8 km of rural roads were rehabilitated against the original target of 7.2 km and the revised target of 5.8 km. The ICR clarifies that the original revised target was based on incorrectly classified rural roads. • 42.7 km of non-rural roads were rehabilitated as compared to the original target of 34.9 km and the revised target of 32.7 km. Additional works (mostly in Betio) increased the total length of non-rural road rehabilitated by 10 km to 42.70 km. • 56.8 km of footpaths (target 67 km) and 116 speed humps (target 56) were constructed to address road safety measures. The ICR (page 40) clarifies that due to a design change, 10.2 km of footpath were dropped (in some locations, mostly at Temaiku, where there is still single direction footpath) from the target of dual direction footpath. The target for speed humps was set based on design drawings, which didn’t show the locations of all humps. • A Road Safety Action Plan (RSAP) was completed by March 2014, but the implementation of its recommendations had largely been stagnant for four years, including the establishment of the office for road safety. The GoK has implemented only some of the measures, such as the roadworthiness verification by police. • Road safety equipment (safety vests, radar guns and breath testing equipment) was purchased to support police capacity for enforcement. • The ICR (para 29) notes that the technical design also included climate adaptation features and technical specifications accounted for persons with disabilities, by applying ‘universal design’ principles. In addition, an innovative pavement for low-volume roads - geocell concrete pavement - was piloted for the first time in the Pacific region. This was used to seal 7.3 km of pothole-ridden, unsealed feeder roads at a cost 28 percent less than chip seal surfacing and 47 percent less than the cost of asphalt concrete (ICR, para 32). Outcomes •The roads in good and fair condition as a share of total classified roads increased from 18% in 2010 to 90% in 2018, in line with the target (this was a core indicator added in 2015). Page 5 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) • The average travel speed on the section between the St. Anne's and Ananau Causeway increased from 20km/h to 31km/h, which is below the target of 40km. The ICR clarifies that the recorded speed reflects the actual maximum speed limit enforced on the section. •60,936 people (31,737 female) benefit from an improved connectivity under the project (the target was 60,000). • After some delays, the Road Safety Legislation prepared by the project was enacted by parliament into law in November 2017. This allows the Kiribati Police Service (KPS) to better enforce regulations to control dangerous driver behavior such as speeding and drunk driving, which pose significant risks to all road users. While there were no road safety indicators and targets defined at the outcome level, the ICR (para 38) mentions that the modest implementation of mitigating measures for road safety likely contributed to the increase in road accidents and fatalities in 2017 compared to the years immediately prior to the road upgrade. Rating Substantial PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Help strengthen road financing and maintenance capacity. Rationale Theory of change: Through technical assistance, the project aimed to investigate maintenance financing reforms, create microenterprises for routine maintenance, and agree on a plan for reform of road sector management and financing in order to help strengthen road financing and maintenance capacity. Outputs • The ICR (para 35) reports that the project required through an ADB covenant that: “from fiscal year 2013, shall allocate annually at least $1,500 per kilometer for maintenance of sealed roads in South Tarawa and $500 per kilometer for maintenance of unsealed roads, and increase the allocation annually thereafter at the rate of inflation”. The ICR does not state if this covenant was complied with. Page 6 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) • A planned study to identify cost recovery options for the provision of access to and maintenance of roads was not delivered under the project. As explained by the ICR (para 35), the scope was expanded to cover all transport modes and was eventually funded by another World Bank-financed project (Kiribati Aviation Investment Project-KAIP), to reduce the project's cost. The Transport Sector Strategic Development Plan funded under the Kiribati Aviation Investment Project (KAIP) did not specifically identify a financing plan. It recommended two land transport related reforms: (i) the creation of a Transport Coordination and Development Unit; and (ii) the formation of a Road Transport Division within the Ministry of Communications, Transport and Tourism (MCTTD). However, final plans were not developed, and the key reform was yet to be implemented. • Training activities were carried out that helped set up regular maintenance on project roads. • Four reports on procurement, inspection, technical, and managerial aspects as well as manuals on routine maintenance for sub-contractors were delivered by a micro-enterprise specialist. Outcomes The main outcome indicator was to have the reform of road sector management and financing under implementation. As evidenced by the above, final plans for the reform have not been developed, and key reform on financing is yet to be defined and implemented. The ICR (para 35) notes that a policy dialogue and raising awareness on road financing needs and setting up of a road maintenance system led to preparation of a draft Cabinet Paper on potential revenue streams for routine maintenance. Overall, this falls short of the intended result, i.e., the GOK's agreement on plan for the road sector management and financing reform and its implementation. With regard to maintenance practices, 48.6 km of roads were under regular maintenance at project closure, representing 87 percent of the target. This was 7.4 km less than the original 56 km planned to be included under maintenance, due to changes in the targets based on actual needs on the ground. A local contractor was appointed in late 2018 to undertake the routine maintenance works for South Tarawa, with the Ministry of Infrastructure and Sustainable Energy (MISE) responsible for managing the contract. The planned introduction of 'micro-enterprises' for maintenance did not succeed, and the target to have 3 micro- enterprises for routine road maintenance in South Tarawa commercially active for two or more consecutive years was not achieved. The maintenance approach was changed from using “micro-enterprises” to subcontracting. Due to concern from GoK of the higher burden of managing many small groups, the decision was made to instead increase the value of maintenance contracts and make them available to small formal contractors. As noted by the ICR (para 37), the cost of this approach is very high compared to what would typically arise with microenterprises, and given the long-term financial commitments, this modality may not be sustainable. Rating Page 7 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) Modest PHREVDELTBL PHOVRLEFFRATTBL Rationale The project improved the condition of South Tarawa’s main road network to a substantial extent. However, strengthening road financing and maintenance capacity was assessed as modest for the lack of progress on the planned reform of road sector management and financing. The overall efficacy rating is substantial, albeit there are clear moderate shortcomings in terms of the strengthening of road financing. Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Economic analysis An ex-post economic internal rate of return (EIRR) for the roads rehabilitation is estimated at 17.4 percent, which is lower that the EIRR of 29.4 percent estimated at the time of AF in 2015. At appraisal, the project was estimated to have an ERR of 40.3 percent and a net present values (NPV) of US$ 18.49 million (PAD, page 11). When using a 12 percent discount rate, as per the World Bank's economic analysis policy applicable at appraisal in 2011, the project has an NPV of $17.6 million at completion. Considering a 6 percent discount rate as aligned with the latest WB economic analysis guidelines, the NPV is $54.1 million. WB Highway Development and Management Model (HDM-4) was used to compare ex-ante and ex-post EIRRs. The overall economic benefits of the project derived from savings in travel time; and savings in vehicle operating cost, were both due to the improved quality of the roads. The ex-post EIRR is lower due to: (i) changes to the project scope that resulted in different road sections, (ii) a lower traffic growth than forecast at appraisal (except for Betio Link Road), and much higher actual costs of road works. There was also higher growth in heavy truck traffic, especially in Betio, which would accelerate the need for some re-sealing within 8- 10 years (ICR, Annex 4). Operational/ Administrative Efficiency There was a significant cost overrun that necessitated additional financing to ensure the completion of the planned activities. This was due to significantly higher bids compared to the original estimates of road works. As explaned by the AF Project Paper (2015), there were several key reasons for high bids that included: a remoteness cost premium due to high transportation costs of materials; inflation in costs of materials, design changes, and currency fluctuations; and unforeseen site conditions and/or circumstances (e.g. unusual rainfall, Page 8 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) dredged material delays). Other issues that negatively affected the project's efficiency were: (i) the high staff turnover in key positions, (ii) instances of unsatisfactory performance of the design and supervision consultant (e.g., not properly identifying underground services), coupled with low technical capacity of the client (e.g. Public Utility Board had difficulty supporting the contractor with relocation of underground services), and (iii) unsatisfactory project management and lack of familiarity with procurement processes. All these led to delays and extension of the project completion date by two years. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 94.00 Appraisal  40.30 Not Applicable 95.00 ICR Estimate  17.40 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The relevance of the project development objectives is rated high. The project substantially helped to improve the condition of South Tarawa’s main road network, albeit there were moderate shortcomings in strengthening road financing and maintenance capacity. Efficiency is assessed as modest due to lower ex-post economic rate of return, high cost overruns, as well as operational inefficiencies that led to a two-year project extension. The overall outcome is moderately satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome Financial. Road maintenance will be key for the sustainability of project investments. First steps were made in setting up the routine maintenance system and the road maintenance arrangements were in place for 2019. The Page 9 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) long-term sustainability is fully reliant on the availability of sufficient annual national budget support, for which some proposals are under discussion. A Cabinet Paper, which proposes increases in user charges from direct beneficiaries of the road improvements - such as vehicle registration fees and fuel taxes, is under consideration to increase maintenance funding. If approved, this should facilitate consistently adequate allocations to the Ministry of Infrastructure and Sustainable Energy (MISE) for routine and periodic road maintenance. Environmental. Climate change- induced sea level rise and storm surge could pose a risk to the sustainability of the investment. It will be critical for the Government to monitor these impacts and to ensure that the necessary infrastructure investments are made. Temporary coastal protection works were done at multiple sites along the road to allow for road construction to occur (e.g. construction of sand-cement bag enabling seawalls). With a limited design life, these works do not provide sufficient long-term protection against erosion and that the temporary works would need to be replaced with permanent structural protection.Failure to construct more resilient protection will result in continued erosion and damage to the road. To address this, coordination is necessary between the activities financed by various donor projects. Road safety. Although road safety features were incorporated into the roads design, there is a higher risk of increased number of accidents due to increase in traffic and average speed. This underscores the importance of implementing the measures recommended in the road safety action plan, and specifically: (i) establishing the road safety office and appointing an office manager to lead implementation of the plan; (ii) allocating of resources for road safety measures and enforcement in the roads budget; (iii) establishment of a data collection system that makes it possible to understand the underlying causes of the accidents. Institutional. Strong political ownership, institutional leadership, close stakeholder coordination, as well as collaboration and communication between ministries influence the successful implementation of reform initiatives. Continued dialogue with high-level leadership is required to sustain and advance the road safety agenda and the road maintenance agenda. Ongoing contractual dispute between the Ministry of Public Works and Utilities (MPWU) and the Contractor. At the time of publication of the Implementation Completion and Results Report (ICR), a contractual dispute between MPWU and the contractor was under arbitration. The Recipient would inform the Bank regarding the outcome of the arbitration when the matter is resolved, as well as any impact this may have on the investment. 8. Assessment of Bank Performance a. Quality-at-Entry This was the World Bank’s first engagement in the transport sector in Kiribati. The project's strategic relevance, country context and environmental and social development issues were clearly identified at entry. A beneficiary survey was specifically done to inform the technical design and address local community concerns. Availability of materials, climate change, road safety and gender aspects were considered, and resulted in unique design features such as: (i) narrowing of the road in some places to Page 10 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) ensure footpaths; (ii) successful piloting of geocell concrete pavements; and (iii) inclusion of female workers as part of the three sub-contractors engaged for routine maintenance. Similarly, to account for flooding or other events caused by climate change, the design included concrete u-drains, and some sea walls. The project was prepared in a short period of nine months (concept to approval), and as the ICR (para 57) puts it, there was a trade-off between speed of preparation and implementation readiness. With no domestic capacity for civil works of this magnitude, no capacity in the GoK for managing such a project, as well as no history of equivalent civil works, the project faced a number of challenges in implementation. Due to lack of data on unit costs, the initial cost estimate by the design and supervision consultant was inaccurate and resulted in a request for restructuring and then additional financing. According to the AF Project Paper (2015), comparisons with similar projects in the region identified the project cost as an outlier in terms of high costs. While a high standard of technical specification was used during appraisal, the resulting scale of investments was initially underestimated. Quality-at-Entry Rating Satisfactory b. Quality of supervision The project team carried out 23 implementation missions during a 7-year implementation period, of which 17 were jointly carried out with the co-financiers. Missions undertook site visits and discussions with contractors and consultants. There was a low turn-over of TTLs (two), throughout the project lifetime. The WB team's implementation support was adequate. The project faced initial delays due to significant capacity constraints within the GoK and a lack of familiarity with Bank guidelines and procedures. As a result of the early poor performance, delays in the project, and the funding shortfall identified in the bidding process, in December 2012 the project was rated moderately unsatisfactory with regard to progress towards achievement of the PDO, and unsatisfactory with regard to implementation progress. An Action Plan was put in place in late 2012 to address performance challenges leading to a number of changes, including restructuring the PMU arrangements, and embedding an internationally experienced technical auditor in the MPWU to assist with the project management and quality control (AF PP, 2015). Critical issues that emerged during implementation and for which the WB was able to provide additional guidance and persuade the client to agree with were: (i) compliance of safeguards documents with the WB safeguard policy; (ii) improvement of the bidding documents; and (iii) improvement of the reporting of performance and progress. Additional financing and project restructuring were processed in 2015 to ensure the completion of the project activities to achieve the development objective. As reported by the ICR (para 91), although some activities were at that time dropped from the project, provisions were made for them to be covered either by other ongoing WB financed projects in the country at that time, or by other development partners. Page 11 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design Two outcome indicators were defined at appraisal that were linked with each of the objectives: (I) number of km of the paved road network rehabilitated and suitable for only routine maintenance, and (ii) an agreed plan for reform of road sector management and financing under implementation (PAD, Annex 1). The target for the rural roads rehabilitated indicator was based on incorrectly classified rural roads. The reporting was to be undertaken by the MPWU in the form of semi-annual Project Reports. Most of the data for the monitoring and evaluation was to be gathered by the project’s design and supervision consultant and was within the capacity of the MPWU to supply. In 2015, the Results Framework was revised. The first indicator (referring to “km of paved road network rehabilitated”) was split into two indicators (“kilometers of rural roads rehabilitated” and “kilometers of non-rural roads rehabilitated”). Despite the change in the indicator, when compared to the original target value, the value of the combined revised outcome targets was slightly increased, from 39.5km to 40.1km. The second indicator was kept the same, but the target year of achievement of the objective was postponed by one year. b. M&E Implementation During implementation, missions consistently identified that project reporting was unsatisfactory. To address this, the 2015 restructuring included revising the PMU arrangements, and embedded the project monitoring and evaluation as part of the responsibilities of the Kiribati Fiduciary Services Unit (KFSU), which was established in the Ministry of Finance and Economic Development (MFED). Starting with 2016, Aide Memoires noted an improvement in the quality of the project progress reporting, but issues continued to persist until project closure (ICR, para 74). Overall M&E utilization was hindered by systematic delays with reporting and issues with the quality of reports. The continuous WB and ADB supervision and mission reports, including detailed Action Plans for Page 12 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) PMU follow-up, provided input to the M&E efforts. Throughout the project lifetime all indicators included in the Results Framework were routinely updated and reported in the ISRs. c. M&E Utilization It is evident from the ICR that he indicators were used to track achievement of the PDO and report on the results in the ICR. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified Category B under due to the anticipated environmental impact. Two safeguard policies were triggered: Environmental Assessment (OP/BP 4.01) and Involuntary Resettlement (OP/BP 4.12). Environmental Assessment. The ICR (para 80) reports that "the environmental safeguards compliance was overall Satisfactory". The Environmental Management Plan (EMP) was prepared and disclosed on September 6, 2010. The EMP was subsequently updated and redisclosed on October 1, 2014, to reflect the final project designs and other developments. Issues regarding: (i) bio-security checks of imported aggregate materials; (ii) spills and lead management; and (iii) lack of secure storage for unexploded ordinance were raised in the first years of implementation, but they had all been addressed by October 2014. In May 2018, inappropriate asphalt disposal was noted, which was not in accordance with previous agreements. The Environmental and Conservation Division (ECD) advised there was not a significant environmental issue, and at project closure, discussions were ongoing with the contractor to reach a resolution that is acceptable to MISE and ECD. The contractor’s health and safety performance was regarded to be best practice, with Aide Memoires repeatedly commending the contractor’s occupational health and safety (OHS) procedures, including the site safety induction training, and excellent site management. Involuntary Resettlement. Residential land was to be permanently acquired for about 200 bus stopping places, 17 drainage easements and 56 locations for minor realignments of the road. In total, the affected strips of private land amounted to 8,256m2 and 131 trees belonging to 348 households, but these represented a minor portion of affected households’ yards with no serious impacts on incomes or livelihoods. The Resettlement Action Plan (RAP) was disclosed in Kiribati and at the WB InfoShop on May 2, 2013. The RAP was based on the Resettlement Policy Framework (RPF) disclosed during project preparation, and consistent with ADB’s Safeguard Policy Statement (2009) and WB’s safeguard policy on involuntary resettlement (OP 4.12). Page 13 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) The ICR (para 82) reports that the GoK was effective in addressing land acquisition. Compensation of 317 project affected persons (PAPs) for loss of land and assets, mainly for loss of fruit bearing trees, had been paid (or secured in an escrow account for 22 PAPs). A non-governmental organization (NGO) was recruited by the project to monitor RAP implementation and to ensure that any grievances that arose were properly addressed. The Grievance Redress Mechanism (GRM) was accessible to PAPs in person, on the phone or in writing. The latest progress report indicated that 111 complaints had been raised since the commencement of the project. There had been evidence of complaints resolution throughout, including an independent review by the NGO which had confirmed that all grievances at the time of their assessment had been appropriately addressed. The ICR (para 83) also adds that with help of an NGO, the project undertook an HIV/AIDS information and education campaign for the consultant and the contractor’s employees and sub-contractors, as well as local communities in the proximity of the project site. The ICR does not report on the compliance with 'Involuntary Resettlement' safeguards policy. b. Fiduciary Compliance Financial Management (FM). The FM performance was “Moderately Satisfactory” throughout the life of the project, however there was an ongoing issue with contract management. The numerous errors in the Commitments Register and failure to incorporate contract variations resulted in a difficulty to accurately determine the financial position of the more complex contracts. The final FM review downgraded the FM performance to ‘Moderately Unsatisfactory’ as the financial position could not be accurately determined. The lack of clear understanding of the project budget continued to be noted as an issue up to the project closing. Interim Financial Reports were generally late but usually only by a few days, partly because they were often required to be resubmitted. Audit reports were mostly received on time and were unqualified except for the first audit report which covered the period from the start of the project to December 31, 2012 (ICR, para 85). Procurement. Specific procedures between the WB and ADB were agreed upon for: bidding for the civil works (the procurement process followed both WB and ADB guidelines; no-objections from both ADB and the WB had to be requested), recruiting the design and supervision consulting firm (terms of reference were jointly developed between WB, ADB and GoK; procured in accordance with ADBs Guidelines), recruiting technical assistance for institutional reform and road safety action plan, (terms of reference were jointly developed between WB, ADB and GoK; procured in accordance with WB Guidelines), as well as for contractual/financing and disbursement arrangements for the civil works component. The ICR (para 84) reports that procurement delays contributed to the slow startup of the project during the first two years of implementation, but by 2015 procurement activities under the project were largely completed with only a few minor items and technical assistance assignments yet to be procured. At the commencement of the project, delays in updating procurement plans and submitting progress reports were noted. In 2013, KFSU Page 14 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) was described as not operating correctly. The hiring of a dedicated procurement officer in the KFSU to support the project subsequently contributed to improved performance. c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory --- Satisfactory Bank Performance Satisfactory Satisfactory --- Quality of M&E Modest Modest --- Quality of ICR High --- 12. Lessons The ICR identified a number of lessons which could be categorized under five headings: (i) using projects to introduce improved practices; (ii) identifying opportunities for leveraging and supporting other projects; and, (iii) establishing foundations for future collaborations. Lessons were also drawn based on: (iv) the challenges of remoteness; and (v) capacity limitations. IEG summarizes them below: • Roads projects could be innovative in its design by introducing such improved practices as catering for persons with disabilities, gender targeting, protecting vulnerable users through hard infrastructure, awareness of the future impact of climate change in selecting appropriate pavements, and focus on occupational health and safety. The project design terms of reference called for ‘universal design’ principles to be applied but it is also important to undertake thorough technical reviews of design documents in the procurement stage to make sure these principles are properly considered. Gender sensitive surveys were translated into project activities (e.g. location of bus stops and street lighting, employment opportunities for women during construction). A focus on ‘hard’ infrastructure is a valuable Page 15 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) starting point for all road safety activities. Proactive review of the records for injuries and monitoring compliance with occupational health and safety under the project helped ensure this remained a priority for the contractor. • When the government face challenges in implementing multiple projects with complementary activities, practical approaches can be developed to improve coordination between donors and between projects. For example, funds can be combined with a project funds into a single procurement package, engaging one contractor. This not only can reduce the procurement effort, but also simplify supervision and safeguards, reducing the burden on both government and donors. Procurement and financial management activities of donor- financed projects can be transferred to a central project management unit if established in the government. • Joint co-financing —particularly for fragile states—is a useful way to reduce the implementation burden on limited capacity implementation agencies, however, it needs to be adapted to the specific project arrangement. The concept of having a ‘lead donor’ responsible for clearing all safeguard documents and procurement documents – after giving due consideration of the views of other donors- proved to be a pragmatic solution. Conducting most missions jointly under this project, with aide memoires signed by all three donors, sent a clear and consistent message to the government regarding the project’s issues and progress. The use of joint co-financing, however, increased the demands on the project team as they needed to liaise with other donors and include the additional ‘layers’ of clearances. The nature of the co-financing should be tailored to the project specifics and in some instances setting up framework arrangements with other development partners is key to reducing the burden on low capacity implementing agencies. • In remote small island states, such as Kiribati and other Pacific Island Countries, higher cost contingencies need to be factored into project design. At the time of bidding for the road works under this project in Kiribati, the contractors priced the remoteness and risk of the project fundamentally differently, as evidenced by the bids ranging in price by 30% (and also being 20% above the Engineer’s Estimate). This pattern has been observed across the Pacific and it would be valuable for further analysis to be done to investigate in detail how cost estimates could be made more reliable, both at appraisal and at the time of bidding. A substantial remoteness cost premium should be included in the cost estimates. • Capacity limitations requires certain approaches and can be factored in project design. Under this project, project timeline was unrealistic. The success of the project is directly dependent on the project manager and the local implementation team, and often a scale- up of donor activity competes for a few people available to perform these roles. Intensive implementation support is required in the early years. Efforts of development partners should continue to build capacity of local specialists and facilitate transfer of skills from international specialists and advisors. Technical expertise for monitoring of consultants is required when governments lack capacity. Expect the unexpected: civil works contracts should anticipate a “time” contingency in the implementation schedule, in addition to the cost contingencies. Page 16 of 17 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Kiribati Road Rehabilitation Project (P122151) 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is clear, succinct, and consistent, both internally and with the guidelines. It is results-oriented. The thorough analysis and the quality of evidence provide a solid basis for drawing conclusions. The report candidly discusses the issues that affected the project's preparation and implementation and provides additional evidence that is relevant to the project's results, in addition to the indicators in the results framework. It also provides important details and analysis of reasons for high cost overruns. The report offers a useful and rich discussion of findings and lessons derived from the project experience. Overall, all the sections in the ICR are well -argued and provide comprehensive assessments. IEG disagreed with the substantial rating for the second PDO of strengthening road financing and maintenance capacity as the intended outcome indicator was not achieved. a. Quality of ICR Rating High Page 17 of 17