Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) Report Number: ICRR0021773 1. Project Data Project ID Project Name P096931 TO-Transport Sector Consolidation Country Practice Area(Lead) Tonga Transport L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-57300,IDA-D0910,IDA-H4160,TF- 19,688,701.51 99585 Bank Approval Date Closing Date (Actual) 28-Jul-2008 31-Dec-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 0.00 10,300,000.00 Revised Commitment 19,740,000.00 10,300,000.00 Actual 19,688,701.51 10,300,000.00 Prepared by Reviewed by ICR Review Coordinator Group Elisabeth Goller Kavita Mathur Ramachandra Jammi IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives The project development objective (PDO) as stated in the 2008 Financing Agreement (FA) (page 5) was "to assist the Recipient to establish and consolidate the operations of the newly-created Ministry of Transport as a unified transport sector policy, planning and regulatory ministry and to improve the level of compliance of the civil aviation and maritime subsector entities with international safety and security standards. Page 1 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) The PDO revised in 2010 was "to assist the Recipient to develop its transport sector to have: (i) stronger policy, planning and regulatory institutions and framework; (ii) improved safety and security facilities and compliance with international safety and security standards; and (iii) greater domestic capacity for road rehabilitation and maintenance" (July 29, 2011 Amendment to FA). b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 13-Dec-2010 c. Will a split evaluation be undertaken? No d. Components A - Establishment of a sustainable transport policy, and institutional, and operational framework (cost at appraisal US$1 million, cost at appraisal of second Additional Financing (AF) in 2015 US$ 1.59 million, actual cost US$1.71 million). Activities under this component aimed at enabling the creation of a sustainable transport sector. They included the (i) formulation of a national transport sector policy and planning framework, and (ii) the preparation of sustainable investment plans for the aviation, maritime and road/land transport sub-sectors, which consisted of (a) an aviation sector strategic plan, (b) a medium-term sustainable development and investment and financing plan for Fua’amotu International airport, (c) a sustainable investment and financing plan for medium term infrastructure needs of inter-island shipping and realistic medium term planning scenarios for the potential future state and operational needs of the inter- island shipping market, and (d) options for sustainable financing of the road maintenance needs of the country. B - Compliance with Mandatory Safety and Security Standards (cost at appraisal US$2.73 million, cost at appraisal of second AF in 2015 US$7.62 million, actual cost US$5.82 million). This component provided high priority strategic investments to meet mandatory safety and security standards required under international agreements, treaties and obligations. It included the following activities: (i) Aviation sector regulatory surveillance and compliance obligations review and support, (ii) Navigational aids and equipment at Fua’amotu International Airport, (iii) Fire tender at Fua’amotu International Airport, (iv) Replacement of security (baggage screening) equipment and emergency power equipment, (v) Improvements to all ferry terminals, (vi) Marine safety and environmental protection equipment, and (vii) Reform of motor registry system. C - Supporting the Transition towards Sustainability in the Transport Sector (cost at appraisal US$600,000, cost at appraisal of second AF in 2015 US$8.82 million, actual cost US$7.73 million). This component provided further investments consistent with the policy/planning frameworks and investment plans developed in Component A for a more sustainable transport sector. It comprised a package of strategic items (technical assistance and targeted investments) consistent with the policy/planning framework and medium-term sustainable investment plans developed in Component A and support to the Page 2 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) Government of Tonga in the early stages of implementing key initiatives towards a more sustainable transport sector, as well as consolidate the transition to the unified Ministry of Transport. Tentatively the following activities were identified at appraisal: (i) Legislative framework for transport sector planning and policy, (ii) Resurface the inter-island (domestic) ferry terminal at Nuku’alofa, (iii) Installation of solar lighting at primary ferry terminals, and (iv) Additional investment in safety and security equipment at airports. D - Project Implementation Support (cost at appraisal US$630,000, cost at appraisal of second AF in 2015 US$1.73 million, actual cost US$3.69 million). This component provided additional funding for project management support during implementation. The project components did not change, but the successive restructurings and AFs (see below) added 21 new activities, expanded two existing activities, and dropped four activities. Note. The cost figures at appraisal and at project close do not include contingencies of US$490,000 and US$740,000, respectively. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost The actual project cost was US$23.02 million (ICR, page 2), which is in line with the appraisal estimate of the second AF of US$23.09 million (2015 Project Paper (PP), page 23). The original appraisal estimate was US$6.26 million. Financing The actual project financing for a total of US$19.69 million came from (i) an IDA grant of US$5.23 million, (ii) an IDA grant of US$2.06 million, (iii) an IDA credit of US$2.07 million, and (iv) a Pacific Regional Initiative Fund (PRIF) grant of US$10.30 million from the Australian Government (Operations Portal). This is 5 percent more than the appraisal estimate of the second AF of US$18.77 million (2015 PP, page 23). The larger amount of available financing derived from exchange rate gains. Recipient Contribution The actual Recipient contribution was US$3.33 million, in line with the appraisal estimate of the second AF of US$3.32 million (2015 PP, page 23). The Recipient contribution was in the form of the road maintenance fund replenishment and consumption tax refund, exemption from custom tax and duties, and budget support through the Ministry of Infrastructure and Tourism (ICR, para 17). The original Recipient contribution was US$1.08 million. Page 3 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) Dates, Project Restructurings, and AFs The project was approved on July 28, 2008, became effective on December 12, 2008, and closed on December 31, 2018 (ICR, page 2). The original closing date was December 31, 2011. The project was restructured six times, including two AFs (ICR, page 10). The December 13, 2010 AF and level 1 restructuring introduced a road maintenance program under component C, which focused on local private sector participation in road maintenance, and scaled up the investment in the maritime sector under component B. It (i) increased the project cost by US$10.3 million to US$17.09 million (this amount came from an PRIF grant), (ii) modified project activities, costs and indicators to target current government strategies and priorities, particularly related to maritime safety and new investments in road maintenance, (iii) revised the PDO to reflect the new activities, (iv) extended the closing date from December 31, 2011 to December 31, 2013, (v) changed the project category for environmental assessment purposes from C to B to cover larger civil works activities for road maintenance, and (vi) included additional procurement methods. At the time of this AF and restructuring, US$1.32 million of a total financing of US$19.69 million (6.7 percent) was disbursed. The June 5, 2012 level 2 restructuring harmonized the IDA and PRIF FAs and responded to the evolving organizational structure in the Government of Tonga. It (i) changed the implementing ministry’s name from Ministry of Transport to Ministry of Infrastructure, (ii) harmonized the disbursement categories in the FAs, (iii) included retroactive financing to the PRIF FA, and (iv) changed the reporting requirements. The August 1, 2013 level 2 restructuring (i) extended the closing date to December 31, 2015, (ii) reallocated funds among project components to provide support to the newly created and consolidated Ministry of Infrastructure, and (iii) modified the results framework to reflect these changes and to include new IDA core indicators. The February 14, 2014 level 2 restructuring extended the geographic scope of the components to finance urgent reconstruction works at the Ha’apai airport facilities following Cyclone Ian. The December 29, 2015 AF and level 2 restructuring added US$4.0 million to scale up the maritime and land transport safety aspects of existing activities and increase the effectiveness and extent of the project’s impact. It (i) made changes to the project components, (ii) revised the results framework, (iii) adopted the most recent procurement guidelines, (iv) and extended the closing date from December 31, 2015 to June 30, 2018. The results framework changes mainly clarified indicators and added several new ones. The June 22, 2018 level 2 restructuring extended the closing date of the IDA AF from June 30, 2018 to December 31, 2018 because of unexpected delays in the contract for maritime safety works. The 2010 AF and restructuring increased the cost and scope of the project. It did not lower the project's level of ambition. It changed the PDO by (i) enhancing the clarity of the two original objectives, (ii) expanding the scope of one of these original objectives, and (iii) adding a new objective to capture the results of the new activities. The 2010 AF and restructuring took place early on in project implementation. The 2015 AF and none of the other restructurings lowered the levels of ambition of the project. Based on the above, the application of a split rating is not warranted. Page 4 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) Note: The key project dates in the main text and the data sheet of the ICR are different. The ICRR uses the dates from the ICR data sheet because they are in line with the dates in the Operations Portal. 3. Relevance of Objectives Rationale The relevance of the PDO is rated high. Country context: Transport plays an essential role for Tonga with its isolated and dispersed population settlements, but the institutional arrangements in the sector at appraisal did not provide an adequate long- term basis for its management. Safety and security standards required under international agreements, treaties and obligations in the air and maritime sectors were not complied with due to underinvestment and a lack of maintenance. In addition to accidents, this could have induced some airlines to stop their flights to the country. Tonga had no local road construction and maintenance capacity. Road maintenance was largely absent and only key roads, mostly on the main island, were periodically rehabilitated by international contractors, which is a costly and inefficient practice. Against this background, setting up the policy, planning, regulatory, and institutional framework for the sector and building capacity, improving safety and security of transport facilities to comply with international standards, and creating greater domestic capacity to rehabilitate and maintain the road network were critical and highly relevant. Alignment with strategy: The importance of the PDO for the country was reflected in the government's vision for the transport sector articulated in the 8th National Strategic Development Plan. It was also part of the Bank's 2006-2009 Regional Engagement Framework for the Pacific Islands. This Framework outlined the likely allocation of IDA resources for transport infrastructure and a considerable potential to develop a long-term partnership in the sector. The PDO remained well aligned with successive national development strategies. Improving air connectivity, strengthening planning and maintenance, especially in the road and maritime subsectors, ensuring passenger safety, and building capacity are still priorities included in both the 2015-2025 Tonga Strategic Development Framework and the National Infrastructure Investment Plan. The PDO is also in line with the FY2017-2021 Regional Partnership Framework for Kiribati, Republic of Nauru, Republic of The Marshall Islands, Federated States of Micronesia, Republic of Palau, Independent State of Samoa, Kingdom of Tonga, Tuvalu, and Vanuatu, including the objective of “increasing access to basic services and improving connective infrastructure” and “strengthening the enablers of growth opportunities”. Previous sector experience: In 2004, IDA supported a joint review of Tonga’s entire transport sector, the responsibility for which was scattered throughout a number of different Ministries, and recommended strategies and options for improving sector performance. The IDA-supported Tonga Transport Sector Review (TTSR-TF052906) was completed in 2005 and many of its recommendations were adopted as government policy (PAD para 3). Rating Relevance TBL Page 5 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) Rating High 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective To develop the recipient's transport sector to have stronger policy, planning, and regulatory institutions and framework. Rationale According to the original theory of change, technical assistance (TA) to prepare legislative pieces, policies, and strategic and investment plans, TA through capacity building and training, and TA through technical and regulatory reviews were expected to lead to (i) the consolidation of all transport policy, regulatory, and planning functions under one adequately staffed ministry, (ii) the creation of a road maintenance fund, and (iii) an improved policy framework, with draft plans and strategies implemented or under implementation and laws and policies approved or submitted for approval. This, in turn, was to create stronger transport sector policy, planning, and regulatory institutions and frameworks. This theory of change did not change with the AFs, which only intensified the support to the Government of Tonga and added financing for physical investments. Outputs and intermediate results The ICR does not distinguish between outputs financed with resources of the original project and the AFs. However, most of the outputs related to objective 1 were already envisaged under the original project and the AFs provided additional resources to scale-up some of the activities. According to the ICR (pages 15, 16, 49, and 50), the following main outputs and intermediate results were achieved:  Transport Sector Policy and Planning Framework, National Transport Sector Strategy, and implementation plan drafted (the Bank task team noted that they were not yet adopted and that government was still working on them based on the documents prepared under the project);  Tonga Airport Limited core functions and responsibilities defined and Airport Authority Act submitted to the Cabinet (the task team noted that the Act is currently undergoing lengthy discussions in the government's legal department);  Investment plans drafted for all three transport subsectors and the Fua'amotu international and Lupepau'u airports;  Aviation and Maritime Sector Strategic Development Plans drafted;  Civil Aviation Act declared effective in 2014; Page 6 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931)  Cabinet Decision on administration of the outer island ports and maritime investments approved;  Amendment bills and regulations drafted for the following: port management bill, road bill (act), traffic bill (act), amendment to port authority bill, amendment to wharves bill traffic regulations drafted (the task team noted that the traffic regulations/bill are currently undergoing lengthy discussions in the government's legal department);  Aviation sector legislation and regulatory framework review conducted;  Transport sector review carried out;  Training of management advisor;  Study proposing institutional arrangements and funding options for sustainable road maintenance completed;  Road asset management system developed;  Activities associated with the change management process completed. The Ministry of Transport and the Ministry of Public Works were merged to become the Ministry of Infrastructure in 2012 and then the Ministry of Infrastructure and Tourism (MOIT) in 2015;  Divestment plan for 15 non-core activities of the MOIT and support to the transfer of key items such as concessioning of the dormant state quarry operations to the private sector;  Road construction equipment hired out to contractors;  Information technology (IT) strategy prepared, including an accounting system implemented in the MOIT;  Strategic staffing plan for the MOIT prepared;  Support provided by experts and advisors hired by the project for the preparation of the relevant legislation to implement reforms;  General and specific support provided to build capacity, for instance through the project's maritime advisors and the technical advisors on roads, and enhance accountability and transparency, for instance through audits;  Half of the senior position vacancies in the MOIT filled, including the Director position for Land Transport Division, which is only partially compliant with the indicator target of all senior positions filled with qualified staff (the Bank task team confirmed that the senior staff hired is qualified); and  28 percent of the total MOIT staff was daily paid staff at project closing, which is considerably higher than the target of not more than 5 percent. Page 7 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) Note: In this ICRR, MOIT is used to refer to the project implementation agency even if initially it was the Ministry of Transport and later the Ministry of Infrastructure. Outcomes With the exception of pending approvals of legislative and regulatory measures, lack of completion of the change management process for the maritime subsector department, and the systematic replenishment in the road maintenance fund, the objective was achieved. Hence, its efficacy is rated substantial. The outcome target of having "one transport ministry responsible for transport policy and planning" was achieved. According to the ICR (paras 24 and 55 b), the project was instrumental in facilitating the change management process and integration following the merger of the Ministry of Works and the Ministry of Transport to form the Ministry of Infrastructure and later the MOIT. The MOIT regrouped all the responsibilities of the transport sector under one institution, including functions of sector policy, regulation, and planning. The project supported the preparation of legislation to address the regulatory oversight and safety of the transport sector and the strengthening of enforcement capacities. Evidence of the MOIT's enhanced enforcement capacity provided in the ICR (para 24) comes from the successful detainment of five unseaworthy vessels in Tonga waters during project implementation. The change management process to separate the regulatory and operating responsibilities of the maritime subsector department was not completed (ICR, para 89). The target of having "at least 50% of staff positions filled according to strategic staffing plan", related to the outcome indicator of "change management process and creation of appropriate transport policy and planning units", was exceeded. By project end, about 70 percent of staff vacancies were filled according to the strategic staffing plan associated with the change management process. However, the ICR (para 26) points out that there were staff vacancies and high turnover at senior and management level throughout project implementation. With the support of the project, core MOIT divisions were created for Maritime and Ports, Civil Aviation, Land Transport, Policy and Planning, Building and Control, and Corporate Services. In terms of capacity building, the project strengthened the regulatory and enforcement capabilities in the aviation and maritime sectors through the development of manuals, plans, and training. The project also helped develop an asset management capacity through the transport asset management system, hardware, and training. The outcome target of having the "transport policy framework developed and subsector policies defined" was also largely achieved. The project supported the preparation of a large number of transport-related legislation, regulations, plans, and policies listed in the output section above. The ICR (para 91) notes that most of the key transport bills were approved. It mentions that the Civil Aviation Act and the MOIT's 2015-2018 Corporate Plan were approved and that the Airport Authority Act was submitted to Cabinet as of November 2018. It also mentions that the transport policy framework and a national transport sector strategy prepared were still to be confirmed at ministerial level by project closing. The ICR does not provide details on the approval of other legislative, regulatory, and planning pieces. The target of "20 percent of overall road maintenance activities financed by the fund", related to the outcome indicator of "road maintenance fund created and operational", was exceeded. In 2013, the fund was set up as a revolving fund to be replenished through revenues from fuel levy, select fees, and fines with the support of Page 8 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) the project. By project closure, the available resources for the last year of project implementation from 2017 to 2018 were TOP$10.73 million. This corresponded to 43 percent of the annual maintenance needs and is significantly higher than the target of covering 20 percent of these maintenance needs, which were estimated at TOP$6.8 million. Rating Substantial OBJECTIVE 2 Objective To develop the recipient's transport sector to have improved safety and security facilities and compliance with international safety and security standards. Rationale The original theory of change for this objective was that (i) TA, works and the acquisition of goods and equipment to improve air safety, (ii) works and the acquisition of goods and equipment to improve maritime safety, (iii) maintenance, and (iv) environmental protection activities would lead to (a) International Civil Aviation Organization (ICAO) certification of the Fua'amotu airport and (b) ports and airports equipped with navigational aids, tools and features, fire-fighting and rescue stations, and security and emergency power equipment. This, in turn, was expected to lead to improved safety and security facilities and the compliance with international standards. With the AFs, the original theory was expanded to include a road safety perspective. The revised theory of change was that the above mentioned activities and the new road safety activities would lead to (a) International Civil Aviation Organization (ICAO) certification of the Fua'amotu airport, (b) ports and airports equipped with navigational aids, tools and features, fire fighting and rescue stations, and security and emergency power equipment, and (c) safer roads. This, in turn, was expected to lead to improved safety and security facilities and the compliance with international standards. Outputs and intermediate results The ICR does not distinguish between the outputs financed with resources of the original project and the AFs, and the distinction below represents an estimate of the IEG evaluator based on the project description in the PAD, the RPs, and the ICR (pages 11 and 12). The following outputs and intermediate results were achieved under the original project (ICR, pages 17, 18, 50, and 51): Air safety  Investment plans drafted for Fua’amotu airport and for the maritime sector, contributing to the identification and prioritization of needs to meet international standards;  Aviation sector legislation and regulatory framework review and strengthening of the regulatory oversight of the aviation sector, which included aspects related to aviation safety;  TA and works to obtain the ICAO certification for Fua'amotu airport carried out; Page 9 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931)  Navigational aids and safety equipment installed at the airports in Tongatapu and Ha’apai;  Safety training delivered, including with firefighting training equipment, fire hoses, etc. Maritime safety  One working boat for the Niuas island, which according to the task team was necessary to safely transfer passengers and cargo from the Otuanga’ofa to the island;  Maritime policy frameworks reviewed to ensure strong compliance with the International Maritime Organization (IMO) regulations, assessment of maritime safety, including of passenger vessels, and detailed designs for safety improvement works; and With the AFs the following additional outputs were obtained (Note: Some of the outputs below were conceived under the original project but received substantial additional financing and hence are listed here): Air safety  New fire station constructed at Fua’amotu airport, access road rehabilitated, terminal building expanded, and screening area equipped with new security screening equipment and emergency power generator; Maritime safety  Navigational aids, marine safety, and maintenance and environmental protection equipment installed in the five island groups of ‘Eua, Tongatapu, Ha’apai, Vava’u and Niuas and safety works completed at the ferry terminals, with the installation of 86 navigational lights, 18 buoys, 16 buoy anchors, 6 towers, pavement and capping beam works, and concrete slab construction to wharves and passenger and cargo facilities in selected ports and ferry terminals;  Equipment for the Tonga Maritime Polytechnical Institute installed;  Development and provision of a coastal radio watch system with associated training;  Operations manuals developed for the Maritime and Port and Civil Aviation Divisions;  Safety campaigns and capacity building activities carried out; Road safety  Road safety audits and works completed on the Airport Nuku’alofa main road;  Three road safety campaigns completed. This activity was led by women with a major emphasis on children and interventions in schools; Page 10 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931)  Legislation on seat belt requirement submitted to the Cabinet (the task team noted that this legislation is currently undergoing lengthy discussions in the government's legal department); and  TA by the road safety advisors and media consultants carried out. Outcomes This objective was fully achieved, and its efficacy is rated substantial. In line with the outcome indicator target, the Fua'amotu international airport received the ICAO certification on August 27, 2010. It was the first Pacific Islands' airport to receive such certification. According to the ICR (para 30), the airport certification manual was regularly updated, the certification valid for two years was renewed, and training was undertaken as needed. The project contributed to the achievement of objective 2 through the needs identification, TA, the construction of the new fire station, the expansion of the existing terminal building, works and equipment on the transit screen area as well as navigational aids, lights, fenders, and safety and protection equipment. The outcome target of "80 percent of maritime infrastructure under the Transport Sector Consolidation Project's AF has safety measures completed" was exceeded with all maritime infrastructure under the project's AF having the planned safety measures. These included equipment, such as navigational aids, marine safety equipment, maintenance and environmental protection equipment, and improvement works. The project also helped improve road safety through a mix of TA and analytical work, road improvement works, and road safety awareness and education campaigns. For the indicator of improved road safety management systems, the target of "two road safety campaigns completed and improved compliance of seat belt usage rate in safe corridor demonstration program" was partially achieved. The ICR (page 46) notes that three road safety campaigns were completed and the road asset management system can be used to identify priority road safety interventions. With respect to the indicator of "improved compliance in terms of seat belt usage", the ICR points out that there was no definition of safe corridor demonstration program, but legislation on the seat belt requirement was validated at ministerial level and submitted to the Cabinet by project closure. The task team noted that this legislation is currently undergoing discussions in the government's legal department. Rating Substantial OBJECTIVE 3 Objective To develop the recipient's transport sector to have greater domestic capacity for road rehabilitation and maintenance. Rationale The 2010 AF and restructuring added this objective. Its theory of change was that (i) the provision of goods and equipment to improve the road maintenance capacity, (ii) training, among others, of local contractors and consultants, (iii) the implementation of a road safety program, including road safety improvement works Page 11 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) and public awareness and education campaigns, and (iv) the replenishment of the road maintenance fund would lead to enhanced domestic road rehabilitation capacity, more regularly maintained roads by domestic contractors, and greater safety awareness. Outputs and intermediate results The following outputs and intermediate results were achieved under the this objective (ICR, pages 16, 17, 51, and 52):  Study proposing institutional arrangements and funding options for sustainable road maintenance completed;  Local road construction industry assessed and detailed action plan to strengthen it prepared (the task team confirmed that these activities were carried out with project funds);  Domestic contractors and consultants trained in road maintenance techniques, business management, and bidding knowledge;  Otta seal method introduced as an effective low-cost treatment for low-volume roads introduced, using aggregates available from quarries in Tongatapu and the outer islands;  11 Otta seal contracts financed for more than TOP 4.0 million;  Eight companies that participated in the procurement process for road maintenance works active;  Road network condition survey carried out;  Road materials testing laboratory received equipment, hardware, upgrades, and training;  Traffic counts completed;  Sustainable road maintenance planning and investment plan drafted;  14 routine maintenance contracts financed for more than TOP 2.7 million;  40 km of rural roads rehabilitated under the project, which exceeds the revised target of 35 km, but is significantly lower than the original target of 75 km. The ICR (page 44) notes that the original target of 75 km referred to "rural roads maintained" and when this indicator was updated to reflect the Bank's core indicator "Roads rehabilitated, rural", it was estimated that 35 km could be targeted for rehabilitation, which is more costly than maintenance, and  5 km on non-rural roads rehabilitated, in line with the project end target. Page 12 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) Outcomes This objective was achieved with minor shortcomings in achieving the indicator targets, and it is rated substantial. 77 km of classified public roads were designed, supervised, and maintained by Tongan companies, which slightly exceeds the outcome target of 75 km. In addition, the task team specified that local contractors also carried out the regular maintenance of 488 km of roads. This slightly exceeds the target of 462 km. The ICR (para 36) explains that this was a huge achievement because before the project, there was hardly any routine or periodic maintenance and instead international contactors rehabilitated roads periodically, which was costly and unsustainable. The achievements above were possible because of (i) the analysis of the institutional arrangements and funding options for sustainable road maintenance mentioned under objective 1, (ii) the assessment of the local industry, (iii) the action plan to strengthen it, (iv) the comprehensive training program for contractors and consultants, (v) a convinced government believing in outsourcing, and (vi) financial resources and mechanisms to secure the funding. It needed a Land Transport Department within the MOIT with systems, processes and trained staff to manage the road maintenance program. However, the operational procedures to systematically replenish the road maintenance fund through dedicated revenues were not effective by project end, and the fund had to rely on the regular annual budget allocation process. This diminishes the certainty on the availability of resources for maintenance and hence limits the contractors' ability to plan ahead and make investments in machinery and equipment. The project helped create the necessary capacity in the Land Transport Department to assess road network conditions, prepare and manage annual routine maintenance contracts, prepare a periodic maintenance programs, manage resealing, and minor rehabilitation works contracts. The project also assisted the Land Transport Department with road asset management processes to prepare the annual road programs, including road inventory and condition databases, planning tools, technical standards, and procurement and contract administration procedures (ICR, para 34). By project end, 42 percent of the total classified roads of 640 km were in good and fair condition, which is significantly lower than the target of 70 percent. The ICR (page 45) explains that the baseline was estimated and never confirmed. 76 percent of the classified road network was in good and fair condition after the initial two years of road maintenance financed by the project. From 2013 to 2016, road maintenance was underfunded, which affected the condition of the road network. Despite the increase in maintenance activities in 2017, by project closure in 2018, the project was not back on track. The ICR also clarifies that the MOIT now has the capacity to monitor the condition of the road network and prioritize interventions based on an effective road asset management. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Page 13 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) Rationale The second objective was fully achieved. The first and third objectives were achieved with minor shortcomings. Hence, the overall efficacy is considered substantial. Overall Efficacy Rating Substantial 5. Efficiency Economic efficiency At appraisal, only a qualitative assessment of the likely costs and benefits of the proposed project activities was carried out. A cost-benefit analysis (CBA) was not considered suitable given the nature of these technical assistance activities (PAD, para 26). The efficiency of the additional activities introduced with the 2010 AF was also assessed in qualitative terms (2010 PP, paras 38 to 40). An economic evaluation was carried out for the investments introduced with the 2015 AF. This analysis assumed a total cost of US$5 million and an analysis period of 10 years. It focused on the possible disruption of port operations due to a lack of maritime safety and the impact of safer roads. The analysis showed an economic internal rate of return (EIRR) of 21 percent and a net present value (NPV) of 2.03 million (2015 PP, page 11). By project end, CBAs were carried out for the three transport subsectors. They covered a time horizon of 20 years (2009-2028), used a discount rate of 3.2 percent, in line with the 2016 Guidance Note on Discounting Costs and Benefits in Economic Analysis of World Bank Projects, and a standard factor for the conversion from financial to economic costs of 0.90. The task team confirmed that the methodology used was the same as for the 2015 AF, but it has more details on key assumptions, which were simplified in the 2015 AF analysis. For land transport/road subsector, the benefits came from reduced vehicle operating costs, less accidents, and emergency reconstruction costs avoided due to improved climate resilience. The key assumptions provided in the ICR (page 56) are reasonable. The analysis showed an EIRR of 12.4 percent and a NPV of US$6.79 million. The task team confirmed that the sensitivity analysis carried out showed an EIRR above the discount rate of 3.2 percent for all hypotheses tested. For the maritime subsector, the benefits derive from the improved port infrastructure, which enables port operators to enhance their passenger vessels and cargo handling operations. The key assumptions provided in the ICR (page 56) are reasonable. The analysis showed an EIRR of 15.9 percent and a NPV of US$6.27 million. The task team confirmed that the sensitivity analysis carried out showed an EIRR above the discount rate of 3.2 percent for all hypotheses tested. For the aviation subsector, the benefits came from the compliance with the ICAO standards and from improved safety and security at Fua’amotu International Airport. This was expected to avoid potential flight disruptions, which could have happened without the project from 2015 onward. The assumptions in the ICR (page 56) appear reasonable. The CBA showed an EIRR of 19.9 percent and a NPV of US$1.32 million. A sensitivity analysis was carried out for different timings of the potential disruptions and their magnitude (i.e. percentage of Page 14 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) demand loss) to test the robustness of the results. This analysis showed an EIRR above the discount rate of 3.2 percent for all hypotheses tested. Overall, the project had an EIRR of 14.3 percent and a NPV of US$14.38 million. This does not include the benefits of the strengthening of the regulatory framework through the project. Administrative efficiency The project had two AFs and six restructurings. These AFs and restructurings were necessary and justified. The total implementation period was ten years. As pointed out in the ICR (page 21), the first project phase, between 2008 and 2010, was a technical assistance project. The second phase, between 2010 and 2015, was when significant investments took place, and the last phase, between 2015 and 2018, had a mix of activities to scale up safety and security improvement activities. The project changes justified the longer than usual implementation period. However, project implementation was slow initially because the MOIT was not adequately staffed and Tonga Airport Limited lacked familiarity with Bank procurement procedures. Further delays occurred mainly because of the late effectiveness of the PRIF grant and the late shipping of supplies. The project's administrative costs were 123 percent higher than the 2015 AF appraisal estimate of US$1.73 million. The task team explained that the 2015 AF did not adjust the project implementation support costs under component D in line with the actual needs. They also explained that this item covered the cost of the operation of the project support team, which was US$475,000 annually when the team included an international advisory and half of this cost, when it did not. It also covered the cost of international advisors used under component A. Overall efficiency The EIRR was lower than at the time of the 2015 AF, but still well above the discount rate of 3.2 percent, and the shortcomings in terms of administrative efficiency were relatively minor. Hence, the overall efficiency is considered substantial. Note: The coverage/scope percentage of 69 percent below was obtained by adding the cost of the components B and C, which according to the ICR (page 12) constituted the investment costs for the three subsectors, and dividing it by the project cost of US$23.02 million minus the counterpart funds of US$3.33 million. These counterpart funds were in form of tax reduction, road maintenance fund replenishment, and other exemption and were not figured into the economic cost of components B and C. Efficiency Rating Substantial Page 15 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 69.00 ICR Estimate  14.30  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome With highly relevant objective, and substantial efficacy and efficiency, the overall project outcome is rated satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The main risks to the development outcomes include the following:  Lack of approval of the legislative and other documents prepared under the project. The ICR (para 91) notes that on the legislative side, most of the key transport bills were approved but some remains in draft and risk not to be completed. Similarly, the national transport policy framework and strategy still need to be finalized and confirmed at ministerial level.  Loss of capacity in the MOIT. According to the ICR (para 86), capacity and coordination within the MOIT still need strengthening. The MOIT relies on a limited number of technical staff with inconsistent supervision and management, and half of the senior positions are still vacant. The Land Transport Department has a small number of engineers and trained technical staff and needs to ensure that many essential technical positions are filled. Its capacity could also be compromised by the retirement or resignation of senior staff.  Discontinuation of outsourcing practice to the private sector. The local private sector road maintenance and rehabilitation capacity can only be sustained if the MOIT continues to outsource road works. The ICR (para 85) notes that (i) the project encouraged the MOIT to do so, (ii) currently force account is only used for emergency works and there are plans to outsource even these works to contractors, and (iii) the Ministry is testing performance-based maintenance contracts. However, in the longer run, the continuation of outsourcing is entirely up to the MOIT and its strategy for road works and maintenance. Page 16 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931)  Insufficient equipment to adequately undertake routine and periodic road maintenance. Significant capital investments and operational and maintenance funds are needed to acquire the equipment for road works, such as for aggregate extraction, chip sealing, and bituminous asphalt sealing. Several contractors have invested in equipment for road maintenance, such as bitumen sprayers, aggregate spreaders and rubber-tyred rollers based on the MOIT’s plans to outsource road maintenance works. However, they expressed concerns that the government might not continue with the outsourcing practice. Other contractors held back from investing in equipment for the same reason (ICR, para 86).  Inadequate road maintenance funding. Maintenance is dependent upon the availability of funding. The allocations of resources to the road maintenance fund, even if they exceeded the respective indicator targets, corresponded to 43 percent of the overall maintenance needs only. The allocation is also dependent upon the annual budget negotiation process.  Inadequate maintenance of navigation and safety equipment and works in the maritime sector. Since the condition of ferry terminals was poor at appraisal because of missing equipment and disrepair, there is no certainty that the project equipment and works will adequately be taken care off. 8. Assessment of Bank Performance a. Quality-at-Entry The Bank performance at entry is rated satisfactory. The Bank team soundly grounded the original project design in the 2005 Tonga Transport Sector Review and the government's initial institutional reform. The Bank team set realistic objectives and adopted a straightforward project design with a reasonable balance between TA activities and investments. Adequate attention was given to safeguards and fiduciary issues. The Bank team also carried out a comprehensive risk assessment. The implementation arrangements with the MOIT and Tonga Aviation Limited as implementation agencies reflected the institutional configuration at that time. Due to an oversight, Tonga Aviation Limited was not part of the project steering committee, which negatively affected communication during implementation. The original M&E design had shortcoming (see section 9). The Bank team prepared two AFs, which were based on analytical work prepared during project implementation and corresponded to real needs in the country. The Bank team carried out the necessary technical, fiduciary, social, and environmental evaluations, including a change in the environmental assessment classification, for the AFs, made minor adjustments, and upgraded the risk rating. The Bank team also adjusted the original M&E design in a largely satisfactory way. Quality-at-Entry Rating Satisfactory Page 17 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) b. Quality of supervision The Bank performance during supervision is rated satisfactory. According to the ICR (para 79) the Recipient showed high appreciation for the Bank team's strong presence and the diligent supervision support. The Bank team was candid in its project ratings, using them to signal the main project problems and issues. The Bank team showed flexibility in adapting the project design to new requirements and the availability of additional funding resources. Instead of preparing a new project, they selected the more efficient AF approach to use the PRIF grant. The Bank team closely monitored the safeguards and fiduciary performance, among others, identifying a case of misprocurement (see section 10 below). The Bank team also regularly monitored project outputs and results, carrying out impact assessments of specific activities, such as on the change management process support. Finally, the Bank team played an important role in the coordination with counterparts and donors to ensure the complementarity of activities and avoid duplications. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The design of the original outcome indicators, baselines, and targets was imperfect and confusing. On page 23 of the PAD, four project outcome indictors are listed, but they have no baselines and end targets. These indicators are different from the single project outcome indicator listed on page 26. The latter has a baseline and an end target, but the end target is identical to the indicator itself. In addition, the indicator was nearly identical to the PDO statement and hence not adequate to measure the achievement of the PDO. The 2010 AF and restructuring revised the indicators from page 23 of the PAD without changing their substance and added two new indicators to capture the additional project activities. It also devised baselines and end targets for all indicators. The 2013 restructuring added the IDA core indicators. With the 2015 AF and restructuring, the existing system of indicators was further finetuned to enhance measurability without changing their substance. New indicators were added to capture the increased ambition of the project. This left a total of 12 PDO level indicators, three of which were core indicators, and three intermediate results indicators. Not all of the PDO level indicators were outcome indicators. However, overall the indicators were adequate to capture the project outcomes. According to the PAD (page 24), the MOIT and the project support team were responsible for M&E. The ICR (para 60) notes that the M&E arrangements relied on the following mechanisms and tools: (i) general Page 18 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) follow up by the MOIT through the project support team with continuous data collection and periodic consolidation of the information from the subsectors on the basis of a planned work program and disbursement schedule, (ii) improvement to the data collection system, (iii) strategic monitoring by the project steering committee, and (iv) Bank supervision missions and annual review meetings to monitor progress, compliance with schedules, and the need to take actions to meet project goals. b. M&E Implementation M&E data was collected and shared through quarterly progress reports submitted on time. This data was also reported in the aide memoires. The project steering committee met at least twice a year to assess the progress in project implementation. In addition to these monitoring tools, the Bank team also carried out impact assessments of specific project activities and used a detailed monitoring table to track the progress in the implementation of the recommendations of the change management process. c. M&E Utilization The Government of Tonga and the Bank team used the M&E data on project performance and results to inform project management and decision-making. The ICR (para 63) notes that the M&E data enabled the Bank's and the government's project support teams to assess the progress towards achieving the indicator targets and PDOs and detect areas that needed additional implementation support. They also showed to some extent the different levels of performance of the transport subsectors. As an example, the ICR (para 64) mentions that the teams detected the high staff turnover in the MOIT at senior and managerial levels through the indicator on staffing recruitment and appropriateness. The M&E data was also used as basis for the AFs and project restructurings. The Government of Tonga, the Bank team and the Australian Government, as co-financier, conducted several sector reviews and technical audits during project implementation. These provided the basis for constant readjustments, extensions, and expansion of activities. For instance, in 2016, the Government of Tonga used project funds to carry out a Transport Sector Review to assess the achievements of the road sector reform process and the investments carried out and derive recommendations to further support the project and the MOIT’s objectives and long-term outcome (ICR, para 67). M&E Quality Rating Substantial 10. Other Issues a. Safeguards The original project was classified as category C for environmental assessment purposes because it was expected to have minimal or no adverse environmental impacts. The following safeguards policy was Page 19 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) triggered: environmental assessment OP/BP4.01 (PAD, para 41). Through the 2010 AF, the project was reclassified as category B because of the inclusion of road maintenance and rehabilitation. The Recipient prepared an Environmental Management Plan in 2010 (2010 PP, para 27). According to the ICR (para 69), the project followed World Bank safeguards policies and procedures, no safeguards incident or accident occurred, and all the required and appropriate environmental safeguard documents were developed, published, and duly applied. The Environmental Management Plan was integrated into the technical designs and civil works contracts, with an explicit requirement for the contractors to prepare their environmental management plans. The training of domestic contractors in road maintenance works included environmental safeguards modules. The Bank team encountered the use of heterogeneous environmental standards in the operation of quarries during site visits in 2017. As a result, the MOIT made sure that the same safety and environmental standards the World Bank helped enforce in the Ahononou quarry were applied elsewhere, liaising with the other relevant line ministries, such as Labor and Land, Surveys and Natural Resources. The project was not expected to require land acquisitions or resettlements. The ICR (para 70) notes that no activity that required land acquisition or involuntary resettlement took place. It points out that the project applied strong, effective, and inclusive communication/consultation processes, such as in the case of the institutional and legislative reform work and the maritime safety and road safety campaigns. The project had a simple grievance redress mechanism, with complaints to be sent to the general e-mail addresses of the MOIT and the World Bank. The ICR (para 70) mentions one complaint related to the Ahononou Quarry, which was resolved. This complaint went to the Tonga Court of Appeal in March 2017 and was then withdrawn by written consent of all parties. b. Fiduciary Compliance Financial management: According to the ICR (para 73), financial management compliance was generally satisfactory. A review of the financial management arrangements in 2012 highlighted the need for better monitoring of contract payments and for a system and accounting software to monitor financial commitments against disbursements and contract payments. The project financed an asset register system that improved monitoring and a procurement administration system for planning, monitoring, and reporting. IFRs and financial audits were presented on time. The task team confirmed that all financial audits had unqualified opinions. During the last year of project implementation minor shortcoming in signing financial and other documents were identified. Procurement: In terms of procurement, the ICR (paras 71 and 81) reports a case of misprocurement in a US$990,000 contract procured by Tonga Airport Limited. In this case, the winning bid contained a material deviation from the terms and conditions of the contract, which was inconsistent with the bidding documents and not mentioned in the bid evaluation report submitted to the Bank team. The evaluation of the case showed an oversight during bid evaluation as a result of the limited procurement capacity of Tonga Airport Limited. Therefore, on an exceptional basis, the respective grant funds were not cancelled. Page 20 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The following are lessons from the ICR with adaptions in language based on the discussions with the task team:  The promotion and use of local resources for road maintenance requires adequate policies to manage these resources. The project made it a priority to use and promote local resources. This included (i) the training and use of the domestic industry for road maintenance and quarry operations, (ii) the adaption of tools to the local context, such a simple and practical road asset management system, and (iii) the use of local materials, such as introducing Otta seals which were most appropriate given local aggregate supplies. This created employment and business opportunities, affordability of road maintenance, and improvement in accountability, transparency, environmental and social safeguards considerations. However, the policies to better manage these local resources still need to be established. For example, high-quality materials should be restricted for use in asphalt concrete for roads and cement concrete for buildings and structures. In addition, there should be a regulation that encourages a more extensive use of Otta seals.  For countries exposed to frequent natural disasters, a zero-budget disaster recovery contingency funds component can help mobilize resources for reconstruction quickly without delaying project implementation. Under this project, when Tropical Cyclone Ian devastated part of the country in 2014, the project had to be restructured to broaden its geographical scope to carry out aviation investments in the affected areas. This delayed project implementation and the disaster response.  Similarly, for such countries, to incorporate climate resilience in civil works and maintenance of roads is essential. During project implementation, the country was hit twice by cyclone. This made it evident that, given the climate risks Tonga faces, mitigation/adaptation measures should be considered to ensure the planning and Page 21 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) sustainability of civil works and maintenance services in the long term. Climate mitigation/adaptation were not aimed at when the project was prepared.  Analytical work, technical studies, and additional audits can provide an anchor for the reform process and shield it from political interference and changes in leadership. When the Ministers changed, it frequently happened that the new Minster wanted to change the course of the reform. In such cases, additional studies or audits of the activities already carried out helped take advantage of the reform principles but realign the reform process. For instance, when the Minister wanted to return to in-house road maintenance, an additional study helped to convince him about the advantage of outsourcing to the private sector.  Focusing on rural roads should be balanced with attention to urban roads needs. Most road investments under the project focused on rural roads, but with increasing urban traffic and population expansion, it became clear that urban roads planning and maintenance need adequate attention, too. Areas where more work is likely needed are (i) physical investments for pedestrian mobility, including sidewalks and safe crossings with many co-benefits linked to climate change, disability and health, and (ii) the drainage of roads in urban areas. Surface flooding (including over footpaths) because of blocked drainage was an issue. This area of work would require clear division of roles and responsibilities between the central level and the municipality level.  Training and strong Bank support are needed in low capacity countries, especially if the implementation agencies are not familiar with Bank rules and procedures. In this case, the misprocurement under a contract bid by Tonga Airport Limited was caused by their lack of familiarity with Bank rules and procedures. More training right from the beginning, the support from an experienced procurement specialist, and more hand holding by the Bank team might have avoided it. 13. Assessment Recommended? Yes ASSESSMENT_TABLE Please Explain According to the ICRR, this project has successfully transformed the transport sector in Tonga, laying the basis for increased safety and security, especially in the air and maritime sectors, and creating a road maintenance capacity and culture in a small, fragile, and low capacity country. It would be important to see if the reforms and changes stand the test of time. If yes, valuable lessons could be drawn for many other countries. 14. Comments on Quality of ICR The ICR is clear, comprehensive, and well written. It candidly presents problems and shortcomings. The ICR gives a detailed picture of the project implementation experience and results. It contains many summary tables, Page 22 of 23 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review TO-Transport Sector Consolidation (P096931) which are essential to understand this complex project, which had two major AFs. It also contains pictures, which show the key transformations that occurred under the project. The ICR clearly identifies the project's theory of change and uses it as a guidance in the assessment of the project results, which is outcome oriented. The CBAs are adequately done. However, although the lessons section in the ICR is extensive, its quality is weak. Several lessons combine different project experiences, but the lesson only refers to one of them. A few lessons are not underpinned by the evidence in the ICR. The task team noted that management asked them to write the lessons in form of recommendations for other countries in similar situations. The ICR is also provides limited information on the status of the different draft legislative measures produced under the project. a. Quality of ICR Rating Substantial Page 23 of 23