REPUBLIC OF NORTH MACEDONIA Regional and Local Roads Program Support Project Report No. 132581 DECEMBER 20, 2018 © 2019 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: The World Bank does not guarantee the World Bank. 2019. Republic of North accuracy of the data included in this work. The Any queries on rights and licenses, including Macedonia—Regional and Local Roads Program boundaries, colors, denominations, and other subsidiary rights, should be addressed to Support Project. Independent Evaluation information shown on any map in this work do World Bank Publications, The World Bank Group, Project Performance Assessment Report not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC 132581 Washington, DC: World Bank. World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No: 132581 PROJECT PERFORMANCE ASSESSMENT REPORT REPUBLIC OF NORTH MACEDONIA REGIONAL AND LOCAL ROADS PROGRAM SUPPORT PROJECT (IDA CREDIT NO. 75320) December 28, 2018 Financial, Private Sector, and Sustainable Development Independent Evaluation Group This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without authorization. Currency Equivalents (annual averages) Currency Unit = Macedonian Denar (MKD) 2008 $1.00 43.2 MKD 2009 $1.00 42.6 MKD 2010 $1.00 46.9 MKD 2011 $1.00 47.7 MKD 2012 $1.00 47.2 MKD 2013 $1.00 44.6 MKD 2014 $1.00 50.6 MKD 2015 $1.00 56.5 MKD Abbreviations ASR Agency for State Roads CPS Country Partnership Strategy EIRR economic internal rate of reform EU European Union ICR Implementation Completion and Results Report IEG Independent Evaluation Group M&E monitoring and evaluation OP operational policy PDO project development objective PPAR Project Performance Assessment Report PESR Public Enterprise for State Roads ZELS Association of the Units of Local Self-Government (Republic of North Macedonia) All dollar amounts are U.S. dollars unless otherwise indicated. Fiscal Year Government: January 1 – December 31 Director-General, Independent Evaluation Ms. Sophie Sirtaine (acting) Director, Financial, Private Sector, and Sustainable Development Mr. José Carbajo Martínez Manager, Sustainable Development Ms. Midori Makino Task Manager Mr. Ramachandra Jammi ii Contents Preface ................................................................................................................................................................vii Summary .......................................................................................................................................................... viii Lessons ................................................................................................................................................................. x 1. Background and Context ..........................................................................................................................1 Role of the World Bank ..............................................................................................................................2 2. Relevance of the Objectives and Design.............................................................................................3 Objectives .......................................................................................................................................................3 Relevance of the Objectives .....................................................................................................................5 Relevance of the Design ............................................................................................................................6 Monitoring and Evaluation ....................................................................................................................... 7 3. Implementation ............................................................................................................................................9 4. Achievement of the Objectives ............................................................................................................ 12 Original Objective ...................................................................................................................................... 12 Outputs ........................................................................................................................................................................ 13 Outcomes ................................................................................................................................................................... 14 Revised Objective....................................................................................................................................... 18 Outputs ........................................................................................................................................................................ 18 Outcomes .................................................................................................................................................................. 20 5. Efficiency ...................................................................................................................................................... 22 6. Ratings .......................................................................................................................................................... 23 Outcome ...................................................................................................................................................... 23 Risk to Development Outcome ............................................................................................................ 23 World Bank Performance ....................................................................................................................... 24 Quality of Supervision ............................................................................................................................. 25 Borrower Performance ............................................................................................................................ 26 Government Performance.................................................................................................................................... 26 Implementing Agency Performance................................................................................................................. 26 7. Lessons.......................................................................................................................................................... 28 Bibliography..................................................................................................................................................... 30 iii iv Boxes Box 4.1. Results from the Beneficiary Survey on Local Roads ........................................................... 16 Tables Table 1.1. Road Network Condition, 2013 ..................................................................................................2 Table 4.1. Illustrative Data from the Road Asset Management System Road Network (km) .. 19 Appendixes Appendix A. Basic Data Sheet ................................................................................................................... 31 Appendix B. Project Cost and Financing ............................................................................................... 34 Appendix C. Road Roughness Index for Rehabilitated Regional Roads ................................... 35 Appendix D. Observations from Road Section Visits ....................................................................... 36 Appendix E. Breakdown of Municipality Budget Allocation from Central Budget, 2018 ... 41 Appendix F. List of Persons Met............................................................................................................... 43 Appendix G. Borrower Comments .......................................................................................................... 44 This report was prepared by Ebru Karamete, Ramachandra Jammi, and Nol Binakaj, who assessed the project in July 2018. The report was panel reviewed by George T. Keith Pitman and peer reviewed by Elisabeth Goller. Richard Kraus provided administrative support. Principal Ratings Indicator ICR ICR Review PPAR Outcome Moderately satisfactory Satisfactory Satisfactory Risk to development Substantial Substantial Substantial outcome Bank performance Satisfactory Satisfactory Moderately Satisfactory Borrower performance Satisfactory Satisfactory Satisfactory Note: The Implementation Completion and Results Report (ICR) is a self-evaluation by the responsible Global Practice. The ICR Review is an intermediate Independent Evaluation Group product that seeks to independently validate the findings of the ICR. PPAR = Project Performance Assessment Report. Key Staff Responsible Task Manager or Division Chief or Project Leader Sector Director Country Director Appraisal Andreas Schliessler Motoo Konishi Jane Armitage Completion Liljana Sekerinska Pierre Guislain Ellen. E. Goldstein v IEG Mission: Improving World Bank Group development results through excellence in independent evaluation. About This Report The Independent Evaluation Group (IEG) assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the World Bank’s self-evaluation process and to verify that the World Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20–25 percent of the World Bank’s lending operations through fieldwork. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or World Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, interview World Bank staff and other donor agency staff both at headquarters and in local offices as appropriate, and apply other evaluative methods as needed. Each PPAR is subject to technical peer review, internal IEG panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible World Bank Country Management Unit. The PPAR is also sent to the borrower for review. IEG incorporates both World Bank and borrower comments as appropriate, and the borrowers’ comments are attached to the document that is sent to the World Bank’s Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://ieg.worldbankgroup.org). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current World Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, country assistance strategies, sector strategy papers, and operational policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared with alternatives. The efficiency dimension is not applied to development policy operations, which provide general budget support. Possible ratings for outcome: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, highly unsatisfactory. Risk to development outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for risk to development outcome: high, significant, moderate, negligible to low, and not evaluable. Bank performance: The extent to which services provided by the World Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan or credit closing, toward the achievement of development outcomes). The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. Borrower performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for borrower performance: highly satisfactory, satisfactory, moderately satisfactory, moderately unsatisfactory, unsatisfactory, and highly unsatisfactory. vi Preface This is a Project Performance Assessment Report (PPAR) prepared by the Independent Evaluation Group (IEG) of the World Bank Group on the Regional and Local Roads Program Support project in the Republic of North Macedonia (P107840). The project was approved on May 13, 2008, for a total cost of $112.3 million, which was supported by a World Bank loan of $105.2 million. The project cost at completion was $112.3 million, of which $91.3 million was financed by the World Bank. The project closed on December 31, 2015, two years and five months later than originally scheduled. The PPAR provides insights into promoting access and reducing the cost of access to basic services and economic markets through road works. It contributes to the Independent Evaluation Group (IEG) strategic evaluation area of inclusive and sustainable economic growth. The PPAR also adds to the evidence base for a potential future assessment of the World Bank’s support for rural roads across client countries. The assessment is based on a review of relevant documentation, interviews with World Bank staff at headquarters and in the country office, and the findings of an IEG mission that visited North Macedonia during July 2018. Project performance was discussed in interviews with officials of the local and central government, the implementing agency Public Enterprise for State Roads, the North Macedonian Chamber of Commerce, the United Nations Development Program, local contractors, and staff of the World Bank’s country office. Appendix F lists the persons met during the mission. Their cooperation and assistance in preparing the report are gratefully acknowledged. As per standard IEG procedures, a copy of the draft PPAR was sent to government officials and implementing agencies for their review and comments received are attached in Appendix G. vii Summary This PPAR assesses the development effectiveness of the Regional and Local Roads Program Support project in North Macedonia, which was approved in 2008. The original development objective of the project, “to reduce cost of access to markets and services for communities served by regional and local roads,” was revised through a level I restructuring in 2013 “to reduce the cost of safe access to markets and services for communities served by regional and local roads in North Macedonia’s territory, and to improve institutional capacity for investment planning and road safety.” The revised objective thus introduced the element of road safety to access, as well as institutional capacity for investment planning and road safety. Because North Macedonia is a landlocked country with a trade-dependent small economy, its transport sector development emphasis has been to ensure good connectivity to neighboring countries to promote exports and foreign direct investment. After the 2008 financial crisis, the two Trans-European Network transport corridors, VIII and X, received a large share of the national and donor or international financial institutions’ investment resources to build and rehabilitate infrastructure. The North Macedonian government also invested significant resources into other national (that is, primary) roads. However, secondary and local roads, which totaled 13,029 kilometers, received less attention—mainly funding of routine and some limited periodic maintenance. As a consequence, the roads’ overall condition had slowly deteriorated over time. This in turn affected poor households’ and other socially vulnerable groups’ access to markets and services. At project preparation, approximately 33 percent of the regional and local road network was in good condition, and 42 percent and 27 percent were in fair and poor condition, respectively. Through improving regional and local roads, this project sought to improve access to services and promote wider economic development. The Ministry of Transport and Communications oversees policy and strategic guidance in the roads sector. Road sector management was carried out by the Agency for State Roads (ASR), which was transformed in 2013 into the Public Enterprise for State Roads (PESR), a managerially and financially independent entity mandated to plan, construct, reconstruct, and rehabilitate the national and regional roads, collect tolls, and prepare road development and financing plans. Municipalities are responsible for the local road network in their jurisdictions. viii Project Performance The relevance of both the original and revised project development objectives is rated high because of their alignment with the country’s and the World Bank’s strategies and priorities at project preparation and project closure. The project development objectives remained relevant to the two Country Partnership Frameworks covering FY11–18. The growth and competitiveness pillar of the latest Country Partnership Framework recognizes that improving road infrastructure is a key government priority for overcoming the disadvantages of a small landlocked economy. The relevance of the original and revised project objectives is rated modest and substantial, respectively. The original project objective was expected to be achieved through the rehabilitation and periodic maintenance of selected regional and local roads, along with institutional support to strengthen the government’s capacity to manage and maintain the road network. However, a lack of emphasis on building the capacity of municipalities (which were responsible for managing local roads) was a significant shortcoming. The revised project objective was therefore more in line with the added focus on road safety and institutional capacity for investment planning. It included new road safety activities, improvement for regional roads, and an asset management system that would improve prioritization and planning of road management activities. The achievement of the original project development objective—to reduce the cost of access to markets and services for communities served by regional and local roads—is rated substantial. The targeted reduction in the cost of access to markets and services (as measured by vehicle operating expenses on roads improved by the project) was achieved; costs decreased by 11 percent, marginally above the planned 10 percent. The achievement of the revised project development objective is rated modest. The first part of the revised objective was the same as the original objective, and therefore the same results apply. However, there were shortcomings regarding the second part of the revised objective, improving institutional capacity for investment planning and road safety. Although the PESR’s capacity was strengthened regarding the deployment of the Road Asset Management System and its use to produce a five-year rolling strategic program, these results have yet to be incorporated into the planning and decision- making process for road management. In addition, there was limited evidence of progress in improving road safety. Efficiency is rated substantial for each of the original and revised project objectives, based on favorable economic rates of return, while there were some operational inefficiencies that were caused by lack of detailed designs for road works and initial staff capacity constraints of the implementing agency. ix The outcomes for the original objectives are rated satisfactory and the revised project objectives are rated moderately satisfactory. The overall project outcome is rated satisfactory, based on a disbursement-weighted average of the outcome ratings for the original and the revised objectives. Risk to development outcome is rated substantial. A major risk to the development outcome of the project is the uncertainty surrounding the ongoing provision of resources and the capacity for road maintenance. The necessary political will and financial resources to fund these activities appear to be uncertain. Formal institutional arrangements have yet to be consolidated to ensure adequate budget allocations for maintenance. As well, municipalities still need support to improve their capacity to manage and maintain their local road networks. The ongoing World Bank–financed National and Regional Roads Rehabilitation Project (P148023; FY14–19) is expanding the use of the Road Asset Management System tool and contributing to PESR's financial and management capacity for maintenance planning and implementation. Bank performance is rated moderately satisfactory, mainly due to some shortcomings in quality at entry, including the lack of detailed designs for regional roads at project effectiveness, which led to a prolonged implementation phase. Quality of supervision is rated satisfactory; the project team provided timely support and strong technical guidance during project implementation. Borrower performance is rated satisfactory. The government and PESR demonstrated commitment to the project through relevant sector reform and by following the project team’s recommendations on addressing implementation issues in a timely manner. The government complied with all the agreed legal and financial covenants. The performance of the Agency for State Roads and the PESR in procurement, monitoring, and evaluation was adequate. Lessons • Objective criteria developed and applied in a participatory manner can support a transparent framework to allocate investments and maintenance funds in the roads sector. This project developed a multicriteria framework (including length of the road segments, number of registered vehicles, fuel consumption, surface size, and population) and applied it in partnership with municipalities and local communities. The framework was later adapted into a law to distribute the central budget for road works among municipalities. • The decentralization of responsibilities to local governments needs to be accompanied by the availability of commensurate resources and capacity building. The responsibility of managing and maintaining local road networks was devolved to municipalities through territorial reforms in North Macedonia beginning in the early 2000s. However, with limited funds and limited capacity, municipalities continue to face challenges carrying out their mandate. • Road safety and road design elements need to be jointly integrated into the project design and monitoring framework to mitigate risks to the effectiveness of road projects. Under this project, road safety aspects were not consistently followed initially but were introduced only after project restructuring. A better approach is to include road safety elements in the project development objective and then integrate them into the project design and monitoring framework. • Road project appraisal requires sufficient time and technical due diligence to ensure effective and timely project implementation. The shortcomings experienced in quality at entry relating to the lack of detailed designs at project commencement, and inadequate provision of capacity building for municipalities, might have been overcome if greater time and technical due diligence had been committed to these matters at project appraisal. José Carbajo Martínez Director, Financial, Private Sector, and Sustainable Development xi 1. Background and Context 1.1 Because the Republic of North Macedonia is a landlocked country with a trade- dependent small economy, its transport sector development emphasis has been on ensuring good connectivity to neighboring countries to promote exports and foreign direct investment. After the 2008 financial crisis, the two Trans-European Network transport corridors, VIII and X, received a large share of the national and donor or international financial institutions’ investment resources to build and rehabilitate infrastructure. North Macedonia government also invested significant resources into other national (that is, primary) roads. 1.2 An enhanced transport network contributes to alleviating poverty in North Macedonia by providing better access to national and international markets for the rural population.1 Thus, a government priority has been to upgrade and rehabilitate road infrastructure to improve prospects for economic growth. At project appraisal, approximately 40 percent of the population in North Macedonia lived in rural areas, of which two-thirds were classified as poor. Most rural households depend largely on crop and livestock production for their income. 1.3 Roads in North Macedonia, including motorways, are categorized as follows: (i) national roads, which primarily connect to neighboring countries but also to the largest regional centers in North Macedonia; (ii) regional roads, which connect two or more municipalities and secure critical in-country connectivity; and (iii) local roads, which serve municipal traffic. North Macedonia’s road network comprises a total length of 14,182 kilometers, including 242 kilometers of motorways, 911 kilometers of national roads, 3,771 kilometers of regional roads, and 9,258 kilometers of local roads. 1.4 The Ministry of Transport and Communications oversees roads sector strategic guidance and policy. As of 2013, management of national and regional roads has been entrusted to the managerially and financially independent Public Enterprise for State Roads (PESR),2 which is mandated to plan, construct, reconstruct, and rehabilitate the national and regional roads and to collect tolls. Local roads are entrusted to municipalities. National and regional roads receive funding from the state budget; these funds are planned and implemented by the Fund for National and Regional Roads (FNRR). Local roads management are financed by municipalities, which set aside funds within their budgets for this purpose. In addition, on a yearly basis all municipalities receive a transfer from the state budget specifically for the maintenance of local roads. 1.5 Road maintenance is carried out by the public enterprise Makedonija Pat, which operates as PESR’s direct contractor. Efforts to restructure the institutional setup of road maintenance and introduce open competition in maintenance are currently ongoing. 1 Steps toward the full commercialization of the maintenance sector have been taken through PESR-financed rehabilitation programs since 2008; these include a significant rehabilitation of regional roads that has been carried out by the private sector. 1.6 At project preparation, attention to regional and local roads was mainly for routine and some limited periodic maintenance, and their overall condition had slowly deteriorated over time. Approximately one-third of the regional and local road network (totaling 13,029 kilometers) was in good condition, with 42 percent in fair condition and 27 percent in poor condition (table 1.1). The improvement of these roads was required to improve access to markets and services for the poor and for other socially vulnerable groups, stimulate local economic development, and sustain rural communities and smaller towns. Table 1.1. Road Network Condition, 2013 Good or Fair Main Road Length Paved Paved Condition Network (km) (km) (percent) (percent) National roads and 1,112 945 85 91 motorways Regional roads 3,721 3,021 81 75 R1 regional roads 2,041 1,889 93 82 R2a regional roads 1,680 1,131 67 68 Total 4,833 3,966 82 Source: PESR 2013. Note: This report is produced by the Road Asset Management System developed via the project support and does not cover local roads. a. R2 regional roads are narrower in general with less traffic compared with R1 regional roads. Role of the World Bank 1.7 The World Bank has four other investment operations that deal directly with the roads sector in North Macedonia or have related elements (table 1.2). The Regional and Local Roads Program Support Project (P107840) attempts to reduce the cost of safe access to markets and services for communities served by regional and local roads in the country and improve institutional capacity for investment planning and road safety. The Road Upgrading and Development Project (P149955) focuses on improving transport connectivity for road users along corridor VIII, and on improving the asset management and planning functions of PESR. 1.8 The Second Municipal Services Improvement Project (P154464) will help improve transparency, financial sustainability, and inclusive delivery of targeted 2 municipal services in the participating municipalities. The Second Trade and Transport Facilitation Project (P091723) facilitates the movement of trade between the borrower and neighboring countries in Southeast Europe through the removal of selected border- zone infrastructure bottlenecks and by improving the efficiency and quality of road and rail services. Table 1.2. World Bank Investment Projects in the Roads Sector and Related Areas World Bank Commitment Project Title and Identification Number Duration ($, millions) Road Upgrading and Development Project (P149955) FY16–20 90.95 Second Municipal Services Improvement Project (P154464) FY16–21 28.04 Second Trade and Transport Facilitation Project (P091723) FY07–12 20.00 National and Regional Roads Rehabilitation (P148023) FY15–19 70.98 Note: FY = fiscal year. 1.9 The World Bank joined the West Balkans Investment Framework after its formation in 2009. The framework blends grants and loans to expedite priority investments in key sectors of borrowing countries. Because the World Bank does not have full member status with the framework, it does not have access to the 20 percent grant to enhance its lending to borrowing countries under the framework. This puts the World Bank at a disadvantage in terms of offering more attractive lending terms to borrowing countries such as North Macedonia. 2. Relevance of the Objectives and Design Objectives 2.1 The original project development objective (PDO) was “to reduce cost of access to markets and services for communities served by regional and local roads” (World Bank 2008c, page 5). The objective was revised as part of the restructuring in 2013 to “reduce the cost of safe access to markets and services for communities served by regional and local roads in the Guarantor's territory, and to improve institutional capacity for investment planning and road safety” (World Bank 2013b, page12). The revised objective thus introduced the element of road safety to access, as well as institutional capacity for investment planning and road safety. 2.2 The project included three components: 1. Rehabilitation and periodic maintenance of regional roads (appraisal cost: $50.5 million; actual cost: $46.1 million). Provision of financing to cover the rehabilitation and maintenance of about 330 kilometers of paved road sections identified through a 3 European Commission–financed study, representing approximately 8.7 percent of all regional roads, between 2008 and 2012. During the level 1 restructuring in 2013, the number of kilometers of the regional roads to be financed was reduced from 330 to 284; this new figure reflected the cost of the actual work completed and some road safety activities that were added, including the installation of traffic signs, road markings, and guardrails; the rehabilitation of regional roads prone to landslides; and the repair of bridges (World Bank 2013a). 2. Rehabilitation and periodic maintenance of local roads (appraisal cost: $50.5 million; actual cost: $42.8 million). Provide funding for the rehabilitation and maintenance of approximately 420 kilometers of local roads (5 percent of all local roads) selected by municipalities over the 2008–12 period, including the preparation of bidding documents through provision of technical assistance. 3. Institutional support (appraisal cost: $3.0 million; actual cost: $2.3 million). Strengthen the government’s capacity to manage and maintain the country’ road network through the FNRR and through providing various types of institutional support, including (i) an institutional analysis to identify weaknesses in the FNRR and define actions to overcome the weaknesses; (ii) supporting the implementation of the new road law and the National Road Transport Strategy; (iii) supporting financial and technical audits; (iv) providing training and office and information technology equipment; and (v) conducting any other activities that may be identified later. During the level I restructuring in 2013, component 3’s description was adjusted to reflect a new lender-to-borrower relationship and specific new activities. The description now reads: “Strengthening the Guarantor's and the Borrower's capacity to manage and maintain the road network through the provision of: (i) advisory services to the Borrower and the Ministry of Transport and Communications; (ii) training staff of the Borrower and the [Ministry of Transport and Communications]; and (iii) office and information and technology equipment, and vehicles” (World Bank 2013b, page 12). 2.3 Financing and duration dates. The project was financed by an International Bank for Reconstruction and Development loan of $105.2 million. No government funds were envisaged at appraisal, and the actual contribution was $16.7 million from the government through a value-added tax exemption. The World Bank loan of $91.3 million was 98.75 percent disbursed. The project was approved on May 13, 2008 and closed on December 31, 2015. The closing dates were extended twice, bringing the total extension period to 29 months and the total implementation period to seven years and five months. 4 2.4 Restructuring. The project went through two restructurings: 1. Level I: In December 2013, the government adopted changes to the Law on Public Roads that transformed the Agency for State Roads (ASR) into PESR, provided the latter with greater autonomy, and separated it from the central budget, thus removing the cap on disbursements from the loan and facilitating speedy implementation. PESR was designated the new borrower, with the government now acting as the guarantor for the loan. This change necessitated a level 1 restructuring. The World Bank confirmed PESR’s ability and capacity to implement the project subject to compliance with strict financial and debt indicators that were introduced into the project’s monitoring and evaluation (M&E) system as financial covenants. No changes were made to the safeguards and procurement arrangements. As well, it was confirmed that PESR would able to service its debts, including the International Bank for Reconstruction and Development loan, and would be subject to close monitoring of its financial and debt indicators. Some project performance indicators were added, and some dropped to reflect the new legal and institutional arrangements. The PDO was revised to reflect the newly identified activities on road safety and investment planning capacity. Because of implementation delays, the loan’s closing date was extended from July 31, 2013, to July 31, 2015. 2. Level II: On April 9, 2015, a level 2 restructuring was conducted to extend the loan closing date again, from July 31, 2015, to December 31, 2015. The extension was needed to complete the civil works, which had been delayed because of unusually heavy rainfall in the previous years as well as delays due to problems with designs. Relevance of the Objectives 2.5 At appraisal, and at present, the PDOs (both original and revised) were highly relevant to the country strategy for developing its transport sector, as well as to the World Bank’s country strategies. 2.6 North Macedonia’s National Transport Strategy, which was adopted in 2007, confirmed the need for investments in roads to ensure better communication among the regional centers within the country, as one of the main tools to promote North Macedonia’s competitiveness in international markets and to support harmonious development of the country as whole. The strategy stipulated that investments should focus on the maintenance and preservation of and repairs to existing roads and on enhancing the functionality of the existing road network, with a very limited expansion of that network. 5 2.7 The Country Partnership Strategy (CPS) for FY07–10 included support for road infrastructure through the implementation of a program to stop the degradation of the “lower-level” (that is, the secondary and tertiary level) road network. The Feeder Roads Project was one key activity; thus, preparation of the project was in direct response to the CPS. The PDOs supported two pillars of the CPS: (i) fostering economic growth, creating jobs, and improving the living standards of all North Macedonians; and (ii) improving the governance and transparency of public service delivery to support the market economy. The CPS noted that structural reforms and institutional strengthening were needed to improve efficiency in transport asset conservation; this was based on well-informed programming and budgeting of maintenance needs. 2.8 From 2014 to 2018, the government’s program focused on improving the living standards of North Macedonians through better road and utility infrastructure. The rehabilitation of regional and local roads was a key part of this program. The revised PDO’s emphasis on improved institutional capacity for investment planning was also consistent, as it included a focus on transparent and efficient work of the government and public administration. 2.9 The PDO remained highly relevant to the ensuing CPS for FY15-18. The Growth and Competitiveness pillar of the later CPS recognized that better road infrastructure was key to overcoming the disadvantages of a small landlocked economy and this was still the Government’s top priority. The Government has had a new ambitious program of investments in the sector (i.e. construction of new national, regional and local roads and their rehabilitation), and the design and implementation of this program require not only financial resources but also expertise on strategic areas such as road safety, climate resilience and evidence-based investment policy. Thus, the revised objectives were aligned with all these three strategic areas. 2.10 The relevance of original and revised objectives is rated high. Relevance of the Design 2.11 Under the original PDO, activities focused on rehabilitation and periodic maintenance of regional and local roads; these were mostly civil works related to localized repairs, replacement asphalt resurfacing or regravelling, and upgrades. This work was complemented by institutional support for strengthening the government’s capacity to manage and maintain selected portions of the road network, to be identified through a European Commission–financed study and by municipalities through a participatory approach. These activities were meant to contribute to improved conditions and quality of the targeted regional and local roads, which would, in turn, 6 lead to a reduction in vehicle operating costs, thereby reducing the cost of access to markets and services for communities served by these regional and local roads. 2.12 The revised PDO’s activities focused on road safety and the institutional capacity for investment planning. After the project’s restructuring, the slightly revised project design introduced some new activities on road safety improvement for regional roads and on the establishment of an asset management system that would contribute to improved investment planning capacity (World Bank 2013b). However, the design did not include any specific activities on road safety improvements for local roads or on capacity building for road safety. The restructured project included road safety elements for regional roads only (which accounted for approximately 40 percent of the road works’ total length). The follow-up project, the National and Regional Roads Rehabilitation (P148023; FY15–20) is supporting PESR and the National Council for Road Safety to improve the safety elements of all road infrastructure in the country. 2.13 One important shortcoming of the project’s design was the lack of a specific provision for institutional capacity building in the country’s municipalities. In retrospect, given that the originally planned length of rehabilitation works for local roads under the project far exceeded the planned length for the regional roads, this provision should have been included. Municipalities were responsible for preparing the road works’ designs, but capacity building was primarily carried out with the support of PESR. The participation of municipalities and local communities in supervision was limited to a written opinion issued by the municipality at the completion of the road works. As well, no institutional strengthening was envisaged to improve their capacity to carry out road rehabilitation works. This was a significant weakness of the project’s design. 2.14 M&E design included a set of sound indicators that were measurable and relevant to the PDO but could have been enhanced by including indicators for tracking maintenance on rehabilitated roads and estimating beneficiaries from the road improvements (see the Monitoring and Evaluation section). 2.15 Because of these shortcomings, the original design’s relevance is rated modest, and the revised design’s relevance is rated substantial. Monitoring and Evaluation 2.16 Design. The implementing agency PESR (formerly ASR) used an existing M&E system to monitor regular activities related to national and regional roads. The system was then expanded to cover the M&E of the local roads under the project (World Bank 2008b). The PDO indicators included vehicle operating costs, length of road rehabilitated (regional and local), and institutional indicators (added via restructuring) on the Road 7 Asset Management System, financial management, and road safety. The intermediate outcome indicators were, in general, causally linked to the PDO indicators and reflected the outputs of the project substantially. Baseline data were adequate and data-collection methods were clear. The results framework was revised at the level 1 restructuring in 2013. The revision included two new PDO indicators to monitor the achievement of institutional strengthening activities under the project, in addition to the reduction in vehicle operating costs indicator. As well, two intermediate outcome indicators were dropped, eight new indicators were added, and a target date for one of the original intermediate outcome indicators was extended. Most of these revisions helped improve the M&E framework. 2.17 In addition, as part of the level 1 restructuring and in view of the PESR’s new status, two financial ratios were added through legal covenants to help monitor the financial standing of the PESR. Although they were not a part of the results framework, these covenants were the key monitoring instrument used to track the financial performance of the enterprise and to encourage the PESR to start preparing regular projections for future periods that would enable timely investment and maintenance plans (see more details about the ratios in the Implementation and Fiduciary Compliance sections). 2.18 A few areas needed further improvements. For example, the project did not include any indicators to monitor maintenance of the rehabilitated road sections. The assessment of vehicle operating costs could have been supported by additional indicators, such as the reduction in travel times, since the cost reduction achieved a result of reduced travel times, for example. This was to be obtained through the socioeconomic impact assessment that the project planned to carry out. Outcome or intermediate outcome indicators to measure road safety on the rehabilitated roads (for example, accident reports) could also have been included. Finally, indicators that measured the number of beneficiaries of the regional and local roads supported by the project and whether these roads served to connect these beneficiaries to markets and services could have also been consulted. 2.19 Implementation. In general, key M&E data were collected in a timely manner. The implementing agency relied on reports from construction supervision teams and municipalities to collect data on road works completion and their quality. A technical audit was not conducted until October 2011, approximately three years after the start of the implementation process (the effectiveness date), due to the delay in hiring technical audit consultants. However, subsequently, audit reports were produced every six months. 8 2.20 Beneficiary feedback was collected by the Socio-Economic Impact Assessment of the Local Roads (2014), the Impact Assessment of a Demonstration Road Safety Project, and a stakeholder workshop that took place during the midterm review. Six focus groups were interviewed as a part of the first study: women entrepreneurs, young people, the elderly, farmers, and both unemployed and employed individuals. 2.21 Use. The M&E process identified a series of gaps affecting implementation and achievement of the development outcome, such as lack of road safety consideration, landslide issues, and the need to strengthen road maintenance arrangements. These findings informed the restructuring of the project to address these issues. The process also informed the World Bank’s discussions with the government about the need for a systemwide approach to addressing these challenges. This same approach was adopted in a follow-up World Bank project, the National and Regional Roads Rehabilitation Project, which was approved in 2014. 2.22 The project’s M&E is rated substantial. 3. Implementation 3.1 Quality of design and road works. The lack of detailed design for regional road rehabilitation led to construction delays, and thus to extended implementation periods for selected roads. Additional time and resources were needed to provide for site- specific interventions and to complete the road works. 3.2 The initial designs also did not include a full consideration of road safety, and therefore needed additional enhancements such as road signs, marking, and guardrails. A road safety consultant was hired subsequently to create a plan that incorporated essential road interventions into the scope of work, and to ensure that the design complied with newly adopted road safety guidelines in line with European Union (EU) regulations. A road safety demonstration project was added for demonstration purposes. 3.3 Due to the wide variation in capacity among municipalities, the local roads component took longer to launch, as the responsibility for preparing the designs was given to the municipalities. Most of the local roads’ designs were not ready by project start, as only urban and large municipalities had the budget to invest in and produce a pipeline of projects. Due diligence checks by the World Bank clearly outlined the need for more details in a number of the designs, namely that the quantities in the bills aligned with actual conditions at the site. As a result, ASR (later, PESR) engaged consultants to assist in the review and correction of submitted designs and prepare guidelines for design of local roads improvements complemented by a workshop. 9 3.4 Climate events. During 2010, landslides and erosion that occurred because of extraordinary heavy rainfall negatively affected the roads. As a result, ASR agreed to carry out remedial works on the project sections affected by the landslides, conducted a study to identify vulnerable road infrastructure, and prepared a detailed design of prioritized sections prone to landslides. The implementation of the measures identified by the study was financed by the National and Regional Roads Rehabilitation Project, which was approved in 2014. 3.5 Progress on institutional support activities. A European Bank for Reconstruction and Development grant financed an overall sector assessment and identified actions to strengthen road institutions following the amendments made to the Law on Public Roads in 2009. The recommendations of the study, which was completed in 2011, helped inform the detailed scope of works under component 3. Progress on their implementation was quite limited until the level 1 restructuring because PESR’s oversight of the implementation of components 1 and 2 demanded most of their available resources. 3.6 Environmental and social safeguards compliance. The following safeguards were triggered in the project: Environmental Assessment (Operational Policy [OP] 4.01), Natural Habitats (OP 4.04), Physical Cultural Resources (OP 4.11), and Involuntary Resettlement (OP/Bank Procedure 4.12). The civil works for regional roads, mostly repair or replacement of structural layers and drainage structures, followed by full asphalt resurfacing, fell under environmental categories B or C. However, as no local roads were defined at project preparation, the project was assigned an environmental category A as a precautionary measure. 3.7 A sectoral environmental assessment was prepared; it contained provisions that addressed OP 4.04 and OP 4.11. The assessment also provided an environmental assessment and management framework for the undefined activities under component 2 and included a resettlement policy framework that governed land use and land ownership issues under the project. Project restructurings did not trigger any additional safeguard policies. 3.8 Because the project did not finance the construction of new local roads, neither resettlement nor acquisition of land was required. There were no complaints received from local residents during the construction period. 3.9 Implementation supervision reports rated safeguard compliance as satisfactory in general for all triggered safeguard policies.3 The project implementation involved proper screening of project activities, development of appropriate environmental due diligence documents, incorporation of those documents into the bidding documents, 10 and ensuring that environmental provisions were an integral part of construction contracts. During project implementation there were some issues in terms of compliance with OP 4.01, mainly related to poorly maintained shoulders and verges, vulnerable slopes, and poor environmental compliance by some of the suppliers that provided construction materials to contractors at the project sites. The World Bank team advised the borrower and the implementing agency on measures to correct these issues and strengthen their monitoring capacity to ensure that the identified mitigation measures and any other recommendations by the World Bank were duly implemented. IEG’s mission was informed by the implementing agency that all these issues were addressed adequately. 3.10 Environmental and social safeguards were streamlined into PESR, which established a separate unit for this purpose two and a half years after effectiveness. Currently, two dedicated employees carry out due diligence for environmental and social safeguards and raise awareness and conduct training for contractors on their roles and obligations under safeguards. The safeguards staff have participated in social safeguards and resettlement training organized by the World Bank and the University of Groningen as well as in training for social assessments. 3.11 Fiduciary compliance. Financial management performance ranged from satisfactory to moderately satisfactory throughout project implementation. The 2013 and 2014 audit reports for PESR included two qualified opinions suggesting that there were several deviations from requirements in terms of the method of revaluation of plant, property and equipment, and accounting treatment of construction in progress. PESR developed action plans to correct these issues. After the project was restructured and the legal status of the ASR was changed to become PESR, a twofold assessment was carried out to confirm that acceptable financial management arrangements were in place and to ensure that the new entity was financially viable enough to take over the implementation and to repay and service the loan. To monitor PESR’s financial status, two new financial covenants were included: (i) a debt service coverage ratio (DSCR; the ratio of net operating income to debt service for the year [principle and interest]) of not less than 1.2, and (ii) a current ratio (the ratio of current assets including estimated net revenues for the next year to current liabilities) of not less than 1. By the time the project closed, DSCR was 2.7 and current ratio was 1.4. 3.12 The DSCR and current ratio continue to meet or exceed targets. As at the end of 2017, DSCR was slightly above target, at 1.3 (target >1.2) and current ratio was also above target, at 1.9 (target >1). 3.13 Procurement. The supervision reports rated procurement management satisfactory to moderately satisfactory. Even so, there was some initial delay in hiring 11 technical audit consultants to inspect the completed works for the implementing agency. The procurement contracts were subject to the World Bank’s prior approval, and one post review that was carried out in 2014 found no major issues with the procurement process or contract management. The quality of bidding and proposal documents, as well as evaluation reports, improved during implementation and there were no major deviations from the agreed-on procedures. Procurement under the project was carried out in accordance with the agreed-on provisions in the loan agreement and in accordance with the procurement guidelines. However, at the end of the project, capacity for procurement was still rated as moderately satisfactory, as reportedly PESR still required regular guidance from the World Bank team. 4. Achievement of the Objectives Original Objective 4.1 The degree to which the project's original development objective was achieved— to reduce the cost of access to markets and services for communities served by regional and local roads—is rated substantial. 4.2 IEG’s site visits. The IEG mission visited a sample of seven road segments in five municipalities to assess project outputs and outcomes. The municipalities and road segments were selected to provide a balanced mix of the following: (i) geographic location and terrain; (ii) economic activity (agriculture, historic sites, agro- tourism); (iii) population size; and (iv) proximity to urban centers. The seven road segments totaled 88.1 kilometers in length (four regional road sections totaling 77 kilometers, out of 284 kilometers); and three local road sections totaling 10.9 kilometers, out of 433 kilometers) (table 4.1). 4.3 The IEG mission used a set of structured questions for discussion for interviewing officials of government departments and agencies and municipalities (appendix D). Similarly, a basic checklist was used to interview small-business persons, farmers, and road users for their observations on road quality, improved services, and economic benefits from road rehabilitation under the project. 12 Table 4.1. Road Segments Covered by Site Visits Local Road Government Regional Length Condition at Unit Road Segment Terrain or Local (km) Appraisal Mavrovo Mavrovo Mountainous Regional 6.0 Poor Debar Konjare to Debar Flat Local 3.4 Very poor Debarto to Struga Mountainous Regional 26.3 Poor Struga Dolna Belica to Oktisi Flat Local 2.6 Very poor Stracin (M-2) to Probistip Hilly Regional 26.3 Poor Probistip Ratavica to Probistip Hilly Local 4.9 Very poor Ilinden Ilinden to Kalugerec Hilly Regional 19.0 Poor Outputs Rehabilitation Works: Although the project did not achieve the original output target for regional roads rehabilitation, the target for local roads rehabilitation was met: o Rehabilitation/reconstruction of about 290 km of regional roads thus falling short of the original target of 330 km. o Rehabilitation of 443 km local paved and unpaved roads, selected by municipalities through a participatory approach, exceeded the target of 420 km. 4.4 Road condition and quality. The roughness of the regional roads, as measured by the International Roughness Index, was estimated to have been reduced by 80 percent or more (many of these roads were unpaved before the project). The IEG team obtained the current data on road conditions using the index, which show that 85 percent of the rehabilitated are in good condition, 2 percent are in very good condition, and 13 percent are in fair condition. IEG could not obtain similar data on local roads (table 4.2 and appendix C). Table 4.2. Road Condition for Regional Roads Rehabilitated Under the Project Road Condition Length (km) Percent Very good 5 2 Good 241 85 Fair 38 13 Total 284 100 Source: PESR, August 2018. *Data were available for only 284 kilometers of road, although 290 kilometers were rehabilitated. 4.5 Road safety–related activities: The project completed and implemented designs for traffic signs and road markings on rehabilitated regional roads in accordance with 13 EU guidelines for road signs and equipment. The existing signage and road markings on segments of the regional roads that the IEG traveled were clear and in good condition, although it could not be confirmed they adhered to EU guidelines. Guardrails on the section going to Struga, however, were raised according to the EU guidelines. 4.6 Vulnerable road-side slopes were still apparent on certain road segments, although IEG was informed by PESR that the advice provided by the World Bank team on measures to address these issues were adequately addressed. For example, although two landslides occurred in Mavrovo National Park, adversely affecting that road segment’s planned timeline and cost, appropriate measures were taken to resolve the issues. On another segment (Probistip to Stracin), improper water drainage caused erosion, leading to a landslide. Although the municipality used its funds to address the problem temporarily, fixing the damage completely requires a larger investment (appendix D). Outcomes 4.7 Vehicle operating costs on regional roads decreased by 11 percent, exceeding the 10 percent target (from Euro 1.18 per vehicle kilometer to Euro 1.05 per vehicle kilometer at project completion). According to the most recent figures for selected roads, vehicle operating costs on average were 84 percent of the baseline amount (ranging from 81– 92 percent; table 4.3). 4.8 Beneficiary survey results. The 2014 beneficiary survey assessing local roads rehabilitation works found that the project reportedly improved access to health care, social and educational facilities, other municipal services, and markets. However, average time savings were not provided (box 4.1). 14 Table 4.3. Vehicle Operating Costs on Regional Roads, 2018 Ratio of Road Road Average Road VOC Euro VOC to Type Number Region and Name IRI* Condition per Vehicle Baseline Skopje R1 P1106 Ilinden to Kalugjerec 1.9 Good 1.0 0.82 R1 P1205 Stracin to Probistip 2.0 Good 1.1 0.92 Bitola R1 P1308 Makazi to Granica 1.5 Good 0.9 0.79 R1 P1201 Struga to Debar 1.6 Good 1.0 0.84 R1 P1306 Prilep to Krivogashtani 2.0 Good 0.9 0.80 R1 P1303 Prilep to Mak Brod 1.7 Good 1.0 0.82 Stip R2 P2136 Rzhanichno to Sveti Nikole 2.4 Good 1.0 0.87 R1 P1304 Prevalec to Smojmirovo 2.2 Good 1.0 0.83 Veles R1 P1401 Strumica to Dojran 1.9 Good 1.0 0.83 R1 P1105 Davidovo to Rabrovo 3.0 Fair 1.0 0.81 R1 R1103 Lakavica to Negotino 1.9 Good 1.1 0.91 Weighted average 2.0 1.0 0.84 Source: PESR Notes: IRI = International Roughness Index; VOC = vehicle operating costs. 4.9 The beneficiary survey also indicated that in some areas, key parts of the network were still in poor condition (parts that could not be addressed by the project), limiting access for community members (box 4.1). The IEG mission site visit also revealed a similar problem: A local road section in Debar municipality ended two kilometers before reaching the village. IEG was informed that the funds were not sufficient to finish the entire section. This raised some concerns about the selection process for some of the local roads that were included in the project (appendix D). 15 Box 4.1. Results from the Beneficiary Survey on Local Roads A beneficiary survey was commissioned by the Public Enterprise for State Roads (PESR) in 2014 that covered 21 municipal roads rehabilitated by the project in 10 randomly selected municipalities. The survey comprised focus group data (two in each municipality; in total, 20 focus groups with 135 participants) and semistructured interviews with municipal officials, transporters and businesses. Based on the interviews and data received from the municipal authorities and focus groups, the survey concluded that rehabilitated roads significantly improved the network coverage in all local places. Results indicated that, without exception, the roads rehabilitation facilitated and improved access to health care, social and educational institutions, economic opportunities, municipality authorities, markets, and extra-curricular activities. The survey concluded that there was still local roads network in need of rehabilitation in each area visited and this negatively impacted access for the entire community. In the areas in which the beneficiary survey took place, the entire network was 140 kilometers long; the project rehabilitated 45 percent and about 36 percent was still in need of rehabilitation (no asphalt). Thus, the remaining rehabilitation needs still limited the accessibility of many community members. For example, the survey site visits revealed that in Stenje the non-asphalted 500 meters connected the main street with the tourist infrastructure (motel and beaches on the lake). The condition of this non-asphalted road (with dust, mud, and large holes) was below standard, particularly for a tourist area. In Misleshevo, the non-asphalted 500-meter-long road prevented loading and unloading of trucks directly from and to the local textile factory, causing additional expense. In Veleshta, residents started to build the local streets by themselves in despair at the current situation. Regarding road safety, even though all 135 interviewees thought that roads rehabilitation improved traffic participants’ safety, there were several weaknesses. In almost all road sections assessed, there was either none or very old traffic signalization equipment, the road width was very narrow (between 3 and 5 meters), and there were no road bays to allow passing between vehicles as required by by-laws. Also, there was a lack of pedestrian crossings, sidewalks, berms, protection from rockslide, and protective fence barriers on the bridges. No bridges were rehabilitated as part of the rehabilitation, all of which substantially decreased road participants’ safety level. Source: Socio-Economic Impact Assessment of the Local Roads Rehabilitated Under the Regional and Local Roads Program Support Project, PESR, 2014. 4.10 IEG mission site visits and observations on project outcomes: Feedback to the IEG mission from small-business persons, farmers, and road users is in line with the findings of the beneficiary assessment survey. The feedback relates to easier and safer access to surrounding areas for transporting agricultural produce and accessing services and recreation areas. Some of the observations included the following: • Pavement quality and smoothness of drive (good, fair, or poor). Most road segments have good pavement quality with few defects or cracks, providing smooth driving conditions. Exceptions included some water-related damaged 16 and intermittent thin surface cracks ranging from one to three feet in length in the Mavrovo National Park segment. A section of the Ratavica-to-Probistip segments had also been affected by uncontrolled water drainage that caused some erosion of the road surface. Although the municipality used its funds for a temporary repair, full and sustainable repair needs a larger investment. • Verge clearing and grass cutting (good, fair, or poor). On the whole, greater attention needs to be paid to verge clearing and grass cutting on the site visit road segments. These measures can only be considered adequate in the Stracin- to-Probistip and the Ilinden-to-Kalugerec segments. In the Ratavica-to-Probistip segment, there were many spots with overgrown grass and small shrubs. In the remaining segments, the verge was in fair condition. In the Mavrovo segment, some drainage and culverts need attention. • Signage, guardrails, and so on. Existing signage and markings were clear and visible. However, there was no readily available information as to whether national or EU requirements as applicable in this regard had been met. On the Debar-to-Struga segment, guardrails were in good condition and were reported by officials to be in line with EU standards. The Ratavica-to-Probistip segment had clear signage and markings. Sections that experienced landslides have been repaired and reinforced, with additional precautions taken to minimize the impact of future incidents. • Transit time. Overall, there were significant reductions in transit time for most of the road segments. The transit time for the Debar-to-Struga segment (26 kilometers) has been reduced significantly, from 55 minutes to 25 minutes. For the Ilinden-to-Kalugerec segment, transit time was reduced from 3 hours to 2.5 hours. The Dolna Belica-to-Oktisi segment was originally a dirt track but is now on par with the other improved local roads. It has also seen a steep decrease in transit time, although a baseline was not available for comparison. No baseline estimates were available for the Konjare-to-Debar, Stracin-to-Probistip, or Ratavica-to-Probistip segments. However, the IEG mission was able to traverse these segments at 55 kilometers to 60 kilometers per hour. On the Mavrovo segment, no major reduction in time was reported; better road quality and safety were stated as the main gains from upgrading this section. • Impact on services and economic activity and feedback from road users: Anecdotal evidence points to positive impacts on services and economic activity, although no specific comparative data could be collected during the site visits. Staff at the Mavrovo tourist office told the IEG mission that upgrading of the segment provides greater convenience for residents and tourist traffic through the area. For example, upgrades to the Dolna Belica-to-Oktisi segment have 17 reduced the transit time between the two villages. Officials and local road users reported an increase in traffic on this segment over the years, and new construction along the length of the road. The Konjare-to-Debar road segment passes through a cultivated area and can reasonably be expected to have had a positive impact on transporting agricultural inputs and produce. However, this paved stretch ends two kilometers short of a beneficiary village; the rest of the road is an unfinished gravel-and-dirt track. Local officials were not able to confirm why the road segment was not planned to be extended to the village at the outset. For the Stracin-to-Probistip segment, no specific information could be obtained, but much of the road segment serves a densely populated area with significant traffic. Road users noted that the Ilinden-to-Kalugerec segment adds value by improving connectivity to six villages and the nearby regions of Petrovec, Kumanovo, and Skopje. Finally, the Ratavica-to-Probistip segment has seen an increase in traffic from the widening of the road from 3.5 meters to 6 meters, thus increasing its usability and quality. The investments in new retaining walls in the initial part of the road and the widening of a poorly designed curve have contributed to a perception of increased safety along some stretches of the road. Revised Objective 4.11 After the level I restructuring in 2013, the project’s objective was amended to add two subobjectives: (i) “to reduce the cost of safe access to markets and services for communities served by regional and local roads in the Guarantor's territory” and (ii) “to improve institutional capacity for investment planning and road safety”: • Restructured objective 1 was similar to the original objective, except for the addition of the term “safe.” No systematic data were available to track the safety of access, however. Judging mainly by the outcomes discussed under the original outcomes, the achievement of this objective is rated substantial. • The degree of achievement of the project's restructured objective 2—to improve institutional capacity for investment planning and road safety—is rated modest. Outputs 4.12 Objective allocation of funds: The local roads component involved an objective approach to funds allocation among municipalities, as agreed to during preparation. A formula that took into account various factors, such as the length of the road network, the number of registered vehicles, fuel consumption, and surface size and population, was applied to set the project resource allocation among the 81 municipalities. The formula was subject to comprehensive consultations with the Ministry of Transport and 18 Communication, the Ministry of Finance, the Association of the Units of the Local Self- Government (Združenie na edinicite na lokalnata samouprava; ZELS) and municipal governments to ensure agreement among all parties and avoid implementation bottlenecks due to complaints about allocations. No complaints arose during the lifetime of the project. The Law on Public Roads adopted the formula and it was applied in the central budget transfer arrangements between the government and municipalities for maintenance of the local road network, thus ensuring equity. Road Asset Management System: A road database was established on the basis of a condition survey of the core road network. This survey contained key information on nearly all motorways and national roads (road sections, lengths, condition, and road safety data) and on 80 percent of regional roads. It did not include any data on local roads. Highway database management software was installed and the staff of PESR were trained in its usage with the purpose of establishing the Road Asset Management System (table 4.4). Table 4.1. Illustrative Data from the Road Asset Management System Road Network (km) Road Class Two-way Ramps Rest Areas Total AA 42.7 9.5 489.1 AB 419.0 15.5 2.1 489.0 AM 235.1 3.4 1.3 239.8 R1 1,788.7 1.7 1,790.4 R2 1,262.7 1,262.7 R29 295.6 295.6 Total 4,001.0 63.3 12.8 4,566.6 Source: PESR, August 2018. Note: AA, AB, and AM are main roads; R1, R2, and R29 are regional roads. 4.13 A financial module of enterprise resource planning system was established and made operational in PESR, accommodating detailed financial records by source of funding and automation of report generation while improving the capacity to control revenue collection. 4.14 A road rehabilitation and strengthening design study, and guidelines for the identification and design of improvements to local roads, were prepared. 4.15 EU guidelines were adopted for road signs and equipment-based design on traffic signs and road markings for regional roads rehabilitated under the project. 4.16 PESR’s environmental management capacity was strengthened through the preparation of the National Handbook on the Environmental Management of Roads Projects, 19 delivery of training sessions, and assistance toward establishing a unit dedicated to environmental and social safeguards. The unit is now staffed and operational. Outcomes 4.17 Use of the Road Asset Management System for investment planning: One of the key capacity building activities was the establishment of the Road Asset Management System, which was developed and used by PESR to produce a Five-Year Rolling Strategic Program, meeting the target set in the restructuring paper dated May 28, 2013. The IEG mission was informed that the network data had not been systematically updated, nor had the most current road conditions and traffic volumes and the database been actively used, after the project was closed. A follow-up project, Road Upgrading and Development Project (P149955), is attempting to improve the database by adding sophisticated tools (such as visualization tools and maps) and to help use the database in investment decisions. 4.18 Municipality capacity for managing and maintaining local roads: There is evidence that PESR has improved its capacity to manage and implement road investments (on contract management, supervision, road safety, procurement, and safeguards). However, the project did not include any institutional capacity building activities for municipalities, except initial support by PESR for design works for local road rehabilitation. This is a crucial gap, because the municipalities are responsible for managing and maintaining their local road network. During IEG’s mission, meetings with three municipalities (out of 81), as well as ZELS, revealed that municipalities lack the knowledge and financial resources to adequately manage their network. Some specific weaknesses include the following: • Municipalities did not build the needed capacity for contract management and supervision of works, as this was the responsibility of PESR under the project. • Currently, no institution or agency oversees monitoring the entire local road network in the country. A database for local roads is needed to track local road conditions and prioritize investment decisions accordingly. ZELS informed IEG that at the request of the government, the association sometimes attempts to collect information on local roads from the municipalities, but that these were ad hoc cases and that it was left to the discretion of the municipality to provide information. • Maintenance of local roads is also inadequate in general because of limited financial capacity. The local road financing mechanism, which was agreed on during the project preparation and introduced by law for the central budget transfer for local roads financing, still applies and provides equitable distribution of financial resources. IEG was informed that the formula developed was a one- 20 time exercise and that although many of the factors might have changed over time (for example, traffic flow, road conditions, and so on), the formula has not been updated. In addition, the amount of the transfer is inadequate as per IEG’s discussions with stakeholders. The total amount transferred from the central budget was approximately $5.4 million in 2018, an amount that has stayed the same since 2008 (see appendix E). Feedback from the respondent municipalities revealed that the transferred amount in some cases was barely sufficient to cover emergency situations. 4.19 Maintenance of regional roads. Road maintenance for regional roads is handled by Makedonija Pat, which operates as PESR’s direct contractor. The IEG mission was informed that at the current levels of performance, PESR will not be able to keep up with road maintenance, especially during winter. The IEG mission observed several sections in which verge clearing and grass cutting appeared to be behind schedule. Overgrown vegetation can affect sightlines or introduce the risk of tree limbs falling on the road, potentially affecting road user safety. In addition, insufficient removal of vegetation may affect drainage and thus create a hazard during storms (appendix D). Follow-up projects (table 1.2) are attempting to improve the institutional setup for road maintenance through the introduction of open competition and by utilizing the private sector’s construction capacity for road maintenance. 4.20 Road safety. Not enough data were available to conclude that road safety had been improved on the road sections supported by the project; as well, accident data could not be obtained to assess if road safety had improved. As discussed earlier, the restructured project included provision for traffic signs, road markings, and lighting to be installed on critical sections of the regional roads rehabilitated under the project in accordance with EU guidelines. Several concerns were raised by beneficiaries about road safety on local roads, including the lack of traffic lights, narrow roads, lack of curbs, pedestrian crossings, and so on (box 4.1). 4.21 In sum, even though PESR’s capacity has been strengthened to a certain extent under the project, these efforts need to be furthered. In addition, there are substantial needs for Municipalities to manage and maintain their networks. Although the Road Asset Management System was completed, IEG’s mission was informed that the system was not used as a tool for investment decisions and that the database is not updated periodically. Road safety issues on local roads, as well as limited road maintenance efforts on both regional and local roads, are ongoing. Considering these shortcomings, achievement of this objective is rated as modest. 21 5. Efficiency 5.1 The efficiency in achieving the original PDO and the revised PDO is rated substantial. 5.2 Economic efficiency. The ex post economic analysis was done after the project closed. The analysis replicated the ex ante economic internal rate of reform (EIRR) results per road section calculated at appraisal using the Highway Design and Maintenance Standards Model and then repeated the calculation on each road section,4 considering actual rehabilitation costs and traffic growth rate from 2009 to 2014. The Highway Development and Management Model was based on the following assumptions: (i) a 4 percent annual traffic growth rate for all vehicles during the evaluation period, based on the estimated gross domestic product growth rate for North Macedonia by the International Monetary Fund from 2018 to 2022 of 3.7 percent per year (actual traffic grew by 2.3 percent during the project period; this figure was used for the ex post analysis); (ii) an 8 percent discount rate; (iii) a 20-year evaluation period based on common practice; and (iv) actual average rehabilitation cost of Euro 0.101 million per kilometer, which was similar to the appraisal estimate (Euro 0.100 million per kilometer). Compared with a without-project alternative, the rehabilitation alternative was defined as performing recurrent maintenance until the road is in such poor condition that it must be reconstructed. The project result includes future periodic maintenance works, including periodic maintenance when a road reaches a fair condition. All project regional roads achieved an adequate EIRR, higher than the adopted 8 percent discount rate. The overall EIRR of the project roads was 48 percent, which is lower than the ex-ante EIRR of 74 percent but nevertheless very satisfactory. 5.3 An ex ante economic evaluation was not performed for local road works, but an ex post economic evaluation was conducted based on the average characteristics of these roads, which carry approximately 500 vehicles per day.5 The average improvement road works cost was Euro 0.075 million per kilometer. The overall EIRR of the local roads program is 28 percent. Considering both the regional and local road works, the overall ex post EIRR of the project is 38 percent—demonstrating the project’s substantial economic efficiency. 5.4 Administrative and operational efficiency: The original project closing date of July 31, 2013, was extended twice, for a total of 29 months, to December 31, 2015. The reasons for the extensions were mostly related to external factors, including delays because of the annual cap on loan disbursement imposed by the government to reflect its worsened fiscal conditions caused by the 2008 financial crisis affecting Europe, and delay of civil works due to unusually heavy rainfall. Other reasons for the delays was 22 related to weak project planning—that is, the lack of detailed design for regional road rehabilitation. For local roads, the delays were even longer due to the municipalities’ lack of capacity to design their road work projects. This was exacerbated by initial staff limitations at PESR to manage and supervise works for the municipalities. 5.5 While there were some administrative and operational inefficiencies, the overall efficiency of the project is rated substantial based on favorable economic rates of returns. 6. Ratings Outcome 6.1 The relevance of both the project's original and revised objectives is rated high. The relevance of the original design is rated modest and the revised design is rated substantial. The efficacy for the original objective—to reduce the cost of access to markets and services for communities served by regional and local roads—is rated substantial. Following restructuring, objective 1—to reduce the cost of safe access to markets and services for communities served by regional and local roads in the guarantor’s territory—is, as before, rated substantial. The efficacy of the objective added after restructuring, objective 2—to improve institutional capacity for investment planning and road safety—is rated modest. Both objectives were achieved with substantial efficiency. The outcomes for the original and revised objectives are rated satisfactory and moderately satisfactory respectively. Because the project’s disbursement at the time of the PDO revision was approximately 64 percent of total disbursement, the project’s overall outcome is rated as satisfactory, based on a disbursement-weighted average of the outcome ratings for the original and the revised objectives. Risk to Development Outcome 6.2 Inadequate financial resources for maintenance. The regional roads improved by the project are routinely maintained under regular annual contracts with Makedonija Pat; the company has had difficulty carrying out this function adequately, however. Therefore, under the new operation, National and Regional Roads Rehabilitation (P148023), the World Bank is supporting PESR to develop maintenance performance criteria that can be used in annual maintenance contracts with Makedonija Pat. This will help PESR to better monitor the quality of maintenance performed and, in the medium term, move toward performance-based maintenance. 23 6.3 PESR’s financial situation. The IEG mission obtained the most recent financial covenants and verified that financial ratios are in line with required targets. The follow- up project also included regular monitoring of compliances with the financial covenants so that PESR will be able to identify and address any potential financial issues in due course. However, the availability of necessary political will and financial resources for the funding of maintenance activities is uncertain because there are no formal institutional arrangements to ensure that adequate budget allocations will be made to the maintenance. Another factor is the possibility that PESR’s financial allocation to maintenance could decrease in 2019 when the repayment of a loan provided by China commences (World Bank 2016, page22). 6.4 Maintenance of local roads. During the project, capacity building activities were organized with all mayors and technical municipal staff to raise awareness about the importance of maintenance. There is still a need for further improvement in this area, however. The IEG mission was informed by officials of the municipalities it visited that their available funds make it difficult to perform even routine maintenance. This is despite the mechanism established by law for allocating central budget funds to supplement municipal resources for local roads management. 6.5 Road asset management activities that began under this project will be furthered under the follow-up project, National and Regional Roads Rehabilitation and Road Upgrading and Development Project, through further improvements and widening of the Road Asset Management System application. However, municipalities still need extensive support to improve their capacity to manage and maintain their network. 6.6 Overall risk to development outcome is rated substantial. World Bank Performance Quality at Entry 6.7 The project was designed as part of a reform process developed through an intensive sector dialogue with the government in coordination with the World Bank and the European Commission and with financial support from the EU. The World Bank’s long-term involvement in the transport sector in North Macedonia (the World Bank has financed six transport projects since 1995) and lessons learned from analytical work, including the Regional Study on Feeder Roads in the Western Balkans were generally reflected in the overall design (World Bank 2008a). This work confirmed the overall poor state of regional roads in the region and their key role in internal mobility. The design of the local roads component partially reflected lessons learned from the involvement of municipalities and local communities in project design and implementation. Another lesson incorporated from other World Bank projects was the importance of having a line 24 agency as the implementing agency instead of establishing a separate a project implementation unit. 6.8 The project design was based on a strategically relevant objective and a set of project activities logically linked to the PDO. A consultative process was followed in the identification of the local roads component and in related fund allocation among all 81 municipalities (an innovative approach with a formula, which considered various factors including political acceptability, was used). Safeguards management mechanism and fiduciary arrangements were adequately set up.6 Supervision of all civil works, including local roads, was delegated to the implementing agency to reduce governance risks that may have resulted if local governments, without such capacity, were involved in making direct payments to contractors. Although this was a sound arrangement for accountability purposes, it did not help build the capacity of municipalities. 6.9 The relatively short duration of project preparation (that is, six months from concept to approval), though commendable, contributed to some of the shortcomings in the following ways: (i) detailed designs for regional roads were not adequately completed before implementation, and this led to a prolonged period of implementation because of the additional time and resources needed from the World Bank and the road agency to complete good-quality work; (ii) road maintenance and road safety issues were not adequately incorporated into the original design; (iii) institutional capacity strengthening activities could not be sufficiently included for municipalities to support them to manage and maintain their network as per their mandate7; and (iv) the project’s M&E indicators had some shortcomings especially in respect of measuring road safety and road maintenance. 6.10 On balance, the World Bank’s quality at entry is rated moderately satisfactory. Quality of Supervision 6.11 Regular supervision missions were carried out and 13 implementation status and results reports were filed during seven years of implementation (approximately two supervision missions per year). Aide memoires were comprehensive and included detailed action plan guidance on resolving shortcomings and planned activities. It was reported to IEG that the World Bank team was responsive to both PESR’s and the government’s request for technical advice and project modifications. The project had three team leaders in seven years, which did not affect project implementation adversely. 6.12 The World Bank team was proactive. Specifically, it engaged a road safety consultant to prepare road safety audits and to include necessary road interventions into the scope of works when it was revealed that initial designs did not include typical road 25 safety aspects. The project also supported the implementation of mitigation measures to address the impact of landslides. 6.13 The World Bank team was also collaborative during institutional reforms initiated by the government, particularly regarding the development of the new legislation to establish PESR in 2013 that gave the institution legal and managerial autonomy in managing the state road network. This required the level 1 restructuring of the project and the agreement of strict debt and financial indicators for the new agency, assigning the loan from the government to PESR as the new borrower. The Bank together with the counterpart carried out this restructuring process. The process attempted to address the insufficient road maintenance aspects during the level I restructuring through support for the establishment of the Road Asset Management System. This had limited impact, however. 6.14 The quality of supervision is rated satisfactory. 6.15 Based on moderately satisfactory quality at entry and satisfactory quality of supervision, overall Bank performance is rated moderately satisfactory. Borrower Performance Government Performance 6.16 The government proved its commitment during design and implementation through its well-formulated program and the sectoral reform goals that informed the project. The amendments to road legislation that were planned during the design phase were completed early on in the implementation process. The reform was an important move toward strengthening the planning and management of road investments by making them more reliable and transparent. The government supported project preparation by providing adequate project management resources. It also actively involved municipalities in the consultation process, which largely informed the preparation of the project design for local roads. 6.17 The government complied with all legal and financial covenants (which were created after PESR became the borrower). Financial ratios were still being met at the time of IEG mission. 6.18 Government performance is rated satisfactory. Implementing Agency Performance 6.19 The implementing agency of the project was the ASR when the project was approved, and then became PESR after December 2012. The implementing agencies 26 managed a large volume of civil works and ensured their quality, were proactive in improving and maximizing municipal participation in the consultation process for the design of the local roads component and implemented the World Bank’s recommendations on addressing project implementation issues in a timely manner. 6.20 Due to insufficient assessment by the World Bank of the implementing agency’s capacity constraints, PESR’s staffing levels were not fully sufficient at times to manage the project and meet its other responsibilities, including other donor- and government- funded projects. PESR’s oversight of local roads rehabilitation, which is outside its mandate, consumed more of its time than necessary due to the weak design capacity of some municipalities. The recruitment of additional qualified staff, particularly engineers, was considerably delayed, mainly due to their unavailability in the broader labor market. IEG was informed that, currently, PESR has 60 full-time professional employees, up from 34 at project appraisal. The hiring of consultants to assist with safeguards was also delayed by almost 2.5 years, but a safeguards unit with two staff was established after that time and those staff were adequately trained by the World Bank. However, there was insufficient support from PESR management, particularly social safeguards. 6.21 Performance on procurement and M&E by both ASR and PESR was in general adequate, although there was some initial delay in monitoring road works' compliance with contractual specifications due to the delayed hiring of technical audit consultants. Subsequently, key M&E data were collected in a timely manner. The implementing agency relied on reports from construction supervision teams and municipalities to collect data on road works completion and their quality. A technical audit was not conducted until October 2011, about three years after the start of the implementation process (the effectiveness date), due to ASR’s weak capacity. Performance on financial management also had some shortcomings. The PESR audit reports for 2013 and 2014 had two qualified opinions suggesting that there were several deviations from requirements in terms of the method of revaluation of plant, property, and equipment, and of the accounting treatment of constructions in progress. PESR developed action plans to correct these, however. 6.22 The implementing agency performance is rated satisfactory. 6.23 Based on satisfactory government and the implementing agency performance, overall borrower performance is rated satisfactory. 27 7. Lessons 7.1 Objective criteria developed and applied in a participatory manner can support a transparent framework to allocate investments and maintenance funds in the roads sector. This project developed a multicriteria framework (including length of the road segments, number of registered vehicles, fuel consumption, surface size, and population) and applied it in partnership with municipalities and local communities. The framework was later adapted into a law to distribute the central budget for road works among municipalities. 7.2 The decentralization of responsibilities to local governments needs to be accompanied by the availability of commensurate resources and capacity building. The responsibility of managing and maintaining local road networks was devolved to municipalities through territorial reforms in North Macedonia beginning in the early 2000s. However, with limited funds and limited capacity, municipalities continue to face challenges carrying out their mandate. 7.3 Road safety and road design elements need to be jointly integrated into the project design and monitoring framework to mitigate risks to the effectiveness of road projects. Under this project, road safety aspects were not consistently followed initially but were introduced only after project restructuring. A better approach is to include road safety elements in the project development objective and then integrate them into the project design and monitoring framework. 7.4 Road project appraisal requires sufficient time and technical due diligence to ensure effective and timely project implementation. The shortcomings experienced in quality at entry relating to the lack of detailed designs at project commencement, and inadequate provision of capacity building for municipalities, might have been overcome if greater time and technical due diligence had been committed to these matters at project appraisal. 1Absolute poverty in 2008 was measured at either 9 percent, using a regional poverty line of $2.50 per day, or 37 percent, using a regional poverty line of $5.00 per day (World Bank 2014b). 2As the legal successor to the Agency for State Roads (ASR), PESR took over all the responsibilities of the implementation agency for the project. The ASR (transformed from Fund for National and Regional Roads in June 2008) ceased to operate as of December 2012. 3 Supervision of environmental safeguards was conducted on average twice a year. 28 4The Highway Development and Management Model performs a cost-benefit economic evaluation of road works project alternatives, estimating benefits in terms of reduction in vehicle operating costs, travel time, and accident costs for road users. 5 Traffic volume is linked to road maintenance and vehicle operating costs. 6The Law on Public Roads adopted the formula to be applied in the central budget transfer arrangements between the government and municipalities for maintenance of the local road network, thus ensuring objectivity and fairness in fund allocation for maintenance needs. 7 It is noted that the Bank has two active projects supporting municipalities in the country: Municipal Services Improvement (P096481); and Second Municipal Services Improvement Project (P154464). However, the impacts of these projects on the management of local roads are not yet clear. 29 Bibliography World Bank. 2007. Former Yugoslav Republic of Macedonia—Country Partnership Strategy for FY07– FY10. Washington, DC: World Bank Group. ———. 2008a. Improving the Management of Secondary and Tertiary Roads in the South East Europe Countries. Washington, DC: World Bank. ———. 2008b. “Former Yugoslav Republic of Macedonia—Regional and Local Roads Program Support Project.” Project Appraisal Document Report 42200-MK, World Bank, Washington. ———. 2008c. Former Yugoslav Republic of Macedonia—Loan Agreement 7532-MK Conformed Regional and Local Roads Program Support Project. Washington, DC: World Bank. ———. 2011. Former Yugoslav Republic of Macedonia—Country Partnership Strategy for FY11–FY15. Washington, D.C.: World Bank Group. ———. 2013a. “Former Yugoslav Republic of Macedonia—Regional and Local Roads Program Support Project.” Restructuring Paper Report RES10209, World Bank, Washington, DC. ———. 2013b. Former Yugoslav Republic of Macedonia—Loan Assignment and Assumption Agreement. L7532-MK Regional and Local Roads Program Support Project. Washington, DC: World Bank. ———. 2014a. Former Yugoslav Republic of Macedonia—Country Partnership Strategy for FY2015–18. Washington, DC: World Bank. ———. 2014b. “Former Yugoslav Republic of Macedonia—National and Regional Roads Rehabilitation Project.” Project Appraisal Document, Report 86302-MK, World Bank, Washington, DC. ———. 2015a. Former Yugoslav Republic of Macedonia—Public Expenditure Review: Fiscal Policy for Growth. Washington, DC: World Bank. ———. 2015b. “Former Yugoslav Republic of Macedonia—Regional and Local Roads Program Support Project.” Restructuring Paper Report RES10209, World Bank, Washington, DC. ———. 2015c. “Former Yugoslav Republic of Macedonia—Road Upgrading and Development Project.” Project Appraisal Document, Report PAD1117. Washington, DC: World Bank. ———. 2016. “Former Yugoslav Republic of Macedonia—Regional and Local Roads Program Support Project.” Implementation Completion and Results Report, ICR3666, World Bank Washington, DC. ———. 2017. “Former Yugoslav Republic of Macedonia—Regional and Local Roads Program Support Project.” Independent Evaluation Group, Implementation Completion and Results Review Report ICRR0020321, World Bank, Washington, DC. 30 Appendix A. Basic Data Sheet Regional and Local Roads Program Support Project (IDA Credit No. 75320) Table A.1. Basic Information Regional and Local Roads Country North Macedonia Project name: Program Support Project Project ID P107840 L/C/TF number(s): IBRD-75320 ICR date 06/20/2016 ICR type: Core ICR Lending instrument Sector investment and Borrower: North Macedonia maintenance loan Original total Disbursed amount Commitment ($, 105.20 92.27 ($, millions): millions) Revised amount 105.20 ($, millions) Environmental category: A Implementing agency: Public Enterprise for State Roads Note: The Implementation Completion and Results (ICR) is a self-evaluation by the responsible World Bank Global Practice. Note: As of 2019 the “Former Yugoslav Republic of Macedonia” is the Republic of North Macedonia (North Macedonia). Table A.2. Key Dates Process Date Process Original Date Revised/Actual Date Concept review 12/11/2007 Effectiveness: 10/17/2008 10/17/2008 Appraisal 03/17/2008 Restructuring(s): 06/11/2013 Approval 05/13/2008 04/09/2015 Midterm review: 11/29/2010 03/07/2011 Closing: 07/31/2013 12/31/2015 31 Table A.3. Staff Time and Cost World Bank Budget Only Staff time Costa Stage of Project Cycle (no. weeks) ($, thousands) Total Lending, 52.86 188.17 Supervision and ICR FY09 42.03 123.04 FY10 46.27 178.90 FY11 38.95 143.66 FY12 28.52 121.81 FY13 35.05 132.72 FY14 33.77 90.40 FY15 21.94 42.37 FY16 12.88 44.27 Total 259.41 877.17 Note: ICR = Implementation Completion and Results Report. a. Including travel and consultant costs. Table A.4. Task Team Members Responsibility or Name Title a Unit Specialty Lending Diomedes Berroa Lead Specialist OPSOR Procurement Specialist Zarko Bogoev Operations Officer ECSEG - HIS Operations Officer Arcadii Capcelea Senior Environmental Specialist GEN03 Environmental Safeguards Specialist Bekim Imeri Senior Social Development Specialist GSU03 Social Safeguards Specialist Stephen Muzira Senior Transport Specialist GTI01 Transport Specialist Andreas Schliessler Lead Transport Specialist GTI05 Task Team Leader (from 04/10/2008 to 06/11/2009) Marie Laygo Program Assistant GTI03 Program Assistant George Banjo Senior Transport Specialist ECSTR Task Team Leader (from 06/11/2009 to 08/12/2013) Liljana Sekerinska Senior Transport Specialist GTI03 Transport Specialist and Task Team Leader after 08/12/2013 Supervision or ICR Diomedes Berroa Lead Specialist OPSOR Procurement Specialist 32 Responsibility or Name Titlea Unit Specialty Bekim Imeri Senior Social Development Specialist GEN03 Social Safeguards Specialist Zarko Bogoev Operations Officer ECSEG - HIS Operations Officer Arcadie Capcelea Senior Environmental Specialist GEN03 Environmental Safeguards Specialist Gulana Enar Senior Environmental Specialist GEN03 Environmental Hajiyeva Safeguards Specialist Aleksandar Senior Financial Management GGO21 Financial Management Crnomarkovic Specialist Specialist Anneliese Viorela Financial Management Analyst GGO21 Financial Management Voinea Analyst Plamen Stoyanov Senior Procurement Specialist GGO06 Procurement Specialist Kirov Antonia G. Viyachka Procurement Specialist GGO03 Procurement Specialist Marie Laygo Program Assistant GTI03 Program Assistant Luan Aliu Program Assistant ECCMK Program Assistant Note: ICR = Implementation Completion and Results Report. a. At time of appraisal and closure, respectively. 33 Appendix B. Project Cost and Financing Table B.1. Key Project Data Actual or Current Actual as Percent Appraisal Estimate Estimate of Appraisal Financing ($, millions) ($, millions) Estimate Total project costs 105.2 91.3 87 Loan amount 105.2 91.3 87 Cofinancing Cancellation 13.9 13.9 Table B.2. Cumulative Estimated and Actual Disbursements Disbursements FY08 FY09 FY10 FY11 FY12 FY13 FY14 FY15 Appraisal estimate 15 45 70 90 105.2 105.2 105.2 105.2 ($, millions) Actual ($, millions) 5.2 23.5 47.7 59.5 63.7 75.4 84.07 91.5 Actual as percent 35 52 68 66 61 72 80 87 of appraisal Date of final FY08 FY09 FY10 FY11 FY12 FY13 FY14 disbursement 34 Appendix C. Road Roughness Index for Rehabilitated Regional Roads Weighted Type Road No. Designation of the section Length (km) IRI Class Average IRI Ilinden-Kal R1 P1106 1256; Drachevo 1 (Jurumleri) - Markova Sushica 11 1.87 Good R1 P1106 1257; Markova Sushica - Dolno Sonje 5 1.72 Good R1 P1106 1258; Dolno Sonje - Nova Breznica 10 1.51 Good Rzanicino-Sv.Nikole R2 P2136 1510; Rzhanichno - Sredno Konjare 9 2.42 Good R2 P2136 1511; Sredno Konjare - Gorno Konjari 3 2.51 Fair R2 P2135 1508; Gorno Konjari - Gorobinci 21 3.96 Fair Lakavica-Negotino R1 R1103 1237; Leskavica - Negotino 24 1.86 Good Davidovo-Rabrovo R1 P1105 1248; Davidovo - Udovo (avtopat) 1 2.99 Fair R1 P1105 1249; Udovo (avtopat) - Valandovo (Brajkovci) 10 1.91 Good R1 P1105 1250; Valandovo (Brajkovci) - Rabrovo 4 2.10 Good Stracin - Probistip R1 P1205 1310; Stracin (Krilatica) - Zhivalevo 13 2.01 Good R1 P1205 1311; Zhivalevo - Kratovo 5 2.28 Good R1 P1205 1312; Kratovo - Probishtip 16 1.95 Good Struga - Debar R1 P1201 1284; Dobovjani 2 (Jablanica) - Drenok 18 1.60 Good R1 P1201 1285; Drenok - Dzhepishte 12 1.80 Good Makazi-Border R1 P1308 1381; Makazi 1 (Kazhani) - Makazi 2 (Kozjak) 1 1.49 Good R1 P1308 1382; Makazi 2 (Kozjak) - Asamati 8 1.43 Good R1 P1308 1383; Asamati - Kurbinovo 1 (Pretor) 1 1.25 Very good R1 P1308 1384; Kurbinovo 1 (Pretor) - Slivnica 4 1.28 Very good R1 P1308 1385; Slivnica - Ljubojno 8 1.56 Good R1 P1308 1386; Ljubojno - Drzhavna granica MK/GR 5 1.70 Good Prilep-Mak.Brod R1 P1303 1354; Varosh - Dolneni 1 (Novoselani) 11 1.66 Good R1 P1303 1355; Dolneni 1 (Novoselani) - Ropotovo 6 1.49 Good R1 P1303 1356; Ropotovo - Suvodol 15 1.64 Good Prilep-Krivogastani R1 P1306 1372; Prilep 4 (G. Konjari) - Krivogashtani 2 19 1.96 Good Prevalec-Smojmirovo R1 P1304 1360; Prevalec 2 (Kochansko) - Vinica 1 (Gradec) 4 2.15 Good R1 P1304 1361; Vinica 1 (Gradec) - Vinica 2 (Blatec) 6 2.66 Fair R1 P1304 1362; Vinica 2 (Blatec) - Most na reka Pekljanska 2 1.86 Good Strumica-Dojran R1 P1401 1412; Strumica 4 (ciglana) - Tri Vodi 5 1.86 Good R1 P1401 1413; Tri Vodi - Rabrovo 13 1.82 Good R1 P1105 1251; Rabrovo - Furka 9 2.26 Good Mavrovo R2 P2235 1531; Mavrovi Anovi - Mavrovo 4 3.49 Fair R2 P2235 1532; Mavrovo - Bunec 3 3.31 Fair Total (km) 284 1.98 Good Source: PESR Macedonia, August 2018. Note: IRI Class <1.3 very good, >1.3 IRI Class<2.3 good, IRI Class>2.3 fair. IRI = International Roughness Index. 35 Appendix D. Observations from Road Section Visits Road Sections: Mavrovo (National Park) length: 6 km width: 5 m. (regional road) Smoothness of drive (good/fair/poor): Good; few defects visible on pavement surface; some portions had water damage Verge clearing and grass cutting: (good/fair/poor): fair, some observed drainage/culverts need attention; blockages/diversions may have been done by unauthorized private builders. Other: Guardrails appear in good condition; existing signage and markings were clear and visible; Sections that experienced landslides have been repaired and reinforced, with additional precautions for minimizing impact of future incidents. Transit time: No major reduction in time was reported; better road quality and safety were stated as the main gains from upgrading this section. Impact on services and economic activity/ feedback from road users: IEG mission met with Mavrovo National Park office staff. The upgrading of segment provides greater convenience to residents and tourist traffic through the area. Mavrovo Road Sections: Konjare Debar length: 3.4 km width: 5 m (local road) Smoothness of drive (good/fair/poor): Good; Verge clearing and grass cutting: (good/fair/poor): fair; Other: The paved road segment ends 2 km short of a beneficiary village. The remaining 2 km is an unfinished gravel/dirt track. Local officials were not able to confirm the reason why the road segment was not planned to be extended to the village at the outset. 36 Transit time: There was no baseline available to assess the improvement in transit time. Impact on services and economic activity/ feedback from road users: The road passes through cultivated area, and can reasonably be expected to have had positive impact on transporting agricultural inputs and produce. Konjare Debar Road Sections: Debar -Struga length 26 km, width (regional road): Smoothness of drive (good/fair/poor): Good; There were no visible signs of damage to the pavement.. Verge clearing and grass cutting: (good/fair/poor): Fair. Verge was mostly clear, with some patches requiring trimming/cutting. Other: Guardrails were in good condition and were reported by officials to be in line with EU standards. Transit time: Transit time reduced significantly from 55 minutes to 25 minutes. Impact on services and economic activity/ feedback from road users: During the implementation of the works, a couple of landslides occurred, impacting the foreseen timeline and cost. The affected sections of the road have been reinforced, to minimize damage in future incidents. 37 Debar-Struga Road Sections: Dolna Belica-Oktisi length 2.6 km, width:6 m (local road) Smoothness of drive (good/fair/poor): Good; pavement looked in good condition; Verge clearing and grass cutting: (good/fair/poor): Poor; clear need for better upkeep. Transit time: Improved a lot because it was dirt road before, but no figure could be given; Impact on services and economic activity/ feedback from road users: The road segment connects two villages. Officials reported an increase in traffic over the years, and new constructions along the length of the road. Dolna Belica-Oktisi Road Sections: Stracin-Probistip length 26.3 km, width: 6 m (regional road) Smoothness of drive (good/fair/poor): Good; Verge clearing and grass cutting: (good/fair/poor): good Transit time: no baseline for comparison. 38 Impact on services and economic activity/ feedback from road users: No specific information could be obtained, but much of the road segment serves a densely populated area. Stracin-Probistip Road Sections: Ratavica-Probistip length 4.9 km, width: 6 m. (regional road) Smoothness of drive (good/fair/poor): Fair; some unattended spots with defects in the pavement. Verge clearing and grass cutting: (good/fair/poor): Poor; Other: A section of the road has been affected by uncontrolled water drainage that has cause some erosion of the road surface. Although the municipality used their funds for a temporary repair, full and sustainable repair needs a larger investment. Clear signage and markings. Transit time: no baseline information available. Impact on services and economic activity/ feedback from road users. Based on the flux of traffic the project increased the width of the road from 3.5 meters to 6 meters thus increasing the usability and quality of the existing road. The investments were in new retaining walls in the initial part of the road and widening of one of the biggest curves to increase safety. 39 Ratavica-Probistip Road Sections: Ilinden – Kalugerec length 19 km, width: 5 m (regional road) Smoothness of drive (good/fair/poor): Good; Verge clearing and grass cutting: (good/fair/poor): poor; in need of attention Transit time: Reduced from 3 hours to 2.5 hours. Impact on services and economic activity/ feedback from road users: Road users noted that the road segment adds value by improving connectivity to 6 villages, and the nearby regions of Petrovec, Kumanovo and Skopje. Ilinden – Kalugerec Source: Independent Evaluation Group. 40 Appendix E. Municipality Budget Allocation Table E.1. Breakdown of Municipality Budget Allocation from Central Budget, 2018 41 42 Appendix F. List of Persons Met World Bank Marco Mantovanelli, Country Manager Liljana Sekerinska, Senior Transport Specialist Bekim Imeri, Senior Social Development Specialist Ministry of Finance Suzana Peneva, State Advisor Branimir Jovanovik, Advisor Rilind Kabashi, Advisor Dejan Nikolovski, Head, International Finance Relations and Public Debt Department Ministry of Transport and Communications Jasmina Kirkova, State Counselor Ljuba Silijanoska, State Counselor Public Enterprise for State Roads Zoran Kitanov, Director Zoran Slamkov, Project Coordinator North Macedonia Chambers of Commerce Maja Saveska, Advisor Ljupka Dimoska, Coordinator Association of Local Government Units Dusic Perisic, Executive Director United Nations Development Programme Anita Kodzoman, Program Officer Mavrovo National Park Cane Petreski, Head of Technical Issues for the Park Municipality of Struga Vladislav Zhupan Municipality of Probistip Boban Stojmirovski Municipality of Ilinden Zika Stojanovski, Mayor 43 Appendix G. Borrower Comments 44 45 46