Document of The World Bank Report No: ICR00003145 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-45490 IBRD-77510 IBRD-79360 TF-94897) ON A CREDIT IN THE AMOUNT OF SDR16.1 MILLION (US$25.0 MILLION EQUIVALENT) AND TWO ADDITIONAL FINANCING LOANS IN THE AMOUNT OF US$76.6 MILLION TO THE REPUBLIC OF ARMENIA FOR THE LIFELINE ROADS IMPROVEMENT PROJECT June 11, 2014 Transport Sector Unit Sustainable Development Department Europe and Central Asia Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 11, 2014) Currency Unit = Armenian Dram (AMD) US$ l.00 = AMD 413.2 US$ 1.00 = SDR 0.65 FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS ARD Armenian Roads Directorate CAS Country Assistance Strategy DBST Double Bituminous Surface Treatment EIRR Economic Internal Rate of Return EMF Environmental Management Framework EMP Environmental Management Plan EFC Emergency Financing Credit FA Financing Agreement FMS Financial Management Specialist GDP Gross Domestic Product GOA Government of Armenia IDA International Development Association IFI International Financial Institution ISR Implementation Status Results LRN Lifeline Road Network LRIP Lifeline Roads Improvement Project MCC Millennium Challenge Corporation MOF Ministry of Finance MoTC Ministry of Transport and Communications PP Project Paper PPIAF Public Private Infrastructure Advisory Facility RRRP Rural Road Rehabilitation Project TPIU Transport Project Implementation Unit Regional Vice President: Laura Tuck Country Director: Henry G. R. Kerali Sector Director: Laszlo Lovei Sector Manager: Juan Gaviria Project Team Leader: Vickram Cuttaree ICR Team Leader: Rodrigo Archondo-Callao ii REPUBLIC OF ARMENIA Lifeline Roads Improvement Project CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 7 3. Assessment of Outcomes .......................................................................................... 16 4. Assessment of Risk to Development Outcome......................................................... 20 5. Assessment of Bank and Borrower Performance ..................................................... 21 6. Lessons Learned ....................................................................................................... 23 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 24 Annex 1. Project Costs and Financing .......................................................................... 26 Annex 2. Outputs by Component ................................................................................. 28 Annex 3. Economic and Financial Analysis ................................................................. 29 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 34 Annex 5. Beneficiary Survey Results ........................................................................... 36 Annex 6. Stakeholder Workshop Report and Results................................................... 39 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 40 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 51 Annex 9. List of Supporting Documents ...................................................................... 52 MAP A. Basic Information LIFELINE ROADS Country: Armenia Project Name: IMPROVEMENT PROJECT IBRD-77510,IBRD- Project ID: P115486 L/C/TF Number(s): 79360,IDA-45490,TF- 94897 ICR Date: 06/11/2014 ICR Type: Core ICR REBUBLIC OF Lending Instrument: ERL Borrower: ARMENIA Original Total USD 25.00M Disbursed Amount: USD 101.50M Commitment: Revised Amount: USD 101.51M Environmental Category: B Implementing Agencies: Ministry of Transport and Communications Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 01/13/2009 Effectiveness: 04/10/2009 04/10/2009 Appraisal: 01/14/2009 Restructuring(s): Approval: 02/24/2009 Mid-term Review: Closing: 12/31/2010 12/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Substantial Bank Performance: Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Moderately Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Moderately Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Public administration- Transportation 2 2 Rural and Inter-Urban Roads and Highways 98 98 Theme Code (as % of total Bank financing) Rural services and infrastructure 100 100 E. Bank Staff Positions At ICR At Approval Vice President: Laura Tuck Shigeo Katsu Country Director: Henry G. R. Kerali Asad Alam Sector Manager: Juan Gaviria Henry G. R. Kerali Project Team Leader: Vickram Cuttaree Christopher B. Bennetts ICR Team Leader: Rodrigo Archondo-Callao ICR Primary Author: Rodrigo Archondo-Callao F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The project development objective is to upgrade selected sections of the Lifeline Road Network and create temporary employment in road construction. Revised Project Development Objectives (as approved by original approving authority) ii (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : Travel time improvement to drive the length of the section over current situation Value quantitative or 0.0 20.0 20.0 58.5 Qualitative) Date achieved 06/06/2008 12/31/2009 12/31/2013 12/31/2013 Comments 293% achievement. Measurements done by supervision consultants for all (incl. % project roads. Measured in percent reduction in travel time. achievement) Indicator 2 : Number of person-days/month of jobs created Value quantitative or 0 7,650 36,650 39,855 Qualitative) Date achieved 06/06/2008 12/31/2009 12/31/2013 12/31/2013 109% achievement. A typo presents this indicator as person-months in the Comments Project Papers. In reality, the targets for job creation and the job creation (incl. % monitoring was done in person-days/month. Measurements done by supervision achievement) consultants for Indicator 3 : Transport cost reductions on project roads Value quantitative or 0 20.0 20.0 25.8 Qualitative) Date achieved 06/06/2008 12/31/2009 12/31/2013 12/31/2013 Comments 129% achievement. Measurements done using the HDM-4 model based on (incl. % measured roughness values. Measured in percent reduction in transport costs. achievement) Indicator 4 : One Pilot "safe villages" program implemented Value quantitative or No Yes Yes Qualitative) Date achieved 06/06/2008 12/31/2010 12/31/2013 Comments (incl. % 100% achievement. Measured as Yes or No. achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : Number of km upgraded Value 0 100 430 446 iii (quantitative or Qualitative) Date achieved 06/06/2008 12/31/2009 12/31/2013 12/31/2013 Comments (incl. % 104% achievement. Measurements done by supervision consultants. achievement) Indicator 2 : Roads in good and fair condition as a share of total classified roads measured Value (quantitative No Yes Yes Yes or Qualitative) Date achieved 06/06/2008 12/31/2010 12/31/2013 12/31/2013 Comments 100% achievement. Measurements done by ARD. Measured as Yes or No. The (incl. % 2013 value is 54.1 percent. achievement) Indicator 3 : Share of rural population with access to an all-season road measured Value (quantitative No Yes Yes Yes or Qualitative) Date achieved 06/06/2008 12/31/2010 12/31/2013 12/31/2013 Comments 100% achievement. Measurements done by ARD. Measured as Yes or No. The (incl. % 2013 value is 51.5 percent. achievement) Indicator 4 : Road Safety Audit Manual is developed Value (quantitative No Yes Yes or Qualitative) Date achieved 06/06/2008 12/31/2010 12/31/2013 Comments 100% achievement. Measured as Yes or No. The Road Safety Audit Manual was (incl. % developed in 2010. achievement) Indicator 5 : Establishment of a data collection unit in ARD Value (quantitative No Yes Yes or Qualitative) Date achieved 06/06/2008 12/31/2011 12/31/2013 Comments 100% achievement. Measured as Yes or No. The data collection unit was (incl. % established in 2011. achievement) Indicator 6 : New road geometry design standards developed Value (quantitative No Yes No or Qualitative) Date achieved 06/06/2008 12/31/2012 12/31/2013 Comments 0% achievement. Measured as Yes or No. Activity was dropped out during (incl. % project review in 2011. achievement) iv G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 04/13/2009 Satisfactory Satisfactory 0.00 2 07/31/2009 Satisfactory Satisfactory 12.76 3 11/06/2009 Satisfactory Satisfactory 24.40 4 05/07/2010 Satisfactory Highly Satisfactory 44.91 5 12/10/2010 Satisfactory Highly Satisfactory 62.02 6 07/09/2011 Satisfactory Satisfactory 72.50 7 04/18/2012 Satisfactory Satisfactory 85.13 8 12/25/2012 Satisfactory Satisfactory 96.85 9 06/26/2013 Satisfactory Satisfactory 98.80 10 12/28/2013 Satisfactory Satisfactory 101.39 H. Restructuring (if any) Not Applicable I. Disbursement Profile v 1 Project Context, Development Objectives and Design 1.1 Context at Appraisal 1.1.1 Country Background. Armenia suffered from severe economic contraction in 2009 following the global economic crisis. The real Gross Domestics Product (GDP) declined by about 14.6 percent in 2009 (Figure 1 below), mainly caused by the decline in the construction sector. The economy resumed growth in 2010 and has maintained a recovery path with a growth rate of 2.1 percent in 2010, 4.6 percent in 2011, and 7.2 percent in 20121. The recession had a major impact on local employment, disrupted the poverty reduction momentum of Armenia and increased poverty for the first time since the late 1990s. During the peak of the recession, about 42 percent of Armenian households reported of suffering from the negative consequences of the global economic crisis. The poverty headcount increased from 28 percent of the population in rural areas in 2008 to 36 percent in 2010. The Government of Armenia (GOA) reacted to the economic crisis by initiating a recovery strategy to protect the poor and support local employment by improving connectivity to markets and generated local employment. Figure 1: GDP Change and Inflation 20.0 15.0 10.0 5.0 Percent (%) 0.0 2005 2006 2007 2008 2009 2010 2011 2012 -5.0 -10.0 -15.0 -20.0 Percent change of Gross Domestic Product Average consumer prices inflation Source: IMF Outlook April 2014 1.1.2 Road Sector. Armenia is a landlocked country in a strategically important location of the South Caucasus in Eurasia. The country shares its borders with four countries: Georgia, Iran, Azerbaijan and Turkey. However, only two of these borders are open: the southern border with Iran and the northern border with Georgia. The borders with Turkey and Azerbaijan were closed in the early 1 The April 2014 IMF World Development Outlook forecast the growth rate from 2014 to 2019 to be 4.7 percent annually. 1 1990s as a result of the Nagorno-Karabash conflict. In addition, Armenia has very challenging topography and weather conditions during winter – the terrain is very mountainous, which results in high transport costs and expensive infrastructure maintenance and development. 1.1.3 The transport sector has represented an average of 6.7 percent of GDP over the last seven years. The total length of the road network is 7,704 km, excluding urban roads, with less than half being in good or fair condition2. The network is classified into interstate (1,686 km), republican (4,056 km) and local roads (1,962 km), of which republican roads are subdivided into republican main roads (1,814 km) managed by the Ministry of Transport and Communications (MoTC) and other republican roads (2,242 km) managed by the Regional administrations (Marzes). Most of the road network was built in the 1960s and 1970s. The percentage of paved roads, at 93 percent, is high compared to other developing countries, but is in line with most European countries. The majority of republican and local roads have deteriorated since independence and only 52 percent of the republican roads are currently in good or fair condition. The average daily traffic of republican roads is around 8000 vehicles. 1.1.4 The MoTC is responsible for the transport sector in Armenia and for managing interstate and republican main roads (1,814 km). Marzes and local communities are responsible for maintaining about 55 percent of other republican roads (2,242 km) and all local roads. The Armenian Road Directorate (ARD) is responsible for maintaining interstate and republican main roads. In 2008, the GOA launched the Lifeline Road Development Program, introducing the concept of lifeline roads, comprised of about 4,000 km, mainly current republican roads under the responsibility of ARD. The lifeline roads concept ensures at least one access road to all 960 communities in Armenia, which is in line with the GOA’s transport strategy of focusing on improving key international roads network and rehabilitating the lifeline roads network (LRN). In international terms, the rate of deaths per capita and per vehicle in Armenia remains much higher than in best performing countries. Armenia’s deaths per 100,000 people are 10.5, compared with the average of 6.0 for the European Union member states. The deaths per 100,000 people increased from 10.3 in 2005 to 13.7 in 2008 and since 2008 decreased to 10.5 in 2012. Pedestrians account for 44 percent of the deaths. 1.1.5 Rationale for Bank Assistance. In response to the global financial crisis in 2009, the GOA requested support on three fronts: (i) a Small and Medium Enterprises credit line, to support real sector activity; (ii) budgetary support; and (iii) urgent support for small-scale infrastructure to generate increased employment. The Lifeline Roads Improvement Project (LRIP) was part of this requested program and was prepared under the International Development 2 In 2008, 32.2 percent of republican roads were in good or fair condition. In 2013, the percentage increased to 51.5 percent. 2 Association (IDA) “Financial Crisis Response Fast-Track Facility” utilizing Operational Policy (OP) 8.0, Rapid Response to Crisis and Emergencies. The rationale for the Bank assistance was to mitigate the impact of the global financial crisis in Armenia, create employment and respond to the urgent request of support from the GOA, and use the Bank’s global knowledge to strengthen ARD’s capacity to improve efficiency through implementing cost effective pavement designs and to improve road safety. The LRIP had two additional financings approved within two years to increase the scope of the road works. 1.1.6 LRIP was consistent with the 2004-08 Country Assistance Strategy (CAS) and addressed one of its three objectives: “promoting private sector growth by strengthening the financial sector, improving public sector management, and reducing infrastructure bottlenecks”. The CAS and Progress Report specifically included improving road transport infrastructure as a key outcome indicator. The Lifeline Roads Improvement Project Additional Financing (LRIP-AF) and the Lifeline Roads Improvement Project Additional Financing Two (LRIP- AF2) were aligned with the Country Partnership Strategy (CPS) for FY2009- 12, which emphasized that “the road network is of particular importance for a landlocked country with partly closed borders. Investment and operating costs are high, and strategic interests indicate investments in both transit transport corridors as well as in rural or lifeline roads”. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 1.2.1 The Project Development Objectives (PDO) of the LRIP, which was approved on February 24, 2009, were: (i) to upgrade selected sections of the Lifeline Road Network; and (ii) create temporary employment in road construction. 1.2.2 The indicators proposed for the monitoring and evaluation of the project included: (A) Project Outcome Indicators:  Travel time improvement to drive the length of the section over current situation; and  Number of person-days/month of local jobs created. (B) Intermediate Outcome Indicators:  Number of km improved;  Low cost pavement options developed, including increased use of labor;  Capacity building through training; and 3  Vehicle procured for supervision3. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 1.3.1 The Project had an Additional Financing (AF), approved on August 27, 2009, in which the PDO is the same as under LRIP. 1.3.2 The results framework and the monitoring indicators were updated to monitor the rehabilitation of an additional 140 km of Lifeline Road Network and to monitor other road sector strengthening activities. The intermediate outcome indicators “Low cost pavement options developed, including increased use of labor”, “Capacity building through training”, and “Vehicle procured for supervision” included in LRIP were removed from the results monitoring reflecting a change in the focus of the institutional strengthening activities to address road safety. The additional indicators include: (A) Project Outcome Indicators: 4  One Pilot of “safe villages” program implemented . (B) Intermediate Outcome Indicators:  Road Safety Audit Manual is developed;  Roads in good or fair condition as a share of total classified roads measured; and  Share of rural population with access to an all-season road measured. 1.3.3 The project had a Second Additional Financing (AF2), approved on July 15, 2010, in which the PDO is the same as under LRIP. 1.3.4 The results framework and the monitoring indicators were updated to monitor the rehabilitation of an additional 140 km of Lifeline Road Network and to monitor other road sector strengthening activities. The additional indicators include: (A) Project Outcome Indicators:  Transport cost reductions on project roads. (B) Intermediate Outcome Indicators:  New road geometric design standards developed; and 3 In the PP, this intermediate outcome indicator was included on the Arrangements for Results Monitoring table, but was not included on the Results Framework table. 4 In the LRIP-AF Project Paper, this project outcome indicator was defined as an outcome indicator on the Revised Arrangements for Results Monitoring table, but it was defined as an intermediate outcome indicator on Revised Results Framework table. The LRIP-AF2 Project Paper confirms that this is an outcome indicator. 4  Establishment of data collection unit in ARD. 1.4 Main Beneficiaries 1.4.1 The primary target groups were the rural communities living along the project roads that directly benefited from (i) the creation of employment for the rural population, and (ii) better access to social services and economic markets. The project improved the livelihood of the local population by creating temporary employment and improving rural connectivity. The project was expected to have a direct and indirect temporary employment generation impact. Road rehabilitation creates direct temporary employment for skilled and unskilled labor and also creates employment in the production and transportation of road construction materials. The overall project benefited around 945,000 persons living along the project roads and nearby communities, as planned. The LRIP benefited 203,000 persons, the LRIP-AF 216,000 persons and the LRIP-AF2 540,000 persons5. 1.4.2 The secondary groups of beneficiaries include MoTC, ARD, and the local construction industry. The project strengthened the capacity of MoTC and ARD to manage their road assets including road safety and road asset management considerations and improved efficiency by promoting lower cost rehabilitation options and the use of international design standards. Moreover, the project helped local construction industries weather the economic crisis6 and improve their technical skills, by emphasizing the role of technical supervision done by an international consultant company as an instrument for technology and knowledge transfer. 1.5 Original LRIP Components (as approved) 1.5.1 Component 1: Rehabilitation of the Lifeline Road Network (US$30.0 million including contingencies): This component was expected to rehabilitate approximately 100 kilometers of the LRN. The road sections were located in seven Marzes. The component included: (i) civil works for road rehabilitation; (ii) consultancy services for the construction supervision, design author supervision 7 , and technical auditing of rehabilitation works; (iii) updating the original Millennium Challenge Corporation (MCC)-financed 2007 designs and environmental documents; and (iv) project implementation expenses including, but not limited to, funding project audits, incremental 5 These values were estimated by the TPIU during the project implementation. 6 Nine of 23 contractors did not have any other contracts than those offered under the stimulus package and the rural roads contracts accounted for 22 percent to 27 percent of the gross revenue of all the contractors benefiting from the program. 7 According to the legislation of the Republic of Armenia, a fee of up to 0.6 percent of the construction costs was paid to the original designers to ensure compliance with the designs, as well as updating of designs to reflect any changed field conditions. 5 operational implementation costs and additional costs for intensified project supervision. 1.5.2 Component 2: Technical Assistance (US$0.4 million including contingencies): The project was expected to include technical assistance for strengthening ARD capacity, including but not limited to: (i) a study to review low cost pavement options for Armenia to explore options for different pavement types and ways to increase labor based activities; (ii) updating designs for approximately 100 kilometers of LRN roads for a potential future project; (iii) procurement of a vehicle for field supervision; and (iv) related training. 1.6 First Additional Financing Components 1.6.1 The LRIP was approved on February 24, 2009 and became effective on April 10, 2009 with a Credit (Credit: IDA-4549-AM) amount of Special Drawing Rights (SDR) 16.1 million (US$25.0 million equivalent). An additional loan amount of US$36.6 million was approved on August 27, 2009 to finance the scaling up of the project’s activities through rehabilitating an additional 140 kilometers of the LRN. There were no changes in the PDO and the results framework and the monitoring indicators were revised to reflect the increased project scope and attention to road safety activities. There were no changes applicable to the institutional arrangements, financial management, and disbursement arrangements. The closing date was extended by one year, from December 31, 2010 to December 31, 2011. 1.6.2 Component 1: Rehabilitation of 140 kilometers of selected sections of lifeline roads (US$43.93 million, including contingencies). The LRIP-AF expected to rehabilitate in total 29 sections of lifeline roads in seven Marzes. This component included: civil works for the roads rehabilitation; supervision of the works; design supervision; technical auditing of rehabilitation works; and project implementation expenses included but not limited to, funding technical and financial audits, monitoring and evaluation, and incremental operating costs. This component financed the “safe villages” pilot program, which supports rural communities in implementing road safety measures recommended by road safety audits and the National Road Safety Strategy8. The “safe villages” pilot program supports the villages selected on a demand- driven basis in installing road safety measures, especially around schools, as required by a road safety audit. To be eligible for funding, villagers must prepare and start implementing their own road safety campaigns. The LRIP-AF provided necessary support to villagers. 8 Developed under Policy and Human Resources Development Fund Grant (TF 0577789) for Preparation of Traffic Management and Safety Project, and submitted to GOA for adoption. 6 1.6.3 Component 2: Technical assistance (US$1.7 million). This component financed capacity development in alternative pavement designs (based on the recommendations of the study under the original project); the procurement of laboratory and testing equipment, and vehicles for supervision; training; and designs for future investments. This component supported the development of a road safety audit manual to provide checklist for assessing design and inspecting project sites throughout different phases of a road project, including assessment of existing roads. 1.7 Second Additional Financing Components 1.7.1 Another additional loan amount of US$40.0 million was approved on July 15, 2010 and became effective on November 15, 2010 to finance the scaling up of the project activities through rehabilitating approximately 190 kilometers of the LRN. There were no changes in the project’s PDO. The results framework and the monitoring indicators were revised to reflect the increased project scope and attention to road asset management activities. There were no changes to the implementation arrangements. The closing date was extended by two years, from December 31, 2011 to December 31, 2013 to allow for all activities to be completed and fully cover the defect liability period for two years. 1.7.2 Component 1: Rehabilitation of approximately 190 kilometers of selected sections of lifeline roads (US$45.8 million, including contingencies). The LRIP-AF2 expected to rehabilitate about 190 kilometers of lifeline roads. This included 178 kilometers of road sections for which rehabilitation works were suspended around 2008 due to a shortage of donor funding, and a 12 kilometer section of the H46 road to the town of Halidzor. This component included civil works for the rehabilitation of the roads; supervision of the works; technical auditing of rehabilitation works; and project implementation expenses including, but not limited to, funding technical and financial audits, monitoring and evaluation and incremental operating costs. 1.7.3 Component 2: Technical assistance (US$1.1 million). This component was expected to finance the development of new road geometric standards, feasibility studies and designs of future investments and the procurement of road data collection equipment. 2 Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 2.1.1 Project preparation built upon the road rehabilitation program funded by MCC. The preparation of LRIP was initiated in late 2008 with a very tight preparation and implementation schedule in response to the emergency request from GOA. The LRIP Board Approval was on February 24, 2009, the Project Agreement was signed on February 26, 2009 and ratified by Parliament on March 17, 2009, and the project became effective on April 20, 2009. 7 Procurement of all civil works have been completed by April 2009 and contracts awarded by May 19, 2009. Contractors mobilized promptly and started works immediately after the project effectiveness. The project preparation and design were built on the experience with the program for rehabilitation of LRN roads funded by MCC9. The LRIP roads were selected based on their: (i) having been identified as priorities in the MCC program; (ii) located in areas facing increasing unemployment; (iii) having sufficient economic returns to justify the investment; and (iv) readiness to be implemented rapidly. Detailed designs of project roads prepared by MCC in 2007 were reviewed and revised in light of European standards and to ensure consistency with actual traffic, enhance sustainability and improve road safety. The preparation of the additional financings was done based on current road condition and traffic surveys and new designs including the principles agreed on when the original project was prepared. The main reason behind the two successive additional financings in so short time was to increase the scope of the road works. Having two separate additional financings nearly duplicated the resources and efforts in terms of project preparation. The quality at entry is rated satisfactory. 2.1.2 Project design tailored to an emergency project. The LRIP design was very simple and realistic in terms of components and involved organizations in line with the emergency nature of the project. LRIP had two components, which did not include complex activities. Because of the need to rapidly bid the roads and have fast disbursements (LRIP was the first of the IDA Fast Track Facility operations through to become effective), the project had intensive supervision and very close monitoring of project implementation. The main focus of the LRIP was to start the rehabilitation of the roads as soon as possible, maximize the employment generation, and improve the technical quality of the road works. The focus of the LRIP-AF and LRIP-AF2 increased to include concerns about road safety in Armenia and the availability of survey equipment for a road asset management system. The LRIP activities under the institutional strengthening component and later on more activities added to the same component under the LRIP-AF and LRIP-AF2 were modest and in areas where the project could rapidly add value to the institutional capacity of ARD. The institutional strengthening activities focused on: (i) adopting cost effective designs; (ii) improving road safety, and (iii) ensuring proper road construction quality 10 . The rationale for adding additional activities to the project institutional strengthening components during the additional financings was that the LRIP components were focused on quick execution of emergency 9 The Millennium Challenge Corporation (MCC) prepared in 2007 a Rural Road Rehabilitation Project (RRRP) in Armenia designed to rehabilitate up to 943 km of LRN roads. However, in June 2009 the MCC made the decision to discontinue funding any further road rehabilitation, after rehabilitating 24 km, due to concerns about Armenia’s democratic governance. 10 The condition of the rehabilitated roads is overall very satisfactory, particularly considering the rapid preparation schedule. The average roughness of the rehabilitated roads is 3.1 IRI, m/km. 8 rehabilitation works of good quality and once this was achieved there was the opportunity to tackle road safety and sustainability issues. 2.1.3 The financial arrangements were simple and only involved Bank financing. The implementation arrangements were not complex, with the MoTC having the overall responsibility for implementation of the project. The MoTC delegated to the Transport Project Implementation Unit (TPIU) and ARD for the project implementation. The TPIU was established in 2000 as a state institution to implement World Bank financed transport projects. Currently, it manages International Financial Institution (IFI) funded transport projects, with exception of Asian Development Bank projects, and transport projects executed with local financing. The project was mainly implemented by the TPIU that was well qualified and has overseen well the project preparation and implementation. The project was classified through environmental screening as Category B. An Environmental Management Framework (EMF) was developed for the project. Site–specific Environmental Management Plans (EMPs) were prepared for each project road section. No environmental safeguards other than OP/BP 4.01 were triggered. Overall the project has not faced major environmental11, social or fiduciary issues. 2.1.4 The LRIP PDO consisted of two clearly defined development objectives. The LRIP had two outcome indicators corresponding to the two development objectives. The AFs added two outcome indicators to better measure the economic impact of the road works and a road safety activity. The project outcome indicators measured well the outcomes of the PDO. However, while the LRIP PP Results Framework defined the targets and stated that the employment generation was going to be measured in person-months, in reality the employment generation targets and monitoring was measured in average person-days/month12. The LRIP intermediate outcome indicators measured the number of kilometers improved and the implementation of institutional development activities. The project PDO did not change under the LRIP-AF and LRIP-AF2. The results framework and monitoring indicators were updated under the LRIP-AF and LRIP-AF2 to consider the escalation of the project civil works and new institutional development activities. The LRIP-AF and LRIP-AF2 Project Papers did not correct the typo related to the units of the targets measured and the monitoring of employment generation were being measured in average person-days/month13 and not in person-months. The typo 11 From September 2009 to November 2010, minor issues in the area of EMP compliance and works of drainage and sidewalks were identified and then fully addressed. Thus, during this period the environmental rating was downgraded to moderately satisfactory. 12 To achieve a target of 46,650 person-month of temporary employment with the rehabilitation of 430 km of roads is unrealistic. Assuming that the labor works for six months on the rehabilitation of the project road, results on 14.2 persons working on each kilometer of the rehabilitated roads continuously for six months, which is very high. For similar projects, this value has a range from 2 to 5 persons per km. 13 To proper monitor the number of jobs created during the implementation of the project; this task was included in the ToR of the Technical Supervision consultant as a part of Supervision tasks during 9 was identified only at the ICR stage, but the targets were met and exceeded. Overall, the results frameworks reflected properly the PDO and the project activities. 2.1.5 Government’s commitment was strong along project cycle. The GOA demonstrated a strong commitment to the project through efficient project preparation and implementation14. The project was prepared and delivered to the board within two months15, which was an extremely tight and challenging timeline. The GOA also closely cooperated with the Bank during the preparation of the relevant safeguard documentation. The GOA adopted a Rural Infrastructure Strategy and Action Plan, and started its implementation prior to the project with MCC support, and thus, the GOA pre-identified priority LRN rural road links thereby ensuring LRIP roads would maximize improved basic access to rural populations. The GOA recognized the importance of maintaining the same TPIU team throughout project preparation and implementation to ensure continuity. The GOA reiterated its commitment to rehabilitate the LRN roads and strengthening the capacity of ARD, by requesting a follow-up Bank project to further rehabilitate 170 km of lifeline roads - the Lifeline Road Network Improvement Project (LRNIP) that was approved by the Board on January 31, 2013. However, the GOA was not able to allocate proper maintenance budget for the interstate and republican roads during the duration of the project. There were no co-financiers to the whole project16. 2.1.6 Project risks identified. The LRIP PP defined the overall risk rating as moderate because of: (i) the accelerated processing timetable; (ii) potential construction quality risks; (iii) sector risks; and (iv) fiduciary risks. Due to the successful implementation of the project, the identified accelerated processing timetable and construction quality risks did not materialize. Given the emergency nature of the project and short time frame for implementation, the LRIP was not setup to address crucial sector issues. The corruption risk was identified as significant and a series of mitigation measures were incorporated, which were effective for the proper procurement and financial management processes with the project. The LRIP-AF and LRIP-AF2 did not identify any significant risks. Residual fiduciary risks were assessed as moderate based on the mitigation measures implemented under LRIP. The risk of the provision of construction. The consultant included statistics on the temporary employment generation through civil works in Monthly Progress Reports and the Completion Report. 14 There was a period where counterpart funds flows were slow caused by quicker than anticipated implementation of civil works. GOA reacted with large counterpart allocations to solve the issue. 15 The identification mission was on January 10, 2009 and the Board approval was on February 24, 2009. 16 The Bank coordinates road sector activities with other prominent partners in Armenia, such as Asian Development Bank and European Investment Bank, but they were not involved on the LRIP. 10 counterpart funding be delayed was identified and was mitigated by a dated covenant included in the legal agreement17. 2.2 Implementation 2.2.1 The overall implementation progress was rated in the Implementation Status Results (ISR) reports satisfactory throughout May 2010, when it was upgraded to highly satisfactory because a study18 on employment generation found that there have been more jobs created, as well as more roads improved, than originally envisaged and in shorter time frame, and the good responsiveness of the TPIU and ARD to addressing the Bank's recommendations. The highly satisfactory rating of project implementation was maintained up to July 2011 when it returned to satisfactory up to the end of the project. The downgrade of the rating was done to better reflect the fact that the TPIU became severely overloaded with additional responsibilities not related to any Bank-funded projects that started to affect its responsiveness to Bank requests, but without causing major implementation delays19. In addition, the lack of sufficient GOA funding for road maintenance was found to affect the sustainability of the rehabilitation works and the project covenants 20 . Thus, it was agreed to incorporate maintenance sustainability in the design of the LRNIP. 2.2.2 Efficient implementation of civil works. Within eight months after the project became effective, about 150 km of LRN roads were rehabilitated and by the end of the project 446 kilometers were rehabilitated. The project outperformed the target (430 km) by using cost savings to rehabilitate an additional 16 km. In addition, local designers and contractors successfully applied new technologies and appropriate and cost-effective design standards based on international practices, which strengthened the sustainability of investments and improved road safety. 2.2.3 Attention to road safety. During the project period a National Road Safety Secretariat21 was established, a "safe villages" program was implemented and a Road Safety Audit Manual was prepared. The village of Gyularabad was the pilot for the "safe villages" program that included civil works related to road traffic calming measurements, footway construction and awareness campaigns 17 The GOA was required to deposit 20 percent of the counterpart funding within two weeks after the ratification of the Loan Agreement. 18 Report on Job Creation Impact Assessment by Arsen Petrosyan, World Bank, 2009. 19 This was resolved by the hiring of additional staff by the TPIU. 20 The GOA complied with all legal covenants or they were not yet due up to March 2012 when the legal covenants I.A.3 and I.A4 became partially complied with up to the end of the project. These covenants indicate that the Recipient and Borrower shall ensure that adequate budgetary resources are made available to the road sector for maintenance of Interstate and Republican Roads in the Borrower's annual budget. 21 The Armenia’s Road Safety Secretariat was established with support from a grant (TF094897) from the Bank-administered Global Road Safety Facility (GRSF) in the amount of US$100,000. In addition to a Director, the Secretariat has four full time staff. 11 in the village, resulting in lower traffic speeds within the village. As a result of this pilot, the LRNIP extended the “safe villages” program to most of the villages covered by the LRNIP roads. The Road Safety Audit Manual report was completed and submitted to the Bank on June 2, 2011 and road safety audits are being done for all designs prepared for IFIs since 2010. Recent improvements in road safety can be attributed to a number of factors: (i) road safety is recognized as a problem by the GOA; (ii) a Secretariat for the National Road Safety Council of Armenia was established; (iii) seat belt wearing has been improved dramatically; (iv) road safety audits were introduced; and (v) the “safe villages” approach was introduced. 2.2.4 Low cost pavement options. The project has taken a progressive approach to improve the design standards over project implementation. The project promoted the use of more efficient designs based on traffic and road conditions and paid particular attention to providing proper drainage to the roads. A study to review low cost pavements options and identify areas for improvements was programmed under the LRIP. The work started and a first draft report was produced, but the study was terminated on February 25, 2012, with agreement of the Bank 22 , due to poor performance of the Consultant. A study for the development of new road geometric design standards was programmed under LRIP-AF2, but the activity was canceled in 2011 at an early stage, before a TOR was prepared. The activity was canceled and funds reallocated because the work done on the rehabilitation designs addressed the issue of proper geometric design standards for Armenia. LRIP has mainstreamed the use of modern design standards and LRNIP designs include now the identification and evaluation of cheaper pavement options. LRIP financed a pilot road rehabilitation with Double Bituminous Surface Treatment (DBST) pavements23, which is a less expensive technical solution compared with traditional Asphalt Concrete pavements. In addition, the pavement layers used for Asphalt Concrete rehabilitations are now being designed adopting international standards resulting in reduced thicknesses 24 and costs savings; and gravel pavements are being considered a valid option for low volume roads. Therefore, there has been real progress in the use of more efficient design standards in 22 The objective of the study was to review existing standards in Armenia and identify areas for improvement. The Bank agreed that the bidding for this contract did not have to be re-launched and the unspent amount used to finance capacity building activities for ARD/MOTC on designs standards because the identification of lower cost options was already achieved through the project. 23 The use of DBST was piloted on two sections covering 16.5 km of roads. The contract includes rehabilitation, routine maintenance and winter maintenance. The DBST pavement of a small section (1 km) that was rehabilitated with DBST was damaged by alcohol/acid from trucks waiting to serve a wine factory, but this is considered an isolated issue. Overall the quality of the DBST pavements is satisfactory after observing the road condition after two winter seasons. 24 The Republic of Armenia Construction Norms IV-11.05.02-99 for reconstruction, used before the project, calls roads with less than 1,000 AADT for two asphalt concrete surface layers of 3 cm and 6 cm, on top of a 4 cm bituminous base, which is on top of additional subbase layers from 8 cm upwards. With project support, now according to the traffic of the roads, only one asphalt concrete layer may be used with a thickness as low as 4 cm, representing substantial savings. 12 Armenia and the overall quality of the designs of rehabilitation works in Armenia was greatly improved with support from the project. 2.2.5 Mid-term review. A mid-term review was not required for LRIP because the lending instrument was an Emergency Recovery Loan. The lending instrument for the LRIP-AF and ARIP-AF2 was Specific Investment Loan. No formal mid-term review was carried out for the project, though the 2011 supervision missions carried out more in-depth review of the project. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 2.3.1 M&E Design. The overall M&E function was the responsibility of TPIU since the start of the project. The TPIU relied on reports of the construction supervision consultants to monitor the physical works and outcomes in terms of travel times, road roughness and employment generation. Quarterly and later semiannual reports on implementation progress and monitoring indicators were submitted to the Bank. The Results Frameworks consisted of a list of outcome indicators and intermediate results for monitoring progress and outcomes. The design included sufficiently appropriate specific indicators for each component, and covered physical works as well as implementation of institutional development activities. The indicators were simple to collect. 2.3.2 Two sector indicators 25 were introduced in 2009 as per Bank institutional requirements in the ISRs. However, while the Bank Core Sector indicators are measured in percentage, the project indicators were defined to indicate only if measurements are being done (Yes or No). The Results Framework (RF) was updated under the LRIP-AF and LRIP-AF2 to reflect additional roads and changes to the project activities. The indicators were appropriate to measure the PDO. The PDO objective to upgrade selected sections of lifeline road network was measured by the project outcome indicators “travel time reductions on project roads” and ”transport cost reductions on project roads” , and the objective to create temporary employment in road construction was measured by the project outcome indicator “number of person-days/month of local jobs created”. The intermediate outcome indicators were used to monitor the project activities. The intermediate outcome indicator “New road geometric design standards developed”, introduced under the LRIP-AF2, was not achieved because the activity was canceled. The intermediate outcome indicators “Low cost pavement options developed, including increased use of labor”, “Capacity building through training”, and “Vehicle procured for supervision”, defined on the LRIP RF were removed from the RF on the LRIP- AF. 2.4 Safeguard and Fiduciary Compliance 25 Two indicators were added: (i) Roads in good or fair condition as share of total classified roads measured, and (ii) Share of rural population with access to an all-season road measured. 13 2.4.1 Environmental Safeguards. The project was classified as Environmental Category B, with minor expected negative impacts on the environment. In compliance with the World Bank OP/BP 4.01 Environmental Assessment, a framework Environmental Management Plan (EMP) was developed at the project preparation stage and site–specific EMPs were produced for each individual road selected for rehabilitation. The framework EMP was disclosed and consultations on environmental aspects of the Project implementation were carried out. Site-specific EMPs were discussed with local communities residing close to road sections to be rehabilitated to ensure that those most directly affected by the project could — by voicing their opinions and suggestions — increase the quality and efficiency of the designs, and minimize negative impacts of construction and operation of the roads. Site-specific EMPs were included into bidding documents and incorporated into contracts for the provision of works to make adherence to the proposed mitigation measures mandatory for contractors. Suggestions from local communities — such as provision of additional accesses and sidewalks within settlements — were incorporated into designs. Safeguard compliance was rated as “satisfactory” for most part of the Project life. During the September 2009 supervision mission, minor shortfalls were identified in compliance with the requirements of EMPs, and in the quality of arranging drainage infrastructure and sidewalks. Thus, the environmental rating was downgraded to "moderately satisfactory". An Action Plan was prepared to address outstanding issues and to enhance the capacity of contractors and supervisors with regards to environmental performance and monitoring. Detailed instructions were provided to improve the quality of sidewalks and drainage systems. As a result, the environmental rating returned to "satisfactory" during the November 2009 supervision mission upon confirmation that the Action Plan had been fully implemented and the minor issues identified earlier had been adequately addressed. The project's environmental category and safeguard policies triggered originally for it remained the same for LRIP-AF and LRIP-AF2, because the project continued to provide support to rehabilitation of the existing roads. 2.4.2 Social Safeguards. The roads selected for rehabilitation were entirely on public land and no resettlement or land acquisition was involved in any of the sites. This was confirmed through careful review of designs and visits to all project roads. Based on the experience with the LRIP implementation, the grievance mechanisms and beneficiary communications of the LRNIP were improved to promote bottom-up accountability and transparency, and encourage stakeholder feedback26. 26 New communication tools are included such as new roadside billboards, complains telephone number and handling procedures, and official letters to community leaders. 14 2.4.3 Procurement. LRIP was prepared under the OP 8.00 and for which various procurement policy exceptions were allowed. The LRIP-AF and LRIP-AF2 followed standard Bank procurement procedures. Annual procurement post reviews confirmed that all procurement activities related to works contracts were carried out satisfactorily. The Bank observed that the number of bids per Lot decreased over the years27 in part due to the increase in the cost per Lot28. A key aspect is the fact that the road transport market has been limited throughout the past five years. As a result, only a limited number of contracts are awarded each year. This affects companies when it comes to prove similar past experience. In addition, the uncertainty associated to the future (contracts are for one year or least), does not create incentives for investments in equipment. 2.4.4 Financial Management. The Financial Management (FM) arrangements at TPIU, including planning and budgeting, accounting, financial reporting, funds flow, external auditing, and staffing remained satisfactory and acceptable to the Bank during preparation and implementation of the project. TPIU has adequate internal controls in place. The level and timelines of government counterpart contribution (GCF) was satisfactory, except for FY2010 when some delays in GCF were observed due to delayed approval of the project budget. TPIU is in compliance with the financial covenants. Particularly, adequate FM arrangements under the project were maintained. The project’s annual audited financial statements with the auditors’ unmodified opinions were always received on time. The project IFRs have always been received on time and were almost always acceptable to the Bank. 2.5 Post-completion Operation/Next Phase 2.5.1 The maintenance of the project roads is under responsibility of ARD and the Marzes. ARD uses five-year performance based contracts (PBC) for all routine maintenance works but payments are determined annually based on budget available. The budget allocation for maintenance of the lifeline roads is insufficient to cover all the needs. The next Bank involvement on Armenia in the transport sector is the LRNIP, which was built on the success of previous engagement. Its PDO is to improve access of rural communities to markets and services through upgrading of selected lifeline roads, and to strengthen the capacity of the MoTC to manage the lifeline road network. This would be achieved by: (i) improving about 170 km of lifeline roads, including road safety features piloted under LRIP, (ii) the institutional strengthening of MoTC 27 The first 25 Lots of LRIP had an average number of bids per Lot of 3.0 and a minimum number of bids per Lot of 2. The first year program of LNRIP had seven Lots with an average number of bids per Lot of 2.1and a minimum number of bids per Lot of 1. 28 The length of average Lots has been increasing, from 3.7 km in 2009 up to 12.7 km in 2012; average cost also has been increasing from US$1.02 million in 2009 up to US$2.50 million in 2012. 15 and ARD in road asset management related activities, and (iii) the implementation of two pilots on the use of microenterprises for basic routine maintenance and one pilot on performance-based contract for rehabilitation and maintenance. There is an ongoing policy dialog with the MoTC and the Ministry of Finance to address the issue of sustainable road maintenance. 3 Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 3.1.1 The PDO design and implementation of the LRIP and the AFs remain relevant to the Sustainable Development Program of the GOA. The PDO is closely aligned to the strategic objectives of the CPS for Armenia, that is, Strengthening Competitiveness for Post-Crisis Growth. The lifeline roads are referenced in the CPS Progress Report (FY09-FY13) presented to the Board in June 2011. The report highlights the importance of continued investments in lifeline roads rehabilitation in an effort to “further strengthen the CPS goals aiming at addressing vulnerability, while building the foundations for competitiveness and medium-term growth”. 3.1.2 The designs of the LRIP and the AFs are also found relevant because their components are consistent with the PDO. The civil works component sought to contribute to the achievements of the PDO in terms of rehabilitating lifeline roads and generating temporary employment. The institutional components sought to improve the quality of road designs, improve road safety and increase the technical capacity of ARD. Implementation arrangements were appropriately chosen to facilitate the implementation of the components. The choice of a lending instrument for LRIP – Emergency Recovery Loan – was appropriate to address the urgent need of support for generation of temporary employment as a result of the 2008-09 financial crises. The choice of the lending instrument for the AFs – Specific Investment Loan – was appropriate to support the additional investments in the road sector and institutional development components. 3.2 Achievement of Project Development Objectives 3.2.1 The achievement of the PDO is rated satisfactory. The rating is based on the evaluation of the achievement of the PDO, its indicators and outcomes as of December 31, 2013. The ratings consider the two objectives of the PDO: 3.2.2 Upgrade selected sections of lifeline road network (Fully Achieved). All indicators have been exceeded. In total 446 km of lifeline roads were upgraded under the LRIP and the AFs compared to a target value of 430 km. Travel 16 times29 were reduced by 58.5 percent compared to a target of 20.0 percent. The average roughness of the project roads reduced from 11.1 IRI, m/km, to 3.1 IRI, m/km, representing a 25.8 percent reduction in road user transport costs for medium trucks30 compared to a target value of 20.0 percent. If the project was not implemented, transport costs on the project roads would have increased by around 40.0 percent. The unit costs per km of the road works remained reasonable compared to the benefits and international benchmarks. Moreover, the project enabled testing less expensive solutions (DBST) for road rehabilitation that could be mainstreamed in the future. The non-development of the geometric standards did not negatively impact the achievement of this PDO because the use of proper geometric standards was addressed on the project designs. 3.2.3 Create temporary employment in road construction (Fully Achieved). The indicator on employment generation was exceeded. The LRIP and the AFs created about 39,855 person-days/month of temporary construction jobs compared to a target value of 36,650 person-days/month31 , of whom 60-70 percent were hired from local villages and 70 percent unskilled. Considering about US$500 paid per month to each worker, the total amount of direct income transfer to workers is approximately US$5.0 million in addition to indirect transfers not calculated here. In addition, jobs were created for production and transportation of construction materials32. The temporary labor after the completion of the project improved their technical skills increasing their employment opportunities. The project started the introduction of the concept of utilizing micro-enterprises 33 for routine maintenance of roads in Armenia using local labor, which could address the issue of providing permanent jobs to local labor. 29 Travel times and road roughness were measured for all project roads by the road works Supervision Consultant before the road work started and after the completion of the road works. 30 Unit transport costs, in US$ per vehicle-km, were computed using the Highway Development and Management Model for the ex-post economic evaluation. For a medium truck, the unit road user costs are US$0.40 per vehicle-km and US$0.54 per vehicle-km, with and without the project respectively. 31 To convert from average person-days/month to person-months one needs to divide by 24 (number of labor days per month) and multiply by 6 (average number of months of employment for rehabilitation works). Thus, 36,650 person-day/month corresponds to around 9,163 person-months and 39,855 person- day/month corresponds to around 9,964 person-months. The achieved 9,964 person-months corresponds to employment for 3.7 persons per km of rehabilitation works, which can be compared to the achievement of the Georgia SLPR2 project that that created 13,431 person-months of temporarily employment (for 904 km of rehabilitation works) or employment for 2.5 persons per km. 32 The study “Report on Job Creation Impact Assessment” by Arsen Petrosyan financed by the Bank in 2009 indicates that for every 100 persons-months of direct employment generated by rehabilitation works, 40 person-months are created for administration and production and transportation of construction materials. 33 In 2010 with a PPIAF trust fund, a study on Improving Routine Maintenance of Local Roads in Armenia was done that introduced the concept of micro-enterprises. As a result, a pilot for routine maintenance through micro-enterprises of lifeline roads is being financed under the LRNIP. 17 3.2.4 In addition to achieving the RF targets, the project strengthened ARD’s capacity to improve efficiency through implementing cost effective pavement designs and to improve road safety. Rehabilitation works are now designed according to traffic and include proper roads safety elements. 3.3 Efficiency 3.3.1 The efficiency in achieving the PDO is Satisfactory in terms of economic justification. An ex-ante cost benefit analysis was done at the appraisals of LRIP, LRIP-AF and LRIP-AF2 for the proposed first year program roads. The ex-post analysis reevaluated the first year program roads that were rehabilitated under the project considering actual rehabilitation costs per kilometer and annual traffic growth rates from 2009 to 2013. 3.3.2 The overall ex-post Economic Internal Rate of Return (EIRR) (18.1 percent) for the rehabilitation works is slightly lower than the overall ex-ante EIRR (18.4 percent), but above the 12 percent threshold confirming the economic justification of the project (Table 1). Efficiency was affected by slightly higher rehabilitation costs (overall 5 percent higher than the appraisal estimates), which were compensated by actual traffic growth rates observed from 2009 to 2013. Detailed economic analysis is available in Annex 3. Table 1: Ex-Post Economic Evaluation Results First Year Actual Actual Appraisal Ex-Post Program Length Investment EIRR EIRR Roads (km) (US$ million) (%) (%) LRIP 58.50 15.69 17.9 17.7 LRIP-AF 45.14 14.92 15.9 13.8 LRIP-AF2 49.30 13.75 21.1 23.1 Total 152.94 44.36 18.4 18.1 Source: World Bank Ex-Post Economic Evaluation 3.4 Justification of Overall Outcome Rating 3.4.1 The overall outcome is rated Satisfactory taking into account the substantial relevance of the PDO and achievement of Outcome Indicators, satisfactory overall design rating for relevance and implementation progress, and satisfactory economic efficiency in achieving the PDO. The project met all its physical targets. Similarly, the project substantially met its institutional development objectives34..The summary of the assessment is provided on Table 2. 34 Local designers and contractors started to successfully apply new technologies and cost-effective design standards strengthening the sustainability of investments and improving road safety for pedestrians. A road safety audit manual was prepared and is being used by MoTC for IFI projects. A data collection unit in ARD was established and is operational. 18 Table 2: Assessment of the Overall Outcome LRIP and AFs PDO 1 Relevance of the PDO, design, and implementation Relevant 2 Achievement of the PDO Satisfactory 3 Efficiency Satisfactory 4 Overall Rating Satisfactory Source: ICR Team 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 3.5.1 There was no study on poverty impacts or gender aspects undertaken during the preparation of the project. However, it is believed that the project facilitated the provision of social services and livelihood opportunities, created temporary jobs in construction, reduced transport costs and supported economic activities in Armenia35. (b) Institutional Change/Strengthening 3.5.2 Notable progress was achieved in: (i) implementing new technologies and cost- effective design standards, (ii) improving road safety by preparing and effectively adopting a Road Safety Audit Manual and piloting a “safe villages” program that is being mainstreamed under LRNIP, (iii) strengthening the ARDs capacity for monitoring the network and planning and programming road investments through the establishment of a road data collection unit, and (iv) strengthening environmental and social safeguards. (c) Other Unintended Outcomes and Impacts (positive or negative) 3.5.3 There were no noticeable negative impacts and on the positive side, at least three practices from the project – namely, road safety awareness campaigns, employment generation, and rehabilitated roads – have received wide recognition by road users and the public at large. The project has set the stage for the discussion on the sustainability of road maintenance in Armenia and the possible implementation of different roads maintenance contracts modalities, which are being addressed under the LRNIP36. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops 35 Paragraph 3.6.2 presents some findings on income on the project roads found on an impact evaluation study funded by MCC. 36 During the implementation of LRIP, the benefits of long term area wide Performance Based Contracts (PBC) including rehabilitation and maintenance works were presented to MoTC. The LRNIP will pilot a three year pilot rehabilitation and maintenance contract and a pilot routine maintenance contract through micro-enterprises. In addition, the LRNIP will finance the design of a pilot and feasibility of a PBC. 19 3.6.1 A Project Social Assessment37 of the impact of the project on communities was financed by cost savings from the project and executed by the TPIU. The study was done after the completion of the project and concluded that overall the project was effective. The study found that as a result of the project implementation there is: (i) a reduction in vehicle operating costs (both transportation fees and vehicle maintenance expenses); (ii) increase in the frequency of traffic; and (iii) a significant reduction on time spent on the roads. However, in terms of economic investments and volume of sales of agricultural products, there were no statistically significant differences probably given the short-term period of the study. Other results are available in Annex 5. 3.6.2 The MCC commissioned a study for the evaluation of its Rural Road Rehabilitation Project (RRRP)38 that included the roads financed by the Bank under LRIP. The study found significant impacts on household’s perception of road quality and their use of the roads. Treatment households were 40 percent more likely than households in the comparison group to rate lifeline roads as good or excellent. Treatment households were 19 percent more likely than households in the comparison group to rate transportation services as good or excellent. Treatment households were 20 percent less likely than households in the comparison group to encounter market access problems. However, the study found (i) no statistically significant differences of income and income- generation opportunities; and (ii) households did not report social infrastructure was more accessible. The latter probably result from the fact that household level incomes did not report total incomes (formal and informal) and missed reporting differences in incomes due to remittances, which are probably quite relevant in such analysis. Also, the report concludes that the analyses took place just after the road investments were completed and did not allow sufficient time for the benefits to materialize. Annex 5 gives other results. 4 Assessment of Risk to Development Outcome Rating: Substantial 4.1.1 The risk that the achieved development outcomes may not be sustained is Substantial. Provision of maintenance for the rehabilitated roads is needed to ensure that the current good road condition is maintained over time. The GOA budget allocations for routine maintenance for interstate and lifeline roads have not been sufficient in recent years39, which compromised the sustainability of 37 Lifeline Roads Improvement Project Social Assessment Qualitative Survey Report, Hrachya Petrosyan, TPIU, December 2013. 38 “Evaluation of an MCC-and World Bank-Financed Rural Road Rehabilitation Project in Armenia”, First Draft, November 12, 2013, Mathematica Policy Research. 39 In 2012, US$24.9 million was allocated to routine maintenance for 5,742 km of interstate and lifeline roads, and no funds were allocated to periodic maintenance. Such budget allocation for maintenance of lifeline roads was insufficient, resulting in only half of the roads receiving winter maintenance and 26 percent of lifeline roads under the responsibility of the Marzes receiving summer maintenance. 20 the project investments. Moreover, the Marzes often lack the financial resources and capacity to conduct routine maintenance. The lack of sustainability of road maintenance due to insufficient budget allocation is a chronic challenge in Armenia and it puts at great risk the sustainability of the road improvements carried out under the project. Notwithstanding an important sign of progress is that since March 2012, the legal covenants I.A.3 and I.A4 became partially complied with up to the end of the project. These covenants indicate that the Recipient and Borrower shall ensure that adequate budgetary resources are made available to the road sector for maintenance of Interstate and Republican Roads in the Borrower's annual budget. To address the issue of sufficient maintenance requires better assessment of the maintenance needs and options for financing of the sector, including for the roads under the responsibility of the Marzes. The project set the stage for the discussions on more sustained funding for road maintenance and concrete initiatives are underway toward that sustainability such as the a study on road sector sustainability 40 , the development of a road asset management system and survey equipment, and the consideration for new contacting methods like output and performance based road contracts and micro-enterprises for maintenance. 4.1.2 With regard to sustaining the policy and institutional reforms, positive steps were taken in 2009 on Road Safety, with the approval of a National Road Safety Strategy, the creation of the National Road Safety Council (NRSC) and the adoption of a five-year action plan. The NRSC has the mandate to implement the action plan and has received support from the Bank41 to improve coordination activities, strengthen capacity in road safety design within ARD and support approved road safety programs. However, the weak institutional arrangement of the Road Safety Secretariat, as a result of which this Secretariat is currently underfunded, understaffed, and almost dysfunctional to date, compromises the sustainability of the road safety program. 5 Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry – Satisfactory 5.1.1 The Bank’s performance during the preparatory periods was satisfactory with regard to strategic relevance and approach, technical, financial and economic analysis, and paying adequate attention to fiduciary and safeguards 40 The Road Financing Study, included on LRNIP, has as objectives to recommend: (i) the level of financing needed for maintenance and rehabilitation works; (ii) how all types of roads and types of road works should be financed; (iii) how all types of roads and all types of road works should be managed; and (iv) an appropriate legal framework for managing roads. 41 A grant from the Bank-administered Global Road Safety Facility (GRSF) and LRIP have supported the road safety action plan implementation. 21 arrangements. The Bank’s performance during the preparation of the LRIP was particularly highly satisfactory. The LRIP was approved under a very tight schedule and winter conditions, which required an experienced team, strong leadership and practical decisions. The LRIP-AF and LRIP-AF2 followed a more normal preparation schedule, and thus the team focused more on the institutional issues. Insufficient foresight was observed on part of the Bank for not having combined the two AFs into a single AF to reduce the processing burden. (b) Quality of Supervision – Satisfactory 5.1.2 The quality of the supervision is rated satisfactory. The Bank had been fully engaged with the client during the implementation of all planned activities, paying due attention to technical and institutional development issues. The frequent supervision missions during the first year of the project were appropriate considering the emergency nature of the project. A formal mid- term review of the AFs could have strengthened the project implementation. The Bank team was responsive to the TPIU and ARDs request for advice. The LRIP and the AFs had five task managers in five years, linked to Bank staff moving to other regions, while the TPIU manager remained the same during this period, which indicates insufficient Bank planning. The high number of task managers did not impact much the successful implementation of the project, but had an impact on the borrower’s perception of the Bank’s organization. Having different task managers helped change the focus of the project from emergency response and engineering aspects to road safety and maintenance contracting options and sustainability aspects. (c) Justification of Rating for Overall Bank Performance - Satisfactory 5.1.3 Overall, Bank performance is rated Satisfactory. The Bank assisted the Borrower in the adequate preparation of the project. The Bank also actively addressed implementation problems through proactive and intensive supervision. 5.2 Borrower Performance (a) Government Performance – Moderately Satisfactory 5.2.1 The GOA demonstrated a strong level of commitment and ownership toward the project design, preparation and implementation. The concept of the LRN was defined by the GOA before the project preparation. The GOA responded in a positive manner when the issues of counterpart funding, EMP compliance and adequate staff of the TPIU were raised by the Bank. However, the GOA did not fully ensure that adequate budgetary resources are made available to the road sector for maintenance of Interstate and Republican Roads in the Borrower's annual budget, which compromises the sustainability of the investments. 22 (b) Implementing Agency or Agencies Performance – Satisfactory 5.2.2 The project was implemented by a Transport Project Implementation Unit (PIU), which functioned well, delivered overall satisfactory project outcomes. The TPIU has experienced staff and knows well Bank procedures. There were no English language constraints. When needed, the staff of the TPIU was increased to cover its additional obligations. The TPIU was managed, during the entire period of the project preparation and implementation, by one experienced engineer with good knowledge of Bank and Armenian procedures. The TPIU handled well safeguards and fiduciary issues. (c) Justification of Rating for Overall Borrower Performance – Moderately Satisfactory 5.2.3 Considering that the performance of the GOA is moderately satisfactory, the overall performance of the Borrower is rated as Moderately Satisfactory. While its implementing agency performance is satisfactory, the impact of the GOA performance was considered higher than the implementing agency performance on achieving the project objectives. 6 Lessons Learned 6.1.1 The Crisis and Emergencies Bank instrument and procedures offered advantages to assist in the Government’s situation. The project was possible because the GOA had pre-identified critical road links by adopting a Rural Infrastructure Strategy and Action Plan and started its implementation as conditions of adjustment credits, and availability of basic data and designs that only needed to be updated. The Bank assigned a very experienced project preparation team, mobilized international experts quickly and provided extensive supervision during implementation. Incremental improvements were made during project implementation adjusted to the current conditions (for example: move to more efficient designs) because of the phased approvals. The project used technical supervision as a means of technology transfer and project monitoring by employing for construction supervision an international consultant who had a local-sub-consultant and encouragement for collaboration by ARD between the supervision consultant, technical auditor and contractors. An experienced TPIU regularly visited contractors to oversee works and was supportive of the adoption of new technologies. 6.1.2 The lessons learnt from implementation of the project are: (i) road investments can support temporary employment generation and provide quick economic support to rural areas; (ii) incremental improvements can be made during project implementation (e.g., move toward more efficient designs) if done step by step and adjusted to current conditions; 23 (iii) road safety audits can be successfully mainstreamed to be undertaken on construction and rehabilitation works when there is Government ownership; (iv) there is a need to develop in-house capacity of the road administration so design engineers and contractors can be properly supervised; (v) strengthen the technical capacity of local engineers and contracting capacity is needed to implement new technologies. (vi) on emergency projects, make use of “walkover” surveys to help detect and address design oversights, as they can be on projects that need to be prepared quickly; (vii) the project should construct pavement trial sections before introducing new pavement technologies; (viii) support for road safety programs are easily obtained from communities, which consider road safety a major concern; (ix) technical assistance activities, which require revision of standards or practices should always be designed with close collaboration between international and local consultants to avoid international consultants working without benefiting from local experience; (x) periodic maintenance funding is critical for the sustainability of investments for gravel roads where grading and gravel resurfacing are needed to maintain proper levels of service as observed on project gravel roads; (xi) there is a need to review the post-qualification requirements, contracts sizes, unit rates, and disclosure requirements for rehabilitation works to increase the competition level; and (xii) many TTLs during project implementation should be avoided by better Bank planning. 6.1.3 The project showed that to obtain support from local communities where possible, the designs should provide sidewalks in build-up areas (roadside strip development) and villages to improve access to public facilities and ensure pedestrian safety, and the project should promote local employment on the rehabilitation works. The use of cement concrete guardrails (labor intensive construction method) was experimented during the first year implementation of the project but this method was not mainstreamed due to its negative effect on snow removal activities. 7 Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 7.1.1 The Borrower submitted to the Bank its Completion Report (CR) on April 22, 2014. Please refer to Annex 7. The Borrower submitted on June 5, 2014 clarification comments on the content of the Bank’s ICR that were incorporated in the final ICR. The Borrower concurred with the Bank’s proposed rating (Satisfactory) of the Project Outcome, but contested the “moderately satisfactory” rating for Government Performance that resulted in 24 the “moderately satisfactory” rating for overall performance of the Borrower, indicating that the MoTC gradually increases budgetary resource allocation to the routine maintenance of Interstate and Republican Roads42. The ICR reflects the budgetary resource allocation to routine maintenance during the overall project period. (b) Co-financiers Not Applicable (c) Other partners and stakeholders Not Applicable 42 In the approved 2014 state budget the Interstate and Republican Roads routine maintenance financing is AMD 5.675 billion or 106.4% of 2013 actual financing (AMD 5.334 billion). According to the 2015-2017 Midterm Expenditures Framework, the following financing is projected for the routine maintenance of Interstate and Republican Roads: 2015 - AMD 7.635 billion (143 percent of 2013 financing); 2016 - AMD 8.197 billion (153.6 percent of 2013 financing); and 2017 - AMD 8.863 billion (166.2 percent of 2013 financing). 25 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Appraisal Latest Percentage Estimate Estimate Of Components (USD millions) (USD millions) Appraisal Civil Works, Design and Supervision 117.19 124.00 106% Goods and Technical Assistance 5.01 1.88 38% Total Baseline Cost 122.20 125.88 103% Contingencies 3.71 0.00 0% Total Project Costs 125.91 125.88 100% Front-end fee IBRD 0.19 0.19 100% Total Financing Required 126.10 126.07 100% LRIP Appraisal Latest Percentage Estimate Estimate Of Components (USD millions) (USD millions) Appraisal Civil Works, Design and Supervision 29.27 30.08 103% Goods and Technical Assistance 0.40 0.39 97% Total Baseline Cost 29.67 30.47 103% Contingencies 0.73 0.00 0% Total Project Costs 30.40 30.47 100% Total Financing Required 30.40 30.47 100% LRIP-AF Appraisal Latest Percentage Estimate Estimate Of Components (USD millions) (USD millions) Appraisal Civil Works, Design and Supervision* 42.12 44.85 106% Goods and Technical Assistance 3.51 0.87 25% Total Baseline Cost 45.63 45.73 100% Total Project Costs 45.63 45.73 100% Front-end fee IBRD 0.09 0.09 100% Total Financing Required 45.72 45.82 100% * Including contingencies LRIP-AF2 Appraisal Latest Percentage Estimate Estimate Of Components (USD millions) (USD millions) Appraisal Civil Works, Design and Supervision 45.80 49.06 107% Goods and Technical Assistance 1.10 0.62 56% Total Baseline Cost 46.90 49.68 106% Physical Contingencies 2.98 0% Total Project Costs 49.88 49.68 100% Front-end fee IBRD 0.10 0.10 100% Total Financing Required 49.98 49.78 100% 26 (b) Financing Appraisal Actual/Latest Percentage Estimate Estimate of Source of Funds (USD millions) (USD millions) Appraisal Borrower 24.50 24.57 100% International Bank for Reconstruction and Development 76.60 76.50 100% International Development Association (IDA) 25.00 24.99 100% Total Funding 126.10 126.07 100% Original Appraisal Actual/Latest Percentage Estimate Estimate of Source of Funds (USD millions) (USD millions) Appraisal Borrower 5.40 5.49 102% International Bank for Reconstruction and Development International Development Association (IDA) 25.00 24.99 100% Total Funding 30.40 30.47 100% AF1 Appraisal Actual/Latest Percentage Estimate Estimate of Source of Funds (USD millions) (USD millions) Appraisal Borrower 9.12 9.22 101% International Bank for Reconstruction and Development 36.60 36.60 100% International Development Association (IDA) Total Funding 45.72 45.82 100% AF2 Appraisal Actual/Latest Percentage Estimate Estimate of Source of Funds (USD millions) (USD millions) Appraisal Borrower 9.98 9.87 99% International Bank for Reconstruction and Development 40.00 39.91 100% International Development Association (IDA) Total Funding 49.98 49.78 100% 27 Annex 2. Outputs by Component Table 1. Project Outcome and Intermediate Outcome Indicators First Second Status Original AF AF At Baseline Target Target Target Closing Remarks Project Outcome Indicators Travel time improvements to 0% 20% 20% 20% 58.5% Target achieved. drive the length of the sections over current situation Number of person-day/month 0 7,650 17,650 36,650 39,855 Target achieved. of jobs created One Pilot of “safe villages” No Yes Yes “Safe villages” program program implemented successfully implemented. Transport cost reduction on 0% 20% 25.8% Target achieved. project roads Intermediate Outcome Indicators Component 1: Rehabilitation of the Lifeline Road Network Number of km upgraded 0km 100 km 240 430 446 km The targets for the km km upgrading of the Lifeline Road Network were achieved, exceeding expectations. Roads in good and fair No Yes Yes The roads condition is condition as a share of total being measured. This is classified roads measured a monitoring indicator for LRNIP. The 2013 value is 51.5 percent. Share of rural population with No Yes Yes The share of rural access to an all-season road population is being measured measured. This is a monitoring indicator for LRNIP. The 2013 value is 54.1 percent. Component 2: Technical Assistance Road Safety Audit Manual is No Yes Yes The Road Safety Audit developed Manual was developed in 2010. International consultant conducted training and introduced main principles of road safety audit to MoTC, ARD, PIU, Traffic Police and design consultant’s staff. Establishment of a data No Yes Yes The data collection unit collection unit in ARD was established in 2011. New road geometry design No Yes No Was dropped out during standards developed project review in 2011. 28 Annex 3. Economic and Financial Analysis 1. The Cost-benefit analysis done at appraisal for the LRIP, LRIP-AF and LRI-AF2 was carried out using the Highway Development and Management Model (HDM-4), which simulates highway life cycle and vehicle operating conditions and costs for multiple road design and maintenance alternatives. The main economic benefits of the project are vehicle operating cost and travel time savings. The main economic costs of the project are road rehabilitation and maintenance costs. The vehicle operating cost savings were estimated using the consumption of resources for each component of vehicle operating cost based on the surface condition and geometry of each road section, and the characteristics of representative vehicles. Travel time savings are obtained when road improvements lead to an increase in vehicle speeds, and thus reducing the journey times of passengers. The ex-post evaluation of the project was done also using the HDM- 4 model, but taking into account current rehabilitation costs and traffic growth rates from 2009 to 2012. 2. For the LRIP, 12 road sections were evaluated at appraisal yielding an overall EIRR of 20.3 percent43. For the LRIP-AF, 15 road sections were evaluated at appraisal yielding an overall EIRR of 16.2 percent. For the LRIP-AF2, 8 road sections were evaluated at appraisal yielding an overall EIRR of 21.1 percent. Some road sections evaluated at appraisal were dropped from the project or were merged to form lots. Table 1 presents the road sections evaluated at the appraisals and that were included in the project44. For these road sections (160 km totaling US$44.27 million or 38 percent of the project costs), the overall ex-ante EIRR is 18.4 percent, the average traffic is 342 vehicles per day and the average upgrading cost is US$276,788 per km. Table 3: Economic Evaluation done at Appraisal Length Traffic Investment Investment EIRR (km) (AADT) (US$ M) (US$/km) (%) 1 Vardenut-link to M3 Highway 3.50 363 0.97 277,143 27.2 2 Voskehask-link to M7 Highway 2.00 180 0.56 280,000 14.9 3 Karchaghbyur-link to M11 Highway 1.30 166 0.36 276,923 14.3 4 Akunk-Vardenis 2.10 143 0.58 276,190 18.8 5 Chkalov-link to M6 Highway 2.40 165 0.67 279,167 21.6 6 Dzoragyugh-link to M10 Highway 5.00 200 1.39 278,000 17.2 7 Aygabats-Gywmi 10.40 206 2.89 277,885 16.3 8 Zorakert-Ardenis-Aghvorik-Tavshut-link to 10.10 249 2.81 278,218 23.7 43 The LRIP PP indicates an overall ERR of 20.3 percent, but the examination of the ERRs computed per road section indicate that this value is not correct. The correct overall ERR for the LRIP is 17.9 percent if one takes into account the ERRs per road section presented on Table A9.2 of the PP and weights them by the investment. Table A9-2 presents 13 road sections, but links 8 and 9 are the same road section; thus, one has 12 road sections. 44 The EIRR for LRIP-AF (15.9) on Table 3 is different from the PP EIRR (16.2 percent) because it takes into consideration only the 12 road sections included in the project, instead of the 15 road sections presented at the PP. 29 M1 Highway 9 L.Karmiraghbyur-Berd 7.20 190 2.00 277,778 16.3 10 Chochkan-Pokr Ayrum 7.90 151 2.20 278,481 15.5 11 Kurtan-Gyulakarak 8.10 181 2.26 279,012 20.2 12 Aghavnadzor-link to M2 Highway 6.00 186 1.67 278,333 7.6 First Year Program LRIP 66.00 202 18.36 278,182 17.9 1 Brnakot – Sisian 4.75 253 1.14 240,000 15.3 2 H3, Yeravan - Garni – Geghard 3.03 891 1.67 549,600 22.1 3 H62, Sisian – Aghitu 4.68 270 1.26 268,440 15.9 4 M12 – Hartashen 5.00 237 1.25 250,600 15.6 5 Alvanq - M17 2.60 172 0.56 216,000 13.9 6 M17 – Meghri 3.00 228 0.72 240,000 15.3 7 Shvanidzor - M17 1.68 151 0.36 214,800 13.0 8 M2 – Agarak 2.18 231 0.49 225,600 14.8 9 M2 - Kavjut – Geghavanq 6.00 165 1.42 237,120 13.2 M2 - Levaz - Vahravar - Gudemnis - Kuri – 10 Karjevan 4.33 156 1.04 240,000 12.9 11 M2 – Vardanidzor 0.60 75 0.05 90,600 15.5 12 Tcambarak – Vahan 6.10 263 1.54 252,000 16.6 First Year Program LRIP AF1 43.95 263 11.50 261,718 15.9 1 M2 – Halidzor 12.00 631 4.34 361,446 19.0 2 M3 – Paghaghbyur 8.50 537 2.25 265,060 21.9 3 T-4 - 16 Gavar – Noratus 2.10 578 0.56 265,060 21.7 4 M10 – Lchashen 1.70 1,421 0.45 265,060 42.5 5 M4 – Sevaberd 8.50 752 2.25 265,060 25.3 6 M4 - Sevaberd M4 – Sevaberd 7.80 334 2.07 265,060 17.1 7 M9 Talin – Dashtabem 5.30 527 1.40 265,060 20.4 8 Agarak – Karchevan 4.10 537 1.09 265,060 16.5 First Year Program LRIP AF2 50.00 595 14.41 288,193 21.1 Total 159.95 342 44.27 276,788 18.4 Source: World Bank PPs. 2008 Traffic for LRIP, 2009 Traffic for LRIP-AF and 2010 Traffic for LRIP-AF2 3. Table 2 presents the actual rehabilitation costs per road section and the ratio between the actual and appraisal cost per km. The actual costs per km of the LRIP were on average 96 percent of the appraisal estimates, the actual costs per km of the LRIP-AF were on average 126 percent of the appraisal estimates, and the actual costs per km of the LRIP-AF2 were on average 97 percent of the appraisal estimates. Overall, for the actual 153 km, the actual costs per km increased by five percent compared with the appraisal estimates. In addition, the table below shows the actual traffic in 2013 and the annual traffic growth since appraisal. The traffic on the LRIP roads increased by 3.5 percent from 2008 to 2013. The traffic on the LRIP-AF roads increased by 5.6 percent from 2009 to 2013. The traffic on the LRIP-AF2 roads increased by 6.3 percent from 2010 to 2013. Table 4: Actual Investment Costs and 2013 Traffic Length Investment 2013 Traffic (US$ Act/App Growth (km) M) (US$/km) Ratio (AADT) (%/year) 1 Vardenut-link to M3 Highway 3.50 0.94 268,571 0.97 415 2.7% 30 2 Voskehask-link to M7 Highway 2.10 0.68 323,810 1.16 208 2.9% 3 Karchaghbyur-link to M11 Highway 1.90 0.57 300,000 1.08 189 2.6% 4 Akunk-Vardenis 2.10 0.83 395,667 1.43 166 3.0% 5 Chkalov-link to M6 Highway 2.40 0.41 171,498 0.61 191 3.0% 6 Dzoragyugh-link to M10 Highway 4.90 0.92 187,755 0.68 245 4.1% 7 Aygabats-Gywmi 10.40 2.87 275,962 0.99 235 2.7% Zorakert-Ardenis-Aghvorik- 8 Tavshut-link to M1 Highway 2.10 0.74 352,381 1.27 256 0.6% 9 L.Karmiraghbyur-Berd 7.20 1.39 193,056 0.70 205 1.5% 10 Chochkan-Pokr Ayrum 7.80 1.97 252,564 0.91 188 4.5% 11 Kurtan-Gyulakarak 8.10 3.13 386,420 1.38 273 8.6% 12 Aghavnadzor-link to M2 Highway 6.00 1.24 206,667 0.74 252 6.3% First Year Program LRIP 58.50 15.69 268,248 0.96 240 3.5% 1 Brnakot - Sisian 4.70 1.69 359,574 1.50 298 4.2% 2 H3, Yeravan - Garni - Geghard 3.20 0.93 290,625 0.53 1411 12.2% 3 H62, Sisian - Aghitu 4.70 1.60 340,426 1.27 308 3.3% 4 M12 - Hartashen 5.10 1.12 219,608 0.88 261 2.4% 5 Alvanq - M17 2.60 0.83 319,231 1.48 196 3.3% 6 M17 - Meghri 3.00 1.04 346,667 1.44 256 2.9% 7 Shvanidzor - M17 1.68 0.46 273,810 1.27 185 5.2% 8 M2 - Agarak 3.56 0.72 202,247 0.90 263 3.3% 9 M2 - Kavjut - Geghavanq 6.00 2.50 416,667 1.76 189 3.5% M2 - Levaz - Vahravar - Gudemnis - 10 Kuri - Karjevan 4.30 1.71 397,674 1.66 175 2.9% 11 M2 - Vardanidzor 0.60 0.17 283,333 3.13 91 5.0% 12 Tcambarak - Vahan 5.70 2.15 377,193 1.50 296 3.0% First Year Program LRIP AF1 45.14 14.92 330,527 1.26 327 5.6% 1 M2 - Halidzor 12.70 4.97 391,339 1.08 785 7.6% 2 M3 - Paghaghbyur 4.40 1.41 320,455 1.21 563 1.6% 3 T-4 - 16 Gavar - Noratus 2.20 0.20 90,909 0.34 877 14.9% 4 M10 - Lchashen 1.70 0.25 147,059 0.55 1508 2.0% 5 M4 - Sevaberd km 6+800 km 15+300 8.50 0.81 95,294 0.36 799 2.0% 6 M4 - Sevaberd km 15+300 km 23+100 8.00 1.57 196,250 0.74 651 24.9% 7 M9 Talin - Dashtabem 7.70 2.93 380,519 1.44 559 2.0% 8 Agarak - Karchevan 4.10 1.61 392,683 1.48 568 1.9% First Year Program LRIP AF2 49.30 13.75 278,905 0.97 715 6.3% Total 152.94 44.36 290,065 1.05 413 4.8% Source: TPIU 4. Table 5 presents the length and actual costs of the project roads. The average rehabilitation cost per km of the LRIP, LRIP-AF and LRIP-AF2, is US$243,215 per km, US$326,316 per km, and US$231,427 per km respectively, with an overall cost of US$261,431 per km. LRIP-AF unit costs are higher because all LRIP-AF roads required rehabilitation, while some LRIP-AF2 roads required periodic maintenance. 31 Table 5: Project Costs Number Number Length Total Investment Investment of Roads of Lots (km) (US$ million) (US$/km) LRIP 18 30 118.2 28.7 243,215 LRIP-AF 30 34 126.5 41.3 326,316 LRIP-AF2 20 25 201.8 46.7 231,427 Total 68 89 446.5 116.7 261,431 Source: TPIU 5. As can be seen in the table below45, since 2009 the average length of the lots increased, from 3.69 km up to 12.66 km; average cost per lot also increased from US$1.02 million up to US$2.50 million. Table 6: Evolution of Average Costs and Length per Lot since 2009 Average Average Average Total Number Cost per Lot Length per Lot Cost per Km Investment Year of Lots (US$ million) (km) (US$/km) (US$ million) 2009 43 1.02 3.69 277,360 44.0 2010 32 1.30 4.55 286,518 41.7 2011 5 1.69 5.60 302,500 8.5 2012 9 2.50 12.66 197,845 22.5 Total 89 1.31 5.01 261,431 116.7 Source: TPIU 6. The LRIP appraisal economic analysis was done considering the relation between annual traffic growth rates and estimated annual GDP growth rates. It was assumed at appraisal that the annual growth rate in 2009 was zero, and the annual growth rates for 2010, 2011 and 2012 were 4.4, 5.5 and 6.6 percent per year respectively (6.6 percent after 2012). The LRIP-AF and LRIP-AF2 lowered the estimated annual growth rates to 1.65 percent in 2010 and 5.5 percent onwards. The table below presents the estimated annual traffic growth rates and the actual traffic growth rates found on the lifeline roads since 2008. The ex-post economic evaluation was done with the actual traffic growth rates from 2008 to 2013 and assuming an annual traffic growth rate of 5.7 percent after 2013 considering that the IMF46 predicts an average annual growth in GDP of 5.2 percent per year from 2013 to 2018. Table 7: Traffic Annual Growth Rates (%) 2009 2010 2011 2012 LRIP estimates 0.00 4.40 5.50 6.6 onwards LRIP-AF estimates 0.00 1.65 5.5 onwards LRIP-AF2 estimates 0.00 1.65 5.5 onwards LRIP: actual from 2008 to 2013 = 3.5% per year on average LRIP-AF: actual from 2009 to 2013 = 5.6% per year on average LRIP-AF2: actual from 2010 to 2013 = 6.3% per year on average 45 2009 data include LRIP and LRIP-AF, 2010 data include LRIP-AF and LRIP-AF2, and 2011 and 2012 data include only LRIP-AF2. 46 IMF World Economic Outlook Database October 2013. 32 Source: World Bank PPs and TPIU 7. The ex-post economic evaluation was done with the actual costs per km and actual traffic growth rates. The table below shows the ex-post economic evaluation results. The overall ex-post EIRR for the three projects is 18.1 percent that is slightly lower than the overall EIRR based on the appraisal estimates (18.4 percent). The ex-post EIRR of the LRIP-AF (13.8 percent) is lower than for other projects due to the higher upgrading costs per km, but still is higher than 12 percent. The ex-post economic evaluation shows that the original project and the two additional financings were economically justified. Table 8: Ex-Post Economic Evaluation Results First Year Actual Appraisal Ex-Post Program Length Investment EIRR EIRR Roads (km) (US$ million) (%) (%) LRIP 58.50 15.69 17.9 17.7 LRIP-AF 45.14 14.92 15.9 13.8 LRIP-AF2 49.30 13.75 21.1 23.1 Total 152.94 44.36 18.4 18.1 Source: ICR ex-post economic evaluation 33 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Christopher R. Bennett Senior Transport Specialist EASNS TTL Tamara Sulukhia Sector Leader ECSSD Co-TTL Asif Faiz Consultant ECSTR Highway Engineer Alexander Astvatsatryan Senior Procurement Officer ECSO2 Procurement Officer Anarkan Akerova Council LEGCF Council Anders Bonde Consultant ECSTR Highway Engineer Ani Balabanyan Senior Energy Specialist ECSEG Operations Officer Senior Financial Management Financial Management ECSO3 Arman Vatyan Specialist Specialist Arthur Kochnakyan Energy Economist ECSEG Financial Analyst Coral Bird Program Assistant ECSTR Program Assistant Darejan Kapanadze Senior Environmental Specialist ECSEN Environment Elizabeth C. Wang Senior Financial Officer ECSTR Senior Financial Officer Garik Sergeyan E T Consultant ECSO3 Financial Management Junior Professional ECSTR Gibet Camos-Daurella Junior Professional Associate Associate Hannah M. Koilpillai Senior Finance Officer ECSO3 Senior Finance Officer Economist / Social Development Economist / Social EASTS Satoshi Ishihara Specialist Development Specialist Supervision/ICR  Christopher R. Bennett Senior Transport Specialist EASNS TTL Satoshi Ishihara Social Development Specialist EASTS Co-TTL Olivier P. Le Ber Lead Transport Specialist MNSTI TTL Petrus Benjamin Gericke Lead Transport Specialist AFTTR TTL Vickram Cuttaree Senior Infrastructure Economist MNSTI TTL ICR TTL and Principal Rodrigo Archondo-Callao Senior Highway Engineer ECSTR Author Alexander Astvatsatryan Senior Procurement Specialist ECSO2 Procurement Darejan Kapanadze Senior Environmental Specialist ECSEN Environment Jiangbo Ning Highway Engineer ECSO3 Highway Engineer Senior Financial Management Arman Vatyan ECSO3 Financial Management Specialist Junior Professional Alejandro Lopez Martinez Junior Professional Associate ECSTR Associate Asif Faiz Consultant ECSTR Highway Engineer David Silcock Consultant ECSTR Road Safety Elena Y. Chesheva Transport Specialist SASDT Operations Officer Ani Balabanyan Senior Energy Specialist ECSEG Operations Officer Junior Professional Gilbet Camos-Daurella Junior Professional Associate ECSTR Associate Anil Somani Consultant ECSTR Environment Arthur Kochnakyan Consultant ECSTR Financial Analyst Henry Kerali Country Director ECCU3 Sector Manager I.U.B. Reddy Senior Social Development Specialist SASDS Social 34 Irina Tevosyan Program Assistant ECCAR Program Assistant Marie A. Laygo Program Assistant ECSSD Program Assistant Armine Aydinyan Consultant ECSHD Procurement Cesar Queiroz Consultant ECSTR Highway Engineer Zaruhi Tokhmakhian Senior Infrastructure Specialist ECSUW Operations Officer Marinos Skempas Consultant ECSTR Highway Engineer Nargis Ryskulova Operations Officer ECSTR Operations Analyst Chukwudi Okafor Senior Social development Specialist AFTCS Social Junior Professional Michael Butler Junior Professional Associate ECSTR Associate Garik Sergeyan E T Consultant ECSO3 Financial Management Sergo Tsipa Consultant ECSTR Road Engineer Sarah Michael Senior Social Development Specialist ECSSO Social Cecilia Corvalan Senior Transport Economist LCSTR Transport Economist (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only)* Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY09 36.42 187.89 FY10 19.25 118.57 FY11 0.00 -1.89 Total: 55.67 304.57 Supervision/ICR FY09 6.14 42.79 FY10 38.32 172.31 FY11 33.66 141.70 FY12 20.54 104.86 FY13 23.43 87.07 FY14 13.99 56.24 Total: 136.08 604.97 *Includes budget allocations for LRIP AF-1 and LRIP AF-2 preparation 35 Annex 5. Beneficiary Survey Results 1. A social assessment qualitative 47 done after the completion of the project, evaluating the LRIP’s impact on communities concluded that overall the project was effective. The survey was conducted in eight rural communities of eight regions of Armenia with a total number of 48 focus groups and in-depth interviews. The study found that as a result of the project implementation there is: (i) a reduction in vehicle operating costs (both transportation fees and vehicle maintenance expenses); (ii) increase in the frequency of traffic; (iii) a significant reduction of time spent on the roads, and (iv) communities residents have been involved in the construction works as a temporary labor force with reasonable remuneration. However, in terms of economic investments and sales of agricultural products, there was no significant growth in the population’s living standards observed in the short-term period of the study 2. The previous condition of the roads included in the survey was estimated extremely bad by all beneficiaries, with high presence of potholes and absence of pedestrian sidewalks. Before road rehabilitation, community residents visited hospitals only in extreme cases, while after road rehabilitation; access to hospitals is not an issue anymore. During project design, public awareness meetings were held with project implementers, local authorities and community representatives. In general, women’s involvement in public awareness meetings was very low. In all communities some part of the residents, mostly man, has been involved in the construction works as labor force. Taxi traffic has grown in all beneficiary communities and in some communities fees for taxi services have been reduced twice. The rehabilitation of roads had positive and negative impact on private entrepreneurs who own stores. The positive impact is the growth in the number of supplier of goods; the negative impact is the decrease of resident clients due to benefits of new created opportunities to access larger markets outside of their communities. 3. Most of the survey respondents indicated that as a result of the rehabilitation an increase in volume of agricultural product sales is not noticeable from the point of view of profits partly due to disproportional inflation, which did not allow having a significant increase in revenues. Among the beneficiary communities, there is no any noticeable change in agricultural and non-agricultural incomes. As a result of the rehabilitation, traffic increased and the communication between friends and relatives became easier. The road rehabilitation has enhanced not only connections within communities but also connections with regional centers and other cities. Reduction in travel time to health infrastructure reduced health hazards and risks. Both passengers and drivers have mentioned that a result of the rehabilitation works stress was reduced. The project had positive impact on those communities where schools were located nearby reconstructed roads by solving the issues of: (i) economic problems with eroded clothes and shoes; (ii) daily problems with collected dirt on clothes and shoes; (iii) personal safety problems; 47 Lifeline Roads Improvement Project Social Assessment Qualitative Survey Report, Hrachya Petrosyan, TPIU, December 2013. Study financed with savings from the project and executed by the TPIU social specialist with the assistance of consultant researchers. 36 and (iv) education problems displayed by being late or absent from the lessons. The compliance of LRIP with beneficiaries' needs was highly valued by all participants in the survey. Good condition of the road was considered as one of the most important pillars of community life and is one of the basic preconditions for raising living standards of the community. Survey participants indicated that all residents of the community benefited from the program, especially indicating the drivers and entrepreneurs. 4. LRIP rehabilitated lifeline roads that were identified under the Millennium Challenge Corporation (MCC) project titled “Rural Road Rehabilitation Project (RRRP)” defined in 2007. The RRRP was designated to rehabilitate up to 943 km of lifeline roads, but the project was canceled48 after rehabilitating only 24.4 km. Thus, the LRIP took over the rehabilitation of priority lifeline roads not rehabilitated under the RRRP. Despite the low number of kilometers rehabilitated under the RRRP, the MCC commissioned a study49 for the evaluation of its rural roads project. The study included the roads financed by the World Bank under LRIP. 5. The data for the RRRP evaluation came from the Integrated Living Condition Survey (ILCS). The ILCS is an annual national household survey fielded by the National Statistics Service of Armenia. The MCC provided funding and increased the sample size and longer questionnaire from 2007 to 2011 to facilitate the RRRP evaluation focusing on lifeline roads communities. The treatment group included communities served by 27 roads rehabilitated under RRRP or LRIP, and the comparison group included communities served by 29 roads included in the RRRP plans, but were not ultimately rehabilitated with MCC or World Bank Financing. 6. The study found significant impacts on household’s perception of road quality and their use of road links. Treatment households were 40 percent more likely than households in the comparison group to rate regional roads as good or excellent (see table below). Treatment households were 19 percent more likely than households in the comparison group to rate transportation services as good or excellent. Treatment households were 20 percent less likely than households in the comparison group to encounter market access problems. The study also observed a positive and statistically significant impact on the likelihood of utilizing roads for shopping or visiting relatives and other purposes (16 percent of difference) other than getting to work or buying and selling agricultural projects. However, the study found: (i) there were no marked impacts on use of roads for income-generating reasons; (ii) households did not report that social infrastructure was more accessible; and (iii) there were no observed impacts on income within the short evaluation time frame. 48 In June 2009, the MCC Board made the decision to discontinue funding any further road rehabilitation under the RRRP due to concerns about Armenia’s democratic governance. 49 “Evaluation of an MCC-and World Bank-Financed Rural Road Rehabilitation Project in Armenia”, First Draft, November 12, 2013, Mathematica Policy Research. 37 Table 9: Impacts on Perceived Road Quality and Utilization (Percentage of Respondents Reporting) Treatment Comparison Group Group Regional Roads are Good or Excellent 52% 12% Transportation Services are Good or Excellent 41% 22% No Market Access Problems 91% 71% Use Roads to Shop, Visit Relatives and Other Purposes 88% 72% Source: MCC Study 38 Annex 6. Stakeholder Workshop Report and Results Not Applicable 39 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Part I. Project Objectives: The Project development objective (PDO) of the Lifeline Roads Improvement Project (LRIP) for Armenia was to upgrade selected sections of the lifeline road network (LRN) and create temporary employment in road construction. In response to the request of the Government of Armenia (GoA), the original LRIP (US$25 million equivalent IDA Credit) was prepared under the "IDA Financial Crisis Response Fast-Track Facility" utilizing OP 8.0, Rapid Response to Crisis and Emergencies, in order to mitigate the impact of the global financial crisis in Armenia. The LRIP was approved by the Board on February 24, 2009, and became effective on April 10, 2009. The Project Development Objective of the LRIP is to upgrade 100 km selected sections of the LRN and create temporary employment in road construction. On August 27, 2009, the Board approved the First Additional Financing (LRIP-AF1) in the amount of US$36.6 million (IBRD- 7751-AM). The LRIP-AF1 aimed to scale up the project's activities through rehabilitating an additional 140 km of the LRN. The LRIP-AF1 became effective on November 20, 2009. On July 15, 2010, the Board approved the Second Additional Financing (LRIP- AF2) in the amount of US$40.0 million (IBRD-7936AM). The LRIP-AF2 aimed to scale up the project's activities through rehabilitating an additional 190 km of the LRN. The LRIP-AF2 became effective on November 15, 2010. In overall, the project aimed to rehabilitate about 430 km of the LRN and create 36,650 person-months of temporary jobs in road construction by the closing date, i.e. December 31, 2013. These objectives were achieved through the implementation of the following components: Credit No 4549-AM (original project) Component 1: Rehabilitation of the Lifeline Road Network (US$30.0 m including contingencies): Rehabilitation of approximately 100 km of the LRN. The road sections are located in the seven different regions of the country and their lengths vary between 1 and 14 km. The component includes: (i) civil works for road rehabilitation; (ii) consultancy services for the construction supervision, design author supervision, and technical auditing of rehabilitation works; (iii) updating of the original MCA-financed 2007 designs and environmental documents to meet the requirements of the project; and (iv) project implementation expenses including, but not limited to, funding project audits, incremental operational implementation costs and additional costs for intensified project supervision. Component 2: Technical Assistance (US$0.4 m including contingencies): Technical assistance, including but not limited to: (i) a study to review low cost pavement options for Armenia which will explore options for different pavement types and ways to increase labor based activities; (ii) updating of designs for a potential future project; (iii) a vehicle for field supervision; and (iv) related training. Financial results: 40 Appraisal Latest Estimate Estimate Components (USD millions) (USD millions) Civil Works, Design and Supervision 30,0 30,08 Goods and Technical Assistance 0,4 0,39 Total Baseline Cost * 30,4 30,47 Total Project Costs 30,4 30,47 Total Financing Required 30,4 30,47 *Including contingencies Appraisal Estimate Actual/Latest (USD Estimate Source of Funds millions) (USD millions) Borrower 5,4 5,49 International Bank for Reconstruction and Development International Development Association (IDA) 25 24,99 Total Funding 30,40 30,47 Loan No 7751-AM (additional financing) Component 1: Rehabilitation of 140 km of selected sections of the lifeline roads. This component comprises civil works for the rehabilitation of LRN roads; supervision of the works; design author supervision; technical auditing of rehabilitation works; and project implementation expenses including, but not limited to, funding technical and financial audits, monitoring and evaluation, and incremental costs. These operations did not require any land acquisition. This component also financed the “safe villages” pilot program which would support rural communities in implementing road safety measures recommended by the road safety audit and the National Road Safety Strategy. Component 2: Technical assistance. This component financed the procurement of vehicles for supervision; training; and design for future investments. This component also supported the development of road safety audit manual, tailored to the conditions in Armenia, to provide checklists for assessing designs and inspecting projects sites throughout different phases of road project development, including assessments of existing roads. Financial results: 41 Appraisal Latest Estimate Estimate Components (USD millions) (USD millions) Civil Works, Design and Supervision* 42,12 44,85 Goods and Technical Assistance 3,51 0,87 Total Baseline Cost 45,63 45,73 Total Project Costs 45,63 45,73 Front-end fee IBRD 0,092 0,092 Total Financing Required 45,72 45,82 Include 2.5 percent contingencies Appraisal Estimate Actual/Latest (USD Estimate Source of Funds millions) (USD millions) Borrower 9,12 9,22 International Bank for Reconstruction and Development 36,6 36,6 Total Funding 45,72 45,82 Loan No 7936-AM (second additional financing) Component 1: Rehabilitation of approximately 190 km of selected sections of lifeline roads. This component comprises civil works for the rehabilitation of the roads; supervision of the works; technical auditing of rehabilitation works; and project implementation expenses including, but not limited to, funding technical and financial audits, monitoring and evaluation, and incremental operating costs. This operation will not require any land acquisition. Component 2: Technical assistance. This component would finance the development of new road geometric standards, feasibility studies and designs of future investments and the procurement of road data collection equipment. Financial results: 42 Appraisal Latest Estimate Estimate Components (USD millions) (USD millions) Civil Works, Design and Supervision 45,8 49,06 Goods and Technical Assistance 1,1 0,62 Total Baseline Cost 46,9 49,68 Physical Contingencies 2,98 Total Project Costs 49,88 49,68 Front-end fee IBRD 0,10 0,10 Total Financing Required 49,98 49,78 Appraisal Estimate Actual/Latest (USD Estimate Source of Funds millions) (USD millions) Borrower 9,98 9,87 International Bank for Reconstruction and Development 40 39,91 Total Funding 49,98 49,78 Part II. Level of objective achievement The DPO overall targets for the upgrading of the LRN and creating temporary employment were achieved exceeding expectations. About 446 km of the LRN were rehabilitated and about 40,000 person-months of jobs have been created, allowing the Government to swiftly mitigate the negative impact of the financial crisis. Basic access to social services and livelihood opportunities has been improved by connecting rural poor to the main road network. Reduction of travel time by about 58.5 percent (in average) along with improved riding quality on the project roads resulted in substantial reduction of transport cost of road users and helped to reduce poverty among the rural population in Armenia. The Project has been recognized for achieving outstanding results receiving the WB Vice-President Award for Improving Live of People in Europe and Central Asia. The Project Social Assessment (conducted in December 2013) has also confirmed the Project beneficiaries’ satisfaction with the Project objectives achievement:  roads rehabilitation resulted in vehicle costs, travel time and road accidents significant reduction, traffic and agricultural products sells increase, improved road safety and reliable access to social services (schools, hospitals, polyclinics, community centers) 43  communities residents have been involved in the construction works as a temporary labor force with reasonable remuneration. The overall compliance of the LRIP objectives with the needs of the beneficiary communities and their residents was highly appreciated. Part III. Output by Components Rehabilitation of the Lifeline Road Network selected road sections: a) Technical aspects: The expected outcome of the rehabilitation of approximately 430 km of roads were achieved exceeding expectations. The project has mostly repaired road pavements on existing alignments and some structures and proper drainage and safety facilities have been installed. About 446 km of the selected sections of the LRN were rehabilitated with satisfactory quality. This was confirmed by the Technical Supervision Consultants and independent Technical Auditors. Share of roads in good or fair condition after project made 51.5% compared to the 32.2 % before the project. According to the Project Social Assessment, the all beneficiaries have described condition of the Project roads as follows:  before the Project: "deteriorated, bumpy, without sidewalks and safety guardrails roads in very poor conditions"  after the Project: "nice road providing riding comfort and improved safety, especially where the “safe village program” implemented". Some pictures showing condition of the Project roads before and after the Project are provided in the Annex 7-1. b) Environmental and social management: The Project had positive impact on the livelihood of local population and generated about 40,000 person-months temporary employment providing important sources of revenues. The “safe village program” through supporting small road safety civil works (improved pedestrian crossings, speed management and proper horizontal and vertical signage) combined with awareness campaigns at the community level was implemented in several villages to improve safety for vulnerable road users. The all civil works were conducted within the existing Right-of-Way and did not require land acquisition or result in any other impact to trigger the Bank’s Social Safeguard Policy (OP 4.12). The project was classified as Environmental Category B. The Environmental Management Framework was disclosed in country and then a stakeholder consultation meeting was held. Site-specific Environmental Management Plans (EMPs) were prepared 44 for each road section prior to the beginning of works and disclosed to, and discussed with, local stakeholders. The EMPs were included as part of the bidding documents to ensure that contractors are aware of their responsibilities. The Technical Supervision consultants monitored compliance with OP 4.12, EMF and EMPs throughout the project implementation. Some minor environmental issues related to the disposal of construction wastes noted and were followed by swiftly implemented remedial measures. 1. Capacity strengthening in road sector The Project had also positive and sustainable impact on the capacity of the Ministry of Transport & Communication of Armenia (MoTC) and overall road sector:  a data collection unit was established in the Armenian Roads Directorate State Non-Commercial Organization (ARD), equipped with the Falling Weight Deflectometer (procured under the Project), road roughness measurement and traffic counting devices, and regularly collects data for road surveys and works prioritizations  the Transport Projects Implementation Unit State Institution of MoTC (Transport PIU) and ARD, local design consultants as well as many local construction industries have gained experience on the cost effective pavement designs, job creating technologies and “safe village program” and continue to apply this experience under the ongoing the WB and GoA funded road projects  a Road Safety Audit (RSA) Manual was developed and training was delivered to the Transport PIU, ARD, Traffic Police and local design consultants engineers. RSA Manual is being used for IFIs funded road projects. The Project Performance Indicators matrix is provided in the Annex 7-2. Part IV. Assessment on the Borrower’s Performance 1. Assessment on the Government Performance The commitments of the GoA towards achieving project objectives were fulfilled during the project implementation. The GoA has carried out the Project through the Ministry of Transport & Communication of Armenia (MoTC) in accordance with the provisions of the respective Financial Agreement for original project and the Loan Agreements for two additional financings. The use of standards recommended by the European Committee for Standardization on design and construction of roads and highways (in addition to the national standards) in the design and construction of road works under the Project was permitted. The adequate budgetary resources were made available for the Project in due time as the GoA counterpart funding. The GoA has cleared the all contracts above USD 1 million without delays. 45 2. Assessment on the Implementing Agency Performance Project implementation agencies: The MoTC, as a policy making authority for the transport sector, had the overall responsibility for implementation of the LRIP and has delegated the Project management and day-to-day implementation to the Transport PIU and the ARD (only technical aspects for period 2009-2010). The Project was successfully implemented by the Transport PIU. A proper internal control system capable of providing timely information and reporting on the Project was put in place. Project financial and implementation progress reports were always submitted on-time and in proper format. Project independent auditors always expressed clean (unqualified) opinion. The WB regular supervision missions, financial management and procurement reviews have not identified any significant weakness and overall Implementation and Project risks were rated as Low or as Moderate. According to the Project Social Assessment, the Project activities and the effectiveness of the carried out activities and the Project management were highly estimated by almost all beneficiaries. V. Lessons Learnt The following lessons were learnt from the implementation of the LRIP:  moving towards more efficient design standards taking into consideration the traffic on the roads and the current conditions to determine the most economically viable technical solution  incorporation of routine summer and winter maintenance in contracts for rehabilitation of road sections as a potential for more efficiency gain in road works  successful “Safe Village program” (combination of traffic calming measures, footway construction and an education program in the village school) has received enthusiastic support from the local mayor, the school headmasters and people in the village, and should be taken on board in the designs for road where they pass through villages  Road Safety Audit is a useful tool to improve road safety aspects of road rehabilitation designs and should be undertaken on large road construction and rehabilitation projects. 46 Annex 7-1: Some pictures showing condition of the Project roads before and after the Project Road Gavar -Karmirgyugh Before Project After Project Road Chinari-Aygedzor-Artsvaberd-Verin Karmiraghbyur-Berd Before Project After Project Road Mastara-Dzoragyugh-Zovasar-Garnahovit Before Project After Project 47 Road H-7-Charentsavan-M-4 Before Project After Project Road Ddmashen-Zovaber-Hrazdan Before Project After Project 48 Annex 7-2: Performance Key Indicators matrix Project Performance Indicators Year Data Collection and Reporting Type of Data Frequency Data Responsibility Outcome (Target or and Collection for Data Indicators Actual) Baseline 2009 2010 2011 2012 2013 Reports Instruments Collection Project Outcome Indicators: 20% 20% 20% 20% 20% Target 0 Travel time Reduction Reduction Reduction Reduction Reduction Upon improvement to completion Travel time Supervision drive the length 40% 43% 36,8% 58.2 % 58.5 % of each survey consultant of the section Actual Reduction Reduction Reduction Reduction Reduction section over current situation Number of Target 0 10 650 22 650 34 650 36 650 36 650 Supervision Supervision person-months Periodic consultant Actual 12 000 27 000 30 400 39 486 consultant of jobs created 39 855 report One Pilot of Target Yes Project “safe villages’ Periodic Monitoring Transport PIU Actual Yes Yes program Report 49 implemented Intermediate Outcome Indicators: Component 1 Target 0 150 290 400 430 430 Annual Supervision Number of km Project Supervision consultant upgraded Actual 153 289 332 433 446 progress consultant report reports Component 2 Target Yes Project Road Safety Upon Monitoring Audit Manual is Transport PIU Actual Yes completion Report developed Establishment of Project Upon a data collection Target Yes Monitoring ARD completion unit in ARD Report 50 Annex 8. Comments of Co-financiers and Other Partners/Stakeholders Not Applicable 51 Annex 9. List of Supporting Documents 01A: PAD Lifeline Roads Improvement Project, February 6, 2009 01B: PP Lifeline Roads Improvement Project Additional Financing 1, June 30, 2009 01D: PP Lifeline Roads Improvement Project Additional Financing 2, June 18, 2010 01E: PAD Lifeline Road Network Improvement Project, December 9, 2012 02A: ISR Sequence 1, 04/13/2009 02B: ISR Sequence 2: 07/31/2009 02C: ISR Sequence 3: 11/06/2009 02D: ISR Sequence 4: 05/07/2010 02E: ISR Sequence 5: 11/18/2010 02F: ISR Sequence 6: 06/15/2011 02G: ISR Sequence 7: 03/14/2012 02H: ISR Sequence 8: 12/30/2012 02I: ISR Seqquence 9: 06/14/2013 02J: ISR Sequence 10: 12/10/2013 03A: 2008-05 AM 03B: 2009-01 AM 03C: 2009-03 AM 03D: 2009-05 AM 03E: 2009-09 AM 03F: 2010-02 AM 03G: 2010-03 AM 03H: 2011-03 AM 03I: 2011-09 AM 03J: 2012-01 AM 03K: 2012-07 AM 03L: 2013-05 AM 03M: 2013-11 AM 04A: TOR for Low Cost Study 04B: TOR for Road Safety Audit 05A: Draft Final Report Low Cost Study 05B: Road Safety Audit Manual 05C: Report Evaluation of MCC and Bank Projects 05D: Report Job Creation Impact Assessment 05E LRIP Social Assessment Report 52 IBRD 40913 To Shulaveri 45° 44° 46° 47° 48° To Dmanisi Bagrateshen GEOR G I A GE OR GI A To Bagdanovka ARMENIA LIFELINE ROAD Dzoramut Noyemberian Bavra Tashir IMPROVEMENT PROJECT Zorakert Alaverdi Tavshut To Kazakh Ashotsk 41° SHIRAK Step'anavan (LRIP) 41° Amasiya Verdaghbiur Toumanian Arevashogh LO RRI TAVUSH ROADS FINANCED UNDER LRIP Ijevan Berd Spitak ROADS FINANCED UNDER LRIP-AF Gyumri Vanadzor ROADS FINANCED UNDER LRIP-AF2 To Kars Dilijan Akhurit ROAD SECTIONS FOR LRIP Tsakhkahovit Artsvashen PROJECT LENGTH (KM) Artik Aparan Krasnosel'sk Maralik LRIP 118.2 Sevan Tsakhkadzor AZE R B AI JAN LRIP-AF 126.5 Aragats Hrazdan LRIP-AF2 201.8 TOTAL 446.5 Verin Talin R A GAT SO T N A RA Akhuri Aragats Agarak Yegvard KOTAYK’ Gavar LAKE Getap Ashtarak SEVAN an Mayakovski PRIMARY ROADS MAIN TOWNS/VILLAGES Antarut R. (DUAL CARRIAGEWAYS) MARZ (PROVINCE) CAPITALS Abovian Zovashen GEGHARK’UNIK’ Mets Mazra PRIMARY ROADS (OTHER) NATIONAL CAPITAL Echmiadzin Oktemberjan YEREVAN Getashen RAILROADS Garni Vardenis MARZ (PROVINCE) BOUNDARIES Bagaran Martuni RIVERS Armavir Dzhrarat Sarnaghpyur INTERNATIONAL BOUNDARIES GetashenMarkara RESERVOIRS 40° Karakala Massis ARARAT ARMAVIR Araks R. Artashat 40° Vedi YEREVAN Ararat VAY O T S ’ Jermuk 0 10 20 30 40 Miles TURKEY Yeraskh DZOR 0 10 20 30 40 50 60 Kilometers Yegegnadzor Getap Vaik To Shusha To Shakhtakhty Angekhakot Tegh Ara Sisian Kornidzor Zabukh Vo ks R ro t an Goris . To Nakhichevan R. ARMENIA ZE Tatev A RB A SYUNIK’ IJ AN Kapan Kadzh ISLA MIC To Zangelan Kajaran REPUBLIC 39° 39° O F IRA N To Ordubad Megri To Zangelan This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information Araks R shown on this map do not imply, on the part of The World Bank . GSDPM Group, any judgment on the legal status of any territory, or any 44° 45° 46° 47° Map Design Unit endorsement or acceptance of such boundaries. 48° APRIL 2014