Document of The World Bank Report No: ICR2936 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-7458) ON A LOAN IN THE AMOUNT OF US$11.7 MILLION TO THE REPUBLIC OF CHILE FOR THE INSTITUTIONAL STRENGTHENING OF THE MINISTRY OF PUBLIC WORKS TECHNICAL ASSISTANT PROJECT July 28, 2014 Sustainable Development Department Transport Unit Latin America and the Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 11, 2014) Currency Unit = Chilean Pesos (CLP) 1.00 CLP = US$ 0.001803 US$ 1.00 = CLP 554.48 FISCAL YEAR ABBREVIATIONS AND ACRONYMS BP Bank Policy IFC International Finance Corporation M&E Monitoring and Evaluation IMF International Monetary Fund CPS Country Partnership Strategy IT Information Technology DPL Development Policy Loan MOF Ministry of Finance FDI Foreign Direct Investment MOPW Ministry of Public Works FM Financial Management NCB National Competitive Bidding GDP Gross Domestic Product OP Operational Procedure GIS Geographic Information System PAD Project Appraisal Document GNP Gross National Product PIU Project Imp1ementation Unit HH Herfindahl-Hirschman Index PDO Project Development Objective IADB Inter-American Development Bank R&D Research and Development International Bank for Reconstruction and IBRD SSS Sole-Sourced Selection Development ICB International Competitive Bidding TAL Technical Assistance Loan IDA International Development Association Vice President: Jorge Familiar Calderon Country Director: Livia M. Benavides Sector Manager: Aurelio Menendez Project Team Leader: Daniel Alberto Benitez ICR Team Leader: Daniel Alberto Benitez ii CHILE Institutional Strengthening of the Ministry of Public Works CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 8 3. Assessment of Outcomes .......................................................................................... 13 4. Assessment of Risk to Development Outcome......................................................... 18 5. Assessment of Bank and Borrower Performance ..................................................... 19 6. Lessons Learned ....................................................................................................... 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 22 Annex 1. Project Costs and Financing .......................................................................... 23 Annex 2. Outputs by Component ................................................................................. 24 Annex 3. Economic and Financial Analysis ................................................................. 34 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 35 Annex 5. Beneficiary Survey Results ........................................................................... 37 Annex 6. Stakeholder Workshop Report and Results................................................... 38 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 39 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 47 Annex 9. List of Supporting Documents ...................................................................... 48 MAP iii A. Basic Information Ministry of Public Country: Chile Project Name: Works Institutional Strengthening TAL Project ID: P102931 L/C/TF Number(s): IBRD-74580 ICR Date: 09/19/2013 ICR Type: Core ICR Lending Instrument: TAL Borrower: REPUBLIC OF CHILE Original Total USD 11.70M Disbursed Amount: USD 6.84M Commitment: Revised Amount: USD 11.70M Environmental Category: C Implementing Agencies: Ministry of Public Works Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 08/10/2006 Effectiveness: 02/27/2008 02/27/2008 Appraisal: 03/13/2007 Restructuring(s): Approval: 06/14/2007 Mid-term Review: 06/08/2011 08/08/2011 Closing: 01/31/2013 01/31/2014 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Negligible Bank Performance: Moderately Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Satisfactory Satisfactory Moderately Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Moderately Overall Borrower Satisfactory Performance: Satisfactory Performance: iv C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Central government administration 100 100 Theme Code (as % of total Bank financing) Administrative and civil service reform 29 29 Legal institutions for a market economy 14 14 Managing for development results 14 14 Other public sector governance 14 14 Regulation and competition policy 29 29 E. Bank Staff Positions At ICR At Approval Vice President: Jorge Familiar Calderon Pamela Cox Country Director: Livia M. Benavides Axel van Trotsenburg Sector Manager: Aurelio Menendez Jose Luis Irigoyen Project Team Leader: Daniel Alberto Benitez Jordan Schwartz ICR Team Leader: Daniel Alberto Benitez ICR Primary Author: Mirtha Pokorny F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The Project Development Objective is to assist the Government in achieving a sustainable and integrated approach to the planning of infrastructure; levels of service standards for infrastructure as well as model contracts and procedures that promote competition and assign risks appropriately; an integrated project management process; a v regulatory framework for all of MOPW’s sectors of responsibility that increases consumer confidence in infrastructure service provision; and the legal and organizational separation of the areas of planning, contracting, execution and regulation of infrastructure provision. Some of the original PDOs and intermediate indicators were found difficult or impossible to measure. Thus in 2009 the Borrower and the Bank agreed to a revised set of indicators. (See Section 1.3 of the ICR) The new indicators were reflected in the Aide Memoire dated November 20, 2009 and, although they were never formally changed, they became the framework on which the project was monitored. The information below is based on the agreed revised indicators. a) PDO Indicator(s) Original Target Actual Value Formally Values (from Achieved at Baseline Value Revised approval Completion or Target Values documents) Target Years New planning methodology for infrastructure functioning and incorporated in the Indicator 1 investment decision of the MOPW Share of projects approved based on integrated planning methodology Projects approved based on the new planning 25% n.a. 93% methodology Date achieved 2008 2014 Fully achieved. This component, one of the main linchpins of the project, has helped to successfully establish a new paradigm in MOPW’s planning, introducing a new methodology that integrates regional needs and captures the synergies of different projects within a development program. Several consulting contracts contributed to Comments develop the methodology and apply it to different programs (e.g. City Ports, Meterx, (incl. % City Plans, and Systems of Territorial Information). By 2011 thirteen Regional achievement) Infrastructure Plans and Hydrological Management Plans were developed and approved. By 2013 the new methodology was fully mainstreamed and 99% of the investments included in the 2014 budget come from regional plans developed with the new methodology. vi Actual Value Original Target Formally Revised Achieved at Baseline Value Values (from Target Values Completion or approval documents) Target Years Greater competition in bidding supported by fair risk-sharing arrangements embedded in contracts and consumer responsive service targets Indicator 2 More competition in biddings to be reached by (i) improvements in bidders’ registry (reglamento de obras y consultorías) and (ii) improved standard bidding documents Public works and 80% of new works concession contracts and services See description using the new contracts based on below standard bidding standard bidding documents documents Date achieved 2008 2009 2014 Partially Achieved. MOPW went through a lengthy and extensive revision of the Comments regulations governing public works and consulting services and made proposals that (incl. % were internally approved in MPOW to revise the contractors and consultants registries achievement) and bidding documents. The process was slowed down by the Contraloría office, and is still under revision. Original Target Actual Value Formally Values (from Achieved at Baseline Value Revised approval Completion or Target Target Values documents) Years Greater responsiveness to consumer concerns related to the levels of service; and greater oversight of performance by public sector service providers as well as Indicator 3 concessionaires Number of concession contracts under the jurisdiction of the MOPW that incorporate (explicitly) service standards 3 contracts (2010), 2 Public works and contracts concession contracts using See description below (2011) and 2 explicit service standards contracts (2012) Date achieved 2008 2009 2014 Fully achieved. By the end of 2013 MOPW had entered the following number of contracts with explicit service standards: (a) five contracts of airport concessions; (b) Comments five concession contracts of interurban roads with partial definition of service standards; (incl. % (c) 111 contracts of performance-based contracts and ad-measure contracts, of which achievement) contracts by service levels comprise 13,371.53 km or 71.7% of the country’s paved network; (d) concession contracts for the three hospitals (Sotero del Rio, Santiago Occidente and the Geriatric Hospital). vii Original Target Actual Value Formally Values (from Achieved at Baseline Value Revised approval Completion or Target Values documents) Target Years Reduction in variance between values established at contract signature and final values at contract execution Indicator 4 Reduction in the variance between costs forecasted when contract is signed and actual cost of works Variance of cost when 15% More than 32% contract is signed and (as revised) actual cost of works in 2011 Date achieved 2008 2009 2013 Fully Achieved. In 2010 MOPW completed the base line built with contracts Comments implemented in 2005, 2006 and 2007. By 2011 projects managed under the integrated (incl. % management system and completed that year had a cost of 32% below the base line achievement) cost. Projects completed in 2012 had a reduction in cost of 14% with respect to the projects completed in 2011. Original Target Actual Value Formally Values (from Achieved at Baseline Value Revised approval Completion or Target Values documents) Target Years Normative and organizational structure that provides clear separation and definition of roles and responsibilities for planning, contracting, works execution Indicator 5 and oversight Regulatory oversight system in place to be used by entity independent of planning and execution divisions of MOPW Chile does not have a regulatory oversight system suitable for an entity System in See description independent of the planning place below and execution divisions of MOPW Date achieved 2008 2009 2013 Partially Achieved. MOPW developed and implemented oversight systems for 20 interurban road contracts, six urban road contracts and two public transport corridors. They also completed and implemented the necessary forms for contract oversight of Comments concessioned airports (e.g. information screens, escalators, bathrooms, etc.), and for (incl. % contract oversight of rural potable water. In all cases complaints forms have been achievement) completed and being used for remedial actions. Several web platforms were created for the oversight efforts and their effectiveness were confirmed during the reconstruction period after the 2010 earthquake. b) Intermediate Outcome Indicator(s) viii Original Target Actual Value Formally Values (from Achieved at Baseline Value Revised approval Completion or Target Values documents) Target Years Strategic and tactical plans approved – number of Investment Plans (National Component 1 and Regional) elaborated 16 Investment Plans (1 Master No investment plan and Plan and 15 16 Investment Plans Not defined regional plans exist Regional implemented Investment Plans) Date achieved 2008 2009 2013 Comments Fully Achieved. A Master Plan and 15 Regional Investment Plans were developed (incl. % using the new integrated planning methodology. Two regional plans were elaborated achievement) using a more demanding service standard (Arica – Parinacota and Los Ríos) Original Target Actual Value Formally Values (from Achieved at Baseline Value Revised approval Completion or Target Values documents) Target Years Identify, define and adopt service quality standards in a wide set of work Component 2 typologies under the responsibility of the MOPW Adopted target: 13 9 type of works type of works with No model for quality with identified identified services standards per type of work n.a services and and standards being exists. standards being elaborated – see elaborated. description below Date achieved 2008 2009 2014 Substantially Achieved. By 2013 MOPW had completed the identification services and relevant quality attributes as well as standards of service for the following types: (a) airports under concession; (b) public buildings; (c) roads under concession; (d) maintenance of paved roads not under concession; (e) rain waters; (f) hospitals under Comments concession; (g) identification of users and services of small fisheries. MOPW is well (incl. % advanced in the definition of standards for (h) projects of lake, river and maritime achievement) connectivity for isolated communities; (i) rural potable water; (j) border passes under concession (Paso de los Libertadores); (k) urban renewal projects (Proyecto Marga Mar); (l) public assets building; (m) port infrastructure of border coasts; and (n) irrigation works. ix Original Target Actual Value Formally Values (from Achieved at Baseline Value Revised approval Completion or Target Values documents) Target Years Reduction in the variance of construction time between the time envisaged when Component 3 contract is signed and actual construction time Variance of time when 15% 19% contract is signed and actual (as revised) construction time in 2011 Date achieved 2011 2013 Fully Achieved. The base line was completed in 2011 using contracts completed in Comments 2005, 2006 and 2007. Projects completed in 2011 under the integrated management (incl. % system showed a marginal (1%) difference in completion time when compared to the achievement) base line. Projects completed in 2012 showed a difference of 19% with respect to projects completed in 2011. Original Target Actual Value Formally Values (from Achieved at Baseline Value Revised Target approval Completion or Values documents) Target Years Component 4 Develop and implement a system for the evaluation of users’ satisfaction Develop a model for the evaluation No model for the evaluation See description of users’ of users’ satisfaction exists. below satisfaction for 8 types of works Date achieved 2008 2009 2012 Substantially Achieved. Efforts to develop and implement systems for evaluating users’ satisfaction started fully in 2009 with the system developed and tested through surveys in 12 interurban concessioned roads, three sections of interurban roads under Comments the administration of the Regional Directorates of Roads, and four highways. By 2013 (incl. % MOPW has completed the models for evaluating users’ satisfaction and the base line achievement) for comparisons for six different types of service: (a) Interurban concessioned roads, (b) Interurban not concessioned roads, (c) concessioned urban highways, (d) connectivity of isolated areas’ (d) concessioned airports of the primary network, and (e) Non-concessioned airports of the primary network. x G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 09/11/2007 Satisfactory Satisfactory 0.00 2 04/28/2008 Satisfactory Satisfactory 0.00 3 10/30/2008 Satisfactory Satisfactory 0.00 4 05/06/2009 Satisfactory Satisfactory 0.00 Moderately Moderately 5 11/11/2009 0.00 Unsatisfactory Unsatisfactory Moderately Moderately 6 04/09/2010 0.00 Unsatisfactory Unsatisfactory Moderately Moderately 7 11/24/2010 1.19 Unsatisfactory Unsatisfactory Moderately 8 05/21/2011 Moderately Satisfactory 1.58 Unsatisfactory 9 06/29/2011 Moderately Satisfactory Moderately Satisfactory 1.58 10 01/30/2012 Moderately Satisfactory Moderately Satisfactory 2.36 11 11/10/2012 Moderately Satisfactory Moderately Satisfactory 3.97 12 06/26/2013 Moderately Satisfactory Moderately Satisfactory 3.97 13 01/04/2014 Moderately Satisfactory Moderately Satisfactory 6.81 H. Restructuring (if any) By mid-2012 the Borrower requested and the Bank agreed to a one- year extension of the loan with a new closing date of January 31 2014. The objective of the extension was to allow further disbursements on long-term consultancies financed under the loan and also included a few changes in the procurement rules and the implementation of Anti- corruption Guidelines as described below. The Procurement and Consultant Guidelines were updated to reflect the versions dated January 2011, which allows the use of framework agreements, known as “Convenios Marcos” in Chile, for the procurement of goods, works and non-consultant services. In addition, the Bank agreed to raise the maximum limits for using ChileCompra for consultant services, from US$100,000 to US$300,000. In addition, the definition of “Convenios Marco” was amended to include non-consultant services. Finally the Anti- corruption Guidelines dated October 15, 2006 and revised in January 2011 was to apply to the Project to be consistent with the updated version of the Procurement and Consultant Guidelines. xi Disbursement Profile xii 1 Project Context, Development Objectives and Design 1.1 Context at Appraisal In the early 1990s, the Government of Chile took stock of its infrastructure needs and estimated its infrastructure deficit at US$11 billion with additional losses due to lack of competitiveness of US$1.7 billion annually. 1 In determining how to respond to this need for massive investments and significant improvements in service delivery, the Government also recognized that the public sector did not have the resources to carry out the major works required, nor could it commit its budgetary resources exclusively to infrastructure investments given the country's social investment needs. To help face this challenge, the Government first called on the private sector to take part in the building, maintenance and operation of major road works. In the years since its inception, the concession program has expanded both in terms of sectors and design. Chile had at the time of appraisal (and still has) what may be the world's most extensive concession program encompassing roads, ports, railroads, bridges, tunnels, reservoirs, jails and canals. The Ministry of Public Works' (MOPW) concessions portfolio-including roads and highways, bridges, airports, dams, canals, prisons, and other public edifices-has risen in real terms from about 450 billion pesos in 1995 to over 1 trillion pesos in 2005 . More strikingly, the share of private sector participation in investments had increased from 9 percent to 65 percent over the course of the 1995-2005 decade. In addition to the growing concessions program, at the time previous to project appraisal the Government of Chile had announced a significant expansion in the public financing of infrastructure. The intent was to provide Chile's regions and export products with greater access to markets and to further improve the levels of basic services for Chilean consumers. The administration recognized that both its growing public investment and concession program would require greater attention in terms of planning, contract design, project management and oversight in order for them to maximize their potential as contributors to higher growth levels. The Chilean administration had identified specific constraints to the operational and institutional structure of the MOPW that it believed had to be addressed in order for Chile to reach new level of efficiency and accountability in infrastructure investment. Those constraints included: Planning: 1 Sectors covered by Ministry of Public Works include transport assets such as inter-urban roads, urban roads, highways, bridges, and airports; water resource infrastructure such as dams, storm drainage systems, and canals; rural infrastructure such as small water systems, ferry landings, and fishing ports; and public buildings such as jails, hospitals, and schools. Of transport, water resource and public building contracts, only ports are concessioned and monitored by a separate agency. There are separate regulatory agencies for privatized utilities such as water supply, telecommunications and energy 1 • Lack of medium and long-term planning with an integrated geographic perspective oriented to the competitiveness of Chile's productive sectors. • Low level of interaction with other relevant stakeholders in infrastructure sector and a lack of societal or consumer voice in project planning. • Investment programs which are contract-driven rather than integrated along a supply chain, leading to coordination difficulties and inefficient expenditures. Standards, Contracts and Competition: • Lack of explicit service standards embedded in concessions or public works due to an historic focus on input-driven rather than performance-based contracts. • Uneven levels of competition during bidding processes for either concessions or public works exacerbated by lack of information correlating outcomes to contract size, structure, design elements or bidding procedures Project Management: • Low levels of accountability related to project outcomes due to an atomized approach to project management with different individuals responsible for each stage of project development. • Poor availability of information about project costs, contract management or design and construction issues. Oversight and Regulation: • Institutional inability to provide effective oversight of contracts and concessions according to agree upon levels of service, let alone cost elements. • Lack of ability to consider consumer concerns or even to assure public access to information in the oversight of services. Organizational Reform: • An organizational structure that does not provide for clear separation of roles and responsibilities among the four major areas of activities-planning, contract design, project management and regulation. Measures to address these challenges made up the core components of the Institutional Strengthening Program of the Ministry of Public Works. The Government of Chile requested that this reform initiative be supported with a Bank Development Policy Loan (DPL) and, the subject of this report, a Technical Assistance Loan (TAL). 2 The DPL 3 intended to achieve explicit short term outputs and defined a subsequent series of tasks that would require several years to complete and would be implemented under the TAL. Under the DPL the following tasks were undertaken: (i) completion of competitiveness 2 http://www.mop.cl/modernizacion/Paginas/Inicio.aspx 3 Institutional Strengthening of the Ministry of Public Works Development Policy Loan, 2007. Report No: 39306-CL 2 plans for key clusters; (ii) development of sample performance standards for high and low volume roads, for both concessions and public works contracts, and (b) review of bidding processes and recommendations on new procedures; (iii) completion of design for regulatory accounting system; and (iv) determination of standards to be regulated for roads and one other sector. Bank's involvement in this initiative was sought to leverage the Bank's experience in logistics and territorial planning, in the design and execution of public-private contracts, in project management and in the development of regulatory frameworks for infrastructure. In addition, the Bank offered a comparative advantage in its ability to draw upon global knowledge, sustain a policy dialogue, and exploit synergies with other operations and technical support to Chile including, but not limited to, the Infrastructure for Territorial Development Project, the TranSantiago Urban Transport DPL and TAL, and the fee-for-service technical assistance on the valuation of contingent liabilities in infrastructure - see Annex 9 for a complete list of projects. Finally, both the Ministry of Finance and the MOPW were attracted by the synergies that could be gained from having both a DPL and a TAL to finance the key components of Institutional Strengthening Program. The thematic and substantive linkages of the two projects were expected to provide an effective framework for monitoring and evaluating the implementation of the reform program. 1.2 Original Project Development Objectives (PDO) and Key Indicators The Project Development Objective was to assist the Government in achieving a sustainable and integrated approach to the planning of infrastructure; levels of service standards for infrastructure as well as model contracts and procedures that promote competition and assign risks appropriately; an integrated project management process; a regulatory framework for all of MOPW's sectors of responsibility that increases consumer confidence in infrastructure service provision; and the legal and organizational separation of the areas of planning, contracting, execution and regulation of infrastructure provision. Output and Outcome Indicators for monitoring the progress of the objectives centered on: (a) increased return of infrastructure projects due to the new planning methodology; (b) greater competition in bidding supported by fair risk-sharing arrangements embedded in contracts and consumer responsive service targets; (c) greater responsiveness to consumer concerns related to the levels of service; and greater oversight of performance by public sector service providers as well as concessionaires; (d) reduction in variance between values established at contract signature and final values at contract execution; and (e) normative and organizational structure that provides clear separation and definition of roles and responsibilities for planning, contracting, works execution and oversight. The methodology for calculating the baseline and subsequent indicators would be updated to reflect the availability and reliability of data as well as the most appropriate approach for monitoring of change. The baseline for each indicator was to be determined during the first four months of the project after effectiveness. The project would also 3 monitor progress towards the expected outcomes through the medium-term indicators for each of the components as seen in the next paragraph. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification The PDO remained unchanged during the life of the project. With respect to the key indicators, they were found to be difficult to measure and, in some cases, impossible to establish a strong causality between project and outcome. In May 2009 the PIU and the Bank agreed on a revised set of indicators which were included in the Aide Memoire dated November 20, 2009. Although the changes, summarized in the table below, were never formalized, the new agreed indicators were the ones used by MOPW and this ICR to assess the achievements under the project. 4 ORIGINAL REVISED (a) PDO Indicators (a) PDO Indicators Increased return of infrastructure projects due to the New planning methodology for infrastructure new planning methodology functioning and incorporated in the investments decisions of the MOPW Greater competition in bidding supported by fair risk- sharing arrangements embedded in contracts and Unchanged consumer responsive service targets Greater responsiveness to consumer concerns related to the levels of service; and greater oversight of Unchanged performance by public sector service providers as well as concessionaires Reduction in variance between values established at Unchanged contract signature and final values at contract execution Normative and organizational structure that provides clear separation and definition of roles and Unchanged responsibilities for planning, contracting, works execution and oversight (b) Intermediate Indicators b) Intermediate Indicators Strategic and tactical plans approved (Component 1) Unchanged Share of works contracts and concessions based on Identify, define and adopt service quality measurable, output-oriented standards and greater standards in a wide set of works typologies competition in bidding supported by fair risk-sharing under the responsibility of the arrangements embedded in contracts and consumer- MOPW. responsive service targets. (Component 2) Reduction in variance between unit values for Reduction in the variance of construction time 4 The five PDO Indicators are respectively measured by: i) Share of projects approved based on integrated planning methodology; ii) More competition in biddings to be reached by (i) improvements in bidders’ registry (reglamento de obras y consultorias) and (ii) improved standard bidding documents; iii) Number of concession contracts under the jurisdiction of the MOPW that incorporate (explicitly) service standards; iv) Reduction in the variance of the difference between costs forecasted when contract is signed and actual cost of works; and v) Regulatory oversight system in place to be used by entity independent of planning and execution divisions of MOPW. Note: the Result Framework in the PAD and the revised Result Framework do not provide a methodological description on how these five indicators should be measured. 4 construction/investment established at contract between the time envisaged when contract is signature and final values at project execution. signed and actual construction time (Component 3) Greater responsiveness to consumer concerns Develop and implement a system for the related to levels of service; and greater oversight of evaluation of users’ satisfaction performance by public sector service providers as well as concessionaires. (Component 4) A legal and organizational framework in place that provides for clear separation of roles and Unchanged responsibilities. (Component 5) 1.4 Main Beneficiaries The immediate beneficiary of the project would be the MOPW and its ability to respond more effectively to the needs of the users of public infrastructure. The main beneficiary of the project is the Chilean population through the design and implementation of projects focusing on regional development and equity, a more efficient use of private capital in providing public services, and a strong focus on user satisfaction. Ultimately, those benefits are expected to stimulate economic activities and spread the benefits of the project to the entire population in Chile. In addition, improvement of efficiency of public works management would save public expenditures, which leads to benefits to all taxpayers. 1.5 Original Components (as approved) The operation had five components, four of which address the core aspects of MOPW's business activities: planning; setting standards for contracts, and conducting bids; managing projects; and regulating or overseeing ("la fiscalización de") both concessions and public works. A fifth component (Organizational Reform) was to bring coherence to the program through the development of a legal and organizational structure that defined the expected new roles and responsibilities of the MOPW. Each corresponding component as described in the Loan Agreement is presented below. Section II of the PAD provides a more detailed description of the project. Part I: Integrated Planning 1. Design of a Long-Term Planning Methodology for MOPW's Infrastructure Development: a) Design of a methodology for the planning process of infrastructure investments at the strategic level (national level and long-term planning) as well as at the tactical level (sub- national level and medium-term planning). b) Design of an environmental evaluation methodology to be included in the planning process and provision of training on such methodology. c) Design of a strategy of transition towards implementation of the planning methodology referred to in (a) above, and for the monitoring of such planning process. 5 2. Implementation of a Long-Term Planning Methodology: a) Development of components of a strategic national infrastructure plan using the methodology developed under Part I.1.(a) above. b) Design and implementation of an information system, database and geographic information system (GIS). c) Design of policies for long-term planning in the areas of land use, resettlement, transport and water resources management. d) Implementation of the transition strategy referred to in Part I.1.(c) above, including implementation of an internal and external communication plan, and dissemination of the various infrastructure plans developed under Part I.2.(a) above through workshops, courses and seminars. 3. Carrying out of studies and training related to the development of a sustainable and integrated approach to the planning of infrastructure, as proposed by the Borrower and agreed to by the Bank. Part II: Standards, Contracts, Competitiveness 1. Establishment of Service Standards: a) Establishment of a methodology for setting, evaluating and updating MOPW's standards of service. b) Development of a standard-of-service manual that assigns standards according to type of contract, sub-sector, cost or cost categories and demand levels. c) Training of MOPW's and other selected agencies' staff in the use of service standards. d) Strengthening the capacity of the MOPW's staff to apply the service standards referred to in Part II.1.(a) above, to the design of infrastructure projects. 2. Contracts and Competitive Processes: a) Analysis of MOPW's current contracts and bid processes as well as those used in international best practices. b) Development of a database of MOPW's public works contracts and concessions. c) Assessment of the variables of contract design and bidding processes against their ability to generate competition and investment results. d) Design of model contracts for each primary type of modality of contracting of infrastructure works, and each sub-sector, based on international best practices. e) Analysis of appropriate risk allocations in contracts by modality and sub-sector. 3. Carrying out of studies and training related to the development of service standards for infrastructure as well as model contracts and procedures that promote competition and assign risks appropriately, as proposed by the Borrower and agreed to by the Bank. 6 Part III: Project Management 1. Design of an Integrated Project Management Model: a) Detailed design of the MOPW's project management model. b) Establishment of information and software tools to accompany the project management model designed under (a) above. 2. Implementation of a Project Management Model: Carrying out of communications and capacity building plans to accompany the implementation of the project management model developed under Part III.1 (a) above. 3. Carrying out of studies and training related to the development of an integrated management process in MOPW, as proposed by the Borrower and agreed to by the Bank. Part IV: Strengthening of Regulation of Infrastructure Works 1. Development of roles and regulations for the oversights of public works contracts and concessions, including: a) An operations manual to lay out the procedures for such oversight; and b) Economic and financial models, and their accompanying regulatory accounting manuals, for public-private partnerships for selected infrastructure sectors. 2. Carrying out of studies and training related to the development of a regulatory framework for MOPW's sectors of responsibility that increases user confidence in infrastructure service provision, as proposed by the Borrower and agreed to by the Bank. Part V: Organizational Reform 1. Norms and Regulations: a) Provision of technical assistance for the redrafting of the bill for the establishment of oversight mechanisms for public contracts and concessions. b) Provision of technical assistance for drafting or redrafting, as the case may be, of other laws and regulations needed for implementation of the Program. 2. Institutional Development: a) Definition of alternative arrangements for MOPW's reform in accordance with the proposed legal changes referred to in Part V.1 above, including: b) Design of changes to the institutional structure of the MOPW consistent with any legal or regulatory reforms implemented under the Program. c) Design of the work scope and institutional arrangements that would integrate the management of water resources. d) Establishment of: (i) a cadaster or registry of existing MOPW norms; (ii) procedures that regulate the creation, evaluation and approval of new norms; (iii) procedures for populating, maintaining, disseminating and updating information systems; and (iv) a web-based database of the new norms and procedures. 7 e) Development of a human resource strategy based on performance for MOPW. f) Development of an information technology strategy to support the quality and efficiency of MOPW's business processes. 3. Carrying out of studies and training related to the development of an organizational separation of the planning, contracting, execution and regulatory functions of infrastructure provision, as proposed by the Borrower and agreed to by the Bank. 1.6 Revised Components The project components remained unchanged throughout the life of the project. 1.7 Other significant changes By mid-2012 the Borrower requested and the Bank agreed to a one- year extension of the loan with a new closing date of January 31st, 2014. The objective of the extension was to allow further disbursements on long-term consultancies financed under the loan and to include changes in the procurement rules and the implementation of Anti-Corruption Guidelines as described below. The Procurement and Consultant Guidelines were updated to reflect the versions dated January 2011, which allows the use of framework agreements, known as “Convenios Marcos” in Chile, for the procurement of goods, works and non-consultant services. In addition, the Bank agreed to raise the maximum limits for using ChileCompra for consultant services, from US$100,000 to US$300,000. In addition, the definition of “Convenios Marco” was amended to include non-consultant services. Finally the Anti- Corruption Guidelines dated October 15, 2006 and revised in January 2011 was to apply to the Project to be consistent with the updated version of the Procurement and Consultant Guidelines. 2 Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry The ICR finds the quality at entry as moderately satisfactory, since: Project design was in line with the main pillars of the Bank’s strategy for Chile. By supporting the implementation of the Government's Institutional Strengthening Program, the TAL was in line with the goals and strategic approach set out in the joint IBRD-IFC Country Partnership Strategy (Report No 38691-CL). The objective of the CPS was to support Chile's efforts to converge with OECD income levels and living standards by (i) accelerating growth and (ii) building a more equitable society. The Bank Group strategy was based on the maintenance of close engagement with the authorities, which 8 in recent had yielded important results in support of key Government programs and through flexible, timely support that helped to advance Chile's development vision A strong policy dialogue and analytical underpinning contributed to a sound project design. The objectives and actions of the project were the result of more than a year of work at MOPW and from consultations with relevant external stakeholders. The effort was coordinated by a multidisciplinary team composed of MOPW staff with specific responsibilities for each technical area of reform. That team identified the main shortcomings of the MOPW and developed the proposals for their improvement. The Bank supported the work of these teams with technical and regulatory advice since the early stages of the reform’s inception. The result was a very comprehensive and detailed set of objectives and tasks that provided an excellent blueprint for a complex institutional reform. The choice of lending instruments contributed to project continuity. The TAL was presented to the Bank’s Board of Directors together with the DPL. The Ministry of Finance and the MOPW, as already mentioned, were attracted by the synergies that could be gained from having both loans to finance several of their key initiatives. First, because the thematic and substantive linkage of the two projects was expected to provide an effective framework for monitoring and evaluating the implementation of the Ministry’s reform program for a period long enough that exceeded one administration. Second, by having both a DPL and TAL, the political profile of the Bank-financed projects increased and made possible a contingent role of the Bank as an honest broker between the Ministries of Finance and Public Works when the reform suffered from delays during implementation. The quality at entry would have deserved a satisfactory rating if it had included a more adequate results framework. Some of the originally proposed PDO and intermediate indicators were difficult to measure and unrealistic in terms of timing. As shown in Section 1.3 preceding and discussed in more detail in section 2.3 below, some of the indicators had to be revised to make them more precise, measurable and in some cases, achievable within the time frame of the project. 2.2 Implementation The project was approved by the Bank's Board on June 14th, 2007 and it became effective on February 27, 2008. Implementation progressed steadily throughout the life of the project, albeit at different pace during different periods. In particular, internal resistance to the modernization program, changes in government administrations and staff directly involved with project implementation, at times lack of adequate funding, and poor disbursement rates that jeopardized Bank involvement, contributed to slower 9 project implementation. 5 However, in spite of its rocky beginnings, these problems were eventually overcome and the modernization program, although a work still in progress, has succeeded in bringing profound changes to MOPW’s view of its role in a world in which users have a stronger voice and increasing demands. The following paragraphs summarize the most salient aspects of the implementation process. Resistance to change. The launching of the project by its original champion (minister Bitar during the first Bachelet administration) was immediately faced with strong opposition from MOPW’s staff. Unions interpreted the proposed reforms as implying privatizations and the redundancy of some services. Senior management within MOPW realized that the reform principles were not properly communicated and slowed down implementation. The impasse allowed further work with the unions to convey the message that the objective of the reform was to improve the efficiency of public spending and that no redundancies were being considered. Even when the issue of potential redundancies was put to rest, there was (and, with lesser strength still is) some institutional inertia to change the vision from a provider of hard infrastructure to a provider of high quality infrastructure services in those remaining few areas that are still not fully mainstreamed. Team leaders have been key in involving MOPW senior and junior staff (both in Santiago, where the headquarters of the MOPW are located and in all the country’s Regions) in the modernization process. They have fostered implementation of the reform by organizing trainings, strengthening the dialogue and building coordination stages among areas that had traditionally worked somewhat in isolation in order to bring everybody on board of the institutional changes. Changes in administrations and staff. During the first two years of project implementation the high rate of turnover of PIU’s coordinators considerably slowed down progress in the modernization program. Since appraisal to late 2009 the project had four coordinators, an average of one every six months. Subsequently, the upcoming elections and expected change of the administration in March 2010 coupled with a change in priorities to focus on a stimulus package to increase economic activity, contributed to further delays. The elections brought about a change in the coalition in power (the new one being from the center-right) after more than 20 years of a center-left coalition and the new government was faced immediately with the devastating earthquake that hit Chile in February, 2010. After the first few months of the new administration, the Modernization Program received a strong endorsement. The tools developed and implemented by the Program were tested, fully utilized and proved effective in the emergency reconstruction projects that followed the February earthquake, a fact that contributed to the renewed strong Government support of the program. The MOPW undertook several actions to increase the implementation pace of the modernization efforts, including providing a stronger institutional support with its transfer from the Directorate General of Public Works to the Undersecretary of Public Works and the appointment of a new highly effective project coordinator. 5 It should be noted that progress on project implementation was fairly independent from the rate of disbursements during the first half of execution, since in Chile loan funds are not additional to the Ministry’s budget. 10 Disbursements and Project funding. The first two years of the Modernization Program were implemented without Bank funding, with the first disbursement of loan funds taking place in mid-2010. The poor disbursement rates and the slowing down of project implementation during 2009 caused the revision of the PDO rating to MU. Supervision missions identified the problem as a worrisome lack of communication with the MOF with respect to how budget allocation mechanisms work in Chile when funds from a Bank loan are available. Bank funds are not additional to budget allocations and the latter were many times assigned to areas of the program that were not eligible for Bank financing. The dismal disbursement rates were the subject of the mid-term review and the subsequent meetings involving the Bank’s country director in which the Bank team agreed with the PIU and MOF that if changes were not made before June 2011 the Project would be canceled. The agreement included to: (1) appoint a new Project Coordinator, with knowledge of the Modernization Program and credibility within the organization; (2) request at least one disbursement before June 2011; (3) define the new administration’s prioritization areas and launch an activity (consultancy, workshop) in at least one of these areas; and (4) provide a stronger commitment to the Modernization Program. Authorities in the Ministry of Public Works, confirming the importance that was attributed to the continued presence of the Bank for the success of the Modernization Program complied extensively and fast with the agreements made. The actions undertaken included: (1) a new and highly effective Project Coordinator, a long term qualified official in the Ministry was appointed; (2) a new request for retroactive disbursement was presented to the Bank; and the Minister gave a stronger institutional support to the Modernization Program by transferring the management from the Directorate of Public Works to the Undersecretary of Public Works. In practice, this implies giving more visibility and power to the PIU. Moreover, the Strategic Directorate that was created as a high level coordinating unit to monitor the progress of the Modernization Program started to meet again once a month, after 15 months of discontinuity. Several articles about the Modernization Program were published in many newspapers as a result of interviews with the Undersecretary, providing clear support to the Modernization Program. Disbursement issues were satisfactorily addressed to a great extent through the already discussed loan restructuring. By the beginning of 2013 disbursements reached 34 percent of loan funds and accelerated considerably during the year. By the loan closing date US$8.24 million were disbursed (70 percent of the loan) and the remaining US$3.45 million were canceled. Lower than expected budget allocations also contributed to slow the implementation of the Program. An estimated front-loaded cost structure resulted inadequate to finance a program in which the initial stages were devoted to communicating visions and goals, seeking consensus, discussing strategies and alternative methodologies, seeking information from different international experiences and subsequently develop time consuming terms of reference for complex studies and/or technical assistances. Since the initial allocations were not fully utilized, and the MOF relied on past execution rates to 11 define subsequent allocations, funding was decreasing when the needs increased. Budget allocations during 2009 and 2010 were at a level that jeopardized the continuation of the program. Again, these issues were resolved once discussions were held to restructure the project and place it in the right track. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization As already discussed, the result framework of the TAL included targets that were not measurable or lacked a clear distinction between end-of program outcomes monitorable at the time of completion and outcomes that would materialize in the longer term as the reform process consolidates. In addition, some of the indicators for the medium term outcomes were hard to measure, not very realistic, and misleading. 6 These drawbacks of the M&E framework were identified very early during the project execution and by 2009 a revised set of more realistic and measurable indicators were prepared by MOPW and agreed with the Bank. The revised set of medium term outcomes helped to narrow down and provided useful milestones for each policy area objectives. The fine tuning of the indicators to ensure their alignment with the expected outcomes, the availability of data for their periodic monitoring, and the establishment of baseline values was carried out as part of the supervision activities of the TAL. The process of revision of the M&E and its implementation was useful to strengthen the dialogue with the counterpart and identify new opportunities for support. Both the interim semiannual reports prepared by the MOPW and the supervision missions were particularly useful to collect data relating to the project implementation. The missions provided the framework for an in depth follow- up of the project components’ progress and served to highlight pending issues to the authorities. In addition the supervision missions gave the Bank and the MOPW the opportunity to deepen the policy dialogue and identify areas of joint cooperation that were not identified during project preparation. Finally, a field visit conducted during project supervision served to corroborate the process of the reform implementation in the regions. 6 One PDO indicator was the increase in the rate of return (of an investment project) due to a better planning methodology. Even when some positive correlation may occur, the causality between the new planning methodology and the rate of return could be very difficult of being proven. Moreover, the indicator does not provide guidance if the rate or return should be measured ex ante or ex post (once the project is operative). For instance, some typical issues in project preparation such as the underestimation of cost and overestimation of demand can be avoided by the implementation of the new planning methodology. However, the rate of return may decrease during the project preparation if cost is higher and demand is lower. A second indicator was defined to capture the increase in market competition due to a new contract and process standardization measured by a Herfindahl index (HHI). While the contract and process standardization can lead to a more transparent and simple procurement process, the capacity to affect the market share of public contractors can be negligible or impossible to capture. Indeed, the entry of new contractors to the market of public works, if occurs, can take some time and be affected by other variables. 12 2.4 Safeguard and Fiduciary Compliance Limitations imposed by the Bank’s procurement rules contributed to the initial substantial lag in disbursements. The first disbursement took place in October 2010, more than two and a half years after the loan became effective. Initially, one of the main reasons for disbursement delays was the role played by Contraloría (auditing body of Chile) who added at least 6 months to the procurement process of each consulting work. Eventually, Contraloría agreed to work on a standard form for Bank-financed consultancies. Subsequently, and as the pace of implementation improved, the limitations were imposed by the Bank: i.e. matters related to anticorruption clauses in contracts and the limit of US$ 100,000 for contracts using the ChileCompra system. 7 The Borrower repeatedly stressed the importance of revising the procurement procedures and the positive impact that it would have on disbursements, as several contracts could not be financed by the loan due to that particular limitation. On December 12, 2012 the Loan Agreement was revised to extend the original closing date to January 31st, 2014 and to update the Procurement and Consultant Guidelines to reflect the versions dated January 2011, which allows the use of framework agreements, known as “Convenios Marcos” in Chile, for the procurement of goods, works and non-consultant services. In addition, the Bank agreed to raise the maximum limits for using ChileCompra for consultant services, from US$100,000 to US$300,000. In addition, the definition of “Convenios Marco” was amended to include non-consultant services, and the Anti- corruption Guidelines dated October 15th, 2006 and revised in January 2011 was to apply to the Project to be consistent with the updated version of the Procurement and Consultant Guidelines. MOPW had adequate financial management systems, software, and trained staff. No systemic problems were identified during implementation. The rating of the project, however, remained as MS (moderately satisfactory) mainly because of the high rotation of the financial management staff. This issue impacted the preparation of financial reports, which presented problems and needed some corrections to be accepted. Contraloría, the National Audit Office, also made comments requesting some changes and improvements in the presentation of the information. Finally, the correction of some issues took time and a few issues were repeated in other reports. 2.5 Post-completion Operation/Next Phase The vehicle used for the implementation of the Institutional Strengthening Program of the MOPW (Programa de Modernización del MOP) ends at the end of June 2014. The long- 7 The hiring of consultant firms with estimated cost less than US$100,000 was to be generally carried out using national procedures which include advertisement and processing through the Chilecompra Portal: such procedures - that may include open competition are not consistent with the short listing procedures as specified in the Bank's guidelines but in Chile such procedures ensure transparency and quality service and therefore, the Bank allowed the direct contracting of the firms thus selected. 13 term sustainability of the achievements seems to be guaranteed by the fact that the program was a mean to improve the functioning of the different units of MOPW. That is, each unit involved in the program was the final recipients of the technical assistance and they are advancing independently with cementing their new role. The MOPW, however, is interested in continuing with institutional strengthening in two areas where the Bank might be requested to provide support in the near future: i) contract management in infrastructure concessions; and ii) institutional set up of the water sector. The first topic refers to the need to improve the oversight function of the concession unit and it was identified as a priority by the recently appointed management of MOPW. On the water sector, some progress was achieved during the program but there are other reforms that required further support and the timing of the project was not suitable for their implementation. 3 Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation The main objective of the TAL remains extremely relevant to Chile’s current agenda. The modernization program of MOPW remains very much in line with the government’s focus on (i) investment decisions based on programs fostering equity, territorial integration and regional development; (ii) a more equitable distribution of public-private risks and better oversight of the provision of services; and (iii) the shift towards a performance based infrastructure management as measured by users’ satisfaction. The recent reelection of President Bachelet who’s administration was the original champion of the modernization process, provides further enhancement to the relevance of the objectives which have been already mainstreamed in the operations of MOPW. Chile’s continued efforts to converge with OECD countries performance provide a favorable support for the reform. Finally, MOPW has an undisputed role in the Chilean economy as a vehicle for implementation of countercyclical policies. 8 The need to improve the efficiency and performance of the MOPW becomes even more important in the light of its increasing role as growth and employment generator. The objective of the loan also remains relevant for the achievement of the Bank strategic objectives, which emphasized the need to continue to focus on the modernization of the state through a close engagement with the Chilean authorities. Likewise the project objectives are fully consistent with those set in the Country Partnership Strategy (CPS) endorsed by the Board in May 2007 and in February 2010 9 , contributing directly to objectives of supporting the efficiency of infrastructure and public services in addition to decentralization and regional development. 8 Chile has been cited as an example of how to implement adequate countercyclical economic policies. See for instance the article titled “Stimulating” published by The Economist on February 19, 2009 (available at http://www.economist.com/world/americas/displaystory.cfm?story_id=13145570) 9 World Bank 2011. Chile Country Partnership Strategy 2011-2016. Report No. Report No. 57989-CL 14 3.2 Achievement of Project Development Objectives (PDO) PDO Indicator 1: “New planning methodology for infrastructure functioning and fully incorporated in the investment decision of the MOPW” The project fully achieved all the intermediate institutional targets leading to MOPS’s full adoption of a novel integrated planning. The new methodology derived from a strategic long-term view of regional development and was brought down to the tactical selections of project with an integrated geographic perspective oriented to the competitiveness of Chile's productive sectors and the equitable and sustainable access to goods and services to all the country’s territories. The new methodology, now being used in 99% of the MOPW’s newly approved projects, includes a high level of interaction with other relevant stakeholders in the infrastructure sector and a societal or consumer voice in project planning. The integration of the investment programs along a supply chain is expected to substantially decreased coordination difficulties and inefficient expenditures. All MOPW’s projects included in Chile’s 2014 budget come form strategic investment programs evaluated using the new methodology. In 2010, with the new administration, the Bank was requested to pay increased emphasis on water resource management under the integrated planning activities. A large consultancy launched in 2011 produced a manual being successfully used for the planning and management of rainwater in large urban centers, thus contributing to the objective of improved planning in a one of MOPW’s critical priority areas. PDO Indicator 2: “Greater competition in bidding supported by fair risk-sharing arrangements embedded in contracts and consumer responsive service targets”. This objective was partially achieved. The objective was fully achieved with respect to the incorporation of standard of services in concessions and civil work contracts. Progress in ensuring greater competition in the bidding process centered in a series of studies and proposed revisions to reduce the excessive restrictions that limited the inclusion of participants in the registry of consultants and construction companies, to increase the precisions of the bidding documents and to reduce discretionary decisions. The proposed regulations for public works incorporating the new proposals were presented to the Contraloría in 2012. Contraloría raised several comments on the document but stop short of its approval. Clearly this is the most contentious area were many strong interests are at stake and decisive political support is necessary for any meaningful change. Nevertheless, even without the actual formal revision of the Regulation of Public Works (Reglamento de Obras Públicas), some progress was made in the standard bidding documents, such as the handling of the two- envelope system, the improvement of the polynomial adjustment method and the dropping of the publication of MOPWs project budget. Further progress and full approval of a new Reglamento is a challenge left to the new Bachelet administration. 15 PDO Indicator 3: “Greater responsiveness to consumer concerns related to the levels of service; and greater oversight of performance by public sector service providers as well as concessionaires” This objective was fully achieved. This was probably one of the more complicated objectives in terms of resistance to a complete change in the conceptual and methodological model to implement a service- oriented approach to public works. After a process of identifying service levels and standards for the design, construction and exploitation of each type of works, these are being introduced as explicit obligations in an increasing number of concession contracts for airports, roads, hospitals, etc., actually surpassing the projections of the intermediate targets. By the end of 2013 MOPW had entered more than 120 contracts with explicit service standards which, in addition to several contracts in hospital and airport concessions, include performance-based and ad- measurement contracts comprising 13,371.53 km or 71.7% of the country’s road network. In addition MOPW successfully developed and implemented through surveys a system for the evaluation of user’s satisfaction in at least four types of services and is continuing with the task of developing the systems for other services under their responsibilities PDO Indicator 4: “Reduction in variance between values for construction/investment established at contract signature and final values at project execution.” The objective was fully achieved. Reductions in variance between costs for construction/investment established at contract signature and final values at project execution were confirmed by comparing contracts carried out before and after the full institutionalization of the new project management approach. The creation of Project Management Offices at the central and regional levels in 2010 and the full adoption of the “project manager” figure increased accountability and reduced atomized decisions. The new organization is being supported by the development of a solid digital platform to provide rapid information on project costs, contract management and/or design and construction issues. The effectiveness of the new project management system was fully proved during the reconstruction efforts following the 2010 earthquake, a fact that was instrumental in securing the new administration support for the project. Objective 5: “Normative and organizational structure that provides clear separation and definition of roles and responsibilities for planning, contracting, works execution and oversight” Partially achieved. The Concession Law, one of the targets in this component, was approved and made public as Law 20.410 on January 1st, 2010. However, the main laws changing MOPW’s organization (the Superintendence Law and the Organic Law of MOPW) were not pursued and were not considered a Government priority. Instead, MOPW is implementing a strong oversight effort through contract oversight systems that are already in place for concessioned airports (e.g. information screens, escalators, etc.) and for contract oversight of rural potable water. 16 3.3 Efficiency The project should be considered as highly cost effective since costs should be exceedingly compensated by improved capacity reflected in fiscal savings and efficiency gains. The MOPW manages a budget for 2014 of about US$ 3.3 billion of which about US$ 2.4 billion are allocated to investments. The cost of the project, at a total of US$ 29.15 million, represents less than 1% of the budget of one year, yet it was sufficient to introduce deep and permanent improvements in efficiency and service levels. The already confirmed reduction in the costs and execution times of contracts that are being derived from the adoption of an “integrated management system” would exceedingly compensate for the costs of the project. Additional gains, probably more sizable but more difficult to quantify, should be derived from the selection of projects that focus on equity and the social benefits that were previously ignored. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory The ICR considers the overall outcome of the project as satisfactory since (a) project objectives remain highly relevant to the development of public infrastructure in Chile and the government’s pursue of growth with equitable regional development; (b) the project demonstrated satisfactory achievements of its DOs, especially in regard to full institutionalization of integrated planning, efficient performance based management of projects and services, a more substantive oversight of the provision of services, and the inclusion of user satisfaction as a key measure of performance; and (c) the project passed the test of approval of administrations covering a wide range of the country’s political spectrum and is fully embedded in the modus-operandi of MOPW. Moreover, the project was instrumental in bringing about the modernization of MOPW. Without the project, and thus the Bank involvement, it is unlikely that the Modernization Program would have been kept in the agenda and in the country’s budget. Even when the financial contribution of the loan was at times minimal, MOPW insisted that the continuous presence of the project was vital for the continuation of the reform program. The satisfactory rating, however, does not coincide with the last ISR’s rating of Moderate Satisfactory, which unfortunately gives inordinate weight to the poor disbursement performance of the project rather than to the achievements of the development objectives and was not revised when disbursements picked up. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development The operation built the foundations for generating an indirect impact on poverty alleviation by inducing greater efficiency, effectiveness and accountability in the utilization of public resources. It also broadened participation in public infrastructure investment, and fostered transparency and accountability in infrastructure service delivery (principally by enhancing oversight, and access to information). In addition, the 17 project promoted instruments such as strategic environmental assessments that have already fostered (and will continue to foster) the integration of environmental aspects into the planning process. All the above, will help Chile to achieve OECD standards for public works projects that contribute to sustainable development. (b) Institutional Change/Strengthening The operation’s institutional impact was and will be substantial. It established a new institutional paradigm for the MOPW. This includes a new planning methodology designed to provide an integrated strategic vision for all types of infrastructures. The operation introduced the profile of the project manager which was key to create an integrated project management approach that covers the full life cycle of projects. It also designed a Project Management Office (PMO) to improve the management and oversight of the portfolio of projects under the responsibility of the MOPW. The operation further developed an integrated technological platform that is building a database for monitoring the performance of concessions and public works. (c) Other Unintended Outcomes and Impacts (positive or negative) One interesting outcome that was not included in project design was the establishment of the Public Works Institute in the MOPW. The reform process required a substantial amount of training on planning, project management, contract standards, concessions, information technology, etc. As it was found that many of the professional being sought in academia for providing training were members of the MOPW, the government decided to create the in-house training Institute, a move that is resulting in the continuous upgrading of the professional skills in the Ministry. Finally, the relevance of this project for the Bank agenda should be highlighted as Chile is an example for the Bank in the area of infrastructure service provision. The Bank has constantly relied on Chile to illustrate to other countries in different regions how infrastructure reforms should be conducted, in particular in the area of attracting private sector financing and operators through public private partnerships. World Bank staff and public officials from East Asia and South East Asia have conducted study tours to learn from the Chilean experience. Thus, the Bank will benefit significantly from the MOPW modernization process as it will provide valuable lessons for other countries that request the assistance of the Bank to follow a similar path of reforms. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops N.A. 4 Assessment of Risk to Development Outcome Rating: Negligible 18 The sustainability of the achievements of the TAL operation is deemed substantial, given that most of the objectives have been fully institutionalized and became an integral part of MOPW’s modus operandi. While to continue to move forward and deepen the modernization efforts, particularly for the still incomplete legal and regulatory changes, would clearly require the presence of a reform-minded management team and strong political support progress made to date is most likely to endure changes in leadership. There are several factors that mitigate the risk associated with potential changes in leadership. In addition to the strong ownership of the new MOPW’s vision within the institution, risks are reduced by: i) the already strong support from different administrations; (ii) the users pressures for reform resulting from improved performance; iii) the heightened role of public consultation organisms; and iv) the strong ownership within MOPW’ of its strategic plans. Finally, the risk of cultural resistance and vested interests within the institution opposing project implementation, probably the most tangible one during the early years of project implementation, has been largely contained. A well-established organizational culture and self-interest inertia originally resisted institutional change within the MOPW. Mitigation measures aimed at fostering stakeholders’ participation and promoting change management served to counter-balance this resistance to change. The gradual integration of new profiles (such as the project manager) and the progressive definition of new institutional bodies for supervision and coordination, also served to mitigate this risk. Addressing the risks imposed by interests outside the institution to move forward with the legal and regulatory changes still necessary to fully complete the modernization program will require further work, particularly to strengthen the communication channels between the executive and the legislative branches of government to gather support for the modernization of the Ministry’s legal structure. 5 Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: moderately satisfactory The rating for Bank performance during the preparation phase is moderately satisfactory. The justification for this rating is as follows: acknowledging the complexity of the reform, the Bank assigned a highly qualified team with extensive experience in the relevant policy areas. The team was able to maintain a close collaboration with the Minister and his team from the beginning, and responded promptly and with high levels of flexibility to their requests for assistance in certain topics (particularly the drafting of the Concessions and Superintendence of Public Works Law) at the outset of the reform process. Experts from different parts of the world participated actively in preparation 19 missions and workshops, providing ad-hoc advice directly to the Minister and his cabinet on specific issues. The team’s appropriate skill mix helped deliver a well-designed project and provided value added to the Borrower. The Bank’s performance assisting the Borrower in identifying and appraising the project to ensure consistency with the CAS and with government’s priorities, and in a manner that would ensure attainment of expected outcomes, was also very valuable. Performance at entry could have been satisfactory if design of the M&E framework had included more relevant and measurable indicators. (b) Quality of Supervision Rating: moderately satisfactory Traditional supervisory activities to monitor progress toward mid-term outcomes were carried out and missions traveled to the field more than twice a year, with additional regular Bank support provided directly by the Project team. Beyond the focus on regular supervision matters, there was a strong interest on the part of the Bank in assessing impact of project activities and, in doing so, maintaining central government focus in the reform agenda. To ensure a feasible impact analysis, objective indicators, and the availability of the required data to monitor their evolution throughout project implementation, the M&E framework for the project was revised with Chilean counterparts. In addition the supervision team was very proactive and successfully intervened to keep the project alive when the Government support faltered. Supervision missions did play the productive envisaged role of “honest broker” between MOPW and MOF, and all of the MOPW officials interviewed for this ICR coincided in that the presence of the Bank was instrumental in saving the project when the political support was at its lowest. The less than satisfactory rating is due to the following: (i) the revision of the result framework was never formalized. Although a problem of form more than substance, the change would have improved the rather confusing presentations of the result framework in the ISRs, which mixed the original and revised indicators; and (ii) the supervision team relied heavily on disbursements rather than on achievements to evaluate project performance. (c) Justification of Rating for Overall Bank Performance Rating: moderately satisfactory. Based on the above, the ICR considers the overall Bank performance as moderately satisfactory. 5.2 Borrower Performance (a) Government Performance Rating: satisfactory Government performance is rated satisfactory due to the commitment and involvement in MOPW’s reform initiative, as evidenced by: (i) the active participation of the Ministry of Finance from project inception. The positive commitment of the Ministry of Finance, given the structure of power in Chile, is a sine qua non condition for the successful 20 implementation of any institutional reform in Chile; (ii) the explicit support from the highest levels of government to the reform program in public speeches and press releases; and (iii) the valuable contributions and participation from staff of other public agencies in multiple initiatives of the reform program (in particular, involvement of multiple agencies in the seminars carried out by MOPW under the Integrated Planning policy area). On the other hand, the poor budget execution of the reform program during its first two years prompted MOF to reduce budget allocations at a time in which the program was starting to take off. These shortages of funds contributed to slowing down the implementation of the project. (b) Implementing Agency or Agencies Performance Rating: satisfactory Implementing agency performance is rated as satisfactory. From the beginning of project implementation, MOPW has assigned a qualified team to lead the initiatives under each policy area. These teams have worked hard in the production of terms of reference, and in the organization of trainings and other events; strengthening the dialogue and building coordination stages among Ministry areas that had traditionally worked in relative isolation. The Project Coordinating Unit has complied with all of its fiduciary responsibilities in a satisfactory manner, and played an important role in providing relevant information to assess the attainment of outputs established for the ICR. (c) Justification of Rating for Overall Borrower Performance Rating: satisfactory Based on the above, the ICR considers the Borrower performance satisfactory. 6 Lessons Learned • An intensive early period of communication and participation as part of the process of problem recognition and problem definition is fundamental for ensuring support to institutional changes. MOPW underestimated the need for launching an effective process of communication for gathering staff backing for the modernization program. Even once the initial concerns for job stability were addressed, the acceptance of new conceptual and behavioral norms in a big and powerful institution proved a difficult task that was eventually overcome by moving from a “diffusion of communications” to a more participatory approach for setting common goals. • The endorsement and strong ownership of the leaders of an institution are crucial to ensure the continuity and success of major reforms. After Minister Bitrán, who was the original reform champion, left office in January 2008, immediate support by his successor, Minister Bitar, was decisive to providing continuity to the modernization program. This endorsement was critical at a time 21 when support to the reform was waning within the MOPW due to weaknesses in the internal communication of the reform’s principles and objectives among ministry’s stakeholders. The priority of the program was further and definitely confirmed in 2010 when the administration changed hands, thus ensuring its continuity. • The cost curve of institutional reform should take into account the lead- time necessary for consensus building and task preparation. The cost curve estimated for the TAL was front- loaded. During the initial years, however, the needs focused on internal deliberations on visions, strategies and goals. These required the help of individual consultants to define future assistances and develop complex Terms of Reference for more resource consuming consultancies. Thus, actual need for funds increased with time (rather than decrease as projected). This discrepancy between actual and expected costs was at times a source of project underfunding. • The Bank should evaluate more closely the procurement thresholds or consider more flexible procurement rules for countries with a proven record of controlling corruption. The procurement limits originally established for the use of ChileCompra, were at the center of the poor disbursement record of the project. Its revision and the use of limits more according to the procurement record of Chile (which probably should have taken place as soon as the problem was identified), proved instrumental in turning around disbursement s performance. 7 Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies No comment was provided – see Annex 7. (b) Cofinanciers (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) 22 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) 1. Integrated Planning 7.59 4.12 -45.7% 2. Standards, Contracts, and 4.85 -74.9% Competition 1.22 3. Integrated Project 4.00 +57.1% Management 6.28 4. Regulatory Strengthening 6.23 1.74 -72.1% (fiscalización) 5. Organizational Reform 6.50 9.13 +40.5% 6. Administrative costs and 6.19 6.66 +7.5% contingencies Total Project Cost 35.36 29.15 -17.6% (b) Financing Appraisal Actual Percentage of Source of Funds Estimate (USD Appraisal (USD millions) millions) Borrower 23.66 20.80 -12.1% International Bank for Reconstruction 11.70 8.35 -28.6% and Development 23 Annex 2. Outputs by Component This annex lists the main goals achieved in each component. For a more detailed description including intermediate taken steps, studies, and training, etc. please consult the Modernization Program progress reports that MOPW prepared semiannually for the World Bank. COMPONENT 1: INTEGRATED PLANNING (Appraisal estimated cost: US$ 7.59 million; actual cost: US$ 4.12 million ) INFRASTRUCTURE INTEGRATED PLANNIG: Implementation of a full planning process for infrastructure services, which support the decision making process for investment by the Ministry of Public Works aligned with the strategic criteria and the vision of the country. A planning process which also strengthens competitiveness, which enhances the productive development of each territory, and ensures its inhabitants equal access to goods and services in a sustainable and integrated manner. OBJECTIVES ACHIEVEMENTS • Prospective Development • Set up of the Integrated Planning o Development of Country Vision Committee at the MOP. 2020 and its relation to • Development of 5 sectorial investment infrastructure policies. o Design of policies for long-term • Director Plan with long-term vision. planning concerning: land use, • Approval of a methodology for relocation of settlements, investment planning. transportation, water resource management and sustainable • Development of 15 regional plans with territorial development. stakeholder engagement. Dissemination and implementation of 15 Regional o Production cluster’s analysis Infrastructure Plans and of Water and its impact on infrastructure Resource Management for 2021. o Elaboration of studies and Definition of the Agenda for the projects delivery of training related to of the Regional Strategic Plans. the development of a sustainable • Installation of the Land Information and integrated approach to System MOP. infrastructure planning. • Establishment of the MOPW Investment • Planning Model and Process Management process in conjunction with o Installation of participatory the services, under the Quality Initiatives Management System MOP, and its full o Design of methodologies to review. promote a participatory planning • Development of three new MOPW process for infrastructure policies. Elaboration of three inter- investments ministerial public policies: Urban • Transversal Support Systems Development; Rural Development and o Development and Regional Development. implementation of an • Development and use of evaluation information system and a methodology for appraisal plans. database for a Territorial • Delivery of two training cycles Information System “Sistema de "Strengthening Skills and Specific 24 Información Territorial (SIT)” MOPW Planners Competencies” o Capability building for the developed by DIRPLAN - MOPW development and Services. implementation of a sustainable • Creation of an Observatory for and integrated approach to Infrastructure and Water Resource infrastructure planning by the Management MOP. MOP. • Progress in implementing processes associated with Ministerial planning. Completion of MOPW Studies. • Increase of the investment budget coming from plans in 2014 to 99 %; which corresponds to 93% in number of projects. WATER REGULATION: Strengthening of existing institutions, improvement of internal management and improvement of information delivery on Dirección General de Aguas. OBJECTIVES ACHIEVEMENTS • Strengthening existing institutions • Diagnosis of Water Resources. • Improvement of internal • Development of two draft laws and progress management in its processing (modification of the water • Improving of information code and penal code). delivery on Dirección General de • Reduction of time processing applications. Aguas. • Launch of the Maintenance Management System “Sistema de Gestión del Mantenimiento (SGM)” in the Hydrometeorological Network: Phase I of the Maintenance Management System at the regions: Metropolitana, Coquimbo and Nivel Central and Phase II of the System for six more regions (XV, I, II, III, V and VI). • Development of a national water policy. • Publication of three regulations (groundwater, major works and ecological flow). • Launch of White Glacier inventory. • Launch of Phase II Maintenance Management System DGA. • Launch of National Water Resources Strategy • Study "Improving the Institutional Framework for Water Management.” • Study "Preparing Plan for Institutional Development of Water Resources" 25 • Publication of new modules of the National Water Bank including online historical information. • Implementation of the National Water Information System “Sistema Nacional de Información del Agua (SNIA)”. COMPONENT 2: STANDARDS AND CONTRACTS (Appraisal estimated cost: US$ 4.85 million ; actual cost: US$ 1.22 million ) STANDARD: Establishment of a methodology to define, evaluate and update the standards and levels of service by MOPW works typologies, for incorporation into the bidding projects or other regulatory instruments. OBJECTIVES ACHIEVEMENTS • Establishment of a methodology • Identification of standards and levels of to define, evaluate and update the service for 13 construction types and service standards by types of incorporation of explicit standards of service works of MOP in the new concession bidding projects. • Contracts designed through the • Development and dissemination of concession system incorporating concessioned airports Improvement Plan the concept of service standards based on the results of the satisfaction survey of airport users. Preparation of a second concessional airports improvement plan from a second study of airport user satisfaction. • Development of procurement policy by service levels. • Developing a roadwork improvement Plan based on the results of an urban highways and roads users satisfaction survey. • Preparation of Rainwater Manual. • Preparation of Tender of Road Works Concession, incorporating explicit standards and payment methods according to the service levels achieved. PROCUREMENT SYSTEM: Development of a procurement system by service levels, to increase competition in the public works market and allocate risks appropriately. OBJECTIVES ACHIEVEMENTS • Improvement of the procurement • Started CGR analysis for improvement system of the Ministry of Public works’ regulation. Works ¨Ministerio de Obras 26 Públicas (MOP)¨, increasing • MOPW internal work for improvement of market competitiveness through consulting rules and beginning of an analysis regulatory changes with Controller “Contraloría” to amend the • Efficiency and effectiveness of recruitment rules of consulting works. the procurement system • The amended of the regulations for public improvement through works contracts was obtained from standardization of contracts and Controller “Contraloría”. bidding. • Strengthening of the legal support of the procurement system through the restructuring of the ¨Fiscalía de Obras Públicas¨. • MOPW procurement processes improvement through the incorporation of technology. COMPONENT 3: INTEGRATED PROJECT MANAGEMENT (Appraisal estimated cost: US$ ; actual cost: US$ ) Design and implement an integrated view of project management, modernizing the design process, the construction, the maintenance and the operation of the works, from contract management to a project-based management. OBJECTIVES ACHIEVEMENTS • Implementation of the concept of • Project Management Office (PMO) in project managers operation across the services, integrated into the work of the Project Management • Training on Project Management Division of the DGOP. techniques. • Extension of the SGM to airport and port • Implementation of technological infrastructure. tools GIP. • Increase of relevant projects managed under • Implementation of Project GIP methodology. Management Office (PMO) • 500 MOPW professionals training in project • Implementation of world- class management, achieving certification of 50 of management for managing them. infrastructure assets. • Incorporation into Maintenance Management System of 5 new regions at Roads Department ¨Dirección de Vialidad” (Tarapacá, Antofagasta, Coquimbo, O'Higgins and Magellan), and the Departments of Airports and Port Works “Dirección de Aeropuertos y Dirección de Obras Portuarias”. • Preparation of DOP Cadastre. 27 • Project Management training. Methodology PMI. PMBOOK. PMP. • Implementation and use of digital work book in new projects. • GIP methodology integration with the Quality Management System to achieve unique standardization. • Strengthening coordination in services project management control and DGOP. • Progress in DAP Cadastre, so to start the SGM. • DOP Manual. • Implementation of Maintenance Management System at Roads Department ¨Direction de Vialidad”, in the whole country under the Conservation Mode by Direct Administration. COMPONENT 4: PUBLIC INFRASTRUCTURE REGULATION STRENGTHENING (Appraisal estimated cost: US$ ; actual cost: US$ ) FISCALIZATION: Strengthening of oversight and control mechanisms for compliance with the service standards of MOPW public works through the generation and application of tools and methodologies that supports the audit function and the protection of user rights. OBJECTIVES ACHIEVEMENTS • Strengthening of monitoring and • Control System used in 20 Interurban Road control mechanisms for contracts “Viales Interurbanos” (OVI), in 6 compliance with the service urban highway contracts “Viales Urbanos” standards of MOPW public (OVU) and in 2 public transport corridors works, in their operation phase, “Corredores de transporte público” (CORR). through the generation and • This Control System for Interurban Road, application of tools and Urban Highways and Transport corridors methodologies that support the concession contracts is used in 28 contracts. audit function and the rights Beginning the trial run of the applications: protection of user rights. Nonconformance Module and Field Inspection and Field Inventory Module in all of 28 contracts. Thus 100 % of Road Contracts with Operational Consulting are using the new Control System in White March phase. • Control system in use in 37 contracts of road and airport concessions. • Development and use of Management and Inspection of Rural Drinking Water “Gestión 28 e Inspección de Agua Potable Rural (APR)”. • Implementation of nonconformities Module in OVI, OVU and CORR, to display the efforts undertaken from the findings detected in periodic audits and the corrective actions. • Integration of the prison infrastructure to the operating contracts information system. SATISFACTION: Evaluation and monitoring of the satisfaction of users of public infrastructure services system implementation, to provide feedback throughout the life cycle of the MOPW works. OBJECTIVES ACHIEVEMENTS • Increase of user confidence in • Seminar and publication "Quality of Service quality provision. Quality in Public Infrastructure” services are provided throughout • Identification of services, its quality infrastructure. attributes and its indicators and standards, so that to be incorporated into the design, construction and operation of works, both in terms of preservation and operation for 15 types of infrastructure. • Creating overarching goal of Standards and Levels of Service in contracts, so that it commits Administrations to at least consider explicit definitions of service in their strategic products. • Development of methodology for data collection, diagnosis and implementation of indicators and standards. • Development of evaluation models and satisfaction surveys for 6 types of works. COMPONENT 5: ORGANIZATIONAL REFORM (Appraisal estimated cost: US$ ; actual cost: US$ ) RULES AND REGULATIONS: Improve the quality of regulation and institutionalization of the Ministry, in order to adapt to the new requirements of infrastructure and water resources management, in a safe, sustainable and improved form. OBJECTIVES ACHIEVEMENTS • Design of the Superintendent of • Law 20.410 to amend the Concessions Public Works “Superintendencia de Law (Ley de Concesiones). Amendment to Obras Públicas” and construction the Concession Bylaw (Reglamento de la 29 and implementation of the basis for Ley de Concesiones) with complete this operation. processing. • Support of the Processing of draft • Damming Law regulating flood control laws submitted to Congress. function and its bylaw completed • Systematization of MOPW processing. Normative. • Changes to the institutional structure of MOP, strengthening the functions of planning and implementation of projects. STRATEGIC MANAGEMENT: Establishment of a systematic process of ministerial strategic planning, which defines the objectives and priorities for the organization, with a measurement system that provides timely information to support decision making by defining, implementing and monitoring indicators performance. OBJECTIVES ACHIEVEMENTS • Design and implementation of an • The MOPW has a mission and a vision, as integrated dashboard to support the well as a Ministerial Strategic Map and modernization strategy of the Strategic map for each of the Services. Ministry of Public Works. • Implementation and operation of the • Alignment: Improvement of PMG Indicators Management System “Sistema Management Program - MOPW de Gestión de Indicadores (SGI)” at Modernization Program. Ministerial level. • Implementation of technological • Generating Performance Collective tool that facilitates data analysis for Agreement 2013 “Convenio Desempeño decision making Colectivo 2013” reports through SGI. • Maintenance and improvement of the SGC Ministerial and expanding its scope in the central level. CITIZEN RELATIONSHIP: Installation of an integral model of citizen relationship for the administration and management of the information entered by ministerial communication channels in order to better meet the needs of citizens. OBJECTIVES ACHIEVEMENTS • Design of a comprehensive citizen • Implementation of the MOPW relationship model. strategy of information and citizen • Creation of a Citizen Relationship and attention, establishing effective Ministerial Information Management Unit mechanisms for communication and the network responsible by service. and interaction between the MOPW and citizenship • Web MOPW and related sites construction. • Implementation of a document • Integration of citizen and stakeholders 30 management model for the flow of requests in a single technology Ministry. The model is designed to platform. ensure coverage of the needs of • Update the corporate image for MOPW documental information, evidence citizen attention. and transparency required by the Ministry of Public Works. • Publication of the Manual of Public Relations and Ministerial Information Management. • National training workshop on transparency and document management. • Design a model of document management. • Design of a new technological platform for citizen attention. • Implementation of MOPW new citizen service platform to manage applications and record attention services. • Implementation of IVR telephone central in regions, to improve telephone assistance. • Tender for the construction of the document management platform. HUMAN RESOURCES: Strengthening of people management to support the achievement of corporate goals, intervening human resources subsystems to achieve greater efficiency and to adapt to new needs. Promotion of change management actions needed for the success of the program. OBJECTIVES ACHIEVEMENTS • Working environment stabilization • Formulation of Collective Transverse through concrete actions Target HR 2010-2013 “Meta Colectiva • Management and correction of HR Transversal de RRHH 2010-2013” and subsystems: implementation of actions under the works • Design and implementation of a program. recruitment and selection • Establishment and operation of the Bureau mechanism that ensures to have the of Strategic HR “Mesa Estratégica de best for each position RRHH“ and inter-service Work Teams. • Training System focused on • Design management model by ministerial updating technical knowledge and competencies. skills development • Formulation of HR Plan 2010-2014 “Plan • Implementation of positions Director de RRHH 2010-2014”. through the Public Management System • Definition of key positions. Development • Increase HR information of 194 key positions profiles and 321 not • Promotion and propitiation of key positions. knowledge management in the • MOPW Induction Program design. MOP • Implementation of the performance management improvement process. 31 • Edition of Manual HR Competency Management • Tendering for the development project of the new ministerial Intranet. • First measurement of work environment at ministry level. • Discussion of work environment measurement results, and analysis workshops throughout the country. • Development of information architecture and implementation of the new ministerial intranet platform. • Management Skills Program for key positions occupants. • Workforce Development Management Program for MOPW employees. • Transversal Training Program • Design of Human Resources Strategic Framework 2014 – 2017 “Marco Estratégico de Recursos Humanos 2014 – 2017”, which includes Director Plan, Policies , Strategic Plan , Comunicational Strategy and Policy and Communicative Quadrennial Strategic Directions 2014- 2017. • Operational improvements in HR subsystems • Design of the Work Environment Ministerial Strategy “Estrategia Ministerial de Clima”, defining guidelines and development program of actions to improve working environment. • Formulation and design of HR communication strategy • Start-up and implementation of the ministerial intranet, content management, and training management platform. • Delivery of laptops to regional intranet editors. • Improvements in operational HR Subsystems: Transposition Portal Public Employment; Approval of tender forms of training; Definition of Child charges; User profiles Charges; MOPW Training Program Performance Evaluation. 32 INFORMATION SERVICES: Development of a strategy for information technology and communication to support the quality and efficiency of the processes of ministerial management. OBJECTIVES ACHIEVEMENTS • Design and implementation of • Development of the technological tools business solutions considering required by other components. improvements in processes relying • Digital Control System implementation on information and and monitoring invoices and other tax communications technologies documents (SC and SDT) at regions. • Development of Web Contractor Registration System. • Implementation of the Airport Pavement Management System. • Expansion of SC and SDT to central level. • Acquisition of technological tool to generate Ministerial reports. • SGM Roads “Vialidad” and DGA • IVR Call Center • Software Audit • Improvements to the Tax Receipt of Documents 33 Annex 3. Economic and Financial Analysis The project should be considered as highly cost effective since costs should be exceedingly compensated by improved capacity reflected in fiscal savings and efficiency gains. The MOPW manages a budget for 2014 of about US$ 3.3 billion of which about US$ 2.4 billion are allocated to investments. The cost of the project, at a total of US$ 29.15 million, represents less than 1% of the budget of one year, yet it was sufficient to introduce deep and permanent improvements in efficiency and service levels. The already confirmed reduction in the costs and execution times of contracts that are being derived from the adoption of an “integrated management system” would exceedingly compensate for the costs of the project. Additional gains, probably more sizable but more difficult to quantify, should be derived from the selection of projects that focus on equity and the social benefits that were previously ignored. 34 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Julieta Helena Abad-Oholeguy Consultant LCSEG Ana Maria Grofsmacht Procurement Specialist LCSPT Jose L. Guasch Consultant LCSPF Marta Elena Molares-Halberg Consultant LEGLE Andres Prizarro Consultant SASDI Tomas S. Serebrisky Senior Infrastructure Economist LCSSD Jordan Schwartz Manager TWISI TTL Emily Sinnott Senior Economist ECSH4 Alejandro Roger Solanot Sr Financial Management Specialist LCSFM Supervision/ICR Julieta Helena Abad-Oholeguy Consultant LCSEG Daniel Jorge Arguindegui Procurement Specialist LCSPT Jose A. Barbero Consultant LCSTR Daniel Alberto Benitez Senior Transport Economist LCSTR TTL Carter J. Brandon Lead Economist LCSSD Fernando Crespo Senior Transport Economist LCSTR (former TTL) Jose L. Guasch Consultant LCSPF Ana Lucia Jimenez Nieto Financial Management Specialist LCSFM Efraim Jimenez Consultant EASIS Ana Maria Kuschnir Program Assistant LCC7C Marta Elena Molares-Halberg Lead Counsel LEGES Xiomara A. Morel Sr Financial Management Specialist LCSFM Andres G. Pizarro Consultant SASDI Geise B. Santos Program Assistant LCSEN Jean-Jacques Verdeaux Senior Procurement Specialist LCSPT Francisco Rodríguez Procurement Specialist LCSTR Santiago Scialabba Program Assistant LCC7C Tomas S. Serebrisky Senior Infrastructure Economist LCSSD (former TTL) (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY07 1.20 4.44 FY08 Total: 4.44 Supervision/ICR FY07 0.00 FY08 10.78 53.91 35 FY09 8.82 59.44 FY10 13.86 73.31 FY11 18.06 100.36 FY12 18.61 84.42 FY13 26.58 119.22 FY14 9.50 59.64 Total: 550.30 36 Annex 5. Beneficiary Survey Results N.A. 37 Annex 6. Stakeholder Workshop Report and Results N.A. 38 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR Programa de Fortalecimiento Institucional INFORME DE CIERRE PROYECTO DE FORTALECIMIENTO INSTITUCIONAL DE ASISTENCIA TÉCNICA DEL MINISTERIO DE OBRAS PÚBLICAS – (TAL) N° 7458 – CH CONTRIBUCIÓN DEL PRESTATARIO Subsecretaría de Obras Públicas Unidad Ejecutora Ministerio de Obras Públicas 39 1. INTRODUCCIÓN El Programa de Fortalecimiento Institucional del Ministerio de Obras Públicas (MOP), fue una reforma integral que buscó fortalecer las áreas estratégicas del Ministerio, para planificar integralmente los proyectos de infraestructura y recursos hídricos, realizar contratos con estándares de servicio, hacer más eficiente la gestión de los proyectos y mejorar la capacidad de regular y fiscalizar las obras públicas que mejorarán la calidad de vida de la ciudadanía. A partir del 2007, el Ministerio de Obras Públicas inició el proceso de modernización, para potenciarse como un proveedor de obras y servicios de infraestructura pública más eficaz y eficiente, con mayor capacidad de planificación y gestión, al servicio del bienestar de las personas y de la competitividad de los sectores productivos del país. Esto ha significado una transformación en el sentido y forma de proveer obras y servicios de infraestructura pública al país; un cambio que significó pasar de sólo ejecutar contratos a gestionar obras al servicio de la ciudadanía. Esta estrategia de cambio que incluye la declaración de su propuesta de valor a la ciudadanía, la revisión y rediseño de sus procesos de negocio, con énfasis en fortalecer las competencias de sus 8.700 funcionarios (aproximadamente). 2. ANTECEDENTES DEL PROGRAMA Y ROL DEL TAL El Programa de Fortalecimiento Institucional del Ministerio de Obras Públicas o también llamado, al interior del Ministerio, Programa de Modernización, se financió parcialmente con los recursos de un convenio de préstamo de asistencia técnica suscrito con el Banco Internacional de Reconstrucción y Fomento (BIRF), Banco Mundial, hasta por un monto de 11.7 MMUS$, con un plazo de ejecución entre los años 2008 y 2012 y, que en su último año de ejecución, se extendió hasta el 31 de enero del 2014. Dicho programa, dada su magnitud y alcance, toda vez que actúo sobre la gestión interna del Ministerio, objeto y sujeto de esta transformación, fue relevado y se le denominó como un proyecto de Estado, incluido en las metas gubernamentales ministeriales. Los procesos de transformación que se impulsaron ya están en etapa de consolidación, operan en forma transversal al interior del Ministerio y pasaron a ser parte de las prácticas permanentes. Sus principales líneas de acción fueron: • Fortalecimiento de la institucionalidad de aguas. • Consolidación del modelo de planificación integrada con visión territorial. • Estudios para mejorar el reglamento de obras y consultorías. 40 • Consolidación el funcionamiento de las Oficinas de Gestión de Proyectos. • Administrar proyectos bajo la metodología de Gestión Integrada de Proyectos. • Implementación del Sistema de Gestión del Mantenimiento. • Avanzar en el modelo integral de relación ciudadana. • Consolidar los proyectos tecnológicos desarrollados. • Avanzar en el modelo de gestión por competencias. La Subsecretaría de Obras Públicas, desarrolló durante estos años una restructuración funcional, cumpliendo con la misión institucional, permitiendo instalar procesos y unidades que mejoraron la gestión interna de la institución a nivel ministerial e implicó el desarrollo de la infraestructura pública, y el uso sustentable y eficiente de los recursos hídricos. Hoy gran parte de las iniciativas se encuentran en su fase de término o con un avance significativo, salvo las que por su planificación natural exceden el término del programa, como por ejemplo: los avances en el Sistema de Gestión del Mantenimiento en otros Servicios Ejecutores MOP, la Actualización de la Información de Infraestructura Aeroportuaria, las mejoras en el Sistema de Gestión Documental; y el segundo estudio de la Dirección de Planeamiento para el análisis y medición de variables e indicadores de planes territoriales de infraestructura y gestión de recursos hídricos MOP El Programa de Modernización ya es una realidad, ya es parte del MOP, y desde la Subsecretaria se consolidó como aquel proyecto que permitió, con la ayuda de los funcionarios, mantener como práctica habitual el mejoramiento continuo de nuestro accionar. Con ello, se crearon instancias de trabajo y nuevas estructuras, que permiten mantener a través del tiempo las líneas de trabajo impulsadas: • Comité de Planificación Integrada (Dirección de Planeamiento) • Unidad de Estándares (Dirección General de Obras Públicas) • 7 Oficinas de Gestión de Proyectos PMO (Dirección General de Obras Públicas) • Unidad de Gestión Ministerial (Unidad de Monitoreo Ministerial de la Subsecretaría) • Unidad de Relación Ciudadana y Gestión de la Información (Subsecretaría) • Academia de Obras Públicas (Subsecretaría) 3. CUMPLIMIENTO DE LOS OBJETIVOS Y PRINCIPALES AVANCES EN LAS MATERIAS Se planteó como desafío mostrar resultados que avalen al Ministerio como un proveedor, gestor de obras y servicios de infraestructura pública, más eficaz y eficiente, con una 41 clara orientación hacia el usuario, focalizado en su gestión interna, potenciando 4 ejes: planificación de infraestructura y gestión de recursos hídricos, desarrollo de estándares, gestión de proyectos y fiscalización. De acuerdo a esto, a continuación se muestran los principales avances. Con respecto a instalar un modelo de planificación integrada, el objetivo principal fue contribuir a un enfoque sostenido integrado de la infraestructura pública y del recurso hídrico, con un diagnóstico inicial en la cual destacaba una planificación no sistémica, sin mirada de largo plazo ni multisectorial, sin coordinación con otros actores relevantes. Al cierre del proyecto, se ha conseguido diseñar e instalar un proceso de planificación ministerial orientado abordar los nuevos desafíos de provisión de servicios de infraestructura, generando un alineamiento estratégico en torno a resultados e impactos para el desarrollo territorial, logrando con esto construir una visión estratégica de desarrollo país de mediano y largo plazo a fin de proveer servicios de infraestructura con un sentido estratégico. Entre las políticas desarrolladas, cabe destacar que se cuenta con una propuesta de una Política de Financiamiento para la Infraestructura MOP, que incluye en primer término un análisis y diagnóstico de las necesidades financieras para resolver las brechas de infraestructura en el país, detectadas por diferentes estudios e instituciones, luego las alternativas y modelos de financiamiento basadas en la experiencia nacional e internacional y finalmente una propuesta de alternativas y posibles cursos de acción en los ámbitos fiscales como en el campo de la colaboración público-privada. También se finalizó la Política Nacional de Servicios de Infraestructura de Borde Costero, desarrollada por la Dirección de Obras Portuarias con apoyo de Dirección de Planeamiento, en la cual se proponen los lineamientos de acción que deben enmarcar el desarrollo de las diferentes tipologías de infraestructura de borde costero en cada uno de los cuatro ámbitos considerados: Cobertura, Sostenibilidad Económica, Calidad y Sustentabilidad. En la segunda dimensión de trabajo, se realizó el seguimiento de la Política de Conservación Vial, elaborada en tres etapas, entre el 2010 y el 2012, etapas que responden a la problemática de las redes pavimentadas, redes de caminos básicos y redes no pavimentadas respectivamente. Así mismo, se realizaron actividades de seguimiento de la propuesta de política intersectorial de Infraestructura Aeroportuaria. En ambos casos se verificó que en términos generales, en aquellas intervenciones a las cuales hace referencia la política, están en consonancia con los lineamientos de política planteados originalmente. En el ámbito de la modernización se desarrollaron planes de inversión, de carácter regional, estos planes y su cartografía se difundieron en el sitio web del Ministerio, en el portal de internet de la Dirección de Planeamiento y en el Observatorio de la Infraestructura y de los Recursos Hídricos: http://sit.mop.gov.cl/observatorio ; ambos sistemas fueron desarrollados en el marco del Programa y permiten facilitar el acceso y la integración de la información espacial, tanto a nivel institucional como con los propios ciudadanos. 42 También en el marco de la modernización de la institucionalidad del agua y del recurso hídrico la situación inicial era débil, con tiempos de respuesta a solicitudes demasiado extensos, con información respecto de disponibilidad y calidad del recurso deficiente, incompleto y disgregado. Ante ese escenario, entre los años 2010 y 2011 se realizó, en conjunto con el Banco Mundial, un informe de diagnóstico de la gestión de este recurso en Chile, que sirvió de base para la formulación de una política y estrategia nacional sobre esta materia. En la preparación de esta estrategia, se definió, como uno de los desafíos claves, el mejoramiento de la institucionalidad de aguas del país. En este mismo contexto se desarrollaron dos estudios: “Mejoramiento del marco institucional para la gestión del agua” y la “Preparación del Plan de Desarrollo Institucional de los Recursos Hídricos”, ambos financiados parcialmente con aportes del Banco Mundial. Por otra parte, la componente de estándares y contratos radicada en la Dirección General de Obras Públicas, declaraba al inicio del proyecto que la gran mayoría de los contratos de obras del ministerio no incluía compromisos explícitos de servicio y solo respondían a requisitos constructivos. Por tanto, el ministerio no contaba con mediciones respecto a la percepción de los usuarios sobre la calidad de los servicios que presta la infraestructura pública del país. En este contexto se trabajó en la identificación de estándares y niveles de servicio para un total de 11 tipologías de obra, 41 contratos con estándares explícitos, se elaboró y difundió un plan de mejoras de aeropuertos concesionados, se incorporaron estándares de servicios explícitos en las nuevas licitaciones de proyectos concesionados, además de elaborar una política de contratación por niveles de servicio, junto con un plan de mejoras de obras viales. Además durante la ejecución del proyecto fue posible elaborar el Manual de Aguas Lluvia: Manual de drenaje urbano y la guía para el Diseño, Construcción, Operación y Conservación de Obras Marítimas y Costeras. En cuanto a la componente de contratos, poco se avanzó; sólo se trabajó en el análisis y de las mejoras del reglamento de obras del ministerio y una modificación al reglamento de consultorías, ambas materias que aún se mantienen en análisis. Antes del inicio del programa, existía una cartera de proyectos desagregada, sin visión de conjunto, no se lograba visualizar las diferentes etapas de un proyecto: factibilidad, diseño y construcción; por esa razón se estableció en la orgánica del programa una componente Gestión Integrada de Proyectos (GIP), con el fin de integrar las diversas etapas de la obra, controlar las variables de costos, calidad y plazos utilizando herramientas tecnológicas y una nueva metodología de administración de manera de contar con un perfil de jefe de proyecto, profesionales altamente calificados, impulsando con ello mejores prácticas en la realización de contratos de obra pública. Se implementaron las oficinas de gestión de proyectos (PMO), cuya función principal fue estructurar y desarrollar un soporte metodológico, alineado con la visión de GIP instaurando buenas prácticas y metodologías que ayuden a las autoridades y a los equipos responsables de la ejecución de la inversión, haciendo más eficaz y eficiente su labor. 43 En materia de fiscalización se diseñó un sistema informático que buscó fortalecer los mecanismos de fiscalización y control, para el cumplimiento de los estándares de servicio de las obras públicas. En cuanto a gestión interna, se avanzó en la elaboración de un modelo de gestión estratégica Balanced Scoredcard Ministerial, sistema de reportería ministerial y mejoras en los sistemas de información tales como: portal web, intranet, portal de relación ciudadana. También se implementó un modelo integral de atención ciudadana, incorporando competencia para gestionar los procesos que nos permiten responder oportunamente a las demandas de la ciudadanía. En materias de modernizar la gestión de personas, para fomentar mayor eficiencia y aportar en el logro de los objetivos institucionales, en el marco de la subcomponente de recursos humanos se creó la Academia de Obras Públicas, cuyo objetivo es brindar una asesoría de apoyo para el desarrollo de las capacidades de formación de los Inspectores Fiscales. 4. DESEMPEÑO DEL BANCO Durante el plazo de ejecución del préstamo se realizaron varios cambios de Gerente de Proyecto, tanto al interior del Banco, como en la Unidad Ejecutora. Ante cada reemplazo el Banco Mundial realizó una misión, para presentar a cada nuevo jefe de proyecto, cada cambio consideró profesionales calificados y dispuestos a trabajar con la Coordinación del Programa. Gracias a los acuerdos sostenidos entre el Gobierno y el Banco, fue posible desarrollar y continuar con los procesos de mejora, implementados en el marco de la modernización. En este contexto, el Banco Mundial colaboró en la supervisión del proyecto, revisando aspectos del ámbito de gestión financiera y adquisiciones. Sin embargo, faltó más apoyo en materias de carácter técnico, para orientar, contribuir e instalar los avances del proyecto propiamente tal. 5. DESEMPEÑO DEL PRESTATARIO El ciclo de Administración del Estado de Chile, comprende cuatro años de gobierno, un periodo muy breve de gestión, esto provocó una dificultad en el funcionamiento, puesto que mediante cada cambio debíamos presentar los objetivos y lineamientos a nuevas autoridades. Por nuestra parte, debemos reconocer que se debió impulsar un plan comunicacional continuo y permanente, para enfrentar una significativa resistencia al cambio, debido a que el proyecto era innovador y pretendía modificar la forma de trabajo, de cultura y cambio de paradigma en el ámbito de cómo ejecutar las tareas del ministerio. En relación a la ejecución del proyecto su desempeño fue satisfactorio, calificación y 44 opinión consensuada con el Banco Mundial. Esto representa que el Programa de Modernización quedó inserto a nivel ministerial, es decir, las componentes que se crearon pasaron a la línea y a la fecha de cierre de nuestro informe son parte de las unidades funcionales, dichos objetivos fueron cumplidos gracias al compromiso de las autoridades de turno, quienes generaron bases de lineamientos transversales e involucraron proactivamente a los funcionarios. En lo que respecta al préstamo, su ejecución fue lenta, podemos mencionar que una de las razones fue el bajo umbral de desembolso, para consultorías o estudios bajo las normas de adquisiciones nacionales, las que no podían pasar los USD$100.000, esto significó realizar una serie de gestiones tendientes a modificar el convenio, para lograr una mayor ejecución y ampliar el umbral de desembolsos, acciones que se contrataron en el último año de ejecución. Es importante señalar que dichas modificaciones podrían haberse realizado mucho antes. No obstante, dicho cambio aportó positivamente a la ejecución financiera, al cierre del préstamo se desembolsó un total de $ 4.194.389.839 de pesos chilenos, lo que representa una utilización del préstamo de un 70,4%. 6. LECCIONES APRENDIDAS La dirección del proyecto, específicamente la Unidad Ejecutora se estableció inicialmente en la Dirección de Contabilidad y Finanzas, luego de un año fue trasladada a la Dirección General de Obras Públicas. En septiembre 2010, el Ministro de Obras Públicas decidió traspasar la coordinación técnica y administrativa a la Subsecretaría, a partir de enero 2011. Este cambio obedeció al rol transversal y estratégico del Servicio, así como por su mayor nivel de decisión y relevancia dentro de la orgánica Ministerial. Este cambio fue muy relevante, para alcanzar los objetivos propuestos, por tanto una lección aprendida que debemos tener presente para proyectos de la misma envergadura, será ubicar la Unidad Ejecutora en lo más alto del organigrama. Este programa se centró en las áreas de la pertinencia, transparencia en la planificación, contratación, gestión, conservación y explotación de las inversiones, ya sean financiadas con fondos fiscales, por concesión u otra modalidad. También se consideraba la separación institucional del rol de fiscalización de la calidad de servicio y la promoción de garantías explícitas de calidad de servicio a proveer con las obras. En este marco, las restricciones en procedimientos operativos y la nueva estructura institucional, requería de un gran apoyo en los procesos de cambio, debido que consideraba modificaciones al interior de la organización, precisamente en la forma de gestionar sus procesos y resultados asociados. El paso de una situación a otra supone un cambio importante que necesita y requiere un acompañamiento comunicacional significativo, una gestión del cambio previo al inicio del proyecto, lo que no ocurrió en este caso. Otra lección a considerar que destaca positivamente fue la factibilidad de realizar modificaciones al convenio de préstamo, lo que le otorgó flexibilidad en dos aspectos: una modificación funcional en las componentes, un cambio en su estructura, dentro del marco regulatorio del convenio y la segunda fue ampliar el plazo del préstamo, junto con 45 el umbral de desembolsos al amparo de las normas nacionales. Ambas reformas fueron advertidas durante su ejecución, lo que significó una flexibilidad en el contrato y un acuerdo entre las partes. En términos operativos, la administración y ejecución de un proyecto, se identifica por llevar un alto flujo de documentos y ejecutar un gran número de acciones de carácter administrativo, lo que conlleva a un manejo considerable de información, en donde se ven involucrados funcionarios de todas las unidades y una gran coordinación interna, tareas que demandan una gran carga de trabajo y que no están ajenas a falencias. Por lo anteriormente expuesto, se determinó impartir capacitaciones internas a todas las contrapartes técnicas de convenios vigentes, junto con revisar procedimientos y generar manuales operativos. En temas de indicadores de gestión es preciso señalar que son una herramienta de control que permite observar la información desagregada y su estado de avance. En base a lo citado, es importante destacar que los indicadores deben ser factibles de alcanzar, de acuerdo a actividades planteadas y no tan solo traducidas a un número o reporte, sino que deben servir de ayuda para fomentar el mejoramiento de los resultados del proyecto, impulsando la eficiencia, eficacia y productividad de los procesos involucrados. 7. CONCLUSIONES En términos generales, es preciso señalar que durante la ejecución del Programa de Fortalecimiento Institucional, se presentaron debilidades que afectaron el cumplimiento de los objetivos propuestos inicialmente, tales como el recambio de autoridades y coordinadores, las dificultades de modificar normas y reglamentos; y la falta del acompañamiento comunicacional de forma permanente y continua. Al margen de las dificultades que se presentaron, es preciso señalar que los procesos de mejoras que se realizaron al interior del ministerio, permitieron cambiar el paradigma y pasar de sólo construir obras a ser un organismo público orientado al servicio de la comunidad, con la capacidad de instalar nuevos procesos en el ámbito de planificación estratégica, con objetivos y prioridades, contando con una metodología moderna de gestión de proyectos nos lleva a pasar de una administración fragmentada de contratos a una gestión integrada de los proyectos, para concluirlos a tiempo y en los costos estipulados, disminuyendo sus riesgos, proporcionando información confiable, pertinente y oportuna para la toma de decisiones, instrumentos que permiten rendir en forma responsable cuenta pública de la gestión institucional. 46 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders N.A. 47 Annex 9. List of Supporting Documents − World Bank 2011. Chile Country Partnership Strategy 2011-2016. Report No. Report No. 57989-CL − World Bank 2007. Project Appraisal Document “Institutional Strengthening of the Ministry of Public Works Development Policy Loan” Report No: 39306-CL − World Bank 2007. Chile Country Partnership Strategy 2007-2011 − World Bank 2005. Project Appraisal Document “Santiago Urban Transport Technical Assistance Project”, Report No: 31742-CL − World Bank 2004. Project Appraisal Document “Santiago Urban Transport Deferred Drawdown Sector Adjustment Loan”, Report No: 28520-CL − World Bank 2004. Project Appraisal Document “Infrastructure for Territorial Development Project”, Report No: 30463-CL 48 MAP 49