COLOMBIA Public and Private Paths to Sustainable Water Supply and Sanitation in Colombia (1999-2011) Report No. 111404 DECEMBER 29, 2016 © 2016 International Bank for Reconstruction This work is a product of the staff of The World RIGHTS AND PERMISSIONS and Development / The World Bank Bank with external contributions. The findings, The material in this work is subject to copyright. 1818 H Street NW interpretations, and conclusions expressed in Because The World Bank encourages Washington DC 20433 this work do not necessarily reflect the views of dissemination of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they represent. noncommercial purposes as long as full attribution to this work is given. The World Bank does not guarantee the accuracy of the data included in this work. The Any queries on rights and licenses, including boundaries, colors, denominations, and other subsidiary rights, should be addressed to information shown on any map in this work do World Bank Publications, The World Bank not imply any judgment on the part of The Group, 1818 H Street NW, Washington, DC World Bank concerning the legal status of any 20433, USA; fax: 202-522-2625; e-mail: territory or the endorsement or acceptance of pubrights@worldbank.org. such boundaries. Report No.: 111404 PROJECT PERFORMANCE ASSESSMENT REPORT COLOMBIA CARTAGENA WATER SUPPLY, SEWERAGE, AND ENVIRONMENTAL MANAGEMENT PROJECT (IBRD-45070 IBRD-74040) WATER SECTOR REFORM ASSISTANCE PROJECT (IBRD-70770) WATER AND SANITATION SECTOR SUPPORT PROJECT (IBRD-72810) December 29, 2016 Financial, Private Sector, and Sustainable Development Independent Evaluation Group Currency Equivalents (annual averages) Currency unit = Peso (Col$) 1999 US$1.00 Col$1,870.5 2000 US$1.00 Col$2,202.5 2001 US$1.00 Col$2,306.3 2002 US$1.00 Col$2,859.9 2003 US$1.00 Col$2,789.4 2004 US$1.00 Col$2,420.0 2005 US$1.00 Col$2,283.4 2006 US$1.00 Col$2,226.5 2007 US$1.00 Col$1,990.4 2008 US$1.00 Col$2,168.0 2009 US$1.00 Col$2,042.5 2010 US$1.00 Col$1,924.9 2011 US$1.00 Col$1,938.5 2012 US$1.00 Col$1,767.0 2013 US$1.00 Col$1,928.6 2014 US$1.00 Col$2,375.3 2015 US$1.00 Col$3,174.3 Abbreviations and Acronyms ACUACAR Aguas de Cartagena (Cartagena’s water and sewerage company) ACUAS Empresas Departamentales de Acueducto y Alcantarillado (Departmental Water and Sewerage Companies) APL adaptable program loan CARDIQUE Corporación Autónoma del Canal de Dique (the Regional Environmental Authority in Cartagena) CAS Country Assistance Strategy CONPES National Council of Political and Social Economy CPS Country Partnership Strategy CRA Comisión Reguladora de Agua Potable y Saneamiento (Potable Water and Basic Sanitation Regulatory Commission) DANE National Department of Statistics ERR economic rate of return FINDETER Financiera de Desarrollo Territorial (Regional Development Financing Institution) FONADE Fondo Financiero de Proyectos de Desarrollo (Colombian Fund for Development Projects) GDP gross domestic product IBRD International Bank for Reconstruction and Development ICAM Indicador de Calidad de Agua Marina ICR Implementation Completion and Results Report ICRR Implementation Completion and Results Report Review IEG Independent Evaluation Group INSFOPAL Instituto de Fomento Municipal (Municipal Development Institute) IRCA Risk Index of Water Quality for Human Consumption ISR Implementation Status and Results Report INVEMAR Institute of Marine and Coastal Investigations (Institute of Marine and Coastal investigations) M&E monitoring and evaluation MAVDT Ministerio de Ambiente, Vivienda y Desarrollo Territorial (Ministry of Environment, Housing and Regional Development) O&M operation and maintenance PAD Project Appraisal Document PDA departmental water plan PDO project development objective PME Programe de Modernización Empresarial (Corporate Modernization Program) PPAR Project Performance Assessment Report SGP Sistema General Participación (General Revenue-Sharing System) SIU Superintenedence of Public Services SSPD Superintendencia de Servicios Públicos Domiciliarios (Superintendence of Domestic Public Services) UFW unaccounted-for water WSRAP Water Sector Reform Assistance Project WSS water supply and sanitation (services) WSSSP Water and Sanitation Sector Support Project Fiscal Year Government: January 1–December 31 Director-General, Independent Evaluation : Ms. Caroline Heider Acting Director, IEG Public Sector Evaluation : Mr. Stoyan Tenev Manager, Sustainable Development : Ms. Midori Makino Task Manager : Ms. María Elena Pinglo iii Contents Contents ............................................................................................................................. iii Principal Ratings ............................................................................................................ vi Key Staff Responsible .................................................................................................. vii Preface................................................................................................................................ ix Summary ............................................................................................................................. x 1. Background and Context................................................................................................. 1 Country Context .............................................................................................................. 1 Sector Background .......................................................................................................... 1 Institutional and Regulatory Framework ........................................................................ 2 Private Sector Participation in the Water Sector ............................................................ 3 2. Cartagena Water Supply, Sewerage, and Environmental Management Project (1999- 2009)..................................................................................................................................... 4 Background and Context ................................................................................................ 4 Objectives, Design, and Relevance................................................................................. 5 Implementation ............................................................................................................... 9 Safeguards and Fiduciary Compliance ......................................................................... 10 Achievement of the Objectives ..................................................................................... 10 Efficiency ...................................................................................................................... 17 Ratings .......................................................................................................................... 18 Outcome .................................................................................................................... 18 Risk to Development Outcome ................................................................................. 18 Bank Performance ..................................................................................................... 19 Borrower Performance .............................................................................................. 19 Monitoring and Evaluation ....................................................................................... 20 3. Water Sector Reform Assistance Project (2001–10) .................................................... 21 Background and Context .............................................................................................. 21 Objectives, Design, and Relevance............................................................................... 21 Implementation ............................................................................................................. 27 Safeguards and Fiduciary Compliance ......................................................................... 27 Achievement of the Objectives ..................................................................................... 28 Efficiency ...................................................................................................................... 34 This report was prepared by Maria Elena Pinglo and Vibecke Dixon, who assessed the projects in February-March 2016. The report was peer reviewed by Andreas Rhode and panel reviewed by John Eriksson. Romayne Pereira provided administrative support. iv Ratings .......................................................................................................................... 34 Outcome .................................................................................................................... 34 Risk to Development Outcome ................................................................................. 35 Bank Performance ..................................................................................................... 35 Borrower Performance .............................................................................................. 36 Monitoring and Evaluation ....................................................................................... 37 4. Water and Sanitation Sector Support Project (2005–11) .............................................. 37 Objectives, Design, and Relevance............................................................................... 38 Implementation ............................................................................................................. 41 Safeguards and Fiduciary Compliance ......................................................................... 42 Achievement of the Objectives ..................................................................................... 42 Ratings .......................................................................................................................... 45 Outcome .................................................................................................................... 45 Risk to Development Outcome ................................................................................. 45 Bank Performance ..................................................................................................... 46 Borrower Performance .............................................................................................. 47 Monitoring and Evaluation ....................................................................................... 48 5. Lessons .......................................................................................................................... 49 References ......................................................................................................................... 51 Appendix A: Basic Data Sheet ......................................................................................... 53 Appendix B: Operational and Financial Data of Water Sector Reform Assistance Project ............................................................................................................................................ 62 Appendix C: Water and Sanitation Support Project Methodological Approach .............. 66 Introduction ................................................................................................................... 66 Limitation...................................................................................................................... 66 Sample Selection........................................................................................................... 66 Sample Selection of Households for Survey ................................................................ 66 Appendix D: List of Persons Met ..................................................................................... 69 Boxes Box 1: Project Components, Costs, and Subcomponents ................................................... 7 Box 2: Components and Costs .......................................................................................... 23 Box 3: Components and Costs .......................................................................................... 39 v Tables Table 1: Cartagena Water Supply and Sewerage Coverage ............................................. 11 Table 2: Target Population—Percentage by Stratum ....................................................... 11 Table 3: Water Quality in the Area of Effluence .............................................................. 13 Table 4: Degree of Contamination of Water along the Beaches of Punta Canoa ............. 13 Table 5: Water Quality in the Ciénaga de la Virgen Lagoon ........................................... 14 Table 6: Operational Indicators ........................................................................................ 15 Table 7: Financial Sustainability Outcome Indicators ...................................................... 16 Table 8: Number of Private Sector Participation Contracts Signed under the Project ..... 28 Table 9: Average Operational and Financial Ratios ......................................................... 29 Table 10: Energy as a Percentage of Total Cost ............................................................... 30 Table 11: Water Supply Coverage (percentage) ............................................................... 31 Table 12: Sewage Coverage.............................................................................................. 32 Table 13: Continuity of Water Supply .............................................................................. 32 Table 14: Quality of Water: IRCA.................................................................................... 33 Table 15: Population: Percentage of Strata in Service Area ............................................. 33 Table 16: Household Beneficiaries of All Projects........................................................... 43 Table 17: Household Beneficiaries of IEG-Selected Projects .......................................... 44 Figure Figure 2.1: Water Produced and Water Consumed per Household (cubic meters) .......... 16 vi Principal Ratings Cartagena Water Supply, Sewerage, and Environmental Management Project ICRa ICR Reviewa PPAR Outcome Satisfactory Moderately Satisfactory unsatisfactory Risk to development outcome Moderate Significant Low Bank performance Satisfactory Moderately Satisfactory unsatisfactory Borrower performance Satisfactory Moderately Satisfactory unsatisfactory a. The Implementation Completion Report (ICR) is a self-evaluation by the responsible World Bank department. The ICR Review is an intermediate IEG product that seeks to independently verify the findings of the ICR. Water Sector Reform Assistance Project ICRa ICR Reviewa PPAR Outcome Satisfactory Moderately Moderately satisfactory unsatisfactory Risk to development Significant Significant Substantial b outcome Bank performance Satisfactory Satisfactory Moderately unsatisfactory Borrower performance Satisfactory Moderately Moderately satisfactory satisfactory a. The Implementation Completion Report (ICR) is a self-evaluation by the responsible World Bank department. The ICR Review is an intermediate IEG-product that seeks to independently verify the findings of the ICR. b. According to new 2015 IEG classifications, substantial replaces significant rating. Water and Sanitation Sector Support Project ICRa ICR Reviewa PPAR Outcome Moderately Moderately Moderately satisfactory satisfactory unsatisfactory Risk to development Moderate Moderate Substantialb outcome Bank performance Moderately Moderately Moderately satisfactory satisfactory unsatisfactory Borrower performance Moderately Moderately Moderately satisfactory satisfactory unsatisfactory a. The Implementation Completion Report (ICR) is a self-evaluation by the responsible World Bank department. The ICR Review is an intermediate IEG-product that seeks to independently verify the findings of the ICR. b. According to new 2015 IEG classifications, substantial replaces significant rating. vii Key Staff Responsible Cartagena Water Supply, Sewerage, and Environmental Management Project Division chief/ Project Task manager/leader sector director Country director Appraisal Menahem Libhaber Danny M. Leipziger Andres R. Solimano Completion Greg J. Browder Guang Zhe Chen Gloria M. Grandolini Water Sector Reform Assistance Project Division chief/ Project Task manager/leader sector director Country director Appraisal Menahem Libhaber Danny M. Leipziger Olivier Lafourcade Completion Greg J. Browder Guang Zhe Chen Laura Kullenberg Water and Sanitation Sector Support Project Division chief/ Project Task manager/leader sector director Country director Appraisal David N. Sislen John Henry Stein Isabel M. Guerrero Completion Taimur Samad Greg J. Browder Gloria M. Grandolini viii IEG Mission: Improving World Bank Group development results through excellence in evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank’s self-evaluation process and to verify that the Bank’s work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEG annually assesses 20-25 percent of the Bank’s lending operations through field work. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. To prepare a Project Performance Assessment Report (PPAR), IEG staff examine project files and other documents, visit the borrowing country to discuss the operation with the government, and other in-country stakeholders, and interview Bank staff and other donor agency staff both at headquarters and in local offices as appropriate. Each PPAR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the PPAR is commented on by the responsible Bank department. The PPAR is also sent to the borrower for review. IEG incorporates both Bank and borrower comments as appropriate, and the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEG Rating System for Public Sector Evaluations IEG’s use of multiple evaluation methods offers both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEG evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEG website: http://worldbank.org/ieg). Outcome: The extent to which the operation’s major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance, efficacy, and efficiency. Relevance includes relevance of objectives and relevance of design. Relevance of objectives is the extent to which the project’s objectives are consistent with the country’s current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Relevance of design is the extent to which the project’s design is consistent with the stated objectives. Efficacy is the extent to which the project’s objectives were achieved, or are expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings for Outcome: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings for Risk to Development Outcome: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loan/credit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision. Possible ratings for Bank Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower (including the government and implementing agency or agencies) ensured quality of preparation and implementation, and complied with covenants and agreements, toward the achievement of development outcomes. The rating has two dimensions: government performance and implementing agency(ies) performance. Possible ratings for Borrower Performance: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. ix Preface This is a Project Performance Assessment Report (PPAR) of three projects in the water and sanitation sector in Colombia supported by the World Bank: (i) the Cartagena Water Supply, Sewerage, and Environmental Management Project (the Cartagena Project), which was approved in July 1999 and closed in June 2009; (ii) the Water Sector Reform Assistance Project (WSRAP), approved in October 2001 and closed in October 2010; and (iii) the Water and Sanitation Sector Support Project (WSSSP), approved in March 2005 and closed in March 2011. The Cartagena Project had a total cost of US$126.86 million and was supported by two International Bank for Reconstruction and Development (IBRD) loans totaling US$85 million. The WSRAP had a total cost of US$81.54 million and was supported by an IBRD loan of US$35.68 million, and the WSSSP had a total cost of US$107.1 million and was supported by an IBRD loan of US$70 million. The assessment is based on a review of all relevant documentation, interviews of Bank staff at headquarters and in the country office, and the findings of an Independent Evaluation Group (IEG) mission that visited Colombia during March 6–19, 2016, to discuss outcomes with officials engaged with the projects, representatives of the government, staff of the Bank resident mission, and other stakeholders, such as municipalities and operators. The mission was supported by two local consultants: one gathered recent data on project performance for the WSRAP and the other headed a team of enumerators to gather new data for 24 of the subprojects under the WSSSP. The list of persons met during the mission is attached in appendix D, and their cooperation and assistance in preparing the report is gratefully acknowledged. The report presents a detailed assessment of the three operations using standard IEG methodology. All three projects were selected for this PPAR based on their emphasis on sustainability and the introduction of private sector participation. In addition, there was an explicit Regional request to review the Cartagena Project post-closure. Lessons learned from the assessment of the three projects will be used as inputs to IEG’s forthcoming evaluation of the World Bank Group’s assistance to the water and sanitation sector. Following IEG practice, copies of the draft report were sent to government officials and implementing agencies but no comments were received. x Summary Water supply and sanitation in Colombia have improved in recent decades. Between 1990 and 2010, access to improved sanitation increased from 67 percent to 82 percent, and access to improved water sources increased from 89 percent to 94 percent (WHO/UNICEF 2010), but coverage in rural areas still lags behind. The three projects covered by this assessment—the Cartagena Water Supply, Sewerage, and Environmental Management Project (the Cartagena Project); the Water Sector Reform Assistance Project (WSRAP); and the Water and Sanitation Sector Support Project (WSSSP)—are among the second generation of water supply and sanitation (WSS) projects that benefited from the lessons learned in the 1990s from Bank-supported WSS projects in Colombia. Although somewhat different in reach, focus, nature, and geographical coverage, the Cartagena Project and the WSRAP portray a somewhat logical development in the Bank’s support to Colombia’s water sector. They tried to replicate the private sector–operator model, but on a smaller scale, in medium-size and small municipalities. In contrast, the WSSSP focused on building infrastructure, and support was extended to the public sector as well as to utilities run by the private sector. The Bank’s design of these projects created a synergy with the changes and developments within Colombia’s water sector and its reforms. It accomplished this by focusing on the provision of water supply and sewerage services to the most underserved sections of Colombian society—the poor segments of the income strata, some rural areas, and small and medium-size cities. While on different scales, the three projects shared components of what was, at the time, an innovative focus on private sector participation, including (i) sector reform to support private sector participation and (ii) strengthening of private sector participation entities, as well as supporting the actual expansion of water supply and sanitation services. In addition to detailed assessments of the three operations using standard Independent Evaluation Group (IEG) methodology, the report includes data from multiple sources that were used to triangulate results. These include structured interviews with the operators and municipalities and information received from the implementing agencies at the national level; data reported to the Superintendence of Public Services (SIU); sector reports and concession contract information; and information from the National Department of Statistics (DANE) and the environmental supervision agency. The quality of the data is relatively good for subprojects run by medium- to large-size utility operators, and it has been crucial in supporting the ratings and conclusions of this Project Performance Assessment Report (PPAR). However, for the WSSSP, data for utilities located in small and remote areas were difficult to obtain. In these cases, the IEG team relied on structured interviews, field visits to a stratified sample of 24 projects, and surveys around the area of influence. The Cartagena Water Supply, Sewerage, and Environmental Management Project (1999–2009) The project development objectives were to (i) improve the water and sewerage services in the project area and the sanitary conditions of its poorest population; (ii) facilitate the environmental cleanup of water bodies surrounding the project area (Cartagena Bay, the Caribbean beaches, and Ciénaga de la Virgen Lake); and to (iii) improve the xi sustainability of water and sewerage services in the borrower’s territory through a private sector participation model. All three objectives were achieved, and some were surpassed. Water and sewerage services and the sanitary conditions of Cartagena’s poorest population were improved. The IEG mission found that water supply was at 100 percent for the population living in the project area, with continuous supply of water of high quality, and sewerage coverage was at 93.6 percent of the project area population (the target was 90 percent). IEG also found that the project had continued to serve the poorest segments of the population—85 percent of the customers of the implementing agency, Aguas de Cartagena (ACUACAR), belong to the three lowest socioeconomic strata in Colombia. Facilitation of the environmental cleanup was a success, and all the water bodies surrounding Cartagena were significantly cleaner than before project start-up. In addition, the project’s monitoring and evaluation (M&E) system was well designed, and the IEG mission found that ACUACAR continued to systematically collect and manage the monitoring data that was used for baselines and post-completion monitoring. Ratings The overall project outcome, based on relevance, efficacy, and efficiency, is rated satisfactory. Relevance of the objectives and design are both rated substantial. Achievement of two objectives was rated high, and one was rated substantial, since all objectives were achieved or surpassed. Efficiency is rated substantial, and risks to development outcome are rated negligible, since ACUACAR had proven to be an efficient and sustainable mixed-enterprise (public-private) model that survived numerous shifts in political administrations. Both Bank and borrower performances are rated satisfactory. The Water Sector Reform Assistance Project (2001–10) The project development objectives of the Water Sector Assistance Project (WSRAP) were to: (a) Support water sector reform in the borrower’s territory by facilitating an increased participation of the private sector in the management and operation of water utilities, with the intention of: (i) creating and maintaining an appropriate environment for improving the efficiency and sustainability of such water utilities and (ii) providing participating municipalities with financial support to ensure the viability of their water utilities. (b) Expand the coverage of water supply and sewerage services provided in participating municipalities. (c) Facilitate access to water and sewerage services by the population of low-income areas in such participating municipalities. (d) (i) Improve environmental protection practices in connection with the provision of water and sewerage services in the borrower’s territory, and (ii) define rural water and sanitation policies and develop adequate methods for increasing coverage for both water and sewerage services in the borrower’s rural sector. The first objective was substantially achieved. The IEG mission found that, on average, the water utilities in the program have realized operational and financial improvements xii since the initiation of their contracts, yet some are still experiencing losses (see table 3.2 and appendix B for more details). The second objective, concerning water supply and sewage coverage, was modestly achieved. While water supply coverage surpassed the original targets, the target for sewage coverage was not met. The primary target group was 700,000 people, and the project exceeded this target with its 1.6 million beneficiaries. The objective to increase in water supply coverage exceeded targets and was therefore substantially achieved. From an average base of 62 percent, there has been an overall increase in water supply coverage of 23 percent since the initiation of the project. Four of the subprojects still had low water supply coverage (47 percent, 64 percent, 65 percent, and 67 percent of target populations, respectively), but all of them had steadily, albeit slowly, increased water coverage. The remaining subprojects achieved a coverage of 72 percent or higher. The average coverage in all the project areas increased to 85 percent by 2014, which is 10 percent lower than the national average of 95 percent. However, this is a substantial achievement, although all targets (of 90 percent and 100 percent of water supply coverage) had not yet been met. In contrast, the subobjective to increase sewerage coverage was modestly achieved. Project targets were not met. Current data show that sewerage connections increased 30 percent from project start-up, and that coverage ranges between not having been implemented (in 10 subprojects), to 9 percent, 14 percent, 16 percent, and 48 percent in four subprojects, respectively, to 73–97 percent coverage in the remaining subprojects. The average coverage in the participating municipalities was 60 percent, compared with the average target of 87 percent. The third objective—of poverty focus—was substantially achieved. The IEG mission found that 90–100 percent of project beneficiaries belonged to the three lowest socioeconomic strata in Colombia. The fourth objective was modestly achieved. A decision-support tool was developed under the project, but it is unclear to what degree the tool was disseminated and used. Similarly, the rural water sanitation policy was formally defined after project completion, but given the significant time since project closure and the limited resources devoted to this activity, it is not clear to what extent activities under the project contributed to the rural water and sanitation policy formulation. Ratings As explained above, of the four objectives, two were substantially achieved and two were modestly achieved. Relevance of objectives is rated high, as they were fully in line with both national and Bank priorities at initiation, throughout the project, and at closure, and the objectives remain relevant today. Relevance of design is rated modest. Although the project’s logical chain was generally clear and convincing and the components were necessary and sufficient to reach the objectives, there was one objective that lacked project development objective (PDO) indicators in the project design. Efficiency was rated modest, mainly due to the long delay of project implementation, the increase in the budget, and the lack of evidence for the efficiency analysis presented. Risk to development outcome is substantial due to the moderate risk to financial sustainability. Both Bank and borrower performance are moderately unsatisfactory. The project’s xiii shortcomings are considered moderate, and the overall outcome of the project is therefore rated moderately unsatisfactory. The Water and Sanitation Sector Support Project (2005-11) The project development objective for the first phase of the adaptable program loan (APL), as formulated in the loan agreement (World Bank 2005a, section 2) was “to improve the access to water supply and sanitation services in rural and urban communities throughout the borrower’s territory.” The PDO regarding improved provision of water supply and sanitation services was modestly achieved. Because the project had not defined which indicators signified improved access, there was no indication that the upgraded civil works led to reliable and safe water supply and sanitation. The IEG field visit also looked at coverage and continuity of water supply of WSS utilities. The study found that for five out of seven subprojects (for which there was information available), there was an improvement of water supply coverage by an average of 30 percent by project completion, However, 7 subprojects is too small a proportion of 24 or 88 subprojects to claim representativeness. Ratings The overall outcome rating is moderately unsatisfactory. While relevance of objectives is rated substantial, relevance of design is rated modest due to a weak project design, including an inadequately designed results framework. Achievement of the project development objective of improved provision and access to water supply and sanitation is rated modest, because the project reached most of its targets by closure. Efficiency is rated modest due to weak evidence of efficiency. The key lessons derived, based on the findings of this PPAR, are the following: • Consolidation of smaller WSS operations servicing poorer neighborhoods can foster economies of scale and cross-subsidization in achieving financial sustainability at the aggregate level. The success stories of smaller operators relate to private sector operators in large urban areas extending their services to nearby small towns. In the case of the WSRAP, what worked was managed through consolidation of smaller operations. The utility Triple A began operating in neighboring municipalities that were considered poor. To make service provision profitable, Triple A designed and proposed a regional tariff methodology aimed at cross-subsidizing service costs between municipalities served by the same provider. • Small municipalities with limited service coverage require large capital investments, and because of their lack of financial autonomy, government subsidies would be required until full cost recovery is achieved. In the case of large operators such as Cartagena, all costs were to be paid by users, while small operators normally receive a government contribution to cover investment costs (the user charges that they collect cover administration and maintenance costs). The contractual arrangements required private operators to invest in the supply systems to a limited extent, but the major investments had to be financed through capital subsidies from the government. In practice, because the investment needs of the small and medium-size operators were not adequately assessed, the capital investment received was insufficient, and in many cases had to be renegotiated with the government. xiv • It is crucial to phase and/or sequence project activities to ensure achievement of the objectives of utility operators. The WSRAP and WSSSP were simultaneously trying to build the capacity of operators with no operational capacity to immediately implement an expansion of WSS services and improve service quality. This can happen in cases such as Cartagena, where the operator of the project, ACUACAR, was already experienced and ready to undertake activities to improve access and quality of services. Because it takes time to build capacities, one approach might be to move more slowly, emphasizing capacity strengthening first, and then to focus on implementation. Another approach could be to ensure that the operator selected has adequate experience and management capacity. • Management capacity of the utility can be strengthened by contracting a competent operator. Selection of operators for the water utility companies in WSRAP was based mainly on providing the lowest cost for the government. The quality of the operator was an important omission in the bidding requirements. This led to a focus on the short-run financial benefit for the government, rather than on selecting an operator with at least the minimum experience and quality capacities needed to run the water utilities. • A carefully designed M&E system and comprehensive planning are essential for success. The Cartagena Project included relevant and systematic design and implementation of M&E throughout the project period, while the WSRAP and WSSSP suffered from a relatively weak M&E framework and poor M&E implementation. In the case of the WSSSP, the demand-driven selection process resulted in a piecemeal operation, with subprojects financing components or parts of an overall system. A carefully designed M&E system with specified required indicators for the different types of interventions would have been essential in order to enable adequate monitoring of the project outcomes. For small-scale, demand-driven projects, substantial capacity building in addition to careful M&E design is critical for success. Stoyan Tenev Acting Director Financial, Private Sector, and Sustainable Development Evaluation Independent Evaluation Group 1 1. Background and Context Country Context 1.1 Colombia is the third-most populous country in Latin America, after Brazil and Mexico, with a population of 47.6 million. According to the World Bank’s World Development Indicators, Colombia’s population has been growing at 1–2 percent annually since 2004. Historical data on the distribution of the country’s population shows decreasing growth rates for both urban and rural populations. Colombia has an annual precipitation of 3 billion meters in the continental area, making it a water-rich country. Colombia is characterized as an upper-middle-income country, based on its 2013 gross domestic product (GDP) per capita of US$8,030. The country’s GDP has grown significantly, at an average annual rate of 5 percent since 2004 (World Development Indicators, various years). This growth is explained by good performance in sectors such as construction, agriculture, social services, and financial institutions (DANE 2014, first quarter GDP statistics, 2014). Unemployment rates have been falling since 2008, and currently stand at about 10 percent (World Development Indicators, various years). Sector Background 1.2 Water supply and sanitation (WSS) in Colombia have improved in recent decades. Between 1990 and 2010, access to improved water sources increased from 89 percent to 94 percent, and access to improved sanitation increased from 67 percent to 82 percent (WHO/UNICEF 2010). Coverage in rural areas still lags behind. 1.3 The sector was centralized in the 1950s, and was then decentralized from 1976 onward. Private sector participation was introduced in the mid-1990s. Up until the 1950s, the municipalities were responsible for the provision of basic water and sanitation services in Colombia. A centralized system was adopted in the 1950s, and the Municipal Development Institute (INSFOPAL) was established, with the responsibility of managing and maintaining the infrastructure in its member communities. The Departmental Water and Sewerage Companies (ACUAS) were created, consisting of departments, municipalities, and INSFOPAL. In 1976, ACUAS was transformed into the Sanitary Works Companies, which became responsible for the financing, planning, development, and management of public water services in most municipalities (Acuavalle 2007). However, some municipalities remained responsible at the local level, and municipal companies were created. The most notable of these were the public companies in Medellín, which was a municipality-owned, multisector utility created in 1955. At the national level, the water sector was administered by the Ministry of Health (IDB 2005). 1.4 The sector was in crisis in the late 1980s. There was low investment, poor cost recovery, and poor service delivery throughout the country. The government dissolved INFOSPAL in 1987 and handed the responsibility for water service provision back to the municipalities, with some exceptions. The institutional responsibility at the national level was transferred from the Ministry of Health to the Ministry of Economy. This change was more than an administrative one; it introduced a different focus and perspective to the sector—private sector participation. The financing responsibility for the sector was shifted to 2 FINDETER, the Regional Development Financing Institution, which was a second-tier financial intermediary for regional and local infrastructure investment. The water sector was in a transitional stage until 1994, when the National Planning Department was assigned the function of planning and technical assistance to the sector with the emergence of private sector participation. 1.5 The 1991 Colombian Constitution defined a model of increased decentralization for the provision of public services and promoted a shift away from direct provision of services by municipal government, toward one where service is provided by public or private companies. 1.6 According to a World Bank study (World Bank 2004), the Colombian water sector is characterized by a high degree of fragmentation, which has made it difficult to realize economies of scale. There are 1,123 municipalities in Colombia. Eight hundred of these, predominantly rural, do not have a water company. There are water companies (public/private/mixed, municipality-driven or community organized) in 362 of the larger municipalities. To address the water problems in the rural areas, the creation of regional companies has been suggested. This is also part of Colombia’s National Development Plan for 2014–18. Though the plan has not yet reached regional consolidation, the biggest public and private operators have already shown interest in expanding to neighboring municipalities. According to the 2015 Superintendence for Domestic Public Services (SSPD 2015) sector report, 17 percent of the biggest operators reach 70 percent of the regional consumers. Institutional and Regulatory Framework 1.7 The liberal governments of the 1990s pursued a policy of economic modernization. They established a comprehensive new sector policy that aimed at increasing water and sanitation investments through targeted transfers to municipalities and improving service quality and efficiency by promoting private sector participation in the poorest parts of the country, where public utilities were not performing well. Autonomous regulatory agencies were established at the national level to increase cost recovery, and cross-subsidies were established in the form of area-based tariffs, which were intended to benefit the poorest areas. 1.8 The responsibility for regulating water services was vested in two separate institutions at the national level. The Potable Water and Basic Sanitation Regulatory Commission (CRA) defines criteria for efficient service provision and sets the rules for tariff revision, but is not in charge of controlling the application of these rules. This is the responsibility of the SSPD, a multisector regulatory agency. In 2011, the Vice Ministry for Water and Sanitation was moved to the newly created Ministry of Housing, Cities, and Territories. In 2011, the General Revenue-Sharing System was established. This is a tax- sharing system that determines the resource distribution of national government transfers to the municipalities, part of which is allocated to the water sector. 1.9 Law 142 of 1994 defined a new tariff scheme that allowed cost recovery under principles of efficiency, economic and financial sustainability, neutrality, transparency, and integrity. The CRA methodology prompts operators to behave as if they were in a competitive market so as to offer good-quality services with efficient tariffs. The 2004 tariff 3 scheme, submitted by the National Council of Political and Social Economy (CONPES), CONPES number 287, was superseded in 2014 by a new scheme (CONPES 688). The new formula included past and future investments on the basis of regulated capital and recognizes a 10-year net present value of depreciation and includes a return of assets. This formula only applies to the biggest operators (222). Together, these operators attend to 362 municipalities, the equivalent of 80 percent of the urban population. The operators of the smaller municipalities (800) do business under a simpler tariff framework. 1.10 Water tariffs are subsidized. From the supply-side perspective, operators may receive subsidies funded through the national budget to cover the costs of investment when users cannot afford to cover them through the tariff. The government operates with six socioeconomic strata, where stratum 4 is the “neutral” stratum, paying the actual tariff. Strata 1–3 are subsidized by the government. The law stipulates guidelines for how much in subsidies should be allocated to each stratum. Stratum 1 receives 70 percent subsidies, so they pay only 30 percent of the tariff; stratum 2 receives 40 percent subsidies, paying 60 percent of the tariff; and stratum 3 receives 15 percent subsidies, paying 85 percent of the tariff. The actual percentages are not obligatory; each municipality may decide the allocation of subsidies locally. Strata 5 and 6 pay a “solidarity tax.” The criteria to determine what stratum people belong to is set according to house standards per block, and does not take income into account. It is up to each municipality to set the strata. The tariff subsidies are drawn from the General Revenue-Sharing System (SGP), from strata 5 and 6, and from the industrial water users’ tax. 1.11 In an effort to reduce the regulatory risk, private sector participation contracts operate under a “contractual tariff scheme,” as contemplated in the law (Article 87.9, paragraph 1 of Law 142/94). This means that operators would use the fixed tariff specified in their contracts for the life of the contract. These tariffs follow the same principles as the regulated tariff. In some instances, contractual tariffs have been subject to revisions between the operator and the municipality. Operators without contractual tariffs may follow CRA’s regulated tariff scheme. Private Sector Participation in the Water Sector 1.12 The District of Cartagena was one of the first municipalities in Colombia to introduce private sector participation in the water sector, following the 1994 Public Services Law. A concept paper for the Cartagena Project was drafted in 1995 and approved in 1999. The Bank’s participation was considered essential to help consolidate the sector reform and establish much-needed credibility for the mixed-enterprise model in Cartagena. Motivated by the experience in Cartagena, the World Bank Group supported an additional project that encouraged the introduction of private sector participation in the water sector through the Water Sector Reform Assistance Project in 1998. 1.13 The World Bank’s Private Participation in Infrastructure Database shows private investment commitments in WSS of US$1,069 million from 1995 to 2015 through 58 transactions, including 29 classified as concessions, 27 as management and lease contracts, and 2 as greenfield projects. Colombia accounts for only 3 percent of total private sector investment volume in the Region (17 percent by number). According to the SSPD report 4 (SSPD 2015), 66 percent of total operators are public, 22 percent are private, and the remaining 12 percent are mixed. Except for Brazil, there is a downward trend in terms of investment volume and number of projects in the sector, in line with matured Latin America and Caribbean Region peers such as Mexico, Brazil, and Chile. 2. Cartagena Water Supply, Sewerage, and Environmental Management Project (1999-2009) Background and Context 2.1 Cartagena is located on Colombia’s northwestern Caribbean coast, with approximately 1 million permanent inhabitants. It was founded by Spanish colonizers on June 1, 1533, and served a key role in administration and expansion of the Spanish empire. Today, Cartagena’s old town, with its typical colonial architecture, attracts more than 1 million tourists each year and is one of Colombia’s most significant tourist attractions. Cartagena is surrounded by water—the Caribbean Sea to the north, Cartagena Bay to the west, and the Ciénaga de la Virgen Lagoon to the east. 2.2 In December of 1994, after years of inefficiencies in the provision of WSS services, the district shut down the municipal utility and created one of the first public-private partnerships in Colombia, Aguas de Cartagena (ACUACAR). Aguas de Barcelona was selected as the partner of the municipality, and the mixed enterprise was created with initial capital of US$4 million. The District of Cartagena was responsible for capital investments to expand the existing WSS system, while ACUACAR was responsible for capital expenditures to improve it. 2.3 Despite significant progress in improving the efficiency and quality of WSS service provision by 1999,1 Cartagena still faced many challenges, including insufficient water supply coverage and insufficient sewerage services, especially in the poorest areas around Ciénaga, where open sewage canals in the streets were common, and an inadequate domestic 1 At appraisal, ACUACAR had been in operation for three-and-a-half years and it had already achieved significant improvements in operational performance and service quality. From 1996 to 1999, water and sewerage coverage increased from 71 to 80 percent and 61 to 65 percent, respectively; 24-hour continuity of services went from 60 to 70 percent of the customers; water meter installation increased from 77 to 99 percent of the connections, and the number of clients increased from 178,000 to 363,000. 5 wastewater management system was discharging untreated wastewater into the Bay, Ciénaga, and inner-city water courses. 2.4 To address these WSS challenges, the District of Cartagena and ACUACAR developed a water and wastewater master plan and implementation strategy consisting of two projects in two stages: The first, financed by the Inter-American Development Bank for US$24.3 million, entailed the expansion of sewage systems in the southeast and southwest areas. The second, financed by the World Bank for US$85 million, was more comprehensive. It entailed the expansion of both water supply and sewage systems, construction of conveyance systems, treatment plants, and a submarine outfall. 2.5 The Bank’s participation in this project at this critical juncture was considered to be essential to help consolidate the sector reform and establish much-needed credibility for the mixed-enterprise model in Cartagena. Despite the substantial improvements in the late 1990s, the reform process was considered vulnerable to political interference from future mayors, since the implementation process was still in its early stage, and the District owned 51 percent of ACUACAR’s shares. Objectives, Design, and Relevance OBJECTIVES 2.6 The project development objectives (PDOs), as stated in the legal agreement (World Bank 1999b), section 2, were to: (i) improve the water and sewerage services in the territory of the borrower and the sanitary conditions of the borrower’s poorest population; (ii) facilitate the environmental cleanup of water bodies surrounding the territory of the borrower (Cartagena Bay, the Caribbean beaches, and Ciénaga de la Virgen Lake); and (iii) improve the sustainability of water and sewerage services in the borrower’s territory through a private sector participation model. 2.7 The formulation of the PDOs in the project appraisal document (World Bank 1999a, p. 2) is similar, and provides useful specifications: objective (i) was to be achieved through expanding water and sewerage coverage, particularly in the city’s poorest neighborhoods, and objective (iii) was to be achieved by leveraging Bank support to shore up the private sector participation model pioneered by ACUACAR, the city’s mixed-capital water and sanitation utility. 2.8 The primary beneficiaries targeted to benefit from increased sewerage and water supply coverage were 80,000 people located in Cartagena’s poorest neighborhoods, classified among the lowest-income consumer categories in Colombia—that is, strata 1–3 (of 6 strata in total). The targeted beneficiaries increased to 272,700 during project implementation through the inclusion of additional communities. The entire population of Cartagena, which amounted to 750,000 permanent inhabitants, and the 700,000 tourists who visited each year at the time of appraisal, were considered secondary target groups, as they would benefit from the improvements in the reliability of water supply service, and especially from the environmental improvements that wastewater collection, treatment, and safe disposal systems would bring to the Caribbean beaches, the Ciénaga, and Cartagena Bay. 6 COMPONENTS 2.9 The Cartagena Project had eight components: two entailed expansion of the water and sewage systems through upgrading and new construction; four aimed to reduce the impact of wastewater discharge through the construction of the wastewater conveyance system, treatment installations, and submarine outfall and control of wastewater discharge; and two related to activities to mitigate environmental and social impacts and to enhance project management and supervision (see box 1 for project components). 7 Box 1: Project Components, Costs, and Subcomponents Expansion of the water supply system (Appraisal, US$8.62 million; actual, US$13.69 million). This component included the following subcomponents: (i) expansion and improvement of the water production system; (ii) increase of the water coverage in the city; (iii) replacement of primary distribution mains; (iv) mitigation of environmental impact of water treatment sludge; (v) installation of remote control systems; and (vi) carrying out a plan to reduce unaccounted-for water. Expansion of sewerage system in the Ciénaga de la Virgen Drainage Basin (Appraisal, US$30.16 million; actual, US$33.87 million). This component included the following subcomponents: (i) enhancement of conveyance capacity of existing sewage collectors in the southwest, southeast, and central parts of the city that were draining to Ciénaga at the time of project start-up; (ii) expansion of secondary sewerage network in the southwest, southeast, and central parts of the city, as well as the Boquilla area, that were draining to Ciénaga at the time of project start-up; and (iii) construction of new pressure lines and pumping stations; and (iv) construction of new gravity collectors in residential areas. Construction of the main wastewater conveyance system (Appraisal, US$22.81 million; actual, US$29.86 million). The component had the following subcomponents: (i) upgrading of the Paraíso pumping station; (ii) construction of the pipeline from the Paraíso pumping station to the treatment plant site; and (iii) construction of 23.85 kilometers of effluent pipeline from the treatment plant to the submarine outfall at the Caribbean shoreline. Construction of the wastewater treatment installation (Appraisal, US$22.81 million; actual, US$15.36 million). The component included the construction of a preliminary treatment plant to remove floatable materials, grease, oil, sand, and grit. Treatment facilities included six rotary screens to remove rags, floatable material, and large solids, and two vortex-type grit-chambers. Construction of submarine outfall (Appraisal, US$18.11 million; actual, as of October 2011, following the Implementation Completion and Results Report [World Bank 2010], the actual figure had amounted to US$28.2 million, which was still subject to change pending completion of the construction). This component included the construction of the submarine outfall for the safe discharge of the pretreated effluent to the Caribbean Sea near Punta Canoa. Total outfall length was 2,850 meters, and the discharge point (diffuser area) would be submerged at a depth of 20 meters. Industrial wastewater discharge control (Appraisal, US$0.5 million; actual, US$0.23 million). This component addressed the issues related to industrial wastes discharged to the municipal sewerage network and included: (i) a survey to identify key sources of industrial pollution in the city of Cartagena; (ii) establishing a system for regulating the discharge of industrial wastes, either to the sewerage system or to receiving bodies; (iii) establishing a system for auditing the status of industrial wastes discharge; (iv) defining strategies to control small and dispersed sources of industrial pollution discharging to the sewerage networks (gasoline stations and mechanical repair shops); and (v) providing technical assistance in selection and design of pretreatment processes. Environmental and social component (Appraisal, US$2.75 million; actual, US$2.39 million). This component would implement mitigation measures of the project’s environmental and social impacts. The Environmental Management Program included: (i) environmental supervision during construction; (ii) restoration and conservation of the Ciénaga de la Virgen Nature Reserve; (iii) a monitoring program before and after construction of the marine outfall to study what happened to the coliforms and other contaminants discharged through the outfall; and (iv) an environmental institutional-strengthening program. The Social Management Program included: (i) organization and strengthening of the communities of La Boquilla, Punta Canoa, Las Palmeras, El Pozón, and Manzanillo del Mar; (ii) construction, rehabilitation, and equipping of community centers in La Boquilla, Punta Canoa, and Las Palmeras; (iii) supporting in-house basic sanitation in La Boquilla; and (iv) strengthening of the community 8 relations unit of ACUACAR. The loan agreement was amended during implementation to include an additional subcomponent: a social consultation and communications program. Project management, technical assistance, studies, design, and supervision (Appraisal, US$7.6 million; actual, US$13.84 million). The component included: (i) project management; (ii) design and supervision of the water supply system works; (iii) design and supervision of the sewerage system works; (iv) design of the main wastewater conveyance system, treatment installations, and submarine outfall; (v) supervision of the main conveyance system works; and (vi) procurement audits. Source: World Bank 1999a, pp. 7-8, 1999b, schedule 2 RELEVANCE 2.10 Relevance of objectives is rated high. The objectives were highly relevant at entry. The lack of reliable access to water and sanitation and the severe environmental contamination had long been of concern because of the health hazard they represented, in addition to being a critical constraint to the city’s potential for economic growth. The project was strongly aligned with the government’s national priorities in WSS, which started during the late 1990s and continue to the present. 2.11 The objectives were—and remain—relevant. The Colombia Country Assistance Strategy (CAS) of 1997 (World Bank 1997) identified the poor and deteriorated state of the country’s infrastructure as a critical constraint to economic growth, and the government’s strategy emphasized private sector participation to expand WSS service and improve the wastewater treatment infrastructure. The project contributed to the CAS objectives of improving infrastructure services to contribute to sustainable development. The subsequent CASs reaffirmed the relevance of the sector and the selection of Cartagena as a case to be replicated. The 2012–16 Country Partnership Strategy (World Bank 2011a) further endorsed the water sector as contributing to one of its three priority areas: “sustainable growth with enhanced climate change resilience” under “improved sustainable urban development.” 2.12 Relevance of design is rated high. The PDOs are clear and specific, and the components and activities were necessary and sufficient to achieve the objectives. The project’s logical chain is convincing, in that it displays a clear cause-and-effect relationship between the outputs, expected outcomes, and objectives. The implicit goal of the project was to facilitate improvements in health and environmental pollution. Private sector participation was expected to effectively operate and maintain the WSS utilities, and the environmental and social efforts would enable a cleaner environment. With decentralization, the lack of municipal capacity to operate and manage water utilities became apparent, and improvement in efficiency was necessary. Based on recent studies and reports (World Bank, 2009), it was assumed that utilities would run better and more efficiently with the inclusion of private operators. 2.13 Monitoring and Evaluation (M&E) Design: The project appraisal document (PAD) was formulated in 1999 and there have subsequently been changes in the Bank’s M&E guidelines, and the Implementation Status and Results Reports have been introduced. The key performance indicators for the PDOs presented in the PAD (World Bank 1999a, annex 1, p. 29) are (i) environmental indicators that measure sewerage coverage and volume of sewage treated; (ii) “utility-based” indicators that measure the operational and financial 9 efficiency of ACUACAR; and (iii) financial indicators of the District of Cartagena. The key indicators were appropriate to measure the degree of achievement of the PDO. 2.14 The design of the project was sound. Its components and activities were necessary and sufficient to achieve the PDO, and the results framework (in the form of a log frame) in the PAD (annex 1, p. 29) was adequate and covered the main elements of the project. The log frame also included relevant M&E measures for each of its parts. Although there was M&E under project management, technical assistance, studies, design, and supervision component, there is no clear indication of budget allocation for this task. Implementation 2.15 All project activities had been successfully implemented by 2013, although many of them were severely delayed. The project was approved in July 1999 and closed in June 2009, four-and-a-half years later than originally scheduled. 2.16 Several factors contributed to the delays: (1) strong opposition to the submarine outfall construction from special interest groups and local communities in the north zone contributed to a lengthy authorization process for the environmental license (27 months after loan effectiveness) through several appeals; (2) an Inspection Panel Investigation (lasting 20 months); and (3) several unsuccessful bidding processes for the construction contracts that needed rebidding (partly due to price increases between preparation and implementation). 2.17 Three amendments to the loan agreement were issued during implementation. The first amendment (2003) was to finance water supply works to poor communities in the north zone of Cartagena and sewerage works to complete the wastewater collection system in the southeastern zone. These two works were to be financed by the District, but because of the economic crisis that hit Colombia in 2003, the District could not pay for them. The second amendment (2006) involved converting the financial clause of a variable-spread loan to a fixed-spread loan to provide the borrower with access to the Bank’s financial products to manage the volatility of interest and currency exchange rates. The last amendment (2009) was to increase the Bank’s financing percentage of the category of works from 75 to 100 percent to allow full disbursement of the loan and to reallocate funds between categories to adjust the allocation to the project’s actual expenditures. 2.18 The submarine outfall has been fully operational since January 2013. ACUACAR, under the regulatory supervision of CARDIQUE (the Regional Environmental Authority in Cartagena), regularly undertakes extensive water-quality monitoring around the outfall discharge area. The monitoring program follows international standards and indicates that outside the prescribed mixing zone, there is no discernible impact on seawater quality. Outside of the 500-meter mixing zone around the outfall diffuser, biochemical oxygen demand and suspended solid concentrations are equal to ambient seawater quality levels, and total coliform levels are less than 5 NMP/100 ml2, which is an extremely low level, and considered suitable for human contact. 2 Most Probable Number (MPN) of viable cells in 100 mL of sample. 10 Safeguards and Fiduciary Compliance 2.19 Environmental Safeguards: The project was classified as a category A project because of the potential significant adverse environmental impacts during construction and operation if these risks were not properly mitigated. At the same time, the project had potentially significant beneficial environmental and health impacts, because it provided sewerage collection and treatment in the City of Cartagena. A comprehensive environmental assessment and an environmental management plan were developed during preparation, reviewed by an international panel of experts, and shared with the public, following Bank policies. According to the Implementation Completion and Results Report (ICR), the environmental management plan was implemented satisfactorily, with close supervision from Bank staff and CARDIQUE, and it included measures to mitigate environmental impacts during implementation (World Bank 2010, p. 10, para. 44). A comprehensive program for monitoring the quality of seawater and the seabed before and after construction of the outfall and the Ciénaga de la Virgen restoration and conservation program led to the designation of Ciénaga as a protected area. Recent monitoring shows that the environmental cleanup of Cartagena’s surrounding waterways has been successful. 2.20 Social Safeguards: The social assessment conducted during project preparation indicated that the project would benefit the poorest communities in Cartagena and that no resettlement or other negative impact would result. The Mitigation and Community Development Program focused on strengthening community organization and consolidating urban and rural neighborhoods. Although resettlement activities were not expected during project preparation, the involuntary resettlement policy was triggered because 37 families informally settled on District property on the planned route of the onshore pressure pipeline. CORVIVIENDA, a housing program run by the District, in collaboration with ACUACAR and Bank social specialists, prepared and implemented a resettlement plan that included resettlement of 24 eligible families and economic compensation for the remaining families. No other safeguard policies were triggered. Achievement of the Objectives OBJECTIVE 1 2.21 Objective 1 was to “improve the water and sewerage services in the territory of the borrower and the sanitary conditions of the borrower’s poorest population” (World Bank 1999a. p. 2). The IEG mission found that the World Bank support contributed to improve coverage, quality, and reliability of water and sewage services in the borrower’s poorest populations. At project completion, all works planned under component A to expand the water supply system were completed. From a baseline of 80 percent, there was universal water service coverage throughout the city at project completion. The primary targets for both water supply and sewerage improvements were 80,000 people located in Cartagena’s poorest neighborhoods. This target was surpassed—272,700 people received services from the project through the inclusion of additional communities. At closure, almost all customers (99.5 percent) received 24-hour water supply service, compared with 95 percent prior to the project. The percentage of the city’s population with sewerage connections was four points 11 short of the target at project closure and increased to 93.6 percent in 2014, 3.6 points higher than the target of 90 percent (table 2.1). Outputs 2.22 Project achievements in water supply and sewerage coverage are presented in table 1, which shows the figures for 1994 (before project start-up), 2009 (project closure), and 2014 (most recent figures). The table shows that ACUACAR maintained and further improved the accomplishments following project completion. Table 1: Cartagena Water Supply and Sewerage Coverage Baseline, Completion, Target, Indicator 1994 1999 2009 2004 2014 Urban population 656,632 778,915 899,200 947,606 Water supply coverage (%) 65.0 80 100 89 100,0 Sewerage coverage (%) 60.0 69 86 90 93.6 Water supply connections (no.) 65,000 121,965 188,355 na 217,045 Sewerage connections (no.) 60,000 95,871 161,092 na 201,880 Water supply network na na 489.4 1,418.2 1,634.3 (kilometers) Sewage supply network na na 500 na 1,118 (kilometers) Water continuity (24 hours) (%) 75 95 99.5 98 99.5 Unaccounted-for water (%) 48 44 42.9 34 38.5 Source: National Department of Statistics. Note: SIU: Unique system of information of the Superintendence of Residential Public Services. na = Not available. 2.23 The project retained a strong poverty focus throughout implementation and after completion. Furthermore, analysis by stratum shows that most of the population reached (66.9 percent) belonged to the lowest socioeconomic strata,3 levels 1 and 2 (of 6 in total), categorized as the very poor (table 2). Table 2: Target Population—Percentage by Stratum 2009 2014 1999 All Stratum 1 Stratum 2 Stratum 3 All Population 750,000 899,200 947,606 Socioeconomic strata (%) 85 37.9 29.0 18.0 84.9 Source: National Department of Statistics. 3 Colombia uses a socioeconomic stratification system that was implemented in the 1980s to classify urban populations into different strata with similar economic characteristics. The system classifies areas on a scale from 1 to 6, with 1 as the lowest income area and 6 as the highest. In 1994 this stratification policy was made into law. The system is organized so that the people living in the upper layers (strata 5 and 6) pay more for services such as electricity, water, and sewage than groups in the lower strata. 12 2.24 The entire population of Cartagena was considered to be a secondary target group, as they would benefit from the improvements in the reliability of water supply service, and especially from the environmental improvements that wastewater collection, treatment, and safe disposal systems would bring to the Caribbean beaches, the Ciénaga, and the Cartagena Bay. Outcomes 2.25 The successful achievements of all outputs, as demonstrated above, led to the achievement of the intended outcomes of expansion of improved water and sewerage services for the borrower’s poorest population. 2.26 Achievement of objective 1 is rated high, because the project not only achieved but surpassed the targets, the coverage has expanded further, and the poverty focus was maintained after project closure. OBJECTIVE 2 2.27 Objective 2 was to “facilitate the environmental cleanup of water bodies surrounding the territory of the borrower (Cartagena Bay, the Caribbean beaches, and Ciénaga de la Virgen Lake)” (World Bank 1999a, p. 2). Achievement of objective 2 is rated substantial. The objective was achieved only after project completion because the system could not be put into operation until the construction of the submarine outlet was completed. During construction of the submarine outfall, in December of 2010, there was an accident. The contractor tried to install the 4-kilometer long and 1.8 meter in diameter submarine outlet in one piece. The submarine outfall was floating between two boats, but there were bad weather conditions, and the workers had to release the outfall into the sea for the boats not to tilt. This was a serious blow for the project, especially since all the other construction work (the Paraíso pumping station, the 23.85-kilometer long conveyance system, and the 1.6 m3/s4 wastewater treatment plant) had been constructed and were scheduled to be put into operation upon installation of the submarine outfall. ACUACAR was able to collect 2 kilometers of the system and raised its own funding to reconstruct and install the submarine outfall, using the same technology envisioned under the project. ACUACAR sought technical support from its parent company Aguas de Barcelona, two external consultants, and Hazen and Sawyer. The outfall was completed in September 2012 and became fully operational in January 2013. Outputs 2.28 All works planned under component B to expand sewerage systems in the Ciénaga Basin were completed. The works under component D, construction of wastewater treatment installations, and works under component F, control of industrial wastewater treatment installations, were all completed and waiting for the installation of the submarine outfall to be put into operation. Outcomes 4 Cubic meter per second. 13 2.29 The following tables and figures demonstrate the success of the environmental cleanup of Cartagena’s waterways. Table 3 shows that there is no significant contamination compared with the baselines in the area of influence of the submarine outlet except in microbiological bodies, which are represented by high concentrations of fecal and total coliforms during transition5 and the rainy season. However, this is mostly the product of the use of samples from the point of discharge of the submarine outfall. Excluding this point, average concentrations of coliforms drop down significantly (see last three columns of table 3). It is also important to guarantee that water is safe for recreational use of the beaches in the area of influence. Table 4 uses the INVEMAR water quality compliance methodology to assess the microbiological quality of water along the beaches of Punta Canoa. It reveals that the 88 samples taken from 4 locations showed water quality in compliance at 80 percent or above (compliance was 90 percent or above in 3 of the 4 locations), According to the methodology of Marin and others, 2001, at these degrees of compliance, the beaches were not considered contaminated and were safe for recreational use (last column of table 4). Table 3: Water Quality in the Area of Effluence Operation without point of Baseline, Operation, discharge, 2003 2014–15 2014–15 Variable Dry Trans. Rain Dry Trans. Rain Dry Trans. Rain Fecal coliforms <2 <2 <2 30 3,112 390 28 13 69 NMP/100mL 1/ Total coliforms 41 651 124 43 3,120 391 41 15 69 NMP/100mL 1/ Enterococcus na na na 14 33 17 12 28 15 UFC/100mL 2/ Source: ACUACAR. Note: Figures were collected by ACUACAR, but were then analyzed and validated by CARDIQUE through technical note 447 in 2015 (Cardique, 2015). Trans. = transition; rain = rainy season. na = Data not collected. Note 1/ NMP/100mL = Most Probable Number (MPN) of viable cells in 100 mL of sample Note 2/ UFC/100mL = Colony Forming Units per 100 mL of sample Table 4: Degree of Contamination of Water along the Beaches of Punta Canoa Number In of Not in In compliance Stations samples compliance compliance (%) Result E12 22 2 20 90.90 Not contaminated E15 22 4 18 81.80 Not contaminated E16 22 2 20 90.90 Not contaminated E17 22 2 20 90.90 Not contaminated Sources: ACUACAR Note: Figures were collected by ACUACAR, but were then analyzed and validated by CARDIQUE through technical note 447 in 2015(Caridique, 2015). 5 Transition is a period in between the dry and rainy seasons. 14 2.30 Similarly, the Ciénaga de la Virgen Lagoon results summarized in table 5 suggest improvements after the start-up of the submarine outlet in 2013, particularly in the presence of coliforms in the transition and rainy seasons. For example, the average presence of fecal coliforms decreased from a baseline of 651 to 41 (NMP/100mL) during transition season, and from a baseline of 4,796 to 827 (NMP/100mL) in the rainy season. The cleanup of the Ciénaga de la Virgen can be attributed to both the World Bank project, which stopped the sewage disposal into the lagoon, and to a Dutch project, which started operations in 2000, for US$24 million, and opened the lagoon up to the west to bring in seawater. Therefore, baseline results in table 5 are measured three years after the Dutch project started operation. Another indicator of quality of water (ICAM) shows improvements in the Ciénaga de la Virgen Bay before and after the start-up of the submarine outlet. Table 5: Water Quality in the Ciénaga de la Virgen Lagoon Baseline, Current, 2003 2014–15 Variable Dry Transition Rain Dry Transition Rain Phosphates (mg/L) 1.93 0.10 0.15 0.10 0.06 0.08 1/ Fecal coliforms 78 651 4,796 626 41 827 (NMP/100mL) 2/ Total coliforms 729 2,837 8,951 626 49 827 (NMP/100mL) 2/ Source: ACUACAR. Note: Figures were collected by ACUACAR, but were then analyzed and validated by CARDIQUE through its technical note 447 in 2015(Cardique, 2015). Note 1/ mg/L = Milligrams per liter Note 2/ NMP/100mL = Most Probable Number (MPN) of viable cells in 100 mL of sample. 2.31 Achievement of objective 2 is rated substantial. In spite of the additional funding required to finish the submarine outfall, the project reached all targets and has demonstrated results in terms of a cleaner environment in Cartagena’s surrounding water bodies. OBJECTIVE 3 2.32 Objective 3 was to “improve the sustainability of water and sewerage services in the borrower’s territory through a private sector participation model” (World Bank 1999a, p. 2). Sustainability was measured on the basis of operational and financial efficiency. 2.33 Operational Sustainability: The managerial and operational capacities of water supply and sewage services were strengthened, and their financial management was improved. The infrastructure that was funded contributed to increased water production and distribution and reduction in unaccounted-for water. ACUACAR has shown steady improvements in its operations, mostly brought about by significant emphasis on water management due to project intervention before closure, and by ACUACAR itself after closure. Most notable are improvements in water and sewage coverage, with improved continuity of services and a significant reduction of water losses. This was largely attributed 15 to a decrease in water consumption per household as a result of the projects’ metering programs and public awareness campaigns about water conservation (table 6, figure 1). Table 6: Operational Indicators Baseline, Completion, Target, As of Indicator 1994 1999 2009 2004 2014 Metered connection (%) 45 93 100 98 100 Capacity, water supply treatment 165,000 183,906 196,210 na 270,000 (m3/day) Volume of water produced na 230,000 246,000 289,000 296,000 (m3/day) Volume of treated water distributed na 194,000 195,000 249,000 283,333 (m3/day) Response time to breaks (days) na 1.3 0.2 1.2 0.6 Water continuity (24 hours) 75.0 95.0 99.5 98.0 99.5 Staff on payroll/1,000 connections 2.8 2.5 2.8 2.22 (number) Bill collection rate (%) 64.0 na na na 89.5 Unaccounted-for water (%) 48.0 44.0 42.9 34.0 38.5 Source: ACUACAR. Na. = not available m3/day = Cubic meter per day. 16 Figure 1: Water Produced and Water Consumed per Household (cubic meters) Sources: SIU: Unique system of information of the Superintendency of Residential Public Services 2.34 Financial Sustainability: At project closure, the operating margin, net profit margin, return on equity, and uncollectable fees all met their targets, and the remaining three indicators (debt service coverage, debt-to-assets, and current ratio) were close to meeting theirs. As of 2014, ACUACAR had shown further improvements and a stronger financial position (table 7). Table 7: Financial Sustainability Outcome Indicators Year 1999 2009 2004 2014 Outcome indicator Baseline Completion Target Actual Operating margin (%) 15.3 21 12.6 22 Net profit margin (%) 7.1 8.9 7.5 12 Return on equity (%) 17.3 22 14.4 30 Current ratio (times) 1.26 0.98 1.14 0.97 Debt service coverage (times) 1.46 3.4 3.99 4.43 Debt/assets (%) 68.9 65.9 45.6 59.9 Uncollectable, 18 months (%) 8.0 6.0 8.0 7.8 Source: SIU: Unique System of information of the Superintendence of Residential Public Services. Information from the contract studies. Note: Current ratio = current assets/total liabilities. 2.35 In addition, Cartagena was able to maintain the private sector participation model over seven political administrations. The contract was recently expanded for another 13 17 years, making it the longest-running water utility contract in Colombia, in operation for a total of 39 years. According to a sectoral report (SSPD, DNP 2015) the average tariff for a sample of 222 Colombian water companies is about 64.6 pesos per month, 12 percent lower than in Cartagena. This might indicate that Cartagena prices are less favorable to the consumer. However, the overall average tariffs (taking into account inflation) have been kept the same since 1994. The project contributed to an improvement of ACUACAR’s operations and financial efficiency by enabling the company to consolidate the sector reform and establish credibility for Cartagena’s mixed-enterprise model. 2.36 Achievement of objective 3 is rated high, based on the figures presented above. ACUACAR’s private sector participation model has not only demonstrated that it is sustainable and strong, it has also achieved considerable improvements in the quality and efficiency of services. Efficiency 2.37 The implementation of components C, D, and E, which constitute the wastewater management system and represent about 50 percent of project cost, faced significant delays. These were caused by an extended holdup in getting the authorization of an environmental license for these works, an unsuccessful bidding process that required rebidding, and an Inspection Panel investigation followed by the implementation of a Bank management action plan to respond to the Inspection Panel report (World Bank, 2005) . The delays amounted to four-and-a-half years. The Bank provided two extensions to complete the works and meet the PDO: a two-year extension in June 2005, when about 40 percent of the loan amount had been disbursed, and another two-year extension in May 2007, when 70 percent of the loan was disbursed. The project closed on June 28, 2009, as scheduled in the second loan extension, without being fully executed because of delays in completing two contracts (the land conveyance pipeline and the submarine outfall). Another extension was not considered necessary, because the loan was fully disbursed. The borrower and the Bank agreed to complete all the project works after this with counterpart funds. ACUACAR completed the construction and installation of the conveyance pipeline and the submarine outfall with its own funding, and the wastewater network and submarine outfall have been fully operational since January 2013. 2.38 The ex post economic rate of return was calculated for a 15-year period and was estimated at 16.9 percent, just above the expected rate of 16 percent. The benefits for water and sewage components are estimated based on costs avoided by project beneficiaries. The project benefits for the water component included the elimination of water rationing and the expansion of water service. For the sewage component, the benefits are associated with the elimination of the alternative on-site sanitation solutions. Of these, the greatest benefit was in rationing and expansion of water service, since aspects of the sewage component had yet to materialize. Five years after project completion, the intended benefits at appraisal materialized as a result of the completion of the submarine outfall and the construction and installation of the conveyance pipeline. The project’s components related to all three project objectives have had benefits greater than expected at appraisal. A number of intangible benefits—better governance and accountability and ACUACAR’s financial performance— surpassed targets at completion and have continued to improve over the years. The project 18 brought efficiency gains for ACUACAR in operation, in the commercial system, and in water usage. Water losses were reduced, consumption per connection decreased, the customer database improved, and revenues increased. In addition, ACUACAR met all requirements in the World Bank loan agreement. 2.39 Project efficiency was rated modest in the ICRR (World Bank 2010) because of the noncompletion of the submarine outlet and the significant implementation delays. The submarine outlet has now been completed and is working well, contributing to the full achievement of objective 2. Although the implementation delays were significant, 4.5 years in total, they were partly caused inadequate consultations with stakeholders at the outset and by factors outside the control of the Bank and the borrower that could not have been foreseen at appraisal. Furthermore, as demonstrated above, ACUACAR is working efficiently, and the model has proven sustainable over time. Efficiency is thus rated substantial. Ratings OUTCOME 2.40 Outcome is rated satisfactory. All three objectives were achieved and/or surpassed, and achievement of one objective was rated substantial, and two were rated high. Furthermore, relevance of objectives and relevance of design were also both rated high, because the objectives were well in line with national and Bank priorities, both at entry and at closure, as well as at present, and the project’s theory of change was clear, logical, and convincing. Finally, efficiency is rated substantial, because the implementation delays are considered to be caused partly by matters outside the control of the borrower and the Bank and could not have been foreseen at appraisal, and because of ACUACAR’s demonstrated ability to operate efficiently, delivering high-quality services, and the demonstrated sustainability of the mixed-enterprise private sector participation model they are using. RISK TO DEVELOPMENT OUTCOME 2.41 Risk to development outcome is rated low, because the project’s achievements have been maintained and improved five years after project closure. 2.42 Technical Sustainability: This refers to the risk that the investments will not be sustainable because the technical choice was inappropriate or the systems are not properly maintained. This risk is rated low, since the water and sewage networks and wastewater management system are proven technologies that have been operating well for several years. The submarine outfall is operating well, with a carefully designed technology. 2.43 Financial Viability: This refers to the risk that ACUACAR will not be able to generate and obtain the financial resources to operate and maintain the facilities constructed under this project and build new ones to continue providing high-quality service. This risk is rated low, because ACUACAR has proven to be a financially viable enterprise. After the accident with the submarine outfall, ACUACAR proved to be a financially strong enterprise. They took it upon themselves both to finance the new outfall and to complete the works. 19 2.44 Government Commitment: This refers to the risk that the District will not support the institutional private sector participation model and that political objections will arise. This risk is rated low, because ACUACAR has now survived seven different political administrations. Furthermore, ACUACAR’s contract was recently extended for another 13 years, to a total of 39 years (thus far), and is the longest-running water utility contract in Colombia’s history. BANK PERFORMANCE 2.45 Quality-at-entry is rated satisfactory. The project was very well prepared over a period of four years. A concept note was written in 1995, and several preparatory studies had been undertaken prior to project appraisal and approval in 1999. These covered engineering, institutional, tariff, and demand-side issues in depth. The PAD (World Bank 1999a) reflects this in its thoroughness. The project’s objectives were consistent with both national development priorities and the Bank’s CAS. Crucial features, such as technical, financial, economic, environmental, social, institutional, and fiduciary aspects were adequately taken into account. Lessons learned from earlier projects in Colombia were considered and incorporated into the project design. Furthermore, the Bank ensured that the project had a clear poverty focus and that ACUACAR’s efficiency gains benefited the poor. The M&E framework was detailed with relevant indicators, paving the way for the technically sound, comprehensive M&E system now executed by ACUACAR. 2.46 However, the Inspection Panel case presented by poor communities in the north zone of Cartagena illuminated the need for a more comprehensive social analysis and communication strategy to dissipate doubts and create a better understanding of the project. The risks posed by vested interests and delays in environmental license authorization were not identified at approval, which led to significant delays during project implementation and a number actions that delayed the project. 2.47 Supervision is rated satisfactory. A single task manager oversaw the entire operation from project preparation to closing. This ensured continuity of dialogue and uninterrupted supervision. The task team maintained continuous monitoring of the project and policy dialogue at a high level with national authorities. Sufficient budget and staff resources were allocated, and the project was adequately supervised. The Bank team’s management of the Inspection Panel investigation was adequate, and the subsequent action plan was approved by the Bank’s Board and appropriately executed by the task team. However, there were some delays due to procurement complications, because initial cost estimates of two of the four contracts were too low, which required subsequent revisions that demanded greater oversight by staff. 2.48 Overall Bank performance is rated satisfactory. BORROWER PERFORMANCE 2.49 Government performance is rated satisfactory. There was strong commitment from the government during preparation. The Department of National Planning and the Ministry of Finance remained strongly committed to the project. This ensured that the District honored its commitment during the last loan amendment to complete the works with counterpart 20 funding. However, local commitment was variable during implementation because of successive changes in administration. For example, financing for the north zone water and sewerage project as indicated in the loan agreement was withdrawn, which decreased support to the urban rehabilitation and community development programs. 2.50 Implementing agency performance is rated satisfactory. The District of Cartagena was the borrower, and the project was implemented by ACUACAR. This public-private entity demonstrated commitment to fulfilling the PDOs and provided adequate internal staff and resources to ensure success, despite the numerous challenges experienced during implementation. ACUACAR complied with all Bank loan covenants and progress reports and provided technical input and support during the Inspection Panel investigation. ACUACAR also provided additional financial and human resources to implement the subsequent Management Action Plan. Furthermore, ACUACAR took the responsibility for the construction and financing of a new outfall to complete the project, demonstrating managerial proficiency and financial strength. 2.51 Overall borrower performance is rated satisfactory. MONITORING AND EVALUATION 2.52 M&E Design: Despite the lack of a dedicated M&E subcomponent in the project, monitoring of implementation and results was detailed and appropriate. The 29 indicators in the PAD’s results framework were specific in quantity, quality, and time, except for two of the outcome indicators associated with the first PDO: (i) evacuation of wastewater in open channels on the street and (ii) discharge of wastewater to the Ciénaga through the storm drainage system. 2.53 M&E Implementation: The Bank, ACUACAR, and CARDIQUE collected baseline data and tracked performance indicators during the implementation of the project. ACUACAR submitted periodic reports to the Bank, including project activities and calculation of performance indicators. ACUACAR and CARDIQUE were implementing a water quality monitoring program along the beaches of Cartagena, the Bay, and the Ciénaga to measure the before-and-after effects for the project and compliance with Colombian water quality standards. To simplify reporting of the PDO indicators, the project team reported on three main indicators in the implementation status reports (ICR, 2010): (i) water coverage; (ii) sewerage coverage; and (iii) continuity of water supply services. Although these three indicators were appropriate, they were focused on the first PDO (improved WSS services), and did not address the second and third PDOs (environmental cleanup and sustainability). However, the IEG mission found well-supported evidence of achievement of these PDOs as well. 2.54 M&E Utilization: M&E information was used to provide feedback to ACUACAR, the District, and other agencies on issues pertaining to project implementation and sector performance. The Bank team also used the M&E information to report progress and support decision making. Furthermore, service quality indicators played a key role in enhancing ACUACAR’s internal transparency and responsiveness. 21 2.55 The IEG mission found that ACUACAR’s current monitoring of performance and results, including the environmental situation in the surrounding waterways, constituted good practice. Detailed information on all relevant indicators (such as quality of water, continuity, bill collection rate, nonrevenue water, and environmental measurements) were measured and updated continuously and were readily available. M&E is rated substantial. 3. Water Sector Reform Assistance Project (2001–10) Background and Context 3.1 In 1998, the Corporate Modernization Program (PME) was established in the Ministry of Economic Development for the introduction of private sector participation in the water sector. The PME’s objective was to provide technical assistance to decentralized entities, such as municipalities and regions, to partner with the private sector. The PME focused on the Caribbean coastal region, where coverage and quality in the water sector were poor, and institutional weaknesses evident. The PME provided financial incentives for improved performance by the operators, while the national government provided most of the investment financing. 3.2 The World Bank–financed Water Sector Reform Assistance Project (WSRAP) was designed to support the PME and concentrated on the departments of the Caribbean coast. The project design was partly motivated by the positive experience from the two previous private operator contracts, signed for Cartagena in 1995 and for Barranquilla in 1996, although this was considered a pilot project in introducing specialized operators to small and medium-size cities. Objectives, Design, and Relevance 3.3 The loan agreement (World Bank 2001b, schedule 2, p. 29) states that the project had four objectives: • The first objective is “to support water sector reform in the borrower’s territory by facilitating an increased participation of the private sector in the management and operation of water utilities, with the purpose of: (i) creating and maintaining an appropriate environment for improving the efficiency and sustainability of such water utilities; and (ii) providing participating municipalities, which choose to participate in the project, with financial support to ensure the viability of their water utilities.” • The second objective is “to expand the coverage of water supply and sewerage services in participating municipalities.” • The third is “to facilitate the access to water and sewerage services to the poor population of low-income areas in such participating municipalities.” • The fourth is (i) “to improve environmental protection practices in the borrower’s territory; and (ii) to define rural water and sanitation policies and to develop adequate 22 methods for increasing coverage, both in connection with water and sewerage services in the borrower’s rural sector.” 3.4 The objectives as stated in the PAD (World Bank 2001a, annex 1, p. 38) were similar to the formulation in the loan agreement, but lacked the fourth set of objectives (improvement of environmental protection practices and definition of rural water and sanitation policies). This assessment is made against the objectives described in the loan agreement. 3.5 The objectives were to be achieved by providing: (i) technical assistance for incorporating private sector participation in the management and operation of the water and sewerage services in the utilities of about 3 medium-size cities or regional associations of municipalities with populations of up to about 300,000 and in about 15 small municipalities with populations up to 12,000 and (ii) financial support to the utilities that had successfully incorporated the private sector and were to be directed toward benefiting the poor, as well as ensuring financial viability of the utilities. 3.6 The project comprised four components related to (i) investment in infrastructure; (ii) strengthening environmental management capacity with a focus on wastewater; (iii) developing rural water and sanitation policies; and (iv) strengthening project management capacity and training (box 2). 23 Box 2: Components and Costs 1. .Investments in water supply and sanitation works in medium-size cities and in small municipalities (Appraisal, US$61.8 million; actual, US$75.55 million). This component had five subcomponents and included: (a) loans to partially finance the public-sector funding requirements of the water and sanitation infrastructure in 2 or 3 medium-size cities and 15 small municipalities in the Caribbean region and the execution of respective works. The participation of cities and municipalities was demand-driven and based on specified eligibility criteria; (b) execution of works financed by the private sector and by Law 60 resources (Law 60 stipulates that 60 percent of decentralized social expenditures, including water, be distributed to municipalities on the basis of the number of people with unsatisfied basic needs) in medium-size cities and in small municipalities. As part of the private sector participation transaction, participating municipalities would be asked to commit a portion of their Law 60 resources to contribute to the works. 2. Environmental management capacity strengthening of the sector, with a focus on wastewater management (Appraisal, US$0.8 million; actual, US$0.38 million). This component had eight subcomponents: (i) providing advice on modifying and strengthening key environmental regulations, especially the existing water quality and effluent discharge regulations; (ii) establishing sectorwide criteria for the environmental management of water and sanitation projects (siting, construction, and operation) and completing the existing sector technical specifications; (iii) defining environmental requirements to be included in bidding documents and private sector operation contracts; (iv) defining methodologies and programs for environmental audits of water and sanitation facilities; (v) strengthening the environmental management capabilities of the sector as a whole, especially the Directorate of Water and Sanitation in the Ministry of Economic Development; (vi) providing advice on streamlining the environmental licensing procedures for water and sanitation projects in regional environmental agencies; (vii) sponsoring training programs on the environment for water and sanitation system operators; and (viii) defining public participation and community consultation criteria and procedures for water and sanitation projects. 3. Development of a rural water and sanitation policy (Appraisal, US$0.9 million; actual, US$0.21 million). This component comprised technical assistance to support the development of the rural water and sanitation sector policy, strategy, and methods for increasing water and sanitation coverage in the rural sector. 4. Project management and training (Appraisal, US$6.5 million; actual, US$5.4 million). This component encompassed technical assistance to carry out the processes of incorporating the private sector in the management and operation of the water and sanitation utilities in medium-size cities and technical assistance to carry out the processes of incorporating the private sector in management and operation of the water and sanitation utilities in about 15 small municipalities. The project did not have a dedicated subcomponent for M&E. Source: World Bank 2001a, pp. 52-54, 2001b, schedule 2. 3.7 The key design element of the project was the requirement to utilize private operators in order to access loan funds. Although the project design built on the successful introduction of private operators in Cartagena and Barranquilla, introduction of private operators in small and medium-size cities was a much riskier undertaking. The municipalities might not want to participate in the program, and the private operators might not find it desirable to work in poor, small and medium-size cities in the Caribbean region, which had historically suffered from poor governance. The project design ensured that excessive risk could not be shifted to 24 the private operator until the deteriorated systems had been improved to the point where the operating revenue streams had stabilized at a sustainable level. The project included key design elements such as (i) public financing to rehabilitate the run-down systems in targeted municipalities; (ii) the use of domestic, rather than international, private operators in order to build a national pool of small and medium-size private operators; and (iii) continuous technical assistance from the national government’s project implementation unit to the small- scale private operators. 3.8 The primary target group was 700,000 people in low-income communities in Colombia’s Caribbean coastal departments. These populations were typically assigned to the three lowest socioeconomic strata in Colombia’s six-point classification system. The strata definition for the primary target group was not revised during implementation, but the size of the group increased to approximately 1.6 million because of the participation of a much higher than expected number of medium-size municipalities. 3.9 The project had a flexible design, which permitted a larger number of contracts and investments to be carried out for the medium-size cities than originally planned, because of the lower demand than anticipated from smaller municipalities. With regard to the subprojects under the program, of the total cost of US$76 million, 78 percent (US$60 million) was funded equally through the International Bank for Reconstruction and Development (IBRD) (US$30 million) and the government of Colombia (US$30 million), and the remaining 22 percent (US$16 million) was funded by the private sector. RELEVANCE OF OBJECTIVES 3.10 The relevance of objectives is rated high. The project objectives were—and remain— relevant to both national and Bank strategies. Expansion and improved efficiency of the water sector were high on the Colombian government’s agenda throughout the project period, and they still are. The promotion of private sector participation in public services management had already been established in the national constitution of 1991, and Law 142 of 1994 enabled the contracting of private enterprises and promoted competition among providers of WSS services. The 1997 Bank CAS for Colombia referred to the national priorities regarding private sector participation: “A key in the government’s strategy to improve the quantity and quality of infrastructure has been encouraging and facilitating private sector participation in infrastructure—to increase efficiency, mobilize additional resources, reduce risks assumed by the public sector, obtain the benefits of competition, and to reallocate public resources to the social sectors” (World Bank 1997, p. 14, para. 29). 3.11 The project played an instrumental role in strengthening the government’s PME and helped lay the groundwork for the national policy on the departmental water plan program. Private sector participation was further strengthened through Law 1058 of 2012, which allowed public-private cooperation in infrastructure projects, including those in the water and sewage sector. The current Colombian National Development Plan (2014–18) provides additional regulation for private and public participation in the water sector. More recently, the central government has delegated the management of programs promoting private sector contracting in the water and sanitation sector, such as the PME, to the departmental water plans. 25 3.12 At the time of appraisal, the 1997 CAS for Colombia (World Bank 1997) identified expansion of “coverage of water supply, sewerage and sewage treatment, and solid waste services” as one of six areas that needed “continuous effort” (p. 14, para. 29). Both the subsequent 2002 and 2008 CASs continued to support infrastructure under the sustained equitable growth pillar, particularly clean water supply and sewerage and wastewater treatment, especially for the poorer strata of the population. The most recent Country Partnership Strategy (2012–16; World Bank 2011a) focuses on sustainable growth, with enhanced climate change resilience as one of its three pillars. It specifically identifies some of the remaining challenges as expanding water and sanitation access in rural areas and wastewater management. The third pillar of the strategy, “inclusive growth with enhanced productivity,” includes a focus on public-private partnerships in infrastructure sectors. 3.13 The project objectives are consistent with the emphasis and goals of supporting rural and small cities’ water supply, with a special focus on private sector participation, found in the various CASs and the Country Partnership Strategy. Both Bank and government strategies have consistently emphasized the need for enhanced protection and management of the environment. The project objectives therefore remain highly relevant to the priorities of both the country and the Bank. RELEVANCE OF DESIGN 3.14 Relevance of design is rated modest. The causal chain between the activities funded under the project and the expected outcomes was generally clear and convincing, except for the fourth objective, to (i) improve environmental protection practices and (ii) to define rural water and sanitation policies and to develop methods for increasing coverage, neither of which had PDO indicators in the project design. 3.15 The private sector participation model: The project designed two models for service delivery: 1. The “construction-with-operation” model. This was designed for small municipalities, generally with a population under 12,000. The national government would define the investment program, and the municipality would then contract a “constructor-operator” for both the infrastructure construction in the first 2–3 years and operation of the system for 10 or more years. The municipality would be the employer and the “constructor-operator” would be the contractor in this model. This was not realistic, given the poor capacity and resources of small municipalities. 2. The “operation-with-investment model.” This was designed for medium-size municipalities, and it followed a more traditional concession model. The national government would bid for the concession contract based on the “lowest subsidy” required to meet the predefined investment amounts. In this model, the operator would serve as the employer for all construction contracts. During implementation, the government decided to focus more on medium-size cities than on small municipalities (contrary to the project design), and thus the operation-with-investment model was the model most commonly used in this project. 3.16 The Independent Evaluation Group (IEG) mission considers some of the project’s implementation challenges the result of weaknesses in the project design. In an effort to build 26 local capacity to operate unattractive utilities in small municipalities, the construction-with- operation model was created. But the expectation that a construction contractor could become an operator might have been overly ambitious at the time of project approval. The design did not take into account that infrastructure construction and utility operation require people with very different sets of professional skills. 3.17 The initial works planned fell short of the substantial infrastructure investment needed to operate the systems adequately. Out of the 12 contracts signed, 3 were terminated due to problems with financial viability, 5 were renegotiated to extend the concession duration or to obtain additional resources, and the rest received additional resources. 3.18 IEG found that the limited diagnostic work prior to contracting led to inaccurate assessment of the investment needs and had an effect on the inaccurate estimation of the contractual tariffs. Many operators found that the infrastructure needed substantial investments that had not been taken into account in the calculation of the tariffs, and they struggled to reach cost recovery within the limits of the fixed tariffs. The ICR team noted that operators who had managed to renegotiate their contract and linked their tariff regime to national regulations had achieved, or were close to achieving, full cost recovery levels at the time of project closure. In contrast, operators that still had tariffs linked to the original contract were generally struggling financially.6 Now, however, more than five years after project closure, almost all the operators (with the exception of one) have managed to free themselves from the contractual tariff. The one that continues with a fixed tariff has been able to do so because of the capacity of the operator to negotiate additional resources from the government. 3.19 M&E Design: The results framework prepared at project appraisal identified the linkages between the objectives, outputs, intermediate outcomes, and the expected impacts, but the indicators did not necessarily match the corresponding objectives. There was no indicator corresponding to the objective of improved environmental protection practices, and the policy objective was simply the finalization of a document, which is an output and not a final outcome. The results framework was detailed. Baseline and target values for each year for both the medium-size cities and the small municipalities were defined in a systematic 6 On page 4 in the financial analysis for the ICR evaluation (World Bank 2011b), it is stated: “Tariffs and transfers from municipalities were initially set on the contracts, yet along the time, some municipalities have agreed with the operators to adjust tariffs according to National regulatory framework and to negotiate on a year basis the corresponding transfers to pay off the subsidies given to low income customers through tariffs. Depending on these agreements the sharing of both sources of revenues (tariffs and transfers), and the resulting cost recovery vary widely among municipalities. In general, those that have adjusted tariffs to regulatory framework (Soledad, Arjona, and Sincelejo) have achieved or are close to achieving full cost recovery and they have also decreased the dependence on revenues from municipality, and have improved their financial results. Meanwhile, those with tariffs and transfer negotiated on a year basis with the mayors, or with tariffs linked to contract, have neither reached full cost recovery, nor achieved positive returns. La Linea and San Onofre, are exceptions, given that they have tariffs according to contracts, yet show profits; the reason is found on the type of contract, which corresponds to construction-operation contracts. Under this contract the utilities’ profit can depend highly on the transfers from the national government, 68% and 53% of the respective revenues come from the payments made by the government for the construction of the works.” A similar analysis and conclusion are presented in the ICR’s annex 4, p. 43. 27 manner for each subset of indicators, such as service to all, service to the poor, efficiency of service, and sustainability of service. Implementation 3.20 The project was approved in October 2001, and became effective in June 2002. The closing date was extended from June 30, 2007, to October 31, 2010 (a delay of 3 years and 4 months, or 40 months), through three extensions. This was mainly because of the extensive delay in two subprojects; (i) the original operator for Magangué was replaced in 2010 due to implementation delays and (ii) the La Linea subproject in Bolívar experienced delays in the installation of a critical pipeline caused by complications with a parallel highway construction project. The project experienced a number of adjustments during implementation, highlighted below. 3.21 The Ministry of Economic Development was merged with the Ministry of Environment, Housing, and Regional Development (MAVDT) when a new presidential administration came into office in 2002. This led to a high turnover of staff and frequent management changes, which proved disruptive to project implementation. The project survived through four different political administrations, each of which required time to learn the project’s design and implementation status. Project implementation and staffing only stabilized with the appointment of the first vice-minister for water in 2006. 3.22 As a result of the slow project start-up, US$4.31 million of budgeted resources were “lost” and subsequently cancelled from the US$40 million loan, resulting in a final disbursement from the IBRD of US$35.69 million. This was the result of regulations of the Colombian central government budget, into which all disbursements under World Bank– financed projects are fully integrated and where any unused funds cannot be transferred to a new fiscal year. This means that when disbursement rates are slower than planned, the proportion of the annual appropriations for investments under the central government budget that cannot be utilized within a given year is lost and cannot be recuperated through budget appropriations in later years. 3.23 There was a shift to operations in medium-size cities at the expense of the smaller municipalities. Only three medium-size cities were expected to participate in the project under the operation-with-investment model. However, both private sector and municipal interest in the smaller municipalities were less than originally anticipated, and the MAVDT focus on medium-size cities diverted attention from the smaller municipalities during the early phases and resulted in an expansion of the project. This shift created a need for more financing, and the national government increased its contribution from US$30 million to US$45.9 million for the entire PME. The project loan resources were spread out in smaller amounts to a greater number to medium-size cities. In contrast, only 7 small municipalities participated under the construction-with-operation model, under 3 separate contracts, although 15 municipalities had originally been planned for inclusion. Safeguards and Fiduciary Compliance 3.24 Safeguards: The project triggered OP 4.01 Environmental Safeguards, and was classified as a category B project. Most of the investments were related to the rehabilitation 28 of existing water supply and sewerage systems, and no significant negative environmental impact was expected. No wastewater plants were built under this project. According to the ICR (World Bank 2011b), each of the construction contracts included standard environmental and safety requirements, and the contracts were supervised by independent construction supervisors to ensure compliance with environmental and safety requirements. 3.25 The IEG mission, however, found that the construction supervisors were contracted by the municipalities to oversee contract compliance, and they were paid by the same operators they were hired to oversee, which could potentially compromise the quality of contract compliance control. 3.26 Fiduciary Compliance: There were some challenges with fiduciary compliance during implementation. The Government Official Information System was not adjusted to capture all the project transactions, so manual records had to be kept, and some deficiencies in the internal controls were found. Furthermore, there were delays in the submission of withdrawal applications. Fiduciary compliance improved toward the end of the project. Achievement of the Objectives OBJECTIVE 1 3.27 The first development objective was to “create and maintain an appropriate environment for improving the efficiency and sustainability of water utilities.” (World Bank 2001b, Schedule 2, p. 1). This objective was modestly achieved, based on the analysis of the operators’ current degree of efficiency and sustainability. Outputs 3.28 Number of Private Sector Participation Contracts Signed: As shown in table 8, 11 operation-with-investment contracts were signed, compared with the target of 2, and 3 construction-with-operation contracts were signed, against a target of 15. Thus, fewer construction-with-operation contracts in smaller cities and many more operation-with- investment contracts in medium-size cities were signed, compared with the original plan. The reason for this shift in focus from smaller cities is outlined above. The shift to medium-size cities also contributed to a substantial increase in beneficiaries, from the original target of 700,000 to an actual number of 1.6 million. Table 8: Number of Private Sector Participation Contracts Signed under the Project Type of contract City size Target Contracts achieved Number of municipalities Operation-with- Medium- 2 11 PSP contracts 18 investment size cities Construction- Smaller size 15 3 PSP contracts 7 with-operation cities Total 17 14 25 Note: PSP = private sector participation. 29 3.29 By project closure, 36 municipalities in the Caribbean region had private operators under the government’s Modernization Program, and 25 of these were supported by the World Bank. 3.30 Creation of an Appropriate Policy Environment: The IEG mission found that the project contributed to the strengthening of the government’s Modernization Program. Two legislative reforms and three executive decrees were adopted by the government during project implementation to improve and facilitate an enabling environment for private sector participation. Outcomes 3.31 Regarding the efficiency and sustainability of the water utilities, the IEG mission found that, on average, some operators have realized operational and financial improvements since the initiation of their contracts, but others are still experiencing losses (see table 9 and appendix B for details). While most operators have decreased their working ratio,7 only one (Insergroup), a contractor-operator, reached the expected target in 2014. Further analysis indicates that the main driver for reaching the working ratio target is the contractual model (constructor-operator), which is designed to allow the operator to generate revenues not only from the operation, but also from the construction of civil works. Other indicators, such as operating margin and net margin, show, on average, an improvement from the baselines, but some still show significant difficulty in becoming sustainable. Table 9: Average Operational and Financial Ratios Baseline, Completion, Target, 2001 2009-10 2010 2013-14 Working ratio 1.07 0.92 0.70 0.96 Constructor-operator 0.93 0.84 0.85 0.83 Operator-with-investment 1.14 0.96 0.63 1.05 Bill collection rate 24% 65.6% na 80.6% Constructor-operator na 37.0% na 68.5% Operator-with-investment 24% 75.1% na 85.8 Operating margin -0.07 0.08 na 0.04 Constructor-operator 0.07 0.17 na 0.21 Operator-with-investment -0.14 0.04 na -0.05 Net margin 0.01 0.11 na 0.25 Constructor-operator 0.01 0.11 na 0.19 Operator-with- investment -0.29 -0.03 na 0.17 EBITDA margin na na na 0.06 Constructor-operator na na na 0.23 Operator-with- investment na na na -0.03 Sources: SIU: Unique system of information of the Superintendence of Residential Public Services. Information from the contract studies. Note: Includes only two constructor-operators and five operators-with-investment for which consistent available data was obtained. However, one of the contractor-operator’s contracts was terminated in 2015. na = Data not collected or not defined at appraisal. 7 Working ratio measures the relation between operating costs (without deprecation) and operating revenues. 30 3.32 In general, the operators that have not achieved a solid continuity of services have difficulty increasing tariffs, and therefore they are the ones that have little margin to invest (Aquaseo, Acualco, Aguas de la peninsula). In contrast, operators that have achieved reliable continuity of service have been able to increase tariffs, and as a result have significantly improved their operations, reaching financial sustainability. That is the case with the Aguas de la Sabana and Triple A operators. Triple A, a knowledgeable operator, consolidated services of six municipalities into one, which led to the significant improvement of working ratios and other financial indicators, and ultimately to improving the financial viability and the capacity to generate additional resources in these six municipalities. 3.33 High dependency on energy is another factor affecting long-term sustainability. Representatives of several of the water utilities interviewed by the IEG mission reported that their energy costs were the single largest expenditures in their budgets. Table 10 shows that energy expenditures range from 13 to 36 percent of the utilities’ total costs. To be able to provide a continuous water supply, the utilities are dependent on regular power supply for pumping the water from the source to the plant, and then to the consumers. The IEG mission found that power cuts are frequent in some places, and only one of the water utilities under this project has their own generator. This means that the water supply is discontinued whenever there is a power cut. Many utilities reported that in addition to the substantial electricity expenditures, lack of access to continuous power is a significant challenge that directly affects their ability to deliver a continuous water supply, and in turn contributes to lower bill-collection rates, since people are less willing to pay the water bills when the supply is intermittent. Table 10: Energy as a Percentage of Total Cost Utility Percent Constructor-operator 21 Giscol SA 13 Insergrup 28 Operator-with-investment 26 Acualco 36 Aguas de la Sabana 28 Triple A 14 Sources: SIU: Unique system of information of the Superintendency of Residential Public Services. Information from the contract studies. OBJECTIVE 2 3.34 Objective 2 was to “expand the coverage of water supply and sewerage services provided in participating municipalities” (World Bank 2001b, Schedule 2, p. 1). The overall water supply objective was substantially achieved. By type of intervention, it was substantially achieved for medium-size cities (investment-with-operator model), and modestly achieved for small cities (constructor-operator model). In contrast, the coverage of sewage services was negligibly achieved for both types of interventions. Given the 31 substantial achievement of water supply coverage and the negligible results for the sewage coverage, the overall objective was rated modestly achieved. Outputs 3.35 Outputs related to this objective are: • Routine upgrades and rehabilitation of existing water supply and sewage systems in all the participating municipalities. • The primary target group was 700,000 people at appraisal, and the project beneficiaries rose to 1.6 million, partly due to the inclusion of more medium-size cities than originally planned. The total number of people with access to improved water supply substantially exceeded the targets. Outcomes 3.36 Water Supply: While on average there has been an overall 23 percent increase in coverage since the initiation of the project, about 40 percent of the targets available (7 of 17) have not yet been met. The average current coverage in all the project areas is 85 percent, which is 10 percent lower than the national average. Results by type of model show less improvement in municipalities with the constructor-operator model (13 percent) than with the operator-with-investment modality (27 percent). The IEG mission found that water supply coverage increased during project implementation, and, with two exceptions, continued to increase after completion of the project (see table 11). Four of the subprojects still have low water-supply coverage (47 percent, 64 percent, 65 percent, and 67 percent of the target population), but all of them have steadily, albeit slowly, increased water coverage. The remaining subprojects have a coverage percentage of 72 percent or higher. Table 11: Water Supply Coverage (percentage) Baseline, Completion, Target, By municipality 2001 2009-10 2010 2013-14 Constructor-operator 0.64 0.66 na 0.77 Operator-with- 0.62 0.84 0.95 0.89 investment Total 0.62 0.79 0.95 0.86 Sources: World Bank 2001a, 2011b; SIU: Unique system of information of the Superintendency of Residential Public Services. na: No target defined at appraisal. 3.37 Sewerage: The IEG mission found that in only 1 of the 14 subprojects with targets were these targets met. Current data show that sewerage connections for medium-size cities have increased only 10 percent since project start-up, and there is a range in coverage from not having been implemented (six subprojects), to 43–97 percent coverage in the remaining subprojects. While no data were available for small municipalities, evidence gathered from structured stakeholder interviews confirmed little to negligible progress on sewage connections. The average coverage in the participating municipalities is 42 percent, and the average target was 88 percent. See table 12 and appendix B for details. 32 Table 12: Sewage Coverage Baseline, Target, By municipality 2001 2010 2013-14 Constructor-operator 0.00 na 0.09 Operator-with-investment 0.32 0.88 0.42 Total 0.32 0.88 0.42 Sources: World Bank 2001a, 2011b; SIU: Unique system of information of the Superintendence of Residential Public Services. na: No target defined. 3.38 Continuity of Water: The percentages represent the weighted average of a 24-hour day. Although it was below the targets, there was a substantial improvement in continuity of water supply for 7 of the 17 medium-size cities (operator-with-investment model). On average, they improved from a baseline of 13 percent to 58 percent at project completion, and continued to improve to 78 percent as of 2014. Five municipalities operated by Triple A, two municipalities operated by Uniaguas, and one operated by Aguas de la Sabana achieved between 97 and 100 percent continuity of water service by 2014. In contrast, five other medium-size municipalities only have water service 50 percent of the day or less. Water supply continuity in smaller cities (constructor-operator model) improved only marginally, from a baseline of 8 to 41 percent at project completion, and to 44 percent as of 2014. In general, results are still below the Colombian norm of 100 percent and the 100 percent average of 64 of the biggest water companies in Colombia. Table 13: Continuity of Water Supply Baseline, Completion, Target, By Municipality 2001 2009-10 2010 2013-14 Constructor-operator 0.08 0.41 na 0.44 Operator-with-investment 0.13 0.58 0.97 0.78 Total 0.13 0.53 0.97 0.69 Sources: World Bank 2001a, 2011b; SIU: Unique system of information of the Superintendence of Residential Public Services. na = No target defined at appraisal. 3.39 Water Quality: Water quality was not included as part of the PDO, so there was no indicator specified in the PAD that related to it directly. Nevertheless, monitoring of this indicator was initiated in 2007 by the National Institute of Health through the Risk Index of Water Quality for Human Consumption (IRCA8), which defines the physical, chemical, and bacteriological quality of water through monthly samples. The data collected through the IRCA for each of the participating municipalities is presented in table 14. 3.40 Water quality is supposed to be regularly measured by all water utilities, but the EG mission noted that water quality data is missing for two municipalities. Results show that 8 The IRCA defines water quality risk as the following: (i) 0–5 percent (risk free), (ii) 5.1–14 percent (low), (iii) 14.1–35 percent (medium), (iv) 35.1–80 percent (high), (v) 80.1–100 percent (unsanitary). 33 water quality with the investment-with-operator model is considered to be “without risk” in 15 of 16 municipalities with available data. In contrast, the municipalities under the operator- and-constructor model are considered, on average, “medium risk,” which is below the acceptable standard in Colombia; it is low in three municipalities, medium in six municipalities, and high in one municipality. Table 14: Quality of Water: IRCA By municipality 2009-10 2013-14 Risk level Constructor-operator 0.32 0.25 medium Operator-with-investment 0.06 0.03 no risk Total 0.13 0.09 low Sources: World Bank 2001a, 2011b; SIU: Unique system of information of the Superintendence of Residential Public Services. OBJECTIVE 3 3.41 Objective 3 was to “facilitate the access to water and sewerage services by the population of low-income areas in participating municipalities (World Bank 2001b, Schedule 2, p. 1). This objective relates to the project’s poverty focus. The achievement of this objective is substantial, because the IEG mission found that more than 90 percent of the population in the participating municipalities is in the three lower economic strata. Outputs 3.42 The IEG mission found that the great majority (91–100 percent) of the population in the participating municipalities are categorized in the lowest three socioeconomic strata (strata 1–3). This demonstrates that the project has managed to keep a poverty focus. See table 15. Table 15: Population: Percentage of Strata in Service Area Population Strata 1 Strata 2 Strata 3 Constructor-operator 98,211 85% 13% 2% Operator-with- investment 1,555,662 46% 44% 9% Total 1,653,873 57% 35% 7% Sources: National Department of Statistics, DANE, and SIU. 34 Outcomes 3.43 The project had a clear poverty focus. Between 90 and 100 percent of the beneficiaries belong to the three lowest socioeconomic strata. OBJECTIVE 4 3.44 Objective 4 was to (i) “improve environmental protection in the borrower’s territory; and (ii) to define rural water and sanitation policies.” (World Bank 2001b, Schedule 2 p. 1) The achievement of these objectives is rated modest. Outputs 3.45 A decision-support tool was developed under the project to facilitate the planning of smaller municipal wastewater treatment plants. Generic plans, costs, specifications, and manuals were developed for 13 treatment technologies. But the decision-support tool was not formally disseminated or applied, and thus did not have the intended effect of improving environmental protection practices. 3.46 Although some policy directions were identified at project completion, the MAVDT’s rural water program became inactive, and a national rural water policy was not formally defined. The IEG evaluation team found that the government released the 2010–14 national rural water policy, published in 2011, a year after project completion. However, it is unclear to what extent the activities realized under the project contributed to the formulation of the rural water and sanitation policy. Efficiency 3.47 An economic rate of return was calculated at appraisal. A traditional economic analysis was not carried out at project closure, however, because of the large number of subprojects and the data-intensive nature of the exercise. While the ICR (World Bank 2011b) reports that there is a common recognition that providing water and sanitation services results in health and economic benefits, no evidence of these benefits in the context of this project is presented in the ICR. According to the ICR, financial sustainability remains a concern (p. iv, under indicator 6). 3.48 The project was delayed by 40 months, and the total cost of US$81.54 million was higher than the estimated US$70 million at appraisal. These factors reduce the efficiency of the project, which is thus rated modest. Ratings OUTCOME 3.49 Objective 1 (efficient and sustainable water utilities) was substantially achieved. Objective 2 (increased water and sewerage coverage) was modestly achieved. Water supply coverage surpassed the original targets mainly through the inclusion of many more medium- size cities at the cost of smaller municipalities. The target for sewerage coverage was not met. The poverty focus of objective 3 was substantially achieved, and achievement of the fourth objective, to improve environmental protection practices and to define rural water and sanitation policy, was modestly achieved, because in the case of environmental protection 35 practices, a decision tool was developed during the project, but it is unclear to what extent it was used after project completion. In the case of rural water and sanitation policy, output was achieved after project completion, but the contribution of the project is unclear. 3.50 Relevance of objectives is rated high, because they were fully in line with both national and Bank priorities at initiation, throughout the project, and at closure, and the objectives remain relevant today. Relevance of design is rated modest. Although the project’s logical chain was generally clear and convincing, and the components were necessary and sufficient to reach the objectives, there was one objective that lacked PDO indicators in the project design. Efficiency was rated modest, mainly due to the long delay in project implementation, the increase in the budget, and the lack of evidence for the efficiency analysis presented. The project’s shortcomings are considered substantial, and the overall outcome of the project is therefore rated moderately satisfactory. RISK TO DEVELOPMENT OUTCOME 3.51 The risk to development outcome is rated modest. 3.52 Policy Risk: The government has shown significant commitment to the project, both by putting in place an adequate policy framework and by allowing the linking of government resources to private sector involvement. There is, however, a risk that future governments might not follow through with this policy. This risk is considered to be modest. 3.53 Contractual Risk: Many of the contracts have been altered through several negotiated amendments, especially with the entry of new municipal administrations. The ICR (World Bank 2011b) reported some examples of cessation of contracts because of the inability of the water utilities to comply with them. But the IEG mission found that although a small number of the utilities had been closed down because they were not able to operate efficiently, the majority of the utilities are still operating and have maintained and increased improvements five years after project completion. The contractual risk is therefore considered to be modest. 3.54 Financial Risk: The financial risk continues to be modest. The average working ratio of 0.96 is above the target of 0.7, indicating that some operators are continuing to struggle financially. However, operators that are able to cross-subsidize from bigger to smaller operators have better prospects than the smaller ones. Most of the operators are now following the national tariff regulatory regime. Only the ones that are able to show real improvements in service have been able to increase their tariffs. Continued government support and ability to negotiate government transfers are also crucial. BANK PERFORMANCE 3.55 Quality-at-entry is rated moderately unsatisfactory. The Bank team incorporated relevant experience with private sector participation into the project design and provided adequate technical support and model bidding documents for the municipalities and the private sector. The PAD (World Bank 2001a) included details of the project concept and the two types of novel model contracts: (i) operation-with-investment and (ii) construction-with- operation. However, the design relating to the construction-with-operation contracts was 36 unrealistic. It was based on the assumption that contractors/construction companies could “turn into” water utilities/operators once the construction work was done, and it did not take into consideration the very different skills needed for construction and for operation. Risks were identified at appraisal, and mitigation measures established for most of them. However, the project design did not take into account risks related to electricity supply and prices or the time needed for adequate diagnostic studies and the bidding processes. It also did not give adequate consideration to the difficulty of attracting private operators to the smaller cities. 3.56 The IEG mission further found that the time needed for the bidding process was underestimated, and the strong preference to opt for the lowest bids rather than considering relative quality and capability of contractors and operators may have contributed to lower achievement of project outputs and outcomes. The time provided for the diagnostic studies was also limited, which resulted in poor or incomplete diagnostics, and this, in turn, created difficulties for the operators, because the state of the infrastructure was inferior to that anticipated at contract signing. 3.57 Supervision is rated moderately satisfactory. While the team changed right before project completion, there was a continuation of project team leadership throughout preparation and most of the implementation period. The Bank team provided adequate support to the operators and MAVDT during implementation. Twenty supervision missions were conducted throughout the 10 years of project operation (2001–10). 3.58 Overall Bank performance is rated moderately unsatisfactory. BORROWER PERFORMANCE 3.59 Government performance is rated moderately satisfactory. During appraisal, the government was committed to the project and its reform elements. It adopted a series of resolutions and decrees to strengthen its reform program related to private sector participation and disseminated the policy widely to generate demand from the municipalities to participate in the project. The government contributed more than half of the total project costs from its national budget and increased its contribution by 53 percent during implementation, with actual counterpart funding of US$45.86 million, compared with the originally planned US$30 million, which shows dedication. 3.60 A new administration took office shortly after project approval in 2001, and the subsequent departmental restructuring contributed to delays in project implementation because of insufficient staffing and frequent management changes. It was not until the formal establishment of a vice minister of water in the MAVDT in 2006 that project staffing and management stabilized, and implementation started to accelerate. The government continued to support the project during the remaining implementation period. While policy directions were identified, MAVDT’s rural water program remained inactive, and a national rural water policy was not formally defined. 3.61 The government’s focus on medium-size cities diverted the project’s focus from the smaller cities, leaving them (originally the project’s target populations) without access to the interventions. 37 3.62 Implementing agency performance is rated moderately satisfactory. For the constructor-operator model designed for the smaller cities, project implementation was the responsibility of the municipalities. The IEG mission found that very few contracts were signed for smaller cities, and the few contractors operating in these areas generally showed poor performance. Many of the contracts had to be renegotiated, and a number of technical issues were encountered by inexperienced operators. However, in the medium-size cities, utilizing the operation-with-investment model, project implementation was the responsibility of the operators. Their performance varied, with some utilities, such as Triple A and Aguas de la Sabana, performing very well, while others performed less well. However, the IEG mission found that although a small number of the utilities had been closed down because they were unable to operate efficiently, the majority of the utilities are still operating and have maintained and increased improvements five years after project completion. 3.63 The overall borrower performance is rated moderately satisfactory. MONITORING AND EVALUATION 3.64 M&E Design: The results framework in the PAD identified the key performance indicators as outputs, outcomes, and impacts that were generally appropriate, but the indicators were not designed to match the corresponding objectives. The objective related to “improvement in environmental protection” had no corresponding indicator, and the policy objective was formulated to be the finalization of a document (which, technically speaking, is an output). Baseline and target values for each year were defined for both the small and the medium-size municipalities for each subset of indicators, which facilitated the monitoring of progress during project implementation. 3.65 M&E Implementation and Utilization: There is limited information on the project’s M&E implementation in the ICR. There is only an indication that not much monitoring work was undertaken for the various subprojects, because the ICR team had to do substantial data collection work to deal with the lack of M&E data. The IEG mission found that because of the limited capacity of the municipalities and the operators, the project experienced difficulties in implementing the M&E activities systematically. For the assessment of the project’s achievements for this PPAR, the IEG team relied on data collected from the SSPD, which is of acceptable quality, but with the caveat that operators often fail to regularly report to the agency. These indicators are used by the implementing agency to monitor management capacity or performance, and they could prompt a decision to decertify the operator. Each utility is supposed to report on specific indicators regularly to the SSPD. 3.66 Quality of M&E is rated modest. 4. Water and Sanitation Sector Support Project (2005–11) 4.1 In 2002 the government of Colombia allocated US$500 million for investment projects in many sectors, including water and sanitation. Following public consultation activities, the government launched a water public works program (the Program) to channel capital grant resources to improve access and quality of WSS services. The Program was estimated to cost US$180 million, and the government requested support from the World 38 Bank to finance part of the Program. This led to the Bank-supported Water and Sanitation Sector Support Project (WSSSP). 4.2 At the time of appraisal of the WSSSP in 2004, Colombia had made progress in the expansion of WSS services in urban areas, where coverage levels had reached 97 percent for water and 90 percent for sewerage. Conversely, in peri-urban and rural areas, sewerage facilities were still inadequate and about 50 percent of the drinking water was below standard quality levels. Less than 10 percent of the domestic wastewater was treated. Water supply was intermittent, and rationing was common. Water and sanitation coverage in rural areas were 55 and 15 percent, respectively. 4.3 The WSSSP was to support Colombia’s objective of investing in infrastructure and building institutional, monitoring, and oversight capacity. The project sprung from the government’s public works program and did not emphasize institutional reform. Objectives, Design, and Relevance OBJECTIVES 4.4 The project was the first of a proposed three-phase adaptable program loan (APL). The PDO for the overall program was to “improve access to water supply and sanitation throughout Colombia” (World Bank 2005c, p. 6). 4.5 The PDO for the first phase of the APL, as formulated in the loan agreement (World Bank 2005a, section 2) was “to improve the access to water supply and sanitation services in rural and urban communities throughout the borrower’s territory.” The PDO for the first phase of the APL as formulated in the PAD (World Bank 2005c, p, 8) was more detailed: “to improve the provision of water supply and sanitation services in Colombia in a financially efficient and sustainable manner, through the provision of capital investment subsidies for poverty-focused coverage expansion, and service quality improvement.” This PPAR assesses the project achievements toward the PDO as formulated in the loan agreement, but recognizes that the spirit of the objective as elaborated in the PAD captures the need to ensure sustainable access more clearly. COMPONENTS 4.6 The project had two components, as detailed in box 3. 39 Box 3: Components and Costs Component 1: Municipal Water Supply and Sewerage Infrastructure (Appraisal, US$92.2 million; actual, US$105.2 million). Subcomponent 1A: Infrastructure and Municipal Utility Reform and Development. This included costs of rehabilitation and construction of civil works (primary and secondary distribution networks for water supply and sewerage; wastewater treatment plants), engineering designs, and environmental and social assessments. The subcomponent also provided technical assistance to strengthen and modernize the utility companies and bring them to reasonable levels of management, operational efficiency, and financial viability, and potentially facilitate private sector participation. Subcomponent 1B: Supervision and Technical Assistance. This subcomponent financed activities related to enterprise development and reform, including equipment, technical assistance, training and studies. It provided hiring of independent supervision consultants to oversee construction and implementation at the municipal level, and private sector consulting firms to strengthen subproject design. Component 2: Program Management, Benchmarking, Monitoring and Evaluation Component (Appraisal, US$1.2 million; actual, US$1.5 million). This component financed consultant services and equipment to strengthen the Directorate of Potable Water and Basic Sanitation (DAPSBA) to implement and supervise the project and to provide technical assistance to the participating municipal utilities. The component also supported the development and implementation of an institutional framework for monitoring and evaluating subproject execution and performance. The component also funded technical consultants to support the Ministry of Environment, Housing, and Regional Development (MAVDT) to review the viability of proposed subprojects and to manage the safeguards aspects (social and environmental) of the project. Sources: World Bank 2005a, Schedule 2, 2005c, pp. 10–12. 4.7 The total project costs at closing were US$107.1 million, 14 percent more than the appraisal estimate of US$93.8 million. Civil works accounted for 88 percent of the actual project costs. The IBRD loan of US$70 million was fully disbursed, and there was no other external financing. The government contribution of US$37.1 million was 56 percent more than the appraisal estimate of US$23.8 million. This was due to the protracted implementation period and to the higher-than-estimated costs of the subprojects. The project closing date was extended three times (April 21, 2009, April 29, 2010, and in December 2011), for a total of 23 months. Although the project closing date was on March 31, 2011, the last extension to December 2011 was made to allow for the completion of 13 subprojects, implementation of a Remedial Action Plan for safeguard management, and an independent end-project evaluation. RELEVANCE 4.8 Relevance of objectives is rated substantial. The project’s development objectives were consistent with the broad priorities endorsed in the CASs of 2002, 2005, and 2008 (World Bank 2002, 2005b, 2008), which emphasized sustainable growth and building efficient, accountable, and transparent governance systems in the public services. 4.9 The project’s objective to provide capital investment subsidies for the poor was also relevant to Colombia’s water sector strategy at the time of appraisal. An estimated 10 40 million people lacked access to piped water and 15 million were without sewerage systems. The project targeted municipalities with a high number of poor household (in Colombia, referred to as population strata 1, 2, and 3). 4.10 At completion, the project objectives were still relevant to the Country Partnership Strategy (CPS) 2012-16 (World Bank 2011a), which emphasized sustainable urban development; environmental management; and improved water, sanitation, and wastewater management. The project’s objective to improve the provision of water supply and sanitation services in Colombia was in line with the country’s National Development Plan for 2010–14, which places a strong emphasis on improving WSS services and retaining central elements of the previous administration’s water policy and program. 4.11 Relevance of design is rated modest. The project’s components and activities of providing for physical infrastructure and technical assistance were relevant and sufficient to achieve the objective of improved access to water and sanitation. The design in the PAD (World Bank 2005c) shows a reasonably clear causal chain linking the interventions to the overall objective of improving and expanding water and sanitation services. However, the results framework was weak, focusing on outputs at the subproject level (that is, completion of works) rather than on outcomes. The project design was complex, especially with regard to the implementation arrangements. There was a multiplicity of players; the Ministry of Housing, Environment, and Regional Development (later it changed its name to the Ministry of Housing, City, and Territory); the municipalities; the implementing agency, Financial Funds for Development Projects (FONADE), which had fiduciary responsibility for civil works in Colombia; public and private water operators; two groups of contracted agents to act as third-party validators; and an array of contractors and consultants who helped with the project implementation. 4.12 The two principal groups of players were the ministry and the beneficiary municipalities/small towns and local communities. Technical assistance was provided to strengthen both entities, but given the large variability in capacity among the municipalities, the design did not provide for adequate support to the weaker municipalities and local governments, and issues related to land acquisition and easements were not adequately factored into the project design. 4.13 The indicators and target values identified at appraisal were based on 37 pre- identified core subprojects. However, the project did not specify which of the core indicators were to be applied to the project as a whole, and which were to be collected from the subprojects. 4.14 This was a demand-driven project, and the number of municipalities (550) that applied for funds was higher than expected. At the subproject level, M&E design was weak, with poorly defined outcome indicators that were not well-linked to the PDO. Much of the project entailed local-level construction and management of extensive civil works (for pipelines, holding stations, and wastewater treatment plants) in dispersed geographic areas, and monitoring of the program and subprojects was thus highly dependent on local-level 41 efforts. At subproject level, objectives were focused on outputs (that is, the completion of the works), rather than on outcomes. Implementation 4.15 The ministry developed a technical screening and prioritization process, called la Ventanilla Unica (the “Single Window”), to ensure that all the necessary documents and studies (such as feasibility, environmental, and social) of the selected subprojects were considered. However, given the limited timeframe and the significant number of projects, the depth of the review was limited to a desk review of the required documents. The Ventanilla Unica allowed the ministry to detect issues early in the project cycle, although some faulty designs were not detected and had to be corrected during construction. The approved subprojects were executed by the municipalities with independent supervisors. The municipalities hired contractors for the construction works, while FONADE contracted independent supervisors and acted as a fiduciary agent, disbursing directly to the municipality contractors after approval from the independent supervisors. The complexity of this arrangement, coupled with the weak capacity of municipal teams to manage the procurement processes, caused implementation delays. 4.16 The subprojects were to be executed in 16 months (less than 1.5 years), but took 44 months on average (nearly 4 years). This was partly due to the challenges faced by the municipalities in obtaining right-of-way permits and acquiring land in a timely manner. Further contributing factors were suspension of contracts because of design errors, contractor insolvency, and delays in obtaining counterpart funds. In addition, heavy rains in 2010–11 caused floods in several of the project areas, which caused delays. While the implementation of all Bank-financed activities was complete at project closure, there were nine subprojects that needed additional time to achieve their expected results. 4.17 The APL programmatic approach was abandoned in 2008, when MAVDT adopted a new policy, the departmental water plans (PDAs). The PDAs were meant to provide incentives for promoting regional-level management of water and sanitation services, achieving greater economies of scale and regional agreement on the use of financial resources. With the PDAs, the provincial level of government (departamento) became the regional interlocutor between the MAVDT and the municipalities. The shift in responsibility for the regulatory framework and support for the municipalities to the PDAs meant a change in implementation arrangements for the subprojects that adhered to the PDAs. The first phase required municipalities to relinquish control of how the resources assigned to them were distributed within the municipality. In the second phase, the PDA’s boards conducted a diagnostic exercise of the infrastructure. In the third phase, departmental water boards designed a plan for works and investments. In the final phase, PDAs supported and monitored the performance of operators throughout their department. 4.18 As a result of this new policy, the national approach supported by the APL program was no longer relevant. Instead of APL 2 and 3, a follow-on stand-alone project on solid waste management was developed. 42 Safeguards and Fiduciary Compliance 4.19 The project triggered the following safeguard policies: Environmental Assessment (OP 4.01), Cultural Property (OP 4.11), Involuntary Resettlement (OP 4.12), and Indigenous Peoples (OP 4.10). Framework documents (for Indigenous Peoples, Involuntary Resettlement, and Environmental Assessment) were prepared for a category B project, since the subprojects were not yet defined during project preparation. Compliance with the safeguard policies was part of the Ventanilla Unica screening process of the subprojects, and standard environmental and social requirements were built into construction contracts and municipality agreements. Despite these efforts, the Bank found weaknesses in the implementation of the safeguard policies. The weaknesses were addressed, and a satisfactory Implementation Status and Results Report (ISR) rating for the safeguards was achieved at completion. 4.20 Environmental Safeguards: Because of the weaknesses detected, the Bank team and the ministry implemented an action plan to ensure compliance with the environmental safeguards, including environmental supervision templates to be used by MAVDT and FONADE field supervisors. Following this, no major environmental impacts, serious accidents, or complaints related to environmental work within the subprojects were recorded. 4.21 Social Safeguards: The mid-term review revealed substantive and procedural weaknesses in the application of the involuntary resettlement and indigenous peoples’ policies. The ministry and the Bank designed a remedial action plan for each of the relevant subprojects, including specific actions regarding land acquisition, easements, resettlement, and indigenous peoples. A final report at project completion concluded that all activities were completed, and a high percentage of cases were resolved; the ministry completed 80 out of 103 parcels (80 percent compliance) and 1,093 of 1,177 easements (93 percent compliance) satisfactorily. 4.22 Fiduciary compliance: Financial management arrangements were satisfactory throughout project implementation. The external auditors issued unqualified opinions without exception on the project financial statements for the audit periods 2006 to 2011, and MAVDT implemented their recommendations in a timely manner. 4.23 Procurement performance was satisfactory throughout project implementation and there were no cases of misprocurement. MAVDT and FONADE trained the staff in the municipalities and oversaw the procurement processes. Achievement of the Objectives OBJECTIVE Achievement of PDOs is rated modest, based on the degree of achievement of the three subobjectives during the IEG-commissioned review. Urban water supply (urban water) was achieved, but urban sanitation (urban sanitation) and rural water supply and sanitation (rural WSS) fell short of their targets at the time of the IEG PPAR mission. In addition, the IEG team considers that the spirit of the objective as elaborated in the PAD points to ensuring 43 efficient and sustainable access. However, such data was not made available, and the scant sample data collected provides little evidence that sustainable and efficient access has been ensured. Outputs 4.24 During the project period, a total of 296 subproject applications from municipalities and communities were approved; 88 (from an original estimate of 96) were funded by the project. The subprojects were implemented in 96 municipalities in 22 of Colombia’s 32 states. Seventy-four of the subprojects were in peri-urban and urban areas, while 14 were in rural areas. Water supply represented the majority of projects (62), with only 24 subprojects in sanitation and sewerage and 2 subprojects had both water and sanitation works. 4.25 There were 18 subprojects (out of the 88) with an institutional-strengthening component, which primarily dealt with the institutional reform of the utilities, and only four included explicit subprojects focused on capacity building. Outcomes 4.26 According to figures provided by the ministry, about 1.4 million people had been served by the upgraded civil works provided by the project. 4.27 The urban access subobjective was achieved. The targets9 set for access to urban water services (against a target of 800,000) were exceeded—1,115,766 people received improved access to water services. The urban sanitation subobjective was not achieved. Only 264,973 people—against a target of 400,000—had improved sewerage. According to the data provided, only 25,000 (against the targeted 60,000) new consumers were connected to sewerage and 47,000 (against a target of 40,000) new consumers were connected to water. The rural WSS subobjective was achieved. The targets set for access to rural WSS were exceeded; 33,547 people (against a target of 20,000) received improved or new access to services. (See table 16). Table 16: Household Beneficiaries of All Projects Number of Baseline, Project completion, Target, subprojects 2004 2012 2010 Rural WSS 14 na 33,547 20,000 Urban Water 51 na 1,115,766 800,000 Sanitation 23 na 264,973 400,000 Total 88 Sources: World Bank 2012; Post-Completion Evaluation (OTSCORP 2012). na = No baseline defined at appraisal. 9 There were no baselines presented at project appraisal. 44 4.28 However, the IEG review of 24 subprojects found that, overall, the targets for two of the three sub-objectives were not achieved.10 There was an improvement in the urban water supply sub-objective, which is explained by four urban water services projects that finalized construction and started operations after project completion, resulting in the overall achievement of expected urban water services targets. In contrast, neither the urban sanitation nor the rural WSS subprojects met expected targets (table 17). These results were also corroborated with the results of surveys that were done randomly around the area of project influence. Table 17: Household Beneficiaries of IEG-Selected Projects Baseline, Project completion, Target, Actual, Number of subprojects 2004 2012 2010 2016 Rural WSS 6 na 2,307 6,670 2,842 Urban Water 14 na 67,385 89,052 140,890 Sanitation 4 na 3,108 12,314 9,329 Total 24 Sources: World Bank 2012; Post-Completion Evaluation (OTSCORP, 2012), data as of March 2016, collected during field visit. na = No baseline defined at appraisal. 4.29 Because the project had not defined which indicators signified improved access, there was no indication that the upgraded civil works led to reliable and safe water supply and sanitation. IEG’s field visits also looked at coverage and continuity of water supply of WSS utilities. The study found that for five of seven subprojects (for which there was information available), there was an improvement in water supply coverage of 30 percent by project completion, which resulted in six subprojects reporting between 90 and 100 percent coverage, and one reporting 80 percent. 4.30 Regarding the evolution of continuity of water supply (measured in percentage of hours per day), the study found information for 12 of the 19 projects that had a water supply focus. While there was an improvement in both rural and urban projects at project completion, results are in line with improvement shown by the small private operators in the WSRAP project, but still below the norm of 100 percent. Improvement continued in most of the cases through March 2016. Efficiency 4.31 Efficiency is rated modest. While overall the economic rate of return (ERR) calculated at approval was higher than estimated, the calculations are not comparable and represent only about 8 percent of the total number of subprojects (88) implemented. 10 Methodology for sample selection, limitations, and the questionnaire of the IEG survey are detailed in appendix C. 45 Therefore, the characteristics of the water schemes in these subprojects are not generalizable for the whole project. 4.32 Comparison is not possible, because the ex-ante and ex post ERRs were calculated based on different criteria. The PAD calculated the estimated ERR for a sample of six subprojects, all for water supply, in three urban and three rural localities. The estimates were derived from actual calculations of subprojects that had already been approved. The estimated returns for each of these six projects ranged from 16.6 percent to 203 percent. The ICR calculated the ex post return on investment based on a sample of seven subprojects, five urban and two rural. They included one sewerage, one water and sewerage, and five water supply subprojects. Except for one project that had an estimated return of 5 percent due to higher costs per connection than expected, the estimated return of these projects ranged from 23 to 57 percent. According to the ICR calculations (World Bank 2012, p. 40), the ERR for the sample of subprojects was 27 percent. 4.33 In addition, the subprojects were expected to take 16 months to complete. Instead, they took 44 months on average, and the overall project closed 23 months behind schedule. The delay was caused by a combination of issues, including difficulties of obtaining right-of- way permits (for easements), land acquisition, and resettlements in accordance with social safeguards. Other issues included logistical problems in completing a large number of subprojects scattered across the country, the need to raise counterpart funding brought about by a 14 percent increase in project costs above the appraisal estimate, weak implementation capacity at the local level, and complex implementation arrangements. As a result of these factors, fewer subprojects were completed than originally planned. Ratings OUTCOME 4.34 The overall outcome rating is moderately unsatisfactory. Relevance of objectives is rated substantial, Relevance of design is rated modest. The rating for the achievement of the PDO for “improved provision and access to water supply and sanitation” is modest. Efficiency is rated modest due to weak evidence. RISK TO DEVELOPMENT OUTCOME 4.35 The risk to development outcome is rated substantial because of the weak institutional capacity at the local level and the lack of information to assess the financial capacity to maintain the WSS systems over the longer term. In addition, the government ownership risk increased significantly when the new PDA policy put in place caused implementation delays. 4.36 Institutional risk: Government’s efforts to provide water and sanitation services to underserved populations in smaller communities has not been sufficiently followed up at the local level, partly due to low capacity and skills at the municipal level. 4.37 Financial risk: The IEG mission did not find sufficient information on operations and maintenance (O&M) expenditures to permit an estimate of the risks that the O&M expenditures might not be covered. A key assumption at appraisal was that the creation of 46 autonomous public utility companies, involving, where feasible, private sector participation, was expected to bring greater efficiency and sustainability to the sector. However, the IEG mission did not find sufficient evidence to establish whether—or to what degree—the 53 municipalities that undertook institutional reforms achieved the expected improvement in the operational efficiency of their utilities, which were to be measured in terms of lower unaccounted-for water (UFW), a higher bill-collection ratio, and reduced operating costs. Few utilities adopted these criteria as a condition for receiving funding. Combined with the weak M&E framework at the subproject level, this meant that these indicators were not measured in 50 of the 53 municipalities. 4.38 The IEG mission found that the proportion of subprojects measuring UFW was still low in 2016. In the three municipalities that did track UFW at project closure, water loss was reduced by 9 percent. Municipalities funded by the project did not adopt achievement of collection ratio targets as a condition for obtaining funding, although a review of 11 subprojects found little improvement in collection ratios. The IEG mission found that of nine subprojects, four improved between 27 to 80 percent, three worsened between 9 and 30 percent, and two remained the same. This indicates the high uncertainty about the financial sustainability of the water supply and sanitation schemes constructed under the project. 4.39 Government Ownership Risks: Support to improve access to water supply and sanitation through a three-phased water sector APL was weakened after the MAVDT adopted the new policy of using PADs. The project’s APL programmatic approach was abandoned in 2008. The changes have affected the sector program, because the second- and third-phase APLs were cancelled. Moreover, there are six subprojects under the project that have entered into a PDA, and it is expected that more municipalities involved in the project will become part of this framework. However, despite the strong policy support, the PDA is confronting its own challenges in successfully consolidating and achieving economies of scale. BANK PERFORMANCE 4.40 Quality-at-entry is rated moderately unsatisfactory. The project design was complex and had a multiplicity of players, and the design did not provide for adequate support to the weaker municipalities and local governments. Issues surrounding land acquisition and easement were not factored into the project design, and this turned out to be one of the main reasons for the implementation delays. The results framework did not specify indicators for the overall project and for each subproject, resulting in weak monitoring of the project. Furthermore, the programmatic approach designed was abandoned in 2008, because the national approach was no longer suitable when MAVDT adopted the new policy of the PDAs. 4.41 Supervision is rated moderately unsatisfactory. The IEG mission found that the Bank provided adequate support to the ministry and to FONADE, especially with regard to procurement and financial management. However, the Bank could have done more to assist the ministry in helping to strengthen capacities at the local level, as well as to develop an appropriate M&E system, with a baseline, meaningful target, and efficiency and service quality indicators at the subproject level, as well as with the collection of data. 47 4.42 The Bank undertook 12 supervision missions over the course of the 6 years of project implementation. Initially, insufficient attention was given to safeguard issues and the need for more technical assistance at the municipal level. Following the mid-term review, social remedial action plans were jointly developed by the Bank and the ministry for subprojects involving resettlement and indigenous peoples. However, the Resettlement Action Plan was not approved until April 2010, resulting in unresolved resettlement and easement issues in a number of subprojects at project closure. 4.43 Overall Bank performance is rated moderately unsatisfactory. BORROWER PERFORMANCE 4.44 Government performance is rated moderately satisfactory. Commitment from all levels of government to the project was high, and the sector ministry was actively engaged in the project design and in overseeing overall project implementation. With FONADE overseeing the overall governance and administration of funds to the municipalities and the procurement and management of independent consultants, the ministry was free to focus on the technical aspects of the project. The ministry engaged closely with FONADE and the municipalities, and actively participated in resolution processes (including those on environmental and social safeguards) that were carried out during construction. Without this level of engagement, the substantial volume of public works would not have been achieved. Ministry engineers systematically reviewed subprojects and conducted site visits to validate applications submitted to the Ventanilla Unica. The sheer numbers of applications (approximately 550) and the geographical dispersion made it a challenging task. The ministry also adopted a series of resolutions and decrees to strengthen private sector participation in the water sector. 4.45 Together with the Bank, the ministry supported the establishment of the Ventanilla Unica, created by statute to be a one-stop, technical screening body for subproject applications from the municipalities. It functioned well, and the ministry institutionalized its use to all water and sanitation subprojects in the country. 4.46 The ministry could have played a stronger role in guiding the strengthening of the capacity of municipalities in the design and engineering studies for the subprojects and in the development of an adequate M&E system with baseline and quality indicators for the subprojects. 4.47 Implementing agency performance is rated moderately satisfactory. FONADE was the implementing agency. With a solid reputation for managing development funds in a variety of sectors, and avoiding political and elite capture, it put in place a solid governance structure with a high level of transparency and accountability. It had the confidence of all stakeholders. The IEG mission found that FONADE executed its responsibilities satisfactorily. Independent project supervision contractors hired by FONADE played a fiduciary role in ensuring the funds were delivered to the municipalities and in the approval of the contractors. They played an important role in ensuring that the subprojects were 48 executed in accordance with adequate engineering standards. FONADE also maintained a website providing details about ongoing projects, which enhanced transparency. 4.48 At the same time, there were capacity issues that caused implementation delays—for example, safeguard requirements were stipulated for subprojects, but the municipalities did not have the capacity to apply them. Other capacity issues included: the need for technical redesign of subprojects; resolution of land acquisition issues and compliance with social safeguards; management of construction works; and implementation of institutional reform of water and sanitation utilities. The MAVDT and, to a lesser extent, FONADE were assigned to provide capacity building. There were 18 subprojects (out of the 88) with an institutional-strengthening component that mostly dealt with the institutional reform of the utilities, and only 4 of these made explicit the additional capacity building provided. 4.49 Due to the implementation delays and problems in complying with the Bank’s social safeguards policies, the implementation agency performance is rated as moderately unsatisfactory. 4.50 Overall borrower performance is rated moderately unsatisfactory. MONITORING AND EVALUATION 4.51 M&E Design: The IEG mission found that at the subproject level, M&E was weak, with poorly defined outcome indicators that were not well linked to the PDO. Due to the diverse geographical areas for subproject implementation, monitoring of the subprojects was highly dependent on local-level efforts. Indicators for water quality, continuity, and coverage were not mandatory, and municipalities frequently did not include these in their monitoring and reporting. The IEG mission found that the superintendence had now included such indicators in their current M&E system, but the data is incomplete because of the failure of some projects to report on them. 4.52 M&E Implementation: At the mid-term-review, some of the population numbers and targets were revised. A field verification of population data was conducted by the ministry at project closure, improving the reliability regarding the level of achievement of the first sub-objective. The project relied on M&E practices and systems used by the ministry and FONADE. The ministry tracked specific subproject results and number of beneficiaries (monitoring data was to be reported by the projects themselves to the superintendence). Output data on institutional strengthening activities for the municipalities and utilities were systematically collected by the ministry, while FONADE’s information management system (GEOTEC) tracked works contracts and financial flows. 4.53 M&E Utilization: FONADE’s management information system was made available on a public website with detailed accounts for each subproject. However, the IEG review did not find evidence of use of this information to modify implementation or relocate resources at a subproject level. M&E is rated Negligible. 49 5. Lessons 5.1 The key lessons derived, based on the findings of this PPAR, are the following: • Consolidation of smaller WSS operations servicing poorer neighborhoods can foster economies of scale and cross-subsidization in achieving financial sustainability at the aggregate level. The success stories of smaller operators relate to private sector operators in large urban areas extending their services to nearby small towns. In the case of the WSRAP, what worked was done through consolidation of smaller operations. The Triple A began operating in neighboring municipalities that were considered poor. To make service provision profitable, Triple A designed and proposed a regional tariff methodology aimed at cross-subsidizing service costs between municipalities served by the same provider. • Small municipalities with limited service coverage require large capital investments. Because of their lack of financial autonomy, government subsidies would be required until full cost recovery is achieved. In the case of large operators such as Cartagena, all costs were to be paid by users. Small operators normally receive a government contribution to cover investment costs (the user charges that they collect cover administration and maintenance costs). The contractual arrangements required private operators to invest in the supply systems to a limited extent, but the major investments had to be financed through capital subsidies from the government. In practice, because the investment needs of the small and medium-size operators were not adequately assessed, the capital investment received was insufficient, and in many cases it had to be renegotiated with the government. • It is crucial to phase and/or sequence project activities to ensure achievement of the objectives of utility operators. The WSRAP and WSSSP were simultaneously trying to build the capacity of operators with no operational capacity to immediately implement an expansion of WSS services and improve service quality. This can work in cases such as Cartagena, where the operator of the project, ACUACAR, was already experienced and ready to undertake activities to improve access and quality of services. Because it takes time to build capacities, one approach might be to move more slowly, emphasizing capacity strengthening first, and then to focus on implementation. Another approach could be to ensure the selection of an operator that has adequate experience and management capacity. • Management capacity of the utility can be strengthened by contracting a competent operator. Selection of operators for the water utility companies in WSRAP was based mainly on opting for the lowest cost for the government. However, the quality of the operator was an important omission in the bidding requirements. This led to a focus on the short-run financial benefit for the government rather than on selecting an operator with a minimum of experience and the quality capacities needed to run the water utilities. • A carefully designed M&E system and comprehensive planning is essential for success. The Cartagena Project included relevant and systematic design and implementation of M&E throughout the project period, while the WSRAP and WSSSP suffered from a relatively weak M&E framework and poor M&E implementation. In the 50 case of the WSSSP, the demand-driven selection process resulted in a piecemeal operation, with subprojects financing components or parts of an overall system. A carefully designed M&E system with specified required indicators for the different types of interventions would have been essential to enable adequate monitoring of the project outcomes. For small-scale, demand-driven projects, substantial capacity building in addition to careful M&E design is critical for success. 51 References Acuavalle, S.A. S.E.P. 2007. “Historia desde 1959 a 2003.” http://www.valledelcauca.gov.co/asamblea/publicaciones.php?id=9133 B. Marín, Garay, T., y A. Vélez. 2001. “Contaminación marino-costera en Colombia. Informe del estado de los ambientes marinos y costeros en Colombia. Programa Calidad Ambiental Marina” – INVEMAR. 101- 114. DANE (National Department of Statistics, Bank of the Republic of Colombia). 2014. “Economic and Financial Data for Colombia.” http://www.banrep.gov.co/estad/dsbb/imfcolom.htm SSSP (Superintendence of Domestic Public Services), DNP (National Planning Department). 2015. “Informe Sectorial los Servicios Públicos Domiciliarios de Acueducto y Alcantarillado.) DNP (National Planning Department). 2014. “Política para el suministro de Agua Potable y saneamiento Básico en la zona Rural.” DNP (National Planning Department). 2015. “Ley 1753 de 2015 por la cual se expide el Plan Nacional de desarrollo 2014-2018 “Todos por un Nuevo país” Various years, SIU (Unique system of information of the Superintendence of Residential Public Services) http://www.sui.gov.co/SUIAuth/logon.jsp CARDIQUE (Corporación Autónoma del Canal del Dique), 2015. “Concepto Técnico No. 447” IDB (Inter-American Development Bank). 2005. “Buenas Prácticas para la Creación, Mejoramiento y Operación Sostenible de Organismos y Organizaciones de Cuenca – Colombia.” ———. 2006. “When a Water Meter is Worth More than a House.” Webstories. http://www.iadb.org/en/news/webstories/2006-07-01/when-a-water-meter-is-worth-more- than-a-house,3250.html World Bank. 1997. Colombia—Country Assistance Strategy. Report No. 17107. Washington, DC: World Bank. ———. 1999a. “Colombia—Cartagena Water Supply, Sewerage, and Environmental Management Project.” Project Appraisal Document. Report No. 18989. World Bank, Washington, DC. ———. 1999b. “Conformed Copy—L4507—Cartegena Water Supply, Sewerage, and environmental Management Project—Loan Agreement.” Legal Department. World Bank, Washington, DC. ———. 2001a. “Colombia—Water Sector Reform Assistance Project.” Project Appraisal Document. Report No. 21868. World Bank, Washington, DC. ———. 2001b. “Conformed Copy–L7077–Water Sector Reform Assistance Project–Loan Agreement.” Legal Department. World Bank, Washington, DC. ———. 2002. Colombia—Country Assistance Strategy. Report No. 25129. Washington, DC: World Bank. ———. 2004. Colombia: Recent Economic Developments in Infrastructure (REDI): Balancing Social and Productive Needs for Infrastructure. Volume 2, Main Report. Report No. 30379. Washington, DC: World Bank. 52 ———. 2005. Colombia—Cartagena Water Supply, Sewerage, and Environmental Management Project.” Inspection Panel Report. Report No. 32034-CO. World Bank, Washington DC. ———. 2005a. “Conformed Copy–L7281–Water and Sanitation Support Project Project–Loan Agreement.” Legal Department. World Bank, Washington, DC. ———. 2005b. Colombia—Country Assistance Strategy Progress Report for the Period FY03– FY07. Report No. 32999. Washington, DC: World Bank. ———. 2005c. “Colombia—Water and Sanitation Sector Support Project.” Project Appraisal Document. Report No. 29601. World Bank, Washington, DC. ———. 2006. “Local Solutions Improve Water Supply and Sanitation Services in Colombia.” Water Supply and Sanitation Feature Stories. http://siteresources.worldbank.org/INTWSS/Resources/colombia.pdf ———. 2008. Colombia—Country Partnership Strategy. Report No. 42847. Washington, DC: World Bank. ———. 2009. “Public-Private Partnerships for Urban Water Utilities. A review of Experiences in Developing Countries”. World Bank, Washington, DC ———. 2010. “Colombia—Cartagena Water Supply, Sewerage, and Environmental Management Project.” Implementation Completion and Results Report. Report No. ICR1053. World Bank, Washington, DC. ———.2010. “Colombia—Cartagena Water Supply, Sewerage, and Environmental Management Project.” Implementation Completion and Results Review Report. Report No. ICRR. World Bank, Washington, DC. ———. 2011a. Colombia—Country Partnership Strategy for the Period FY2012–2016. Report No. 60620. Washington, DC: World Bank. ———. 2011b. “Colombia—Water Sector Reform Assistance Project.” Implementation Completion and Results Report. Report No. ICR965. World Bank, Washington, DC. ———. 2012. “Colombia—Water and Sanitation Sector Support Project.” Implementation Completion and Results Report. ICR 1316. World Bank, Washington, DC. ———. 2013. “Colombia—Water Sector Reform Assistance Project.” Implementation Completion and Results Report Review. Report No. ICRR13656. World Bank, Washington, DC. ———. Various years. World Development Indicators. http://data.worldbank.org/data-catalog/world- development-indicators OTSCORP, 2012. “Evaluación Final del Programa de Apoyo al Sector de Agua Potable y Saneamiento Básico” WHO/UNICEF (World Health Organization/United Nations Children’s Fund). 2010. “Joint Monitoring Program for Water Supply and Sanitation.” Coverage estimates for improved drinking water and coverage estimates for improved sanitation. http://www.wssinfo.org/ 53 Appendix A: Basic Data Sheet CARTAGENA WATER SUPPLY, SEWAGE, AND ENVIRONMENTAL MANAGEMENT PROJECT Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 116.32 128.32 110 Loan amount 85 85 100 Cofinancing 0 0 0 Cancellation - - - Cumulative Estimated and Actual Disbursements Fiscal year 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Appraisal 1.8 13.3 37.9 61.7 75.8 83.1 85.0 85.0 85.0 85.0 85.0 estimate (US$M) Actual (US$M) 0 9.2 9.2 16.2 25.1 31.6 39.1 49.3 62.9 71.9 85.0 Actual as % of 0 69 24 26 33 38 46 58 74 85 100 appraisal Project Dates Original Actual Initiating memorandum 11/29/1995 Board approval 07/20/1999 Signing 12/10/1999 Effectiveness 01/19/2000 Closing date 12/31/2004 06/28/2009 54 Staff Inputs (staff weeks) Staff time and cost (Bank budget only) US$ thousands (including travel Stage of project cycle Number of staff weeks and consultant costs) Lending FY96 8.01 FY97 1.67 FY98 82.33 FY99 125.86 FY00 13 43.18 FY01 0 FY02 0 FY03 0 FY04 0 FY05 0 FY06 0 FY07 0 FY08 0 Total: 13 261.05 Supervision/ICR FY96 0 FY97 0 FY98 0 FY99 2.21 FY00 8 45.37 FY01 9 59.82 FY02 19 103.22 FY03 21 99.42 FY04 13 73.34 FY05 9 70.8 FY06 10 72.57 FY07 8 84.67 FY08 9 60.33 FY09 6 28.7 Total: 125 961.5 Note: FY = fiscal year. 55 Team Composition Name Title Unit Lending Menahem Libhaber Task Manager (Sr. Sanitary Engineer) LCSUW Yoko Katakura Financial Analyst CSFDR Caroline Van Den Berg Economist ETWWA Franz Drees-Gross Water Supply and Sanitary Engineer BUEWB Juan David Quintero Sr. Environmental Specialist EASER Maria Elena Castro Social Specialist Livio Pino Financial Management Specialist Maria Victoria Lister Quality Assurance Officer Efraim Jimenez Sr. Procurement Specialist Jose Augusto Carvallho Sr. Counsel Issam Abousleiman Disbursement Officer Carl Bartone Peer Reviewer CONS Robert Taylor Peer Reviewer CIADR Silvia Delgado Staff Assistant LCSUW Martha Gonzalez Staff Assistant LCSUW Supervision/ICR Anna Wellenstein Sector Manager SACIA David N. Sislen Sector Leader LCSSD Menahem Libhaber Task Manager LCSUW Guang Z. Chen Sector Manager LCSUW Jeannette Estupinan Financial Management Specialist LCSFM Diana Ortiz Zuluaga E T Consultant LCSUW Laura Kullenberg Senior Operation Officer LCSSD Juan David Quintero Sr. Environment Engineer EASER Franz Drees-Gross Sector Leader LCSSD Marco Zambrano Consultant Greg Browder Task Manager LCSUW Carmen Yee-Batista CO-Task Manager LCSUW Luz Maria Gonzalez Consultant LCSUW Rosa Bellido Program Assistant LCSUW 56 WATER SECTOR REFORM ASSISTANCE PROJECT Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 70 81.54 116.5 Loan amount 40 35.69 89.2 Cofinancing 30 45.66 152.2 Cancellation - 4.31 - Cumulative Estimated and Actual Disbursements Fiscal year 2002 2003 2004 2005 2006 2007 2008 2009 Appraisal 1.7 10.5 23.5 34.0 38.5 40.0 40.0 40.0 estimate (US$M) Actual 0 1.5 3.9 11 16.9 26.9 34.7 35.7 (US$M) Actual as 0 14 17 32 44 67 87 89 % of appraisal Project Dates Original Actual Initiating memorandum 11/21/2000 Negotiations 06/05/2001 Board approval 10/25/2001 Signing 12.17/2001 Effectiveness 06/18/2002 Closing date 06/30/2007 10/31/2010 57 Staff Inputs (staff weeks) Staff time and cost (Bank budget only) Number of staff US$ thousands (including travel Stage of project cycle weeks and consultant costs) Lending (fiscal year) 2000 6.27 20.2 2001 24.03 56.5 2002 14.11 19.1 Total 44.41 96 Supervision/ICR (fiscal year) 2003 11.98 26.8 2004 12.96 42.3 2005 10.65 21.3 2006 13.24 23.3 2007 6.76 9 2008 4.58 36.8 2009 5.95 41.3 2010 12.82 12.1 2011 13.74 16.8 Total 92.68 230.2 58 Team Composition Name Title Unit Lending Menahem Libhaber Lead Water and Sanitation Specialist LCSUW Fernando Troyano, Private Sector Development Consultant Specialist Paula Pini Social and Community Specialist LCSUW Juan David Quintero Sr. Environmental Specialist LCSUW Maria Angelica Sotomayor Sr. Economist LCSUW Luz Maria Gonzalez Consultant Supervision/ICR Greg Browder Lead Water and Sanitation Specialist LCSUW Carlos A. Uribe Water and Sanitation Engineer LCSUW Eric Dickson Urban Economist LCSUW Juan Camilo Gil Consultant Jose Martinez Senior Procurement Specialist LCSPT Luz Maria Gonzalez Consultant Luz Zeron Financial Specialist LCSFM Carmen Yee-Batista Water and Sanitation Specialist LCSUW 59 WATER AND SANITATION SECTOR SUPPORT PROJECT Key Project Data (US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate Total project costs 93.8 107.1 114.8 Loan amount 70 70 100 Cofinancing 23.8 37.7 156 Cancellation - - - Cumulative Estimated and Actual Disbursements Fiscal year 2005 2006 2007 2008 2009 Appraisal estimate (US$M) 0 12 36.0 55 70 Actual (US$M) 0 0 24.6 70 70 Actual as % of appraisal 0 0 68 127 100 Project Dates Original Actual Initiating memorandum 12/16/2003 Negotiations Board approval 03/22/2005 Signing 06/02/2005 Effectiveness 08/31/2005 Closing date 04/30/2009 3/31/2011 60 Staff Inputs (staff weeks) Stage of project cycle Staff time and cost (Bank budget only) Number of US$ thousands (including staff weeks travel and consultant costs) Lending (fiscal year) 2003 3 16.75 2004 37 180.28 2005 36 226.63 Total 76 423.66 Supervision/ICR (fiscal year) 2004 -0.65 2005 1 19.94 2006 13 63.79 2007 16 126.72 2008 34 168.54 2009 18.18 98.92 2010 29.12 153.63 2011 17.91 78.42 2012 3.85 26.03 Total 133.06 735.34 61 Team Composition Name Title Unit Lending Oscar E. Alvarado Sr Water & Sanitation Spec. SASDU Luiz C. Gazoni Consultant LCSFM Menahem Libhaber Consultant LCSUW Patricia Lopez Martinez Sr Financial Analyst LCSUW Luis M. Schwarz Senior Finance Officer CTRFC David N. Sislen Sector Leader LCSSD Cristina Velazco-Weiss Program Assistant MDD Supervision/ICR Ana Maria Aristizabal Consultant LCSUW Daniel J. Boyce Country Program Coordinator SACNA Greg J. Browder Lead Water and Sanitation Spec LCSUW Claudia Mylenna Cardenas Garcia Consultant LCSFM Jeannette Estupinan Financial Management Specialist LCSFM Joseph Paul Formoso Lead Finance Officer CTRDM Natalie Giannelli Junior Professional Associate LCSUW Clara Hortensia Gomez Hernandez Consultant LCSUW Henry Laino Consultant LCSUW Menahem Libhaber Consultant LCSUW Patricia Lopez Martinez Sr Financial Analyst LCSUW Jose M. Martinez Senior Procurement Specialist LCSPT Jesus Martinez ET Consultant LCSPS Alejandro Meleg Consultant LCSUW Jean-Roger Mercier Consultant IPN Luz Meza-Bartrina Sr. Counsel LEGAF Kirsten L. Oleson Environmental Spec. LCSEN Diana Ortiz Zuluaga Consultant LCSUW Gabriel Penaloza Procurement Analyst LCSPT Kennan W. Rapp Social Development Spec. LCSDE Ernesto Sanchez-Triana Lead Environmental Specialist SASDI Luis M. Schwarz Senior Finance Officer CTRFC Carlos A. Uribe ET Consultant LCSUW Carlos Vargas Bejarano Consultant LCSUW Meike van Ginneken Sr Water & Sanitation Spec. AFTUW Patricia Acevedo L Language Program Assistant LCSUW 62 Appendix B: Operational and Financial Data of Water Sector Reform Assistance Project Water Supply Coverage (%) By Municipality Baseline 2010 2013 or 2014 Target Constructor operator 0.64 0.66 0.77 na Calamar 0.78 0.82 0.88 na Mahates 0.71 0.72 0.85 na San Estanislao de Kostka 0.93 0.92 0.72 na San Onofre 0.84 0.83 0.86 na Santa Rosa 0.00 0.00 0.75 na Soplaviento 0.81 0.86 0.89 na Villanueva 0.42 0.44 0.47 na Operator with investment 0.62 0.84 0.89 0.95 Arjona 0.61 0.63 0.65 0.90 Baranoa 0.65 0.96 1.00 0.95 Cerete 0.81 0.91 0.92 1.00 Cienaga de Oro 0.85 0.84 0.87 1.00 Corozal 0.73 0.83 0.99 0.96 El Banco 0.40 0.95 nm 0.90 Magangué 0.53 0.64 0.65 na Maicao 0.41 0.67 0.81 0.95 Polonuevo 0.78 0.92 1.00 0.95 Sabanagrande 0.87 0.93 0.97 0.95 Sahagun 0.85 0.93 0.98 1.00 San Carlos 0.89 0.89 0.89 1.00 San Juan de Nepomuceno 0.39 0.80 nm 0.96 San Marcos 0.21 0.99 0.92 0.90 Santo Tomas 0.51 0.88 1.00 0.95 Sincelejo 0.80 0.83 0.99 0.98 Soledad 0.60 0.96 0.98 0.94 Turbaco 0.25 0.64 0.67 0.90 Grand Total 0.62 0.79 0.86 0.95 63 Sewage Coverage (%) By Municipality Baseline 2014 Target Constructor operator 0.00 0.09 na Calamar na na na Mahates na na na San Estanislao de Kostka na na na Santa Rosa na na na Soplaviento na na na Villanueva na na na San Onofre 0.00 0.09 na Operator with investment 0.32 0.42 0.88 Arjona 0.00 0.00 0.70 Baranoa 0.00 0.14 na Cerete 0.55 0.55 1.00 Cienaga de Oro 0.56 0.48 1.00 Corozal 0.74 0.97 0.84 El Banco 0.24 0.37 na Magangué 0.00 0.16 na Maicao 0.33 0.73 na Polonuevo 0.00 0.00 0.85 Sabanagrande 0.65 0.88 0.85 Sahagun 0.74 0.83 1.00 San Carlos 0.00 0.00 1.00 San Juan de Nepomuceno 0.22 0.00 0.96 San Marcos 0.00 0.00 0.75 Santo Tomas 0.40 0.78 0.85 Sincelejo 0.74 0.96 0.90 Soledad 0.55 0.73 0.90 Turbaco 0.00 0.00 0.70 Grand Total 0.32 0.42 0.88 Working Ratio Operator Contract Type Municipality Año Inicio Inicio 2009 o 2010 2013 o 2014 Target Operagua el Banco Operador con inversión El Banco 2003 nm nm nm nm Soplaviento Santa Rosa Giscol SA Constructor operador 2008 0.92 0.77 0.93 0.90 Villanueva San Estanislao de Kostka Calamar nd nd nd Giscol Dique SA Constructor operador 2009 0.92 Mahates nd nd nd Aquaseo Operador con inversión Magangué 2007 nd nd 1.11 nd Arjona 1.45 0.87 Acualco Operador con inversión 2004 1.19 0.70 Turbaco 0.87 0.94 Aguas de la Costa Operador con inversión San Juan de Nepomuceno 2002 nm nm nm nm Baranoa 1.55 1.37 2003 Polonuevo 2.44 1.54 Triple A Operador con inversión Sabanagrande 1.89 1.40 0.75 0.60 2002 Santo Tomas 2.20 1.78 Soledad 2002 0.77 0.80 Cerete nd nd Cienaga de Oro nd nd Uniaguas Operador con inversión 2004 0.90 0.60 San Carlos nd nd Sahagun nd nd Aguas de la Peninsula Operador con inversión Maicao 2001 0.69 0.97 1.57 0.60 Aguas de la Mojana Operador con inversión San Marcos 2002 nd nd 0.82 nd Insergrup Constructor operador San Onofre 2006 0.94 0.90 0.65 0.80 Sincelejo 0.86 0.63 Aguas de la Sabana Operador con inversión 2003 0.69 0.60 Corozal 1.06 1.15 64 Operating margin Operator Contract Type Municipality Año Inicio Inicio 2009 o 2010 2013 o 2014 Operagua el Banco Operador con inversión El Banco 2003 nm nm nm Soplaviento 2008 Santa Rosa Giscol SA Constructor operador 8% 23% 7% Villanueva San Estanislao de Kostka Calamar 2009 nd nd Giscol Dique SA Constructor operador 8% Mahates nd nd Aquaseo Operador con inversión Magangué 2007 nd nd -11% Arjona 2004 -45% 13% Acualco Operador con inversión -19% Turbaco 13% 6% Aguas de la Costa Operador con inversión San Juan de Nepomuceno 2002 nm nm nm Baranoa 2003 -55% -37% Polonuevo -149% -54% Triple A Operador con inversión Sabanagrande 2002 -89% -40% 25% Santo Tomas -120% -78% Soledad 2002 23% 20% Cerete 2004 nd nd Cienaga de Oro nd nd Uniaguas Operador con inversión 10% San Carlos nd nd Sahagun nd nd Aguas de la Peninsula Operador con inversión Maicao 2001 31% 3% -57% Aguas de la Mojana Operador con inversión San Marcos 2002 nd nd 18% Insergrup Constructor operador San Onofre 2006 6% 10% 35% Sincelejo 2003 14% 37% Aguas de la Sabana Operador con inversión 31% Corozal -6% -47% Net margin Operador Tipo Contrato Municipio Año Inicio Inicio 2009 o 2010 2013 o 2014 Operagua el Banco Operador con inversión El Banco 2003 nm nm nm Soplaviento Santa Rosa Giscol SA Constructor operador 2008 1% 22% 7% Villanueva San Estanislao de Kostka Calamar nd nd Giscol Dique SA Constructor operador 2009 7% Mahates nd nd Aquaseo Operador con inversión Magangué 2007 nd nd -7% Arjona -59% 5% Acualco Operador con inversión 2004 18% Turbaco 5% -3% Aguas de la Costa Operador con inversión San Juan de Nepomuceno 2002 nm nm nm Baranoa -56% -38% 2003 Polonuevo -151% -56% Triple A Operador con inversión Sabanagrande -90% -40% 13% 2002 Santo Tomas -121% -79% Soledad 2002 5% 5% Cerete nd nd Cienaga de Oro nd nd Uniaguas Operador con inversión 2004 17% San Carlos nd nd Sahagun nd nd Aguas de la Peninsula Operador con inversión Maicao 2001 2% 2% 4% Aguas de la Mojana Operador con inversión San Marcos 2002 nd nd 12% Insergrup Constructor operador San Onofre 2006 0% 0% 42% Sincelejo -3% 19% Aguas de la Sabana Operador con inversión 2003 32% Corozal -26% -91% 65 Bill Collection Ratio (%) Operador Tipo de Contrato Inicio 2009-10 2013-14 Constructor operator na 37% 69% Giscol SA Constructor Operator na 26% 28% Giscol Dique SA Constructor Operator na na 88% Insergrup Constructor Operator na 48% 89% Operator with Investment 24% 75% 86% Aquaseo Operator with Investment na na 74% Acualco Operator with Investment 0% 95% 100% Aguas de la Costa Operator with Investment na na na Triple A Operator with Investment 10% 92% 93% Uniaguas Operator with Investment 0% 65% 93% Aguas de la Peninsula Operator with Investment na 16% 89% Aguas de la Mojana Operator with Investment 70% 92% 60% Operagua el Banco Operator with Investment na na na Aguas de la Sabana Operator with Investment 40% 91% 94% Total Average 24% 67% 81% 66 Appendix C: Water and Sanitation Support Project Methodological Approach Introduction As part of the PPAR exercise, the IEG field mission for the Water and Sanitation Sector Support Project (WSSSP) visited a number of subprojects and conducted interviews. In addition, the mission aimed to obtain feedback from the beneficiaries of the water and sanitation support provided by the project through a survey. The team interviewed a random sample of 720 households (equivalent to 30 households around the area of influence) using a questionnaire (attached). Limitation Eighty-seven subprojects received World Bank Group support, but some detailed information, such as achievement of objectives and assessment of completion, was only available for 48 subprojects for which an assessment was done at the time of project completion. Sample Selection Of the 48 subprojects for which a follow-up review was done at project completion (2010), a stratified sample of households was taken according a typology of the projects by urban or rural location and type of activity, as well as the degree of homogeneity of projects in the different geographical areas. The PPAR team sought a fair representation of all the project typologies by location (rural, urban) and type of support (water, sanitation). A random selection was done for each of the categories, resulting in the number of projects for each category listed in table C.1. Table C.1: Stratified Sample of 24 Projects Water Water supply and supply Sewage sewage Total Rural 12 1 1 14 Urban 50 23 1 74 Total population 62 24 2 88 Rural 7 1 8 Urban 30 10 0 40 Subsample reviewed at 38 10 1 48 project closure Rural 5 1 6 Urban 14 4 0 18 Subsample of reviewed 19 4 1 24 by IEG Sample Selection of Households for Survey To select the number of households, it was initially determined that 30 households corresponds statistically to the minimum size of sample required for each project in its area of influence. The formula used to calculate the simple size: 67 Where: n = sample size, z = confidence level, P= probability of occurrence, Q = complement of P, ɛ = expected error, N = size of the universe z2 2 PQ n = ε z 2 PQ 1+ ε 2N Using the parameters of the table below, we obtained a size of 30, which should be used under the assumption that population is small and homogenous, and that it is known the event occurred. Parameters Confidence level 95% 1,96 Expected error 10,0% P 90% q 10% N 250 Sample size 30 In theory, this corresponds to a random systematic sample. However, in practice the selection was not done with the rigor needed, because it required a recount of the households in the area of influence. The team sought to survey households in different sectors around of the area of influence. 68 Households Questionnaire 69 Appendix D: List of Persons Met Name Designation Organization Government officials Julio Cesar Aguilera Wilches Expert Commissioner Regulatory Water Commission (CRA) Mauricio Santa Maria Director National Planning Department (DNP) Beatrice Giraldo Contractor of Water and National Planning Department Sanitation (DNP) Diego Polania Subdirector de Agua y National Planning Department Saneamiento (DNP) Diego Fernando Rojas Lara Coordinator-Operator Ministry of Environment, Housing Management and Regional Development (PMA)(MAVDT) Jorge Silva Coordinator-Operator Ministry of Environment, Housing Management and Regional Development (MAVDT) Edilma Nieto Advisor Ministry of Environment, Housing and Regional Development (MAVDT) Guillermo Andrés Arcila Hoyo Subdirector PME Ministry of Housing (MINVIVIENDA) Luis Fernando Ulloa Vergara Vice President of Regional Development Financing Projects Institution FINDETER Jorge Enrique Angel Advisor Regional Development Financing Institution FINDETER Diana Tabera Manager Regional Development Financing Institution FINDETER Luis Fernando Ramos Director Superintendence (SSPD) Private operators John Montoya Canas General Manager Aguas de Cartagena SA. ESP Jose Eliecer Zapata Pinedo Head of Environmental Aguas de Cartagena SA. ESP Management Wilmer Corpas Romero Coordinator of Punta Aguas de Cartagena SA. ESP Canoa treatment plant Miguel Enrique Alfonso Fleites Project Manager Aguas de Cartagena SA. ESP Gregorio Otero Revollo Project Manager Aguas de Cartagena SA. ESP Carlos Uribe Z. Director Acualco, Arjona y Turbaco Edgardo Burgos General Manager Uniaguas SA ESP Jairo Tatil Users’ association rep. Uniaguas “vocal de control” Fabio Aranque de Avila Director Aguas de la Sabana Juan Carlos Salgado B. Manager Aguas de la Sabana Alma Ramos Director of Aguas de la Sabana Communications Jorge Carillo Velilla Technical Manager Aguas de la Sabana 70 Juan Pablo Nunez Planning Manager William Tabares Poroa Director Aguas de la Sabana Andres Manuel Pertuz Hurtado Users’ association rep. Aguas de la Sabana “vocal de control” Municipalities Elizabeth Guardo Lazo Secretaria de la Arjona Alcaldeza Yorio Yoniero Yamayo Advisor Arjona Dario Romero Planning Manager Arjona Jacobo Quissep Espinosa Major Sincelejo Vice-alcaldeza Cartagena World Bank Group Staff Greg Browder Task Team Leader Menahem Liebhaber Task Team Leader Carlos Uribe Resident Sector Expert (retired) Luis Alfonso Pinzon Environmental Specialist Carmen Yee Batista ICR Author Phillipe Marin Book Author Issam A. Abousleiman Country Manager