ICRR 13656 Report Number : ICRR13656 IEG ICR Review Independent Evaluation Group 1. Project Data: Date Posted : 05/16/2013 Country : Colombia Project ID : P065937 Appraisal Actual Project Name : Water Sector Reform US$M ): Project Costs (US$M): 70 81.54 Assistance Project L/C Number : L7077 Loan/ US$M ): Loan /Credit (US$M): 40 35.68 Sector Board : WAT US$M): Cofinancing (US$M ): 30 45.86 Cofinanciers : Board Approval Date : 10/25/2001 Closing Date : 06/30/2007 10/31/2010 Sector (s): Water supply (49%); Sanitation (49%); Central government administration (2%) Theme (s): Other urban development (25% - P); Access to urban services and housing (25% - P); Other financial and private sector development (24% - P); Environmental policies and institutions (13% - S); Rural services and infrastructure (13% - S) Prepared by : Reviewed by : ICR Review Group : Coordinator : Midori Makino Robert Mark Lacey Soniya Carvalho IEGPS1 2. Project Objectives and Components: a. Objectives: The Project Development Objectives (PDO) as stated in the Loan Agreement (LA) Schedule 2 were: "(a) to support water sector reform in the Borrower’s territory by facilitating an increased participation of the private sector in the management and operation of Water Utilities, with the purpose of (i) creating and maintaining an appropriate environment for improving the efficiency and sustainability of such Water Utilities; and (ii) providing Participating Municipalities, which choose to participate in the Project, with financial support to ensure the viability of their Water Utilities; (b) to expand the coverage of water supply and sewerage services provided in Participating Municipalities; (c) to facilitate the access to water and sewerage services by the population of low income areas in such Participating Municipalities; and (d) (i) to improve environmental protection practices in connection with the provision of water and sewerage services in the Borrower ’s territory; and (ii) to define rural water and sanitation policies, and to develop adequate methods for increasing coverage, both in connection with water and sewerage services in the Borrower’s rural sector". The PDO as stated in the Project Appraisal Document (PAD) page 1 were: "(i) to support water sector reform by facilitating the incorporation of the private sector into the management and operation of water utilities participating in the project for the purpose of creating an enabling environment for improving the efficiency and sustainability of these utilities; and by providing them with financial support to ensure the viability; (ii) expand the coverage of water and sewerage services provided in participating municipalities; and (iii) facilitate the access of populations in low income areas of participating municipalities to water and sewerage services ". This Review uses the objectives as stated in the LA as these contain a fourth set of objectives (improvement of environmental protection practices and definition of rural water and sanitation policies ) which was not included in the PAD's statement. b.Were the project objectives/key associated outcome targets revised during implementation? No c. Components: The project had four components : Component A . Investments in Sub -projects in medium size cities and small municipalities (original cost: $61.8 million, actual cost: $75.55 million). This component (with five sub-components) included: (a) loans to finance partially the public sector funding requirements of the water and sanitation infrastructure in two to three medium sized cities and 15 small municipalities in the Caribbean region, and fund medium size cities and small municipalities located outside of the Caribbean region and the execution of respective works . The loan amount allocated to each city that is partially funded would be based on a combination of the financial model developed during the preparation of the respective private sector participation process as well as what is requested by the winning bidder . The participation of the cities and municipalities was demand driven, and based on certain eligibility criteria . (b) Execution of works financed by the private sector and by Law 60 resources (Law 60 stipulates that 60 percent of decentralized social expenditures, including water, be distributed to municipalities on the basis of the number of people with unsatisfied basic needs) in medium size cities and in small municipalities . As part of the Private Sector Participation (PSP) transaction, participating municipalities would be requested to commit a portion of their Law 60 resources to contribute to the works. Component B . Sector environmental management capacity strengthening (original cost: $0.8 million, actual cost: $0.38 million): (i) providing advice on modifying and strengthening key environmental regulations, especially the existing water quality and effluent discharge regulations; (ii) establishing sector wide criteria for the environmental management of water and sanitation projects (siting, design, construction and operation ), completing the existing sector technical specifications; (iii) defining environmental requirements to be included in bidding documents and private sector operation contracts; (iv) defining methodologies and programs for environmental audits of water and sanitation facilities; (v) strengthening the environmental management capabilities of the sector as a whole, especially the Directorate of water and Sanitation in the Ministry of Economic Development; (vi) providing advice on streamlining the environmental licensing procedures for water and sanitation projects in regional environmental agencies; (vii) sponsoring training programs on environment for water and sanitation system operators; and (viii) defining public participation and community consultation criteria and procedures for water and sanitation projects . Component C. Rural water and sanitation policy development (original cost: $0.9 million, actual cost: $0.21 million): technical assistance to support the development of the rural water and sanitation sector policy, strategy and methods for incrementing water and sanitation coverage in the rural sector . Component D . Project management and training (original cost: $6.5 million, actual cost: $5.4 million): project management and training; technical assistance to carry out the processes of incorporating the private sector in the management and operation of the water and sanitation utilities in medium size cities; and technical assistance to carry out the processes of incorporating the private sector in the management and operation of the water and sanitation utilities in about 25 small municipalities. d. Comments on Project Cost, Financing, Borrower Contribution, and Dates: As a result of the slow project start -up, US$4.31 million of the budgeted government resources were cancelled from the loan. The actual borrower (Government of Colombia) contribution was US$45.86 million, 153 percent of the appraisal estimate of US$30 million. With regards to the sub-projects carried out under the project, out of the total US$ 76 million, 78 percent or US$60 million were funded by IBRD and the government of Colombia (US$30 million each), and the remaining 21 percent or US$16 million were funded by the private sector . The project closing date was extended from June 30, 2007 to October 31, 2010 through three extensions primarily due to two sub-projects that experienced significant delays . In Magangue, the selected operator caused significant delays in implementation, and was replaced by a new operator in 2010. La Linea sub-project in Bolivar also experienced delays in the installation of a critical pipeline due to complications with a parallel highway construction project. 3. Relevance of Objectives & Design: a. Relevance of Objectives: The Country Assistance Strategy at the time of project appraisal in 1997 had the objective of achieving sustainable growth by improving infrastructure services, and both the 2003 and 2008 Country Assistance Strategies supported infrastructure, in particular, drinking water, sewerage, and wastewater treatment for expanded access to the poor under the sustained equitable growth pillar . The most recent Country Partnership Strategy, prepared in June 2011 focused on sustainable growth with enhanced climate change resilience as one of its three pillars, and included adoption of sound water sector policies as an area in which the Government of Colombia has been a leader . It specifically identified the remaining challenges to improve multi -city service quality, access in rural areas, and wastewater management. The third pillar of the Strategy, - inclusive growth with enhanced productivity -, included a focus on public-private partnerships in infrastructure sectors . The objectives are, moreover, consistent with the Country Assistance Strategies' and Country Partnership Strategy's emphasis and goals of supporting rural and small cities water supply, with a special focus on private sector participation. Bank and government strategy has also consistently emphasized the need for enhanced protection and management of environment. The project objectives therefore remain relevant to the country's priorities . Relevance of objectives is rated as substantial . b. Relevance of Design: The results framework prepared at project appraisal adequately identified the linkages between the objectives, outputs, intermediate outcomes, and the final impacts . The causal chain between the activities funded under the project and the outcomes to be achieved was generally clear and convincing, except for the objective related to targeting the lower income populations in the participating municipalities, which did not have a related output in the project design. The first objective related to the creation of an enabling environment for improving the efficiency and sustainability of the utilities included components to pilot private sector participation and related sector reform activities. The second and third objectives related to water and sanitation access were to be attained through the associated investments such as network extensions and reduction in accounted for water . The fourth objective included development of a decision making tool for wastewater investments . The project was flexibly designed to allow for a larger number of contracts and investments carried out for the medium sized cities than originally planned because of the lower demand from smaller municipalities . Overall relevance of design is rated as substantial . 4. Achievement of Objectives (Efficacy): The statement of objectives contains a mix of outputs and intended outcomes . The review will assess the achievement of the four main intended outcomes – efficiency/sustainability, extended coverage, access of the poor, and environmental protection – while the components, including increased private sector participation, form part of the results chain to achieve those outcomes . 1. Creating and maintaining an appropriate environment for improving the efficiency and sustainability of Water Utilities : Substantial . Outputs Number of Private Sector Participation contracts signed : For the medium sized cities, the project had the target of signing two PSP operations with investment contracts, but exceeded this target and signed 11 PSP contracts with 18 municipalities. These contracts were signed during the early phase of the project . For the small municipalities the project only signed 3 PSP construction-with-operation contracts with 7 municipalities against the target of signing 15 PSP contracts. Under the overall Corporate Modernization Program of the Ministry of Environment, Housing, and Regional Development, 36 municipalities in the Caribbean region now have private operators, and 25 of these municipalities were supported under the World Bank funded project . This corresponds to more than a third of the cities in the Caribbean region . Creation of an appropriate policy environment : According to the ICR, the project played an instrumental role in strengthening the Ministry of Environment, Housing, and Regional Development's Corporate Modernization Program and helped lay the ground work for the national policy on the Departmental Water Plan program . During the course of the Project a number of legislative reforms and executive decrees were adopted to improve the enabling framework and sustain the reforms . By 2007, the national government created the Departmental Water Program (CONPES Policy Paper 3463) which channels national government funds for water and sanitation through the department and requires municipalities to ensure water utilities sustainability stimulating private sector participation. Outcomes Efficiency of the water utilities in terms of staff per 1000 connections : There were significant reductions in the number of staff per 1000 connections, from 7.6 to 4.5, almost achieving the target of 4 making the utility's operation more efficient. Sustainability of the water utilities as indicated by the working ratio : Working ratio indicates the water utility's ability to cover its expenses through revenues, and is defined as operating costs /operating revenues. While project supported actions reduced the working ratio from 0.96 at appraisal to 0.84 at project closure, fully recovering their operating costs through tariff revenues, the target 0.6 was not achieved. According to the ICR (page iv), "financial sustainability remains a concern ." Almost 70 percent of the municipalities which involved the private sector have been supported by World Bank financing, This, coupled with the above progress to achieve the outcome indicators have shown that the project has contributed to creating and maintaining an appropriate environment for improving the efficiency and sustainability of water utilities. 2. Expand the coverage of water supply and sewerage services provided in participating Municipalities : Substantial . Outputs Project funded routine upgrades and rehabilitation of the existing water supply and sewerage systems of all participating municipalities, and for the medium sized cities the scope of work were determined by the private operators. Specifically, the investments supported under the project include reduction in technical water leakage and unaccounted for water, rehabilitation of wells, pumping stations, expansion of storage tanks, upgrading and extension of distribution networks etc . The target set at appraisal was to reach at least 100,000 inhabitants of medium sized cities . Some calculations were made based on the data provided in Annex 2 of the ICR and it was found that the population benefiting from water supply access through the project interventions in El Banco of Magadlena, Soledad of Atlantico, Maicao of La Guajira, and Sincelejo of Sucre (all medium sized cities) alone totaled 247,780, while 101,574 people have benefited from sewerage access . These numbers indicate that the objective of reaching at least 100,000 inhabitants in the medium sized cities has been exceeded substantially . The total number of people gaining access to improved water supply and sewerage services has considerably exceeded the target set at appraisal. The ICR (on page 14) reports that the project supported; (a) the preparation, publication, and dissemination of the document "public policy guidelines for drinking water and sanitation for rural Colombia "; (b) a rural health inventory in 18 departments of the country; and (c) a series of capacity building exchanges and workshops at the community, local government, and international levels . While policy directions were identified, the Ministry of Environment, Housing, and Regional Development's rural water program was inactive and a national rural water policy was not formally defined Outcomes Coverage rates for water supply infrastructure : The project increased the water coverage from 61 percent at appraisal in fiscal year 2001 to 84 percent at closure in fiscal year 2010, exceeding the target of 80 percent. Coverage rates for sewerage infrastructure : The project increased the sewerage coverage from 40 percent at appraisal in fiscal year 2001 to 50 percent at closure in fiscal year 2010, but did not reach the target of 65 percent. Continuity of water supply service : The project significantly increased the percentage of continuous water supply service from 2 percent at appraisal in fiscal year 2001 to 49 percent at closure in fiscal year 2010, although it did not achieve the target of 80 percent. Comparing the coverage increase as well as the continuity of service in two categories of municipalities, the medium sized cities exceeded the target of increasing the water supply coverage, and substantially increased the continuity of water supply service, but fell short of increasing the sewerage coverage . The small cities could not make any progress in increasing the coverage for water supply or sewerage or the continuity of water supply service but the overall achievement of this objective is substantial . 3. Facilitate the access to water and sewerage services by the population of low income areas in participating municipalities : Substantial . Outputs Water supply schemes rehabilitated by the private operators were in the municipalities where 90 percent of the population was officially categorized as "poor" (page 27 in Annex 2 of ICR). Outcomes The ICR does not report on the number or the share of low -income households that gained access to water and sewerage services because more than 90 percent of the population in participating municipalities is in the lower economic brackets. Assuming that 90 percent of the 247,780 people in Atlantico, Maicao of La Guajira, and Sincelejo of Sucre alone benefited from water supply access and 101,574 people benefited from sewerage access through the project, it is likely that the target of reaching 40,000 inhabitants of low income areas in participating municipalities has been exceeded. 4. Improve environmental protection practices in connection with the provision of water and sewerage services : Negligible . Outputs A decision-support tool was developed to facilitate the planning of smaller municipal wastewater treatment plants . Generic plans, costs, specifications, and manuals were developed for over 13 treatment technologies, Outcomes This tool was not formally disseminated or applied and therefore the objective of improving the practices was not achieved. 5. Efficiency: An economic rate of return was calculated at project appraisal, but according to the ICR a traditional economic cost-benefit analysis was not carried out at closure because of the large number of sub -projects and the data- intensive nature of the exercise . While the ICR reports that there is a common recognition that providing water and sanitation services provides significant health and other economic benefits, there is no evidence in the context of this project that points to such economic benefits . The ICR assessed the efficiency of project investments as qualitatively measured by an engineer on the ICR team as a type of least-cost economic analysis. The ICR team developed a scoring system for qualitatively assessing investment efficiency of six sub -projects in 15 cities, out of 25 participating cities. The ICR explains that there are four categories of investment efficiency, but does not elaborate how the assessment is made to define a real, limited, modest, or positive impact on the conditions of service that affect the efficiency of the investments . According to this analysis, the operators in medium -sized cities generally made efficient investment decisions according to the pre-defined four point scale defined according to whether investments have generated a positive, modest, limited, or no impact on the conditions of service to the users . The exercise also assessed whether or not the investments were based on proper planning and prioritization of cost effective solutions to achieving the project objectives and related outcomes. The ICR states that this analysis is based on the professional findings of a recognized expert in Colombian water supply systems with extensive experience and familiarity of water systems along the country's Caribbean coast, but the ICR lacks the methodology and information that back the conclusions drawn by the expert . While the overall rating for project efficiency using a population weighted scoring system was high according to this analysis, the ICR does not mention whether the selected systems represent all project funded activities and it does not explain how the six sub-projects reviewed for efficiency were selected . The project took more than three years longer to complete than planned, and the total cost, at $ 81.54 million was higher than $70 million that was estimated at appraisal . These factors also reduce the efficiency of the project . On balance the overall efficiency rating is modest . ERR )/Financial Rate of Return (FRR) a. If available, enter the Economic Rate of Return (ERR) FRR ) at appraisal and the re- re -estimated value at evaluation : Rate Available? Point Value Coverage/Scope* Appraisal Yes 28.12% 88% ICR estimate No * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome: Both the relevance of the objectives and that of design are substantial . With regards to efficacy, the objectives related to efficiency/sustainability, expanding the coverage of water supply and sewerage services, and facilitating the access to water and sewerage services by the population of low income areas in participating municipalities - were substantially achieved . However, the remaining objective related to improving environmental protection practices was rated negligible. Efficiency is rated modest. As a result, there were moderate shortcomings in the operation's achievement of its objectives . Therefore the overall outcome rating is moderately satisfactory . a. Outcome Rating : Moderately Satisfactory 7. Rationale for Risk to Development Outcome Rating: The risk to development outcome rating is significant , based on the assessment of the policy, contractual, and financial risks. Policy Risk : The government has shown strong support for the key policy elements of this project through formally adopting the policies related to private sector participation in its National Commission for Economic and Social Policy (known by its Spanish acronym , CONPES) Document 3385 of 2005. It has also adopted the promotion and consolidation of improved operational management as one of the main objectives of the departmental water plans . Therefore the political risk is considered low. low Contractual Risk : Most of the operator contracts have experienced multiple amendments through renegotiation, especially when the municipal administration changes . In addition, there have already been a number of sub -projects in which the original operator was unable to comply with contracts, forcing a cessation of the contracts . The contractual risk is therefore considered significant . Financial Risk : According to a comprehensive financial analysis of five operators covering 14 municipalities, the working ratio which measures the utility's ability to cover operating costs from its revenues has improved only slowly, and it did not achieve the targets set for project closure . This is because some utilities rather than linking their tariff regime to national regulations, linked them to the original contract or negotiated tariffs with the municipality . The financial risk is therefore rated significant . a. Risk to Development Outcome Rating : Significant 8. Assessment of Bank Performance: a. Quality at entry: The Bank team had incorporated relevant experience regarding private sector participation in the project, and adopted the model to the local context through encouraging the use of domestic rather than international private operators and through providing adequate technical support and model bidding documents for the municipalities and the private sector. The PAD included details of the project concept and the implementati on procedures of the two different types of contracts with the private operators . Relevant risks were identified at appraisal, and mitigation measures were established for most of them . Although the PAD did not take full account of the difficulty of attracting the private operators in the small cities, or the increasing government focus on the medium sized cities, flexible and programmatic design allowed implementation to be adjusted to the pace of the private sector's capacity and interest . Although there was a discrepancy in the statement of objectives between the PAD and the Loan Agreement (see Section 2a above), relevant components were included and indicators defined to achieve all the objectives in the Loan Agreement. M&E design had some shortcomings : there were no indicators for the objective of improving environmental practice and indicators in general were not selected to map to the objectives . at -Entry Rating : Quality -at- Satisfactory b. Quality of supervision: There was a continuation of project team leadership throughout preparation and implementation, up to almost project completion, and the Bank team provided good support to the operators and th e Ministry of Environment, Housing, and Regional Development during project implementation. Adequate project supervision was carried out and the findings were recorded in the system periodically . On balance, the quality of supervision is rated satisfactory . Quality of Supervision Rating : Satisfactory Overall Bank Performance Rating : Satisfactory 9. Assessment of Borrower Performance: a. Government Performance: The Government was committed to the project and its reform elements . It adopted a series of resolutions and decrees to strengthen its reform program related to private sector participation . The Government also disseminated its policy widely which generated demands from the municipalities to participate in the projec t. The Government contributed more than half of the project costs from its national budget and increased its contribution by 53 percent, with actual counterpart funding of $ 45,86 million, compared to $30 million at appraisal. A new Administration took office shortly after project approval in October 2001 and the departmental restructuring that took place subsequently contributed to some initial delays in project implementation due to insufficient staffing and frequent management changes . However, the formal establishment of a Vice Minister of Water in the Ministry of Environment, Housing, and Regional Development helped to stabilize project staffing and management, and implementation accelerated after 2006. The Government continued to support the project through close supervision during the remaining implementation period . A decision-support tool developed to facilitate the plan ning of smaller municipal wastewater treatment plants was not formally disseminated or put into practice as had been intended . While policy directions were identified, the Ministry of Environment, Housing, and Regional Development's rural water program remained inactive, and a national rural water policy was not formally defined . Government Performance Rating Moderately Satisfactory b. Implementing Agency Performance: Project implementation in terms of contracting private sector operators was under the responsibility of the municipalities in the case of the constructor operator model for the small cities, generally with a population under 12,000. In the case of the operation with investment model for the medium -sized municipalities, the operators would bid for the concession contracts that were designed by the municipal Government in their investment programs. In the former, the performance of the municipalities were not positive as there were very few contracts signed and the investments made in those few municipalities where they were signed are not performing well . There were many instances in which contracts needed to be renegotiated, and there were a number of technical issues that were encountered by inexperienced operators . There were some weaknesses in M&E implementation, and the overall implementing agency performance is rated moderately unsatisfactory . Implementing Agency Performance Rating : Moderately Unsatisfactory Overall Borrower Performance Rating : Moderately Satisfactory 10. M&E Design, Implementation, & Utilization: a. M&E Design: The results framework in the PAD identified the key performance indicators as outputs, outcomes, and impacts that were for the most part appropriate, but the indicators were not separated to match the corresponding objectives . Improvement in the environmental protection practice did not have an indicator, and the policy objective was the finalization of a document. Baseline and target values by year for the medium sized cities and small municipalities were defined for each sub-set of indicators, such as service to all, service to the poor, efficiency of service, and sustainability of service in a systematic manner . This facilitated the monitoring of the progress during project implementation. b. M&E Implementation: Limited information on M&E implementation was available in the ICR . As a result of the capacity limitations of the participating municipalities, and the consequent difficulty in implementing their M&E activities in a systematic manner, the ICR team was obliged to carry out a significant data collection exercise at the end of the project . c. M&E Utilization: Limited information on M&E utilization was available in the ICR . M&E Quality Rating : Negligible 11. Other Issues a. Safeguards: The project was classified as category B for Environmental Assessment purposes because most of the investments comprised rehabilitation of existing water supply and sewerage systems without any significant anticipated negative environmental impact. According to the PAD (page 32), only one safeguards policy was triggered – Environmental Assessment (OP 4.01). No wastewater treatment plants were built under the project . Each of the construction contracts included standard environmental and safety requirements, and the contracts were supervised by independent construction supervisors . According to the ICR (page 6), there was compliance with environmental safeguards policies and safety requirements . b. Fiduciary Compliance: Financial Management : The ICR reports that the external auditors issued unqualified opinions on the projects' financial statements for the audit periods 2006-2009. Some internal control and non-compliance findings were corrected by the end of the project . The last two audit reports were submitted late and the team had once rated financial management moderately unsatisfactory in the supervision reports because; (a) the Government official information system was not capturing all the project transactions, thus manual records had to be kept, (b) there were deficiencies in the internal controls, and (c) there were delays in the submission of withdrawal applications . These deficiencies were subsequently addressed, and the FM rating was upgraded to moderately satisfactory in the supervision reports six months prior to project closure . Procurement : For the medium sized cities, the Bank provided no objection to the selection of the operators based on the procurement procedures agreed with the Government . During project implementation, after 18 municipalities had signed contracts with private operators, some problems were encountered with regard to further contracts , related to; (i) no bidder or only one bidder; (ii) concession contracts that were signed but failed; and (iii) directly negotiated contracts rather than competitive bidding . The award of the concession contracts needs the approval of both the Ministry and the Bank, which helped to ensure a certain level of contract integrity and transparency . Works, goods, and service contracts were implemented by the municipalities and this was done smoothly because adequate technical support was provided by the Ministry . The project's procurement was gen erally rated highly satisfactory or satisfactory throughout the life of the project and there were no cases of mis-procurement. One of the sub-projects (La Linea) was the subject of an INT investigation in early 2009. According to the ICR (page 7) "the INT report has been completed and concluded that there was no evidence of malfeasance". c. Unintended Impacts (positive or negative): There were no unintended impacts identified . d. Other: 12. 12. Ratings : ICR IEG Review Reason for Disagreement /Comments Outcome : Satisfactory Moderately The relevance of objective and Satisfactory relevance of design are both substantial. Efficacy for three of the four objectives is rated substantial while one is rated negligible. Efficiency is rated modest. There were moderate shortcomings in the operation's achievement of its objectives and efficiency. Therefore the overall outcome rating is moderately satisfactory. Risk to Development Significant Significant Outcome : Bank Performance : Satisfactory Satisfactory Borrower Performance : Satisfactory Moderately The Government did not show full Satisfactory commitment to environmental protection practice and rural water and sanitation policies. In addition, the municipalities did not perform well in achieving the efficiency and sustainability of the water systems covered under the project, thereby not achieving the related project objectives . Quality of ICR : Satisfactory NOTES: NOTES - When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. - The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons: Several lessons were learned from this project based on the ICR : An enabling policy environment and government commitment are critical for successful engagement of private sector operators . The private operators have generally performed well in medium sized cities because of the government's commitment to establish and disseminate relevant sector policies, and support to provide adequate financing and supervision assistance to the municipa lities and the private operators . The small towns component, however, did not perform well, in part because the smaller municipalities did not attract the private operators, but also because the Government did not have a clear strategy or commitment to make the policy work. Investment planning and selection of the operators are key to successful implementation of private sector participation . The project encountered problems with the quality of the operators as the selection method was based on the lowest cost subsidy by t he operators and not on the necessary qualifications . In addition, the project ran into problems during the investments because of technical issues such as inadequate raw water supply source and frequent power outage . Regulation is critical for ensuring financial sustainability . The project showed that cost recovery performance was best when the tariffs were set according to the national regulatory framework . In contrast, tariff regimes formulated at the start of the concession contract were often based on inaccurate investment and operational assumptions that resulted in operator under -funding. Likewise, tariffs which are based on direct negotiations with municipal governments are often disadvantageous to operators and based on political considerations rather than actual financial needs . 14. Assessment Recommended? Yes No Why? Although the operation-investment models for the medium sized cities seemed to work efficiently , the key performance indicator targets were not achieved and the risk to development outcome is significant . It would therefore be desirable to verify the ratings through a Project Performance Assessment Report to evaluate the performance of the project municipalities some years after project completion . Such in depth review could also tease out the differences in performance across municipalities to understand better under what context this strategy works, and to learn more about environmental management . 15. Comments on Quality of ICR: The ICR is in general well written and it is concise. The efficacy section, which assesses the achievement of project objectives, had enough evidence on the achievement of key performance indicators . Risk to development outcome was also assessed in a systematic manner for each perceived risk . The efficiency section however did not carry out a quantitative assessment which would have been valuable for comparing the results at project completion stage against the ERR at project appraisal stage . The efficiency section instead included a qualitative assessment, but the methodology for selection of sub -projects and the information used for the assessment were not provided . a.Quality of ICR Rating : Satisfactory