Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) Report Number: ICRR0021685 1. Project Data Project ID Project Name P112901 PK: Punjab Cities Governance Improvement Country Practice Area(Lead) Pakistan Social, Urban, Rural and Resilience Global Practice L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-51530 30-Jun-2017 140,897,780.07 Bank Approval Date Closing Date (Actual) 11-Sep-2012 30-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 150,000,000.00 0.00 Revised Commitment 147,255,452.62 0.00 Actual 140,897,780.07 0.00 Prepared by Reviewed by ICR Review Coordinator Group Cynthia Nunez-Ollero John R. Eriksson Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives According to both the Financing Agreement (FA, p. 4) and the Project Appraisal Document (PAD, paragraph 17), the Project Development Objectives (PDOs) were "to (i) support the Project Implementing Entity's cities in strengthening systems for improved planning, resource management, and accountability; and (ii) improve the Project Implementing Entity's capacity to respond promptly and effectively to an Eligible Crisis or Emergency." Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) There were no emergencies declared during the life of the project. The second part of the PDO is therefore not assessed by this review. The review will assess the following objectives:  to support the Project Implementing Entity's cities in strengthening systems for improved planning  to support the Project Implementing Entity's cities in strengthening systems for resource management  to support the Project Implementing Entity's cities in strengthening systems for accountability b. Were the project objectives/key associated outcome targets revised during implementation? No c. Will a split evaluation be undertaken? No d. Components 1: Performance Grant (US$145.0 million at appraisal, US$136.2 million actual). This component financed annual performance grants to the five largest cities in Punjab Province - Lahore, Rawalpindi, Multan, Faisalabad, and Gujranwala. These grants supported governance in (i) urban planning and resource management, and (ii) transparency and voice. Targets for first year performance were established at appraisal. Cities applied for these annual performance grants after meeting targets validated by independent teams deployed by the Government's Monitoring and Evaluation (M&E) Directorate General of the Planning and Development Department (P&DD). 2: Project Implementation and Capacity Building (US$5.0 million at appraisal, US$8.7 million actual). This component financed training and technical assistance directed at the implementing unit, the provincial Urban Unit (UU) of the Planning and Development Board of the provincial Government of Punjab. This component also financed training and technical assistance for City Program Units (CPUs) of the five participant cities to strengthen their financial, social, and environmental management systems. The training and technical assistance supported (I) city-wide resource planning; (ii) procurement; (iii) own source revenue enhancement; (iv) citizen participation and the use of social accountability tools; and (v) management of social and environmental impacts. This component also financed the UU's automation of the Urban Immovable Property Tax (UIPT) and contracts for the independent validation of disbursement- linked indicators achieved by cities to qualify for the annual performance grants. 3: Contingent Emergency Response (CER) (US$0 million at appraisal, US$0 million actual). CER was included following World Bank guidelines on investment projects in Pakistan, so that there could be a rapid reallocation of resources in case of a natural disaster, emergency, or catastrophic event. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: The total project cost was US$154.0 million and at project closing the total project cost reached US$144.9 million. The remaining SDR1.97 million (or US$1.41 million) was cancelled because exchange rate gains generated savings in the final year (ICR, paragraph 42). Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) Financing: The International Development Association (IDA) financed this Specific Investment Loan credit for US$150 million. Borrower Contribution: The Borrower committed and disbursed contributions of US$4.0 million that were all allocated to technical assistance activities under capacity building and the operational support for the project activities. Dates: The project was approved on September 11, 2012 and became effective on February 14, 2013. A Mid Term Review (MTR) was completed on March 21, 2016. The original closing date was June 30, 2017. A level 2 restructuring was approved on May 1, 2017 to extend the closing date by one year, to June 30, 2018. The extension was to allow (i) the city governments and entities to develop and adopt Integrated Development and Asset Management Plans (IDAMPs) and use the remaining performance grants to finance priority expenditures identified in these IDAMPs; and (ii) to institutionalize the project interventions following changes in the local government system and help establish new city government structures. 3. Relevance of Objectives Rationale The PDO was relevant to Pakistan's development strategy, Vision 2025. As people migrated toward urban areas, cities would strive to become "smart cities," able to adapt to increasing complexity and demand for communicating knowledge. Urbanization would be an important driver of growth. Urban cities would respond with improved governance and urban planning systems. Digitizing the land registration system also was an expressed priority of Vision 2025. The PDO was also relevant to the Punjab Provincial Government's Growth Strategy 2018. This strategy highlighted the role of cities as engines of growth and identified the reforms and public investments needed that mirror the activities under this project. The PDO remained relevant to the World Bank Group's Country Partnership Strategy (CPS) FY15-20. Goal 14 envisaged the "adoption of performance and transparency mechanisms in selected institutions to address specific problem areas of weak implementation capacity, low accountability, and lack of transparency of public institutions." This goal was to be rated against an "increased number of departments adopting citizens feedback and performance management systems." This project contributed to Results Area IV - Service Delivery - through improvements in governance in selected Punjabi cities. These improvements included Outcome 2.1 - Improved business environment for the private sector; Outcome 4.1 - Improved public resource management; Outcome 4.4 - Adoption of performance and transparency mechanisms in selected institutions; and Outcome 4.5 - Improved urban management in cities. The PDO was realistic and not overly ambitious. The activities were adequately funded. Lessons learned from other similar operations in both South Asia and Africa positively contributed to the realism of the PDO as clarified by the Task Team in their email of May 30, 2019. Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective • to support the Project Implementing Entity's cities in strengthening systems for improved planning Rationale The Theory of Change. The following activities (inputs) were designed to strengthen the planning system of cities: (i) prepare consolidated Annual Development Plans of new, on-going, and planned investments categorized by sector, entity, and which ones were new, rehabilitated (or refurbishment), or replacement assets; (ii) prepare asset inventories owned by the city government and water and sanitation agencies (WASAs); and (iii) prepare energy audits of WASA equipment. These inputs were to lead to outputs such as Annual Development Plans, asset inventories, and energy audits that identified inefficiencies and helped devise priorities for maintenance and replacement needs of WASA equipment. The inputs were necessary and sufficient to support the outputs. The outputs led to expected outcomes of evidence-based development planning at the city level called the Integrated Development and Asset Management Planning (IDAMP) method. IDAMP promoted a three-year window for planning urban investments. The IDAMP was based on best practices and included (i) asset inventory development; (ii) adopting standard metrics; (iii) topological (Geographic Information System or GIS) mapping; (iv) estimating costs to maintain current and future level of service using a life cycle management strategy; and (v) financial strategy (medium term financial framework). These inputs were necessary to achieve the outputs. The outputs led to the expected outcomes. OUTPUTS: Unless noted otherwise, the targets for all Outputs and Outcomes for Objectives 1-3 were achieved from a baseline of zero.  Each city prepared a consolidated Annual Development Plan for four years, achieving target.  Each city prepared a 3-year rolling IDAMP (FY2017-18 and FY2018-2019), achieving target.  City entities developed standardized procedures for planning, procurement, and contract management, achieving target.  Cities and WASAs prepared phased action plans to extend their service delivery area after adopting demarcated city boundaries for urban planning purposes, achieving target. OUTCOMES:  80 percent of the development and asset maintenance expenditures of each city were spent according to the three-year IDAMP achieving target. Each IDAMP was integrated, based on evidence, and multi- year. Previous plans were single year, not necessarily based on evidence.  Funding streams and amounts were disclosed and together with IDAMPs led to development decisions that were coordinated between the cities and the provincial government. Investment Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) planning was informed by the publicly disclosed provincial transfers. The ICR, however, was not clear on how many funding streams were consolidated, how many were found superfluous or eliminated.  City entities adopted and put into practice standardized procedures for planning, procurement, and contract management. Local Governments were also strengthened on their social and environmental management systems and these safeguards were applied to all investments in their respective action plans. Action plans for the phased extension in service delivery within the demarcated city boundaries were implemented meeting the targets for year 1 extension. These outcomes substantially achieved the PDO. Undefined funding streams that were eliminated/consolidated was a minor shortcoming. Rating Substantial OBJECTIVE 2 Objective • to support the Project Implementing Entity's cities in strengthening systems for resource management Rationale The Theory of Change: The following activities (inputs) were designed to strengthen the resource management system of cities: training and technical assistance that would result in outputs such as plans and standard operating procedures. These plans and procedures were implemented to generate increases in own source revenues, transparency in procurement, and increase in accountability to its citizens. These activities and inputs were necessary to result in the outputs, which, in turn, were to lead to the outcomes under this objective. OUTPUTS:  City entities developed, approved, and operationalized standardized procedures for planning, procurement, and contract management. Each procurement plan conformed to the Punjab Public Procurement Regulatory Act (PPPRA). City officials were trained on their implementation. Procurement reference materials were disseminated.  Cities prepared consolidated city-level reports on the flow of funds from the Provincial Government to urban areas. The Provincial Government informed the cities of transfers made at the time of transfers. The disclosure of all transfers and amounts from the provincial Finance Department to WASAs were reported to city governments. These funding streams were reflected on the websites of both entities.  The Urban Immoveable Property Tax (UIPT) was at closure fully automated in the five participant cities exceeding the original target of at least 75% completion at closure. Another city, Sialkot also benefited from the automation of its UIPT. The project supported the digitization of maps under the UIPT system. This entailed the scanning and digitization of property maps, electronic translation of ownership and taxation data, field surveys to validate and update the records and add any missing units, and the development of Management and Geographic Information Systems (MIS and GIS) that are currently being used by the Excise and Taxation Department for UIPT administration and collection. Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901)  Cities prepared action plans for extending the service delivery area after adopting city boundaries for urban planning purposes.  Cities and WASAs prepared action plans for enhancing Own Source Revenues (OSRs). City Governments and WASAs adopted Action Plans for enhancing self-collected OSR during the second year of implementation. The City Governments conducted field surveys to update the consumer base for: (a) licensing fees on trades and vocations; (b) advertisements and billboards; (c) rents from municipal properties; and (d) classification/re-classification of land use.  WASAs conducted consumer surveys to detect illegal connections and to verify records on the type of connections (residential, commercial, or industrial). OUTCOMES: Unless noted otherwise, the targets for all Outputs and Outcomes for Objectives 1-3 were achieved from a baseline of zero.  The approved procurement SOPs were applied to all contracts of city governments and WASAs resulting in the transparency and efficiency of the procurement process bringing savings of US$4.8 million (estimated vs. contract costs). The Government has required that procurement SOPs and safeguard requirements be applied to all local government investments, not just for rehabilitation and deferred Operations and Maintenance (O&M) but also new investments and regardless of funding source. The technical assistance component of the project supported training and technical assistance for improved Public Financial Management and social and environmental management, which contributed to these outcomes.  Amounts and predictability of funding streams from the Provincial Government to cities and WASAs were publicly disclosed. The baseline was that City governments were not aware of the magnitude of development expenditure in the city across multiple streams, including those received by WASAs. This reform (predictable periodic disclosure of funding streams), informed the 3-year rolling IDAMPs (investment plans), which reflected improved coordination between the provincial and the city governments.  UIPT collections have improved with average annual increases of US$1.3 million annually (2014- 2018) in tax collections from urban immoveable property. The average annual growth rate of 5.1 percent in the 5 years prior to the project reached 5.8 percent during the project years. The Government of Punjab has scaled up this digitized UIPT system across Punjab.  Cities operationalized action plans for raising OSRs with revenues reaching US$40.7 million at project closing from US$16.9 million at project start (no target provided). Revenue collection improved and arrears were recovered through electronic billing; detection and regularization of illegal water connections; installation of water meters; and rationalization of user tariffs. Collections improved in other avenues of OSR such as from sewerage services. New fees and licenses at the city level included rental income from municipal properties, fees from transport terminals including parking fees, license fees on trade and vocations, commercialization fees, and advertisement fees. These fees generated additional annual own-source revenues.  WASA revenues from water charges, sewerage and drainage charges, and miscellaneous charges or other income, increased from US$34.9 million in 2012-2013 to US$56.2 million by project closing.  Energy audits reported savings from reduced energy bills. The audits identified inefficient equipment and scope for power factor corrections. The ICR did not provide a baseline or target value of the savings. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901)  Institutions prepared urban plans that extended service delivery. Cities achieved integrated planning by implementing these action plans prepared by the various institutions. The outcomes substantially achieved the PDO. There were two minor shortcomings that did not detract from the substantially achieved PDO: (i) missing baseline and target values in energy savings; and (ii) comparing the amount of own source revenues mobilized under the project to total revenues available to the cities to show project impact. Rating Substantial OBJECTIVE 3 Objective • to support the Project Implementing Entity's cities in strengthening systems for accountability Rationale The Theory of Change: The following activities (inputs) were designed to strengthen the accountability system of cities: training and capacity building of officials on systems and procedures that need to be in place to comply with the Punjab Transparency and Right to Information Act of 2013 as well as subsidiary legislation under the 2013 Local Government Act. These activities led to outputs such as mechanisms for improved public disclosure and a system for managing grievances and its resolution. The outputs led to the desired outcomes under this objective of establishing accountability and transparency in government transactions. The inputs were necessary to achieve the outputs and these outputs in turn were to achieve outcomes. OUTPUTS:  Each city prepared a Grievance Redress Mechanism (GRM) with service standards and documented processes, achieving target.  Mechanisms in each city were developed and approved to monitor and resolve complaints and grievance redress related to municipal services, achieving target.  Cities and WASAs developed and approved mechanisms, including websites, for disclosing public information in accordance with Punjab's Right to Information Act, achieving target.  A communications strategy was developed and implemented by the Urban Unit of the Planning and Development Board of the Provincial Government of Punjab. Numerous communication tools including tickers on TV; posters and banners; and consultations were deployed to reach out to the citizens.  Over 170,000 complaints were resolved by WASA and more than 2,000 by the city governments. OUTCOMES: Unless noted otherwise, the targets for all Outputs and Outcomes for Objectives 1-3 were achieved from a baseline of zero.  All outcomes under Objective 1 remained relevant to Objective 3. The project strengthened systems in city governments and WASAs for transparency and downward accountability to citizens in the five cities. Downward accountability refers to elected officials being held accountable by their constituents. Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) These systems comply with the requirements of the Punjab Transparency and Right to Information Act 2013 and subsidiary legislation under the LG Act.  Mechanisms for improved public disclosure and access to information were operationalized in all cities, and were responding to information requests from journalists, academics, lawyers, NGOs, citizens, etc. Proactive disclosure was also undertaken on the websites of city governments and WASAs.  Existing complaint resolution systems run by the Government of Punjab (such as the Chief Minister’s, Chief Secretary’s and DCO’s complaints cells and “Zimmedar Shehri”) were linked to automated systems developed at city entities to allow rerouting of complaints to the agency responsible for the service and reporting.  Operationalization of the one-window complaint center increased the resolution rate during 2015-17 - 175,000 complaints were received by WASAs and CDGs, of which 98.35 percent were resolved, 1.53 percent were pending, and 0.12 percent were rejected. Complaints and grievance management systems developed and operationalized in the five city governments and WASAs provided citizens with multiple channels to register complaints and to provide feedback, including walk-in complaint centers, dedicated telephone lines and designated staff, and web-based solutions.  A third-party survey was conducted in the five cities in January-February 2019 to assess the quality and effectiveness of the complaints and grievance redress mechanisms. Almost 99 percent of the respondents were aware of these mechanisms, 92 percent were satisfied with the ease of access using the indicators such as suitability of channels or modes, the conduct of officials and their competency. Over 78 percent were satisfied with the process of registering complaints. According to the survey, female respondents were more likely to be satisfied or highly satisfied with the available systems than male respondents. Overall, more than half of the respondents were satisfied with the complaints and grievance redressal systems even though there were wide disparities in the results by agencies. The WASA Gujranwala and Metropolitan Corporation Lahore registered high dissatisfaction. Notwithstanding the high dissatisfaction with one WASA and the Lahore Metropolitan Corporation, the large majority of survey respondents registered a relatively high level of satisfaction with accountability mechanisms initiated under the project. The above outcomes substantially achieved the PDO with minor shortcoming in describing both the upward and downward accountability achieved by the project. Upward accountability in this case would refer to how local city officials would hold the provincial and national entities accountable within the context of the project. Rating Substantial OVERALL EFF TBL OLD Rationale The capacity building activities of the project were technical assistance focused on (i) city-wide resource planning; (ii) procurement; (iii) own source revenue enhancement; (iv) citizen participation and use of social accountability approaches and tools; and (v) management of environmental and social impacts. Performance grants were released to the cities and city entities based on achieving disbursement linked indicators. These outputs - mechanisms, operating procedures, and investment plans - were implemented to achieve outcomes that strengthened the core business systems and capacity of city governments and WASAs to deliver services to their Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) constituents by improving governance, decision making, resource planning and management, consolidating and strengthening revenue mobilization, and increasing transparency and accountability. Even with the transition from City District Governments to Municipal Corporations under the Local Government Act and a turnover of officials in the last two years of project implementation, the project received adequate training to address the risks to the sustainability of project outcomes. All three objectives were rated substantial notwithstanding the minor shortcomings noted under each objective. Overall Efficacy Rating Substantial 5. Efficiency Economic and Financial Efficiency: A cost benefit analysis was not prepared at appraisal because funding went to improved maintenance of assets and technical assistance, rather than new infrastructure. (PAD, paragraph 73). Benefits were expected to include improved efficiencies in service delivery, better managed cities, that would then lead to savings and improve cities' finances (PAD, paragraphs 71-73). Financial efficiencies were expected from increased property tax revenues because of the strengthened systems and processes introduced by the project. In addition, capacity improvements in maintenance of water and sanitation would lower operational costs. The project was expected to provide technical support and build capacities of cities to select cost effective investments and enable cost-benefit analysis at closing. A cost benefit analysis was conducted at project closing. Cost was assumed to be the US$150 million original IDA loan commitment. Benefits included the following (ICR, paragraphs 40-43):  increase in own source revenues of city governments and WASAs,  increase in cities collection of urban immovable property tax,  rise in efficiency of employees and capital assets and systematic improvements from training and capacity building of employees,  cost savings in contracts awarded as a result of good procurement practices, savings in out of pocket costs for citizens due to the establishment of complaint and grievance redress mechanisms,  savings in maintenance and repairs and the depreciation costs of the existing capital stock of city governments and WASAs based on integrated spatial planning and asset management using common city boundaries  having fiscal space through an improved intergovernmental finance system. The International Monetary Fund defines fiscal space as the budgetary room that allows a government to provide resources without undermining fiscal sustainability. This budgetary room is based on the availability of current and future expenditures and revenues. The provincial government in this case, reports all transfers to the cities. The cities then rely on the predictability of these transfers to allocate them efficiently. This factor improves the credibility of City District Governments to deliver services. Annex 4 of the ICR describes metrics and estimates of the increase of each of these benefits over the life of the project. Overall ex-post efficiency of the project was measured using net present value, internal economic rate of return, and benefit cost ratio. The benefit cost ratio registered at 3.5 if the actual US$136.2 million project cost was used rather than the total US$150 million project cost. The project achieved a 45.8 percent economic internal Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) rate of return, with net present value of US$333 million, using a 7.5 percent discount rate. This discount rate was aligned with the interest rates on recent foreign loans of the Government of Pakistan (ICR, Annex 4, paragraph 4). Administrative and Operational Efficiency: The date of effectiveness was delayed by the two months that the federal Ministry of Law took to clear the legal documents. This delay cascaded to a full year of implementation and disbursement cycle delay since the annual assessment and performance grant cycles were synchronized with the July 1 to June 30 city and water and sanitation agency budget cycles. In 2013, the Government enacted a new local government legislation that revamped structures, jurisdictions, and mandates. In 2016 new local elections were held. In 2017, the newly elected officials assumed office. The new system called for Metropolitan and Municipal Corporations (MCs) to replace the City District Governments (CDGs). The new MCs separated the urban from the rural institutional structures resulting in a readjustment of the urban boundaries and plans revised. l . All CDG staff was replaced by new MC staff, which required new training. Several core municipal functions were handed over to the new agencies, authorities, and companies with unclear accountability to the city governments. Several sources of revenues were handed over to the new institutions. These negatively affected the revenue enhancement plans prepared by the city governments since these sources were no longer available to them. In addition, the project closing date was extended by a year to allow time to train new staff of the newly established MCs and implementation of the IDAMPs. The Government of Punjab's Finance Department, on average delayed its release of project funds by 8 months. This delay negatively affected the cities' implementation and procurement abilities. Unspent balances from earlier tranches delayed final disbursement. The follow-on Punjab Cites Program now includes a legal covenant for the timely release of funds from the province to local governments. Notwithstanding the shortcomings in administrative and operational efficiency, overall efficiency (including significant ex poste estimated economic and financial efficiency) is judged to be Substantial. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 94.00 ICR Estimate  45.80  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) The Relevance of Objective was rated substantial. The efficacy of all three objectives was rated substantial. Efficiency was also rated substantial. Outcome is therefore satisfactory. a. Outcome Rating Satisfactory 7. Risk to Development Outcome The following posed risks to development outcome:  Government Commitment and Institutional Risks: The new provincial and local governments were committed to strengthening urban governments. This commitment required a carefully managed transition. Several regulatory issues were identified that needed to be addressed. These included (i) non-optimal urban boundaries, (ii) mandate fragmentation among various new agencies, and(iii) weak institutional accountabilities for urban management and service delivery. The Local Government and Community Development Department identified several reforms to address these risks and included these in the new local government legislative framework to be applied to all cities. These included: (i) drafting new boundaries for cities consistent with built up areas and for use in the next local government elections; (ii) strengthening accountabilities of service delivery companies and authorities to these local governments; (iii) empowering the office of newly elected mayors; and (iv) mainstreaming and scaling up improved systems introduced under the project (e.g., standard budgeting and accounting practices).  Governance Risks: Gains in improved governance systems during the first four years of project implementation were negatively affected by the 2017 transition to the new local government system. To mitigate this risk, the Urban Unit and the City Program Units adjusted the implementation of their systems and procedures to match the revised mandates and jurisdictions of successor entities and train new staff. The Urban Unit is now an independent, public sector owned company. 8. Assessment of Bank Performance a. Quality-at-Entry The Bank identified this operation to be strongly aligned with both the Bank's and the Government's development strategy. The project was designed to focus on the five largest cities in the province. Both the provincial and the national governments acknowledged that cities significantly contributed to the country's economic growth. Cities were estimated to contribute 78 percent of the country's Gross Domestic Product (GDP). In the case of Punjab province, urban areas contribution estimated 52-58 percent of the province's GDP. The project identified the following obstacles for these cities to be engines of growth - (i) fragmented mandates and inconsistent jurisdictional boundaries; (ii) weak systems for resource planning and management, including unpredictable capital investment funding and low own source revenues; and (iii) lack of accountability to citizens. The project design, seen as the initial steps to becoming "smart" cities, focused on improving systems for planning, resource management, and Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) accountability to citizens. Project design also incorporated mutually reinforcing elements - capacity building efforts would improve institutional performance to justify disbursements of grants that meet performance indicators. Quality at entry was enhanced by the Government's establishment of the Urban Unit under the province's Planning and Development Department. This Urban Unit assessed the capacity of the five participant City District Governments, benchmarked service delivery targets, and identified lack of funds as the reason behind deferred maintenance of infrastructure assets. This led to defining which expenditure would be eligible under the project. In addition, City Program Units in the five city governments were established as the decentralized representatives of the provincial Urban Unit to help meet disbursement linked indicators. The year-one results of seven disbursement linked indicators were envisaged to be achieved during preparation, verified at effectiveness and to lead to disbursements of performance grants to the cities. The ICR reported that these year-one results were achieved by negotiations among the relevant parties (ICR, paragraph 55). The project design benefited from the experience of the Punjab Municipal Services Improvement Project and similar operations in South Asia and Africa. In their May 30, 2019 email, the project team specified that design elements from the West Bengal project to improve Gram Panchayats, the Kerala project to improve municipal governments, the Metro Colombo Project, which focused on large cities, and the Tanzania Urban Local Government Strengthening Program, which simplified its disbursement process, all informed the project design. This project originally considered including more than the five target cities. When a phase 2 of the Punjab Municipal Services Improvement Project included other cities, this project refocused only on the five largest cities of the province. At appraisal, the technical, institutional, fiduciary, safeguards, and economic aspects of the project were assessed with the intention to ensure the achievement of the PDO. Risks were identified as part of capacity assessments and mitigating measures included in design. The Director General for Monitoring and Evaluation (M&E) verified the achievement of output and outcome indicators. These indicators were clearly linked to the project results framework. A minor shortcoming in the design of the results framework was the lack of baseline values for the indicators. Quality-at-Entry Rating Satisfactory b. Quality of supervision The Bank team adequately supervised the project with 11 implementation support missions (ISMs) over the six years of project implementation. With a focus on achieving the PDO, these ISMs and additional technical missions that included international experts, assessed progress and candidly flagged issues that posed risks to sustaining project outcomes. For example, during the last two years of the project, when the implementation of the new Local Government Act introduced new structures, jurisdictions, functions, Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) and human resources, the Bank team flagged the associated issues of suboptimal boundaries, fragmented mandates, staff turnover, and financial management systems. The comprehensive Mid Term Review (MTR) provided corrective measures, particularly following the impact of the implementation of the 2013 Local Government Act (amended in 2016). These included accelerating development and operationalization of the Integrated Development and Asset Management Plans of those cities that were lagging at the time of the MTR, the release of the undisbursed funds from the Provincial Government to the cities, and appointing officials into vacancies with the requisite technical skills. In addition, additional technical assistance was identified as part of a framework to make the new municipal service provision companies accountable to city governments under the new Local Government Act, enhance their financial sustainability, and review municipal functions and mandates to be assigned to the appropriate administrative level. After loan closing, and prior to the completion of the ICR, the Bank team briefed the new provincial government to ensure awareness of critical mitigating measures to sustain project outcomes. The Government requested Bank team assistance in developing a province-wide Urban Sector Reforms Roadmap. This led to a strategic dialogue where Punjab cities were compared to regional and global comparators, opportunities and challenges faced and actions to be taken by the cities to become the promised engines of growth. The dialogue led to the preparation of the follow-on project, Punjab Cities Program (ICR, paragraph 87 and reconfirmed by the project team in its May 30, 2019 email). Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The objectives were clear and specific. The theory of change was sound. Key activities were expressed in seven intermediate results indicators and two outcome indicators. These results and outcomes were adequately described in the results framework. The indicators covered all the outcomes of the PDO. The intermediate outcomes captured the contribution of the project activities and results. The indicators used were mostly qualitative but detailed quantitative indicators were further explained in Annex 4 of the ICR. All were relevant to meeting the objectives. A lack of baseline values for the indicators were a minor drawback. For example, share of own source revenues in overall revenues of city and WASA budgets. However, the annex on Efficiency (ICR, Annex 4) provided some level of baseline such as those referring to the UIPT. Another drawback in transparency and accountability indicators was the focus only on downward accountability (e.g., surveys on the grievance mechanisms), which may have been by design but not noted Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) in the ICR. Institutionally, the implementing entity, the Urban Unit of the Planning and Development Board, Provincial Government of Punjab was responsible for the overall M&E functions. b. M&E Implementation The M&E system was implemented as designed, even though gaps in baselines for some components may not have been presented in the ICR. The indicators in the Results Framework were measured and reported as designed. Some indicators needed to be revised during implementation because of the launch of the 2013 Local Government Act. Some core municipal functions were removed from the mandate of city governments and given to new entities with unclear accountability mechanisms. Corrective timebound actionable measures such as staffing and clarifying accountabilities of WASAs to city governments were addressed in the MTR. The Director General for M&E conducted timely annual assessments. Local governments provided information on achieving the direct linked indicators and the intermediate results indicators. The Urban Unit reported on project progress on a regular and satisfactory manner, A firm was hired to conduct the beneficiary survey of outcome indicator 2. This survey was conducted in January February 2019 (see last Outcome under Objective 2 above). c. M&E Utilization M&E data informed the disbursement decisions under the Performance Grants component of the project and was used in key decisions, such as in the MTR. For example, a framework of relevant follow-on technical assistance was developed in response to the institutional changes introduced by the implementation of the 2103 Local Government Act. Institutionally, the M&E functions remain with the Urban Unit of the Planning and Development Board, and led by the Director General for M&E. M&E data was used to provide evidence of project outcomes and supported information to strengthen the follow-on project. For example, the timely release of funds from the province to local governments became a legal covenant in the follow-on project. M&E Quality Rating Substantial 10. Other Issues a. Safeguards Environmental Safeguards: At appraisal, the project was classified as category B and triggered only one safeguard policy - OP/BP 4.01 - Environmental Assessment. An Environmental and Social Management Framework (ESMF) was prepared and disclosed in compliance with World Bank safeguard policies. The ESMF was implemented by 19 full time staff in the Urban Unit and in the City Program Units, Safeguard documents and clauses were included in the bid documents and contracts. Its implementation was part of Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) the overall scope of work. The UUs and CPUs conducted some 22 training sessions with 607 people from the local governments, WASAs, contractors and other stakeholders participating. Committees formed in each entity, met 176 times to review, approve, and monitor compliance with environmental and social management plans. Training on safeguard issues was held regularly at multiple levels leading to greater awareness of mitigation measures. An independent third-party contractor was hired to annually evaluate ESMF performance. These third-party validations reported that CPU staff raised environmental and safeguards awareness among officials and contractors. These reports were also used to encourage cross-entity learning and drawing lessons for future similar operations. Social Safeguards: The project did not require land acquisitions nor report any impact on livelihoods. Neither an Abbreviated or Resettlement Action Plans were required. The ICR mentioned minor issues of non-compliance that were subsequently resolved, as clarified by the project team May 30, 2019 email. b. Fiduciary Compliance Financial Management: Under the project, an internal audit section was established at the Urban Unit supported by a qualified audit staff as mitigation measure against the risk identified during appraisal. . Semi-annual internal audit plans using a risk-based audit approach were instituted. The internal audit reports highlighted control weaknesses. There were no cases of fraud or corruption. Three were, however, delays in the release of project funds from the Finance Department of the Government of Punjab. The ICR reported that the project complied with all financial covenants, submitted acceptable quarterly interim financial reports and audited financial statements (ICR, paragraph 79). However, the ICR also reported that audited financial statements for the year ended June 30, 2018 have not been submitted. The final review of audited financial statements for the year ended June 30, 2018 showed that the Director General Audit, Punjab Auditor General of Pakistan, submitted an audit report based on draft financial statements. Both the revised audited financial statements and annual auditor's report were expected by April 30, 2019. When the local governments were restructured in 2017, the financial management systems of the City District Governments were transferred to the divisional level. The municipal corporations that succeeded city district governments adopted a computerized financial management and reporting system developed by an earlier Bank supported project. The financial management systems were not yet fully operational by project closing because of a lack of qualified accounting staff in the municipal corporations. The municipal corporations depended on their municipal staff for financial management - municipal finance officers for accounting and recording, and local fund audit staff for pre-audit. Procurement: Improvement in procurement was one of the pre-specified results in resource planning and management that the five cities needed to achieve to qualify for annual performance grants. City District Governments and other City entities would have applied transparent Standard Operating Procedures (SOPs) for procurement and contract management. These SOPs included planning, bidding, evaluation, Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) approvals and award, contract management, payments and contract closing. The SOPs further delineated roles and responsibilities within jurisdictions along with service standards. The improved procurement led to savings of about PKR1 billion or 8.9 percent of the estimated total value of the contracts over the project implementation period (ICR, Table 7, paragraph because of efficiencies in public procurement practice 74). Savings were used for additional projects. Lapses were encountered such as in one instance, a municipal corporation did not apply procurement rules adequately because of the change in local government structure and leadership, as well as high staff turnover. Some bids were cancelled before contracts were awarded. In the initial years, procurement plans only covered those activities to be funded by the Bank rather than entire budgets. Corrective measures were undertaken by conducting rigorous ex-post reviews. There were no major non-compliance issues encountered. All procurement was carried out under national competitive bidding or shopping. The Bank conducted prior review of the first three contracts. Subsequent annual plans were approved by the Bank. The Urban Unit provided regular support and supervision of the Municipal corporations and WASAs. Overall, the project complied with the financial covenants. Acceptable interim quarterly financial reports were submitted. Annual audited financial statements were submitted, except audited financial statements for the year ended June 30, 2018 (ICR, paragraph 79). c. Unintended impacts (Positive or Negative) --- d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory Quality of M&E Substantial Substantial Quality of ICR --- Substantial 12. Lessons The ICR offered eight lessons from this operation that could benefit other similar operations in the future. Some of these, slightly edited, are presented below:  Applying project-funded interventions, such as procurement, financial management, environmental and social management systems, into all development spending by city Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) governments and water and sanitation agencies can provide official and staff with relatable tangible benefits. For example, in the case of environmental and social safeguards systems, cities and water and sanitation agencies internalized safeguard considerations in its municipal development processes by strengthening their compliance monitoring and meeting mandatory reporting requirements. In the case of improvements in the financial system, for example, substantial increases in revenues and savings in expenditures were realized. Cities and WASAs increased their own source revenues, reduced inefficient expenditures in energy and premiums in delayed maintenance of service delivery assets.  Performance grant mechanisms require predictability in the flow of funds. Delayed releases of promised resources affect the planning and implementation of planned investments and affect the credibility of local authorities who wish to establish accountability . As a result of the experience in this project, the follow-on Punjab Cities Program provided a more effective and predictable mechanism in the relevant legal documents.  A dearth of skillsets needed to successfully achieve the objectives of an urban and regional reform program need to be accompanied by training and capacity building endeavors. For example, in this project, when newly elected local governments took office in January 2017 and all key City District Government officials were remapped to provincial departments, there was an immediate need to bring new officials who were unfamiliar with the project systems up to speed. New and extensive training programs for new officials were implemented over the last year of the project, which contributed to the need to extend the project by a year. The project team reported in its May 30, 2019 email that the new officials assumed their responsibilities and continued to operate the systems even after project closing.  By using their own systems and procedures, local governments have greater chances of adopting and institutionalizing how resources are mobilized and spent to implement decentralization. For example, in this project, to improve its own source revenue mobilization, the City Governments conducted field surveys to update its consumer base for (i) licensing fees on trades and vocations; (ii) advertisements and billboards; (iii) rents from municipal properties; and (iv) classification of land use. WASAs conducted consumer surveys to identify leakages from illegal connections and to verify connection types. Resources improved because of electronic billing. Illegal water connections were detected. Water meters were installed. Tariff rates were rationalized. City governments prepared consolidated Annual Development Plans using specified templates reflecting all on- going, new, and planned investments categorized by sector, by entity, and whether or not these were new, rehabilitation, or replacement assets. WASAs prepared detailed asset inventories to identify inefficiencies, and prioritize maintenance and replacement needs. These data sets were the evidence used to create the three-year rolling integrated development expenditure and asset management plans. 13. Assessment Recommended? No Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review PK: Punjab Cities Governance Improvement (P112901) 14. Comments on Quality of ICR The ICR was concise and followed OPCS guidelines. The report focused on results and was internally consistent. There was strong evidence to support the narrative that supported the ratings. The report was candid, pointing out the operational and administrative drawbacks during implementation. Another point of candid assessment referred to the changes in regulatory environment and institutional mandates that posed significant risks to development outcome. The quality of evidence was robust with annexes supporting the analysis. There were only minor shortcomings in the completeness of data. Additional detailed information was provided in the Team's message of May 30, 2019 and is available from IEG. Lessons were based on evidence and analysis. For example, the reduced ability of recipient cities to implement their planned investments because the provincial Finance Department periodically delayed transferring performance grants led to stipulating a legal covenant in the follow-on Punjab Cities Program. a. Quality of ICR Rating Substantial Page 18 of 18