Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) Report Number : ICRR0021525 1. Project Data Project ID Project Name P099979 IN: CBldg for Urban Development Country Practice Area(Lead) India Social, Urban, Rural and Resilience Global Practice L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-49970 30-Jun-2016 22,253,370.81 Bank Approval Date Closing Date (Actual) 21-Jul-2011 30-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 60,000,000.00 0.00 Revised Commitment 44,999,999.87 0.00 Actual 20,759,256.87 0.00 Prepared by Reviewed by ICR Review Coordinator Group Victoria Alexeeva Vibecke Dixon Christopher David Nelson IEGSD (Unit 4) 2. Project Objectives and Components a. Objectives Original Project Development Objective (PDO): “To assist the Recipient in improving the systems and skills of select urban local government bodies [ULBs] with respect to urban management and urban poverty reduction, as well as to support the Recipient’s implementation of various urban policy and institutional reforms” (Financing Agreement dated December 8, 2011, p.4). According to the project appraisal document (PAD, p.5), the PDO was “that: selected ULBs will have improved their systems and skills for urban management and urban poverty reduction”. Page 1 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) Revised PDO: “To assist the Recipient improve planning for urban management in select urban local bodies and in the roll out of national urban missions” (Restructuring paper dated June 27, 2016, p. 6). b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 07-Jul-2015 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components Original components 1. Capacity Building for Strengthened Urban Management (appraisal US$37.5 million; actual US$14.2 million). This component was to provide demand-driven technical assistance for States and urban local government bodies on various urban management topics, including financial management, urban planning, improvement in service delivery, and local governance. 2 . Capacity Building for Effective Urban Poverty Alleviation and Monitoring (appraisal US$18.5 million; actual US$0). Activities focused both on central- and local-level support and included (1) Preparing poverty reduction strategies for States and urban local bodies. (2) Establishing, administering and operating a challenge fund for urban poverty alleviation whose objectives were to: (i) assist eligible entities with recognizing and implementing innovative practices regarding urban poverty alleviation; and (ii) support government departments, institutions, and parastatals in adopting and scaling up “best practices” associated with these innovative practices. (3) Creating a network of practitioners from among the urban local bodies participating in the Challenge Fund to promote information sharing and capacity building in urban poverty alleviation. (4) Developing training materials on urban poverty and service delivery. (5) Strengthening the Ministry of Housing and Urban Poverty Alleviation (MOHUPA) capacity for policy analysis and monitoring and evaluation. 3 . Implementation Support (appraisal US$4 million; actual US$6.5 million). This component was to support a national Project Management Unit (PMU) for providing overall technical and managerial assistance to the two implementing ministries the Ministry of Urban Development (MOUD) and MOHUPA during implementation, in the areas of (i) pipeline development; (ii) quality assurance; (iii) procurement and procurement advisory services; (iv) FM; (v) reporting; (vi) M&E; and (vii) project administration. Page 2 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) Revised components At the time of project restructuring in December 2013, Components 1 and 2 were revised in terms of size and scope, due to significant delays in the implementation. By the time the project became effective in January 2012, the Government’s urban development program supported by the project was reaching completion (March 2012). While the project preparation began in 2006, it was approved only in July 2011. Initially there were two projects planned (US$40 million Credit to the MoUD and a parallel US$20 million project was being prepared to support MoHUPA); the combination of these two projects led to subsequent delays. As several activities (i.e., double entry accounting and city development plans) had been undertaken by the government on their own, both components were downsized, and activities removed and new added. US$15 million was cancelled across two components. Specific changes included, • Under Component 1, the provision of training to municipal officials was added along with direct support to the MoUD for strengthening the ministry’s capacity for policy analysis, monitoring and evaluation. • Under Component 2, the ULB-level activity for preparation of urban poverty alleviation strategies was removed, as these had already been completed under the Government’s national poverty mission.The activities implemented by MoHUPA were re-oriented towards support for strengthening various institutions involved in capacity building of ULBs and implementation of the Ministry’s schemes (Restructuring Paper, 2013). At the time of the project restructuring in 2015, the following changes were made: • Component 1 was revised in response of the Government’s request to support the new national urban missions of the Atal Mission for Rejuvenation and Urban Transformation (AMRUT) and Smart Cities by (a) providing access to ULBs to consultants to assist them in the planning exercises; (b) funding activities which enabled cities to increase own revenues (property tax, advertisement tax, and Value Capture Financing [VCF]); and (c) facilitating access to private finances through credit rating reports and transaction advisory support. The funding of Component 1 was increased form the revised US$30.5 million to US$41 million using cancelled funds under Component 2. • Component 2 was dropped, as no implementation had commenced under Component 2, and the Government had undertaken urban poverty reduction-related measures on its own. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Total project cost: The total project cost at closure was US$20.8 million or 1/3 of the originally committed funds of US$60 million. The project scope was significantly reduced during implementation for the slow pace of implementation. Financing: The project was financed through an IDA technical assistance (TA) credit in the amount of US$60 million. US$15 million was cancelled in 2013, and only US$20.8 million was utilized out of the revised amount of US$45 million at closure. Page 3 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) Borrower contribution: None was planned, and none materialized. Dates: The project was approved on July 21st, 2011 and effective on January 27th, 2012. The original closing date was June 30th, 2016 and the actual closing date was June 30th, 2018, i.e., the project was extended for two years. There were three project restructurings with significant changes to the design and scope of the project. Below are the details: • Under the 1st project restructuring in December 2013, the key changes were: (i) partial cancellation of US$15 million from the Credit to reflect the likely savings estimated based on variation of exchange rates and reduction in the scope of activities under Component 2; (ii) revision of the activities under Components 1 and 2; and (iii) corresponding revision of the project's results framework and disbursement schedule (see M&E section). • Under the 2nd restructuring approved in July 2015, the key changes were (i) revision of the PDO; (ii) cancellation of Component 2 (implemented by MoHUPA) and transfer of Component 2 funds to Component 1 (implemented by MoUD); (iii) revision of the results framework; (v) addition of the description of activities to be undertaken to support national urban programs under Component 1; and (vi) extension of the project closing date by two years to June 30, 2018. • Under the 3rd restructuring in July 2016, the key changes were: (I) change in institutional arrangements to allowing decentralized procurement to address delays in procurement, and (b) change in intermediate indicators. 3. Relevance of Objectives Rationale Original PDO The Government of India launched its flagship urban development program-the Jawaharlal Nehru National Urban Renewal Mission (JNNURM) in December 2005 to address the country’s urbanization challenges to help manage the urban space and alleviate urban poverty. The Mission targeted ULBs that could access funds for investment and capacity building in return for a commitment to adopt the obligatory reforms over a period of seven years. As of March 2011, the Government of India (GOI) had committed to provide US$ 14.43 billion in federal resources for these investments for qualifying ULBs for a seven- year period. States and ULBs were then expected to match the federal grants from own-source funds, the capital markets, public private partnerships (PPP), and bilateral and multilateral agencies (PAD, p.4). The program was managed by India’s Ministry of Urban Development (MoUD) and mandated reforms at the state and local level. The main elements of the reform program related to urban management were: (i) adoption of modern accrual based double entry system of accounting; (ii) introduction of a system of e- governance using IT applications, such as Geographic Information Systems (GIS); (iii) reform of property tax; and (iv) levy of reasonable user charges for municipal services. In addition, the JNNURM was designed to assist in pro-poor development of Indian ULBs by enabling the provision of basic services to the poor and supporting integrated development of slums. It was identified at appraisal that the weak capacity of ULBs Page 4 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) was slowing the implementation of both the urban management and poverty reduction reforms in many ULBs (PAD, p.4). This World Bank project was designed to support the GoI’s flagship program to promote urban reforms. The project was prepared under the framework of the World Bank Group’s FY9–12 India Country Assistance Strategy (CAS). Building institutions was one of the key priorities of the CAS, as it recognized the importance of (a) sound governance and institutional arrangements and (b) capacity of local governments, which together promote an enabling environment for the provision of public services. The project objectives were also aligned with the GoI’s Eleventh Five-Year Plan (2007–12), which prioritized strengthening decentralization and empowering local governments for service delivery and improved governance. However, by the time the project was approved, it had few linkages to the JNNURM, the national reform program it was intended to support. Due to delays in the project approval and implementation, the Government had already undertaken urban poverty-related activities and was nearing the completion of the JNNURM. Relevance of the original objective is therefore rated modest. Revised PDO The urban poverty alleviation-related activities were dropped, and the PDO was revised to reflect these changes. MoHUPA received a grant of US$19 million from the U.K. Department for International Development (DFID) for capacity building that overlapped with the project and it was no longer interested in IDA's credit financing. Furthermore, the term "reform" was dropped due to the JNNURM completion , and the Government of India announced the Smart Cities Mission and the National Rejuvenation Mission as part of a set of four new national urban programs. The revised objectives were relevant to the reduced project scope and new priorities of the government following completion of the JNNURM. They were consistent with the Bank Group’s India Country Partnership Strategy (CPS) for FY201-17, which stated that “accelerating urbanization is central to India’s growth, development and poverty reduction”. It would contribute to CPS Engagement Area 2, Spatial Transformation: strengthened institutional capacity of urban governments and improved urban services. Relevance of the revised objective is rated substantial. Rating Substantial 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Improving the systems and skills of select ULBs in urban management. Rationale Page 5 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) This was a technical assistance (TA) project that included a range of capacity building activities in financial management, urban planning, service delivery, and governance that were expected to improve systems and skills of select ULBs in urban management and urban poverty reduction, as well as implementation various urban policy and institutional reforms. The original theory of change was premised on the notion that the above capacity building activities and support in implementation would lead to 20 ULBs having implemented (i) at least two urban management reforms; and (ii) urban poverty reduction strategies. TA outputs and related results • Under the JNNURM, City Development Plans (CDPs) were prepared for 63 cities in 2007–08. These were updated for 30 cities as targeted under the project and were to form the basis for further plans. However, these Plans were dropped as requirements from the new national programs of AMRUT and Smart Cities and thus not utilized by the cities in a manner initially envisaged. The project team subsequently clarified that the national government pre-identified water and sanitation as the priority investment area under AMRUT and asked cities to provide investment plans largely focusing on this priority area. • Training was provided to 28,000 of municipal /elected officers on urban governance and management (target was 2,500) in 22 ULBs (target was 20). The ICR (page 34) clarifies that this was part of individual training plans where 6 institutes were shortlisted by the ministry and State/ULBs sent their officials on a needs basis. • Studies for reduction in non-revenue water (NRW) supply were completed in six cities originally (the target was five) with the objective of assisting the cities in developing a strategy for reduction of NRW. All the six cities, developed and implemented performance-based contracts for water supply. The findings of these studies also became the model provided by the Ministry for the Request for Proposal (RFP) for water supply contracts under AMRUT. Thus, any of the 500 cities that proposed to fund water supply contracts under AMRUT were to follow guidelines of this RFP. Based on demand, NRW studies were completed for an additional 61 cities under the project. • Studies were undertaken for sixteen cities to conduct a comprehensive assessment of the status of sanitation services in the cities and to undertake a city-wide sanitation planning exercise which can be incorporated in future investment plans. However, due to changing investment priorities of the cities, the sanitation plans remained largely unused. • The property tax assessment was conducted in six cities with a view to improve property tax collection (without increasing rates of tax) through widening of tax base by detecting under-assessed and non- assessed properties and included improvements to tax administration by computerization of property tax records, introduction of online assessment and collection, and establishing a dispute redress mechanism. This exercise resulted in an average of 73 percent increase in the property tax base and an over 300 percent increase in the property demand. All six cities implemented the reports under the state/central funds, albeit this was below the targeted 10 ULBs. In response to the demand, studies were later conducted in over 50 cities under the project. • The project introduced the double entry accounting system in 8 ULBs, which was below the targeted 15. Double entry accounting was a mandatory reform under the JNNURM; however, it was only partially implemented in most ULBs. Page 6 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) There were mixed results in relation to the targets; while some were underachieved, others were significantly overachieved. On balance, provided evidence-related findings under the property tax assessment and demand-driven studies that led to implementation of the performance-based contracts for water supply, the achievement of this objective is rated substantial. Rating Substantial PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective Improve planning for urban management in select urban local bodies. Revised Rationale The revised results framework had a narrower focus on the activities related to improving planning for urban management. Technical assistance in preparation of city plans, GIS mapping, credit rating exercise, and training were to improve planning for urban management in select ULBs. • Trainings Needs Assessment was conducted and an ‘Integrated Capacity Building Framework’ was prepared by the MoUD. The framework detailed training in four key municipal areas: (a) Finance and Revenue, (b) Engineering and Public Health, (c) Town Planning, and (d) Administration. The ministry short-listed 6 institutes across states to conduct training in these areas. By the end of the project, about 28,000 ULB personnel had been trained under the project, as compared to the original target of 2,500. The ICR (page 57) reports that on average, around 90% of the participants felt confident to apply the knowledge gained with or without further revision, according to the evaluation carried out by the National Institute of Urban Affairs (NIUA). The following studies were carried out to improve planning for urban management: • Property tax studies were undertaken in six cities that, according to the assessment carried out under the project, resulted in significant increases in the property tax base and demand, consequently resulting in the overall increase in the ULB's own source revenues (ICR, Appendix 1). Property tax contracts were implemented across 55 cities in 12 states. Studies were carried out in 88 ULBs for the preparation of comprehensive Advertisement Guidelines to improve revenues from advertisement tax. The project also provided TA to ULBs in preparing and maintaining a computerized database of all advertisement sites. An assessment was conducted on a sample of five cities that showed a median increase of 52 percent in advertisement tax revenue on the implementation of study recommendations (ICR, Appendix 2). Advertisement tax was implemented across 88 cities in 11 states. Double entry was implemented across 8 cities in 5 states. Overall, this was a total of 151 ULBs, surpassing the project target of 15 ULBs having prepared plans for improved delivery of financial systems-property tax/double entry accounting. Page 7 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) • Reports were prepared for the following areas: (a) non-revenue water studies were completed for a total of 67 cities under the project in response to the demand; (b) city sanitation in 16 cities; (c) city wide drainage in one city; and solid waste management in 9 cities. Overall, 93 ULBs had prepared plans for service delivery- water and sanitation/drainage; this was significantly higher than the targeted 20 ULBs. • For credit ratings, detailed terms of reference were prepared and rolled it out across 300 cities. While all 300 cities commenced the credit rating activity under the project, eventually the project funded 117 reports. The remaining were funded by the cities out of their own funds. The cities of Pune, Bhopal, and Indore undertook credit rating and issued municipal bonds in 2018. • A Livability Index calculation was prepared to rate and benchmark 116 major Indian cities. The cities were rated on parameters such as local governance, social infrastructure, education, employment, health, safety, security, physical infrastructure such as housing, availability of open spaces, land use, energy, availability of water, solid waste management, and pollution. The objective of the index was to help ULBs determine their livability status and to improve their score on the index. The Livability Index is planned by the ministry as an annual exercise and incentive funds in AMRUT are linked to the improvement of the city in the index rankings. • Geographic information system (GIS) mapping was cancelled. On balance, based on the achievement of targets as well as the utilization and assessment results of the studies, the achievement of this objective is rated substantial. While the ICR (page 19) notes that most studies were expected to require approvals and additional steps from states/ULBs for further implementation that may involve institutional and regulatory reform, it is reasonable to conclude that the project contributed substantially to improving planning for urban management in selected ULBs. Revised Rating Substantial PHEFFICACYTBL Objective 2 Objective Improving systems and skills of select ULBs with respect to urban poverty reduction. Rationale The related activities on the implementation of urban poverty reduction strategies in 20 ULBs were dropped and not implemented under the project. The project did not have any direct impact on poverty reduction in the urban sector in India. Rating Negligible PHREVDELTBL PHINNERREVISEDTBL Objective 2 Revision 1 Page 8 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) Revised Objective Assist in the roll out of national urban missions. Revised Rationale • A total of 39 cities (against the target of 50 cities) under the project prepared the City Strategic Plans and all were financed under the Smart Cities Mission. The Smart Cities Mission was a competition-based program by the MoUD, in which 100 cities were invited to apply through a detailed proposal, i.e., the City Strategic Plans. Eventually, the Government financed almost all of the 100 cities. • Technical assistance was provided to cities for developing a Value Capture Financing (VCF) framework (including procedural, legal, and institutional aspects), to effectively capture the additional land/property value being generated through their public investments under the Smart Cities Mission. At the time of preparation of the ICR, the VCF studies in 70 cities had been initiated under the project and the final reports were produced by June 2018. Ghaziabad had already implemented impact fees based on the VCF study. However, the activity was eventually not funded from the project. • The project financed the credit rating of 40 cities. The cities of Pune, Bhopal, and Indore undertook credit rating and issued municipal bonds in 2018. The ICR (page 30) adds that the issuance of municipal bonds has been limited in India with only 30 municipal bonds issued by 15 ULBs by 2018. This is due to a lack of credit assessment and proper fiscal management. Of these 15 ULBs, the project assisted three ULBs. Overall, apart from the target of cities with City Strategic Plans, other targets are not available, and it is unclear from the project's results framework what the planned key activities were. Overall, the project underachieved its target of funding the preparation of 50 Smart Cities Proposals (39 were funded), and the Value Capture Activity initiated under the project was not financed due to contract management issues. The rating is modest for lack of evidence for the achievement of this objective. Revised Rating Modest PHEFFICACYTBL Objective 3 Objective Supporting implementation of various urban policy and institutional reforms. Rationale A total of 5 national-level studies were funded by the project, as targeted, to contribute to national urban policy and programs. A Rapid Baseline Assessment of Urban Management Capacity was carried out that included a detailed survey of 30 cities. It was followed up by a Training Needs Assessment that surveyed 15 cities. The study eventually formed the basis of the ‘Integrated Capacity Building Framework’, which trained ULB personnel under the project in municipal finance, planning, and governance. A study on Land-Based Fiscal Tools was carried out; it became the basis of the Value Capture Finance Policy which was rolled out by the ministry in 2017 under AMRUT and provided ULBs with a framework to leverage land resources to Page 9 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) finance infrastructure. A Municipal Cadre Study was done; by 2017, 21 of 29 states had formed municipal cadres based on the guidelines and recommendations from this study. A Livability Index to rate and benchmark 116 major Indian cities was completed (discussed under Revised Objective 1). Overall, as noted by the ICR (page 16), the project provided guidance and inputs towards urban policy and institutional reforms, however it did not support their implementation as specified in the objective. Urban development falls under the control of state governments in India and central framework documents are only guidance documents for the states. The final adoption and implementation of these are to be done at the state and local level. The project did not provision for support to assist states and ULBs to roll out the outputs. Rating Modest PHREVDELTBL PHOVRLEFFRATTBL Rationale The overall efficacy under the original objective is assessed as modest; while the achievement of the first objective of improving the systems and skills of select ULBs in urban management is rated substantial, the achievement of the second objective related to urban poverty reduction is negligible, and the third objective of assisting in the roll out of national urban missions is rated modest for lack of evidence. The overall efficacy under the revised objective is assessed as modest. The support of rolling out of national urban missions was not clearly defined and the target for city plans was only partially achieved. So while the project significantly contributed to improving planning for urban management in select urban local bodies, as evidenced by utilization of the studies and findings from relevant assessments, there were substantial shortcomings for two of the objectives. Overall Efficacy Rating Primary reason Modest Low achievement 5. Efficiency No analyses to measure efficiency were carried out to under this TA Loan at both project appraisal and closure. The ICR (page 21) explains that economic analysis was not undertaken at appraisal, as while the areas were outlined, no studies had been identified, and thus, no clear link could be established between costs and expected economic benefits. At closure, the ICR only addressed such issues as efficiency in the use of project resources, project execution compared to plans, and the opportunity costs of delays and no execution. Under the original PDO Operational/Administrative Efficiency Page 10 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) There were several factors that contributed to the low efficiency before the project restructuring and revision of the PDO: (i) delays in the appraisal and effectiveness led to significant changes in the project design and scope, and cancellation of the whole component as the related activities had been carried out by the Government; (ii) there were substantial delays in the project implementation for reasons that included frequent changes of staff that affected the decision making at the MoUD, lack of coordination between two ministries, and slow procurement processes. Considerable time and costs were expended in the technical preparation of terms of reference and specifications during implementation. After three years of implementation before the project restructuring in 2015, only 17 percent of the funds were disbursed. Efficiency under the original PDO is rated negligible. Under the revised PDO Operational/Administrative Efficiency The project pace of implementation speeded up with decentralized procurement, however, it was not sufficient to utilize the full amount of the already reduced funding. About half of the revised commitment was not utilized by closure (out of US$ 24 million that remained unutilized, US$12 million accounts for savings due to devaluation of the rupee). With the reduced scope, the project was still extended by an additional two years. Efficiency under the revised PDO is rated negligible, largely for operational and administrative inefficiencies but also for lack of other measures of efficiency. The project team could have attempted to estimate cost effectiveness or carry out a comparative analysis with respect to similar activities in other projects, in particular taking into account the reported benefits under selected TA activities (e.g., property tax increase, impact fees, municipal bonds). Efficiency Rating Negligible a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Point value (%) *Coverage/Scope (%) Available? 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 11 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) Under the original PDO, with modest relevance and efficacy, and negligible efficiency, the overall rating is Highly Unsatisfactory. Under the revised PDO, with substantial relevance, modest overall efficacy and negligible efficiency, the overall rating is Unsatisfactory. Based on a split rating methodology prior and after the project restructuring, when a disbursement share was at 17 percent, the overall project rating is Unsatisfactory. a. Outcome Rating Unsatisfactory 7. Risk to Development Outcome Institutional. The Government commitment to the urban agenda remains strong as evidenced by the continuation of the national programs until 2021. However, as many studies provided inputs to central level policy, they may not trickle down to the state level, unless supported by the government. Financial. The project -financed studies need additional funding requirements to support the implementation, which, as noted by the ICR, may be difficult to raise by some ULBs. Technical. Enhanced capacity through the training of central, state and ULB officials conducted under the project may be affected by routine rotation and reassignment of personnel. 8. Assessment of Bank Performance a. Quality-at-Entry The project was the first Bank engagement with the Ministry of Urban Development. Consultations were held at both national and local levels starting from 2004, and project preparation began in 2006. It took almost five years to prepare involving three Bank TTLs and several changes of the main point person at the counterpart Ministry. Combining two separate projects with different ministries also contributed to the extended preparation time involving coordination efforts and a change in the implementation structure. Despite the long preparation period, there were major shortcomings related to project design: • Project design was too ambitious, spanning two ministries, which required coordinated implementation efforts and significantly diverse areas of support. • The project risk assessment and mitigation measures were inadequate. Most of the risks identified in the PAD were related to the poor procurement capacity and contract management. The mitigation Page 12 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) measures of the project to address coordination issues by (a) clearly earmarking funding and activities under different components, (b) agreeing on a set of 20 cities for the project, and (c) sharing a common PMU proved to be inadequate. Significant emphasis was given to the PMU as a solution to the majority of the key issues raised. • The procurement capacity of the ministries was underestimated. While the target ULBs were determined after consultations and a needs assessment exercise, the procurement decisions on contracts were retained at the central ministry, resulting in little ownership at the local level. While a PMU was put in place to assist the ministries, the design was not consistent with their internal fiduciary systems. • The project was only partially ready for implementation at the time of approval. The identification process of the ULBs began only after the project approval. Consequently, the first two years of implementation were devoted to the start-up of the project activities, including finalization of the selection of participating cities by the implementing ministries, signature of memoranda of understanding with the cities, and hiring of consultants to provide TA to the ULBs. • The M&E design had significant shortcomings (see Section 9). Quality-at-Entry Rating Unsatisfactory b. Quality of supervision As reported by the ICR (page 28), the project was closely monitored by the Bank team throughout the implementation period. The three TTLs and all core team members were based in Delhi for almost the entire duration of the project. Implementation status reports (ISRs) were regularly filed (a total of 13). Starting from 2014, the implementation ratings in the ISRs continued to remain moderately unsatisfactory to highly unsatisfactory. The initial period of implementation was characterized by a slow pace of disbursement. In addition to continuing preparatory activities as discussed above in Quality at Entry, no significant activities were funded under the project for almost a year and half when the ministry was planning to roll out new urban programs, and the cities were to identify activities to implement. The team restructured the project three times to keep the activities relevant and expedite the project implementation. After the second restructuring, there was a demand for project activities, however the ministry did not have enough capacity to procure a large number of contracts (the aggregate number of these studies were estimated at over 300) or monitor their quality and outputs . As procurement and consequently disbursement was an issue through the project duration, supervision focused extensively on how to expedite it. The third restructuring put in place a decentralized procurement system with a clear division of responsibilities at the central and local levels. No safeguard policies applied to the project. As reported by the ICR (page 29), the World Bank’s FM and procurement specialists provided extensive support and closely monitored the fiduciary compliance of the project activities with World Bank policies. Page 13 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) Overall, the supervision team managed to address certain shortcomings at design, in particular the implementation arrangements that included one ministry and streamlining the procurement processes at local level. At the same time, the project’s efficiency remained low, with two additional years of implementation and significantly reduced scope, the project could not utilize half of its already reduced commitment. The ICR reports that a cancellation was discussed in June 2017 but not implemented as the Ministry expected to use the funds and savings resulting from devaluation of the rupee for additional activities. In addition, the M&E design indicators were changed in all three restructurings; however the outputs and targets were not aligned with the increase of funding under component 1 and the overall funds available under the project. Also, as noted by the ICR (page 26), while extensive with several review committees and reports, the M&E arrangement did not adequately address the needs of a project which required dynamic monitoring of contracts and quality assessment. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Unsatisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The original results framework had two PDO outcome indicators that were not clearly defined, i.e., ULBs will be implementing (i) at least two urban management reforms covering FM, urban planning, service delivery and governance, and (ii) urban poverty reduction strategies. Four intermediate level indicators were linked to components 1 and 2 each; while some were precise (e.g., number of ULBs able to finalize their accounts under accrual accounting within a period of one year of close of FY after initial installation of new accounting system), others were less so just referring to the ability of demonstrating improved property tax management or 100 percent degree of operationalization of diagnostic and reduction strategy (PAD, Annex 3). The PMU was responsible for the results monitoring framework on behalf of MOUD and MOHUPA. b. M&E Implementation The project established a three tier monitoring structure: (i) Project Management Unit (PMU) provided day to day monitoring and oversight; (ii) a Project Steering Committee (PSC) chaired by the Project director from the Ministry met regularly at an interval of 3 to 6 months to review implementation; and (iii) the Project Management Board (PMB) provided macro level guidance on the project, meeting once a year. Activities financed under the project kept changing from inception until project closure. The results framework was also revised each time at both outcome and intermediate level during project restructurings, without significant improvements. The two new outcome indicators linked to the revised PDO were output-oriented, i.e., (i) Page 14 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) number of ULBs with City Strategic Plans financed under the Smart Cities Mission, and (ii) Number of policy level studies completed and disseminated (Project Restructuring Paper, 2015, Annex I). The ICR reports (page 26) that, while the results indicators measured the outputs that were achieved, there was no critical analysis on (i) the quality of each output and utilization/impact, (ii) the reasons for outputs were not being produced, and (iii) measures to be taken for delivery of outputs and the effective utilization of these outputs. In summary, the monitoring structure and evaluation arrangements were ineffective. c. M&E Utilization As the ICR (page 26) reports, based on feedback received through the M&E reports on training and financial management reports, support for the roll out of the new national urban missions was included in the revised PDO. However, given the weaknesses in M&E Design, the M&E reports could not be utilized to assess progress toward the achievement of the PDO. M&E Quality Rating Negligible 10. Other Issues a. Safeguards This TA project was assigned Category C at appraisal and did not trigger any safeguard policies. b. Fiduciary Compliance Financial Management. The ICR reports (page 28) that the project complied with the designed financial management systems and the fund flow mechanism in a satisfactory manner, albeit with delays. The project largely submitted accurate and timely Interim Financial Reports (IFRs) for Component 1 as per due dates, whereas those of Component 2 were delayed. The annual external audits were largely prepared on time and the submissions of external audit reports were mostly satisfactory. There were a few delays on account of MoHUPA, when the project had two implementing agencies. The Bank followed up with the Ministry as well as the Department of Economic Affairs to resolve these. Audit observations in the external audit did not raise serious concerns. Procurement. The project followed the procedures of e-Procurement from July 2014 using the Central Procurement Portal of National Informatics Centre. Procurement remained a critical issue at preparation and throughout the implementation stage. The two central ministries were responsible for the entire procurement. They had poor capacity and procurement remained a key bottleneck throughout the project. The ICR (page 27) reports that procurement was largely rated unsatisfactory throughout the project. This was mostly on account of Page 15 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) the low levels of procurement during majority of the project period. After the third project restructuring in 2016, the decentralized procurement helped speed up the implementation, however, as noted by the ICR (page 27), it was difficult to decentralize procurement end-to-end at an advanced stage of project implementation. Thus, the final verification of outputs and payment remained with the Center. The Center had to coordinate with ULBs and verify outputs to pay 312 contracts. The PMU manpower remained the same and processing of outputs and payments took significant time. Thus, many ULBs that initially procured services under the World Bank project eventually opted out and made final payments out of other funding sources. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Unsatisfactory Unsatisfactory --- Bank Performance Unsatisfactory Unsatisfactory --- IEG agrees with the rating of Quality of M&E Modest Negligible the ICR that is negligible (ICR, p.27). Quality of ICR Substantial --- 12. Lessons IEG selected three lessons from the ICR, with some adaptation of the language: • Provision of technical assistance is not a guarantee of interest on the part of local bodies to undertake capacity-building exercise. Under this capacity building project in India, given the large deficit in basic service delivery, urban local bodies (ULBs) are predominantly concerned with building infrastructure and the day to day service delivery. While officials may appreciate the long term positive impact from technical assistance, they might not prioritize it. Knowledge exchange workshops could help to create incentives of undertaking technical assistance. Outputs should enable ULBs to leverage other funds/programs, planning for service delivery which is critical to the city and for which investments have been identified, increase in own source revenues and trainings linked to current work programs. Being able to clearly identify such incentives has two benefits: (a) increase in the likelihood that TA activities will be in Page 16 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) areas which are prioritized for on-ground implementation and (b) World Bank-funded activities becoming upstream works which leverage further funding or enable ULBs to do so. • Funding of ‘wholesale’ TA may bring larger benefits versus ‘retail’ TA. In India, most state-level urban projects have TA components and these have been formulated as comprehensive studies, which are implemented across all ULBs in the states. The impact is considered deep and sustained. Many of them have similar scope to what the project intended to fund, e.g., double entry accounting, geographic information system (GIS) mapping, e-governance systems, property tax enhancement, city level service planning, and training. Consequently, the contracts are large, implemented across a range of ULBs to enable cross- learning and coordination, and anchored in a department that has the mandate and capacity to implement those. If working directly with ULBs, TA assistance could provide a ‘package’ of assistance to few ULBs versus several dispersed studies to a large number of ULBs. • There is a trade-off between complexity of fiduciary requirements and efforts and ownership of outputs produced. This TA project attempted to fund a large number of consultancies/studies. The decision was made to have centralized procurement as it simplifies fiduciary assessments during preparation and monitoring during implementation. Consequently, the centralized implementation structure became a critical bottleneck, and the project introduced the decentralized procurement. The ownership of outputs was higher when the ULBs (beneficiaries) were made a party to the procurement and also led to an increase in demand for TA as ULBs were able to customize contracts based on their needs. The downside of decentralized procurement was the increased need for monitoring the contracts. Thus, this trade-off between fiduciary requirements and demand and ownership for TA should be carefully assessed and implementation arrangements should be designed accordingly. This approach should be combined with increased PMU staff for fiduciary monitoring. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR is outcome-oriented and written in line with the OPCS/IEG harmonized guidelines. It provides a thorough and diligent account of the completed TA activities and studies, complemented by an analysis of the results and outcomes achieved from the project TA outputs (beyond the results framework). The ICR provides a candid analysis of issues that affected the project implementation and achievement of the objectives. Page 17 of 18 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review IN: CBldg for Urban Development (P099979) Lessons are based on the project experience. At the same time, the report seems to be overcritical in the efficacy section basing its judgement largely on the expected long-term impacts of the studies rather than targets that could be possible to achieve within the project timeframe. In addition, a brief description of the results and outcomes would be better suited in the main text rather than referencing them in Appendices (e.g., assessment of training, NRW studies, tax increase). Annex 3 Table on Project Cost by Component erroneously indicates that the actual total project cost was US$45 million at closure. a. Quality of ICR Rating Substantial Page 18 of 18