Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004587 IMPLEMENTATION COMPLETION AND RESULTS REPORT IDA 5213 ON A CREDIT IN THE AMOUNT OF SDR 32.5 MILLION (US$ 50 MILLION EQUIVALENT) TO THE Republic of South Sudan FOR THE Local Governance and Service Delivery Project September 10, 2019 Urban, Resilience And Land Global Practice Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective February 28, 2019) Currency Unit = South Sudanese Pound (SSP) SSP 155.08 = US$1 US$1.40 = SDR 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Hafez M.H. Ghanem Country Director: Carolyn Turk Regional Director: Ede Jorge Ijjasz-Vasquez, Simeon K. Ehui Practice Manager: Meskerem Brhane Task Team Leader(s): Zishan Faiza Karim, Makiko Watanabe ICR Main Contributor: Andrea Fitri Woodhouse ABBREVIATIONS AND ACRONYMS BDC Boma Development Committee CA Coordinating Agency CDD Community Driven Development CEN Country Engagement Note CPA Comprehensive Peace Agreement CPF Country Partnership Framework ESMF Environment and Social Management Framework FCV Fragile, Conflict, Violence GRM Grievance Redress Mechanism ICR Implementation Completion Report IFR Interim Unaudited Financial Report INDH National Human Development Initiative IPF Investment Project Financing IPP Indigenous People’s Plan ISN Interim Strategy Note ISR Implementation Status Report LGA Local Government Act LGB Local Government Board LGSDP Local Governance and Service Delivery Project M&E Monitoring and Evaluation MDTF-SS Multi-Donor Trust Fund for South Sudan MOFEP Ministry of Finance and Economic Planning PBFM Planning, Budgeting, Financial Management PDC Payam Development Committee PDG Payam Development Grant PDO Project Development Objective PMT Project Management Team PMU Project Management Unit PST Project Supervision Team SIDA Swedish International Development Agency SMoLG State Ministries of Local Government TPM Third Party Monitoring WDR World Development Report TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL......................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ........................................................... 10 II. OUTCOME .................................................................................................................... 11 A. RELEVANCE OF PDOS ........................................................................................................ 11 B. ACHIEVEMENT OF PDOS (EFFICACY)................................................................................... 11 C. EFFICIENCY ....................................................................................................................... 17 D. JUSTIFICATION OF OVERALL OUTCOME RATING................................................................. 20 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ........................................................................ 20 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21 A. KEY FACTORS DURING PREPARATION ................................................................................ 21 B. KEY FACTORS DURING IMPLEMENTATION ......................................................................... 22 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 23 A. QUALITY OF MONITORING AND EVALUATION (M&E) ......................................................... 23 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ................................................. 24 C. BANK PERFORMANCE ....................................................................................................... 25 D. RISK TO DEVELOPMENT OUTCOME.................................................................................... 26 V. LESSONS AND RECOMMENDATIONS ............................................................................. 26 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 28 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 38 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 40 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 41 SUB PROJECT COSTS AND BENEFITS .......................................................................................... 42 SUMMARY ............................................................................................................................... 46 SENSITIVITY ANALYSIS .............................................................................................................. 47 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 48 ANNEX 6. FURTHER INFORMATION ON CONTEXT & FACTORS AFFECTING IMPLEMENTATION49 ANNEX 7. THIRD PARTY MONITORING SUMMARY ............................................................... 59 ANNEX 8. DATA SOURCES & RELIABILITY FOR EFFICACY ANALYSIS ........................................ 63 SOURCE.................................................................................................................................... 64 MEASURES ............................................................................................................................... 64 METHODS & COVERAGE ........................................................................................................... 64 LIMITS ...................................................................................................................................... 64 The World Bank Local Governance and Service Delivery Project (P127079) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P127079 Local Governance and Service Delivery Project Country Financing Instrument South Sudan Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Related Projects Relationship Project Approval Product Line Supplement P129581-South Sudan 29-Jun-2012 Recipient Executed Activities Local Governance and Service Delivery Preparation Organizations Borrower Implementing Agency Republic of South Sudan, Ministry of Finance and Ministry of Finance and Economic Planning Economic Planning Project Development Objective (PDO) Original PDO To improve local governance and service delivery in participating counties in South Sudan. Page 1 of 70 The World Bank Local Governance and Service Delivery Project (P127079) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing P127079 IDA-52130 50,000,000 50,000,000 45,891,392 P127079 TF-18138 6,980,505 6,980,505 6,980,505 Total 56,980,505 56,980,505 52,871,897 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 DENMARK, Govt. of 13,500,000 0 0 NETHERLANDS, Govt. of THE (Except for MOFA/Min of 25,000,000 0 0 Dev. Coop NORWAY, Gov. of (except for Ministry of Foreign 10,000,000 0 0 Affairs) Total 48,500,000 0 0 Total Project Cost 105,480,505 56,980,505 52,871,897 KEY DATES FIN_TABLE_DAT Project Approval Effectiveness MTR Review Original Closing Actual Closing P127079 28-Mar-2013 21-Feb-2014 04-Sep-2016 31-Dec-2018 28-Feb-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 19-Dec-2018 51.50 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Implementation Schedule Page 2 of 70 The World Bank Local Governance and Service Delivery Project (P127079) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Satisfactory Modest RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 03-Aug-2013 Satisfactory Satisfactory .59 02 30-Mar-2014 Satisfactory Satisfactory 1.42 03 10-Aug-2014 Satisfactory Moderately Satisfactory 5.30 04 11-Feb-2015 Satisfactory Satisfactory 8.25 05 11-Jun-2015 Satisfactory Satisfactory 12.50 06 11-Dec-2015 Satisfactory Moderately Satisfactory 20.03 07 29-Apr-2016 Satisfactory Moderately Satisfactory 29.82 08 28-Dec-2016 Moderately Satisfactory Moderately Satisfactory 33.83 09 31-Jan-2017 Unsatisfactory Unsatisfactory 34.38 Moderately 10 27-Sep-2017 Moderately Satisfactory 42.12 Unsatisfactory Moderately 11 11-May-2018 Moderately Satisfactory 47.63 Unsatisfactory Moderately 12 27-Nov-2018 Moderately Satisfactory 50.96 Unsatisfactory SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 12 Other Agriculture, Fishing and Forestry 12 Page 3 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Public Administration 34 Sub-National Government 34 Social Protection 30 Social Protection 30 Transportation 12 Rural and Inter-Urban Roads 12 Water, Sanitation and Waste Management 12 Other Water Supply, Sanitation and Waste 12 Management Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Public Sector Management 29 Public Administration 29 Administrative and Civil Service Reform 9 Municipal Institution Building 20 Social Development and Protection 59 Social Inclusion 13 Participation and Civic Engagement 13 Fragility, Conflict and Violence 46 Conflict Prevention 23 Post-conflict reconstruction 23 Urban and Rural Development 12 Urban Development 12 Municipal Finance 12 Page 4 of 70 The World Bank Local Governance and Service Delivery Project (P127079) ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Hafez M. H. Ghanem Country Director: Bella Deborah Mary Bird Carolyn Turk Director: Ede Jorge Ijjasz-Vasquez Simeon Kacou Ehui Practice Manager: Rosemary Mukami Kariuki Meskerem Brhane Zishan Faiza Karim, Makiko Task Team Leader(s): Tesfaye Bekalu Wondem Watanabe ICR Contributing Author: Andrea Fitri Woodhouse Page 5 of 70 The World Bank Local Governance and Service Delivery Project (P127079) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. The Local Governance and Service Delivery Project (LGSDP) was launched during the period of immense optimism that followed South Sudan’s independence in July 2011 . After decades of war between the pro-independence Sudan People’s Liberation Army and the Government of Sudan, the 2005 Comprehensive Peace Agreement (CPA) augured a time of relative peace and stability. Poverty rates decreased, GDP grew, and the government benefited from windfall oil revenues and a massive influx of external grants. Receipts at the close of the Multi Donor Trust Fund for South Sudan (MDTF-SS) stood at US$728 million,1 25 percent of which was from the government. 2. The new country urgently needed to improve basic infrastructure and services—and establishing a functioning state at the local level was critical for this. Civil war had left the new country with some of the worst socioeconomic conditions in the world. South Sudan thus needed to deliver a peace dividend in the form of basic infrastructure. Yet local governments, which were primarily responsible for delivering upon this, had little capacity to do so. 3. The government thus embarked on a rapid state-building agenda, bolstered by substantial financial support from international partners. This included a focus on strengthening local government. In 2009, during the post-CPA period when Southern Sudan was preparing for independence, the government passed the Local Government Act (LGA). 2 This had provisions to establish inclusive local governance structures in payams (sub-counties) and bomas (village clusters) and to devolve core government functions to the country’s ten states, 79 counties, and payam and boma sub-structures. County governments were responsible for delivering basic services. To support them, the government planned to provide “county block grants” to finance county infrastructure and sought donor funds to build local government capacity and rehabilitate community infrastructure through payam block grants. (Further detail is in the annexes). 4. After decades of war, the government also needed to build state legitimacy and reduce the risks of further conflict . At independence, South Sudan was deeply fractured. Conflict had been a matter not only of resistance to Khartoum or of intra-elite competition but also of local competition over resources. Such tensions were manifested in border disputes, cattle rustling, and conflict between pastoralists and agriculturalists; these were intensified by war and the spread of modern arms into communities. 5. LGSDP was envisaged as a flagship in the World Bank’s South Sudan portfolio. In April 2012, South Sudan became the newest member of the World Bank Group. In February 2013, two months after LGSDP appraisal discussions, the World Bank released its Interim Strategy Note (ISN) for FY2013-14. The ISN argued that South Sudan needed to lay the foundations for effective and accountable institutions, which would use the country’s resources to respond to citizen needs. The ISN focused on building legitimate institutions to mitigate the drivers of instability, including by developing local government capacity to deliver services. LGSDP aimed to do exactly this. It became effective in February 2014. 3 6. South Sudan’s promise at independence, however, rapidly gave way to civil war and economic collapse, leading the project to adjust from supporting the rebuilding of state capacity to focusing on services for poor, rural populations living amid civil war. Yet amid this unraveling, much of the optimism around South Sudan’s nationhood persisted. In December 2013, just before the project became effective, the country’s fragile political settlement fell apart. War broke out and spread, increasing local and intercommunal conflict.4 By late 2017, 7.5 million of South Sudan’s estimated population of 12.3 million needed humanitarian assistance and 4 million were forcibly displaced. 5 By 2018, some Page 6 of 70 The World Bank Local Governance and Service Delivery Project (P127079) 380,0006 persons had lost their lives. The country’s already high rates of gender-based violence grew worse, along with rape, gang rape, and sexual slavery of women, young girls, and young boys.7 The economy collapsed, inflation soared, and the currency plummeted. The government also announced the creation of new states and counties, causing considerable upheaval in local governance arrangements. Yet despite this volatility, there was a continued sense of optimism and hope in the international community that these issues would be short lived. 7. LGSDP provides lessons for World Bank operations in FCV environments that experience repeated cycles of violence . In such settings, institutions have often been decimated, leading to trade-offs between the competing aims of delivering services quickly and building a functioning state. The project took a hybrid approach, prioritizing both building local government and delivering community-prioritized services. It was designed as the first phase of a 15-year institution- building engagement and explicitly took on the principles of the New Deal for Engagement in Fragile States and the lessons of the 2011 WDR on Conflict, Security, and Development, which emphasized the need for FCV operations to have flexible approaches to risk and long horizons, to avoid building unmet expectations, and to be responsive enough to local conditions to enable national confidence-building efforts to be supported effectively. LGSDP had a sensitive project design and an experienced team. And yet, the reality of changing conditions on the ground led to mixed results. Theory of Change (Results Chain) 8. A theory of change was not required in the PAD, so this ICR reconstructs one based on project documents and discussions with the task team. The project aimed to support the higher-level objectives of supporting nation-building and peace, through catalyzing the establishment of responsive, legitimate, accountable local government institutions. It sought to achieve the objective through: (a) improving local governance (engaging citizens in making decisions about their own development and building the capacity of local government institutions to plan, manage, and oversee local development) and (b) improving service delivery (by supporting county governments with basic infrastructure). These medium-term outcomes were captured in the PDO. 9. The project aimed to improve local governance through strengthening community engagement and local government capacity in planning, managing, and overseeing local development. To strengthen community engagement, facilitating partners (NGOs) conducted training, conflict mapping, and other facilitation (activities) to help communities set up and participate in inclusive planning processes via functioning boma and payam development committees (outputs). This, in turn, would lead to strengthened community engagement in planning, managing, and overseeing local development (outcome) in support of the higher level objectives of a more legitimate system of local governance and greater peace. These boma and payam development committees were not parallel structures but were part of the local government system provided for in the LGA. To strengthen local government capacities, the county governments would be provided with payam block grants (outputs), and government staff would be coached, mentored, trained, and learn on the job in managing these payam block grants (activities). This experience would result in improvements in their capacity to plan, implement, and oversee local development (outcomes), and thus to better functioning, more accountable, and legitimate government, and thereby nation-building and greater peace (higher level objectives). 10. The project aimed to improve service delivery by expanding basic infrastructure. Strengthened community engagement and improved local government capacity were meant to facilitate this in the project, as communities themselves would identify their own infrastructure priorities and county governments would manage the grants under which these infrastructure sub-projects could be funded. County governments would spend the payam block grant funds (activities) to finance rehabilitated or newly built community-prioritized basic infrastructure (outputs), the supply of which was expected to lead to improved service delivery (outcomes). Over time, this too was expected to contribute Page 7 of 70 The World Bank Local Governance and Service Delivery Project (P127079) to nation-building (higher level objectives), on the assumption that seeing visible improvements in service delivery was likely to lead citizens to have confidence in and trust the state. Figure 1: Theory of Change 11. Underlying the theory of change were several assumptions about South Sudan’s security and economic situation and its political economy. The most critical were that: (a) funds would flow through the government system mostly as planned; (b) there would be enough security for the project to operate; (c) the economy would be stable enough for labor and raw materials to be available and affordable; and (d) the system of local government would remain sufficiently stable for improvements in its capacity to be sustained. In moving from the PDOs to the higher-level objectives, the theory of change also contained less critical assumptions about how societies transition from conflict to sustained peace, for example: (a) citizens who get better services are more likely to trust government, (b) citizens who plan jointly engage less in conflict, and (c) that nations tend to consolidate peace gains when their institutions of government are good at what they do and accountable to citizens. Project Development Objectives (PDOs) 12. The project development objective (PDO) was to improve local governance and service delivery in participating counties in South Sudan. Key Expected Outcomes and Outcome Indicators 13. The PDO statement includes two objectives with three main outcome areas. Page 8 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Table 1: PDO objectives and outcome areas Objectives Outcome areas Objective 1: To improve local governance in  #1 Strengthen community engagement in planning, managing, participating counties in South Sudan and overseeing local development  #2 Improve local government capacities in planning, managing, and overseeing local development Objective 2: To improve service delivery in  #3 Improve service delivery participating counties in South Sudan 14. These were measured using the following key PDO indicators. Table 2: Key PDO indicators Objectives & indicators Objective 1: To improve local governance in participating counties in South Sudan Outcome area 1: Strengthened community engagement in planning, managing, and overseeing local development PDO I #1 Percentage of intended beneficiaries that are aware of project information and project-supported investments (core sector indicator) PDO I #2 Percentage of participating payams with functioning payam development committees PDO I #3 Percentage of subprojects or investments for which arrangements for community engagement in post-project sustainability and/or operations and maintenance are established (core sector indicator) PDO I #6 Number and percentage of project beneficiaries who are female 8 (core sector indicator) Outcome area 2: Improved local government capacities in participating counties in South Sudan PDO I #4 Percentage of counties remaining eligible in subsequent year for the payam development grant. Objective 2: To improve service delivery in participating counties in South Sudan Outcome area 3: Improved service delivery PDO I #5 Percentage of financed subprojects functioning and delivering services to communities one year after completion (core sector indicator) Components 15. To achieve these aims, the project had four components:  Component 1: Block grants to counties for payam development. Estimated cost (IDA & TF): US$30,000,000; actual cost (IDA & TF) US$8,787,310: This provided county governments with “payam development grants” (PDGs) to finance community-selected basic infrastructure. Responsibility for budgeting, procurement, accounting, and overseeing implementation rested with county governments. The component aimed to: (a) provide an incentive for community members to participate in planning and overseeing local development activities; (b) help build county government staff capacity by enabling them to ‘learn by doing’ in managing budgeting, procurement, accounting, and implementation; and (c) expand service delivery in communities through small-scale infrastructure. Page 9 of 70 The World Bank Local Governance and Service Delivery Project (P127079)  Component 2: Community engagement. Estimated cost (IDA & TF): US$25,400,000; actual cost (IDA & TF) US$11,894,580: This aimed to support citizens to plan, implement, and oversee local development activities in the boma, payam, and county levels. The component supported community members to (a) map local conflict; (b) establish or validate representative development committees in bomas and payams, (c) identify priority infrastructure investments through a facilitated local planning and prioritization process; (d) manage and oversee local public infrastructure investments to ensure quality and access; and (e) have a means of providing feedback and resolving grievances. The planning process and resulting payam action plans were intended to be integrated into county development plans and provide a framework for other development investments.  Component 3: Institutional strengthening. Estimated cost (IDA & TF) US$22,000,000; actual cost (IDA & TF) US$15,457,081: This aimed to increase the capacity of county governments to plan, implement, and oversee local development activities, in accordance with their responsibilities under the 2009 Local Government Act. The component financed technical assistance and training for state and county government staff on: (a) participatory local development planning and budgeting; (b) financial management; (c) procurement; (d) technical (engineering) aspects of local infrastructure planning and implementation; (e) monitoring and reporting; f) environmental and social safeguards management; (g) social accountability; and (h) communication and information dissemination. Component three enabled government staff to better manage the block grants to counties to finance the infrastructure investments (component one) identified through community engagement (component two).  Component 4: Project Management. Estimated cost (IDA & TF) US$21,100,000; actual cost (IDA & TF) US$15,454,436: This component supported the Ministry of Finance and Planning (MoFEP), the Local Government Board (LGB), and State Ministries of Local Government (SMoLG) to manage and report on project implementation and to support and monitor the performance and outputs of the PDG and associated TA at community and county levels. This component financed the staffing, equipment, and operational budgets for a dedicated Project Management Unit (PMU) at central level, supported by state level personnel, to (a) manage the project; (b) contract for spot checks and audits; (c) implement a grant monitoring system, project monitoring system as well as project evaluations based on data collection at mid-term; and (d) implement, in collaboration with the CA, a feedback and grievance redress mechanism for the project. 16. The estimated and actual resource allocations for each component differed. For component 1, PDG disbursements were suspended twice (first to manage exchange rate fluctuations and then to manage conflict-related fiduciary risks), while disbursements across the other components (which mainly consisted of overhead for facilitating partners, the technical assistance firm, and the PMU) continued. The disbursement in component 3 for institutional strengthening and project management are therefore higher and consistent with the projects expected disbursement whereas the disbursement for component 1 is less than what was expected during project design. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 17. There were no changes in the PDOs, outcome targets, PDO indicators, or components throughout the life of the project. At the start of the project, bilateral donors withdrew most of their financing. Given this financing gap and the uncertainties posed by South Sudan’s crises, between July 2016 and August 2018, Bank management discussed, but did not pursue, a restructuring with additional financing. Instead, the decision was to finance a new project and close the existing one. Page 10 of 70 The World Bank Local Governance and Service Delivery Project (P127079) II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating Rating The overall relevance of the PDOs to the World Bank Country Engagement Note for the Republic of South Sudan (FY18-19) is substantial. The ratings for each objective are as follows.  Objective One: improve local governance in participating counties in South Sudan: substantial  Objective Two: improve service delivery in participating counties in South Sudan: high The objective of improving local governance is assumed to have somewhat more weight in contributing to the aggregate outcome, yielding a consolidated rating of substantial. 18. Although the PDO was highly relevant to the World Bank’s ISN at appraisal, the country soon began to experience repeated cycles of violence. Whereas the Bank’s strategy at appraisal9 focused on building legitimate long-term institutions, the strategy at closing10 was sympathetic to the view that ‘international actors were rushing too fast toward development and state-building without consolidating peace.’11 The CEN acknowledged that South Sudan needed to build institutions and left open the possibility of an institutional development project under appropriate conditions, 12 but otherwise made little reference to institution-building, focusing on livelihoods and service delivery. 19. The PDO at closing was thus less relevant than at appraisal, and overall is rated as substantial:  Objective 1 has substantial relevance. Although improving local governance is still critical, the 2018 CEN does not emphasize building institutions; thus implying that enabling government officials to learn by managing grants delivered through country systems no longer has ‘high’ relevance. The CEN nevertheless emphasizes the importance of service delivery and acknowledges the key role of institutional development in delivering services. Consequently, its relevance is still substantial, rather than negligible. The project also was designed as part of a long-term incremental process, which aimed to put in place building blocks for governance capacity in tandem with service delivery.  Objective 2 continues to have high relevance. At closing, the service delivery needs of the population were higher than before the conflict. The CEN emphasized this fact, highlighting that less than 2 percent of roads were paved and three in four people lacked access to sanitation.13 One of the two objectives in the CEN was, therefore, to ‘support basic service provision for vulnerable populations.’ Improving service delivery thus remains highly relevant. B. ACHIEVEMENT OF PDOs (EFFICACY) 20. This ICR triangulates several data sources to assess project efficacy, despite the M&E challenges resulting from an active conflict environment. South Sudan’s civil war led to security restrictions for World Bank staff, preventing field visits during implementation and ICR preparation. In 2017 the project rolled out portfolio-wide geo-enabling of sub- projects and third-party monitoring through the International Organization for Migration (IOM). These actions, and a PMU-conducted beneficiary survey, has enabled the ICR to verify project outputs and triangulate information. The main sources and limitations of evidence are presented in Annex 8 and a summary of the third-party monitoring findings are presented in Annex 7. Page 11 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Assessment of Achievement of Each Objective/Outcome Rating The overall efficacy of the project was modest. The ratings for each objective are as follows.  Objective 1: To improve local governance in participating counties in South Sudan: modest  Objective 2: To improve service delivery in participating counties in South Sudan: modest Although the project partly, substantially, or fully met its PDO targets (Table 4), it rolled out in far fewer counties than planned (Table 3) due to conflict and project financing gaps, but was not restructured. This affects the rating. Table 3: Planned vs. actual geographic scope by component* Component Objective/outcome area Target Actual counties counties PDG block grants Improve service delivery 40 14 Community Improve local governance (strengthen community 40 25 engagement engagement) Institutional Improve local governance (improve local government 40 2314 strengthening capacities) The project’s detailed county selection criteria aimed at increasing scope sequentially and incrementally. These included: population, accessibility, security, other donor assistance, and administration capacity. Each year, a joint government team traveled to states and counties to apply these criteria and identify counties to be covered the next year. * Third Party Monitoring covered 9 out of the 14 counties that received PDG block grants, 5 of the 25 counties provided community engagement support, and 6 of the 23 counties provided institutional strengthening support. Page 12 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Table 4: Achievement of PDO indicators, disaggregated by outcome area* PDO # Description Baseline Target Actual Objective One: Improved local governance Strengthened community engagement (outcome area 1) PDO I #1 Intended beneficiaries that are aware of project information 0% 60% 62% and project supported investments PDO I #2 Participating payams with functioning PDCs (%) 6%15 80% 67.7% PDO I #3 PDG projects for which arrangements for community 0% 80% 52%16 engagement in post project sustainability and/or O&M are established (%) PDO I#617 Direct project beneficiaries (number) aggregated by sex (female 0% 48.1% 774,466 (Female and male %) 47%, male 53%18 Strengthened local government capacities (outcome area 2) PDO I #4 Counties remaining eligible in subsequent year for the PDG (%) 0 80% 44.4% Objective 2: Improved service delivery Improved service delivery (outcome area 3) PDO I #5 Percentage of financed subprojects functioning and delivering 0 80% 90% services to communities one year after completion *The percentages in the results framework are proportions only for counties where the project rolled out, not all counties. Objective 1: Improve local governance in participating counties in South Sudan Rating: Modest: the operation partly achieved its objective of improving local governance (it was more successful at strengthening community engagement than improving local government capacities). 21. The PAD envisaged improving local governance as both a community and local government endeavor : It involves ‘strengthening community engagement’ and ‘improving local government capacities’ in planning, implementing, and overseeing local development. Community engagement is seen to be strengthened to the extent that functioning local planning, implementation, and accountability processes are established, which include a range of community members and members of vulnerable social groups and provide a forum for transparency and accountability. Local government capacities are seen to be improved to the extent that county government functionality and capabilities improve. Outcome area 1: Strengthened community engagement 22. The project exceeded or substantially met its PDO indicators related to strengthened community engagement (Table 4) but did this in far fewer counties than planned (Table 3). Where it rolled out, the project substantially succeeded at enabling citizens to become more engaged in planning, implementing, and overseeing local development. The project almost met its target for the percentage of payams with functioning PDCs (67.7 vs 80 percent, PDO I #2).19 Community members participated in implementation by clearing sites, creating pathways, providing food for laborers, and collecting stones, sand, timber, and wood. They also participated in oversight: the project partly met its target for the percentage of sub-projects with arrangements for post-project sustainability and/or operations and maintenance (52 vs 80 percent, PDO I#3). This was corroborated by TPM, which found that more than half of subprojects had a user fees system to fund maintenance costs. Although the borrower’s report found uneven adoption of O&M (attributing this to O&M not being Page 13 of 70 The World Bank Local Governance and Service Delivery Project (P127079) in implementation plans and community members being insufficiently trained), this should not be interpreted as evidence of poor citizen engagement as the project originally conceived of O&M as a local government responsibility. 23. The quality and inclusiveness of this participation was substantial . The beneficiary assessment found that the project exceeded its target for beneficiary awareness of the project (6220 vs 60 percent, PDO #1). Although this number holds limited weight (its underlying sampling did not enable representativeness beyond the sample of respondents), the qualitative evidence strongly corroborated it, finding that respondents understood the project’s purpose and knew they had a right to participate and be members of the BDC through a democratic process. 21 Other evidence also indicated success. In every county, community engagement activities began with conflict mapping and a campaign aimed at reaching different micro-social groups, including “women, youth, internally displaced people (IDPs), refugees, returnees, people with disability, minority tribal groups, and others”. An independent assessment 22 of the project’s community engagement methodology found that project processes were inclusive. This extended to gender. Project administrative data shows that the project substantially met the 50 percent threshold for women’s participation despite a context of strong patriarchal social norms (women comprised 41 percent23 of PDC members and 34 percent of BDC members). The beneficiary assessment corroborated these levels of women’s participation, though found lower levels of women’s participation at payam level.24 The TPM report also corroborated these gender figures, found the project to have a ‘great degree of inclusivity’,25 and found that people with disability were included in BDCs and PDCs, though it could not say if disability representation was proportional. The beneficiary assessment also perceived the quality of participation to be high, noting that the project represented a shift from “the traditional way of decision making,” where “chiefs and cultural leaders were central to the process,” and from “traditional societal norms where men dominate.” 24. There were, however, shortcomings. The project’s grievance-handling mechanism (GRM) had some deficiencies. The project responded to 91 percent of grievances, surpassing its 80 percent target (IR 11). However, the beneficiary assessment found that only two in ten people were aware of the project’s GRM, suggesting that although the project itself was responsive, its potential as a vehicle for accountability was limited. The beneficiary assessment also found that grievances to county offices did not get adequate feedback. 26 Furthermore, an independent assessment27 of the project’s community engagement methodology found mixed reviews for GRM, as did third party monitoring. 25. The existing evidence does not enable this ICR to compare outcomes in participating versus non-participating counties for the purposes of attribution. The committees did not replace pre-existing bottom-up governance structures but were introduced in the context of a top-down system predicated upon the power of traditional chiefs, in a context of patriarchal social norms and, although there were other community-oriented development projects in South Sudan, 28 these were not linked to the formal local government system. It is therefore probable that, in counties where BDCs and PDCs were established, LGSDP contributed to improving community engagement. Outcome area 2: Improved local government capacities 26. The project partly achieved its PDO indicator related to improved local government capacities (Table 4), but did so in fewer counties than expected (Table 3). Where it rolled out, county government functionality and capabilities improved, but with varying success across types of skills and limited sustainability, indicating modest success. Before the project, local government capacities were low. Counties did not follow proper procurement or financial processes and had poor PBFM (planning, budgeting, and financial management), procurement, and infrastructure design, appraisal, implementation, and maintenance capacities.29 The project partly succeeded in changing this. A 2017 Local Government Performance Assessment (LGPA) found that 44.4 percent of assessed counties attained above-average ratings, enabling them to remain eligible the following year for the payam development grant, meaning the project fell Page 14 of 70 The World Bank Local Governance and Service Delivery Project (P127079) short of its 80 percent target for PDO indicator #4. In addition, only 11 percent of assessed counties attained a clean report (IRI #13). However, the project surpassed its intermediate results indicator target for ensuring that participating counties submitted approved budget plans each year (100 percent actual against a target of 85 percent) (IRI#12). Finally, some capacities improved more than others. The TA firm hired to train local government officials also tracked its own key performance indicators, which showed that PBFM and procurement capacity improved more than infrastructure design, appraisal, implementation, and maintenance capacities. The LGPA corroborated these patterns. 30 27. Perceptions data corroborated such improvements in local government capacity. County officials interviewed as part of the study31 on institutional strengthening reported that the project had improved executive management performance and enabled county and state officials to have a ‘better understanding of proper PBFM, planning, and infrastructure processes.’ They perceived the project’s impact to be substantial, noting that their counties now had core operational procedures and systems in place, held regular planning and budget meetings, and had functional financial and other reporting structures. These findings were corroborated by the project’s beneficiary assessment, which noted that local government staff now came up with work plans and budgets and kept books of accounts, and that county procurement procedures had changed from a system where politicians were the primary decision makers to one where ‘procurement officers… do market analysis, collect quotations and conduct proper evaluations.’32 28. These apparent improvements, however, need to be weighed against the upheavals in local government caused by South Sudan’s expansion of states and counties, the TPM findings on local governance - which showed mixed results, and the borrower’s report - which highlighted challenges in building capacities of staff with limited minimum skills . According to the borrower’s report, the expansion of states and counties led to significant turnover of local government staff (with the implication that the gains from their training might not be sustained), and the country’s runaway inflation led to lowered morale among government officials, whose salaries did not keep pace. This staff turnover was confirmed by the TPM report. The TPM report also identified mixed results with local governance. In most of the six examined counties, county staff members were able to describe procurement and other such procedures, indicating that they had been appropriately trained. Most county staff, however, were not able to show related documents to the third-party monitors, because they had poor filing systems or said they were not authorized to do so. The TPM report noted that all counties lacked good filing systems, and that documents such as budgets and financial reports were written in English, something that created difficulties for those county staff who did not speak the language. Finally, the borrower’s report highlighted the limitations of a classroom and on-the-job-training model of capacity building for county officials who often lacked minimum technical qualifications, implying that the project’s expectations in this regard were too high—citing the mid-term review view that for such staff ‘no amount of skilled training can take effect.’ Objective 2: Improve service delivery in participating counties in South Sudan Rating: Modest: the operation partly achieved its objective of improving service delivery 29. The PAD envisioned improved service delivery in terms of expanded basic infrastructure. The key expected results were expanded access to services, to be measured by the percentage of financed subprojects functioning and delivering services to communities one year after completion—a PDO level outcome indicator. Outcome area 3: Improved service delivery 30. The project met its PDO indicator targets related to improved service delivery (Table 4) but did so in fewer counties than planned (Table 3: Planned vs. actual geographic scope by component*). Outputs as reported by the borrower were as follows. By the time the project closed, 439 sub-projects were implemented, though 30 of these were abandoned by Page 15 of 70 The World Bank Local Governance and Service Delivery Project (P127079) contractors due to inflation, and one was not functioning. 408 sub-projects were thus completed and functioning, enabling the project to substantially meet its intermediate results target of building 500 sub-projects.33 The project built 69 primary health care units, 30 primary health care centers, 142 classroom blocks in 60 primary schools, three primary access roads of about 180 km, 105 boreholes, three local markets, one livestock market, 202 ventilated improved pit (VIP) latrines, and one storm water dyke. 656,023 beneficiaries benefitted from PDG subprojects over the four years. Thus, the project met its PDO target for improved service delivery, despite rolling out to fewer counties than planned due to conflict, insecurity, and project financing gaps. 90 percent of financed subprojects were functioning and delivering services to communities one year after completion.34 There were, however, some issues with operations and maintenance: for boreholes and hand pumps, finding spare parts on the market was difficult. Technical repair was also a challenge: this was due to the lack of available resources in government and reduced intergovernmental fiscal transfers following macroeconomic crisis. 31. These outputs were broadly corroborated by TPM. IOM examined the 293 sub-projects which had been completed when the third-party monitoring began (the remainder were constructed later so were not included in the TPM sample). 33 of these could not be accessed because of conflict or a lack of all-season road access during the rainy season. Out of the remainder, IOM found that 97 percent existed, while 3 percent (eight sub-projects) had either been deferred (because of the rainy season) or constructed through the LGSDP participatory development planning process but using funds from other development partners (in other words, not directly from LGSDP money but still as an outcome of the improvements it brought to community capacity for local development planning). TPM found that 95 percent of the subprojects were completed at the time of monitoring, with the remaining 5 percent late because of insecurity. TPM also found that overall, 86 percent of completed and verified subprojects were delivering services. 14 percent were not, or not yet, offering a service at the time of the visit. Among completed sub-projects not offering a service, almost 38 percent were boreholes or hand pumps and 35 percent were latrines. Interviews conducted for this ICR highlighted challenges with obtaining spare parts for boreholes and hand pumps, which could explain this. Table 5: Planned vs. implemented sub-projects as reported by borrower Planned sub-projects Implemented Deferred Reason for deferral 2013/14 (completed between 32 32 0 FY13/14 and FY 15/16) (31 functioning) 2015/16 (completed in 2016/17) 307 269 38 Insecurity 2016/17 (completed in 2017/18) 756 138 618 Lack of financing (108 completed, 30 abandoned) Total 1095 439 656 32. The project’s beneficiary assessment found that community members were broadly satisfied with the quality of services. Community members were asked to rate the structure, functionality and utilization of sub-projects. The percentage of focus group participants rating the quality of infrastructure as good was as follows: markets (78 percent), schools (75 percent), health centers (73 percent), boreholes (66 percent) and dykes (67 percent); the rating was low for the one road that was visited (0 percent), because it had broken culverts and so was flooded during the rainy season. 35 The beneficiary assessment reported that community members appreciated LGSDP, to the extent that the news of its closure caused concern among the community members. Page 16 of 70 The World Bank Local Governance and Service Delivery Project (P127079) 33. During implementation, the project took measures to improve the quality of and access to infrastructure, increasing the project’s per capita allocation to enable this. This was verified by the TPM. As the initial round of LGSDP rolled out, the project learned that its per capita allocation for PDGs was insufficient for implementing subprojects that met environmental standards, were of good construction quality, and enabled community members to fully use and access the service.36The project therefore doubled its per capita allocation from US$3.5 per capita to US$7 to improve standards. The TPM found that most types of sub-projects were built according to these improved standards. 34. The project did not collect data on the numbers of people who were able to access infrastructure, but the beneficiary assessment and TPM data indicated that sub-projects had increased access . The TPM report noted that “100 percent of all the various subprojects were easily accessible to the general populace living within some kms of the sub-projects.” Meanwhile, perceptions data from the beneficiary assessment indicated that subprojects had indeed increased access.37 The mechanisms for this varied. The construction of school classroom blocks was reported to have increased the number of children reporting to schools, though the expansion of access was somewhat constrained by community norms holding children back to help graze cattle, by early marriage for girls, and the distance of schools from communities. The construction of health centers was reported to have eased access to health services, reduced travel distance, and to have ensured the provision of some services that were previously not offered, namely child immunisation services, nutritional services for children, and diagnosis of malaria—but access was sometimes constrained by challenges such as the limited availability of health workers and drug stocks. Boreholes were reported to have improved access to clean water, enabled children to go to school by reducing the time needed to collect water, and reduced consumption of contaminated water from open wells and rivers, but the ratio of boreholes to people was sometimes high, which was reported as a source of conflict. C. EFFICIENCY Assessment of Efficiency and Rating Rating The overall efficiency of the project is modest. Despite this, completed sub-projects had a high overall (economic) internal rate of return. However, a reduction in geographic coverage and in outputs due to economic collapse and conflict, along with lowered overall efficiency due to exchange rate and conflict-related gaps in disbursement, caused a less-than-efficient overall use of funds. 35. This efficiency analysis is based on data observed during the project’s life and at closing, albeit with limitations related to South Sudan’s FCV context. Data to measure project performance and progress were gathered through audits, budget and performance reports, term reviews, and annual assessments. (The project was audited annually, and participating counties were audited at the end of each fiscal year). However, the outbreak of violence during the project’s life limited such reporting, with substantial impact on the assessment of the project. 36. An economic and financial analysis to assess the economic viability of the implemented and completed subprojects was carried out for this ICR. Specifically, an ex-post38 cost-benefit analysis39 was carried out based on observed and relevant empirical data. Due to the economic and security situation, a very conservative discount rate of 6 percent was used for the analysis, in accordance with the World Bank guidelines on social discount rate 40 and investments analyzed over the expected life span of each of the sub-projects, estimated in 20 years. Page 17 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Table 6: Planned versus actual expenditure, US$ Project Cost Actual IDA & Donor End of Project’s Component (as per PAD at Appraisal) Commitment Total Expenditure IDA TF IDA TF IDA TF 1 Block grants to counties $14,800,000 $15,200,000 $5,900,902 $2,930,084 $5,857,226 $2,930,084 2 Community engagement $12,600,000 $12,800,000 $10,950,000 $972,663 $10,921,917 $972,663 3 Institutional strengthening $10,900,000 $11,100,000 $13,545,259 $1,992,098 $13,464,983 $1,992,098 4 Project management $11,700,000 $9,400,000 $14,632,364 $1,085,660 $14,368,776 $1,085,660 Exchange loss. $4,971,575 $4,971,575 $50,000,000 $48,500,000 $50,000,000 $6,980,505 $49,584,477 $6.980,505 Total $98,500,000 $56,980,505 $56,564,982 Source: Government of South Sudan Data (2019) 37. No ex ante economic analysis was carried out during appraisal, so this ICR conducts an ex-post assessment. Because no ex ante economic analysis was done, no anticipated cost-benefit indicators such as the project’s expected net present value (NPV) and IRR were available for an ex ante/ex post comparative analysis of costs and benefits. However, assessing a project’s economic impact remains a key indicator of success. This ICR thus conducts an ex post assessment. 38. The analysis shows a positive NPV and a robust IRR, suggesting that the efficiency of the project in improving service delivery was high in areas where the project rolled out. This held across different types of sub-projects: each of the proposed initiatives, namely the construction/rehabilitation of water boreholes, VIP latrines, schools, primary health care units, market sheds, and roads, shows a positive NPV and a robust IRR, which confirm their economic viability (Table 2 – detailed economic and financial analysis in Annex 8). Ultimately, the project presents a general weighted average IRR of 49.88 percent, which reveals the overall viability of LGSDP. The efficiency of this would normally be considered high (efficiency exceeds expectations). However, the shortfall in geographic coverage and outputs due to conflict and insecurity means that the project’s efficiency in improving service delivery can only be considered high in the areas where these sub-projects were completed. Page 18 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Table 7: Cost-benefit analysis of selected sub-projects Discount Rate 6% VIP Latrine Classroom Primary Health Market Shed Water Borehole Road Cost of Capital (one 5-toilet (Primary Education) Care Unit (5,000 (one 50-stall (one unit) (14.5 Km) 10% block) (one unit) people capacity) market) Investment $6,266 $29,544 $14,446 $74,638 $31,799 $36,324 NPV $26,954 $60,202 $96,499 $621,943 $104,496 $150,989 IRR 41.06% 24.03% 58.58% 70.73% 33.51% 39.99% Figure 2: Budgeted/actual PDG disbursements Figure 3: Periods of low or suspended PDG disbursements * Note: from 2017 onwards, budgeted disbursement assumed that additional financing would come through, so are high, and do not match the original projections in the estimated component costs. 39. The efficiency of the project in improving local governance was modest. Improvements in local governance materialized in fewer counties than expected, and, for local government capacities, with varying success across type of function and capability and risks to sustainability. Although there are no cost comparators to enable analysis of costs versus norms in the sector, it is likely that efficiency for these components was modest, due to three periods during which disbursements were low, suspended, or when the project was making projections based on the expectation of additional financing. During these periods, expenditures for the institutional strengthening and project management components of the project continued, even though PDG disbursements were paused (Figure 3). The first downturn in disbursement took place between February 2015 and May 2016 (15 months). Between February 2015 and December 2015, disbursements were low because of conflict in Western Equatoria State and Lakes States, and between December 2015 and May 2016, PDG disbursements were put on hold to help the government deal with large exchange rate fluctuations and the plummeting value of the South Sudan Pound. (The details of these exchange rate fluctuations are found in the ‘Factors Affecting Implementation’ section of this ICR). The second downturn in PDG disbursements took place between July 2016 and February 2017 (eight months). During this time, the World Bank stopped PDG disbursements to minimize conflict-related fiduciary risk, allowing them to resume only in March 2017, after additional fiduciary measures were introduced. Finally, from July 2017 onwards, PDG disbursements were lower than the PMU expected because of the funding shortfall and the fact that the additional financing did not go through. During these periods of low or suspended disbursements, expenditures on project management and institutional strengthening necessarily continued, which meant that the proportion of the overall budget spent on these components was higher than those of other components: actual expenditures for institutional strengthening and project management were 70 Page 19 of 70 The World Bank Local Governance and Service Delivery Project (P127079) and 74 percent of the original estimates respectively, compared to only 29 percent for block grants and 47 percent for community engagement (Table 8, Annex 3). D. JUSTIFICATION OF OVERALL OUTCOME RATING 40. The overall outcome rating of the project is moderately unsatisfactory. This is based on a relevance rating of substantial, an efficacy rating of modest, and an efficiency rating of modest. This yields a consolidated outcome rating of moderately unsatisfactory. E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 41. The project took place in a country context with widespread GBV and patriarchal social norms. The project did not directly address GBV, the risk of which increased throughout conflict, but attempted to address women’s participation by requiring 50 percent of committee members to be women. Evidence on the quality of this participation was mixed. The beneficiary assessment found that community members thought committees had been formed fairly, allowing all groups, including women, to be represented. Anecdotal evidence from ICR interviews indicated that the women’s participation requirement helped women identify priority plans that reflected their and their families’ needs, but also indicated that women did not necessarily speak up at meetings despite attending. The community engagement assessment also identified challenges, such as scheduling conflicts, limited interest from women, and social conservatism—issues that LGSDP staff responded to by changing meeting times and making other adjustments. Institutional Strengthening 42. The effects of the project on institutional strengthening are captured in the efficacy section of this ICR. Mobilizing Private Sector Financing 43. The project was not designed to mobilize private sector financing. Poverty Reduction and Shared Prosperity 44. The project was intended to improve access to services (through provision of improved infrastructure), thereby reducing the deprivations faced by poor and vulnerable households. Impact evaluations of CDD programs in other countries have shown a positive effect on household income: for example, a National Human Development Initiative (INDH) impact evaluation of a CDD project in Morocco showed an average household income increase of 21 percent. To the extent that poverty is understood as a phenomenon of multi-dimensional deprivation, any increase in access to clean water, schooling, health care, or other services provided by LGSDP sub-projects constitutes a reduction in the deprivations faced by such households, and thereby a reduction in poverty. Other Unintended Outcomes and Impacts 45. Although the project did not include conflict reduction in its PDO, it nevertheless incorporated conflict management skills into its community engagement methodology. This involved training community members in analyzing and resolving local conflict and enabling BDCs and PDCs to put together a work plan for community awareness of conflict management and community reconciliation. Qualitative evidence from facilitating partners suggests that, despite the overall increase in conflict in South Sudan at the national level, this conflict training had positive results at the local level. The borrower’s report noted several cases where this training had helped community members resolve disputes with the help of the BDC, for example over dowries and water resources, and cases where BDCs were used as fora for wider advocacy over local issues, such as against forced marriage of young girls. Although such cases were reportedly Page 20 of 70 The World Bank Local Governance and Service Delivery Project (P127079) common, and highlighted during interviews for this ICR, they were not consistently documented by facilitating partners, so are likely under-reported; they were, however, documented in case studies prepared by the PMU. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 46. The task team had strong FCV, community engagement, and local governance experience and made efforts to align LGSDP with the country context and the New Deal principles for engagement in fragile states . The team contained experts on forced displacement and conflict, intergovernmental fiscal transfers, local governance, and CDD. During scoping and preparation, the task team laid out the program’s principles, emphasizing that it would be government- owned and led, would balance demands for a tangible ‘independence dividend’ with the need for longer-term institution-building, and would aim to empower local communities and build local social capital through CDD, thus strengthening community-state linkages and legitimacy.41 The team drew on lessons from the South Sudan MDTF, which, based on its experience in aiming to build government capacity during the post-CPA period, emphasized realism.42 47. The project design incorporated rigorous analysis and pilot testing. This included a review of existing CDD operations (July 2012)43 which incorporated lessons learned from a range of different CDD approaches in other countries in the region; an analysis of local socio-political organization and its implications for CDD 44; and analyses of incorporating forced displacement and human rights issues into operations. The design of the community engagement methodology was field tested through a pilot conducted in eight counties (two in each of four states) through a separately-funded Fast Track Initiative funded by the Swedish International Development Agency. 48. However, some aspects of project design made the project more vulnerable to upheaval . The first was the decision to have a hybrid local governance/CDD model with interdependent components. The project was designed to provide block grants via intergovernmental fiscal transfers, rather than directly to communities via parallel structures, in order to help establish local government via the LGA. It saw South Sudan as a country that needed to deliver early benefits to communities despite weak government capacity but also strengthen government institutions and legitimacy over time. Combining features of traditional local governance and a CDD operation was thought to be the best way to achieve these goals. This was appropriate to the project’s aims but made the project more vulnerable to upheaval, as it made the project components interdependent: any disruption in local government transfers would delay building infrastructure and undermine the success of the community engagement component. The project thus became vulnerable to delays in government’s budgeting and flux in local government arrangements. 49. The risk mitigation framework required more flexibility. The pre-appraisal mission of July 2012 explicitly identified the risk of systemic failures associated with (a) frequent or continuous security threats preventing the project implementation or supervision; (b) a deepening fiscal crisis leading to an erosion of public sector capacity, which would undermine project implementation through state and county governments, and (c) a large-scale social crisis stemming from inflationary pressures, deepening food insecurity, or widespread insecurity. The aide memoire noted that views differed on how likely these failures were, but that there was agreement on elaborating a contingency component to provide for robust contingency measures, which would also establish ways to channel additional donor funds to communities with an expanded investment menu. The PAD allowed enough flexibility for ongoing implementation in Page 21 of 70 The World Bank Local Governance and Service Delivery Project (P127079) the case of substantially deteriorating country conditions—but in practice, these only allowed the project to pause, suspend, or move operations if such deteriorating conditions applied to a small number of states and counties. B. KEY FACTORS DURING IMPLEMENTATION Table 8: Impact of conflict, economic collapse, and local government upheaval on LGSDP Development Undermines critical Impact assumption that… Civil war PDO efficacy and effectiveness  Leads donors to  Funds will flow through  Project financing gap causes project to roll out to fewer withdraw/reduce funding the system as planned counties than planned, undermining outcomes  Leads conflict areas to be  Project mainly in west and south of the country deferred/excluded from  There will be enough  Period of no disbursement and additional fiduciary project security for the project to requirements lead to delays (undermining efficiency),  Leads to evacuation of operate disrupts PDG disbursements, and creates procurement Bank & project staff and delays. No additional financing means projects go restrictions on field travel unfunded. This undermines citizen confidence,  Leads to period of no undermining the link between this and the project’s disbursement and higher-level objectives of building state legitimacy additional fiduciary M&E quality measures by World Bank  Limits travel to field sites & limits ability of PIU to recruit an international expert of sufficient caliber Bank performance  Task team limited to remote supervision. Ameliorated somewhat by third party monitoring Macroeconomic crisis PDO efficacy and effectiveness  Leads to austerity budget  Funds will flow through  Lack of CDG undermines outcome area 2 (improvements (no CDGs) the system as planned in local government capacity)  Fall in SSP value & high  Higher costs for raw materials (because many inflation leads the cost of  Labor & raw materials will imported)—affects outcome area 3 (improving service raw materials to rise be affordable and available delivery) Delays in government  Delays in budget ceiling leads to ‘rainy season delay’, budget ceiling each year affecting outcome area 3 (improving service delivery) from 2013 onwards Upheaval in local government PDO effectiveness and efficiency  Staff turnover, uncertainty,  Structure & functions of  Undermines outcome area two (improvements in local low morale local government will governance) remain stable 50. During LGSDP’s implementation, conflict erupted, the economy collapsed, and the government drastically changed the local government structure. Together, these developments threatened many of the critical assumptions underlying the project (see Annex 6 for more detail). Much of the northern part of the country was unable to participate in the project due to a lack of functional local government and access, despite being in dire need of improved service delivery. Instead of being a national project as originally intended, LGSDP instead was concentrated in the relatively more peaceful southern and western parts of the country and all counties supported by the project were Page 22 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Government controlled throughout project implementation. When seen in the context of the theory of change, this means that the move from the project outcomes to the higher-level objectives of building a national system of decentralized governance and service delivery was disrupted. The collapsing economy and hyperinflation also meant that the project could not operate on its original budget. Finally, upheaval in local governance led to high staff turnover and disrupted capacity building. 51. Conflict caused World Bank staff to be evacuated and to face security restrictions, and disrupted travel for facilitating partners, the TA firm, and project staff, disrupting M&E and supervision. The war caused World Bank staff to be evacuated for long periods (December 2013 to April 2014 and July 2016 to April 2017), and, at other times, face tight security restrictions on field travel,45 limiting the ability of the task team to supervise the project. Insecurity and conflict also caused delays and difficulties for PMU, facilitating partner, TA firm, and state project coordination and support office staff in reaching field sites, leading to stop/start implementation or higher security costs. 52. Conflict also meant that the project got less donor money, disbursements paused, and additional fiduciary measures were introduced. At the start of the project, the Netherlands and Norway withdrew their commitments, and Denmark reduced its funding, leaving the project with a funding gap of US$41.5 million (Annex 3). Yet conflict and local realities meant the project needed more money, not less: the higher cost of delivery, combined with inflation, led to a projected cost overrun of US$33.5 million and therefore a total financing gap of US$75 million. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 53. The project’s M&E design reflected the challenges of designing M&E frameworks that are realistic but still fit-for- purpose in low-capacity FCV environments. The core underlying elements of the M&E design were appropriate: the theory of change for the project was clear, and was reflected in the results framework, which had SMART indicators and had baseline, mid-term, and end targets for all but two results indicators. All outcomes of the PDO statement were included in the results framework.46 Several of the results framework indicators (particularly for those measuring strengthened community engagement) were also core sector indicators (Table 2). M&E Implementation 54. There were significant shortcomings in M&E implementation capacity and systems . Although the PMU initially had an M&E staff member of international caliber, the staff member soon left, and the project was unable to hire qualified staff following the outbreak of conflict. The project had only one local M&E officer and seven state monitoring and information officers with limited technical expertise working as data collectors. Information on component two was collected by aggregating reports from facilitating partners. 55. These constraints were reflected in results framework reporting. The project struggled to obtain reliable data for the results framework and ISRs. From December 2014 onwards, implementation support missions consistently highlighted the need to improve M&E, emphasizing the need for the project to establish a data collection system across the key results indicators. Although the project then attempted to improve data collection (efforts that were acknowledged in subsequent ISM aides-memoires), these efforts remained insufficient. 56. Meanwhile, from March 2015 onwards, the task team was unable to travel to field sites—something the team used alternative means to try to overcome. As noted in the Outcome section, the monitoring was ultimately done through Page 23 of 70 The World Bank Local Governance and Service Delivery Project (P127079) multiple means including geo-enabled monitoring, reports from implementing partners, third party monitoring by IOM starting in 2017. This enabled them to collect data through multiple means and remotely supervise projects. M&E Utilization 57. There were also significant shortcomings in M&E utilization. Take-up of M&E findings was limited. The assessment of M&E contained in the abovementioned report found that “Looking at most reports, the consultant observed that they are very output oriented…’ and that ‘none of the senior management staff reports in the interviews or day-to-day discussion. No management staff reported that she/he used them for decision-making purposes.” Justification of Overall Rating of Quality of M&E 58. Given the significant shortcomings of the M&E system, the project’s M&E is rated as modest. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 59. The project triggered several safeguard policies. These were: (i) Environmental Assessment (OP/BP 04.01). LGSDP was categorized as an environmental Category B-Partial Assessment, as the small-scale basic infrastructure built by the project was expected to generate limited and localized environmental impacts to the nearby biophysical environment. The project’s negative environmental impacts, such as noise or air pollution and construction debris were limited, mostly reversible, localized, temporary, and large mitigated through sensible construction management techniques and diligent environmental management practices. (ii) Indigenous Peoples (OP/BP 4.10). The majority or all people in the project area met the definition of indigenous peoples. Therefore, per the requirements of OP/BP 4.10, elements of an Indigenous People’s Plan (IPP) were included in the overall project design, rather than presented in a separate IPP. The project embedded the basic principles of OP 4.10 of a free, prior, and informed consultation leading to broad community support for the project through its emphasis on reaching vulnerable social groups, including ethnic/tribal/kinship minority groups, ensuring their representation in community organizations, considering their needs and priorities, and ensuring equitable benefits and an accessible grievance redress and feedback mechanism. (iii) International Waterways (BP/OP 7.50). Sub-projects under component 1 included the rehabilitation or drilling of boreholes, and it was initially expected that the project would support small-scale irrigation projects. The project did not implement sub-projects that would have triggered OP/BP 4.11 on Physical Cultural Resources or OP/BP 4.12 on Involuntary Settlement. 60. The project prepared an Environmental and Social Management Framework (ESMF) to manage safeguard concerns. This was in compliance with OP/BP 4.01. It invested in building the environmental and social safeguards capacity of state and county officials and provided safeguards training materials to facilitating partners for community training. The project screened sub-projects to ensure they met environmental and social standards as a precondition for funding. 61. An independent environmental and social safeguards audit found project performance on safeguards was satisfactory and in compliance with Bank policies. The audit, conducted for the mid-term review, established that good practices were in place: the PMU had institutionalized and operationalized LGSDP implementation to ensure compliance with the GRSS regulatory framework, World Bank policies, and international best practice. There were no cases of involuntary resettlement. However, there were some areas of non-conformance. These included workers not wearing protective equipment, night drilling, noise pollution, a fuel spill in one location, and a lack of first aid boxes in another. 62. The project complied with all legal and financial covenants. A July 2017 financial management review found that all financial audits had been submitted on time, issues raised in audit reports had been adequately addressed, and interim- unaudited financial reports (IFRs) were being prepared on a quarterly basis for review by the Bank. Procurement was Page 24 of 70 The World Bank Local Governance and Service Delivery Project (P127079) consistently carried out in accordance with Bank guidelines. After the conflict intensified in July 2016, the Bank put in place additional fiduciary and procurement measures. C. BANK PERFORMANCE Quality at Entry 63. The project’s quality at entry was moderately satisfactory. The project was highly relevant to the needs of the new country and the World Bank’s country strategy. Its approach was informed by the New Deal principles and aimed to help South Sudan transition out of conflict effectively. The project was prepared by a team with an appropriate skill set and within a reasonable 16 month timeframe, as measured by the gap between the identification mission in November 2011 and board approval in March 2013. This was long enough to enable the team to commission appropriate preparatory assessments to ensure technical rigor and context-specificity, but timely enough for the project to be responsive to the country’s needs. (It took another 11 months, however, for the project to become effective due to the time taken for securing parliamentary approval of the project). (Annex 6). 64. The project was informed by rigorous analysis of the social and development context and adhered to environmental, social, and fiduciary standards. It also had strong government ownership . Indeed, it was one of the government’s primary vehicles for building the new nation and bringing aid on budget to enable it to be country-owned and led. 65. There were, however, shortcomings in quality at entry. First, although the core principles underlying the M&E design (the results framework) were fit for purpose, the project did not invest adequately in building the M&E capacity of the borrower or adequately help the PMU to develop an appropriate M&E plan. Second, although the project identified the risks of conflict or economic crisis, it did not anticipate the flux in local governance arrangements or the scale with which such conflict might materialize, and it relied too much on the assumption of bilateral donor financing. The project’s risk mitigation measures were aimed at enabling flexibility but were still too rigid: they required restructuring, and so did not afford the project enough flexibility to course-correct in the iterative manner required in a volatile FCV context. Quality of Supervision 66. The performance of the World Bank in supporting the project to achieve its PDO outcomes during implementation was moderately satisfactory. The Bank team stayed focused on development impact, was responsive to the borrower, was candid in its reporting, and enjoyed an excellent working relationship with the client. However, civil war constrained the task team’s ability to supervise the project fully. The task team also should have restructured the project early (see below). The disbursement gaps caused pauses while additional fiduciary measures were put in place. These gaps undermined the project’s ability to achieve its PDO outcomes. The promise of additional financing also led the client to use faulty assumptions in its financial predictions; its abrupt cessation undermined transition arrangements. 67. The team conducted regular implementation support missions, which were focused on development impact and which paid adequate attention to identifying and resolving issues. Supervision missions took place regularly (see Annex 6 for the schedule). Aides-memoires throughout the supervision period correctly identified the major issues facing the project. Ideally, the Bank team would have been stronger in its own internal M&E capacity to provide more dedicated hands-on support to the borrower, who identified the desire for more M&E support in its borrower’s report. 68. LGSDP should have been restructured earlier. Ideally this would have been around January/February 2014, soon after the first round of conflict had taken place and it was clear that bilateral donors were withdrawing, leaving a significant financing gap. The Bank was indecisive on restructuring as it was not clear whether the conflict would be sustained or if a peace settlement could be quickly negotiated.47 At each stage, the Bank remained optimistic that these Page 25 of 70 The World Bank Local Governance and Service Delivery Project (P127079) circumstances might improve, and the project was not restructured. It was also not clear if the financing gap would persist, as the European Union at the time (through 2014) strongly signaled their interest in contributing (which eventually did not materialize), and later the World Bank itself began the process of additional financing, which would have negated the financial need to restructure. Moreover, the scope of the proposed additional financing restructuring should have been expanded to include an amendment to the local governance PDO given flux in local governance arrangements—but during that period, the Bank believed such issues could be overcome through clustering states and counties. In both cases, there were too many variables in future arrangements, and too much unpredictability in the country, for the Bank to make what, in retrospect, would have been the correct decision. Justification of Overall Rating of Bank Performance 69. The quality at entry of the project and the quality of supervision are moderately satisfactory. The overall World Bank performance rating is moderately satisfactory. D. RISK TO DEVELOPMENT OUTCOME 70. The risk to the project’s development outcome is high. The latest South Sudan peace deal could unravel. The deal, signed in September 2018, has, for now, reduced the fighting between the main power blocs; conflict remains in areas controlled by groups not party to the peace agreement. However, commentators suggest that the peace agreement has some inherent contradictions48 and does not reflect a true political settlement. Important issues around security, federalism, and devolution of powers are unresolved: the deal has not addressed issues of security, internal boundaries, subnational governance and how power and resources will be shared among South Sudan’s regions and ethnic groups. 49 71. A return to conflict would directly undermine project outcomes—as could continuing uncertainty over the country’s internal boundaries. There are three possible scenarios. The first is if conflict were widespread and serious enough to result in forcible displacement of people in LGSDP communities or for warring parties to destroy basic infrastructure financed through the project. The second is if conflict—or other developments such as instability over the border in Sudan—were severe enough to precipitate another macroeconomic or fiscal crisis, contributing to unpredictability in the intergovernmental fiscal transfer system. The third is continuing uncertainty in the country’s governance arrangements, especially over internal state boundaries. 72. There are also local risks to project outcomes stemming from possible reductions in community financing. BDC structures created by the project lie dormant unless money flows through them. The beneficiary assessment found that many BDCs had not seen funding for a while. This could ‘project-ize’ the BDCs, limiting their use as a wider platform for development planning. Second, there is a risk that communities who engaged in community planning without getting project funding will lose faith in participatory planning and trust in government unless they get future funding. V. LESSONS AND RECOMMENDATIONS 73. In FCV settings, the World Bank needs more flexible instruments. Although LGSDP should have restructured early, it is difficult for Bank Teams to know in real time what changes in an FCV country’s volatile context are likely to persist, affect PDO outcomes, and thus require restructuring. It is thus important for projects to be able to take on risk, see what works, and adapt to changes in the environment. The current IPF mechanisms, restructuring requirements, internal incentives, and culture still emphasize ex-ante project design and ex-post evaluation, so do not adequately enable this. Page 26 of 70 The World Bank Local Governance and Service Delivery Project (P127079) 74. Building in a robust risk management mechanism is crucial for operations in FCV settings. Although the team built extensive safeguarding and fiduciary measures into the project design, in reality (especially given difficulties with supervision and reliance on third party monitors), the project was not able to mitigate against all risks. Some risks—for example, the risk of widespread GBV or rent-seeking or corrupt behavior—are difficult to manage, particularly when project sites cannot be visited. Situations on the ground are often volatile, making it difficult to anticipate all risks ex- ante. It is important to acknowledge that in FCV contexts, residual or unanticipated risks are common. Teams should therefore re-evaluate the on-the-ground situation regularly and adapt the design and implementation approach accordingly. 75. It is possible to achieve results in FCV settings, especially through projects with a CDD component. Despite its challenges, LGSDP enabled the World Bank to stay engaged in South Sudan. This is particularly remarkable given the projects’ focus on local governance, compared to projects delivering emergency health support and social protection. Although civil war and economic collapse limited the overall outcome rating to modest, where it did roll out, the project delivered significant results, even with conflict elsewhere in the country. As in many FCV countries, there was huge spatial variation in conflict in South Sudan, enabling development in more peaceful areas even when others were inaccessible. Projects with a CDD component can be a useful entry point for staying engaged and building state-society relations even when other development interventions that rely on a more functioning centralized state are more challenging to implement. LGSDP’s experience also shows that local government capacity in individuals (who consistently reported benefiting from such training) can be built even capacity building in systems is hard to maintain. 76. Early post-conflict transitions present choices about how to appropriately sequence and manage trade-offs between supporting CDD and state-building. The 2011 WDR on conflict emphasizes that fragile contexts require delivering services to support early confidence-building, but also require the less visible, longer-term work of building institutions and strengthening government systems. Yet in practice, this presents trade-offs: to what extent should early interventions focus on meeting people’s needs but set up parallel community systems, or, in contrast, support local institution-strengthening but possibly make the project more susceptible to security and fiduciary risks or experience delays in service delivery? LGSDP aimed for a hybrid model as part of a longer 15-year engagement. In retrospect, however, some of its early project design choices (relying on intergovernmental fiscal transfers and a centralized technical assistance model of local governance strengthening) were overly ambitious. An alternative might have been to build local governance capacity via a “barefoot engineers” or “barefoot administrators” model of learning on the job and being less dependent on intergovernmental transfers. This might have made it more resilient to volatility. 77. M&E and supervision are challenging in FCV settings, and so need to be strengthened, including through providing task teams with detailed guidance on M&E design, emphasizing early third-party arrangements and iterative conflict and social analysis. Task teams in FCV settings need detailed, hands on guidance about how to identify appropriate results indicators, set targets, and monitor in such settings. Incorporating third party arrangements into projects early is also critical in FCV settings where there are perceived risks that project funds might be diverted or where conflict can affect project performance at the local level: this can help overcome information gaps and trust deficits and help task teams to better supervise and be more responsive to changes in local conditions. In the case of LGSDP, third party monitoring corroborated the PMU’s own reporting: incorporating this earlier would have helped the task team, PMU, and CMU considerably. . Page 27 of 70 The World Bank Local Governance and Service Delivery Project (P127079) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To improve local governance in participating counties in South Sudan Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Intended beneficiaries aware of Percentage 0.00 60.00 60.00 62.00 project info. and project investments (%) 15-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Participating payams with Percentage 6.00 80.00 80.00 67.70 functioning Payam Development Committees 15-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 [Participation] Comments (achievements against targets): Page 28 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Sub-projects with post-project Percentage 0.00 80.00 80.00 52.00 community engagement or O&M arrangements (%) 15-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Counties remaining eligible in Percentage 0.00 80.00 80.00 44.40 subsequent year for the Payam Development Grant [County 15-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 Performance] Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Direct project beneficiaries Number 0.00 4100000.00 4100000.00 774664.00 15-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 Female beneficiaries Percentage 0.00 50.00 48.10 47.00 Page 29 of 70 The World Bank Local Governance and Service Delivery Project (P127079) 15-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 Comments (achievements against targets): Objective/Outcome: To improve service delivery in participating counties in South Sudan Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Financed subprojects Percentage 0.00 80.00 80.00 100.00 functioning and delivering services to communities one 15-Feb-2012 26-Feb-2013 31-Dec-2018 24-Apr-2018 year after completion [Service Delivery] Comments (achievements against targets): A.2 Intermediate Results Indicators Component: Block grants to counties for payam development Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Comp 1 - CDG and PDG Yes/No N Y Y Y Indicative Planning Figures published for counties by 21-Feb-2014 26-Feb-2013 31-Dec-2018 28-Feb-2019 official due date Page 30 of 70 The World Bank Local Governance and Service Delivery Project (P127079) [Transparency] Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Comp 1 - Eligible and reporting Percentage 0.00 100.00 100.00 100.00 compliant counties being sent planned PDG disbursement 21-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 within 30 days of due dates [Predictability] Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Comp 1 - Eligible counties with Percentage 0.00 90.00 90.00 80.00 variance between budgeted and actual PDG expenditures 21-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 not exceeding 35 (percentage) [Execution] Comments (achievements against targets): Page 31 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Improved basic services Number 0.00 0.00 500.00 439.00 infrastructure in counties covered by the Program - 21-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 Disaggregated list of investments by type and sector Comments (achievements against targets): Component: Community engagement Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Participants in consultation Number 0.00 0.00 500000.00 135449.00 activities during project implementation (number) 21-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 Participants in consultation Number 0.00 0.00 250000.00 63778.00 activities during project implementation - female 21-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Page 32 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Grievances registered related Percentage 0.00 80.00 80.00 91.00 to delivery of project benefits addressed (%) 21-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 Comments (achievements against targets): Component: Institutional Strengthening Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Comp 3 - Participating counties Percentage 0.00 85.00 85.00 100.00 submitting approved (or passed) and complete plans 12-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 and budgets by October 1 Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Comp 3 - Participating counties Percentage 24.00 85.00 85.00 11.00 with “clean” (not adverse or disclaimed) audit opinion on 21-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 grant expenditures Comments (achievements against targets): Page 33 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Comp 3 - Subprojects adhering Percentage 0.00 70.00 70.00 86.00 to technical standards and completed within one year of 21-Feb-2012 26-Feb-2013 31-Dec-2018 28-Feb-2019 grant disbursement Comments (achievements against targets): Page 34 of 70 The World Bank Local Governance and Service Delivery Project (P127079) B. KEY OUTPUTS BY COMPONENT Objective 1: Improved Local Governance (Outcome area 1: Strengthened Community Engagement) 1. Intended beneficiaries that are aware of project information and project supported investments 2. Participating payams with functioning Payam Development committees 3. PDG projects for which arrangements for community engagement in Outcome Indicators post project sustainability and/or operations and maintenance are established (%) 4. 4. Direct project beneficiaries (number) aggregated by sex (female and male %) 1. Participants in consultation during activities during project implementation Intermediate Results Indicators 2. Grievances registered related to delivery of project benefits that are actually addressed (%) 1. 117,458 people (53,680 men and 63,778 women) participated the project’s community engagement activities as part of consultations.50 2. 1,280 community members (856 male and 424 female) were trained in conflict resolution and community reconciliation, conflict analysis, the roles of boma and payam development committees (BDCs and PDCs) in conflict resolution, and community awareness.51 3. The project also set up 855 development committees (700 BDCs and 155 Key Outputs by Component PDCs52) to enable community members to plan, implement, and oversee (linked to the achievement of the Objective/Outcome 1) local development investments. 4. 7,621 community members (5,017 male and 2,604 female) participated as members of the BDCs. 5. 1,826 people (1,080 male and 746 female) participated as members of the PDCs.53 6. 616 project supervision teams (PSTs) or project management teams (PMTs)54 were set up at community level to oversee sub-project implementation, 376 of which were activated.55 Page 35 of 70 The World Bank Local Governance and Service Delivery Project (P127079) 7. Over 4000 community members (2528 male and 1564 female) participated in these committees.56 8. 123 user group committees were set up to enable community members to engage in post-project sustainability or operations and maintenance of sub-projects.57 (Outcome area 2: Improved local government capacities) Outcome Indicators 1. Counties remaining eligible in subsequent year for the PDG (%) 1. CDG and PDG indicative planning figures published for Counties by official due dates (yes/no) 2. Eligible and reporting compliant counties being sent planned PDG disbursement within 30 days of due dates (%) 3. Eligible counties with variance between budgeted and actual PDG Intermediate Results Indicators expenditure not exceeding 35% 4. Participating counties submitting approved and passed, complete plans and budgets by Oct 1st, every year (%) 5. Participating counties with ‘’clean audit opinion report on grant expenditure 1. 14 training handbooks and user guides for 18 thematic modules covering planning, budgeting and financial management (PBFM); procurement; and infrastructure were developed.58 2. 119 classroom training workshops were delivered to an average of 41 Key Outputs by Component officials each.59 (linked to the achievement of the Objective/Outcome 2) 3. 474 local government officials were trained in at least one subject. 4. 16 counties received county ‘starter kits’ consisting of furniture, ICT equipment, and other office items. 5. Financial audits were conducted in 15 counties 6. Nine counties had their performance assessed. Page 36 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Objective 2: Improved service delivery (Outcome area 3: Improved service delivery) 1. Percentage of financed subprojects functioning and delivering services to Outcome Indicators communities one year after completion 1. Subprojects adhering to technical standards and completed within one Intermediate Results Indicators year of grant disbursement (%) Key Outputs by Component 1.Disaggregated list of investments by type and sector (number) (linked to the achievement of the Objective/Outcome 3) Page 37 of 70 The World Bank Local Governance and Service Delivery Project (P127079) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Zishan Faiza Karim, Makiko Watanabe Task Team Leader(s) Ocheng Kenneth Kaunda Odek, Pascal Tegwa Procurement Specialist(s) Stephen Diero Amayo Financial Management Specialist Tracy Hart Environmental Safeguards Specialist Martin Onyach-Olaa Team Member Varalakshmi Vemuru Social Safeguards Specialist Eric Brintet Team Member Grace Tabu Felix Team Member Florence Poni Team Member Droma Bank Dominic Kat Team Member B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 19.125 164,675.48 FY13 70.201 330,484.06 FY14 0 -290,617.64 FY17 0 -7,722.35 Total 89.33 196,819.55 Supervision/ICR FY12 1.400 133,056.91 Page 38 of 70 The World Bank Local Governance and Service Delivery Project (P127079) FY13 12.400 187,975.72 FY14 117.963 514,586.90 FY15 101.863 550,423.93 FY16 80.735 421,448.27 FY17 78.204 178,814.99 FY18 69.700 144,829.52 FY19 42.453 209,950.93 FY20 0 162.91 Total 504.72 2,341,250.08 Page 39 of 70 The World Bank Local Governance and Service Delivery Project (P127079) ANNEX 3. PROJECT COST BY COMPONENT Table 8: Project Cost by Component Amount at Approval Actual at Project Percentage of Approval Components (US$M) Closing (US$M) (US$M) Component 1: Block Grants 30.00 8.79 29% to Counties and Communities Component 2: Community 25.40 11.90 47% Engagement Component 3: Institutional 22.00 15.46 70% Strengthening Component 4: Project 21.00 15.45 74% Management Support Total 98.50 51.60 52% Page 40 of 70 The World Bank Local Governance and Service Delivery Project (P127079) ANNEX 4. EFFICIENCY ANALYSIS 78. The economic and financial analysis conducted as part of the South Sudan ICR’s efficiency section suggests that each of the project’s proposed initiatives is economically viable at a discount rate of 6 percent over 20 years. The analysis employed a 10-year-average inflation rate of 2.18 percent and an estimated cost of capital for South Sudan of 10 percent. 79. The results of the analysis are conservative. Each of them presents far larger economic returns in terms of indirect social benefits (described in the “Sub-project costs and benefits” section), but these are not readily quantifiable. Nevertheless, the project presents an overall weighted average IRR of 49.88 percent, which reveals the economic viability of the LGSDP initiative. 80. The analysis employs a simplified cost-benefit methodology, where solely direct cost and monetary value of selected benefits are compared, and net present value (NPV) and economic internal rate of returns (IRR) are presented. In fact, the completed sub-projects have several indirect benefits that are not quantifiable in monetary terms, also due to data constraints. For this reason, they cannot be accounted for, both in net present value calculations and in the sensitivity analysis. Nonetheless, they can be listed among the project benefits. 81. The analysis could benefit from actual implementation costs, while benefits could only be estimated based on existing empirical evidence. 82. The main hypothesis, as set up in the PAD in 2013, is that appropriate design and implementation of various investments would significantly increase the social and economic conditions of communities benefiting from LGSDP. The sub-projects’ incremental benefits relate to improving the socio-economic conditions of the populations in the areas of interventions through access to clean water and health conditions (water boreholes, VIP latrines, and health centers), better and comprehensive education (schools), and both upgraded infrastructures and channels for trade to maximize productivity, efficiency, and economic returns in agriculture (roads and markets). 83. Furthermore, improving local government capacities is another important objective, achievable as a project’s positive externality. Specifically, local administrations are expected to improve in participatory local development planning and budgeting, public financial management, procurement, technical/engineering aspects of local infrastructure planning and implementation, monitoring and reporting, environmental and social safeguards management, social accountability, and communication and information dissemination. 84. An additional externality may be that investments in the listed areas would significantly contribute to the creation of an effective labor market, which in turn would lead to building self-reliant communities. 85. The assessed interventions include a total of six sub-projects across different sectors. Both NPV and IRR for each of the sub-projects and the project’s weighted average IRR show the investments and the overall initiative are economically viable (Table 9). Page 41 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Table 9: Cost-benefit analysis of selected sub-projects Primary Classroom Health Care Water VIP Latrine Market Shed Discount Rate 6% (Primary Unit (5,000 Road Borehole (one (one 5-toilet (one 50-stall Cost of Capital 10% Education) people (14.5 Km) unit) block) market) (one unit) capacity) Investment (US$) 6,266 29,544 14,446 74,638 31,799 36,324 NPV (US$) 26,954 60,202 96,499 621,943 104,496 150,989 IRR 41.06% 24.03% 58.58% 70.73% 33.51% 39.99% Source: Team calculations based on Government and empirical data SUB PROJECT COSTS AND BENEFITS Water Borehole – sub-project investment average: US$6,266 86. As per Government of South Sudan (GoSS) data, costs for the construction or rehabilitation of a water borehole amounted on average to US$6,266.60 The life span of such an intervention is estimated at 20 years, provided regular annual maintenance to keep it operational. Empirical evidence shows that operating and maintenance costs amount, on average, to US$0.60 per user.61 A water borehole is expected to serve 417 people, which according to the International Organization for Migration (IOM) is the average village population in the areas of the analysis. 62 As such, annual operation and maintenance (O&M) costs for one water borehole has been calculated at US$250, on average. 87. Facilitated access to clean water generates several economic and social benefits (Table A.2). Among them, the most important ones appear to be improved health63 and time saving. The analysis reveals an estimated economic return of US$1,381 per water borehole due to health-related improvements and US$1,257 from improved time management. The total estimated benefit per borehole is therefore calculated at US$2,638, on average. 88. Specifically, according to the World Health Organization (WHO), access to clean water reduces diarrhea morbidity by 25 percent. Considering five diarrhea-related deaths per village per year and a statistical value of life per year-life saved of US$2,394 as per Viscusi,64 with a conservative effectiveness rate of success assumed at 50 percent, the economic benefit of improved health conditions amounts to US$2,071, on average. 89. Furthermore, time saved due to proximity of water source is estimated to amount to 4.5 hours per week per household,65 which equals to 234 hours of time saved per year per household. Accounting for an average 60 household per village and a per capita GDP per hour of US$0.09, the estimated economic benefits due to time saving amount to an average US$1,257 per water borehole/village per year. Page 42 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Table 10: Direct and indirect benefits of water borehole sub-project Water Borehole Discount rate 6% Indirect Benefits - Children’s Development: Time saved can be used for children's welfare and up- Investment US$6,266 bringing, gaining new skills, training or participation in associations. - Reduction of conflicts and tensions over scarce availability of water NPV US$26,954 - Reduced costs of treatment for poor quality water-related illnesses - Reduction of greenhouse emissions with compensatory reforestation around IRR 41.06% water points. Source: Team calculation based on GoSS and empirical data 5-toilet VIP Latrine Block – sub-project investment average: US$29,544 90. The construction of one ventilated improved pit (VIP) latrine entailed an average investment of US$29,544. O&M are calculated at US$2.72 per user per block.66 Considering an estimated 104 users per five-toilet VIP latrine block, annual O&M are calculated at US$284, on average. 91. The use of VIP latrines is critical to dramatically reduce mortality due to sanitation- and hygiene-related issues. According to an International Water Association study,67 improved pit latrines reduce diseases attributed to unsafe sanitation and lack of hygiene by 31 percent. Such a benefit translates into nine lives saved per five-toilet VIP latrine block per year. Considering a statistical value of life per year-life saved of US$2,394 for South Sudan (Viscusi), in monetary terms that amounts to US$6,735, assuming the intervention is 33 percent effective. Table 11: Direct and indirect benefits of VIP latrine sub-project 5-Toilet VIP Latrine Block Discount rate 6% Indirect Benefits - Increased overall sense of well-being: safety and security, comfort, convenience, and Investment US$29,544 privacy - Contribute to create a hygiene-oriented mindset with the ultimate consequence of increased conditions in communities NPV US$60,202 - Reduction of infections, particularly respiratory and helminth, which remain the leading cause of child deaths globally (Black et al. 2010) - Reduced childhood undernutrition because of repeated bouts of diarrhea and nematode infections (WHO, 2008) IRR 24.03% - Reduced Gender-Based Violence and attacks associated with the absence of toilets due to increased privacy and proximity of sanitation points to frequented areas. Source: Team calculation based on GoSS and empirical data Page 43 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Primary Education Classroom – sub-project investment average: US$14,446 92. Annual operating and maintenance costs of primary education may amount, on average, to US$350 per student. 68 In the CBA, a classroom is assumed to be attended by 30 students, considering a 10-classroom primary school vis-à-vis a population of 0-14 age of 175 per village. In such a scenario, classroom O&M amount, on average, to US$10,500. 93. On the other hand, primary education entails several benefits, many of which can be quantified and expressed in monetary terms. Among them, the most important appear to be reduction in child mortality and maternal deaths, and an increase in income per year of schooling.69 94. Specifically, the Global Partnership for Education determines a 50 percent reduction in child mortality in case of mothers able to read. Also, a 67 percent reduction in maternal deaths if a mother has completed primary education. Finally, a 10 percent increase in income per year of schooling completed.70 In monetary terms, such benefits amount to US$18,476 per class per year, on average. Table 12: Direct and indirect benefits of primary school sub-project Primary Education Classroom Discount rate 6% Indirect Benefits - Higher attendance of students and teachers lead to improved learning Investment US$14,446 outcomes and building solid social capital - Poverty reduction as demonstrated by higher income per year of schooling. - Parental education positively affects the child’s educational level NPV US$96,499 - Boosts technological change by diffusion of knowledge - Reduction of financial burden through efficient access to education for girls, rural communities, ethnic minorities, and migrants IRR 58.58% - Reduction in criminal activity due to both increased time dedicated to school participation and increased level of education. Source: Team calculation based on GoSS and empirical data Sub-project: Primary Health Care Unit – sub-project investment average: US$74,638 95. The PHCU sub-project has been analyzed over an estimated life span of 20 years and assumed to serve communities at the boma level (approximately 5,000 people). Its annual O&M costs have been calculated at $21,000, on average.71 Total benefits have been estimated as a consequence of reduced maternal mortality and morbidity-related deaths due to both preventative and curative actions generated by the proposed initiative. In particular, considering an estimated annual 176 newborns per boma and, as per Lincetto,72 a reduction of maternal mortality at birth by 33 percent as a consequence of health care interventions both in terms of antenatal care (ANC) and curative interventions, 58 maternal lives are saved. At a statistical value of life per year–life saved for South Sudan estimated at $2,394 (Viscusi) and considering a very conservative effectiveness rate of 10 percent, the economic benefits due to reduction in maternal mortality amount, on average, to US$13,891 per Boma per year. Furthermore, preventative and curative care is conservatively expected to reduce morbidity-related deaths by 25 percent. As such, the estimated number of survival due to health care is calculated at 239 individuals per Boma per year, on average. Such a benefit translates into an economic benefit of US$57,185 per Boma per year, at an effectiveness rate of 10 percent. 96. Total benefits amount to US$71,076 per year, on average. Page 44 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Table 13: Direct and indirect benefits of primary health care unit sub-project Primary Health Care Unit Discount rate 6% Indirect Benefits Investment US$74,638 - Diminishing overall budget burden as the result of increased immunization and reduction of NPV US$621,943 severe illness. IRR 70.73% - Improved fight against HIV/AIDS as well as treatments of those infected Source: Team calculation based on GoSS and empirical data Sub-project: Market Shed – sub-project investment average: US$31,799 97. According to a report on market opportunities in South Sudan, 73 annual costs to operate and maintain a market stall amount, on average, to US$9.02 (inflation adjusted). Assuming an average 50 stalls per market shed, O&M can be assumed to be approximately US$451 per year. 98. In terms of benefits, a market shed is expected to generate US$15,210,74 on average, in direct taxes (estimated at 15 percent).75 Table 14: Direct and indirect benefits of market shed sub-project Market Shed Discount rate 6% Indirect Benefits - Well defined markets (and location) can help the government regulate and coordinate trade Investment US$31,799 with the consequence of increasing revenues while reducing costs associated with monitoring and tax collection. NPV US$104,496 - Organized market sheds reduce security risks faced by isolated market stalls and, if combined with infrastructures such as improved roads, can increase trade activity, this way IRR 33.51% making both traders and consumers better off. Source: Team calculation based on GoSS and empirical data Sub-project: Road – sub-project investment average: US$36,324 99. According to a World Bank study,76 recurrent costs to operate and maintain a paved road have been calculated at US$850 per Km per year. The Naliel-Lomuta road sub-project comprises a 14.5-Km road. Accordingly, such an infrastructure requires an average annual cost of US$12,325, if the road is to be maintained fully active and operational over an estimated life span of 20 years. 100. On the other hand, the benefits appear to be much higher, by only considering returns from increased economic activity and decreased accidents as a consequence of using a paved road. Another World Bank report 77 indicates that paved roads increase trade and economic activity by 10 percent. Using market stalls as a proxy, such a benefit is estimated in an average US$2,880 increase in economic activity per Boma. 101. Also, paved roads reduce mortality and disability due to car- and traffic-related accidents.78 Specifically, the estimated deaths and healthy life years lost in Eastern Sub Saharan Africa is approximately 13.2 per year per km. Accounting for that, the proposed sub-project is expected to save 191.4 lives per year. With a statistical value per year-life saved of US$2,394 and considering a conservative effectiveness rate of 5 percent, benefits in monetary Page 45 of 70 The World Bank Local Governance and Service Delivery Project (P127079) terms amount to US$22,911 per year on average. Total benefits of the road sub-project are calculated at US$25,791 per year, on average. Table 15: Direct and indirect benefits of paved road sub-project Paved Road Discount rate 6% Indirect Benefits - Transportation of commodities to/from the areas of the project, and access to markets (this is Investment $36,324 a large positive impact) - Better access to community facilities and services such as health centers, schools, and places of worship, amongst others - Creation of job opportunities to the local unskilled and skilled person in the community NPV $150,989 - Provision of markets for locally available resources needed for road construction, rehabilitation, and recurrent maintenance materials (i.e. sand, gravel, etc.) - Boost tourism potential in the region through improved access to tourist destinations - Improved road drainage infrastructure and general discharge of storm water from the road/carriageway IRR 39.99% - Less dust in the air from rugged, unpaved roads - Increased presence of government due to improved regional and inter-community connections - Cost-effective ability to manage the territory with fewer resources Source: Team calculation based on GoSS and empirical data SUMMARY 102. The analysis confirms that the completed sub-projects can substantially contribute to shared economic growth and improved access to services like health, education, sanitation, and hygiene, and ultimately to better social and economic outcomes. 103. Furthermore, as per the project development objective (PDO) of improving local governance and service delivery by strengthening community engagement and local government capacities in the planning, implementation and oversight of local development activities, local administrations and communities were evidently able to provide strong management, coordination, and oversight of the various initiatives at different stages of implementation, from design to completion. 104. Remarkably, the project presents an overall weighted average economic IRR of 49.88 percent, which confirms its economic viability. 105. Finally, as mentioned, each of the sub-projects presents several intangible benefits that add up to the reported figures and therefore make the returns much higher both in economic and social terms. Page 46 of 70 The World Bank Local Governance and Service Delivery Project (P127079) SENSITIVITY ANALYSIS 106. A sensitivity analysis has been conducted to assess the effect of possible variations in benefits and costs on both NPV and IRR for each of the sub-projects being evaluated. All results confirm the economic viability of each of the sub-projects under any circumstances (Table 16). 107. In fact, notwithstanding an increase in costs79 by 20 percent or a decrease in the total estimated benefits by 20 percent, the results remain robust. Moreover, a third scenario that contemplates a 20 percent increase in costs combined with a 20 percent decrease in benefits still shows robust NPV at 6 percent discount rate and IRR, vis-à-vis a cost of capital of 10 percent. Table 16: Sensitivity analysis: effect of variations in costs and benefits on project's NPV and IRR Discount Rate 6% Classroom Primary Water VIP Market Paved (Primary Health Cost of Capital 10% Borehole Latrine Shed Road Education) Care Unit Investment $6,266 $29,544 $14,446 $74,638 $31,799 $36,324 COST +20% SCENARIO NPV $26,258 $59,413 $67,287 $563,519 $89,440 $116,700 IRR 40.24% 23.82% 43.69% 64.98% 29.96% 32.97% BENEFIT -20% SCENARIO NPV $19,614 $41,464 $45,098 $424,203 $62,180 $79,237 IRR 32.35% 19.00% 32.28% 51.25% 23.39% 25.14% COST +20% AND BENEFIT -20% SCENARIO NPV $18,919 $38,252 $15,886 $365,779 $47,124 $44,948 IRR 31.51% 18.78% 16.48% 45.49% 19.63% 17.63% Source: Team calculations based on Government and empirical data Page 47 of 70 The World Bank Local Governance and Service Delivery Project (P127079) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS Page 48 of 70 The World Bank Local Governance and Service Delivery Project (P127079) ANNEX 6. FURTHER INFORMATION ON CONTEXT & FACTORS AFFECTING IMPLEMENTATION CONTEXT 108. In July 2011, after six years as an autonomous region within Sudan, the Republic of South Sudan became the world’s newest independent state. But decades of civil war between the pro-independence Sudan People’s Liberation Army 80 (SPLA) and the Government of Sudan had left the new country with some of the worst socioeconomic conditions in the world. At independence, more than half the population lived in extreme poverty, one in 50 women died from pregnancy or childbirth81, and one in ten infants82 died before the age of one. The population had devastatingly low access to infrastructure and services. Less than two percent of the country’s roads were paved; only four in ten households lived within five kilometers of a health care facility; three quarters of the population had no access to a sanitation facility of any kind; and the country had the lowest access to piped water and stand posts in sub-Saharan Africa. 109. The country thus badly needed to improve basic infrastructure and services. But local governments, which were primarily responsible for delivering upon this, had little capacity to manage this. Although South Sudan’s civil service had tens of thousands of staff,83 decades of neglect by Khartoum had decimated the historic remnants of local government, leaving local governments weak, fragile, and ill-equipped to perform their core administrative functions. International assistance did little to change this. During and immediately after the wars leading up to the 2005 Comprehensive Peace Agreement, most assistance to South Sudan took the form of humanitarian aid to conflict- affected populations, which mostly bypassed and was delivered in parallel to government. 84 This, combined with a legacy of poor governance and a political economy wherein many local government staff positions were given as rewards to former combatants, meant that local government staff had little experience at planning, managing, and overseeing the local development cycle: in other words, at much of their core business in governing. 110. The government therefore also needed to improve local governance. It had begun to prepare for this before independence, when the country was an autonomous region within Sudan but preparing to separate from it. During this period, the Government of Southern Sudan began to lay out its policy architecture for local governance. In 2009, it passed the Local Government Act,85 which specified how the local government system would be structured and what each level of government would do. The act contained provisions to establish inclusive local governance structures in payams (sub-counties) and bomas (village clusters) and to devolve core government functions to the country’s ten states, 79 counties, and these payam and boma sub-structures. Under the law, the primary responsibility for delivering basic services was held by county governments. To support them in doing this, the government planned to provide them with block grants (“county block grants”) to pay for county infrastructure. It also planned to seek donor funds to build and rehabilitate community infrastructure through a system of payam block grants. 111. After decades of war, the government also needed to build state legitimacy and reduce the risks of conflict. At independence, South Sudan was deeply fractured at different levels of society. Conflict had been a matter not only of resistance to Khartoum or of intra-elite competition over power between different factions of the Sudan People’s Liberation Movement, but also of local competition over resources. Such tensions were manifested in border disputes, violent cattle rustling, and conflict among and between pastoralists and agriculturalists, but were intensified by war and the spread of modern arms into communities. War also left behind a deeply traumatized population, whose grievances began increasingly to be expressed in ethnic terms, and who had little trust in or experience of a functioning Page 49 of 70 The World Bank Local Governance and Service Delivery Project (P127079) state. The drivers of conflict thus ran deep. The new government needed to ensure that its development efforts did not exacerbate these, and ideally could mitigate them, while shoring up its legitimacy. 112. In keeping with experience from other fragile states, the government sought a community-driven approach to meeting these aims. In 2011, the government expressed interest in seeking financing from the World Bank Group for a project that could help improve local governance and service delivery, thereby delivering an ‘independence dividend’, building state effectiveness, and reducing local conflict. This proposed expansion of services was consistent with the ‘taking towns to the people’ vision of the Sudan People’s Liberation Movement, which emphasized the primacy of bringing health, education, infrastructure, and basic services to people in its vision of nation-building. The Local Governance and Service Delivery Project (LGSDP) sought to improve local governance and service delivery by: training and building the capacity of local government staff; enabling them to ‘learn by doing’ in managing block grants for infrastructure; establishing inclusive governance structures in bomas and payams to enable citizens to engage in development; and building and rehabilitating basic infrastructure. In some ways, the project was a classic post-conflict, community-driven nation-building project, but with two critical differences. Rather than focusing on communities alone, it had a strong focus on building local government capacities. It also provided block grants not directly to communities, but through local government via the intergovernmental fiscal transfer system. 113. The project was part of wider government attempts to strengthen South Sudan’s aid infrastructure in line with global agreements on supporting transitions out of conflict. At the High-Level Forum for Aid Effectiveness in Busan in 2011, the g7+ group of 19 fragile states, development partners, and international organizations endorsed the New Deal for Engagement in Fragile States, a landmark global agreement for improving donor assistance to fragile and conflict- affected countries. The New Deal recognized that transitioning out of fragility was long, political work requiring country leadership and ownership, and that much donor assistance in fragile settings had been only modestly effective. It thus aimed to change this. As part of this, it proposed moving away from a donor-led model of such assistance and towards a country-led model where development assistance was predictable, accountable, and funded largely through country systems. South Sudan emphasized these principles in its 2011 Aid Strategy.86 The strategy set out two new aid instruments for donors to support its aims: budget support, to fund national and state-level services, and a Local Support Services Aid Instrument (LLSAI), a mechanism to fund health, education, and local governance and service delivery through intergovernmental fiscal transfers. LLSAI created an entry point for the World Bank’s dialogue with the government around LGSDP. 114. The project was envisaged as a flagship in the World Bank’s South Sudan portfolio. In April 2012, South Sudan became the newest member of the World Bank Group, and in February 2013, two months after the LGSDP appraisal discussions, the World Bank released its Interim Strategy Note (ISN) for South Sudan for FY2013-14. The strategy was underpinned by the New Deal principles and the World Bank’s 2011 Africa strategy. It was also informed by the 2011 WDR on Conflict, Security, and Development, which emphasized the bumpy, cyclical nature of post-conflict transitions and the importance of building institutions and taking flexible approaches to risk. In keeping with this, the ISN argued that South Sudan’s central challenge was to lay the foundations for effective and accountable institutions, which would use the country’s own resources to respond to citizen needs. The ISN thus focused on the longer-term endeavor of building legitimate institutions to mitigate the drivers of instability, including by developing the capacity of local government to deliver services. LGSDP—which aimed to do exactly this—was thus a flagship in the portfolio. It was explicitly designed as the initial phase of a longer term, 10-15-year engagement to establish responsive, effective, sustainably resourced, accountable local institutions. Page 50 of 70 The World Bank Local Governance and Service Delivery Project (P127079) 115. The risks, however, were high. At appraisal, the country faced severe macroeconomic, fiscal, and political stresses. South Sudan was highly dependent on oil revenues,87 but could not agree with Sudan on how much it needed to pay to transport its oil through Sudan, so had to stop producing and exporting oil in January 2012. This, and a subsequent drop in global oil prices once the country had resumed producing oil, caused the economy to contract and drained public finances, leading the government to put in place an austerity budget.88 Meanwhile, the country faced political challenges. At appraisal, the political bargains underlying peace were fragile, and key players within the Sudan People’s Liberation Movement (SPLM)—notably President Salva Kiir and the Vice-President, Riek Machar, who were from different ethnic groups—maintained strong rivalries, which risked spilling over into active conflict and exacerbating ethnic tension. There were also wider risks of elite capture and corruption. The project appraisal document recognized this environment of high risk, highlighting the possibility of governance weakness, elite capture of resources, fiscal risks to budgets, and insecurity. But, at appraisal, these had not yet threatened the critical assumptions underlying the project, and so did not affect the project design or theory of change. Factors affecting implementation 116. Even before LGSDP became effective, these risks began to materialize. In December 2013, SPLA government forces loyal to the president, Salva Kiir, clashed with rebel SPLA-In Opposition (SPLA-IO) armed forces loyal to the former vice president, Riek Machar. This sparked the third civil war in South Sudan’s post-colonial history, one that soon spread to Unity, Jonglei, and parts of Upper Nile state. On the day the project became effective, none of the World Bank LGSDP task team were the country: instead, they were in Kenya, where they had been since being evacuated in December 2013 because of the outbreak of civil war. The first implementation support mission, in March 2014, was held not in Juba but in Nairobi. 117. After LGSDP became effective, South Sudan’s civil war intensified. Although the two parties signed a peace deal in August 2015, renewed clashes took place in Juba in July 2016. These intensified the war, which metastasized and spread to much of the country, including states in Equatoria, the southern third of the country, which until then had been peaceful. As it grew, the civil war became increasingly complex, morphing from a conflict primarily between rival groups of national elites to one where inter-communal violence, organized along ethnic lines and perpetrated by civilians, was common. 118. During the same period, South Sudan’s economy collapsed. The country’s macroeconomic crisis had begun during project preparation, when South Sudan shut down oil production and exports following its dispute with Sudan about border issues and oil transit fees. For an economy so dependent on oil revenues, this was devastating: in 2012, GDP dropped by an estimated 52 percent.89 With shrinking revenues, the government was forced to adopt an austerity budget, which led the economy to contract, people’s purchasing power to shrink, foreign reserves to be hit, and the currency to weaken. During implementation, these problems worsened. In 2014, the international price of oil dropped by over half, causing the country’s fiscal deficit to widen (by 2016, it was at 40 percent of GDP). The government attempted to ease this by floating the exchange rate in December 2015. The value of the South Sudan pound (SSP) plummeted. In December 2015, it opened at the parallel market rate of SSP 18.5 per dollar, but by August 2016 had fallen to SSP 70 per dollar and by mid-May 2017 to SSP 150 per dollar. This improved the government’s fiscal position, but such gains were eroded by further decreases in oil prices, government overspending on budgets, runaway security spending, and civil servant pay increases. Meanwhile, inflation spiraled: the annual Consumer Price Index (CPI) Page 51 of 70 The World Bank Local Governance and Service Delivery Project (P127079) increased from 1.7 percent in 2014 to about 30 percent in 2015 and 480 percent in 2016, eventually coming down to 155 percent between July 2016-July 2017. Prices rose across the country, though in some areas more than others. 119. Meanwhile, the country’s local government system underwent massive upheaval. In October 2015, two months after the August 2015 peace deal, President Kiir announced a decision to expand the number of states in South Sudan from 10 to 28, a decision thought to be driven by the need to respond to demands for greater decentralization, reinforce his patronage network, and tilt the balance of power in his favor. In April 2016, the number of states increased again to 32, and the number of counties from 79 to 318. With new counties came the need for county administrations and staff. This introduced enormous change and uncertainty into the country’s system of local government, as there was significant staff turnover and uncertainty among staff about whether they would remain working in their counties or be moved to new ones. The new states were organized along ethnic lines. Their creation is thought to have exacerbated South Sudan’s conflicts further.90 120. Together, these developments threatened many of the critical assumptions underlying the project. They made it harder for the project to achieve its objectives effectively and efficiently, affected the quality of M&E and Bank performance, and increased the risks to the sustainability of the project’s development outcomes. 121. The main pathways through which this took place were as follows. (a) Civil War 122. South Sudan’s civil war undermined the assumption that the project would have enough security to operate . Much of the northern part of the country was unable to participate in the project—despite being in dire need of improved service delivery. After the first wave of conflict, the project’s intended roll-out into Jonglei and Upper Nile State was deferred, and, though the PMU continued to assess these states in subsequent years to see if they were safe enough to operate in, the continued spread of conflict meant these were never included. Insecurity led the project to also be deferred to the end of the project in six counties, Magwi, Yei, Maridi, Ezo, Nzara, and Nagero, and to be suspended and then restarted in Ibba. The project made significant attempts to adapt to this context of conflict and risk. In April 2016, eight months after the August 2015 peace deal, the project put together a “high-risk areas” strategy for the project. The Options and Operating Modalities for LOGOSEED in Insecure Areas paper outlined the circumstances under which the project could operate in conflict-affected areas and what measures it would have to take to do so. Yet ultimately, Jonglei and Upper Nile remained too insecure for the project to operate in. Instead of being a national project as originally intended, the project instead was concentrated in the more peaceful southern and western parts of the country. When seen in the context of the theory of change, this means that the move from the project outcomes to the higher-level objectives of building a national system of decentralized governance and service delivery was disrupted. 123. Conflict also led to disruptions in travel for other critical actors, disruptions in the movement of goods, and displacement. Ongoing insecurity and communal conflicts throughout the project were common. Communal conflicts first affected the project in Lakes State but also in Western and Eastern Equatoria and parts of Warrap, and there was widespread insecurity due to armed gangs in Eastern and Western Equatoria. This ongoing insecurity and conflict caused delays and difficulties for PMU, facilitating partner, TA firm, and state project coordination and support office staff, as well as contractors, in reaching field sites, leading to stop/start implementation of the project. In some areas, it led contractors to abandon project sites, or led to displacement, as beneficiaries of sub-projects moved to new Page 52 of 70 The World Bank Local Governance and Service Delivery Project (P127079) locations. It also led to a need to provide security, as in some cases convoys and armed escorts had to be arranged for field missions. This, along with stop/start implementation, led to higher costs. 124. Conflict also disrupted supervision and M&E. The war caused World Bank staff to be evacuated and face tight security restrictions. After the December 2013 clashes, World Bank staff were evacuated to Nairobi, where they remained through to the end of April 2014. Thereafter, staff faced tight security restrictions. After the renewed clashes in July 2016, World Bank staff were again evacuated to Nairobi, and remained there for 10 months through to April 2017. (PMU staff were also evacuated from the country, but for only two weeks; facilitating partner staff were also evacuated.) During that time, the World Bank office in Juba was closed, so the project was supervised remotely from Nairobi (the mid-term review was a reverse mission). After the World Bank office in Juba re-opened, World Bank staff faced tight security restrictions and could not in practice travel outside Juba, and missions to South Sudan were restricted in number given the need for staff to stay on the World Bank Juba compound, which could only accommodate three visitors at a time. This limited the ability of the task team to monitor and supervise the project as effectively as they could without such restrictions, causing them to rely more heavily on PMU staff for information. The CMU attempted to help task teams overcome these supervision and monitoring challenges by investing heavily in third party monitoring, but such third-party monitoring usually has limits, as staff remain a degree of separation removed from field realities and must rely on the interpretations of others. Conflict also led to disruptions in M&E, something discussed later in this ICR. 125. Finally, South Sudan’s civil war disrupted the project’s critical assumption that funds would flow as planned : at the start of the project, two bilateral donors pulled out. During project design, the governments of the Netherlands, Norway, and Denmark committed to contribute a total of US$48.5 million to the project in addition to IDA’s US$50 million, bringing the project total to US$98.5 million. Faced with South Sudan’s volatility, however, these donors became concerned about political risk and value for money. This led the Netherlands and Norway to withdraw their commitments to the project, and Denmark to reduce its funding to US$6.9 million. The project thus faced a funding gap of US$41.6 million. 126. Yet conflict and local realities meant the project needed more money, not less. After learning what was needed to deliver infrastructure of acceptable quality and accessibility, the government increased the project’s per capita allocation from US$3.5 to US$7.0, leading estimated project costs to rise. Conflict and macroeconomic crisis, however, exacerbated this: the need to start and stop implementation and ensure security increased delivery costs, as did inflation and the plummeting value of the SSP, which caused the cost of (mainly imported) raw materials to increase. This resulted in a projected cost overrun to the project of US$33.5 million, increasing the total project cost from US$ 98.5 million to US$ 132 million, and increasing the financing gap to US$ 75 million (US$41.5 million + US$ 33.5 million). 127. The project expected to fill most of this gap through additional financing. In June 2017, 18 months before the project was due to close, the government began discussions with the World Bank to secure an additional US$65 million for the project to fill all but $10 million of the expected financing gap. Over the year that followed, the task team worked intensively with the government to prepare this additional financing, under which the PDO was expected to stay the same but the results framework was expected to be revised to accommodate changes in the country context (Box 1). The relevant aides-memoires show that the task team expected that, in addition to introducing changes in scope and expected results, additional financing would fund a “rigorous impact evaluation” of the project, put in place a “more Page 53 of 70 The World Bank Local Governance and Service Delivery Project (P127079) robust data collection system”, and strengthen line ministry participation in operations and maintenance, which had been curtailed by the country’s fiscal deficit.91 Box 1: Proposed additional financing The proposed additional financing project paper sought a level one restructuring to:  Extend the project closing date from 31 December 2018 to 31 December 2010  Revise the results framework to accommodate changes in the country context, changes in end of year targets, and the reduction in scope from 40 to 27 settlements. It was expected that the intermediate results indicators related to the predictability of fiscal transfers would be removed given that the PDG disbursement procedures had changed.  Expand the project to include an urban settlement, triggering OP/BP 4.12 on Involuntary Resettlement Source: Draft project paper for additional financing 128. However, this additional financing did not go through, leaving the project with a significant funding gap. In August 2018, the World Bank decided not to pursue additional financing and announced that project would instead need to close as planned in December 2018. The changes expected under the additional financing, including in the results framework, thus did not materialize, though the project did ultimately get a two-month extension to February 2019, to enable it to wrap up its administration connected to closing. 129. The project thus rolled out to fewer counties and built less infrastructure than planned, undermining its ability to meet all outcome areas, particularly that of improving service delivery . As is detailed in the efficacy section of this ICR, the project had to roll out to far fewer counties than planned. Component one, which aimed to improve service delivery through building basic infrastructure finance by payam development grants, was particularly affected, as the project was only able to build infrastructure in 14 counties, not the original target of 40, and the funds available to the PDG over the five-year period meant that 618 projects needed to be deferred. Although the project had planned to use US$30 million for component one, only US$5.9 million was used from the IDA allocation of US$ 14.8 million, with some monies reallocated to other components of the project. Yet because the project was not restructured, the achievement of the PDOs was judged against the original criteria. Components two and three also rolled out to fewer counties than expected. 130. This risked undermining citizen confidence in the development planning process, and therefore increased the risks to the project’s desired development outcome. Interviews conducted for this ICR in Juba with PMU and facilitating partner staff consistently reported that citizens in locations that waited too long for infrastructure—or that had been through the community engagement activities but might never see infrastructure funded—were become dissatisfied with the project and risked losing trust in it—and thereby losing trust in the participatory planning processes set up by the project and, by extension, the state. 131. The civil war also changed the World Bank’s calculus about risks to project finances in South Sudan in ways that similarly disrupted project fund flows. The civil war was accompanied by a sharp increase in security spending, as government spent on the war effort. In this context, development partners became increasingly concerned that development financing could be diverted: in other words, that aid money could be used to buy weapons. The World Bank thus decided to put in place additional fiduciary measures for projects. The risks to projects were later outlined Page 54 of 70 The World Bank Local Governance and Service Delivery Project (P127079) in the 2017 Country Engagement Note for South Sudan, which rated the fiduciary risks to World Bank projects in South Sudan as high across the portfolio. 132. This led to gaps in LGSDP disbursements until additional fiduciary measures could be put in place. Between the second major outbreak of conflict in July 2016 and March 2017, the World Bank made no disbursements for the PDG, leading to huge gaps between expected and actual disbursements. The gaps in disbursements slowed down implementation of component one, which focused on building infrastructure. As a result, contractors abandoned 30 subproject sites, some threatening to take legal action against the project. 618 sub-projects—more than half of the 1095 originally planned for—that had not been initiated were indefinitely delayed. 133. These extra fiduciary measures unlocked disbursements but were reportedly time-consuming to follow—and undermined some of the project’s local governance improvement aims. The main changes are outlined in Table 17. Interviews conducted for this ICR showed that the additional fiduciary measures were considered necessary by PMU staff, but that they were also time-consuming to follow. The borrower’s report stated that “the combined effect of suspending payments, and then instituting new measures, without exception, on all payments including for those that had already been submitted, meant further delays by contractors in accessing sub-project resources which they so desperately required to complete sub-projects.” Interviews conducted for this ICR also indicated that the additional fiduciary measures handicapped the capacity building component of the project: the requirement to pay contractors only upon receipt of work instead of on a 50/50 basis undermined the government’s ability to learn by doing in managing funds. Table 17: LGSDP additional fiduciary and procurement measures Fiduciary Use of direct payments Most non-PDG payments in the project now required TTL approval Disbursements based on Disbursements shifted from being based on interim unaudited financial reports to being statements of expenditure based on statements of expenditures, allowing for more control and oversight Lowered ceiling of designated DA ceiling lowered from two quarters of forecasted expenditures to a ceiling of US$500,000. account This reduced the amount of the funding the project had access to at any given time. PDG paid as needed Payam development grants paid after contractors have completed work and invoiced, as an advance as designed originally. This avoids funds sitting idle in bank accounts County inspectors of works Additional staff hired by the technical assistance firm to provide extra verification of sub- contracted by the TA firm to verify project progress, helping safeguard the LGSDP project coordinator from any potential physical progress of sub-projects external pressure/influence County withdrawal of PDGs from Counties require a supporting letter from the project coordinator to commercial banks before bank accounts withdrawing their PDG grants for payment to contractors Conversion of PDG PDG converted from US$ to SSP immediately prior to disbursement to counties to avoid loss of value Procurement Procurement reviews All procurement activities regardless of value, nature, categories, or packages treated as prior reviews Page 55 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Figure 4: War, economic collapse, and local government upheaval & LGSDP Page 56 of 70 The World Bank Local Governance and Service Delivery Project (P127079) (b) Macroeconomic crisis 134. Macroeconomic and fiscal crisis disrupted the critical assumptions that funds would flow through the system as planned and that materials and labor would be available and affordable. There were two main issues. First, with an austerity budget, the government could not disburse CDGs, and made no such disbursements throughout the life of the project. The CDG, however, was a critical part of the theory of change: the idea was that county government officials were supposed to learn on the job in managing not simply payam development grants but also county development grants. Without CDGs, they were not able to do this. In addition, the borrower’s report noted that the government integrated planning system was delayed every year from 2013/2014 onwards. Because PDG sub- projects were funded as part of annual county plans and budget, which was a precondition to trigger release of PDG funds, this meant that PDG funds were not provided on time to start building subproject infrastructure during the dry season. PDG sub-projects were thus tendered and awarded when the rainy season was approaching, leading to delays in sub-project execution. 135. Second, the plummeting value of the SSP caused disbursements to be suspended for the PDG between December 2015 and May 2016 while the project could work out an ‘in between’ exchange rate and caused the cost of (mostly imported) raw materials to rise. Contracts and payments were made in South Sudan pounds, so these fluctuations were problematic—it means that contractors were at risk of losing out because they had to buy mostly imported raw materials, the costs of which rose; there were also shortages of construction materials and fuel. Making payments in USD, however, was against South Sudan government policy, so the World Bank was not able to pay out in USD; nor were PDG sub project costs able to be pegged against US$. The World Bank attempted to negotiate with MOFEP on this but was not able to secure an exception. Eventually the PMU worked out an ‘in between’ exchange rate with the banks, but this only solved part of the problem. Fluctuations in the exchange rate also meant that SSP would sit in accounts rapidly losing value. The Bank's additional fiduciary measures, which disbursed funds not on a 50/50 basis but on based on need, brought some relief, as it means money wasn’t sitting for long in bank accounts losing value. However, financial crisis also caused bank branches to close, leading contractors to depend on project payments to complete works. 136. Macroeconomic crisis also reportedly contributed to a downturn in staff motivation . The borrower’s report noted that staff became demoralized due to a staggering erosion of their income due to high levels of inflation. Staff absenteeism in offices thus became high, and undermined county staff enthusiasm for the on the job training they received. (c) Local government upheaval 137. The proliferation of states and counties in South Sudan disrupted the project’s critical assumption that there would be enough continuity in local government structure, functions, and personnel for capacity improvements to be sustained. The borrower’s report (and interviews for the ICR) noted that there were significant staff movements because of these changes, and, perhaps even more importantly, uncertainty. Staff moved between states and counties, and within counties from one position to another. In some cases, county staff were promoted to state level positions, leaving county level vacancies. The borrower’s report reported that at the end of the project, less than 10 Page 57 of 70 The World Bank Local Governance and Service Delivery Project (P127079) percent of LGSDP trainees were still in the positions for which they had been trained. This raises a serious question over whether the training provided by the project was something that could be sustained. Page 58 of 70 The World Bank Local Governance and Service Delivery Project (P127079) ANNEX 7. THIRD PARTY MONITORING SUMMARY 138. The International Organization for Migration (IOM) conducted third party monitoring (TPM) for LGSDP from July 2018 to March 2019. The IOM TPM team examined 260 out of a sample of 293 sub-projects, conducted 31 focus group discussions with BDC members, 16 focus group discussions with PDC members, and did six institutional strengthening assessments. 139. The TPM findings broadly corroborated the evidence as reported by the borrower on the extent to which the project achieved its outcomes, though with some minor variations. The TPM report found evidence of strengthened community engagement in local development and was able to verify the outputs related to improved service delivery, though found mixed evidence on the effectiveness of institutional strengthening. Highlights of these findings are as follows: Outcome area 1: Strengthened community engagement 140. TPM found evidence of citizens becoming more engaged in implementing and overseeing local development. Over half of the subprojects visited by third party monitors had a system in place to fund maintenance costs through user fees. Such maintenance was reported to be carried out by local government and community members. Figure 5: Sub projects with user fees for maintaining infrastructure. Source: IOM third-party monitoring Payment status for classroom Payment for borehole use construction use 70% 58% 57% 60% 60% 50% 43% 42% 50% 40% 40% 30% 20% 30% 10% 20% 0% 10% Fees charged for Fees not being 0% classroom charged for Fees charged for Fees not being constructions use classroom borehole use charged for constructions use borehole use Payment Status for health facility use 94% 100% 80% 60% 40% 20% 6% 0% Non fees payment Fees payment for for health facility health facility use use 141. TPM also found the project’s participatory processes to be inclusive. The TPM report found a ‘great degree of inclusivity’, and corroborated the project’s figures on gender inclusivity, finding that 37 percent of BDC members Page 59 of 70 The World Bank Local Governance and Service Delivery Project (P127079) were women, and 47 percent of PDC members. The report also reported that disabled people were included in the BDCs and PDCs but could not report with certainty on the extent to which disabled people were represented in proportion to their share of the population. Outcome area 2: Improved local government capacities 142. Third party monitoring identified some mixed results with institutional strengthening. The monitoring examined six counties. In most of these counties, county staff members were able to describe procedures and refer to plans that indicated that they had been appropriately trained: for example, they described proper procurement procedures. Most counties, however, were not able to share documents with the third-party monitors, because they had poor filing systems or said they could not share them (for example, budgets). The IOM report noted that all counties lacked good filing systems, and that documents such as budgets and financial reports were written in English, something that created difficulties for county staff, who had difficulties communicating in English. The third party monitoring also identified problems with staff turnover at county offices. Outcome area 3: Improved service delivery 143. Project outputs were broadly corroborated by third party monitoring. IOM examined 260 of a sample of 293 of the project’s sub-projects, with the remainder not being accessible because of rainy season restrictions on mobility or because of insecurity. It did not report ‘ghost’ projects. Of the verified sub-projects, 97 percent existed, while 3 percent (eight) were either not constructed (because of indefinite rainy season deferrals) or constructed outside LGSDP. 95 percent of the subprojects were completed at the time of monitoring, with the remaining 5 percent late because of insecurity. Table 18: Sub projects delivering services (Source: IOM third party monitoring) Complete Subproject service offering status Complete subprojects NOT offering services by type 100% 86% 38% 90% 40% 35% 80% 35% 30% 70% 25% 60% 18% 20% 50% 15% 10% 6% 40% 3% 5% 30% 0% 20% 14% 10% 0% Complete subproject Complete subproject not offering a service offering a service 144. The project’s findings of an increase in functioning basic infrastructure were corroborated by third-party monitoring. IOM’s third-party monitoring report found that overall, 86 percent of completed and verified subprojects were delivering services. 14 percent were not offering a service. Among completed sub-projects not Page 60 of 70 The World Bank Local Governance and Service Delivery Project (P127079) offering a service, almost 38 percent were boreholes or hand pumps and 35 percent were latrines. Interviews conducted for this ICR, highlighted challenges with obtaining spare parts for boreholes and hand pump, which could explain this. 145. TPM verified that most types of sub-projects were built according to improved standards. LGSDP took measures to improve the quality of and people’s access to infrastructure, increasing the project’s per capita allocation from US$3.5 to US$7 to enable this. This was to ensure that sub-projects met environmental standards, were of a good enough quality, and enabled community members to use and access the service: for example, ensuring that schools were not simply built, but also furnished with desks, benches, and a writing board, and that latrines were built for students and staff within the school premises. Third-party monitoring found that most types of sub-projects were built according to these improved standards. Table 19: Sub-projects built to improved standards (Source: IOM third party monitoring) Classrooms with latrines Classrooms with chairs and desks 100% 88% 80% 72% 90% 70% 80% 60% 70% 60% 50% 50% 40% 40% 28% 30% 30% 20% 20% 12% 10% 10% 0% 0% Classrooms with no Classrooms with Classrooms with chairs Classrooms without latrines latrines and desks chairs and desks Health Facilities with Latrines Health Facilities with Incinerators 100% 80% 86% 68% 90% 70% 80% 60% 70% 60% 50% 50% 40% 32% 40% 30% 30% 14% 20% 20% 10% 10% 0% 0% Health facilities with health facilities Health facilities with Health facilities without latrines without latrines incinerators incinerators Page 61 of 70 The World Bank Local Governance and Service Delivery Project (P127079) 146. Third party monitoring found no deviations in technical standards. The IOM TPM team with the technical oversight from an engineer took some measurements of the lengths, widths and heights of some of the subprojects like classrooms, latrines and market sheds. No significant deviations were noted in terms of measurements except with differences of a few centimeters. Page 62 of 70 The World Bank Local Governance and Service Delivery Project (P127079) ANNEX 8. DATA SOURCES & RELIABILITY FOR EFFICACY ANALYSIS Table 20: Source & limitations of data, indicators for strengthening community engagement PDO Source & limitations PDO I #1 Source: Midterm review, June 2017. Limitations: Somewhat limited geographic coverage (six counties in three states), though care was taken to ensure representativeness. PDO I #2 Source: Project administrative data: monthly progress report, quarterly reports, PDC meeting minutes, PDC attendance log sheets. Limitations: Somewhat limited coverage. Only 93 of 155 PDCs supported by the project were assessed.92 PDO I #3 Source: Project administrative data: O&M plans in project profiles (Form E); community user group committee list and minutes; project completion report/certificate; project hand over report; county quarterly reports. Limitations: Limited sectoral scope. The data represent only water sector sub-projects. In other sectors, it was assumed the responsibility for sustainability lay with government, as the project was funding existing and functional small-scale infrastructure such as schools. PDO I #6 Source: Project administrative data I#10 IR Source: Project administrative data: Monthly and quarterly reports from facilitating partners I #11 IR Source: Project administrative data: grievance registers, grievance reports Table 21: Sources & limitations of data, indicators for improving local government capacity PDO/IR Source & any limitations PDO I #4 Source: Results of annual audit and performance assessment. 4 out of 9 assessed counties attained an above average result from the Local Government Performance Assessment of FY16/17. I#7 IR Source: PDG -MoFEP IPF letter. CDG-MoFEP County budget and CDG guidelines. I#8 IR Source: PDG disbursement letters from LGB. Comments: In 2018, conflict caused additional fiduciary measures to be introduced, 93 so PDG disbursements changed, rendering this indicator less relevant. I#9IR Source: Audit reports. Quarterly and annual budget performance reports. I#12 IR Source: County annual budgets. County Audit and performance assessment reports. I#13 IR Source: Audit and performance assessment reports Page 63 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Source Measures Methods & coverage Limits World Vision Focuses on Mixed methods – quantitative Some possibility of bias and a lack of measures of survey + qualitative interviews. representativeness. Although care End of Project Evaluation strengthened Quantitative survey had appears to have been taken in April 2018 community purposive sampling of sites, and selecting sites and ensuring a engagement but convenience sampling of minimum sample size of also monitors respondents within bomas. respondents, the respondents PDC within each boma were selected establishment through convenience and not and community 12 payams in 3 former counties random sampling, which introduces satisfaction with in Warrap State: Gogrial West, a possibility of bias and a lack of services Twic, Tonj representativeness. Limited geographic coverage. Only examines areas covered by World Vision, one of several project facilitating partners. Mid-term review of Local Assesses entire Key document analysis Somewhat limited geographic Governance and Service project coverage, though care appears to SWOT analysis Delivery Project have been taken to ensure these Qualitative research (focus were representative September 2016 group discussions & key informant interviews) Observation: transect walks 6 counties in three states: former Lakes State: Rumbek East, Yirol West, Wulu Warrap State: Gogrial West, Twic Eastern Equatoria: Kapoeta East, Ikwoto Page 64 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Cowater End of Objective 1: Report by the firm hired to Assignment Report: Improving local implement Component 3, Component 3, governance Institutional Strengthening, Institutional based on their own internal Strengthening of Local monitoring Governments May 2018 23 counties in 7 states Central Equatoria: Rejaf, Yei River Eastern Equatoria: Ikwoto, Kapoeta East, Kapoeta North, Magwi Lakes: Rumbek East, Wulu, Yirol West, Rumbek Center Northern Bahr el Ghazal: Aweil East, Aweil North Warrap: Gogrial West, Twic, Tonj North Western Bahr el Ghazal: Jur River, Wau Western Equatoria: Ezo, Ibba, Mvolo, Maridi, Nagero, Nzara Results, Outcomes, and Objective 1: Document analysis Limited geographic coverage of Impact of the Project’s Improving local qualitative work. Institutional governance Strengthening Activities Qualitative research: (Helge Rieper) Possible bias in the survey: 369 FGDs and interviews in Juba and county officials were targeted, with April 2018 in 4 counties in 3 states: 81 (22 percent) officials responding. Lake: Rumbek East, Rumbek Although this is over one in five Center respondents, it may be that county officials who liked the project were Western Bahr el Ghazal: Jur more likely to respond. River Warrap: Twic Page 65 of 70 The World Bank Local Governance and Service Delivery Project (P127079) Survey: 81 respondents in 12 counties Aweil East, Aweil North, Gogrial West, Ibba, Jur River, Kapoeta East, Kapoeta North, Rajaf, Rumbek East, Twic, Wulu, Yirol West Beneficiary assessment Lakes: Rumbek East Although numbers are reported in the assessment, they represent the Warrap: Twic entire population of focus group Western Bahr el Ghazal: Jur respondents only (i.e. those people River who were interviewed), not the broader population: the numbers Northern Bahr el Ghazal: Aweil are not based on a representative East survey. Eastern Equatorial State: Ikwoto Western Equatorial State: Ibba Central Equatorial State: Juba Page 66 of 70 The World Bank Local Governance and Service Delivery Project (P127079) 1 “Independent Evaluation of the Multi-Donor Trust Fund-South Sudan (MDTF-SS).” Fafo Institute for Applied International Studies, July 25, 2013. 2 The Local Government Act, 2009 (n.d.). http://southsudanhumanitarianproject.com/wp- content/uploads/sites/21/formidable/The-Local-Government-Act-2009.pdf . 3 The time between approval and project effectiveness was because of the time needed to secure parliamentary approval for the project 4 Source: Armed Conflict Location Event and Data ACLED – acleddata.com, as reported in the Country Engagement note FY18- 19. 5 2 million were IDPs and 2 million were refugees or asylum seekers elsewhere. 6 https://crises.lshtm.ac.uk/2018/09/26/south-sudan-2/ 7 “Economic and Social Costs of Violence Against Women and Girls, South Sudan.” UK Aid, University of Limerick, IPSOS, IPSOS Mori, T-Cons, NUI Galway, March 2019. https://www.whatworks.co.za/documents/publications/301-10079-nuig-vawg- south-sudan-technical-report/file. 8 This measures an aspect of objective two, improving service delivery, since it captures access by women to services, as well as an aspect of objective one, improving local governance, since it captures participation by women in community engagement 9 FY13-14 Interim Strategy Note (ISN), approved on 28 February 2013 10 FY18-19 CEN approved on 16 January 2018 11 FY18-19 CEN, referring to analysis in Aiding the Peace.2010. NORAD Country Evaluation Brief, 2016 12 “Country Engagement Note for the Republic of South Sudan.” World Bank Group, November 7, 2017, p. 18 13 CEN, page 9, based on World Bank and NBS survey data statistics, complied in the 2015 South Sudan SCD. 14 One of these counties, Ezo in Western Equatoria, was suspended due to security conditions after the end of 2015. 15 This was based on a sample of 47 payams in counties which participated in the Fast Track Initiative (FTI), which acted as a pre-project pilot for the community engagement component of the project. 16 The underlying figures are as follows: FY14/15 = (24/32); FY 15/16 = (81/131); FY 16/17 = (0/38). Total: 105/201 = 52.2% 17 This PDO indicator is also relevant for improved local government capacities and for the second objective, ‘improving service delivery in selected counties’. The underlying figures will be disaggregated and discussed in the text 18 This is based on a total of 774,366 beneficiaries of the project, 367,432 of whom were female, and 406934 of whom were male. 19 Consolidated numbers from facilitating partner reporting. Functioning’ was defined in the results framework as ‘meeting criteria on: composition (e.g. participation of vulnerable groups) and frequency of meetings with quorum’, so is a good indication of how participatory these processes were and how engaged community members were in them over time, though it does not shed light on the quality of that participation. The assessment covered WBG, Lakes, Warrap, and NBG States. 20 Ibid, p. 27 21 Ibid, p.29 22 “Assessment of LGSDP Community Engagement Methodology.” Forcier Consulting, 2016. 23 746 out of 1826 PDC members were women and 2604 out of 5017 BDC members were women 24 BDCs visited as part of the assessment consisted of 34 percent women, 53 percent youth, 14 percent elderly, and 4 percent people with disabilities, whereas PDCs consisted of 33 percent women, 54 percent youth, 12 percent elderly, and 5 percent people with disability (Beneficiary Assessment of the Local Governance and Service Delivery Project (LGSDP).” Pazel Conroy Consulting, November 2018 25 37 percent of BDC and 47 percent of PDC members were women 26 Data quality: the beneficiary assessment figures are representative only of the focus group discussion participants, not the broader population, so should be interpreted as insights into the mechanisms around grievance-handling rather than as the last word on them. 27 “Assessment of LGSDP Community Engagement Methodology.” Forcier Consulting, 2016. 28 A background study conducted as part of project preparation, “Mapping of community-based approaches in South Sudan.” Elsebeth Krogh, June 2012, identified NGOs, CBOs, and UN agencies who took a community-based approach to service delivery Page 67 of 70 The World Bank Local Governance and Service Delivery Project (P127079) in South Sudan. For example, the BRIDGE project, funded by USAID, established community action groups to aggregate and articulate community needs. Although the BRIDGE project aimed to foster linkages between these groups and local government, the groups were not part of the local government system as laid out in the LGA. 29 Rieper, Helge. “Local Governance and Service Delivery Project, South Sudan: Results, Outcomes, and Impact of The Project’s Institutional Strengthening Activities (Component 3),” April 2018.See also “LOGOSEED - Component 3 Institutional Baseline Assessment Report.” Cowater International, March 2015. 30 “Local Government Performance Assessment of Counties in the Republic of South Sudan.” Ministry of Finance and Planning and Local Government Board, Republic of South Sudan, July 2017, p. 12. 31 Rieper, Helge. “Local Governance and Service Delivery Project, South Sudan: Results, Outcomes, and Impact of the Project’s Institutional Strengthening Activities (Component 3),” April 2018. 32 “Beneficiary Assessment of the Local Governance and Service Delivery Project (LGSDP).” Pazel Conroy Consulting, November 2018, p. 48 33 The project substantially achieved this despite having to defer 618 sub-projects (Table 5) because of a lack of project financing, conflict, or insecurity; this is because the project planned to build 1095 sub-projects based on an expectation of additional financing, a number that was substantially above its intermediate results target of 500 sub-projects. 34 This calculation excludes the subprojects completed in the last year of the project, because a year has not gone by since they were built. 35 Some caution should be used in interpreting these findings, though, since it representative only of all focus group participants in the beneficiary survey, and not of the broader population. 36 Schools that were expanded for example needed additional tables and chairs for the students. 37 Data quality: the qualitative findings from the beneficiary assessment are reliable, but one should not infer that the perceptions of respondents on changes in access necessarily represent actual changes in access. 38 Investments already made for currently implemented sub-projects. 39 The cost-benefit analysis does not include the “dyke”, “incinerator”, and “culvert” sub-projects as data are not available. 40 Discounting Costs and Benefits in Economic Analysis of World Bank Projects, Feb 2016 41 “Aide Memoire: Scoping Mission for a Proposed Local Governance and Service Delivery Program.” World Bank, September 9, 2011. Also “Aide Memoire: Preparation Mission for the Proposed Local Governance and Service Delivery Program.” World Bank, March 23, 2012. 42 “Project Appraisal Document on a Proposed Credit in the Amount of SDR32.5 Million (US$50.0 Million Equivalent) to the Republic of South Sudan for a Local Governance and Service Delivery Project.” 43 Krogh, Elsebeth. “Mapping of Community-Based Approaches in South Sudan: Background Study for the Design of the Local Governance and Service Delivery Program (LGSDP) of the Government of the Republic of South Sudan in Cooperation with the World Bank.” Copenhagen Consultants, June 2012. 44 Pendle, “Local Socio-Political Organization and Implications for Community-Driven Development in South Sudan: An Analysis of Existing Literature.” 45 The last field visit by the Bank team was conducted from March 24-26, 2015. 46 However, the results framework did not have indicators to capture an increase in access to, rather than simply coverage of, services, and so was a somewhat incomplete measure of improved service delivery 47 The December 2013 political conflict largely between the SPLM and SPLM- IO. It was only later in 2014 when the conflict became much more fractured to include additional groups such as the Equatorians. 48 “South Sudan: The Perils of Payroll Peace.” London School of Economics, March 2019. 49 “Salvaging South Sudan’s Fragile Peace Deal.” International Crisis Group, March 13, 2019. 50 See aide memoire 51 “Local Governance and Service Delivery Project Implementation Completion Report (ICR) from the Borrower.” Ministry of Finance and Planning and Local Government Board, Republic of South Sudan, February 2019. 52 Ibid, Annex 3. Figures are also in ‘LOGOSEED Project overview, 8 Oct 2018’. 53 Ibid, Annex 3 54 These were temporary committees (expected to be dissolved once they had completed their task) through which citizens Page 68 of 70 The World Bank Local Governance and Service Delivery Project (P127079) could oversee implementation of sub-projects. 55 The remainder were not yet activated pending sub-project implementation. Figures are from the LOGOSEED Project Overview, 8 October 2018, presented as part of an ISM. The Borrower’s ICR report, “Local Governance and Service Delivery Project Implementation Completion Report (ICR) from the Borrower.” Ministry of Finance and Planning and Local Government Board, Republic of South Sudan, February 2019,” appears to contain earlier figures. 56 “Local Governance and Service Delivery Project Implementation Completion Report (ICR) from the Borrower.” Ministry of Finance and Planning and Local Government Board, Republic of South Sudan, February 2019. 57 Figures are from the LOGOSEED Project Overview presentation, 8 October 2018, presented as part of an ISM 58 End of Assignment Report: Component 3 - Institutional Strengthening of Local Governments. Cowater International, May 2018, p.13 59 Ibid, p. 13 60 Bugnion, C (1998). Use of cost and effectiveness indicators to evaluate ECHO funded humanitarian emergencies 61 A Systematic Review: Costing and Financing of Water, Sanitation, and Hygiene (WASH) in Schools. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5409642/ 62 Village Assessment Survey County Profiles. https://iomsouthsudan.org/tracking/sites/default/publicfiles/documents/Lakes.pdf 63 Water, sanitation and hygiene links to health: https://www.who.int/water_sanitation_health/publications/facts2004/en/ 64 W. Kip Viscusi* and Clayton J. Masterman Income Elasticities and Global Values of a Statistical Life https://law.vanderbilt.edu/phd/faculty/w-kip-viscusi/355_Income_Elasticities_and_Global_VSL.pdf 65 Progress on Drinking Water, Sanitation and Hygiene: 2017 Update and SDG Baselines. https://theconversation.com/women- still-carry-most-of-the-worlds-water-81054 66 A Systematic Review: Costing and Financing of Water, Sanitation, and Hygiene (WASH) in Schools. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5409642/ 67 Source: Sanitation and Hygiene in Africa Where Do We Stand? https://www.communityledtotalsanitation.org/resource/sanitation-and-hygiene-africa-where-do-we-stand-analysis-africasan- conference-kigali-rwanda 68 A Systematic Review: Costing and Financing of Water, Sanitation, and Hygiene (WASH) in Schools. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5409642/ 69 Global Partnership for Education. https://www.globalpartnership.org/education/the-benefits-of-education 70 Ibid 71 UKAID COST ANALYSIS OF THE ESSENTIAL PACKAGE OF HEALTH SERVICES (EPHS) https://assets.publishing.service.gov.uk/media/57a089e8e5274a27b2000305/Somalia-cost-analysis-of-the-essential-package- of-health-services-.pdf 72 Lincetto, O., Mothebesoane-Anoh, S., Gomez, P., and Munjanja, S. (2006). Antenatal care. Opportunities for Africa’s newborns, pages 51–62. World Health Organization. http://www.who.int/pmnch/media/publications/aonsectionIII_2.pdf 73 Sudan: Generating market opportunities. https://reliefweb.int/report/sudan/sudan-generating-market-opportunities 74 UN South Sudan COUNTRY PROGRAMME PERFORMANCE SUMMARY 75 Deloitte South Sudan. https://www2.deloitte.com/content/dam/Deloitte/global/Documents/Tax/dttl-tax- southsudanhighlights-2015.pdf 76 World Bank: Why road maintenance is important and how to get it done 77 World Bank Scaling up and coordinating investments in physical structures and infrastructure. http://documents.worldbank.org/curated/en/854221490781543956/122290272_201711346032501/additional/113851-PUB- PUBLIC-PUBDATE-2-9-2017.pdf 78 Why road maintenance is important and how to get it done. https://siteresources.worldbank.org/INTTRANSPORT/Resources/336291-1227561426235/5611053- 1231943010251/TRN4_Road_Maintenance.pdf 79 Only Operation and Maintenance (O&M) costs have been observed considering this is an ex post analysis, with investments already made (fixed). Page 69 of 70 The World Bank Local Governance and Service Delivery Project (P127079) 80 The Sudan People’s Liberation Army was the armed guerrilla group advocating for independence against Sudan. When South Sudan became independent, it became the national army. The name of the army has since been changed to the South Sudan People’s Defense Force. 81 The country had a maternal mortality rate of 102 per 100,000 live births, the highest in the world. 82 The country had an infant mortality rate of 102 per 1,000 live births. 83 “South Sudan Governance Analysis: Building Sustainable Public Sector Capacity in a Challenging Context.” World Bank, January 2017. 84 The Aid Strategy for the Government of the Republic of South Sudan (November 2011) highlighted the fact that although South Sudan received over $3 billion in international assistance between 2005 and 2010, in 2009, only 45 percent of this was aligned with government expenditure priorities. 85 The Local Government Act, 2009 (n.d.). http://southsudanhumanitarianproject.com/wp- content/uploads/sites/21/formidable/The-Local-Government-Act-2009.pdf . 86 http://grss-mof.org/wp-content/uploads/2011/11/RSS_Aid-Strategy.pdf 87 In 2010, oil revenues accounted for 98 percent of public revenues and 71 percent of GDP (NBS 2011, as outlined in the PAD). 88 Although the two countries did manage to agree on oil transit fees by August 2012, macroeconomic instability and austerity continued. 89 https://databank.worldbank.org/data/source/2?country=SSD 90 “The 28 States System in South Sudan.” Stimson Center, August 9, 2016. 91 See aide-memoire of 11-15 June 2017 (LGSDP Implementation Support and Additional Financing Preparation), which details what is expected to be funded under additional financing. 92 These were in WBG, Lakes, Warrap, and NBG states. 63 out of 93 PDCs were found to have functioning committees. 93 Instead of PDG disbursement triggering tendering and contracting, Counties were communicated confirmation of PDG sub- projects and permitted to tender, contract and execute subprojects. PDG was then released at the time when payments are due to contractors upon receipt of invoices for approval. Page 70 of 70