Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004751 IMPLEMENTATION COMPLETION AND RESULTS REPORT IDA-52840-001 ON A CREDIT IN THE AMOUNT OF SDR14 MILLION (US$21 MILLION EQUIVALENT) TO THE REPUBLIC OF SOUTH SUDAN FOR A SAFETY NET AND SKILLS DEVELOPMENT PROJECT August 28, 2019 Social Protection & Jobs Global Practice Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective August 15, 2019) Currency Unit = South Sudanese Pound (SSP) SSP159.218 = US$1 US$1.374 = SDR 1 FISCAL YEAR July 1 - June 30 Regional Vice President: Hafez M. H. Ghanem Country Director: Carolyn Turk Regional Director: Dena Ringold Practice Manager: Robert S. Chase Task Team Leaders: Endashaw Tadesse Gossa, Nadia Selim ICR Main Contributors: Steisianasari Mileiva, Suzana Abbott ABBREVIATIONS AND ACRONYMS AAH-I Action Africa Help-International AGI Adolescent Girls Initiative BDC Boma Development Committee CEN Country Engagement Note ESMF Environmental and Social Management Framework ESMP Environmental and Social Management Plan ESSAF Environmental and Social Screening Assessment Framework FCV Fragility, Conflict and Violence FM Financial Management GBV Gender-based Violence GNI Gross National Income GRSS Government of Republic of South Sudan GRM Grievances Redress Mechanism HFS High Frequency Survey HH Household ICR Implementation Completion and Results IDA International Development Association IDP Internally Displaced Person IMAC Inter-Ministerial Approval Committee IP Implementing Partner IRI Intermediate Results Indicator ISN Interim Strategy Note ISR Implementation Status and Results Report LGSDP Local Governance and Service Delivery Project M&E Monitoring and Evaluation MAFS Ministry of Agriculture and Food Security MGCSW Ministry of Gender, Child and Social Welfare MIS Management Information System MoAFCRD Ministry of Agriculture, Forestry, Cooperative and Rural Development MFP Ministry of Finance and Planning MoLPSHRD Ministry of Labor, Public Service and Human Resource Development NGO Non-Government Organization PAD Project Appraisal Document PDC Payam Development Committee PDO Project Development Objective PIU Project Implementation Unit RF Results Framework SNSDP Safety Net and Skills Development Project SNV SNV Netherlands Development Organization SP Social Protection EFCRP Emergency Food Crisis Response Project SSP South Sudanese Pound SSSNP South Sudan Safety Net Project TST Technical Support Team TTL Task Team Leader UNHCR United Nations High Commissioner for Refugees UNOPS United Nations Office for Project Services WASH Water, Sanitation and Hygiene WDR World Development Report TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 5 A. CONTEXT AT APPRAISAL .........................................................................................................5 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) .......................................9 II. OUTCOME .................................................................................................................... 11 A. RELEVANCE OF PDOs ............................................................................................................ 11 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 11 C. EFFICIENCY ........................................................................................................................... 16 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 17 E. OTHER OUTCOMES AND IMPACTS (IF ANY) ............................................................................ 17 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 20 A. KEY FACTORS DURING PREPARATION ................................................................................... 20 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 21 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 23 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 23 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 24 C. BANK PERFORMANCE ........................................................................................................... 26 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 27 V. LESSONS AND RECOMMENDATIONS ............................................................................. 28 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 31 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 40 ANNEX 3. PROJECT COST BY COMPONENT ........................................................................... 42 ANNEX 4. EFFICIENCY AND FINANCIAL SUSTAINABILITY ANALYSIS ........................................ 43 ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 53 ANNEX 6. REVISION OF INTERMEDIATE RESULTS INDICATORS .............................................. 61 ANNEX 7. PUBLIC WORKS LOCATIONS BEFORE AND AFTER THE SEPTEMBER 2017 PROJECT RESTRUCTURING ................................................................................................................. 62 ANNEX 8. LESSONS LEARNED AND RECCOMENDATIONS ...................................................... 63 ANNEX 9. BIBLIOGRAPHY ..................................................................................................... 67 The World Bank Safety Net and Skills Development (P143915) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P143915 Safety Net and Skills Development Country Financing Instrument South Sudan Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Republic of South Sudan Ministry of Agriculture and Food Security Project Development Objective (PDO) Original PDO Provide access to income opportunities and temporary employment to the poor and vulnerable and put in place building blocks for a social protection system. Page 1 of 67 The World Bank Safety Net and Skills Development (P143915) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 21,000,000 21,000,000 19,353,761 IDA-52840 Total 21,000,000 21,000,000 19,353,761 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 21,000,000 21,000,000 19,353,761 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 21-Jun-2013 10-Nov-2014 15-Jan-2018 31-Dec-2017 28-Feb-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 09-Sep-2017 2.75 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Reallocation between Disbursement Categories Change in Implementation Schedule 18-Dec-2018 19.15 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Satisfactory Moderately Satisfactory Modest Page 2 of 67 The World Bank Safety Net and Skills Development (P143915) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 04-Jan-2014 Moderately Satisfactory Moderately Satisfactory 0 Moderately 02 30-Apr-2014 Moderately Unsatisfactory 0 Unsatisfactory 03 08-Jan-2015 Moderately Satisfactory Moderately Satisfactory 0 04 30-Jun-2015 Moderately Satisfactory Moderately Satisfactory 1.43 05 04-Jan-2016 Moderately Satisfactory Moderately Unsatisfactory 1.55 06 20-Jun-2016 Moderately Satisfactory Moderately Satisfactory 2.02 07 22-Dec-2016 Unsatisfactory Unsatisfactory 2.41 08 26-Jun-2017 Unsatisfactory Unsatisfactory 2.41 09 21-Dec-2017 Moderately Satisfactory Moderately Satisfactory 3.47 10 10-Apr-2018 Moderately Satisfactory Moderately Satisfactory 11.38 11 04-Jan-2019 Moderately Satisfactory Moderately Satisfactory 19.15 12 28-Feb-2019 Moderately Satisfactory Moderately Satisfactory 19.31 SECTORS AND THEMES Sectors Major Sector/Sector (%) Social Protection 100 Social Protection 75 Public Administration - Social Protection 25 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Page 3 of 67 The World Bank Safety Net and Skills Development (P143915) Social Development and Protection 0 Social Protection 60 Social Safety Nets 30 Social protection delivery systems 30 Human Development and Gender 0 Gender 10 Labor Market Policy and Programs 30 Labor Market Institutions 15 Active Labor Market Programs 15 Environment and Natural Resource Management 0 Climate change 56 Mitigation 56 Private Sector Development 100 Jobs 100 ADM STAFF Role At Approval At ICR Regional Vice President: Makhtar Diop Hafez M. H. Ghanem Country Director: Bella Deborah Mary Bird Carolyn Turk Director: Ritva S. Reinikka Dena Ringold Practice Manager: Lynne D. Sherburne-Benz Robert S. Chase Endashaw Tadesse Gossa, Task Team Leader(s): Endashaw Tadesse Gossa Nadia Selim Suzana Nagele de Campos ICR Contributing Author: Abbott Page 4 of 67 The World Bank Safety Net and Skills Development (P143915) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. South Sudan1 was facing formidable development challenges at the time of the Appraisal of the Safety Net and Skills Development Project (SNSDP, the Project) in May 2013. Two years after its independence, South Sudan was experiencing on-going insecurity and violence, contributing to an environment of protracted conflict. As described in the Project Appraisal Document (PAD), cross-border conflicts persisted in disputed areas between rebel groups and armies of Sudan and South Sudan. The violence displaced about 500,000 people across the country between 2011 and 2013, exacerbated by an influx of returnees following the signing of the Comprehensive Peace Agreement in 2005. This triggered rapid changes in the country, deepening the vulnerability of South Sudanese households. Several factors contributed to South Sudan’s vulnerability: disparities in income and access to resources; climatic shocks; food insecurity; scarcity of social services; inadequate physical, economic and administrative infrastructure; and lack of viable employment opportunities, especially for unskilled labor. 2. These developmental challenges were exacerbated by chronic poverty. In 2011, the country’s Gross National Income (GNI) per capita was US$ 1,2202 and slightly over half of its population lived below the poverty line.3 Poverty rates varied greatly across regions and rural poverty was twice that of urban areas. Female headed households were among the poorest, with 56.9 percent below the poverty line compared to 48.1 for male headed households. War-induced poverty, displacement and trauma weakened kinship and community ties, endangering principles of participation, inclusion, transparency and accountability. 3. Human development indicators placed South Sudan among the poorest countries in Africa, and basic infrastructure was lacking. Maternal mortality was the highest in the world, infant mortality was 102 per 1,000 live births, and one third of all children under five were underweight. Moreover, less than one third (27 percent) of adults over 15 were literate, less than half of primary school aged children were enrolled, and only ten percent of those that were completed it. Further, only 44 percent of households lived within a five-kilometer radius of a health care facility, at least 30 percent of the population lacked access to safe water, and only seven percent had access to improved sanitation facilities. The rural population (approximately 80 percent) was often isolated across large swathes of land where infrastructure provision was, and continues to be, extremely poor. The country was young: 72 percent of the population was under the age of thirty. Unemployment was highest among youth, with around one fifth of those under 24 recorded as unemployed. Youth unemployment, particularly among men, was recognized as a potential trigger for social instability. 4. The Government of Republic of South Sudan (GRSS) in the South Sudan Development Plan (SSDP) 2011-2013 envisioned a united and peaceful new nation. The SSDP focused on building strong foundations for good governance, economic prosperity and enhanced quality of life for all and identified the importance of social protection (SP) in the achievement of the country’s development objectives. Specifically, it aimed to reduce risk, vulnerability, poverty, and economic and social exclusion throughout the country through SP interventions. 1 South Sudan became the world’s newest nation on July 9, 2011 and joined the World Bank Group as its newest member in Spring 2 012. 2 Atlas method. GDP and GNI were estimated to decrease substantially in 2012 and only partly recover in 2013. 3 Calculated at 72.9 SSP (approximately US$24) per person per month. Page 5 of 67 The World Bank Safety Net and Skills Development (P143915) 5. Nevertheless, as a post conflict and fragile state, the country had limited capacity by the Government, civil society, private sector and community at large to undertake necessary reforms and implement development programs. In particular, the Government suffered from weaknesses in administrative capacity, human and financial resources, and fiduciary accountability. Humanitarian aid financed by donors and delivered by international aid agencies and non- governmental organizations (NGOs) was the primary mechanism for providing basic services, including SP services, to the poor. But, humanitarian aid had limited coverage, and there was a concern that it could be detrimental to the state building process by undermining national and local capacities. 6. There was an urgent need to provide direct support to the poorest and most vulnerable households and put in place a predictable and reliable national safety net system. Given the poverty and vulnerability of the South Sudanese population, and the continuing social, political and economic instability, there was a need for interventions to address short-, medium- and longer-term development challenges. In the immediate term, a labor-intensive public works program was considered to be a practical short-term stabilizing force, as it would provide employment opportunities to the poorest and most vulnerable households, while at the same time support valuable investments in community assets (i.e., feeder roads, drainage systems, etc.). In the medium and longer term, there was a need to address “employability in fragile situations where violence may have eroded skills and work practices” (WDR, 2011) by providing skills development and putting in place the building blocks for a SP system to address vulnerabilities better. 7. The World Bank’s Interim Strategy Note (ISN) for South Sudan for FY13-14 focused on helping the country move from fragility to stability. The ISN had two clusters: (i) supporting activities which establish accountable state institutions that can deliver basic services to underserved populations, both by building central state capacity to manage the macro- economy and national fiduciary and statistical systems, and by developing local government capacity to deliver services, and (ii) building public and private institutions that can generate and sustain improved livelihoods and create jobs for the population. The SNSDP was expected to contribute directly to the objectives of both of these clusters. Theory of Change (Results Chain) 8. The SNSDP PAD did not describe a Theory of Change4. For this Implementation Completion and Results Report (ICR), the Project’s Theory of Change is constructed ex-post and has been retrofitted from the Project’s Results Framework (RF) and its detailed description. 9. Figure 1 below presents project activities, outputs, outcomes and its (long-term) expected impact. Activities under capacity building and project management had an overall impact on all outcomes. They are hence not connected to a single outcome in Figure 1. Project Development Objectives (PDOs) 10. The PDOs were to provide access to income opportunities and temporary employment to the poor and vulnerable and put in place building blocks for a SP system. Building blocks were defined as a SP policy, and operational tools to improve beneficiary identification and registration, monitor the program, and ensure that cash is transferred to eligible beneficiaries in a secure, transparent and timely manner. 4As per OPCS guidelines, including the theory of change became mandatory for PADs as of May 2018. The SNSDP PAD was approved by the World Bank’s Board of Directors in June 2013. Page 6 of 67 The World Bank Safety Net and Skills Development (P143915) Figure 1. Original Theory of Change Source: original for ICR Key Expected Outcomes and Outcome Indicators 11. The key expected outcomes as captured in the PDO were: (i) access to income opportunities and temporary employment to the poor and vulnerable, and (ii) building blocks for a SP system in place. The PDO indicators, as defined in the PAD, are presented below. The Project’s RF also contained 11 Intermediate Results Indicators (Annex 1). • Number of direct project beneficiaries, of which female (%) (core indicator); • Social protection policy developed and submitted for discussion; • Beneficiaries of public works complying with eligibility criteria, and • Targeting mechanism to select poor youth for skills development training designed and tested. Components 12. The Project comprised of three components, as presented below. The first component was designed to meet the expected outcome of building blocks for SP system in place, while the second and third components were envisaged to contribute to the expected outcome of providing poor and vulnerable households the access to income opportunities and temporary employment. The activities under these components were estimated at US$20 million. Given the high economic and security volatility in the country post-independence, an additional US$1 million was left unallocated for physical and price contingencies. Detailed project costs are presented in Annex 3. Page 7 of 67 The World Bank Safety Net and Skills Development (P143915) 13. Component 1: Social Protection System and Project Management (US$5.5 million equivalent). This Component was envisaged to develop the basic features of a SP system, and build necessary implementation and coordination capacity within the GRSS. The first year of implementation was meant to strengthen the capacity of the lead implementation agency, the Ministry of Agriculture, Forestry, Cooperatives, and Rural Development (MoAFCRD),5 by establishing a Technical Support Team (TST);6 recruiting Implementing Partners (IPs) NGOs; developing necessary implementation tools and procedures; monitoring and evaluation; and designing a communications strategy and plan. 14. Sub-component 1a, Support for a SP System, aimed to support the GRSS to draft and endorse a SP policy and put in place a set of basic operational tools as the foundation of a SP system. The tools were to include: (a) a common targeting mechanism; (b) a MIS, which could also serve as the basis of a beneficiary registry in the future; (c) a Grievance Redress Mechanism (GRM); (d) an accountable payment transfer mechanism; and (e) monitoring and evaluation (M&E) tools, including targeting assessment, beneficiary feedback, and Community Score Cards. Together, these aimed to ensure that: (a) safety net programs reach the poorest and most vulnerable; (b) the Government had an adequate mechanism to gather information and monitor progress; and (c) there was a formal system to transfer benefits in a secure, timely and predictable manner. 15. Sub-component 1b, Capacity Building and Project Management, aimed to strengthen the project management and supervision capacity. Successful implementation and management of project activities required ownership and support by a number of ministries, namely the Ministry of Gender, Child and Social Welfare (MGCSW), MAFS, and the Ministry of Labor, Public Service and Human Resource Development (MoLPSHRD). The MGCSW was responsible for the development of the Social Protection Policy, as well as supporting the establishment of the basic operational tools and coordination and governance structures. The MAFS was responsible for coordinating Public Works (Component 2), and the MoLPSHRD was to support Skills Development (Component 3), while the Ministry of Finance and Planning was to provide overall guidance. Field level implementation was supported by the IPs, in collaboration with state and county level government and MAFS staff. With overall fiduciary and project management responsibilities, the MAFS established a Project Implementation Unit (PIU) at the national level, staffed with consultants and government-assigned personnel for key project management functions. This sub-component financed activities to strengthen the management, coordination, and implementation capacity of the various ministry staff and the PIU. 16. Component 2: Public Works (US$11 million equivalent). This Component aimed to provide temporary seasonal income to poor and vulnerable households through participation in labor intensive public works focused on building, repairing, or maintaining local infrastructure that would benefit the community at large. Specifically, this Component meant to finance: (a) wages/transfers (US$3 per day per beneficiary) for 10 days/month for three months7 in a year in rural counties and four months in urban counties; (b) equipment, materials and skilled labor necessary to complete the selected public works with reasonable quality8; and (c) the administrative cost of IPs responsible for daily implementation and supervision. 17. Component 3: Skills Development (US$3.5 million equivalent). This Component aimed to provide poor and vulnerable youth aged 18-30 years with access to livelihood opportunities through comprehensive skills training,9 based 5 Then become the Ministry of Agriculture and Food Security (MAFS). 6 TST comprised focal points from the participating ministries to facilitate the efforts by the Project Implementation Unit (PIU) and worked across line ministries given the multi-sectoral nature of SP. 7 The three months would coincide with the peak hunger months, i.e., right before the agricultural season that ranges from December to April. 8 These could include hand tools that would be used by participants and material inputs that some of the public works would require, and skilled labor such as masons and carpenters who could provide technical supervision and guidance to the public works participants. 9 These included: sector or trade specific skills, basic soft skills, life skills, financial literacy skills, and entrepreneurial skills Page 8 of 67 The World Bank Safety Net and Skills Development (P143915) on market demand. Thus, a market assessment/skills gap study and youth needs assessment study was to be conducted in the six targeted counties to identify the range of trades for the skills training. The skills training was to be provided by non-profit and private sector agencies. Upon completion of the training, the IPs were to support beneficiaries to start or engage in income generating activities. This Component was to be a pilot carried out in two phases following a “learning- by-doing” and adaptation approach. During the first phase, training was supposed to be provided in two counties and Juba city to 600 youths; the second phase was supposed to further target more beneficiaries in the phase one locations, and scale up to the four remaining counties to benefit a total of 2,900 youths. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION (IF APPLICABLE) 18. The Project was restructured twice. The first restructuring addressed delays, implementation challenges and increased assistance needs that emerged following the outbreak of conflict in July 2016.10 Under this restructuring (September 2017), changes included, among others, the elimination of Component 3 (Skills Development), a reallocation of funds among components, a revision of PDO and intermediate indicators and targets, and an extension of the Credit’s Closing Date. The second restructuring (December 2018) was made to extend the Credit’s Closing Date. Details of the two restructurings are presented in Table 1 below. Table 1. Restructuring under the Project and changes introduced ISR Disb. Date(s) Change Ratings Reason for Restructuring & Key Changes (US$ M) DO* IP* The restructuring adapted the Project to focus primarily on Public Works (Component 2) to more effectively respond to the alarming food insecurity needs on the ground, requiring a scale-up and rapid response. The US$3.5 million funds from the Skills Development (Component 3) and the unallocated amount of US$1 Changes in: Results million were reallocated to the Public Works component, scaling it up to US$15.5 Framework, million. This allowed the SNSDP to provide rapid assistance to a greater number Components and of poor and vulnerable households than originally envisioned (i.e. 34,000 Sep Cost, Credit Closing MS MS 2.75 households vs. 25,000 households). In addition, the Credit’s Closing Date was 2017 Date, Reallocation of extended by one year to December 31, 2018 to ensure that all planned activities Funds, could be completed, accommodating the nearly one-year pause in Implementation implementation due to insecurity and violence. Further, the RF was updated to Schedule reflect both the changes to the components and the extended closing date, i.e. the PDO indicator and intermediate indicators related to the Skills Development component were removed, while targets for the other remaining activities were adjusted, as appropriate. The Government requested a two-month Closing Date extension to allow for Dec Closing date MS MS 19.15 successful completion of key activities, including those related to safeguards, 2018 fiduciary, and M&E activities, and further disbursement of funds. *DO: Development Objective; IP: Implementation Progress 10 The 2016 Juba conflicts were a series of clashes in Juba between rival factions of the Sudan People's Liberation Army (SPLA) loyal to President Salva Kiir and Vice-President Riek Machar, respectively. Overall, 300 people were killed, approximately 36,000 civilians fled parts of the city and there was massive destruction of properties. Page 9 of 67 The World Bank Safety Net and Skills Development (P143915) Revised PDO 19. The Project’s PDO was not revised. As per the Theory of Change, both Components 2 and 3 were designed to contribute to the outcome of providing poor and vulnerable households with access to income opportunities and temporary employment (see Para 12). As a result, the deletion of Component 3 during the first restructuring did not necessitate a change of PDO, since Component 2 continued to support that outcome fully. Revised PDO Indicators 20. Revisions to PDO Indicators are presented in Table 2 below. The two Intermediate Results Indicators (IRIs) related to the Skills Development activities were deleted, and targets for the other IRIs were revised upwards to reflect additional funding. See Table 1 in Annex 6 for complete list of the revised IRIs. Table 2. Revision of PDO Indicators End Target as per End Target after Sep No. PDO Indicators Unit of Measure PAD, June 2013 2017 restructuring 1. Direct project beneficiaries Number 25,000 34,000 2. Female beneficiaries Percentage 30 50 Social protection policy developed and submitted Text SP policy submitted No change 3. for discussion for discussion Beneficiaries of public works complying with Percentage 85 No change 4. eligibility criteria Targeting mechanism to select poor youth for skills Text Targeting mechanism Deleted 5. development training designed, tested and finalized/revised finalized/revised Revised Components 21. As described above, the Project’s Component 3, Skills Development, was removed from the Project.11 Component 2, Public Works was scaled up with funds originally allocated to Skills Development activities and the unallocated funds. Other Changes 22. The duration of participation in public works by beneficiaries was adjusted following the reduced period of overall project implementation and a UN Agency (United Nations Office for Project Services, UNOPS) became the IP for the six locations outside Juba.12 The public works duration was reduced from 10 days/month for three months in a year in rural counties and four months in urban counties for a total of three years (totaling 90-120 days of work) to 15-20 work days per month in one year (totaling 90 days) for both urban and rural areas given the wide-spread prevalence of poverty and vulnerabilities across the country and the reduced project implementation period, affected by multiple outbreaks of violence in the country. Specifically, following the July 2016 conflict, many organizations, including one of the two originally 11 By the time of the first restructuring, this component was yet to start. It took longer than anticipated to procure a service provider for implementation of the planned activities as there were not many skills training programs for youth in the country. The relatively slow progress and the rapidly increasing vulnerabilities following the July 2016 outbreak of violence led to a decision to cancel this component, allowing more people to benefit from public works. The Task Team assessed that the worsening macroeconomic conditions (characterized by triple digit inflation and a rapidly devaluing exchange rate) and limited job opportunities would not allow successful implementation of the skills component as originally designed. 12 The other six locations/counties were: Torit, Bor, Pibor (replacing Ayod where conflict took place) Kapoeta East, Gogrial West and Tonj South. Page 10 of 67 The World Bank Safety Net and Skills Development (P143915) hired IPs, left the country. This situation led to a need for adjusting the Project’s original plan of hiring multiple organizations as IPs in each of the Project’s seven locations. Instead, the UNOPS was selected to implement the public works component outside Juba (Section III B, paras. 61 to 62). Rationale for Changes and Their Implication on the Original Theory of Change 23. The rationale of the first restructuring is presented above. The Original Theory of Change was affected in that the input, output and outcome specific to the Skills Development Component were removed. Therefore, the revised Theory of Change is very similar to what was presented in Figure 1 above without the last boxes under Activities and Outputs that relate to the Skills Development Component. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 24. Rating: High. The Project’s PDO was relevant at the time of appraisal and continued to be relevant at completion. At Appraisal, the Project’s objectives and design were relevant to South Sudan’s development priorities and consistent with the World Bank’s FY13-14 country ISN. At completion, the PDO remained well aligned with the strategies outlined in the World Bank’s Country Engagement Note (CEN) FY18-19 for South Sudan, namely: (a) support to basic services for vulnerable populations, and (b) support to food security, livelihoods and basic economic recovery. The CEN acknowledges that SP interventions are key to supporting livelihoods and generating early economic recovery. At completion, the SNSDP responded to both of the CEN objectives and contributed to the achievement of its target by supporting 53,163 households (355,237 individuals) with access to income opportunities and temporary employment through public works, and by establishing the foundation for a SP system in the country. Despite changes in the Project’s components due to the restructuring, the relevance of its PDO was High throughout implementation. The poverty rate in South Sudan remains extremely high and thus social safety net interventions continue to be relevant. The World Bank is responding to this need by preparing a follow-on IDA-financed project scheduled for consideration by the Board of Executive Directors in FY20. B. ACHIEVEMENT OF PDOs (EFFICACY) 25. The SNSDP achieved its PDO, as measured by the PDO indicators and IRIs. Annex 1 summarizes the Project’s progress towards the PDO and IRIs in its Results Framework (RF), as well as its outputs. The Project successfully achieved important results in an extremely challenging environment characterized by fragility, weak capacity, conflict and insecurity, and political and economic uncertainties throughout its implementation. 26. The two distinct objectives of the SNSDP PDO (para. 11) will be rated separately. The Project’s implementation was characterized by two distinct phases: initial implementation before the first project restructuring in September 2017 and then after that restructuring. Before the 2017 restructuring, implementation of a somewhat ambitious project was challenging (paras. 61 to 64), with progress limited almost exclusively to policy actions in support of Objective 2. Activities in support of Objective 1, under both Components 2 (Public Works) and 3 (Skills Development) stalled; hence, progress towards this objective was Modest before the restructuring (para. 40). Although the PDO remained unchanged Page 11 of 67 The World Bank Safety Net and Skills Development (P143915) throughout, the Project’s RF (including indicator targets) was adjusted to align project design to the realities on the ground better by narrowing its scope in terms of the types of interventions (but not in terms of the number of beneficiaries). Therefore, this section evaluates Efficacy both before and after the 2017 restructuring, as per the World Bank’s ICR Guidelines.13 The Project’s Outcome rating is derived by weighting the share of disbursements made in the periods before and after approval of the restructuring (Table 3) The section below describes progress towards the PDO, and against PDO and IRI targets. Assessment of Achievement of Each Objective/Outcome Objective 1: Provide Access to Income Opportunities and Temporary Employment to the Poor and Vulnerable 27. The Project was successful in providing access to income opportunities and temporary employment to 53,163 poor and vulnerable households of which 74 percent were women, greatly exceeding the revised targets of 34,000 households and 50 percent women beneficiaries.14 The independent project evaluation carried out in Juba15 found positive results in terms of the impact of cash transfers. Thirty-six percent of respondents in Juba bought food for their households with the cash, while 26 percent and 22 percent paid school fees and medical bills, respectively. Eighty-nine percent indicated that their household income increased, and of these, 82 percent reported that the increased income led to greater frequency and better quality of meals. A number of beneficiaries also reported to the ICR mission that they used the cash transfer to save and start small scale income generating activities. 28. In total, about US$11 million was transferred to beneficiaries as public works wages for a little over four million total person days worked, exceeding the IRI target of 3.6 million days. The works resulted in the rehabilitation/maintenance of 517 community assets and innovations in group farming in rural areas, contributing to increased household food availability. Beneficiaries and community members also mentioned that enhanced community assets, such as roads and clearing of drainage channels, have supported mobility and access to markets, improved delivery of social services, and reduced incidents of communicable diseases. These results were achieved mostly after Project activities resumed in May 2017 and the Project was restructured. At the time of restructuring, the Project had only provided temporary employment to 4,865 beneficiaries (60 percent were women) through participation in public works in Juba municipality and Torit county16 (See Annex 7 for Maps of public works locations before and after the September 2017 restructuring and overlaid with conflict areas). Also, the skills development component had yet to recruit a service provider for implementation and thus did not contribute to the achievement of Objective 1. 29. These results were achieved using a community-based approach that empowered communities to take a lead in managing and implementing the public works component. This was facilitated by establishing community level structures in line with the Local Governance Act (2009) and building community and local government capacity to effectively undertake their roles and responsibilities, closely tied with community mobilization and sensitization to deepen understanding of safety net design and operational processes. While the number of beneficiaries supported and community assets maintained were important in improving household and community welfare in the immediate term, establishing and strengthening capacities of local structures were deemed by the Government to be much more important 13 World Bank (2018), Bank Guidance on ICR for Investment Project Financing (IPF) Operations, Page 17. 14 The revised targets had been established taking into consideration that additional funding was assigned to the public works component. 15 HUB Consulting (2018), End of Project Evaluation: Juba Public Works Project. 16 This was around 20 percent of the original end target, leading to Modest efficacy rating in Table 3. Beneficiaries who were to receive US$3/day for a total of 90 days’ work, received this amount only for 60 days’ work in Juba and 21 days in Torit due to the outbreak of conflict in June 2016. A total of 205,700-person days of work for 56 subprojects had been completed and a total of South Sudanese Pound (SSP) 12,684,000 had been transferred as direct wage payments. Page 12 of 67 The World Bank Safety Net and Skills Development (P143915) in the longer term in addressing fragility in South Sudan given the very weak capacity base. Additional information on the performance and outcomes of the local structures are provided under Objective 2 below. Objective 2: Put in Place Building Blocks for a Social Protection System 30. Starting from a context where no social protection system was in place in the country, the Project successfully put in place the building blocks of a future national SP system. Unlike activities under the public works component, activities in support of this objective were implemented well before the conflict, as envisioned under the Project’s implementation plan. 31. Social Protection Policy. As a first step, the MGCSW developed the SP national policy framework, which was approved by the Council of Ministers and launched by the MGCSW in May 2016.17 The development and approval of the policy signaled strong commitment to establishing a SP system in the country18. The MGCSW also established, staffed and equipped a SP Unit in the MGCSW to spearhead and coordinate SP-related activities. To that end, the SP Unit established the National Social Protection Technical Working Group, with support from the World Bank and the United Nations International Children’s Fund (UNICEF), to bring together the Government, humanitarian and development actors for strategic coordination and alignment among SP system and program interventions. Discussions with the World Food Programme (WFP) and UNICEF representatives during the ICR mission affirmed their appreciation toward MGCSW’s leadership and commitment to sustain the SP Working Group. 32. Targeting. At appraisal, South Sudan did not have in place a rigorous and transparent mechanism to prioritize participation of extreme poor households in a development program. The Project introduced a three-step targeting process to select project beneficiaries, namely geographical and administrative to select the states, counties and communities, as well as community-based to select households.19 A Beneficiary Assessment carried out in three sample locations at the end of the project, found that 80 percent of those surveyed were "satisfied with the categories of households selected as these were deemed to be the most vulnerable."20 Using this as a proxy to report against the relevant PDO indicator, the Project was close to achieving the target which was set at 85 percent. Further, there was a general concern among respondents across sites that the Project targeted a very small proportion of vulnerable households, given the enormous needs. All respondents were satisfied with the involvement of communities and their leaders in the selection of project beneficiaries. 33. MIS. The development of the MIS was fundamental for beneficiary management and monitoring of implementation progress, evidence-based decision-making, and program oversight and accountability. Data collection, storage and processing by the MIS was already developed and functional21 (intermediate indicator) at the time of project restructuring. An independent MIS Assessment,22 together with an independent financial management (FM) Audit conducted towards the end of the Project, identified areas for further development in the MIS related to its proper management, administration and usage.23 17 This PDO indicator was met ahead of schedule, i.e. the policy was approved by the Council of Ministers in April 2015 (before 2017 restructuring). 18 This was driven by the MGCSW, with support from the World Bank through the SNSDP and the UNICEF. 19 See Annex 4. 20 The study is based largely on qualitative information, and the sample size was small. This means that extrapolation to be representative of all the seven locations would not be feasible. Nevertheless, its findings are reported here as it was a project activity. 21 It was initially developed as web-based information system, managed by the PIU at the MAFS, with basic functions for beneficiaries’ registration, sub-project recordings, and beneficiaries’ attendance and payment processing. 22 This was conducted between December 2018 and January 2019. 23 These included, among others, issues of data privacy and security; internal data quality checks and controls; synchronization of the MIS with the Page 13 of 67 The World Bank Safety Net and Skills Development (P143915) 34. Payment mechanism. The payment mechanism was established, in partnership with a commercial bank, based on two-factor authentication. The payment procedure involved three parties and the World Bank, allowing for rigorous checks and balances.24 According to the commercial bank’s records, 63.6 percent of payments were received on time (intermediate indicator), while others experienced delays due to MIS glitches, internet connectivity issues, limited access due to weak infrastructure, insecurity and/or the rainy season, and low staffing. The MIS was upgraded to include biometric identity verification and geo-tagging of payment locations, which was innovative in a capacity constrained fragile environment such as South Sudan. The biometric capacity strengthened the registration and payment verification system, which aimed to reduce potential risks of fraud, thereby greatly enhancing transparency and accountability in the utilization of resources. The payment mechanism developed under this project can be used by other cash transfer programs in the country and will contribute to the significant reduction of fiduciary risks. 35. The Beneficiary Assessment revealed that the majority of respondents (78 percent) were unsatisfied with the payment process. Specifically, beneficiaries were unhappy about the long time it took to receive money, noting that delays in payment disrupted their cash flows, caused challenges with their creditors, and forced them to constantly borrow. However, while beneficiaries reported delays in payments, records of the payment agency that were provided to the ICR team show that 63.6 percent of payments arrived at the communities on time (within 15-30 days of completion of the work), as stipulated in the Project Operations Manual. However, challenges such as the rainy season, absent beneficiaries, difficult terrain, connectivity, and verification issues often required additional time to deliver cash in the hands of the beneficiaries. Further, the IPs failed to communicate the payment schedule properly and the beneficiaries expected to receive payment immediately upon completion of work. The survey indicated that beneficiaries were happy with the use of biometric system as it reduces potential fraud or corruption opportunities. 36. Implementation arrangements for safety net delivery. The management, coordination and implementation capacity of the various ministry staff and the PIU was strengthened through a number of activities. Twinning arrangements of national PIU staff with international experts, resulted in the PIU’s transition from one staffed fully by international consultants to one with a majority of South Sudanese staff. Other activities included technical assistance, knowledge exchange and study tours of ministries’ staff, which led to strong commitment and ownership of the Project by the Government, as demonstrated by the high level joint Ministerial visit by the four Ministers, Juba Deputy Mayor and other senior officials to public works location in Juba city. Further, the institutional arrangement involving multiple ministries motivated the four relevant ministries to coordinate and collaborate closely to ensure effective implementation, leading to a holistic and multi-sectoral implementation approach. 37. The Project also involved NGOs, a United Nations agency and a commercial bank, to support implementation on the ground. This provided a mutual opportunity for the PIU and these agencies to learn new skills and build capacity for future, similar interventions. For example, engaging with the commercial bank in the cash transfer facilitated them to develop new business opportunities by reaching clients they were not serving before. Conversely, it strengthened understanding and capacity by the PIU regarding how to deliver cash in a transparent and accountable manner using technology such as biometric verification mechanisms. Similarly, it enabled UNOPS to strengthen its understanding of safety nets and of transparent, participatory, and accountable community-based delivery mechanisms. biometric system; reconciliation of payments; restricted access and audit trails; lack of hardware, software and human capacity in the Government to assume full responsibility for the MIS and the management of its data. The developer continues to host and manage the MIS fully under another World Bank-funded project in the same PIU. 24 These included: (a) generation of payroll by IPs; (b) review by the PIU; and (c) actual payment to the beneficiaries by the commercial bank. Page 14 of 67 The World Bank Safety Net and Skills Development (P143915) 38. In addition, the MAFS established local level oversight and coordination structures to support implementation of the public works component, in line with a community-based implementation approach. These included: State Technical Committee, County/Municipality Core Team, Payam/Block and Boma/Quarter Council Development Committees, Appeals Committees, and Community Supervision Teams, established by the IPs at the state, county/municipality, payam/block and boma/quarter council25 levels, with support from the PIU. These committees took the lead in mitigating risks and ensuring smooth implementation, and served as a liaison between the project and the communities. At first, these arrangements were considered as overly complex for South Sudan. However, in the end, the MAFS informed the ICR mission that they found these local level institutional structures to be highly helpful for project management as it promoted a culture of accountability and transparency. Further, given the high levels of mistrust in communities in South Sudan, exacerbated by weak or absent local governance structures, participatory approaches taken by the Project were cited to contribute towards improved trust among the beneficiaries and local authorities. It was felt that in the future, these structures could be utilized by other development projects and serve as the foundation for a local governance structure, assuming that the capacities built in these structures will endure despite changes in the national political context. 39. GRM. The GRM was a community-based structure26 that aimed to address grievances regarding the Project by beneficiaries and communities, as well as mitigate against social risks, tensions, disputes, and in extreme cases, violence. As such, Appeals Committees were set-up and trained in all the participating bomas and quarter councils and consisted of well-respected non-beneficiaries selected from and by the community. The GRM in Juba also piloted the inclusion of gender-based violence (GBV) focal points within the GRMs to sensitize the community, Appeals Committee members, chiefs, and local leaders on GBV and gender issues. The community-based GRM was considered to be functional by project completion, and was found to be largely capable of addressing complaints raised by beneficiaries including incidents of violence in Kapoeta East (see para. 68). Nonetheless, weaknesses exist, particularly in terms of its reporting capacity to identify and mitigate emerging social risks (see more details in Lessons and Recommendation Section). Box 1. GRM in SNSDP The GRM in SNDP was a community based, formalized way to accept, assess, and resolve community feedback or complaints. The lowest level of the GRM consisted of the Group Leader of each public works beneficiary group. If a Group Leader was unable to resolve an issue, it was then escalated to the Community Supervision Teams. If they were unable, it was then escalated to the Appeals Committees, often the most important level of the GRM as most beneficiaries opted to go directly to the Appeals Committee. Each Committee consisted of 5 to 7 people from the community who were not beneficiaries of the project but command respect in the community where the project was implemented. All grievances and complaints were received and recorded by the elected focal person/secretary of the Appeals Committee. The Appeals Committees met twice in a month to handle grievances/complaints recorded within the month. If the committee felt a complaint could not be resolved at the Quarter Council/Boma level, the committee forwarded the case to the Boma Development Committee (BDC)/Quarter Council Development Committee (QCDC). If it could not be resolved at the BDC/QCDC level, it was then escalated to the Payam Development Committee/Block Development Committee, then the County Core Team/Municipality Core Team, and finally to the PIU. At the beginning of implementation, it was common to have a higher number of grievances especially on beneficiary registration process and name replacements, then the number of complaints declined as the public works activity progressed. 25 Payam and boma are South Sudanese administrative units in rural areas. Their equivalent in urban areas are blocks and quarter councils. Payam is the official administrative unit below the county (municipalities in urban areas), and the boma is the smallest administrative unit, below the payam. A boma is typically a collection of villages. 26 It comprises structures at the lowest level (that is, group leaders of beneficiaries and Appeals Committees) to the highest level (that is, the implementing agency). Page 15 of 67 The World Bank Safety Net and Skills Development (P143915) Justification of Overall Efficacy Rating 40. The Project’s Efficacy is rated as Substantial, based on the following ratings for achievement of each of its objectives, both before and after the restructuring. Progress towards Objective 1, Provide Access to Income Opportunities and Temporary Employment to the Poor and Vulnerable, was Modest before the restructuring and Substantial thereafter. Progress towards Objective 2, Put in Place Building Blocks for a Social Protection System, was Substantial both before and after the restructuring. Overall (throughout implementation as a whole), Efficacy is rated Modest before restructuring in view of limited progress towards Objective 1, but Substantial thereafter. C. EFFICIENCY Assessment of Efficiency and Rating 41. The economic and financial analysis in the PAD estimated the Project’s benefits for both the public works and skills development components based on: (a) qualitative economic justification and rationale,27 (b) justification for government involvement in achieving project objectives; and (c) assessment of the relative cost effectiveness of each component. As the skills development component was eliminated in the restructuring, the Project’s efficiency is rated on the basis of: (a) efficiency of the public works component, and (b) efficiency with which the Project was implemented. The full economic and financial sustainability analysis is provided is Annex 4. 42. The analyses suggest that the public works component was cost effective compared to earlier and similar programs in the region and the world. The estimated cost-effectiveness of the public works component is 0.42. This means that it costs US$1 for every US$2.36 transferred in wage benefits to the intended recipients. This estimate compares very well to the overall cost effectiveness of similar programs, despite South Sudan’s very high operational costs. At appraisal, it was estimated that it would cost US$1 for every US$2.07 transferred in wage benefits. Thus, the Project was actually more cost effective than estimated at appraisal. The cost-effectiveness of the program is primarily due to the transfer sizes and high labor intensity. 43. For the public works component, the Project resulted in a higher labor intensity than originally estimated. Project data estimate a labor intensity of 0.70 at the end, implying that 70 percent of program costs went to wage transfers to project beneficiaries.28 This result is better than the 0.67 that was assumed at the appraisal stage and is on par with or in some cases even better than similar projects in the Africa region and in the world (Annex 4). 44. The Project was implemented efficiently and utilized all the allocated funds well. When the Project was restructured in September 2017, only US$2.75 million equivalent of the Credit had been disbursed. After restructuring, implementation scaled up rapidly, and the Project implemented and disbursed in less than 1.5 years close to all of the allocated budget.29 Further, project management was efficient, as the PIU responsible for project management, including fiduciary management, was shared among three projects. There was no ineligible expenditure throughout project implementation. 27 The PAD noted that a full cost-benefit analysis of the components was not possible because of the uncertainties about the magnitude of long-term benefits, given the lack of data and few experiences with similar projects in the South Sudanese context. 28 This computation does not include US$340,774 which project documents show as not spent by end of February 2019. If this amount is considered spent on wage transfers, then the labor intensity would increase to 0.72. However, it is not clear whether this amount was saved on Component 1 or Component 2; thus, it was not factored into the labor intensity analysis. 29 Exact number and percentage not yet available. Page 16 of 67 The World Bank Safety Net and Skills Development (P143915) Assessment of Efficiency and Rating 45. Overall, the Project’s Efficiency is rated Substantial, based on its cost effectiveness and high labor intensity, as well as the efficiency with which it was implemented following the first restructuring. D. JUSTIFICATION OF OVERALL OUTCOME RATING 46. The overall outcome of the Project is rated as Moderately Satisfactory (MS). The Project’s relevance was High and efficiency was Substantial at the end of the Project, as described in the corresponding sections above. Substantial progress toward Objective 2 before restructuring, followed by progress towards Objective 1 due to timely restructuring, allowed the Project to achieve its overall development objectives at closing with moderate shortcomings. As weighted by disbursement before and after restructuring30, the project’s final overall outcome rating is MS as detailed in Table 3 below. Table 3. Overall Outcome Rating Relevance Efficacy by Objective* Efficiency Overall Value of the Share of Weighted Final Overall of PDOs Outcome Disbursed IDA Disburse Value Outcome Rating Rating31 Credit (US$) ment Before 1) Modest MU (3) 2,750,000 13.1% (3*0.131) 0.393 + 3.178 = Restructuring 2) Substantial = 0.393 3.571, rounding Overall: Modest it to 4.0 High Substantial = MS After 1) Substantial MS32 (4) 16,682,43633 79.4% (4*0.794) Restructuring 2) Substantial = 3.178 Overall: Substantial *Objective 1: Provide access to income opportunities and temporary employment to the poor and vulnerable; Objective 2: Put in place building blocks for SP system E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 47. As reflected in recent analytical work carried out by the World Bank, the Project was notable for taking into account the gender gaps and addressing them to the benefit of female beneficiaries.34 The Project’s design emphasized the participation of women through the PDO indicator of number of female beneficiaries,35 and by primarily targeting poor female-headed households. The number of female participants in the Project’s public works exceeded its original 30 World Bank. 2018. Bank Guidance: Implementation Completion and Results (ICR) Report for Investment Project Financing (IPF) Operations. Issued and Effective: September 27, 2018. 31 Following the formula outlined in Appendix H of the World Bank’s Guidance for ICR for IPF Operations, the Outcome rating for the period bef ore restructuring is Moderately Unsatisfactory (MU) while it is Moderately Satisfactory (MS) after restructuring. 32 Following moderate shortcomings in achievement of one or more of the objectives/outcomes used in the assessment of the overall Efficacy. 33 Disbursement data from the credit account shows US$389,006 undisbursed by the time of the ICR writing. Further, US$1.17 million is recorded as a loss due to the variations in the SDR to US$ exchange rates over the life of the credit. 34 World Bank, 2018, Strengthening Gender Equality in Social Protection Programs in South Sudan. The report includes a portfolio review to analyze the integration of gender equality in 24 World Bank-financed projects whose main objective is poverty reduction. Fourteen of the projects were implemented in South Sudan, and the remaining 10 in similar fragile and/or conflict affected countries. Out of the 14 projects in South Sudan, the SNSDP was highlighted as notable for taking into account the socio-economic gaps between men and women and establishing quota for female beneficiaries as part of the PDO indicators. The Project was also one of two in South Sudan that took into consideration the constraints for women’s economic participation and incorporated design features accordingly. 35 Initial target was 30 percent, then following good progress at the time of project restructuring, it was revised to 50 percent. Page 17 of 67 The World Bank Safety Net and Skills Development (P143915) target, affirming the body of knowledge that public works are not biased toward male labor.36 Discussions with IPs and beneficiaries37 revealed that the targeting process usually took place in the morning and that was helpful for targeting women since they were usually available at homes or neighborhoods during that time. Meaningful participation of women in public works activities was achieved through: (a) participation of women in the selection of public works activities and in the community structures;38 (b) work activities and payment sites were in close proximity to beneficiaries’ houses; (c) light and appropriate work for women, including consideration for pregnant and lactating mothers included in the work norms; (d) selection of work hours taking into account women’s household obligations; (e) provision of work site shelter for children during work hours; and (f) zero tolerance towards violence. In addition, the Project recognized that there could be a potential for GBV given the pervasiveness of the issue and especially as cash transfers can potentially put women at risk as recipients and/or worsen intra-household tensions.39 To address this concern systematically, the Project tried to raise awareness on GBV and other gender-specific issues among stakeholders during community mobilization and sensitization, in addition to piloting GBV focal points in the Juba GRM. Institutional Strengthening 48. Possibly more impressive than the number of beneficiaries supported, and community works financed were the Project’s achievements in terms of institutional strengthening that were especially impressive in the complex environment in which these were put in place. The Project helped put in place and strengthened an institutional process to not only support its implementation, but to implement other, future community driven development-type projects. This institutional process strategically brought together three relevant government ministries, with overall project coordination provided by the MAFS, and involved collaboration with state and county level staff that worked with communities to help identify beneficiaries and prioritize works. This effective institutional process, with the IPs working in collaboration with state and county level government and MAFS staff, helped support execution on the ground using a conflict sensitive implementation approach. With overall fiduciary and project management responsibilities, the MAFS established a PIU at the national level, staffed with consultants and government-assigned personnel for key project management functions, and a TST. Specific capacity building measures, such as the TST’s twinning arrangements with international experts, knowledge exchange, and study tours, resulted in the PIU’s transition from one staffed fully by international consultants to one with a majority of South Sudanese staff. Moreover, working closely with international NGOs, a United Nations agency and a commercial bank opened a mutual opportunity for the TST and these agencies to learn new skills and build capacity on how to work with communities and implement public works in rural, remote, and conflict-affected parts of the country, for future similar interventions. Mobilizing Private Sector Financing 49. The Project did not mobilize private sector financing. Poverty Reduction and Shared Prosperity 50. The Project’s primary target beneficiaries were poor and vulnerable households with limited or no income and employment opportunities. It applied robust criteria for geographic, administrative, and a participatory, inclusive and transparent community-based targeting process to identify beneficiary households for the Public Works Component. 36 See for example: del Ninno, C., K. Subbarao and A. Milazzo. 2009. How to Make Public Works Work: A Review of the Experiences. 37 During the ICR mission in February 2019. 38 Particularly the Appeals Committees, to the extent was possible considering social and cultural norms. 39 For example, in one location, there was a complaint from a female beneficiaries’ husband regarding his wife’s access to money through her participation in the work. This complaint was satisfactorily resolved by the Appeals Committee through a constructive dialogue. Page 18 of 67 The World Bank Safety Net and Skills Development (P143915) 51. A micro-simulation exercise of the Project’s poverty impact was carried out for this ICR, following the same methodology used for the simulations done at appraisal.40 The simulation41 found that given the extremely high poverty rates in South Sudan, even a relatively small transfer could have a non-negligible impact on poverty rates. Despite minor differences in daily rates among states42, the impact of the wage transfer was relatively uniform across the three states, ranging from a reduction of 0.08 percentage points in Central Equatoria to 0.1 percent in Warrap. The reductions in the poverty gap were also similar, ranging from 0.001 to 0.003. The simulation also explored the potential poverty impacts if the same transfers were expanded to cover all poor households in Central Equatoria, Eastern Equatoria and Warrap. It showed that a full expansion of the program would, all else held equal, result in a reduction of final poverty rates ranging from seven to almost 20 percentage points among these three states. Details of this exercise are provided in Annex 4. 52. Nevertheless, meaningful changes in poverty from safety nets would require peace and stability. Even if the fiscal situation allows for the public works program to benefit all poor individuals (82 percent of South Sudanese or 10 million people were living under the international poverty line in 2016),43 shocks such as conflict outbreaks would pull down those just above the poverty line. For example, from 2015 to 2016 combined conflict shocks and near hyperinflationary conditions, increased the poverty headcount by 16 percentage points. In states covered by the South Sudan’s High Frequency Survey (HFS), a modest 10 percent consumption shock would risk pushing about 160,000 people into poverty. 44 Other Unintended Outcomes and Impacts 53. The rehabilitated community assets resulting from the public works supported increased service delivery and access to markets, particularly in urban areas. Testimonies from beneficiaries in Juba indicated that improved roads enabled water trucks to enter neighborhoods that were previously unpassable. These helped in reducing costs for business and households to purchase water. Rural areas also reported increased market activities, especially around payment times, as traders could more easily access rural communities and were more assured of commerce from the cash transfers. Also, the opening of ditches and drains contributed to fewer illnesses due to improved sanitation in the communities, namely malaria that was caused by mosquitos breeding on stagnant water. 54. Trainings on Water, Sanitation and Hygiene (WASH) and financial literacy were not part of the original design but were undertaken in Juba City as a pilot with positive results. This was an initiative taken by the Juba IP, in consultation with the PIU, in response to interests by beneficiaries. Training participants reported improved awareness of hygiene practices (i.e. ensuring cleanliness when breastfeeding), as well as of the importance of saving. With the skills learned through the financial literacy trainings, some of them reported engaging in small-scale income generating activities.45 These outcomes may contribute to better human capital outcomes, leading to longer term economic growth through reductions in disease incidents and increased economic activities and savings, lack of which impose large constraints on the long-term economic growth in South Sudan. These trainings were not implemented by UNOPS in the other six project 40 Results for the ex-post simulation is based on the actual number of reported beneficiaries, wage rates and hours worked, while the results at appraisal calculated the poverty impact as if the program were being implemented so as to cover the entire population, statewide. 41 Carried out for three states where data is available. 42 On average, beneficiaries in Central Equatoria worked 74.66 days at US$3 a day. In Eastern Equatoria the corresponding numbers were 72.43 days at either US$3 per day (Torit county) or US$2.4 per day (Kapoeta East county). In Warrap the average beneficiary worked for 80.23 days at a rate of US$2.4 per day. 43 The total cost of the program per year would be approximately US$145 million, that is 24 percent of the Government’s overall 2018 budget or 6 percent of 2016 GDP. 44 https://blogs.worldbank.org/africacan/how-conflict-and-economic-crises-exacerbate-poverty-in-south-sudan? 45 Discussion with beneficiaries in Juba during the ICR mission in February 2019. Page 19 of 67 The World Bank Safety Net and Skills Development (P143915) locations. Nevertheless, UNOPS staff reported organic engagement in savings and group-based businesses by the public works participants. 55. Engagement in public works was reported to provide beneficiaries with a platform for greater interaction and dialogue among the diverse ethnic groups, contributing to strengthened community cohesion and social unity. Nine out of ten of the respondents of the independent evaluation in Juba felt that the Project united people of different ethnic groups. Although systemic evaluations were not undertaken in the other locations, anecdotal evidence reports the same. In environments such as that in South Sudan where distrust among different communities is a key factor in local violence, this increased collaboration and trust among communities is foundational towards reinforcing peace dynamics. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 56. The SNSDP, the first World Bank-financed SP project in South Sudan, was prepared in less than one year. Given the poverty and vulnerability of the population, there was an urgent need to provide direct support to the poorest and most vulnerable households, while putting in place policy, operational and governance elements of a national safety net system in the medium and longer term. The World Bank team worked closely with the Government and partners in preparing an operation that would address poverty and provide temporary employment opportunities, especially to women and youth46, and also support needed investments in community assets. 57. The Project’s design incorporated lessons learned from safety net type interventions in the humanitarian sector and other related projects in South Sudan. The experience and lessons from on-going safety net type interventions at the time of SNSDP preparation highlighted: (a) the need to establish a robust and coherent SP system, supported by a national SP policy, operational tools and governance mechanism for more predictable, reliable, coordinated, and longer term safety net support; (b) the value of using inclusive and transparent community-based targeting when poverty data is not available and administrative capacity is low; (c) the need to complement technical training with life and entrepreneurial skills training for effective skills development targeted at the poor; and (d) the need for functional and effective delivery tools to track implementation progress and impact of interventions, and to mitigate error, fraud and corruption risks at different levels of implementation. 58. Implementation experience of the South Sudan Emergency Food Crisis Response Project (EFCRP; P113586) indicated that twinning government staff with PIU consultants is effective in strengthening institutional capacity in the long run. Moreover, the EFCRP and the Adolescent Girls Initiative (AGI) demonstrated that Government-led coordination of NGOs contracted for project implementation produced positive results. The AGI further highlighted the need to provide start-up grants along with skills development to poor youth and women given that the private sector is nascent in the country, employment opportunities are limited, and the poor lack access to funds from banks or micro finance agencies. Lastly, the Project also learned from the Local Governance and Service Delivery Project (LGSDP; P127079) and incorporated its design principles on community-based delivery mechanisms that had already been field tested. The Project and the LGSDP were meant to be mutually reinforcing and informed by each other’s implementation experience. 59. The Project’s implementation arrangements requiring a multi-sectoral approach involving various Government ministries at different levels, IPs and the private sector were especially effective, but in retrospect, the Project’s design 46 Youth targeting was meant to be under the Skills Development component. Page 20 of 67 The World Bank Safety Net and Skills Development (P143915) was ambitious in view of the extremely weak institutional capacity and difficult operating environment . The implementation arrangements were well designed, involved the responsible agencies at the various levels of government, and were designed to capitalize on the knowledge and support that could best be provided by IPs. They were sufficiently flexible so that they could be adapted to implementation challenges when these arose (i.e., reducing the number of IPs). The technical assistance that was planned to support these agencies was also well defined and appropriate, and therefore was well utilized. The implementation arrangements, coupled with the technical assistance provided by the Project, helped put in place a process that not only benefited project implementation, but could also potentially support implementation of other community driven development-type (and other) projects in the future. Nevertheless, the Project’s original design, with a policy and system building component and two distinct types of interventions (public works and skills development), was ambitious considering the volatile operating environment and the fact that implementing capacity was only just being developed under the Project. Because of this, the Project required a timely restructuring, eliminating one of the originally envisaged interventions, for it to successfully achieve its objectives. 60. The Project’s overall risk rating was correctly identified as High. The Project was implemented in a difficult environment, and thus experienced inevitable delays, particularly due to the 2016 violence. While there were few factors that could mitigate these risks, the Project’s design was kept simple and well-structured to address identified risks. B. KEY FACTORS DURING IMPLEMENTATION 61. Delayed project effectiveness was the first challenge the SNSDP faced. It took 17 months after approval by the World Bank’s Board of Directors for the Credit to become effective.47 Generally, the Government requires three government bodies to review and approve projects financed by development partners before the signing of financing agreement, namely the: Inter-Ministerial Approval Committee (IMAC), Council of Ministers, and the Parliament. Thus, a lengthy signing and effectiveness process could have been anticipated. However, this was exacerbated by a Government reshuffle and the conflict that broke out in December 2013 soon after project approval. The team then had to acquaint the new counterparts with the Project and to ensure their ownership and commitment. 62. The Project faced a very challenging environment throughout its implementation. The political economy was exceptionally challenging, including high levels of violence, repeated cycles of conflict, and inaccessibility outside Juba. In addition, the Government had limited implementation capacity, weak fiduciary accountability and risk of elite capture of project benefits, although no fiduciary breaches were uncovered during project implementation and financial audits were unqualified. When implementation finally began, priority was given to implementing public works subprojects in stable and secure areas to allow for rapid implementation and safety of project staff, with implementation starting in two of the seven locations. It was expected that the political, economic and security situations would gradually improve to enable implementation in the remaining counties. However, further implementation delays occurred due to the outbreak of conflict in July 2016, and all project implementation was paused accordingly, with the understanding that implementation would resume once the security and implementation conditions were more favorable. Consequently, the PIU was scaled down, strengthened fiduciary and operational measures were established, and Project procurement was adjusted to require prior review for all project activities (see more in the Fiduciary Section). 63. Following the conflict, many organizations, including one of the two IPs, left the country. This decreased the availability of IPs when project implementation resumed, posing potential delays in on-boarding new IPs for scaling up Project activities in the six locations outside Juba that lacked an IP. The United Nations Office for Project Services (UNOPS) 47 SNSDP was approved by the Board in June 2013 and became effective in November 2014. Page 21 of 67 The World Bank Safety Net and Skills Development (P143915) was selected to implement the public works component in the six counties outside Juba given its nation-wide access, ability to scale up rapidly and provide economies of scale, and ensure fiduciary accountability. This arrangement provided higher reassurance of implementation, especially as the World Bank’s movement was limited to inside Juba due to security concerns. 64. Figure 2 illustrates the planned implementation timeline, the periods of conflict mentioned and the actual implementation timeline and the respective ISR ratings. For example, due to the extremely difficult implementation conditions on the ground post July 2016 conflict, the key project ratings, including DO and IP ratings were downgraded to Unsatisfactory. Figure 2. Actual vs. Planned Implementation, and respective ISR ratings Source: original for ICR 65. The macroeconomic situation deteriorated significantly as a result of the conflict. By August 2016, South Sudan displayed all the signs of macroeconomic collapse, with output contracting, and inflation and parallel exchange rate market premium spiraling. Gross Domestic Product estimated to have contracted by about 11 percent in FY2016/17 due to conflict, oil production disruptions and below-average agricultural production. On the demand side, exports and household consumption declined, while government consumption increased due to spending on defense and security operations. Deficits associated with the war and security led the Government to cover its costs by printing money, driving up inflation—the Consumer Price Index increased by 480 percent in 2016 and by 155 percent in July 2016-June 2017. The SSP depreciated from SSP18.5 per US$1 in 2015 to SSP172 per US$1 by August 2017. The rapid currency fluctuation made pegging public works wages to an absolute SSP amount difficult as the transfer amounts to the beneficiaries had to be constantly changed to make them equivalent to US$3 (Section IV B). 66. Poverty soared over the Project’s life, increasing the relevance of the cash transfers provided through the public works activities. Poverty rates soared from about 50.6 percent in 2013 to 70 percent in 2016 and 82 percent in 2017, and to an estimated 88.7 percent in 2018. The macroeconomic collapse, high inflation and increasing food prices led to a sharp increase in poverty, particularly in urban areas. Inequality amongst the poor also worsened and the poverty severity index doubled from 2015 to 2016. Consequently, estimates indicate that real incomes have fallen nearly 70 percent since the onset of the conflict in 2016. Many South Sudanese have lost a large share of their income- generating assets and now rely overwhelmingly on aid. To address this reality, the Government requested that the project restructuring concentrate activities on and increase resources to the public works component. 67. The implementation of public works in the six counties outside Juba was significantly more challenging than in Juba. Rural areas are more remote and difficult to access, lack basic infrastructure, and have limited community capacity and structures. For example, deliveries of input tools in rural areas are highly difficult seasonally due to the lack of access during the heavy rainy season. Also, social and security risks in these locations tended to be higher, particularly following the July 2016 conflict. Greater community mobilization and sensitization efforts were therefore needed in Page 22 of 67 The World Bank Safety Net and Skills Development (P143915) these areas to deepen understanding of development activities and build ownership of and commitment to the Project. 68. Increased risks of implementation in rural areas were demonstrated by isolated incidents of violence in Kapoeta East, which led to the suspension of public works activities in two bomas. Extreme levels of poverty, food insecurity and illiteracy, the nomadic nature of certain project communities, mistrust of development support resulting from past negative experiences, as well as implementation challenges related to delays in payments and crowd management during payments were only some of the factors that created a difficult environment for project implementation in remote rural areas. Specifically, these challenges led to three incidents of violence against UNOPS staff in Kapoeta East,48 which were resolved to the satisfaction of the MAFS and the World Bank through actions taken by the established Appeals Committees of the GRMs in the relevant bomas, with the support of the local structures (i.e. PDCs/BDCs) and the Kapoeta East County Government. The PIU, UNOPS, Kapoeta East County Government and the World Bank agreed jointly on the need for remedial actions, including a zero-tolerance policy towards violence. Activities in the two bomas were permanently stopped, as violence of staff of different donor projects had become recurrent, while implementation resumed in the third boma where the conditions were deemed more favorable. In addition, as preventative measures, a letter from the MAFS emphasizing the zero-tolerance policy and possible suspension of activities if violated was sent to all the Project County Governments, and community sensitization and mobilization was further strengthened across all project locations. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 69. The Project’s RF included relevant indicators across the three originally designed components and the two project development objectives. The PIU together with relevant ministries’ staff were primarily responsible for regular monitoring of activities and outputs, supported by the IPs. Each of the indicators had clear frequency, data source and target and baseline values. Three indicators related to the skills development component were deleted following the first project restructuring. The planned M&E methods and tools were: (a) the MIS; (b) an independent beneficiary satisfaction survey, with particular focus on targeting and transfer mechanisms; (c) community score cards on the Project’s performance; (d) an independent FM audit; and (d) and independent evaluations by the IPs. M&E Implementation 70. There was systematic updating of the Project’s RF through the higher than usual numbers of World Bank supervision missions and regular video conferences, though with some limitations. Regular data collection and reporting were mainly coordinated and conducted by the PIU, which undertook implementation support and monitoring missions to the counties. The PIU recruited an M&E Specialist and a private IT firm to design the MIS. However, the quality of the M&E implementation suffered as the M&E Specialist left early on and the PIU was unable to recruit another full time M&E professional.49 IPs conducted regular monitoring, through their field teams, and submitted reports to the 48 In summary: (a) detainment of UNOPS team leader for less than 30 minutes by a few youths who were not beneficiaries; (b) slapping of UNOPS community mobilizer by a beneficiary incited by rumors; and (c) detainment and beating of UNOPS team leader by a police officer due to missing beneficiaries’ names from the payroll during payment. 49 An acting arrangement was put in place. The Torit County Coordinator became the M&E focal point as his responsibility reduced due to UNOPS Page 23 of 67 The World Bank Safety Net and Skills Development (P143915) PIU for quality assurance. An innovation in M&E piloted in Juba was the GIS-enabled biometric registration and cash transfer mechanism. Cash transfer data were collected and compiled by the commercial bank contracted by the Project. An independent evaluation for public works implementation in Juba city was commissioned by Action Africa Help- International (AAH-I), however the same approach could not be implemented in the other six counties, as implementation by UNOPS was completed too close to the project’s closing date. Additionally, an independent beneficiary satisfaction verification exercise, with particular focus on targeting and transfer mechanisms, was conducted using community score cards method, only one-time toward the end of project implementation instead of being an annual exercise, as per the M&E design. Also, at project closing, the Government submitted its Project Completion Report and Borrower’s ICR contribution (Annex 5). M&E Utilization 71. Throughout implementation, various M&E tools provided data and information for monitoring the Project’s results and implementation progress. During each World Bank supervision mission, progress was tracked through the RF and actions were agreed to improve the lagging indicators. The MIS was a key source of data, though at times it needed to be complemented with other sources of data for accuracy and analysis, such as the progress report from the IPs. The absence of a full-time M&E Specialist in the PIU limited the richness of the analyses and use of the various data collected. Implementation of public works in the rural and remote locations benefited from emerging lessons from the early implementation experiences in Juba and the Project’s Mid-Term Review. The beneficiary feedback survey was completed the same month as the project closing, and thus, the recommendations would mainly be useful for future operations. Justification of Overall Rating of Quality of M&E 72. The overall rating for quality of M&E is Modest. The design of Project’s M&E framework was robust but there were shortcomings in M&E implementation and utilization. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 73. Environmental and social safeguards. The Project was categorized under environmental category B given that the proposed public works activities were low-scale and localized. Relevant environmental and social safeguard policies50 were triggered at appraisal and no waivers of the World Bank’s safeguards policies were required. OP 4.12 (Involuntary Resettlement) was not triggered, as it was expected that land needed for public works would be provided voluntarily. The World Bank portfolio-wide Environmental and Social Screening and Assessment Framework (ESSAF) containing broad parameters of requirements, processes and responsibilities informed the Project design, in lieu of the traditional project- specific safeguards instruments. Based on the ESSAF, an Environmental and Social Management Framework (ESMF), that included an Integrated Pest Management Plan, was prepared. Further, Environmental and Social Management Plans (ESMPs) were also prepared and adopted based on the ESMF once public work activities were selected to ensure that all public works activities were carried out in accordance therewith. 74. From an environmental safeguards perspective, the Project was a good example of a social safety net project in full implementation role in Torit County after the restructuring. 50 OP/BP 4.01: Environmental Assessment, OP/BP 4.09: Pest Management, OP/BP 7.50: Projects on International Waterways, and OP/BP 4.09: Indigenous People. On OP/BP 7.50, subsequently, the task team obtained approval for exception to riparian notification from the Africa Vice President because the proposed activities would be limited to the improvement or rehabilitation of existing water delivery schemes. Page 24 of 67 The World Bank Safety Net and Skills Development (P143915) terms of design, implementation, monitoring, and reporting of site-specific environmental management plans. It was also a good practice example in terms of clarity and thoroughness in rejecting unsuitable subprojects. The Project articulated well the justification for certain subprojects, and the operation and maintenance requirements to ensure their sustainability. Implementation and monitoring of safeguards measures were facilitated by an international safeguard consultancy firm hired by the PIU to support three World Bank-financed projects in the MAFS, including SNSDP, as the government lacked the necessary environmental safeguards skills. The firm seconded two international safeguards consultants to the PIU who supervised the SNSDP safeguards implementation, provided training and mentoring and worked with environmental and social safeguards officers at the IPs. Social safeguards issues were mainly managed through the GRM, as described in Efficacy Section under Objective 2. The incidents of violence against UNOPS staff in one of the project locations, though not anticipated in the risk assessment at project preparation, were resolved accordingly as discussed in Section III.B para. 68. 75. The World Bank provided regular supervision of compliance with the applicable safeguards policies and the Government produced a safeguards closure report. The report outlined challenges faced during implementation, particularly on social safeguards issues, including: (i) lack of a community mobilization strategy and tailored communication materials; (ii) no specific training and sensitization of GBV of IP staff prior to implementation; (iii) limited PIU resources to ensure regular M&E of safeguards across project locations; and (iv) unsystematic management of voluntary land donation. The Project’s lessons learned were key in the development of safeguards instruments for a future social safety net operation in South Sudan. 76. Fiduciary. The PIU hired a Finance Specialist and a Procurement Officer to support the three World Bank-financed projects in the MAFS, including the SNSDP. Following the outbreak of violence in 2016, additional fiduciary risk mitigation measures were adopted by the Project to mitigate against heightened fiduciary and safeguards risks, ensure accountable and effective delivery mechanisms, and enhance monitoring and evaluation of implementation progress. These included: (a) direct payments to service providers; (b) switching from report-based disbursement system to the transaction-based Statement of Expenditure disbursement system; and (c) World Bank prior review for all project procurement requests regardless of the amount involved. In addition, the PIU was also asked to provide the following: (i) report on actual cash balances in DA, supported by certified copies of bank statements, (ii) contract register for all ongoing firms and individual contracts, (iii) expenditure plan and cash forecast for next 2 months, and (iv) delay all procurement activities initiated that were not yet concluded, with exceptions of contract renewal with PIU consultants. Further, the currency devaluation made pegging public works wages to the SSP difficult as the amounts transferred kept having to be changed to make them equivalent to US$3. As a result, IP contracts had to be amended constantly as they included the cash transfer amount in SSP. After restructuring, these terms were modified to transfer US$3 equivalent in SSP based on the prevailing exchange rate at time of transfer. 77. To respond to the need for greater assurance and efficiency in verifying tens of thousands of payments transactions in a high-risk environment, the project team was proactive in rolling-out biometric registration. Prior to the use of biometric data, verification was done manually requiring intensive time and effort, and enhanced the risk of human error, as well as fraud and corruption. The Project however experienced challenges of data mismatch between the biometric system and the MIS due to lack of full integration of the two systems. To ensure appropriate ownership, custody and security of beneficiary data collected as part of the project, legal advice was sought towards the end of the project to inform future iterations of the MIS. 78. Throughout implementation, FM and Procurement performances were rated as Moderately Satisfactory, except in the initial period due to the effectiveness delay and in the aftermath of the 2016 clashes. During these periods, ratings Page 25 of 67 The World Bank Safety Net and Skills Development (P143915) were downgraded to Moderately Unsatisfactory,51 since the Project was not being implemented, and Credit funds were not being disbursed. Given the heightened insecurity and the continued volatility of the exchange rate, the fiduciary risk rating was increased from Moderate to High. No waivers were required of the World Bank’s fiduciary policies, no ineligible expenditures were presented, interim unaudited financial reports were presented in a timely fashion and the final annual audit report for the period ended March 31, 2019 provided an unqualified/clean opinion, with few issues highlighted in the management letters.52 C. BANK PERFORMANCE Quality at Entry 79. Quality at Entry is rated Moderately Satisfactory. The preparation team helped to design a project that responded to the Government’s needs and considered lessons learned from other related World Bank-financed projects in the SP sector and in South Sudan. Preparation built upon the design of the LGSDP that had been field tested, which helped to reduce the Project’s preparation schedule – it was prepared in less than one year – and to standardize implementation approaches across projects. While the preparation team aimed to ensure that the Project was relatively simple to implement and monitor, its design was nonetheless ambitious in view of South Sudan’s extremely weak institutional environment. The support to establishing the building blocks of a SP system was carefully considered, with objectives that could reasonably be implemented from scratch. The fact that the Project required the September 2017 restructuring was precipitated by escalating immediate needs and rapidly changing dynamics on the ground following the July 2016 violence. While not attributable to preparation quality, per se, the need for this restructuring reflects the preparation team’s over estimation of the Government’s implementation capacity. Quality of Supervision 80. The Quality of the World Bank’s Supervision was Satisfactory. To say that supervision of this project was challenging would be an understatement. Security issues precluded field visits in the six counties outside Juba, and even resulted in the temporary relocations of World Bank staff from the Country Office from December 2013 to May 2014 and from July 2016 to April 2017. Given that full implementation only got underway after the operating environment was more favorable in May 2017 and the Project was restructured shortly thereafter, the Mid-Term Review was delayed until January 2018. 81. The Project’s restructuring was timely and well prepared. The restructuring was well prepared and responded to the country’s needs, as requested by the Government. Through this restructuring, the Project’s design emphasized simplicity, a characteristic that was especially important given the volatile environment in which it was being implemented. It reduced the Project’s scope to that which could be reasonably accomplished during its implementation thereby assuring that it would be able to achieve its objectives, as measured by its key indicators. 82. The supervision team produced twelve ISRs (at least one for each semester of project implementation) and carried out supervision missions in line with World Bank guidelines, when feasible. In fact, after the first restructuring supervision was much more frequent (i.e. about every two months) to ensure successful completion of all activities by project closing. Despite the challenges of supervision by World Bank team outside Juba, the fact that UNOPS 51In 2017, disbursement ratio was about 2 percent before picking up steadily until 98 percent at project closing in February 2019. 52These include weaknesses in the process of handing over assets, delayed implementation of previous audit recommendations, failure to maintain a comprehensive risk management strategy and continued reliance on manual accounting system for project financial reporting purposes. Page 26 of 67 The World Bank Safety Net and Skills Development (P143915) implemented the public works activities in areas outside Juba was reassuring. UNOPS is a trusted UN partner with proven capacity to reach and implement effectively across the nation in difficult and remote areas. Challenges in field supervision prompted the team to find innovative ways of ensuring close supervision. Specifically, the team held regular bi-weekly video conference meetings with the PIU, the IPs and the commercial bank to identify and resolve issues promptly. Presence of one of the Task Team Leaders in Nairobi further facilitated close coordination and supervision, and daily implementation support to the PIU and the IPs, supported by key Country Office staff. Supervision missions comprised the necessary team members, including fiduciary and safeguards specialists, and findings and agreements of the missions were detailed in quality Aide-Memoires and management letters. Supervision was proactive and provided realistic feedback to management, as well as guidance to counterparts. Project reporting was thorough, candid, and ratings in ISRs were mostly adjusted to reflect challenges and progress in implementation, except before the conflict in 2016 when the team was slow in downgrading project ratings to reflect growing concerns at the time. During the ICR mission, the PIU expressed appreciation for the strong implementation support provided by the World Bank team, as also recorded in the Borrower’s ICR. 83. The World Bank’s implementation support benefited from continuity in Task Management. The World Bank Task Team Leaders (Co-TTLs) and core members remained the same until project completion. This continuity from preparation through completion was helpful in providing effective implementation support. Justification of Overall Rating of Bank Performance 84. The Overall Rating of World Bank Performance is Moderately Satisfactory in view of the Moderately Satisfactory rating for Quality at Entry and the Satisfactory rating for Supervision, and determined by the Project’s Overall Outcome rating (as required by the ICR Guidelines in the event of split ratings for World Bank performance).53 D. RISK TO DEVELOPMENT OUTCOME 85. The Risk to Development Outcome is High. Risk to Development Outcome is evaluated from two angles. First, the risk that the works financed by the Project will not be maintained, and second, the risk that the Social Protection Policy Framework supported by the Project cannot continue to be operationalized, further expanded and developed. 86. The risk that works financed by the Project will not be maintained is mitigated by the high ownership of the subprojects by the communities, as they were identified as priorities by them. Aligned with global experiences in public works,54 the Government’s Project Completion Report indicated that several of the works put in place under the Project were being maintained by the communities even after completion. In addition, the Government’s report and the ICR mission demonstrated that community members organized themselves and maintained assets in their neighborhood voluntarily after closure of the public works activities without any additional funding. However, these positive developments rely on communities’ own proactivity and motivation as there was no systematic enforcement operations and maintenance plans or monitoring of sub-projects after the works were completed. Given the deteriorating economic conditions, members are increasingly forced to devote all their time to finding livelihood options, and may not have the time or the capacity to work to maintain community assets without pay, as the opportunity cost would be too high. 87. The possibility that the Social Protection Policy Framework supported by the Project will continue to be 53World Bank ICR Guidelines (September 2018), page 23. 54 Community involvement in project selection creates ownership of the asset created and may lead to better maintenance of the assets (del Nino, C., K. Subbarao and A. Milazzo. 2009. How to Make Public Works Work: A Review of the Experiences). Page 27 of 67 The World Bank Safety Net and Skills Development (P143915) operationalized, expanded and developed is threatened by several factors. Key among these are: (a) uncertain prospects for durable peace; (b) further deterioration of the macroeconomic conditions and the Government’s highly constrained fiscal conditions; and (c) risk of interrupted international assistance. As such, efforts to maintain and expand SP initiatives would depend more on continued peace55 and the availability of financing from donors. Admittedly the public works program is only one of several different programs that could be used to fight poverty, and its impacts go beyond just reducing income poverty at the beneficiary household level. In the best-case scenario, public works should be used in conjunction with other programs to reduce poverty, as well as more selectively to: (a) deal with income and livelihood shocks. e.g., in areas affected by drought, (b) improve community assets in a cost-effective manner, and (c) help build capacity in remote areas. Even in the best of macroeconomic scenarios, the Government is simply not in a position to scale up the public works program without continued external assistance (Annex 4). Nonetheless, the policy framework, institutional arrangements and administrative tools established serve as a foundation of a national safety net system, and could continue supporting the delivery of public works programs and other future SP initiatives by the World Bank and South Sudan’s other partners. These efforts would, in turn, need to be expanded and accompanied by complementary efforts aimed at human capital accumulation through initiatives in education and health to ensure that the livelihoods of this and future generations are improved on a sustainable basis, and not be dependent upon continued social safety net assistance. 88. Despite the internal conflict and deteriorating macroeconomic situation, the Government has sustained its commitment to social protection as an important part of its strategy to fight poverty and promote resilience. Its SP policy framework, approved in 2014 with Project support, the National Social Protection Policy Framework (NSPPF) was developed to “respond to and address the multiple vulnerabilities faced by South Sudanese citizens, with a particular focus on the poorest and most excluded sectors”, The recently-approved National Development Strategy 2018-2021, “Consolidate Peace and Stabilize the Economy”, contains six strategic deliverables several of which address the priorities supported by the Project, including: (a) create enabling conditions for and facilitate the voluntary return and integration of displaced South Sudanese; (b) ensure secure access to adequate food, and (c) restore and expand the provision of basic services. 89. A proposed US$40 million IDA Credit for a South Sudan Safety Net Project (SSSNP), currently under preparation, aims to address urgent needs through a scale-up of cash transfers, while also rehabilitating community assets and strengthening local level capacity and safety net delivery tools, thereby contributing to humanitarian- development linkages and longer-term development. The SSSNP builds upon the lessons learned under the Project such that in addition to providing cash support to poor and vulnerable households through public works activities, it would also provide direct income support to selected households who lack able-bodied members to participate in public works. Further, it would strengthen safety net delivery tools, especially the GRM, the MIS, the beneficiary payment system and community engagement and outreach. V. LESSONS AND RECOMMENDATIONS 90. This Project offers numerous lessons that may be relevant for future operations in South Sudan as well as in other countries implementing investment projects in an FCV environment. The most important of these lessons are 55 The Revitalized Agreement on the Resolution of the Conflict in South Sudan (R-ARCISS), signed by the Government, the opposition and civil society on September 12, 2018 in Addis Ababa is seen as a fresh opportunity for South Sudanese to build durable peace in the country. The main challenge going forward is to ensure all parties to the conflict remain committed to implement the new peace agreement. For more, see Annex 4, under financial sustainability. Page 28 of 67 The World Bank Safety Net and Skills Development (P143915) summarized below; the full set of lessons from the Project’s design and implementation are presented in Annex 8. 91. Implementation of development projects in an FCV context needs to be flexible in the midst of volatile conflict dynamics. This is critical in ensuring that project interventions actively support efforts aimed at restoring peace, establishing stability and promoting medium- to longer-term development. Project design needs to be simple and flexible, not only to be quick in adjusting to realities, capacity/resource constraints, and risks on the ground, but also to recognize that anything can change at a moment’s notice. SNSDP went through a timely project restructuring to remove a stalling component and focus its efforts to what was feasible and considered to be most beneficial given the conflict and economic volatility that started in July 2016. 92. The use of third parties as IPs can be very effective and supports the principle of doing no harm in delivering development projects in conflict settings, especially in FCV situations where Government capacity and systems are nascent, and access to project locations are limited. The engagement of NGOs and UNOPS as IPs, supported by a commercial bank as the payment service provider, augmented the capacity gap in the MAFS, enabled access to remote and insecure areas, ensured transparency and accountability, mitigated high fiduciary risks, assured neutrality of service delivery and distribution of resources, and earned communities’ trust in the Project. The risk of potential manipulation of the targeting process and elite capture of project benefits were high, but no significant irregularities were uncovered during the review process. The payment procedure in SNSDP involved three parties and the World Bank, allowing for rigorous checks and balances, consisting of: (a) generation of payroll by IPs (AAH-I and UNOPS); (b) review by the PIU; (c) actual payment to the beneficiaries by the commercial bank; and (d) ex-post review by the World Bank. 93. Having one or more independent agencies monitor project operational processes and implementation progress is important in fragile contexts with limited access by the World Bank. The SNSDP had not envisioned the need for an independent third-party monitor during preparation, but when the security context deteriorated, priority was given to transferring as much of the available funding as possible to the beneficiaries, as UNOPS systems and capacities were assessed to be adequate in ensuring delivery of results and fiduciary accountability. While reporting by the IPs were verified by the PIU through monitoring missions to project locations, a third-party monitor in future operations would provide a robust check on reports by the IPs and the PIU, as well as identify areas where inconsistencies over approaches or measurement techniques are evident so that additional capacity building or other adjustments can be made quickly. 94. When the basics of a social protection system is absent, and poverty and need is widespread, a project does not need to be “state-of-the-art” to produce results. Perfecting targeting system is not required where poverty56 is prevalent, as is the case in South Sudan. In a context with near universal poverty and lack of disaggregated data, such “state-of-the-art” targeting methodologies may not be practical, and could in fact be counterproductive as it could delay delivery of safety net assistance and exacerbate grievances within the selected community, thereby hurting the legitimacy of the program. Rather, it is more critical to establish a targeting mechanism that is accepted by beneficiaries and stakeholders as fair and transparent and that ensures that inclusion errors are minimized as much as possible. Given that there will likely be a large segment of the target population left out due to funding constraints, the reasons for exclusion need to be clear from the beginning, and every effort should be taken to minimize political/elite-based manipulation, including through sensitization, public verification, and GRM. 56The Project did not make a distinction between poor and vulnerable households at design as they were both seen as potential beneficiaries as poverty is widespread in South Sudan. Nevertheless, future safety net operations, especially when data is available, may consider avoiding using the term poor and vulnerable interchangeably and clearly defining whether the target groups are the poor, the vulnerable (typically those above the poor and closer to the poverty line), or both. Page 29 of 67 The World Bank Safety Net and Skills Development (P143915) 95. Mitigation of social risks and the GRM. The community-based GRM structure encountered limitations, particularly in its capacity to mitigate against social risks. Key among these included infrequent, incomplete, and inconsistent reporting of complaints received, which limited timely identification and implementation of issues and mitigation measures. Weak capacity and motivation by GRM Appeals Committee members resulted in poor understanding and limited functionality of the GRM. Also, social norms around GBV made it difficult to openly discuss these issues within a community and likely resulted in underreporting. The lack of referral mechanisms and complementary support made it difficult to address the few GBV cases that were reported. Follow-up operations should consider strengthening such GRM platform, with attention paid to its functionalities in addressing GBV incidents, closely tied to deepened community engagement, citizen participation and capacity building of local structures. 96. Fully functional MIS and biometric tools. The SNSDP MIS and biometric tools were important in supporting effective beneficiary management and payment, as they supported demographic and biometric data collection, storage, and retrieval. Further, these databases constitute an important step towards a beneficiary registry that can be used as a national registry in the future. However, they suffered from weaknesses, particularly in terms of operations, data mismatches, business continuity disaster recovery and reporting capacities. Data protection and security was of particular concern and internal checks and controls were inadequate. Lack of offline functionality, given poor or no Internet connection, also was a major challenge. Future operations that would support the MIS and biometric tools should anticipate the data privacy and security aspects by engaging legal experts early on in project preparation. 97. In-depth community mobilization and sensitization, together with continuous communication with beneficiaries and dissemination of the program, are imperative to ensure buy-in and minimize grievances, especially in an environment characterized by weak capacities and high risks due to fragmented communities with high levels of mistrust. For communities to engage effectively however, concerted efforts should be provided to strengthen their capacity. Local level engagement requires time and up-front planning, especially in communities where capacities are very limited. For example, under the SNSDP, there was no capacity or understanding in many communities of how community- based decision-making structures work. If some form of indigenous systems and structures do exist, it is critical to strengthen them instead of replacing them new with project-specific ones. 98. Additional lessons and recommendations, described in detail in Annex 8, include: • A project design that empowers women and supports them to address their specific needs and challenges is an effective way to promote gender equity and inclusion, but project design needs to incorporate measures to mitigate potential GBV that may result; • Complementary trainings such as basic financial literacy, early childhood development messaging, WASH and training against GBV have contributed to women’s empowerment; and • The Government must gradually take ownership over safety net assistance such that support is provided through Government-led national systems with full government ownership. . Page 30 of 67 The World Bank Safety Net and Skills Development (P143915) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Provide access to income opportunities and temporary employment Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 25000.00 34000.00 53163.00 10-Nov-2014 31-Dec-2017 31-Dec-2018 28-Feb-2019 Female beneficiaries Percentage 0.00 30.00 50.00 74.00 31-Dec-2017 31-Dec-2018 Comments (achievements against targets): Target Exceeded. On the direct project beneficiaries, the last ISR used data from the PIU as of February 22, 2019. Since then, the PIU conducted further data analysis and clarified that 53,290 beneficiaries were registered for the public works project, however only 53,163 were engaged in the actual works with the difference being duplications, pastoralists who moved to another area and stopped working after only few days and did not come for payments. On percentage of female beneficiaries, the last ISR averaged estimated percentage of female participation in each seven counties/municipality which resulted into 75.8 percent; for the ICR, exact numbers of female participants were available and the correct percentage is 74.0 as recorded here. Page 31 of 67 The World Bank Safety Net and Skills Development (P143915) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Beneficiaries of public works Percentage 0.00 85.00 85.00 80.00 complying with eligibility criteria 10-Nov-2014 31-Dec-2017 31-Dec-2018 28-Feb-2019 Comments (achievements against targets): Target Partially Achieved. The measurement of this indicator deployed a proxy measurement through the result of an independent qualitative beneficiary satisfaction survey (conducted in November 2018 and report completed in February 2019) which found that 80 percent of those surveyed were “satisfied with the categories of households selected as these were deemed to be the most vulnerable.” Neverthe less, it is important to note that poverty was an important eligibility criterion (eight out of the nine targeting criteria contains poverty, see section II.B) and poverty was near universal (approximately 80 percent between 2016 and 2018). Objective/Outcome: Put in place the building blocks for a social protection system Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Social Protection Policy Text no social protection SP policy submitted Social protection developed and submitted for policy in place for discussion policy framework ( discussion which is SP policy for South Sudan) approved by council of Ministers and has been launched. Basic operational tools such as the MIS, GRM, Page 32 of 67 The World Bank Safety Net and Skills Development (P143915) targeting, and payment mechanism, as well as local level oversight and coordination structures, have also been established. 10-Nov-2014 31-Dec-2017 28-Feb-2019 Comments (achievements against targets): Target Achieved. The target had been met at the time of the project restructuring in September 2017. The last ISR included the following information “Basic operational tools such as the MIS, GRM, targeting, and payment mechanism, as well as local level oversight and coordination structures, have also been established” that speaks about the other building blocks for a social protection system. A.2 Intermediate Results Indicators Component: Social Protection System and Project Management Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion MIS systems for public works Text 0 MIS systems fully Management and skills training developed functional Information System and functional has been developed and is functional Page 33 of 67 The World Bank Safety Net and Skills Development (P143915) 10-Nov-2014 31-Dec-2017 28-Feb-2019 Comments (achievements against targets): Target Partially Achieved. By the closing date, the MIS (for public works only, because the skills training was not implemented) still required significant improvements in order to be considered as fully functional. The MIS independent assessment report (completed in January 2019) noted that one month before project closing date, the MIS was undergoing modification and/or changes to enhance its performance and make it more user-friendly. The report underscored issues in the MIS, including on data privacy and security, internal data quality checks, synchronization of the MIS with the biometric system, restricted access and audit trails, and lack of hardware, software and trained staff to take over the MIS system. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Cash transfers to Percentage 0.00 80.00 80.00 63.60 beneficiaries made on time 10-Nov-2014 31-Dec-2017 31-Dec-2018 28-Feb-2019 Comments (achievements against targets): Target Partially Achieved. The measurement was set as cash paid to beneficiaries fifteen days after a work cycle ended. In those fifteen days, the payment verification process took place by three parties: the implementing partners generated the payroll and input data in the MIS, then PIU performed data checking and sent payment instruction to the payment provider who performed another check and prepared the required logistics to get to the beneficiaries. This supposedly rigorous process was not always completed in fifteen days. Biometric registration and verification (only started in May 2018) helped, but there were also issues such as limited data download ability due to inadequate internet capacity and postponements during heavy rain period. Overall timeliness, as per recorded by the payment provider, was 63.6 percent. Page 34 of 67 The World Bank Safety Net and Skills Development (P143915) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Percentage of selected Percentage 0.00 90.00 100.00 100.00 Bomas that post list of approved beneficiaries in 10-Nov-2014 31-Dec-2017 31-Dec-2018 28-Feb-2019 public places Comments (achievements against targets): Target Achieved. The target had been met at the time of the project restructuring in September 2017. This indicator continued to be measured and in Bomas with high number of illiterate people (such as in Kapoeta East County), the list was also announced at a public meeting. Component: Public Works Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Beneficiaries of Safety Nets Number 0.00 21500.00 34000.00 53163.00 programs (number) 10-Nov-2014 31-Dec-2017 31-Dec-2018 28-Feb-2019 Beneficiaries of Safety Nets Number 0.00 6450.00 17000.00 40394.00 programs - Female (number) 10-Nov-2014 31-Dec-2017 31-Dec-2018 28-Feb-2019 Page 35 of 67 The World Bank Safety Net and Skills Development (P143915) Beneficiaries of Safety Nets Number 0.00 21500.00 34000.00 53163.00 programs - Cash-for-work, food-for-work and public 10-Nov-2014 31-Dec-2017 31-Dec-2018 28-Feb-2019 works (number) Comments (achievements against targets): Target Exceeded. On the direct project beneficiaries, the last ISR used data from the PIU as of February 22, 2019. Since then, the PIU conducted further data analysis and clarified that 53,290 beneficiaries were registered for the public works project, however only 53,163 were engaged in the actual works with the difference being duplications, pastoralists who moved to another area and stopped working after only few days and did not come for payments. On percentage of female beneficiaries, the last ISR averaged estimated percentage of female participation in each seven counties/municipality which resulted into 75.8 percent; for the ICR, exact numbers of female participants were available and the correct percentage is 74.0 as recorded here. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of work days Number 0.00 2190000.00 3600000.00 4032670.00 created 10-Nov-2014 31-Dec-2017 31-Dec-2018 28-Feb-2019 Comments (achievements against targets): Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 36 of 67 The World Bank Safety Net and Skills Development (P143915) Number of public works Text 0 TBD 150 517 projects 10-Nov-2014 31-Dec-2017 31-Dec-2018 28-Feb-2019 Comments (achievements against targets): Target Exceeded. The number used in the last ISR (510) was data from the PIU as of February 22, 2019. Since then, the PIU conducted further review and clarified that the total number of public works project was 517 as recorded here. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Beneficiary satisfaction with Percentage 0.00 80.00 80.00 52.67 program 10-Nov-2014 31-Dec-2017 31-Dec-2018 28-Feb-2019 Comments (achievements against targets): Target Partially Achieved. The measurement was set as average of aggregated percentage of beneficiaries that ranked very good, good and fair in the areas of targeting mechanism and selection, payment mechanism and timeliness and functioning of the oversight and coordination structures as per an independent qualitative beneficiary satisfaction survey (conducted in November 2018 and report completed in February 2019). The number used in the last ISR (73.70). Further analysis clarified that the satisfaction rate is 52.67 as recorded here. Page 37 of 67 The World Bank Safety Net and Skills Development (P143915) B. KEY OUTPUTS BY COMPONENT Provide Access to Income Opportunities and Temporary Employment to the Poor and Vulnerable 1. Direct project beneficiaries (number) 1.1. Female beneficiaries (percentage) Outcome Indicators 2. Beneficiaries of public works complying with eligibility criteria (percentage) 1. Beneficiaries of safety nets programs (number) 1.1 Beneficiaries of safety nets programs – female (number) 1.2 Beneficiaries of safety nets programs – cash-for-work, food-for- work and public works (number) 2. Number of work days created Intermediate Results Indicators 3. Number of public works projects 4. Beneficiary satisfaction with program (percentage) 5. Number of persons who complete the skills training program (disaggregated by youth and women)57 6. Number of evaluation and studies conducted on skills development58 1. Cash transferred to public works beneficiaries 2. Trainings on financial literacy and Water, Sanitation and Health Key Outputs by Component (WASH) delivered to public works beneficiaries (linked to the achievement of the Objective/Outcome 1) 3. Community infrastructure (road, drainage, public facilities, etc.) rehabilitated or maintained Put in Place Building Blocks for a Social Protection System Outcome Indicators 1. Social protection policy developed and submitted for discussion 57 This indicator was deleted in September 2017 project restructuring. 58 Ibid. Page 38 of 67 The World Bank Safety Net and Skills Development (P143915) 2. Targeting mechanism to select poor youth for skills development training designed, tested and finalized/revised59 1. MIS systems for public works and skills training developed and functional Intermediate Results Indicators 2. Cash transfers to beneficiaries made on time (percentage) 3. Selected bomas that post list of approved beneficiaries in public places (percentage) 1. Social protection policy 2. MIS system Key Outputs by Component 3. Payment system (linked to the achievement of the Objective/Outcome 2) 4. Targeting tool 5. List of beneficiaries 6. Appeals committees as platform for grievances 59 This indicator was deleted in September 2017 project restructuring. Page 39 of 67 The World Bank Safety Net and Skills Development (P143915) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation60 Endeshaw Tadesse Gossa Task Team Leader Richard Olowo Procurement Anjani Kumar Procurement Adenika Sherifat Oyeyiola Financial Management Antonia T. Koleva Operations Bedilu Amere Reta Environmental Safeguards Varalakshmi Vemuru Social Safeguards Hassine Hedda Finance Siobham McInerney-Lankford Counsel Krishna Pidatala ICT Albino Okeny Olak Communications Yoseph Abdissa Deressa Governance and Accountability Begashaw Wukaw Woldu Public Works Museme Munira Issa Gender Muderis Abdulahi Mohammed Institutional Arrangements Colin Andrews Social Protection Emily Weedon Chapman Social Protection Diego Garrido Martin Monitoring and Evaluation Thembi Malena Kumapley Administrative Support Akwii Anne Kennox Administrative Support Philip Blue Consultant Maikel Lieuw-Kie-Song Consultant Ashutosh Raina Consultant Espen Beer Prydz Consultant 60 As listed in the Project Appraisal Document, pages. viii and ix. Page 40 of 67 The World Bank Safety Net and Skills Development (P143915) Supervision/ICR Endashaw Tadesse Gossa, Nadia Selim Task Team Leader(s) Ocheng Kenneth Kaunda Odek, Pascal Tegwa Procurement Specialist(s) Stephen Diero Amayo Financial Management Specialist Antonia T. Koleva Team Member Tracy Hart Environmental Specialist Begashaw Wukaw Woldu Team Member Steisianasari Mileiva Team Member Eric Brintet Team Member Simon Sottsas, Desta Solomon Social Specialist Droma Bank Dominic Kat Team Member Mariam Denise Brain Team Member Ashutosh Raina Consultant B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY13 50.139 417,485.63 Total 50.14 417,485.63 Supervision/ICR FY14 17.500 130,740.84 FY15 35.290 205,767.57 FY16 21.680 140,325.41 FY17 20.195 109,881.44 FY18 31.528 212,271.49 FY19 37.626 225,308.31 FY20 .225 1,545.14 Total 164.04 1,025,840.20 Page 41 of 67 The World Bank Safety Net and Skills Development (P143915) ANNEX 3. PROJECT COST BY COMPONENT Amount at Amount after Sep Actual at Project Percentage Percentage of Components Approval 2017 Restructuring Closing (US$M) of Approval Sep 2017 (US$M) (US$M) (%) Restructuring (%) Social Protection System 5.50 5.50 2.72 49.36 49.36 and Project Management Public Works 11.00 15.50 16.71 151.98 107.86 Skills Development 3.50 0 0 0 0 Unallocated 1.00 0 0 0 0 Total 21.00 21.00 19.43 92.54 92.54 Page 42 of 67 The World Bank Safety Net and Skills Development (P143915) ANNEX 4. EFFICIENCY AND FINANCIAL SUSTAINABILITY ANALYSIS Introduction The South Sudan Safety Net and Skills Development Project (the Project) was expected to address the urgent needs of poor households in the short term. At the same time the Project would set in place building blocks for a social protection system, including immediate management needs as well as longer-term elements. At appraisal, the Project had three components: (i) social protection system and project management (US$5.5 million); (ii) public works (US$11 million); and (3) skills development (US$3.5 million). The Project budget had US$1 million unallocated at appraisal. The public works component was expected to reach 21,500 households with transfers equivalent to US$3 per household for 30 days per year in rural areas and 40 days per year in urban areas. However, the Project was restructured owing to changing dynamics on the ground, with the resumption of conflict, and changing macro- economic, social, and humanitarian context. This was necessary to ensure that the Project’s design and implementation approach effectively address the changing socio-economic dynamics on the ground, taking into consideration increasing humanitarian needs, extreme levels of food insecurity (with famine declared in parts of the country), and worsening macroeconomic conditions characterized by triple digit inflation and rapidly devaluing exchange rate. It was expected that such a restructuring would better align the project to the conditions on the ground, and therefore contribute to improved implementation going forward. Consequently, the restructuring adapted the Project to focus primarily on public works (Component 2) to more effectively respond to the alarming humanitarian and food insecurity needs on the ground, which require a scaled up and rapid response. The funds from the skills development (Component 3) and the unallocated amount of US$1 million were reallocated to the public works component. With these changes, the public works component was scaled up to US$15.5 million, allowing the Project to provide rapid safety net assistance to a greater number of poor and vulnerable households than originally envisioned. The number of target households increased by 58 percent to 34,000 from 21,500. Project locations were also slightly modified, with Pibor county replacing Ayod county in consideration of security and implementation conditions. Efficiency Analysis Efficiency is a measure of how economically resources and inputs are converted to results. In this section, we briefly demonstrate whether the costs involved in achieving the operation’s objectives were reasonable in comparison with both the benefits and with recognized norms (“value for money”). We present the detail of the efficiency analysis, including the underlying assumptions about costs and benefits, and any other information supporting the analysis. The public works activities completed under the Project are very appropriate in the South Sudanese context. Poverty rates have increased from about 50.6 percent in 2013 to 82 percent in 2017 and an estimated 88.7 percent in 2018. With these developments, estimates indicate that real incomes have fallen nearly 70 percent since the onset of the latest conflict in 201661. In addition, many South Sudanese have lost a very large share of their income-generating assets and now rely overwhelmingly on aid. In a situation where average household consumption has been declining over the past five years due to rising poverty, the introduction of cash wages through the public works program would have knock-on effects in the local economy by increasing purchasing power and stimulating demand in local markets. In this context, the public works program would help to preserve purchasing power and protect participants 61IMF (2019). Republic of South Sudan—Policy Note for the 2019 Article IV Consultation. Page 43 of 67 The World Bank Safety Net and Skills Development (P143915) from the effects of a shrinking economy. Moreover, the subprojects and interventions that focused on road rehabilitation and drainage; maintenance; farming; garbage collection; social infrastructure; tree planting and nursery site establishment and management will have longer term human development outcomes due to the development and preservation of community assets and infrastructure such as roads, improved water sources, and better environmental goods which will provide the building blocks for economic recovery and poverty reduction. Cost-effectiveness of the Public Works Program This section estimates the cost-effectiveness of the public works program in comparison to other programs. Using a methodology developed by Ravallion (1999)62 and adapted to reflect the post conflict characteristics of South Sudan, the cost-effectiveness of the program estimates how much it costs to transfer US$1 to beneficiaries based on three variables: (i) labor intensity; (ii) targeting performance; and (iii) net wage gain from the program, which are assessed in greater detail below. The analyses suggest that the Project’s public works component is cost effective in comparison to earlier and similar programs in the Africa region and the world. The estimated cost-effectiveness of the public works component is 0.42. This means that it costs US$1.00 for every US$2.36 transferred in wage benefits to the intended recipients. This estimate compares very well to the overall cost effectiveness of similar programs, despite South Sudan’s very high operational costs. At appraisal, it was estimated that it would cost US$1.00 for every US$2.07 transferred in wage benefits. Thus, the restructuring and the actual implementation modalities on the ground made the Project more cost effective. The cost-effectiveness of the program is primarily due to the high labor intensity, and the precision of the targeting. The assumptions underlying this analysis are summarized in the subsequent paragraphs and correspond to rows in Table 1. Table 1: Estimated cost-effectiveness and comparison to other programs South Sudan Liberia India Ethiopia Bangladesh (SNSDP) (CfWTEP) (NREG) (PSNP) (ESG) Labor Intensity 0.70 0.68 0.71 0.85 0.65 Targeting Performance 0.85 0.80 0.70 0.87 0.80 Net Wage Gain 0.90 0.93 0.50 0.75 0.50 Cost-Effectiveness Ratio 0.42 0.40 - 0.47 - Share of indirect benefits to the poor 0.85 0.80 - 0.85 - Cost of transfer US$1 2.36 1.98 4.02 1.80 1.85 Annual cost per household US$28.9 - - - - Labor intensity is the share of wage costs relative to overall program costs. For the public works component of the SNSDP, the project budget suggests a labor intensity of 0.70, implying that 70 percent of program costs go to wage transfers to project beneficiaries63. This is higher than the 0.67 that was assumed at the appraisal stage. Moreover, it is at par or in some instances even performs better that similar projects in the region and in the world (Table 1). In Liberia, for example, the Cash for Work and Temporary Employment Project (CfWTEP) had a labor intensity rate of 0.68; the National Rural Employment Guarantee Scheme in India had a labor intensity rate of 0.71; the Productive Safety Net Program in Ethiopia has an estimated labor intensity rate of 0.85; and the Bangladesh Food for Work 62Ravallion, Martin (1999) Appraising Workfare. World Bank Policy Research Working Paper No. 1995. The World Bank: Washington DC. 63This computation does not include $340,774 which project documents show as not spent by end of February 2019. If this amount is considered spent on wage transfers, then the labor intensity would increase to 0.72. However, it is not clear whether this amount was saved on component 1 or component 2; thus, it was not factored into the labor intensity analysis. Page 44 of 67 The World Bank Safety Net and Skills Development (P143915) Program had a labor intensity rate of 0.65. Targeting performance is the extent to which the program benefits reach intended beneficiaries (such as the poor) and can be expressed as a ratio of the final beneficiaries that are eligible for the program. The Government’s Project Completion Report indicates that the Project reached 53,136 beneficiary households, with a total of 355,871 individuals in the seven project locations over period of three years. Thus, the Project reached 57 percent more households during the implementation than what was targeted at appraisal (34,000 households after restructuring). The project created more than 3.8 million total work days against original target of 3.5 million and maintained 517 community assets against original target of 150. In addition, about 74 percent of the beneficiary household heads were female, helping to close the gender gap in project targeting. Table 2 shows the aggregated beneficiary details by location and gender of the household head. Table 2: Beneficiary location and gender Location Individuals Households % Female Juba 118,694 15,627 73.3 Torit 45,127 7,378 76.1 Bor 42206 5833 86.3 Pibor 32,714 5,802 63.2 Kapoeta East 34,245 6,857 82.2 Tonj South 41,503 5,833 68.1 Gogrial West 40,748 5,833 69.8 Total 355,237 53,136 74.2 The targeting criteria was established at the national level through participatory process facilitated by the Ministry of Agriculture and Food Security (MAFS). The targeting criteria had three components namely; geographical, administrative, and community-based targeting. It involved vulnerability mapping of the areas by the local authorities (quarter councils/bomas) followed by household targeting using the community structures. The targeting process followed the following criteria: (i) Poor households with no employment; (ii) high dependency ratio; (iii) Child or elderly headed household; (iv) female headed households; (v) households with chronically ill or disabled member; (vi) household with no literate member; (vii) households hosting returnees and or internally displaced persons (IDPs), and (viii) households residing in makeshift or temporary shelter. The household survey findings indicate that 99.8 percent of the beneficiaries in Juba were identified, selected and registered by the Project through house to house visits. While it might not have been possible to have the timeframe to conduct house to house visits when UNOPS implemented the public works in the six other locations, the beneficiary targeting criteria and selection process was designed to objectively and transparently identify eligible public works beneficiaries who were the poorest and most vulnerable within the community. As a requirement, the names of the selected and registered beneficiaries were posted in a public space to ensure that beneficiaries meet the set criteria. Given the very high poverty rates and general levels of deprivation in South Sudan and especially in the project areas, combined with community-based targeting method and the targeting criteria used, a high share of benefits was able to reach the poor. The poverty headcount – measuring the proportion of the population living under the international poverty line of US$1.90 Purchasing Power Parity (PPP) (2011) – was estimated at 82 percent Page 45 of 67 The World Bank Safety Net and Skills Development (P143915) in 2017, placing South Sudan among the poorest countries in the world64. Projections indicate that the poverty rate could have reached 89 percent in 201865. With this poverty rate, it is estimated that 80 percent of the beneficiaries in the public works component complied with the eligibility criteria.66 Net wage gain is the share of the gross wage received by the poor which is a net gain, after considering any foregone income resulting from participating in the program. In the case of South Sudan, it can be assumed that the opportunity cost of participating in the program is very low for several reasons. Most importantly, the poverty rate in South Sudan is very high. The non-oil sector’s contribution to overall economic activity is very small and the size of the formal sector has been shrinking for the past three years, limiting opportunities for wage employment. While unemployment rates are low, many households are engaged in low productivity agricultural activities which account for two thirds of total employment in the country. More than 8 in 10 households rely on own account agricultural production as the primary source of livelihood. Consequently, formal labor force participation rates are very low, with only 59 percent of the age working population engaged in the labor market. The situation is estimated to have become worse, given the precarious economic situation and the recent shocks to the agricultural sector. Importantly, the interventions targeted the very poor and vulnerable households with limited or no income, which means that they were unlikely to give up much income to participate in the program. Overall, the net wage gain is estimated at 0.90 (i.e., for each dollar earned, on average 10 cents are lost from foregone potential income generating activities). Experiences from other public works and cash transfer programs in South Sudan suggest that beneficiary households are investing cash receipts from social assistance in activities and assets that can help improve productivity. In a program conducted by Save the Children in 2010 in Baac Payam, Aweil East County, Northern Bahr el Ghazal, it was found that the most important priorities for use of cash transfer were expenditures on education, health and assets67. An evaluation of another cash transfer program implemented by ACF in Warrap State, South Sudan in 2009 finds substantial returns on investment in the cash grants, estimating them to more than 300 percent over a 3-year period.68 Indirect benefits are those that would accrue to the poor from the assets created or services provided. In South Sudan, these are estimated to be very high as poverty is high and there are large infrastructure and service delivery deficits. South Sudan is one of the most underdeveloped countries in the world, with structural poverty, especially in rural areas. Access to basic community infrastructure is very low. Poor conditions related to access to sanitation and hygiene is a large problem in South Sudan. Only about one in eight people had access to improved sanitation in 201669 in South Sudan. While about 7 in 10 people had access to safe water in 2016, this level of access ranks South Sudan among some of the lowest performing countries in Sub Saharan Africa (SSA). More than 17 out of 20 South Sudanese people reside in rural areas according to the 2008 Housing and Population Census. The rural population is often 64 World Bank (2017). South Sudan poverty assessment. Impact of Conflict and Shocks on Poverty 65 World Bank (2019). South Sudan Macro-Poverty Outlook. 66 This is the result of an independent qualitative beneficiary satisfaction survey (conducted in November 2018 and report completed in February 2019) which found that 80 percent of those surveyed were “satisfied with the categories of households selected as th ese were deemed to be the most vulnerable.” 67 Save the Children (2010) Improving Income and Food Security through Cash Transfers in Southern Sudan: An Evaluation of the Save the Children’s Pilot Project in Northern Bahr el Ghazal State, South Sudan 68 ACF International (2010) Programme Evaluation: Cash Grant Supported Income Generating Activities Twic and Gogrial West Counties, Warrap State, Southern Sudan. 69 World Bank (2017). South Sudan Poverty Assessment. Impact of Conflict and Shocks on Poverty Page 46 of 67 The World Bank Safety Net and Skills Development (P143915) isolated across large swathes of land where infrastructure provision is extremely poor. South Sudan has the lowest road density in SSA with only about 200 km of paved roads, accounting for an estimated two percent of all roads. Seasonal weather and powerful floods, therefore, often leave much of the country totally inaccessible for months at a time. The Government’s Project Completion Report indicates that several of the sub-projects carried out have led to preservation of community assets and infrastructure (Table 3). With these, we estimate that indirect benefits to the poor were very high, estimated at 0.85, signifying that the Project was very beneficial to the communities and some of the benefits will be transferred beyond of the lifetime of the Project itself. Table 1: Number of completed sub-projects Sub-projects No. completed Road rehabilitation and maintenance 344 Drainage 68 General cleaning (markets, hospitals, schools) 37 Farming 24 Garbage collection 20 Social infrastructure 10 School fencing, landscaping, and gardening 4 Tree planting 5 Nursery site establishment and management 3 Airstrip maintenance 1 Stream embankment 1 Total 517 Source: Republic of South Sudan, Project Completion Report. While the public works were able to provide additional income to households, making it easier to spend on basics such as food, school fees, and medical bills, the improved community assets led to improved service delivery, including access to markets and other basic services. In addition, the public works program will lead to the development and accumulation of community assets such roads, improved water sources, and better environmental goods which will contribute to building resilience for the community in which the Project was implemented. Simulating Impact on Poverty and Consumption Simulating the welfare impacts of the wage transfer from the public works program The most recent dataset that is representative at the state level is used to simulate the welfare impact of the wage transfer from the public works program. The third round of the South Sudan High Frequency Survey (HFS) from 2016 is used as the basis for the micro-simulations. CPI data from the National Bureau of Statistics is used to adjust the 2016 consumption measures to their real 2018 equivalents. Coverage of the HFS in 2016 included the former states of Central Equatoria, Eastern Equatoria and Warrap. Jonglei was not included because of security concerns, and therefore the micro-simulations in this section do not cover this state as consumption data are not available. Poverty rates in South Sudan soared between 2009, the year of the National Household Baseline Survey, and 2016. In 2009, poverty was concentrated in Northern Bahr el Ghazal, Unity and Warrap. In 2016, the persistent conflict and near-hyperinflationary conditions had led to poverty rates in excess of 75 percent in much of the country, with the Page 47 of 67 The World Bank Safety Net and Skills Development (P143915) notable exception of Western Equatoria (Figure 1). Figure 1 Poverty headcount in 2016: HFS and satellite imputation Source: South Sudan Poverty Assessment 2017. Note: The HFS 2016 covered 7 out of the 10 former states. In non-HFS states (Jonglei, Unity and Upper Nile) poverty rates were imputed based on satellite imagery. The micro-simulation exercise in this report follows a similar methodology to the simulations that took place at the Project’s Appraisal stage. One difference, however, is that the results reported here are based on the actual number of reported beneficiaries, wage rates and hours worked, while the results in the Project Appraisal Document calculated the poverty impact as if the program were being implemented so as to cover the entire poor population, statewide. Naturally, the poverty impacts are more muted when applying our approach, but they are able to provide a more accurate picture of the true welfare impacts of the Project. These micro-simulations are the result of a static exercise, meaning that general equilibrium, multiplier and behavioral effects have not been considered. In this sense, the Project’s poverty impacts can be interpreted as lower bounds of the Project’s true overall poverty impact. Simulations using the HFS data assuming full coverage of the poor at the state level are presented at the end of this section. Although the Project’s implementation took place at the county level, the simulation is based on data that are representative at the state level. In states which had different wage rates in different counties, the weighted average by number of beneficiaries is used to calculate a state-wide average wage transfer. In 2018, there were 6,140 beneficiaries in Central Equatoria (0.4 percent of the state population), 12,926 beneficiaries in Eastern Equatoria (1.15 percent of the state population), and 5,791 beneficiaries in Warrap (0.47 percent of the state population). On average, beneficiaries in Central Equatoria worked 74.66 days at US$3 a day. Page 48 of 67 The World Bank Safety Net and Skills Development (P143915) In Eastern Equatoria the corresponding numbers were 72.43 days at either US$3 per day (Torit county) or US$2.40 per day (Kapoeta East county). In Warrap the average beneficiary worked for 80.23 days at a rate of US$2.40 per day. The results in Table 4 show the micro-simulation-based changes in the poverty rate, the poverty gap and the squared poverty gap as a result of the wage transfers of the program. As mentioned above, the most recent poverty rates in South Sudan are extremely high. Partly because of this, even a relatively small transfer can have a non-negligible impact on the poverty rate. The proportion of the population that was poor was nearly 80 percent in Central Equatoria, over 95 percent in Eastern Equatoria, and a little over 85 percent in Warrap. The impact of the wage transfer was relatively uniform over the three states for which was can calculate estimates, despite the fact that there were some differences in the daily rates. The poverty rate was reduced by 0.1 percentage points in Warrap, 0.09 percentage points in Eastern Equatoria, and 0.08 percentage points in Central Equatoria. The reductions in the poverty gap are also similar, with the biggest decrease in Eastern Equatoria. Table 2: Simulated poverty rates with and without the program Poverty rate Poverty gap Poverty gap squared Former state Est. Sim. Diff. Est. Sim. Diff. Est. Sim. Diff. Central Equatoria 79.82 79.74 -0.08 0.425 0.424 -0.001 0.287 0.286 -0.001 Eastern Equatoria 95.49 95.40 -0.09 0.657 0.654 -0.003 0.522 0.519 -0.003 Warrap 85.76 85.66 -0.1 0.529 0.527 -0.002 0.402 0.400 -0.003 Note: Calculations based on project beneficiary data and HFS 2016 data. Consumption aggregates are adjusted to their real 2018 equivalents using headline CPI for South Sudan. Simulating the welfare impacts of a full expansion to the poor population Another potentially interesting way of thinking about the micro-simulations is to consider what the potential poverty impacts would be if the same public works wage transfers were to be expanded to cover all poor households. This, of course, would provide an upper bound of the potential poverty impact of the widest possible program, while maintaining the same payment schedule, number of days worked and acknowledgement that general equilibrium effects and behavioral changes are ignored. As shown in Table 5, the poverty reduction impacts of scaling up the Project to cover all poor households (while holding everything else constant) are enormous. This is unsurprising given that (i) poverty rates are extremely high to begin with; (ii) the poverty line in South Sudan corresponds directly to the US$1.90 PPP international poverty line, and (iii) the daily wage rates of US$3 or US$2.40 multiplied by approximately 80 days represents a significant injection of income over the year for poor households. The biggest impact of scaling up the Project to fully cover poor households is in Central Equatoria, which had the lowest poverty rate of the three states to begin with. The simulation shows that a full expansion of the public works program would, all else held equal, result in a final poverty rate of 60 percent. This is a very large reduction from the initial rate of about 80 percent. The simulated reduction in Warrap is smaller at about 13 percentage points, while in Eastern Equatoria – the poorest of the three states – the reduction is a little over 7 percentage points. Changes in the poverty gap are quite similar between the three states, despite all of them starting off at different levels. The changes in this measure range from a reduction of 0.221 in Warrap to a reduction of 0.259 in Eastern Equatoria. Page 49 of 67 The World Bank Safety Net and Skills Development (P143915) Table 3: Simulated poverty rates with and without the program assuming full statewide targeting of the poor Poverty rate Poverty gap Poverty gap squared Former state Est. Sim. Diff. Est. Sim. Diff. Est. Sim. Diff. Central Equatoria 79.82 60.11 -19.71 0.425 0.195 -0.230 0.287 0.090 -0.197 Eastern Equatoria 95.49 88.36 -7.13 0.657 0.398 -0.259 0.522 0.228 -0.294 Warrap 85.76 72.5 -13.26 0.529 0.308 -0.221 0.402 0.171 -0.231 Note: Calculations based on project beneficiary data and HFS 2016 data. Consumption aggregates are adjusted to their real 2018 equivalents using headline CPI for South Sudan. Simulations assume perfect and complete targeting of poor households and are therefore the upper bound of poverty impacts. Financial sustainability South Sudan’s fiscal space has deteriorated significantly over the past years due to a combination of the effects of conflict, lower oil production and lower oil prices that have resulted into a near collapse of the economy. The combined effects of these factors pose major financial sustainability risks for the interventions beyond the life cycle of the projects. The economy has been contracting on average by 9.5 percent since 2015 and is estimated to have contracted by 3.5 percent in 2018. Coupled with economic mismanagement, many years of conflict have eroded the productive capacity of South Sudan. Conflict persists across the country despite the peace agreement and is the major driver of the economic collapse. If the peace agreement is respected by all parties and conflict does not recur, the economy is projected to grow by 1.8 percent during 2019. However, a less positive outlook could emerge if the peace agreement falters, with growth barely reaching 0.3 percent in the absence of progress in the non-oil sectors. Net oil production has plummeted, and little oil revenues are expected to be available to fund the government budget as net oil revenue continue to be used to settle oil advances. Oil production is estimated to have decreased to about 120,000 barrels per day in 2018 down from 165,000 barrels per day in 2014, itself less than half of the peak production before independence in 2011. Given the dire financing shortfalls, the Government has faced difficulties in meeting its payment obligations. The Government has been building arrears by defaulting on the timely payment of civil service salaries and accepting advance oil payments. Civil service salaries were in arrears of three to five months by March 2019, creating tensions among the civil service. The Government must find new sources of revenue to finance the peace process. The authorities had planned a supplementary budget of SSP44 billion for the National Pre-transition Committee to meet peace-related costs, but such plans were rolled over into the next financial year due to lack of resources. The overall budget for the public works project was US$15.5 million, of which US$10.9 million or 70.8 percent was spent on activities that directly benefited the households. The public works implementation was three years. Total annual transfers are therefore estimated at US$3.63 million per year of beneficiary activities. The public works program benefited 53,136 households and 355,871 individuals, resulting in average project transfers of US$68.1 per benefiting household per beneficiary year (or US$10.2 per benefiting individual per beneficiary per year). At the current scope and scale of the Project the costs would correspond to 0.86 percent of the Government’s 2018 annual budget and 0.20 percent of 2016 GDP, thus having only minor budgetary implications. This compares well with Ethiopia’s PSNP program with costs corresponding to 4.3 percent of its annual budget and 1.2 percent of GDP70. However, these estimations should be interpreted with caution given South Sudan’s recent fiscal outturns and limited data credibility. Budgets in South Sudan tend to be grossly under resourced leading to large gaps between estimates and actual revenues/expenditures. 70Sources: Ethiopia Productive Safety Net Programme (PSNP): Prospects for graduation and sustainability beyond 2014 & PSPN project documents in World Bank project database. Page 50 of 67 The World Bank Safety Net and Skills Development (P143915) However, if the program were scaled nationally to benefit all poor household in the country, it would not be financially feasible within the current fiscal envelope. If a similar program were scaled to benefit all poor individuals in South Sudan (82 percent of South Sudanese or 10 million people were living under the international poverty line in 2017), the total cost of the program per year would be approximately US$145 million, assuming similar costs of implementing the program in all parts of the country. While costs are likely to be higher in certain regions, a scaling of the program is also likely to save overall costs from some economies of scale, thus scaling the cost linearly is a reasonable assumption. A total cost of US$145 million would correspond to 24 percent of the Government’s overall 2018 budget or 6 percent of 2016 GDP. Given South Sudan’s current dire economic and fiscal situation, large scale social assistance programs in the country may not be financially sustainable. As discussed, the major risk to future sustainability is shaped by the dire economic situation and uncertainty around sustained peace and economic recovery. Sustainability will also dependent on expected earnings from oil. Whereas South Sudan has instituted the National Revenue Authority, non-oil revenues are not expected to contribute a sizeable proportion of the resource envelop in the short run given the very small size of the non-oil economy. In addition, it will take time to build strong and independent systems for revenue mobilization. The 2018 budget estimates indicate that oil constitutes about 70 percent of anticipated central government revenues. However, the current budget estimates indicate gross mismatches between planned expenditures and available revenue sources. With the 2018 budget increasing by more than 60 percent on the previous year, it is not clear how the increase will be financed. Ongoing payments to Sudan and the cost of the Nile Petroleum Corporation (Nilepet) fuel subsidy reveal that relatively few net oil revenues are available to fund government spending. According to the Government, the net oil revenues available to fund the budget in 2017/18 are estimated at SSP71,860 million, which is equal to just 37 percent of the expected gross oil revenues for the fiscal year. However, the Government must repay the oil advances, estimated at SSP11,100 million, which were taken from Trafigura, a multinational commodity trading company, to help fund the expenditures of the previous fiscal year. While Government has committed to stop the massive subsidies to Nilepet71, it is still early to determine the extent to which this will be followed through. The fuel subsidies to Nilepet were estimated at US$74.7 million during 201872, corresponding to 52 percent of annual total program costs if the program were to be scaled to benefit all poor individuals in South Sudan. Therefore, removing all fuel subsidies and eliminating oil-related transfers to Nilepet would free substantial resources that would be invested in a Safety Net project to benefit the poor. Despite the lack of credible data on the true government finance position, the fiscal deficit is projected to rise to 4.1 percent in 2019 from 3.1 percent of GDP in 2018, due to the anticipated increased spending on the peace process, lower than anticipated oil revenues, and slow progress in raising non-oil revenues73. These experiences have demonstrated the vulnerability of South Sudan’s fiscal space to loss of oil revenues. Maintaining government revenues from oil and diversifying oil revenue from the non-oil economy are therefore essential to make such interventions at a large scale financially sustainable in the country in the future. Therefore, given the limited fiscal space in South Sudan and the negative economic outlook, the Project may not be financially sustainable. However, safety net projects are needed to support the reconstruction of livelihoods in South Sudan, given the high importance of the Project and the large estimated economic returns. However, in fragile and 71 Budget Speech FY2018 72 IMF (2019). Republic of South Sudan—Policy Note for the 2019 Article IV Consultation 73 World Bank (forthcoming). South Sudan Economic Update. Page 51 of 67 The World Bank Safety Net and Skills Development (P143915) conflict countries such as South Sudan, safety net financing highly depends on donors funding. Given that the poverty and vulnerability is very high in South Sudan the need for a safety net program is essential. Therefore, future sustainability of the Project should be considered conditional on continued support through development assistance, which is justified for the case of South Sudan. In addition, the Government should consider taking firm and decisive steps to reform the fuel subsidy program and improve oil revenue governance and use the savings to finance better targeted safety net programs. This is especially important since the fuel subsidy program benefits the wealthy at the expense of the poor and most vulnerable. Page 52 of 67 The World Bank Safety Net and Skills Development (P143915) ANNEX 5. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS The Borrower submitted two documents that are relevant for this section, namely the Project Completion Report (February 2019) and the Borrower’s Implementation Completion and Results (ICR) Report (March 2019). These two documents delivered the same key messages with difference in granularity and depth of elaboration. Both reports used the same source of data. This section provides short summary of the two documents. The Project had three main components namely, (1) Social Protection System and Project Management, (2) Public Works Component and (3) Skills Development Component. The Project was suspended in October 2016 following the conflict in Juba, which occurred in July 2016. However, following the improvement in security situation in the country, implementation resumed in May 2017. In 2017, the Government restructured the project to extend the implementation period to December 2018, removed the Skills Development component and its funds added to the public works component. Highlights of Achievements According to the Borrower’s ICR, the Project achieved the project development objectives (PDOs) satisfactorily. Despite the on-going insecurity, political uncertainties and macroeconomic crisis, as well as the recent halting of implementation, the Project achieved key results and intermediate outcomes. The PDOs were more relevant than ever given the worsening humanitarian and economic conditions in the country which are leading to alarming levels of food insecurity, rising poverty and deepening vulnerabilities. A total of 53,290 beneficiary households (over than 350,000 individuals) were registered for the public works project, however only 53,16374 beneficiary households were engaged in the various public works sub-projects, with about 74 percent being female. This created 4,032,670 workdays and wage transfer amounting to US$10,888,684. These numbers are higher than the end targets of 34,000 households, 50 percent female and 3,600,000 workdays. The details breakdown of the information above are in the table below: Number of Wage Rate Created Work Amount Paid S/# Location Year Beneficiaries (US$) Days (USD) 1 Juba 2016 3,487 3.00 180,826 222,638 2 Juba 2017 6,000 3.00 568,179 1,704,534 3 Juba 2018 6,140 3.00 458,668 1,376,004 4 Torit 2016 1,378 3.00 29,196 31,335 5 Torit 2018 5,992 3.00 479,935 1,439,805 6 Tonj S. 2018 5,791 2.40 465,948 1,118,274 7 Gogrial W. 2018 5,829 2.40 466,279 1,119,070 74 The difference between those registered and those engaged was about 127 beneficiary households who did not get to work and hence were not paid. Some of the 127 beneficiary households initially registered were found to be pastoralist (particularly in Kapoeta East) and hence may have moved after their livestock, while others were due to duplicate at the time of registration in Pibor and others just failed to turn up for payments after working few days. Page 53 of 67 The World Bank Safety Net and Skills Development (P143915) 8 Bor 2018 5,831 3.00 466,570 1,399,710 9 Pibor 2018 5,781 3.00 460,582 1,381,747 10 Kapoeta E. 2018 6,934 2.40 456,487 1,095,569 Total 53,163 4,032,670 10,888,684 The income from the public works wages helped beneficiaries access necessities such as food, clothes, and medicine, as well as pay for their children’s education. The financial literacy training provided by the implementing partner (IP) helped beneficiaries to manage their earnings better. Some beneficiaries have also used part of their earnings from the public works to start small income generating activities. Furthermore, the Project fostered strong ownership and commitment from the community, national and local government. Communities and local authority’s representatives worked hand in hand to ensure targeting of beneficiaries is fair, transparent and inclusive. Project resources were used effectively and efficiently without leakages, and grievances were heard and addressed timely and judiciously. Community assets were built/rehabilitated which in turn improved access to basic services. For example, roads maintenance through public works improved access for trucks delivering water and the price of water reduced from the level before the improvement of the roads. Solid waste disposal and opening of drainage lines contributed to cleanness and environmental sanitation of the neighborhoods where the public work activities took place. In addition, water, sanitation and hygiene (WASH) training provided during the public works gathering raised awareness, and community members organized themselves and conducted cleaning of their neighborhood voluntarily with no payment from the Project. The Borrower’s ICR noted that the Project achieved the expected efficiency from the public works part and even exceeded expectations. Sub-projects were completed well before project completion, whereas the estimated benefits from Project are around the expected benefits/impact. Extra benefits came from relocation of the budget for Component 3 to Component 2 as well as the additional benefits from better coordination, capacity building and road supervision. The other aspects of efficiency, particularly on implementation, have also been improved through institutional strengthening and business process of MAFS, including public works prioritization, planning, budgeting, procurement, contract management, IP capacity building and monitoring among others. Despite the above results, challenges persist, particularly related to the extremely high levels of need vis-à-vis the limited funding. Communities continued emphasizing and urging the Government to both increase coverage within a community and expand to other locations within the country. Assessments by Project Components Component 1: Social Protection Systems Building and Project Management Under this component, the Project achieved the following: • Put in place a targeting and grievance redress mechanisms (GRM) which viewed by beneficiaries and the local authorities as fair, transparent and inclusive. For example, of the total beneficiaries, 74 percent are females (against the end target of 30 percent) with beneficiaries representing diverse ethnic and religious backgrounds. The targeting mechanism worked well, as demonstrated by the high score (80 percent) in the beneficiary’s satisfaction Page 54 of 67 The World Bank Safety Net and Skills Development (P143915) survey/Community Score Cards conducted by an independent consultant. The mechanism is also found to be inclusive and gender sensitive, as the beneficiaries represent a wide range of ethnicities living in communities and most of them are females. Grievances related to inclusion/exclusion of beneficiaries, which continued to be one of the biggest complaints of the Project, and any other issues, are addressed through the GRM. • Developed a biometrically integrated Management Information System (MIS) for beneficiary registry, attendance, payment records and completion status of sub-projects. MIS was developed for the monitoring of the beneficiary registry and progress of public works sub-projects. The MIS system is functional, and all beneficiary information, attendance, payment records, and completion status of sub-projects were included. Key challenges were identified, and an independent MIS systems analysis was undertaken to identify current gaps and solutions, as well as to provide guidance on a longer-term MIS strategy based on the needs of the social protection sector. • Developed safe and transparent public work wages transfer mechanism through commercial banks using two- factor authentication (attendance and biometric identification). An accountable payment transfer mechanism, based on beneficiary biometric identification and attendance records, was established. The payment procedure involved manual generation of payroll by IPs, which then reviewed by the PIU, and then actual payment of the beneficiaries is done by the commercial bank. • Supported the MGCSW in drafting and dissemination of the National Social Protection Policy Framework. Specifically, the NSPPF was finalized and formally launched in May 2015. The NSPPF has been disseminated to all relevant stakeholders in Juba and to the states. Additionally, A Social Protection Unit in the MGCSW to lead SP related activities has also been formed, staffed, and equipped. • Strengthened the PIU and built capacity of government officials through study tours, participation on safety net core courses and community of practices organized by the World Bank. Component 2: Public Works The Public Works component provided the mechanism for conditional cash transfer through creation of temporary employment for the target beneficiary households in the communities. Generally, the communities selected sub-projects, which benefited the larger community (such as road rehabilitation, farming, drainage works), but in particular the target households through the cash transfers. Beneficiary households were engaged on the public works activities for period of 15-20 days in a month, working 4 hours per day after which they were paid for the days worked. The public works component implementation was completed well in all locations. At completion date, 53,290 poor and vulnerable beneficiary households (about 350,182 individuals) were registered but 53,163 households benefited compared to the end target of 34,000 households. About 74 percent of the beneficiaries were female compared to the end target of 50 percent. They were engaged in 517 Public works sub-projects and US$10,888,684 million were transferred for 4,032,670 total person days in the 7 project Counties. Majority of the sub projects identified by communities as priorities are in road clearance and maintenance, followed by drainage works maintenance of public facilities, garbage collection, tree planting and nursery. These works were undertaken successfully through the procurement of necessary tools and inputs valuing at US$973,006 which are also used on a rotational basis by all the community households for personal use, benefitting others more greatly beyond the Project. Page 55 of 67 The World Bank Safety Net and Skills Development (P143915) The cash transfers led to positive impacts on the lives of the beneficiaries, including increased purchasing power for basic necessities, such as food and medicines, ability to pay for school fees and uniforms, and make home improvements. Enhanced community assets contributed to increased access to services and mobility due to better road networks, and crime. People could easily go to the markets, water trucks could come to their neighborhood, and public areas are clean. It also empowered beneficiaries to engage in small scale income generating activities such as setting up tea kiosks, selling vegetables etc. Youth, who would otherwise be idle and potentially active in criminal activities, were productively engaged. Household capacity, specifically for those in Juba, was strengthened for better decision making through trainings in financial literacy and sanitation and hygiene trainings, such that incidences of communicable diseases were decreasing. Child care centers at work sites allowed both mothers with young children, as well as the elderly who take care of them, to participate and earn wages. Participation in public works contributed to an attitudinal change of the communities to continue to work in their area beyond the official work days. Most importantly, engagement in public works provided a platform for greater interaction and dialogue, contributing to local level peace building and stabilization efforts by creating a sense of unity and social cohesion among the diverse ethnic groups. Fiduciary and Safeguards Financial Management. The Project implemented the financial management arrangements as per the financing agreement, disbursement letters and other instructions issued by the Bank Task Team and Fiduciary team. Since effectiveness in November 2014, the Project prepared timely Interim Financial Reports (IFRs) for which all clearances were received. External audits were undertaken for all financial years of project implementation and the Project received unqualified audit opinions without any reservations. The Project disbursed over 97 percent of the total financing as of February 20, 2019. The Project maintained a sound internal control system ensuring that only eligible payments were prepared and approved by the authorized personnel. The internal audit was not undertaken during the project life, but steps were undertaken to have a staff assigned within the Ministry for Internal Audit. Procurement. At project preparation, the Government of South Sudan did not have a procurement legal framework; a national Procurement Bill was in draft and by project closure the Bill had not been assented by the President as required to facilitate its application in management of project procurement activities by the Ministry of Agriculture or by the Implementing Partners. MAFS through the PIU therefore conducted procurement in accordance with the World Bank Procurement Guidelines as provided in the Project Appraisal Document and the Financing Agreement. The Bank procurement tracking system was applied, initially through PROCYS and subsequently through the STEP. The assets acquired under the project were compiled and documented into an asset register; and the Government in consultation with the World Bank agreed to hand-over the assets as Government contribution to the follow up Public Works project. Environmental Safeguards. In compliance to OP/BP 4.01, the sub-projects were screened using the Environmental and Social Safeguards checklist where the impacts were assessed. Environmental and Social Management Plan (ESMP) was prepared in a participatory manner as an instrument for the implementation of the mitigation measures. In order to ensure implementation of the ESMP, a monitoring plan with verifiable indicators was prepared. In line with the World Bank policy of disclosure, the ESMP was published in the National print media and disclosed onto the WB Website. The Natural Habitats (OP/BP 4.04) and Forests (OP/BP 4.36) were not triggered but advisory notes in line with National laws like Forest Act, were disseminated to all stakeholders to ensure that subprojects were not implemented in the natural habitats nor forests. In this regard the Project did not have any impacts on the health and quality of natural habitats and forests. Page 56 of 67 The World Bank Safety Net and Skills Development (P143915) There were some cases of soil erosion, but using scour checks, composting and mulching, its impacts were mitigated. This was the same with cases of wastes generated either through road works or agricultural activities where the wastes were either composted to generate organic manure for enriching the soils or appropriately disposed to the public waste dumping site. In carrying out both public road works or agricultural activities, some vegetation was cleared by ensuring that it was limited to project area without destroying the soil structure and water retention capacity of the soils. To ensure that these mitigation measures were implemented, inspections, supervision field visits, training and sensitizations were carried out by the PIU and IPs in all the seven sites guided by the Environmental and Social safeguards specialists and officers. Social Safeguards. The rating to compliance is high as there were no social ills that affected project implementation despite that fact that ethnic identity is a strong feature in South Sudan, including in rural and agricultural areas. The Indigenous Peoples (OP/BP 4.10) policy was triggered as analysis from social assessments carried out confirms that most people in the project area are expected to meet the requirements of OP/BP 4.10. Given that agricultural land ownership and agricultural practices are all strongly related to ethnic identity and traditions, the Project embedded the basic principles of OP/BP 4.10 of a free, prior, and informed consultation leading to broad community support, as part of the implementation arrangements. Indeed, elite capture by certain ethnic groups and the relative vulnerability of the minor groups could lead to capture of benefits and potential exclusion from participation and benefits. Therefore, the Project applied vulnerability approach to mitigate this risk through carefully defined eligibility criteria for targeting the poorest and mechanisms to reach the vulnerable groups. The Physical and Cultural Resources (OP/BP 4.11) were not triggered but given the strong cultural attachment which were not well documented, the chance find procedures were included as a precaution as part of the ESMF. The most outstanding pillar for the management of the social ills especially grievances was the GRM that had structures from the group level to the national level. In view of the strong GRM, where Appeals Committees were formed with the mandate to receive and resolve grievances, all grievances received were resolved in time to avoid escalation. Majority of the grievances (about 80 percent), rotated around delayed payments because people had urgent needs of food, medical care, school fees and other domestic needs. Such grievances when received by the Appeals Committee, mechanisms would be put in place to ensure that payments are speeded up and in case there are complications then beneficiaries were engaged to ensure they understand the situation. Another driver of grievances was the exclusion and inclusion because everyone is vulnerable. Therefore, in line with the indigenous peoples (OP 4.10), the selection criteria ensured that there was a clear criterion targeting the most vulnerable populations. Other grievances like quarreling, late coming and alcoholism were addressed using the group norms, which were enshrined into the GRM. World Bank’s Supervision and Implementation Support The Bank’s supervision was effective in supporting MAFS’s implementation of the Project. With good understanding of the weaknesses and strengths of the implementing agency, the Bank’s team was able to address the issues detected in a collaborative fashion, with the Bank and the client working together to find amicable and practical solutions. This collaborative approach led to the right capacity building support, mentorship and training (e.g. on procurement and FM in the beginning of the project, technical aspects and on safeguards at preparation and during implementation). The capacity enhancement also went beyond the project. The Bank also facilitated training in GIS for remote monitoring in Nairobi Kenya to enhance project data collection, analysis and reporting and also knowledge exchange activities with neighboring countries, such as Tanzania to share and exchange experiences. Page 57 of 67 The World Bank Safety Net and Skills Development (P143915) Mid-term review (MTR). A thorough and well-documented MTR was carried out towards the mid of January 2018, which confirmed that (a) the project design was still relevant and practicable, (b) risks identified during appraisal were being effectively managed, and (c) implementation was on track towards attainment of the PDO. The assessment concluded that implementation was still appropriately focused not only on public works, but also on systems development and strengthening. The MTR team identified areas in need of further support and/or strengthening, particularly at the quarter council level, was proactive in identifying resources and approaches to provide such support and monitored progress for the duration. South Sudan based implementation support. Given that the Bank’s team was largely in the region, responses to issues did not have to wait for the next mission. Problems could be discussed and handled expeditiously and given a high frequency of dialogue often could be solved before they became problematic. Technical assistance through video conferencing and sometimes phone calls could be made available whenever needed, as opposed to waiting for the next mission. Frequent field implementation support was especially helpful in facilitating coordination and dialog between MAFS and the selected project counties and municipalities during project implementation. However, the inability of the World Bank team to conduct field monitoring/verification and support visits in other areas beside Juba was observed as a challenge. Technical support and flexibility. The Bank’s team was well placed to provide technical support to help MAFS cope with unexpected challenges and changes, i.e. cost overruns and financing gap due to crisis. The team supported the MAFS in adjusting the project design within the original scope to consider restructuring and reallocation of funds from Component 3 and savings from Component 1 to Component 2, a more realistic design that resulted in higher efficiency and more beneficiaries targeted. Various technical assistances were also mobilized to support the MAFS, such as strategic advisory services, internal control system and tools development, Integrated information Management System (MIS), and Financial Management System among others. According to the Borrower’s ICR, the Bank’s overall performance was rated Satisfactory. Challenges The following are some challenges that were noted in the Project Completion Report. All of them were addressed, at varying degrees, during implementation. Exchange rate issues: This affected progress on wage payments as households in some locations were demanding to be paid in USD or at the black-market rates, as well as higher costs for all field transactions, which were done in SSP. To address this, continued engagement/awareness of key stakeholders and households on the agreement and rate to be used, more rigorous coordination with local stakeholders was done. Budget deficit and costs changes: The Project budget did not cover a number of critical issues, which include field operations, some personnel, tools and their transportation and increase in the number of households. The budget for the implementation in the six locations by UNOPS had to be realigned. Fuel, cost and scarcity also affected the projects activities. This was addressed through contracting USD based UNMISS to provide fuel in all the six field locations, which provided more stability and certainty of both pricing of the fuel as well as availability. Seasonality: There was also limited budget for transportation and the seasonality issue further compounded this (start of the rains hence road delivery was no longer possible), which came in since procurement for the six locations was delayed. This resulted in the project resorting to use of cargo plane to send the materials to the field which was very costly. The Page 58 of 67 The World Bank Safety Net and Skills Development (P143915) rainy season also affected public works and the movement of staff. Staffing levels and capacity: The Project had fewer staff than what was required to execute all M&E activities and hence casual staff had to be recruited at some critical stages e.g. during biometric registrations. However, this could not be continued, as the budget could not allow. Also, some staff did not have the capacity and understanding of the MIS and some M&E processes and hence the Project had to commit a lot of time and resources on training of all M&E Staff. MIS glitches: For most of the M&E activities the MIS was required, however due to several issues the system experienced many glitches, which resulted in the slow payment of households. Continuous discussions and engagement were done with the MIS developer to address the challenges as well as that no system upgrades will be done during the day and that if they are any upgrades to be done prior communication should be done. Internet and power: Internet connectivity issues and availability hampered the processes in a great way. There were two field locations which had no internet completely, and hence the Project had to buy new VSAT equipment. The other four locations had connectivity although they had numerous challenges as well. To cater for this, staff had to be brought to Juba to do the data uploads as well as sending of the biometrics devices for downloading of the registration data and uploading of payrolls etc. For the locations with power issues, solar and generators put which had power issues, agreements in place and this was done. Community dynamics: Biometric registration was also delayed due to cultural issues in some locations (e.g. Kapoeta) were communities believed the machines would draw blood samples and hence a lot of awareness raising had to be done. The devices could not read some of the fingerprints either due to them being worn out or due to the application of henna on the fingers by women. The project had to do a lot of awareness raising and use alternatives. During registration, some fingerprints were not recorded well and this affected payment since they could not be read. The team had to redo the fingerprints during the payment process. Low literacy levels: This had a major effect on the ability of committees to mark attendance and impatience of households on the payment process after completion of a work cycle resulting in demand for payment the same day they finished work. UNOPS resorted on the piloting of use of numbers to identify households and flexibility on payment as well as realigning end dates of each cycle for easy of payment Beneficiary payment delays: Due to MIS glitches, internet connectivity issues and low staffing levels among other issues, beneficiary payments could not happen according to plan during the first cycle. Measures were then put in place to ensure that attendance upload is done on a daily basis, that payment plan be made and agreed upon by all stakeholders and that additional staff be brought on board to assist in the process. Handling of public works tools: In most of the project locations, there were not storage provided to ensure safekeeping of the tools. The beneficiaries and the IP had to device means of safe storage of the tools. Due to lack of stores, either the community members stored the tools in a common location (usually a house to one of them) or each of the members kept the tools. In Juba, a few stores were constructed in 2017 and are used to store the tools. There was agreement between the IP and the local authority at the time of handover. Lessons Learned Key lessons from project preparation and implementation experience included but not limited to the following: Page 59 of 67 The World Bank Safety Net and Skills Development (P143915) Straightforward project design supported easy adjustments during the life of the project. Clear, not overly ambitious targets were set and sufficiently straightforward to be easily adjusted when required, e.g., after the crisis and to accommodate restructuring and reallocation of funds from component 3 to component 2. The Project design provided a clear line-of-sight from outputs to objectives, which made it possible for the project to be easily adapted to unexpected challenges. The approach will likely continue to be relevant in the South Sudan context. From the Project’s experience, capacity building takes time and will likely require continuing support due to staff turnover and movements. Institutional leadership by MAFS was key to the success of the project. Many aspects of the project have an impact at all levels, which was possible because MAFS took the lead in donor coordination, financial management practice, safeguards implementation and project oversight etc., beyond the project. This would not be possible without the institutional leadership and ownership of government through the MAFS. Innovative implementation turned the crisis into an opportunity. Because of the need to respond to the urgent needs triggered by the crisis which caused high demand for public works component which was more relevant to suit the context, the Project used the opportunity to restructure and reallocate more funds to the component 2 tackle the increased vulnerability. Inclusion of contingency during budgeting process. With the complexities, which are there in South Sudan. It is important to have a contingency line in the budget to cater for unforeseen costs. For example, the assumptions at the start of the project were that all materials would be delivered by road. However, due to delays the deliveries had to be done by air and this resulted in huge deficits. Monthly or bi-monthly payment cycle to public works beneficiaries. Under the current project it was agreed that beneficiaries should be paid within two weeks after the completion of each work cycle. This proved not only tedious in terms of payment, but also unfeasible in terms of expectation management among the beneficiaries who end up disgruntled when payments are not made in time, despite the very limited time for preparation and apparent challenges with the MIS, etc. Similar projects in future should look into extending this period, to either monthly or bi-monthly payment cycle and awareness should be continuous to continue to raise the beneficiary knowledge on the processes. Public works-type project to have bigger window during the dry season. It was noted that enough time should be given to planning processes to ensure the smooth implementation of the Project. This should also take into account seasonal factors, which may affect project delivery. For this type of project, it is recommendable to start around October to allow for a bigger window during the dry season for delivery of goods as well as implementation of subprojects, which may not be feasible during the rainy season. Strong relationship and trust fostered sustainability. Institutional capacity building was developed hand-in-hand with strong relationship and trust between MAFS and the Bank’s team. With flexibility and focus on sustainability at the institution level, the approach that the World Bank task team used in providing implementation support has built and strengthened relationship and trust, which has provided opportunities to introduce innovative approaches including video conferencing. The Project is a good example of how the institutional strengthening, improved business processes, and the capacity of MAFS to manage the Project had sustainably evolved and expanded beyond national level to state level. Page 60 of 67 The World Bank Safety Net and Skills Development (P143915) ANNEX 6. REVISION OF INTERMEDIATE RESULTS INDICATORS At project restructuring (September 2017), in addition to the revisions of PDO indicators, the two Intermediate Results Indicators (IRIs) related to the Skills Development activities were deleted, and targets for the other IRIs were revised upwards to reflect additional funding. The table below outlined the detailed revisions. Table 1. Revision of Intermediate Results Indicators End Target as per End Target after Sep No. Intermediate Results Indicators Unit of Measure PAD, June 2013 2017 restructuring 1. Beneficiaries of Safety Net programs Number 21,500 34,000 2. Beneficiaries of Safety Net programs – Female Number 6,450 17,000 Beneficiaries of Safety Net programs – Cash for Number 21,500 34,000 3. work, food-for-work and public works 4. Number of work days created Number 2,190,000 3,600,000 5. Number of public works projects Text TBD 150 6. Beneficiary satisfaction with program Percentage 80.00 No change MIS systems for public works and skills training Text MIS systems fully Deleted the second 7. developed and functional functional part of skills training 8. Cash transfers to beneficiaries made on time Percentage 80.00 No change Percentage of Bomas that post list of approved Percentage 90.00 100.00 9. beneficiaries in public places Number of persons who complete the skills training Number 3,500 Deleted 10. program (disaggregated by youth and women) Number of evaluation and studies conducted on Number 3 Deleted 11. skills development Page 61 of 67 The World Bank Safety Net and Skills Development (P143915) ANNEX 7. PUBLIC WORKS LOCATIONS BEFORE AND AFTER THE SEPTEMBER 2017 PROJECT RESTRUCTURING Project Counties marked by blue-colored borders in the maps below were the public works locations. Page 62 of 67 The World Bank Safety Net and Skills Development (P143915) ANNEX 8. LESSONS LEARNED AND RECCOMENDATIONS 1. Addressing fragility and being flexible in the midst of volatile conflict dynamics. Implementation of development projects in a FCV context means that it needs to go beyond simply addressing development gaps to also considering specific fragility and conflict dynamics. This is critical in ensuring that project interventions actively support efforts aimed at restoring peace, establishing stability and promoting medium- to longer-term development. Project design needs to be simple and flexible, not only to be quick in adjusting to the realities, capacity/resource constraints, and risks on the ground, but also to recognize that anything can change and deteriorate at a moment’s notice. This was evidenced by the experience of the Project in the 2016 conflict in particular, which required adjustment and changes to the project design and implementation approach in a timely and flexible manner through project restructuring. 2. Third parties for implementation. The use of third parties as IPs can be very effective and supports the principle of doing no harm in delivering development projects in conflict settings, especially in FCV situations where Government capacity and systems are nascent, and access to project locations are limited. The engagement of NGOs and UNOPS as IPs, supported by a commercial bank as the payment service provider, augmented the capacity gap in the MAFS, enabled access to remote and insecure areas, ensured transparency and accountability, mitigated high fiduciary risks, assured neutrality of service delivery and distribution of resources, and earned communities’ trust in the Project. For example, the payment procedure in SNSDP involved three parties and the World Bank, allowing for rigorous checks and balances. These included: (a) generation of payroll by IPs (AAH-I and UNOPS); (b) review by the PIU; and (c) actual payment to the beneficiaries by the commercial bank. Also, implementation of public works in the locations outside Juba benefited from emerging lessons from AAH-I’s experiences in Juba and the field visit during Project’s Mid-Term Review. 3. Third party monitoring. Having one or more independent agencies to monitor project operational processes and implementation progress is also important in fragile contexts with limited access by the World Bank. The SNSDP had not envisioned the need for an independent third-party monitor during the preparation of the project, as the security situation at that time was favorable. Later, when the context deteriorated, priority was given to transferring as much of the available funds as possible to the beneficiaries, as UNOPS systems and capacities were assessed to be adequate in ensuring delivery of results and fiduciary accountability. While reporting by the IPs were verified by the PIU through monitoring missions to project locations, a third-party monitor in future operations would provide a robust check on reports by the IPs and the PIU, as well as identify areas where inconsistencies over approaches or measurement techniques are evident so that additional capacity building or other adjustments can be made quickly. It would also facilitate knowledge sharing across project areas. 4. “Good-Enough” project design for delivery of support and achievement of results. When the basics of a social protection system is absent, and poverty and need is widespread, a project does not need to be “state-of-the-art” to produce results. For example, a perfecting targeting system is not required where poverty75 is prevalent, as is the case in South Sudan. In a context with near universal poverty and lack of disaggregated data, such “state-of-the-art” targeting methodologies may not be practical, and could in fact be counterproductive as it could delay delivery of safety net assistance and exacerbate grievances within the selected community, thereby hurting the legitimacy of the program. Rather, it is more critical to establish a targeting mechanism that is accepted by beneficiaries and stakeholders as fair 75The Project did not make a distinction between poor and vulnerable households at design as they were both seen as potential beneficiaries as poverty is widespread in South Sudan. Nevertheless, future safety net operations, especially when data is available, may consider avoiding using the term poor and vulnerable interchangeably and clearly defining whether the target groups are the poor, the vulnerable (typically those above the poor and closer to the poverty line), or both. Page 63 of 67 The World Bank Safety Net and Skills Development (P143915) and transparent and that ensures that inclusion errors are minimized as much as possible. Given that there will likely be a large segment of the target population left out due to funding constraints, the reasons for exclusion need to be clear from the beginning, and every effort should be taken to minimize political/elite-based manipulation, including through sensitization, public verification, and GRM. 5. Sufficient time to transfer knowledge. In capacity constrained environments, the development of even the simplest SP delivery tools requires time and transfer of knowledge to local counterparts to ensure their appropriate operation and maintenance. In this sense, teams facing these implementation constraints should estimate more time and handholding for introducing new information technology, providing increased capacity development for specialized roles and functions such as on MIS and biometrics. During implementation of the Project, the MIS developer was not on the ground due to security concern, though support was provided through various communication means. This however limited the developer’s understanding of the nuanced challenges facing project implementation in South Sudan, as well as their ability to provide ‘in-real-time’ support to issues. Further, it also hindered the PIU’s ability to fully take advantage of the developer’s knowledge and expertise, and learn from them in close working arrangements. Future similar projects should therefore consider the MIS system developer to be in close physical proximity to the implementation team (i.e. in the same country or region) to provide timely maintenance and operation support, adjusted to the local context. 6. Effective, fair, and transparent payment system. An effective, fair and transparent payment system is required since nothing is more important to beneficiaries than reliable and timely payments. As was the case under the SNSDP, a project may start with a basic payment system to ensure immediate transfer of benefits to meet urgent needs, and then continue to build upon it. As part of the payment process, the use of third parties as implementation partners can be very effective, especially in situations of high fiduciary risk, limited Government capacity and infrastructure, and nascent (or absent) systems. However, fraud and corruption by, and accountability of, payment agents need to be assured to avoid beneficiaries being cheated, as was the case in a few isolated incidents in the SNSDP where payment agents charged beneficiaries a “commission fee.” Therefore, similar SP projects need to train payment agents on error, fraud and corruption and ethical practices, as part of payment contracts, as well as sensitize beneficiaries of their entitlements. 7. Further, the definition of “on time” payments in the Operational Manual needs to accurately reflect reliable and timely payments to beneficiaries – i.e., not as disbursements to payment providers or payments reaching a community (as under the Project) – but as payments received by beneficiaries. Schedule of payments needs to consider the logistical difficulties of moving cash in FCV settings. These issues need to be communicated and discussed with the beneficiaries to mutually agree on delivery frequency and duration, being very clear on the tradeoffs between frequent but smaller payments (and possible delays) and less frequent and larger payments (likely with fewer delays). Such understanding and agreement would be key in minimizing grievances and ensuring smooth implementation. For example, the three isolated cases of violence against project staff in the SNSDP were all related to unaddressed grievances arising from delayed payments. 8. Close collaboration and coordination with local authorities and community leadership. The Project’s experience highlights the benefits of prioritizing local level coordination structures and building their capacity to support safety net and other interventions more effectively. Partnership between local authorities, the implementing agency and the private sector (i.e. payment service providers) can be very effective in safety net operations in fragile environments, as they can provide layers of checks and balances which minimize errors, fraud, and corruption and alleviates undue influence by one actor engaged in implementation. Coordination with other partners on the ground is also helpful in avoiding duplication and ensuring consistency and efficiency of efforts. Close collaboration with local Page 64 of 67 The World Bank Safety Net and Skills Development (P143915) authorities and community leadership, as well as coordination with other partners on the ground, was key in ensuring smooth implementation, and mitigating social risks under the Project. The structures put in place under the SNSDP also contributed towards a more accountable and transparent governance approach to project management. 9. In-depth community mobilization, communications and sensitization. In-depth community mobilization and sensitization, together with continuous communication with beneficiaries and dissemination of the program, are imperative to ensure buy-in and minimize grievances, especially in an environment characterized by weak capacities and high risks due to fragmented communities with high levels of mistrust. For communities to engage effectively however, concerted efforts should be provided to strengthen their capacity. Local level engagement requires time and up-front planning, especially in communities where capacities are very limited. For example, under the SNSDP, there was no capacity or understanding in many communities of how community-based decision-making structures work. If some form of indigenous systems and structures do exist, it is critical to strengthen them instead of replacing them new with project-specific ones. 10. In Project locations where community mobilization and sensitization were not adequately conducted, the SNSDP experienced higher levels of implementation challenges, at times culminating in escalated tension and violence between project staff and community members. This means adequate training for staff and development of some type of a beneficiary engagement and outreach plan with specific focus on unity, inclusion, and gender issues. Ethical protocol and training for community engagement on issues related to GBV, in particular, should be established to ensure consultations activities are undertaken in line with global standards and best practice. Related to this, when poverty is widespread, and resources are limited, continuous communication with beneficiaries and dissemination of the program are required. Under the Project, the Beneficiary Assessment and interviews during the ICR mission revealed that beneficiaries requested an expanded program, both in terms of beneficiaries and time. This suggests that upfront communications should be undertaken to explain the development and phasing of the program, including possibilities for expansion, if applicable. 11. A functioning GRM. A functioning GRM is critical for addressing grievances, and strong capacity by Appeals Committee members is key to ensuring its full functionality. As participation in the Appeals Committee is a voluntary endeavor, it is necessary to provide minimal incentive package to compensate for its members time. Adequate resources should also be allocated to develop and implement a training plan for GRM structures, complemented by communication and community mobilization. Under the Project, lack of compensation for duties was raised as one of the biggest challenges by committee members in the Beneficiary Satisfaction survey. Innovative, non-salaried and sustainable forms of compensation should be explored (i.e. project shirts, most valuable member awards etc.). 12. Inclusion and empowerment of women. A project design that empowers women and supports them to address their specific needs and challenges is an effective way to promote gender equity and inclusion. The SNSDP specifically targeted women, and was extremely successful in doing so (Section II E). Further, evidence shows that transfers provided to women not only improves their welfare both economically and socially but also that of their households as well. Nevertheless, women’s empowerment can lead to increased intra-household tension and partner violence. Cash transfers can make them targets. Worse, social norms and fears often preclude women from reporting these occurrences, and projects need to incorporate measures to mitigate GBV resulting from activities through preventative measures (e.g., training of staff and community focal points, community sensitization, selection of work and payment sites, etc.). Under the SNSDP, GBV focal points were piloted in the Juba GRM, but lack of information on referral services proved to be a challenge—further efforts at dissemination and communication are needed. Page 65 of 67 The World Bank Safety Net and Skills Development (P143915) 13. Complementary “Cash-Plus” support. Trainings such as basic financial literacy, early childhood development messaging, WASH and training against GBV have contributed to women’s empowerment. These trainings were provided as complementary services to public work participants who were predominantly women. The beneficiary feedback survey confirmed that these trainings were well appreciated and there were also high interests to have more trainings, such as on gender-based violence. Discussions with beneficiaries during the ICR mission recognized these trainings as key benefit for participating in the public works, in addition to the additional income. Training recipients reported improved awareness and application of the skills learned. 14. Finally, the Government must gradually take ownership over safety net assistance such that support is provided through Government-led national systems with full government ownership. Still, in South Sudan as in many other FCV and low-income/capacity non-FCV countries, putting in place a fully functioning safety net will likely only happen over a much longer-term. While the focus of safety nets should be to address immediate needs, short-term provision of cash cannot build resilience, particularly given frequent shocks, and, as a next step, design should have a longer-term vision in terms of the building blocks needed to establish a national SP system. A Government-led national SP system could serve to channel Government and donor funding, and complementary technical support, in a coordinated manner. Page 66 of 67 The World Bank Safety Net and Skills Development (P143915) ANNEX 9. BIBLIOGRAPHY del Nino, C., K. Subbarao and A. Milazzo. 2009. How to Make Public Works Work: A Review of the Experiences. World Bank’s Social Protection and Labor Policy and Technical Notes. HUB Consulting. 2018. End of Project Evaluation: Juba Public Works Project. Juba, South Sudan. Republic of South Sudan, Ministry of Agriculture and Food Security. 2019. Assessment of the Beneficiary Satisfaction of the Safety Net and Skills Development Project: Final Report. Juba, South Sudan. Republic of South Sudan, Ministry of Agriculture and Food Security. 2019. Borrower’s Implementation and Completion Report for Safety Net and Skills Development Project (P143915). Juba, South Sudan. Republic of South Sudan, Ministry of Agriculture and Food Security. 2019. Assessment of Management Information System (MIS) for Public Works. Juba, South Sudan. Republic of South Sudan, Ministry of Agriculture and Food Security. 2019. Project Completion Report: Safety Net and Skills Development Project (SNSDP). Juba, South Sudan. Villalobos, Blanco and Bassett. 2010. Management Information Systems for CCTs and Social Protection Systems in Latin America: A Tool for Improved Program Management and Evidence-Based Decision Making. World Bank. 2013. Project Appraisal Document for $21 million IDA Credit to the Government of South Sudan for the Safety Net and Skills Development Project. World Bank. Multiple Years. Implementation Status and Results (ISR) Reports for the Safety Net and Skills Development Project (P143915). Washington DC, USA. World Bank. Multiple Years. Aide Memoires of the Safety Net and Skills Development Project (P143915) Missions. Washington DC, USA. World Bank. 2018. Implementation Completion and Results (ICR) Report for Investment Project Financing (IPF) Operations. Washington DC, USA. World Bank. 2011. World Development Report. Page 67 of 67