Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) Report Number : ICRR0021553 1. Project Data Project ID Project Name SL:Decentralized Service Delivery Prog P119355 2 Country Practice Area(Lead) Additional Financing Sierra Leone Social Protection & Labor P156548,P162615 L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IDA-H7390,TF-12665 31-Dec-2015 28,729,342.95 Bank Approval Date Closing Date (Actual) 20-Dec-2011 29-Jun-2018 IBRD/IDA (USD) Grants (USD) Original Commitment 26,000,000.00 2,889,331.02 Revised Commitment 30,007,921.50 1,501,057.50 Actual 28,729,342.95 4,603,763.14 Prepared by Reviewed by ICR Review Coordinator Group Katharina Ferl Judyth L. Twigg Eduardo Fernandez IEGHC (Unit 2) Maldonado 2. Project Objectives and Components a. Objectives According to the Project Appraisal Document (PAD, p. 7) and the Financing Agreement of February 14, 2012 (p. 5), the objectives of the project were: "to (i) strengthen government capacity to manage decentralized services; (ii) improve availability and predictability of funding for Local Councils (LCs); and (iii) strengthen the Recipient’s intergovernmental fiscal transfer system. For purposes of this assessment, objectives (ii) and (iii) are discussed and rated together, as strengthening of the fiscal transfer system meant improving availability and predictability of LC funding as part of this project. Page 1 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) Two outcome targets were revised downward, and therefore a split rating is performed. The project was the second phase of a planned three-phase Adaptable Program Loan (APL) whose overall objective was the same as for the second phase. b. Were the project objectives/key associated outcome targets revised during implementation? Yes Did the Board approve the revised objectives/key associated outcome targets? Yes Date of Board Approval 06-May-2016 PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? Yes d. Components The project included four components: Component 1: Grants to Local Councils (LCs) (appraisal estimate US$22.9 million, actual US$16.6 million): This component was to finance grants to LCs to complement inter-government fiscal transfers, both flowing through the same mechanism that was part of the comprehensive decentralization policy informing the revision of the Local Government Act (LGA) launched in February 2004. The grants covered five sectors: health and sanitation, education, rural water, solid waste management, and social assistance services for the disabled and other vulnerable groups. The funds were to be provided based on the LCs' budgets and local development plans to finance service operations and civil works. During the restructuring in June 2012, this component received an additional US$5.4 million through a Trust Fund from the European Union (EU). Component 2: Capacity development and technical assistance to strengthen LCs' and ministries, departments, and agencies' (MDAs') capacity (appraisal estimate US$3.0 million, actual US$5.0 million): This component was to finance LCs to carry out their core functions and central government capacity to provide adequate strategic guidance and oversight to LCs. Capacity building was also to be supported to MDAs for provision of strategic guidance and oversight to LCs. Component 3: Results and social accountability (appraisal estimate US$4.1 million, actual US$4.7 million): This component was to support a focus on community monitoring throughout the project, to be scaled up gradually to cover all five sectors over the project implementation period. It was to support Page 2 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) monitoring of results at the LC level and also support LCs in implementing a results-based approach with service providers. Component 4: Project management (appraisal estimate US$1.3 million, actual US$2.2 million): This component was to finance administrative costs related to the implementation of grants to LCs, capacity development, and social accountability components (Components 1, 2, and 3). It was also to finance specific operational costs of the Integrated Project Administration Unit (IPAU), as well as capacity building to strengthen such areas as fiduciary management, intergovernmental coordination, monitoring, administration, supervision, and auditing. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project Cost: At approval, total project cost was estimated at US$31.4 million. Actual cost was US$28.73 million. The project experienced exchange losses from the SDR to the US$ in the amount of US$1.8 million. Also, the project had to reimburse the EU for ineligible expenditures in the amount of US$1.4 million. Financing: The project was financed through a US$26 million grant from the International Development Association, of which US$24.1 million was disbursed. The project also received an EU Trust Fund of US$5.4 million, of which US$4.6 million was disbursed. Borrower Contribution: The Borrower planned to contribute US$105 million, which materialized. Dates: The project was restructured five times: • On August 31, 2012, the project was restructured to adapt disbursement arrangements. • On January 30, 2015 the project was restructured to reallocate funds between disbursement categories and to change the closing date from December 31, 2015 to December 31, 2016, to allow for completion of project activities that had been delayed due to an Ebola outbreak. • On May 6, 2016 the project was restructured to modify the Results Framework: i) decrease the target values for PDO Indicator 3, “Local Councils (LCs) with Comprehensive Local Government Performance Assessment System (CLoGPAS) social accountability aggregate score over 60,” and PDO indicator 4, “citizens satisfied with service delivery in key sectors”; ii) change data sources for two intermediate outcome indicators; and iii) drop two intermediate outcome indicators. • On December 8, 2016, the project was restructured to change the closing date from December 31, 2016 to October 31, 2017 to allow for the completion of activities in the post-Ebola time frame. • On October 23, 2017, the project was restructured to reallocate funds between disbursement categories and to change the closing date from October 31, 2017 to June 29, 2018 to allow for disbursement of the remaining project funds. Page 3 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) 3. Relevance of Objectives Rationale After a decade of civil war, Sierra Leone started to make improvements in terms of increasing economic growth and reducing poverty. Between 2003 and 2011, the poverty ratio declined from 66.4 percent to 52.9 percent, and the Gross Domestic Product grew at a rate of 15.2 percent in 2012 and 20.1 percent in 2013. However, the country still experienced low human development outcomes, lack of essential social services, and weak governance. In order to address these challenges, the government initiated in 2004 decentralization reforms with the approval of the LGA. The LGA established 19 elected LCs and respective Ward Committees (WDCs) and implemented a legal framework for the devolution of a large number of functions. Also, the LGA aimed to improve the transparency of Sierra Leone’s intergovernmental transfer system and to reduce resource and expenditure inequalities between regions. The decentralization process was supported by the World Bank’s first phase of the Decentralized Service Delivery Program (APL I, 2009- 2011). The objectives of the project were in line with the Bank’s most recent Country Assistance Strategy (FY10- FY13), which identified decentralized service delivery to be a key element for improving basic services in education, health, and water and sanitation. The project’s objectives were also aligned with the Bank’s 2018 Systematic Country Diagnostic, which stressed the importance of finalizing the devolution of competencies to LCs in line with the LGA, and of providing more reliable inter-governmental transfers and social accountability measures. In addition, the project’s objectives were in line with the Bank Africa Region's 2012-2022 Social Protection Strategy, which supports increasing access to and quality of basic services. Rating High 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective Strengthen government capacity to manage decentralized services (original outcome targets) Rationale Page 4 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) The project’s theory of change linked the provision of workshops, LC capacity-building grants, upgrading of the M&E system, and piloting of social accountability mechanisms with strengthened capacity to manage decentralized services. Outputs • Subsidiary agreements were concluded consistently and in a timely manner between LCs and service providers. However, results were not consistently monitored by LCs. • 448 civil works sub-projects were implemented across 1,139 sites. These sub-projects included rehabilitation of 122 primary health clinics and hospitals; rehabilitation and building of 129 primary schools, junior secondary schools, and adult literacy centers; and rehabilitation and construction of 177 water supply systems. Also, six infrastructures on solid waste/social welfare were financed. • In addition to civil works activities, the project funded procurement of drugs and medical supplies, vehicles, power supply and equipment, and medical equipment; training of health workers on family health and infection prevention/control; chlorination of wells and other water sources; and the purchase of solid waste collection vehicles and equipment. Funds were also provided for specialized educational support for children with disabilities, as well as support for survivors of Ebola and their families. • 98.2 sub-projects were finalized, and 97.3 percent of supported facilities were functioning by the time the project closed. • All 19 LCs had integrated development plans, budgets (finalized in accordance with sector plans), and procurement plans (which included LC transfer and project funds), achieving the target of 19 LCs. All 19 LCs had January to December budget execution rates of 90 percent of available funds, achieving the target of 19 LCs. All 19 LCs were producing annual updates on the implementation progress of the development plans by project closure, achieving the target of 19 LCs. All 19 LCs’ M&E units received, processed, and transmitted data related to sector activities to relevant stakeholders, achieving the target. • All MDAs in the project’s sectors (Ministries of Education, Health and Sanitation, Water, and Social Assistance) requested and received demand- (US$133,000 in total in two tranches) and supply-driven (US$50,000 per LC) grants for the technical supervision of LCs. The ICR (p. 18) stated that during the first phase of the program, grants for MDAs were available but were not requested. • A disaggregated list of project-related investments (by type and sector) was kept at all times during project implementation by LC civil work officers, achieving the target. • The number of LCs assessed in the Comprehensive Local Government Performance Assessment Survey (CLoGPAS) increased from 5 out of 19 in 2011 to 10 out of 15 in 2018. The CLoGPAS is a composite instrument that measures institutional capacity at LCs by evaluating performance on management, transparency, planning, financial management, and procurement. CLoGPAS scoring assessed how well LCs performed their duties and whether they exhibited functional capacities that were supposed to be in place according to the LGA and other regulations. Page 5 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) • The project facilitated a South-to-South knowledge exchange between the project and the Ethiopia Promoting Basic Services Program Phase 3, resulting in lessons being incorporated into Sierra Leone’s decentralization strategy and strengthening of LC capacity. • An Integrated National Public Service Survey (INPSS) was conducted in May 2016 and was part of a partial Impact Evaluation in June 2018, achieving the target this survey being conducted at least once during project implementation. • A Community Monitoring Intervention (CMI) was taking place in four LCs and four sectors by the end of project implementation. The CMI used scorecards that allowed citizens to evaluate services in the health, solid waste management, water, and social assistance sectors and was managed by LCs with assistance from the government. • The percentage of ward committees that performed spot checks on service providers increased from 46 percent in 2012 to 95 percent in 2018, surpassing the original target of 75 percent and the revised target of 65 percent. Outcomes • The LCs' aggregate CLoGPAS score increased from 42 percent in May 2012 to 60.2 percent by the end of 2017, not achieving the original target of 75 percent. • The percentage of citizens who were satisfied with service delivery in key sectors increased from 37 percent in 2012 to 85 percent in 2018, surpassing the original target of 70 percent and the revised target of 60 percent. Progress of the overall program as it moved from APL II to APL III was to be assessed by four national-level indicators on service delivery: • The proportion of clinics with essential drugs in stock increased from 32 percent in 2007 to 56 percent in 2018, surpassing the target of 50 percent. • The percentage of rural households with access to an improved water source increased from 52 percent in 2008 to 67.8 percent in 2018, surpassing the target of 65 percent. • The percentage of people with access to improved health and sanitation services increased from 30 percent in 2007 to 92.47 percent in 2018, surpassing the target of 48 percent. • The percentage of schools with Teaching and Learning Materials (TLM) was 58 percent in 2018, not achieving the target of 80 percent. This indicator lacked a baseline because TLMs were not being distributed before project implementation. Page 6 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) Although the original target for improvement in aggregate CLoGPAS score was not met, other targets for citizen satisfaction and service delivery were surpassed. These achievements are indicative of substantially strengthened capacity to manage decentralized services. Rating Substantial PHREVDELTBL PHINNERREVISEDTBL Objective 1 Revision 1 Revised Objective Strengthen government capacity to manage decentralized services (revised outcome targets) Revised Rationale Outcomes • The LCs' aggregate CLoGPAS score increased from 42 percent in May 2012 to 60.2 percent by the end of 2017, achieving the revised target of 60 percent. • The percentage of citizens who were satisfied with service delivery in key sectors increased from 37 percent in 2012 to 85 percent in 2018, surpassing the revised target of 60 percent. Revised Rating Substantial PHEFFICACYTBL Objective 2 Objective Improve availability and predictability of funding for LCs/Strengthen the Recipient’s intergovernmental fiscal transfer system Rationale The project's theory of change linked the transfer of funds to LCs and monitoring of regular window transfers from the central level to LCs with strengthening of the intergovernmental fiscal transfer system, enhancing the predictability and availability of funding for LCs. Outputs, in addition to capacity-building activities listed under Objective 1: Page 7 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) • All LCs received transfers from the project and the EU Trust Fund within each quarter from November 2012 through September 2014. However, LCs did not receive transfers within each quarter of each fiscal year from the Consolidated Revenue Fund (the government’s budget) during project implementation, not achieving the target of doing so. However, funds were provided once per semester, which was more frequently than during APL I. Outcomes • 20.73 percent of domestic revenues were transferred to LCs during FY 2017, not achieving the target of 40 percent. However, the ICR (p. 15) stated that in three out of five FYs during which the indicator was calculated, the target was met. The target was achieved in December 2015, the project’s original closing date, with 49.3 percent. • The percentage of funds transferred to LCs as a percentage of government expenditure increased from 6 percent in 2012 to 6.78 percent in 2018, surpassing the target of 6 percent. The decrease in transfer of domestic revenues to LCs during the last two years of the project is indicative of instability and lack of predictability in funding of LCs, and of insufficient strengthening of the intergovernmental fiscal transfer system. Achievement of this objective is therefore rated Modest. Rating Modest PHREVDELTBL PHOVRLEFFRATTBL Rationale Though the intergovernmental transfer system was only modestly strengthened and therefore there was only modest improvement in the availability and predictability of funding for decentralized entities, the capacity of those entities to manage services improved substantially. Overall efficacy is therefore rated Substantial under both the original and revised outcome targets. Overall Efficacy Rating Substantial PHREVISEDTBL 5. Efficiency Page 8 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) The PAD did not conduct a traditional economic analysis, because at appraisal it was not pre-determined how the different LCs would allocate resources. Instead, the PAD presented a summary of the economic impact of the range of possible expenditures in the priority sectors of health and sanitation, education, and rural water. Also, the PAD provided a discussion of the positive impact of social accountability as a monitoring tool. The ICR (p. 20) stated that efficiency-related data was limited under the project, making the conducting of an economic analysis difficult. Instead, the ICR assessed the operational efficiency of the project. Out of 448 civil works completed, only eight sub-projects were not finalized at project closing, and only three sub-projects experienced cost overruns of more than five percent compared to their initial contract amounts. The ICR also provided a comparison of unit costs in the construction of health centers and schools (57.8% of total project costs) across Africa that were constructed by the same engineer. It cost only US$9,319 on average to construct a new school in Sierra Leone, compared with much more expensive average new school construction costs of US$65,752 in Guinea Bissau, US$32,943 in Comoros, and US$29.230 in Madagascar. Similarly, the construction of a new health center cost on average only US$9,350 in Sierra Leone, compared with US$18,867 in Madagascar. According to the ICR (p. 20), administrative costs under components 2 and 4 were higher than expected due to implementation delays, requiring three project restructurings to reallocate funds from component 1 to components 2 and 4. The implementation delays were mainly due to ineligible expenditures under the EU Trust Fund, resulting in the freezing of funding until the issue had been addressed. In the absence of a formal economic analysis, and with increased administrative costs due to implementation delays, overall efficiency is rated Modest. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Page 9 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) The objectives were highly relevant to country context and to government and Bank strategy. Though the intergovernmental transfer system was only modestly strengthened and therefore there was only modest improvement in the availability and predictability of funding for decentralized entities, the capacity of those entities to manage services improved substantially. Overall efficacy is therefore rated Substantial under both the original and revised outcome targets, though only marginally so. Efficiency is rated Modest due to the lack of a formal economic analysis and implementation inefficiencies. These ratings are indicative of moderate shortcomings in the project's preparation and implementation under both the original and revised outcome targets. Outcome is therefore rated Moderately Satisfactory. a. Outcome Rating Moderately Satisfactory 7. Risk to Development Outcome The project was able to consolidate key institutional structures and build capacity within LCs; in particular, the Decentralization Secretariat (DecSec), which implemented the project and linked the government to LC, was mainstreamed into the Ministry of Local Government. According to the ICR (p. 30), DecSec staff turnover was low during project implementation, but several key employees of the agency were replaced after closure. Given the importance of this institution in the decentralization process, this turnover might have a negative impact on development outcomes. Another risk might be the availability of sufficient funding for DecSec’s operation. The ICR (p. 31) stated that the current government does not have a parliamentary majority, and even though cutting the budget of the LGA is unlikely to happen, the country has a weak track record of following statutory provisions. Providing the LCs with insufficient transfers or recentralizing devolved functions would present a significant risk to development outcomes. 8. Assessment of Bank Performance a. Quality-at-Entry According to the ICR (p. 29), the project built on previous Bank-financed operations, the Institutional Reform and Capacity Building Project (2004-2008) and the first phase of the program (APL I, 2009- 2011). The project’s design took into account lessons learned from the first phase of the program, including the importance of signing LC agreements with service providers, better integration of sector plans into LC budgets and LC development plans through a new indicator, incorporation of the Environmental and Social Management Framework (ESMF) and Resettlement Policy Framework (RPF) Page 10 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) to ensure the project’s compliance with safeguard policies for construction activities, and social accountability activities ensuring a bottom up approach to service delivery. Also, while the M&E framework built on the first phase’s systems, it was modified to better capture intended outcomes. However, data collection depended on several different sources, which resulted in data collection challenges. Several risks, which were identified and mitigated during the first phase of the project, were also addressed during this second phase. According to the ICR (p. 23), several mitigation measures were implemented: supporting the enhancement of coordination between LCs and WDCs; clarification of responsibilities of LCs and MDAs through the introduction of subsidiary agreements; hiring of environmental and social safeguards officers at DecSec and all LCs to reduce the risk of changes in land use; and close monitoring of sub-projects by implementing agencies and LCs, resulting in lower than expected deviations in delivery and quality of sub-projects. Quality-at-Entry Rating Moderately Satisfactory b. Quality of supervision According to the ICR (p. 29), the Bank continuously monitored project implementation and produced Implementation Status and Results Reports (ISRs) as planned. Supervision missions cooperated effectively with implementing agencies, and findings were reported in aide memoirs and management letters. However, although the ICR (p. 29) stated that performance reporting was candid, data on indicators were not always methodologically consistent across project documents and ISRs. The Bank team tried to mitigate this issue by searching for funding to develop a Management Information System (MIS) to consolidate data. However, due to budget issues the development of a MIS was postponed. The Bank team provided appropriate technical assistance in several areas such as M&E, civil engineering, financial management, procurement, safeguards, social accountability, capacity building, and governance. The Bank team restructured the project five times to adapt project activities in line with government efforts to address changing circumstances such as the Ebola outbreak. The Bank team was able to seek additional funding through a grant to address financing gaps that the project could not cover. It also addressed an issue of ineligible expenditures by reimbursing the EU (see Section 10b) and taking measures to ensure that this issue would not happen again. Quality of Supervision Rating Satisfactory Page 11 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) Overall Bank Performance Rating Moderately Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The objectives of the project were clearly specified, though there was redundancy in the second and third objectives. The theory of change specifying how key activities would lead to intended outcomes was reflected in the Results Framework. The selected indicators were specific and measurable, and all had a baseline. The project’s M&E design built on the first phase of the project and used similar evaluation instruments and data sources. The Results Framework benefited from data gathered by the CLoGPAS and INPSS, allowing the use of methodological frameworks that were already used in the first phase and providing comparable data. However, the M&E design did not centralize all required reports at the local level, resulting in a large number of reports including LC reports, LC M&E reports, LC output reports, etc. The following triggers were proposed for movement from Phase II to Phase III, which is still under consideration: • At least 30 percent of domestic revenues (excluding wages, interest obligations, and statutory transfers to Recipient’s National Revenue Agency and Road Fund) transferred to LCs on an annual basis, excluding project funds. • Transfer of funds to LCs equaling at least 6 percent of total government expenditures. • 75 percent of LCs with CLoGPAS results and social accountability aggregate scores over 60 percent. • At least two rounds of CLoGPAS (or lighter version of CLoGPAS) completed, as well one round of a National Services Perception Survey (NSPS). • Lessons from pilot phase of community scorecards prepared and community scorecards implemented. b. M&E Implementation The project’s Results Framework was revised during the May 2016 restructuring to decrease the target values for two outcome indicators. Also, the data sources for two intermediate outcome indicators were changed, and two intermediate outcome indicators were dropped. According to the ICR (p. 25), data from the Page 12 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) government and external firms was triangulated by the Bank to provide more realistic accounts. Progress reports were submitted by M&E officers at the Program Fiduciary Management Unit (PFMU) and DecSec. However, the ICR (p. 26) stated that these reports were not always submitted in a timely manner, and several exchanges with the Bank team were required to complete them. The PFMU was responsible for ensuring compliance with the Grant Agreement and had a tool to track indicators to inform the Bank’s ISRs. Sub-projects were supervised by the Bank and the client. A civil works specialist from the Bank verified the information in the consolidated sub-projects list during supervision missions and developed a consolidated database of all sub-projects funded by the project. According to the ICR (p. 26), LC reports were submitted more frequently than LC M&E reports and LC output reports but were not consistently submitted throughout project implementation. In 2017, the Bank introduced a new format for LC reports. However, since most of the funds had already been disbursed and grant funds had been reduced, a consistent adaptation of the new format across all LCs did not materialize. According to the Bank team (March 25, 2019), the validity and accuracy of M&E data varied depending on the instrument used. The project factored in this uncertainty by producing several independent assessments and by triangulating results of the monitoring instruments used. Overall, the data was found to be reliable. c. M&E Utilization According to the ICR (p. 26), M&E data informed all project restructurings and was shared with the public frequently to increase transparency. For example, scores assigned by CLoGPAS to LCs were published in the press, radio, and television, resulting in best performing LCs being praised in public. The Bank created a blog and a documentary on the project, with the latter aired on national television. However, the ICR (p. 26) also stated that M&E utilization experienced some challenges with data sharing between different levels such as line ministries, decentralization units, LCs, and WDCs, resulting in limited utilization of data. According to the ICR (p. 26), the M&E system lacked a shared standardized platform with parameters that provided alerts when information was not delivered in a timely manner. M&E Quality Rating Modest 10. Other Issues a. Safeguards The project was classified as environmental assessment category B and triggered the Bank’s safeguard policies OP/BP 4.01 (Environmental Assessment) and OP/BP 4.12 (Involuntary Resettlement). The project developed an ESMF and RPF to provide guidance, mitigation, and management of the triggered safeguards. The ICR (p. 27) stated that, since most of the sub-projects were located within already purchased lands and Page 13 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) were being used by public agencies, no follow-up preparation of resettlement action plans, abbreviated resettlement action plans, environmental and social impact assessments, and environmental and social managements plans was needed. DecSec hired a safeguards specialist, and the LCs hired environmental and social safeguard officers who received training and mentoring. The ICR (p. 27) stated that Bank specialists monitored safeguards on routine field visits every six months. Several issues were identified: i) limited sub-project monitoring by LCs in some localities; ii) improper documentation for voluntary land donations; and iii) development of dumpsites into landfill sites. The project addressed these issues by providing supervision vehicles to LC safeguard officers to overcome previously limited means of transportation to visit sub-projects. Additional training was also provided to LCs, and additional supervision missions were conducted by the Bank’s safeguard specialist to ensure the proper implementation of land donation guidelines. Also, the government registered dumpsite sub-projects with the Sierra Leone Environmental Protection Agency to ensure appropriate monitoring. According to the ICR (p. 28), implementation of OP/BP 4.01 was rated Moderately Satisfactory and OP/BP 4.12 was rated Satisfactory throughout project implementation. The project complied with all safeguard policies and produced a safeguards closure report. b. Fiduciary Compliance Financial Management According to the ICR (p. 28), project audit reports had unqualified opinions and were submitted in a timely manner. At the project’s mid-term, an in-depth financial management review was conducted, confirming that financial management systems at the project coordination unit and at the LCs met the Bank’s minimum requirements. The project’s financial management experienced some issues with grant disbursements due to late submission of expenditure returns for the EU Trust Fund by several LCs. The ICR stated that this issue was addressed in a satisfactory manner. Also, the project faced an ineligibility claim by the EU Trust Fund due to the late use of funds. The client had wrongly assumed that funds that were transferred from the government to LCs were considered as expenditures; however, according to the Bank’s disbursement policy, grants were only considered eligible when they were accounted for by the end-beneficiary based on actual expenditures. The partial reimbursement amounted to US$1.4 million in October 2017. In order to avoid this issue in the future, the project ensured that Interim Financial Reports were cleared by the financial management team at the PFMU prior to providing Bank clearances, and it provided additional quarterly financial management supervisions as well as capacity building and mentoring of LCs. The final external audit of the EU Trust Fund was conducted in December 2013, and no issues of ineligible expenditures were identified. Procurement According to the ICR (p. 29), the project built on the procurement capacity of APL I and was rated Satisfactory in all ISRs. The implementing agency prepared procurement documents that were Satisfactory Page 14 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) throughout implementation. Comments by the Bank were addressed in a timely manner. The ICR (p. 29) stated that ISRs and aides-memoire reported minor delays in drafting and submitting Terms of Reference and technical specifications during the first semester of 2014. The Bank provided the procurement officers of the LCs with training on the new bank procurement framework. c. Unintended impacts (Positive or Negative) None reported. d. Other --- 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Moderately Outcome Moderately Satisfactory --- Satisfactory Moderately Bank Performance Moderately Satisfactory --- Satisfactory Quality of M&E Modest Modest --- Quality of ICR High --- 12. Lessons The ICR (p. 31-32) provided useful lessons learned, adapted here by IEG: • Developing data-sharing mechanisms through an integrated Management Information System (MIS) across different government agencies, Local Council (LCs), and service providers to better understand, promote, and manage resources and local services is critical. In this project, gathering and disseminating data to inform decision making was challenging between the different levels such as line ministries, decentralization units, LCs, and Ward Committees. The M&E system lacked a shared standardized platform with parameters that provided alerts when information was not delivered in a timely manner. • Coordination and knowledge exchange between LCs are critical for building capacity in all entities. In this project, LCs benefitted from LC officers rotating from one LC to another. More intensive and Page 15 of 16 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review SL:Decentralized Service Delivery Prog 2 (P119355) formal fostering of peer-to-peer learning, mentoring between officers, and exchange visits of best-performing officers/managers between LCs would be useful to ensure capacity building in all LCs. • A decentralized government framework can reduce social tensions in post-conflict countries with deep geographic and ethno-political divisions. In Sierra Leone, the decentralization process introduced services closer to beneficiaries, increased citizen engagement and ownership of local agendas, and mitigated tensions between ethnic groups, fostering cooperation among these groups. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR provided a comprehensive overview of project preparation and implementation. It was internally consistent and candid, provided excellent sourcing of all the data, and included an explicit assessment of the quality of the data. Furthermore, it used data outside the Results Framework to compensate for inadequate project M&E. In addition, the ICR provided an explicit attribution discussion and demonstrated the results chain well. Also, it included several useful and interesting lessons that drew directly from the main-text discussion. It did not include a formal economic analysis, but overall, the quality of the ICR is rated High. a. Quality of ICR Rating High Page 16 of 16