Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004522 IMPLEMENTATION COMPLETION AND RESULTS REPORT IDA-51220 ON A CREDIT IN THE AMOUNT OF SDR 11 MILLION (US$17 MILLION EQUIVALENT) TO THE REPUBLIC OF SIERRA LEONE FOR THE SIERRA LEONE (SL) PUBLIC SECTOR PAY AND PERFORMANCE PROJECT March 29, 2019 Governance Global Practice Africa Region CURRENCY EQUIVALENTS (Exchange Rate Effective {November 13, 2018}) Currency Unit = Sierra Leonean Leones (SLL) SLL 8,450 = US$1 US$1.39 = SDR 1 FISCAL YEAR ABBREVIATIONS AND ACRONYMS AfDB African Development Bank AGD Accountant General Department ASSL Audit Service Sierra Leone BoSL Bank of Sierra Leone CAS Country Assistance Strategy CSSC Civil Service Steering Committee DFID Department for International Development DLI Disbursement-Linked Indicator EU European Union EVD Ebola Virus Disease FA Financing Agreement GDP Gross Domestic Product GoSL Government of Sierra Leone HRMO Human Resources Management Office ICR Implementation Completion and Results Report IDA International Development Association IFC International Finance Corporation IFMIS Integrated Financial Management Information System IL Investment Loan IMF International Monetary Fund IPAS Individual Performance Appraisal System IRI Intermediate Results Indicator ISR Implementation Status and Results Report JCAS Joint Country Assistance Strategy LT Leadership Team LTA Local Technical Assistant M&E Monitoring and Evaluation MDAs Ministries, Departments, and Agencies MEWG M&E Working Group MOFED Ministry of Finance and Economic Development MTR Mid-Term Review P&PP Pay and Performance Project PAD Project Appraisal Document PCN Project Concept Note PDO Project Development Objective PforR Program-for-Results PFMICP Public Financial Management Improvement and Consolidation Project PIM Project Implementation Manual PRAMS Procurement Risk Assessment and Management System PRIMA Portfolio and Risk Management PSC Public Services Commission PSR Public Sector Reforms PSRU Public Sector Reform Unit RRA Rapid Results Approach RRI Rapid Results Initiative SCPSR Steering Committee on Public Sector Reforms SIL Sector Investment Lending SLL Sierra Leonean Leones SOE State-Owned Enterprise TA Technical Assistance Regional Vice President: Hafez M. H. Ghanem Country Director: Henry G. R. Kerali Senior Global Practice Director: Deborah L. Wetzel Practice Manager: Hisham Ahmed Waly Task Team Leader(s): Smile Kwawukume ICR Main Contributor: Shomikho Raha, Daniel Nogueira-Budny TABLE OF CONTENTS DATA SHEET ...............................................................................................................................1 EXECUTIVE SUMMARY ........................................................................................................... 5 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 7 A. CONTEXT AT APPRAISAL .........................................................................................................7 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION .............................................................. 13 II. OUTCOME .................................................................................................................... 15 A. RELEVANCE OF PDOs ............................................................................................................ 15 B. ACHIEVEMENT OF PDOs (EFFICACY) ...................................................................................... 16 C. EFFICIENCY ........................................................................................................................... 24 D. JUSTIFICATION OF OVERALL OUTCOME RATING .................................................................... 27 E. OTHER OUTCOMES AND IMPACTS ......................................................................................... 27 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 29 A. KEY FACTORS DURING PREPARATION ................................................................................... 29 B. KEY FACTORS DURING IMPLEMENTATION ............................................................................. 30 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 31 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................ 31 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE ..................................................... 35 C. BANK PERFORMANCE ........................................................................................................... 35 D. RISK TO DEVELOPMENT OUTCOME ....................................................................................... 39 V. LESSONS AND RECOMMENDATIONS ............................................................................. 39 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 42 ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 59 ANNEX 3. DATES AND AMOUNTS OF DLI DISBURSEMENTS ................................................... 61 ANNEX 4. EFFICIENCY ANALYSIS ........................................................................................... 64 ANNEX 5. RECRUITMENT INFORMATION ............................................................................. 67 ANNEX 6. PAY HARMONIZATION AND THE ESTABLISHMENT OF A WAGE COMMISSION ....... 69 ANNEX 7. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ... 70 ANNEX 8. SUPPORTING DOCUMENTS .................................................................................. 71 The World Bank SL Public Sector Pay & Performance (P128208) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P128208 SL Public Sector Pay & Performance Country Financing Instrument Sierra Leone Investment Project Financing Original EA Category Revised EA Category Not Required (C) Not Required (C) Organizations Borrower Implementing Agency Ministry of Finance and Economic Development - Human REPUBLIC OF SIERRA LEONE Resources Management Office Project Development Objective (PDO) Original PDO Improve competitiveness in pay, performance management and accountability of, and increase staffing of middle and senior staff in, the civil service in Sierra Leone. Page 1 of 72 The World Bank SL Public Sector Pay & Performance (P128208) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 17,000,000 17,000,000 13,970,762 IDA-51220 Total 17,000,000 17,000,000 13,970,762 Non-World Bank Financing 0 0 0 Borrower/Recipient 0 0 0 Total 0 0 0 Total Project Cost 17,000,000 17,000,000 13,970,762 KEY DATES FIN_TABLE_DAT Approval Effectiveness MTR Review Original Closing Actual Closing 31-May-2012 23-Oct-2012 10-Feb-2014 31-Oct-2015 30-Jun-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 24-Mar-2014 8.12 Change in Results Framework Change in Loan Closing Date(s) Change in Implementation Schedule 09-May-2015 9.19 Change in Loan Closing Date(s) 30-Oct-2017 12.69 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Moderately Unsatisfactory Moderately Unsatisfactory Modest Page 2 of 72 The World Bank SL Public Sector Pay & Performance (P128208) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 13-Nov-2012 Satisfactory Satisfactory 0 02 18-Jun-2013 Satisfactory Satisfactory 4.39 03 30-Nov-2013 Satisfactory Satisfactory 8.12 04 25-Apr-2014 Satisfactory Satisfactory 8.12 05 20-Aug-2014 Satisfactory Satisfactory 8.12 06 14-May-2015 Satisfactory Satisfactory 9.19 07 15-Jan-2016 Satisfactory Satisfactory 9.19 08 16-Jun-2016 Satisfactory Satisfactory 9.69 09 02-Jan-2017 Moderately Satisfactory Moderately Satisfactory 12.69 10 15-Aug-2017 Moderately Satisfactory Moderately Satisfactory 12.69 11 14-Feb-2018 Moderately Satisfactory Moderately Satisfactory 12.69 SECTORS AND THEMES Sectors Major Sector/Sector (%) Public Administration 100 Central Government (Central Agencies) 90 Sub-National Government 10 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Public Sector Management 100 Public Administration 100 Administrative and Civil Service Reform 100 Page 3 of 72 The World Bank SL Public Sector Pay & Performance (P128208) ADM STAFF Role At Approval At ICR Regional Vice President: Obiageli Katryn Ezekwesili Hafez M. H. Ghanem Country Director: Yusupha B. Crookes Henry G. R. Kerali Senior Global Practice Director: Marcelo Giugale Deborah L. Wetzel Practice Manager: Anand Rajaram Hisham Ahmed Waly Task Team Leader(s): Roberto O. Panzardi Smile Kwawukume ICR Contributing Author: Shomikho Raha Page 4 of 72 The World Bank SL Public Sector Pay & Performance (P128208) EXECUTIVE SUMMARY The Sierra Leone Public Sector Pay and Performance Project (P&PP, P128208) was a US$17.0 million Specific Investment Loan made up of disbursement-linked indicators (DLIs) and a technical assistance component. It was approved on May 31, 2012 and closed on June 30, 2018, after three Level 2 restructuring that included extensions. Its project development objective (PDO) was, to improve competitiveness in pay, performance management and accountability of, and increase staffing of middle and senior staff in, the civil service in Sierra Leone. The context, within which the P&PP was designed, changed significantly during project implementation, but the project did not adequately adapt to the changes to improve its effectiveness . At appraisal in 2012, Sierra Leone was undergoing an economic recovery and rapid growth was projected to continue. Since the number of staff in Grades 6-10 was very limited, low pay and an unfair pay structure discouraged qualified candidates from entering the public sector, and performance management was weak, there was broad consensus on the need to modernize its civil service. However, twin exogenous shocks—the Ebola epidemic and the collapse in international iron prices in 2014—led to the economy in 2015 having contracted by 21 percent. This changed context had significant effect on the assumptions of fiscal space and political appetite for pay reform. However, the project never adequately addressed this disconnect and the project closed after extensions of 32 months with three DLIs unachieved. Project performance is summarized below. Relevance Efficacy Efficiency Overall Bank M&E Quality Outcome Performance Substantial Modest1 Modest Moderately Moderately Modest Unsatisfactory Unsatisfactory The overall outcome of the P&PP was Moderately Unsatisfactory. The PDO remained relevant throughout the life of the project. Achievement of the PDO, however, was modest. There was negligible impact regarding improving the competitiveness of civil service pay: the new pay structure, which would ensure more equitable and competitive pay to civil servants, was never approved. Improvements to both the performance management and accountability of the civil service were modest. Substantial progress was still made in increasing staffing of middle- and senior-level staff in the civil service: the project helped register both real and percentage increases in the hiring of technical and professional staff to fill the “missing middle.” However, project efficiency was only modest, as the anticipated counterpart funding did not materialize, and substantial implementation delays contributed to the need for project extensions beyond October 2015 till June 2018. At close of project, 10 out of 13 DLIs were deemed achieved with 2 out of 5 PDO indicators were assessed met. Bank performance for the project was rated Moderately Unsatisfactory. The amount of staff weeks dedicated to project implementation was significantly less than the amount of supervision originally 1The overall rating was based on average of the ratings of the PDO’s four sub-objectives. These ratings were: Negligible, Modest, Modest, and Substantial. Page 5 of 72 The World Bank SL Public Sector Pay & Performance (P128208) envisioned. Implementation Status and Results Reports (ISRs) noted implementation challenges, but the task team was slow to reflect actual concerns related to project implementation by revising project ratings downward, which happened relatively late. Changes to the Results Framework arising from the first restructuring were not properly captured within the system, complicating the monitoring of modified indicators. Advance disbursements against DLIs were not systematically tracked to prevent potential ineligible expenditures arising from failure to achieve DLIs. Critically, the Bank project team could have been more proactive to discuss with government counterparts the need to restructure the project. Especially the pay reform aspects of the project needed to better reflect the profound changes to the underlying assumptions in project design at appraisal. The project’s M&E quality was Modest. The M&E framework was well established during project preparation. Implementation, however, was weak: there was irregular monitoring of the Results Framework or reporting of indicator values on quarterly progress reports. A rapid results approach facilitated the achievement of many activities, but was unable to address many of the implementation- related delays that plagued the project. Furthermore, M&E did not appear to inform project management or decision-making processes. Key lessons learned that arise from this ICR include:  A significantly changed country context during project implementation that questioned the key assumptions at project appraisal required that the project design (and DLIs) be adapted sufficiently through restructuring to the changed context in order to ensure project effectiveness;  The overall size of project financing, as well as the value attached to individual DLIs, should have been commensurate with the scope and ambition of the reform;  Following the first restructuring, the project did not sufficiently link the horizontal reforms of institutional strengthening of civil service recruitment and performance to a tangible results- based focus on outcomes within a vertical line ministry to clearly demonstrate the impact of such reforms, including any related links to the core service performance mandate of that line ministry;  The use of a results-based financing instrument combined with the use of country-based systems, especially for the first time, required additional caution, guidance, and training to be built into project implementation;  Disbursements based on individual DLIs ideally should not have been complicated by withdrawal limits set based on the grouping of DLIs, and instead there was need for greater focus on tracking disbursement of advances made against DLIs until these advance payments could be adequately deemed as eligible expenditures;  When DLI disbursements are made into the Consolidated Fund, the pricing of challenging DLIs that require the coordination of multiple agencies is critical — and the project design may have needed additional monetary incentives built into such DLIs for participating agencies;  Non-technical aspects and political economy risks to the reform path needed to have been explicitly identified and addressed in the project, such as related to pay reform and mainstreaming of LTAs. Page 6 of 72 The World Bank SL Public Sector Pay & Performance (P128208) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL 1. The Sierra Leone (SL) Public Sector Pay and Performance Project (P&PP, P128208) was designed in the context of Sierra Leone’s economic recovery, projections of continuing rapid growth, and an increasing realization of the need to modernize its civil service. The Government’s revenues were projected to increase two-to-threefold in the next few years, with iron ore exports expected to drive economic growth. This imminent windfall from exports led to rising expectations that civil service salaries would be increased. In parallel, the inadequacy of Sierra Leone’s civil service was becoming increasingly apparent to the Government of Sierra Leone (GoSL). 2. There was growing recognition of the need to rebuild the civil service after years of development work ‘bypassing’ the core civil service during and following the conflict . A functioning civil service, it was argued, would help to address the serious development challenges plaguing the country, which was ranked 180 out of 187 countries in the 2011 United Nations Human Development Index. Further, the weak public sector was a constraint on the performance of the rest of the World Bank’s country lending portfolio. Previously, the country’s civil service had once been one of the most highly respected civil services in West Africa (Blum et al., 2017). 3. At appraisal in 2012, Sierra Leone’s public sector suffered a deficit in public sector capacity and performance, diagnosed as particularly severe due to a ‘missing middle.’ Specifically, the number of staff in Grades 6–10 was very limited, with only 11 percent of the total civil service workforce (see Table 8 change in civil service composition). Low pay discouraged qualified candidates from entering the public sector.2 This was compounded by an unfair pay structure that resulted from: (a) the establishment of autonomous agencies and commissions that performed key functions exempt from civil service rules, (b) the growth of ‘outside grade’ appointees who were paid at discretionary salaries, and (c) the hiring of highly paid local technical assistants (LTAs) by development partners to support implementation of externally-financed projects.3 4. One rationale for the project was to help strengthen the institutions and processes that were tasked with recruitment of the civil service, which would in turn reduce ad hoc patronage appointments. At the time of project appraisal, large numbers of civil servants within the GoSL had been appointed under the one-party state outside of the formal recruitment process, including for reasons of patronage — not merit. A culture of weak and opaque performance management within the civil service was suggested as an additional disincentive to attracting higher numbers of potential applicants. The Project Appraisal Document (PAD) mentioned a growing sense of demoralization among civil servants, given a lack of extrinsic incentives for performing their jobs well. 5. The World Bank-supported project was intended as a strategic part of the GoSL’s own Medium- Term Reform Framework (2012–2015) to improve public sector governance. The Medium-Term Public 2 The PAD noted that the GoSL has been forced to offer an average of US$200 per month top-ups to regular civil service wages to attract qualified staff to professional positions. 3 These three phenomena were referred to collectively as the ‘shadow civil service.’ Page 7 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Sector Reform (PSR) involved: (a) launching pay reforms to improve competitiveness in the public sector, (b) rationalizing staff by filling existing critical skill gaps, (c) minimizing waste and inefficiencies through improvements to the integrity of payroll and personnel records, (d) developing the institutional capacity to drive reforms, (e) introducing a performance management system, and (f) implementing comprehensive training and staff development programs. 6. The P&PP was expected to leverage US$17.0 million equivalent of International Development Association (IDA) resources as a Specific Investment Loan (SIL) using a results-based investment lending modality through disbursement-linked indicators (DLIs) and a technical assistance (TA) component. The rationale for using a results-based approach was primarily in recognition of the potential that such an approach has for promoting collaborative working within the Government, as well as for addressing the collective action problem. It was expected that results-based financing would mitigate the high risks posed by a fragmentation of functions across many different ministries, departments, and agencies (MDAs) that did not sufficiently coordinate with one other. The project was approved by the World Bank Board on May 31, 2012 and became effective on October 23, 2012.4 Although it was scheduled to close on October 31, 2015, numerous issues — including the Ebola Virus Disease (EVD) outbreak, procurement delays, and the failure to make implementation headway on a series of activities — led to three project extensions. Theory of Change (Results Chain) 7. Although the project documents did not explicitly include a theory of change, which was not a required World Bank procedure at the time of project preparation, the rationale for the intervention included in the Project Appraisal Document (PAD) provides an explanation of the expected results. Based on this information, a results chain was constructed to link project inputs to outputs and outcomes (see Figure 1). 8. The project’s subcomponents and expected outcomes were based on the GoSL’s PSR framework. Given this foundation within the Government’s strategic plan, the project had a clear internal logic. Figure 1 details the results chain, visualizing project subcomponents, inputs, Project Development Objectives (PDOs), and outcomes. Component 2, which focuses on the provision of TA to support Component 1 activities, was expected to facilitate the achievement of the PDO and the intermediate results indicators (IRIs). Therefore, it was not included in the project’s Results Framework. 4The effectiveness deadline of 90 days after signing of the Financing Agreement (FA) was not achieved due to issues relating to the finalization of the financial management section of the Project Implementation Manual (PIM). The World Bank subsequently granted a one-month extension to allow the GoSL to finalize it. Page 8 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Figure 1. Project Results Chain Note: GoSL= Government of Sierra Leone; HRMO= Human Resources Management Office; LTA=local technical assistants; PDO= Project Development Objective. 9. A critical assumption implicit in the project design and the results chain was that key activities, which were prerequisites for later ones, would be delivered in a timely, sequential manner. The project had an implicit sequential logic in that the implementation of later activities were predicated on the successful implementation of earlier ones. For instance, the ability of civil servants to be paid according to the new grading and pay structure was dependent on the successful delivery of the job evaluation and labor market survey, on which the new, approved structure would be based. The fact that this survey was significantly delayed by endogenous and exogenous factors (explained later in the Implementation Completion and Results Report [ICR]) caused implementation delays to pay structure reform. 10. Another fundamental assumption was that the country’s rapid economic growth during the project period would create the fiscal space for the new pay and grade structure, as well as the expansion of the civil service. In contrast, the PAD noted that real GDP growth was expected to remain high, as revenues from the natural resource sector were expected to boost domestic revenues by approximately 95 percent from 2010 to 2014. Likewise, the contribution of iron ore to GDP was projected to grow at 50 percent in 2012. The Country Assistance Strategy (CAS) Project Report, written shortly after the PAD, also noted this positive economic trend. However, the twin shocks of the Ebola epidemic and the downward trend in international iron prices in 2014 led to a severe contraction of the economy. The rapid growth expected at the time of appraisal did not materialize and the GDP instead contracted by 21 Page 9 of 72 The World Bank SL Public Sector Pay & Performance (P128208) percent in 2015.5 Lower revenues and substantial expenditure needs, followed by a sharp depreciation of the leone and double-digit inflation, resulted in substantial increases in the existing wage bill as a percentage of GDP, as well as a freeze on recruitment to the civil services called for by the International Monetary Fund (IMF). As such, the GoSL was forced to rethink its civil service reform plans. (See Annex 1).6 Project Development Objectives (PDOs) 11. The PDO was to improve the competitiveness in pay, performance management, and accountability of, and increase staffing of middle and senior staff in, the civil service of Sierra Leone. Key Expected Outcomes and Outcome Indicators Box 1. PDO Indicators PDO Indicator Status at Project Closing 1. Civil servants in Grade 6 and above are paid in accordance with the approved pay Not Achieved structure. 2. Priority vacancies have been filled in accordance with the Annual Recruitment Achieved Plans and approved recruitment procedures. 3. Annual Performance Appraisal Reports of civil servants in Grades 7 to 10 in pilot Dropped ministries are of appropriate quality.a 4. Annual Performance Reports for civil servants in Grade 11 and above in Pilot Achieved Ministries on Performance Contracts are of appropriate quality. 5. All Ministries’ performances against Performance Targets have been evaluated Dropped jointly by the recipient and relevant non-state actors. 6. Direct project beneficiaries (number), of which (%) are female. Not Achieved 7. Recruits to priority vacancies after January 1, 2013, retained in civil service Not Achieved Note: a. PDO Indicators 2 and 5 were dropped in the restructuring following the Mid-Term Review (MTR), as explained in the following paragraphs. An additional indicator (Indicator 7, above) was added, that is, 80 percent recruits to priority vacancies after 1 January 2013 are retained in the civil service. 12. The project initially had 15 DLIs uniformly weighted at US$1 million each (regardless of the ambition of the DLI), with the option of receiving 50 percent advances against each DLI. With an original three-year timeframe, the project design grouped DLIs by years (for example, Year 0 to Year 3), following a sequential logic. After the first restructuring (detailed below), the DLIs associated with the Performance Management Subcomponent (1.3) were altered. Three of the Year 3 DLIs were rescheduled to Year 4 (2015), the overall number of DLIs was reduced from 15 to 13, and DLIs 3.2–3.6 were altered or dropped. 5In 2014, there was a complete collapse of the price of iron ore which declined by 47 percent. 6Annex 1, Figure 1.1 details the actual versus projected wage bill from 2012 to 2020 and Figure 1.2 details the actual wage bill as a percentage of GDP versus projections from 2012 to 2020. Substantially fewer domestic resources were spent on the medium- term PSR than planned. In 2015, a freeze on civil service hiring effectively halted the Government’s efforts to fill the ‘missing middle.’ Page 10 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Annex 1 details the changes to the DLIs following the project restructuring, as well as the status of the DLIs at project closure. 13. Changes to the Results Framework were captured in the first restructuring Project Paper 7 and Amendment to the Financing Agreement (FA), but they were never formalized within the World Bank system. As a result, all subsequent Implementation Status and Results Reports (ISRs) continued to include indicators from the original Results Framework. This further complicated the ability of both the client and the World Bank to systematically monitor the 50 percent advances and reimbursements against the achievement of DLIs, given that these should have been correctly captured as indicators in the revised Results Framework. 14. By the close of the project period, based on the verification protocol, 10 DLIs were achieved out of a total of 13 DLIs listed in the amended FA following the 2014 project restructuring . Table 1 lists the final status of the 13 DLIs. Table 3 shows this discrepancy between the number of DLIs achieved versus the amount of funds disbursed for achievement of the DLIs. Annex 3 provides more details about this issue.8 Table 1. Final Status of DLIs (Post-Project Restructuring) DLIS achieved 1. The HRMO has conducted and completed a remuneration survey (DLI 1.1). 2. The appropriate open, competitive and merit-based recruitment procedures have been designed by the PSC in collaboration with the HRMO and approved by the CSSC (DLI 2.1). 3. Seven staff have been recruited to the PSC and trained to manage the recruitment and selection process (DLI 2.2). 4. At least 60 percent of priority vacancies have been filled in accordance with the Annual Recruitment Plans and the approved recruitment procedures (DLI 2.3). 5. At least 80 percent of priority vacancies have been filled in accordance with the Annual Recruitment Plans and the approved recruitment procedures (DLI 2.4). 6. At least 90 percent of priority vacancies have been filled in accordance with the Annual Recruitment Plans and the approved recruitment procedures (DLI 2.5). 7. Appropriate guidelines for performance contract management for civil Servants in Grades 11 and above in all Ministries have been prepared by the HRMO and approved by the CSSC (DLI 3.1). 8. All pilot Ministries have completed one annual cycle of the performance appraisal process for civil servants in Grades 11 and above (DLI 3.2). 9. Supervisors in pilot Ministries at the central, regional and district levels are trained in performance appraisal skills (DLI 3.3). 10. At least 75 percent of Annual Performance Reports for civil servants in Grade 11 and above in pilot Ministries are of appropriate quality (DLI 3.4). DLIs not achieved 11 All civil service jobs have been evaluated in accordance with a Job Evaluation Scheme and assigned to a grading structure, as approved by the Civil Service Steering Committee (CSSC) - DLI 1.2. 12 One hundred percent of civil servants in Grades 6 and above are paid in accordance with the approved 7 The ICR team was never able to locate the processed Project Paper for the first restructuring; as such, it worked under the assumption that the Draft Paper found in the Operations Portal was the final version. 8 Annex 3, Table 3.1 details the dates and amounts of DLI disbursements, and Table 3.2 lists the DLI transfer amounts by date and type. Page 11 of 72 The World Bank SL Public Sector Pay & Performance (P128208) pay structure - DLI 1.3. 13 The LTA Mainstreaming Policy and Action Plan have been implemented - DLI 2.6. Note: CSSC= Civil Service Steering Committee; DLI= Disbursement-linked indicator; HRMO = Human Resources Management Office; LTA= Local technical assistant; PSC = Public Services Commission. 15. The TA intended to support the entire program was to be monitored by the Results Framework; therefore, no indicators were specifically assigned to Component 2. A significant part of the TA, for instance, went toward the establishment and completion of job evaluations, which was IRI 2 under Component 1. Components 16. The project was designed around two components. The first component was composed of three interrelated subcomponents that used results-based financing to support the implementation of program activities from the GoSL’s PSR. The second component involved TA in support of implementation of the first component (Table 2). Table 2. Project Components and Subcomponents Component 1: Support to the GoSL’s Reform Program (US$15 million)  Subcomponent 1.1: Pay Reform (3 DLIs; US$3 million). The objective was to: (a) attract and retain key professional and managerial staff, and (b) motivate all civil servants to perform their jobs to an acceptable standard.  Subcomponent 1.2: Recruitment and Staffing (6 DLIs; US$6 million). The objective was to create a capable civil service of the right size and job composition to deliver its core functions as assigned by the Government.  Subcomponent 1.3: Performance Management and Accountability (6 DLIs; US$6 million, revised to 4 DLIs in 2014; US$4 million) The objective was to: (a) improve the performance and productivity of ministries and individual civil servants, (b) increase citizen trust and confidence in the Government, and (c) strengthen the accountability of ministries to the executive and citizens. Component 2: Technical Assistance (US$2 million)  Subcomponent 2.1: Strategic Communications Program. Activities included developing and implementing an appropriate communications program to enhance the transparency of the PSR, among other things.  Subcomponent 2.2: Enhancing Social Accountability. Activities involved developing and implementing appropriate measures to enhance the accountability of the recipient’s civil service, including: (a) the public disclosure of PSR-related information, (b) the provision of a recruitment complaint handling mechanism, (c) the institutionalization of ‘Meet the Public’ forums on PSR, and (d) the preparation of Citizen’s Charters in all MDAs.  Subcomponent 2.3: Job Evaluation and Labor Market Survey. Activities included: (a) conducting a comprehensive job evaluation exercise covering all civil service jobs based on an appropriate factor- based scheme; (b) building skills within the civil service to conduct a job analysis and evaluation; (c) obtaining comparative pay data for a select number of civil service benchmark jobs; (d) examining the future demand and supply of skills in the economy needed by the civil service; (e) designing new pay structure(s) and a new grading structure for the civil service based on the information from the job evaluation and market survey; (f) developing an implementation plan for phasing in the new structure in accordance with priorities and wage bill constraints; (g) preparing administrative, maintenance, and Page 12 of 72 The World Bank SL Public Sector Pay & Performance (P128208) appeals procedures; and (h) proposing organization and staffing structures to maintain the job evaluation system.  Subcomponent 2.4: Implementation Support. Activities included providing technical advisory services to: (a) the PSC and HRMO for their proper implementation of Component 1, and the Public Sector Reform Unit (PSRU) for its proper implementation of the rest of Component 2; (b) monitoring, evaluation, and verification; (c) a short-term consultancy for procurement; and (d) additional, unanticipated support to implementing agencies for Component 1. 17. Under Component 1, the project supported the achievement of 10 DLIs — but with a total disbursement of US$11,974,840.529 (which includes 50 percent advances made against 3 DLIs which were not reported as achieved). In October 2013, there was a 50 percent advance disbursement against five Year 2 DLIs, with a total amount of US$2.5 million. One of these five DLIs was deemed ‘achieved’ and disbursed an additional US$500,000, for a total of US$1 million for the DLI. The tracking of the advances against the remaining four DLIs was inadequate. Changes in the first restructuring led to one of the four DLIs being dropped in the revised FA. At the close of the project, the three DLIs not achieved all had 50 percent advances disbursed against these DLIs, which were not subsequently adjusted. Table 3 shows this discrepancy between the number of DLIs achieved as compared to the amount of funds disbursed for achievement of DLIs. Annex 3 details this issue in more depth.10 Table 3. Disbursement for DLIs Achieved and Not Achieved Amount Disbursed to Amount Disbursed to Client Category of Disbursement Client against DLIs against DLIs not Achieved (in Achieved (in US$) US$) 10 DLIs achieved (US$1.0 million per DLI × 10) 10.0 million 11 3 Advances against DLIs not achieved (US$0.5 1.5 million million per DLI × 3) 1 Advance against DLI that was dropped (US$0.5 0.5million million per DLI × 1) Subtotal 10.0 million 2.0million Total Actually Disbursed in Component 1 (in US$) 11,974,840.52 Note: The total disbursed was marginally less than US$12 million due to exchange rate losses of US$25,159.48 during the first disbursement of US$3 million on achievement of three Year 0 DLIs. B. SIGNIFICANT CHANGES DURING IMPLEMENTATION Revised PDOs and Outcome Targets 18. The PDO and outcome targets were not revised throughout the life of the project. 10 Annex 3, Table 3.1 details the dates and amounts of DLI disbursements, and Table 3.2 lists DLI transfer amounts by date and type. 11 This includes both advances (where provided) and reimbursements for the DLIs. Page 13 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Revised PDO Indicators 19. The restructuring included changes to PDO indicators (Box 1), which were never updated within the World Bank system. This contributed to poor monitoring, as ISRs continued to reflect the original Results Framework. The revised results matrix in the 2014 Project Paper was adopted and used by the client, the PSRU. However, there was no systematic use, updating, and tracking of data in the Results Framework during the project duration. 20. Following the 2014 restructuring, a new PDO indicator was added, related to the recruitment subcomponent; one PDO indicator on the performance management subcomponent was dropped; and another changed to an IRI. Following the first restructuring, changes were made to the PDO indicators and further modifications to the IRIs, as indicated in Table 4. Table 4. Changes to PDO Indicators in the First Restructuring (May 2014) Recruits to priority vacancies after January 1, 2013, retained in civil service PDO indicator added All ministries’ performances against performance targets have been evaluated jointly PDO indicator dropped by the recipient and relevant non-state actors. Annual Performance Appraisal Reports of Civil Servants in Grades 7 to 10 in Pilot PDO indicator changed Ministries are of appropriate quality. to IRI Revised Components 21. The project components were never revised during the life of the project. Other Changes 22. The project’s closing date was initially extended by one year and DLIs were altered due to significant delays in achieving the Year 2 (2013) targets. On March 4, 2014, the GoSL requested a 12- month extension to the closing date, that is, from October 31, 2015 to October 31, 2016, as well as an alteration of the Results Framework. The Project Paper details the changes to the project, including: (a) an extension of the closing date for a period of 12 months; (b) changes to the DLIs, their time framework, and PDO and intermediate indicators (mentioned above); and (c) revisions to the cost categories between withdrawals to accommodate the altered DLIs and time frame. Expenditure categories within the TA component were collapsed into one single category, including an additional one for sub-grants. 23. The project’s closing date was subsequently extended by an additional year due to continued implementation delays, which were then exacerbated by the EVD outbreak. Following a formal request from the Government on March 26, 2015, a second Level 2 restructuring was processed, extending the project closing date from October 31, 2016 to October 31, 2017; no other changes were made to the project at that time. The extension was justified by the fact that many key activities were proceeding at a slower pace and with fewer participants due to public health measures resulting from the Ebola epidemic. The collection of data for the job evaluation survey was also delayed.12 12Project Paper Report No. RES18991 and April 23, 2015, Memo to the World Bank Regional Vice President to extend the closing date of the project. Page 14 of 72 The World Bank SL Public Sector Pay & Performance (P128208) 24. Following a third formal request from the GoSL on October 12, 2017, the closing date was extended for another eight months to provide the Government more time to achieve the four outstanding DLIs (1.2, 1.3, old 2.5/new 2.6, and new 3.4) before project closure. The GoSL had requested an extension to the closing date of 12 months, that is, from October 31, 2017 to October 31, 2018. However, the World Bank agreed to an extension of only eight months and processed a no-cost extension third restructuring on October 30, 2017.13 The stated reason for the continued implementation delays was, again, the earlier Ebola outbreak. In particular, the consulting firm contracted to conduct the job evaluation survey, KPMG, took significantly more time than planned to complete the work. There were additional, avoidable delays on the part of the GoSL in mobilizing counterpart funding of US$158,720 critical to the implementation of the job evaluation survey (this delay is explained in more detail in the Efficiency section). Rationale for Changes and Implications for the Original Theory of Change 25. As noted, the rationale for the no-cost extensions was the slow implementation of project activities. Reasons for this include the Ebola epidemic, delays in (or limited amounts of) counterpart funding disbursements (to support project-financed activities), and procurement-related delays regarding the consultant contract for the job evaluation survey. 26. Alterations to the PDO indicators during the first restructuring had direct implications on the original theory of change. The dropping of PDO Indicator 3 did not affect the project’s ability to achieve the PDO because PDO Indicator 4 also measured movement toward the ‘performance management’ sub- objective of the PDO (see Figure 2 below for a breakdown of the PDO by PDO indicator). As such, the objective was maintained. However, with the dropping of PDO Indicator 5, the project was left with no PDO indicator to measure the PDO dimension pertaining to ‘accountability’.14 II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 27. The PDO was consistent with the World Bank’s CAS Progress Report for Sierra Leone (FY10–13), which remained current at project closing. The World Bank, the International Finance Corporation (IFC), and African Development Bank’s (AfDB) Joint Country Assistance Strategy (JCAS) for the Sierra Leone Progress Report, published in 2012, reported on the GoSL’s new emphasis on ‘building capacity for improved governance and service delivery within government and within civil society.’ The PDOs were directly relevant to systematically addressing the issues of efficiency, capacity, and effectiveness of the 13Project Paper Report No. RES29587. 14The addition of the new PDO indicator—Recruits to priority vacancies after 1 January 2013 retained in civil service—further contributed to the measurement of the ‘increase staffing of middle and senior staff’ portion of the PDO, which was already being measured by PDO Indicator 2. Page 15 of 72 The World Bank SL Public Sector Pay & Performance (P128208) public sector as flagged by the GoSL. This was a key policy reform priority and was subsequently reflected in the CAS Progress Report. 28. The PDO is composed of four different sub-objectives that are rated to determine achievement of the overall objective. Overall, DLIs directly contributed to the PDO; however, they did not always link up to the PDO indicators. Figure 2 breaks down the PDO component parts. It also illustrates how there were no PDO indicators to measure Subobjective 3 after the restructuring. Table 5 details the sub- objective and overall PDO ratings. The project achieved 10 of the 13 DLIs. Of the five active PDO indicators by the end of the project, two were achieved, two were not assessed, and one was not achieved. Figure 2. Breakdown of PDO Sub-objectives Mapped to Indicators 29. The PDO remains very relevant at project closing. Although the project contributed toward a job evaluation survey, the competitiveness of pay in the Sierra Leone civil service remains a critical issue. Indeed, there is a need for a more harmonized pay and grade structure, which was not implemented during the project period. The GoSL continues to be concerned with increasing staffing, especially through a renewed commitment to recruit large numbers of teachers, health workers, and police, as indicated in the 2018 budget speech of the Finance Minister. While systems for individual performance management for Grades 6 and above of the civil service have been created through the project, the focus on ministry- level performance and further institutionalization of individual performance to support ministry-specific targets remain highly relevant. After the first restructuring, accountability systems through wider public engagement were inadequately addressed, and remain important. 30. In view of this, the relevance of the PDO is assessed as Substantial. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome Table 5. Breakdown and Summary of PDO Efficacy Ratings Overall PDO Sub-objective Rating Rating 1: To improve the competitiveness of civil service pay in Sierra Leone Negligible Modest15 15 Annex 1B presents more details about the data used to inform the efficacy analysis. Page 16 of 72 The World Bank SL Public Sector Pay & Performance (P128208)  New pay structure not finalized, salaries not commensurate with job responsibilities.  Increase in analytical skills and knowledge about remuneration, job, and labor market through completion of a job evaluation survey.  Fifty-seven LTAs ‘mainstreamed’ with former salaries undermining salary harmonization — instead of the project’s expected ‘mainstreaming’ of LTAs within a new pay and grade structure.  Authority to change payroll records transferred from the Accountant General Department (AGD) to the HRMO, but challenges remain regarding the accountability of the monthly payroll. 2: To improve the performance management of the civil service of Sierra Leone  Culture of performance management introduced in Grades 7+, Grades 1–6 launch expected in 2019.  Performance appraisal and promotions are more rule-bound and transparent. Modest  Performance appraisal compliance and quality are low, but sanctions have been useful.  The integrated system of performance management linking ministerial performance contracts with individual performance contracts or targets still does not exist. 3: To improve the accountability of the civil service of Sierra Leone  The evaluation of ministerial performance by non-state actors was dropped from the project.  Payroll integrity issues have led to greater fiscal space through wage bill savings. Modest  Grades 7+ recruitment and selection procedures are competitive and merit based.  Large numbers of civil servants are still being hired through non-PSC recruitment procedures. 4: To increase staffing of middle- and senior-level staff in the civil service of Sierra Leone  Institutionalization of recruitment prioritization through annual Substantial manpower plans before the announcement of jobs.  A total of 805 prioritized ‘critical vacancies’ in the ‘missing middle’ were filled. Page 17 of 72 The World Bank SL Public Sector Pay & Performance (P128208)  A total of 602 or 75 percent of these critical vacancies filled through the project remain active in the civil service as of September 2018 (PDO target was 80 percent retained PDO Sub-objective 1: To improve the competitiveness of civil service pay in Sierra Leone (Negligible) 31. The new pay structure has not been approved in the project period; as such, the project had no effect on ensuring that civil servants are paid more equitably and competitively . The structure is still pending approval by the CSSC. The main reason for the lack of progress on this front is that the GoSL does not have the fiscal space necessary to afford the subsequent increases in salary and allowance payments that would result from its adoption16— particularly given calls from the IMF and other development partners for Sierra Leone to control its wage bill. Thus, the GoSL’s civil service pay scale remains compressed, thereby discouraging much-needed professionals and managers from entering the service due to uncompetitive wages. The GoSL’s immediate priority is also to establish a Wages and Compensation Commission to enable pay harmonization (see annex 6 for details). 32. Analytical work, including the job evaluation and labor market surveys, have established a framework to allow for the future harmonization of the civil service pay and grade structure. The survey helped map the skillset available within the country in relation to those that will be needed by the civil service in the medium to long term. The satisfaction level of civil servants was assessed through an employee satisfaction survey, which then offered strategies to improve staff retention and productivity. Market data regarding remuneration were ascertained by the remuneration survey to benchmark comparable roles and responsibilities within the private sector and other public sector institutions. Simultaneously, the capacity of the GoSL was built to conduct future job analyses and evaluations. However, the project missed the opportunity to establish a Wage Commission as an interim step needed to implement pay harmonization — although project TA supported study tours specifically for this purpose to Ghana and Kenya. 33. The ‘mainstreaming’ of 57 LTAs17 before finalization of the pay structure led to the institutionalization of positions within the civil service that fall outside of the established grades and salaries (and are not in compliance with a new pay structure, as planned). 18 This influx of highly paid, long-term consultants into the civil service at their same salaries has paradoxically distorted pay harmonization. In 2017, the Ministry of Finance and Economic Development (MoFED) increased the salaries of many civil servants in Grades 11 and above to the high level of LTAs as a way of addressing this pay discrepancy. 34. The authority to change personnel records and payroll has been vested in the HRMO, but challenges remain. Effective since January 2014, this transfer of responsibility has increased the authority of the HRMO to maintain payroll, although it is still reliant on the AGD for database management and the 16 An amount of US$144.2 million was flagged as necessary to spend on implementation of a new pay structure in the medium- term PSR (2011–2015). 17 A total of 50 LTAs in the Ministry of Finance and Economic Development (MoFED); 5 in the PSRU; and 2 in the Ministry of Land, Country Planning, and Environment. 18 It was this aspect that prevented the World Bank from signing off on the achievement of this DLI. The Government’s explanation for moving ahead before finalization of a new pay structure (which remains outstanding) was that it had to move to formalize their contracts because it could not afford to potentially lose such high-value consultants. Page 18 of 72 The World Bank SL Public Sector Pay & Performance (P128208) generation of certain reports.19 Critically, mutual accountability between the AGD and the HRMO still needs further definition. Additionally, integrity measures have led to tangible improvements to payroll. However, such measures are no substitute for the integration of personnel and payroll systems to allow for automated data reconciliation. PDO Sub-objective 2: To improve the performance management of the civil service of Sierra Leone (Modest) 35. A culture of performance management has been introduced for Grades 7+ of Sierra Leone’s civil service, with plans to launch the system in phases to the remaining grades starting in 2019 . The project supported the strengthening of the HRMO’s Performance Management Unit. All civil servants Grades 11+ sign annual contracts, whereas those in Grades 7–10 have performance management appraisals with their supervisors. The project plans to introduce the Individual Performance Appraisal System (IPAS) to 12 pilot ministries were achieved. Also, the HRMO introduced performance management to all MDAs by offering training to a total of 657 supervisors (330 to pilot MDAs and 326 to non-pilot MDAs) (Table 6). Since 2017, the HRMO has also started offering on-site, on-demand training for managers in other public institutions outside of the civil service, including numerous state-owned enterprises (SOEs). Demand for this service is such that SOEs pay nominal fees to the HRMO’s Performance Management Unit—which was reconstituted by the project—for such training. These fees help to offset the associated costs. Table 6: Number of Supervisors Trained in IPAS Pilot MDAs Non-Pilot MDAs Total Number Trained 2014 43 0 43 2015 201 121 322 2016 86 0 86 2017/2018 92 114 206 TOTAL 422 235 657 Source: HRMO. 36. Performance appraisals, work plans, and promotions are now more rule-bound and transparent. The IPAS has phased out the opaque Annual Confidential Report. Staff performance appraisals have been standardized and reports are shared with employees. 37. The quality of and compliance with the IPAS is slowly improving due to the new rewards/compliance system. Some MDAs treat performance management as a compliance issue. In this context, some appraisal reports are also of poor quality. The training of supervisors in the usage of the 19 A meeting entitled “Payroll Management Issues” (June 25, 2013), chaired by the HRMO Director General, defined the complementary new roles of the HRMO and AGD regarding payroll management. It was decided that the HRMO will be responsible for: (a) entering all data regarding changes to payroll (including adequate supporting evidence for such changes); and (b) sending copies of the changes to the AGD for finalization. The internal auditor will monitor the payroll change and compare the payroll to the changes. The auditor will then note that the said change has not been tampered with and the AGD will check the change. The AGD will then run the payroll and describe the financial implications with a report to the HRMO. Page 19 of 72 The World Bank SL Public Sector Pay & Performance (P128208) IPAS is likewise hampered by resource constraints, with supervisors reporting that they are unable to complete their work due to limited MDA resources. The initial focus has been on ensuring greater compliance in adopting the IPAS through the incentive of an additional month’s salary. The HRMO and PSC have worked together to ensure compliance by requiring performance appraisal reports before processing promotions. The HRMO, in conjunction with the CSSC, has established six criteria with which to objectively judge the quality of appraisal reports. Even after the corresponding DLIs were achieved, the HRMO continues to provide external quality assurance to performance contracts for Grades 11+, which are conducted by the Cabinet Secretariat.20 38. Ministerial performance contracts have not been integrated into the civil service performance management system due to a political decision to keep them separate. The project dropped activities around ministerial performance contracts based on a March 2014 request from the GoSL that the State House preferred to keep ministerial contracts separate from the project. In 2018, the new government reconstituted the Performance Management Directorate, and is in the process of relaunching ministerial performance contracts. PDO Sub-objective 3: To improve the accountability of the civil service of Sierra Leone (Modest) 39. As the primary activities contributing to achievement of this sub-objective were removed from the project during the first restructuring, the project did little to help achieve this sub-objective . Non- state actors are not evaluating ministries’ performance jointly with the GoSL, which was a PDO indicator before the first restructuring. 40. Improved accountability of the civil service was partially achieved through payroll integrity activities funded by the European Union (EU), and were buttressed through project activities under Sub- component 1.1. The transfer of payroll management to the HRMO from the AGD led to greater oversight of payroll and a clearer, more direct line of accountability. The project-supported payroll integrity measures have led to regular cleanings of the payroll, as well as deletions of individuals who should not be getting paid. These measures helped to increase the government’s fiscal space (see annex 4, Efficiency Analysis, for more details). 41. The development and increased use of the Merit-Based Recruitment Handbook has helped to institutionalize open, competitive, and merit-based recruitment and selection procedures for Grades 7+. This Handbook, developed in 2012 by the PSC in close collaboration with the HRMO, replaced an unsystematic and opaque recruitment system. There are now clearly established rules governing the advertising of vacancies and short-listing of candidates through an objective matrix; a double-blind written test (for Grades 6–8) developed and graded by independent adjudicators; an interview process; and reference and background checks before an offer of appointment is made. MDAs with a formal vacancy position liaise with the PSC to jointly develop the terms of reference for the open position. The Handbook also establishes the right of candidates to complain if they feel that they have been 20 For feasibility reasons, the HRMO decided not to evaluate the quality of performance reports for civil servants in Grades 7–10. Instead, it limited their intervention to boosting the capacity of managers and supervisors to engage in performance management in their respective MDAs. In response, the project dropped these activities during the restructuring. The HRMO does, however, monitor compliance with the IPAS. It has found numerous challenges with the adoption of the system, including inadequate resources to implement targets, a lack of clear job descriptions and schedule of duties for staff, and heavy workloads because of understaffing across MDAs in the regions (June 2018, HRMO Monitoring Report for the regions). Page 20 of 72 The World Bank SL Public Sector Pay & Performance (P128208) discriminated against or unjustly treated. In this regard, the PSC is required to retain all application documents for 12 months for monitoring purposes. Finally, the PSC has reported that it uses this Handbook for all hires. 42. The impact of this open, competitive, and merit-based recruitment system remains limited for two reasons. First, the HRMO, not the PSC, is the authority that recruits officers in Grades 1–5. Although it relies on the Handbook, it does not systematically use it for all recruitments.21 Second, there are numerous MDAs with their own service commissions responsible for recruitment, such as the Ministries of Health and Education, the Political Parties Registration Commission, the National Revenue Authority, and the National Public Procurement Authority. These other authorities engaged in recruitment do not adhere to the recruitment guidelines of the PSC Handbook. Because large numbers are recruited outside of PSC guidelines (only a limited number of civil servants—significantly less than 16.2 percent, which is the total number of civil servants in Grades 6 and above—are recruited through the Handbook), the PSC currently plans to table a draft Public Service Act Bill. This Bill is designed to negate these other commissions. In this manner, all civil service recruitments will be centralized in one single institution. PDO Sub-objective 4: To increase staffing of middle- and senior-level staff in the civil service of Sierra Leone (High) 43. The project helped the GoSL to start prioritizing among civil service job vacancies, a critical measure given the Government’s limited resources. Before the project, there was no prioritization of MDA positions to be filled, exacerbating the country’s problem with the ‘missing middle.’ This had led to ad hoc, off-budget recruitments throughout the course of the fiscal year, as money became available. There was no prioritization of hires, which led to the opening of non-strategic recruitments at the expense of strategic ones. To achieve DLIs 2.3, 2.4, and 2.6, however, the GoSL had to provide the World Bank with a list of priority hires for a given year. This necessitated the development of lists of ‘priority appointments’ as part of the manpower plans by the HRMO and the MoFED.22 44. In consultation with the MDAs, the HRMO created a prioritized list — focused on technical and professional positions (for Grade 6 and above) — in the eight priority ministries identified in the previous government’s Agenda for Change. From these priority lists, the number of new hires would be based on the overall resource envelope from the MoFED, a result of AGD macroeconomic forecasts, and the wage bill projection (Table 7). There has been a hiring freeze since 2015. In this regard, the HRMO notes that this policy is still in effect through Manpower Hearings and Reports. 21 Although the Constitution of Sierra Leone (Act 6 of 1991) confers the power to recruit civil servants on the PSC, it was decided by the GoSL that the PSC should instead play an oversight role and delegate such responsibilities to the HRMO in phases, including its authority to recruit officers. The HRMO currently recruits Grades 1–5, but Grades 6 and above continue to be recruited by the PSC. 22 For Year 1 (2012), for example, of the 1,084 total requests for positions received by the HRMO, only 350 appointments could be created, given the availability of funds as reported by the MoFED. Page 21 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Table 7. Prioritization of Hires Target Number of Vacancies on GoSL Year Prioritized List of Hires Priority Hires for the Establishment List Project 2012 5,666 350 210 2013 5,111 350 280 2014 8,509 350 315 Source: Government of Sierra Leone, Manpower Reports (2012, 2013, and 2014). 45. The priority to fill vacancies in the ‘missing middle’ led to an increase—both in real terms and as a percentage—in the number of technical and professional staff within the civil service. Figure 3 shows the number of priority hires during the three first years of the project. Of the 805 priority recruits, 732 (91 percent) were still active civil servants as of March 2016. By September 2018, three months after the close of the project, 602 recruits (or 75 percent of the original recruits) were still active civil servants although the target of this PDO indicator was 80 percent of project-supported recruits retained. 46. Table 8 details the change in the composition of the civil service from 2008 to 2018 . The hiring freeze has limited the ability of the Government to continue filling vacancies in the short term, but promotions have helped improve the civil service’s grade composition. Page 22 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Figure 3. Number and Grade of Priority Hires into the ‘Missing Middle’ under the Project 23 Source: Adapted from HRMO data. Table 8. Composition of Civil Service: 2008, 2011, and 2018 2008 2011 2018 Category Grades Numbers % of Total Numbers % of Total Numbers % of Total Low 1–5 13,255 92.2 11,881 87.3 14,129 81.6 Middle 6–10 995 6.9 1,559 11.4 2,837 16.4 High 11 and up 134 0.9 177 1.3 204 1.2 Total 14,384 100.0 13,617 100.0 17,308a 99.2 Source: Adapted from PAD table 1, with 2018 HRMO data. Note: The total figure includes 138 civil servants categorized as ‘other’ (meaning either out of scale or at Grade 20). Justification of Overall Efficacy Rating 47. Given the ratings of the four PDO subcomponents, the project’s overall efficacy rating is Modest. It should be noted that the relatively low efficacy rating has much to do with the fact that, despite the significant macroeconomic effects of the twin shocks on the Sierra Leonean economy, the project nonetheless did not undertake in subsequent restructurings a critical revisiting of project activities and expected outcomes, especially related to pay reform. Given the severely limited fiscal space that the Government was faced with, pay structure reform should have been scaled back and replaced with a more feasible reform in order to ensure some sort of progress regarding addressing competitiveness of civil service pay. 23Due to difficulties in obtaining data about the breakdown of hires, not all 805 recruitments are represented here. Also, due to the restrictions on travel and gathering for the 2014–2015 DLI round, the World Bank accepted internal promotions (‘Upward Mobility Policy’) as a legitimate way of achieving the DLI target. See annex 5, Table 5.1 and Figure 5.1 for more information. Page 23 of 72 The World Bank SL Public Sector Pay & Performance (P128208) C. EFFICIENCY Assessment of Efficiency and Rating 48. Because the project focused on capacity building and TA, efficiency was determined mostly by implementation efficiency. It was not considered feasible to do a full economic analysis (the other half of an efficiency assessment) at project design for such a project. Consequently, no cost-benefit or financial analysis was conducted as part of project preparation and included in the PAD. Annex 4 includes more information about the ICR’s efficiency analysis, including planned versus actual costs, staff retention, and cost/time overruns. 49. The PAD centered the efficiency of using World Bank resources of US$17 million to help leverage the Government’s larger medium-term PSR program, totaling an anticipated US$203 million . The PAD noted that the World Bank’s modest US$17 million in resources (US$2 million of which was for TA) exhibited high value for money as it supported a far larger commitment by the GoSL of US$186 million toward the 2011–15 Medium-term PSR Program. 50. The ICR team was unable to determine how much the GoSL and its partners actually spent on the medium-term program; however, counterpart funding for the project remained small . The GoSL spent US$634,575.54 of counterpart funding for the seven agencies associated with the P&PP, 24 as well as an additional US$158,720 payment to KPMG to offset the exchange-rate related shortfall. In addition, the HRMO calculated that the monthly cost of 732 of the 805 ‘missing middle’ civil servants hired by the project in 2013–2014 totaled the equivalent of US$206,580 per month in net pay, which is just short of US$2.5 million per year. A very rough estimate of the overall amount of counterpart funding spent on the salaries of these civil servants during the project duration would amount to US$10 million. 25 51. One clear efficiency gain from the project has been the cleaning up of the payroll, which led to significant government savings. Payroll integrity activities were included as part of the project design. Specifically, these activities centered around building the capacity of the HRMO to take on management of the civil service payroll. However, it should be noted that these payroll integrity-related activities were also European Union (EU)-funded. In addition, a biometric census was supported through a separate World Bank project, the Public Financial Management Improvement and Consolidation Project (PFMICP). 52. According to HRMO data, considerable improvement has been achieved in maintaining the accuracy and credibility of the payroll following the project’s support to a HRMO payroll unit beginning in January 2014. With the transfer of the function on payroll integrity from the AGD to the HRMO, a number of payroll verification and reconciliation exercises led to substantial integrity gains through the weeding out of staff wastages (see Table 9), including through the establishment of an audit trail for all changes made to the payroll. 53. These integrity activities led to the average civil service payroll declining by 3.5 percent, 5.2 percent, 16.7 percent, and 3.6 percent for 2014, 2015, 2016, and 2017, respectively . Although the specific fiscal space created by these actions was not accessible, the HRMO estimated that total deletions For the specific breakdown of counterpart funding to implementing agencies, see Table 4.1 of annex 4. 24 Apart from the US$10 million in salaries and approximately US$0.8 million in a direct project payment, counterpart funding 25 would also include the annual budgets for the PSRU, the HRMO, and the PSC. Page 24 of 72 The World Bank SL Public Sector Pay & Performance (P128208) from payroll freed up the equivalent of US$796,687 in 2016, US$492,849 in 2017, and US$632,626 in 2018. Furthermore, taken cumulatively to date, this is the equivalent of US$4,008,383 in fiscal space over the past three years.26 Table 9. Annual Deletions from Civil Service Payroll (2014-2017) Year Reason for Deletion from Payroll Total 2014 2015 2016 2017 Deaths, resignations, dismissals, abandoned duty, secondment, and 320 552 2,340 375 3,587 so on Retirees 413 551 1,275 267 2,506 Total 733 1,103 3,615 642 6,093 Source: HRMO. 54. The project’s upstream interventions supported the creation of a more professional and effective civil service, which should lead to greater efficiency through improved sector performance and service delivery outcomes. While efficiencies were difficult to quantify, the project did help to generate substantial efficiencies for the GoSL by putting into place incentives to attract capable individuals into the GoSL’s civil service, including rewards for strong performance. The project succeeded in developing open and merit-based recruitment practices, filling GoSL-identified ‘critical vacancies’ using approved recruitment procedures. The project also introduced a performance management framework culture into the civil service. 55. The project encountered substantial implementation delays that led to an extension of 32 months — despite no additional financing (or substantial restructuring). Although delays are not always inefficient, this project’s impact relied heavily on the timely sequencing of activities and outputs. For instance, the delayed implementation of the job evaluation survey created substantial inefficiencies by delaying delivery and finalization of the new pay structure.27 The client moved forward with the filling of vacancies—as well as the integration of LTAs into the civil service—without any integration into a new pay and grade structure. This was critical to the project’s theory of change toward the PDO objective of pay competitiveness for attracting new recruits.28 Recruitments into the ‘missing middle’ remained until the close of the project under the old structure. 26 For the specific breakdown of fiscal space created by deletions from payroll between 2016 and 2018, see Table 4.22 of annex 4. 27Annex 4 includes a detailed explanation of the delays to the job survey and resulting pay structure. 28The mainstreaming of LTAs under their existing salaries created substantial inefficiencies in terms of government expenditures related to the wage bill — not only because of the effect of these 57 individuals, but also with the subsequent harmonization of permanent secretary salaries that took place in 2017. Page 25 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Figure 4. DLI Disbursementsa Source: PAD, First Restructuring Paper, Client Connection Disbursement Summary. Note: a The Amendment to the Financial Agreement that resulted from the first restructuring post facto altered the amount from the third disbursement (Category 1c), under which US$2.5 million had already been disbursed (October 10, 2013). This explains why the actual curve shows a higher disbursement amount than the restructured curve. Actual disbursements also include 50 percent advances, as and when they were made. Figure 4 does not include TA disbursement. 56. The project’s end disbursement rate of 87.6 percent—even after the project’s extension by 32 months—is a sign of inefficient implementation. The fact that the project closed with the achievement of 10 of the 13 DLIs — despite being extended for an additional period of almost three years — calls into question the project’s overall implementation efficiency. 57. An additional issue concerns the over-reporting of eligible disbursements against DLIs, with US$2 million disbursed as 50 percent advances against unachieved DLIs. The project allowed for 50 percent advances against DLIs based on projected expenditures in accordance with the Eligible Expenditure Program. However, the project FA deemed withdrawals under Component 1 of the project conditional on the verification and/or satisfactory evidence that DLIs are achieved in order for disbursements to be deemed eligible expenditures. The efficiency rating of this project is directly affected as the eligible disbursements are lower than currently reported. 58. By project closure, 3 of the 13 DLIs (down from the original 15) were still not achieved. The third and final restructuring provided the time (eight months) for the GoSL to achieve DLI 3.4. The reasons for the non-achievement of three DLIs at closing are listed in Table 10. None of the DLIs are deemed scalable. Therefore, in accordance with the project FA, disbursements are only eligible against DLIs that are fully (and not partially) achieved. Table 10: Reasons for Outstanding DLIs DLI 1.2 At project closure, the CSSC had not yet finalized a new pay and grade structure. The new salary and pay structure was completed significantly later than planned, that is, in October 2017. These delays Page 26 of 72 The World Bank SL Public Sector Pay & Performance (P128208) were due to the fact that the DLI could only be achieved following the job evaluation survey and implementation of other key activities from the Pay Reform Strategy. DLI 1.3 This DLI ensures that 100 percent of civil servants in Grade 6 and above are paid in accordance with the approved pay structure. It was predicated on DLI 1.2, which stipulates that to implement the new job structure, it has to be finalized and approved first. There are currently no plans for this approval, given the fiscal implications of implementation of the new structure, as well as the GoSL’s immediate priority to first establish a Wages and Compensation Commission. DLI 2.6 The DLI was not met because the ‘mainstreaming’ of LTAs for existing salaries was not signed off by the PSC and integrated into an agreed upon pay and grade structure. The CSSC had approved the National Policy on the Absorption of LTAs into the Sierra Leone Civil Service in August 2016. Delays arose in ensuring the PSC Action Plan for LTA Mainstreaming, as the PSC expressed concern about retaining the high salaries of the LTAs. 59. In view of this information, the project’s efficiency is rated Modest. First, IDA funding was important in achieving key results in recruitment to the ‘middle levels’, thereby gaining fiscal space through improvements in the integrity of the payroll. However, the project did not leverage a larger, medium-term PSR program with over 90 percent of additional funding from the GoSL, as anticipated in the PAD. Second, implementation delays led to a number of inefficiencies. Third, despite 32 additional months, the project closed with the achievement of ten DLIs, three not achieved. D. JUSTIFICATION OF OVERALL OUTCOME RATING 60. The ICR team rates the overall outcome of the project as Moderately Unsatisfactory. While PDO relevance was Substantial, efficacy and efficiency were Modest, resulting in an overall rating of Moderately Unsatisfactory. E. OTHER OUTCOMES AND IMPACTS Gender 61. The project did not attain its PDO gender target of 30 percent women as direct project beneficiaries. At project closure, the PSRU had not evaluated the PDO indicator measuring the overall number of direct project beneficiaries, including the percentage of women. Table 11, developed by the ICR team, shows a gender breakdown of direct project beneficiaries based on available data of staff recruited in Grades 6 and above during the project period. Table 11. Gender Breakdown of Direct Project Beneficiaries Type of Beneficiary # Female # Male Total % Female PSRU core staff recruits 1 6 7 14 HRMO performance management recruits 2 10 12 17 HRMO human resource officer recruits 10 21 31 32 Priority hires (2013–2015) 132a — 805 16 LTAs mainstreamed 16 41 57 28 Page 27 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Supervisors trained in IPAS — — 657 — TOTAL At least 161 — 1,569 10 Source: ICR team and data from the PSC, the HRMO, and the PSRU. Note: a. Of the 732 individuals found to be still within the civil service as of March 2016, 132 were female. 62. A gender analysis of the project,29 prepared in late 2015, noted that one quarter of appointees were female. The breakdown was 22.4 percent, 22.5 percent, and 26.0 percent in 2014, 2015, and 2016, respectively. This is significantly lower than the overall percentage of female civil servants, reported as 45 percent. These lopsided appointments were identified too late to be corrected as part of the project’s recruitment of the ‘missing middle’. This was in part due to austerity measures put in place by the GoSL in 2016. It should be noted, though, that the analysis did inform government officials of the need to make a more concerted effort to recruit women into the civil service. Institutional Strengthening 63. The project built and strengthened a number of institutions. Within the HRMO, the Performance Management and Payroll Directorates were developed to oversee the introduction of a performance management system and payroll management, respectively. The PSC increased its capacity to engage in recruitment, and strengthened the GoSL’s recruitment procedures. The PSRU also became a permanent fixture within the civil service in 2018. The Cabinet Secretariat strengthened its role in performance management, particularly by establishing a strong working relationship with the HRMO regarding the better integration of performance contracts and IPAS, as well as ensuring of quality control. Other Unintended Outcomes and Impacts 64. The project was an early example of a results-based lending instrument being used as an incentive for the relevant stakeholders to work together for the achievement of DLIs; however, the impact was limited because there was no significant direct benefit to individual implementing agencies. To achieve the DLIs, collaboration was needed between various government agencies, including: the PSRU, the PSC, the HRMO, MoFED, the AGD, and the Cabinet Secretariat. However, representatives from both the PSC and the PSRU noted the lack of direct incentives in the results-based financing mechanism of the project. A critical assumption in the project design was that individual DLIs would sufficiently incentivize closer coordination within the fragmented government structure. During this ICR process, the client noted that the pay-off was a public good (that is, payments into the Consolidated Fund, which are indirect and non-excludable) and the achievement of the DLIs did not result in any proportional budgetary increase for the implementing agencies engaged in delivering the DLIs. In fact, counterpart funding made available to implementing agencies (US$634,575.52) was only a fraction of what was disbursed into the Consolidated Fund (approximately US$12 million through Component 1). 65. This project serves as the first (and, as of today, only) World Bank project in Sierra Leone to use country systems; this has led the Bank of Sierra Leone (BoSL) to start offering memorandum accounts. In 2012, the project was forced to open a leone-denomination account to use the BoSL. This was due in large part to the issue of exchange-rate-related shortfalls that occurred within this project during the first 29 PSC, 2015. Gender Analysis for the Sierra Leone Pay and Performance Project 2012–2015 . Page 28 of 72 The World Bank SL Public Sector Pay & Performance (P128208) disbursement of US$3 million as retroactive financing of achievement of three Year 0 DLIs when the project became effective. The BoSL started providing the option for development partners to open memorandum accounts, that is, leone-denominated accounts whose value in local currency was determined by the current U.S. dollar equivalent. For instance, if the leone depreciated, the amount of leones would automatically increase to match the value of the original U.S. dollar equivalent. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 66. Aligning with the medium-term PSR time frame contributed to an ambitiously short time frame for initial project DLIs. The project was designed to contribute to the achievement of the Government’s Medium-term Strategy (2011–2015). This, in turn, led to the overambitious nature and initial short-term length of the project for three years (2012–2015) that was not commensurate with the size of project finance. During this time, the GoSL was expected to complete 100 percent implementation of pay reform and deliver publicly accountable results of performance management contracts with the pilot ministries. 67. The project design choice of results-based financing using DLIs was very deliberate and justified as a risk mitigation measure. The task team noted during preparation that the PSR is generally difficult. As such, it was expected to be even more difficult in a weakly institutionalized and under-performing system such as the one in Sierra Leone. The choice of a results-based instrument was primarily premised as a mitigation measure against this real risk of slow or weak progress. During initial preparation (Project Concept Note [PCN] review), the Program for Results (PforR) instrument was also considered. However, the risks were considered too high because the PforR was approved by the World Bank Board at the same time. Thus, there were no assessments yet from the first use of the PforR lending instrument. 68. The task team was guided to develop a Specific Investment Loan (SIL), which combined a results- based component complemented by a small but strategically targeted TA component . It was also based on the assessment that a traditional Investment Loan (IL) would not help in providing the fiscal space which the Government needed, as the resources would be effectively tied to inputs. 30 69. The political economy of pay reform was not sufficiently factored into the project design. The project preparation acknowledged the importance of the broader political economy of the country. The political consequences of substantially altering the GoSL’s civil service pay and grade scale was not however properly accounted for — especially if an absence of rapid economic growth (as projected at appraisal) did not generate the fiscal space for higher salaries as part of pay reform. Nor were the complications around altering the salaries of well-placed, highly paid project staff and government consultants (the LTAs), many of whom themselves were central to the LTA mainstreaming process, fully appreciated. The implementation of a new pay and grade structure also warranted the establishment of a Wages Commission (see annex 6) that the HRMO is presently requesting, which was not part of the project design. 30 Minutes of PCN Review, Regional Operations Committee (ROC) meeting, and peer review comments, archived in WBDocs. Page 29 of 72 The World Bank SL Public Sector Pay & Performance (P128208) B. KEY FACTORS DURING IMPLEMENTATION (i) Factors Subject to Government Control 70. Use of country systems. Three major country-system-related factors improved with time or have since been addressed (Table 12).31 Table 12. Challenges Addressed in the Use of Country Systems Issues Faced in Use of Country Systems Solution Exchange-rate related shortfalls due to the BoSL’s There are now memorandum accounts, which would requirement that the special account be in leones. prevent such shortfalls. Initial delays in receiving MoF clearances before Clarification that special accounts should not need processing of payment vouchers, as Integrated such clearances; additionally, the new administration Financial Management Information System (IFMIS) has has entrusted the AGD with the authority to sign off yet to be launched in the PSRU. on payments. Delays due to cash rationing of the Consolidated Fund. Special accounts should not be subject to the Government’s Cash Committee oversight. This issue was addressed once the AGD and BoSL understood that the project account held restricted, special purpose funds. 71. Lack of strategic leadership. Whereas inter-ministerial meetings at the technical level took place regularly, Leadership Team (LT) meetings, which were to be chaired by the Minister of Finance, 32 occurred infrequently. Greater high-level support and strategic leadership would have helped to ensure smoother and swifter project implementation. Most critically, it would have allowed for greater adaptability to the country’s changing fiscal and economic scenario. 72. Insufficient counterpart funding. The implementation of certain activities was delayed due to insufficient counterpart funding. Numerous implementing agencies informed the ICR team that the GoSL did not make available sufficient funds to engage in the project activities necessary to ensure achievement of the DLIs. For instance, the HRMO’s lack of funds to engage in complementary activities to implement the job survey by the contracted consultant firm (KPMG) caused considerable delays. The TA contract with KPMG supported by the project only covered fees for their services. However, the implementation of the job survey required the recruitment of job analysts, as well as costs for travel and quality assurance. This meant that the Government needed to allocate additional funds (Table 4.2, Annex 4). These counterpart funds were delayed until Q1 of 2017. 31 The project’s special account within the BoSL was in leones, and key consultant contracts were in U.S. dollars. Funds to cover the KPMG contract were disbursed in 2015. However, the final deliverables—and subsequent tranche payments—were only completed in 2018. During this time, inflation and currency depreciation led to a budget shortfall within the TA component once the leones had to be converted back into U.S. dollars to pay the international consulting firm. 32 The team also included the Chairman of the PSC, the General Director of HRMO, the Accountant General, the Secretary to the Cabinet and the Head of the Civil Service, the Director of the Performance Service Delivery Directorate (Office of the Chief of Staff), and the PSRU Director. Page 30 of 72 The World Bank SL Public Sector Pay & Performance (P128208) 73. Weak M&E reporting. Inadequate monitoring and reporting of the project Results Framework meant that the modified DLIs, as well as the advance payments against the DLIs before restructuring, were not properly tracked and addressed during project implementation. 74. Election-related disruptions. In the final year of the project, implementation of the pay reform- related DLIs was delayed in the run-up and immediately following national elections. (ii) Factors Subject to World Bank Control 75. Inadequate implementation support and supervision. The increased level of supervision necessary for the implementation support plan in the initial years was not provided. The first restructuring was unnecessarily complicated. While DLIs were the principal focus of the task team, reporting on IRIs of the Results Framework was done through ISRs. However, the ISRs were not updated to reflect the modified Results Framework of the PDO and intermediate indicators. As such, they were not tracked. 76. Addressing the lack of familiarity regarding the results-based financing modality . The nature of the instrument and the legal responsibilities for the client, as prescribed in the FA, were new for the World Bank team and for the client. Better guidance during implementation to properly monitor progress through the formally agreed indicators and the need for tracking disbursement of advances made against DLIs would have provided the opportunity to follow through on ineligible expenditures before project closure. 77. Need for adaptability to the country context. Significant changes to the Sierra Leone country context made it exceedingly difficult for the project to be able to deliver on the agreed-upon activities and DLIs, especially those related to pay reform. At the second project extension in 2016, the World Bank may have lost the opportunity to simplify the project design and better adjust DLIs to the exogenous shocks related to the fall in iron-ore revenues and the EVD outbreak. These events significantly impacted the treasury and the availability of fiscal space necessary for the upward revision of salaries. (iii) Factors Outside the Control of Government 78. The Ebola outbreak. The EVD epidemic caused a significant disruption in the functioning of the GoSL, including all of its projects, both because of the resources required by the coordinated response to combat the health and humanitarian crisis as well as the response itself (that is, quarantines, limits on gatherings in public spaces, and so on). The resulting need for more health care professionals also led to a significant influx of workers, increasing the wage bill beyond earlier forecasts. 79. The collapse of iron ore revenues affected projected economic growth rates. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 80. The project’s M&E framework was established in the PAD, and was well developed in the PIM . The PDO and intermediate indicators were derived from the PSR’s Results Framework. The initial six PDO Page 31 of 72 The World Bank SL Public Sector Pay & Performance (P128208) indicators fully encompassed all parts of the PDO (see Figure 2). The IRIs encompassed the scope of activities necessary for achievement of the PDO. Activities undertaken through the TA/Component 2 of the project (12 percent of IDA financing) were expected to directly contribute to progress in the Results Framework, which covered only Component 1. 81. Lead implementation agencies were clearly identified and tasked with progress on Results Framework indicators. As the project implementation agency, the PSRU was tasked with supporting coordination around the overall monitoring of the Results Framework of the project. This involved coordination among the four key implementing agencies: the HRMO for pay reform (Subcomponent 1.1); the PSC for recruitment and staffing (Subcomponent 1.2); and both the Cabinet Secretariat and the HRMO for performance management (Subcomponent 1.3). The PSRU was primarily responsible for TA support and project management (Component 2), although the HRMO later took the lead in managing the large TA contract with the consultant (KPMG) regarding the job evaluation survey. The project budgeted for one M&E officer within the PSRU, but resources could have been used as needed under Component 2 to support M&E. 82. The PSRU was listed as the primary institution responsible for data collection for all PDO, DLI, and other intermediate indicators; data sources and secondary responsible parties were listed in the text, as well as in the Results Framework. The PSRU was to rely on M&E officers within each implementing agency for data collection to track the progress of each activity on the reform path. The PSRU, in turn, would conduct monthly monitoring exercises and collate data on the progress of activities and indicators for milestones. M&E Implementation 83. During project implementation, the PSRU received external M&E training implemented through the World Bank-managed Rapid Results Approach (RRA). As there was no project reporting on the Results Framework, the PSRU established a M&E Working Group (MEWG) in 2014. Table 13. M&E Reporting Mechanisms: Planned Versus Actual Reporting Planned Objective Actual Frequency Arrangement Frequency Leadership Team Monitor progress of DLIs (and report to Monthly (Year 1), Irregular and the President every quarter). Bimonthly infrequent afterwards Quarterly progress Current status of indicators according to Quarterly Quarterly; however, reports the Results Framework (detailed in M&E quarterly reports did reports, but synthesized within progress not include updates reports). on the status of the Results Framework MEWG meetings Support the collection and dissemination Quarterly Irregularly, meetings of information to facilitate evidence- were informal based decision-making and accountability across MDAs. Rapid Results Facilitate the timely implementation of 100-day time Weekly coaching and Initiative (RRI) teams key activities through the designation of frames presentation of goal goals, breakdown of tasks, development status, but marked Page 32 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Reporting Planned Objective Actual Frequency Arrangement Frequency of implementation/coordination teams, lags in targeted time and definition of time lines. frame Verification of DLI Collect and collate data and verify Every July As and when DLIs achievement whether DLIs were achieved. were reported achieved 84. All activities and the one PDO indicator associated with ‘accountability’ were dropped; as such, the PDO should have been revised during the restructuring to reflect the narrowing of project objectives. The EU-funded integrity activities did indeed lead to greater accountability. However, the original project design and documents made clear that the ‘accountability’ aspect of the PDO was to be measured by those inputs around greater transparency and institutionalization of non-state actor oversight, as well as feedback about ministerial performance contracts. Therefore, the sub-objective on the accountability of the civil service should have been removed when relevant project activities — related to ministerial performance contracts and their monitoring by non-state actors — were dropped (along with their DLIs). 85. The task team failed to capture in the World Bank system the changes made to project indicators in the 2014 restructuring; this was never addressed in either of the two subsequent restructurings that provided no-cost extensions to project duration. Therefore, none of the PDO indicators and DLI changes agreed upon in the first restructuring were reflected in the ISRs. 86. The capacity of the PSRU to engage in M&E implementation was low, and quarterly progress reports were inconsistent in quality and sometimes delayed (particularly in the beginning of the project). The PSRU’s M&E officer left in 2014 and was never replaced. The PSRU assigned the additional role of serving as project M&E officer to one of the Rapid Results Initiative (RRI) coaches. The first Project Paper noted that “one of the weaknesses of project implementation to date has been a narrow focus by the implementing agencies on the DLIs to the exclusion of the IRIs which are necessary steps to their accomplishment” (p 11). In general, the quality and timeliness of such reports were inconsistent, a fact that was flagged by the task team in the 5th ISR and elsewhere, but never fixed. 87. The RRA, introduced into the project in 2012, facilitated activity implementation (particularly regarding the eventual delivery of the job survey), but was unable to solve all implementation-related delays. The RRA is a practical operational approach for achieving implementation within a time frame of 100 days. RRI work plans, with milestones disaggregated by implementing agencies, were developed and implemented in parallel to the project’s existing Results Framework and M&E design. The RRI was lauded for helping improve the understanding of inter-agency linkages to achieving project results, particularly toward the beginning of the project.33 88. The tracking of DLI-related data and RRI-related tracking of activity implementation crowded out the monitoring of the Results Framework. Quarterly progress reports by the Government project team did not provide updates on the project’s Results Framework, and separate M&E reports were never 33Even with this tool, however, key project activities were not delivered on time. Of the 20 milestones in the last RRI Work Plan, ending in June 2018, five actions were outstanding and four were in progress, with eleven accomplished or ongoing (this last number includes the six that were completed, as well as the five ‘in progress’ activities that were ongoing activities). Page 33 of 72 The World Bank SL Public Sector Pay & Performance (P128208) developed. The PSRU ensured that the verification protocols for the DLIs were adhered to, and that the achievement of the DLIs was reported.34 However, the monitoring of non-DLI PDO and intermediate results indicators (IRIs), for example, was not included in quarterly progress reports. The sixth Aide Mémoire (June 2014) noted, “there is too much emphasis on detailed activities and not enough on results … there is no overall work plan for the project which shows when the intermediate results and final results (including DLIs) are planned to be achieved” (p 8). The final Government ICR shared with the World Bank does not report on the status of the PDO or the non-DLI IRIs after 2015. M&E Utilization 89. M&E did not appear to inform project management or decision-making processes. DLI-related activities were regularly monitored. However, the ICR team did not find evidence that the little M&E data that exists was used as a project management or decision-making tool to course-correct project implementation. No overall work plan for the project was found detailing when and how intermediate and PDO indicators were going to be achieved. 90. The project’s Leadership Team, which had been tasked to regularly monitor progress toward the achievement of the DLIs, convened inconsistently — and less frequently than had been planned. Chaired by the Minister of Finance and including representation from the three implementing agencies, the Leadership Team was supposed to meet once a month during the first year of the project, and at least every two months subsequently. Its objective was to monitor progress toward the achievement of the DLIs and project indicators and make necessary course corrections to ensure that the project was on track to achieve its PDO. In practice, though, the Leadership Team met much less, sometimes going nine months without convening. When it did meet, though, the Leadership Team did not routinely receive quarterly progress projects.35 91. It does not appear that M&E data have been used to offer evidence beyond the achievement of project outputs. The ICR team could not find evidence of project M&E being used to inform broader discussions regarding civil service reform or ways to improve the overall efficiency of the public sector. One of the exceptions has been information on the improvements achieved in the accuracy and credibility of the GoSL civil service payroll, a result of the ongoing payroll integrity activities supported by the P&PP. Justification of Overall Rating of M&E Quality 92. Project monitoring focused primarily on the DLIs; the project M&E framework was only partially implemented and poorly utilized. Changes to the DLIs during the first restructuring introduced added complexity to the Results Framework. However, the failure to enter these changes into internal World Bank systems made the monitoring of indicators even more complicated. Quarterly progress reports from 34 The PSRU did publicly share the status and achievement of the DLIs (as well as the status of implementation of key activities), as part of Subcomponent 2.2. However, this was done primarily through one-way communications in the form of a radio program (with no feedback mechanism). 35 “Quarterly progress reports do not appear to be routinely provided for the Leadership Team…The PSRU and the Leadership Team are not meeting regularly to review progress against the results framework for the project.” Sixth Aide Memoire (June 2014). “Since the second quarter of this year, the World Bank has been hammering the way they would like the PSRU as coordinator to monitor and report on the implementation of the project.” Seventh Aide Memoire (March 2015). “The Mission however recommends that future reports should include the status of the Project Results Framework.” Eleventh Aide Memoire (June 2017). Page 34 of 72 The World Bank SL Public Sector Pay & Performance (P128208) the client did not include updates on the project’s Results Framework, and the PSRU never reported on two non-DLI PDO indicators. 93. Based on this analysis, the overall quality of project M&E was rated Modest. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE 94. No significant environmental or social issues arose during project implementation . The project had received a Category C for environmental assessment purposes, in accordance with World Bank OP 4.01 (Environmental Assessment), as no social or environmental safeguards policies were triggered by the project. 95. Several fiduciary issues were noted.  Procurement. A procurement assessment during appraisal found that the PSRU had inadequate experience and capacity to conduct procurement activities under the project. The hiring of a part-time procurement specialist mitigated this risk, but a May 2017 Procurement Risk Assessment and Management System (PRAMS) scored project procurement as Moderately Unsatisfactory; the task team then downgraded the project’s procurement score to Moderately Unsatisfactory in February 2018. Procurement delays also contributed to a delay in activities, such as the hiring of the consultant for the job evaluation survey.  Financial management. Portfolio and Risk Management (PRIMA) assessments consistently rated the project’s financial management as Satisfactory, although with some delays in reporting. However, the following issues arose during the ICR: o The World Bank failed to track progress on advances to DLIs, a core compliance issue within the results-based financing modality. This became problematic given that advances were made against four DLIs that were either never achieved or cancelled during the first restructuring. o The annual external audit covered the expenditures undertaken within the TA Component 2 of the project. The annual audit of the budget by the Audit Service of Sierra Leone reported on the P&PP, but it did not include a distinction between disbursements recorded as advances against non-achieved DLIs and disbursements recorded as reimbursements on achieved and verified DLIs. C. BANK PERFORMANCE Quality at Entry 96. The World Bank contributed significant up-front investment in project preparation . The World Bank task team worked closely and extensively with the GoSL, as evident from the high levels of staff time and costs during project preparation (comprising five two-week missions between September 2011 and April 2012). This involved building trust between the MoFED, the HRMO, the PSC, and the PRSU. The World Bank team built upon analytical work on public sector challenges previously financed by the United Page 35 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Kingdom’s Department for International Development (DFID). 97. The project was expected to address coordination risks principally through the incentive of the DLIs. The project design addressed the significant identified risk of fragmentation of core civil service- related functions across multiple government entities through the adoption of a results-based mechanism. Sierra Leone’s public sector has a complex distribution of responsibilities and multiple reporting lines between the Cabinet Secretariat, the PSC, the HRMO, the PSRU, and the MoFED (Roseth and Srivastava, 2015), which were not directly addressed by the project. 98. The project design of 15 DLIs, equally weighted and uniformly priced, was aimed at bringing simplicity to the implementation of a new results-based instrument using country systems. The task team additionally front-loaded DLIs as a way of motivating implementing agencies: a total of US$3.0 million—three Year 0 DLIs—were designed to be disbursed as retroactive financing within the first few months of the project. The experience of project implementation suggests that the monetary incentives linked to the DLIs — further complicated by 50 percent advances made against the DLIs — may not have been sufficient to address coordination challenges in achieving the most challenging DLIs, which required greater coordination between several agencies. 99. At the design stage, the P&PP did not sufficiently factor in the practical guidance/training needed at implementation, both for the client and within the World Bank, in managing DLI-based disbursements and related advance payments against these DLIs . This was the first use of both country systems and results-based financing using DLIs at a time when the use of such a lending modality was new for the World Bank in the Africa region — and completely new for the country context and client. The PAD’s assumption that there would be a significantly increased number of World Bank missions and more intense supervision in the initial years did not happen in practice. The PAD was relatively weaker in practical guidance built into the project implementation design relating to controls and checks to be performed by the technical task team, the loan department, as well as required training. There was need for additional training/guidance both within the World Bank and for the client, especially when managing DLI-based disbursements (for advances and reimbursements). Quality of Supervision 100. The amount of staff weeks dedicated to project implementation was significantly less than the amount of supervision originally envisioned. In the PAD, it was estimated that project supervision would require 86 weeks of staff time. In practice, with 32 extra months of project implementation, a little over 92 weeks were charged by the time of project completion. Given procurement delays and implementation difficulties associated with achieving the DLIs, more staff time would have been required to ensure proper supervision. It is unclear, for instance, why only less than six weeks of staff time was charged to the project in FY17 (see Figure 2.1 in annex 2). There was no additional financing or changes to project components and activities following the first restructuring. Therefore, the 32 additional months should have necessitated additional ISR missions and supervision time. 101. ISRs were published regularly, but World Bank implementation support was not as intensive as originally planned. The task team had noted in the PAD that the project would need more intensive technical support and facilitation of intra-governmental communications, particularly during the first year of effectiveness. This did not happen in practice. Although no additional supervision budget was provided Page 36 of 72 The World Bank SL Public Sector Pay & Performance (P128208) to the task team in the initial years, actual supervision costs during later years of the project were also less than budgeted (for example, the FY17 supervision budget had a burn rate of 32 percent). Also, no in- depth Mid-Term Review (MTR) was carried out for the project, although the first restructuring occurred immediately after the fifth regular implementation support mission at the mid-point of the original project time frame. There were no ISRs between August 20, 2014 and May 14, 2015, coinciding with the worst of the Ebola epidemic. 102. The World Bank did draw attention to implementation challenges in project ISRs, but it was slow to revise downward the DO and IP ratings. Aide-Mémoires and ISRs flagged implementation challenges and delays, including delays in Leadership Team meetings, the lack of follow-up, and weak monitoring of the Results Framework. However, despite significant delays to key activities and DLIs, the World Bank maintained DO and IP ratings of Satisfactory until January 2017. This was after the project had extended its closing date twice because of slow project implementation that had led to direct risks of the project not being able to achieve its PDO indicators and DLIs (see Figure 4). Relatedly, the World Bank team only downgraded procurement to Moderately Satisfactory in August 2017, and then Moderately Unsatisfactory in February 2018.36 103. The World Bank team did not adequately update changes to the DLIs and the Results Framework that affected project reporting in the ISRs. The 2014 project restructuring introduced complications to the DLI framework that were not addressed during the project implementation period. The overall number of DLIs decreased from 15 to 13 — without explaining how or why the price for each DLI nevertheless remained, in effect, US$1 million for the remainder of the project.37 The revised FA also makes no note of whether the remainder of project funds, after reducing the number of DLIs from 15 to 13, was to be adjusted if each DLI remained valued at US$1 million. Aide-Mémoires and ISRs regularly mis- number PDO and intermediate indicators. This was partially the result of an overly complicated revision to the DLIs (for instance, swapping DLIs 2.5 and 2.6). The May 2016 Aide-Mémoire references a DLI (3.4) that had been dropped in 2014. 104. The World Bank team did not systematically track the 50 percent advances made against Year 2 DLIs throughout the project duration. In October 2013, the project disbursed US$2.5 million as 50 percent advances against five Year 2 DLIs. Subsequent to the first restructuring, the revised FA of May 1, 2014 dropped one of the DLIs which had an advance disbursed against it. The restructuring paper and the revised FA do not explain how the advance made against this dropped DLI was to be adjusted or refunded. The project did achieve one of the original Year-2 DLIs regarding recruitment to the ‘middle grades, and was reimbursed for the remaining amount of US$500,000. However, the 50 percent advances against the remaining three DLIs were never systematically tracked by the project team until the project closed. 105. The World Bank team did not report on the modified Results Framework following the first restructuring, and it did not sufficiently use the Results Framework in dialogue with the government during project implementation. The project had a clearly defined Results Framework and monitoring table of PDO level results indicators and IRIs. During the 2014 project restructuring, there were 36The PRAMS Assessment of May 2017 rated project procurement as Moderately Unsatisfactory. 37In reducing the number of DLIs from 15 to 13, there were three options for adjustments: (a) reduce the US$15 million credit amount allocated to Component 1; (b) reallocate the balance among the remaining DLIs, thereby increasing the individual price of DLIs to US$1.25 million if they remained uniformly weighted; and (c) substantially increase the TA allocation in Component 2 by reallocating from Component 1. None of these options were selected. Page 37 of 72 The World Bank SL Public Sector Pay & Performance (P128208) modifications to the PDO level indicators, and additional IRIs were included. In fact, the restructuring paper notes that “one of the weaknesses of project implementation to date has been a narrow focus by the implementing agencies on the DLIs to the exclusion of the IRIs which are necessary steps to their accomplishment.” However, as of the time of this ICR, there is only weak evidence of the project systematically reporting on IRIs and PDO indicators of the modified Results Framework, such as the retention of staff recruited. 106. ISRs and Aide-Mémoires regularly noted lapses in project governance activities (Table 14). It is unclear if these notifications had an impact; as such, delays continued throughout most of the project. Table 14. Governance Reporting Arrangements Reporting Planned Objective Actual Frequency Arrangement Frequency Steering Committee Strategic leadership Quarterly Unknown for Public Sector (proceedings and outcomes Reform (SCPSR) reported to the President and meetings donors) Technical team Ensure timely project Monthly Monthly meetings implementation CSSC Approve key deliverables, and As needed As needed receive progress reports regarding project implementation Leadership Team Monitor progress of DLIs (report Monthly (Year 1), Irregular; reports were to President every quarter) then bi-monthlya disrupted in Year 1 by the elections, and did not occur monthly.b They became regular in 2015, but without high-level representation Note: a. In the 9th ISR Aide-Mémoire, the World Bank requested quarterly meetings; b. The 4th ISR mission noted that four months had gone by without a meeting. Justification of Overall Rating of Bank Performance 107. The overall rating of World Bank performance is Moderately Unsatisfactory. During project preparation, the World Bank processes for quality control, as well as the significant investments of additional mission costs and staff time, were important for project design. However, this project implemented a new design for the country (and even in the region) of results-based financing using DLIs, with the additional use of 50 percent advances disbursed against these DLIs before DLI achievement. In addition, country systems were used for disbursements. Therefore, the project required more careful supervision than a regular Investment Project Financing (IPF) project. 108. Key weaknesses during implementation pulled the overall World Bank performance rating down.  First, there was no explicit link between documented disbursements under Component 1 to a clear logic of the number of DLIs already verified as achieved. This also contributed to the Page 38 of 72 The World Bank SL Public Sector Pay & Performance (P128208) poor tracking of disbursement of advances against DLIs. Reducing the total number of project DLIs from 15 to 13 in 2014 — without any corresponding adjustments to DLI pricing — is a symptom of the inattention of World Bank supervision and management regarding the project’s DLI-related disbursements under Component 1.  Second, World Bank implementation support did not sufficiently address weak reporting on the progress of IRIs, and the detailed Results Framework. Also, it did not address how the TA rapid results activities sufficiently contributed to progress on the project’s Results Framework indicators.  Third, the project DO and IP ratings were downgraded from Satisfactory in January 2017, but these ratings should have been downgraded earlier. As such, the ISRs could have been used to better flag implementation challenges and weak achievement of DLIs to both World Bank management and to the client. This would have included raising the possible need for further restructuring of the DLIs from 2016 onwards, due to the changed country context from the time of project appraisal. D. RISK TO DEVELOPMENT OUTCOME 109. The risk to development outcome is Modest. In future, it is unlikely that the results achieved by the project will be reversed. A framework for performance management has been introduced within the GoSL’s civil service (IPAS); compliance and quality remain low, but appear to be increasing — including as prerequisites for promotions and pay incentives. The recruitment process, while temporarily suspended due to the hiring freeze, has nonetheless been modernized and systematized through prioritization of hires and professionalization through transparent and merit-based procedures and guidelines. In this context, the percentage of middle-grade civil servants compared to the lower grades has improved considerably since 2011 (at 16.4 percent, up from 11.4). The outcomes most likely to be at risk (due to fiscal constraints) include approval of a new pay and grade structure based on the job evaluation TA supported through the project. However, the Government did not approve a new pay reform, and civil servants were not paid based on the revised structure during the project duration. V. LESSONS AND RECOMMENDATIONS 110. The following key lessons and recommendations from the P&PP project could help inform design and implementation of similar projects in the future:  Particularly during MTRs, projects (and DLIs) need to be adapted to a significantly changed country context when the key assumptions at project appraisal have changed. Sierra Leone in 2015/16 was a vastly different country from when the project was initially designed. Exogenous factors disrupted the country’s civil service plans, belied fiscal assumptions and wage bill forecasts, and necessitated a rethinking of the project design assumptions regarding pay increases across the civil service through pay reform. The restructuring of the project DLIs in 2014, was before the exogenous shocks of Ebola and fall in iron-ore production. Project DLIs could have been reassessed in subsequent years given the changed country context, especially during the processing of two further extensions to the project timeline. This project would have benefited greatly had there been Page 39 of 72 The World Bank SL Public Sector Pay & Performance (P128208) a MTR undertaken by an independent and experienced civil service reform expert, which assessed the project to identify improvements necessary in project design, implementation arrangements, and areas for strengthening support from the Bank, and/or other proactive restructuring.  The overall size of a project, as well as of individual DLIs, should be commensurate with scope and ambition. In order to ensure the necessary buy-in from relevant stakeholders, financial resources need to be large enough to provide the incentive necessary for reform. Pay reform is an especially complex area with the P&PP having weighted the project financing less towards this project component compared to the level of ambition involved.  Clearly linking horizontal civil service institutional reform to vertical sector ministry outcomes can demonstrate a more tangible results-based focus of such reforms on outcomes. The reform areas addressed by the P&PP were very relevant and continue to remain so. Links to vertical ministry performance in pilot ministries, while present in the original design, became much weaker after the first 2014 project restructuring. Consequently, results-driven DLIs, such as increased recruitments and performance of individual civil servants in Grade 11+ were not linked to any demonstrable improvements of core service performance mandate of any vertical line ministry or specific sector institutional performance outcomes.  The use of a results-based financing instrument combined with the use of country-based systems, especially for the first time, require additional caution, guidance, and training (applicable for both World Bank staff and for government counterparts). Project preparation and design can be technically sound for DLIs, with legal safeguards built into the FA. However, project design may still underestimate the familiarization with rules and controls for eligible disbursement against DLIs, including for tracking and documentation of advance payments. Such rules and controls need to be followed during implementation by World Bank staff and the government. For a project using country systems for the first time, additional reviews may be necessary at the project preparation stage. These could include the potential for exchange-rate-related losses, or clarity about the functioning of a special account within the Consolidated Fund.  Parallel TA and enhanced supervision could have helped overcome collective action problems and coordination challenges hampering critical reforms from progressing. Additional supervision was envisaged during project design, but unfortunately never materialized. Such supervision could have helped ensure greater project results, as the Bank can effectively play an important role of external broker to help overcome collective action problems and mobilize relevant stakeholders together around the need to engage in critical reforms.  Disbursements based on individual DLIs should not be complicated by withdrawal limits set by the grouping of DLIs. The creation of subcategories of withdrawal amount limits by grouping DLIs, as done in the original P&PP FA, can create unnecessary confusion in documenting disbursements disconnected to DLIs (Annex 3; Table 3.3). This is especially the case when the withdrawal limits do not account for additional advances disbursed against a future group of DLIs. The unnecessary focus on withdrawal subcategory limits (and concerns of overdrawing against these) can distract from a sole focus on tracking and documenting individual DLI-based advances and reimbursements based on the verification protocol. Page 40 of 72 The World Bank SL Public Sector Pay & Performance (P128208)  When DLI disbursements are made into the Consolidated Fund, the pricing of challenging DLIs that require the coordination of multiple agencies is critical — and may need additional monetary incentives built into the DLIs for participating agencies. In the P&PP, uniformly weighted DLIs did not sufficiently acknowledge more challenging coordination or the higher level of commitment required to achieve certain individual DLIs. Results-based financing may not in itself address coordination risks, as assumed during P&PP design. In this regard, additional mitigation measures may be needed, including prioritizing the functioning of leadership arrangements to support coordination outlined in the project implementation plan.  Non-technical aspects and risks to the reform path need to be explicitly identified in the project design. Beyond the country-level political economy factors, the P&PP did not sufficiently account for political economy factors surrounding the implementation of a new pay and grade structure — especially if the economic growth and fiscal space projections at the time of project appraisal did not hold true during implementation. The pricing of DLIs also need to account for additional weight given to those that are more politically challenging.  The Results Framework and intermediate indicators can be especially relevant in ensuring the internal logic of sequencing in the reform path. In the case of the P&PP, greater focus on the Results Framework and intermediate indicators may have better highlighted the issues of fiscal space, for instance, in realistically assessing the feasibility of pay reform. Conversely, it is important for DLI- based projects not to be subsumed by a singular focus of reporting on DLIs alone.  Technical assistance for RRIs needs to be consistently grounded in relation to the logic of progress against indicators within the Results Framework. The RRI was widely seen as instrumental in ensuring the implementation of a number of specific tasks/activities within the P&PP. However, linking these short-term activities to broader results through the project’s Results Framework could have better determined the relevance of the RRI tasks for project results. . Page 41 of 72 The World Bank SL Public Sector Pay & Performance (P128208) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To improve the competitiveness of civil service pay in Sierra Leone Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Civil Servants in Grade 6 and Percentage 0.00 100.00 100.00 0.00 above are paid in accordance with the approved Pay 12-Mar-2012 30-Jun-2015 31-Oct-2017 29-Dec-2017 Structure. Comments (achievements against targets): Not achieved Objective/Outcome: To improve the performance management of the civil service of Sierra Leone Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Annual Performance Reports Percentage 0.00 80.00 80.00 100.00 for Civil Servants in Grade 11 and above on Performance 20-Apr-2012 30-Jun-2015 31-Oct-2017 29-Dec-2017 Contracts in Pilot Ministries are of an appropriate quality. Page 42 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Comments (achievements against targets): Achieved Objective/Outcome: To improve the accountability of the civil service of Sierra Leone Objective/Outcome: To increase staffing of middle- and senior-level staff in the civil service of Sierra Leone Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Priority Vacancies filled in Percentage 0.00 90.00 90.00 100.00 accordance with the Annual Recruitment Plans and the 12-Mar-2012 30-Jun-2015 31-Oct-2017 29-Dec-2017 Recruitment Procedures Comments (achievements against targets): Achieved Unlinked Indicators Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Direct project beneficiaries Number 0.00 1800.00 1800.00 1800.00 12-Mar-2012 30-Jun-2015 31-Oct-2017 29-Dec-2017 Female beneficiaries Percentage 0.00 30.00 30.00 40.00 12-Mar-2012 30-Jun-2015 31-Oct-2017 29-Dec-2017 Comments (achievements against targets): Achieved Page 43 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Annual Performance Percentage 0.00 80.00 80.00 0.00 Appraisal Reports of Civil Servants in Grade 7 to 10 in 12-Mar-2012 30-Jun-2015 31-Oct-2017 29-Dec-2017 Pilot Ministries are of appropriate quality. Comments (achievements against targets): Not achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion All Ministries’ performances Yes/No N Y Y Y against Performance Targets have been evaluated jointly 31-Dec-2012 30-Jun-2015 31-Oct-2017 29-Dec-2017 by the Recipient and relevant non-state actors Comments (achievements against targets): Achieved A.2 Intermediate Results Indicators Component: Improve competitiveness in civil service pay Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 44 of 72 The World Bank SL Public Sector Pay & Performance (P128208) HRMO has conducted and Yes/No N Y Y Y completed a remuneration survey 12-Mar-2012 30-Jun-2015 31-Dec-2015 14-Feb-2015 Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Job evaluation set-up Yes/No N Y Y Y completed 12-Mar-2012 30-Jun-2015 29-Jun-2018 29-Dec-2017 Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion All Civil Service jobs have Yes/No N Y Y Y been evaluated in accordance with a Job 12-Mar-2012 30-Jun-2015 29-Jun-2018 29-Dec-2017 Evaluation Scheme and assigned to a Grading Structure approvedby the CSSC. Comments (achievements against targets): Achieved Page 45 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion New civil service pay Yes/No N Y Y N structure and phased plan for implementation 13-Mar-2012 30-Jun-2015 29-Jun-2018 29-Dec-2017 approved by Civil Service Steering Committee and Ministry of Finance & Economic Development Comments (achievements against targets): Not achieved Component: Increase recruitment of middle and senior staff to the civil service Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion The appropriate open, Yes/No N Y Y Y competitive and merit-based Recruitment Procedures 12-Mar-2012 30-Jun-2015 31-Dec-2012 23-Oct-2012 have been designed by PSC in collaboration with HRMO and approved by CSSC. Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Page 46 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Recruitment strategy Yes/No N Y Y Y approved by Civil Service Steering Committee 12-Mar-2012 30-Jun-2015 31-Dec-2012 30-Mar-2015 Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Seven staff recruited to PSC Number 0.00 7.00 7.00 7.00 and trained to manage recruitment and selection 13-Mar-2012 30-Jun-2015 31-Dec-2015 23-May-2016 Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Annual recruitment plan Yes/No N Y Y Y developed by HRMO 13-Mar-2012 30-Jun-2015 31-Dec-2015 03-Sep-2015 Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion LTAs have been integrated in Number 0.00 0.00 60.00 0.00 Page 47 of 72 The World Bank SL Public Sector Pay & Performance (P128208) accordance with an approved 13-Mar-2012 30-Jun-2015 31-Oct-2017 16-Jun-2017 Mainstreaming Policy and Action Plan. Comments (achievements against targets): Not achieved Component: Improve performance management and accountability Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Appropriate guidelines for Yes/No N Y Y Y staff performance appraisal for civil servants in grade 13-Mar-2012 30-Jun-2015 31-Dec-2012 23-Oct-2012 seven to 10 jobs in all ministries prepared by HRMO and approved by Civil Service Steering Committee Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Staff in MDAs HR PM units Number 0.00 22.00 22.00 22.00 recruited and trained for performance management 13-Mar-2012 30-Jun-2015 31-Dec-2013 23-Oct-2013 (disaggregate by recruitment and training) Page 48 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Staff in MDAs HR PM units Number 0.00 22.00 22.00 22.00 recruited 13-Mar-2012 30-Jun-2015 31-Dec-2013 23-Oct-2012 Staff in MDAs HR PM units Number 0.00 22.00 22.00 22.00 trained for performance management 13-Mar-2012 30-Jun-2015 31-Dec-2013 14-Feb-2014 Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Staff in HRMO PM unit Number 0.00 12.00 12.00 12.00 recruited and trained for performance management 13-Mar-2012 30-Jun-2015 31-Dec-2013 23-Oct-2012 (disaggregated by recruitment and training) Staff in HRMO PM unit Number 0.00 12.00 12.00 12.00 recruited 13-Mar-2012 30-Jun-2015 31-Dec-2015 14-Feb-2014 Staff in HRMO PM trained Number 0.00 12.00 12.00 12.00 for performance management 13-Mar-2012 30-Jun-2015 31-Dec-2015 14-Feb-2014 Comments (achievements against targets): Achieved Page 49 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Appropriate guidelines for Yes/No N Y Y Y Performance Contract management for Civil 13-Mar-2012 30-Jun-2015 31-Dec-2015 03-Sep-2015 Servants in Grade 11 and above in all Ministries have been prepared by HRMO and approved by CSSC. Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Publication of (i) all Yes/No N Y Y N Ministries’ Performance Targets; and (ii) achievement 13-Mar-2012 30-Jun-2015 31-Dec-2015 29-Dec-2017 against Performance Targets Comments (achievements against targets): Achieved Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion All Pilot Ministries have Yes/No N Y Y Y completed one annual cycle of the Performance Appraisal 13-Mar-2012 30-Jun-2015 31-Dec-2015 31-Oct-2016 Process for Civil Servants in Page 50 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Grades 7 to 10 Comments (achievements against targets): Page 51 of 72 The World Bank SL Public Sector Pay & Performance (P128208) B. KEY OUTPUTS AND OUTCOMES BY SUBOBJECTIVE: EFFICACY ANALYSIS Table 1.1. Efficacy Table on Key Indicators of the Results Framework Indicator Baseline Target Achievements Comments PDO Subobjective 1: To improve the competitiveness in pay of the civil service of Sierra Leone HRMO has conducted and completed a No Yes Achieved Remuneration survey delivered in 2012. remuneration survey (DLI 1.1) All Civil Service jobs have been evaluated in No Yes Not achieved Delivery is contingent upon finalization of the new pay accordance with a Job Evaluation Scheme and structure, which was never finalized. assigned to a Grading Structure approved by the CSSC (DLI 1.2) Civil Servants in Grade 6 and above are paid in 0% 100 % Not achieved: This indicator was predicated on approval of the new job accordance with the approved Pay Structure 0% structure, which was never approved (PDO Indicator 1, DLI 1.3) LTA Mainstreaming Policy and Action Plan have No Yes Not achieved The GoSL believes that it achieved this; the World Bank been implemented (DLI 2.5) does not, because LTAs were mainstreamed before finalization of the new pay structure (which is still pending) Finalization of new pay structure for GoSL civil Structure Structure Not achieved New pay structure, which was delayed by the job survey, service not finalized was never finalized by the CSSC developed Payroll System transferred from the Accountant No Yes Achieved The authority to change personnel records and payroll is General Office to HRMO now fully vested in the HRMO. Policy, Processes and Action Plan for appointing No Yes Achieved The CSSC approved the plans. LTAs to established posts approved by the CSSC PDO Subobjective 2: To improve the performance management of the civil service of Sierra Leone Annual Performance Appraisal Reports of Civil 0% 80 % Dropped This PDO indicator/DLI was dropped with the restructuring, Servants in Grade 7 to 10 in Pilot Ministries are as the HRMO realized that it did not have the resources to of appropriate quality (PDO Indicator 3, old DLI evaluate so many reports (instead, the HRMO limits its 3.3) interventions to training supervisors in MDAs on appraisal reports). Annual Performance Reports for Civil Servants 0% 80 % Achieved. This The HRMO continues to evaluate quality of Cabinet in Grade 11 and above on Performance revised to PDO Secretary’s performance contracts for civil servants in Contracts in Pilot Ministries are of an 75% Grades 11+ Page 52 of 72 The World Bank SL Public Sector Pay & Performance (P128208) appropriate quality (PDO Indicator 4, new DLI Indicator/DLI 3.4) was achieved. Appropriate guidelines for Performance Guidelines Guidelines Achieved Performance Contract Management Guidelines approved Contract Management for Civil Servants in not prepared by the CSSC. Grade 11 and above in all Ministries have been prepared. and prepared by HRMO and approved by CSSC (DLI approved. 3.1) Appropriate guidelines for staff performance No Yes Achieved Report forms, guidelines, and database finalized. appraisal for civil servants in Grades 7 to 10 jobs in all ministries prepared by HRMO and approved by CSSC All Pilot Ministries have completed one annual No Yes Achieved The Cabinet Secretary now oversees performance cycle of the Performance Appraisal process for contracts for all civil servants in Grades 11+ Civil Servants in Grade 11 and above (new DLI 3.2) All Pilot Ministries have completed one annual No Yes Achieved Performance appraisal process for civil servants in Grades cycle of the Performance Appraisal Process for 7–10 has been institutionalized and replaced the Annual Civil Servants in Grades 7 to 10 Confidential Report Supervisors in pilot Ministries at Central, 0% 80% in 12 Achieved This new DLI was not only achieved, but the HRMO Regional and District level trained in pilot expanded IPAS training to all ministries and certain SOEs. performance appraisal skills (New DLI 3.3) ministries SOEs pay HRMO a nominal fee for this service Staff in MDAs HR PM recruited and trained for 0 22 Achieved 31 HR staff hired and trained, of which 11 (35%) were performance management (disaggregate by female recruitment and training) Staff in HRMO PM unit recruited and trained for 0 7 Achieved 12 staff hired for PM unit, of which 2 (17%) were female. performance management (disaggregated by recruitment and training) All Pilot Ministries have agreed annual No Yes Achieved The HRMO rolled out annual performance targets to all performance targets for individual civil servants MDAs after achieving the 12 pilot ones. in Grade 7 and above PDO Subobjective 3: To improve the accountability of the civil service of Sierra Leone All ministries’ performances against No Yes Dropped The PSRU requested to drop related activities as State Performance Targets have been evaluated House opted not to combine ministerial performance jointly by the Recipient and relevant NSAs (PDO targets with the broader civil service performance Indicator 5) management. Page 53 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Publication of (1) all Ministries’ performance No Yes Dropped As mentioned above, the GoSL decided not to unite the targets; and (2) achievements against two separate performance management systems, so no Performance targets (Year 3) activities around this area were implemented Deletions from civil service payroll due to n.a. n.a. 3,587 civil Payroll integrity activities led to the deletion of 3,587 deaths, resignations, dismissals, abandoned servants individuals from payroll. duty, Secondment, and so on. removed Deletions from civil service payroll due to n.a. n.a. 2,506 retirees Payroll integrity activities led to the deletion of 2,506 retirement removed individuals from payroll. PDO Subobjective 4: To increase staffing of middle and senior staff in the civil service of Sierra Leone Priority Vacancies filled in accordance with the 0% 60% (Y1), Achieved: The HRMO and PSC prioritized hiring for the three years in Annual Recruitment Plans and the Recruitment 80% (Y2), Target hires question and successfully hired mid-level candidates to fill Procedures (PDO Indicator 2) 90% (Y3) were achieved the positions (210 in 2013, 280 in 2014, and 315 in 2015). for all three This process was halted following the hiring freeze (which years. remains in effect as on September 30, 2018). The gender breakdown is unknown, but well below 30%. Recruits to priority vacancies after 1 January 0% 80% Achieved as in The PSRU has not conducted analysis on the numbers since 2013 retained in civil service (new PDO March 2016, 2016; HRMO has been asked to re-run the numbers to Indicator) when it was extend the analysis to end of project (June 2018). determined that 90.9% of the 805 recruits (732) were still active civil servants. The appropriate open, competitive, and merit- No Yes Achieved The CSSC approved the new recruitment procedures. based Recruitment Procedures have been designed by PSC in collaboration with HRMO and approved by CSSC (DLI 2.1) Recruitment strategy approved by the CSSC No Yes Achieved The CSSC approved multiyear guidance for annual staffing plan. Seven staff recruited to the PSC and trained to 0 7 Achieved 12 staff were recruited and trained in the end; 2 of the 12 manage recruitment and selection (DLI 2.2) were female. Human resource plan based on MTEF budget No Yes Achieved The number of priority vacancies in annual recruitment ceilings prepared by HRMO plan reflects those that will be funded in the budget. Page 54 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Annual Recruitment Plan based on rigorous No Yes Achieved. The HRMO conducts manpower hearings with MDAs, priority setting developed by the HRMO develops Manpower Report on prioritized hires. Figure 1.1. Wage Bill: Actual and T+1, T+2, T+3, and T+4 Projections (in leones)a Source: Annual Budget Reports. Note: a. 2015 Budget Report missing Page 55 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Figure 1.2. Wage Bill: % of GDP and Forecast % of GDP at T+1, T+2, T+3, and T+4 (2011–2020) Source: Annual Budget Reports Page 56 of 72 The World Bank SL Public Sector Pay & Performance (P128208) C. STATUS OF DLIs Table 1.2. DLIs (Including Changes Following Restructuring) # Original DLI Alteration to DLI (if any) Following Restructuring Notes 1 DLI 1.1: HRMO has conducted and completed a remuneration survey - — 2 DLI 1.2: All Civil Service jobs have been evaluated in accordance with a - Not Job Evaluation Scheme and assigned to a Grading Structure approved achieved by the Civil Service Steering Committee (CSSC) 3 DLI 1.3 (PDO Indicator 1): 100% Civil Servants in Grade 6 and above are - Not paid in accordance with the approved Pay Structure achieved 4 DLI 2.1: The appropriate open, competitive and merit-based - — Recruitment Procedures have been designed by PSC in collaboration with HRMO and approved by CSSC 5 DLI 2.2: Seven staff have been recruited to PSC and trained to manage - — recruitment and selection 6 DLI 2.3 (PDO Indicator 2): At least 60% of Priority Vacancies have been - — filled in accordance with the Annual Recruitment Plans and the approved Recruitment Procedures 7 DLI 2.4 (PDO Indicator 2): At least 80% of Priority Vacancies have been - — filled in accordance with the Annual Recruitment Plans and the approved Recruitment Procedures 8 DLI 2.5: LTA have been integrated in accordance with an approved Swapped. New DLI 2.5: At least 90% of Priority Vacancies have Mainstreaming Policy and Action Plan been filled in accordance with the Annual Recruitment Plans and the approved Recruitment Procedures 9 DLI 2.6 (PDO Indicator 2): At least 90% of Priority Vacancies have been Swapped. New DLI 2.6: LTA Mainstreaming Policy and Action New DLI filled in accordance with the Annual Recruitment Plans and the Plan have been implemented 2.6 not approved Recruitment Procedures achieved 10 DLI 3.1: Appropriate guidelines for Performance Contract Management - - for Civil Servants in Grade 11 and above in all Ministries have been prepared by HRMO and approved by CSSC Page 57 of 72 The World Bank SL Public Sector Pay & Performance (P128208) # Original DLI Alteration to DLI (if any) Following Restructuring Notes 11 DLI 3.2: All Pilot Ministries have completed one annual cycle of the Old DLI 3.2 dropped. New DLI 3.2:a All Pilot Ministries have Performance Appraisal Process for Civil Servants in Grades 7 to 10 completed one annual cycle of the Performance Appraisal process for Civil Servants in Grade 11 and aboveb 12 DLI 3.3 (PDO Indicator 3): At least 80% of the annual Performance Old DLI 3.3 dropped. New DLI 3.3: Supervisors in pilot Appraisal Reports for calendar year 2014 for Civil Servants in Grade 7 to Ministries at Central, Regional and District level trained in 10 in Pilot Ministries are of appropriate quality performance appraisal skillsc 13 DLI 3.4 (PDO Indicator 4): At least 80% of the annual Performance Old DLI 3.4 slightly revised: New DLI 3.4: At least 75% of Reports for calendar year 2014 for Civil Servants in Grade 11 and above annual Performance Reports for Civil Service in Grade 11 and on Performance Contracts in Pilot Ministries are of appropriate quality above in Pilot Ministries are of appropriate qualityd 14 DLI 3.5 (PDO Indicator 5): All ministries’ performances against Droppede Not Performance Targets have been evaluated jointly by the Recipient and achieved relevant non-state actors 15 DLI 3.6: Publication of (1) all Ministries’ performance targets and (2) Dropped Not achievements against Performance Targets achieved Note: a. This was an existing intermediate indicator (but not DLI) before the restructuring. b. The rationale behind turning this existing intermediate indicator into a DLI was, “because it is a higher priority to implement performance appraisal at the management level (above grade 11)” (Project Paper, p 20); furthermore, “If management appraisal is introduced first, it provides a basis for cascading objective setting and appraisal at lower levels” (p 24). The original DLI 3.2 was shifted down to an intermediate results indicator and rescheduled for Year 3 (2014) due to “delays in the initiation of the new performance appraisal process, which in turn were caused by delays by MoFED in releasing resources for training. It has been removed as a DLI because it is now considered more important to incentive the introduction of appraisals for management grades (Grade 11 and above)” (p 24). c. The original DLI was dropped because it was not going to be achieved due to the explanation given for the revisions to DLI 3.2. d. The Project Paper did not explain the rationale for the downward revision to the percentage (from 80 percent to 75 percent). e. The Project Paper does not explain why DLIs 3.5 and 3.6 were dropped as DLIs. Interviews with key stakeholders, as well as the Minutes of the April 11, 2014 Meeting of the Technical Committee of the project, explained that these indicators were dropped, along with the respective activities around ministerial performance targets, because the State House Performance Management Directorate, which had already designed and rolled out such a system in 2008, was championing the system itself, complicating the project’s ability to integrate into the emerging civil service system. Page 58 of 72 The World Bank SL Public Sector Pay & Performance (P128208) ANNEX 2. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Supervision/ICR Smile Kwawukume Task Team Leader(s) Innocent Kamugisha Procurement Specialist(s) Sydney Augustus Olorunfe Godwin Financial Management Specialist Roberto O. Panzardi Team Member Fatu Karim-Turay Team Member Victor Boakye-Bonsu Team Member Adama Davida Taylor Team Member Allan Dunstant Odulami Cole Team Member Anita Bimunka Takura Tingbani Environmental Specialist Charles Ankisiba Social Specialist B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY12 27.135 327,626.22 FY13 0 0.00 Total 27.14 327,626.22 Supervision/ICR FY13 16.726 158,031.40 Page 59 of 72 The World Bank SL Public Sector Pay & Performance (P128208) FY14 15.023 139,516.44 FY15 18.767 142,140.54 FY16 21.426 148,503.86 FY17 5.775 32,465.30 FY18 14.500 85,141.13 FY19 7.850 56,146.85 Total 100.07 761,945.52 Figure 2.1. Staff Time and Cost Source: Operations Portal. Note: * FY12 wholly and exclusively encompasses project preparation costs. Page 60 of 72 The World Bank SL Public Sector Pay & Performance (P128208) ANNEX 3. DATES AND AMOUNTS OF DLI DISBURSEMENTS Table 3.1. Dates and Amounts of DLI Disbursements Actual Date Actual Date of Total Disbursed of Advance DLI Reimbursement to Client (US$, Comments (US$, (US$, millions) millions) millions) DLI 1.1: HRMO has conducted and 1.0 0 1.0 completed a remuneration survey March 1, 2013 DLI 2.1: The appropriate open, competitive, and merit-based US$3.0 million 1.0 Recruitment Procedures have been 0 1.0 disbursed March 1, 2013 designed by PSC in collaboration with together for all HRMO and approved by CSSC three Year 0 DLIs DLI 2.2: Seven staff have been 1.0 recruited to PSC and trained to 0 1.0 March 1, 2013 manage recruitment and selection DLI 2.3: At least 60% of Priority US$1.0 million in Vacancies have been filled in 0.5 0.5 advances for two accordance with the Annual March 14, October 13, 1.0 Year 1 DLIs made Recruitment Plans and the approved 2013 2013 together Recruitment Procedures DLI 3.1: Appropriate guidelines for US$1.0 million in Performance Contract Management 0.5 0.5 reimbursements for Civil Servants in Grade 11 and March 14, October 13, 1.0 for two Year 1 above in all Ministries have been 2013 2013 DLIs made prepared by HRMO and approved by together CSSC DLI 1.2: All Civil Service jobs have been evaluated in accordance with a Job 0.5 Evaluation Scheme and assigned to a October 10, 0 0.5 Grading Structure approved by the 2013 Three DLIs were CSSC never achieved DLI 1.3: 100% Civil Servants in Grade 6 0.5 but US$1.5 and above are paid in accordance with October 10, 0 0.5 million advances the approved Pay Structure 2013 made Old DLI 2.5 (New DLI 2.6): LTA 0.5 Mainstreaming Policy and Action Plan October 10, 0 0.5 have been implemented 2013 DLI 2.4: At least 80% of Priority Vacancies have been filled in 0.5 0.5 accordance with the Annual October 10, October 17, 1.0 Recruitment Plans and the approved 2013 2016 Recruitment Procedures Old DLI 3.2: All Pilot Ministries have 0.5 Dropped but completed one annual cycle of the October 10, 0 0.5 US$0.5 million Performance Appraisal Process for 2013 advance made Civil Servants in Grades 7 to 10 Page 61 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Actual Date Actual Date of Total Disbursed of Advance DLI Reimbursement to Client (US$, Comments (US$, (US$, millions) millions) millions) Old DLI 2.6 (New DLI 2.5): At least 90% of Priority Vacancies have been 0.5 0.5 filled in accordance with the Annual January 25, 1.0 June 28, 2016 Recruitment Plans and the approved 2016 Recruitment Procedures New DLI 3.2: All Pilot Ministries have 1.0 completed one annual cycle of the 0 October 17, 1.0 Performance Appraisal process for 2016 Civil Servants in Grade 11 and above New DLI 3.3:a Supervisors in pilot 1.0 Ministries at Central, Regional and 0 October 17, 1.0 District level trained in performance 2016 appraisal skills New DLI 3.4: At least 75% of annual Performance Reports for Civil Service 1.0 0 1.0 in Grade 11 and above in Pilot July 5, 2018 Ministries are of appropriate quality TOTAL 12.0 Note: a. As neither advances nor reimbursements were made for old DLIs 3.3 or 3.4, they were not included on this list. Likewise, as neither advances nor reimbursements were made for DLIs 3.5 or 3.6 before they were dropped, they were not included on this list. Table 3.2. DLI Disbursements by Transfer Amount and Date Transfer Amount Type Date Received Explanation No. (US$) 0016 1,000,000 Reimbursement June 27,2018 100% reimbursement for new DLI 3.4 October 17, 100% reimbursement for new DLI 3.2, new DLI 3.3; 0013 2,500,000 Reimbursement 2016 50% reimbursement for DLI 2.4 0011 500,000 Reimbursement June 28, 2016 50% reimbursement for new DLI 2.5 0010 500,000 Reimbursement January 25, 2016 50% advance for new DLI 2.5 October 10, 50% advances for DLIs 1.2, 1.3, old 2.5, 2.4, and old 0006 2,500,000 DA-A 2013 3.2 October 10, 0004 1,000,000 Reimbursement 50% reimbursement for DLIs 2.3 and 3.1 2013 0003 1,000,000 DA-A March 14, 2013 50% advances for DLIs 2.3 and 3.1 0001 3,000,000 1A March 1, 2013 100% reimbursement for DLIs 1.1, 2.1, and 2.2 Source: World Bank. Table 3.3. The final disbursement status by category & sub-category Category Category Description Amount in XDR 1 Eligible Expenditures Program under Part 1 of the Project (a) First Withdrawal 2,634,647.28 (b) Second Withdrawal 1,640,059.62 Page 62 of 72 The World Bank SL Public Sector Pay & Performance (P128208) (c) Third Withdrawal 743,356.62 (d) Fourth Withdrawal 1,293,317.40 (e) Fifth Withdrawal 1,252,178.73 (f) Sixth Withdrawal 709,793.73 Goods, non-consulting services, consultants' services, 2 1,380,375.45 Operating Costs, and Training under Part 2 of the Project DA-A* Designated Account (83,928.70) DA-B* Designated Account (886.12) Total Disbursement 9,568,914.01 Undisbursed balance cancelled as on October 31, 2018 1,431,085.99 Total Financing Amount 11,000,000.00 Page 63 of 72 The World Bank SL Public Sector Pay & Performance (P128208) ANNEX 4. EFFICIENCY ANALYSIS 1. As no economic or financial rate of return was conducted during project preparation, the ICR determined efficiency solely based on implementation efficiency. Implementation efficiency involves actual versus planned costs, the extent to which key activities were implemented on time, the extent of staff retention, the existence and amount of procurement issues and delays, and cost overruns, among others. 2. Actual administrative costs were slightly hired than originally planned. Financing for Component 2, TA, was revised slightly upward during the first restructuring (from SDR 1.20 million to SDR 1.29 million), when the number of DLIs and withdrawals was altered. As mentioned earlier, World Bank supervision costs in the form of staff weeks was higher than originally planned—92.22 weeks as opposed to 86; however, the project was extended from three years to almost six years. Supervision costs—staff time plus travel and consultant costs—totaled slightly less than US$700,000 (plus US$327,626 spent during project preparation). 3. Actual component costs were in line with estimated component costs, as the bulk of project activities were financed through DLIs. As mentioned above, the first restructuring did allocate a slightly larger amount to Component 2 than originally planned; however, this amount was minimal. 4. The fact that the project required 32 additional months to achieve what it set out to do suggests significant inefficiency. The Ebola epidemic certainly was unplanned and caused significant disruption; however, even after three no-cost extensions, the project was unable to achieve all 13 DLIs (which were, to be sure, down from an original 15 at project effectiveness). 5. The project placed an emphasis on filling critical vacancies in the public sector, leading to greater efficiencies in terms of the GoSL’s ability to engage in its everyday functions. While it is difficult to quantify, ensuring that competent individuals are brought into the civil service to fill critical professional and technical roles in ‘the missing middle’ is crucial to ensuring that necessary tasks are completed in a timely manner. Box 4.1. Delays in the Implementation of the Jobs Survey and Resulting Job Structure The jobs survey was supposed to be implemented early in the project. Part of the delay was due to the Ebola epidemic; however, that masks serious avoidable delays in preparation and procurement of the activity:  Insufficient counterpart funding was made available to the HRMO for logistics related to the job survey. Partial blame for this can be attributed to a misunderstanding of what would be covered by the vendor (that is, implementation costs) and what was expected from the Government counterpart.  There was initial disagreement as to the scope of the project (that is, whether to include teachers, military, and police).  The devaluation of the leone from when World Bank funds were disbursed for the jobs survey (February 4, 2015) to its actual delivery—in 2017, due to the reasons mentioned above—led to a shortfall of SLL 1,166,341,996 (US$159,722 equivalent) that prevented the PSRU from being able to pay the vendor. a The GoSL had to step in to incur the costs of the seventh tranche, of US$158,720. Page 64 of 72 The World Bank SL Public Sector Pay & Performance (P128208)  The resource shortfall to conduct the survey led to significant disagreement among implementing agencies about whether to repurpose counterpart funds to be reallocated to this output. Obtaining consensus on this issue delayed moving forward with the activity.  Contract negotiations with KPMG were postponed to allow the GoSL to address concerns raised on the evaluation methodology used in the process.  Ebola-related delays led to the lapse of the vendor contract before all deliverables were concluded, necessitating additional contract negotiations.  The CSSC delayed considering approval of the job structure in the run-up to the 2018 national elections, as many of the stakeholders were involved in campaigning. Also, it was deemed wise to allow the new administration a role in the selection and finalization of the new structure to ensure political buy-in. Note: a. The project had a special account with the BoSL in leones, as Sierra Leone’s IFMIS only recognized local currency figures at the time and there was no option for a foreign currency account (this is no longer the case, as since 2017 the BoSL provides the option of opening memorandum accounts that guarantee the U.S. dollar amount in leones). World Bank disbursements to cover TA activities would thus be immediately converted from U.S. dollars into leones. Given the country’s high rate of inflation, once the leones were finally converted back into U.S. dollars to pay KPMG for services rendered, the amount in the leone-denominated account was far less than needed to make the U.S. dollar-payment. To avoid more exchange rate-related losses, the project’s Disbursement Letter was modified on April 4, 2018 to include a fourth disbursement method, Direct Payment, to allow for the direct payments to vendors of contracts in U.S. dollars. The last three tranche payments to KPMG were thus paid directly from the World Bank through direct disbursement (of U.S. dollars, the currency of the contract). Table 4.1. GoSL Counterpart Funding for Pay and Performance Project (in US$) a Implementing 2014 (Q1) 2015 (Q1) 2015 (Q2) 2017 (Q1) Total Percentage Agency HRMO 77,543.66 10,116.23 62,296.40 127,568.34 277,524.64 49.60 PSRU 51,928.27 10,218.42 41,530.93 — 103,677.62 14.70 PSC 51,695.77 9,196.58 41,530.93 — 102,423.28 14.50 MoFED 25,819.63 3,372.08 20,765.47 — 49,957.18 7.10 Cab Sec 25,847.89 2,299.14 20,765.47 — 48,912.50 6.90 Office of Chief 25,643.64 2,299.14 20,765.47 — 48,708.26 6.90 of Staff AGD — 3,372.08 — — 3,372.08 0.50 Total 258,478.86 40,873.67 207,654.67 127,568.34 634,575.54 100.00 Source: PSRU and AGD. Note: a. Amounts converted into U.S. dollars based on the following BoSL exchange rates: US$1 = SLL 4,406.55 (2014); US$1 = SLL 4,868.66 (2015); and US$1 = SLL 7,303.15. Page 65 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Table 4.2. Fiscal Space Created by Deletions from Payroll (2016–2018) Year Number of Leavers Savings (SLL) Savings (US$) January 118 223,048,919 40,210.73 February 130 227,897,268 41,084.78 March 253 431,688,952 77,823.86 April 619 853,596,914 153,884.43 May 206 284,512,400 51,291.22 June 2061 1,683,491,655 303,495.88 July 87 136,485,180 4,605.22 August 122 362,028.19 65.27 September 107 154,950,890 7,934.18 October 120 142,050,516 25,608.53 November 103 169,933,551 30,635.22 December 64 111,202,574 20,047.34 Subtotal for 2016 4,419,220,847 796,686.65 January 26 41,272,160 5,499.29 February 98 128,983,563 17,186.35 March 71 213,533,251 28,452.13 April 141 225,967,533 30,108.93 May 54 73,543,771 9,799.30 June 116 259,407,604 34,564.64 July 44 83,278,304 11,096.38 August 75 1,947,385,983.00 259,478.48 September 27 83,808,705 11,167.05 October 34 249,407,805 33,232.22 November 34 202,570,905 26,991.46 December 29 189,670,806 25,272.59 Subtotal for 2017 3,698,830,390 492,848.82 January 413 1,988,983,538 247,017.33 February 142 288,580,556 35,839.61 March 517 791,084,969 98,247.02 April 113 374,178,153 46,470.21 May 104 171,829,660 21,340.00 June 288 846,571,437 105,138.03 July 110 204,762,470 25,430.01 August 158 427,910,912 53,143.43 Subtotal for 2018 5,093,901,695 632,625.65 Total (2016–2018) 13,211,952,932 1,922,161.12 Source: HRMO. Exchange rate from July 1 of calendar year used. Page 66 of 72 The World Bank SL Public Sector Pay & Performance (P128208) ANNEX 5. RECRUITMENT INFORMATION 1. The project was designed to support the GoSL deliver portions of its civil service reform strategy. However, the project did not attempt to achieve all of the results of the three areas from the strategy (namely, pay reform, recruitment and staffing, and performance management); it was assumed that those results outside of the project—directly linking the human resource database and payroll and basing promotions on open, competitive, and merit-based promotion procedures—would be independently pursued and achieved by the GoSL. 2. Over the years of the project (2012–2018), the GoSL recruited more civil servants than those counted toward achievement of the three recruitment DLIs, 805. Due to poor record keeping, the ICR team had difficulty determining the exact number of hires. HRMO provided data for 853 of such hires within Grades 6–10. Figure 5.1 categorizes these hires in grades and years. Figure 5.1. GoSL Recruits into the ‘Missing Middle (2012–2015) a Note: a. Number includes both external recruitments (710), as well as internal promotions (146) into the ‘missing middle.’ During this time, the Health Service Commission recruited a number of individuals, Grades 4–7, but not under the scrutiny and standards of the PSC; as such, those Grades 6–7 hires were not counted toward achievement of the DLI (see Aide Memoire 4 for more information). 3. Of these hires, 146 were internal (that is, promotions). In 2014, during the Ebola crisis, the client and task team decided that, due to travel restrictions and the risk of contagion that complicated the holding of interviews, internal promotions would be counted toward achievement of the DLIs. Table 5.1 breaks down these recruitments into internal versus external hires. Table 5.1. Total GoSL Recruitments to the ‘Missing Middle’ Years Internal External Total 2012 99 151 250 2013 0 181 181 Page 67 of 72 The World Bank SL Public Sector Pay & Performance (P128208) 2014 47 295 342 2015 0 80 80 Total 146 707 853 Page 68 of 72 The World Bank SL Public Sector Pay & Performance (P128208) ANNEX 6. PAY HARMONIZATION AND THE ESTABLISHMENT OF A WAGE COMMISSION 1. The Pay Reform Subcomponent of the P&PP intended to address disparities in the GoSL’s civil service pay system. Due in large part to the failure to finalize and implement a new pay structure, the objective of this subcomponent was not achieved. However, a number of other factors contribute to the continued disparity in pay across the public service. Box 6.1. HRMO Analysis of Factors Contributing to Disparity in Pay across the GoSL Public Sector  Job Evaluation Report not implemented.  Internal pay relativities within the pay structure are highly inequitable because the grading system in the current grade structure was not based on an empirical analysis of the jobs.  Similar posts and employees with similar qualifications and competencies in the civil service attract different pay scales due to ad hoc and uncoordinated arrangements involving multiple players such as the Ministry of Finance, the Re-grading Committee, HRMO, Office of the President, and so on.  The evolution of LTAs and/or contract staff in post-war Sierra Leone Public Service.  The LTAs/contract workers retained their salaries on a personal basis even on their absorption/mainstreaming into the permanent and pensionable establishment of the civil service.  Disparities in pay have also been further compounded by the fact that the Acts establishing some subvented agencies of the Government have given them the power to set their own pay without reference to any agency such as the PSC or HRMO to ensure pay relativism and equity is maintained.  Disparities in pensions paid to retirees due to the existence of several laws governing the administration of pensions, for example, Judges Act of 1983, Cap. 173 of the Laws of Sierra Leone (1960), NASSIT Act of 2001, and Teachers’ Pension Act of 1966.  NASSIT Act of 2001 did not repeal these existing laws, hence the payment of multiple pensions to retirees thereby putting pressure on the wage bill.  Disparities in pensions are alarming: there are pensioners receiving below SLL 20,00 per month—with some as low as SLL 3,287—while the highest pension is SLL 31,026,852.  Political pensioners represent 1.28 percent of the total pension roll of 18,430, but monthly pensions paid to them represent 44 percent of the total pensions paid to retirees. Source: HRMO Director General “Update on the Establishment of a Wages and Compensation Commission to Address the Current Disparities in the Pay System.” 2. To address these pay disparities, a wages and compensation commission is needed. As part of the P&PP, a delegation conducted study tours in Ghana and Kenya in 2013 and 2014, respectively. A study was then commissioned to determine an action plan for harmonizing salaries in subvented agencies; however, those recommendations have not yet been implemented. In August 2017, the Ministry of Finance agreed to the establishment of a wages and compensation commission and three months later, the Cabinet approved its establishment and enactment, as well as the move to harmonize existing laws governing the payment of end service benefits. Procurement of consulting service to provide technical support on the development of an institutional framework and the drafting of a wages and compensation bill is currently ongoing. Page 69 of 72 The World Bank SL Public Sector Pay & Performance (P128208) ANNEX 7. BORROWER, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS 1. The ICR of the PSRU, Office of the President (PSRU), GoSL, included the following lessons learned, as detailed in the following paragraphs. 2. There has been a stronger inter- and intra - agency collaboration and increased trust and communication among the Implementing Agencies. However, the amount of time and energy required should not be under-estimated. 3. There was a need for a broader stakeholder engagement during project designs where all streams of required activities of a project/program were clearly identified for proper role clarifications and fiscal management. Otherwise, prior unidentified activities could have posed funding constraints and implementation delays especially where the project designs required funding from different parties 4. There was a need for in-depth review of the context and culture of the prevailing situation at the project design stage. Experience have shown that though implementing agencies were willing to change, they were also handicapped by systemic problems. 5. There was a need for Strategic Leaders to be more assertive and visible to enable staff to comply with policies and guidelines that support achievement of activities. This has been more evident with the introduction of the new Performance Management system in the Civil Service which has experienced implementation delays. 6. Institutionalized the RRA. In building both capacity and a result-oriented culture, there was a need to introduce management technique that has shown positive result in the implementation of the indicators/activities of the P&PP. Therefore, the Rapid Result Approach (RRA) which has been utilized in the project introduced by the World Bank was proven to be appropriate. The approach was initially focused on MDAs that provide key public services. The RRA is a management technique that breaks down long term plan into 100 -day projects 7. Exchange Rate Loss- Whilst the use of country system has it benefit, we came to the realisation that this had an alarming impact on the project especially since 2015. The project lost a significant amount of funds on exchange rate as majority of funds allocated under the Technical Assistant (TA) component were tied to payment of KPMG, the Consultant for the JE & LMS which contract was denominated in USD. Page 70 of 72 The World Bank SL Public Sector Pay & Performance (P128208) ANNEX 8. SUPPORTING DOCUMENTS Ascendant and Company. 2017. “Sierra Leone Pay and Performance Project Performance Review of Compliance with GoSL Performance Management Policy for Civil Servants in Grade 11 and Above in 12 Pilot MDAs.” Blum, Jurgen, Rene Marcos Ferreiro-Rodriguez, and Vivek Srivastava. 2017. Building Public Services in Postconflict Countries: A Comparative Analysis of Reform Trajectories in Afghanistan, Liberia, Sierra Leone, South Sudan, and Timor-Leste. World Bank: Washington, DC. HRMO (Human Resources Management Office). 2016. Civil Service Performance Management Policy. HRMO. 2018. “Update on the Establishment of a Wages and Compensation Commission to Address the Current Disparities in the Pay System.” HRMO. Staff Performance Appraisal Report Form A - Grades 1–6. HRMO. Staff Performance Appraisal Report Form B - Grades 7–10. PSC. 2015. Gender Analysis for the Sierra Leone Pay and Performance Project 2012—2015 . PSRU. Minutes of Meetings of Leadership Team on the Pay and Performance Project (various). PSRU. 2012. Project Implementation Manual. PSRU. 2018. Report on the Three-Day Retreat Held on the Implementation of the Job Evaluation and Labor Market Reports for the Sierra Leone Civil/Public Service. PSC. 2012. Recruitment Handbook: An Open, Competitive, and Merit-Based Recruitment and Selection Procedure Handbook. Roseth, Benjamin, and Vivek Srivastava. 2015. “Engaging for Results in Civil Service Reforms: Early Lessons from a Problem-Driven Engagement in Sierra Leone.” Journal of Management and Strategy 6: 2. World Bank. 2012. CAS Progress Report. World Bank. 2012. ISDS Concept Stage. Report No. ISDSC263. World Bank. 2012. PAD. World Bank. Project Paper Report No. RES18991. World Bank. Project Paper Report No. RES29587. World Bank, International Finance Corporation (IFC), and African Development Bank (AfDB). Joint Country Assistance Strategy (JCAS). Page 71 of 72 The World Bank SL Public Sector Pay & Performance (P128208) Page 72 of 72