Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan FSF (P168274) Report Number: ICRR0021846 1. Project Data Project ID Project Name P168274 Afghanistan FSF Country Practice Area(Lead) Afghanistan Governance L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) TF-A9129 31-Jan-2019 100,000,000.00 Bank Approval Date Closing Date (Actual) 14-Nov-2018 31-Jan-2019 IBRD/IDA (USD) Grants (USD) Original Commitment 100,000,000.00 100,000,000.00 Revised Commitment 100,000,000.00 100,000,000.00 Actual 100,000,000.00 100,000,000.00 Prepared by Reviewed by ICR Review Coordinator Group Jorge Garcia-Garcia Clay Wescott Malathi S. Jayawickrama IEGEC (Unit 1) 2. Project Objectives and Components DEVOBJ_TBL a. Objectives To support improved predictability of recurrent expenditure payments of the Government of the Islamic Republic of Afghanistan during the period of political transition (p. 11, PAD). b. Were the project objectives/key associated outcome targets revised during implementation? No Page 1 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan FSF (P168274) c. Will a split evaluation be undertaken? No d. Components Support to Core Government Functions The component financed payment of civilian recurrent expenditures during a period of political transition. Its purpose was to prevent any delay in paying recurrent expenditures that could disrupt delivery of essential services and cause social unrest. The component supported prioritizing salary payments to ensure their delivery. To withdraw the funds the government had to fulfill three conditions upon project effectiveness:  Actual revenue collected equals at least 80 percent of the revenue target for FY2018 agreed with the International Monetary Fund (IMF) during the fourth Extended Credit Facility (ECF) review. The total amount of the target was Afghan Afghani (AFN) 137.6 billion.  There are no civil servant’s salary claims pending with the Treasury for more than 10 working days.  The average Treasury cash balance between August 10 and November 10, 2018, is not less than the floor of AFN 10 billion agreed with the IMF during the fourth ECF review. e. Comments on Project Cost, Financing, Borrower Contribution, and Dates The project cost $100 million and the International Development Association (IDA) financed it with a Grant from the Afghanistan Reconstruction Trust Fund. The project was approved on November 14, 2018, became effective on November 21, 2018 and was closed on January 31, 2019. 3. Relevance of Objectives Rationale The country’s electoral experience of 2014 showed that during periods of uncertainty, like elections, the government ran short of cash for paying the bills because it did not collect enough revenues to pay for its expenses. To ensure the government would have the necessary liquidity to pay the salaries, the project had three conditions that could be easily achieved and verified before the fiscal year ended. The conditions were associated with fiscal stability, necessary for building fiscal institutions and improve financial management. By emphasizing fiscal management, the project supported the fiscal performance improvement plan and complemented the support from three instruments established in 2017: the US$100 million Fiscal Performance Improvement Support Credit, the Advisory Facility for the fiscal performance improvement plan, and the Investment Program Development Policy Grant for US$300 million per year. The objectives were aligned with the World Bank’s Country Partnership Framework for Afghanistan and with the government’s Afghanistan National Peace and Development Framework. The project supported: (a) the Country Partnership Framework’s pillar building strong and accountable institutions which sought to Page 2 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan FSF (P168274) improve public financial management and the performance of key government institutions; and (b) the Afghanistan National Peace and Development Framework program which sought a credible and implementable national budget, essential for a well-functioning government. Afghanistan has two pervasive and intertwined problems: internal conflict and fragility. Building effective institutions is necessary to address these problems. The project sought to prevent the conflict and violence that could be triggered by the failure to deliver essential services because public servants were not paid their salaries. Rating Relevance TBL Rating Substantial 4. Achievement of Objectives (Efficacy) EFFICACY_TBL OBJECTIVE 1 Objective Improved predictability of recurrent expenditure payments of the Government of the Islamic Republic of Afghanistan during the period of political transition. Rationale Disbursement of the funds was conditional on the government meeting three targets for: revenues, minimum cash balance held by the Treasury, and payment by the Treasury to civil servants within 10 days of the day salaries are claimed. Revenue: The expected result was to collect revenues of, at least, AFN 137.6 billion which is an amount equivalent to 80 percent of the AFN 172 million revenue collection agreed to with the IMF in its fourth review of the ECF. By November 10th, 2018 the government had collected revenues of AFN 147.78 billion, exceeding the target. The IMF estimates the government collected AFN 189.9 billion of revenues in FY2018 (IMF Country Report No. 19/157, p. 26). The expected result was achieved. Cash balance target: The expected result was for the Treasury to have a minimum average cash balance (without restrictions) of at least AFN 10 billion during the period August 10th to November 10th, 2018. The actual average cash balances were AFN 21.46 billion, exceeding the AFN 10 billion floor. The large cash buffer reduced the likelihood of the government having to ask the donors for emergency financial assistance. The expected result was achieved. Timely payment of civil servants' salaries: The expected result was that as of November 10th, 2018 the Treasury would pay civil servant’s salary claims within less than 10 working days. Payment was verified with the Afghan Financial Management Information System, which showed there were no claims outstanding on that date. The payment system had already been working well: between December 22, 2017 and November Page 3 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan FSF (P168274) 22nd, 2018, only 82 of 112,516 payment transactions processed for payroll were delayed more than 10 days. As of April 2019, all salaries were paid on time. The expected result was achieved. Rating Substantial OVERALL EFF TBL OBJ_TBL OVERALL EFFICACY Rationale --- Overall Efficacy Rating Substantial 5. Efficiency The project was completed on time, within budget and the results exceeded the expected outcomes. Given the nature of the project, the project appraisal document did not have estimates of efficiency. The following reasons suggest the project’s efficiency is high:  Per dollar of project cost, the revenue collected and the cash balances outstanding exceeded their expected values by 10 and 100 percent;  Civil servants received their salaries within the 10 days period agreed to; in addition, salaries were paid on time between January and April 2019. As a result, the households of civil servants were better off because they did not have to disrupt their consumption plans;  The project showed a way to strengthen fiscal institutions while preventing a repeat of the experience in 2014: low revenue, dwindling cash reserves, and growing salary arrears. By forcing more discipline in managing revenues and salary expenditures, the project helped to put in place a coherent framework for preventing future potential liquidity shortages. Efficiency Rating Substantial a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Page 4 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan FSF (P168274) Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0  Not Applicable 0 ICR Estimate 0  Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome The objective was achieved in an efficient manner. The project helped ensure the predictability of the government’s recurrent expenditure payments during a period of political transition. The government collected more revenues and maintained a higher level of cash balances than expected, thereby permitting the payment of civil servants' salaries within 10 days after the claims became effective. Because salaries were paid on time: (a) civil servants' households were better off than they would have been otherwise; (b) Afghan households, enterprises and family businesses were better off because they had access to government services without interruption. Finally, the institutional arrangements put in place led to a better management of government revenues and expenditures. a. Outcome Rating Satisfactory 7. Risk to Development Outcome Sections 4 on efficacy and 6 on outcome show that the expected outcomes were exceeded and the potential risks identified at the time of evaluation did not materialize. The project was a one–time event that lasted three months. The expected benefits have been obtained already, and there is no risk of losing them. The risk to development outcome is negligible to low. 8. Assessment of Bank Performance a. Quality-at-Entry The FSF was financed by the Afghanistan Reconstruction Trust Fund (ARTF). Managed by the World Bank since 2002, the ARTF has had two main funding windows: 1) baseline financing provided support to government's recurrent costs with few conditions attached; and 2) the incentive program introduced in 2008 provided annual incentive payments in exchange for completed reforms. Since 2008, the balance of funding has increasingly shifted to the incentive program. In 2018, the baseline window was replaced by the FSF, so that all ARTF funds are now conditional and based on completed reforms. This transition to Page 5 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan FSF (P168274) results based financing has been well designed to enable the government to have enough time to build up sufficient capacity to succeed in such a program. The Bank designed the project in a way that guaranteed the government could meet the disbursement conditions while ensuring fiscal stability, timely payment of salaries and delivery of services without interruption. The revenue and cash balance conditions were part of the agreement reached between the authorities and the IMF during the fourth review of its ECF, a program which the government had been complying with. The targets could be verified and the steps required to do so were appropriate. In sum, the conditions ensured the project objectives were achieved. The design considered disbursing 40 percent of the grant as an advance and 60 percent after the conditions were verified. The two-tranche disbursement plan did not work because of the short life of the project; a single, 100 percent disbursement was made once the conditions for releasing the funds of the grant were met. It is possible that if sufficient time had been allowed for preparing the operation, the split disbursement could have been carried out, but there is no evidence that disbursing in a single tranche impinged on project results. Quality-at-Entry Rating Satisfactory b. Quality of supervision Supervision was appropriate. All the funds were disbursed after verifying achievement of the targets. The Oversight Committee comprised of three Director Generals in the Ministry of Finance oversaw project implementation, reviewing the evidence on revenue collection, cash position and timely salary payment prior to informing IDA. IDA staff reviewed and verified the evidence but they could not verify the quality of the data on the Treasury’s cash position because the Ministry of Finance lacked a system to keep up with the outputs of the Afghanistan Management Information System. IDA staff also verified in the financial reports of the management information system the details on salary claims pending with the Treasury, concluding that as of November 10, 2018 there were no outstanding claims. Quality of Supervision Rating Satisfactory Overall Bank Performance Rating Satisfactory 9. M&E Design, Implementation, & Utilization a. M&E Design The project objective was clearly specified, the indicators selected were clear, specific, measurable, achievable, relevant, time-bound and appropriate to measure the achievement of the objective. The text in Page 6 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan FSF (P168274) the Bank Performance section of the ICR concludes that M&E design was sound and aimed at collecting and verifying the information required to establish fulfillment of the disbursement conditions. A Third-Party Monitoring Agent would certify the fulfillment of the withdrawal conditions. The cash balance report from the Da Afghanistan Bank [the country’s Central Bank] was to be compared with the information in the Afghanistan Management Information System. b. M&E Implementation The government collected the information on time and reviewed its quality. The Third-Party Monitoring Agent and IDA staff verified the information. IDA and the Monitoring Agent concluded that the data received were reliable and of good quality, but they could not verify this to its full extent as mentioned above under Quality of Supervision. The short time span between reception and verification of the information and project termination prevented IDA and the Treasury from sharing it with donors beforehand. Donors received the findings and information during a joint donor and government meeting. Although no rules were broken –the project did not consider delivering the information to donors in advance– distributing the information during the meeting forced donors to raise their questions publicly leaving the impression among some government officials that donors did not fully trust the Monitoring Agent. c. M&E Utilization The World Bank used the data collected to assess achievement of outcomes and to authorize disbursement of the grant. M&E Quality Rating Substantial 10. Other Issues a. Safeguards The project did not trigger social and environmental safeguards b. Fiduciary Compliance The project complied with fiduciary requirements. The monitoring agent verified the expenditures reported, reviewed the withdrawal applications and verified fulfillment of the conditions for releasing the funds of the grant. Page 7 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan FSF (P168274) c. Unintended impacts (Positive or Negative) None d. Other None 11. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Satisfactory Satisfactory Bank Performance Satisfactory Satisfactory M&E implementation had some Quality of M&E High Substantial drawbacks. Quality of ICR --- Substantial 12. Lessons The lessons from the ICR are summarized below:  The Fiscal Sustainability Framework design can be an important instrument under the right circumstances. Under circumstances where budget support is needed but the authorizing environment or fiduciary controls are weak, it is appropriate to use short-term, reasonably easy-to-achieve targets as a steppingstone to transitioning to conditions that will have a longer policy term impact.  Changes to long-established programs need to be adequately socialized over time. After 16 years of unconditional baseline support from the Afghanistan Reconstruction Trust Fund, transitioning to a new modality requires enough time and socialization for all parties to become comfortable and adjust to the new reality. The dependency can be lessened by programs that help increase government revenues.  In fragile environments, preparing for the worst case scenario may be the best option. The negative impacts of preparing for a fiscal shock that does not materialize are minimal compared with the risks of not being prepared.  Identifying and implementing a precautionary fiscal response mechanism gives confidence to the Government and partners when a fiscal shock is feared. By implementing the FSF Project, the Government was able to confidently move forward with Page 8 of 9 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review Afghanistan FSF (P168274) recurrent payments and avoid holding back expenditures out of fear that it would face a revenue shortage in the later months of the year. The review draws the following lessons:  It is possible to withdraw unconditional support and replace it with conditional support that could help develop better institutions without disrupting normal government functions. These findings have implications for how to proceed with the new program built around the Incentive Program Development Policy Grant and for how the baseline facility and the Ad-Hoc Payment Facility are managed in the future.  Implementing the transition from unconditional to conditional support may be fast. While it may take time to wean aid recipients from unconditional to conditional support, it is possible to prepare a project in a short time where conditions can be implemented and the objectives achieved, and where precedents for good financial management are established. 13. Assessment Recommended? No 14. Comments on Quality of ICR The ICR describes in a clear and concise manner about the project origins, the choice of design, the reasons for it and the results achieved. The analysis explains well the problems, their causes and their solutions; the graph on the results chain gives a useful picture. The ICR describes fully the monitoring and evaluation processes and the actions taken to verify the evidence received from the government. a. Quality of ICR Rating Substantial Page 9 of 9