66193 A NEW INSTRUMENT TO ADVANCE DEVELOPMENT EFFECTIVENESS: PROGRAM-FOR-RESULTS FINANCING Operations Policy and Country Services December 29, 2011 ABBREVIATIONS AND ACRONYMS BP Bank Procedures CCT Conditional cash transfer CAS Country Assistance Strategy CODE Committee on Development Effectiveness CPS Country Partnership Strategy CSO Civil Society Organization DLI Disbursement-linked indicator DPL Development policy lending EC European Commission EFA FTI Education for All—Fast-Track Initiative FM Financial management GAC Governance and Anticorruption GAVI Global Alliance for Vaccines and Immunization GPOBA Global Program on Output-based Aid IBRD International Bank for Reconstruction and Development IAD Internal Audit Department ICR Implementation Completion and Results Report IDA International Development Association IDB Inter-American Development Bank IEG Independent Evaluation Group IL Investment lending INT Integrity Vice Presidency IP Inspection Panel ISR Implementation Status and Results Report M&E Monitoring and evaluation MCC Millennium Challenge Corporation OBA Output-based aid OP Operational Policy Statement OPCS Operations Policy and Country Services PAD Program Appraisal Document PCN Program Concept Note PID Program Information Document PRSP Poverty Reduction Strategy Paper RBFH Results-based Financing in Health SAI Supreme audit institution SWAp Sectorwide approach TA Technical Assistance A NEW INSTRUMENT TO ADVANCE DEVELOPMENT EFFECTIVENESS: PROGRAM-FOR-RESULTS FINANCING CONTENTS Executive Summary ..................................................................................................................... iii  I.  Introduction ........................................................................................................................1  II.  Responding to Demand and Building on Experience .....................................................3  A. Demand for Program-based Lending ..............................................................................3  B. Summary of Bank Experience.........................................................................................4  C. Lessons from Other Development Partners ....................................................................5  D. Missed Opportunities ......................................................................................................6  III.  Overview of Proposed Program-for-Results Financing .................................................8  IV.  Preparation of Program-for-Results Operations ..........................................................14  A. Technical Assessment ...................................................................................................15  B. Assessment of the Program’s Fiduciary Systems .........................................................18  C. Assessment of the Management of Environmental and Social Effects .........................19  D. Integrated Risk Assessment ..........................................................................................21  V.  Appraisal and Approval of Program-for-Results Operations .....................................22  A. Program Action Plan and Capacity Building ................................................................22  B. Disbursement-Linked Indicators (DLIs) and Disbursement Arrangements .................23  C. Program Audit ...............................................................................................................25  D. Economic Evaluation ....................................................................................................26  E. Management Assessment of Benefits and Risks ...........................................................26  VI.  Supporting Implementation of Program-for-Results Operations ...............................27  A. Implementation Support ................................................................................................28  B. Performance Issues and Remedies ................................................................................29  C. Role of Corporate Oversight Units................................................................................30  VII.  Program-for-Results Policy and Operational Framework ..........................................31  VIII.  Working in Partnership ..................................................................................................32  IX.  Challenges and Risks .......................................................................................................33  X.  Next Steps and Request for Approval ............................................................................36  Annex A. Summary of Consultations .........................................................................................39  Annex B. Summary of Experience .............................................................................................59  Annex C. Draft Operational Policy Statement (OP) 9.00........................................................69  Annex D. Draft Bank Procedures (BP) Statement 9.00 ............................................................74  Annex E. Proposed Anti-Corruption Guidelines ......................................................................85  Annex F. Amendments to World Bank General Conditions ...................................................90  Annex G. Illustrative List of Possible Program-for-Results Operations ................................93  Annex H: Changes to Be Made in OPs, BPs, and Operational Memoranda to Reflect Program-for-Results Financing ..................................................................................................98  Annex I. Initial Implementation Review of Program-for-Results Operations.....................102  A NEW INSTRUMENT TO ADVANCE DEVELOPMENT EFFECTIVENESS: PROGRAM-FOR-RESULTS FINANCING EXECUTIVE SUMMARY 1. Program-for-Results is a proposed new instrument designed to enable the Bank to better respond to changing development needs, meet demand from client countries, and enhance development effectiveness. It links disbursements to achievement of results that are tangible, transparent, and verifiable. By directly supporting government programs, Program-for-Results will help countries strengthen institutions, build capacity, and enhance partnerships with stakeholders to achieve lasting impact. Designed to accommodate a broad range of countries, sectors, and programs, Program-for-Results will also enable the Bank to leverage its own financing and partner with other development organizations in supporting country programs. This paper describes the instrument and requests approval by the Executive Directors that will allow Management to roll it out. 2. Responding to Demand. Client countries are increasingly implementing their own programs for development and poverty reduction and are asking development partners for financing and expertise to improve those programs’ effectiveness and efficiency in achieving results. The Bank’s ability to meet this demand has been constrained by the limitations of its existing investment lending (IL) and development policy lending (DPL) instruments. Neither instrument fully allows supporting a government program of expenditure, building institutional capacity, and tying financing to achievement of results. With Program-for-Results, the Bank would have three complementary lending instruments to offer client countries: policy support (i.e., DPL), project support (i.e., IL), and program support. 3. Building on Experience. The design of the Program-for-Results instrument benefitted from the invaluable experience the Bank has gained in designing and implementing IL and DPL operations: specifically, technical and design issues, results definition, fiduciary systems, environmental and social impacts and risk management, and improving policy environments. It also built on decade-long experience—both within and outside the Bank—with sector-wide, program-based, and results-focused operations. Some key lessons incorporated into the design include the importance of: using the institutional set-up of the government program; ownership of results framework; and transparency and accountability such as having credible verification processes. 4. Feedback from Consultations. Feedback from extensive external and internal consultations highlighted broad support for the instrument, especially its focus on results, leveraged impact, and strengthening of country institutions. Many welcomed the flexibility of the instrument and the opportunity to use it in areas beyond the education and health sectors. They stressed the importance of programs remaining government-owned, and of using program systems to produce results that are clearly defined, measurable, and verifiable. At the same time, concerns were raised regarding the perceived dilution of the Bank’s commitment to fiduciary and safeguards standards. Some asked for a pilot phase and close monitoring of the roll-out. iv 5. Key Features of Program-for-Results. Program-for-Results would have the following features: (a) Finances and supports borrowers’ programs. These programs, comprising expenditures and activities, can be ongoing or new, sectoral or sub-sectoral, and national or sub-national programs, as well as community development programs. The Bank assesses the quality of the program, including its results framework, expenditure framework, supporting systems, and the scope for systems-strengthening measures. (b) Disburses upon achievement of program results. Disbursements will be determined by the achievement of monitorable and verifiable indicators, rather than by inputs. Together with funds from other sources, Bank disbursements will finance the borrower’s expenditure program rather than being linked to individual transactions. (c) Focuses on strengthening the institutional capacity needed for programs to achieve their desired results. A priority area for both preparation and implementation support will be to strengthen the capacity of the institutions to implement the program, thereby enhancing development impact and sustainability. Program governance will be strengthened by focusing on transparency, accountability, and participation. (d) Provides assurance that Bank financing is used appropriately and that the environmental and social impacts of the program are adequately addressed. The Bank will assess the program’s fiduciary and environmental and social management systems. It will agree with the borrower on the additional measures needed to provide assurance that the loan proceeds are used for program expenditures, that these expenditures are incurred with economy and efficiency, and that potential impacts to the environment and affected people are adequately addressed. 6. Program Monitoring. The monitoring and verification of results is an essential feature of the instrument. Moreover, DLIs will require a credible verification process that is acceptable to the Bank and is agreed at the time of appraisal. The verification process would also include independent/third party monitoring, where appropriate. DLIs will be made public and their progress will also be reported in the implementation reports. Program-for-Results documents will be available to the public, giving stakeholders access to information about the performance of the public institutions and programs. 7. Appropriateness of Program-for-Results Instrument. The Bank will determine the choice of lending instrument for specific countries, sectors, and programs in the context of its Country Assistance Strategy/Country Partnership Strategy (CAS/CPS), and its assessment of the country’s policies, programs, and institutional capacity. Program-for-Results has the potential to make significant development impacts, yet it also poses risks. Certain high-risk activities will be excluded from Program-for-Results operations: activities that pose a risk of potentially significant and irreversible adverse impacts on the environment and/or affected people (activities classified as Category A under IL); and activities that involve procurement of works, goods, and services under contracts whose estimated value exceeds specified monetary amounts. v 8. Processing of Program-for-Results Operations. For each Program-for-Results operation the Bank will carry out a process of identification, preparation/assessment, appraisal, and implementation support. A central focus of preparation will be the identification of disbursement-linked indicators (DLIs), each with a verification protocol to ensure that a credible mechanism is in place for monitoring and verifying its achievement. Appraisal of each operation will be informed by rigorous assessments in three main areas that will be applied to the overall program and its expenditures. The technical assessment focuses on the strategic relevance and technical soundness of the program and its expenditure framework, the results framework, and the M&E arrangements. The fiduciary assessment, covering the procurement and financial management arrangements, seeks to make sure that program funds are used appropriately. The environmental and social systems assessment seeks to make sure that the potential environmental and social impacts and risks are adequately addressed. These assessments will identify measures to enhance performance, build capacity, and mitigate key risks, and will be reflected in an integrated risk assessment. The resulting action plan will be reflected in the legal agreement between the Bank and the government. 9. Capacity Building. Program-for-Results focuses on the behavioral and institutional changes that are required to achieve results and manage associated risks. Hence it is expected that many Program-for-Results operations will require some level of capacity-building activities, which will be informed by the technical, fiduciary, and environmental and social systems assessments. Capacity building support will be provided through different modalities, from direct technical assistance and training to specific actions or DLIs to strengthen performance. 10. Implementation Support. During implementation, the Bank team will monitor the overall progress of the program, associated expenditures, and the achievement of results (including the DLIs). It will monitor progress in implementing the action plan, changes in the program’s risks, and compliance with the provisions of the legal agreements. Technical support from the Bank team will focus on improving systems performance and resolving implementation issues. Program-for-Results will be subject to the same corporate oversight functions as other Bank lending instruments, and the Bank will retain the right to carry out investigations that it judges to be necessary and to sanction entities that are found to have engaged in fraud or corruption. The Bank’s debarment list will apply to Program-for-Results operations. 11. Management/Board Oversight of the Rollout of Program-for-Results. The Bank will roll out the new instrument cautiously, aiming to find a balance between responding to strong client demand and learning from implementation experience. This approach builds on experience with introducing other new initiatives, such as additional financing. Staff will be provided with comprehensive training, technical guidelines, and other tools. In addition, Management intends to put the following in place:  Enhanced corporate oversight arrangements during the initial phase to help ensure appropriate and consistent application of the new instrument.  A strong feedback mechanism so that lessons of experience are used in the guidance to staff and in the review of new operations that will provide inputs to a proposed review of implementation experience. Management welcomes an early evaluation of vi implementation by the Bank's Independent Evaluation Group which would provide a useful independent input for ongoing improvements to the instrument. 12. In addition, Management proposes to limit IBRD/IDA commitments to 5 percent of total IBRD/IDA commitments for the first two years from Board approval. Management would propose to the Board to lift this cap if justified by a review of implementation experience. 13. Request for Approval/Endorsement. Management requests that the Board of Executive Directors (a) approve the content of the draft OP 9.00, Program-for-Results Financing (Annex C of the paper) and of the draft Guidelines on Preventing and Combating Fraud and Corruption in Program-for-Results Financing (Annex E of the paper); and (b) approve the modifications to the General Conditions for Loans (July 31, 2010) and the General Conditions for Credits and Grants (July 31, 2010), substantially in the form set out in Annex F of the paper. In addition, Management requests that the Executive Directors endorse the proposal to limit Program-for- Results commitments as set out in paragraph 111 of the paper. A NEW INSTRUMENT TO ADVANCE DEVELOPMENT EFFECTIVENESS: PROGRAM-FOR-RESULTS FINANCING I. INTRODUCTION 1. World Bank Management is proposing a new financing instrument—Program-for-Results —to respond to borrowing countries’ changing development needs. Under Program-for-Results, Bank support will help countries improve the design and implementation of their own development programs and will finance a portion of the program expenditures. The instrument will focus on development results by linking disbursements to results or performance indicators, which can be outputs, outcomes, or other actions/results that are tangible, transparent, and verifiable. Program-for-Results will work directly with the program’s institutions and systems and will seek to strengthen those institutions’ governance, capacities, and systems over time. By supporting government programs, the new instrument provides an opportunity for the Bank to extend its impact beyond its financing to larger country programs and the performance of country institutions. Finally, Program-for-Results will be an instrument for strengthening partnerships with governments and development partners, allowing the Bank to effectively support larger programs and cofinance in pooled funding arrangements. 2. Context. In March 2010, in updating the Board’s Committee on Development Effectiveness (CODE) on progress on investment lending (IL) reform, Management explained that a new instrument had been conceived in the context of IL reform but, after careful consideration, would be proposed as an instrument separate from and complementary to both IL and development policy lending (DPL). CODE members welcomed the initial discussion and expressed their support for a “program-based� approach as a complement to the “transaction- based� approach of IL. They also commented on the rationale for the new instrument, its expected impact on the way the Bank engages with clients and development partners, the review of defined government expenditure programs, and the fiduciary and environmental and social aspects of such operations. In October 2010, Management presented to CODE a draft concept note for the new instrument. CODE supported the broad directions proposed in the note, including the focus on programs and the clear link to results. Members asked for additional details—particularly with respect to the rationale for a new instrument separate from IL and DPL, the definition and treatment of results and their links with disbursements, the assessments of procurement and environmental and social impact, risk management, and the name of the proposed instrument. Management revised the concept note and, following an informal Board meeting in February 2011, used the note as the basis for extensive internal and external consultations on the proposed new lending instrument. 3. The draft Policy Paper was discussed by CODE on August 3, 2011. The Committee unanimously supported the initiative and agreed that the instrument could advance development effectiveness through its results orientation, focus on capacity building and institutional strengthening, and contribution to the governance and anti-corruption agenda. The Committee stressed the need to maintain flexibility in the design of the instrument, which is essential to be able to apply the instrument to different country circumstances. At the same time, Committee members noted the importance of risk management and the need to proceed cautiously with 2 implementation, while allowing for innovation. Management’s request to proceed to the second phase of consultations was endorsed. A cover note summarizing the issues raised during that discussion was added to the draft policy paper posted for internal and external consultations. 4. Consultations. The Bank carried out internal consultations with the Board and Bank staff—more than 50 sessions with corporate units, Networks, Regions, and field offices. It also held broad external consultations with representatives of governments, development partners, other international organizations, parliamentarians, the private sector, civil society, foundations, academia, and think tanks. The consultation process on the proposed Program-for-Results instrument was divided into two phases: in the first phase, extensive global consultations were held on the Concept Note from March 1 to June 30, 2011. The comments and suggestions from the first phase were reflected in the draft Policy Paper. The draft Policy Paper was posted on the Bank’s external website and formed the basis of the second phase of consultations (August 15 to September 30, 2011).1 Announcements seeking public feedback on the draft Policy Paper were made through several channels in Washington, DC and the Bank’s country offices. The global consultation process—web-based and face-to-face—included 34 client countries, consisting of IDA and IBRD borrowers with large and small operations in all six Regions, fragile states, and 7 donor countries.2 In each consultation meeting, efforts were made to ensure the participation/representation of a range of stakeholders. The consultation plan was made publicly available on the Bank’s external website and outlined the areas for which the Bank was seeking feedback. A dedicated webpage hosted the consultation materials, the majority of which were translated into five languages: Arabic, Chinese, French, Russian, and Spanish. Detailed information about the consultations—participant lists, discussion summaries, and other records of consultation—were made publicly available on the consultation webpage, which was visited by people from 148 countries. 5. Consultation Feedback. Overall, participants expressed broad support for the introduction of the new instrument, viewing it as a natural complement to the Bank’s existing lending instruments. They welcomed the potential of the instrument to help build government systems and felt that, by rewarding performance, it should help strengthen the efficient use of public resources. They agreed that by using the government’s own program systems, Program- for-Results would enhance the focus on institutional strengthening, thereby enhancing development impact and sustainability and creating an enabling environment for partnerships. Regardless of income level, client countries generally welcomed the flexibility of the instrument and the opportunity to use it in sectors beyond education and health. They stressed the importance of country ownership of the program and the need for tangible, feasible, and verifiable results. Participants articulated the importance of a clear definition of results; clarity on roles, responsibilities, and accountabilities; and effective monitoring, evaluation, and verification (including potential involvement by civil society organizations). Several participants asked what checks and balances the Bank would put in place to ensure that money is used for its intended purposes and that environmental and social impacts are systematically identified and addressed—and specifically, how fraud and corruption will be addressed in Program-for-Results, what procurement principles will be applied, how the environmental and social policies 1 While consultations officially ended on September 30th, management continued to accept comments through the end of October. 2 In addition, consultations were held with representatives from 9 Part I countries and 9 development partners. 3 applicable under Program-for-Results will relate to the Bank’s current safeguard policies for investment lending, and what oversight functions will apply to Program-for-Results operations. (A more detailed summary of the feedback from the consultations is provided in Annex A.) Some stakeholders had concerns that the new instrument may dilute the Bank’s fiduciary and safeguards standards and asked for more clarity on exclusions from the instrument. Several stakeholders underscored the importance of adopting a learning approach to the implementation of the instrument in the early years; some asked for a pilot phase. The detailed design of the new instrument described in this paper endeavors to respond to the feedback and questions received. 6. Organization of Paper. Following this introduction, Section II of this paper reviews the demand for Bank financing as reflected in country strategies and the lessons from experience inside and outside the Bank with operations that have programmatic features and which link disbursements to results. Section III provides an overview of the new instrument. Sections IV, V, and VI describe the process for preparing, appraising, and implementing Program-for-Results operations. Section VII outlines the new operational policy and procedures for the instrument, and Section VIII describes how the Bank would work in partnership with others in using the new instrument. Section IX discusses how to address some of the challenges and risks to the further development and introduction of the proposed instrument, and Section X summarizes the proposed next steps and request for approval to the Board. Annex A summarizes the feedback from consultations. Annex B summarizes some of the Bank’s and other development partners’ experience with operations with programmatic investment lending features. Annexes C, D, and E consist of the draft policy, procedures, and anticorruption guidelines, respectively, for the new instrument. Annex F contains proposed changes to the General Conditions that take into account the instrument’s features. Annex G provides an illustrative list of possible Program-for-Results operations for FY12.3 Annex H outlines the required technical (non-policy) modifications to the other relevant OPs, BPs, and Operational Memoranda. Finally, Annex I outlines the approach for an initial review of implementation experiences. II. RESPONDING TO DEMAND AND BUILDING ON EXPERIENCE 7. The Bank does not have a stand-alone financing instrument that supports government expenditure programs and disburses against results. However, demand for this type of lending has been strong, and features of program-based lending have been used in operations that have been approved and implemented over more than a decade. Other development agencies have also been developing more program-based approaches with stronger links between financing and results. A review of experience with these types of operations provides perspective and useful lessons for the design and implementation of Program-for-Results operations (Annex B provides additional detail). A. Demand for Program-based Lending 8. A review of 51 Country Assistance Strategies (CASs)/Country Partnership Strategies (CPSs) discussed by the Board between 2008 and 2010 yields information on country demand for the Bank’s lending, advisory, and other services. With respect to lending, almost all the 3 These operations are in the early stages of preparation, and none has yet been presented to the Board for approval. 4 countries covered expect to continue benefiting from traditional IL, while 36 of the 51 countries (69%) expect to benefit from DPL and 34 (65%) request some form of programmatic lending, program-based approach, or sectorwide approach (SWAp). Demand for program-based lending comes from countries in all income categories and all Regions, as well as from states or subnational levels of government, and includes a broad range of sectors—education, health, social protection, roads, water, energy, urban development, and agriculture. It is also clear that countries want a program-based approach that is integrated, holistic, and focused on performance measurement and results. The programs to be supported need to be owned by the government, aligned with sector and macroeconomic priorities, and focused on system strengthening, with the financing of the expenditure programs integrated into national budgets. Countries would also like such approaches to be used as a mechanism for better coordinating the support of all external partners, including through possible joint funding. B. Summary of Bank Experience 9. Many of the operations with programmatic features have their roots in what has come to be known as “program-based approaches.�4 The numbers of these operations have increased since the 1990s in response to a stronger focus on results and to clients’ increasing demand for support of government programs and assistance in addressing system-level weaknesses. Vehicles used have included SWAps, conditional cash transfers, the Global Program on Output-based Aid, the Results-based Financing in Health initiative, and the Global Partnership for Education. The sectoral, regional, and product design profile of such program-based IL operations has adapted and evolved to meet different country conditions (client demand, development needs, and institutional arrangements) and to focus more on results. Box 1 provides an example of programmatic lending for the education sector in Pakistan. 10. Good Implementation Record. Reviewing the overall performance of programmatic ILs is complicated by the fact that these operations are not separately coded and can therefore be difficult to identify. A sample of 41 such projects identified by the Regions5 revealed that they perform at least as well as the overall IL portfolio; and a more recent analysis of SWAps, conditional cash transfers, and output-based aid confirms that their project performance indicators compare favorably with those of the overall lending portfolio. These types of operations appear to have a stronger results orientation, at least in part because several of them include disbursement-linked indicators (DLIs). Implementation Status and Results reports (ISRs) rated monitoring and evaluation (M&E) as moderately satisfactory or better for 90 percent of these projects by number and 97 percent by value. The types of results have varied and (like other aspects of program design) have been tailored to the development stage of the supported government program and its expenditures. 4 There is no single definition of a program-based approach. The most commonly used description is the OECD- Development Assistance Committee’s: “A way of engaging in development cooperation based on the principle of coordinated support for a locally owned program of development. It includes four elements: leadership by the host country organization; a single program and budget framework; donor coordination and harmonization of procedures; and efforts to increase the use of local procedures over time with regard to program design and implementation, financial management, and monitoring and evaluation.� 5 The review included operations from all Regions (13 from AFR, 3 from EAP, 4 from ECA, 12 from LCR, 3 from MNA, and 6 from SAR) and in education, health, other social sectors, infrastructure, local government/urban, and finance, as well as multisectoral programs. The total lending volume was US$9 billion. 5 11. Lessons. Some key lessons can be drawn from program-based approaches used by the Bank: (a) linking resources to results enhances development effectiveness; (b) an early focus on design and assessment of the program supported produces better results in the long term; (c) precise definitions of the expected results and assignment of accountability are needed; and (d) credible verification of results is essential. At the same time, staff report that the continued process- and input-intensity of the IL product and a lack of clarity on how to approach and how to interpret the current IL operational framework (which was designed for a very different kind of lending) has caused confusion among staff, clients, and development partners, taken time away from focusing on substantive results, and made the processing of such operations costly in time and effort for development partners and countries. Box 1. Programmatic IL: The Experience in Pakistan The Punjab Education Sector Project and Sindh Education Sector Project, approved by the Board in June 2009, use an IL instrument with program-based features to support education reforms in their respective provinces. These projects build on the experience of projects in LCR: Minas-Gerais Partnership II SWAp and Ceara Multi-Sector Social Inclusion Development in Brazil. Minas supported five sectors, including health and education, and Ceara supported human development and sustainable resource management. The Pakistan teams, learning from the LCR operations, designed specific IL credits to support the governments’ programs for primary and secondary education. The projects disburse against eligible expenditure programs based on the achievement of predetermined disbursement-linked indicators (DLIs). The DLIs include intermediate outcomes and implementation performance/institutional change indicators that build incrementally over the life of the project. Evolution of support. The Bank previously engaged in education reform programs in Punjab and Sindh through development policy credits that supported policy reforms in the sector. The program then followed up with a specific investment credit building on the achievements of the development policy credits while focusing on implementation outcomes and outputs through the achievement of measurable targets (DLIs). The new approach also provided enhanced focus on strengthening associated provincial systems (e.g., fiduciary, safeguards, and M&E) through technical capacity building and implementation support. Flexibility in design. The flexibility in design allowed the Bank to support a part of the government’s program and priorities while maintaining a strong focus on the achievement of key results. The DLIs were determined in partnership with the provincial governments, reflecting priority elements in the provincial reform programs. This also allowed for the DLIs to be tailored to the specific needs in each province. Focus on governance and M&E. The use of this lending approach facilitates an emphasis on governance improvements by targeting subprogram design at strengthening education sector governance and institutionalizing accountability in the delivery of education. Some examples:  Merit and need-based teacher recruitment in Sindh;  Implementation of a teacher incentives program in Punjab; and  Strengthening of school-based management, with establishment and capacity building of school management committees in both Punjab and Sindh. A strong focus on M&E is also a part of project design to complement governance reform. The government’s own M&E systems, such as the annual public school census, have been strengthened and are now used to motivate policy and implementation decisions. Also, project design includes third-party validations to evaluate subprogram performance and achievement of DLIs, and evaluations for key interventions to provide valuable information to inform future program directions and decisions. C. Lessons from Other Development Partners 12. The call for more focus on results and for stronger links between financing and results is not unique to the Bank; other agencies have also been exploring such options. A number of global initiatives have been created to support closer links between funding and results—for 6 example, the Millennium Challenge Corporation and the GAVI Alliance (formerly known as the Global Alliance for Vaccines and Immunizations). The Inter-American Development Bank (IDB) has also developed a performance-driven lending instrument, and the sector budget support programs of several development partners have many of the features of programmatic and more results-focused lending (see Annex B). 13. Lessons. While the Bank cannot adopt some of these approaches wholly, they yield several important lessons:  More performance-based financing can incentivize progress on results in terms of outputs and outcomes, and client countries appreciate it.  Program-based approach shifts the focus of operations from definition of inputs toward results.  Flexibility in instrument design is needed to accommodate diverse country and sector contexts.  Keeping project design as simple as possible improves an operation’s focus and the ease of implementation.  The selection of DLIs should reflect the fact that results mean different things at different stages in the program cycle, and outcomes may take many years to achieve.  Data quality and adequate monitoring systems are the cornerstone of a robust focus on results. D. Missed Opportunities 14. While demand for program-based lending is increasing, the Bank’s ability to meet that demand is constrained by the limitations of its menu of lending instruments. While the two existing Bank instruments strive to deliver results and development impact in support of Bank country programs and will continue to be the leading lending instruments, neither fully allows support to be tied to the results from specific programs of expenditures. Therefore, the Bank is missing opportunities to partner with clients and leverage its expertise in the implementation of an entire program. It would improve the efficiency, effectiveness, and governance of development programs and thereby enhance their overall development effectiveness. This would also shift incentives of governments from focus on inputs to focus on outcomes and how to achieve them. By aligning more support directly with government programs, focusing directly on systems improvement and capacity building, and working in partnership with governments and other stakeholders to reorient programs to be more results-based, the Bank has the opportunity to build on its track record of delivering results across a broad range of sectors and countries, and to significantly increase the sustainability of its impact. 15. Application of IL Operational Policies and Procedures. Until now, program-based investment operations have had to be squeezed into the existing IL policies, including the fiduciary and safeguards policies, which are designed for traditional ring-fenced IL projects 7 rather than a results-based programmatic approach. This has created a number of problems (see Box 2). Box 2. Using the IL Instrument in India and Brazil: Achievements and Limitations The India PMGSY Rural Roads Project (P124639), financed by an IDA credit of US$1 billion and an IBRD loan of US$500 million, is supporting the Government of India’s Rural Roads Program, which aims at providing all-season road access to all villages with a population above 500. About 30 percent of India’s villages (or 300 million people) are without all-season access to social services and economic opportunities. Launched in 2000, the Government program has already financed the connection of 70,500 villages through 270,000 km of new or improved rural roads, at a cost of US$14.7 billion. It is estimated that it will cost another US$40 billion to achieve the program’s target by 2020. Bank financing supports implementation of a time-slice of the program in the seven Indian states where most progress will be needed to achieve the program’s targets. Bank funds will be disbursed against results achieved in those states: enhancing connectivity (another 8,200 villages would be connected), improving effectiveness of public administration (including new, more efficient, countrywide specifications applied to manage rural roads and procure civil works), and maintaining the roads (increased portion of the rural road network under maintenance contracts). While the program is generally acceptable to the Bank, Government procedures under the program had to be modified to conform to the Bank’s IL policies, especially in the procurement and social impact areas. The Bank was able to accept a modified version of the program procedures mainly because the contracts under the program are of low value and complexity and have the potential to generate only small negative impacts on the environment and society. The combination of support to a program of small infrastructure expansion with the use of a modified version of program procedures and of results-based disbursements make this operation one of the frontier Bank IL operations providing program-based support. However, this approach came at a cost. First, it excluded all contracts under the program that were ongoing at financing effectiveness, reducing the Bank’s ability to influence the performance of those contracts. Second, both the Government and the Bank spent much of their project preparation resources in finding an acceptable way for the Government to conform to prescriptive IL requirements, instead of partnering to improve the program’s cost- efficiency and sustainability. Third, because of the transaction focus of IL policies, the transaction cost of implementation support will be high: the project is expected to finance over 7,000 contracts, spread over a territory covering about one-third of India. Under the Brazil Minas Gerais Partnership for Development II project, cumulative IBRD lending of $1.5 billion supports the Minas Gerais “State for Results� program for improved public sector management in several sectors and stronger results management by the state government. IBRD resources are disbursed against indicators in six sectors: public sector management, private sector development, health, education, transport, and rural poverty reduction. The operation includes extensive technical assistance, some of it financed under the loan and some provided by Bank staff as part of project support, including in the areas of public procurement and performance- based contracting of infrastructure and services. IBRD provides around 18 percent of program financing. Semiannual disbursements are proportionate to the achievement of indicators, provided that the Government has financed at least 70 percent of total program expenditures. The project has helped the Government implement a broad array of reforms and efficiency improvements under 20 expenditure programs or initiatives. In particular, it has supported the Government in modernizing fiscal management and improving tax efficiency; introducing a results-based public sector management culture; further involving the private sector in the state’s development; improving coverage, quality, and efficiency in health services delivery; achieving faster improvement in school quality and education; improving the condition of the state road network; and providing all-weather access to all municipalities in the state. However, to fit within the parameters of the IL instrument, the Bank had to exclude about 50 percent of the civil works contracts from the program because such contracts, while deemed to follow good procurement practice, did not strictly follow Bank procurement rules. For the same reason, the program could not include any consulting services contracts procured following Government processes. Choosing projects that avoid procurement (and safeguards) for procedural reasons deprives the Bank of the opportunity to support important improvements in both of these critical areas. 8 (a) It often requires a double layer of controls (to ensure the implementation of activities supported by the operation with respect to the requirements of both the government and the Bank), which can impose additional costs on clients without any perceived additional value. (b) It has led to project designs that are biased toward activities that are easier to accommodate within the current Bank procedures rather than being primarily driven by delivering maximum results. (c) “Cherry-picking� specific activity types under a program limits the Bank’s ability to partner with the client on implementing improvements in the effectiveness and efficiency of the entire program. (d) Overlaying concerns about applying Bank rules to a country program and complying with those rules has often resulted in giving greater attention to procedural compliance than to key institutional issues. (e) Uncertainty about how Bank policies and procedures (including those on procurement and environmental and social safeguards) designed for more traditional IL projects should apply to more program-based lending has heightened risks for both client countries and the Bank. (f) When relying on government systems, it has been difficult to use procedures designed for project lending for more program-based and results-focused lending—for example, matching expected results with programmed resources or tracking eligible expenditures to specific outputs and outcomes established in the results matrix. III. OVERVIEW OF PROPOSED PROGRAM-FOR-RESULTS FINANCING 16. In proposing a new instrument, Management is responding to changing demand and a changing country engagement model and drawing on experience with program-based operations. Client countries are increasingly implementing their own programs for development and poverty reduction, programs that are rooted in their own legal, policy, regulatory, and institutional environments. They are asking development partners for finance and expertise to improve such programs’ effectiveness and efficiency in achieving results. In this context, Management believes that the Bank needs a suite of lending instruments that responds to three broad sets of demands from borrowers:6 (a) Policy support operations: operations that support policy and institutional actions to achieve a country’s overall development objectives and provide rapidly disbursing general budget support to help address development financing needs. (b) Project support operations: operations that support specific investment projects and disburse against specific expenditures and transactions. 6 The Bank also offers guarantees to help mobilize private sector finance. 9 (c) Program support operations: operations that support government programs and system strengthening, and that disburse against results. 17. Key Features of Program-for-Results. The Bank responds to the first two sets of demands with the DPL and IL instruments. To address the third group, Management is proposing a new instrument with the following four key features: (a) Finances and supports borrowers’ programs. These programs, comprising expenditures and activities, can be ongoing or new, sectoral or sub-sectoral, and national or sub-national programs, as well as community development programs. The Bank assesses the quality of the program, including its results framework, expenditure framework, supporting systems, and the scope for system-strengthening measures. (b) Disburses upon achievement of program results. Disbursements will be determined by the achievement of monitorable and verifiable indicators rather than by inputs. Together with funds from other sources, Bank disbursements will finance the borrower’s expenditure program rather than being linked to individual transactions. (c) Focuses on strengthening the institutional capacity needed for programs to achieve their desired results. A priority area for both preparation and implementation support will be to strengthen the capacity of the institutions to implement the program, thereby enhancing development impact and sustainability. Program governance will be strengthened by focusing on transparency, accountability, and participation. (d) Provides assurance that Bank financing is used appropriately and that the environmental and social impacts of the program are adequately addressed. The Bank will assess the program’s fiduciary and environmental and social management systems. It will agree with the borrower on the additional measures needed to provide assurance that the loan proceeds are used for program expenditures, that these expenditures are incurred with economy and efficiency, and that potential impacts to the environment and affected people are adequately addressed. In addition to these features, Program-for-Results will enhance the Bank’s ability to partner with other development organizations by pooling resources and focusing directly on capacity building. There will be important opportunities for the Bank to work with other partners at all phases of the development and implementation of Program-for-Results operations, and this should enhance the effectiveness of development partners’ collective efforts. It should also increase efficiency by reducing transaction costs for the government and development partners themselves. 18. Instruments to Respond to Different Development Challenges. The proposed new Program-for-Results instrument will complement, not replace, the Bank’s existing two lending products. While all Bank instruments focus on development results, borrowers will be able to choose from a wider range of instruments to suit their objective, the results they desire, and risks they face. The description below notes the differences and complementarities among the instruments (see also Table 1). 10 (a) Development policy lending. DPL will remain the primary Bank instrument to support policy and institutional actions. It focuses on discrete policy actions that are under the direct control of governments, and it links disbursements to evidence that those actions have been adopted. DPL is a practical and effective way of supporting policy actions that help create the enabling conditions for improving results—for example, when new regulations are required for the better functioning of markets or new policy frameworks are necessary to improve government efficiency. DPLs provide general budget support and do not earmark loan proceeds for specific programs. Moreover, DPLs disburse against specific policy and institutional actions and not against the results or outputs/outcomes associated with sector/program expenditures. (b) Investment lending. IL is the Bank’s main instrument to finance investment projects. It supports a heterogeneous range of operations, but their common characteristic is typically the financing of specific investment activities that involve a set of expenditure transactions, most of which are used for the purchase of works, goods, and services. IL is a practical and effective way of supporting the achievement of results when risk management and controls are needed on the inputs side (e.g., construction, technology) and when technical design and implementation challenges are critical bottlenecks to achieving results. Often such situations involve discrete, one-off activities (for example, the construction of a large infrastructure project or the purchase of expensive and technically complex equipment). By focusing on the proper implementation and risk management of individual transactions, IL operations put the emphasis of Bank-client relations on making sure that the right inputs and technology are in place and the operation is implemented as planned. (c) Program-for-Results. Many of the development challenges countries face cannot be addressed just through discrete policy actions or through the proper technical implementation of a project. For example, improving service delivery (e.g., better maintained roads, functioning schools and health clinics, effective agricultural extension services) may well require both policy actions (e.g., a decentralization law) and some discrete investment activity (e.g., constructing new schools or contracting works for road maintenance). But in many cases, these are not sufficient for the achievement of results. Schools can be built, but teachers may remain absent; health clinics may have new equipment, but essential drugs may not be available at the point of service; and rural roads may remain un-maintained in spite of the existence of contracts. Addressing such bottlenecks involves improvements in the governance of institutions and systems, including capacity building and changes in management practices and behaviors by service providers and users alike. Program-for-Results will be the instrument of choice when the objective is to support the performance of a government program using the government’s own systems, when the results require expenditures, and when the risks to achieving the program’s objectives relate to the capacity of the systems—for example, the M&E, fiduciary, and environmental and social systems—to achieve better results. Program-for-Results will also enhance the Bank’s ability to partner with other development organizations in supporting government programs by pooling resources and focusing more directly on capacity building. 11 Table 1. Complementary Lending Instruments Project support lending Program support lending Policy support lending Category (IL) (Program-for-Results) (DPL) Purpose Supports specific investment Supports government Supports policy and operations programs or subprograms institutional actions Disbursement Disburses against specific Disburses upon achievement Disburses against policy and mechanism expenditures that support the of results and performance institutional actions operation indicators Implementation Bank IL rules and procedures Program systems Country policy processes mechanisms Funds for specific Funds for specific Non-earmarked funds for expenditures expenditure program general budget support 19. The Need for a New Operational Policy and Procedures. The distinct nature of Program-for-Results warrants a dedicated instrument and associated set of operational policies and procedures. Program-for-Results will support a government development program that may predate Bank involvement and involve other development partners; and Bank funding may often represent only a modest part of the total financing, with all partners endeavoring to rely on and improve the program’s own systems and controls. In these circumstances, it is more appropriate to make the performance of the program systems and institutions a central focus of the Bank’s engagement. Therefore, it is not realistic for the program to follow the Bank’s IL policies and procedures. IL policies and procedures were designed to fit a discrete investment project in which close attention to the details of how inputs are obtained is a critical ingredient to success. IL policies are transaction-based and are, in many ways, detached from government legal and other frameworks; and some assume that the Bank funds can be segregated from other sources of finance or earmarked for specific expenditures. Similarly, the operational policy for DPL is not applicable because it focuses on the appropriateness of the macroeconomic framework and on critical policy and institutional actions necessary to enhance growth and reduce poverty, rather than on the quality of specific sectoral or other development programs and their accompanying systems and the measures necessary to improve those programs and systems. Accordingly, it is more appropriate that Program-for-Results be governed by a single dedicated operational policy (OP) and Bank procedures (BP) statement, accompanied by additional guidance to staff.7 20. Support for Country Systems. Effective country systems and institutions are fundamental to the Bank’s development mission. The Bank has joined developing countries and other development partners in endorsing the Paris Declaration on Aid Effectiveness and the Accra Agenda for Action, which express the international consensus in favor of strengthening and expanding the use of country systems. Program-for-Results provides an additional opportunity for the Bank to use the country institutions that are responsible for implementing the program being supported, thereby strengthening their governance and performance. Program-for- Results will therefore enable the Bank to focus more systematically on the strengthening of country systems by improving or enhancing such systems at the sector and program levels. The approach to meeting the Bank’s fiduciary and environmental and social concerns under Program- for-Results will focus on an assessment of the quality of program processes and systems, on the improvements necessary to ensure better systems and results, and on what the Bank can do to 7 In addition, the general Bank OP/BPs will continue to apply to this instrument. 12 help realize such improvements. Implementation support will provide more focused technical advice to improve systems and capacity building to improve performance. While Program-for- Results will contribute to the broad agenda of country systems, it will be different from the Bank’s country systems pilots in two ways. First, it will recognize that institutional performance is not homogeneous within a given country and that it may be possible to work with some institutions even when there are wider national weaknesses. Second, while maintaining the Bank’s overall commitment to high international standards, Program-for-Results will not seek procedural equivalency to the Bank’s policies and procedures designed for IL operations. 21. Governance and Transparency. Program-for-Results will support improvements in governance and transparency. The program-level focus provides an opportunity to build stronger governance systems and institutions across the entire program. By linking disbursements to results, the nature of the relationship between the Bank and the client moves away from individual transactions and focuses on the behavioral and institutional changes that are required to achieve those results. An essential implication of the new instrument is the development of strong and credible means of making program information publicly available8 and monitoring the achievement of results, including through enhancing the role of users and civil society organizations where needed. This approach will greatly contribute to improved accountability. In addition, the Bank is bringing more transparency than ever to its operations. Under Program- for-Results, as the documentation covers the entire program, which will typically be much larger in size than the Bank’s financing, the Bank’s “transparency footprint� will be significantly enlarged. Moreover, publicly disclosed documentation will make the results of the program and its achievements more accessible. This in turn will give stakeholders access to information about the performance of the public institutions and their achievements and will help strengthen the culture of accountability and transparency in these programs. 22. Appropriateness of Program-for-Results. Demand for Program-for-Results comes from a broad range of member countries, including both low- and middle-income countries, and covers a broad range of program types in many different sectors. As with both DPL and IL, there will be no a priori restrictions on the use of Program-for-Results in terms of income categories or sectors. Instead, decisions on the feasibility and scope of Program-for-Results will take account of the following considerations: (a) Consistency with CAS/CPS. The appropriateness of Program-for-Results will be determined in the context of the CAS/CPS and the Bank’s overall assessment of the country’s policies and programs, its institutional capacity to undertake those programs and achieve related results, its commitment to improving those programs and the associated systems, and the risks associated with those programs and systems. (b) Priority and definition of the program. Since Program-for-Results operations are expected to support new or existing programs, the Bank will assess the priority 8 The results framework, the overall expenditures, and the DLIs of the programs will be publicly available. Consultations on the programs’ draft environmental and social systems assessment will be undertaken. Civil society will be involved in program implementation monitoring, when possible. In addition, all key Bank documents in support of programs will be disclosed: the Program Appraisal Document (the results of all assessments, risk ratings, agreed actions in response to assessments, and the results framework), Implementation Status and Results reports, annual audit reports, and Implementation Completion Reports. 13 attached to the program and its expenditures to be supported in the context of a country’s overall development strategy and plans. It will also assess the overall scope of the program. A program could cover an entire sector or subsector, or it could be a subnational program or a multisectoral program. In some cases it will be a “time- slice� of an ongoing program. (c) Constraints to achieving development outcomes. Once the program is defined, a determination of the constraints to achieving the development outcomes will be carried out to decide on the appropriate instrument to use. If the primary bottleneck is one that requires policy actions, it may be more appropriate to use a DPL. If the key bottleneck relates more to the right inputs and technology being in place, then a focus on managing inputs using the IL instrument may be more appropriate. For Program- for-Results, the implementation challenges will be predominantly institutional, with improved overall program performance demanding a focus on incentives and accountability for results. (d) Excluded high-risk activities. Activities that pose a risk of potentially significant and irreversible adverse impacts on the environment and/or affected people (activities classified as Category A under IL), and activities that involve procurement of works, goods, and services under contracts whose estimated value exceeds specified monetary amounts (high-value contracts),9 are not eligible for Program-for-Results financing and will be excluded from Program-for-Results operations. 23. Processing Program-for-Results Operations. For each Program-for-Results operation, as for operations using the Bank’s other lending instruments, the Bank will carry out a process of identification, preparation/assessment, appraisal, and implementation support, described in the next chapters. (a) Identification. In preparing the Program-for-Results concept note (PCN), the Bank will provide a preliminary description and assessment of the program and the results to be achieved, including initial findings on its strategic relevance and economic impact, the level of borrower commitment, the program’s objectives and key results (and associated risks), implementation arrangements, and overall performance (for ongoing programs). (b) Preparation/Assessment. The task team will work in close partnership with the borrower and, as relevant, any other development partners to deepen the assessment of the technical aspects of the program, including results, expenditures and DLIs, and the quality of the systems supporting the program. All of these assessments will involve a rigorous review of fiduciary as well as environmental and social effects management systems, capacity, and performance. 9 The amounts are those, as may be amended from time to time, that require mandatory review by the Bank’s Operations Procurement Review Committee (OPRC): currently $50 million for Works, Turnkey and Supply, and Installation contracts; $30 million for Goods; $20 million for IT systems and Non-consulting Services; and $15 million for Consulting Services. 14 (c) Appraisal. The task team will pull together the findings of the various assessments and review with the government the scope of the program and proposals for program strengthening and capacity building. The Bank will also complete an economic evaluation of the program. (d) Implementation. The Bank will support the borrower in carrying out the improvement measures agreed at appraisal and will monitor the overall progress of the program (including the achievement of results and expenditure performance), as well as changes in the program’s risks and compliance with the provisions of the legal agreements. It will provide support to resolve emerging program implementation issues and help build institutional capacity. 24. Resource Implications. While past experiences with IL projects with Program-for- Results features suggest that costs are within the norm for IL projects, it is difficult to predict the internal budget implications because cost will be influenced by many factors—previous engagement in the sector and with the institutions that will implement the program, capacity of clients, involvement of other donors, and so forth. No additional resources are requested for Program-for-Results. As the instrument is totally demand-driven, Management will make any necessary trade-offs and allocations within the overall country/Regional budget envelopes. IV. PREPARATION OF PROGRAM-FOR-RESULTS OPERATIONS 25. During the preparation of Program-for-Results operations, the Bank will work with government counterparts to assess the scope of the program, including expenditures, expected results, Disbursement-linked Indicators (DLIs), the technical arrangements and supporting systems for achieving the program’s results, and the risks that the program will not achieve those results. In this context, the Bank will conduct three types of assessments: technical (including expenditures and results), fiduciary, and environmental and social. Each assessment will consider the overall quality of the program (including its expenditures) and its systems in its area—the framework/rules, implementation capacity, and actual performance under the program—and will identify improvements, including measures to mitigate major risks that the program will not achieve its expected results. The assessments will screen for excluded high- risk activities (see paragraph 22(d) above), and will provide inputs to the Program-for-Results integrated risk assessment, which will in turn provide feedback on the scope of the Program-for- Results program and the types of mitigation measures that might be needed. To the extent possible, assessments may be based on, or coordinated with, those of other development partners, and may use as inputs any relevant source of information, including existing assessments in earlier operations and analytic work at the country, sector, or program level. As needed, an action plan will be developed for the Program-for-Results operation, including improvements and risk mitigation measures. 26. Strengthening Institutional Capacity. As noted earlier, a key feature of Program-for- Results operations will be the focus of Bank support on strengthening the governance and institutional capacity essential to ensure that the programs achieve their expected results and can be sustained. Perhaps the most important element of capacity building is the focus on the behavioral and institutional changes that are required to achieve results and manage associated risks. The assessments that will be carried out as part of preparation will identify, as appropriate, 15 key program improvements and actions that can be undertaken over the time period of the operation. Since the assessments will focus on the whole program, their recommended measures will help reduce the fragmentation of capacity-building efforts within a Bank country program, focus attention on core issues affecting the performance of the program, and thus enhance the effectiveness and sustainability of these measures. The action plan should also consider improvements to enhance the long-term efficiency of a borrower’s program while taking into account the feasibility of implementation during the operation’s timeframe. 27. Technical Assistance and Capacity Building. In addition to the overall program-level approach of Program-for Results, in many cases the Bank will actively support capacity building through dedicated technical assistance activities. Program-for-Results offers a number of modalities for providing such assistance. Technical assistance, where needed, can be provided as an integral part of a Program-for-Results operation; as a separate stand-alone IL or an IL component within a Program-for-Results operation (i.e., as a hybrid operation); or through parallel efforts financed by development partners. Management will seek to establish a trust fund to support such activities. The trust fund will be demand-driven, depending on the needs of the specific program. It will also be designed to complement, and avoid duplicating, efforts by the Bank or other development partners at the country level. 28. Addressing Material Weaknesses. If any of the assessments (in the technical, fiduciary, and environmental and social areas) reveal material weaknesses, the Bank’s response will depend on the severity of those weaknesses and the potential for addressing them. If the weaknesses are so severe that credible remedial measures at the program level are judged unlikely to work, the Bank may decide not to finance the operation or to use another instrument that can better mitigate the risks. For less severe risks, the Bank will agree with the borrower on measures to address and rectify them before or during program implementation—for example, in one or more of the following ways: (a) Measures could be identified to improve capacity, systems, and procedures, and could be supported by the Bank’s financing, or another donor’s, or from the client’s own resources. Progress in implementing these measures—some of which could be in the form of legal covenants or included in an action plan—would be monitored during implementation. (b) The DLIs could include specific actions, covering the areas of weakness, as results that would be monitored during implementation. (c) Specific actions could be agreed for inclusion in an operations manual for the program (if one is available or required) or in the internal regulations of the client, and would be monitored during implementation. (d) Adjustments could be made to the terms of reference of the audits and/or Bank monitoring during implementation to focus on the areas of weakness or riskiness. A. Technical Assessment 29. The technical assessment is a critical step of the processing of Program-for-Results operations—it includes a dialogue about the strategic relevance of the program to be supported, 16 the results framework, the related expenditures, the design and verification of DLIs, monitoring and evaluation, the technical and institutional dimensions of the program and related governance and capacity issues, and so forth. The technical assessment will address the following kinds of questions: How will programs supported by a Program-for-Results operation be situated in a strategic context in the country? Is the program economically justified?10 How will the expenditure program be framed and addressed? And how will results be defined, measured, and monitored to ensure the desired development impact? The assessment will draw on existing documentation, analytic work, and sectoral best practices as its basis, thereby building on the evaluation tools developed by the Bank and its partners over past decades. The focus on the achievement of results, and its direct link to the disbursement of Bank financing, will reinforce the results orientation of the program while also serving as an indicator that funds are being used appropriately. Based on the assessment’s findings, the Bank and the borrower will agree on specific measures to be included, as needed, in the operation’s action plan and legal agreements. 1. Strategic Relevance, Technical Soundness, and Institutional Quality 30. Strategic Relevance. The Bank will seek to determine that the program will address an important goal in the sector, and that there is a clear rationale for government intervention. This will involve establishing that the issues to be addressed by the program are priority for economic development and poverty reduction in the country. It will also establish that there is a case for government intervention in resolving the issue (for example, addressing market failure, spillovers, redistribution, or social concerns) and identify whether government intervention should take the form of regulation, financing, or outright provision. The possibility of some form of public-private partnership may also be considered. The Bank will then examine whether the proposed program is adequately structured to resolve the issue at hand, taking into account international good practice, and whether there is a significant likelihood that the issue can be substantially resolved as a consequence of the program’s implementation. Much of the information for such an assessment is likely to be available in development plans, sector strategies, and studies. 31. Technical Soundness. Drawing on the appropriate sectoral experts, the Bank will seek to determine that the program is designed and implemented to efficiently produce results and achieve the program’s objectives. The assessment will review whether the nature and technical design of the program activities and expenditures are adequate to reach the program’s objectives; whether the incentives are in place for program stakeholders to contribute effectively to the program’s success; what the quality of the program’s organization is, including its ability to plan, design, execute (including manage and supervise contracts), and monitor program activities; and, for ongoing programs, what the overall performance of those programs has been. Some of the information needed for such an assessment is likely to be available in existing reports and evaluations. The Bank will also be able to rely on the knowledge and experience of internal and external experts, including experts on project management, M&E, and sector-specific technical issues. 32. Institutional Arrangements. The Bank will assess the institutional arrangements for program implementation and monitoring by reviewing whether the necessary entities are 10 For more information on the economic evaluation of the program, see Section V, Subsection D. 17 involved, roles and responsibilities are clearly defined, incentives are adequate to ensure cross- entity collaboration, and the government is committed to the program’s objectives. The assessment will then focus on the institutional arrangements of the program implementing agency, including whether it has the leadership and human resources needed to carry out the program, and whether there are clear processes for decision-making in the agencies and adequate incentives for staff and service providers. In addressing these various dimensions of institutional quality, the assessment will help determine the quality of governance of the implementing agencies in terms of their transparency, accountability, and skills and competence. This institutional assessment will draw on available sector and program studies, the findings of the systems assessments, and feedback from stakeholders and experts. 2. Expenditure Framework 33. Expenditure analysis will review the level, efficiency, and effectiveness of the government expenditures on the entire program, not just on the share or components “financed� by the Bank. The scope of expenditure analysis will be determined by the size and complexity of the government program, not the size of the Bank’s financing. Key questions for the assessment could include the following: Does the budget classification system permit tracking of expenditures and provision of timely information on expenditure composition? Are program expenditures—especially those with direct implications for the long-term sustainability of the program—part of a medium-term expenditure profile that provides financial sustainability and funding predictability, including consistency with the country’s overall financial profile? Is the borrower able to execute the expenditures budgeted under the program, for example, avoiding generating a pattern of arrears and “stops-and-goes� under contracts with the private sector? Do the level of the budget and its allocation across activities adequately reflect the resources required to generate the expected results? What about overall efficiency of expenditures of the program? Does the analysis identify activities in the program that should be excluded based on criteria defined in paragraph 22 (d) above? 3. Results Framework and Monitoring and Evaluation 34. While the Bank gives attention to results in all the operations it finances, programs supported by Program-for-Results will make results the basis for disbursements, for the first time formally codifying the link between the two. Measuring results will move from being just one element of operation design to being the primary driver of Bank financing. Program-for-Results operations will therefore have strong built-in incentives for both the client and the Bank to focus attention on the definition, achievement, and measurement of results. Thus it will be particularly important to have in place effective M&E systems. 35. Framework for Results. The results of the government program will be the results of the Program-for-Results operation. The Bank will assess the program’s results framework that defines program outputs, intermediate outcomes, program outcomes, and goals. While disbursements will be based on a selection of key milestones from the indicators in the results framework—the Disbursement-linked Indicators (DLIs)—a clear understanding of the whole results chain is essential to ensuring that incentives for program improvement and strengthening are in place. 18 36. Results frameworks can be expected to vary in complexity and scope, depending on the definition and scope of the program to be supported, the indicators selected, and the client’s capacity to implement the program and monitor results. For an ongoing program the results framework may be implicit in a government strategy or other document. In such cases the process of defining and assessing the results framework may involve working with the client to frame and refine the program and establish expected results, including any actions or processes that may be required to achieve the program’s objectives. 37. Monitoring and Evaluation Arrangements. The Bank will assess the government’s capacity to monitor, evaluate, and validate program results, reviewing the following with reference to the requirements and results framework of the program to be supported by the Program-for-Results operation: the institutional arrangements for monitoring and evaluating results; existing M&E plans; the quality and reliability of existing M&E products; the framework to support effective M&E and information sharing; staffing and resources to support M&E functions; and capacity development needs. For some government programs the assessment may indicate that existing M&E arrangements and capacity are adequate; but in most cases, it is likely that M&E systems will need to be improved and their capacity strengthened. B. Assessment of the Program’s Fiduciary Systems 38. To make sure that funds are used appropriately, during the preparation of a Program-for- Results supported program, the Bank will assess the program fiduciary systems against the provisions set out in OP/BP 9.00, Program-for-Results Financing. The assessments will build on available analytic work and country diagnostics (including work undertaken by other development partners), and will draw on the Bank’s extensive experience acquired in supporting IL projects and other operations. The scope of the assessments will include the following: (a) A review of applicable rules and procedures, including oversight mechanisms. (b) A review of the capacity of the implementing agencies to follow those rules and procedures, and measures that could be introduced to strengthen that capacity (c) An assessment of the agencies’ performance in implementing these procedures under the program. (d) The definition of improvements to be carried out as part of the Program-for-Results action plan. 39. Fiduciary Principles. From a fiduciary perspective, program systems arrangements will need to be consistent with the key principles of economy, efficiency, effectiveness, accountability, and transparency, the same principles that govern investment lending. Overall procurement arrangements will be assessed as to the extent to which the regulatory and procedural framework, the institutional framework of functional responsibilities and accountabilities, procurement operations, and market practices all are expected to provide “best fit for purpose� through procurement processes that have acceptable levels of planning, bidding, evaluation, contract award, and contract management. The financial management arrangements will be assessed as to the degree to which the relevant planning, budgeting, accounting, internal 19 controls, funds flow, financial reporting, and auditing arrangements provide reasonable assurance11 on the appropriate use of Program-for-Results funds and safeguarding of its assets. 40. Fiduciary Systems Assessment. The assessment will be carried out at the program level using, where available, existing knowledge about country and programs’ systems as well as the implementing agency. The nature and scope of additional analysis to support program design and implementation support will vary among countries, sectors, levels of subnational government, and organizations. The assessment also considers how program systems handle the risks of fraud and corruption, including by providing complaint mechanisms, and how such risks are managed and/or mitigated. Based on the findings of the assessment, and where needed, the Bank and the borrower will agree on measures to be included in the action plan for the operation. 41. The fiduciary systems assessment will focus on the borrower’s regulatory framework, program procedures, and fiduciary management capacity and implementation performance against the requirements of OP/BP 9.00 and the new Anticorruption Guidelines for Program-for-Results (ACGs). Program implementing institutions with demonstrated implementation effectiveness can be expected to manage significantly greater levels of risk, while new programs or newly organized implementing agencies will warrant deeper scrutiny and, depending on program context, may require more risk management and capacity-building activities in the action plan. 42. Coverage of the Assessment. The coverage of the fiduciary systems assessment will depend on the nature and scope of the program. The fiduciary assessment will be carried out at the program level using information available from diagnostics conducted by the Bank or other donors and complemented, as needed, with program-specific fiduciary considerations. Using the findings of the assessment, where needed, the Bank and the borrower will agree on measures to be included, as necessary, in the action plan and legal agreements for the operation and will identify: (a) measures to improve the performance of the program; (b) measures to strengthen design, preparation, implementation, control, and oversight capacity; and (c) any specific mitigation measures to ensure consistency with the fiduciary requirements, taking into account the new ACGs. Progress in implementing these measures would be monitored during implementation. 43. Excluded High Risk Activities. Activities that involve procurement of works, goods, and services under contracts whose estimated value exceeds specified monetary amounts (high-value contracts) are not eligible for Program-for-Results financing and will be excluded from Program- for-Results (see paragraph 22(d)). C. Assessment of the Management of Environmental and Social Effects 44. Program-for-Results will seek to make sure that the environmental and social effects of the program are adequately addressed. During the preparation of a Program-for-Results operation, the Bank will assess, against the requirements of OP/BP 9.00, the degree to which the program systems manage and mitigate the environmental and social impacts of the overall program. The requirements of the assessment of environmental and social systems are guided in purpose and goals by the environmental and social safeguards policies of investment lending projects, 11 Based on the principles and requirements set out in paragraph 7 of OP 9.00, “reasonable assurance� will be determined initially by the task team with appropriate technical inputs, then reviewed and decided upon by management, and finally considered formally by the Board. 20 adapted to the needs of the Program-for-Result instrument. The assessment will also identify and exclude high risk activities as defined in paragraph 22(d). 45. Environmental and Social Principles. The assessment will be based on the following principles:  Environmental and social management procedures and processes are designed to (a) promote environmental and social sustainability in program design; (b) avoid, minimize, or mitigate against adverse impacts; and (c) promote informed decision- making relating to a program’s environmental and social impacts. And where relevant:  The program avoids, minimizes, and mitigates adverse impacts on natural habitats and physical cultural resources resulting from the program.  The program protects public and worker safety against the potential risks associated with: (a) construction and/or operations of facilities or other operational practices under the program; (b) exposure to toxic chemicals, hazardous wastes, and other dangerous materials under the program; and (c) reconstruction or rehabilitation of infrastructure located in areas prone to natural hazards.  Land acquisition and loss of access to natural resources is managed in a way that avoids or minimizes displacement; and affected people are assisted in improving, or at the minimum restoring, their livelihood and living standards.  Due consideration is given to cultural appropriateness of, and equitable access to, program benefits, with special attention provided to the rights and interests of the Indigenous Peoples and to the needs or concerns of vulnerable groups.  The program avoids exacerbating social conflict, especially in fragile states, post- conflict areas, or areas subject to territorial disputes. 46. Systems Assessment. The systems assessment will first take account of the range of environmental and social tasks to be undertaken and the scale of potential effects under the program. The assessment will then focus on the organizational and procedural considerations that are relevant to environmental and social management, as well as on the capacity of the implementing agency to effectively implement necessary actions and on its performance in practice. The assessment will be adapted to the country and program context, and will review the regulatory and policy bases for appropriate environmental and social planning and action, organizational authority and capacity, any interagency coordination arrangements, and the effectiveness of environmental and social management in practice. Key organizational dimensions include, among others, whether the program implementing agency has the necessary authority to manage the environmental and social effects of the program; whether its staffing and skills are adequate; how the program is coordinated; and whether it promotes credibility and accountability. Other dimensions are the effectiveness of the implementation of the existing regulatory framework; the quality of early screening for environmental and social effects and of 21 up-front review of any proposed subprojects; the extent and quality of consultations with stakeholders; and the effectiveness of established grievance procedures. Finally, the assessment includes a review of the arrangements by which program activities that affect communities will be disclosed, consulted upon, and subject to a grievance redress process. Measures to address consultation, disclosure, and grievance should be appropriate to the activities to be supported under the program. This assessment will inform mandatory remedial actions. 47. Consultation and Disclosure. Before Program-for-Results appraisal, the task team makes publicly available the draft environmental and social system assessment. The task team subsequently consults with Program stakeholders on the draft assessment. The stakeholder consultation has the objective of testing the findings of the draft assessment against the knowledge and perceptions of stakeholders who are involved in the government program. Relevant information regarding environmental and social issues will be disclosed through an enhanced Program Information Document (PID), replacing the Integrated Safeguards Data Sheet that was designed specifically for IL operations. The Program Appraisal Document (PAD) will describe the potential environmental and social impacts, program capacity to mitigate impacts, and capacity-strengthening or other specific measures deemed necessary to ensure effective performance. The PID will summarize potential impacts and mitigation arrangements described in greater detail in the PAD. Both PID and PAD will be disclosed in accordance with the Bank’s Access to Information Policy. 48. Excluded High-Risk Activities. Activities that pose a risk of potentially significant and irreversible adverse impacts on the environment and/or affected people (activities classified as Category A under IL) will be excluded from Program-for-Results operations (see paragraph 22(d)). D. Integrated Risk Assessment 49. Risk assessment is an important aspect of Program-for-Results operations. For each operation, an integrated risk assessment will build on the work done in the various assessments outlined above. The assessment will focus primarily on the risks to achieving the development objectives and results. The risk assessment will be a dynamic process, starting in early stages of preparation, and continuing through implementation. It will be an important input for Management to determine and maintain an adequate balance between expected program results and related risks. It will also help staff to continuously monitor the evolution of risks, identify the emergence of new risks, assess progress in implementing risk management measures and their impact, and, as necessary, devise appropriate adjustments to support the achievement of program results. The results of the risk assessment are a key input to the Bank’s decision on whether to go forward with the preparation of a Program-for-Results operation, to authorize the operation’s appraisal, and to determine the level of Bank implementation support to a given government program. If the overall program risk is deemed to be high, Management may decide not to move ahead with preparation of the operation or may raise the decision to the corporate level. 50. Risk Categories. The risk framework will draw on the lessons of the investment lending Operational Risk Assessment Framework, which will be adapted to the program nature of the intervention. It will include key risk categories, such as the following: 22  Operating environment risks, including: (a) country risk, examining, for example, the extent to which macroeconomic conditions may affect prospects for implementation of the program supported and consistency with the government’s overall fiscal program. It will also look at issues like political economy, governance (including fraud and corruption), and the independence of civil society; and (b) stakeholder risk, including risks emanating from the government, the general public, key donors, development partners, or other stakeholders.  Program risks, including: (a) technical risk, relating to the program’s economic rationale, technical soundness, institutional capacity, governance, sustainability, and M&E arrangements; (b) fiduciary risk, relating to the program’s fiduciary systems arrangements, capacity, and performance, as well as to controls and audits; (c) environmental and social risk, relating to the potential impacts of the program, the systems in place, and capacity and performance for avoiding, mitigating, or managing such impacts; (d) DLI risk, relating to the program’s results framework, the type of indicators selected, and the measurement and verification of results, and (e) other program-related risks not covered under the above four dimensions. As governance is a cross-cutting theme, risk evaluation under each of these dimensions will take into account governance risks pertaining to that dimension. V. APPRAISAL AND APPROVAL OF PROGRAM-FOR-RESULTS OPERATIONS 51. In appraising a Program-for-Results operation, the Bank will integrate the findings of the various assessments to determine the overall quality of the program and its associated systems, the borrower’s commitment to and potential for making improvements, and the risks (with and without improvements) that the program will not achieve its expected results. On the basis of these findings, the Bank will agree with government counterparts on the scope of the program to be supported and its objectives, results, and a set of DLIs, as well as on measures, if needed, for systems strengthening and capacity building, including with respect to the governance of the program and M&E. As necessary, these improvements will be described in an action plan for the operation. As part of appraisal, the Bank will also agree with the borrower on the audit arrangements and will finalize the economic evaluation of the proposed operation. Taking all this information into account, Management will decide whether the overall balance between the benefits and risks of the operation justifies proceeding with the negotiations of the operation and presenting it to the Bank’s Board of Directors for approval. A. Program Action Plan and Capacity Building 52. Based on the outcome of the various assessments, the Bank will agree on action plans with the borrower, to the extent possible with input from other development partners. At least three types of improvements may be included in the action plan: (a) Actions to improve the technical dimensions of the program and the formal rules and procedures governing the organization and management of the systems used to implement the program. (b) Actions to enhance the capacity and performance of the agencies involved. 23 (c) Risk-mitigating measures to increase the potential for the program to achieve its results and to address fiduciary, social, and environmental concerns. The action plan will be drafted as part of the preparation/assessment process and further refined during appraisal. 53. Capacity Building. Of particular importance for programs supported by Program-for-Results will be the measures for improving capacity and strengthening institutions. Building on the findings of the assessments, the Bank (and other development partners, as relevant) will work with country counterparts to develop a better understanding of capacity constraints to achieving the specified results, and will then devise measures to, among other things, enhance transparency, strengthen incentives and accountability, and improve the demand-side aspects of governance. B. Disbursement-Linked Indicators (DLIs) and Disbursement Arrangements 54. DLIs play a critical role in Program-for-Results operations because they provide the government with incentives to achieve key milestones in the program’s results framework or to improve performance under the program. They are also the basis for the disbursement of Bank funds. Since Program-for-Results financing is likely to support a diverse set of programs at various stages of development, the types of DLIs need to be sufficiently flexible to respond to the wide range of program circumstances. For instance, a program in the initial stages of implementation may have DLIs that are more focused on basic institutional capacity-building actions and intermediate output indicators, while programs that are well advanced or “mature� in terms of their development may have a larger number of DLIs that are outcome indicators. 55. Definition of DLIs. DLIs can be outcomes (e.g., infant mortality rate, literacy rates, etc.); outputs (e.g., immunization rates); intermediate outcomes, outputs, or process indicators (e.g., confirmation of substantive participation in decision-making by specified communities); or financing indicators (e.g., share of strategic projects in total expenditures). DLIs can also be key actions aiming to address specific risks or constraints to achieving the results (e.g., actions to improve fiduciary systems, environmental and social management, and/or monitoring and evaluation). What is common to all is that the selection of a DLI will be driven by the desired outputs and outcomes of the programs. The selection of DLIs will also take into consideration the practical aspects of measuring, monitoring, and verifying achievement of the results. DLIs will be tangible and verifiable. A given DLI can have one or more values to be achieved over the lifetime of a Program-for-Results operation. 56. Verifying the Achievement of DLIs. Each DLI will be required to have a credible verification protocol that will define the DLI and set out how it will be measured; whether it is discrete (i.e., all or nothing) or scalable; how the verification will be done; and who will be responsible for verifying its achievement. The verification protocol for each indicator will be identified and appraised. Each indicator needs to have a credible mechanism that is acceptable to the Bank and produces the needed information. Depending on the nature of the specific DLI, country circumstances, and the overall context of the program, the verification of DLIs can be carried out in various forms and by various parties—for example, government agencies, semiautonomous entities, statistical or audit entities, and third-party entities such as nongovernmental organizations and representatives of civil society. Not all DLIs will need to use 24 the same verification mechanism. The primary objective is to ensure that a credible mechanism is in place for monitoring, measuring, and verifying the achievement of the DLIs. Moreover, information about DLIs will be in the public domain and accessible to various stakeholders. 57. Scalability of Disbursements. The achievement of each DLI will lead to a pre-defined amount of funds to be disbursed. Hence, in a situation where a number of DLIs were to be achieved, disbursements will be made only on the basis of those DLIs which were actually achieved. Moreover, disbursements against DLIs may be scalable, with the disbursement of financing proceeds proportional to the progress toward achieving the DLI. For example, a DLI could be “increase of 10 percent in the percentage of female students enrolled in primary education.� If the increase was only 8 percent in the defined period, then, for example, 60 percent of the financing corresponding to the particular DLI could be disbursed. Scalability of disbursement may not apply to all DLIs. For instance, if a DLI refers to an action (e.g., modernization of procurement system in place), then it is either done or not. Although the instrument allows for disbursement scalability, if there are no results, there will be no disbursements. 58. Timing. Most borrowers are expected to request that Bank disbursements occur on a regular basis during the implementation period. When determining the timing for achieving an operation’s DLIs, the Bank will take into consideration the borrower’s funds flow needs as well as ensure that the critical steps needed to achieve the DLIs can be achieved by the completion of the Program-for-Results operation. The DLIs, the related results, and the loan amounts allocated to the achievement of those results are recorded in a DLI matrix. In some circumstances, certain results may need to be realized before the legal agreement is signed so that the desired program results can be achieved—for example, establishing credible baseline data or setting up the monitoring arrangements for the program, particularly for newly established programs. In such cases, the Bank will be able to disburse against DLIs achieved between the date of the Program Concept Review and the date of the legal agreement, as long as the aggregate amount of loan proceeds allocated to such DLIs does not exceed 25 percent of the amount of the Bank financing. This disbursement arrangement would affect the definition of the program and would be subject to all standard Program-for-Results financing requirements. 59. Revolving Advances. In some situations, advances may be helpful, or even necessary, for the borrower to finance the activities needed to achieve the results for one or more DLIs. Such situations might occur, for example, in fragile and conflict-affected states or where the Program-for-Results financing is supporting the start-up of a new program in a budget- challenged country. In these cases, advances could be considered to achieve not only the initial set of DLIs but also subsequent DLIs during the implementation period. Such advances should normally not exceed 25 percent of the total Program-for-Results financing. The amount of the advance is deducted (recovered) from the amount to be disbursed under a subsequently met DLI. Further advances can be made once an advance has been recovered, or partially recovered, as long as the overall limit is not exceeded. The combined amount of financing referred to in paragraphs 58 and 59 may not exceed 30 percent of the Program-for-Results financing.12 12 Unless a higher amount is approved by Management. 25 60. Disbursement Arrangements. Program-for-Results supports existing or new programs. As existing programs are usually part of the general budget process, when a DLI is achieved, the Bank will disburse the agreed amount as a contribution to the program that underpins the results. The borrower can request that the Bank loan proceeds be disbursed either into the borrower’s account at the central bank, or in some situations (for example, if the program is defined to be outside the borrower’s general budget or part of a stand-alone revolving road fund, a state-owned entity, or a municipality) into a separate account of the relevant entity. C. Program Audit 61. As part of appraisal, and taking into consideration the nature of the program, country context, and risk assessment, the Bank will agree with the borrower on an audit approach for the program. The scope of the audit approach will be determined by the potential vulnerable areas the fiduciary assessment identified as needing independent review/verification. The audit may not necessarily be undertaken by a single entity. Depending on existing capacity for accountability and oversight, different entities may be in charge of conducting different audits. The approach will be to use and, to the extent possible, strengthen existing program oversight entities. The audit will generally cover the following areas:  Program financial statements audit. The borrower is required to submit annual audited program financial statements after the close of its financial year. Audits need to be carried out by auditors with independence, experience, and capacity acceptable to the Bank, and under terms of reference acceptable to the Bank. The Bank and borrower agree on, and confirm in the legal agreement, the period for receipt of the annual audit reports (e.g., within 12 months after the close of the borrower’s financial year), taking into consideration the country and program context. Countries’ supreme audit institutions (SAIs)13 generally carry out regular financial audits of the execution of the state budget, including the programs supported by the Bank. However, in many cases SAIs do not issue a separate audit report for each individual program; or they may issue such a report, but only with significant delays. The Bank will discuss with the SAI, and reach agreement on, acceptable terms of reference to ensure that the annual program financial statements audit will be done in a timely and reliable manner and the audit report will be made public in a time and manner acceptable to the Bank. The Bank will monitor how the responsible agency(ies) implement the findings and recommendations of the audit report.  Program technical audits/reviews. The annual financial audit may need to be complemented with other audits or independent reviews conducted by the SAI, private audit firms, third-party verification agents, internal audit departments, or other arrangements (e.g., social inspection). These audits/reviews may include such areas as assessment of the performance of environmental and social impacts management; overall best fit for purposes of the procurement under the program, using proxy indicators; physical inspections assessing the quality of goods and works and the 13 If the SAI is unable (because of capacity considerations) or unwilling (because of independence concerns) to audit the program, the borrower and the Bank may agree on alternative audit arrangements for the program (for example, recruiting a private audit firm). 26 reliability of grievance mechanisms; the verification of outputs/results linked to disbursement (using the verification protocol for each DLI); the efficiency and effectiveness of administrative activities; and reviews of fraud and corruption “red flags� and complaint handling mechanisms. D. Economic Evaluation 62. The economic evaluation will take into account the scope of the program, the improvements to be made, the results to be expected, and the conclusions of the various assessments. The overall assessment of the economic justification (or evaluation) of the proposed operation answers the following questions: Is public provision appropriate for the program to be supported by a Program-for-Results operation? Is the current program economically efficient? What will be the development impact of the improved program? What is the Bank’s value- added? 63. Public Rationale and Economic Efficiency. Assessing the rationale for public provision of goods and services is an integral part of the economic evaluation of any government development program. The main rationales for public action—market failures, spillovers, redistribution, social and political concerns, and so on—will have been assessed as part of the strategic relevance assessment of the overall program. The assessment of the program’s expenditure framework should help determine whether an existing program is being delivered efficiently, taking account of available resources, and at a reasonable cost. 64. Development Impact. The economic evaluation should confirm that the program proposed for Program-for-Results support contributes to development, with expected benefits adequately exceeding expected costs. For many programs, this evaluation may take the form of a comparison of the with and without situations, supported wherever possible by evidence from similar programs in the country or elsewhere. If benefits are quantifiable but not in monetary terms, it may be possible to measure them with and without the improved program—for example, a reduction in the incidence of malaria. 65. Bank Value-Added. The Bank’s resources, principally funds and staff, increase the development impact of a Program-for-Results operation when their contribution adds to what can be realized by an exclusive reliance on the client’s own resources or those provided by other development agencies. Conceptually, the Bank’s value-added is the difference in the program’s development impact with and without the Bank’s involvement. It should be clear that the allocation of staff and resources to the Program-for-Results operation under consideration adds significant value and that the specific activities are well chosen. Three broad sets of activities can be identified: training, capacity building, and institutional strengthening; the learning dimension of Program-for-Results operations; and the risk-mitigating measures required under the risk assessment for Program-for-Results operations. E. Management Assessment of Benefits and Risks 66. At various stages in the processing of a proposed Program-for-Results operation, the Bank will assess the overall balance between the benefits and the risks of proceeding with the operation and will come to a considered judgment as to the best way forward. This will be a 27 focus of discussion as early as during the Program-for-Results concept review meeting; depending on the results of that discussion, a decision could be reached early on that the Program-for-Results instrument is not the appropriate one to use. During preparation, the Bank will keep under review the balance of risks, capacity, and rewards and, as necessary, adjust the various dimensions of the proposed operation accordingly. If there are significant changes in some of these critical areas, the team will alert Management to those issues. The balance of rewards and risks will again be considered during the Management decision meeting before appraisal. 67. Program-for-Results Operation Scope, Results, and Risks. The scope and quality of an ongoing program and of its accompanying systems, the borrower’s commitment to and potential to make improvements, the ambitiousness of the improvements planned, the results to be expected, and the risks associated with the program (both with and without improvements) will all influence the balance between results and risks. For example, the broader the scope of the program and the more ambitious the objectives for system-strengthening (e.g., with respect to fiduciary issues and environmental and social impacts), the greater will be the anticipated results—but probably also the greater the risk to achievement of the projected results. Working with government counterparts and other development partners, the Bank will come to an appropriate balance between results and risks—perhaps by adjusting the scope of the program or agreeing on strengthening or other risk-management measures. (The introduction of a large number of special risk management measures would likely signal that the scope of the operation is too ambitious or that the environment is too risky for Program-for-Results to be the most appropriate instrument.) 68. Overall Judgment in Determining the Fit of a Program for Program-for-Results Support. The Bank’s decision on whether to support a borrower program with the Program-for- Results instrument will be based on the results of all the assessments. A number of factors will be considered in determining whether the instrument is the right choice. For instance, if the quality of an existing program or the systems supporting it is low, but the commitment and potential to make major improvements with the support of the Bank and other development partners are high and the activities included in the program are relatively low-risk from a fiduciary, environmental, social, or other perspective, Program-for-Results may well be an appropriate instrument to use. At the other end of the spectrum, in some cases the quality of an ongoing program and its supporting systems may be good but there is relatively little interest in or commitment to making improvements or the activities to be supported by the program are complex and potentially high-risk. If the added value of Bank involvement is small and the risks relatively high, the Program-for-Results instrument may not be appropriate. Over time, the Bank will build up a case-load of experiences and best practices that will facilitate decision-making. Until then, while the instrument is new, enhanced corporate oversight will be put in place. VI. SUPPORTING IMPLEMENTATION OF PROGRAM-FOR-RESULTS OPERATIONS 69. Overall Borrower and Bank Responsibilities. Under Program-for-Results, as for operations under any Bank financing instrument, the borrower will be responsible for the program’s overall implementation. The Bank provides implementation support by: (a) reviewing implementation progress and achievement of the program results and DLIs; (b) providing support on resolving emerging program implementation issues and on building institutional 28 capacity; (c) monitoring the adequacy of systems performance through program monitoring reports and audit reports, field visits, and monitoring compliance with legal agreements; and (d) supporting the borrower in monitoring changes in risks. As appropriate, the Bank will work closely with other development partners during implementation by conducting joint missions and sharing implementation support responsibilities. A. Implementation Support 70. Reviewing Progress. The Bank will review implementation progress and achievement of the program’s results and DLIs. The Bank will support the borrower in identifying bottlenecks, emerging issues, and potential risks to the achievement of the program’s results. Timely review on the basis of program progress and other reports and field visits will enable the Bank to give advice on, and will enable the borrower to implement, any corrective actions needed to achieve the expected results in the face of changing circumstances. In this context, the quantitative/qualitative references developed for monitoring program performance will assist in comparing baseline and implementation information and in identifying areas that appear to be underperforming. For field visits, particularly when program sites are numerous and dispersed across wide areas, the Bank staff will devise a plan to ensure that, over time, they observe a representative sample of areas. Additionally, the Bank will support the borrower in ensuring the continuing adequacy of program performance and underlying systems, and will review financial, technical, and progress reports prepared by the borrower and audit reports prepared by external and internal auditors. The Bank will continue sector/program dialogue throughout the program’s implementation. 71. Capacity-Building Support. Bank task teams will also provide support to the borrower in: (a) undertaking the program action plan and institutional capacity-building activities defined during operation preparation; and (b) identifying and resolving program implementation issues that may arise during the implementation of a Program-for-Results operation. Drawing on the list of technical support activities agreed during the preparation of the operation, the task team and the borrower will confer regularly on the specific activities to be carried out and will structure a joint timetable of concrete actions to be undertaken. As needed, the initial list of technical support activities will be modified to accommodate unforeseen priority needs for technical support from the Bank or other partners. Capacity-building support will be focused on improving governance and performance and on institutionalizing those improvements for medium- and longer-term sustainability. For new programs, the development of a practical and functioning M&E system could be an area of focus for support. 72. Achievement of Results. During the implementation of a Program-for-Results operation, the Bank will review progress toward the achievement of DLIs on the basis of agreed reporting arrangements, including the program’s M&E data in the progress reports. When a DLI has been achieved, the borrower will inform the Bank and provide any evidence agreed in the DLI verification protocol, and the Bank will determine that the DLI has been met. If DLIs were not well defined during the preparation of a Program-for-Results operation, or cannot be met because of unexpected events, the definition of these DLIs may be modified during the operation’s implementation, following normal Bank project restructuring processes. 29 73. Monitoring Risks. During implementation, the Bank will continue to use the integrated risk framework in: (a) monitoring the evolution of risks and identifying the emergence of new risks; (b) assessing progress in implementing risk management measures and their impact, working with the authorities and partners to make adjustments as necessary; and (c) determining the level of staff and Management oversight. 74. Restructuring. If, as part of program implementation support, the Bank identifies significant changes in the program circumstances or risks, or changes in the borrower’s priorities and needs, the program financing arrangements may need to be modified to make the original program objectives and the results framework more realistic, or to increase the development impact of the program. If such changes cannot be accommodated within the framework of the original legal agreement(s), the operation may be restructured, in a similar manner to existing restructuring procedures. 75. Additional Financing. During the implementation of the program, the Bank may provide additional financing to the borrower to finance: (a) unanticipated significant changes to expenditures parameters required to achieve the original program results or DLIs; or (b) new or modified results, to be reflected in new or modified DLIs, that aim to scale up the impact or development effectiveness of the original program The Bank’s decision to extend additional financing will be based on the same considerations as its decision regarding the original program financing. In addition, the Bank will need to be satisfied with the implementation and performance of the original program, and with the results of the assessments related to the new or modified DLIs and results. 76. Implementation Completion Report. As with other lending instruments, all individual operations will be evaluated upon completion. This self-evaluation by Bank teams will analyze the results of each operation as well as efforts to achieve institutional capacity building (building on the various assessments and their outcomes). The review will also look at the performance of the Bank and the borrower in carrying out their respective roles. Within one year after the instrument is launched, a joint working group from OPCS and IEG (with membership from other parts of the Bank) will develop and issue detailed guidelines to staff for this assessment, building on existing methodologies and taking into account the special features of the instrument. B. Performance Issues and Remedies 77. Performance Issues. When performance problems are identified, the Bank will promptly bring them to the attention of the borrower and others concerned. The Bank will also follow up on the results of independent audits and on the functioning of grievance processes. The Bank will support the borrower in taking the steps necessary to address performance issues. If the borrower fails to take corrective action promptly, or if performance continues to lag, the Bank may invoke remedies as specified in the program legal documentation. 78. Remedies and Recourse. The Bank’s standard legal remedies will apply to Program-for- Results operations. Remedies (to be distinguished from not disbursing Bank funds when DLIs are not met) may be applied when standard events (set out in the Bank’s General Conditions, which are part of the legal agreements) and operation-specific events (set out in the legal agreements) occur. During preparation of a Program-for-Results operation, the borrower and the 30 Bank will agree on the specifics of the program to be supported and the fiduciary and environmental and social systems requirements to be adhered to by the client (these may include actions set out in the program action plan). These operation-specific undertakings will be reflected in the legal agreements. During implementation, the Bank will monitor the borrower’s compliance with its legal undertakings. If the borrower does not comply with its legal obligations, the Bank will require it to institute timely and appropriate corrective measures. If such measures are not taken, the Bank may exercise remedies—including suspension of disbursements and cancellation of the loan—to address the situation, reflecting the degree of severity of the borrower’s noncompliance.14 The Bank will also have the right to demand a refund if a disbursement is made in connection with a DLI that is subsequently shown not to have been met, or only partially met. C. Role of Corporate Oversight Units 79. Dealing with Fraud and Corruption Cases. The Bank (through the Integrity Vice Presidency (INT)) will have the right to investigate allegations of possible fraud and corruption related to an operation supported by Program-for-Results. The New ACGs (see Annex E) take account of the fact that, in the context of these operations, the Bank will not be funding specific transactions—that is, Bank funds will not be linked to specific expenditures within a program (see Box 3). In addition, consistent with the overall Program-for-Results approach of using and strengthening overall governance and systems, the program will support government efforts to address and manage fraud and corruption risks, including investigations. Box 3. Overall Approach to Fraud and Corruption in Program-for-Results Program-for-Results provides an opportunity for the Bank to expand how it helps borrowers to fight fraud and corruption. Linking Bank financing to verifiable results is itself one of the indicators that funds are used appropriately. In addition, the Bank supports the borrower in managing the risk of fraud and corruption as one element of the borrower’s overall responsibility for program implementation. The focus is on how the borrower manages all program resources, not only those provided by the Bank. The Bank reviews the systems and processes by which program implementing agencies do this. The results of this review inform the Bank’s overall program risk rating and the plan of actions it agrees with the borrower to respond to those risks (among other things). Programs are subject to the ACGs, which will be complemented, when required, by country- or program-specific protocols agreed between the Bank and the borrower. Even as it encourages and helps borrowers to manage fraud and corruption risks, the Bank has the right to carry out investigations that it judges necessary to fulfill its fiduciary duty under the Articles, and to sanction entities that are found to have engaged in fraud or corruption. The Bank’s debarment list will apply to these operations. 80. Under the ACGs, allegations of possible fraud and corruption that are brought to the attention of the government or INT/the Bank will normally be communicated to the other party, leading to consultation on appropriate follow-up action. The findings by the responsible investigative unit and proposed actions would be shared with the Bank, and, as appropriate, the parties will consult on the corrective actions warranted. The ACGs will also provide that the borrower will ensure that entities debarred and suspended by the Bank do not participate in 14 Section 7.02 (a) through (m) of IBRD’s General Conditions and Section 6.02 (a) through (m) of IDA’s General Conditions set out the events that may lead to suspension: payment failure, performance failure, fraud and corruption, cross-suspension, extraordinary situation, event prior to effectiveness, misrepresentation, cofinancing, assignment of obligations, disposition of assets, membership, condition of borrower or program implementing entity, ineligibility, and additional event. 31 contracts awarded during implementation of the operation during their periods of debarment or suspension. The Bank’s standard remedies under the General Conditions (e.g., suspension of disbursements, cancellation of the loan, repayment of monies already provided) provide sufficient scope for the Bank to respond when issues of fraud and corruption are not being satisfactorily addressed. As needed, detailed arrangements by which the ACGs will be implemented in each program will be, when required, detailed in a protocol to be agreed between the Bank and the borrower. In line with its current practices, INT would be unlikely to investigate all the complaints that may arise in the context of a particular operation. However, the Bank will have the right to conduct its own investigations and will exercise this right as the circumstances require (e.g., the nature of the case, lack of action or capacity of the government) when it believes it necessary to do so to fulfill its fiduciary duty. The Bank will be able to impose sanctions on firms and individuals in accordance with its sanctions policies and procedures. 81. Role of Other Corporate Oversight Units. Program-for-Results will be subject to the same oversight functions as other lending instruments:  The Inspection Panel is an independent accountability and recourse mechanism that may investigate IBRD/IDA-financed projects to determine whether the Bank has complied with its operational policies and procedures, and to address related issues of harm. Hence, people affected by activities included in the program supported by Program-for-Results may submit a Request for Inspection to the Inspection Panel, as per the requirements set out in the Inspection Panel Resolution and its subsequent Clarifications.15  The Internal Audit Department provides objective and independent review and assessment of the World Bank Group’s business activities and controls.  The Independent Evaluation Group is responsible for providing an objective assessment of the results of the Bank Group’s work and identifying and disseminating the lessons of experience. VII. PROGRAM-FOR-RESULTS POLICY AND OPERATIONAL FRAMEWORK 82. Conformity with Articles of Agreement. The Bank’s Articles of Agreement provide sufficient legal basis to allow the Board to approve the new instrument. They provide, in part, that Bank loans should be used for productive purposes, for specific projects (except in exceptional circumstances), for intended purposes with due attention to considerations of economy and efficiency, and for expenses as they are actually incurred. Subject to the Board’s approval of the policy recommendations set out in this paper, Program-for-Results financing would comply with the Articles of Agreement. 15 Resolution IBRD 93-10/IDA 93-6 (the Inspection Panel Resolution); Review of the Resolution Establishing the Inspection Panel - 1996 Clarification of Certain Aspects of the Resolution; and 1999 Clarification of the Board's Second Review of the Inspection Panel. 32 83. Dedicated Operational Policy and Bank Procedure Statements. As discussed above, this instrument warrants dedicated Operational Policy (OP) and Bank Procedure (BP) statements.16 Program-for-Results operations will be subject to and comply with other general OPs and BPs, as described in OP 9.00. Subject to Board approval of the recommendations set out in the MOP and this paper, Management will also, as part of implementing and rolling out the new instrument, make the required technical (non-policy) modifications to the other relevant OPs, BPs, and Operational Memoranda. (see Annex H for more details). From time to time, Management will also issue advisory/technical guidance notes to staff (see paragraph 87). 84. Scope of OP and BP. The draft OP sets out the essential mandatory policy elements and other key components of the new instrument. It describes the general goals and parameters of Program-for-Results financing; the types of programs and expenditures that may be eligible for Bank financing; activities that must be excluded; and the overall disbursement policies. The OP also sets out the policy considerations for Bank financing, focusing on, among other things, strategic, economic, technical, results and risks, and governance considerations (including a requirement to adhere to the provisions of the ACGs). The OP sets out the requirements for the technical, fiduciary systems, environmental and social systems, and integrated risk assessments. The OP also discusses the responsibilities of the Bank and borrower, recourse, remedies, sanctions, restructuring, additional financing, and disclosure requirements. The draft BP spells out the related mandatory procedures and rules that will need to be met during program preparation, appraisal, and implementation. 85. General Conditions. The IBRD General Conditions for Loans and the IDA General Conditions for Credits and Grants need to be amended to take into account the features of the new lending instrument.17 Management will adopt similar modifications in the Standard Conditions for grants and loans financed from Bank trust funds. These modifications to the General Conditions and Standard Conditions will be formally reflected in individual legal agreements for each Program-for-Results operation. 86. Anticorruption Guidelines. New ACGs are proposed to help borrower and Bank staff prevent and combat fraud and corruption that may occur in connection with the preparation and implementation of programs supported by Program-for-Results.18 87. Guidance Notes. Guidance notes to staff have been drafted, focusing on such technical matters as: (a) how to assess a government program and related systems in general and in relation to specific areas such as fiduciary and environmental and social effects; (b) how to assess, mitigate, and monitor risks; (c) how to design and measure results and what to do if results targets are not met; and (d) how to follow up if some agreements are not complied with. VIII. WORKING IN PARTNERSHIP 88. Program-for-Results will enhance the Bank’s ability to work in partnership with governments, other development organizations, and stakeholders. Participants in the external 16 The draft OP and BP 9.00 are included as Annexes C and D. 17 The proposed amendments to the General Conditions are included as Annex F. 18 Proposed anticorruption guidelines are included as Annex E. 33 consultations have recognized this aspect but have raised important questions about how this will work in practice. 89. Government Ownership and Leadership. The Bank will partner with governments in assessing programs and the potential for improvements, and in agreeing with government counterparts on specific improvements and the Bank’s role in supporting those improvements. The Bank will help strengthen the design and implementation of programs. The Bank believes that the longer-term effectiveness and sustainability of such programs requires strong government ownership and leadership. 90. Development Partners. Program-for-Results will help the Bank work better with other partners at all phases of the development and implementation of operations—for example, carrying out the technical and systems assessments and drawing on diagnostic work that development partners have carried out; and appraising Program-for-Results operations, including finalizing the scope and financing of the program and the agreed improvement measures. Development partners will often play an important role in the system-strengthening and capacity- building aspects of a Program-for-Results supported program, both because of their expertise and because they often provide technical support on a grant basis. Program-for-Results will increase efficiency by reducing transactions costs for the government and for development partners. 91. Stakeholders. Program-for-Results will offer opportunities for the Bank to partner with other stakeholders—the private sector, civil society, and others—in helping governments improve the design of their programs and implement them more effectively. Stakeholders may be involved in the Bank’s assessment of the government’s program and of its capacity to manage the program’s environmental and social effects, and in the monitoring and evaluation of programs and their results, including the independent verification of program results and DLIs. In addition, Program-for-Results gives the Bank an opportunity to discuss with government counterparts improvements to the overall governance of programs, including strengthening the demand-side aspects of governance. IX. CHALLENGES AND RISKS 92. Management believes that this proposal responds to client demand and will strengthen the Bank’s contribution to results. At the same time, it presents a range of challenges and risks, which this section discusses. 93. Scope of Program and Systems Assessments. There is a risk that the various technical and systems assessments that are integral to the development of Program-for-Results operations may become excessively complex or detailed, time-intensive, costly, and prescriptive. There is also a risk of inconsistencies in approach and application. 94. The Bank will seek the right balance between preserving the flexibility of the instrument and ensuring that the instrument is used only where it can be expected to achieve the desired results. Guidance to staff on the conduct of the assessments will assist in this area, including by recognizing that assessments should draw on other credible analytic work as well as on the experience of the Bank and other development partners in working in particular countries and particular program areas. In addition, a corporate team will be established to provide oversight 34 on all Program-for-Results operations, supporting teams and helping to ensure consistency and high quality. 95. Specification, Verification, and Achievement of Results. Linking disbursements to results presents a number of challenges. First, specifying the outcomes and outputs that the program is intended to achieve, determining the associated performance measures, and, where relevant, mapping intermediate results/indicators to final outcomes is a relatively new approach worldwide, especially in environments with weak institutions. Second, with Bank financing conditional on measured results, both clients and Bank staff may feel pressured to compromise on results measurement. Third, to the extent that expected results are not achieved and disbursements cannot be made, some clients may face financing gaps in important development programs and challenges in filling those gaps. 96. The Bank will develop appropriate quality mechanisms to address these challenges. It will support clients’ capacity to develop sound results frameworks and to monitor and evaluate such frameworks. Program-for-Results operations will seek to establish independent verification and monitoring mechanisms, with transparent, timely, and accurate reporting and clear requirements for follow-up actions if problems are identified. Making DLIs scalable to allow for partial disbursement in the case of partial achievement of results will also help reduce the financing risks for governments. 97. Adequacy of Fiduciary, Environmental and Social Impact, and Risk Management Measures. Program-for-Results operations will provide assurance that Bank financing is used appropriately and that the environmental and social impacts and risks of the program are adequately addressed. This assurance will be heavily influenced by the system assessments and by efforts to improve the systems’ performance and mitigate risks as well as to monitor such performance throughout implementation. The systems assessment and/or review of program expenditures will also assist in identifying excluded high-risk activities (see paragraph 22 (d)). Still, some may perceive the shift away from transactions and very specific procedural aspects as diluting the Bank’s standards. 98. To address this risk, systems assessments will be carried out rigorously, consistently, and transparently, and a summary will be included in the Program-for-Results operation’s documents and made available to the public. In addition, working on the system level can enhance the overall performance of the program and produce higher sustainability and impact on addressing fiduciary, social, and environmental issues. The additional corporate quality assurance will help ensure that such assessments are done well. In addition, the Bank will provide guidance as well as both hands-on and structured training to staff. 99. Effectiveness of System Strengthening and Capacity Building. Institutional strengthening and capacity building to ensure that programs achieve their expected results is a core feature of Program-for-Results, especially in situations of weak capacity. But it is challenging to build institutions and capacities. Some countries may be unwilling to request Bank financing for technical assistance and other measures necessary to help build capacity. 100. To address this challenge, Bank implementation support activities will focus on capacity building in a broad range of areas, including the program’s results frameworks and its fiduciary 35 and environmental and social systems, processes, and procedures. Under each operation, the Bank will support institutional strengthening and capacity building through an action plan that is agreed during preparation and monitored during implementation, as well as through complementary analytic work and technical assistance. It will develop partnership arrangements with other development partners, which may be in a better position to support the necessary capacity building, including through grant financing. Technical assistance can be provided through a number of modalities (i.e., as investment lending components, hybrids, or part of the Program-for-Results). Management is also seeking to establish a trust fund to support capacity building efforts. 101. Risk of Results or Other Actions Being Imposed on Countries. Program-for-Results operations provide an opportunity to improve development partners’ coordination around government-owned programs, thereby contributing to significant improvements in both the efficiency and effectiveness of development assistance. But in countries where there are many development partners and development assistance accounts for a large proportion of the financing of a government program, there is also a risk that development partners may (perhaps inadvertently) impose results on countries and programs or insist on the use of external review mechanisms of various kinds, thereby creating new forms of conditionality. 102. Governments and development partners all need to be aware of and guard against such risks. Since Program-for-Results has at its center country objectives, programs including their results, and institutions, any imposition of results on countries would be counter to the spirit of the instrument. Similarly, Program-for-Results provides a platform for harmonizing and aligning development assistance around the government’s program and systems, and any deviation from that would imply that another funding modality should be used. 103. Expansion of Bank Corporate Oversight. Although Program-for-Results focuses on systems rather than transactions, and the Bank itself may be financing a small part of a larger program, Program-for-Results will be subject to the same oversight functions as other lending instruments. If such oversight is not carefully interpreted and managed, this could result in expanded Bank oversight of government-owned programs that could undermine the ownership of such programs and discourage governments from making use of the instrument. 104. The clients that request a Program-for-Results operation will likely be those that want to enhance the performance of their overall programs and systems to achieve results that are sustainable and effective. In such situations, the intent of the instrument would be aligned with the government objective. Every effort has been made to get a good balance between ensuring the role of corporate oversight, maintaining strong ownership, and relying on government systems. 105. Readiness for Introduction of the Instrument. The introduction of a new instrument will require that all necessary elements of support are in place. In view of the demand for the new instrument, there is some urgency to proceed with its further development and introduction. 106. While Program-for-Results will be a new instrument, there is broad experience with using Program-for-Results-type features in IL operations in recent years. In many ways, the introduction of Program-for-Results codifies the lessons learned from that experience. Reflecting the results of the consultation process, detailed guidelines to staff have been prepared 36 to accompany the new OP/BP. A structured training program will be in place, and guidance notes, best practices, and tools will also be made available to staff. Moreover, during the initial phase of the rollout, Management will put in place enhanced oversight to ensure effective use of the new instrument—a corporate team (including fiduciary, social, environmental, and technical staff) that will provide hands-on support to all task teams and review every Program-for-Results operation. 107. Risks of “Muddling Along.� There are important challenges and risks that need to be addressed in implementing the Program-for-Results instrument. But there are also major risks in not doing so. As this paper has described, the current “muddle-through� approach is resulting in inconsistent and selective application of programmatic approaches across the institution, with missed development opportunities and significant frustrations to borrowers and Bank staff. It also seems clear that if Program-for-Results were included as one of the IL options,19 it could get caught between the transactions orientation and the policy focus, making it of limited value to many client countries. X. NEXT STEPS AND REQUEST FOR APPROVAL 108. This paper describes the new instrument and how it will work. The final design of the Program-for-Results instrument benefitted from the views of clients and other stakeholders, experiences from inside and outside the Bank in the design and implementation of program- based approaches, and extensive consultations with shareholders and stakeholders. 109. Next steps. If Executive Directors approve Management’s proposal, Management will undertake the following steps to ensure smooth implementation of the new instrument. 110. Enhanced oversight. Drawing on experience with the rollout of other new initiatives, Management will proceed cautiously, putting in place enhanced corporate oversight arrangements during the initial phase. This includes:  All proposed Program-for-Results operations will be reviewed by a dedicated corporate-level team that will include technical, procurement, financial management, environmental, and social specialists.  The first two operations in each Region will have a corporate review meeting (ROC or OC), followed by full Board discussion. 111. Commitment limit. Management proposes to limit IBRD/IDA commitments to 5 percent of total IBRD/IDA commitments for the first two years from Board approval. Management would propose to the Board to lift this cap if justified by a review of implementation experience. 112. Staff guidance and training. Staff will be provided with guidance and support in implementation. Communications and training and orientation will be carried out in close collaboration with the Regions. Training will include formal and informal modalities, e-learning, 19 The current menu of seven IL options includes adaptable program loan (APL), emergency recovery loan (ERL), financial intermediary loan (FIL), learning and innovation loan (LIL), specific investment loan (SIL), sector investment and maintenance loan (SIM), and technical assistance loan (TAL). 37 and face-to-face efforts, and will be targeted to both country-based and Washington-based staff. Targeted support will also be provided to teams working on the early operations. A help desk will be set up to answer questions about Program-for-Results. 113. Communications and outreach. In addition to staff guidance and training, comprehensive internal and external communications and outreach efforts will accompany the introduction of the instrument. These include a range of face-face communications, web-based outreach, and the use of social media to reach a broader global audience. Communication materials for clients and stakeholders will be produced and made available through country offices and the Bank’s external website. Channels to provide feedback (e.g., a link on the Program-for-Results website, e-mail, etc.) will allow interested stakeholders to share implementation experience. The communication strategy will be closely monitored and adjusted as implementation proceeds. Management will consider various options to encourage cross- Regional learning and sharing within the Bank and with other partners—for example, an external seminar series on key aspects of the instrument, special events during the Spring and Annual Meetings, or possible joint events with other development partners. Internal learning opportunities (e.g., clinics and seminars) are also planned. 114. Tools and templates. Management will also provide all the tools—system support and templates—that teams need to carry out their tasks and that Management needs to carry out its quality assurance and monitoring functions. This will build on recent work done as part of IL reform and the Operations and Knowledge Systems Program, which has revamped the IT system and Operations Portal and can be easily adapted for Program-for-Results. 115. Trust fund to support capacity building for Program-for-Results. As CODE suggested, Management will seek to establish a trust fund to support capacity-building efforts for Program- for-Results. 116. Review of implementation experience. After about two years of implementation, Management will carry out an initial review of experience with the new instrument. Using basic descriptive statistics, supplemented by qualitative evaluation techniques, the review will address questions like the following: Is implementation of the instrument going well? What programs are we supporting and in which countries? What types of results are covered in these operations? How are capacity-building efforts addressed? What is the initial reception from the clients where the instrument has been tried? In addition, the review will look into the instrument’s overall management and administrative requirements. Finally, it will pay special attention to fiduciary and social and environmental issues. In preparing the review, Management will solicit feedback from countries, development partners, other stakeholders, and staff (using both surveys and in- depth interviews) (see Annex I for more details). Management welcomes an early evaluation of implementation by the Bank's Independent Evaluation Group which would provide a useful independent input for ongoing improvements to the instrument. 117. Request for Approval/Endorsement. Management requests that the Executive Directors approve: (a) the policy content of the draft Operational Policy (OP) 9.00, Program-for-Results Financing (Annex C of the paper), and of the draft Guidelines on Preventing and 38 Combating Fraud and Corruption in Program-for-Results Financing (Annex E of the paper); and (b) the modifications to the General Conditions for Loans (July 31, 2010) and the General Conditions for Credits and Grants (July 31, 2010), substantially in the form set out in Annex F of the paper. 118. In addition, Management requests that the Executive Directors endorse the proposal to limit Program-for-Results commitments, set out in paragraph 111. ANNEX A. SUMMARY OF CONSULTATIONS 1. The World Bank undertook an extensive global consultation process from March to September 2011 to inform the development of its proposed new instrument—Program-for- Results—seeking views and inputs through internal and external consultations. Overall, feedback from the consultations highlighted broad support for the Program-for-Results financing instrument, with countries and citizens welcoming the approach and the instrument’s potential to enhance development effectiveness. External and internal audiences shared some similar questions—the rationale for the instrument, its relationship with the Bank’s other financing instruments, definition of results and disbursement-linked indicators (DLIs), accountability, monitoring and evaluation, environmental and social issues, fraud and corruption, partnerships, procurement, disbursements, and implementation support. The comments and suggestions from the consultations were taken into account while drafting the policy and guidance notes. 2. Whom did we consult? Wide ranging internal and external consultations have marked the development of the Program-for-Results instrument. Consultations were held in 34 client countries, 7 donor countries, with representatives from 9 shareholder countries, and a large group of development partners. Following the discussion of the Program-for-Results Concept Note by the Bank’s Executive Directors on February 22, 2011, the Bank consulted with representatives of governments, development partners, international organizations, multilateral development organizations, United Nations agencies, parliamentarians, the private sector, civil society, foundations, academia, think tanks, and practitioners. Internal consultations were held in parallel through over 50 specific events and numerous informal meetings. Internal consultations were also often included in country visits, in addition to being held via videoconferencing. Staff also provided feedback via email. 3. Resources and Translations. In addition to the Program-for-Results webpage in English, the Concept Note and the executive summary of the draft policy paper were made publicly available in Arabic, Chinese, French, Russian, and Spanish. A range of materials, including the consultation plan, a fact sheet, frequently asked questions, and presentations were provided. An electronic feedback form and email address provided additional feedback channels. 4. Innovative Outreach Channels. Outreach through Facebook and Twitter opened electronic channels to help guide the public to the Program-for-Results webpage. Statistics show that this approach was fairly successful, with Facebook being the second largest referral to the Concept Note on the Program-for-Results page, preceded only by Google (including all its country domains). Feedback Highlights 5. Broad, Global Support. There was broad support for the proposed instrument, the focus on results, and the World Bank’s responsiveness to demand from borrowers and donors to develop the instrument. Many noted that Program-for-Results would help bring about systemic change in countries and address concerns about capacity development. “This is exactly the kind of flexible instrument that we require from the World Bank that allows us to achieve our development goals,� exemplifies the nature of positive feedback. Most lauded the instrument and felt it was innovative and will lead to better development impact; and others welcomed “the 40 proposed cautious approach.� They welcomed the flexibility and opportunity to use the new instrument in areas beyond the education and health sectors. They clearly articulated a focus on ownership (i.e., that the programs to be supported remain government-owned programs that use government systems) and stressed that the program’s results would need to be clear, feasible, and verifiable. Other comments suggested that the Bank should move faster and without the restrictions on the use of the instrument that were included in the policy document. Some noted that the instrument is long overdue. 6. Concerns. Some representatives of civil society organizations and business groups argued that the Bank is lowering its fiduciary, environmental, and social standards, and that Program-for-Results will not allow the Bank to effectively deal with fraud and corruption. Comments from civil society organizations illustrated their concern: “There is a lack of clarity regarding environmental and social standards that will apply to activities funded by Program-for- Results,� and “Category A activities should be excluded.� They consider the shift to a program- level approach to addressing environmental and social effects as a challenge to accountability. Moreover, many civil society organizations requested clarifications about the role of the Inspection Panel. Business groups were primarily concerned with maintaining opportunities for firms to participate in contracts financed by programs supported with the new instrument and “ensuring that the duty to investigate corruption and fraud allegations is independently implemented by the World Bank.� Some asked the Bank to pilot the instrument before its introduction. 7. Results. One of the most interesting aspects of the consultations was how the discussion on “results� became the focus of the meetings in most countries. Questions were primarily related to the definition of results, how results would be measured and validated, what steps the Bank would take if there were no results, how disbursements would occur, and whether advance financing would be an option for resource-scarce countries. 8. Assessments. Many participants expressed concerns that the fiduciary, environmental and social, and technical assessments could become so demanding that they would discourage the use of the instrument; or that lack of capacity might not allow countries to use this instrument. Participants welcomed the focus on partnerships and capacity building, but sought more details on how this would be implemented in countries. 9. Procurement and Environmental and Social Issues. Procurement and environmental and social issues were of interest and concern to a number of participants. While there was recognition of the need to move from transactions to systems if the Bank was to support borrowers’ programs, there was also concern about the implications for the Bank’s position on these issues. The views were varied and diverse. Some representatives of the private sector felt that the introduction of Program-for-Results could restrict opportunities for business with the Bank. Several civil society organizations felt the Bank would dilute its commitments to environmental and social safeguards, and is trying to sidestep its commitment to protecting people and the environment by what it perceived as the Bank “promoting� the use of the Program-for-Results at the expense of its other instruments. Both groups felt that the Bank should exclude high-risk activities. 41 10. Exclusions. Some countries felt that the exclusion of certain high-risk activities, as articulated in the Program-for-Results policy paper, was unwarranted as it did not take into account their capacity and ability to manage those risks. They felt it might limit the applicability of the instrument and its potential impact. On the other hand, as noted above, others welcomed this and asked that the language regarding the exclusions be clearer in the final policy document. 11. Capacity Building. Capacity building is a core objective of the Program-for-Results instrument, and there were several questions relating to the specific modalities for achieving this objective. How will capacity building be delivered and financed? Will trust funds be available for providing such support? Others noted that improvements in capacity building are not easy to measure and may present some challenges for Program-for-Results. 12. Corporate Oversight. Most participants welcomed the fact that all Program-for-Results programs would be subject to the Bank’s independent oversight agencies, including the Institutional Integrity group (INT), the Independent Evaluation Group, and the Inspection Panel, and would be independently audited. However, some raised a concern that some of these rights will make the instrument too intrusive for the countries. 13. The attached matrix (Table A1), summarizing the main issues raised and the Bank response to those issues, illustrates the range of queries and comments. Conclusion  14. By engaging a wide range of stakeholders in both phases, the consultation process elicited client and stakeholder views to help define the instrument and address concerns about the Program-for-Results instrument. It also helped raise awareness about the rationale for the instrument, and provided information and guidance on the potential benefits it could bring to clients. Most constituencies provided feedback and were supportive of the goals of the instrument as is evident from recent reference to Program-for-Results in the Development Committee Statements (Box A1). 42 Box A1: Summary of Development Committee Statements with Reference to Program-for-Results World Bank Annual Meetings, September 2011  United Kingdom: “The UK looks forward to the introduction of the WBG’s new lending instrument… as a means for the WBG to respond more effectively to changing development needs and demands from client countries.�  Sweden on behalf of Denmark, Estonia, Finland, Iceland, Latvia, Lithuania, Norway and Sweden: “We further strongly welcome the development of the new lending instrument Program-for-Results. We would like to see the instrument rolled out and tested without undue delay. [Program-for-Results] will be a useful instrument for strengthening country systems. It is particularly important to see the combination of results-oriented disbursement with institutional development and capacity-building. This should, in the long run, also strengthen client countries’ work related to governance and anti-corruption.�  New Zealand on behalf of Australia, Cambodia, Kiribati, Korea (Republic of), Marshall Islands (Republic of the), Micronesia (Federated States of), Mongolia, New Zealand, Palau (Republic of), Papua New Guinea, Samoa, Solomon Islands, Tuvalu and Vanuatu: “We also welcome the Bank’s innovation in the development of its new results-based lending instrument. This instrument has great potential for our constituency and we urge the Bank to maintain momentum by absorbing lessons promptly as it is rolled out on the ground.�  Germany: “With a view to enhancing the focus on development results, we welcome the envisaged introduction of the new Program-for-Results financing instrument. We can use it to create a direct link between the disbursement of financial contributions and the achievement of specific results and impacts. The instrument also focuses our attention on institution-building. This is a promising approach which enables us to strengthen partner ownership and take targeted action to help build their capacity and systems and support their reform programs.�  Netherlands also representing Armenia, Bosnia and Herzegovina, Bulgaria, Croatia, Cyprus, Georgia, Israel, Former Yugoslav Republic of Macedonia, Moldova, Montenegro, the Netherlands, Romania, and Ukraine: “We support the World Bank Management’s proposal for a new lending instrument to respond to changing development needs and demand from borrowing countries. We want the instrument to focus directly on improvements in sectorial or other development programmes and place direct emphasis on results by making them the basis for disbursement…Our constituency cautions that any procurement resulting from the use of the new lending instrument should be based on solid systems, with a level playing field for the international business community.�  India: “We are particularly keen to see the introduction and extended use of the new results based lending instrument and greater use of country systems.�  43 Table A1: Summary of Feedback from Public Consultations Feedback Bank Response Rationale for Program-for-Results What is the rationale for The development landscape has changed. Bank clients are increasingly introducing Program-for- implementing their own programs for development and poverty reduction and Results? are asking development partners for finance and expertise to improve such programs’ effectiveness and efficiency in achieving results. Bank Management believes that the Bank should provide lending instruments that respond to three broad demands: project support; policy support; and program support. The first two are addressed by Investment Lending (IL) and Development Policy Lending (DPL) respectively. By supporting a government program, Program-for-Results would fill the existing gap and place more direct emphasis on results by making them the basis for disbursement. It would focus the Bank’s technical and financial support more strongly on institutional development, particularly capacity to monitor results and strengthen the public expenditure systems of a country. It would also enable the Bank to leverage its own financing and collaborate with other development partners through pooling resources and focusing efforts on supporting government programs. How is Program-for-Results With its program-level orientation, Program-for-Results would be different from IL and DPL? complementary to but different than Investment Lending, which supports specific projects and disburses against specific expenditures and transactions, and Development Policy Lending, which supports policy and institutional reforms and provides general budget support. As with the DPL and IL, the decision on the choice of instrument would depend on the client, the development objectives, and nature of development constraints. How does Program-for-Results IL options, including Sector-wide Approaches (SWAps), Adaptable Program relate to current IL options Loans (APLs), and Output Based Aid (OBA), are governed by the IL policy such as SWAp, APL, and framework (i.e. are transaction based and disburse against expenditures OBA? incurred). Some of them have program support features. For instance, SWAps allow for pooled funding, OBA links disbursements to the achievement of specific outputs, and APLs allow for a longer term engagement with a government program. However, these options fall short of achieving what Program-for-Results is designed to do—i.e. support government programs using the program’s institutions and systems, thereby building their capacity, and linking disbursements to achievement of results. The combination of the two features will also help enhance the effectiveness of the total public spending for such programs and strengthen their results orientation. While the design of Program-for-Results has benefitted from the Bank’s experience with these IL options, after careful review of the lessons learned, it was clear that the IL policy framework would not enable an adequate response to client demands for support to their programs. How will Program-for-Results Access to Program-for-Results by any country will depend on a careful work in different country assessment of program systems in terms of performance, capacity and risks, contexts – middle income, low and the potential for improvements. In contexts where the existing systems income, fragile states/conflict- are strong, the Bank may support wide-ranging and ambitious programs. In affected situations? countries with weaker systems and capacity, it is likely that the scope of a Program-for-Results operation may be adjusted to strike the right balance between benefits and risks. Accordingly, how Program-for-Results is used will vary from country to country as well as from sector to sector; but the flexibility of the instrument should enable it to be used in a wide range of country and sectoral situations. 44 Feedback Bank Response Is it possible for countries to If there is a need for additional technical assistance (TA), such support can opt for a hybrid, e.g. Program- be provided through several ways: for-Results with an IL  a separate IL operation that can be processed in parallel; technical assistance  through a TA component in the same operation that would follow IL component? policies; or  through a TA component embedded in the Program-for-Results operation. The decision on the appropriate option would be made jointly by the client and the Bank depending on what works best for the program. Has the Bank taken relevant Yes, the Bank has taken into account the experiences of the European Union; experience from within and Norway; DFID; the IDB; and organizations such as the Center for Global outside the institution into Development; Millennium Challenge Corporation; and others. The Bank has account while designing the also learned from its own work, from the program-based lending it has done Program-for-Results in the past. These experiences—including sector-wide approaches (SWAps), instrument? conditional cash transfers, the Global Program on Output-based Aid, the Results-based Financing in Health Initiative, and the Education for All Fast Track Initiative—have highlighted the limitations of trying to achieve a Program-for-Results type of operation through the Bank’s existing lending instruments. Design of the Instrument What types of The Program-for-Results instrument can be used to support all sectors, new sectors/programs can and existing programs, as well as for a specific geographic location, including Program-for-Results support? subnational, depending on how the program is designed and defined. Support Can they include private to the private sector may be included in a program in a manner similar to sector development programs existing practice in IL. Two types of activities are excluded from Program- and subnational programs? for-Results— activities with potentially significant, irreversible adverse impact on the environment and affected people (activities classified as Category A under IL policy), and activities that involve procurement of goods, works and consultant services under a contract whose estimated value exceeds specified monetary amounts. How will the Bank determine Programs to be considered for Program-for-Results will be typically the suitability of a program for identified through engagement with client countries as part of the country Program-for-Results support? assistance strategy process and policy dialogue. However, if a country requests support using Program-for-Results the Bank will appraise several aspects of the program including: (a) a technical assessment that focuses on the technical soundness of the program, the expenditure framework, and the results framework; (b) a systems assessment that includes fiduciary and environmental and social impacts and risks of the program; and (c) an overall risk assessment. The Bank will also assess capacity to implement and performance of the accompanying systems and discuss with government counterparts how the program could be strengthened. Following these assessments, a program may or may not be considered suitable for Program- for-Results, or actions may be agreed upon to enhance the program’s system and performance, as needed, and identified and included as part of implementation How will Program-for-Results Program-for-Results will finance government programs following the rules ensure that the programs and procedures of the program, not of the World Bank. The results to be remain government owned and achieved would be those of the program, and any institutional strengthening that the Bank is not dictating and capacity building will focus on program institutions and systems. to governments? Moreover, the results framework and choice of indicators will also be based on the program’s results. 45 Feedback Bank Response Can the Bank influence the By working at the program and systems level, Program-for-Results increases design of programs if it is only the potential reach and impact beyond the actual amount of financing providing a small amount of provided by the Bank. For instance, the measures identified in the action plan the financing? of the operation apply to the entire program to be supported regardless of the amount of funds the Bank provides. How will Program-for-Results The focus on strengthening the borrower’s program and capacity increases address the issue of long term the likelihood of the program continuing beyond the finite period of the sustainability of programs? Bank’s financial support. The overall focus on systems and their enhancement will increase the chances of the program achieving results and sustainability over time. The proposed program and The assessments of the technical soundness of the program, the capacity and systems assessments for performance of the program’s systems, and the key risk of the program not Program-for-Results are achieving its development objectives are fundamental to understanding the overly complex. quality of the program and its potential for improvement. They provide assurance that the program has the necessary underpinnings in place for an effective Program-for-Results operation. The complexity of the assessments will be determined by the complexity of the programs being considered for Program-for-Results support and will be made easier and less complex if (as will be the case for most programs) there is other analytic work to draw on, whether from the government, other development partners, or the Bank itself. Results Who will be responsible for The results to be achieved under a Program-for-Results operation are the determining the results results of the government program. The government will therefore be framework for a program? responsible for defining the results. The Bank will review the results from the perspective of relevance, achievability, and monitorability and will discuss the potential for and desirability of any changes with government counterparts. How will the results be As noted above, the results are those of the program supported by Program- defined, measured and for-Results. A specific set of results will be selected for the program, defined verified? in relation to the specific objectives of the program and the expected timelines for achieving those objectives. Indicators will be established for measuring those results. The necessary data collection systems for those indicators will be strengthened or put in place to facilitate timely and accurate reporting. Results monitoring is the responsibility of the program. Disbursement-linked indicators (DLIs) will be linked to key program results. The Bank requires that each DLI has a credible verification protocol that is acceptable to the Bank. Verification mechanisms depend on the nature of the indicator at hand and can include program data if that is deemed acceptable, data provided through audits or by other parts of the government (e.g., Central Bureau of Statistics), or by other bodies independent of the government. The DLIs and their verification, as well as updates on their progress, will be made available in the public domain. How will program monitoring The M&E system (including what will be done and by whom) will be and evaluation (M&E) be typically that of the government and will be carried out by the institution(s) carried out, and by whom? implementing the program. In some cases, it will be possible to rely on an existing system, while in others it will be necessary to enhance the system (including capacity building, or adding new options like using third party verification, etc.) or establish entirely new systems. The Bank will agree with the government on the specifics of the M&E system in the context of finalizing the agreement on a Program-for-Results operation. 46 Feedback Bank Response What is the requirement in M&E is an important component of a Program-for-Results operation. During terms of the M&E system? preparation, the teams will assess the capacity of the M&E system and identify gaps and actions, where needed, to help enhance that capacity. Moreover, the strong focus on results and their use in decision making is likely to strengthen the M&E system. Is it possible to have results Yes, both qualitative and quantitative results are possible. Some DLIs may that are both quantitative and be harder to measure than others but this will be assessed on a case by case qualitative, the latter being basis as there is no “one-size-fits-all� solution for this. In each Program-for- more difficult to measure? Results document, the DLIs will be clearly defined, as well as how they will be measured and verified. That information will also be in the public domain. If there are changes during The policy recognizes that there are many situations where changes to implementation or there are original design may be needed. The restructuring aspects of the policy can be exogenous factors affecting utilized in those situations. The option to restructure is also available for our DLIs, can these be modified? Investment Lending Projects as well. If so, how? Should there be a requirement Program-for-Results is designed to support government programs and their to have one pre-determined own results framework. Ownership of the results is very important for the global results framework for a success of the operation. The M&E for the program will also be assessed and Program-for-Results data requirements identified and, where needed, capacity building efforts of operation? the M&E system supported as well. How can the focus on results There are a number of ways in which the focus on results can help enhance be used to help improve the focus on governance. Linking Bank financing to verifiable results is governance? itself an indicator that funds are used appropriately. In addition, the Bank supports the borrower in managing the risk of fraud and corruption as one element of the borrower’s overall responsibility for program implementation. The focus is on how the borrower manages all program resources, not only those provided by the Bank. In addition the verification protocols for DLIs will assign clear accountability, the disclosure of program results will increase transparency, and the strengthening of institutions and capacity are likely to improve overall governance of the program well beyond the involvement of the Bank. Lastly, Program-for-Results will improve public management and performance and hence increase efficiency and effectiveness of the program being supported. Is there a risk that Program- The challenges in results measurement are several – nature of the sector, stage for Results financing will focus of development of the program and the related nature of the results that are on sectors where achieving desired for the program. For instance, capacity building efforts are generally results may be considered harder to measure than some outputs related to service delivery coverage. easier than others? While this may be a risk, decisions on Program-for-Results depend on the demands and needs of the client, the desire to enhance the performance of the overall program of the government, and the outcome of the appraisal process. The sectors to be supported by Program-for-Results will be determined in the context of the Bank’s CAS/CPS for that country, taking into account government priorities as well as the potential value added of the Bank. The results to be supported will be the program’s results – expected to be challenging but realistic – and will be reviewed by the Bank in the context of its technical assessment of the program. To whom will the results be The results will be the country’s and program’s results, defined by the attributed? government in consultation with the Bank. The results will be attributed to the agencies responsible for implementing the program. 47 Feedback Bank Response How will the Bank evaluate As with the Bank’s other lending instruments, the Bank will assess progress progress? of implementation throughout the operations as part of implementation support. In addition, the Bank will prepare a completion report on the extent to which the operation has achieved its overall objectives and provide reasons for success or lack thereof. These completion reports will draw on the government’s own evaluation of progress as well as on any evaluations conducted by others—evaluations that are expected to be more rigorous because of the stronger focus on results and on monitoring and evaluation. The Bank’s Independent Evaluation Group will also review the performance of Program-for-Results operations as it does for the Bank’s other lending operations. Will there be impact The evaluations of the program will be carried out by the implementing evaluation to enhance learning agency, the Bank, and other partners and strongly centered on impact because in the process? they will focus on the overall development results to be expected from the program. Additionally, in-depth impact evaluations will help further enhance cross-country global learning and the understanding of the performance of the Program-for-Results instrument. The extent and frequency of impact evaluations will depend on the individual programs. Fiduciary and Fraud and Corruption How will the Bank ensure that The Bank is committed to addressing fiduciary as well as environmental and money, people, and the social concerns and will undertake upfront assessments of these issues in environment are protected? determining whether to proceed with a Program-for-Results operation. As necessary, the Bank will also agree with the government on measures to strengthen fiduciary arrangements and/or the arrangements for managing the potential environmental and social impacts of particular programs—measures that may be introduced prior to or as part of the implementation of a Program- for-Results operation. The Bank will make a judgment about the suitability of a Program-for-Results operation taking into account the risks and capacity of programs including fiduciary, environmental, and social aspects. Will there be a competitive Under a Program-for-Results operation, the Bank will assess the program’s bidding process for contracts procurement system. In particular, the system will be assessed against criteria in the program? including: adoption of public advertisement of all bids; no restrictions on competition; fair and transparently applied bid evaluation criteria; acceptable conditions of contract; effective complaint mechanisms; and strong anti- corruption measures. Will bidders have the right to Bidders will have access to complaints mechanism and can complain directly complain under Program-for- to the Bank if they believe the Borrower is unresponsive to their complaints Results? or treated them unfairly under applicable rules and processes. How will Program-for-Results The Bank will agree with the government on measures to address weaknesses address weaknesses in and gaps, and Bank staff will monitor the implementation of those measures program fiduciary systems? as well as the overall performance of the systems assessed during program implementation. If performance is lagging, the Bank will ask the client to take timely and appropriate action; and if performance continues lagging, the Bank will have the right to apply the relevant remedies. What fiduciary principles Program-for-Results will have its own operational policy that will cover inter related to Investment Lending alia fiduciary management. The policy will draw on similar fiduciary will apply to Program-for- principles as those for Investment Lending operations and focus on assessing Results? the quality of government systems and how they can be improved. Hence, the main difference is in terms of “how� such policies are applied at the system level instead of at the level of specific transactions 48 Feedback Bank Response With the shift to systems, will The same legal remedies that are available to IL will also be available to the Bank be able to take action Program-for-Results. If there is non-compliance with contractual if something goes wrong? With commitments between the Bank and the borrower, these remedies can be regard to system performance utilized. Moreover, the Bank will be monitoring system performance beyond preparation, how will (fiduciary, M&E, environment and social) throughout program that be monitored? implementation and will support the borrower in addressing any weakness or new issues that may emerge as needed. What is the difference between Program-for-Results calls for the use of the program’s institutions, assessing the approach to be followed by those against a set of principles and, as needed, agreeing on enhancements Program-for-Results and the and improvements in performance. The country systems pilot for Bank’s country systems procurement and safeguards adopted a different approach – one that looked at approach/pilots? the whole country and that sought policy and procedural equivalence to the Bank’s IL policies. The Program-for-Results approach differs in many dimensions - from the scope of the assessment to the methodology. What is the link between Program-for-Results has important governance dimensions. It looks at the Program-for-Results and the overall performance of the program’s institutions and helps link those more governance and anti- directly to results. It also asks for clear results and for a credible verification corruption (GAC) agenda? process before disbursements are processed. These are at the heart of the GAC agenda. How will fraud and corruption The Bank is committed to the highest standards of integrity and transparency be addressed in Program-for- and to addressing fraud and corruption. The operational policy for Program- Results supported operations? for-Results sets out how issues of fraud and corruption will be addressed. New anti-corruption guidelines for Program-for-Results have been developed. There will be a strong focus in Program-for-Results operations on good governance including provisions for how to handle cases of fraud and corruption. In some Program-for-Results operations, it is expected that progress in fighting corruption and improving procurement will be among the results needed to release Bank disbursements (i.e., they would be part of the Program-for-Results disbursement-linked indicators). New Anti-Corruption Guidelines will be mandatory for all borrowers and will apply to the entire program supported. The Bank, through its independent unit set up for this purpose (INT), will have the right to investigate all allegations of fraud and corruption in the entire program supported, not only those related to Bank financing. The Bank’s debarment list will apply to the entire program. The up-front fiduciary assessment includes the risk of fraud and corruption. The results of the assessment will be included in the program document that is publicly disclosed and will inform mandatory remedial actions, as appropriate. Will INT’s right to investigate The Bank (INT) will reserve the right to conduct its own investigations and be preserved? will exercise this right as the circumstances require (e.g., the nature of the case, lack of action, or the capacity of the government). The Bank will also be able to impose sanctions against firms and individuals in accordance with its sanctions policies and procedure. Why doesn’t the Bank reduce While there are clear exclusion criteria for contracts above specified monetary the thresholds for contracts value, this does not mean that all contracts below that will be financed excluded from Program-for- through Program-for-Results. It is only one of several options the Bank has Results? to finance such contracts. The decision will depend on the nature of the development challenge, and the capacity, performance, and risk of the specific program. 49 Feedback Bank Response Environmental and Social Dimensions Is the Bank abandoning The World Bank is not abandoning safeguards. The Bank’s principles related safeguards? to environmental and social issues will be maintained but realized in a manner appropriate for program-level support. The requirements for the assessment of environmental and social systems are guided in purpose and goals by the environmental and social safeguard policies of Investment Lending projects, adapted to the needs of the Program-for-Results instrument. How will Program-for-Results The Bank is committed to addressing environmental and social issues and ensure that the potential will make an upfront assessment of these issues in determining whether to environmental and social proceed with a Program-for-Results operation to adapt the scope of the impacts are managed program to be supported. Category A activities will be excluded from adequately? Program-for-Results financing. For activities that are not in Category A, the Bank expects that the case by case risk assessment process mandated under Program-for-Results will, in certain circumstances, lead to the determination that the instrument is not appropriate for some types of programs. As necessary, the Bank will also agree with the government on measures to strengthen the arrangements for managing the environmental and social effects of particular programs – measures that may be introduced prior to or as part of the implementation of a Program-for-Results operation. Category A activities are Category A activities are excluded from Program-for-Results. For all other excluded. Why doesn’t the activities, a risk based approach will be adopted (i.e., the system will be Bank also exclude "high Bs" assessed based on its capacity, performance, and risks), and a judgment will from Program-for-Results? be made. There is no classification of “high Bs� in the Bank, only A, B and C. How does the Program-for- The principles incorporated in the Program-for-Results policy are similar in Results policy on purpose and goals to Investment Lending safeguards policies. While their environmental and social application differs from those in Investment Lending, they are intended to issues relate to the existing achieve similar outcomes. Program-for-Results emphasizes upfront safeguards policies for assessments and strengthening of the program system to promote acceptable Investment Lending? Is the and sustainable environmental and social management. In addition, Program- new policy consistent with the for-Results excludes activities with significant and irreversible impacts. existing Investment Lending safeguards policies? How does Program-for-Results The ongoing process for the updating and consolidation of the environmental relate to the process for and social safeguard policies that are used for Investment Lending is updating and consolidation of anticipated to be completed by the end of calendar year 2013. Program-for- the environmental and social Results will have its own policy framework that will include provisions for safeguards policies that is addressing environmental and social issues. The development of the policy currently underway? for the proposed Program-for-Results instrument is separate; however, they have the same objective of integrating environmental and social issues into the preparation and implementation of Bank supported operations. Will the Inspection Panel’s The Inspection Panel will have an oversight role over Program-for-Results. right to investigate be Hence, people affected by activities included in the Program supported by the preserved? Bank under this Policy may submit a Request for Inspection to the Inspection Panel, as per the requirements set out in the Inspection Panel Resolution and its subsequent Clarifications. 50 Feedback Bank Response Will there be any consultations Relevant stakeholders will be consulted regarding the findings of the on the environmental and environmental and social systems assessment of the program proposed to be social aspects in a Program- supported by Program-for-Results financing. When the draft system for-Results operation? assessment is prepared, the Bank will solicit views from stakeholders involved in the government program. The Bank will make draft and final versions of the system assessment available to the public. The systems assessment process itself includes a review of existing borrower consultation arrangements associated with the program (or proposed for a new program) and, if relevant, suggests measures to improve effectiveness. What are the disclosure and The Bank will consult on and disclose its environmental and social systems consultation mechanisms? Will assessment during preparation, and information on performance during local communities and the implementation will be in the Implementation Status and Results (ISR) public have timely access to Reports. The government’s program’s own consultation and disclosure information about programs practices are reviewed in the systems assessment process. Key considerations that may affect them? during the review are whether stakeholders’ views and concerns are solicited in an open and effective manner, and whether these views and concerns are considered in program design and implementation. Depending on the program, the key stakeholders may be local communities. If warranted, the systems assessment may make actions to strengthen consultation practices, and the adoption of these actions will be monitored during implementation. How will Program-for-Results The Bank’s draft and final environmental and social systems assessment will provide information to the be made available to the public. In addition, a summary of the assessment public about environmental will be disclosed in the Program Appraisal Document (PAD). Furthermore and social issues? the Implementation Status and Results Reports (ISRs), which are available to the public, will provide an overview of progress in the implementation of the operation, including agreed actions to improve environmental and social systems performance. Why is OP4.00 Use of Country While Program�for�Results will contribute to the broad agenda of country Systems not used in Program- systems, it will be different from the country systems pilots for safeguard for-Results? polices in several ways. First, the focus is on the program’s systems, not the country’s systems. Second, Program�for�Results excludes high risk activities. Third, while maintaining the Bank’s overall consistency in terms of policy content and principles, as well as the commitment to high international standards, Program�for�Results will not seek procedural equivalency to the Bank’s policies and procedures designed for Investment Lending. How will communities know The procedures for disclosure of information and related consultation at both about sub-program activities? the program and project/subproject level would be described in the Will all communities impacted environmental and social system assessment and will be made publicly by all activities in the program available. The assessment includes a review of the arrangements by which be consulted? program activities that affect communities will be disclosed, consulted upon, and subject to a grievance redress process. The party responsible in the borrowing country for program management and implementation would make information available to the public at both the program and project/subproject level through methods that are appropriate to the scope and nature of the program. As part of program implementation support and supervision, the World Bank would monitor the implementation of these procedures and assist the borrower to facilitate their effective application. 51 Feedback Bank Response What happens if the During preparation of a Program-for-Results operation, the borrower and the government does not Bank will agree on the specifics of the program to be supported and the implement the agreed action fiduciary and environmental and social systems requirements to be adhered to plans? Can the Bank do by the client (these may include actions set out in the program action plan). anything about that? These operation-specific undertakings will be reflected in the legal agreements. When performance problems are identified, the Bank will promptly bring them to the attention of the borrower and others concerned. If the borrower fails to take corrective action promptly, or if performance continues to lag, the Bank may invoke remedies as specified in the program legal documentation. The Bank’s standard legal remedies will apply to Program-for-Results operations. Disbursements and Financing Who will design the DLIs and Program-for-Results operations will include a set of “disbursement-linked on what basis? indicators� (DLIs) — indicators that would be the basis for disbursements. They will be designed by the government in collaboration with the Bank and are a key aspect of the appraisal process of a Program-for-Results operation. The indicators will be tangible, transparent, and verifiable, and will have been generated by expenditures supported by the program. Discussion will need to take place between the Bank team and the government (including Ministry of Finance) to work out the best choice of indicators, timeline and amounts to be linked to DLIs. While DLIs could vary in nature, they will be driven by desired outputs or outcomes (e.g., the number of service connections of a particular specification or number of children vaccinated or confirmed as going to school). Moreover, while it is also desirable for DLIs to be primarily outcomes or outputs, they can also be complemented by intermediate outputs or process indicators (e.g., confirmation of substantive participation in decision-making by specified communities); or financing indicators (e.g., share of strategic projects in total expenditures). DLIs can also be key actions aiming to address specific risks or constraints to achieving the results (e.g., implementation of an information system). The indicators chosen will be those judged to be critical to achieving the outputs and outcomes efficiently and effectively. Are DLIs a form of DLIs are not a form of conditionality as the term is understood in the context conditionality? of the once-used Bank’s Structural Adjustment Loans. The selected DLIs are part of the program’s results which are key milestones in achieving the program objectives. They are defined by the government. Linking the DLIs to disbursement provides an incentive for the program implementing agency to achieve its own results. What happens if agreed DLIs Disbursements will occur when results are achieved and verified. Program- are not achieved? for-Results also includes the concept of scalability—that is, partial disbursements if some DLIs are partially met – as well as the possibility for disbursements against prior results, if countries request it. If DLIs are not achieved, there will be no disbursements unless there are unexpected developments that justify a restructuring of the operation and, in that context, some adjustment of the DLIs. 52 Feedback Bank Response How does verification of DLIs The verification of DLIs has to be credible and acceptable to the Bank. The take place? Who is responsible verification protocol defined for each DLI will be specified in the program for that? documents and be part of the program document and implementation support reports that are in the public domain. The protocol will also specify the source of the information for the verification process. In many cases this will be done through an existing agency in government, a semi-autonomous agency that may have M&E capacity, statistical bodies, or a third party. Based on the agreed verification protocol, government has the responsibility of carrying out the verification, and Bank teams will review whether the information provided is in accordance to the agreed protocol as well as follow up as part of implementation support, which could include technical and other audits. How will Program-for-Results As part of the CAS/CPS agreed with the Bank, countries will choose the mix facilitate predictability of of Bank instruments that best meet their needs. By linking disbursements to funding? intermediate results and other actions, it will be possible to phase the expected timing and level of disbursements over the implementation period of the operation. Scalability (partial disbursements against partially achieved DLIs), is also an option to facilitate predictability of fund flow. Additional details will be provided as part of the new policy that will govern the instrument. What are the possibilities for For a government that needs advance funds to achieve the initial DLIs, advance funding and provisions will be made for advances up to 25 percent of the total Program- scalability of funding? for-Results financing before the achievement of initial set of DLIs as well as subsequent DLIs during the implementation period. The amount of the advance will be deducted (recovered) from the amount to be disbursed subsequently. Further advances can be made once an advance has been recovered, so long as the overall limit is not exceeded. Scalability is also a possibility, as indicated above. How will Program-for-Results Because many results will take a long time to achieve, disbursements can be deal with uncertainties and the based on intermediate indicators and results. The Bank has also clarified that long time horizon required to there will be options with respect to scalability, the use of advances and the accomplish many results? disbursements against prior results – all of which should help reduce uncertainties. How will Bank funds be pooled Program-for-Results would enhance the Bank’s ability to partner with other with the government and other development organizations in supporting government programs by permitting partners? And will there be a Bank financing to be disbursed into a government account and pooled with need for counterpart funding resources from the government and other development partners. Government from the government? counterpart financing, as it is defined in the context of Investment Lending operations to be a “counterpart� to Bank financing, will not be required, as programs supported will be primarily government financed. Will additional Bank resources No additional Bank resources will be made available for Program-for-Results. be available for Program-for- Through the country dialogue and CAS process, countries will choose the Results, and what will be the instrument that best suits their needs. Program-for-Results will have the same terms and conditions of terms and conditions as other Bank financing instruments. Program-for-Results financing? 53 Feedback Bank Response Institution/Capacity Building Does Program-for-Results A key feature of Program-for-Results is institutional and capacity building. assume that adequate capacity Program-for-Results does not assume that adequate capacity is already in is already in place? place. As part of the process of preparing a Program-for-Results operation, the Bank will undertake a rigorous and detailed assessment of client capacity. If adequate arrangements are not yet in place or capacity is weak, the Bank will discuss with the government what measures could be taken prior to and/or as part of implementation to address such weaknesses and enhance capacity. Only if it is not possible to reach an understanding or the risks to implementation remain too high, would the Bank decide that the program is not be ready for Program-for-Results. Because capacity building is an integral feature of Program-for-Results, it is expected that it will be used in many cases to help countries strengthen existing institutions and capacity. How will the Bank provide Program-for-Results will support capacity building in several ways: (a) capacity building to countries supporting a stronger focus on results in the management of such programs; with low capacity to (b) various system assessments and related action plans to improve capacity support/implement results- that might be put in place; (c) preparation and implementation support based programs? activities; and (d) stand-alone technical assistance that can be supported either as part of the program or as a standalone Investment Lending activity. How will Program-for-Results Institutional strengthening and capacity building to ensure that programs help strengthen institutions achieve their expected results is a core feature of Program-for-Results, and build capacity? especially in situations of weak capacity. A priority area for both preparation and implementation support will be to strengthen the institutional capacity of the program’s own systems, such as monitoring and evaluation, fiduciary, and environmental and social impacts and risk management. Capacity building can be financed: (a) as part of the Program-for-Results operation; (b) as a separate IL operation; or (c) as a parallel operation co-financed by development partner(s) that may be in a better position to support the necessary capacity building, including through grants in certain countries. An effort to establish an additional trust fund to support the capacity building efforts is also underway. Will Program-for-Results link Capacity building will take a number of forms: (a) through assessments and capacity building to results? identification of actions to improve system assessments; (b) through implementation support; (c) through additional technical assistance, where needed; and (d) through linking resources to results and alignment of internal incentives to results. Hence, the whole focus on results and institutions will, in some sense, link the two efforts together. Explicit linkages of some capacity building to DLIs is possible and is a choice made on a case by case basis depending on how critical some of the actions needed are to achieving the program objectives. Will there be parallel Program-for-Results will use the program’s systems and implementation institutional arrangements for arrangements. There is no need to develop parallel structures. Clear action the government program and plans with adequate incentive structures will be important to attract, retain Program-for-Results? and motivate staff in circumstances where there is a lack of adequate capacity among existing government staff. Will there be any training for Program-for-Results will use program systems to deliver results and will help local agencies? to build the capacity of those systems and enhance their efficiency. Capacity building programs could include training of various kinds, including training for local agencies. 54 Feedback Bank Response Transparency and Access to Information What transparency measures The Bank’s Access to Information policy will apply to Program-for-Results. are included in Program-for- In line with recent trends in the institution, the Bank is bringing more Results? Does the World transparency than ever to its operations. In the case of Program-for-Results, Bank’s Access to Information as the documentation covers the entire program, which will typically be much policy apply? larger in size than the Bank’s financing, the Bank’s “transparency footprint� is significantly enlarged. Moreover, documentation will make results of the program and their achievements accessible by the public. This in turn gives stakeholders access to information about the performance of the public institutions and their achievements and helps strengthen the culture of accountability and transparency in these programs. If the Bank only contributes to Program-for-Results operations call for the use of programs’ institutions and a small percentage of a systems, assessing those against a set of principles and, as needed, agreeing program, how will on enhancements and improvements in performance at the level of the entire transparency and program, and not just the portion financed by the Bank. As such, accountability apply to the enhancements in transparency and accountability agreed between the Bank whole program? and the borrower during the preparation of a Program-for-Results operation will be implemented by the borrower at the program level, regardless of the size of Bank financing to that program. Partnerships How will Program-for-Results Program-for-Results operations will provide a major opportunity to improve promote partnerships with development partner coordination around government-owned programs, other donors and make the contributing to both the effectiveness and sustainability of development Bank a better partner? assistance. With Program-for-Results, the Bank can be a more effective development partner since it will be able to pool its resources with governments and other development partners. Program-for-Results will also address some of the rigidities in current IL that have been at times perceived as a constraint to good partnership. Will the Bank draw on the Yes, the Bank will build on existing knowledge, assessments, and work of analysis and assessments of others, including development partners and credible country institutions. other partners? Assessments can also be done jointly with other partners. How might civil society As with all development programs, participation of key stakeholders will be organizations (CSOs) and the important in Program-for-Results supported operations. The degree and private sector be directly nature of such participation will be decided at the program level. CSOs, in involved in Program-for- particular, could be involved in the implementation of the program as well as Results supported operations? in its monitoring and evaluation. How will Program-for-Results Using the institutional set-up of the program will help various development be used to co-finance a partners to support the program using the same platform. This will, of course, program when the other require that the various development partners agree to this approach. In the funding organizations involved spirit of partnership, the preparation and implementation support for the do not have a results-based program should be closely coordinated among development partners. instrument? 55 Feedback Bank Response New Policy Framework Why is it necessary to have a Since Program-for-Results will be a new instrument and distinct from both IL new operational policy for and DPL, it will need its own Operational Policy (OP) and Bank Procedure Program-for-Results? (BP) statement, accompanied by additional technical guidance notes or guidelines to staff. The OP will set out the essential components of the new instrument, describing the general purposes of a Program-for-Results operation and covering the regulatory, governance, institutional, financial management, procurement, environmental, and social areas. The OP will make clear that these policies apply to the entire program to be supported by a Program-for-Results operation. The BP will spell out the related procedures that will need to be carried out during preparation, appraisal, and implementation. Will the public have a chance Draft OP/BP 9.00 was made publicly available for comment as part of the to comment on the OP/BP second phase of consultations (August 15 to September 30, 2011). before they are considered for approval by the Board of Executive Directors? The exclusions imposed on The overall scope of a program will be determined primarily on the basis of Program-for-Results will come an assessment of the program’s procedures, capacity, and performance, and at a cost and are likely to the risks associated with using the program’s systems with and without reduce its effectiveness as an improvements. But there are some high-risk activities that are better instrument. They should be addressed within the policy framework of Investment Lending. These include based on risks and client activities that have significant and irreversible impact on the environment and capacity and not decided a affected people (Category A under IL) and high value procurement contracts. priori. While these exclusions might result in missed opportunities, on balance, they have been identified as activities that are best suited for Investment Lending. Does OP 9.00 comply with the The Program-for-Results instrument described in the policy paper complies requirements in the Article of with the Articles of Agreement. The Articles provide, in part, that Bank loans Agreement for both IBRD and should be used for productive purposes, for specific projects (except in IDA that the Bank provide exceptional circumstances), for intended purposes with due attention to funding only for specific considerations of economy and efficiency, and for expenses as they are projects except under special actually incurred. Programs proposed to be financed by Program-for-Results circumstances? will meet these requirements. Thus the Articles provide sufficient legal basis to allow for Board approval of the new instrument. Risks What are the risks facing the The Bank faces a range of risks with respect to the nature of the instrument. Bank in introducing this new These include whether it will be possible to: (a) adequately specify and instrument and how will these monitor results, (b) to effectively address fiduciary and environmental and risks be addressed? social concerns, and (c) to successfully build institutions and capacity. There is also the risk of widely differing expectations of the new instrument. To address these risks the Bank will put in place a new OP/BP, associated guidance notes, support to clients and staff, and extensive communications. In addition, Bank Management is providing enhanced corporate oversight in the early years. 56 Feedback Bank Response Role of the World Bank What will be the Bank’s role in The Bank will be responsible for assessing the quality of the government the preparation, appraisal ,and program to be supported by a Program-for-Results operation and its implementation of Program- associated systems. It will also agree with the government, in the context of for-Results supported appraisal, on any necessary improvement measures and on the Bank’s role in operations? supporting the implementation of those measures. There will be a strengthened focus on implementation support. The Bank will be responsible for providing close and frequent implementation support to government teams implementing Program-for-Results. Work with other development partners will also be critical for implementation support. Does Program-for-Results Based on the experience with programmatic operations supported as part of imply any resource IL, it is not clear that Program-for-Results operations will cost on average implications for the Bank? more than the average for other types of Bank lending. However, it is difficult to predict the resource requirements due to a number of uncertainties. Cost will be influenced by many factors: (a) previous engagement in the sector and with the institutions that will implement the program; (b) capacity of the clients; (c) involvement of other donors, some of whom may “share� the costs of us all getting what we need; (d) and the cautious roll-out and enhanced oversight. However, there is large potential for saving staff time on review of specific transactions (for example small procurement contracts) and shifting these staff resources to direct support for the strengthening of government systems and capacity and achievement of results. Management is not asking for additional resources as a result of the introduction of Program- for-Results but will monitor the cost of preparation and implementation during the initial years of the roll-out Will Bank staff have the A training program on the new instrument will be put in place for staff to necessary training, support ensure that staff are trained on the new policy and its implementation. In and incentive to work on such addition, guidance notes will be provided to staff. In the initial phase, a operations? corporate support team will be put in place to help/guide task teams of new Program-for-Results operations through the process. As for incentives, many staff view Program-for-Results as an opportunity to respond to client demands and needs and do more in terms of institutional capacity building and be better partners. Support and guidance to staff and clarity about their roles will be addressed in the various Program-for-Results supporting documents. What will be the role of the Program-for-Results will be subject to the same oversight functions as other Bank’s oversight units? Bank lending instruments. The independent corporate oversight units— Inspection Panel (IPN), Institutional Integrity (INT), Internal Audit Department (IAD), and Independent Evaluation Group (IEG)—will oversee Program-for-Results operations. What will the Bank’s Bank teams will be responsible for supervising the implementation of supervision process be? Program-for-Results operations, in collaboration with development partners. During implementation support, task teams would focus on reviewing the monitoring and validation of results and DLIs, capacity building efforts, resolving implementation issues, implementation of the agreed action plans, as needed, monitoring overall performance of the systems, and so forth. Will the World Bank carry out Like with other lending operations, an Implementation Completion and final evaluations of programs Results report (ICR) will be prepared by the Bank, covering, among other supported by Program-for- matters, the degree to which the program objectives and results were Results? achieved, the program’s performance and achievement of DLIs among others as well. It incorporates the government’s evaluation of the program, as well as of its own performance and the performance of the Bank. 57 Feedback Bank Response Rollout of the Instrument How will Program-for-Results The proposed Program-for-Results instrument builds on 10 years of be rolled out? And will there experience. Many of the features of Program-for-Results have already been be a piloting stage? piloted through program-based operations under Investment Lending. In the early years, lending will be limited in terms of the proportion of total IDA and IBRD lending. In addition, Bank Management is putting in place enhanced corporate oversight in the initial phase of implementation. The Bank will prepare a progress report after two years of implementation, which will be discussed with the Bank’s Board of Executive Directors and disclosed publicly. What are the expectations for There are no targets for the instrument. However, in the early years, Bank the share of Program-for- Management is proposing a limit on the share of lending to allow for a Results relative to IL and cautious roll out and an opportunity to learn from early implementation DPL? Will there be a cap on experience. Program-for-Results lending? What’s the expected timeline The two-phase global consultation process began in March 2011 and on next steps? concluded at the end of September 2011. The Bank revised/updated the draft Policy Paper taking into account the feedback received throughout the process. 58 List of Countries Included in the Consultations From March to September 30, 2011, the World Bank held consultations with 34 client countries and 7 donor countries, and multi-stakeholder meetings with 9 donor countries, 9 development partners, and several hundred civil society and business representatives. Below is a list of meetings held as part of the consultation process. In addition, the Bank received written comments from numerous individuals, CSOs, and private sector organizations. Client Countries (34 Total) Donor Countries (16 Total) AFR (8 Total) Donor Country Consultations (7 Total) Cameroon (VC with HQ) Belgium Ethiopia Canada Ghana (VC to Senegal) France Malawi (VC with HQ) Germany Mozambique (VC with HQ) Japan Nigeria Switzerland Senegal UK South Africa EAP (4 Total) Multi-Stakeholder Meetings with Donor Countries (9 Total) Indonesia Nordic-Baltic Representatives (Denmark, Estonia, Laos Finland, Iceland, Latvia, Lithuania, Norway, Sweden) Philippines USA Vietnam LCR (11 Total) Development Partners (9 Total) Argentina ADB Brazil AFD Colombia AfDB (VC with Tunis) Dominican Republic AusAID Jamaica DFID Nicaragua ( + Guatemala, Honduras, El Salvador, EC Costa Rica) JICA Uruguay IDB UN/New York ECA (4 Total) Other Consultation Events Romania Center for Global Development Russia (VC to Beirut) IFI Liaison Officers Serbia (VC to Romania) Spring Meetings (global civil society representatives) Turkey Annual Meetings (global civil society representatives) MNA (3 Total) Annual Meetings (government officials and Annual Egypt (VC to Beirut) Meetings delegates) Lebanon Overseas Development Institute Seminar West Bank and Gaza Multiple meetings with US and European Business SAR (4 Total) Representatives Bangladesh (VC to HQ) Bhutan India Pakistan (VC to India) ANNEX B. SUMMARY OF EXPERIENCE 1. In October 2009, OPCIL carried out a review of the Bank’s experience with various lending operations that incorporate programmatic-type features (referred to here as “programmatic IL�): sectorwide approaches (SWAps), conditional cash transfers (CCTs), output-based aid (OBA), results-based financing in health (RBFH), and operations under the Education for All—Fast-Track Initiative (EFA FTI). The review also looked at the approaches of three external agencies created in the last decade that pursue important dimensions of program- based aid—the Global Fund to Fight AIDS, Tuberculosis and Malaria, the Millennium Challenge Corporation (MCC), and the GAVI Alliance. During the development of the proposed Program- for-Results (Program-for-Results) instrument, the Inter-American Development Bank’s Performance-Driven Loans (PDLs) and the European Commission’s General Budget Support were also reviewed. This summary illustrates the demand for results-based instruments and describes approaches and lessons learned. A. Sectorwide Approaches 2. Since the mid-1990s, an increasing number of Bank operations support sectorwide programs. A recent analysis identified 111 IBRD/IDA SWAps approved since 2000, with a combined commitment amount of $16.5 billion. In terms of volume, 46 percent of SWAps are in LCR and 26 percent in AFR, but AFR has the greatest number of projects, 50. Of the 111 SWAps, the largest percentages (by value) were in social protection, education, and public sector governance. 3. Evolution of SWAps. SWAps were first used largely in the social sectors in donor- intensive countries in AFR and SAR, but are now used much more broadly. They have grown strongly in middle-income countries, notably in LCR; and they have grown beyond social sectors to include water, transport, agriculture, and others. The Regions report that client demand for SWAps is strong. SWAps have helped advance donor alignment to country strategies, better coordinate donors, and make significant institutional change and reform in many countries, notably in public financial management. The SWAp concept has been a platform for innovation and greater flexibility. It has also allowed the Bank, other donors, and the countries themselves to focus more strongly on results. A recent analysis of SWAp project performance shows indicators that compare favorably with the overall lending portfolio, with the share of commitments at risk and problem projects significantly lower. 4. Lessons Learned. At the same time, there have been difficulties. Initial assumptions have often proved too optimistic, especially in terms of low-income countries’ institutional capacity to manage larger and often more complex programs. The early momentum in some (mainly low-income) countries has been difficult to sustain. SWAps have increased the transaction costs for donors, particularly the lead donor. New donors, especially in the social sectors, have found the SWAp framework insufficiently flexible to allow the degree of focus they wished to have on the area to which they are dedicated. Some established donors, including the Bank, have judged it appropriate to do stand-alone, parallel interventions alongside SWAps. The absence of clear policies and guidance on SWAps has led to confusion and inconsistent approaches among staff. Bank staff routinely report the internal processing costs of these types of operations to be excessive, diverting time and attention from substantive matters. More 60 recently, staff have expressed a lack of clarity about the applicability of the Bank’s fiduciary and fraud and corruption policies in an IL environment in which the Bank finances broader expenditure programs rather than individual transactions. B. Conditional Cash Transfers 5. CCTs have grown strongly in the last 15 years.1 The Bank estimates that they are being implemented in more than 30 countries, with the heaviest concentration in LCR, but with large- scale programs in Bangladesh, Philippines, and Turkey, and with pilot programs in Cambodia, Malawi, Morocco, Pakistan, and South Africa. In addition, there has been impressive growth in the numbers of beneficiaries reached: Mexico’s PROGRESA, which started with approximately 300,000 beneficiary households in 1997, now covers 5 million households;2 and Bolsa Família in Brazil now covers 11 million families, or around 46 million people—almost one-quarter of the population. Bank support for CCT programs has grown sharply. It is mainly concentrated in LCR, where the Bank has approved $3.8 billion in loans to support 14 operations in nine countries. The largest sectoral concentration is in social protection and education. The Regions report strong demand for further CCT lending. 6. Results and Lessons. CCT operations compare well to the total portfolio, and there is considerable evidence that CCTs have improved the lives of poor people. Transfers generally have been well targeted, have raised consumption levels, and have reduced poverty—by a substantial amount in some countries. CCTs have provided an entry point to reforming badly targeted subsidies and upgrading the quality of safety nets, while making sure that offsetting adjustments have been modest. CCTs also have a strong evaluation culture, well beyond traditional practice in social policy; and this culture is spreading from one CCT to another, as well as to other programs in the same country. At the same time, the evidence on improvements in final outcomes in health and education is more mixed. CCTs have worked best when the supply side has been in place to respond to the demand; in this area various Bank instruments and tools can help countries achieve optimal outcomes when applied to solving the appropriate constraint. Experience also suggests the need to experiment with conditions that focus on outcomes rather than on the use of services alone. A key lesson from the financial management perspective is that no CCT program is immune from error or even some degree of fraud, even with best controls in place; thus an important part of assessing a potential CCT project for Bank support is to determine whether the level of risk is acceptable. C. Output-based Aid 7. OBA, broadly defined, seeks to tie disbursement to the achievement of specific outputs. The Bank-administered Global Program on Output-based Aid (GPOBA) started in 2000 to both directly support OBA activities using trust funds and promote the wider use of OBA in the development community.3 With a modest level of resources, the program used a more focused definition of OBA—a mechanism that ties the disbursement of explicit, performance-based 1 See Conditional Cash Transfers—Reducing Present and Future Poverty, World Bank Development Research Group, February 2009. 2 This program, now known as Oportunidades, is supported by an IBRD loan of $1.5 billion, the Bank’s largest CCT loan. 3 See www.gpoba.org and Output Based Aid: A Compilation of Lessons Learned and Best Practice Guidance, GPOBA/IDA-IFC Secretariat, June 2009. 61 subsidies or grants to the achievement of specified outputs that directly support improved access to basic services by poor households.4 GPOBA has supported such basic services as improved water supply, energy, health care, education, information and communications technology, and transport. It places particular emphasis on explicitly targeting output-based subsidies to the poor. Hence, an OBA subsidy finances the funding gap between the cost of service delivery and the beneficiaries’ ability and willingness to pay user fees for the service. Services are contracted out to a third party (service provider) through either a competitive process or a single-source selection process, in both cases following Bank Procurement Guidelines, with elaborate processes in place to determine the amount of subsidy, share risk, and independently verify the output. 8. Results. The OBA portfolio includes 115 closed and active projects. Of these, 57 projects are funded by IDA, 25 by IBRD, and the rest by trust funds and GEF. The size of the OBA subsidy portfolio is $4 billion, of which IBRD has the largest share ($2.5 billion) despite the smaller number of projects. Most of the projects are in AFR (because of recent piloting efforts by GPOBA) and in LCR (where OBA was first piloted in many sectors). LCR accounts for 51 percent of the total volume of subsidies, while AFR accounts for 23 percent of projects that are active or have closed. Disbursements have similar shares. It is estimated that of the active and closed projects with OBA components, 80 percent of the funds—$3.2 billion—have been disbursed. The largest share of OBA projects, both in terms of volume and number of projects, is under the Transportation Sector Board. The energy and mining, health, and education sectors also have relatively large shares in OBA projects. 9. Lessons Learned. The OBA operations provide important lessons for the further development of results-based financing.  They offer increased transparency through the explicit targeting of subsidies. The focus on subsidies for access is inherently pro-poor: the poorest segments of the population often cannot afford initial access and therefore often do not benefit from subsidies for ongoing service provision. Furthermore, if outputs are explicitly defined, targeting can be made more precise. The process of output verification, which can serve as an additional check on the targeting of subsidies, is helping to provide early evidence that OBA schemes are reaching the poor.  There is increased accountability because performance risk is shifted to service providers: OBA pays providers only after delivery of verifiable access and service.  OBA leverages some private funding, but because of its generally pro-poor nature, private financing leveraged is limited by the extent that user fees can incorporate investment costs while remaining affordable.  Some evidence indicates that output-based payments have led to improvements in operational efficiency and the delivery of innovative, often pro-poor, access-to- service solutions.  Finally, OBA enhances the monitoring of results by paying on verified outputs. Best practice would also use the monitoring platform of OBA beyond just the verification of outputs to check other aspects of service delivery. With OBA schemes, 4 GPOBA also draws a distinction between OBA and output-based disbursement (OBD), defining the latter as disbursement to improve the efficiency of assets or government systems, while OBA tends to support private provision of services. 62 accountability also increases for donors and governments because public funding is linked to delivery of pre-identified outputs, therefore waste or inappropriate allocation of such funding should be minimized.5 10. GPOBA considers that the main challenge for mainstreaming OBA approaches lies in reconciling the Bank’s requirement that investment lending needs to be “expenditure-based� with the “performance-based� nature of OBA and similar mechanisms. It wishes to see the Bank go beyond the guidance it has provided to staff on procurement for OBA. Specifically, it would like to see incentives for innovation and a greater emphasis on outputs rather than inputs, including more amenable fiduciary procedures. D. Results-based Financing for Health 11. In general, results-based financing mechanisms operate on the basis of payment for agreed outputs rather than the traditional payment for inputs. For several years the Bank has worked with other donors to promote results-based financing in health (RBFH), an innovative financing strategy that aims to increase the impact of investments in health by providing a financial or in-kind reward upon achievement of agreed performance goals—payment to a provider, payer, or consumer is made when measurable actions are taken or defined performance targets are achieved.6 RBFH is an umbrella term that includes output-based aid, performance- based financing, provider payment incentives, vouchers, contracting linked to particular targets, and conditional cash payments and transfers to households. In some countries, RBFH may take the form of paying a bonus to health facilities that meet certain quantity or quality targets, such as percentage of women delivering their babies in a facility. Other countries are designing RBFH mechanisms to provide incentives and support to poor people to overcome hidden barriers—such as the cost of transport—to using services. 12. Prevalence, Scope, and Support. A recent stocktaking of Bank RBFH experience identified 260 active and closed projects over the period FY95-FY08. Of these, 15 percent identified an RBFH element, which was considered “substantial� in 60 percent of the cases. Of those considered “substantial,� 43 percent were in LCR and 27 percent in AFR. Most of these projects were approved in the last five years. The mechanisms spanned a wide range—from performance agreements with national and subnational governments, insurance entities, public facilities, private providers, and NGOs; to performance-based worker incentives and arrangements with communities; to conditional cash transfers. The dominant Bank lending instrument was IL. The total value of support by IBRD/IDA was $3.79 billion, and IDA financing represented 68 percent of support in active projects. RBFH received an added impetus in September 2009, when members of the High-Level Task Force on Innovative Health Financing committed a total of $5.3 billion to accelerate progress toward the Millennium Development Goals. The UK and Norway will contribute $420 million to support results-based financing programs and buy-downs to improve maternal and child health, and the Government of Australia will provide AUD336 million over the next four years for performance-linked aid to 5 Output-Based Aid: Lessons Learned and Best Practices (2010). 6 See www.rbfhealth.org and Taking Stock: World Bank Experience with Results-based Financing in Health, Brenzel and others, HDNHE, World Bank, June 2009. 63 help partner governments in Asia and the Pacific. A significant portion of the new funds committed will be channeled through the Bank’s Health Results Innovation Trust Fund. 13. Challenges. Teams working in this area have noted that the Bank is missing opportunities for adding impact through results and have identified several areas for attention: the need for feasible mechanisms for disbursing against results, better analysis, and in-country capacity building, and the need to help staff to put in place adequate monitoring frameworks (including baseline data) and to integrate RBF into bigger operations and initiatives,. E. Global Partnership for Education 14. The Global Partnership for Education (previously Education for All – Fast-track Initiative (FTI)) was created in 2002 to help low-income countries achieve free, universal basic education by 2015.7 Nineteen donors pledged $2.0 billion to the FTI Catalytic Fund between 2003 and 2013. These funds have been almost fully committed to 37 countries (all IDA) as of March 31, 2011, with 48 percent disbursed. The 2011-2014 replenishment is seeking US$2.5 billion in contributions. 15. EFA Compact. Under the EFA Compact, partner countries commit to develop sound education sector programs through broad-based consultation, show commitment to education through strong domestic support, demonstrate results on key performance indicators, and exercise leadership in developing and implementing the program and coordinating donor support. Donors commit to help mobilize resources and make them more predictable, align with country development priorities, coordinate support around one education plan, and harmonize procedures as much as possible. The initiative seeks to apply the 2005 Paris Declaration on Aid Effectiveness. Staff and partners associated with the program report that rigid application of existing Bank fiduciary rules (mainly procurement), or failure to adapt them to the programs being supported, causes unnecessary delay and undermines program implementation and the achievement of results. F. Global Fund to Fight AIDS, Tuberculosis and Malaria 16. The Global Fund to Fight AIDS, Tuberculosis and Malaria is a global public/private partnership—of governments, civil society, the private sector, and affected communities— dedicated to attracting and disbursing additional resources to prevent and treat HIV/AIDS, tuberculosis, and malaria. It works in close collaboration with other bilateral and multilateral organizations. Created in 2002, the Global Fund has become the main source of finance for programs to fight the three diseases, with approved funding of US$22 billion for programs in 150 countries. Globally, it provides one-quarter of all international financing for AIDS, two-thirds for tuberculosis, and two-thirds for malaria.8 17. Operating Model. The Global Fund placed performance-based funding at the heart of its operating model, to ensure that funding decisions are based on a transparent assessment of results 7 See www.educationfasttrack.org and A Fast Track to 2015: Educating the World’s Children for a Better Future, October 2009. 8 See www.theglobalfund.org, Global Fund Results Report 2011. 64 against time-bound targets. Global Fund programs receive initial funding on the basis of the quality of their applications. To receive subsequent financing, they must demonstrate results against defined performance targets. These targets are proposed by the country (for approval by the Global Fund) to ensure that they are appropriate to the national context and local program realities. This system of performance-based funding was developed to  Link funding to the achievement of country-owned objectives and targets;  Ensure that money is spent on delivering services for people in need;  Provide incentives for grantees to focus on programmatic results and timely implementation;  Encourage learning to strengthen capacities and improve program implementation;  Invest in measurement systems and promote the use of evidence for decision-making;  Provide a tool for grant oversight and monitoring within countries and by the Global Fund Secretariat;  Free up committed resources from nonperforming grants for reallocation to programs in which results can be achieved. 18. Fiduciary Procedures. The Global Fund established its own fiduciary arrangements. Its experience illustrates the difficulties posed by equating specific operating procedures (which can quite appropriately vary from country to country, or from organization to organization) with overall fiduciary objectives. G. The Millennium Challenge Corporation 19. The MCC was created in 2004 as an independent foreign aid agency of the United States to help in the fight against global poverty in innovative ways. Rather than delivering development finance in traditional ways, it focuses on good policies, country ownership, and results.9 The MCC places great emphasis on achieving and measuring results, concentrating on identifying activities that have the greatest promise of poverty reduction through growth, measuring progress during implementation, and learning from experience. 20. MCC Approach. The MCC has approved US$7.6 billion in “compact� funding to 21 countries, and US$495 million in “threshold program� funding to a further 21. It has disbursed over US$2 billion. Beneficiary countries are selected competitively on the basis of 17 policy indicators and their policy performance. The countries are required to identify their priorities for achieving sustainable economic growth and poverty reduction, and to develop their MCC proposals in broad consultation within their society. For compact countries, a local, country- owned Millennium Challenge Account (MCA) accountable entity is set up to manage and oversee all aspects of implementation, including overseeing the MCC program and its 9 See www.mcc.gov, 2010 Annual Reports. 65 components, allocating resources, overseeing and implementing a financial plan, approving expenditures and procurements, and being accountable for MCC program results. Funding has been in such sectors as agriculture and irrigation, transportation (roads, bridges, ports), water supply and sanitation, access to health, finance and enterprise development, anticorruption initiatives, land rights and access, and access to education. 21. Lessons Learned. A recent MCC publication highlights the main lessons learned from the agency’s experience thus far.10  Transparency is a big part of accountability for results. Easy access to financial and program information allows partner governments to plan and budget their own development strategies and allows citizens to hold their governments accountable for good investments and results. For this reason, MCC makes information available along the full cycle of country engagement, from country selection through implementation to results.  “Results� means different things at different times in the project cycle. MCC results exist along a continuum—from policy changes countries may make to become compact-eligible, to interim outputs and outcomes as compacts mature, to post- compact increases in incomes. As compacts enter into force, M&E mechanisms begin to track process milestones, such as key contracts signed and program financial progress. As implementation advances, they track output indicators, such as number of farmers trained, hectares planted with high-value crops, miles of road constructed, or land titles granted—because these are the drivers of the income gains MCC investments aim to achieve.  Detailed project design in the early years produces better results in the long-term. In MCC’s start-up years, much of the detailed project preparation work, including establishing key implementation structures, staffing, feasibility and design studies, and environmental and social impact assessment, was delayed until after the compact was signed. As a result, the first years of implementation in the early compacts often were dedicated to detailed program planning and program start-up, rather than to implementing activities that would show significant disbursements and tangible outputs.  Data quality is the cornerstone of robust focus on results. Reporting on results is basically about data—capturing it, vetting it, reporting it publicly, and putting it into context to tell a story about MCC investments. The better the data quality, the better the reporting. Factors that affect data quality are varied, and include the capacity of local survey firms collecting baseline and follow-up data; the degree to which household and targeted beneficiaries are accustomed to formally tracking key indicators; timeliness of data collection; MCA staff previous experience with M&E techniques; and MCA staff ability to manage a growing number of indicators to meet program monitoring and external reporting needs. 10 Principles into Practice: Focus on Results. Department of Policy and Evaluation, MCC (2011). 66 H. The GAVI Alliance 22. The GAVI Alliance (formerly known as the Global Alliance for Vaccines and Immunization) was launched in 2000 as a global health partnership of the private and public sectors to accelerate access to underused vaccines, strengthen countries’ health and immunization systems, and introduce innovative new immunization technology, including vaccines. The partnership comprises developing world and donor governments, private sector philanthropists such as the Bill & Melinda Gates Foundation, the financial community, developed and developing country vaccine manufacturers, research and technical institutes, civil society organizations, and multilateral organizations like the World Health Organization (WHO), the United Nations Children’s Fund (UNICEF), and the World Bank. It aims to contribute to achievement of the Millennium Development Goal for child health—a two-thirds reduction in the number of deaths in children under five years of age by 2015. 23. GAVI Approach. GAVI support is subject to performance monitoring designed to track progress achieved in the previous year, declare planned targets for the following year, and verify the sustainability of existing financing mechanisms. It therefore gives considerable prominence to monitoring through annual progress reporting, data quality auditing, and vaccine management assessment. An Independent Review Committee reviews each annual progress report, making technical comments and suggestions, and recommends one of the following three outcomes to the GAVI Alliance Board—to continue providing support, request clarifications before continuing to provide support, or request the country to resubmit its report as insufficient information has been provided. Independent verification is pursued though data quality audits, which are required for countries receiving immunization services support during the second year.   24. Lessons Learned. A report on the experience of countries that received GAVI support provides some lessons for the development of a results-based financing instrument.11 For example, the predominant concern of many countries was that five years is too short a period to introduce and transfer responsibility for funding new vaccines and attain global targets. Monitoring and reporting processes were considered burdensome and the collection of data difficult in the developing country context. A recommendation from stakeholders was to consolidate reporting to reduce duplication. Another lesson related to monitoring was that the Data Quality Audit process, conducted in Year 2 to verify reporting systems and identify areas of the immunization monitoring and reporting systems that require improvement, was reported to have a catalytic effect on country reporting systems. Finally, countries appreciated the incentive benefits of a performance-based mechanism for GAVI systems support; however, understanding of the precise nature of the investment and reward mechanism was fairly limited. I. Inter-American Development Bank Performance-Driven Loans (PDL) 25. In December of 2003, the Inter-American Development Bank (IDB) approved a proposal for a six-year pilot program for Performance-Driven Loans (PDLs). Designed to tie disbursements to the achievement of actual results, the PDLs focused on project outcomes rather than project inputs and processes. The focus of preparation was primarily on identifying project 11 Lessons Learned from GAVI Phase 1 and design of Phase 2: Findings of the Country Consultation Process. Report to GAVI Secretariat (2005). 67 outcomes, selecting outcome indicators, setting outcome targets, ensuring that a solid and reliable system was in place to track outcome indicators, and verifying the achievement of targets for disbursement. Between 2005 and 2008, the IDB approved nine PDL projects (US$515 million). 26. Lessons Learned. The low use of the PDL is related primarily to two factors—the difficulty of matching expected results with programmed resources, and implementation problems associated with needing to verify expenditures. Establishing a direct relationship between expenditures and performance outcomes is considered challenging, and in many cases unrealistic, particularly in sectors where the IDB and other multiple donors are supporting a national development goal. In the health sector, for example, outputs such as training of health professionals can be costed-out, but the final outcome of reducing mortality rates for children under five years of age is actually achieved through financing a combination of activities and many outputs. Given that the accountability rests with the executing agency to select the most cost-effective input-output mix for achieving the performance targets, the task of costing activities and outputs loses relevance. The fact that PDLs, as investment loans, are required to track eligible expenditures directly related to the outputs and outcomes established in the results matrix constitutes a burden for both the borrower and IDB staff, since PDLs carry the same accounting and reporting requirements as regular investment loans.12 J. European Commission Budget Support 27. The European Commission (EC) defines budget support as the transfer of financial resources from an external financing agency to the national treasury of a partner country, where they form part of the partner country’s global resources and are used in accordance with the country’s public financial management (PFM) system. The EC provides budget support only to countries that meet three eligibility conditions: they have in place (a) a well-defined national or sectoral development or reform policy and strategy, (b) a stability-oriented macroeconomic framework, and (c) a credible and relevant program to improve PFM. General Budget Support (GBS) is a transfer to the national treasury in support of a broad range of development or reform policies set out in a national strategy. Under such programs the focus is on the overall fulfillment of development and reform objectives and important cross-cutting issues with implications for the efficiency and effectiveness of policies in all sectors. Sector Budget Support (SBS) is a transfer to the national treasury in support of a narrow range of development or reform policies set out in a sectoral strategy. Under such programs the focus is on development and reform objectives in a particular sector—water, road maintenance, health, and so on. The EC provides both GBS and SBS as direct, untargeted budget support; there is no procedural distinction between them. 28. In 2004 a group of 24 aid agencies and 7 partner governments commissioned a joint evaluation of GBS,13 which identified several main challenges of GBS: (a) the short-term predictability of GBS has been a frequent problem; (b) GBS has been criticized for bias toward the expansion of public services, without enough attention to quality, and for neglecting private 12 The Evaluation of the New Lending Framework, 2005-2008. Office of Evaluation and Oversight, IDB (2008). 13 The Joint Evaluation of General Budget Support 1994–2004. IDD and Associates (2006). 68 sector growth and development; and (c) corruption is a serious problem, but there was no clear evidence that budget support funds were more affected by corruption than other forms of aid. 29. MDG Contract. In 2009, the EC launched the MDG Contract, a longer-term, more predictable form of general budget support14 with the following key features: (a) a commitment of funds for the full six years of EDF 10; (b) a base component of at least 70 percent of the total commitment, which will be disbursed if there is no breach in eligibility conditions for general budget support, or in the essential and fundamental elements of cooperation; and (c) a variable performance component of up to 30 percent, made up of an MDG-based tranche and an Annual Performance Tranche. Countries eligible for the MDG Contract are those with general budget support programmed under EDF 10 that have a successful track record in implementing budget support, show a commitment to monitoring and achieving the MDGs and to improving domestic accountability for budgetary resources, and have active donor coordination mechanisms to support performance review and dialogue. 30. Although it is still quite early in the implementation of the MDG Contract, several lessons can be learned from the principles guiding the instrument’s design:  Annual monitoring and focus on results. Country performance with respect to key MDG-related results, PFM reforms, and macroeconomic stability are monitored annually in the context of joint reviews of PRSPs and Performance Assessment Frameworks.  Design and implementation aims to support processes of donor harmonization and alignment to minimize the transaction costs and maximize the coherence of donor budget support.  Design is kept as simple as possible to improve the ease and transparency of operation.  Flexibility at country level. To accommodate diverse country contexts, the program incorporates flexibility at the country level in several key areas: the set of indicators to be followed, the precise rules for translating performance assessments into disbursement decisions, and the potential disbursement profile and subdivision of the variable performance element. 14 See http://ec.europa.eu/europeaid/what/millenium-development-goals/contract_mdg_en.htm ANNEX C. DRAFT OPERATIONAL POLICY STATEMENT (OP) 9.00 OP 9.00 – Program-for-Results Financing This operational policy statement (OP) was prepared for use by World OP 9.00 Bank staff and is not necessarily a complete treatment of the subject. ______, 2011 Bank-financed Program-for-Results operations are governed by this OP 9.00, the related BP 9.00, and the following OPs/BPs, as appropriate (including any relevant Operational Memoranda): OP 1.00, Poverty Reduction BP 2.11, Country Assistance Strategies OP/BP 2.30, Development Cooperation and Conflict OP/BP 3.10, Financial Terms and Conditions of IBRD Loans, IBRD Hedging Products, and IDA Credits OP 4.07, Water Resource Management OP 4.76, Tobacco OP/BP 4.20, Gender and Development OP/BP 7.00, Lending Operations: Choice of Borrower and Contractual Arrangements OP 7.20, Security Arrangements OP 7.30, Dealing with De Facto Governments OP/BP 7.40, Disputes Over Defaults on External Debt, Expropriation, and Breach of Contract OP/BP 7.50, Projects on International Waterways OP/BP 7.60, Projects in Disputed Areas OP/BP 8.45, Grants OP/BP 10.20, Global Environmental Facility Operations OP/BP 13.00, Signing of Legal Documents and Effectiveness of Loans and Credits OP/BP 13.16, Country Portfolio Performance Reviews OP/BP 13.30, Closing Dates OP/BP 13.40, Suspension of Disbursements OP/BP 13.50, Cancellations OP/BP 14.10, External Debt Reporting OP/BP 14.20, Cofinancing OP/BP 14.40, Trust Funds BP 17.30, Communications with Executive Directors BP 17.55, Inspection Panel Questions related to this OP and BP 9.00 may be addressed to the OPCS Help Desk. 1. Program-for-Results Bank financing1 aims to promote sustainable development and improve the efficiency and effectiveness of expenditures by: (a) financing the expenditures of specific borrower development programs; (b) disbursing on the basis of the achievement of key results (including prior results) under such programs; (c) using and, as appropriate, strengthening the program systems to provide assurance that program funds are used appropriately and that environmental and social impacts are adequately addressed by such programs; and (d) strengthening, where appropriate, the institutional capacity necessary for such programs to achieve their intended results. 2. Programs supported by Program-for-Results financing (hereinafter “Programs�) have expenditures, activities, and defined results, and promote sustainable development. The Programs may be: (a) new or already under implementation; (b) national, subnational, multisectoral, sectoral, or sub-sectoral in scope; (c) part of broader, longer-term, or 1 Program-for-Results Bank financing refers to the provision of loans, credits, or grants financed by the Bank (including IBRD and IDA) from its resources or from trust funds financed by other donors and administered by the Bank, or a combination of these (hereinafter “Program-for-Results financing�). 70 geographically larger programs; and/or (d) carried out by governmental and/or nongovernmental parties. 3. Subject to the other applicable requirements of this OP, Program-for-Results financing may be extended to any type of expenditures, provided the Program is productive and Program oversight arrangements, including the fiduciary arrangements, provide reasonable assurance that financing proceeds will be used for the purposes for which the financing is granted, with due consideration of economy and efficiency. The amount of Program-for-Results financing should be equal to or less than the total Program expenditures. If by the end of the Program, the amount of Program-for-Results financing disbursed exceeds the total amount of expenditures under the Program, taking into account contributions from other financing sources, the borrower is required to refund the difference to the Bank. 4. Subject to the other applicable provisions of this OP, financing proceeds are disbursed upon the achievement of verified results specified as disbursement-linked indicators (DLIs). Such disbursements are not dependent upon or attributable to individual transactions or expenditures. Under appropriate circumstances, such as to provide the borrower with resources to allow the Program to start or to facilitate the achievement of one or more DLIs, the Bank may agree to disburse a portion of loan proceeds as an advance for DLIs that have not yet been achieved. Considerations for Program-for-Results Financing 5. The Bank’s assessment of a proposed Program is based on various country and Program- specific strategic, technical, and risk considerations. These include the Program’s strategic relevance, technical soundness, expenditure analysis, economic rationale, results framework, fiduciary and environmental and social systems2 and risks, including governance arrangements.3 The assessments evaluate the relevant risks and the scope for improvements and managing such risks, including proposed institution strengthening activities to be undertaken before, if deemed appropriate, and during Program implementation. 6. The technical assessment considers, among other things, the Program’s rationale and its development objectives (taking into account consistency with the Bank’s overall assistance strategy for the member country in question); the borrower’s commitment; relevant institutional and implementation arrangements, including governance; the Program’s activities and expenditures framework; the degree to which the Program aims to achieve specific, measurable, and verifiable results; the monitoring and evaluation arrangements; and the general policy, legal, regulatory, and institutional frameworks relevant to the Program. 2 In this OP, the term “systems� or “Program systems� refers to the systems of the institutions responsible for Program implementation, including the relevant rules, procedures, controls, risk identification and mitigation arrangements, and, when applicable, past implementation experience and record. 3 In this OP, references to the Program governance arrangements and risks refer to: (a) good governance principles (transparency, integrity, accountability and effectiveness); (b) handling the risks of fraud, corruption, coercion, collusion, and similar activities (hereinafter “fraud and corruption�), and how such risks are managed and/or mitigated; and (c) compliance with the requirements set out in the Guidelines on Preventing and Combating Fraud and Corruption in Program-for-Results Financing (________, 2011), as such guidelines may be amended from time to time. 71 7. The fiduciary systems assessment considers whether Program systems provide reasonable assurance that the financing proceeds will be used for intended purposes, with due attention to the principles of economy, efficiency, effectiveness, transparency, and accountability. The Program procurement systems are assessed as to the degree to which the planning, bidding, evaluation, contract award, and contract administration arrangements and practices provide reasonable assurance that the Program will achieve intended results through its procurement processes and procedures. The financial management systems are assessed as to the degree to which the relevant planning, budgeting, accounting, internal controls, funds flow, financial reporting, and auditing arrangements provide reasonable assurance on the appropriate use of Program funds and safeguarding of its assets. The fiduciary assessment also considers how Program systems handle the risks of fraud and corruption, including by providing complaint mechanisms, and how such risks are managed and/or mitigated. 8. The environmental and social systems assessment considers, as may be applicable or relevant in a particular country, sector, or Program circumstances, to what degree the Program systems: (a) promote environmental and social sustainability in the Program design; avoid, minimize, or mitigate adverse impacts, and promote informed decision-making relating to the Program’s environmental and social impacts;  (b) avoid, minimize, or mitigate adverse impacts on natural habitats and physical cultural resources resulting from the Program;   (c) protect public and worker safety against the potential risks associated with: (i) construction and/or operations of facilities or other operational practices under the Program; (ii) exposure to toxic chemicals, hazardous wastes, and other dangerous materials under the Program; and (iii) reconstruction or rehabilitation of infrastructure located in areas prone to natural hazards;  (d) manage land acquisition and loss of access to natural resources in a way that avoids or minimizes displacement, and assist the affected people in improving, or at the minimum restoring, their livelihoods and living standards;   (e) give due consideration to the cultural appropriateness of, and equitable access to, Program benefits, giving special attention to the rights and interests of the Indigenous Peoples and to the needs or concerns of vulnerable groups; and  (f) avoid exacerbating social conflict, especially in fragile states, post-conflict areas, or areas subject to territorial disputes.  9. Activities that pose a risk of potentially significant and irreversible adverse impacts on the environment and/or affected people, and activities that involve procurement of works, goods, and services under contracts whose estimated value exceeds specified monetary amounts (high- value contracts)4 are not eligible for Program-for-Results financing, and are excluded from the Program. 10. The Program integrated risk assessment considers key risks to achieving the Program’s results and development objectives. The integrated risk assessment is informed by the results of 4 See paragraph 5 of BP 9.00. 72 the technical, fiduciary, and environmental and social systems assessments and provides a key input into the Bank’s decision to provide Program-for-Results financing. This decision takes into account country/sector/multisector-specific circumstances, potential benefits of the Program, the needs and capacity of the borrower,5 and the degree to which the Program-for-Results financing and implementation support will contribute to the overall Program objectives and results. Borrower and Bank Roles and Responsibilities in Program Preparation and Implementation 11. The borrower is responsible for preparing and implementing the Program. The Program’s scope and objectives and the borrower’s contractual obligations to the Bank are set out in the legal agreements with the Bank. These obligations include the requirement to maintain appropriate monitoring and evaluation arrangements (including credible DLI verification protocols), fiduciary and environmental and social Program systems, and governance arrangements, and to deal in a timely and effective manner with actual or alleged problems or violations (individual or systemic) in these areas. 12. The Bank assesses and appraises the Program in accordance with this OP, BP 9.00, and other applicable policy, technical, operational, and procedural requirements. In relation to the environmental and social systems assessment, the Bank consults with Program stakeholders and discloses the results and recommendations of its assessment. As appropriate, the Bank agrees with the borrower on specific actions to be taken and arrangements to be maintained during the period of the Program, including measures to address identified weaknesses and risks and to strengthen institutional capacity. 13. The Bank provides implementation support to the borrower by reviewing implementation progress, achievement of the Program results and DLIs, and associated Program risks. The Bank monitors the borrower’s compliance with its contractual obligations, including actions to strengthen institutional capacity. Recourse, Remedies, and Sanctions 14. If the borrower does not comply with its contractual obligations, the Bank consults with the borrower, and requires the borrower to take timely and appropriate corrective measures. The Bank’s legal remedies are specified in the relevant legal agreements and include the right to suspend disbursement and to cancel Program-for-Results financing. The Bank exercises such remedies when warranted and as it deems appropriate, taking into account, among other things, country-, sector-, and Program-specific circumstances, the extent of and possible harm caused by circumstances giving rise to the remedy, and the borrower’s commitment and actions to address the identified problems. 15. The borrower is responsible, among other things, for taking appropriate measures to prevent, detect, and respond to fraud and corruption or allegations of fraud and corruption in the 5 In this OP, unless the context requires otherwise, references to the “borrower� include the borrower of record and any other entities involved in Program implementation. 73 Program. The Bank has the right to investigate allegations of fraud and corruption in the Program and to sanction parties that engage in sanctionable practices. Restructuring 16. During the implementation of the Program, and as part of Bank implementation support, the Program may, with the agreement of the Bank and the borrower, be restructured to strengthen its development impact, modify its development objectives or DLIs, improve Program performance, address risks and problems that have arisen during implementation, make appropriate use of undisbursed financing, extend the financing closing date, or otherwise respond to changed circumstances. A restructuring involving a modification of the original Program objectives is referred to as a Level One restructuring and is submitted for approval by the Executive Directors under absence-of-objection procedures. A restructuring involving any other modification of the Program is referred to as a Level Two restructuring. The authority to approve Level Two restructuring is delegated by the Executive Directors to Management. Management periodically informs the Executive Directors of Level Two restructurings. Additional Program-for-Results Financing 17. The Bank may agree to provide additional Program-for-Results financing to an ongoing Program to meet: (a) unanticipated significant changes to expenditures parameters required to achieve the original Program results or DLIs; or (b) new or modified results, to be reflected in new or modified DLIs, that aim to scale up the impact or development effectiveness of the original Program. The Bank provides additional Program-for-Results financing if it is satisfied with the overall implementation of the original (or restructured) Program. The Bank considers the proposed additional Program-for-Results financing on the basis of, as necessary, updated or additional technical, fiduciary, environmental and social impacts, and integrated risk assessments. Additional Program-for-Results financing is separate and distinct from the original Program-for-Results financing and is submitted for approval by the Executive Directors. Disclosure of Information 18. During Program preparation and implementation support, the Bank discloses Program- related information in accordance with the Bank’s Access to Information Policy. *** ANNEX D. DRAFT BANK PROCEDURES (BP) STATEMENT 9.001 1. The Bank assesses a program proposed by the borrower2 for Program-for-Results Bank financing3 (hereinafter the “Program�) and, upon Program-for-Results financing approval, provides implementation support to the borrower in accordance with the requirements set forth in OP 9.00 and this BP. A. Preparation of the Program Operation 2. The preparation phase includes identification, assessment, and appraisal of the Program, and various interim processing and decision steps such as concept review, decision meeting, negotiations, Board presentation, signing, and effectiveness. Identification to Concept Review 3. At the identification stage, the Bank consults with the borrower on the borrower’s development program and goals, and seeks to identify the Program’s overall parameters, objectives, financing requirements, possible level of Program-for-Results financing, and other general information. After the Bank and borrower have reached preliminary understanding on the Program concept and parameters, Bank budget is made available for further Program preparation, and a task team is formed, led by a team leader and comprising relevant specialists. 4. The task team preliminarily, and in consultation with the borrower: (a) defines the Program and assesses its development objectives, strategic relevance, rationale, and relation to the relevant Country Assistance Strategy (CAS), Country Partnership Strategy (CPS), or similar instrument; (b) identifies the results to be achieved under the Program; (c) identifies the overall Program expenditures, nature of activities and implementation arrangements; (d) estimates the possible scope of Program-for-Results financing; 1 BP 9.00 is derived from and accompanies OP 9.00, and may be further supplemented by guidance and instructions issued from time to time by Management. 2 In this BP, unless the context requires otherwise, references to the “borrower� include the borrower of record and any other entities involved in Program implementation. 3 Program-for-Results Bank financing refers to the provision of loans, credits, or grants financed by the Bank (including IBRD and IDA) from its resources or from trust funds financed by other donors and administered by the Bank, or a combination of these (hereinafter “Program-for-Results financing�). 75 (e) considers whether the Program systems4 in the fiduciary, environmental and social, and governance5 areas are broadly adequate; (f) when the proposed Program is a part of an ongoing program, considers that program’s performance to date and, if applicable, lessons learned from similar programs in the country; and (g) considers the risks to achieving the Program’s objectives and results. 5. In accordance with paragraph 9 of OP 9.00, the task team preliminarily assesses whether the proposed Program may involve activities that: (a) pose a risk of potentially significant and irreversible adverse impacts on the environment and/or affected people; or (b) involve procurement of goods, works, and services under high-value contracts. For the purposes of this BP, “high-value contracts� means contracts with estimated values exceeding the monetary amounts, as may be amended from time to time, that require mandatory review by the Bank’s Operations Procurement Review Committee (OPRC).6 Such activities are not eligible for Program-for-Results financing, and are excluded from the Program. 6. The Bank coordinates its preparatory work with development partners and other parties that may cofinance the Program or be otherwise involved in Program-related activities. 7. After the Program concept is developed, but before further preparation takes place, the task team prepares a Program Concept Note (PCN) that describes the proposed Program, including the information under paragraphs 4 and 5 above. The task team also prepares an initial Program Information Document (PID) that summarizes the main elements of the Program and the proposed Bank financing. The PCN and the PID are submitted for the Concept Review. The Concept Review decides, among other things: (a) whether the Bank should proceed with the preparation of the Program-for-Results financing; (b) the overall parameters of the assessments to be carried out; (c) the level of subsequent Bank review(s);7 and (d) the timetable and resources for Program preparation. 8. Upon receiving authorization to continue with the preparation of the operation, the task team submits the PID for public disclosure. 4 In this BP, the term “systems� or “Program systems� refers to the systems of the institutions responsible for Program implementation, including the relevant rules, procedures, controls, risk identification and mitigation arrangements, and, when applicable, past implementation experience and record. 5  In this BP, references to the Program governance arrangements and risks refer to: (a) good governance principles (transparency, integrity, accountability and effectiveness); (b) handling the risks of fraud, corruption, coercion, collusion, and similar activities (hereinafter “fraud and corruption�), and how such risks are managed and/or mitigated; and (c) compliance with the requirements set out in the Guidelines on Preventing and Combating Fraud and Corruption in Program-for-Results Financing (________, 2011), as such guidelines may be amended from time to time.   6 See OPRC Thresholds. 7 Decision reviews may take place at either: (a) the corporate review level, through a Regional Operations Committee, or the Bank’s Operations Committee; or (b) at a subcorporate level (review by country or regional management). For more details, see the Guidelines and Procedures for Corporate Review of Operations and Country Strategies. 76 Assessment and Appraisal 9. The Bank continues to carry out the Program technical, fiduciary systems, environmental and social systems, and integrated risk assessments (described in Part III of this BP), taking into account the country-, sector-, and Program-specific circumstances, the scope for improvements before and during implementation, the risks to achieving Program objectives and results, and associated risk mitigation measures. The integrated risk assessment is an ongoing process that requires periodic consultations between the task team and Management, and between the Bank and the borrower.   10. After the assessments have been carried out and most design and assessment issues have been resolved, the task team submits for Management’s Decision Review Meeting the draft Program Appraisal Document (PAD), an updated draft PID, and the draft legal agreements.8 The Decision Review Meeting decides on, among other things, the following matters: (a) the ability to achieve Program results and the adequacy of the disbursement-linked indicators (DLIs), and their verification protocols; (b) the adequacy of the recommendations from the assessments, as well as the Program action plan (if required) for enhancing the Program systems and mitigating risks; (c) the overall Program risk assessment; (d) the exclusion from the Program of activities referred to in paragraph 5 of this BP; (e) Program-for-Results financing conditions, including conditions of appraisal, negotiations, and Board presentation, and legal conditions; (f) the proposed implementation support arrangements; and (g) whether the task team should proceed with Program appraisal or whether the Program can be considered already appraised. 11. The task team appraises the Program-for-Results financing to confirm any relevant Program- and financing-related information and resolve any outstanding legal, design, and implementation issues, and then finalizes the draft PAD and draft legal documents. The updated PID is disclosed before the Program appraisal is completed. 12. Generally, the following information is finalized following the appraisal: (a) the Program’s definition, development objective, rationale, and scope, taking into account the provisions of paragraph 9 of OP 9.00 and paragraph 5 of this BP, planned expenditures, financing requirements, and implementation and funds flow arrangements;    (b) the technical, fiduciary, environmental and social systems, and integrated risk assessments carried out and the conclusions of those assessments, and, as necessary, the relevant risk management and other actions undertaken or to be undertaken during the preparation of the Program operation, and/or during Program implementation, to enhance Program systems and performance; 8 When significant Program design or other legally relevant issues remain partially unresolved, the draft legal documents may be excluded from the documents submitted for the decision meeting. 77 (c) the main legal terms and conditions, DLI verification protocols, and/or the Program action plan, as applicable; (d) the results framework and the monitoring and evaluation arrangements, and, as appropriate, baseline references to be used in monitoring implementation and Program systems performance and assessing the development effectiveness of the Program at completion; (e) the DLIs and their credible verification protocols,9 and other disbursement-related provisions; (f) cofinancing or other collaboration arrangements with other development partners and stakeholders; (g) any proposed exceptions to or waivers from Bank policies or procedures; and (h) implementation support arrangements. 13. DLIs and Disbursement. The DLIs are specific, measurable, and verifiable indicators related to and/or derived from the Program development objectives and the results framework.10 14. Subject to the other requirements of OP 9.00 and this BP, the Bank may agree to disburse up to 25 percent of Program-for-Results financing proceeds (unless a higher percentage is approved by Management) on account of the DLIs met by the borrower between the date of the Program Concept Review and the date of the legal agreement for Program-for-Results financing. 15. To provide a borrower with resources to allow the Program to start or to facilitate the achievement of one or more DLIs, the Bank may agree to make an advance payment (following the effectiveness of the Program-for-Results financing legal agreement) of up to 25 percent of Program-for-Results financing (unless a higher percentage is approved by Management) for one or more DLIs that have not yet been met (“advance�). When the DLI(s) for which an advance has been disbursed are achieved, the amount of the advance is deducted (recovered) from the amount due to be disbursed under such DLI(s). The advance amount recovered by the Bank is then available for additional advances (“revolving advance�). The Bank requires that the borrower refund any advances (or portion of advances) if the DLIs have not been met (or have been only partially met) by the Program closing date. 16. The combined amount of financing referred to in paragraph 14 and paragraph 15 of this BP 9.00 may not exceed 30 percent of the Program-for-Results financing (unless a higher percentage is approved by Management). 9 The credibility of verification protocols is finally assessed and confirmed at appraisal, and described in the Program documents, as appropriate. Depending on the nature of the specific DLIs and the overall context of the Program and country circumstances, the verification of DLIs can be carried out in various forms and by various parties, which could include but not be limited to government agencies, semi-autonomous entities, statistical or audit entities, and third-party entities such as nongovernmental organizations and representatives of civil society. For disbursement purposes, the Bank retains the right to make the final decision whether DLIs have been achieved. 10 DLIs may be expressed as outcomes, outputs, intermediate outcomes or outputs, process indicators, or financing indicators. DLIs may also be defined as actions or process results deemed critical for strengthening performance under the Program (this could include actions for improving fiduciary, social and environmental issues and/or monitoring and evaluation), or as indicators of key institutional changes. 78 Negotiations and Board Presentation 17. Following the appraisal, the task team submits for Management approval a request, supported by the relevant documents, to negotiate the Program-for-Results financing. After negotiations have been authorized, the Bank and borrower and any other Program-related parties conduct the negotiations and seek to finalize agreement on the relevant issues and documents. If new substantive issues or significant changes in the design of the Program or Program-for- Results financing are raised during the negotiations, the task team consults with Management. If the negotiations are successfully completed, and there are no Board conditions, the task team finalizes the relevant draft Program-for-Results documents for submission to the Board. If any information in the PAD raises issues of confidentiality or sensitivity, or may adversely affect relations between the Bank and the borrower, and this information is deemed to be relevant to the Executive Directors in their decision-making process, the task team, in consultation with Management, removes this information from the PAD and incorporates it in the draft MOP. 18. After all requirements for Board presentation have been met, the Executive Directors consider the proposed Program-for-Results financing at a Board meeting, following streamlined or regular procedures.11 The final PAD is disclosed in accordance with the Bank’s Access to Information Policy. B. Program Assessments   Technical Assessment 19. The technical assessment is carried out in accordance with the provisions set out in paragraph 6 of OP 9.00 and paragraph 9 of this BP, and addresses the following matters: 20. Strategic relevance, technical soundness, and institutional arrangements. The task team assesses whether:    (a) there is a clear rationale for the Program, with the Program addressing an important development goal, and whether there is potential to significantly improve Program performance;   (b) the Program is consistent with the Bank’s overall assistance strategy for the member country in question (as expressed in the CAS, CPS, or similar instrument);   (c) the technical approach proposed by the borrower under the Program, and the Program’s structure and implementation arrangements (including governance arrangements), are adequate;   (d) the borrower’s institutional capacity is adequate; and   (e) the general legal, regulatory, and institutional environment within which the Program will operate is adequate for Program implementation.  21. Expenditure Framework. The task team assesses the level, efficiency, transparency, and effectiveness of the expenditures included in the Program. This includes consideration of 11 For more details, see the OPCS Processing Steps Guidelines and SECPO guidance detailing eSubmission of Board documents. 79 whether the planned expenditures are adequate to achieve the Program results, whether the medium-term budget is sustainable, and whether there are major discrepancies between budget allocations, releases, and actual expenditures. 22. Results Framework and DLIs. The task team assesses the Program’s: (a) results framework, which may include outcomes, outputs, intermediate outcomes or outputs, including the degree to which the Program aims to achieve these results; (b) monitoring and evaluation framework; and (c) DLIs and the related verification protocols. 23. Economic Evaluation. The task team undertakes an economic evaluation providing the rationale for the public provision of the Program, where appropriate, as well as its expected impact, describing, among other things, the expected added value of Bank support. Evaluation methods range from narrative comparisons of scenarios to quantitative methods. Fiduciary Systems and Environmental and Social Systems Assessments 24. The task team carries out the fiduciary and the environmental and social systems assessments in accordance with paragraphs 7 and 8 of OP 9.00. The assessments consider whether the Program institutions have the capacity to carry out adequate planning, decision making, execution, reporting, monitoring and evaluation, auditing and information disclosure under the Program. The assessments include: (a) a review of the existing systems; (b) identification of areas in which the implementing entities should improve procedures and performance (which may be expressed through the Program’s action plan, as necessary); and (c) inputs to the integrated risk assessment. 25. Fiduciary Systems Assessment. The assessment is carried out in accordance with paragraph 7 of OP 9.00. The task team evaluates the Program fiduciary systems to determine whether they provide reasonable assurance that the Program expenditures will be used appropriately to achieve their intended purpose. The fiduciary systems assessments consider the degree to which: (a) from a procurement perspective there is/are reasonable: (i) arrangements for planning and budgeting; (ii) procurement rules and such rules are easily accessible to the public; (iii) capacity for contract management and administration; (iv) complaint mechanisms, including clarity on how they are utilized; (v) systems for Program oversight and control; and (b) from a financial management perspective: (i) the budgeted expenditures are realistic, prepared with due regard to relevant policies, and executed in an orderly and predictable manner; (ii) reasonable records are maintained and financial reports produced and disseminated for decision-making, management, and reporting; (iii) adequate funds are available to finance the Program; (iv) there are reasonable controls over Program funds; and (v) independent audit arrangements are in place. 26. As part of the assessments, and taking into account the requirements of the Guidelines on Preventing and Combating Fraud and Corruption in Program-for-Results Financing (________, 80 2011) (ACGs), the Bank also assesses the degree to which Program systems handle the risk of fraud and corruption, including complaint mechanisms, and how such risks are managed and mitigated. The Bank and the borrower will, where required, agree on a Program-specific protocol describing the operational arrangements to handle allegations, information, and investigations related to fraud and corruption.12 27. The fiduciary systems assessment provides a reference that is used to monitor fiduciary systems performance during Program implementation and identifies actions, as needed, to enhance the fiduciary systems during Program preparation and implementation (the latter are included in the Program’s action plan). The fiduciary risks and proposed mitigation measures are inputs to the integrated risk assessment. 28. Environmental and Social Systems Assessment. The task team assesses the Program systems for managing environmental and social effects, taking into account, among other things, the capacity to plan, implement, monitor, and report on the environmental and social mitigation measures, the scope for improvements, and the risks and related mitigation measures. 29. The assessment, carried out in accordance with paragraph 8 of OP 9.00, considers, as may be applicable or relevant under particular country, sector, or Program circumstances, to what degree the Program systems: (a) operate within an adequate legal and regulatory framework to guide environmental and social impact assessments at the Program level; (b) incorporate recognized elements of environmental and social assessment good practice, including: (i) early screening of potential impacts; (ii) consideration of strategic, technical, and site alternatives (including the “no action� alternative); (iii) explicit assessment of potential induced, cumulative, and transboundary impacts; (iv) identification of measures to mitigate adverse environmental or social impacts that cannot be otherwise avoided or minimized; (v) clear articulation of institutional responsibilities and resources to support implementation of plans; and (vi) responsiveness and accountability through stakeholder consultation, timely dissemination of Program information, and responsive grievance redress measures; (c) include appropriate measures for the early identification and screening of potentially important biodiversity and cultural resource areas; (d) support and promote the protection, conservation, maintenance, and rehabilitation of natural habitats; avoid the significant conversion or degradation of critical natural habitats; and if avoiding the significant conversion of natural habitats is not technically feasible, include measures to mitigate or offset the adverse impacts of Program activities; 12 Such a protocol may be formulated in various forms, for example as a covenant in the legal agreement between the Bank and borrower, letter or memorandum of understanding, or as part of the minutes of negotiations. 81 (e) take into account potential adverse effects on physical cultural property and provide adequate measures to avoid, minimize, or mitigate such effects; (f) promote adequate community, individual, and worker safety through the safe design, construction, operation, and maintenance of physical infrastructure; or, in carrying out activities that may be dependent on such infrastructure, incorporate safety measures, inspections, or remedial works as appropriate; (g) promote the use of recognized good practice in the production, management, storage, transport, and disposal of hazardous materials generated under the Program; promote the use of integrated pest management practices to manage or reduce pests or disease vectors; and provide training for workers involved in the production, procurement, storage, transport, use, and disposal of hazardous chemicals in accordance with the relevant international guidelines and conventions; (h) include adequate measures to avoid, minimize, or mitigate community, individual, and worker risks when Program activities are located in areas prone to natural hazards such as floods, hurricanes, earthquakes, or other severe weather or climate events; (i) avoid or minimize land acquisition and related adverse impacts; identify and address economic and social impacts caused by land acquisition or loss of access to natural resources, including those affecting people lacking full legal rights to resources they use or occupy; provide compensation sufficient to purchase replacement assets of equivalent value and to meet any necessary transitional expenses, paid before taking land or restricting access; provide supplemental livelihood improvement or restoration measures if taking of land causes loss of income-generating opportunity (e.g., loss of crop production or employment); and restore or replace public infrastructure and community services that may be adversely affected by the Program; (j) undertake free, prior, and informed consultations if the Indigenous Peoples are potentially affected (positively or negatively), to determine whether there is broad community support for the Program activities; (k) ensure that the Indigenous Peoples can participate in devising opportunities to benefit from exploitation of customary resources or indigenous knowledge, the latter (indigenous knowledge) to include the consent of the Indigenous Peoples; (l) give attention to groups vulnerable to hardship or disadvantage, including as relevant the poor, the disabled, women and children, the elderly, or marginalized ethnic groups; and, if necessary, take special measures to promote equitable access to Program benefits; and  (m) consider conflict risks, including distributional equity and cultural sensitivities. 30. The environmental and social systems assessment provides a reference that is used to monitor environmental and social systems performance during Program implementation, and identifies actions, as needed, to enhance the systems during Program preparation and 82 implementation (the latter are included in the Program’s action plan). The environmental and social risks, and proposed mitigation measures, as appropriate, are inputs to the integrated risk assessment. The assessment includes a review of the arrangements by which Program activities that affect communities will be disclosed, consulted upon, and subject to a grievance redress process. Measures to address consultation, disclosure and grievance should be appropriate to the activities to be supported under the Program.   31. Before Program-for-Results appraisal, and as part of the environmental and social system assessment, the task team makes the draft assessment publicly available. The task team subsequently consults with Program stakeholders on the draft assessment. The Bank makes the final assessment publicly available. In addition, the PID, which is made publically available at the concept and appraisal stages, includes information about the environmental and social issues related to the Program. Integrated Risk Assessment 32. At different points during the preparation of the Program-for-Results financing and subsequently during Program implementation, the Bank prepares, with the borrower’s inputs, an integrated risk assessment to identify the major risks that the Program will not achieve its results. The Bank determines appropriate mitigation measures, and monitors the evolution of risks, the implementation of mitigation measures, and their impact, making adjustments as appropriate. C. Implementation Support and Program Completion 33. Borrower’s Role. The borrower is responsible for implementing the Program, monitoring its progress, evaluating results on completion, and meeting the relevant contractual obligations (including the Program action plan) set out or referred to in the legal agreements with the Bank. These responsibilities include the requirement to maintain agreed fiduciary, environmental and social, and risk management systems, and to deal in a timely and effective manner with failures (individual or systemic) in these areas. 34. Bank’s Role. In providing implementation support to the borrower in accordance with paragraph 13 of OP 9.00, the task team pays particular attention to reviewing the monitoring and verification of the Program’s results and DLIs, and the borrower’s compliance with its contractual undertakings in the fiduciary, environmental and social, and risk management areas, including those related to the Program’s action plan. The task team regularly assesses the Program, taking into account the integrated risk assessment developed during the preparation stage, uses the risk assessment to respond to and assist the borrower to respond to pre-identified and new risks, and recommends adjustments to the risk assessment, as may be appropriate. The task team undertakes field visits, liaises as necessary with relevant partners, provides appropriate support to the borrower, and reviews Program audit reports and progress reports prepared by the borrower. The task team regularly reports to Management on Program implementation by sharing the aide-mémoires that reflect review of documents, discussions with the borrower and relevant partners, and visits to Program sites and facilities. 83 35. Program Financial Statements Audits. The borrower is required to submit annual audited Program financial statements after the close of the borrower’s financial year. Audits need to be carried out by auditors with independence, experience, and capacity acceptable to the Bank, and under terms of reference acceptable to the Bank. The Bank and borrower agree on, and confirm in the legal agreement, the period for receipt of the annual audit reports. 36. Additional Reviews. Depending on the nature of the Program and the findings of the assessments, the Program’s annual financial statements audit may need to be complemented by other reviews to address certain risks or to strengthen the Program’s audit/inspection arrangements.13 37. Restructuring. If, as part of Program implementation support, the task team identifies, normally with the borrower’s input, a need to restructure the Program, the restructuring takes place in accordance with paragraph 16 of OP 9.00. 38. Additional Financing. Additional Program-for-Results financing may be provided by the Bank in accordance with paragraph 17 of OP 9.00. Approved additional Program-for-Results financing is formalized through a separate legal agreement or, if appropriate, through amendments to the original legal agreement(s). 39. Fraud and Corruption. Program-for-Results financing is subject to the ACGs. In providing Program-for-Results financing, the Bank has the right to investigate allegations of fraud and corruption in the Program and to sanction parties engaged in sanctionable practices under the Program. 40. The ACGs require the borrower to report complaints and undertake follow-up actions. When allegations of fraud and corruption under the Program are brought to the attention of the borrower or the Bank, the Bank’s Integrity Vice Presidency (INT) informs and consults with as many relevant parties as is appropriate given the nature of the allegations and the impact that such discussions would have on any related investigations. Task team members report suspected or reported cases of possible fraud and corruption to INT. Fraud and corruption investigations may be launched by INT in response to complaints received by the Bank directly or through the reporting on complaints received by the borrower. When the borrower carries out its own investigation into complaints under the Program, the findings by the responsible investigative agency and proposed actions are shared with INT and, as appropriate, other partners. The Bank monitors the progress in the borrower’s investigation.     41. When allegations of fraud and corruption are substantiated by the borrower or the Bank, INT may seek administrative sanctions under the Bank’s sanctions procedures against the individual(s) and/or entity(ies) involved, which may result in: (a) temporary suspension or early temporary suspension in connection with an ongoing sanctions proceeding: (b) debarment or other sanctions: (c) cross-debarment as agreed with other international financial institutions, including multilateral development banks: and (d) debarment by the World Bank Group’s 13 Such reviews may involve the following: procurement performance measures/indicators; physical inspection assessing the quality of goods, works and services; verification of DLIs; the efficiency and effectiveness of administrative activities; reviews of fraud and corruption “red flags� and complaint handling mechanisms. 84 General Services Department. Based on its own findings or those provided by the borrower, INT makes referrals to relevant stakeholders in accordance with its policies and practices.   42. The borrower must undertake not to allow parties on the Bank’s debarment or suspension lists to be awarded contracts under, or otherwise initiate participation in, the Program during their debarment/suspension period. 43. Remedies. The Bank’s legal remedies are specified in the relevant legal agreements. If the task team identifies breaches in the borrower’s compliance with its contractual obligations, including undertakings in the fiduciary, environmental and social, and risk management areas, including those related to the Program’s action plan, the task team informs Management and makes appropriate recommendations. Management determines whether to exercise legal remedies in accordance with paragraph 14 of OP 9.00. Normally, before exercising its remedies, the Bank provides the borrower with an opportunity to take timely and appropriate corrective measures agreed with the Bank.   44. Program Completion Report. On completion of the Program, the task team prepares an Implementation Completion and Results report (ICR). This covers, among other things, the degree to which the Program objectives and results have been achieved and the overall Program performance, including the achievement of the DLIs. The ICR incorporates the borrower’s evaluation of the Program and Program-for-Results financing, as well as of its own performance and the performance of the Bank. Program implementation support and monitoring carried out by the Bank during the implementation period end at completion of the Program. *** ANNEX E. PROPOSED ANTI-CORRUPTION GUIDELINES On Preventing and Combating Fraud and Corruption in Program-for-Results Financing Dated _________, 2011 Purpose and General Principles 1. These Guidelines address fraud and corruption that may occur in connection with the preparation and implementation of programs financed, in whole or in part, by the International Bank for Reconstruction and Development (IBRD) or the International Development Association (IDA) through Program-for-Results financing. They set out the general principles, requirements, and sanctions applicable to such programs. 2. All individuals and entities participating in programs financed through Program-for- Results financing must observe the highest standard of ethics. Specifically, all such persons and entities must take all appropriate measures to prevent and combat fraud and corruption, and refrain from engaging in fraud and corruption, in connection with such programs. Legal Considerations 3. The Loan Agreement1 providing for a Loan2 governs the legal relationships between the Borrower3 and the Bank4 with respect to the Program5 for which the Loan is made. The responsibility for the implementation of the Program under the Loan Agreement rests with the Borrower. The Bank, for its part, has a fiduciary duty under its Articles of Agreement to “make arrangements to ensure that the proceeds of any loan are used only for the purposes for which the loan was granted, with due attention to considerations of economy and efficiency and without regard to political or other non-economic influences or considerations.�6 These Guidelines constitute an important 1 References in these Guidelines to “Loan Agreement� include any Guarantee Agreement providing for a guarantee by the Member Country of an IBRD Loan; Financing Agreement providing for an IDA credit or IDA grant; agreement providing for a project preparation advance or Institutional Development Fund (IDF) grant; Trust Fund Grant Agreement or Loan Agreement providing for a recipient-executed trust fund grant or loan in cases where these Guidelines are made applicable to such agreement; and the Program Agreement with a [Project] Implementing Entity related to any of the above. 2 References to “Loan� or “Loans� include IBRD loans as well as IDA credits and grants, project preparation advances, IDF grants, and recipient-executed trust fund grants or loans for programs to which these Guidelines are made applicable under the agreement providing for such grant and/or loan. These Guidelines do not apply to investment lending (to which separate guidelines apply) or to development policy lending. 3 References in these Guidelines to “Borrower� include the recipient of an IDA credit or grant or of a trust fund grant or loan. In some cases, an IBRD loan may be made to an entity other than the Member Country. In such cases, references in these Guidelines to “Borrower� include the Member Country as Guarantor of the Loan, unless the context requires otherwise. In some cases, the project, or a part of the project, is carried out by a Program. Implementing Entity with which the Bank has entered into a Program Agreement. In such cases, references in these Guidelines to the “Borrower� include the Program Implementing Entity, as defined in the Loan Agreement. 4 References in these Guidelines to the “Bank� include both IBRD and IDA. 5 Reference in these Guidelines to the “Program� means the Program as defined in the Loan Agreement. 6 IBRD Articles of Agreement, Article III, Section 5(b); IDA Articles of Agreement, Article V, Section 1(g). 86 element of those arrangements and are made applicable to the preparation and implementation of the Program as provided in the Loan Agreement. Definitions of Practices Constituting Fraud and Corruption 4. These Guidelines address the following defined practices in connection with the Program:7 (a) A “corrupt practice� is the offering, giving, receiving, or soliciting, directly or indirectly, of anything of value to influence improperly the actions of another party.8 (b) A “fraudulent practice� is any act or omission, including a misrepresentation, that knowingly or recklessly9 misleads, or attempts to mislead, a party to obtain a financial or other benefit or to avoid an obligation. (c) A “collusive practice� is an arrangement between two or more parties designed to achieve an improper purpose, including to influence improperly the actions of another party. (d) A “coercive practice� is impairing or harming, or threatening to impair or harm, directly or indirectly, any party or the property of the party to influence improperly the actions of a party. (e) An “obstructive practice� is (i) deliberately destroying, falsifying, altering or concealing of evidence material to the investigation or making false statements to investigators in order to materially impede a Bank investigation into allegations of a corrupt, fraudulent, coercive, or collusive practice; and/or threatening, harassing or intimidating any party to prevent it from disclosing its knowledge of matters relevant to the investigation or from pursuing the investigation, or (ii) acts intended to materially impede the exercise of the Bank’s contractual rights of audit or access to information. 5. The above practices, as so defined, are sometimes referred to collectively in these Guidelines as “fraud and corruption.� Borrower Actions to Prevent and Combat Fraud and Corruption in Connection with the Program 6. In furtherance of the above-stated purpose and general principles, and except as 7 Unless otherwise specified in the Loan Agreement, whenever these terms are used in the Loan Agreement, including in the applicable General Conditions, they have the meanings set out in paragraph 4 of these Guidelines. 8 Typical examples of corrupt practice include bribery and “kickbacks.� 9 To act “knowingly or recklessly,� the fraudulent actor must either know that the information or impression being conveyed is false, or be recklessly indifferent as to whether it is true or false. Mere inaccuracy in such information or impression, committed through simple negligence, is not enough to constitute fraudulent practice. 87 otherwise agreed in writing by the Borrower and the Bank, the Borrower will: (a) take all appropriate measures to ensure that the Program is carried out in accordance with these Guidelines;  (b) take all appropriate measures to prevent fraud and corruption in connection with the Program, including (but not limited to) adopting and implementing appropriate fiduciary and administrative practices and institutional arrangements to ensure that the proceeds of the Loan are used only for the purposes for which the Loan was granted;  (c) promptly inform the Bank of all credible and material allegations or other indications of fraud and corruption in connection with the Program that come to its attention, together with the investigative and other actions that the Borrower proposes to take with respect thereto;   (d) unless otherwise agreed with the Bank with respect to a particular case, take timely and appropriate action to investigate such allegations and indications; report to the Bank on the actions taken in any such investigation, at such intervals as may be agreed between the Borrower and the Bank; and, promptly upon the completion of any such investigation, report to the Bank the findings thereof;  (e) if the Borrower or the Bank determines that any person or entity has engaged in fraud and corruption in connection with the Program, take timely and appropriate action, satisfactory to the Bank, to remedy or otherwise address the situation and prevent its recurrence;  (f) cooperate fully with representatives of the Bank in any investigation conducted by the Bank into allegations or other indications of fraud and corruption in connection with the Program, and take all appropriate measures to ensure the full cooperation of relevant persons and entities subject to the Borrower’s jurisdiction in such investigation, including, in each case, allowing the Bank to meet with relevant persons and to inspect all of their relevant accounts, records and other documents and have them audited by or on behalf of the Bank; and  (g) ensure that any person or entity debarred or suspended by the Bank is not awarded a contract under or otherwise allowed to participate10 in the Program during the period of such debarment or suspension.  10 For purposes of paragraph 6(g), participation does not include the performance under contracts entered into or other engagements began prior to the date of the Loan Agreement. 88 Sanctions and Related Actions by the Bank in Cases of Fraud and Corruption 7. In furtherance of the above-stated purpose and general principles, and except as otherwise agreed in writing by the Borrower and the Bank, the Bank will: (a) promptly inform the Borrower of all credible and material allegations or other indications of fraud and corruption in connection with the Program that come to its attention, consistent with Bank policies and procedures;  (b) have the right to conduct an investigation of such allegations or other indications, independently of or in collaboration with the Borrower;   (c) inform the Borrower of the outcome of any such investigation, consistent with Bank policies and procedures; and  (d) have the right to sanction, in accordance with its prevailing sanctions policies and procedures, any individual or entity other than the Member Country11—including (but not limited to) publicly declaring such individual or entity ineligible, either indefinitely or for a stated period of time, to (i) be awarded a Bank-financed contract; (ii) benefit from a Bank-financed contract, financially or otherwise, for example as a subcontractor; and (iii) otherwise participate in the preparation or implementation of the Program or any other project or program financed, in whole or in part, by the Bank—if at any time the Bank determines that such individual or entity has engaged in fraud and corruption in connection with the Program.12   8. The Bank also has the right to sanction individuals and entities other than the Member Country if: (a) at any time the Bank determines that such individual or entity has engaged in fraud and corruption in connection with any other Bank-financed activity; (b) another financier with which the Bank has entered into an agreement for the mutual enforcement of debarment decisions has declared such individual or entity ineligible to receive proceeds of financings made by such financier or otherwise to participate in the preparation or implementation of any project financed in whole or in part by such financier as a result of a determination by such financier that the individual or entity has engaged in fraudulent, corrupt, coercive, or collusive practices in connection with the use of the proceeds of a financing made by such financier; 11 For purposes of these Guidelines, “Member Country� includes (i) officials and employees of the national government or of any of its political or administrative subdivisions and (ii) non-autonomous government- owned enterprises. 12 The sanction may, without limitation, also include restitution of any amount of the loan with respect to which sanctionable conduct has occurred. The Bank may publish the identity of any individual or entity declared ineligible under subparagraph 7(c). 89 (c) the Bank’s General Services Department has found the individual or entity to be non- responsible on the basis of fraud and corruption in connection with World Bank Group corporate procurement pursuant to the Bank’s Vendor Eligibility Policy; or (d) at any time the Bank determines that such individual or entity has violated a material term of the terms and conditions of the Bank’s Voluntary Disclosure Program. Miscellaneous 9. The provisions of these Guidelines do not limit any other rights, remedies13 or obligations of the Bank or the Borrower under the Loan Agreement or any other document to which the Bank and the Borrower are both parties. 13 The Loan Agreement provides the Bank with certain rights and remedies that it may exercise with respect to the Loan in the event of fraud and corruption in connection with the Program, in the circumstances described therein. ANNEX F. AMENDMENTS TO WORLD BANK GENERAL CONDITIONS Part I: Amendments to the IBRD General Conditions for Loans (dated July 31, 2010) 1. Throughout the General Conditions, references to “the Project� shall be replaced with “the Program,� references to “the Project Agreement� shall be replaced with “the Program Agreement,� references to the “Project Implementing Entity� shall be replaced with “the Program Implementing Entity,� and references to “Project Report� shall be replaced with “Program Report.� 2. In the Table of Contents, the references to Sections, Section names, and Section numbers shall be modified to reflect the substantive amendments set out below. 3. Section 2.02, Special Commitment by the Bank, shall be deleted and subsequent Sections in Article II shall be renumbered accordingly. 4. In the newly numbered Section 2.02 (original Section 2.03), the heading “Applications for Withdrawal or for Special Commitment� shall be replaced with “Applications for Withdrawal,� and the phrase “or to request the Bank to enter into a Special Commitment� shall be deleted. 5. Original Section 2.04, Designated Accounts shall be deleted, and subsequent Sections in Article II shall be renumbered accordingly. 6. In Section 2.03 (original Section 2.05), Eligible Expenditures, paragraph (a) shall be amended to read: “(a) the payment is for the financing of the reasonable cost of expenditures required for the Program and to be financed out of the proceeds of the Loan in accordance with the provisions of the Legal Agreements.� 7. In Section 2.04 (original Section 2.06), Financing Taxes, the last sentence shall be amended to read: “To that end, if the Bank at any time determines that the amount of any such Tax is excessive, or that such Tax is discriminatory or otherwise unreasonable, the Bank may, by notice to the Borrower, exclude such amount or such Tax from the Eligible Expenditures to be financed out of the proceeds of the Loan specified in the Loan Agreement, as required to ensure consistency with such policy of the Bank.� 8. Section 2.06 (original Section 2.08), Reallocation, shall be amended to read: “Notwithstanding any allocation of an amount of the Loan to a withdrawal category under the Loan Agreement, the Bank may, by notice to the Borrower, reallocate any other amount of the Loan to such category if the Bank reasonably determines at any time that such reallocation is appropriate for the purposes of the Program.� 9. Section 7.01, Cancellation by the Borrower, shall be amended to read: “The Borrower may, by notice to the Bank, cancel any amount of the Unwithdrawn Loan Balance.� 91 10. In Section 7.03, Cancellation by the Bank, paragraph (d), Misprocurement, shall be deleted, and subsequent paragraphs shall be relettered accordingly. 11. Section 7.04, Amounts Subject to Special Commitment not Affected by Cancellation or Suspension by the Bank, shall be deleted, and subsequent Sections in Article VII and references to such Sections shall be renumbered accordingly. 12. In the Appendix, Definitions, the references to Section numbers shall be modified to reflect the substantive amendments set out above. In addition, paragraph 86 (“Special Commitment�) shall be deleted, and all subsequent paragraphs shall be renumbered accordingly. Part II: Amendments to the IDA General Conditions for Credits and Grants (dated July 31, 2010) 1. Throughout the General Conditions, references to “the Project� shall be replaced with references to the “Program,� references to “the Project Agreement� shall be replaced with “the Program Agreement,� references to the “Project Implementing Entity� shall be replaced with “the Program Implementing Entity,� and references to “Project Report� shall be replaced with “Program Report.� 2. In the Table of Contents, the references to Sections, Section names, and Section numbers shall be modified to reflect the substantive amendments set out below. 3. Section 2.02, Special Commitment by the Association, shall be deleted and subsequent Sections in Article II shall be renumbered accordingly. 4. In the newly numbered Section 2.02 (original Section 2.03), the heading “Applications for Withdrawal or for Special Commitment� shall be replaced with “Applications for Withdrawal,� and the phrase “or to request the Association to enter into a Special Commitment� shall be deleted. 5. Original Section 2.04, Designated Accounts, shall be deleted, and subsequent sections in Article II shall be renumbered accordingly. 6. In Section 2.03 (original Section 2.05), Eligible Expenditures, paragraph (a) shall be amended to read: “(a) the payment is for the financing of the reasonable cost of expenditures required for the Program and to be financed out of the proceeds of the Financing in accordance with the provisions of the Legal Agreements.� 7. In Section 2.04 (original Section 2.06), Financing Taxes, the last sentence shall be amended to read: “To that end, if the Association at any time determines that the amount of any such Tax is excessive, or that such Tax is discriminatory or otherwise unreasonable, the Association may, by notice to the Recipient, exclude such amount or such Tax from the Eligible Expenditures to be financed out of the proceeds of the Financing specified in the Financing Agreement, as required to ensure consistency with such policy of the Association.� 92 8. Section 2.06 (original Section 2.08), Reallocation, shall be amended to read: “Notwithstanding any allocation of an amount of the Financing to a withdrawal category under the Financing Agreement, the Association may, by notice to the Recipient, reallocate any other amount of the Financing to such category if the Association reasonably determines at any time that such reallocation is appropriate for the purposes of the Program.� 9. Section 6.01, Cancellation by the Recipient, shall be amended to read: “The Recipient may, by notice to the Association, cancel any amount of the Unwithdrawn Financing Balance.� 10. In Section 6.03, Cancellation by the Association, paragraph (d), Misprocurement, shall be deleted, and the subsequent paragraph relettered accordingly. 11. Section 6.04, Amounts Subject to Special Commitment not Affected by Cancellation or Suspension by the Association, shall be deleted, and subsequent Sections in Article VI renumbered accordingly. 12. In the Appendix, Definitions, the references to Section numbers shall be modified to reflect the substantive amendments set out above. In addition, paragraph 50 (“Special Commitment�) shall be deleted, and all subsequent paragraphs shall be renumbered accordingly. ANNEX G. ILLUSTRATIVE LIST OF POSSIBLE PROGRAM-FOR-RESULTS OPERATIONS The 6 Regions have tentatively identified 13 operations that propose to use the Program-for- Results instrument in the first 12 months after its approval. This list below is meant to be illustrative and is expected to change. These operations are diverse in terms of country income, targeted sectors, and types of program activities. About 60 percent of the financial commitments are IBRD, while the balance is IDA. The sectors represented include health (2), education (1), social and community-driven development (1), local government/decentralization (3), rural water and sanitation (1), public sector management (1), transportation (2), and rural development (1). In addition, there is a multisector, state-level operation that encompasses education, public administration, and transportation. Africa Region Ethiopia Title: Health MDG Support Operation [P123531] Sector: Health, Nutrition and Population Proposed Amount: US$200 million IBRD/IDA: IDA The objective of this operation is to assist Ethiopia in implementing its new health sector strategy (estimated at US$8.4 billion) and to achieve the health MDGs by scaling up government programs aimed at delivering and improving the quality of maternal, child and disease control interventions that will have a high impact on the health MDGs. The main component of the operation would be an MDG cash-on-delivery facility for results to reward performance of scaled-up, high-impact health interventions. Other components could include (a) a demand-based innovation fund for achieving results with the intention of encouraging effective service delivery techniques to achieve results. This component would provide financing for regions to test or carry out innovative ways of delivering services in order to achieve the targets; and (b) strengthening the data system to measure and verify performance. This component would also include evaluation of the impact of the project. Results may be measured relative to number of out-patient visits per capita, contraceptive prevalence rate, proportion of pregnant mothers who received antenatal care, proportion of births attended by skilled health personnel, TB case detection rate and the proportion of 12- to 23-month-old children immunized for Pneta. Sierra Leone Title: Public Sector Pay and Performance Project [P128208] Sector: Public Sector Governance Proposed Amount: US$17 million IBRD/IDA: IDA The project proposes to support the Government of Sierra Leone through results-based disbursement that would reward tangible reforms that address the critical problems of attraction and retention of human capital, streamlining its pay structure and improving motivation for individual staff and agency performance and provision of technical assistance. The operation would be based on a set of principles, objectives and an agreed critical path for achieving the objectives with credible and monitorable mechanisms in place. 94 Tanzania Title: Second Local Government Support Project (LGSP 2) [P118152] Sector: Local Government/Decentralization (Subnational Government Administration) Proposed Amount: US$200 million (with possible co-funding from KfW, AFD, and the Government of Finland) IBRD/IDA: IDA The proposed long-term development program objective is to establish a progressive system that enables urban local government authorities (LGAs) to deliver and maintain small-scale urban infrastructure. The proposed specific objective of the operation is to strengthen the management of public finance, procurement, capital budgeting and accountability mechanisms in all LGAs (rural and urban), and to ensure improved small-scale infrastructure in urban local governments. The higher-level goal is to increase urban economic productivity, service delivery and governance in these areas. The key results areas for the proposed project are (a) strengthened public financial management and procurement systems in LGAs; (b) establishment of a transparent performance-based urban infrastructure grant system and strengthening of own revenue systems; (c) increase in upgraded urban infrastructure and capital budgeting systems; and (d) improved number and gender distribution of direct project beneficiaries. LGSP2 would follow and build on ongoing Bank-financed projects in support of Tanzania’s urban areas. Uganda Title: Uganda Support to Municipal Infrastructure Development (USMID) Project [P117876] Sector: Urban Local Government/Decentralization (Roads and Highways; Subnational Government Administration; General Water, Sanitation and Flood Protection Sector) Proposed Amount: US$150 million IBRD/IDA: IDA The long-term development program objective is to improve municipal management and delivery of urban services. The specific objective is to pilot a small-scale municipal infrastructure financing instrument and enhance management capacity to address service gaps in 14 municipalities. The proposed operation would use the existing intergovernmental system to strengthen capacity at both central and local government for urban development and management. In addition, all pilot municipalities would benefit from some basic minimum institutional strengthening to prepare them for future investment and avoid penalizing weak municipalities. The proposed operation would have 4 components: (i) small-scale municipal infrastructure development; (ii) institutional support to the Ministry of Lands, Housing, and Urban Development; (iii) institutional support to participating municipalities; and (iv) M&E and project implementation support. The key results of the proposed Phase 1 of the operation are expected to be (a) a national municipal infrastructure financing strategy in place, (b) increase in the population living in planned urban settlements (number, %); (c) roads/drainage systems in good or fair condition (Kms, %); (d) increase in municipalities’ own source revenue (UGX, %); (e) waste collected out of total waste generated (Kgs, %), and (f) the direct and indirect project beneficiaries (numbers, %), disaggregated by gender. East Asia and Pacific Region Vietnam Title: Rural Water Supply and Sanitation Operation [P127435] Sector: Water and Sanitation Proposed Amount: US$200 million IBRD/IDA: IDA This operation builds on the ongoing IDA-financed Vietnam Red River Delta Rural Water Supply and Sanitation Project. The operation would support a slice of the ongoing government National Target Program (NTP) on rural water supply and sanitation (RWSS) by supporting 8 provinces in the Red River Delta region with sustainable water supply, sanitation and hygiene services using a results-based approach. The key features would be (a) building on the strong government-led national RWSS program within the existing national framework; (b) developing government institutional capacity, especially at the provincial level; (c) strengthening governmental M&E systems for measuring results, including poverty and health impacts; and (d) scaling up sustainable RWSS services from ongoing Bank-financed projects across the region, with results feeding into the NTP at the national level. 95 Indonesia Title: Local Government and Decentralization II [P123940] Sector: Subnational Government Administration Proposed Amount: US$200 million IBRD/IDA: IBRD The operation would be an expansion of the ongoing Bank-financed Local Government and Decentralization Project. The project focused on strengthening the fiduciary and M&E systems of the government’s special allocation (DAK) grant used by local governments for local development, including infrastructure. The proposed operation would build on lessons from the first project, which helped to strengthen the capacity of the government to conduct the verification of outputs, as well as the M&E and fiduciary systems of the government program. The new operation would help deepen the governance and accountability reforms associated with the use of the DAK grants, and would expand the program to another set of provinces and their districts. The Bank loan would reimburse DAK upon verification of outputs. The new operation would include technical assistance to support capacity enhancement of the local governments. East and Central Europe Region Romania Title: Results-Based Health Sector Reform Project [P121675] Sector: Health Proposed Amount: US$336.838 million IBRD/IDA: IBRD The proposed Program-for-Results operation would enhance the service delivery system, strengthen financing and payment methods, and improve governance in the health system. It aims to develop a coordinated response to the specific health needs of the population and to introduce incentives for higher quality services. The proposed operation would support the Government’s health sector reform efforts with results-based disbursements, directly linked to outputs achieved. Moldova Title: Education Reform Project [P127388] Sector: General Education Proposed Amount: US$30 million IBRD/IDA: IDA The proposed project would support efficiency reforms in the education sector aimed at improving quality and maintaining access to general education. The development objective would be to ensure the quality of, and maintain access to, general education while supporting the efficiency reforms being implemented in the education sector. Latin America and Caribbean Region Brazil Title: Recife - Education and Public Management [P126372] Sector: Multi Sector (Pre-primary Education, Public Administration, General Transportation) Proposed Amount: US$130 million IBRD/IDA: IBRD This proposed operation would support various programs in the municipality of Recife aiming to expand coverage of improved early childhood education and to create conditions more conducive to learning in primary education. The operation would strengthen coverage and quality of both early childhood and primary education and strengthen institutions for more efficient and effective public management. As part of support to existing government programs in early childhood and primary education, the operation would also support the provision of technical assistance to strengthen municipal public sector management in 5 areas: education, human resource policy, public management, financial systems, and transport. 96 Uruguay Title: Transport Infrastructure Program [P125803] Sector: Roads and General Transportation Proposed Amount: US$66 million IBRD/IDA: IBRD Uruguay has one of the densest paved highway networks of Latin America and fast-growing truck traffic. The main challenge of the country’s inter-urban road network is adequate maintenance. One of the core activities of the Uruguay Ministry of Transport and Public Works (MTOP) is to maintain this asset. The proposed operation would support the road maintenance and rehabilitation part of the Uruguay Road Infrastructure program, as defined in the Uruguay 2010-2014 programmatic budget law, which also includes a related results and monitoring framework. The objective of this operation would be to improve transport efficiency by supporting the rehabilitation and maintenance of the national road infrastructure and enhanced public sector capacity to plan, regulate and monitor transport and logistics services. Road maintenance and rehabilitation works are expected to improve road quality and to decrease road users’ transport costs. South Asia Region Bhutan Title: Improving Rural Livelihoods [P123820] Sector: Agriculture Proposed Amount: US$12 million IBRD/IDA: IDA The proposed operation would support the Government’s rural development programs that entail expanding the network of rural roads to improve access to markets; public and private measures to enhance agricultural productivity and strengthen the marketing of agriculture commodities; and expanded opportunities for rural livelihoods, improved technologies, market information systems, skills training and entrepreneurship. The operation would contribute to the Government’s Renewable Natural Resources sector program, the objective of which is to reduce the proportion of rural households living below the poverty line through enhanced food security. The operation would have a regional focus, including some of the most disadvantaged rural areas of Bhutan. Nepal Title: Results-Based Bridges Improvement and Maintenance Project [P125495] Sector: Transportation/Infrastructure Proposed Amount: US$60 million IBRD/IDA: IDA The proposed operation would support the maintenance, rehabilitation and in some cases new construction of bridges in Nepal’s strategic roads network. The operation would essentially address the maintenance backlog of bridges following decades of conflict. The majority of funds would be used for the maintenance and rehabilitation of existing bridges, while some of the remaining funds would be used to finance construction of short-span, potentially up to 100m, bridges on the strategic roads network. Given the poor quality of the existing stock of bridges, priority will be given to backlog maintenance and rehabilitation works and to scaling up the existing routine maintenance activities. Large bridge works would not be eligible for financing under the project. The operation is also expected to include a US$5 million institutional strengthening component which would help build Department of Roads capacity in sustainable planning, development and management of bridges on the strategic roads network. It is envisaged that the potential impacts associated with environmental and social issues will be low, as they would be localized to the vicinity of the bridges. 97 Middle East and North Africa Region Morocco Title: National Initiative for Human Development (INDH) Phase II [P116201] Sector: Social Development Proposed Amount: US$120 million IBRD/IDA: IBRD The proposed operation would build on the experiences of INDH1, a Government program aimed at alleviating poverty and social exclusion through a CDD-type approach. INDH2 is budgeted at US$2 billion over a 5-year implementation period and will be expanded to include additional communes and neighborhoods. The objective of Bank support to INDH2 would be to strengthen social and economic inclusion of the poor and most vulnerable groups by reinforcing participatory approaches and strengthening accountability mechanisms. Key results areas are likely to be: (i) improve livelihoods of INDH targeted population; (ii) strengthen local- and central-level accountability on access to and quality of public investments through consolidation of good local governance and development of performance monitoring tools, with a view to improving the quality, accessibility, and relevance of service delivery and public works and reducing leakages and corruption; (iii) reinforce participation of beneficiaries; and (iv) improve sustainability and alignment of social investments with community needs by introducing institutional mechanisms. ANNEX H: CHANGES TO BE MADE IN OPS, BPS, AND OPERATIONAL MEMORANDA 1 TO REFLECT PROGRAM-FOR-RESULTS FINANCING 1. Footnote 3 of OP 1.00 shall be modified to read: “Some other Operational Manual Statements that are closely related to the Bank’s poverty reduction mission are: BP 2.11, Country Assistance Strategies, OP 2.30, Development Cooperation and Conflict, OP/BP 8.60 Development Policy Lending, OP/BP 9.00, Program-for-Results Financing, and OP/BP 10.00 Investment Lending.� 2. Footnote 13 of OP 2.30 shall be modified to read: “See OP/BP 9.00, Program-for-Results Financing, OP/BP 13.05, Project Supervision, and OP/BP 13.40, Suspension of Disbursements. Where new lending activity in the affected area is not likely to contribute effectively to the Bank’s development objectives, the Bank may use economic and sector work to maintain a knowledge base and prepare for eventual expansion of activities. � 3. In BP 3.10, references to the “project� shall, where appropriate, be replaced with “project/program� and references to the “Project Appraisal Document� shall be replaced with “Project/Program Appraisal Document.� 4. In OP 4.20, the first sentence in footnote 1 shall be amended to read: “‘Bank’ includes IBRD and IDA; ‘loans’ includes IDA credits and IDA grants; ‘project’ includes any project or program financed under a Bank loan, IDA credit or IDA grant, but does not include programs financed under development policy loans. Social aspects of programs supported under Bank development policy loans are addressed in OP/BP 8.60, Development Policy Lending� and the term “project� in paragraph 3 shall be replaced with the term “project/program.� 5. In paragraph 2 of OP 4.76, the term “project� shall be replaced with the term “project/ program.� 6. Footnote 2 of OP 7.00 shall be modified to read: “Unless the context requires otherwise, ‘projects’ includes programs supported by Program-for-Results financing and development policy lending.� In paragraphs 12 and 13 of OP 7.00, the term “Project Agreement� shall be replaced with “Project/Program Agreement.� 7. In paragraph 1(a) of OP 7.20, the term “project� shall be replaced with the term “project/program.� 8. Footnote 1 of OP 7.30 shall be modified to read: “‘Loan’ includes IDA credits and IDA grants; ‘Bank’ includes IBRD and IDA; and ‘project’ includes a development policy lending program supported under a Bank loan or guarantee, and a program supported under Program-for- Results financing. Footnote 1 of BP 7.30 shall be modified to read: “‘Bank’ includes IBRD and IDA; ‘loans’ includes IDA credits and IDA grants; and ‘project’ includes a development policy lending program supported under a Bank loan or guarantee, and a program supported under Program-for-Results financing.� 1 New text added to existing statements is shown as underlined text. The changes to be made will be substantially in the form described in this Annex. 99 9. In paragraphs 2 and 5 of OP 7.40, the term “projects� shall be replaced with the term “projects/programs�. In paragraphs 9 and 10 of OP 7.40 and paragraph 3 of BP 7.40, the term “project� shall be replaced with the term “project/program�. In paragraph 2 of BP 7.40, the term “Project Appraisal Document� shall be replaced with “Project/Program Appraisal Document.� 10. Footnote 1 in OP 7.50 shall be modified to read: “‘Bank’ includes IBRD and IDA; ‘loans’ include IDA credits and IDA grants; and ‘project’ includes all projects and programs financed under Bank loans or IDA credits, but does not include development policy lending programs supported under Bank loans and IDA credits; and ‘borrower’ refers to the member country in whose territory the project is carried out, whether or not the country is the borrower or the guarantor.� Footnote 1 of BP 7.50 shall be modified to read: “‘Bank’ includes IBRD and IDA; ‘loans’ include IDA credits and IDA grants; and ‘projects’ includes all projects and programs financed under Bank loans or IDA credits, but does not include development policy lending programs supported under Bank loans and IDA credits; and ‘borrower’ refers to the member country in whose territory the project is carried out, whether or not the country is the borrower or the guarantor.� In OP/BP 7.50, the term “Project Information Document� shall be replaced with “Project/Program Information Document� and the term “Project Details� shall be replaced with the term “Project/Program Details�. 11. Footnote 1 in OP 7.60 and footnote 1 in BP 7.60 shall be modified to read: “‘Bank’ includes IBRD and IDA; ‘loans’ include IDA credits and IDA grants; and ‘projects’ includes all projects and programs financed under Bank loans or IDA credits, but does not include development policy lending programs supported under Bank loans and IDA credits.� In OP/BP 7.60, the term “Project Appraisal Document� shall be replaced with “Project/Program Appraisal Document� and in BP 7.60, the term “Project Information Document� shall be replaced with the term “Project/Program Information Document�. 12. Paragraph 1 of OP 8.10 shall be modified to read: “The Bank may make a Project/Program Preparation Advance (PPA) from the Project Preparation Facility (PPF) to a borrowing country to finance: a. preparatory activities for investment operations, including preliminary and detailed designs and limited initial implementation activities; and b. preparation of programs to be supported by development policy lending operations; and c. preparation of programs to be supported by Program-for-Results financing.� 13. The last sentence of paragraph 2 of OP 8.60 shall be modified to read: “Any investment lending subcomponent and any Program-for Results subcomponent included in a development policy operation is subject to the relevant operational policies for investment lending and Program-for-Results financing, respectively.� Footnote 3 of OP 8.60 shall be modified to read: “See OP/BP 10.00, Investment Lending, and OP/BP 9.00, Program-for Results Financing.� 14. The last sentence of footnote 1 of OP 10.02 and the last sentence of footnote 1 of BP 10.02 shall be modified to read: “The policy for Development Policy lending, Program-for- Results financing, and Guarantees are set out in OP 8.60, Development Policy Lending, OP 9.00, Program-for-Results Financing, and OP 14.25, Guarantees, respectively.� 100 15. In OP 11.00, a new paragraph 29 shall be added to read: “Program-for-Results Financing 29. The procurement policies applicable to Program-for-Results operations are set out in OP 9.00.� 16. In BP 11.00, a new sentence shall be added at the end of footnote 1 to read: “The procurement procedures applicable to Program-for-Results operations are set out in BP 9.00.� 17. In OP 12.00, a new paragraph 15 shall be added to read: “Disbursement under Program-for-Results financing 15. Disbursement policies related to Program-for-Results financing are set out in OP 9.00.� 18. In BP 12.00, the first sentence of footnote 2 shall be modified to read: “See OP/BP 10.00, Investment Lending: Identification to Board Presentation, for investment loans, OP/BP 9.00, Program-for-Results Financing, for program-for-results loans, and OP/BP 8.60, Development Policy Lending, for development policy loans.� 19. Footnote 2 in OP 13.05, and footnote 1 in BP 13.05 shall be modified to read: “…‘project’ includes investment lending and guarantee operations, grants from IBRD net income and IDA resources, activities financed under trust funds that cofinance projects, and GEF operations of more than US$ 1 million, and does not include operations supported by development policy lending (for information on development policy lending, see OP/BP 8.60, Development Policy Lending), and operations supported by Program-for-Results financing (see OP/BP 9.00, Program-for-Results Financing);…� 20. Footnote 1 in OP 13.16 and footnote 1 in BP 13.16 shall be modified to read: “‘Bank’ includes IBRD and IDA; ‘loans’ includes IDA credits and IDA grants; ‘borrowers’ includes guarantors and public or private subborrowers; and ‘project’ refers to the activities included in an investment operation, the program supported under Program-for-Results financing operation, and the program supported under a development policy lending operation.� 21. In footnote 1 of OP 13.20 and in footnote 1 of BP 13.20, the last sentence shall be modified to read: “The policy and procedures on additional financing do not apply to development policy lending, for which the policy and procedures are set out in OP/BP 8.60, Development Policy Lending, or to Program-for-Results financing, for which the policy and procedures are set out in OP/BP 9.00, Program-for-Results Financing.� 22. In OP 13.25, a new sentence shall be added at the end of footnote 1 to read: “Project’ means a project financed under OP/BP 10.00, and does not include programs financed under Program-for-Results financing (OP/BP 9.00).� 23. Footnote 1 in OP 13.30, and footnote 1 in BP 13.30 shall be modified to read: “…‘project’ includes activities supported by investment loans, programs supported by 101 development policy loans, programs supported by Program-for-Results financing, and activities supported by grants to which OP 13.30 applies; …� 24. The last sentence of footnote 1 of OP 13.50 and BP 13.50 shall be modified to read: “Loan cancellation is also dealt with in OP/BP 9.00, Program-for-Results Financing, OP/BP 11.00, Procurement; OP/BP 13.05, Project Supervision; OP/BP 13.30, Closing Dates; and OP/BP 13.40, Suspension of Disbursements.� Paragraph 3 a. of OP 13.50 shall be modified to read: “disbursements of that amount have been suspended for at least 30 days (see OP/BP 9.00, Program-for-Results Financing, OP/BP 13.05, Project Supervision, and also OP/BP 13.40, Suspension of Disbursements);�. 25. The last sentence of footnote 1 of OP 13.55 shall be modified to read: “This OP does not apply to programs supported by development policy loans and Program-for-Results financing, for which implementation completion reporting requirements are set out in OP/BP 8.60 and OP/BP 9.00, respectively.� 26. The first sentence of footnote 5 of OP 13.60 shall be modified to read: “See BP 2.11, Country Assistance Strategies; OP/BP 8.60, Development Policy Lending; OP/BP 9.00, Program-for-Results Financing; BP 10.00 , Investment Lending: Identification to Board Presentation; BP 13.05, Project Supervision; and BP 13.55 , Implementation Completion Reporting.� 27. Footnote 1 of OP 14.20 and footnote 1 of BP 14.20 shall be modified to read as follows: “‘Bank’ includes both IBRD and IDA; ‘loans’ includes credits and grants; ‘projects,’ ‘operations,’ and ‘investments’ include the activities or facilities financed under investment, Program-for-Results, and development policy loans; ‘Loan Agreements’ includes Development Credit Agreements; and ‘recipient’ refers to the recipient of cofinancing funds, which is usually also the Bank's borrower.� At the end of footnote 11 of OP 14.20, the following sentence shall be added: “Procurement policies and procedures applicable to Program-for-Results operations are set out in OP/BP 9.00.� Footnote 7 of BP 14.20 shall be modified to read: “See BP 9.00, Program-for-Results Financing, and OP/BP 13.55, Implementation Completion Reporting.� 28. In footnote 3 of BP 17.30, the following sentence shall be added before the last sentence: “See also BP 9.00, Program-for-Results Financing.� 29. The first sentence in footnote 7 of the March 3, 2011 Operational Memorandum (Tracking and Recovery of Cash Refunds arising from Ineligible Expenditures) shall be modified to read: “See BP 9.00, Program-for-Results Financing and BP 13.05, Project Supervision.� ANNEX I. INITIAL IMPLEMENTATION REVIEW OF PROGRAM-FOR-RESULTS OPERATIONS 1. Management proposes to rollout the Program-for-Results instrument cautiously in the early years, including a limit on overall lending during that period. Management will propose to the Board to lift the lending limit following an initial review expected after approximately after 2 years of implementation. This note outlines how such a review will be approached. 2. Implementation experience will be limited in the first two years by the proposed lending cap and the time taken for programs to be prepared, approved, and launched. While evaluation of the full impact of the first Program-for-Results programs will likely require more time, it is expected that some valuable lessons can be learned. Many of these will focus on how each operation was prepared and early implementation experience. The initial review will involve basic descriptive statistics supplemented by qualitative evaluation techniques. Among the questions to be answered are:  Are we doing things right?1 Is the instrument being implemented according to the design approved by the Board? What are the challenges faced by borrowers, development partners, and staff?  Where is the instrument being used? What countries and sectors are using the instrument? Is it working across the board or are there specific country or sectoral issues that need to be taken into account?  What is the experience with systems assessments? Are there specific issues related to the assessments—technical, fiduciary (including fraud and corruption), environmental and social, and risk? Are the assessments too intrusive? Are there gaps? What is the early experience with the action plans to enhance capacity and manage risks?  What is the early experience in measuring results? Is the results framework aligned with program objectives? What is the link between program results and DLIs? Is baseline data in place? Are credible verification protocols in place and how are they working in the early stages? Is M&E capacity adequate? What is the status of capacity building measures? Are disbursements taking place as expected? How much are Bank resources being leveraged?  What is the early experience in risk management? Have all activities governed by exclusion provisions been excluded from the supported programs? What is the assessed risk profile of the key dimensions of the risk framework?  What are the views of others? How do clients, development partners, and other stakeholders view the early implementation experience? Has the Bank’s commitment to transparency been manifest in each program? 1 Evaluations try to answer two main questions: (a) are we doing the right things and (b) are we doing things right? The first question will require more time to assess and evaluate and will be addressed in subsequent reviews. 103  What is the level of management and administration effort? What are the resource requirements of preparation and implementation support?  What are emerging early lessons? 3. The preparation of the review will adopt a number of methodologies to help answer the above questions:  Solicit feedback from countries, development partners, and staff. This will be done through a combination of a survey as well as qualitative evaluations  Establish mechanisms for data monitoring.  Conduct in-depth interviews with all stakeholders.  Explore options for cross-regional learning, including with other development partners.  Encourage an on-going dialogue. Maintain an active external website that will allow dissemination and provide an official forum for an on-going discussion.