Rwanda Country Program Evaluation F Y0 9 –17 AN INDEPENDENT EVALUATION © 2018 International Bank for This work is a product of the staff of The RIGHTS AND PERMISSIONS Reconstruction and Development / World Bank with external contributions. The material in this work is subject The World Bank The findings, interpretations, and to copyright. Because The World 1818 H Street NW conclusions expressed in this work Bank encourages dissemination Washington, DC 20433 do not necessarily reflect the views of of its knowledge, this work may be Telephone: 202-473-1000 The World Bank, its Board of Executive reproduced, in whole or in part, for Internet: www.worldbank.org Directors, or the governments they noncommercial purposes as long as full represent. attribution to this work is given. Attribution—Please cite the report as: World Bank. 2018. Rwanda Country The World Bank does not guarantee Any queries on rights and licenses, Program Evaluation, FY09–17. the accuracy of the data included in including subsidiary rights, should be Independent Evaluation Group. this work. The boundaries, colors, addressed to Washington, DC: World Bank. denominations, and other information World Bank Publications, The World shown on any map in this work do not Bank Group, 1818 H Street NW, imply any judgment on the part of The Washington, DC 20433, USA; fax: Cover Photo: Simone D. McCourtie / World Bank concerning the legal status 202-522-2625; e-mail: pubrights@ World Bank. of any territory or the endorsement or worldbank.org. acceptance of such boundaries. Rwanda Country Program Evaluation FY09–17 AN INDEPENDENT EVALUATION March 14, 2019 Careful observation and analysis of program data and the many issues impacting program efficacy reveal what works as well as what could work better. The knowledge gleaned is valuable to all who strive to ensure that World Bank goals are met and surpassed. contents abbreviations  v acknowledgments  vii overview  viii 1. Country Context and World Bank Group Strategy and Program, FY09–17  1 Country Context 2 The World Bank Group’s Strategic Objectives 3 Toward a Consolidated Framework for the Evaluation Period 5 World Bank Group Program Delivery 6 World Bank Group Support in the Broader Development Partner Context 7 2. Overall Assessment of World Bank Group Positioning and Support  9 Strategic Focus and Alignment with Country Goals 9 Appropriateness of the Program Design 11 Results and Performance of the Intellectual Leadership and Convening 12 World Bank Group 2015 14 Impact of the Division of Labor on Bank Group Strategic Positioning Program Flexibility 15 World Bank–IFC–MIGA Collaboration and Synergies 16 AN INDEPENDENT EVALUATION Engagement within the Broader Development Partner Landscape 17 Monitoring and Evaluation Focus and Contributions 18 Capacity Building Focus and Contributions 19 Gender Focus and Contributions 20 Regional Integration Focus and Contributions 21 Implementation and Results 21 Conclusion 22 3. Developments and Contributions in Areas of Strategic Focus  25 Promoting Rwanda’s Economic Transformation for Sustained Growth 25 Reducing Social Vulnerability and Improving the Productivity and Incomes of the Poor 33 Careful observation and Supporting Accountable Governance for Improved Service Delivery 44 analysis of program data World Bank Group Program Ratings under Each Pillar and the many 45 issues impacting program efficacy reveals what *To come. works as well as what could work better. The knowledge gleaned is 4. Conclusions, Lessons, and Recommendations  valuable to all who strive 50 to ensure that World Key Lessons 52 met and Bank goals are surpassed. Recommendations53 Independent Evaluation Group | World Bank Group iii Bibliography 55 Box Evolution of Fertilizer Policies in Rwanda since 2008  Box 3.1.  35 Tables Table 1.1. Evaluation Pillars for the FY09–17 Country Program Evaluation  5 Table 1.2. IDA Financing Volume by Instrument Type (percent)  6 Table 3.1. Bank Group Program Ratings  33 Appendixes* *Available in a separate volume. iv Rwanda Country Program Evaluation, FY09–17 | Contents abbreviations AS Advisory Service ASA Advisory Services and Analytics CAS Country Assistance Strategy CASPR Country Assistance Strategy Progress Report CPAF common performance assessment framework CPE Country Program Evaluation CPS Country Partnership Strategy DfID Department for International Development DIME Development Impact Evaluation DOL Division of Labor DPF development policy financing DPO development policy operation EARP Electricity Access Rollout Program EASSDP Electricity Access Scale-Up and Sector Wide Approach Development Project EDPRS Economic Development and Poverty Reduction Strategies EUCL Energy Utility Corporation GDP gross domestic product ICT information and communications technology IDA International Development Association IEG Independent Evaluation Group IFC International Finance Corporation IFMIS integrated financial management information system IMF International Monetary Fund IPF investment project financing LWH Land Husbandry, Water Harvesting and Hillside M&E monitoring and evaluation MIC middle-income country MIGA Multilateral Investment Guarantee Agency NISR National Institute of Statistics Rwanda ODA official development assistance PforR Program-for-Results Independent Evaluation Group | World Bank Group v PFM public financial management PLR Performance and Learning Review PPP public-private partnership PRSF Poverty Reduction Support Financing RDRP Rwanda Demobilization and Reintegration Program REG Rwanda Electricity Group RSSP Rural Sector Support Project SREP Scaling-Up Renewable Energy Program SWG Sector Working Groups TAS Transformation of Agriculture Sector TVET technical vocational education and training PforR All monetary amounts are in U.S. dollars unless otherwise indicated. vi Rwanda Country Program Evaluation, FY09–17 | Abbreviations acknowledgments This report was prepared by an Independent Evaluation Group (IEG) team led by Florence Charlier and comprising Yumeka Hirano, Takatoshi Kamezawa, Lourdes Pagaran, and Ali Khadr (consultant). In addition to her role as overall task team leader, Florence Charlier led the agriculture and social protection analysis. Yumeka Hirano focused on the business environment and decentralization. Taka Kamezawa led the review of the infrastructure and financial sector. Lourdes Pagaran reviewed the urban sector. Ali Khadr assessed the education and skills, health, and public financial management areas. Yumeka Hirano and Corky De Asis helped prepare the portfolio analysis. Florence Charlier, Ali Khadr, and Yumeka Hirano were responsible for preparing the overall report, and Swizen Rubbani provided key inputs. Carla F. Coles was in charge of administrative aspects, including the organization of the “deep dive” week and document management. Earlier administrative support to the team for the preparation of the approach paper was also provided by Dung Thi Kim Chu. The report was prepared under the guidance and supervision of Auguste Kouame (director) and the overall direction of Caroline Heider (Director-General, Evaluation). External peer reviewers for this evaluation were Aloysius Ordu (former director, Operations Policy and Country Services) and Monika Huppi (Inter-American Development Bank). IEG is grateful to the numerous representatives of the Rwanda government, private sector and civil society organizations, and development partners who provided valuable insights into the World Bank Group’s program in Rwanda. Finally, IEG extends its appreciation and gratitude to the management of the Bank Group institutions and country team members working in Rwanda, both currently and previously, who provided valuable time, information, and feedback to the evaluation team. Independent Evaluation Group | World Bank Group vii overview T H I S C O U N T RY PR O G R A M E VA LUAT I O N reviews the effectiveness of the World Bank Group partnership with Rwanda during FY09–17. It puts particular emphasis on the alignment of Bank Group strategic objectives with Rwanda’s aspirations to rapidly attain middle-income country (MIC) status, and the Bank Group’s contribution to achieving them. Country Context Under Vision 2020, originally prepared in 2000 to consolidate recovery from the 1994 genocide and place Rwanda on a new and ambitious development trajectory, the country aspired to attain (lower) MIC status within two decades—by transforming the economy from one based predominantly on subsistence agriculture to a knowledge-based, regional service hub. With its 2016 gross national income per capita at $700 compared with the (lower) MIC threshold of $1,006, Rwanda now aspires to higher MIC and high-income country status by 2035 and 2050, respectively. During FY09–17, Rwanda implemented two five-year Economic Development and Poverty Reduction Strategies (EDPRSs)— EDPRS (2008–12) and EDPRS 2 (2013–18)—toward the vision. Nested sector-specific strategies and district development plans—reflecting well-advanced decentralization—helped guide EDPRS implementation. A unique homegrown imihigo (performance contract) system has created a culture of results and monitoring and evaluation around the execution of plans. Rwanda has established a track record of remarkable success, especially when seen against the utter devastation emerging from the genocide. With generally sound macroeconomic management backed by steady support from the International Monetary Fund, gross domestic product growth averaged 7.4 percent annually during 2008–17, though it was strongly driven by public investment with official development assistance (ODA), which is unlikely to be sustainable. Strong growth has driven significant poverty reduction—from 57 percent to 39 percent of the population between 2006 and 2014. In parallel, viii x many nonmonetary dimensions of well-being have improved, including infant and maternal mortality, although others, such as primary school completion, lagged. Income inequality remains high. Rwanda’s remarkable progress was made against a backdrop of political stability and continuity of leadership that facilitated social cohesion and discipline. But in spite of strong performance on many cross- country governance indexes and the strong (vertical) accountability fostered by the imihigo performance culture, Rwanda’s standing in cross-country “voice and accountability” and media freedom indexes has remained low. And although reforms have rapidly boosted Rwanda’s Doing Business rankings to second- best in Africa, a complex political economy continues to permeate the business environment. Bank Group Strategy and Program Two successive Bank Group strategy documents set out support for specific parts of the EDPRSs. They were adjusted to reflect implementation experience and government requests, including its 2010 Division of Labor (DOL) exercise. The DOL, revised under EDPRS 2, designated specific sectors for specific donors. With few exceptions, the Bank Group’s focus remained broadly stable through FY09–17 and can be set out under an amalgamated framework. Under a first pillar of promoting economic transformation for sustained growth, it supported infrastructure (including energy and transport as well as—initially— information and communications technology [ICT]); the business environment (including skills development); the financial sector (including rural finance); and (in the latter years) the urban sector. Under a second pillar of reducing social vulnerability and raising the productivity and incomes of the poor, it supported agriculture, health (initially), and social protection—including the demobilization and reintegration of ex-combatants. A third accountable governance pillar was designed to strengthen central and decentralized public financial management (PFM). Under growing International Development Association allocations (and an increasing credit or grant ratio), the World Bank delivered financing commitments of some $2.06 billion for 39 operations (including regional) during FY09–17. The major divergence from plans concerned a third series of Poverty Reduction Support Financing (PRSF) operations, PRSF 8–10, which was to be delivered over FY12–14. Because of a 2012 incident in which Rwanda was thought to have supported rebel activity in the eastern Democratic Republic of Congo, budget support, especially, saw donor cutbacks. PRSF 9 and PRSF 10 were never approved because of a lack of support from shareholders, although the World Bank later built consensus for decentralization-focused development policy financing (DPF). The International Finance Corporation (IFC) committed 15 investments ($92 million in long-term financing) and did 21 Advisory Services (AS) projects. The Multilateral Investment Guarantee Agency (MIGA) issued five guarantees for $125.7 million. World Bank financing was an important but not dominant part of the tightly managed ODA. Independent Evaluation Group | World Bank Group ix Assessment of Strategic Positioning and Overall Program Strategic positioning in relation to higher-order country goals was appropriate. Bank Group objectives were explicitly aligned with those under the EDPRSs, and balanced comparative advantage and government demand. But—at least for most of the evaluation period—the objectives left out some areas that were critical for the country’s rapid progression to MIC status and beyond, such as basic education. The areas of the Bank Group’s focus were clearly critical to Rwanda’s drive for rapid attainment of MIC status. Social protection, agriculture, and energy—where Bank Group technical and financial support was sustained—were vital to the structural transformation of the Rwandan economy and to reducing social vulnerability. Other areas of Bank Group focus, from ICT to health (both early in the period) and the financial sector, PFM, and technical vocational education and training, were also critical in laying the foundations to attain MIC status. Though progress in many of these areas was qualified, Bank Group contributions were instrumental to securing the progress that was made. The Bank Group’s program design was satisfactory. With some caveats, strategies were sound, and the Country Partnership Strategy was explicit on where World Bank–IFC synergies would be pursued. Lending operations were generally of good design, and the blend and sequencing of instruments and activities, which reflected government wishes, showed strengths. In several areas, logical progression from investment project financing (IPF) to Program-for-Results (PforR) financing occurred, indicating less hands-on support as capacity matured. The balance between financing and Advisory Services and Analytics (ASA) was generally good, with analytical work informing financing in many cases. The World Bank played a commendable knowledge-generation role through its ASA, notably in its areas of focus, including agriculture and social protection. In the urban sector, ASA helped rebuild the World Bank’s knowledge after several years’ absence. ASA tasks with broad coverage included regular economic updates, poverty monitoring and analysis, and a high-profile Future Drivers of Growth study begun toward the end of the Country Program Evaluation period. In the financial sector, World Bank ASA and IFC AS represented a major part of its engagement. But more integrative analytical work could have helped draw attention to areas constituting vital foundations for MIC status where less progress was being made, such as basic education. The World Bank also played an important convening role in several areas. In energy, its leadership was instrumental in developing a sectorwide approach among development partners around government plans to expand access. The DOL, though not very restrictive, led to a concentration of International Development Association financing in certain sectors. From FY11 onward, five sectors where World Bank support was explicitly indicated under the DOL—agriculture, energy, the urban sector (after 2013), transport and ICT (before 2013), and social protection—accounted for some 85 percent of its financing outside of PRSF (7 and 8). But the World Bank supported other areas not explicitly indicated under the DOL, such as PFM, technical vocational education and training, the business environment, and the financial sector, x Rwanda Country Program Evaluation, FY09–17 | Overview typically at government request. The DOL did induce an exit by the World Bank from (general, and basic in particular) education and health. Basic education has since lagged in many respects, though this cannot be attributed to the World Bank’s exit. The World Bank program showed flexibility and agility in response to the 2012 ODA cutback (by some 8 percent of gross national income). Although it could not proceed with the originally planned PRSF 9 and 10, many stakeholders credit its decentralization-focused DPF, delivered in record time in late FY13 by consensus with key stakeholders, with having helped decisively stem cutbacks and prevent reversals in development gains. Equally agile was the response to the 2008 food price crisis, where the World Bank quickly delivered DPF that helped fund fertilizer imports. Synergies within the Bank Group were not uniformly pursued or realized across the areas indicated in the Country Partnership Strategy. But there were good practice examples of synergies in agriculture and the business environment. Synergies with other development partners have been generally strong. In the early years, the framework that guided the delivery of budget support (including PRSF) has been cited in several studies of budget support as a model of Paris Declaration principles. Constructive engagement within the broader government-donor architecture—which included a full range of Sector Working Groups—was widely apparent (if not always infallible) and helped coordinate ODA delivery. Bank Group contributions to monitoring and evaluation during the evaluation period were significant. They included support for regular Sector Working Group monitoring, for the National Institute of Statistics Rwanda, and for management information system development in several government departments. Support for impact evaluations—notably through the World Bank’s Development Impact Evaluation initiative—also featured prominently in the program. A conducive environment facilitated these contributions, despite challenges. The program also contributed to capacity development. This featured strongly in many areas of the program, including public resource management, social protection, and agriculture. Systematic, sustained approaches enabled tangible contributions. In agriculture, capacity development extended well beyond government institutions to nongovernment stakeholders, including farmers’ and water users’ associations. Gender-related objectives did not feature explicitly at the overall strategy level but were pursued in certain areas, notably agriculture and social protection. Contributions are likely to be modestly positive, although it is hard to make a conclusive assessment. Though the program made modest contributions to the regional integration agenda, there has been a pattern of major implementation challenges in World Bank projects, although underlying causes vary. Finally, World Bank project outcome ratings during the evaluation period outperformed Africa Region and World Bank–wide averages as well as the corporate target. Portfolio performance has been good. But regional operations have performed less well. IFC investments and AS projects had mixed Independent Evaluation Group (IEG) ratings. Independent Evaluation Group | World Bank Group xi Assessment of Bank Group Support in Specific Areas All areas of Bank Group support under the three pillars are assessed, albeit in greater depth for three areas—social protection, agriculture, and energy—for which the DOL consistently called for Bank Group engagement throughout FY09–17. The appendixes provide a comprehensive assessment of Bank Group support in all areas of focus. Under the first pillar of promoting economic transformation for sustained growth, the Bank Group’s support covered infrastructure (including energy and transport as well as—initially—ICT), the business environment (including skills development), the financial sector (including rural finance), and (in the latter years) the urban sector. In energy, the Bank Group’s focus during the evaluation period was on increasing generation capacity and access to electricity, which began from a very low base. In 2009, the government launched its Electricity Access Rollout Program (EARP), a geographic information system–based spatial network plan to optimize expansion of access. A few years later, targets were updated under EDPRS 2, and rural electricity access targets were subsequently defined using the Sustainable Energy for All Multi- Tier Framework. A combination of generation capacity increases and grid extension investments under the EARP has increased grid connections from 6 percent in 2009 to 33 percent at the end of January 2018. Off-grid access has more than doubled since 2016 and is estimated at 12 percent at the end of January 2018. This puts the nationwide electrification rate at 43 percent, up from 6 percent in 2009. By mid-2017, the vast majority of service delivery units had electricity connections. However, the rapid sector expansion poses fiscal sustainability risks under business as usual. Unit costs are high, in part because Rwanda gave up cheaper regional imports (given challenging regional politics) in favor of increasing domestic generation capacity, often through bilaterally negotiated independent power producer deals. Even when electricity is subsidized, tariffs averaging $0.20 per kilowatt hour make it unaffordable for many and a source of competitive disadvantage for firms. Service costs and the revenue gap are among the top 10 in Sub-Saharan Africa, and low access to off-grid electricity means a large rural-urban divide. As the lead donor in energy, the World Bank’s contributions to the progress were significant. Its active advisory and convening role helped prepare the EARP and promote a sectorwide approach among donors. A MIGA guarantee facilitated an innovative transaction to construct the world’s only industrial-scale methane power plant at Lake Kivu. The PRSF operations early in the evaluation period supported energy reforms, and the associated dialogue contributed to building government capacity, although reversal of early reforms contributed to financial problems, hampering development. Yet through follow-up projects, the World Bank pursued sector reform dialogue, which triggered the 2014 institutional reforms in the power sector and paved the way for the comprehensive and ambitious reform agenda supported through a new development policy operation series, the first of which was xii Rwanda Country Program Evaluation, FY09–17 | Overview approved in late 2017. This is the first energy development policy operation in East Africa to date and addresses head-on the medium-term fiscal sustainability risks that could materialize after 2020. World Bank provided IPF steadily throughout the evaluation period, initially helping to reduce losses and eliminate load shedding and later helping to increase electricity connections. Impact evaluation work shows improved household welfare as a result of electricity access, affordability problems notwithstanding. In other cases, notably with respect to “green” energy solutions, it is too early to assess the results of the numerous initiatives and projects undertaken with World Bank support. Technical assistance components in IPF operations helped strengthen the capacity of the key operational institutions and define energy sector strategies and plans. Stand-alone ASA also represented important World Bank contributions, complementing technical assistance and capacity development initiatives under IPF operations. It will be appropriate to persist with the recent DPF-supported efforts to rationalize the energy sector, balancing further sector expansions against cost, affordability, equity, and fiscal sustainability concerns. IEG rates the extent to which the Bank Group achieved its relevant objectives in energy as satisfactory. In transport and ICT, the Bank Group’s support for roads suffered some implementation delays. Results were otherwise positive, notably in ICT, where World Bank projects contributed to increasing ICT service use and reducing the digital divide. IEG rates the extent to which the Bank Group achieved its relevant objectives in transport and ICT as satisfactory. With respect to education and skills, in spite of some design issues, results associated with World Bank support were largely positive. IEG rates the extent to which relevant Bank Group objectives were achieved as moderately satisfactory. Regarding support for the business environment, there were substantial achievements in improving the business climate. But there were also limitations in its effectiveness, notably regarding regional integration, public-private partnerships, and special economic zones. Achievement of relevant objectives is rated moderately satisfactory. In the financial sector, using relatively modest means, the Bank Group provided sustained support for reforms to good effect. IEG rates the extent to which relevant Bank Group objectives were achieved satisfactory. In the urban sector, it is too early to assess results from the World Bank’s renewed engagement. Based on the relevance of objectives and the efficacy of analytical work, IEG rates progress toward relevant Bank Group objectives moderately satisfactory. The second pillar of reducing social vulnerability and raising the productivity and incomes of the poor covers the Bank Group’s support for agriculture, health (early on), and social protection—including demobilization and the reintegration of ex-combatants. Independent Evaluation Group | World Bank Group xiii In agriculture, the Bank Group’s focus was on sustainably increasing agricultural production and (increasingly in the latter years) on enhancing farmers’ access to financing and markets as well as on strengthening agricultural value chains and on improving nutrition. The Bank Group played a crucial role in supporting agriculture during (and before) the evaluation period. Steady Bank Group analytical work and advisory support helped identify challenges and solutions, informing policy formulation and investment decisions. Long-term programmatic IPF in the form of successive Rural Sector Support Projects helped intensify agriculture in marshlands and adjacent hillsides, with striking results. Equally favorable results were realized under another World Bank IPF operation that contributed to the intensified cultivation of hillsides, which make up much of Rwanda’s arable land. In both the marshlands and hillsides, World Bank IPF helped establish and strengthen water users’ associations. Aside from IPF, in the early years PRSF operations supported agricultural policy reforms, generally with favorable results. But, more recently, the Transformation of Agriculture Sector (TAS) PforR, now the World Bank’s principal financing vehicle, has seen more muted results. Data sources are inconclusive regarding productivity trends, private sector investment and value chain development are below target, and statistical data sources suggest food security has not improved. World Bank, IFC, and MIGA projects have also supported agricultural value chain and agribusiness development. Multisector World Bank IPF supported agriculture-related institutional development, notably in the tea and horticulture sectors. Importantly, an IFC AS project was central to the reform of green leaf tea pricing that helped increase farmer and plucker incomes. And both IFC and MIGA have supported agribusiness firms in the processing industry, with potential benefits for agricultural smallholders. IFC has supported state-of-the-art nutritious food processing that integrates backward links with farmers’ cooperatives supported by the World Bank IPF. The Bank Group’s agriculture program design had numerous strengths, and the transition to the TAS PforR operation as the main financing vehicle was a natural lending instrument progression. But a recent assessment judged its design—notably its scope, which spans the entirety of the government’s program—overambitious. Other deficiencies involved the choice and verification of disbursement-linked indicators. A notable strength of the Bank Group’s agriculture program was its careful balance between investments and capacity building of a wide range of stakeholders and institutions. The program has also increasingly had a gender focus, although little information is yet available on results. The World Bank will need a more phased approach, narrowing the scope of its second PforR operation and following other recommendations of the recent TAS PforR review, such as an increased emphasis on the private sector and strengthening data systems. In light of the more qualified results associated with the Bank Group’s support in recent years, IEG rates the extent to which the Bank Group achieved its relevant objectives in agriculture moderately satisfactory. xiv Rwanda Country Program Evaluation, FY09–17 | Overview In health, innovative and generally effective World Bank support for community health initiatives was delivered early in the period, with good results on pre- and postnatal care. Achievement of relevant objectives is rated satisfactory. In social protection, the World Bank’s focus was on Rwanda’s flagship Vision 2020 Umurenge Program (VUP), as well as demobilization and the reintegration of ex-combatants. The three-part VUP encompasses unconditional direct support to poor people unable to work; public works employing the poor at a guaranteed wage; and financial services providing low-interest-rate credit, though not exclusively to the extreme poor. The VUP is underpinned by community-led sorting of households into ubudehe categories to identify the poor. Although it was technically designated a “silent partner” in social protection, the World Bank’s sustained partnership with Rwanda (which began before the evaluation period) contributed significantly to the progress made and helped promote strong donor coordination. VUP design and rollout received substantial World Bank financial support in the form of three successive programmatic DPF series. Strong complementarity between lending and ASA support helped ensure the steady provision of technical assistance, support for institutional capacity building, and strong feedback loops in VUP rollout. Regular panel data collection and analysis as well as impact evaluation work and other ASA helped inform reforms and targets as DPF support progressed. The World Bank will need to pay particular attention to strengthening local-level capacity as VUP coverage expands, and to ensuring that regular reviews and impact evaluations guide further program extensions and targeting adjustments. Innovative extensions—for example, targeting climate change mitigation or youth—could also be considered. World Bank IPF (also supported by a multidonor trust fund) contributed to progress in demobilizing and reintegrating ex-combatants. Targets on numbers demobilized as well as their productive and harmonious reintegration into civilian life were generally met. IEG rates the extent to which the World Bank achieved its relevant objectives in social protection as satisfactory. Finally, the third pillar of promoting accountable governance focused on strengthening both central and decentralized PFM. In central-level PFM, World Bank support was generally well conceived, and results took a broadly favorable trend, though achievements often fell short of plans. Regarding decentralization, the World Bank’s objectives were largely concerned with PFM at the local level. Results were generally positive, although support could have been more sustained and coordinated for greater impact. The extent to which the Bank Group achieved its relevant objectives is rated moderately satisfactory under both subobjectives. Overall, also taking into account synergies in the Bank Group’s support within pillars, the achievement of objectives is rated satisfactory under pillars 1 and 2 and moderately satisfactory under pillar 3. Independent Evaluation Group | World Bank Group xv Overall Conclusion and Rating Although it is government commitment and leadership that has been at the heart of Rwanda’s success, IEG judges Bank Group positioning in relation to Rwanda’s Vision 2020 goals of attaining MIC status to have been appropriate. Notwithstanding the shortfalls in results under individual pillars or subpillars, IEG considers that the combination of achievement of objectives in individual areas and overall positioning and program delivery warrants an overall rating of satisfactory for the extent to which the Bank Group program met its relevant objectives during the evaluation period. IEG also considers the Bank Group’s performance in designing and implementing its Rwanda program to have been strong. Revitalizing and sustaining progress will require a paradigm shift, with dramatically increased outward orientation anchored in regional integration, a greater role for private investment, and increased domestic resource mobilization. Beginning with the joint World Bank–Government Future Drivers of Growth study, which promises to be highly influential, the Bank Group is uniquely positioned to help Rwanda manage the shift to a new growth paradigm. Key Lessons Certain general lessons emerge from the Rwanda experience: ■■  Strong government leadership and discipline have been an important determinant of development progress and of the effectiveness of the Bank Group’s support and ODA more generally and may limit the transferability of lessons from the Rwanda program to other contexts. ■■ Under strong government leadership, general budget support can be an effective means of  supporting development, particularly when blended with IPF and technical assistance to support the nuts and bolts of reform implementation. ■■ Sustained collaboration among staff is essential to realizing synergies across the Bank Group. The  experience in the agricultural sector suggests that synergies depend on close cooperation and regular interaction between World Bank and IFC staff. ■■ Systematic integration of sector experiences is key to effectively tackling cross-cutting areas.  Although the program stands out for generally good cross-sector collaboration, there is scope for more systematic integration of sector experiences in approaching the Bank Group’s support for certain cross-cutting areas such as PFM, decentralization, capacity development, and gender. ■■ Rapid agricultural modernization has the potential to widen inequality. In Rwanda, policies have  begun moving surplus labor off the farm into the nonfarm economy, including in secondary towns and in Kigali. But as noted in IEG’s 2017 evaluation Growing the Rural Nonfarm Economy, efforts are less likely to result in reduced poverty where high inequality exists, and they may even lead to marginalization of disadvantaged groups. xvi Rwanda Country Program Evaluation, FY09–17 | Overview In countries with limited institutional capacity, placing senior task team leads in the field can help build policy-making capacity and ensure smooth project implementation. The Rwanda program experience suggests that a strong field presence in sectors where the Bank Group leads helps ensure steady engagement in policy dialogue and proactive participation in Sector Working Groups. Recommendations In planning and implementing the next phase of Bank Group support, IEG recommends that the Bank Group’s Rwanda country management team consider the following: ■■ Making systematic use of Rwanda’s government-led donor architecture in determining how  the Bank Group can best respond to priorities arising from findings of the Future Drivers of Growth study. As part of this response, it would be appropriate to continue the selectivity that has characterized Bank Group engagement during the past several years, considering government demand and the comparative advantage of Bank Group institutions as well as their complementarity with other development partners. ■■ Using analytical work strategically to draw timely attention to insufficiently addressed binding  constraints or policy errors that will need to be overcome if Rwanda is to continue smoothly and sustainably on its path toward MIC and eventually high-income country status. The Future Drivers of Growth study is an important step in this direction, and the World Bank is encouraged to continue using knowledge work and policy dialogue to highlight key issues, sharing findings candidly with policy makers, stakeholders, and development practitioners. ■■ Ensuring more systematic World Bank–IFC collaboration to realize synergies—as seen, for  example, in agriculture—more broadly across the Bank Group program, notably in energy, the financial sector, and the urban sector. ■■  Ensuring that the Bank Group’s approaches to key cross-cutting subjects (for example, PFM, decentralization, or capacity development) systematically integrate sector-specific experiences. ■■ Adopting a holistic rural-urban approach to ensure inclusive and sustained territorial transformation  that does not leave the poor behind or widen inequalities. Independent Evaluation Group | World Bank Group xvii 1 T H I S C O U N T RY PR O G R A M E VA LUAT I O N (CPE) reviews the effectiveness of the World Bank Group’s partnership with Rwanda during FY09–17. The report discusses the appropriateness of the Bank Country Context and Group’s strategic focus, particularly in light of Rwanda’s World Bank Group development aspirations, including that of rapid progression Strategy and to middle-income country (MIC) status. It discusses the Program, FY09–17 progress made in the various areas of the Bank Group’s focus, as well as the likelihood that the Bank Group’s program—through its various instruments and activities, including financing, Advisory Services and Analytics (ASA), policy dialogue, and coordination with other development partners—contributed substantively to that progress. It also assesses whether the program as a whole made a tangible contribution to Rwanda’s development goals during the period. The report comprises four chapters. The rest of this chapter reviews the country context and outlines the Bank Group’s strategic objectives and program delivery within Rwanda’s broader official development assistance (ODA) picture. Chapter 2 addresses evaluation questions about the program as a whole, such as whether the Bank Group’s program was adequately tailored to Rwanda’s aspirations to rapidly attain MIC status. Chapter 3 assesses the Bank Group’s contributions in the specific areas in which it engaged, focusing in greater depth on energy, agriculture, and social protection, arguably the flagship areas of Bank Group engagement. (A more comprehensive assessment of the Bank Group’s contributions in all the areas it covered is in appendixes A–M.) Chapter 4 concludes by highlighting key lessons and recommendations. 1 Country Context Rwanda has articulated a clear vision of its development path. Under its Vision 2020 document, originally prepared in 2000 to consolidate recovery from the 1994 genocide and place Rwanda onto a new and ambitious development trajectory, the country aspired to attain (lower) MIC status within two decades—based on a transformation of the economy from one based predominantly on subsistence agriculture to a knowledge-based, regional service hub. It now aspires to higher MIC and high-income country status by 2035 and 2050, respectively. Rwanda has put in place implementation modalities to rapidly achieve its MIC status aspirations. During FY09–17, Rwanda implemented two five-year Economic Development and Poverty Reduction Strategies (EDPRSs)—EDPRS (2008–12) and EDPRS 2 (2013–18)—toward realizing the vision. Nested sector-specific strategies and district development plans—reflecting well-advanced decentralization—helped guide EDPRS implementation. The country’s unique homegrown imihigo (performance contract) system has entrenched a culture of performance, results, and monitoring and evaluation (M&E) around the execution of plans. Rwanda has established a track record of remarkable economic success, especially when viewed against the conditions of utter devastation emerging from the genocide. With generally sound macroeconomic management supported by the International Monetary Fund (IMF),1 gross domestic product (GDP) growth averaged 7.4 percent annually over the evaluation period. The government has kept inflation low, in the 2–6 percent range. Even though the level of public debt has increased in recent years, Rwanda remains rated at a low risk of debt distress.2 Outstanding economic growth has been accompanied by structural transformation. Though agriculture accounts for 75 percent of employment, the share in industry and services has doubled over the past two decades. Growth has been largely driven by public investment with ODA support. However, in recent years some private investment response can be seen to business environment reforms. Progress on poverty reduction and shared prosperity is largely attributable to the strong growth performance, but inequality remains high and prosperity is spatially concentrated. Strong growth has driven significant poverty reduction—from 57 percent to 39 percent of the population between 2006 and 2014.3 In parallel, inequality declined modestly—the Gini coefficient decreased from 0.52 in 2006 to 0.45 in 2014. The reduction in inequality accounted for more than 40 percent of the reduction in poverty (World Bank 2017e, para 10). Nevertheless, inequality remains high for Rwanda’s income level.4 Moreover, economic activity is highly concentrated in Kigali and Rubavu (a secondary city) as well as along the main transport corridors, implying spatial concentration of better living standards. Marked improvements that pushed Rwanda beyond the lower-MIC average in some areas appear in several nonmonetary indicators of welfare, although many challenges remain. It reached near- universal primary school enrollment, though net attendance in secondary schools remains low (see appendix P, table P.3). However, primary completion rates recently deteriorated, learning outcomes remain poor, and access to education is skewed in favor of better-off households. Child mortality has dropped sharply—by two-thirds—and immunization coverage has increased. But chronic malnutrition remains widespread, with persistently high rates of stunting jeopardizing learning abilities. Large-scale 2 Rwanda Country Program Evaluation, FY09–17 | Chapter 1 public investments have improved access to water sources and sanitation, road transport, electricity, and information and communications technologies (ICT), as well as housing conditions. But unpaved rural and feeder roads remain in poor condition, impeding farmers’ connections to markets, although the high cost of energy is a persistent hurdle to enterprises and to access for the bottom 40 percent. Rwanda has seen remarkable political stability and generally strong governance, but it ranks comparatively low on voice and accountability. The remarkable economic and social progress has been achieved against a backdrop of political stability and continuity of leadership that have facilitated social cohesion and discipline. But in spite of both a strong performance on many cross-country indexes of governance and the strong (vertical) accountability fostered by the imihigo performance culture, Rwanda’s standing in cross-country indexes of “voice and accountability” and media freedoms has remained low.5 Rwanda has emerged as a top Doing Business reformer, but political economy factors affect the business environment. The country has implemented a number of regulatory reforms, leading to positive developments in the business environment in recent years. In global terms, Rwanda now ranks 41 in Doing Business 2018, up from 62 in 2016 and 150 in 2008. Yet foreign direct investment levels remain low (3 percent of GDP in 2016) and the private sector’s contribution to the economy is still small. A complex political economy environment has been cited as limiting fair competition and effective implementation of regulatory reforms, with holding companies closely affiliated with the government, the ruling party, and the military playing a dominant role in the private sector.6, 7 Despite its impressive achievements, Rwanda faces multiple challenges to its quest for upper- MIC status by 2035. Structural transformation is still at an early stage. The private sector remains underdeveloped with a narrow manufacturing base, limited diversification of exports, a small financial sector, and an underdeveloped services sector. Sustaining growth and poverty reduction—from already impressive achievements—will require significant structural change in the economy. Most importantly, in the face of declining foreign aid, limited public resources, and increasing government debt, the sustainability of the public investment–driven growth model of the past two decades is in doubt. As discussed in chapter 4, a new growth paradigm based on dramatically increased outward orientation—including higher, more diversified export growth—will be needed. The World Bank Group’s Strategic Objectives Two successive strategy documents steered the Bank Group’s support to Rwanda during the CPE’s FY09–17 period. Both were explicitly aligned with the Rwanda Vision 2020 aspiration of attaining MIC status by 2020. The first, the FY09–12 Country Assistance Strategy (CAS), was prepared in 2008 and de facto extended to FY13. A CAS Progress Report (CASPR) was prepared in FY11. The second, the FY14–18 Country Partnership Strategy (CPS), remains under implementation, having been revised and extended to FY20 by an April 2017 CPS Performance and Learning Review (PLR). Both the CAS and the CPS were joint International Development Association (IDA)-International Finance Corporation (IFC)-Multilateral Investment Guarantee Agency (MIGA) strategies, with the goal Independent Evaluation Group | World Bank Group 3 of drawing public-private synergies and catalyzing private sector resources in support of Rwanda’s development. The FY09–12 CAS was anchored in Rwanda’s 2008–12 EDPRS and its goal of accelerating growth to create employment and generate exports.8 Under its first pillar of promoting economic transformation and growth, the CAS sought to achieve four outcomes: (i) increased agricultural production; (ii) improved access and quality of infrastructure services; (iii) improved environment for private sector development; and (iv) strengthened capacity to manage public resources. Under a second pillar, the CAS sought to consolidate Rwanda’s social progress by targeting selected aspects of social vulnerability. This focused on supporting efforts to help Rwanda get on track on child and maternal mortality goals and promoting peace and social cohesion through demobilization and reintegration. A CASPR adjusted CAS plans, notably to take account of a Division of Labor (DOL) among development partners prescribed by the government in July 2010. The DOL sought to limit each partner to three sectors (exempting support through certain modalities, such as budget support and “silent” partnerships, and for exceptional expenditure such as demobilization and reintegration of ex-combatants as well as regional projects and support to the private sector; see appendix T). Aside from lower transaction costs and improved aid effectiveness, the DOL sought greater evenness in donor coverage of EDPRS priorities. For the World Bank, the DOL prescribed engagement in three sectors—agriculture, energy, and transport (including ICT)—and recorded its involvement in social protection as a silent partner providing budget support. Consequently, the FY11 CASPR amended the program and CAS results framework to better align them with the DOL.9 It noted that although the World Bank would no longer finance some sectors (health and education), it would continue as a silent partner in social protection and would support cross-cutting areas such as skills and demobilization and reintegration. The FY14–18 CPS sought to support selected objectives of the second EDPRS (EDPRS 2). EDPRS 2 was intended to accelerate private sector–led growth and further reduce poverty, including extreme poverty. It set an extremely ambitious inclusive growth agenda to attain the Vision 2020 goal of lower- MIC status by that year. It focused on (i) economic transformation toward industry and services; (ii) rural development to reduce the national poverty rate below 30 percent; (iii) productivity and youth employment; and (iv) accountable governance to improve service delivery as well as citizen participation and satisfaction. In line with DOL revisions reflecting EDPRS 2 priorities, the government asked the Bank Group to remain the lead in energy but not in transport. It also requested that the Bank Group lead in urban development, where it had not been engaged under the CAS (despite some prior engagement). The Bank Group was also asked to remain engaged, but no longer lead, in agriculture, and to continue as a silent partner providing budget support in social protection. It was also asked to support public financial management (PFM) and decentralization, though these were not explicitly recorded in the revised DOL. The proposed program had three objectives: accelerating economic growth, notably through support for energy and urban development; improving the productivity and incomes of the poor through rural development and social protection; and supporting accountable governance through PFM and decentralization. 4 Rwanda Country Program Evaluation, FY09–17 | Chapter 1 The 2017 PLR proposed adjustments in the Bank Group’s approach to engagement; these would sharpen focus on the “how” of enhancing private sector participation in a range of sectors. The CPS was extended to FY20 to align the next Country Partnership Framework with the National Strategy for Transformation, the planned successor to EDPRS 2, as well as to inform the upcoming Systematic Country Diagnostic with the results of the 2018 household survey. The PLR noted a need for the Bank Group to do more to address structural reforms in the energy sector and help transform agriculture. As discussed in chapter 2, some CPS results matrix indicators were amended in an effort to ensure better measurability of outcomes and links to the Bank Group’s program. Toward a Consolidated Framework for the Evaluation Period An amalgam of CAS and CPS objectives is used to cover the FY09–17 period. This CPE uses three interlinked organizing strategic objectives or “pillars” to cover the entire evaluation period (table 1.1). The three pillars—promoting economic transformation for sustained growth, reducing social vulnerability and improving the productivity and income of the poor, and supporting accountable governance—consolidate the Bank Group’s objectives during the CAS and CPS periods (appendix O; table O.1). The first pillar encompasses four areas: (i) improving access and quality of key economic infrastructure services; (ii) improving the business environment, notably through skills development; (iii) deepening the financial sector; and (iv) developing the urban sector. Under the second pillar, the main areas are (i) promoting agriculture, (ii) reducing vulnerability to health risks, and (iii) strengthening and expanding the coverage of the social protection system (including support for demobilization and reintegration of ex-combatants). The third pillar encompasses PFM and decentralization, including citizen accountability. TABLE 1.1 | Evaluation Pillars for the FY09–17 Country Program Evaluation Pillar 1: Promoting Rwanda’s Pillar 2: Decreasing Social Vulnera- Economic Transformation for bility and Improving Productivity and Pillar 3: Supporting Accountable Sustained Growth Incomes of the Poor Governance Infrastructure – Energy, Agriculture Public Financial Management Transport, ICT Business Environment – Skills, Health Decentralization (Including Investment Climate, Public- Citizen Accountability) Private Partnerships Financial Sector – Banking, Social Protection (Including the Rural Finance, Capital Markets Vision 2020 Umurenge Program and Demobilization and Reintegration Programs) Urban Sector – Urban Planning, Housing, Urban Infrastructure Note: ICT = information and communication technologies. Independent Evaluation Group | World Bank Group 5 World Bank Group Program Delivery Bank Group program delivery largely matched plans. Under growing IDA allocations (and a progressive increase in credits relative to grants), some $935 million was committed (including regional projects) over FY09–13 (substantially more than originally foreseen in the CAS), followed by approximately $1.1 billion over FY14–17 (roughly in line with amounts foreseen in the CPS). Altogether, financing of some $2.06 billion for 39 operations was committed, adding to a substantial inherited portfolio of $380 million under 16 projects. Thematically, four Global Practices (as lead) dominated World Bank lending, accounting for some three-quarters of commitments—macroeconomics, trade and investment (reflecting general budget support that in fact supported largely sectoral reforms); social protection and labor; energy and extractives; and agriculture. ASA as well as significant trust fund financing also made up an important part of the World Bank’s program. A detailed description of the Bank Group’s operational program is provided in appendix O. However, although it did not ultimately reduce overall IDA financing provided over FY09–17, the abrupt discontinuation of general budget support after FY12 marked an important deviation from original plans. In the early part of the evaluation period (and even before it), the World Bank delivered a large share of its financing in the form of multisector Poverty Reduction Support Financing (PRSF) operations, delivered (at least) annually. After the delivery of a four-operation series (PRSF 4–7) during 2008–11, in late 2011 (that is, FY12) the World Bank approved the first (PRSF 8) of a further planned three-operation series. However, in 2012, controversy surrounding alleged Rwandan support for rebel activity in the eastern Democratic Republic of Congo led several development partners to suspend or significantly reduce their support. As discussed further in chapter 2, the World Bank was unable to muster sufficient shareholder support to continue its PRSF support, though it ultimately found ways to make the cut in development policy financing (DPF) less drastic, notably by delivering a decentralization-focused DPF operation in late FY13. Nevertheless, there has been a striking reduction in DPF in the latter part of the evaluation period (table 1.3; see appendix O, figure O.3). IFC and MIGA activity was also substantial. IFC committed approximately $92.3 million across 15 investments (long-term financing of loans and equity investments). The largest sector in net commitment is Agricultural and Forestry (50.6 percent), followed by Telecom, Media, and Technology (TMT; 17.1 percent). IFC Advisory Services (AS) comprised 21 projects in sectors such as access to TABLE 1.2 | IDA Financing Volume by Instrument Type (percent) Strategy Period Development Policy Financing Investment PforR Total FY09–13 (CAS) 62.4 37.6 0.0 100.0 FY14–17 (CPS) 29.3 48.9 21.8 100.0 Total 44.6 43.6 11.8 100.0 Source: World Bank Business Intelligence database. Note: CAS = Country Assistance Strategy; CPS = Country Partnership Strategy; PforR = Program-for-Results. 6 Rwanda Country Program Evaluation, FY09–17 | Chapter 1 finance, agriculture, sustainable business advisory, public-private partnership (PPP) in infrastructure, and investment climate reform. MIGA activity in Rwanda during the period consisted of five guarantees totaling $125.7 million. World Bank Group Support in the Broader Development Partner Context The Bank Group has been an important, though not overwhelmingly dominant, financier of Rwanda’s development. ODA was significant throughout the evaluation period and reached levels as high as 20 percent of gross national income at its peak in 2011 (appendix O, figure O.1). The importance of the 2012 cutback in ODA—some 8 percent of gross national income—is also apparent. For the World Bank, IDA disbursement as a share of ODA disbursement to Rwanda of 15.3 percent during FY09–13 and 19.5 percent during FY14–17 were clearly important contributors to overall ODA flows. Nevertheless, a drop-off in IDA disbursements can be seen in FY13–14 (appendix O, figure O. 2). ODA and the Bank Group program were delivered within a sophisticated architecture for government- donor interface. Throughout the evaluation period, a complex but mature and generally well- functioning apparatus facilitated coordinated donor support for the priorities articulated in Vision 2020 and the EDPRSs (including a series of nested sector-specific and district-level development plans). Regularly convened coordination forums have included well over a dozen Sector Working Groups (SWGs).10 These typically have both strategic- and technical-level arms, and some have thematic subgroups. They bring together not only relevant government and development partner representatives, but also other stakeholders, including civil society organizations (CSOs) and private sector representatives. Most SWGs undertake and disclose the regular monitoring of progress in implementing sector strategic plans. The institutional architecture has also included a Donor Performance Assessment Framework to underpin reviews of bilateral and multilateral donors’ performance against a set of established indicators on the quality and volume of development assistance to Rwanda.11 The framework review process was designed to strengthen mutual accountability at the country level. Early in the evaluation period, an important part of the architecture was devoted to harmonizing practices around general budget support. Under the aegis of a Budget Support Harmonization Group, the government and budget support donors established a common performance assessment framework (CPAF), based on the EDPRS Results and Policy Matrix for national-level monitoring. Elements of the CPAF were also drawn from the 2008 Joint Governance Assessment, a joint government-donor review of governance reform implementation in areas related to justice, voice and empowerment, and accountability, as well as the government’s PFM action plan. The CPAF came into use in 2009, and in the ensuing years (with updates) served as a basis for the twice-yearly formal monitoring of progress by the Budget Support Harmonization Group. Independent Evaluation Group | World Bank Group 7 1 A full list of International Monetary Fund arrangements appears in appendix P, table P.4. 2  oncessional loans still constitute the largest share of total debt at 58.1 percent, while guarantees and C nonconcessional debt of public enterprises constitute 20.8 percent. 3 t should be noted that the precise estimates of poverty, and in particular the comparability of poverty headcount I numbers across years, have reportedly been a source of contention between the government and the World Bank in the past. 4 T  he Gini coefficients in Burkina Faso (gross domestic product per capita purchasing power parity constant 2011 international: $1,536) and Madagascar ($1,371), which had income levels similar to Rwanda’s ($1,539), were 35.3 in 2014 and 0.43 in 2012, respectively. 5  he Press Freedom Index of Reporters without Borders ranks Rwanda 156 out of 180 countries for 2018. The T evolution of Rwanda’s percentile ranking on the voice and accountability dimension of the World Governance Indicators (an “index of indexes” on this dimension) is shown in the table: Country 2008 2009 2010 2011 2012 2013 2014 2015 2016 Rwanda 12.0 11.8 11.8 11.7 13.6 15.0 17.7 17. 2 14.8 Kenya 38.9 36.5 39.3 38.0 39.0 41.8 42.9 42.4 41.9 Uganda 31.3 30.8 30.8 31.0 31.9 29.6 30.0 29.1 27.1 Source: Worldwide Governance Indicators. 6  hese observations are based on interviews with stakeholders, including private sector investors, and investment T climate reports and articles such as the U.S. Department of State’s “2013 Investment Climate Statement: Rwanda” and “State-backed corporations vie to dominate Rwanda’s business landscape,” Financial Times, April 23, 2015. 7 A military-affiliated company controls fertilizer distribution in the agriculture sector (see appendix H).  8 T  he Economic Development and Poverty Reduction Strategy was framed around three flagship programs: (i) Sustainable Growth for Jobs and Exports, which supported interventions to develop skills, improve economic infrastructure, promote Science, Technology, and Innovation, and strengthen the financial sector; (ii) the Vision 2020 Umurenge Programme (VUP), which provided public works, credit packages, and direct support to create opportunities for the poorest Rwandans and foster shared growth; and (iii) Governance, to strengthen both political and economic governance and build institutions and the capacity of the state, including further decentralization. 9 I t also aligned them with a common performance assessment framework, a subset of the Economic Development and Poverty Reduction Strategy monitoring framework that had recently begun to be used as a basis for aligned and harmonized development partner provision of budget support to Rwanda. 10 F  ull details on forums as well as on aid policy and procedures can be found on the Rwanda Development Partners website (http://www.devpartners.gov.rw). 11  or instance, the Ministry of Finance and Economic Planning’s 2015 Donor Performance Assessment Framework F for FY2013/14 cites strong and predictable disbursements by the World Bank as compared with other development partners as well as progress in the World Bank’s performance in supporting its division-of-labor-designated sectors and in using country public financial management systems. 8 Rwanda Country Program Evaluation, FY09–17 | Chapter 1 2 T H I S C H A P T E R assesses the Bank Group’s strategic positioning and program delivery relative to Rwanda’s higher-order development goals. The evaluation questions it seeks to answer include the following: How well aligned Overall Assessment was the Bank Group’s strategic focus with the country’s of World Bank Group development objectives, in particular its quest to rapidly attain Positioning and MIC status? Was the Bank Group’s program design, including Support overall packaging and sequencing of activities and products, appropriate in light of its strategic objectives? How well did the Bank Group exercise intellectual leadership and convening? How did the DOL affect the Bank Group’s positioning? How did the program adapt to unanticipated events? How well integrated were IFC and MIGA into the Bank Group’s strategic positioning? How well did the Bank Group work within the broader aid architecture and donor landscape? To what extent did the program support the development of M&E and of client capacity? To what extent did the program advance gender-specific goals? Did the program focus on the regional integration agenda, and how much did it help advance it? And finally, what picture emerges of portfolio management, and how successful were Bank Group operations that exited the portfolio during the evaluation period? Strategic Focus and Alignment with Country Goals The Bank Group’s strategic objectives and areas of focus generally reflected country goals. As indicated in chapter 1, its strategic objectives over the evaluation period were explicitly tailored to (a subset of) Vision 2020 and EDPRS and EDPRS 2 9 goals, including the sector development strategies nested within them.1 Strategic objectives in the FY14–18 CPS evolved relative to those in the FY09–12 CAS to reflect the increased government focus on private sector-driven growth as well as its will to reduce dependency on ODA (especially budget support, viewed as more vulnerable to cuts) and its recognition that meeting growth and export targets would require accelerated regional integration and renewed attention to urban development. The Bank Group’s objectives also struck a reasonable balance between the institution’s comparative advantage and government wishes under the DOL, which itself reflected government views on comparative advantage. At the same time, the commendable selectivity of engagement following the DOL also meant that its objectives did not cover certain critical foundational areas, such as basic education, during much of the evaluation period. In some cases, there may have been a benefit to adjusting the definitions of objectives. For instance, it may have been preferable to more explicitly reflect cost and affordability concerns in the objective of increasing access to electricity. But such cases were relatively few and far between. In general, there is little doubt that the objectives set out in both the CAS and the CPS were highly aligned with the evolving country context and the development aspirations of Rwanda’s leadership. Adjustments to strategies during program implementation helped sustain the alignment of the Bank Group’s goals with Rwanda’s. For instance, the 2011 CASPR appropriately sought to adjust planned Bank Group interventions to reflect greater selectivity of focus—notably on agriculture, energy, and social protection—following the government’s issuance of the DOL in 2010 (see also the discussion later in this chapter). However, the government could have better planned the introduction of the DOL through ex ante consultation with donors. In addition to creating a more trustful environment, this would also have allowed thorough discussion to ensure that the DOL took full account of different partners’ comparative advantage and knowledge accumulation in areas critical to achieving Rwanda’s development objectives. The extension of the CPS through FY20 in the 2017 PLR was also appropriate—it enabled better alignment of the Bank Group’s strategy cycle with the government’s and ensured that the Systematic Country Diagnostic underpinning the next Country Partnership Framework could be based on an up-to-date household living standards analysis. Equally appropriate were the PLR’s proposals to enhance the focus on structural reforms in the energy sector, given the growing concerns regarding the fiscal sustainability of electricity subsidies, and to adopt a multisectoral approach to improving nutrition that included, notably, health, education, and social protection to complement the supply-side approach pursued within agriculture projects (chapter 3). The areas of the Bank Group’s strategic focus bore critical links to Rwanda’s drive for MIC status. Three sectors where the Bank Group provided sustained technical and financial support—energy, agriculture, and social protection—were vital to making progress in the structural transformation of the Rwandan economy and in reducing the vulnerability of its population. For example, increases in agricultural productivity are essential to releasing labor to the service sector, and expansion in access to electricity is a critical determinant of manufacturing and service growth. Beyond this, the Bank Group’s support during the evaluation period focused on several other areas—all essential prior conditions for sustained progress toward MIC status. These included PFM at the national and local levels, development of the technical vocational education and training (TVET) system, the business environment, Rwanda’s road (including rural feeder road) network, and the financial sector. Even 10 Rwanda Country Program Evaluation, FY09–17 | Chapter 2 though the World Bank was absent from health during much of the evaluation period, early on it supported some very innovative interventions to strengthen maternal and child care (appendix I). It also supported ICT in the initial years (appendix C), a particular area of concern for the government in light of the Vision 2020 aspirations of transforming the country into a knowledge-based regional service hub. As discussed in chapter 3, even though progress in many of these areas was qualified, the Bank Group’s contributions were instrumental in securing the progress made. Appropriateness of the Program Design The Bank Group’s strategies were generally sound. Logical links between strategic objectives, results sought, and proposed Bank Group–supported interventions were broadly plausible. With a few exceptions, outcome indicators used to track progress toward the objectives satisfied the SMART criteria.2 In several cases, however, the indicators initially specified in the CAS and CPS were subsequently judged not to be the best fit or information turned out not to be available to allow progress to be gauged; adjustments to the indicators were therefore needed in the CASPR and PLR.3 Only the CPS was explicit regarding where within–Bank Group synergies would be pursued. Both the CAS and CPS acknowledged that intra–Bank Group (particularly World Bank–IFC) synergies were key to crowding-in private sector investments. However, the CAS did not elaborate much on what specific sectors synergies would be pursued in, and planned IFC and MIGA contributions to CAS objectives and results were left unspecified (except for indicators relating to the investment climate for IFC). In contrast, the CPS was more explicit on areas where intra–Bank Group synergies would be pursued for maximum development impact, based on factors such as country needs, government requests (notably via the DOL), and the Bank Group’s comparative advantage. These were identified as energy, ICT, the rural sector (agriculture and infrastructure), urban development, the financial sector, and private sector development. In some areas (energy, the financial sector), the CPS results framework embodied IFC-specific targets. Considering client demand, IFC anticipated that its investments would be concentrated in the financial sector. Although the design of individual lending operations was typically of satisfactory quality, there were exceptions. Results frameworks in most financing operations were adequately constructed and captured progress toward project or program development objectives. In spite of some deficiencies, results frameworks in the PRSF series delivered during the evaluation period were largely adequate, although (from PRSG 5 onward) the need to align the choice of prior actions and outcome indicators with those found in the CPAF may have handicapped quality. For example, several prior actions lacked “depth”—in and of themselves, they did not bring about significant changes in the institutional environment.4 Other investment project financing (IPF) and Program-for-Results (PforR) operation design attributes were likewise largely adequate, although subsequent experience revealed shortcomings in some cases. For instance, a 2017 independent assessment of the Transformation of Agriculture Sector (TAS) PforR financing operation identified several design deficiencies, even abstracting from its excessively broad reach (it funded government programs in agriculture across the board). These included technical assessment gaps and results framework deficiencies (see Independent Evaluation Group | World Bank Group 11 chapter 3 and appendix H). Arguably, such setbacks were all but inevitable as the World Bank put a new lending instrument to use. The blend and sequencing of Bank Group instruments and activities showed considerable strengths and reflected government wishes. As discussed further in the Intellectual Leadership and Convening section, there was generally a good balance between lending and ASA activities, including technical assistance and capacity building support, and intra–Bank Group complementarity was capitalized on in several areas. Regarding the use of financing instruments (if one abstracts from the multisector PRSF financing operations that were used early in the period but discontinued beginning in 2012), there has been a logical shift toward less hands-on, more sectorwide policy- and results-focused forms of financial support as sectors have developed and the associated institutional capacity has matured. In several areas of focus—skills, agriculture, and PFM—PforR financing has supplanted IPF. In energy, DPF has recently succeeded IPF as structural reforms have come to the fore, although in social protection the opposite has occurred. However, the blend and sequencing of World Bank financing instruments have also been (appropriately) driven by the wishes of the government. Since the 2012 incident, it has viewed DPF as vulnerable to external perceptions of the country and has sought to reduce its share in the World Bank’s program: compared with 60 percent in 2012, the share is now expected at about 40 percent. Intellectual Leadership and Convening The Bank Group’s intellectual leadership was apparent in several areas, though ASA was typically guided by SWGs. In several areas where the World Bank provided steady financial support through the evaluation period, it also had a strong, solid stream of analytical work, although the government and other donors exercised a strong influence on the overall work done and on the specific parts allocated to the World Bank. The Bank Group’s involvement in agriculture, for example, saw a steady production of analytics, some of which provided essential foundations for lending (chapter 3 and appendix H). In agriculture and rural development more broadly (including rural transport and finance), work under the World Bank’s Development Impact Evaluation (DIME) initiative provided valuable analysis regarding the impact of Bank Group–supported interventions. An IFC-supported study on green leaf tea pricing was instrumental in preparing a reform of the pricing formula, to good effect (see chapter 3). Similarly, in social protection, regular Vision 2020 Umurenge Program (VUP) household surveys and impact evaluations as well as broader analytical work helped in scaling up the program and fine-tuning policies. In some sectors, World Bank ASA was less consistently visible, but knowledge work was done by other means. In energy, there were few stand-alone analytics, but World Bank lending adequately provided for the technical work needed and advisory services and nonlending technical assistance addressed specific issues that arose. The World Bank also provided essential intellectual input into the government’s 2009 geospatial Electricity Access Rollout Program (EARP), as well as into sector planning more generally. In PFM, there was intensive ASA in the initial part of the evaluation period (much of which was broadly disseminated, at least among government counterparts); however, 12 Rwanda Country Program Evaluation, FY09–17 | Chapter 2 there was less in the latter part (although key analytics such as a Public Expenditure and Financial Accountability assessment were provided for under the work programs decided on in the SWG, as well as under the government’s PFM strategy, which the World Bank played a major role in funding). Similarly, regarding education and TVET, ASA was concentrated early in the evaluation period. As with energy and PFM, analytical work in later years was provided for in the government’s work program, which the World Bank supported through its projects. Other ASA tasks, including the recent high-profile drivers of growth study, had broad coverage. Regular economic updates, each of which featured a special topic, were prepared and widely disseminated throughout the period. A poverty assessment was prepared in FY15 and subsequently published by the National Institute of Statistics Rwanda (NISR). Toward the end of the evaluation period, during a visit by Bank Group President Jim Kim to Rwanda, President Kagame made a request for the World Bank’s analytical support in the preparation of Rwanda’s next long-term development strategy. The World Bank began working (jointly with the government) on the integrative Future Drivers of Growth study, which promises to be influential in shaping future policy directions. In the urban sector, World Bank analytical work was highly relevant and generally of high quality but was completed long after the government had issued its urban policy and strategy. The World Bank’s absence from the urban sector since 2009 had left knowledge and capacity gaps. After the World Bank began leading the urban policy dialogue under the FY14–18 CPS, four notes were prepared to inform the government on the role of urbanization in economic development, job creation, and poverty reduction.5 They provided the basis for a synthesis note (World Bank 2017). A 2015 Employment and Jobs Study also examined the potential role of the six secondary cities in job creation and poverty reduction. However, these ASA tasks, completed between 2015 and 2017, came too late to effectively inform and influence the government’s urbanization policies and strategies, which were issued in 2015 and 2016.6 Nevertheless, the ASA provided the avenue for dialogue with the government and, together with the ongoing discussion on urbanization in the context of the Future Drivers of Growth study, could effectively influence the government’s urbanization approach over time. In other cases, ASA was a key part of the Bank Group’s engagement and leveraged results. In the financial sector, a major part of World Bank engagement has been through analytical work together with intensive (often trust fund–supported) nonlending technical assistance. The FY11 World Bank– IMF FSAP update underpinned the second phase of Rwanda’s financial sector development program (as the original 2005 FSAP had the first phase) and continuing technical support under the Financial Sector Reform and Strengthening Initiative helped prepare and implement it. In other words, although financing (notably the PRSF series)7 played a key role, a major part of the World Bank’s support to the financial sector during the evaluation period took the form of ASA. Together with IFC ASs to and investments in specific institutions, this substantial body of work made a key contribution to the rapid development of the financial sector observed during the period. More integrative analytical work could, however, have made Bank Group support for the attainment of Vision 2020 goals more effective. More regular integrative work of the core diagnostic variety, such as Independent Evaluation Group | World Bank Group 13 Country Economic Memorandums, could have been of significant benefit. In Rwanda and elsewhere, the World Bank is one of very few development partners with a clear comparative advantage in producing high-quality integrative analytics. Such work could have helped take more regular stock of the progress that Rwanda was making toward its MIC aspirations. In particular, it could have helped identify factors liable to restrain progress—such as persistent challenges in basic education learning outcomes—and helped define priorities and trade-offs to address development constraints. The World Bank could also have used analytical work to draw more systematic attention to some of the risks of Rwanda’s governance system; for example, top-down accountability could be complemented with more flexible approaches in which CSOs could have a greater say in formulating performance standards and providing feedback, and public institutions could be given more space to take risks and pilot local initiatives. Similarly, regarding the business climate, the World Bank could have more systematically underscored the importance of a level playing field with genuine competition for private firms in relation to state-owned and -connected enterprises so that the regulatory framework could offer greater incentives for investment. Such work would also have helped cement the World Bank’s role as a trusted development policy adviser to Rwanda. Yet, following the 2007 Country Economic Memorandum, truly integrative analytical work was largely absent from World Bank ASA during the evaluation period, although the regular economic updates (each of which had a special topic) had value. The joint World Bank–government Future Drivers of Growth study and future Systematic Country Diagnostic, aligned with the Country Partnership Framework cycle, should begin to address the longtime gap in integrative work. The World Bank also played an important convening role. In energy, where the World Bank cochaired the SWG, its convening role was particularly instrumental in rallying development partners around a sectorwide approach (SWAp), in contrast to the fragmented approach before the evaluation period. It also led in policy dialogue. These factors contributed to the rapid development of the sector, as discussed in chapter 3. In agriculture, the World Bank played a particularly important role in convening and in policy dialogue up to 2013, when it served as SWG cochair, and it has continued to play an important role since. Impact of the Division of Labor on Bank Group Strategic Positioning The DOL has clearly shaped the thematic composition of World Bank lending deliveries. World Bank lending in areas that have been formally recorded under the DOL—agriculture, energy, transport and ICT, the urban sector, and social protection—has accounted for some 85 percent of (nonregional) lending outside of general budget support (PRSF operations) to Rwanda since FY11 (appendix O). The DOL induced an exit from (general) education and health. Two areas where the World Bank visibly disengaged after the 2010 DOL were general (including basic) education (that is, other than TVET) and health. In education, the troubled implementation of a human development IPF operation that closed soon after the start of the CPE period may have influenced Rwanda’s decision to ask the World Bank not to continue in the sector. However, World Bank support for health has recently resumed in the guise of nutrition and early childhood development. Even in education, recent 14 Rwanda Country Program Evaluation, FY09–17 | Chapter 2 World Bank regional ASA (on basic education) and IPF (on developing higher education centers of excellence) have included Rwanda. Elsewhere, the DOL’s restrictive effects on World Bank engagement appear to have been modest. The World Bank has been engaged in many other areas—including PFM, decentralization, private sector development, the financial sector, and skills development (for example, through TVET)— though several of these were at government request, even if not formally recorded in the DOL. And despite its silent partner designation in social protection, the World Bank’s dominant technical and advisory role has belied the label of passive financier. Most recently, after a decade or more of continuous support through DPF (which the DOL pointedly exempts from needing to support a donor’s “assigned” sectors), its support has shifted to IPF. Although the selectivity of World Bank engagement that the DOL helped bring about was positive, the World Bank arguably should have maintained a base level of knowledge in basic education, even after its exit. Early in the evaluation period, the World Bank provided some support for basic education—including administration of an Education for All Fast-Track Initiative Catalytic Bridge Grant and analytical work (see appendix D). The FY09–12 CAS also had a related outcome indicator—the primary school student-to-qualified-teacher ratio—though the overall CAS-level objective was cast as improving the environment for private sector development. Although the student-teacher ratio improved through 2011, it has since deteriorated. Other indicators, such as the primary school completion rate—which stood at 60.4 percent in 2015 compared with 78.6 percent in 2011—also deteriorated. Just as importantly, recent World Bank work points to very poor learning outcomes in Rwanda’s basic education.8 It would be pretentious, not to mention technically impossible, to attribute poorer outcomes in education to the World Bank’s exit. Yet good basic education (among other factors, such as nutrition) is a fundamental driver of the quality of a country’s human capital, and consequently of long-term competitiveness and growth. In addition, it is a major determinant of the efficacy of Bank Group support for TVET, which is immediately downstream. Maintaining a knowledge base would have enabled the World Bank to signal to the government that deteriorating trends in basic education, particularly for a small landlocked country lacking natural resources, was apt to affect progress toward the country’s goal of MIC status. The World Bank is now having this dialogue through the Future Drivers of Growth study. Program Flexibility The World Bank demonstrated the flexibility to respond to unanticipated events during program rollout. In particular, the World Bank showed agility and resourcefulness in its response to the 2012 ODA cutback following the allegations of Rwandan support for rebel activity in the eastern Democratic Republic of Congo. After the event, donor budget support especially saw severe cuts, and the World Bank was unable to muster sufficient support from key shareholders for its planned PRSF 9 operation. Rwanda’s budget suffered major cuts, leading to widespread concerns that the rapid development gains of preceding years would be reversed. The World Bank took the initiative to consult intensively with key donors and shareholders and was eventually able to garner consensus Independent Evaluation Group | World Bank Group 15 to restore DPF in support of decentralization. The May 2013 operation—prepared in record time— opened the way for a broader resumption of donor support, and stakeholders widely credit the World Bank for its leadership in cushioning the impact of the 2012 incident (appendix M). It is unclear whether the permanent discontinuation of PRSF support has had costs, though it is improbable that the World Bank would have had sufficient support to resume it without at least several years’ interruption. The World Bank also showed agility in its response to the 2008 food price crisis when it mobilized resources from the Food Price Crisis Response Trust Fund (delivered as DPF). The resources were used to boost fertilizer imports that helped to increase crop yields (see chapter 3). World Bank–IFC–MIGA Collaboration and Synergies Intra–Bank Group synergies were not uniformly visible across the program. Coordinated support capitalizing on the strengths of the World Bank, IFC, and MIGA and their complementarity was apparent in some areas, though not systematically across areas of involvement, including those targeted for synergies in the CPS. In energy, transport, the financial sector, and the urban sector, there was scope for more coordinated World Bank–IFC planning and implementation for sector development from both public and private sector perspectives. In energy, early World Bank–IFC collaboration on a methane gas generation project did not pan out, but MIGA later issued a guarantee under a new project sponsor (see chapter 3 and appendix A). In transport, there was only one IFC investment to support the privatization of a logistics company. In the financial sector, IFC met its financial commitment volume targets under the CPS (although these were predominantly short- term trade finance commitments, not IFC’s core product of long-term financing). It also issued a local currency bond, albeit with limited impact thus far on capital markets development (appendix F), with no subsequent corporate bond issuance in the market. However, the extent to which IFC’s financial sector program has been informed by the World Bank’s experience in the sector appears limited. In the urban housing sector, interviews suggest some divergence of ideas between the World Bank and IFC on how best to respond to government requests on affordable financing mechanisms. Good practice examples of synergies are found in agriculture, an area identified in the CPS. Agriculture offers several examples of synergies (chapter 3). Early on, IFC investments and MIGA guarantees reinforced one another in increasing local value-addition in grain milling. More recently, IFC– World Bank synergies have been apparent in TAS PforR implementation, where through AS projects IFC is contributing to advancing the reform agenda in areas ranging from seed and pesticide laws to leasing and land use management. Synergies have also been strikingly apparent in IFC’s AS project in support of an innovative multistakeholder PPP that seeks to link a high-nutrition food processing plant with farmer cooperatives in districts covered by a World Bank project (for maize and soy supply); IFC is also supporting the initiative with short-term funding and risk mitigation to a local bank to enable the cooperatives to access affordable financing. Finally, IFC’s financing of a pricing study for the tea industry complemented efforts under World Bank IPF to promote a market-based tea industry. Intra–Bank Group synergies have also been evident in the strengthening of Rwanda’s business environment. Regarding the business environment, where IFC has been asked to lead, a series of 16 Rwanda Country Program Evaluation, FY09–17 | Chapter 2 IFC AS Investment Climate Reform projects have built on and complemented initiatives supported under World Bank projects before and early in the evaluation period (appendix E). Broadly speaking, a DOL catering to the respective strengths of Bank Group institutions has been discernible. In particular, World Bank diagnostic ASA and IPF support for preparing laws and strengthening regulatory and investment promotion institutions has dovetailed with IFC support for operationalizing laws and regulations, improving public-private dialogue and services to private business, and with catalytic IFC investments and specific ASs to support private ventures. Particularly notable was IFC’s advisory role in securing a PPP transaction in 2015—the country’s first and the first competitively tendered Build, Operate, and Transfer water concession in Sub-Saharan Africa (outside of South Africa). Engagement within the Broader Development Partner Landscape The framework around the World Bank’s PRSF operations was widely considered a model of “partnership” budget support. Early in the evaluation period, PRSF operations were delivered (at least) annually and dominated lending. Their coverage was very wide, extending from agriculture, energy, and other infrastructure areas to the financial sector and public resource management. This general budget support was delivered in tandem with a broader mechanism that represented one of the best examples of Paris Declaration principles at work. These included country leadership of the development agenda, with donors contributing to the partnership through regular and predictable untied financing, but also the associated conditionality, policy dialogue, technical assistance and capacity building, and harmonization and alignment with Rwanda’s own systems and agenda. The CPAF guided M&E of progress in implementing the EDPRS. Despite the obvious costs of keeping such a heavy apparatus going, few have suggested that these exceeded the benefits, imperfections notwithstanding (for example, the policy dialogue around M&E was arguably too process focused). On balance, the aid delivery model appeared to function well until the end of general budget support in 2012. It would, of course, be rash to attribute Rwanda’s rapid progress toward reducing extreme poverty and increasing shared prosperity during the time this aid delivery model was used to the model itself. Nevertheless, it likely facilitated the progress. General budget support aside, constructive engagement within the government-development partner architecture was visible in several areas during the evaluation period, helping to improve the impact of Bank Group support and ODA more broadly. The elaborate and generally well-functioning structure of government-donor-third party forums (including the many SWGs) has facilitated good coordination and alignment in several other areas. The World Bank has capitalized on the architecture with active convening and participation, contributing to the coordination among development partners. In energy, the World Bank’s lead role in developing and rallying donors around the SWAp stands out. In agriculture, the collaboration with other development partners, particularly the United Kingdom’s Department for International Development (DfID), for its TAS PforR, likewise stands out as good practice. In social protection, regular and fruitful dialogue through the social protection SWG was an essential part of an overall World Bank package of support that has included DPF (some of it cofinanced),9 technical assistance, and analytical work. Similar examples of collaboration through the Independent Evaluation Group | World Bank Group 17 SWG structures can be seen in skills development (through TVET) and PFM. Although the latter two areas have not been entirely criticism-free and improvements could be made, it is clear that the World Bank has put significant effort into assuring that SWGs enable substantive policy dialogue and that its support is adequately coordinated with that of others. World Bank–IMF collaboration also appears to be strong, especially in the energy sector. Monitoring and Evaluation Focus and Contributions Rwanda has offered an environment conducive to M&E and evidence-based decision-making. Several features of the country’s institutional environment encouraged a culture that supports M&E. Such features include the regular, in-depth joint monitoring of progress in sector strategy implementation that many of the SWGs undertake. Early in the period (before the end of general budget support), regular CPAF monitoring in the Budget Support Harmonization Group helped gauge progress in EDPRS implementation. The country’s imihigo culture, institutionalized since 2006, has encouraged regular M&E, creating demand for timely and accurate data. With partner support, many government programs have built-in impact evaluations to aid decision-making in fine-tuning interventions. Bank Group contributions to M&E during the evaluation period were significant and included support for SWG activities and statistics. The World Bank contributed in several ways to strengthening M&E and evidence-based decision-making. It has actively supported the preparation of several sector strategies (including the associated M&E frameworks), as well as regular SWG monitoring—in agriculture, energy, social protection, the financial sector, PFM, and elsewhere. It has a long track record of close collaboration with the NISR, notably in the analysis of living conditions survey data (including the 2015 Poverty Assessment). Its Public Sector Governance PforR operation, which supports Rwanda’s 2013–18 PFM Sector Strategic Plan, has devoted a major component to NISR institutional strengthening. Altogether, the World Bank contributed to the steady improvements made in the timeliness, quality, and dissemination of core economic and social statistics. The World Bank also contributed to M&E by supporting administrative information systems development and impact evaluations. The World Bank actively supported and funded the development of several management information systems. These include the Ministry of Finance and Economic Planning’s integrated financial management information system (developed, improved, and extended with support from a series of World Bank operations) and the education sector management information systems (whose development and rollout were notably supported under the PRSF series). Similarly, DPF supported ongoing efforts by the Ministry of Local Government and affiliated agencies to link multiple databases into an overarching data management system, with DfID-funded technical assistance. In energy, the World Bank contributed to the development of management information systems under its IPF operation supporting the SWAp. Just as importantly, the World Bank contributed to the widespread use of impact evaluation in Rwanda’s development programs—either directly through its DIME initiative (as in agriculture, rural roads, and rural finance), or indirectly through support under its projects (as in energy). 18 Rwanda Country Program Evaluation, FY09–17 | Chapter 2 Nevertheless, M&E faces persistent challenges, some of which directly affect World Bank activities. Major support from the World Bank and other partners notwithstanding, weaknesses in M&E remain prevalent. Although undoubtedly useful in enforcing discipline and sustaining momentum, many of the SWG monitoring exercises tended to take on an activity- and process-focused “bean counting” character, making it difficult to keep track of the big picture. The imihigo performance culture is also a double-edged sword: the pressure to tell a “happy story” can create perverse incentives to delay or mask data releases. For instance, several stakeholders expressed concerns that continuing reluctance to release up-to-date food security–related data may reflect less-than-positive developments. Data discrepancies also confounded monitoring: in agriculture, for example, there are persistent divergences in not only the levels but also the trends of several variables (for example, production, yields, and terraced surface area) in data validated through the imihigo monitoring system, NISR seasonal agriculture surveys, and district surveys. Finally, at the local government level, administrative data systems remain in dire need of strengthening. Capacity Building Focus and Contributions Capacity development was explicitly pursued as a cross-cutting, programwide strategic objective over only part of the evaluation period. The FY09–12 CAS (see its annex 4) sought to mainstream assistance for capacity development across all Bank Group interventions through the application of a capacity “filter.” However, little detail was given as to how, in practice, capacity development efforts would be integrated into the country program and how programwide progress would be monitored and reported on (individual areas notwithstanding, capacity development as a cross-cutting issue did not feature in the CAS results framework). That said, the World Bank did some diagnostic work applying the filter approach during the CAS period.10 Under the FY14–18 CPS, capacity development did not feature explicitly as an overarching strategic objective. Nevertheless, there was a strong de facto focus on capacity development in many areas of the Bank Group’s program—notably public resource management, social protection, and agriculture. Both the CAS and CPS pursued explicit objectives of strengthening the management of public resources at central and local levels. During the evaluation period, associated lending was either centrally focused on building capacity or contained significant components intended to do so.11 Equally, whether explicitly reflected at the overall Bank Group strategy level or not, capacity development goals permeated Bank Group financing and ASA activities across multiple areas of Bank Group support, including social protection, agriculture, the business environment, energy, and TVET. In social protection, the World Bank was instrumental in building capacity (through policy dialogue in addition to financing and ASA) since the inception of the VUP, and its efforts were sustained throughout program rollout and fine- tuned. In agriculture, World Bank lending operations promoted initiatives to build capacity among government institutions, but also among farmers and farmer cooperatives, water users’ associations, and private agrodealers and distributors (for agricultural inputs). IFC advisory operations—covering areas as diverse as the water sector PPP and the high-nutrition food processing with backward links to farmers—likewise sought to develop capacity among government and nongovernment stakeholders. Independent Evaluation Group | World Bank Group 19 In sum, the Bank Group likely contributed to a broader environment of capacity development of both government institutions and nongovernment stakeholders. Although the CAS objective of mainstreaming capacity development across the Bank Group’s program using a systematic filter approach was not considered attained,12 the emphasis on building capacity in many areas of the program is likely to have had positive results. Explicit capacity development indicators were not systematically tracked, but where available, sustained progress is apparent. To cite just one example, capacity for service delivery at the district level has clearly improved.13 More broadly, capacity development initiatives in Rwanda appear to have had some payoff in improving the government’s ability to fulfill core functions, including regulation and service delivery. The World Governance Indicators “index of indexes” for government effectiveness places Rwanda at the 58th percentile rank in 2016, compared with 50th in 2009. The improvement primarily reflects government commitment to capacity development but may also be at least partly attributable to the sustained support from development partners, including the World Bank Group. Gender Focus and Contributions Although gender-related objectives did not feature explicitly at the overall strategy level, they were pursued in certain areas, notably agriculture and social protection. Neither the CAS nor the CPS encompassed overall strategic objectives focusing on gender, although both highlighted the importance of supporting the most vulnerable groups—including female-headed households, mothers facing high mortality rates, genocide-survivor widows, and victims of gender-based violence—in reducing social vulnerability and poverty. The CAS did not incorporate gender-related indicators in its results framework. The CPS highlighted gender as an overall cross-cutting issue to be pursued but provided little explanation of how it would be integrated into the program and reported on; however, its results framework included gender-disaggregated indicators, such as the number of female-headed households benefiting from social protection programs and the number of females with access to crop and livestock insurance (although the latter was later dropped).14 This reflected the program’s focus on lending operations in support of social protection and agriculture. Several Bank Group gender-focused analytical products informed the country strategies and operations.15 Although on the face of it the Bank Group program likely made positive contributions, little information is available to support a fully conclusive assessment. In agriculture, for instance, the CPS documented a significant number of female beneficiaries under two major World Bank interventions.16 The subsequent TAS PforR operation took a gender-sensitive approach in the programs it supported and sought improvements in an Index of Women’s Empowerment in Agriculture. The index (piloted in 13 countries) represents an innovative approach that takes account of links among women’s empowerment, food security, and agricultural growth. Based on the 2014 baseline survey, Rwanda ranked second-highest among the 13 countries. However, no update has since been made available to allow developments over time to be measured. In social protection, evidence from a Food and Agriculture Organization study indicates that the VUP, which the World Bank has consistently supported, has increased women’s access to labor earnings and financial services, in turn enabling 20 Rwanda Country Program Evaluation, FY09–17 | Chapter 2 precautionary savings and better coping in the face of shocks. Conversely, the same study found that the VUP only partially promoted economic advancement for women and marginally increased their bargaining power in the household. More important, women who were caregivers (for children or old or sick relatives) were self-excluded from public works programs; consequently, as discussed in chapter 3, a more gender-sensitive enhanced public works program has been launched and is currently in a pilot phase. Regional Integration Focus and Contributions The Bank Group program has made modest contributions to the regional integration agenda, but there is a pattern of major challenges in implementation. Given Rwanda’s landlocked status, the program has rightly paid attention to the need to strengthen trade and infrastructure links with neighboring countries. Aside from covering ICT links, World Bank regional projects sought, among other things, to promote trade insurance, improve road transport links and border posts, and support hydropower generation to benefit multiple countries, including Rwanda. Nevertheless, implementation difficulties and delays have been frequent. An FY01 Regional Trade Facilitation project (whose implementation continued well into the evaluation period) substantially met its objectives of establishing a regional trade insurance agency and promoting trade and investment flows, albeit with significant delays. An East Africa Trade and Transport Facilitation project, likewise approved before the evaluation period but implemented through 2015, experienced delays and failed to establish planned one-stop border posts with Uganda and Kenya. More recently, a 2013 regional hydroelectric project also took significant time to get off the ground, although construction began in 2017. It is, however, difficult to detect a common thread in the implementation challenges that regional operations faced—other than that they are more complex and involve more stakeholders than the single-country projects. Implementation and Results World Bank project outcome ratings significantly outperformed Africa Region and World Bank–wide averages as well as the corporate target. A total of 25 World Bank projects (excluding regional projects in which Rwanda was involved) exited the portfolio during the FY09–17 evaluation period and received Independent Evaluation Group (IEG) outcome ratings (see appendix O, tables O.7 and O.8). Of these, 88 percent received outcome ratings of moderately satisfactory or better (MS+), compared with Africa Region and World Bank–wide averages of 65 percent and 72 precent, respectively, and with the corporate target of 75 percent. Strikingly, below-par (moderately unsatisfactory) outcome ratings were confined to three operations—approved well before the start of the evaluation period— that exited in the early years (FY09 or FY10). These operations covered human resource development (mainly education; see appendix D), demobilization and the reintegration of ex-combatants (appendix K), and HIV/AIDS, and were handicapped by design flaws (for example, excessive complexity or weak results handle). Since FY11, 100 percent of outcome ratings have been MS+. Two projects (in Independent Evaluation Group | World Bank Group 21 agriculture and rural development and in urban development) received highly satisfactory ratings, and all social protection operations (seven projects) were rated satisfactory. Finally, 80 percent of projects that exited during the period received risk to development outcomes ratings of moderate or lower, auguring well for sustainability. World Bank portfolio performance continues to be good. Even at its highest in the early years, the number of at-risk projects (actual or potential “problem” projects) in the portfolio was very small (two projects), and in recent years, there have been no projects at risk. Portfolio management during the period also does not appear to have encountered any significant realism or proactivity issues. Moreover, Implementation Status and Results Reports for projects approved during FY09–17 that remained open as of April 2018 all recorded both development objective (that is, progress toward project development objectives) and implementation progress ratings of MS+. Regional World Bank operations that include Rwanda performed less well. In contrast to the Rwanda-specific portfolio outcomes, only two of the four regional operations that exited over the period and had IEG outcome ratings (50 percent) had ratings of MS+. Similarly, three of the seven regional projects active as of April 2018 had at least one moderately unsatisfactory rating (for development objective or implementation progress or both) in the latest Implementation Status and Results Report. IEG ratings indicate a mixed performance for IFC investments and AS. Only four IFC investments were evaluated during the period and reviewed by IEG; two of these were rated mostly successful or better on development outcome.17 Of the seven AS projects evaluated and reviewed, only three were rated mostly successful or better on development effectiveness. However, one AS project was rated mostly unsuccessful, largely owing to unclear government commitment and associated delays. Nevertheless, it eventually resulted in a pathbreaking PPP transaction. Conclusion The Bank Group’s strategic positioning and program delivery have been strong overall. Areas of focus and the program as a whole were very responsive to Rwanda’s higher-order development goals, including its aspirations to rapidly attain MIC status. They also factored in—and struck a reasonable balance among—the Bank Group’s comparative advantage; criticality of its focus areas with respect to country development goals; program selectivity; government wishes regarding program content (based on the government views of the World Bank’s comparative advantage, and government preferences regarding financing instruments); and guidance or contributions from the government-led donor architecture. The program’s widespread focus on client stakeholder capacity development has been commendable. Finally, with some caveats, program implementation has largely been smooth. 1 E  xamples of the latter include the Strategic Plans for the Transformation of Agriculture in agriculture, the 2011 National Social Protection Strategy, (updated in 2013 to reflect Economic Development and Poverty Reduction Strategies 2 22 Rwanda Country Program Evaluation, FY09–17 | Chapter 2 goals), and the Electricity Access Rollout Program in energy (chapter 4), as well as the public financial management reform strategies (2008–12 PFMRS and 2013–18 PFM SSP; see appendix L). 2  hat is, they were Specific to the objectives toward which progress was to be measured; Measurable; Achievable; T Relevant in the sense of appropriate to measure progress toward the objectives; and Time bound. 3  or example, the Performance and Learning Review replaced a decentralization-related outcome originally sought F in the Country Partnership Strategy, “improved delivery of decentralized services,” with “increased national and subnational transparency and accountability in the use of public funds” and overhauled the associated indicators to better align outcomes sought with what the World Bank could influence. Similarly, the Performance and Learning Review replaced two indicators intended to measure improved rural/small farmer access to inputs, financing, and markets as no baseline and target had been provided in the Country Partnership Strategy. 4 The concept of structural depth was originally developed used in Structural Conditionality in IMF-Supported Programs (IMF 2007). 5 N  ote 1: Urbanization and the Evolution of Rwanda’s Urban Landscape; Note 2: Internal Migration in Rwanda; Note 3: Urbanization, Job Creation, and Poverty Reduction in Rwanda; Note 4: Profiling Secondary Cities in Rwanda—Dynamics and Opportunities. 6 The government was advised by United Nations Habitat. 7  he Poverty Reduction Support Grant/Financing 4–7 supported key financial sector reforms such as the adoption of a T microfinance law, a law on the payments system, and a framework to support small and medium enterprises. 8  n a recent reading test, 85 percent of students were rated “below comprehension” at the end of grade three, and O one-in-six grade 3 students could not answer a single reading comprehension question. 9  or example, the World Bank’s Community Living Standards Development Policy Financing Grant series delivered F during 2009–11 blended $18 million in International Development Association financing ($12 million of which was in grant form) with funding from the World Bank–administered Health Results Innovation Trust Fund, supported by Norway and the United Kingdom. 10 Rwanda Capacity Filter—Sector Analysis (agriculture, energy, and roads) (P124317; FY11). 11 For example, early in the period, the Public Sector Capacity Building Project (implemented over FY05–12) funded multiple institutional development initiatives as well as integrated financial management information system development and training in public financial management (PFM). Beginning in FY14, a Public Sector Governance Program-for-Results operation funded capacity development initiatives in PFM and statistics as part of its broader support for the government’s PFM sector strategic plan. In addition, at the local level, an FY13 Quality of Decentralized Service Delivery Support development policy operation as well as an earlier Decentralization and Community Development Project made substantial contributions to improving capacity at the district level. 12  ee, for example, the discussion in the Country Assistance Strategy Completion Report. The document also noted S that, in light of the emphasis given in the Country Assistance Strategy to capacity building and the associated filter, it would have been useful to reflect this aspect more prominently in the results framework. As it was, the results framework did not include any indicators that were dependent on the successful implementation of the filter. 13  y the time the Poverty Reduction Support Grant/Financing series 4–7 was completed in 2011, the share of districts B achieving at least 80 percent of their service delivery and development targets increased to 67 percent from 60 percent in 2006, meeting the target of 65 percent under the series. By the time of Country Assistance Strategy Completion Review in 2014, the share had reached 100 percent, meeting the Country Assistance Strategy target of 75 percent. Improvements in Public Expenditure and Financial Accountability scores over time are broadly suggestive of improvements in service delivery capacity (government of Rwanda 2016). 14 T  his indicator (indicator 6.2), together with another (indicator 6.3) on the number of smallholder farms that meet new market standards for selected products (of which number of women) were later dropped in the 2017 Performance and Independent Evaluation Group | World Bank Group 23 Learning Review on the (correct) grounds that they had no associated baselines or targets (World Bank 2017e, 25). The replacement indicators had no gender reference. 15 The products include The Adolescent Girls Initiative in Rwanda (World Bank 2015) and Voices of Women  Entrepreneurs in Rwanda (IFC 2009). (See appendix N.) 16  f the 57,000 people who benefited from the first two Rural Sector Support Project operations, series, 42 percent are O female, as are 48 percent of the 19,828 people who have benefited from the Land Husbandry, Water Harvesting, and Hillside Irrigation project (see World Bank 2014), 54). 17  ne, a risk reduction facility for lending to private schools with the Development Bank of Rwanda, was not rated by the O Independent Evaluation Group despite being rated as mostly successful in the Expanded Project Supervision Report as the Development Bank of Rwanda discontinued the facility on the grounds that it was too costly. 24 Rwanda Country Program Evaluation, FY09–17 | Chapter 2 3 T H I S C H A P T E R reviews developments and the Bank Group’s contributions under the three evaluation pillars. It assesses the extent to which, for each pillar, the Bank Group’s support to Rwanda met its relevant objectives, making Developments and substantive contributions to the outcomes targeted in its Contributions in strategies in furtherance of Rwanda’s development goals Areas of Strategic (see table 1.1). Synergies within and across pillars are also Focus discussed. All areas within each pillar are covered, but three major areas of engagement—energy, agriculture, and social protection—are covered in greater depth. More in-depth discussions of the Bank Group’s objectives and contributions under the various subpillars, and their associated ratings,1 are provided in appendixes A–M. Promoting Rwanda’s Economic Transformation for Sustained Growth The first pillar covers the Bank Group’s support in four areas: (i) improving access to and quality of key economic infrastructure services, notably energy, transport, and ICT; (ii) improving the business environment, including skills development; (iii) deepening the financial sector; and (iv) fostering urban development. This section provides a deep analysis on the energy sector, in which the World Bank was designated lead donor for the entire evaluation period. Improving Access and Quality of Key Economic Infrastructure Energy The Bank Group’s objectives during the evaluation period were to increase generation capacity and access to electricity. Under the CAS, the World Bank focused on improving access and quality in key economic infrastructure services, including energy, in support of the structural transformation and competitiveness of the Rwandan economy. Its focus under the 25 CPS was on increasing generation capacity and access to electricity. IFC sought to focus its support on increasing generation capacity, including “green” (hydroelectric and solar) generation. MIGA also sought opportunities to facilitate foreign direct investment in the energy sector. Results under Strategic Objectives Rwanda set out formal plans to increase access to electricity. In 2009, the government launched its EARP, the flagship program for achieving EDPRS electricity access targets (later updated to 70 percent access by 2018, both on-grid and off-grid, under EDPRS 2). In preparation for the EARP, and based on a roundtable discussion that brought together stakeholders to set common goals, the government signed a memorandum of understanding with donors.2 The prospectus, integrating technical, financing, and implementation planning components, was published in March 2009 and came to be known as the EARP.3 The EARP helped address the prior lack of credible electricity access plans, which had led to fragmentation and underfunding. In 2016, the government approved a Rural Electrification Strategy that reframed the 2018 access target by tier levels.4 Generation and connections have seen very substantial increases. Rwanda’s generation capacity roughly tripled during 2010–17 (appendix A).5 New capacity has been financed in large part by the private sector, with 29 independent power producers (IPPs).6 Substantial increases in generation capacity and investments in grid extensions under the EARP have sharply increased electricity connections, albeit starting from a very low base.7 Access (both on- and off-grid) has risen sharply. Grid connections increased from 6 percent in 2009 to 33 percent at the end of January 2018. Off- grid access has more than doubled since 2016 and is estimated at 12 percent at the end of January 2018. This puts the nationwide electrification rate at 43 percent, up from 6 percent in 2009. By mid- 2017, 100 percent of hospitals, 93.2 percent of health centers, 92.1 percent of administrative offices, and 69.9 percent of primary and secondary schools had access to electricity. A 2015 IEG evaluation characterized Rwanda’s rapid increases in access as a success story (World Bank 2017e), and the new SDG7 tracking report highlights Rwanda as a very robust performer globally and one of the strongest achievers in Sub-Saharan Africa.8 After some “back and forth” reforms of energy sector institutions, a sound operational framework has been in place since 2014. Under the Rwanda Electricity Group (REG), utility operations (Energy Utility Corporation; EUCL) were split from energy resource development to allow for clear financial accountability between the revenue-generating electricity business and nonrevenue energy asset development. The government retains ownership of the corporatized entities, but they are governed under company law, not public service law. Since appointing a seasoned expert as chief executive officer in May 2017, REG has made progress, notably reducing system losses, outages, and corruption. Tariff setting is now the responsibility of an independent regulatory agency. However, rapid sector expansion has put fiscal sustainability at risk, a challenge that urgently needs to be addressed. The cost of service is among the top 10 in Sub-Saharan Africa. This high cost of electricity supply is mostly due to the limited availability of domestic low-cost energy resources and high delivery costs deriving from being a landlocked country. It is further exacerbated by expensive IPPs. The revenue gap, despite high tariffs, is also among the top 10 (see Kojima and Trimble 2016). 26 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 Unit costs of electricity were around $0.32 per kilowatt hour in FY16/17 (much higher than in neighboring countries), in part because investment planning did not adhere to least-cost principles. Owing partly to difficult political-economic relations in the region, Rwanda has prioritized expensive domestic solutions over cheaper electricity imports from neighboring countries (see World Bank 2017d, 3). Many new IPPs were added through bilaterally negotiated deals rather than competitive bidding. In FY15/16, REG spent some $60 million on power purchases from IPPs, almost 38 percent of its cost structure. As a result, high customer tariffs notwithstanding,9 REG losses necessitated fiscal transfers to sustain operations. Under a business-as-usual scenario, budget transfers to the electricity sector—currently at 1.4 percent of GDP— risk increasing significantly to more than 4 percent by FY20/21,10 as a number of expensive capital-intensive fossil fuel power plants come online.11 High and increasing cost of service poses equity and competitiveness challenges, and low access to off-grid electricity means a large rural-urban divide. High tariffs ($0.20 per kilowatt hour on average) make electricity unaffordable for many and a source of competitive disadvantage to firms. Even when subsidized, firms pay more for electricity than those in neighboring countries. Electrification concerns mainly grid-connected users in urban areas and remains concentrated in the two top quintiles, with negligible coverage in the bottom 40 percent of the population. The remaining access challenge in electricity is thus primarily a rural and off-grid one.12 World Bank Group Contributions to Results As the lead donor in energy, the World Bank made important contributions to the realized progress, particularly through its convening role. After the July 2008 memorandum of understanding designated it as lead donor, the World Bank played a central role in preparing the EARP, establishing and actively participating in the energy SWG, and reducing fragmentation of aid flows (which had produced few results in the field) by promoting and helping to mainstream and finance a SWAp. Under the 2010 DOL, which designated energy as one of its focus sectors, the World Bank continued in its lead role. During the evaluation period, the World Bank supported energy reforms, and the associated dialogue contributed to building capacity. Early on, the five multisector PRSF operations during 2008–11 supported policy reforms to make the energy sector financially viable. Through policy dialogue, they contributed to raising the awareness and capacity of the government of Rwanda. Key measures included the completion of an electricity tariff study and plans to increase local private sector involvement in the construction and maintenance of electricity distribution. However, a separation of the electricity and water utilities in 2008 was promptly reversed through a remerger in 2010. The underlying cost structure was high owing to the departure from least-cost planning principles, so tariffs could not be set at break-even levels because this would have made electricity wildly unaffordable. Even though regulations were established in law, before 2014, the crisis in energy hampered implementation and development. After the PRSF series’ closure, the World Bank’s sustained policy dialogue triggered the new round of structural reforms in the power sector and raised government awareness on fiscal risks. Through Independent Evaluation Group | World Bank Group 27 the Electricity Access Scale-Up and Sector Wide Approach Development Project (EASSDP; closed in FY18), the World Bank financed the preparation of a detailed road map to implement the reforms approved by the government of Rwanda, in particular action plans to establish and strengthen the capacity of the new institutions (REG, EUCL, and energy resource development).13 The reforms contributed to an independent offtake for private sector contracts.14 The Rwanda Energy Sector Strengthening Project (approved in FY16) helped implement the main components of the action plans in the road map and enabled a smooth transition from the access agenda in EASSDP to the comprehensive sector reform supported by the new development policy operation (DPO) series,15 of which the first operation was approved in December 2017. The operation is the first and only energy DPO in East Africa to date and proactively mitigates the medium-term fiscal sustainability risks that may materialize after 2020.16 The dialogue with the government under the DPO also helped establish more meaningful guideposts for energy sector development, now reflected in the new National Strategy for Transformation.17 The DPO also aims to improve affordability of access by institutionalizing least-cost planning and introducing a new connection policy and a life-line tariff. The World Bank provided IPF steadily during the evaluation period, generally with good results. The FY05 Urgent Electricity Rehabilitation Project contributed to the increase in generation capacity from 41 megawatts to 75 megawatts during 2004–10 and to the elimination of load shedding by 2009. Investments under it helped reduce system outages and technical losses from 25 percent to 16 percent over 2004–10. The FY10 EASSDP, which received additional financing in FY13, exceeded its outcome targets on access.18 It also helped reduce the high unit cost of electricity connections from $1,019 at the feasibility stage to $411 at the end of project execution. A recent REG impact evaluation of the EARP and the EASSDP found that electrification has decreased household monthly energy expenditure (excluding electricity) and biomass collection costs and time. It also found that among connected households increases appeared in (i) income and consumption expenditure; (ii) hours worked per day; (iii) number and value of assets and income from assets; (iv) the time children spent on education and time used for tutoring children;19 and (v) the number of people visiting grid- connected health centers, which drastically reduced childbirth-related deaths. Technical assistance components in IPF operations helped develop plans and strengthen the capacity of the key institutions in the energy sector. Under the EASSDP, the World Bank financed technical assistance to define sector strategy and plans as well as capacity building for key implementation agencies, including the Rwanda Electricity Corporation (which, after merging with then splitting from water again, became REG). The Rwanda Energy Sector Strengthening project had a $20 million sector capacity strengthening component.20 The project is strengthening the EUCL’s capacity, including through the installation of an integrated business management information system, which already has several functions operational.21 The World Bank has begun supporting the regional agenda in East Africa, going beyond physical interconnections and generation projects.22 The World Bank’s regional Rusumo Falls Hydroelectric Project, expected to supply electricity to the national grids of Rwanda and others, has the potential to affect Rwanda’s energy mix and reduce unit costs. The project was approved in August 2013, but construction only began four years later owing to the complex multilateral investments involved. 28 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 The World Bank also contributed to institutional strengthening of the East Africa Power Pool and the creation of a market platform to allow countries to trade electricity in short-term markets.23 The Bank Group also encouraged green energy solutions, though it is too early to assess the full results. Currently, over 50 percent of power generation comes from renewables, and another 25 percent from Lake Kivu gas, and Rwanda’s greenhouse gas intensity is lower than that of most Organisation for Economic Co-operation and Development countries. Several Bank initiatives helped promote clean energy and eliminate reliance on oil-fired power. This included investments in hydro, such as the regional Rusumo Falls project. Another initiative has been the Lake Kivu Methane- to-Power project. IFC was one of its initial potential financiers. Although that investment did not materialize, MIGA facilitated the innovative transaction under a new sponsor with a $95.4 million political risk guarantee of up to 20 years in FY11 and FY12.24 The World Bank also promoted solar panels and policy reforms to promote clean energy.25 Several other Bank initiatives also helped promote “green” solutions.26 Stand-alone ASA represented important World Bank contributions, complementing technical assistance and capacity development initiatives under IPF operations. The World Bank provided important technical contributions in the lead-up to the EARP,27 helping to give the plan many good practice features and facilitating successes under it.28 Throughout the period, it leveraged trust funds to provide advice and technical assistance in many subjects, from renewable energy to strategic reviews of energy sector investments, thereby also contributing to capacity development. For example, a June 2011 Public-Private Infrastructure Advisory Facility electricity diagnostic helped prioritize generation projects to establish a commercially sound generation investment pipeline.29 Finally, although the Bank Group clearly contributed to progress in the energy sector, other partners were also present.30 After the World Bank, the Netherlands and the African Development Bank were major EARP donors. During 2009–14, the World Bank was responsible for almost one-third of donor funding ($396 million) to support EARP implementation, whereas the Netherlands and African Development Bank contributed 11.2 percent and 10.6 percent, respectively. Lessons and Rating A SWAp based on a sound geospatial plan can facilitate rapid expansion in access to electricity. A credible electricity access plan—based on sound geospatial optimization of on- and off-grid expansion in connections—provided the basis for structured engagement by the government with donors and private sector partners, attracting significant financing. This SWAp showed better results than could have been achieved using a project-by-project approach. Prioritizing the connection of public facilities (clinics, primary health centers, and so on) ahead of progress on household connections in the off-grid program helped maximize development impact early on. Rapid increases in access need to remain sensitive to cost and fiscal sustainability concerns. Despite the surge in generation and the access gains that place Rwanda ahead of many of its peers in Sub- Saharan Africa, average consumption lags behind because the high cost of electricity, though heavily subsidized, makes it unaffordable to consumers. Households at the bottom end of the income scale are not covered, and Rwandan firms lose out on competitive advantage. Independent Evaluation Group | World Bank Group 29 IEG rates the extent to which the Bank Group achieved its energy-related objectives as satisfactory. Transport In the transport sector, the Bank Group’s objectives were to improve regional connectivity and access to urban infrastructure. Under the CPS, World Bank engagement shifted toward improving the conditions of rural feeder roads (appendix B, table B.1). The Bank Group engaged by using national and regional IPF for the construction and the rehabilitation of rural, urban, and national transportation infrastructure, although DPF through the PRSF series supported policy reforms covering local road maintenance, including an increase in the fuel levy and the decentralization strategy. All Bank Group transport projects suffered from implementation delays. Nevertheless, the urban infrastructure and city management project helped increase urban transportation services in Kigali and two secondary cities. The Transport Sector Development Project funded the rehabilitation of a northern corridor road with crucial East Africa links, as well as multiyear road maintenance contracts that employed roadside dwellers. Initial results from an impact evaluation of the Rwanda Feeder Road Development Project (World Bank 2017f) indicate that improvement in rural transportation brought several benefits to the beneficiary rural communities (appendix B). IEG rates the extent to which the Bank Group achieved its relevant objectives in transport as satisfactory. ICT In the ICT sector, early in the evaluation period, the World Bank was engaged at the specific request of the government with the objective (set out in the FY09–12 CAS) of increasing ICT composite network coverage. This directly aligned with the Vision 2020 goal of transforming the economy. The World Bank engaged notably through two IPF operations: an e-Rwanda project, its first in East Africa, was implemented during FY07–11. Among other initiatives to strengthen government ICT processes and better deploy ICT for public information access, the project helped develop an innovative e-soko information system on market prices of agricultural commodities that farmers could readily access through mobile phones. The e-soko system was recognized in a Technology in Government in Africa award. Also innovative was the project’s deployment of ICT buses—in effect mobile telecenters and computer labs—that have had wide “reach” and helped bridge the digital divide. Phase 2 of the World Bank’s Regional Communications Infrastructure Program, implemented over FY09–16, helped achieve both the objective of lower prices for international capacity and that of extending the geographic reach of broadband networks in Rwanda. The project contributed to substantial cost savings and price reductions in international access as well as in internet and broadband services. World Bank support through the two projects contributed to increasing ICT service use and reducing the digital divide in Rwanda (appendix C). IEG rates the extent to which the Bank Group achieved its relevant objectives in the ICT sector as satisfactory. Improving the Business Environment, Including Skills Development Skills Development In the education sector, after the introduction of the DOL, the Bank Group’s objectives shifted away from general (including basic) education toward the more cross-cutting theme of workforce skills 30 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 development through TVET. These objectives were relevant, and—with some qualifications—its instruments were well designed and appropriately blended and sequenced. Education- and skills- related measures supported by the PRSF series could have been better focused and had a more tightly knit results framework, but its design was constrained by the need to conform to the CPAF. The later focus on TVET responded to government wishes and, despite some design issues, the World Bank’s Skills Development Project substantially met its objectives—though the absence of outcome indicators related to skills in the CPS weakened the handle on results at the overall program level. The “graduation” to supporting TVET with PforR financing in 2017 was a logical course of action as Rwanda’s TVET system matured. IFC appropriately complemented World Bank support during part of the evaluation period, although synergies were not explicitly sought out or showcased (appendix D). The extent to which the Bank Group achieved its relevant objectives in education and skills warrants a rating of moderately satisfactory. Improve the Business Environment Objectives were to improve the environment for private sector development, including through streamlining the business and regulatory environment, leveraging PPP, and facilitating regional integration in the latter part of the period under evaluation. These objectives were highly relevant to the country context and government strategies throughout the evaluation period. To implement this agenda, the Bank Group provided support through several ASA, IFC Advisory Services and projects, five MIGA guarantees, budget support (through PRSF), two World Bank competitiveness investment projects—seeking notably to strengthen the institutional capacity of priority institutions, promote public-private dialogue, and support tourism and agriculture—and two regional investments for trade facilitation. The significant improvement of Doing Business indicators and favorable trends in several other indicators such as foreign direct investment, trade, business registration, and a couple of successful PPP award cases suggest some improvement in the business environment. A successful case was the joint IFC and MIGA support to a milling company, which boosted noncommodity exports. Nevertheless, state involvement in commercial activities and uneven incentives continue to hinder private sector development. The regional trade facilitation projects, though delayed, contributed to some improvement in trade and investment and helped implement a scheme for the provision of insurance. The Bank Group’s support for promoting public-private dialogue and partnership and supporting special economic zones and agribusiness was less effective (appendix E). Overall, the Bank Group’s achievement of relevant objectives with respect to improving the business environment is rated moderately satisfactory. Deepening the Financial Sector The Bank Group’s objectives were to deepen and broaden the financial sector by improving the legal and regulatory framework for banking, fostering financial inclusion with a focus on access in rural finance, and developing capital markets. The relevance of these objectives was high and fully Independent Evaluation Group | World Bank Group 31 aligned with EDPRS goals. During the evaluation period, the World Bank–administered Financial Sector Reform and Strengthening Initiative supported the preparation and implementation of two government Financial Sector Development Programs. The PRSF supported reforms and ASA provided technical support. The continuous engagement in the financial sector during the evaluation period helped develop banking, microfinance, and capital markets. IFC deployed a substantive package in support of capital markets.31 Its five-year, $23 million equivalent “Umuganda Bond” issuance in May 2014 was a landmark transaction, the first by a nonresident issuer in the Rwandan capital market. Nevertheless, no private entity has issued a corporate bond since. IEG rates the extent to which the Bank Group’s relevant financial sector objectives were achieved as satisfactory. Fostering Urban Development The Bank Group’s CPS objective was to spur urbanization to develop the potential of secondary cities and reap the benefits of increased agglomeration, with a targeted outcome of developing plans for secondary cities. The objective was congruent with government objectives, but the outcome sought was narrowly focused and did not match the scope of Bank Group interventions. Given its long absence from the urban sector, the World Bank had to catch up quickly to build its knowledge base. ASA came late to influence EDPRS 2 and the national urban policies and strategies. However, it provided the basis for dialogue with the government. The World Bank had to prepare an operation in real time and build capacity during implementation, which led to some delays. The overall approach in lending was adequate in terms of the provision of infrastructure and the associated technical assistance for contract management, operations and maintenance, and revenue and expenditure management. The World Bank is sustaining its dialogue on urbanization issues with the government through the preparation of the Future Drivers of Growth study. The impact of this joint work on the government’s future urbanization approach remains to be seen (appendix G).32 Bank Group support in the urban sector is rated moderately satisfactory. The World Bank’s urban development project was only approved in FY16, so its contribution to results does not enter the ratings, unlike other sectors. Instead, the rating is based on the relevance of the World Bank’s early engagement and the quality of its analytical work and policy dialogue in improving urban planning and developing funding mechanisms for affordable housing. Overall Assessment of Pillar 1 In light of the analysis, the extent to which Bank Group contributed to promoting Rwanda’s economic transformation is rated satisfactory (table 3.1). Significant progress was made to expand economic infrastructure, deepen the financial sector, and improve the regulatory framework. Yet significant barriers subdued the private sector response, including the high cost of energy, the large presence of state-owned or -connected enterprises in some sectors, and continued constraints to lending to the private sector, which limits the ability of domestic firms to compete effectively. 32 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 TABLE 3.1 | World Bank Group Program Ratings Pillar or Subpillar Rating Pillar 1: Promoting Rwanda’s Economic Transformation for Sustained Growth Satisfactory Economic Infrastructure — Energy Satisfactory Transport Satisfactory ICT Satisfactory Education and Skills Development Moderately satisfactory Business Environment Moderately satisfactory Financial Sector Satisfactory Urban Sectora Moderately satisfactory Pillar 2: Decreasing Social Vulnerability and Improving Productivity and Incomes Satisfactory of the Poor Agriculture Moderately satisfactory Community Health Satisfactory Social Protection (demobilization/reintegration program) Satisfactory Pillar 3: Supporting Accountable Governance for Improved Service Delivery Moderately satisfactory Public Financial Management Moderately satisfactory Decentralization Moderately satisfactory Source: Independent Evaluation Group. Note: a. This rating assesses the relevance of the World Bank’s early engagement and the quality of its analytical work and policy dialogue. Reducing Social Vulnerability and Improving the Productivity and Incomes of the Poor The second pillar encompasses Bank Group support for (i) promoting agriculture; (ii) reducing vulnerability to health risks; and (iii) strengthening and expanding social protection (including the demobilization and reintegration of ex-combatants). Promoting Agriculture The Bank Group’s areas of focus in the agricultural sector were extensive, reflecting government plans. Under the CAS, the focus was on supporting sustainable increases in agricultural production. In parallel, the World Bank also began promoting the increased commercialization of agriculture to enhance the impact of its support for production increases. Under the CPS, the objective was to help Independent Evaluation Group | World Bank Group 33 improve the productivity and incomes of the poor through rural development. Supporting improved productivity and sustainability remained a key area of focus, but increased emphasis was placed on enhancing farmers’ access to financing and markets, and on improving agriculture value chains. Nutrition also became an area of focus. These areas of focus mirrored those in government plans. Results under Strategic Objectives The evaluation period coincided with specific government plans for agricultural and rural development. Rwanda’s second and third Strategic Plans for the Transformation of Agriculture (PSTA 2 and 3) covered 2008–12 and 2013–18, respectively, and were nested in the country’s broader development strategy as set out in EDPRS and EDPRS 2. The government introduced a Crop Intensification Program to boost agriculture production through designating crop types by region and by subsidizing chemical fertilizers. Agricultural development played a crucial role in reducing poverty and vulnerability. During 2000–14, agricultural growth averaging 5.5 percent was accompanied by poverty reduction of more than 15 points at the national level. At the household level, agricultural production more than doubled and marketed surplus increased. These developments were key drivers of the reduction in poverty, with 45 percent of the total attributable to increased agricultural production (35 percent) and commercialization (10 percent). An additional 13 percent was driven by increased self-employment in small, off-farm nonagricultural businesses. Nevertheless, rural poverty remains widespread and much higher than in urban areas. Early increases in production have not been sustained, and productivity remains low. Significant growth in production and yields under the crop intensification program plateaued in recent years (appendix H, table H. 2).33 Scope for improvement remains high through the greater use of improved inputs and sustainable land husbandry techniques, improved water management and extension services, and increased access to output and financial markets. At their current level, agricultural yields are estimated at some 40–50 percent of their potential as a result of suboptimal use of production factors. Similarly, livestock yields remain consistently low. Adoption of improved technologies in cultivation has been broadened, although it remains below government targets. Soil erosion control, a key driver, is the most frequently used improved technique (appendix H, figure H. 2). Irrigation remains low, even though a 2016 evaluation showed significant favorable impacts on plot use, crop choice, input use, yields, and farmer income. Chemical fertilizer use showed some (very jagged) increases initially, but more recently, uptake has stagnated, in part reflecting regression in the distribution system (box 3.1), lack of access to finance and extension, and the lack of appropriate fertilizers. In contrast, initiatives to foster composting and the use of animal waste as organic fertilizer saw considerable success. Rural access to finance has grown rapidly, though rural infrastructure development (including roads) has been patchy. A 2016 Finscope survey found that 89 percent of Rwandan adults had access to financial services, a major increase from 48 percent in 2008. There is also some evidence that the rural-urban divide is being bridged. A growing number of savings and credit cooperatives, now supervised by Rwanda’s central bank, enabled increases in rural access. But the informal sector 34 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 Box 3.1 | Evolution of Fertilizer Policies in Rwanda since 2008 Pre-2008, the fertilizer market was characterized by exclusive government importation and distribution and very poor farmer repayment rates. After the 2008 food price crisis, measures were taken to change the distribution system. The government continued to administer imports, but distribution was privatized through bulk auctions of fertilizer to the private sector, which managed retail distribution to farmers through agrodealers. A subsidy was intended to compensate for land transport costs from the transit port targeted high-value crops such as rice, maize, and wheat. In 2013, the government took further steps to liberalize the fertilizer market, privatizing importation. Unit subsidies on fertilizers were also reduced, though the range of eligible crops was increased. However, imports and farmer uptake were below expectations. In 2016, in response to reported collusion and subsidy fraud, the government abruptly introduced the Agro Processing Trust Company (APTC), a military-affiliated organization, as the exclusive buyer from importers, based on a fixed margin, to transport fertilizer to sectors. APTC oversees distribution to agrodealers and verifies delivery to farmers. Signs indicate that the APTC- managed distribution system disrupted input markets. For example, in the face of lower margins (by half), agrodealer activity has reportedly dwindled, and farmers have reported cases of distribution delays and reduced choice of fertilizers. District-level surveys also indicate that farmers face difficulties accessing finance and high input prices for fertilizer, pesticides, and seeds, even when subsidized. continues to play an important role in financial inclusion, particularly in rural areas and among women. Access to rural roads remains limited, and further investments are needed to enhance market access and reduce transport costs. An impact evaluation of feeder road interventions in selected sites showed that roads could generate income increases of close to 20 percent for farmers in remote villages (World Bank 2017f). Private investment in agriculture has increased (albeit below target), and some development of agribusiness and export value chains is seen. Under PSTA 3, private sector investment increased, but remains below target. Export growth varied significantly over time (from below 10 percent to more than 20 percent, compared with the ambitious 25 percent target), reflecting vulnerability to climatic conditions, international prices, and disease outbreaks. Export growth for selected crops proxies progress in value chain development. Performance of exports of coffee, tea, pyrethrum, and horticulture has varied over time.34 The tea sector in particular has undergone significant change. During 2001–12, major reforms privatized management and investment in tea estates and factories and strengthened associated policy making and marketing. In 2012, the tea pricing mechanism for Independent Evaluation Group | World Bank Group 35 green leaf was overhauled to ensure fair returns to all stakeholders. In contrast, horticulture has not performed well (exports registered a 9 percent decline since 2014), owing to factors including cold storage infrastructure, export policies, logistics, and land availability. Nutrition remains a major challenge. Despite increases in food production, notably under PSTA 2, in 2012, 17 percent of households had poor (borderline) food consumption patterns because they remained vulnerable to seasonal shortages. The stunting rate is 37.9 percent, and the food security index lies below the average for countries in Sub-Saharan Africa. A 2016 DIME impact evaluation—the results of which cannot be generalized—found that even where the treatment group was better off than the comparison group in income terms, food security status was similar.35 Despite government efforts under its 2013–17 Nutrition Action Plan,36 the percentage of households with an acceptable level of food consumption did not improve during 2010–16. The government has not validated any more recent data on food security, but stakeholders interviewed during the mission felt that systematic delays in the distribution of inputs (in addition to more erratic rainfall) may have increased food security risks in the past two years. World Bank Group Contributions to Results The Bank Group played a crucial role in supporting agriculture throughout (and before) the evaluation period. The World Bank consistently supported the implementation of the PSTAs since PSTA 1, its predominant role in agriculture affirmed by its lead donor designation in the 2010 DOL. During 2008–17, it deployed a combination of significant ASA and financing—five multisector PRSF operations, four agriculture-specific IPF operations, and later a PforR operation—and played an important convening role, serving notably as cochair of the Rural SWG until 2014. Additionally, two multisector governance and competitiveness IPF operations supported policy measures, capacity building, and physical investment to support the general competitiveness of agriculture and to promote value chains and the development of agribusiness. Several other World Bank IPF operations or activities (for example, on feeder roads) were instrumental to the development of agriculture. The World Bank’s support to agriculture was complemented by IFC investments and AS operations as well as MIGA guarantees (appendix E). A steady stream of World Bank ASA helped identify challenges and solutions in Rwanda’s agriculture sector. During (and before) the evaluation period, World Bank analytical work helped build the knowledge base to underpin policy formulation and investment decisions. Examples, among many others,37 include (i) the 2007 Country Economic Memorandum, which analyzed constraints to agricultural growth, providing the foundations for subsequent PRSF-supported policy actions; (ii) a major agricultural sector review in 2014 that assessed alternative growth scenarios and provided market and competitiveness analyses; (iii) a 2017 Agriculture Public Expenditure Review that analyzed consistency between realized spending patterns and stated priorities, recommending improvements in spending quality, decentralization, nutrition-sensitive programming, and “climate-smart” agriculture; and (iv) a comprehensive program of DIME evaluations that has helped track results and inform policy decisions. Long-term programmatic World Bank IPF helped intensify agriculture in marshlands and adjacent hillsides with striking results. Beginning in 2001, the World Bank deployed a three-phase, long- term (15 years, later extended to 17) Rural Sector Support Project (RSSP 1–3) under its Adaptable 36 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 Program Lending instrument to support PSTA implementation.38 The project supported the adoption of sustainable intensification technologies in targeted marshlands and adjacent hillsides and promoted the commercialization of agricultural production. It contributed notably to major increases in areas under irrigation, as well as to improvements in soil conservation and input use, commercialization, and marketing. An impact assessment of RSSP 2 (closed and rated highly satisfactory) showed major increases in crop yields in treated areas, marketing improvements, increased household employment and income, and improved socioeconomic well-being.39 Another World Bank IPF operation helped intensify hillside cultivation, with favorable results. Given Rwanda’s terrain, the majority of its arable land and farmers are found on hillsides. Since 2009, the Land Husbandry, Water Harvesting and Hillside Irrigation (LWH) project has supported a flagship government program that helps to improve sustainable land management, irrigation, and input provision on hillsides.40 In parallel, the project helped to improve access to markets, rural finance, and rural infrastructure to develop commercialization. Among other results, it developed 2,371 hectares of irrigated command areas, sharply reduced soil erosion, and improved input use (including organic fertilizer) and access to extension services, increasing crop production and sales. Under both the RSSPs and LWH operations, the World Bank helped establish and strengthen water users’ associations. The World Bank supported the separation of these associations—in charge of operation, maintenance, and the regulation of water resources and irrigation infrastructure—from the cooperatives’ management. This was a critical achievement; water users’ associations are now well established. Under the RSSP series, where such associations are more mature than under the LWH project, there has been a continuous improvement in water fee collection and better maintenance of irrigation infrastructure in the marshlands covered by the project. Early in the evaluation period, a series of DPF operations supported agricultural policy reforms generally associated with favorable results. During 2008–11, five PRSF operations (PRSF 4–7 and PRSF 8) served as the World Bank’s umbrella support for the implementation of Rwanda’s EDPRS through policy reforms, which aimed, among other things, at sustainable increases in agricultural production. Among the prior actions were steps to reform chemical fertilizer distribution, which resulted in rapid increases in fertilizer use during 2008–12 as a whole, albeit following a very jagged pattern. More recently, World Bank PforR financing experienced more muted results. Beginning in 2014, the World Bank’s TAS PforR operation sought to help foster a results-based approach in supporting the implementation of PSTA 3,41 which addressed the EDPRS 2 foundational goal of increased food and nutrient security. Thematically, the operation’s scope extends to all four PSTA 3 programs (including all 24 subprograms), and contrasts with the more targeted interventions that the RSSP and LWH operations represented. Results in the implementation of the PSTA 3 appear to be lagging in several areas—including those supported by the PforR, such as productivity gains, private sector involvement and value chain development, and nutrition. The centrally planned agriculture policy—through the Crop Intensification Program—is now showing its limits, imposing restrictions on agricultural practices that impede creativity (World Bank 2018 and Del Prete et al. 2017). Successive changes in chemical fertilizer policy have also constrained fertilizer uptake (box 3.1). Independent Evaluation Group | World Bank Group 37 Other World Bank, IFC, and MIGA projects supported agricultural value chain and agribusiness development. The World Bank’s Competitiveness and Enterprise Development project, implemented during 2001–12, helped reduce costs and increase efficiency in the tea industry, one of Rwanda’s two major export crops. The follow-up FY12 Governance and Competitiveness Technical Assistance Project contributed to organizational, strategy, and capacity improvement in agriculture sector institutions, notably regarding horticulture.42 However, this alone was not sufficient to boost exports, given other bottlenecks. Beginning in 2011, an IFC AS operation focused on removing critical constraints to investment and exports in the agribusiness sector, especially in the horticulture and tea sectors. IFC helped review green leaf tea pricing mechanisms. A 2017 impact evaluation showed a steady green leaf tea output supply response, as well as favorable early effects on farmer and tea-plucker incomes and farm-related investments (World Bank and RNAEB 2017). With respect to horticulture, IFC helped develop a land-leasing framework, repeatedly reported major barrier to growth. IFC and MIGA supported agribusiness firms in the processing industry, with potential benefits to agricultural smallholders. For example, IFC and MIGA jointly supported a grain milling company in FY10.43 IFC recently helped structure and fund an innovative PPP that addresses nutrition concerns in parallel with developing agricultural processing. The project established a modern processing plant, now operational, to produce nutritious food for mothers and children. In parallel, IFC recently approved a package of support to farmers’ cooperatives to enable them to supply maize and soy to the plant, thereby strengthening backward links and helping to boost farmers’ incomes.44 Program design, including the blending and sequencing of interventions and Bank Group instruments, showed clear strengths. Strong synergies were apparent among World Bank, IFC, and MIGA activities. The programmatic nature of long-term World Bank financing allowed adaptation to evolving PSTA priorities.45 LWH project design was informed by success factors in fostering intensification of agriculture under RSSP 1 and was appropriately holistic in focus. Postharvest infrastructure and marketing activities addressed downstream value chain gaps (boosting returns to upstream investments). Other activities facilitated sustainable access to rural financial services. The project was also linked explicitly to other Bank Group–supported investments (for example, in feeder roads). TAS PforR financing represented a natural instrument progression, but the design was overambitious and had some shortcomings. It built on a record of effective implementation under the RSSPs and LWH project to help the government strengthen the results focus of its programs. However, a December 2017 review concluded that its design was ambitious, particularly given the capacity and M&E challenges in the sector. A narrower program definition would likely have made for greater strategic leverage and impact. Combined with certain deficiencies, the breadth of program definition would be likely to detract from the achievement of objectives. The deficiencies include gaps in the technical assessment, notably concerning private sector development and the impact of climate change, as well as institutional aspects such as decentralization and the Rwanda Agriculture Board’s mandate. They also include certain shortcomings in the results framework and the disbursement- linked indicator verification process. 38 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 The Bank Group’s program design balanced investments and capacity building and focused increasingly on gender. Building on an RSSP 1 lesson regarding the pivotal role of upfront capacity development in farmer cooperatives, RSSP 2 almost doubled the funding of associated activities and refined their timing. RSSP 3 provided more resources for capacity building activities focusing on value chain development and infrastructure maintenance, critical to the sustainability of the development outcomes. The LWH project placed stakeholder training and institutional capacity building activities (often based on participatory processes) at its core. Discussion with stakeholders noted that the capacity building approach was more successful in Rwanda than elsewhere (Uganda, Kenya), demonstrating the benefit of starting to form small groups that will be federated upward. The TAS PforR operation that pursued capacity development and its results framework also included gender- specific empowerment targets, extending earlier efforts to track gender-disaggregated indicators under the LWH project. However, progress toward the targets has not yet been measured. The Bank Group needs to pay more attention to rural-urban integration. IEG’s Growing the Rural Nonfarm Economy evaluation pointed to evidence of the potential of widening inequality (World Bank 2017e). In Rwanda, the growth in nonfarm employment has been achieved in part through the rapid agricultural modernization program, which has increased production and moved surplus labor off the farm into the nonfarm economy, including into Kigali and secondary towns. But, as the evaluation notes, these efforts are less likely to result in reduced poverty where inequality is high, and may even lead to exacerbated poverty or the marginalization of disadvantaged groups (Negin et al. 2009; and Thorbecke 2013).46 Lessons and Rating A holistic approach encompassing both urban and rural development is important. The full integration of rural and urban development approaches is needed to ensure inclusive and sustained territorial transformation that does not leave the poor behind or widen inequalities. Proactive capacity building among project beneficiaries, starting small, can play an important role in multiplying and sustaining benefits. Both the RSSP and LWH projects in agriculture illustrate the importance of giving high priority to capacity building in project design. Both incorporated participatory and market-responsive value chain approaches to building business skills among farmers. Both projects demonstrate the sustained benefits of empowering farmers to organize into self-help groups that can later federate upward into cooperatives; this proved preferable to starting with excessively large groups at the inception of the project. Scale-up from focused to broad-based programmatic support needs to be paced and balanced. Following the successful implementation of the RSSP and LWH projects, the transition to a full- fledged PforR operation encompassing the government’s entire agriculture strategy may have been too ambitious. Experience under the PforR operation underscores the importance, during the design phase, of embedding the balanced approach that has characterized the past IPF operations in the sector—notably through the selection of disbursement-linked indicators to ensure that capacity building and institutional strengthening remain incentivized, but also by ensuring that a transparent and robust M&E system is in place to verify that these indicators have been implemented. Independent Evaluation Group | World Bank Group 39 IEG rates the extent to which the Bank Group attained its relevant objectives in agriculture as moderately satisfactory. On Bank Group project sites, significant investment helped develop new irrigated land and secure land on steep slopes, foster the use of seeds and fertilizers to boost productivity, and develop and strengthen the capacity of cooperatives. However, the implementation of the overall PSTA 3, supported by the PforR, shows less tangible results at the national level. Productivity gains have leveled off, and further improvement will require a larger role for the private sector to achieve allocative efficiency gains and better vertical coordination through value chains. The public sector would need to more effectively target the provision of public goods to activities with higher returns, and to support the promotion of adaptation to climate change policies. Reducing Vulnerability to Health Risks In health, the World Bank sought in the early years of the evaluation period (under the CAS) to reduce health risks to vulnerable groups. A three-operation Community Living Standards Grant DPF series blended IDA financing with support from Results-Based Financing for Health (a World Bank– administered multidonor fund). It supported policy actions that, among other things, introduced new demand- and supply-side incentives designed to boost quality and uptake of prenatal and postnatal care. For example, the demand-side incentives consisted of gifts in kind (such as baby cloths and water purification tablets) for women seeking antenatal care, in-facility delivery, and postnatal care. Impact evaluation work sponsored by Results-Based Financing for Health, which was, commendably, built into the program, found that the incentives had a significant effect on the timely uptake of care (even when in-kind gifts were out of stock), and the Implementation Completion and Results Report Review for the Community Living Standards Grant series judged the achievement of the series’ health-related objectives to be substantial (appendix I). IEG rates progress toward relevant objectives in health as satisfactory. Strengthening and Expanding Social Protection, Including Demobilization and Reintegration of Ex-Combatants Areas of World Bank focus included Rwanda’s flagship social protection program. The flagship VUP has three parts: direct support (unconditional cash transfers) for those unable to work; employment in public works projects at a guaranteed wage; and financial services (loans provided at low interest rates, though not exclusively to households in extreme poverty). Separately, the Rwanda Demobilization and Reintegration Program (RDRP) provides social protection support—including benefits covering health, education, shelter, and income-generating activities—to eligible ex- combatants and dependents from both the Rwanda Defence Force and armed groups outside the country. Results under Strategic Objectives The VUP was significantly expanded, and it contributed to improvements in the welfare of beneficiaries during the evaluation period. From its origins in 2008 as a pilot focused on 30 high- poverty incidence sectors (subdivisions of districts), coverage of the cash transfer system and public works employment was steadily expanded. By the end of FY16/17, 95,846 ubudehe 1 households (the poorest category) nationwide (that is, in all 416 sectors) were direct support recipients. Public 40 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 works beneficiaries numbered 128,000 households in 240 sectors. Financial services beneficiaries, in contrast, remained roughly constant at some 55,000, following the component’s initiation in 2009, with a drop in FY15/16 because their administration was transferred to local finance cooperatives. The VUP helped beneficiaries meet their immediate needs and increase livestock holdings; the effect was larger for direct support than for public works employment, notably on food security and access to health insurance. Women’s empowerment also experienced some positive effects.47 Significant institutional capacity development at the central and local levels accompanied VUP rollout. The Ministry of Local Government’s capacity for policy formulation and oversight has been significantly strengthened, and the autonomous Local Administrative Entities Development Agency was created and operationalized to implement social protection policy. Budget execution and reporting improved through the production of a consolidated social protection budget, and transparency and accountability have been enhanced through the publication of the social protection budgets and imihigos (performance contracts, including targeted outcomes) for each of the country’s 30 districts. A comprehensive integrated social protection management information system has been designed and launched, although it is still being rolled out. Citizen appeals and complaints mechanisms were established. Guidelines for scaling up social protection programs in response to disasters were also developed. Finally, collaboration with the NISR for the collection and analysis of social protection–related data was institutionalized. A 2014 impact evaluation of the VUP pointed to several weaknesses that the government began addressing, though hard evidence on the effectiveness of the new policies is not yet available. First, the community-led ubudehe household-tagging mechanism had significant errors of inclusion and exclusion in the targeting of cash transfers. In response, the government revised the ubudehe system and moved scale-up plans to areas where poverty is high.48 Second, vulnerability to shocks (mainly natural, economic, and health-related) remained pronounced, particularly among public works employment beneficiaries.49 This suggests a need to sustain VUP support to eligible households over more extended periods.50 To this end, recent initiatives include expanded public works offering more employment and a monthly wage. The government also piloted a “minimum graduation package” scheme in 30 sectors in 2016.51 Third, opportunities for caregiving women (to children or the elderly or both) were more restricted. The extended public works scheme will expand employment opportunities, notably for caregiving women. Finally, more focus on child poverty is needed, including nutritional needs. The government now plans to introduce nutrition grants. Significant progress was made in demobilizing and reintegrating ex-combatants during the evaluation period. Implementation of the RDRP continued successfully. The full caseload of 4,000 Rwanda Defence Force members was demobilized, as were 4,585 adults and 264 children formerly associated with armed groups (out of a maximum of 5,500 targeted). Ex-combatants and dependents were provided with support services and grants to support their reintegration into civilian communities of their choice. Enhanced support, including custom-designed housing, was provided to the disabled. Over 77 percent of ex-combatants who received reintegration grants were economically active, compared with a program target of 70 percent. Almost two-thirds of ex-combatants reported being socially accepted in their communities. Although this is below Independent Evaluation Group | World Bank Group 41 the program target of 80 percent, other indicators, including marriage in the community, point to favorable reintegration outcomes. World Bank Group Contributions to Results The World Bank was consistently a key partner in social protection even before the evaluation period and played a key role in promoting strong donor coordination. With DfID as convener, key development partners regularly met in the social protection donor coordination group. Among other advantages, this helped harmonize their inputs to the government-led social protection SWG, which regularly updated the social protection and financing framework. Hence, its designation as a “silent partner” under the 2010 DOL (and 2013 update) notwithstanding,52 the World Bank provided substantial support, both financial (through DPF) and technical, for the design and rollout of the VUP and for the RDRP. Crucially, strong complementarity between lending and ASA support for the VUP helped ensure strong feedback loops in the rollout of the program. Regular collection and analysis of panel data and evaluation work, for instance, helped inform policy reforms and targets from one DPF operation to the next. Earlier, under the 2005 Decentralization and Community Development Project, the World Bank had provided technical assistance—subsequently complemented by DfID–supported advisers—to develop Ministry of Local Government capacity, contributing to the design and launch of the VUP program in 2007. More generally, through steady, patient dialogue, the World Bank was reportedly influential in shaping Rwandan policy makers’ views throughout the period, notably regarding the importance of prioritizing direct support (unrequited transfers running counter to Rwandan culture) and rolling out geographical targeting. Programmatic DPF supported VUP piloting, rollout, and adjustment, as well as associated capacity development. During 2009–11, a three-operation DPF Community Living Standards Grant series supported the piloting of the VUP in 30 sectors. A second three-operation Support to Social Protection System DPF series over FY12–14 provided support for expansions and adjustments in the VUP, including associated institutional, management, and M&E enhancements—such as the establishment of Local Administrative Entities Development Agency to coordinate core social protection programs. A third DPF series over FY15–17, the three-operation Social Protection System series, supported a further expansion in the VUP to full national coverage, along with additional governance reform (for example, to strengthen citizen engagement). Nearly a decade of World Bank DPF support was followed by an IPF operation in December 2017, which enhanced the focus on child poverty and vulnerability and early childhood development. The project will notably help fine-tune and scale up the expanded public works initiative and introduce nutrition grants.53 ASA informed lending-supported VUP reforms and helped build associated capacity. Analytical work included VUP targeting and poverty surveys, reviews of public expenditure on social protection (2006 and 2010), policy guidelines for sector harmonization (2012), the Social Safety Net Assessment (2012), the 2014 VUP program impact evaluation, and the Family Strengthening Initiative focused on early child development (2017). These helped identify issues to be addressed under the successive DPF series. The Rapid Social Response trust fund provided programmatic technical assistance to help inform key components of the reforms.54 Significant technical assistance from other donors, notably DfID, complemented World Bank support. Equally significant was the informal capacity building support that came through regular dialogue with World Bank staff, which key stakeholders thought particularly effective 42 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 when senior staff were present in the field early in the evaluation period. The array of analytical work was also critical in providing policy options not only to the sector ministry but to the Ministry of Finance, which facilitated dialogue on fiscal space that led to increasing resource allocations to the sector. A World Bank IPF operation was the principal source of external support for the RDRP. A second Emergency Demobilization and Reintegration Project, approved in August 2009 and closed at the end of 2017, provided combined IDA financing (topped up in 2014) and multidonor trust fund grant funding. Significantly, its design took account of the experience under a 2002 predecessor project. That project had suffered from results framework weaknesses, making it a challenge to establish that it had contributed to the successful reintegration of demobilized ex-combatants. The World Bank’s contribution to VUP and RDRP implementation and outcomes—including associated capacity development—was clearly central, but other key partners in social protection included DfID, Sweden, the European Union, and the United Nations Educational, Scientific, and Cultural Organization. DfID in particular has served as lead donor in social protection and provided substantial technical assistance since the inception of VUP. In addition to individual partners’ contributions, the social protection SWG provided an effective forum for dialogue, on occasion exercising decisive influence on policy direction. Lessons and Rating Community engagement helps make for effective social protection. In Rwanda, alongside the establishment of foundational tools for the management of and delivery of services under social protection programs, citizen engagement has been key to the effective deployment of social safety nets. The homegrown approach to tagging and tracking poverty under the ubudehe system of collective problem-solving proved to be a powerful tool to ensure community ownership and empowerment as interventions under the VUP were scaled up. Similarly, the active involvement of host communities has proved to be a key factor in reintegrating ex-combatants into civilian life. Continuous World Bank DPF support for the social protection system over a decade represents an innovative approach that could bring meaningful lessons for the region and beyond. IEG rates the extent to which relevant Bank Group objectives in social protection were achieved as satisfactory. Overall Assessment of Pillar 2 In light of the analysis, the extent to which the Bank Group contributed to promoting Rwanda economic transformation is rated satisfactory (table 3.1). The World Bank contributed substantively to the setting up and progress in building of a social protection system in Rwanda. Ongoing support to further improve and monitor the efficiency of targeting, to consider gender, and to introduce nutrition grants in the social protection program is highly relevant. Results under the Second Demobilization and Reintegration Project are favorable. In agriculture, the Bank Group was instrumental in the policy dialogue with the government of Rwanda and contributed substantively to progress in improving agriculture productivity and commercialization in marshlands and hillside areas under the RSSP and LWH project. The recent PforR showed mixed results: productivity seems to have leveled off recently Independent Evaluation Group | World Bank Group 43 for a number of crops, food security has not improved, and the performance of Rwanda Agriculture Board and progress in decentralization are lagging. Yet, World Bank support to the government’s new agriculture strategy and discussion through the Future Drivers of Growth study rightly tackle challenges to unlocking agriculture’s potential and transitioning to a new private sector–led model. Supporting Accountable Governance for Improved Service Delivery The third pillar covers Bank Group support for PFM and decentralization. Strengthening Public Financial Management The Bank Group’s objectives were to improve transparency, efficiency, value for money, and accountability in public resource use. Objectives were relevant, and World Bank instruments were generally well designed and appropriately blended and sequenced. Parallel support through IPF and DPF was later succeeded—appropriately, especially given the limitations on using DPF after 2012— by PforR financing, and adequate provision was made for ASA. Results were broadly favorable but often fell short of plans. The World Bank also made adequate provision for developing institutional capacity for public resource management; it was a central area of focus in most financing operations as well as ASA. A 2017 Public Expenditure and Financial Accountability assessment report indicated good performance in PFM systems: ensuring aggregate fiscal discipline, strategic prioritization of resource allocation, and efficient and effective use of resources at the public agency level in service delivery. However, the IMF points to the need for continued attention to improving fiscal transparency, including the need to account for fiscal risks and to extend financial reporting systems to local service delivery units such as hospitals and schools (appendix L). The extent to which the Bank Group achieved its relevant objectives in public resource management warrants a rating of moderately satisfactory. Strengthening Decentralization The World Bank’s objectives were to strengthen local-level public resource management through the evaluation period, specifically by enhancing local government tax generation and administration, and to improve subnational transparency, efficiency, value for money, and accountability in the use of public funds. The objectives were highly relevant and fundamental to the country context and government strategies and remained so throughout the evaluation period. The Bank Group supported the government early on through IPF (the decentralization and community development project) and DPF and shifted to support through a PforR after 2015. Capacity development of subnational governments featured centrally in the World Bank’s work throughout. Several ASA tasks, though not primarily focused on decentralization, underpinned World Bank lending. The World Bank also actively participated in the SWG on decentralization and governance. Overall, the packaging and sequencing of activities were appropriate, with greater focus on fiscal decentralization and service delivery in the latter part of the period. The increased resources that local governments have managed shows progress in fiscal decentralization, and the improvements in key governance and institutional indicators and participatory processes indicate positive outcomes; yet challenges remain. Districts’ own revenue 44 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 increased more than threefold over the past decade. But transfers to them remain very rigid, with over 80 percent earmarked. The World Bank contributed to financing and rolling out PFM systems at the local level and advancing capacity. But outcomes on service delivery and accountability show mixed results. Satisfaction on service delivery as observed in the Citizen Report Card has improved since 2014. But a recent assessment finds that “district performances in service delivery are guided by strict central government oversight rather than downward accountability exercised by district councils and citizen’s groups” (MINALOC 2017). Overall the Bank Group’s support could have been more sustained and coordinated across sectors for greater impact. Given the high relevance of objectives and substantial achievements in the results, the achievement of relevant Bank Group objectives in decentralization and public resource management at the local level is rated moderately satisfactory. Overall Assessment of Pillar 3 In light of the analysis, the extent to which the Bank Group contributed to supporting accountable governance is rated moderately satisfactory (table 3.1). Progress was made in strengthening PFM at the central and local levels. But analytical work could have been used to draw clearer attention to the broader unfinished reform agenda in economic governance. The main challenges are related to the enforcement of the regulatory framework, particularly weaknesses in competition policies, including vested interests that hamper private sector development. Others are found in accountability mechanisms between government and citizens.55 World Bank Group Program Ratings under Each Pillar Although successes in its areas of engagement were not unqualified, the Bank Group made significant contributions almost across the board. In no area where it was active during the evaluation period (as enumerated in chapter 1, table 1.2) were the results of its support subpar. Table 3.1 summarizes outcome ratings associated with Bank Group support for each pillar, based on the analysis detailed in the appendixes. Finally, there were synergies within and across pillars. Within pillar 1, under the business environment, World Bank projects in competitiveness also supported improvements in financial sector regulatory frameworks, and IFC AS supported PPP in infrastructure.56 Within pillar 2, synergies were apparent between agriculture projects and social protection programs, with VUP beneficiaries hired to work on terraces or road maintenance, thereby contributing to the outcomes sought in agriculture. More recently, the World Bank embarked on a multisector approach to nutrition challenges through agriculture, social protection, and health projects. Synergies across pillars included the development of infrastructure (in particular, feeder roads) under pillar 1 in areas of World Bank project support for agriculture, covered under pillar 2. They also included the linking of VUP and agriculture project beneficiaries under labor-intensive works to financial institutions by requiring individuals to open accounts to be paid, which proved an innovative way to promote financial inclusion and inject cash into the rural economy that in turn created demand for nonfarm employment. Independent Evaluation Group | World Bank Group 45 1  he ratings—on the standard six-point scale—measure the extent to which (in the Independent Evaluation Group’s T judgment and given the information available) the World Bank Group achieved its relevant objectives in the stated area. 2 T  he memorandum of understanding spells out the aim of improving efficiency, effectiveness, and impact of the Energy Sector Strategy by increasing transparency on all sides; improving the predictability and allocation of financing; and better coordinating the multiple inputs and activities that serve sector objectives. The memorandum of understanding provides a sound foundation for scaling up energy sector interventions to achieve the objectives of the Economic Development and Poverty Reduction Strategy. 3 The report, Rwanda Electricity Access Programme. Volume I: Investment Prospectus, also embodied a geographic  information system–based spatial network plan to optimize expansion in Rwanda through the year 2020 comparing, among others, the costs of electricity supply from alternative sources (on-grid and off-grid). 4 T  he tier levels of access are defined under the Sustainable Energy for All Multi-Tier Framework initiative. The initiative takes a multidimensional view of the energy sector by considering various service levels and attributes such as availability, quality, reliability, health or safety, convenience, and affordability; and it addresses multiple technology options (such as grid and off-grid). Ratings go from tier 1 (minimum 12 watt hours per day) to tier 5 (safe, reliable, and unlimited 24-hour service from a grid system). 5  eneration is sourced in hydropower at 45 percent, oil (heavy fuel oil and diesel) at 27 percent, peat at 7 percent, solar G at 4 percent, lake methane at 14 percent, and imports at 3 percent. 6 As of 2017, 52 percent of capacity is under private ownership. 7 I n early 2009, Rwanda was one of the least-electrified countries in the world with a national electrification rate of approximately 6 percent on-grid (approximately 110,000 households), but this dropped to 1 percent in rural areas. There were no households connected off-grid. 8 See https://trackingsdg7.esmap.org/. 9  subsistence level of electricity (30 kilowatt hours per month) would be unaffordable for more than three-quarters of A those who are still without it. 10 n recent years, fiscal transfers to the energy sector have declined from 2.5 percent of gross domestic product in I FY14/15 to 1.4 percent in FY16/17. 11 T  his is according to preliminary results of the draft Least-Cost Power Development Plan commissioned by the government (see World Bank 2017d, 4–5). 12  mong the urban population, where 77 percent have access to electricity, access is concentrated in higher-access A tiers (that is, higher level of service), but 84 percent of the rural population have no access to electricity (tier 0). 13 T  hose plans included specific activities aimed at improving the operational performance of the utility (Energy Utility Corporation; EUCL) in all business areas to better serve its customers, reduce losses and increase collections in supply, and achieve higher efficiency in the management of corporate resources. 14 Twenty-nine independent power producers are currently responsible for over half the power generation. 15 T  he project supported the definition of an optimal organizational structure for EUCL, the competitive appointment of staff in all positions of the structure, and the incorporation of tools (information systems, revenue protection program, and so on) to support the efficient, transparent, and accountable execution of operations in all business areas. 16  he development policy operation (DPO) directly reduces these future fiscal risks, aiming to cap sector subsidies at T 1.4 percent of gross domestic product. 46 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 17  aving National Strategy for Transformation targets in line with realistic sector plans was a trigger in the DPO policy H matrix. The government now aims at 100 percent access by 2024 and the objective of having 15 percent reserve capacity over and above growing demand. 18  he targets set under the Electricity Access Scale-up and Sector Wide Approach Development Project related to T the number of households connected under the EARP program. The achievement was 810,923 versus a target of 768,000. 19 Number of hours studied at home per day after sunset for children who are in school. 20 S  ubcomponent A-3: Strengthening of Technical Capacity of Key Functions in the EUCL ($5 million equivalent) sought to support the strengthening of the technical capacity in key functions of the EUCL—namely, operations, commercial services, finance, and corporate services. The technical assistance included but was not limited to (i) coaching, mentoring, and enhancing the capacity of EUCL staff in areas of their technical expertise; (ii) assisting EUCL to develop and document functional processes and operational procedures; (iii) assisting EUCL to implement the management information system and the Revenue Protection Program (components A-1 and A-2); (iv) assisting EUCL to collect and keep data records to be used as baseline data in performance targets setting; and (v) through the assistance of a strategy execution consultant, preparing and implementing a corporate strategic plan and developing a performance-based dashboard. 21 S  ome of the functions already at work include (i) a human resources module, including payroll, leave administration, performance management, employee self-service, employee records, and performance management; (ii) finance management, including budget and control, treasury management, financial reporting and delivery of annual and other financial reports, cash flow planning and forecasting, and assets register and control; (iii) procurement, covering the whole procurement cycle; (iv) inventory management; and (v) project management. 22  thiopia-Kenya Interconnector; Proposed Tanzania-Zambia Interconnector; Regional Rusumo Falls Hydroelectric E Project. 23 M  ultidonor Trust Fund, International Development Association Grant to Eastern African Power Pool in Proposed Tanzania-Zambia Project. 24 Project Brief for KivuWatt (www.miga.org/projects/). 25  he 2017 DPO began reforming the legal framework for renewable energy generation, developing grid-connected T hydropower and solar power and removing barriers for off-grid solar energy. 26 I n 2015, with Public-Private Infrastructure Advisory Facility financing, the World Bank supported the preparation of action plans to develop hydropower and solar energy in the country. It also actively assisted the government of Rwanda with drafting the Rural Electrification Strategy that reframed the 2018 access target with regard to the tier level of access as defined by the Sustainable Energy for All Multi-Tier Framework. The World Bank also helped the government of Rwanda prepare its Scaling-Up Renewable Energy Program (SREP) Investment Plan (the Renewable Energy Fund financed by SREP through the World Bank is supporting implementation of the SREP Investment Plan) and is starting to support its implementation, notably the off-grid programs of the Rural Electrification Strategy. Additionally, with Energy Sector Management Assistance Program financing, Rwanda carried out a Sustainable Energy for All Multi-Tier Framework survey (https://energydata.info/dataset/rwanda---multi-tier-framework--mtf-- survey--2018-), the first ever completed in the world, to define priorities for off-grid electrification. 27  he Electricity Access Rollout Program was a national geospatial access rollout plan that combined geographic, T demographic, and technical parameters to scale up access in a least-cost combination of on-grid and off-grid electrification and in a time-bound manner. 28 T  he plan addressed equity and shared prosperity considerations through policies for keeping connection charges affordable for the poor. A substantial off-grid program gave priority to connecting public facilities (schools, clinics, primary health centers, and administrative centers) so that developmental impacts could be attained even ahead of the progress on household connections. Independent Evaluation Group | World Bank Group 47 29 ts results were presented in an Energy Investor Forum held in Kigali in October 2011. It was a timely report since the I purpose of the forum was to attract credible and experienced project developers and investors to the country. 30 O  ther development partners that have supported the Electricity Access Rollout Program are the African Development Bank, the Arab Bank for Economic Development in Africa, the Belgium Technical Cooperation, the European Union, the Netherlands, Japan, the OPEC Fund for International Development, and the Saudi Fund. 31 T  he package included (i) technical assistance provided to the government when it issued it a sovereign bond, which was oversubscribed; (ii) a regulatory and capacity building program delivered by the Regional Program of the International Finance Corporation (IFC), which was instrumental to Rwanda’s CM development and skills building across the board for the Central Bank, Treasurers of Commercial Banks, Brokers, and so on; (iii) issuance of a local bond of $23 million to set the benchmark on the back of government and move the yield curve at five years (currently the yield curve has progressed to 15 years); (iv) an off-shore local currency bond issue of $5 million that sent a signal to investors outside Rwanda that Rwanda is a good investment destination, and that one can invest in local currency and still make a return; (v) the putting together of a nine-month Pan-Africa Program by IFC and the Milken Institute cofunded by governments and IFC and the Milken Institute, which provides practical skills to two selected candidates, in addition to offering two development assignments to a central bank senior officer on the IFC trading floor. 32 The government’s intention is to revisit and update the master plans for Kigali and secondary cities. 33 I n 2007, the government undertook a Crop Intensification Program targeting six priority crops—maize, wheat, rice, Irish potato, beans, and cassava—focusing on input distribution, land use consolidation, proximity extension services, and postharvest handling and storage. 34 In 2013/14 and 2015/16, growth amounted to only 7 percent and 6 percent, respectively, whereas in 2014/15 and  2016/17, growth of 22 percent and 23 percent was recorded, respectively. 35 The overall proportion of food-insecure households was close to 60 percent in the sample. The year 2016 was poor.  The absolute proportion of individuals reporting food insecurity was almost three times higher in 2016 than in 2014 (World Bank 2017g, 39). 36  he design of this component included (i) a scale-up of the kitchen garden program; (ii) improved nutrition knowledge T and practices for food-insecure households; (iii) developing a program of biofortified food and expanding the One Cup of Milk per Child Program; and (iv) maintaining a national strategic food reserve. 37 F  or example, a 2009 Investment Climate Assessment focused on agricultural export promotion needs. Among the World Bank’s regular Economic Updates, the 2011 and 2012 reports also focused on agriculture, discussing challenges in sustaining productivity increases and developing intra-East African Community trade in agriculture to promote regional security. World Bank technical assistance in 2012 helped elaborate priorities relating to climate change and land and natural resource management improvements. 38  ural Sector Support Project (RSSP 2) began in 2008 and RSSP 3 was brought forward to 2012 when RSSP 2 was R completed one year ahead of schedule. The project received additional financing in 2014 and was expected to close in October 2017. 39  or example, the more productive use of extensive fertile marshland areas led to an increase in rice yields from 2.7 tons F per hectare to 5.7 tons per hectare within participating districts and from a low of 1.5 tons per hectare to 6.7 tons per hectare for direct beneficiaries within one to two years (Oxford Policy Management 2012). 40  he project received additional financing in 2013 for more land husbandry work (7,000 hectares in the poorest regions) T and 500 hectares of irrigation. The closing date was extended from June 2014 to June 2017 and in 2017 a one-year extension was requested until June 2018. 41 The project received additional financing in 2017 and was extended until October 2018. 42  he project had a positive effect on the horticulture sector’s ability to organize itself and to design and implement long- T term strategic interventions. 48 Rwanda Country Program Evaluation, FY09–17 | Chapter 3 43  he Multilateral Investment Guarantee Agency provided guarantees for a period of up to 10 years against the risks of T transfer restriction, expropriation, and war and civil disturbance. 44 This is too early to assess whether the project is providing the expected benefits to agricultural smallholders.  45 F  or example, the focus was adjusted from RSSP 1 (initial focus on developing capacity to support the adoption of sustainable intensification technologies in marshlands and adjacent hillsides) through RRSP 2 (the intensification and broadening of support to the commercialization of agriculture following the initial agricultural supply response) and Strategic Plan for the Agriculture Sector 3 (promoting the diversification of economic activities to increase rural incomes). 46 n addition, a World Development article by Dawson, Martin, and Sikor (2015) suggests that these policies may be I exacerbating landlessness and inequality for the poorest. 47  2016 Food and Agriculture Organization report documented some positive impacts of the VUP on women, including A improved access to financial services, technical skills, and wage employment and income (and associated risk management capacity), as well and enhanced self-esteem and participation in mutual support social networks 48  he government of Rwanda reduced the number of categories from 6 to 4, making only the bottom category eligible T for social protection benefits. The government also adopted a new set of criteria to guide the categorization of households into the four new categories. 49  or instance, households that benefited from the VUP for one period typically saw the increase in their livestock F holdings reversed in response to a shock. 50  This would enable more sustainable expansion in their material assets, improve resilience to shocks, and facilitate graduation from poverty. 51  nder it, beneficiaries are given access to a minimum number of public works employment days with on-time U payments, combined with asset transfers and complementary services such as health insurance, skills and financial literacy training, and business or employment opportunities. Households are supported by a case worker to facilitate access to these services and promote graduation. 52 Of course, the Division of Labor expressly did not apply to sector budget support. 53  n additional financing to the Strengthening Social Protection Project ($23 million of grant) has been approved by the A World Bank and preparations are under way for its implementation during FY18–21. 54 C  ontributions included workshops and studies concerning child-sensitive protection; analysis of the main social protection income-generating programs; evaluation of the welfare trajectories of VUP beneficiaries to strengthen the identification of individual-level drivers and obstacles for change; and technical assistance for strengthening the management information system. 55 These issues are now treated in the discussion on state institutions in the Future Drivers of Growth study. 56 T  he Skills for Growth Program-for-Results operation approved in July 2017 is seeking to sharpen the focus of Rwanda’s technical and vocational education and training system on priority economic sectors where skills shortages are evident—including energy, transport, and logistics—and expanding opportunities for individuals to acquire quality, market-relevant skills in these sectors. Independent Evaluation Group | World Bank Group 49 4 G OV E R N M E N T C O M M I T M E N T A N D L E A D E R S H I P have been at the heart of Rwanda’s success. During the evaluation period, Rwanda sustained the vigorous economic growth and social progress that have driven its transformation Conclusions, from a fragile, conflict-affected country to one that is widely Lessons, and regarded as being among Africa’s most dynamic, with high Recommendations potential. Sustained government commitment and effective management of the reform agenda were key factors in this transformation and enabled the Bank Group and other development partners to play a positive role in supporting the country’s development. The Bank Group’s positioning in relation to Rwanda’s Vision 2020 goals was strong. The Bank Group’s strategic objectives over the evaluation period reflected many elements critical to the country’s attainment of MIC status. At the same time, they were responsive to government wishes, notably regarding specialization by development partners under the DOL. The objectives underlying Bank Group support in agriculture, energy, and urban development, in addition to social protection, the business environment (including the financial sector, skills development, and PPP), and PFM, have all been of central relevance to the country’s middle-income aspirations. The Bank Group also paid adequate attention to capacity development objectives, an important foundation for the country’s continuing advancement. Overall, the extent to which the Bank Group’s program met its relevant objectives over the evaluation period warrants a satisfactory rating. The rating reflects both the extent to which objectives were attained under individual pillars and subpillars (see chapter 3) and the Bank Group’s overall strategic positioning and program delivery (see chapter 2). As discussed in chapter 3, progress was made toward relevant Bank Group objectives in all areas of engagement, notably in energy, transport and ICT, agriculture, social protection, the financial sector, and public resource management—areas of engagement that were either explicitly indicated under the DOL or otherwise spanned the 50 entire evaluation period. In many of these areas, of course, progress was qualified. In energy, the notable successes in increasing generation capacity and (on- and off-grid) access to electricity have been tempered by concerns regarding affordability and the risks to fiscal sustainability, now being addressed under the newly approved energy DPO. In transport, implementation delays and capacity-related challenges diluted results. In agriculture, the increased use of improved techniques (in the mold of best practice RSSP and LWH project sites) has been slower than hoped, food insecurity remains significant, and the distribution system for key inputs, notably fertilizer and seeds, shows regression. Nevertheless, taking account of the strength of the Bank Group’s overall strategic positioning and program delivery, including synergies across pillars, IEG judges a rating of satisfactory to be justified. IEG considers the Bank Group’s performance in designing and implementing its Rwanda program strong. Although some questions can be raised, few obvious gaps or cases of bad project design or strategic errors were apparent in the Bank Group’s program. Even where successes were qualified, they were so mostly in spite of the Bank Group’s support rather than because of it. They often reflected factors extraneous to the Bank Group, including changes in leadership and political economy constraints. And the World Bank has proactively sought to address less encouraging developments. In the energy sector, it is now providing programmatic DPF to the government to address the medium-term fiscal sustainability risks that could materialize after 2020. In agriculture, the World Bank through its PforR operation played a key role in engaging the debate on sensitive policy issues that need to be addressed to unleash agricultural potential. These include the removal of obstacles to private sector growth and increased devolution of responsibilities to districts. Together with the findings from a comprehensive agriculture public expenditure review, this policy dialogue has provided key inputs to the government’s development of a new agriculture policy. Revitalizing and sustaining progress will now require a paradigm shift. Rwanda’s growth model for the past two decades depended largely on public investment backed by donor support. The government has in effect taken on the role of catalyst of growth, often seeding business development. Although some increases in private investment were observed in recent years, party- and military-affiliated companies continue to play an important role in the economy (in the views of some, this can have a deterrent effect on private investment). As Rwanda approaches MIC status and the terms of external support progressively “harden,” driving up public debt, the sustainability of this model will come increasingly under strain. The Bank Group is uniquely positioned to help Rwanda manage the shift to a new, private sector–led growth paradigm. Its current engagement with the Future Drivers of Growth study to help shape Vision 2050, a new long-term plan for Rwanda’s development currently under preparation, has given it a role of trusted development partner. The road map toward a new growth model will need to embrace an ambitious reform agenda that addresses key issues and constraints to attain MIC status and sustain progress thereafter. The most important issues include improvements in nutrition and basic education; appropriate pacing and sequencing of urbanization; dramatically stronger outward orientation—including stronger, more diversified export growth—in part through closer regional integration and cooperation; a more strategic and transparent approach to fostering an enhanced Independent Evaluation Group | World Bank Group 51 role for the private sector in the economy; increased domestic resource mobilization; and better exercising of expanded local government functions and responsibilities. The Bank Group program can make strategic contributions to help Rwanda follow the road map and realize the vision. Key Lessons This section reviews lessons that emerge from the Bank Group’s experience in Rwanda. The lessons outlined may be of interest in helping to inform the design and implementation of Bank Group and other development practitioner support in country contexts that are similar to Rwanda’s. They may also be of interest to government authorities and other stakeholders in recipient countries. However, in considering the transferability of lessons, it is important to account for country-specific factors, including the strength of the government’s vision for development and leadership, the efficacy of the donor architecture, and cultural factors that have a bearing on social discipline and performance. Strong government leadership and discipline are critical determinants of development progress. Though it seems trite, this is a key takeaway from the Rwanda program and may place limits on the transferability of the Bank Group’s experiences in the country to other contexts. Strong government leadership at the most senior level has imbued a strong culture of performance and results, buttressed by the unique imihigo system of performance contract implementation and monitoring in which the country’s president takes an active part. These conditions have encouraged a systematic approach to development planning; proactive reform implementation; regular learning from evidence, including impact evaluation as well as other countries’ experiences; and effective coordination of donor support. This governance model has undoubtedly been a key driver of the country’s development gains and of its ability to make effective use of Bank Group support and ODA more broadly. However, as a uniquely “homegrown” model, it may not be easily replicable. In such a context, general budget support can be an effective means of supporting development. General budget support has generally fallen from favor in recent years, both in Rwanda and elsewhere. Yet in the early years of the evaluation period, multisector PRSF operations (with associated mechanisms for alignment and harmonization among development partners) offered a means of providing large sums of financing to Rwanda, at relatively low transaction cost, while encouraging steady implementation of the reform agenda. Reforms supported under the PRSF series and budget support from other partners coupled with investments and technical assistance (supported by World Bank IPF and other donors) were associated with important development gains in many areas. Sustained collaboration among staff is essential to realizing synergies across the Bank Group. The experience in the agricultural sector suggests that synergies depend on close cooperation and regular interaction between World Bank and IFC staff. The broad and in-depth sector knowledge of World Bank staff can play a crucial role in orienting IFC investments and AS projects, which in turn can make important contributions to advancing the sector reform agenda. Systematic integration of sector experiences is key to effectively tackling cross-cutting areas. The Rwanda program stands out for its generally good cross-sector collaboration. Nevertheless, there 52 Rwanda Country Program Evaluation, FY09–17 | Chapter 4 is scope for more systematic integration of sector experiences in Bank Group support for certain cross-cutting areas such as PFM, decentralization, capacity development, and gender. In PFM, for example, a more holistic view of the reform agenda in public procurement would take account of the power purchase agreements that have been entered into with IPPs. Similarly, sector-specific experiences in working with local governments could usefully inform a more systematic approach to supporting decentralization. The rapid agricultural modernization program points to the potential for widening inequality. In Rwanda, agricultural intensification policies have started to move surplus labor off the farm into the nonfarm economy, including in secondary towns and in Kigali. But as noted in IEG’s evaluation Growing the Rural Nonfarm Economy, these efforts are less likely to result in reduced poverty where high inequality exists and may even lead to the marginalization of disadvantaged groups (Negin et al. 2009; Thorbecke 2013). A World Development article by Dawson, Martin, and Sikor (2015) suggests that these policies may be exacerbating landlessness and inequality for the poorest. In countries with limited institutional capacity, basing senior task team leads in the field can help build policy-making capacity and ensure smooth project implementation. The Rwanda program experience suggests that a strong field presence in sectors where the Bank Group leads helps ensure steady engagement in policy dialogue and proactive participation in SWGs. During the early part of the evaluation period, nearly 60 percent of task team leads were based in the country office, but the share has since declined. In some sectors, such as agriculture and social protection, several stakeholders noted that the policy dialogue with the government and donors lost some intensity when the task team lead was no longer based in the country office because it became more dependent on field missions (despite proactive communication via email or calls by the task team lead). In the urban sector, some stakeholders attribute implementation delays to a lack of field presence, active task team lead follow-up notwithstanding. Recommendations In planning and implementing the next phase of Bank Group support, IEG recommends that the Rwanda country management team consider the following: ■■ M  aking systematic use of Rwanda’s government-led donor architecture in determining how the Bank Group can best respond to priorities arising from findings of the Future Drivers of Growth study. As part of this response, it would be appropriate to continue the selectivity of Bank Group engagement that has characterized the past several years, taking into account government demand and the comparative advantage of the Bank Group institutions as well as complementarity with other development partners. ■■  sing analytical work strategically to draw timely attention to insufficiently addressed binding U constraints or policy errors that will need to be overcome if Rwanda is to continue smoothly and sustainably on its path toward MIC and, eventually, high-income country status. The Future Drivers of Growth study is an important step in this direction, and the World Bank is encouraged Independent Evaluation Group | World Bank Group 53 to continue using knowledge work and policy dialogue to highlight key issues, sharing findings candidly with policy makers, stakeholders, and development practitioners. ■■  nsuring more systematic World Bank–IFC collaboration to realize synergies—as seen, for E example, in agriculture—more broadly across the Bank Group program, notably in energy, the financial sector, and the urban sector. ■■ E  nsuring that the Bank Group’s approaches to key cross-cutting subjects (for example, PFM, decentralization, or capacity development) systematically integrate sector-specific experiences. ■■  dopting a holistic rural-urban approach to ensure inclusive and sustained territorial transformation A that does not leave the poor behind or widen inequalities. 54 Rwanda Country Program Evaluation, FY09–17 | Chapter 4 bibliography Bardasi, E., and G. M. Garcia. 2016. Integrating Gender into IEG Evaluation Work. Washington, DC: World Bank. Castalia Strategic Advisors. 2009. Rwanda Electricity Access Programme. Volume I: Investment Prospectus. Wash- ington, DC: Castalia LLP. del Prete, D., L. Ghins, E. Magrini, and K. Pauw. 2018. “Land Consolidation, Specialization, and Household Diets: Evidence from Rwanda.” Discussion Paper 1728, International Food Policy Research Institute, Washington, DC. Dawson, N., A. Martin, and T. Sikor. 2015. “Green Revolution in Sub-Saharan Africa: Implications of Imposed Inno- vation for the Well-being of Rural Stallholders.” World Development 78: 204–218. Gatsinzi, M., R. Sabates-Wheeler, E. Wylde, and S. Yates. 2014. “Challenges of Measuring Graduation: Building Systems for Appropriate Identification of the Poor and Non-Poor.” Paper presented at the Social Protection Conference in Kigali, Rwanda, May 6, 2014. Rwanda, Government of. 2007. Vision 2020 Umurenge: An Integrated Local Development Program to Accelerate Poverty Eradication, Rural Growth, and Social Protection. EDPRS Flagship Program Document. Kigali, Rwan- da: Government of Rwanda. Rwanda, Environment Management Authority. 2015. “State of Environment and Outlook Report 2015 Rwanda. Greening Agriculture with Resource Efficient, Low Carbon and Climate Resilient Practices.” Kigali, Rwanda: Government of Rwanda. Rwanda, Government of. 2016. Public Expenditure and Financial Accountability Assessment 2015. Kigali, Rwanda: Government of Rwanda. Rwanda, Ministry of Finance and Economic Planning. 2014. Donor Performance Assessment Framework FY2013– 2014. Kigali, Rwanda: Government of Rwanda. IMF (International Monetary Fund). 2007. Structural Conditionality in IMF-Supported Programs. Independent Evalua- tion Office. Washington, DC: IMF. ———. 2017. “Rwanda: Selected Issues.” IMF Country Report 17/214, IMF, Washington, DC. Kojima, M., and C. Trimble. 2016. Making Power Affordable for Africa and Viable for its Utilities. Washington, DC: World Bank. Kondelys, F., and M. Jones. 2017. Feedback Matters: Evidence from Agricultural Services. Washington, DC: World Bank. Mason, K. 2015. “State-Backed Corporations vie to Dominate Rwanda’s Business Landscape.” Financial Times, April 23. https://www.ft.com/content/e42773a4-ded5-11e4-b9ec-00144feab7de. Negin, J., R. Remans, S. Karuti, and J. C. Fanzo. 2009. “Integrating a Broader Notion of Food Security and Gender Empowerment into the African Green Revolution.” Food Security 1 (3): 351–360. Oxford Policy Management. 2012. Assessment Report of Rwanda Second Support Rural Project. New York: Oxford Policy Management. Pavanello, S., P. Pozarny, A. P. de la O Caompos, and N. Warring. 2016. Research on Rural Women’s Economic Empowerment and Social Protection: The Impacts of Rwanda’s Vision 2020 Umurenge Programme. Rome: Food and Agriculture Organization. Research Moguls. 2016. “Mid-Term Assessment of The Implementation of the Financial Sector Development Pro- gram II (FSDP II).” Final Report. Kigali, Rwanda: Research Moguls. Independent Evaluation Group | World Bank Group 55 Thorbecke, E. 2013. “The Interrelationship Linking Growth, Inequality and Poverty in Sub-Saharan Africa.” Journal of Africa Economies 22 (suppl 1): i15-i48. United States, Department of State. “2013 Investment Climate Statement: Rwanda.” Bureau of Economic and Business Affairs, February 2013. https://www.state.gov/e/eb/rls/othr/ics/2013/204721.htm World Bank. 2008. “Country Assistance Strategy for the Republic of Rwanda FY2009–2012.” Report 44938-RW. World Bank: Washington, DC. ———. 2013. Youth Employment Programs. An Evaluation of World Bank and IFC Support. Independent Evalua- tion Group. Washington, DC: World Bank. ———. 2014. Rwanda—Country Partnership Strategy FY2014–18. Washington, DC: World Bank. ———. 2015a. Investment Climate Reforms: An Independent Evaluation of Bank Group Support to Reforms of Business Regulations. Independent Evaluation Group. Washington, DC: World Bank. ———. 2015b. Mixed Method Study, A Dialogue on the Welfare Trajectories of VUP Beneficiaries: Identifying Indi- vidual-level Drivers and Obstacles for Change. Washington, DC: World Bank. ———. 2015c. “Rwanda: Employment and Jobs Study.” Report 96504-RW, World Bank, Washington, DC. ———. 2015d. World Bank Group Support to Electricity Access, FY2000–2014. Independent Evaluation Group. Washington, DC: World Bank. ———. 2017a. Growing the Rural Nonfarm Economy to Alleviate Poverty. Independent Evaluation Group. Washing- ton, DC: World Bank. ———. 2017b. Reshaping Urbanization in Rwanda: Economic and Spatial Trends and Proposals. Note 1: Urban- ization and the Evolution of Rwanda’s Urban Landscape. Washington, DC: World Bank. ———. 2017c. “Rethinking Urbanization in Rwanda: from Demographic Transition to Economic Transformation.” Rwanda Economic Update, 11th ed., World Bank, Washington, DC. ———. 2017d. “Rwanda—First Programmatic Energy Sector Development Policy Financing.” Program Document Report 120612-RW, World Bank, Washington, DC. ———. 2017e. “Rwanda—Performance and Learning Review of the Country Partnership Strategy for FY14–FY18.” Report 106731-RW, World Bank, Washington, DC. ———. 2017f. “Rwanda Feeder Roads Development Project Impact Evaluation Report from the 2016 Follow-up Survey.”World Bank, Washington, DC. ———. 2017g. “Land Husbandry, Water Harvesting, and Hillside Irrigation (LWH) Project.” 2016 Survey Report, DIME-GAFSP, World Bank, Washington, DC. ———. 2018. Future Drivers of Growth in Rwanda: Innovation, Integration, Agglomeration, and Competition. Washington, DC: Bank Group. World Bank and RNAEB (Rwanda National Agriculture Export Board). 2017. Impact Evaluation of the Rwanda Green Leaf Tea Pricing Reform. Washington, DC: World Bank. WEF (World Economic Forum). 2017. The Global Gender Gap Report 2017. Geneva: WEF. World Food Programme. 2016. Rwanda 2015—Comprehensive Food Security and Vulnerability Analysis. Data Collected in April–May 2015. Rome, Italy: World Food Programme/VAM Food Security. 56 Rwanda Country Program Evaluation, FY09–17 | Bibliography The World Bank 1818 H Street NW Washington, DC 20433