97401 How Does Knowledge on Public Expenditures Integrate with the Design of Development Policy Operations? IEG Learning product May 27, 2015 This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without authorization. Contents ABBREVIATIONS ................................................................................................................................. III ACKNOWLEDGMENTS ......................................................................................................................... V OVERVIEW........................................................................................................................................... VII 1. INTRODUCTION........................................................................................................................ 1 2. THE APPROACH ...................................................................................................................... 3 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW ........ 5 Trends in the volumes, numbers, IBRD/IDA, and geography of DPOs ..................................................................... 5 Trends in public expenditure knowledge products .................................................................................................... 8 How does knowledge of public expenditure inform DPOs? .................................................................................... 11 4. SELECT FINDINGS FROM CASE STUDIES .......................................................................... 17 Examples of successful contribution of public expenditure work in the design of DPOs ........................................ 19 Examples of missed opportunities .......................................................................................................................... 23 5. CONCLUSIONS....................................................................................................................... 27 REFERENCES...................................................................................................................................... 29 APPENDIX A. COUNTRIES IN WHICH DPOS WERE PRECEDED BY A PER .................................. 31 APPENDIX B. LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS .................................. 37 APPENDIX C. THE ROLE AND INFLUENCE OF PUBLIC EXPENDITURE REVIEWS ON DPO DESIGN (TABLE 2 OF 2) ..................................................................................................................... 51 APPENDIX D. DPOS AND PUBLIC EXPENDITURE REVIEWS AT THE SUBNATIONAL LEVEL .... 57 Boxes Box 2.1. A Systematic Representation of the Focus (top row) and Analytical Approaches ..................... 3 i CONTENTS Tables Table 3-1 Top ten countries by commitments and number of operations (closed operations 2004-2013) ............................................................................................................................................................... 7 Table 3-2. Top ten countries with PERs, 2000-14 .................................................................................. 9 Table 3-3. Number and share of DPOs preceded by PER ....................................................................12 Table 3-4 Similarity index of DPOs and PER, using theme codes .........................................................15 Table 4-1 Integration of PERs with DPOs: Select findings from the case studies (Also see detailed table in Annex 3). ...........................................................................................................................................17 Figures Figure 3-1 Commitments and numbers of DPOs by lending instrument and region (closed DPOs, 2004- 2013) ...................................................................................................................................................... 6 Figure 3-2 Trends in IBRD and IDA DPOs and average commitments (closed DPOs 2004-2013) ........ 7 Figure 3-3 Distribution and trend of DPOs managed by Networks and Sector Boards (closed operations 2004-2013) ............................................................................................................................................. 8 Figure 3-4 Trend and regional distribution of public expenditure knowledge products (2000-2014) ......10 Figure 3-5 Percentage of PERs with specific themes (2000-2014); (n=191) .........................................11 Figure 3-6 Numbers and shares of DPOs with PERs within four years, by world region .......................13 Figure 3-7 Trends in the numbers and shares of DPOs with PERs .......................................................13 Figure 3-8 Comparison of the trend in performance of DPOs with/without a PER .................................14 Figure 3-9 Number of MTEF related prior actions (operations closed 2004-2013) ................................16 Evaluation Managers  Caroline Heider Director-General, Evaluation  Nick York Acting Director, Independent Evaluation Group, Public Sector  Mark Sundberg Manager, Independent Evaluation Group, Public Sector  Zeljko Bogetic Task Manager ii Abbreviations AFR Africa CEMs Country Economic Memoranda CPIA Country Policy and Institutional Assessment DPL Development Policy Loan DPOs Development Policy Operations ECA Europe and Central Asia ESW Economic and Sector Work IBRD International Bank for Reconstruction and Development ICR Implementation Completion and Results Report IDA International Development Agency IDPL Infrastructure Development Policy Loan IEG Independent Evaluation Group MDG Millennium Development Goal MENA Middle East and North Africa MOF Ministry of Finance MTEF Medium-Term Expenditure Framework OPCS Operations Policy and Country Services PEFA Public Expenditure and Financial Assessment PEMSP Public Expenditure Management Strengthening Programme PER Public Expenditure Review PFM Public Financial Management PPAR Project Performance Assessment Report PPP Public Private Partnerships PRSC Poverty Reduction Support Credit RETFs Recipient Executed Trust Funds SAR South Asia Region SPF State and Peace Building Fund iii Acknowledgments This IEG Learning Product was prepared by under the general direction and guidance of Mark Sundberg, Manager, Independent Evaluation Group (IEG). This paper is part of a series of Learning Products (LP) that primarily synthesize existing evaluative material with the aim of gaining new insight into the effectiveness and design of development policies and interventions supported by the Bank from an operational perspective. In this series the focus of the LP is on factors that influence design, policy implementation, and performance of Development Policy Operations (DPOs). This particular report aims to assess the extent of integration of knowledge on public expenditures with the design of DPOs using a combination of portfolio review, case studies, and descriptive statistical analysis to derive insights, lessons, and good practices that could inform the future design of DPOs. The database, analysis, and draft report were prepared by an IEG team consisting of Zeljko Bogetic (team leader), Javier Bronfman, Malathi Jayawickrama, Marko Klasnja, Aghassi Mkrtchyan, Moritz Piatti, Marcelo Selowsky, and Jesse Torrence. The team leader would like to express appreciation to Mark Sundberg for substantive guidance and advice in all stages of this study. Peer reviewer comments from Ismail Arslan, Pedro L. Rodriguez, Shahrokh Fardoust, as well as comments and discussions with Linda van Gelder, Ed Mount field, Manuela Francisco, Peter Moll, and Jasmin Choker are gratefully acknowledged. Much appreciated are important individual contributions of team members who provided building blocks of the complete analysis presented in the report. They are as follows: Moritz Piatti (management, improvements and revisions of the Development Policy Operation (DPO) and Public Expenditure Review (PER) databases based on the early database developed by Jesse Torrence); detailed portfolio analyses of both DPOs and PERs; the author of one case study, and working closely with the team leader and Marko Klasnja on statistical analysis); Malathi Jayawickrama (author of several case studies who also prepared draft synthesis of case studies related to public expenditures and DPOs analyzed in this paper), Marcelo Selowsky (author of several case studies), Javier Bronfman (author of several case studies, who also synthesized case studies for the accompanying paper on macro-fiscal framework), Aghassi Mkrtchyan (author of several case studies), Marko Klasnja (who developed the similarity index between PERs and DPOs and also supported data work and parts of the portfolio analysis), and Jesse Torrence (who developed an early version of database of DPOs and PERs). Team assistance by Aline Dukuze and Yezena Zemene Yimer is gratefully acknowledged. v Overview Integration of knowledge with lending is an enduring theme in World Bank strategies at the corporate and country levels. The Bank lending instruments could be more relevant and produce better results if they incorporate key analytical, country-level knowledge. The goal of this learning product is to contribute modestly to the understanding of how this knowledge-lending integration works in practice and raise questions for further research.1 It examines the extent to which an important type of Bank knowledge—on public expenditures—informs the design of Development Policy Operations (DPOs). It is hoped that this preliminary analysis can provide the IEG and the Bank with useful lessons as it evolves toward an integrated “solutions Bank” and the twin goals. The analysis is not just retrospective. It also seeks to inform the ongoing and future efforts toward integrating knowledge and lending, Systemic Country Diagnostics (SCD), and the design of DPOs. It also aims to inform the OPCS 2015 DPO Retrospective. Public expenditure knowledge for the purpose of this learning product, is identified as the body of Bank knowledge products specifically referred to in their titles as Public Expenditure Review, whether they are multi-sector or sector public expenditure reviews, but they also include other economic and sector work (ESW) which directly or mostly deal with public expenditure issues. This broad definition should capture much of the public expenditure-related analytical work at the Bank. Arguably, efficiency and allocation issues important in the design of DPOs and reforms they support can only be addressed by such integrative, not partial or silo- type knowledge of specific subsector expenditures. 1 This paper is part of a series of Learning Products (LP) that primarily synthesize existing evaluative material with the aim of gaining new insight into the effectiveness and design of development policies and interventions supported by the Bank from an operational perspective. In this series the focus of the LP is on factors that influence design, policy implementation, and performance of Development Policy Operations (DPOs). This particular report aims to assess the extent of integration of knowledge on public expenditures with the design of DPOs using a combination of portfolio review, case studies, and descriptive statistical analysis) to derive insights, lessons, and good practices that could inform the future design of DPOs vii OVERVIEW Six broad messages emerge from this review. First, the Bank has been producing a considerable volume of knowledge on public expenditure, which increased in recent years, and that knowledge was available to inform the majority of DPOs in the majority of countries with DPO operations. The analysis of the 390 DPOs and 404 public expenditure knowledge products (PERs, broadly defined), which were the subject of this review, suggests that there appears to be a solid degree of timeliness, availability, and thematic overlap as well as integration between PER knowledge and DPOs. There is also evidence of continuity of policy dialogue from PERs to DPOs in many cases. This is a reassuring conclusion. It implies that the country-based Bank model which integrates knowledge and lending is working well, in the case of PER knowledge and DPOs. There are good practices worth highlighting as well as weaknesses that offer opportunity for reflection and lessons for the future. Second, public expenditure knowledge informs DPOs largely in the areas of public sector governance, followed by social development, and human development. Macroeconomic and tax policies, in particular, tend to show comparatively smaller overlaps, but this may be because some of the macroeconomic policy agenda is taken up more directly in the macro-fiscal framework as elaborated in the accompanying report. There is a tentative question whether tax policy is an area in which the World Bank has “left the field,” perhaps to the national authorities and the International Monetary Fund (IMF), and whether that should be revisited. Third, deeper dives into case studies of recent DPOs show some good practices. These good practice examples of integration of PER knowledge and DPOs tend to show a continuity of policy dialogue, from PER to DPO, with DPO design directly and explicitly using PER policy recommendations in its prior actions. Such DPOs also tend to have stronger elements of DPO design such as macro-fiscal framework, as shown in the accompanying paper on macro-fiscal frameworks. And they often have better IEG ratings. Fourth, more broadly, good practice DPOs in terms of the integration of PER and DPO knowledge tend to be situated in an environment of long-term Bank engagement and continuous, high-quality policy dialogue (knowledge and lending) based on a relationship of trust. Sound PERs with candid recommendations hitting clearly on important issues stand a better chance of informing DPO and policy reform. Medium-Term Expenditure Frameworks (MTEF), budgeting, Public Financial Management (PFM) issues tend to be some of the common areas in which PERs substantially informed the DPOs, but the policy areas cover the entire spectrum of governance and other thematic areas. viii OVERVIEW Fifth, examples of missed opportunities in integrating PER knowledge with DPOs tend to reflect breaks in institutional memory on the Bank and the government, as well as breaks in the policy dialogue which can come about for many reasons (passage of time between the PER and the DPO, changes in staff on the Bank and the client side, changes in government and political priorities, etc.). One lesson here is that the Bank should pay greater attention to DPO-led reengagements with countries and new DPOs after a longer period of no core diagnostics on public expenditures or prior dialogue and DPOs, and ensure that all relevant Bank knowledge fully informs those DPOs. Sixth, there may be a potential knowledge gap and an opportunity for the Bank to intensify knowledge engagement on public expenditures at subnational levels, both as part of the knowledge and policy dialogue and as a contribution to future DPO development. Finally, in follow up to this learning product, which is focused on the limited questions around the integration of knowledge and DPOs, there is a need for a systematic analysis of the quality of the Bank’s AAA knowledge products. An evaluation of the public expenditure reviews, in particular, would potentially shed further light on how the Bank can continue improving the quality, relevance, and integration of its knowledge products. ix 1. Introduction 1.1 Integration of knowledge with lending is an enduring theme in World Bank strategies at the corporate and country levels. It rests on the widely shared proposition that Bank lending instruments could be more relevant and produce better results if they incorporate key analytical, country level knowledge. To be successful, development interventions must be informed by evidence, and evidence comes from knowledge. This is the idea behind Bank as a “solutions bank,” integrating financial instruments and knowledge products into “development solutions” that deliver results. A recent Independent Evaluation Group (IEG) evaluation suggests that the Bank’s broad economic and sector work and technical assistance strongly informs Bank lending strategies (IEG 2008). 1.2 The goal of this learning product is to help inform the understanding of how this integration works in practice and to raise questions for further research. Specifically, it seeks to examine the extent to which one, albeit important type of Bank knowledge--on public expenditures--informs the design of DPOs, a key lending instrument. This question is relevant primarily for the quality of design of DPOs and their performance. But it also has broader implications for how well the Bank combines knowledge and lending to influence policy, poverty, and distributional outcomes. Finally, it should be noted that this learning product does not evaluate the quality of the Bank’s knowledge on public expenditure per se; it is concerned, instead, with the integration of that knowledge into the design of DPOs.2 1.3 This is not the first IEG learning product focused specifically on the role of the Bank knowledge in development lending.3 It is also limited in scope in that it takes a fresh look at the interactions between one class of the Bank’s core diagnostic reports— 2 Only in the limited number of in-depth case studies, the team has looked at aspects of relevant PER quality. 3 Martin Ravallion has pointed out in recent paper (“Knowledgeable Bankers? The Demand for Research in World Bank Operations,” Policy Research Working Paper 5892, World Bank, December 2011 https://openknowledge.worldbank.org/bitstream/handle/10986/3674/WPS5892.pdf?sequence=1) that the knowledge of operational staff within donor agencies is likely to be important for the quality of development aid. He provides evidence that the stock of prior analytic work by the Bank on a recipient country is a strong predictor of the subsequent quality of its lending operations to that country (Deininger, Squire and Basu, 1998; Wane, 2004) and that the quality of prior analytic work matters to the quality of its projects (Fardoust and Flanagan, 2011). Other related studies analyze aspects of Bank core diagnostic knowledge. For example, Deolalikar (2008) reviews 60 PERs in 2000-07 and identifies common problems and solutions for greater impact and improved effectiveness and equity of public expenditure. An IEG Project Performance Assessment Report (2010) analyzes growth diagnostics in four African countries. A 2008 IEG evaluation of economic and sector work and technical assistance (“knowledge”) in the 2000-06 period finds that the Bank’s knowledge informs the Bank’s lending strategies and the clients even more than through the direct impact of knowledge on the clients. This suggests important synergies in Bank services between knowledge and lending. 1 1.INTRODUCTION the public expenditure review (PER), broadly defined—and a key lending instrument, and it is mainly focused on understanding the scope and sectoral and thematic nature of their integration. It is hoped that even preliminary answers to these questions can provide IEG and the Bank with useful lessons as the Bank evolves toward an integrated “solutions bank.” 1.4 PERs, for the purpose of this learning product, are identified as Bank knowledge products with a specific reference in their titles as Public Expenditure Review, whether they are multi-sector or single sector public expenditure reviews. They also include other analytical documents which deal with public expenditure issues, be they public finance reviews, studies of specific expenditure and debt issues, and even Country Economic Memoranda (CEMs) with a special focus on public expenditures. This broad definition should capture much (though not all) of the public expenditure-related analytical work at the Bank. Arguably, efficiency and allocation issues important in the design of DPOs and the reforms they support can only be addressed by such integrative, not partial or silo-type knowledge of specific subsector expenditures (e.g., tertiary education expenditure review). 2 2. The approach 2.1 The preliminary analysis in this paper combines several approaches (Box2.1). Box 2.1. A Systematic Representation of the Focus (top row) and Analytical Approaches Source: IEG. 2.2 The IEG team analyzed all DPO that were closed in the FY2004-13; a total of 390 DPO’s were analyzed. Also, it analyzed all Public Expenditure Reviews and closely related country diagnostics as per the Governance global practice database of public expenditure reviews (the total of 404 public expenditure knowledge products) during the period 2000-13. For the purpose of this analysis, it is assumed that PER knowledge products may inform a DPO if it precedes the DPO effectiveness by no more than four years. 2.3 The structure of the paper is the following: first, it provides elements of a portfolio review of Development Policy Operations (DPOs) insofar as they relate to the knowledge on public expenditure reviews. 2.4 Second, it documents basic but important metrics of the integration of Public Expenditure Reviews (PERs) and DPOs. For example, how many PERs (defined broadly) precede DPOs in the same and the previous three years in various countries? This indicates the timeliness and synchronization of PER knowledge and DPOs, and identifies those PERs that are most relevant for the design of corresponding DPOs. Essentially, this metric answers some basic questions: in how many and in which DPOs 3 2. THE APPROACH AND MAIN MESSAGES is there an analytical knowledge on public expenditures prepared by the Bank?4 What types of DPOs tend to be more informed by PERs? And do these DPOs show better integration of knowledge with their policy frameworks and greater success in implementation as measured by rating outcomes. 2.5 Third, the paper answers the question in which thematic and policy areas do PERs inform DPOs. Are there areas of particularly strong interest to DPOs? Are there policy areas that appear to be underrepresented given the policy agenda in many DPOs? Statistical tools measuring the degree of correspondence or similarity are also used to assess the thematic “proximity” of PERs and DPOs. 2.6 Fourth, for deeper insights into how this integration works in practice, the report synthesizes insights and lessons from 15 DPO desk case studies; this represents a third of the DPOs closed in FY11-13 and for which there are completed Implementation Completion Report (ICR) reviews. The case studies aim to distill some lessons from good and not so good practice that could inform better integration of PER knowledge and DPOs in the future. They include insights from the recent Project Performance Assessment Reports (PPARs) on long DPO series (Tanzania, Vietnam, and Uganda). . 4The paper considers only the World Bank-prepared public expenditure country focused diagnostics. It does not consider non-Bank knowledge products. 4 3. Knowledge on Public Expenditure and DPOs: A basic portfolio view 3.1 This section provides an overview of Development Policy Operations (DPOs), the World Bank's public expenditure knowledge work, and an assessment of how public expenditure knowledge products may have informed and contributed to DPOs. 3.2 This section is structured as follows: first it provides a review of the DPOs in the time period of 2004-2013. Second, it reviews the World Bank's portfolio of knowledge work on public expenditure in the same period.5 And third, the DPOs and public expenditure knowledge products are analyzed jointly to assess the extent to which DPOs may have been informed by Public Expenditure Review (PER)-type knowledge. Trends in the volumes, numbers, IBRD/IDA, and geography of DPOs 3.3 Data and method. For identification purposes, the detailed Bank project theme report (2c.2.1 lines) was downloaded and customized from Business Warehouse in August 2014. Only DPOs were retained from the database. Independent Evaluation Group (IEG) Implementation Completion Report Review (ICRR) outcome ratings were merged and matched with the DPO database. Observations for DPOs without an IEG ICRR were dropped. Based on this selection process, 390 DPOs were identified for the period of 2004-2013. Next, a database of prior actions was received from Operations Policy and Country Services (OPCS) matching 387 of the 390 DPOs identified through Business Warehouse. Thematic codes were further reviewed manually by the IEG team and broken down into a few subtopics to improve the granularity of the analysis of macroeconomic management issues in DPOs. Two main limitations apply to the underlying data. First, administrative data for a given project are typically recorded at an early stage of preparation, and the record is unlikely to be changed even if there may be significant changes later on. While the thematic focus of projects is typically expected to remain robust between preparation, concept, and board approval, this may lead to some inaccuracies when the project focus was significantly changed during preparation. Second, the number of observations for recent years is significantly lower, as there is a delay in the review of the closed projects. The reader should keep this in mind, 5The period of PERs data spans four years prior to that of DPOs to allow for an analysis of how many DPOs have been preceded by a PER. DPOs cover the period 2004-2013. 5 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW especially for the analysis of trend data which, on average, includes fewer observations in the last three years. 3.4 Volume, numbers, and regional distribution. Two Regions, Europe and Central Asia and Latin America and the Caribbean, have received most DPO lending commitments in terms of volume (almost $18 billion each). Most were International Bank for Reconstruction and Development (IBRD) countries. This is followed by East Asia and the Pacific which has proportionately more International Development Association (IDA) countries. Next is the Africa Region, with mostly IDA countries.6 3.5 In terms of number of DPOs, the Africa Region has more DPOs than any other Region, partly reflecting the large number of countries. The average size of a DPO (in terms of US$) in the Africa Region, however, is significantly lower than that of other Regions, reflecting the comparatively smaller size of African economies and their absorptive capacity (figure 3.1). Figure 3-1 Commitments and numbers of DPOs by lending instrument and region (closed DPOs, 2004-2013) Source: IEG DPO database. 3.6 The average lending amount for IDA countries has remained largely constant over time, but increased dramatically for IBRD countries in the late 2000s. This increase was a result of the global financial crisis when DPOs were used as a countercyclical instrument in IBRD countries. The reversal of these trends in 2012 and 2013 in the 6 There were two recipient-executed trust funds (RETFs). One in Rwanda (2010) and one Burkina Faso (2012), and eight DPOs (e.g. West Bank and Gaza and Burundi) using the state and peace building fund (SPF). These constitute a minor part of the portfolio, both in terms of numbers and volume. 6 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW figures may simply reflect the low number of observations (low number of operations closed and reviewed) and should be taken with caution (figure 3.2). Figure 3-2 Trends in IBRD and IDA DPOs and average commitments (closed DPOs 2004-2013) Source: IEG DPO database. 3.7 The top recipients of DPO financing by volume are IBRD countries. Most notably Mexico and Indonesia each received more than US$ 6 billion, followed by Poland and Turkey, which received close to US$ 5 billion each. The countries that received the largest number of DPOs were Pakistan (16 operations) and Colombia (14 operations) (Table 3.1). Table 3-1 Top ten countries by commitments and number of operations (closed operations 2004- 2013) # Top ten countries by Total Top ten countries by no of No of commitment commitment operations operations (US$ millions) 1 Mexico 6,916 Pakistan 16 2 Indonesia 6,250 Colombia 14 3 Poland 4,996 Mexico 13 4 Turkey 4,900 Morocco 13 5 Colombia 3,650 Vietnam 11 6 Pakistan 2,760 Indonesia 11 7 Vietnam 2,175 Bangladesh 10 8 Ukraine 1,701 Ghana 9 9 Morocco 1,638 Peru 9 10 Romania 1,514 Mozambique / Tanzania 8 Source IEG DPO database. 3.8 The majority of DPOs (70 percent) were managed by the Poverty Reduction and Economic Management (PREM) Network, followed by Human Development (HD) (12 7 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW percent) and the Sustainable Development Network (SDN) (10 percent). Within PREM, most DPOs were managed by economic policy. The Poverty (POV) and Public Sector (PS) practices each managed 12 percent of all DPOs. 3.9 There does not appear to be substantial variation over time regarding the networks managing DPOs. This may reflect, on the demand side, the thematic focus of reforms supported by DPOs, typically concentrated in economic policy and governance issues and, on the supply side, the Bank’s institutional tendency towards a relatively constant division of labor among networks. The notable reduction of PREM-led DPOs from 2011 onward, however, reflects the low number of closed and reviewed operations rather than a trend in the reduction of approvals (Figure 3.3). Figure 3-3 Distribution and trend of DPOs managed by Networks and Sector Boards (closed operations 2004-2013) Source IEG DPO database. Trends in public expenditure knowledge products 3.10 Methods of identifying PERs. This review draws on a data collection effort by the Public Sector Governance Global Practice (GP) on public expenditure knowledge products. All reports between 2000 and 2014 were drawn from a PER Business Intelligence and Image Bank query, supplemented by economic and sector work (ESW) that were not specifically classified as PERs but may also be focused on other aspects of fiscal and expenditure management, such as debt sustainability and macro-fiscal issues. Examples include (but are not limited to) public finance reviews, country economic memoranda, health expenditure and finance studies, and various policy notes with substantial public expenditure content. The Public Sector Governance GP cast the net 8 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW relatively wide so the total number of reports may appear large. Reports included are all those that have a unique identification number in Image Bank. Excluded are reports written in languages other than English, and supplementary volumes. All thematic codes of reports were reviewed, double-checked, and cleaned manually as necessary by the IEG team to allow for the interpretation of the data. 3.11 Volume, numbers, and regional distribution. Over the period of 2000-2014 there have been 404 knowledge products on public expenditures. The longer period is used compared to DPOs to account for linkages with PERs that preceded DPOs in 2004 by four years; those earlier PERs could, presumably, inform the design of DPOs three to four years later. There has been a steady upward trend in the number of PERs delivered over the years. The average number delivered between 2000 and 2009 was 22.3, which then increased to 36.2 per year delivered between 2010 and 2014. 3.12 The largest share of these knowledge products (35 percent) was produced in the Africa Region followed by Europe and Central Asia (27 percent) and East Asia and the Pacific (16 percent). South Asia and the Middle East and North Africa trail notably with five and four percent, respectively. There has been a concentration of PERs in a small number of countries with extensive Bank engagement (Table 3.2). Table 3-2. Top ten countries with PERs, 2000-14 Country Number of reports Indonesia 29 Tanzania 15 Tajikistan 13 Uganda 11 Ethiopia 10 Kyrgyz Republic 9 Philippines 8 Africa, region wide 8 Russian Federation 8 Ghana 7 Source: IEG DPO and PER Database. 3.13 The 404 reports were conducted across 127 countries. The top ten countries with the most reports are listed in table 3.2: Indonesia scores highest with 29 reports, followed by Tanzania (15), and Tajikistan (13). The number of reports has increased significantly from the late 1990s until 2010. The majority of PERs in the Africa Region may reflect the emphasis on basic public expenditure policy trade-offs, poverty, distributional and allocation issues, as well as the need to underpin DPOs by a public expenditure analysis in an environment of limited availability and quality of data 9 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW (figure 3.4). By contrast, in many IBRD countries, such data and analyses may be available on a more regular basis, including analyses by country authorities, perhaps obviating the need for full-fledged or frequent PERs. Figure 3-4 Trend and regional distribution of public expenditure knowledge products (2000-2014) Source: IEG review of Public Sector Governance PER monitor. 3.14 Most PER-type knowledge products reviewed were thematically coded for public sector governance (figure 3.5). Of the 404 reports, 191 were coded for 81 themes by the task team leader. Summarizing these into clusters, close to 90 percent contain public sector governance as thematic cluster. The social development, gender, and inclusion cluster was coded 35 percent of the time, followed by the human development cluster, which was coded in 18 percent of all reviews. In terms of the actual themes of interest in the present analysis, macroeconomic management was coded in 15 percent of documents, tax policy and administration in seven percent, and social safety nets7 in six percent of all reports.8 7 “Macroeconomic management” is included in the “economic management” cluster. “Tax policy and administration” is in the “public sector governance” cluster. And “social safety nets” is in the “social protection and risk management” cluster. 8 Note that reports may receive up to five themes, thus totals exceed 100 percent. 10 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW Figure 3-5 Percentage of PERs with specific themes (2000-2014); (n=191) Source: IEG review of Public Sector Governance PER monitor. 3.15 Importantly, these PER codes are the same theme codes used in the thematic coding of DPOs and prior actions. This allows a relatively accurate “matching” of themes between PERs and DPOs. The notable difference is that operations and ESW are coded by the task team leaders, while prior actions were retro-fitted with codes by OPCS manually. How does knowledge of public expenditure inform DPOs? 3.16 This section uses portfolio data on both DPOs and PERs to identify the overlaps and linkages between the two and ascertain whether PERs could potentially have played an informative role regarding resource allocation issues and policy frameworks in DPOs. Specifically, the analysis of the portfolio tells us how many DPOs were preceded by a PER within an adequate time period, and whether there was thematic overlap between the PER and the prior actions used by the DPO. An in-depth review of whether and how PERs have informed recent DPOs was taken up in the case studies (see chapter 4). 3.17 A total of 390 DPOs and 404 PER knowledge products were analyzed for potential thematic overlaps as follows: if a DPO had a public expenditure knowledge product within four years before DPO project effectiveness, it was considered relevant9. Using this rule, of the 390 DPOs, exactly 265 were preceded by a PER knowledge product, about 68 percent. DPOs were conducted in a total of 103 countries. In 75 (73 percent) of those countries, a knowledge product was potentially informative to the 9 Reports released five years or longer before DPO project inception were deemed too dated to credibly inform the operation, especially since reports use data from two or more years prior to dissemination. 11 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW operation. These numbers suggest that the Bank’s knowledge on public expenditures is highly relevant and could inform the design of operations in the majority of DPOs and countries under analysis. Table 3.3 provides a breakdown of these figures. Table 3-3. Number and share of DPOs preceded by PER Number Share Total No of DPOs 390 100% DPOs with PER 265 68% Total countries with DPOs 103 100% Countries w/ DPO and PER 75 73% Countries w/ DPO but w/o PER 28 27% Source: IEG assessment of DPO and PER data. 3.18 Another way of looking at these data is to view the countries with DPOs matched by recent PERs as those with relatively substantial policy dialogue. PERs are not just policy oriented research products. They are also a process of consultation, discussion, and dialogue that can inform policy reforms underpinning DPOs on a variety of public expenditure issues (e.g., subsidies, wage bill, public investments, budgeting and execution, pro-poor expenditures, equity). Neither PER-type knowledge products nor DPOs can be done without an explicit request from and collaboration with the country authorities—collaboration which evolves through a dialogue on policy reforms, their objectives, policies, trade-offs, and outcomes. And it is through this policy dialogue that knowledge products translate, directly or indirectly, into policy reforms and outcomes. 3.19 Further, while on average, 68 percent of DPOs were potentially informed by a PER, there is considerable regional variation. Europe and Central Asia has the most DPOs with PERs at close to 80 percent, followed by the Africa Region and East Asia and the Pacific, which have 76 percent each. On the other end of the spectrum is the Middle East and North Africa, where only 23 percent of all DPOs have a PER as an analytical underpinning (see figure 6). The Middle East and North Africa Region covers about eight percent of all DPOs, but only constitutes four percent of PERs (see figure 4). As indicated, these figures may also reflect not only the demand for the Bank’s knowledge and DPOs but also the relative breadth and strength of the policy dialogue on policy reform and public expenditure issues in the respective Regions. 12 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW Figure 3-6 Numbers and shares of DPOs with PERs within four years, by world region Source: IEG assessment of DPO and PER data. 3.20 There has been a downward trend over time in the number of DPOs preceded by a PER (Figure 3.7). Between 2004 and 2007, more than 70 percent of DPOs had a relevant PER as an analytical underpinning. This share declined since then to less than 60 percent since 2012. While observations of DPOs were few in 2012 and 2013, this trend was already pronounced in the years prior. Interestingly, there is a spike of DPOs with PERs in 2009-2010 when a number of DPOs were extended in the context of the global financial crisis. Apparently, many of the DPOs in those years had PERs available. Figure 3-7 Trends in the numbers and shares of DPOs with PERs Source: IEG assessment of DPO and PER data. 13 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW 3.21 The trend in the performance of DPOs with PERs (IEG outcome rated moderately satisfactory or higher) has been mixed. There has been a decline in IEG ratings of DPOs with PERs between 2004 and 2008, and in most of these years the performance has been slightly under that of DPOs without a PER. Since 2008, however, the performance of DPOs with a PER improved markedly so that in recent years DPOs with a PER outperformed those without by a small margin (figure 3.8). Figure 3-8 Comparison of the trend in performance of DPOs with/without a PER Source: IEG calculations using Business Warehouse data and the PER monitor. 3.22 Matching DPOs and knowledge products varies by “similarity” of themes. Another way to obtain a metric of thematic integration of PER themes with DPO policy content represented in DPO prior actions uses the statistical-descriptive concept of “similarity.” In other words, we ask the question: to what extent do policy themes/topics in the PER match those in the policy framework in the prior actions of DPOs? To this end, the IEG team developed a “similarity index,” constructed to show the extent to which the portfolio of DPOs match with the respective portfolio of PERs using the themes of DPO prior actions and the first theme (where available) of the PER. This similarity measure ranges between -1 (no similarity) and 1 (high similarity), with higher values indicating greater similarity.10 3.23 Using this approach, the match between DPOs’ policy frameworks in prior actions and PERs is generally quite high with an overall similarity measure of 0.69 (table 3.4) compared with 1.0 indicated “perfect” thematic similarity. Interestingly, 10The similarity index is based on the concept of Euclidian distance. The similarity measure employed is the cosine similarity, which measures the similarity between two vectors showing frequencies of thematic codes in DPOs and PERs. In this case, the prior action theme vector is the vector that counts the number of instances of each theme among all prior actions of a DPO. The PER theme vector does the same for all the themes of the matching PERs. In other words, these vectors capture theme frequencies. The measure then compares the distance between these vectors with smaller distances indicating greater similarity. 14 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW stand-alone operations are somewhat more closely related to the PER than programmatic DPOs, but the difference is small and both types of DPOs have quite high similarity indices.11 Table 3-4 Similarity index of DPOs and PER, using theme codes Similarity measure All 0.69 Stand-alone DPO 0.74 Programmatic DPO 0.52 Below-average size DPO 0.72 Above-average size DPO 0.45 Econ. Policy sector board 0.67 Other sector board 0.60 Source: IEG calculations. 3.24 A number of DPOs feature explicit medium-term expenditure framework (MTEF) content in prior actions. To ascertain such DPO-PER linkages at a more granular level, we also looked at some broad public expenditure policy areas relevant in many countries, such as medium-term expenditure frameworks. A relatively small fraction of DPOs analyzed explicitly mention the medium-term expenditure framework (MTEF) in their prior actions. Out of the 387 operations, 59 cited an MTEF at least once in a prior action, which is about 15 percent of total. In terms of the total numbers of prior actions, MTEF-related ones are much fewer, making up less than two percent of the total. In addition, there appears to be a downward trend of MTEF-related actions over the years, as shown in figure 3.9. This, however, could be interpreted in light of a general reduction in the number of prior actions and, possibly, progress with completion of past MTEF reform actions. Or, it could reflect a reduction in multi-sector operations with multiple MTEF related prior actions.12 11Further, DPOs with higher ICR review ratings appear less closely matched to PER analytical work than DPOs with lower ratings. This may, in part, be explained by a larger number of reports being produced in large countries, not all of which need to be aligned with a DPO and some of which were more aligned with non-DPO instruments (e.g., the count of PERs in Indonesia is 29 and not all can be expected to be related to the 11 DPOs —especially given that many sectoral reports were produced). in only one prior action was a public expenditure review explicitly mentioned, directly 12 Interestingly, linking the policy recommendation of the PER with DPO policy action. The 2007 LAO PRSC III operation required that the “[Ministry of Finance] approved an updated Public Expenditure Management Strengthening Programme (PEMSP) and implementation plan based on 2005/6 PER analysis and demonstrated ongoing progress with its implementation and capacity building .” Interestingly, that particular operation, as with some other DPOs in that country, scored highly on the macro-fiscal consistency index in assessing the quality of the macro-fiscal framework (See the accompanying report on the quality of macro frameworks in DPOs, IEG 2015). So this is the case of a DPO and country where there is clearly a high-quality macro-fiscal framework coupled with a relevant PER knowledge product that was directly and explicitly used in the design of the DPO policy framework. 15 3. KNOWLEDGE ON PUBLIC EXPENDITURE AND DPOS: A BASIC PORTFOLIO VIEW Figure 3-9 Number of MTEF related prior actions (operations closed 2004-2013) Source: IEG review of OPCS prior action database. Note: MTEF related prior actions are those that explicitly feature MTEF in their formulation, either as a process, document, or Government decision regarding the preparation or implementation of MTEF. 3.25 Finally, a very preliminary review of the portfolio or subnational DPOs and PERs (Appendix 3) for the period 2000-13 (PER) and DPOs (2004-13) suggests that those DPOs have taken place in a small number of large, federated countries.13 By far, the largest number of subnational DPOs in this period took place in only one country: Brazil. PERs and DPOs generally tend to overlap in a very small number of countries. Given the importance of the subnational fiscal and development agenda, this may suggest a potential knowledge gap and an opportunity for the Bank to intensify knowledge engagement at subnational levels, both as part of the knowledge and policy dialogue and as a contribution to future DPO development. 13There were other subnational DPOs in other countries in the period preceding 2004 (e.g., India, Pakistan). 16 4. Select findings from case studies 4.1 This section draws on 15 in-depth case studies (including programmatic series; 25 if counted as individual operations) that were written against a common protocol to provide an understanding of where and how knowledge on public expenditures has contributed to the design of Development Policy Operations (DPOs). Case studies were drawn from all Regions and a variety of DPO types that were closed in FY11-13 and had a completed Implementation Completion Report (ICR) review. Although the case studies cover a third of all DPOs meeting the periodicity and ICR review criteria, the intention is to maximize learning and gain in-depth insights rather than fully represent the portfolio regionally and globally. A summary of select findings of these case studies is provided in table 4.1. 4.2 Overall, in most of these cases, there is a Public Expenditure Review (PER) or a related product addressing public expenditure issues related to the content of the DPO. In one case where there is no recent PER—St. Lucia—there is a related poverty assessment closely thematically related to the social sector focus of the DPO. In the case of Brazil, there is no evidence of a PER or a related work, and in Albania, the PER was dated although related to the content of the DPO. Interestingly, in each of these cases where there is no recent DPO, the macro-fiscal frameworks had identifiable weaknesses (see the accompanying report on the macro-fiscal frameworks in DPOs). Table 4-1 Integration of PERs with DPOs: Select findings from the case studies (Also see detailed table in Annex 3). Country, DPL, Fiscal Year Coverage Was there a PER? What was Other Analytical Work (single the focus? (specific/general) on Public Expenditure tranche; Allocation series; sector; general) Albania Social Sector Reform DPL, One-Tranche No recent PER. Some intra-sectoral one-tranche, US$25m, FY11 Social Sector Extensive PER in 2006 analysis on shifting covered public debt, social expenditure insurance system, the health allocations to the poor finance environment including through targeting, but the health insurance institute no discussion of (HII). Although PER was dated, composition or issues of targeting in the social adequacy of protection system and need for expenditure HII reforms were relevant. allocations to social transfer programs and health. Benin PRSC 4-6, series, FY07-10 Series PER in 2004 focusing on social General expenditure trends in the same 17 4.SELECT FINDINGS FROM CASE STUDIES sectors of focus in the PRSC series. Brazil (State of Rio Grande Do One-tranche No PER or state specific social Sul), Fiscal Sustainability DPL, State only sector reports to guide DPO. US$1.1 billion, FY09 Cote d’Ivoire Post Reconstruction One-tranche PER in 2008. Emphasis on and Recovery Grant, US$150 General macro and fiscal issues, exp. million, FY12 and revenue policies, PEFA, and a review of Public Procurement System. Indonesia Infrastructure DPL 4 Series of 4 PER in 2007 Infrastructure (series of 4), amounts of Infrastructure Framework for Policy US$200m, 200m, 250m and 200m, Sector and Action, 2004 FY08-11 Latvia Safety Net and Social Series of 2 PER in 2007 (prepared for IMF Program Sector Reform Program (series of General crisis) addresses long-term 2). US$134.62m, FY10; (Global Crisis) PER 2010 (on social sector structural reforms in US$130.73m, FY11 2008-10 reforms) social safety net response) (SSN), pensions, state-owned enterprises Mexico Green Growth DPL, One-tranche PER in 2009 (on agriculture US$1.503 billion, FY09 Environment and rural development) sector PER in 2004 country-wide Peru Programmatic Fiscal Series of 4 PER in 2002 CEM, CFAA, CPAR, Management and General PER in 2007 Investment Climate Competitiveness DPL, US$200m, Assessment (ICA), 370m (plus supplemental DDO of Poverty Assessment, US$330m), 150m and 100m, FY07- Sector Studies, 11 Financial Sector Assess. Update, PEFA in 2009 Poland Energy Efficiency and One-tranche PER in 2010 Renewable Energy Development, Energy sector DPL, US$1.4 billion, FY11 Romania Financial Management Series PER in 2006 and Social Protection DPL 2 (in a General PER in 2010, in middle of series of 1-3, 2009-11), amounts of series as urgency due to global US$422m, 380m, 560m, FY11 crisis. St Lucia Economic and Social One-tranche No PER Poverty Assessment, DPL, US$12 million, FY10 General SSN Assessment, Tunisia Governance and One-tranche No PER Health Sector Study in Opportunity DPL, US$500 million, General 2006 FY11 Vietnam, Programmatic PRSC 10 Series PER in 2005 covered resource PEFA exercise in 2013 (in a series of PRSCs 6-10), General allocation, PFM challenges. to diagnose PFM US$152 million, FY10 issues. Source: IEG team review of case studies. 18 4.SELECT FINDINGS FROM CASE STUDIES Examples of successful contribution of public expenditure work in the design of DPOs 4.3 Some DPOs clearly show a high degree and relevant integration of PER knowledge into DPO policy frameworks and prior actions. We highlight those cases with emphasis on the specific policy areas and how they contributed to the formulation of DPO design. Among the case studies reviewed, three operations stand out as examples of good practice. Even there, however, there are areas where such integration could have been more pronounced, which is also highlighted. These were the DPOs on Romania, Peru, and Indonesia. 4.4 Romania. Financial Management and Social Protection Series (DPOs 1-3, closed FY11). This DPO series is a good example of incorporating PER recommendations and appropriately sequencing prior actions in the DPO series, and prior actions that are substantive and binding. 4.5 Romania’s PERs in 2006 and 2010 were relevant. The 2010 PER was in the middle of the DPO series and emphasized the urgency of the Bank’s response to the global crisis. The findings of the PER were directly used in the DPO policy areas and prior actions. Policy dialogue on the PER continued and intensified as policy dialogue on DPO reforms. 4.6 The main areas where the Romania PER recommendations were used in the DPO include the following:  Medium-term expenditure framework (MTEF): Financial management was improved under the DPO by explicitly introducing an MTEF. This moved from administrative action in the first DPO, to preparation and discussion of legislation in Parliament in DPL2. The prior actions became increasingly more binding, eventually calling for parliamentary approval of the MTEF in DPL3.  Public Sector Wages: The PER made recommendations for reforms in the structure of wages and the wage-setting process. Reforms in public sector pay, introducing more transparent mechanisms for adjustments in the structure of public sector wages, and merit increases were specific prior actions in the three operations.  Education: The PER suggested more flexibility in allocating teachers according to demand. The second PER, going further, recommended per capitation formulas. The prior action included a submission to Parliament followed by legislation in place in capitation formulas in eight counties. This reform, however, was slow in implementation. It could have been approved a year earlier, particularly as 19 4.SELECT FINDINGS FROM CASE STUDIES the population involved was small. The policy should have ideally been extended to the overall country by DPL2.  Health: Both PERs raised the need to address bias in the allocation of resources toward hospitals versus preventive and primary care (outpatient care). Here, the DPO does not feature strong follow up. But DPL 2 included a prior action requiring a decision to update a Hospital Rationalization Strategy.  Pensions: The PERs address the need to reform the system. The second PER is more forceful in flagging that the system is financially unsustainable. It also stresses the need to act on several fronts, from retirement age, privileges to certain groups, indexation, etc. The DPL includes a prior action on indexation reform. 4.7 Peru. This DPO series (Programmatic, Fiscal Management, and Competitiveness, closed FY11) represents a case of well-timed, several PERs, well integrated into the DPOs, a host of good economic and sector work (ESW) including a Public Expenditure Financial Assessment (PEFA), and good data and analysis which informed the sound design of a series of four DPLs. The operation also resulted in strong implementation. 4.8 Substantial PER-type work preceded the DPO. Peru had PERs in 2002 and 2007, and a PEFA in 2009, in preparation for the Programmatic Fiscal Management and Competitiveness DPL series of four loans in FY07-11. The first operation used the PER directly as a basis for its design. This ensured the continuity of the policy dialogue from the PER to the DPO. The third operation used the PEFA to inform its design (to inform the reform process and indicative triggers). All three studies appear to have informed country authorities and the Bank on the most important aspects of reform, and also provided them with data and timely analysis, which made it possible to formulate DPO prior actions and several indicators. 4.9 The DPOs selected policies on the basis of the analytical work that aimed to determine key policies with a large development impact. Substantial diagnostic work informed the design of the series. Some studies had been completed, while others were underway at DPO preparation. For example, a completed Public Expenditure and Financial Management Assessment and an (at the time) ongoing PER contributed to inform the policy dialogue in the areas of Public Financial Management (PFM) and fiscal transparency. Similarly, the Governance and Governability Nonlending Technical Assistance (NLTA) supported policy analysis related to results-based transfers to subnational governments, which underpinned the reforms supported by the series. On the competitiveness agenda, the Informality ESW addressed the question of how informality affects productivity and competitiveness of small and medium-sized businesses. Overall, the lesson here is that a substantial amount of timely and good 20 4.SELECT FINDINGS FROM CASE STUDIES analytical work formed the basis for this successfully-designed DPL series, with robust links in the results framework, and strong implementation. 4.10 As documented in the accompanying report, this operation also featured a high quality macro-fiscal framework, which contributed to the quality of the overall design and DPO rating. 4.11 Indonesia. Infrastructure DPO series, four operations, closed FY11. This sector DPO series was based on PER findings. It set high reform expectations and it pushed hard for sector reforms amidst challenges. 4.12 Indonesia’s comprehensive, multi-sectoral 2007 PER was timely. At the time, Indonesia was expected to have significant additional fiscal resources to address development needs following the post-crisis period. Although the macro-stability was established, public infrastructure spending had remained at only 2 to 3 percent of gross domestic product (GDP) for many years, well below the needs of a rapidly growing economy and population. There was concern that poor infrastructure—electricity shortages, road congestion, and low access to piped water and treated sewage—had reduced the quality of life and the environment, and would ultimately hamper growth. A major Bank study (2004) had analyzed the problem of infrastructure needs.14 The PER focused on how to build on prior achievements and spend additional public resources effectively and efficiently to improve the quality of social services and close the infrastructure gaps. 4.13 The Indonesia DPL (I-DPL) Policy Matrix was clearly linked to the PER and the 2004 study. The four policy areas of the DPL series (national, subnational spending on infrastructure; private sector financing through public-private partnerships (PPP); and cross-cutting areas of land, environment, and fiduciary), and prior actions covered several key policy proposals. This included expanding the supply of electricity to unserved areas in a cost effective and equitable manner; reducing inefficient and pro- rich fuel subsidies (one of the largest spending items on the budget); and reallocating inefficient and pro-rich electricity subsidies from consumption towards connection to encourage expansion of the electricity network (also recommended in the related Social Impact Assessment). 4.14 Despite the long-standing tensions and turmoil over the policy direction in the power sector, the inclusion of high profile outcomes (greater access and better subsidy 14 Indonesia: Averting an Infrastructure Crisis, A Framework for Policy and Action, 2004. 21 4.SELECT FINDINGS FROM CASE STUDIES targeting) in the policy matrix raised expectations for policy changes.15 Sector policy dialogue was substantive, but ultimately implementation proved difficult. The policy of providing electricity at rates below cost recovery was in direct conflict with the need to increase the supply of and access to electricity at a time of fiscal restraint. Existing consumption subsidies favored the wealthier over the poor. The lack of legislative action to change tariff policy during the first three operations prevented the reform of tariff pricing and limited progress on improving reliability and access to the power supply. 4.15 In hindsight, the Bank overlooked some key factors—that parliamentary approval was required to change below-cost tariffs and electricity prices and that powerful vested interests could block the approval of the law.16 Although the government began to revisit the electricity law to remove the monopoly power of the state-owned electricity company in electricity supply and to allow regional governments to set electricity tariffs, lobbying by some industry groups prevented tariff increases. However, the IDPL continued to push for power sector reforms. Although reforms did not occur during the first three IDPLs and performance targets were not achieved, there was a final breakthrough in the end. 4.16 In mid-2010, during the last IDPL, revisions in electricity tariffs broke the long- standing taboo and made a major reform step in the right direction. Simulations of the social impact of the draft tariff scheme suggest that the top decile’s household consumption share of the electricity subsidies will fall, while the bottom decile increases their share—indicating a more pro-poor tariff structure. Proposed tariff increases would not affect the poorest as they are concentrated in the lower capacity bandwidths. Analysis also shows that the lowest capacity users represent the largest share of residential users and the prospective subsidy changes in 2010 would shift the subsidy emphasis from industry and business to low capacity households. This would suggest a progressive tariff and subsidy scheme. Reforms supported by the series, despite delays and difficulties, should play a role in extending access and reducing regressivity of tariffs in Indonesia. 15Official household access to electricity was 55 percent in 2005 (PER, 2007). Residential consumers who were connected at less than 900 watts received only 41 percent of the State Electricity Company (PLN) subsidy. 16The government provides a subsidy to cover the difference between its costs and regulated tariffs. A key part of the subsidy is received by residential consumers who are unlikely to be poor (with connections greater than 900 watts) and by large industrial and commercial consumers. 22 4.SELECT FINDINGS FROM CASE STUDIES 4.17 This is an example of difficult, well-designed DPO reforms informed by PER analysis that took time to deliver results. It would have been expedient to drop or soften tariff reform from the reform agenda. But the Bank and the authorities’ persistence in the face of difficulties paid off. 4.18 Finally, another lesson is criticality of complementary AAA on public expenditure to drive implementation effectiveness, including political economy analysis, poverty assessment, CEM, which is possible for a large, active Bank client country. Examples of missed opportunities 4.19 In contrast to the above good practice examples, in some DPOs, we identified missed opportunities in the integration of PERs with DPOs. For example, public expenditure knowledge that existed but was not used in the DPO dialogue and design. Or difficult issues in the DPO that could have been resolved and for which there existed a knowledge base, but they remained unaddressed. We highlight, in particular, the operations in Latvia, Albania, Vietnam, and Benin. 4.20 Latvia. Social Safety Net and Social Sector Reform Program, series of two DPOs, closed FY11. A missed opportunity here can be summarized with a question: Could “indicative triggers” have been revised and new actions introduced following Board approval of the first operation in a series if there is compelling new information or a change of circumstances? 4.21 Latvia had two PERs in 2007 and 2010. The 2007 PER had limited overall relevance to the DPL as it had focused on the long-term agenda of improving public efficiencies in general. The subsequent global crisis resulted in major budget cuts in 2009. The Safety Net and Social Sector Reform DPL (FY10) commissioned a PER in the same year to provide analytical underpinning, and to specifically focus on social sector reforms in an environment of a substantially reduced budget envelope. The study scrutinized the ongoing reforms on social protection, health, and education; provided a thorough analysis of challenges in social protection, health, and education; and provided a menu of options for policy action for the medium term. However, as policy actions for the (second) DPL were already designed during first DPL (as indicative triggers) — there was little flexibility to incorporate new actions and recommendations arising from the PER. 4.22 As a result, some of the potentially important reform areas that came out of the 2010 PER that were not included in the first operation did not become reform actions under the second program. For example, the 2010 PER suggested reforms in family state 23 4.SELECT FINDINGS FROM CASE STUDIES benefits that were not well targeted but constituted 37 percent of the social protection budget as of 2010. The PER also suggested a number of reforms in social programs to redirect resources from the nonpoor to the poor. However, this did not become part of the reform program either. There was a de facto break in the continuity of the policy dialogue from the PER to DPOs. And a missed opportunity to improve the composition of public expenditures towards the poor at a time of general budget austerity. 4.23 Albania. Social sector reform DPL (standalone, closed in FY11) is a DPO that does not use or mention a relevant PER. An in-depth PER in December 2006 covered the same areas as the Social Sector Reform DPL in April 2011, and the PER provided a thorough analysis on key social sector reforms. However, there is no mention of the PER in the DPL program document. There is also no indication that PER recommendations informed the DPO design. It is true that the PER was completed over four years before the DPO and perhaps this and the associated break in the policy dialogue may partly explain how the PER fell from the DPO radar screen. Personnel changes in both the government and the Bank during this time also may have contributed to the apparent loss of institutional memory. But the PER content was highly relevant to the DPO and as such it should have been consulted. 4.24 The December 2006 PER may be dated; however, it covered key aspects and issues in the social sector, which were relevant at the time of the DPL. For example, it included the Ndihma Ekonomike Social Assistance Program, equity and efficiency of health sector expenditures (including the Health Insurance Institute), education funding levels and relative efficiency, and the pension program’s challenges. These overlap with some of the main areas and objectives supported by the subsequent DPL, including, policy changes to: improve the effectiveness of social safety nets and enhance the efficiency and equity of health spending in a fiscally sustainable environment. Specific policies under the DPO aimed at strengthening mechanisms to allocate funds and select beneficiaries in the main social assistance program and enhancing the efficiency and equity of health spending. Against this backdrop, the 2011 DPO program document fails to mention the PER and its findings. The only reference to the “PER - Public Expenditure Review” is in the list of acronyms. 4.25 Vietnam: The programmatic PRSC series, PRSC6-10 incorporates the critical recommendations of a PER in its DPO policy matrix at entry. However, integration proved of PER with the initial design of the PRSC was solid, it proved ineffective as the political economy constraints to implementing the program were overlooked (IEG PPAR, 2015, Vietnam PRSC6-10). 4.26 There was an “award-winning” PER in 2005, a major exercise that covered the allocation of public funds and critical areas of public financial management (PFM). The PRSC series included 10 of the key recommendations from the PER and the 2007 Vietnam Development Report in its design. Public expenditure and PFM were key 24 4.SELECT FINDINGS FROM CASE STUDIES policy areas in the series. Of the four prior actions in the first PRSC, three were public expenditure/PFM and focused on the critical areas of improving independence of the State Audit of Vietnam (SAV) and strengthening debt management. There were 10 indicative triggers in subsequent PRSCs 7-10. These were also critical to improving PFM and derived directly from the PER. So what was the problem? 4.27 The policy matrix changed significantly as the dialogue and the series progressed, introducing problems. The relevance and strength of policy actions declined substantially. Key policy actions envisaged at entry in PRSC 6 were never incorporated into the final policy matrix of the PRSC operations. In particular, proposed key policy actions on integration of the MTEF into the budgeting process, reducing off- budget expenditures, and introducing international standards in financial statements were dropped from subsequent PRSC operations, ostensibly because of the lack of reform momentum. The PRSCs continued even as weakening reform performance was reflected in the Country Policy and Institutional Assessment (CPIA) subcomponents on fiscal management and PFM, which declined during the relevant period 2006-2013. Why did the Bank progressively drop the policy actions in the DPOs? It is a question for reflection within a broader context and discussion of the dialogue and Bank strategy in that country. What is clear is that PRSC disbursements were made irrespective of the clear weakening of the otherwise sound policy framework that was well integrated with relevant analytical work. Resource transfer considerations dominated reform objectives. 4.28 The Vietnam Public Investment Reform (PIR) series was implemented in FY 09- 12 in parallel to the PRSCs and it took on some of the issues identified in the PER. The PIR was to support reforms in public investments (identified as a major issue in the PER). This had become a larger issue by 2009, due to problems in the expenditure composition that neglected operations and maintenance expenditures. Although the initial design of the PIR series included the law on public investments (to address this issue) as an indicative trigger, it was dropped from the policy matrix and never became a prior action. 4.29 The PIR series did not achieve any major results in the area of public investments that were highlighted in the PER. The 2005 PER suggested frequent subsequent PERs; however, the authorities indicated no interest in follow up PERs in the period 2008- 2013. There was apparently no interest or political will to delve into the diagnostics and causes of expenditure inefficiencies and resulting recommendations. The 2013 Public Expenditure Financial Assessment (PEFA) revealed that many of the challenges identified in the 2005 PER remained. Despite the PIR, improvements in the public expenditure policy since 2005 have not been substantial, except in the areas of PFM information systems, public debt management, and state audit. 25 4.SELECT FINDINGS FROM CASE STUDIES 4.30 In sum, Vietnam is a case of strong public expenditure AAA, which was well integrated into the initial design of the PRSC, but which produced limited results because of weak implementation and ownership. 4.31 Benin. PRSCs 4-6, closed FY10. The DPO series aimed to improve Millennium Development Goal (MDG) outcomes through increasing the efficiency of public expenditures in water and sanitation, health, and education. A PER 2.5 years before included an in-depth analysis of trends and policy issues in these three sectors critical for the MDGs. Yet the PRSC program documents make no reference to the PER. The DPO policy framework also does not take into account its analysis and recommendations. 4.32 The 2004 PER focused on social expenditure trends in water and sanitation, health, and education, the same areas of focus in the PRSC series. However, the PRSC 1 does not contain any quantitative analysis of the composition and direction of public expenditures, how they evolved over time, and how well they targeted areas of relevance to the MDGs, especially those MDGs where Benin was lagging. One would have expected the first PRSC operation to at least have updated, main budgetary information that was presented in the PER. But the PRSC makes no references to the PER. This is a serious omission considering the PER’s sound analysis and candid assessment that “the share of expenditures in five major health programs is driven by the availability of foreign funds rather than by national priorities; some important causes of child mortality and morbidity are not addressed,” and the fact that a PRSC series explicitly focuses on improving MDG outcomes. 4.33 Other issues raised in the PER that were missed but could have strengthened the policy framework and results of the PRSCs are:  First, despite improvements, infant and child mortality remain high. Very little progress was achieved with maternal mortality and access to sanitation in rural areas. There are substantial differences across income groups in both access to education and child mortality.  Second, special budget lines have increased significantly relative to the spending by health administrators at the regional and central levels. Major health programs are assessed as driven by the availability of foreign funds, rather than by national health priorities. This is a broader indication of the misalignment of the donors’ objectives from those of the government, as well as a missed opportunity to address priority areas such as child mortality and morbidity.  Third, a similar issue is raised in the water sector where the PER notes that “[i]investment in water point is focused in a few regions, due to donor preferences.”  Did the Bank raise the issue of apparently different priorities between the government and donors, at least informally, during the preparation of the series? It is not clear based on the document review. However, it is hoped that these issues were discussed with donors, which is a key point of the Paris Declaration. 26 5. Conclusions 5.1 Six conclusions emerge from this review.  First, the Bank has been producing a considerable volume of knowledge on public expenditure, which increased in recent years, and that knowledge was available to inform the majority of DPOs in the majority of countries with DPO operations. The analysis of 390 DPOs and 404 public expenditure knowledge products (PERs, broadly defined), which were subject of this review, suggests that there is a solid degree of timeliness, availability, relevance, and thematic overlap as well as integration between PER knowledge and DPOs. Most DPOs (some 68 percent) had PERs prior to DPO effectiveness. In most countries with DPOs (73 percent), there are timely PERs preceding DPOs. There is also evidence of continuity of policy dialogue from PERs to DPOs in many cases. This is a reassuring conclusion. It implies that the Bank’s country-based model, which integrates knowledge and lending, is working well in the case of PER knowledge and DPOs. There are good practices worth highlighting as well as weaknesses that offer opportunity for reflection and lessons for the future.  Second, public expenditure knowledge informs DPOs largely in the areas of public sector governance, followed by social development and human development. Macroeconomic and tax policies, in particular, tend to show comparatively smaller overlaps, but this may be because some of the macroeconomic policy agenda is taken up more directly in the macro-fiscal framework as elaborated in the accompanying report. There is a tentative question whether tax policy is an area in which the Bank has “left the field,” perhaps to the national authorities and the International Monetary Fund (IMF), and whether that should be revisited. Also, the decline in the frequency of MTEFs in the policy content of DPOs raises questions of the current state and the future of these medium-term reforms.  Third, deeper dives into case studies of recent DPOs show some good practices. The DPO series in Peru and Romania are such examples. These good practice examples of the integration of PER knowledge and DPOs tend to show a continuity of policy dialogue, from PER to DPO, with the DPO design directly and explicitly using knowledge on public expenditures and policy recommendations in its prior actions. Such DPOs also tend to have stronger, other elements of DPO design such as macro-fiscal frameworks, as shown in the accompanying report on macro-fiscal frameworks. They often show better Independent Evaluation Group (IEG) ratings. There is a “virtuous circle,” from 27 5. CONCLUSIONS knowledge informing the design of lending, and lending based on knowledge producing results.  Fourth, more broadly, good practice DPOs in terms of the integration of PER and DPO knowledge, tend to be situated in an environment of long-term Bank engagement and continuous, high-quality policy dialogue (knowledge and lending) based on a relationship of trust where there are no taboo topics for policy dialogue. Sound PERs with candid recommendations hitting clearly on important issues stand a better chance of informing DPO and policy reform. MTEF, budgeting, and PFM issues tend to be some of the common areas in which PERs substantially informed DPOs, but the policy areas cover the entire spectrum of governance and other thematic areas. However, even in cases of long-term engagement and solid initial integration between knowledge and DPO design, problems with political economy, ownership of reform and shifting policy priorities can undermine impact (as in the case of Vietnam).  Fifth, examples of missed opportunities in integrating PER knowledge with DPOs sometimes reflect breaks in institutional memory in the Bank and the government, as well as breaks in the policy dialogue which can come about for many reasons (passage of time between the PER and the DPO, changes in staff on the Bank and the client side, changes in government and political priorities, etc.). One lesson here is that the Bank should pay greater attention to DPO-led reengagements with countries, and new DPOs after a longer period of no core diagnostics on public expenditures or prior dialogue and DPOs, and ensure that all relevant Bank knowledge fully informs those DPOs.  Sixth, there appears to be a potential knowledge gap and an opportunity for the Bank to intensify knowledge engagement on public expenditures at subnational levels, both as part of the knowledge and policy dialogue and as a contribution to future DPO development. This will vary by country depending on the policy spcace to engage national authorities on subnational reform.  Finally, in follow up to this learning product, which is focused on the limited questions around the integration of knowledge and DPOs, there is a need for a systematic analysis of the quality of the Bank’s AAA knowledge products. An evaluation of the public expenditure reviews, in particular, would potentially shed further light on how the Bank can continue improving the quality, relevance, and integration of its knowledge products. 28 References Deininger, Klaus & Squire, Lyn & Basu, Swati (1998). "Does Economic Analysis Improve the Quality of Foreign Assistance?," World Bank Economic Review, World Bank Group, vol. 12(3), 385-418, September. Deolalikar, Anil B. (2008). “Lessons from the World Bank Public Expenditure Reviews, 2000 -07, for improving the effectiveness of public spending,” Transparency and Accountability Project (TAP), Brookings Institution, Washington D.C. Fardoust, Shahrokh and Ann Elizabeth Flanagan (2013). “Quality of Knowledge and Quality of Financial Assistance: Quantitative Assessment of the World Bank Experience in 2000-06,” unpublished draft paper (May). IEG (2008). Using Knowledge to Improve Development Effectiveness, An Evaluation of World Bank Economic and Sector Work and Technical Assistance, 2000-06, IEG/World Bank, Washington D.C. IEG (2010). World Bank Economic Reports on Growth Diagnostics in Four African Countries: Ghana, Nigeria, Mauritius, and Uganda, Performance Assessment Report (June). IEG (2011). World Bank Support for Revenue Policy Reform in Eastern Europe and Central Asia with Performance Assessment Reports of ESW in Georgia, Kazakhstan and Kyrgyz Republic, IEG. IEG (2015). “The Quality of Macro-Fiscal Frameworks in Development Policy Operations,” IEG learning product, Independent Evaluation Group, The World Bank, Washington D.C. (May). IEG PPAR (2015a). Vietnam: Project Performance Assessment Report of PRSC6-10, IEG (May). IEG PPAR (2015b). Uganda: Project Performance Assessment Report of PRSC5-7 and PRSC8-9 series, IEG (May). Ravallion, Martin (2011). “Knowledgeable Bankers? The Demand for Research in World Bank Operations, Policy Research Working Paper 5892, World Bank, (December). 29 APPENDIX A COUNTRIES IN WHICH DPOS WERE PRECEDED BY A PER HEADING Appendix A. Countries in which DPOs were preceded by a PER Region Country Year Project Name Proj ID Commitment (US$ million) AFR Benin 2007 BJ-PRSC 3 P083313 30 BJ-PRSC 4 (New Series) (FY07) P096928 40 2005 BJ-PRSC II P074313 30 Burkina Faso 2012 BF - EFA/FTI Grant (3rd tranche) P127166 35 2006 BF-PRSC 6 P078996 60 2005 BF-PRSC 5 DPL (FY05) P078995 60 Burundi 2010 BI - ERSGIII-Dev. Policy Loan DPL3 P113235 25 2009 BI - Food Crisis Response DP Grant P113438 10 BI- ERSG II DPL (FY09) P102508 30 Cabo Verde 2009 CV-PRSC 4 -DPL P106502 10 2007 CV-PRSC 3 (last of 1st series) P100807 10 2006 CV-PRSC 2 (FY06) P090875 10 Central African 2011 CAR: EMGRG III DPO IV (FY11) P120534 8.8 Republic 2009 CAR: EMGRG II - DPO III (FY09) P113176 5 2008 CAR: EMGRG 1 - DPO II (FY08) P106458 7.9 Congo, Dem. 2006 DRC - Transitional Support Credit (DPL) P091990 90 Rep. Cote d'Ivoire 2012 CI-Post-conflict Reconst. & Recov. Grant P122800 150 2010 CI-EGRG-Econ. Gov. & Recovery 3 P117281 90 2009 Governance & Recovery Grant II P112368 150 2008 Econ. Governance& Recovery Grant P083583 308 Ethiopia 2005 ET-PRSC 2 DPL (FY05) P077749 130 2004 ET-PRSC 1 (FY04) P074014 120 Gambia, The 2009 GM-Budget Support -DPL P107398 7 Ghana 2010 GH:Natural Resources Env Governance P118188 10 DPO3 2009 GH-EGPRC (fast-track) P113301 300 GH-NREG DPO P113172 10 2008 GH-Environmental Governance (FY07) P102971 20 GH-PRSC 6 DPL (FY08) P103631 100 2007 GH-PRSC 5 DPL (FY07) P099287 110 2006 GH-PRSC 3 DPL (FY06) P078619 125 GH-PRSC 4- DPL P095730 140 2005 GH-PRSC 2 DPL (FY05) P083246 125 Guinea-Bissau 2011 Third Economic Governance Reform EGRG P123685 6.4 3 2010 GW-Econ. Gov. Ref. Grant - EGRG II P114937 6 2009 GW-Economic Governance DPL1 P107493 8 Liberia 2009 LR - RRSP2-Budget Support P113450 4 Madagascar 2008 MG-PRSC 4 - 1st of New Series (FY08) P099420 40 MG-PRSC 5 (FY08) DPL P105135 50 31 APPENDIX A COUNTRIES IN WHICH DPOS WERE PRECEDED BY A PER 2007 MG-PRSC III DPL (FY07) P096102 40 2006 MG-PRSC 2 DPL (FY06) P083326 80 2005 MG-PRSC 1 DPL (FY05)/ P070999 125 Malawi 2013 Malawi - Rapid Response DPG P126155 50 2010 MW-PRSC 3 P117238 54 2009 MW - PRSC 2 P107303 30 2008 MW-PRSC-1 P099313 20 2004 MW-FIMAG SAL (FY04) P072395 50 Mali 2011 ML-PRSC 5 - DPL P122483 70 2010 ML-PRSC 4 - DPL P117270 70.5 2009 Mali - PRSC-3 P113451 65 2008 ML-PRSC 2 (FY08) P103466 42 Mauritius 2008 MU-Dev.Pol.Loan (intermediate) P106650 30 2007 MU-Development Policy DPL (FY07) P101570 30 Mozambique 2012 Moz PRSC VIII P126226 110 2011 MZ-PRSC 7 (second in series) P122470 85 2010 MZ-PRSC 6 (first in new series) P117234 110 2007 MZ-PRSC 3 DPL (07) - 1st in new series P083459 70 2006 MZ-PRSC 2 (FY06) P056201 120 2005 MZ-PRSC (FY05) P075805 60 Namibia 2009 NA-Support of ETSIP1 DPL2 (FY09) P109333 7.5 2007 NA-Education & Training DPL (FY07) P086875 7.5 Niger 2007 NE-Rural & Social Policy DPL 2 (FY07) P098963 50 2006 NE-Rural & Social Policy Reform I (FY06) P096411 50 2005 NE-Pub Expend Reform Crdt (FY05) P083275 40 2004 NE - PEAC II - (Pub.Exp.Adj.Cr.) P069570 65 Nigeria 2010 Fin Sec + Pub Fin Mgmt DPC P117088 500 Rwanda 2010 RW:EFA-FTI Catalytic Fund Bridge Grant P115816 35 2007 RW-PRSG III DPL (FY07) P098129 50 2006 RW-PRSC 2 DPL (FY06) P092944 55 2005 RW-PRSC 1 DPL (FY05) P085192 65 Senegal 2009 SN-DPO fast-track-Public Fin. Support Cr P107288 60 2008 SN-En. Sec. Recov. Dev Policy Financing P105279 80 2007 SN-PRSC III DPL (FY07) P098964 20 2006 SN-PRSC 2 (FY06) P091051 30 2005 SN-PRSC 1 DPL (FY05) P074065 30 2004 SN-Priv Sec Adj Crdt (FY04) P080013 45 Seychelles 2011 SC-DPL 2 P120947 9 2010 SC DPL P114822 9 Sierra Leone 2010 SL:GRGC-3 Suppl.Credit (Crisis Window) P121056 7 SL-GRG 3 DPL P107335 10 2007 SL-GRGG (DPG 1)-Programmatic Series P095575 10 2005 SL-ERRG 4 (FY05) P083477 15 Tanzania 2011 TZ-PRSC 8 (5th and last in 2nd series) P116666 115 2009 TZ-PRSC 6 (3rd of 2nd series) P101229 160 TZ-PRSC 7 (4th in 2nd series) P101230 190 2007 TZ-PRSC 5 (2nd of 2nd series) P095657 190 2006 TZ-PRSC 3 (last) P087256 150 32 APPENDIX A COUNTRIES IN WHICH DPOS WERE PRECEDED BY A PER TZ-PRSC 4 DPL (1st of 2nd series) P095509 200 2005 TZ-PRSC2 (intermediate) P074073 150 2004 TZ-Education Dev. Program II (FY04) P083080 150 Uganda 2008 UG-PRSC 7 (FY08) P101231 200 2007 UG-PRSC 6 DPL (FY07) P090219 125 2006 UG- PRSC 5 P090881 135 2005 UG-PRSC 4 (FY05) P074082 150 2004 UG-PRSC 3 (FY04) P074081 150 Zambia 2012 Zambia PRSC-3 P126349 30 2011 ZM-PRSC 2 P117370 30 2010 ZM-First Poverty Reduction Support Cr. P107218 20 2008 ZM-Econ Mgmt & Growth Crdt 2 P074445 10 2005 ZM-Econ Mgmt & Growth Credit (FY05) P040631 40 EAP Cambodia 2010 KHSmallholder Ag & Social Protection Sup P117203 5 2008 KH-Poverty Reduction and Growth-1 (DPL) P071103 15 Indonesia 2011 ID - Fourth Infrastructure DPL (IDPL4) P118531 200 2010 ID - Third Infrastructure DPL (IDPL3) P115102 250 Indonesia Sixth Development Policy Loan P113638 750 2009 Fifth Development Policy Loan P110191 750 ID-Second Infrastructure DPL (IDPL 2) P111905 200 Public Expend. Supp. Facility (DPL-DDO) P115199 2000 2008 Fourth Development Policy Loan P105637 600 ID-Infrastructure DPL (IDPL) P107163 200 2007 Indonesia-Third Development Policy Loan P100327 600 2006 ID Second Development Policy Loan P096594 400 2005 First Development Policy Loan P092663 300 Lao PDR 2011 Lao PDR PRSO 7 P122847 10 2010 Lao PDR PRSO 5 P110109 20 Lao PDR PRSO 6 P118814 20 2008 Lao PDR PRSO4 P107242 10 2007 LA - PRSO III P096710 10 Mongolia 2011 Mongolia - Development Policy Credit 2 P117421 29.7 2009 MN-Development Policy Credit P115737 40 Philippines 2009 Philippines GFRP DPO ICR P113492 200 2007 PH- Development Policy Loan - DPL I P100706 250 Timor-Leste 2006 TP Consolidation Support Program (CSP) 1 P088181 0.5 2005 Third Transition Support Program P083894 4.96 2004 Transition Support Program II P082190 4 Vietnam 2009 VN - PRSC 8 P111164 350 2008 VN - PRSC 7 P105287 150 2007 Vietnam PRSC 6 P101724 175 VN - Program 135 Phase 2 Support Credit P104097 50 2006 VN-PRSC V P086361 100 2005 Vietnam - PRSC IV P086360 100 2004 Vietnam PRSC III P082759 100 ECA Albania 2011 Social Sector Reform DPL P116937 25 2007 DPL P096205 10 2005 PRSC 3 P083337 10 33 APPENDIX A COUNTRIES IN WHICH DPOS WERE PRECEDED BY A PER 2004 PRSC 2 P077739 18 Armenia 2007 PRSC 3 P093460 28 2006 PRSC 2 P093459 20 2005 PRSC P078673 20 Azerbaijan 2005 PRSC P074938 20 Bosnia and 2010 Public Expenditure Crisis DPL P116951 111 Herzegovina 2004 EMSAC P071039 34 Bulgaria 2009 SIR DPL2 (DDO) P102160 150 SIR DPL3 P115400 200 2007 SIR DPL1 P094967 150 2005 PAL 3 P078675 150 2004 PAL 2 P081637 150 Croatia 2010 Fiscal, Social and Financial Sector DPL P117665 296.75 Georgia 2006 PRSO P088820 20 2004 REFORM SUPPORT P057814 24 Kazakhstan 2010 DPL P119856 1000 Latvia 2011 2nd Safey Net and Social Sector Reform P121796 142.08 2010 Financial Sector DPL (LATVIA) P115709 282.65 Safety Net & Soc. Sector Reform Program P115732 143.9 Macedonia, FYR 2010 Macedonia DPL 1 P116984 30 2008 PDPL 3 P101296 25 Moldova 2010 ECONOMIC RECOVERY DPO P112625 25 2008 PRSC 2 P103941 10 2007 PRSC P099166 10 Poland 2011 ENERGY EFFICIENCY DPL P115426 1114.5 2010 DPL 3 P117666 1331.3 2009 DEVELOPMENT POLICY LOAN P112765 1250 Empl. Entrepreneurship & HCDP DPL P116125 1300.24 Romania 2012 DPL 3 P122222 560.6 2011 DPL 2 P117667 380.5 2010 DPL 1 P102018 422.99 2005 PAL P008791 150 Russian 2005 KAZAN MUNICIPAL DEVT P082018 125 Federation Serbia 2010 YF PFDPL 2 P115958 100 2009 PFDPL P096711 50 2006 PPFDPC1(SERBIA) P089116 55 2005 SAC 2 (SERBIA) P078457 45 Tajikistan 2009 PDPG 3 P106963 20 2008 PDPG 2 P096930 10 2007 TJ Prg. Development Policy Grant P074889 10 Turkey 2010 REGE DPL P112495 1300 2009 CEDPL 2 P096840 500 2008 PPDPL 2 P088837 400 2007 CEDPL P074181 500 2006 TR- PPDPL P071052 500 2004 PFPSAL 3 P082996 1000 Ukraine 2010 FIN REHAB DPL 1 P115143 400 34 APPENDIX A COUNTRIES IN WHICH DPOS WERE PRECEDED BY A PER 2009 Ukraine DPL 3 P107365 500 2008 DPL 2 P096389 300 2006 DPL P079316 251.26 2004 PAL 2 P074972 250 LCR Argentina 2004 AR-Prov. Maternal-Child Hlth Adj P072637 750 PMCHSAL Bolivia 2005 BO-Social Sector Programmatic Credit II P091365 15 2004 BO First Programmatic Bank and Corporate P082781 30 BO-Social Sector Programmatic Credit P087841 25 Colombia 2009 CO 3rd Sust. Dev DPL P101301 450 2008 CO 3rd Business Prod & Efficiency DDO P105029 550 2007 CO 2nd Business Product and Efficiency P095213 300 CO 2nd Sustainable Dev DPL P095877 200 CO- 3rd Prog.Labor and Social Sector P094097 200 2006 CO Business Product and Efficiency DPL I P094301 250 2005 CO- 2nd Prog. Labor & Social Sector Ref P082865 200 CO 2nd Programmatic FSAL P082597 100 CO Prog Dev Policy Ln for Sust. Dev P081397 150 CO Prog Fiscal and Institutional SAL III P084762 100 Costa Rica 2009 CR Pub Fin & Compet. DPL/ DDO P115173 500 Dominican 2005 DO Power Sector Program Loan P082712 150 Republic 2004 DO Social Crisis Response Adjustment Loa P085433 100 El Salvador 2011 SV Public Finance and Social Progress P122699 100 2010 SV Sustaining Social Gains P118036 100 2005 SV (CRL2) Program. Broad-Based Growth P093133 100 DP Guyana 2006 GY Poverty Reduction and Public Mgmt. P078703 9.6 Haiti 2011 HT- Emergency DPO P118239 30 2010 HT 3rd Econ. Governance Reform P117944 12.5 Operation Honduras 2012 HN 1st Progr. Reduc. Vulner. Growth DPC P127331 86 2011 HN Emergency Recovery Dev Policy Credit P121220 74.7 2005 HN First Prog Fin Sec Dev Pol Credit P083311 25 2004 HN PRSC P074758 58.8 Jamaica 2009 JM Fiscal & Debt Sust. DPL P101321 100 Mexico 2011 MX MEDEC Low-Carbon DPL P121800 401.002507 MX Strength. Business Env for EcoGrowth P112264 751.8797 2010 MX DPL Adapt. Climate Change in WtrSct P120134 450 MX Economic Policies DPL P118070 1503.75 MX Framework for Green Growth DPL P115608 1503.75 2009 MX Environmental Sustainability DPL P095510 300.75188 2008 MX Affordable Housing DPL III P101342 200.51 MX Climate Change DPL/DDO P110849 501.25 2006 MX Aff. Housing & Urban Pov. Red. DPL II P089852 200.51 MX Competitiveness DPL P098299 300.76 MX ENVDPL II P079748 200.51 MX Finance and Growth DPL I P097159 501.26 2004 MX Afford.Housing & Urb.Pov.Prog. SECAL P070371 100 35 APPENDIX A COUNTRIES IN WHICH DPOS WERE PRECEDED BY A PER Nicaragua 2009 NI Development Policy Credit P106747 20 2004 NICARAGUA PRSC I P082885 70 Paraguay 2009 PY Public Sector DPL P113457 100 Peru 2011 PE 4th Prog. Fiscal Mgmt DPL P116214 100 2010 PE 3rd Prog Fiscal Mgmt DPL P106720 150 2006 PE Third Progr. Decentral. & Compet. DPL P089826 150 2005 PE PROGRAMATIC REFORM FOR P083949 100 GROWTH II PE-Programmatic Social Reform Loan IV P083968 100 2004 PE Program. Decent. & Competitiveness P082871 150 PE-Programmatic Social Reform III P078951 150 Uruguay 2005 Uruguay Social Program Support Loan DPL P095028 75.38 MNA Djibouti 2008 Djibouti - Food Crisis Response Dev. P112017 5 Egypt, Arab Rep. 2010 Financial Sector DPL III P120470 500 Lebanon 2008 LB - Reform Implementation DPL P094288 100 West Bank and 2011 WBG:PRDP Support III P118593 40 Gaza 2009 WBG PRDP Support II P113621 40 2008 WBG PRDP Support P111078 40 Yemen, Rep. 2008 RY-INSTITUTIONAL REFORM DPG P101453 50.93 SAR Bangladesh 2007 BD DSC IV/DPL P074801 200 BD Railway Reform Programmatic P100330 40 DevPolicy Education Sector Dev.Support Credit III P102541 100 2006 Bangladesh Development Support Cr. III P090832 200 Education Sector Dev. Support Credit II P084567 100 2005 Development Support Credit II P083887 200 Education Sector Dev. Support Credit P077789 100 Nepal 2004 Poverty Reduction Support Credit I P074685 70 Pakistan 2007 NWFP Second Development Policy Credit P097471 130 PEDPC-IV P101243 100 PK PRSC II P090690 350 Punjab Irrig DPL II P102333 100 Sindh DPC P100846 100 2006 NWFP DPC I P090689 90 PK Punjab Education DPC-III P097636 100 PK -Punjab Second Education Sector DPC P090346 100 PK Punjab-Irrigation Sector DPL P096962 100 2005 Pakistan PRSC I P078806 300 PK Banking Sector Dev. Policy Credit P083079 300 2004 NWFP SAC II P079635 90 Punjab Education Sector Adjustment Credi P083228 100 36 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS Appendix B. List of public expenditure knowledge products Note, if reports consists of various volumes, only volume one is presented. Region Country Year Document Name Report Number SAR Afghanistan 2010 Afghanistan - Country overview and role of public expenditure 72359 Afghanistan - Education sector 72364 Afghanistan - Expenditure framework and public financial 72361 management Afghanistan - Public expenditure review 2010 : second generation 53892 of public expenditure reforms Afghanistan - Public expenditure trends and fiscal sustainability 72360 Afghanistan - Security Sector 72362 Bangladesh 2010 Main Report 47767 2003 Bangladesh - Public expenditure review 24370 Bhutan 2013 Bhutan - Human development public expenditure review ACS2873 Maldives 2002 Maldives - Public expenditure review 24238 Nepal 2011 Nepal : Public expenditure review - Roads 66303 2000 Main Report 20211 Pakistan 2013 Pakistan - Punjab social sector public expenditure review 78748 2013 Pakistan - Khyber Pakhtunkhwa : public expenditure review 77337 2004 Strategic issues and reform agenda 25665 2001 Pakistan - Reforming Punjab's public finances and institutions 20981 Sri Lanka 2014 Investing in the Future : Education expenditures and financing in AUS3408 Sri Lanka 2014 Sri Lanka - Health PER : investing in Sri Lanka’s health system AUS2874 2006 Sri Lanka - Selected public expenditure issues 2003/2004 36630 AFR Africa 2013 Guinea - Basic agricultural public expenditure diagnostic review 89509 (2003-2012) : main report Malawi - Basic agricultural public expenditure diagnostic review 89511 (2000-2013) : final report 2013 Senegal - Basic agricultural public expenditure diagnostic review : 89512 final report 2013 Strengthening national comprehensive agricultural public 83163 expenditure in Sub-Saharan Africa : basic agricultural public expenditure diagnostic review - Ghana 2013 Burkina Faso - Basic agricultural public expenditure diagnostic 90813 review (2004-2012) : main report 2012 Togo - Basic agricultural public expenditure diagnostic review 73138 2011 More, better, or different spending? Trends in public expenditure 67321 on water and sanitation in Sub-Saharan Africa 2011 Sierra Leone - Public expenditure review for water and sanitation 64895 2002 to 2009 Angola 2007 Policy briefing 39710 37 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS Angola;Chad 2005 Angola - Public Expenditure Management and Financial 29036 Accountability Benin 2004 Benin - Enhancing the effectiveness of public spending - a review 29656 of three sectors Botswana 2010 Botswana - Public expenditure review 53959 Burkina Faso 2008 Burkina Faso - Rural water and sanitation sector : public 43912 expenditures review 2005 Burkina Faso - The budget as centerpiece of PRSP 29154 implementation - Public Expenditure Review Burundi 2013 Burundi - Public expenditure review : strengthening fiscal ACS5393 resilience to promote government effectiveness 2012 Burundi - Public expenditure review : fiscal challenges, security ACS3988 and growth issues in Burundi 2010 Burundi - Improving the efficiency of public investment : public 56716 expenditure review 2008 Republic of Burundi - Public Expenditure Management and 42160 Financial Accountability Review (PEMFAR) : improving allocative efficiency and governance of public expenditure and investing in public capital to accelerate growth and reduce poverty Cabo Verde 2009 Synthesis 44350 2006 Cape Verde - The challenge of increasing fiscal space to meet 34523 future pressures : public expenditure review Cameroon 2010 Cameroon - Fiscal policy for growth and development 48433 Central African 2012 Central African Republic - Public expenditure review (PER) : 63517 Republic creating fiscal space to transition out of fragility through growth and poverty reduction Central African 2008 Zambia - Making basic education services more effective in 54500 Republic;Zambia Zambia : improving delivery, tracking school level expenditures Chad 2011 Republic of Chad public expenditure review update : using public 57654 resources for economic growth and poverty reduction Congo, Dem. 2008 Congo, Democratic Republic of - Public expenditure review (PER) 42167 Rep. 2006 Executive summary 34937 Congo, Rep. 2014 Republic of Congo - Enhancing efficiency in education and health AUS5649 public spending for improved quality service delivery for all : a public expenditure review of the education and health sectors Cote d'Ivoire 2008 Main Report 44429 2003 Cote d' Ivoire - Strengthening public expenditure management and 27141 controls : public expenditure review Eastern Africa 2010 Enabling reforms : a stakeholder-based analysis of the political 74548 economy of Tanzania's charcoal sector and the poverty and social impacts of proposed reforms Equatorial 2010 Equatorial Guinea : public expenditure review 51656 Guinea Eritrea 2008 Eritrea - Health and education sectors : public expenditure review 38698 Ethiopia 2010 Ethiopia - Public finance review 2010 54952 2010 Addis Ababa city : public expenditure review 67926 2010 Benishangul-Gumuz Regional State : Public finance review 67928 2010 Oromia region study case : public finance review 67929 The Amhara regional report : public finance review 67927 2009 Ethiopia - Public finance review 50278 38 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS 2008 Ethiopia - Agriculture and rural development : public expenditure 41902 review for 1997-98 and 2005-06 2004 Public spending in the social sectors 29338 2001 Main Report 23351 2000 Main Report 20810 Gabon 2012 Gabon - Public expenditure review : better management of public 62548 finance to achieve millennium development goals Gambia, The 2011 The Gambia - 2010 public expenditure review update 59309 2006 Gambia - Fiscal developments and the agriculture sector : public 67703 expenditure review update 2005 The Gambia - Recovering Fiscal Discipline - Public Expenditure 31454 Review Ghana 2011 Ghana - Joint review of public expenditure and financial 67466 management 2009 Main Report 47639 2008 Main Report 40676 2008 Ghana - Rural water and sanitation sector : public expenditures 43909 review 2006 Main Report 36384 2005 Main Report 32595 2000 Ghana - Public expenditure review 1999 : improving education 59491 through increased, efficient, and effective utilization of resources Guinea 2004 Guinea - Strengthening public expenditure management for 27347 poverty reduction and growth : public expenditure review Guinea-Bissau 2007 Guinea-Bissau - Public expenditure review (PER) update : 35871 enhancing growth and fiscal adjustment through civil service reform 2004 Guinea Bissau - The challenge of restoring budgetary discipline - a 27175 public expenditure review Kenya 2013 Kenya - Comprehensive public expenditure review 2013 : eye on 82678 budget - spending for results 2005 Kenya - Public Expenditure Review 2004 - Report on the structure 29421 and management of public spending 2001 Kenya - Review of the water supply and sanitation sector 22182 Lesotho 2014 Lesotho - Public expenditure review 71973 Liberia 2013 Liberia - Public expenditure review : options for fiscal space 75049 enlargement 2013 Liberia - Public expenditure review note : meeting the challenges 71009 of the UNMIL security transition Strengthening national comprehensive agricultural public 83165 expenditure in Sub-Saharan Africa : agriculture sector public expenditure review - Liberia 2012 Liberia - Public expenditure review : human development 70980 2009 Liberia - 2008 public expenditure management and financial 43282 accountability review Madagascar 2007 Executive summary 38687 2005 Madagascar - Public Expenditure Review (PER) 2004 - The 30331 challenge of poverty reduction Malawi 2013 Malawi - Public expenditure review 79865 89185 2007 Malawi - Public expenditure review 2006 40145 39 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS 2001 Malawi - Public expenditures - issues and options 22440 Mali 2008 Mali - Rural water and sanitation sector : public expenditures 43911 review Mauritania 2011 Mauritania - Public expenditure review : update 62082 2007 Mauritania - Improving budget management to promote 36385 sustainable development and reduce poverty : public expenditure review Mauritius 2013 Mauritius policy notes : building analytical capacity to raise public 77778 sector efficiency 2004 Mauritius - The New Economic Agenda and Fiscal Sustainability 26152 Mozambique 2011 Mozambique - Analysis of public expenditure in agriculture 59918 2010 Mozambique - Public expenditure review for the water sector 54692 2003 Mozambique - Public expenditure review : Phase 2 - sectoral 25969 expenditures 2001 Mozambique - Public expenditure management review 22985 Namibia 2008 Namibia - Implementing the agenda of the Namibian ministry of 42472 environment and tourism : a rapid country environmental analysis with a public expenditure review for aligning policy, institutional and financing priorities 2005 Namibia human capital and knowledge development for economic 59166 growth with equity Niger 2014 Niger - Security sector public expenditure review 83526 2013 Niger - Public Expenditure Review (PER) 2012 76851 2009 Main Report 48234 2004 Niger - Public Expenditure Management and Financial 29752 Accountability Review Nigeria 2011 Nigeria - State level public expenditure management and financial 67934 accountability review : a synthesis report (Anambra, Bayelsa, Ekiti, Kogi, Niger, Ondo, and Plateau) 2010 Nigeria - Lagos rolling first public expenditure review 67936 2008 Nigeria - Agriculture public expenditure review 44000 2008 Executive summary 42418 2006 Nigeria - Strengthening the Nigerian forestry sector to enable 32406 sustainable forestry and revenue generation in productive forests Rwanda 2010 Rwanda - Technical assistance for public expenditure 54811 management : a policy note 2003 Rwanda - Public expenditure reform for sustaining poverty 25914 reduction : a public expenditure management review Senegal 2005 Senegal - Enhancing the efficiency of public investmnet - Public 32479 Expenditure Review (PER) 2004 Senegal - Public expenditure review 29357 Seychelles 2014 Seychelles - Programmatic public expenditure review policy notes 85290 - health education and investment management 2011 Seychelles selected issues : social protection, labor market, and 57113 public enterprise reforms - second public expenditure review 2009 Seychelles - Public expenditure review 49273 Sierra Leone 2010 Sierra Leone - Public expenditure review 52817 2004 Sierra Leone - Public expenditure review : from post-conflict 29075 recovery to sustained growth South Africa 2012 Health expenditure and finance in South Africa 69890 40 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS Sudan 2014 Sudan - State-level public expenditure review : meeting the ACS8803 challenges of poverty reduction and basic service delivery 2011 Sudan - Public Expenditure Tracking Survey (PETS) : case study 67712 of the health sector 2011 Public Expenditure Tracking Survey (PETS) for Northern Sudan : 67937 case study of the health sector 2009 Sudan - GoNU budget note series : World Bank review of 2008 67933 budget performance and 2009 budget preliminary analysis 2007 Sudan - Public expenditure review : synthesis report 41840 Swaziland 2006 Main Report 35318 2000 Swaziland - Reducing poverty through shared growth 19658 Tanzania 2014 Tanzania - Strengthening national comprehensive agricultural 87819 public expenditure in Sub-Saharan Africa : national agricultural input voucher scheme (NAIVS) 2012 Rapid budget analysis 2012 : synoptic note 89141 2012 United Republic of Tanzania - Public expenditure review 2011 72119 2011 United Republic of Tanzania - Public expenditure review 2010 64585 2010 United Republic of Tanzania - Public expenditure and financial 56313 accountability review 2009 2009 Tanzania - Public expenditure review of the water sector 50905 2009 Tanzania - Public expenditure and financial accountability review 48042 2008 2007 United Republic of Tanzania - Public expenditure and financial 67931 accountability review for FY07 2006 Tanzania - Public Expenditure and Financial Accountability Review 36642 : FY05 2004 Public expenditure review for FY04 67930 2003 Tanzania - Public Expenditure Review FY03 - Managing public 26807 expenditures for poverty reduction report on fiscal developments and public expenditure management issues Tanzania - Zanzibar Public Expenditure Review 2003 - Laying the 27504 foundations for improved public expenditure management 2002 Tanzania - Public expenditure review FY02 - report on fiscal 24475 developments and public expenditure management issues : an external evaluation 2001 Tanzania - Public expenditure review 23812 2001 Main Report 22078 Uganda 2013 Service delivery with more districts in Uganda : fiscal challenges ACS4421 and opportunities for reforms - public expenditure review 2010 Uganda - Strengthening the effectiveness of the Public Investment 58565 Program : public expenditure review 2010 Uganda - Agriculture public expenditure review 53702 53704 2010 Uganda - Public expenditure review : strengthening the impact of 55672 the roads budget 2009 Uganda - A public expenditure review 2008 : with a focus on 49361 affordability of pay reform and health sector 2008 Uganda - Sustainable Land Management Public Expenditure 45781 Review (SLM PER) 2007 Executive summary 40161 2004 Main Report 29377 41 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS 2003 Uganda - Public expenditure review 2003 27135 2001 The budget and medium-term expenditure framework in Uganda 23439 Zambia 2012 Zambia - Public expenditure review : resource allocations and 67935 financial flows in the water sector in Zambia 2010 Zambia - Impact assessment of the fertilizer support program : 54864 analysis of effectiveness and efficiency 2009 Zambia health sector public expenditure review : accounting for 47110 resources to improve effective service coverage 2006 Executive summary and main report 36552 2002 The impact of cash budgets on poverty reduction in Zambia : a 68547 case study of the conflict between well-intentioned macroeconomic policy and service delivery to the poor 2001 Zambia - Public expenditure review : public expenditure, growth 22543 and poverty - a synthesis ECA Albania 2014 Toward a sustainable fiscal policy for growth 82013 2008 Albania - Local finance policy note : programmatic public 44109 expenditure and institutional review 2006 Overview and policy options 36453 2001 Executive summary 21857 Armenia 2014 Armenia - Public expenditure review : expanding the fiscal 88586 envelope 2012 Fiscal consolidation and recovery in Armenia : impact of the global 72880 crisis on a small open economy 2011 Synthesis report 62587 2010 Armenia - The 2008-09 global economic crisis, policy responses, 55011 and household coping strategies 2003 Armenia - Public expenditure review 24434 Azerbaijan 2013 Azerbaijan - Food security and livelihoods : public expenditure 78836 review Azerbaijan - Healthy development improving outcomes in the 78834 health sector : public expenditure review Azerbaijan - The oil nexus : public expenditure review 78835 2003 Azerbaijan - Public expenditure review 25233 Belarus 2013 Belarus - Public expenditure review : enhancing public services in 74148 times of austerity 2011 Belarus public expenditure review : fiscal reforms for a sustainable 63566 economic recovery 2003 Belarus - Strengthening public resource management 26041 Bosnia and 2012 Bosnia and Herzegovina - Challenges and directions for reform : a 66253 Herzegovina public expenditure and institutional review 2006 Bosnia and Herzegovina - Addressing fiscal challenges and 36156 enhancing growth prospects : a public expenditure and institutional review 2002 Bosnia and Herzegovina - From aid dependency to fiscal self- 24297 reliance : a public expenditure and institutional review Bulgaria 2012 Bulgaria - Public expenditures for growth and competitiveness 62774 2010 Bulgaria - Public expenditure review for agriculture and rural 54903 development 2008 Bulgaria - Resourcing the judiciary for performance and 42159 accountability : a judicial public expenditure and institutional review 2007 Overview 38570 42 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS 2006 Bulgaria - Public Finance Policy Review 33992 2002 Bulgaria - Public expenditure issues and directions for reform - a 23979 Public Expenditure and Institutional Review Croatia 2014 Croatia - Justice sector public expenditure and institutional review : ACS7424 resourcing the justice sector for efficiency and performance Croatia - Public finance review : restructuring spending for stability 78320 and growth 2008 Croatia - Restructuring public finance to sustain growth and 37321 improve public services : a public finance review 2001 Croatia - Regaining fiscal sustainability and enhancing 22155 effectiveness - a public expenditure and institutional review Czech Republic 2001 Czech Republic - Enhancing the prospects for growth with fiscal 22114 stability - a public expenditure review Georgia 2014 Strategic issues and reform agenda 78143 2012 Georgia - Managing expenditure pressures for sustainability and 73701 growth : public expenditure review Georgia - Public expenditure review : managing expenditure 74474 pressures for sustainability and growth 2002 Georgia - Public expenditure review 22913 Kazakhstan 2006 Kazakhstan - Reforming intergovernmental fiscal relations 33709 2000 Summary report 20489 Kosovo 2010 Kosovo - Public expenditure review 53709 2007 Kosovo - Policy note on public investment management 68419 2006 Overall fiscal situation 32624 Kyrgyz Republic 2014 Kyrgyz Republic - Public expenditure review policy notes : 88978 education Kyrgyz Republic - Public expenditure review policy notes : health 88979 Kyrgyz Republic - Public expenditure review policy notes : 88975 intergovernmental fiscal relations Kyrgyz Republic - Public expenditure review policy notes : 89007 pensions Kyrgyz Republic - Public expenditure review policy notes : public 89008 investment management Kyrgyz Republic - Public expenditure review policy notes : public 88976 wage bill Kyrgyz Republic - Public expenditure review policy notes : social 89022 assistance Kyrgyz Republic - Public expenditure review policy notes : 88977 strategic setting 2004 Main Report 28123 Latvia 2010 Overview and summary 56747 2002 Latvia - Public expenditure review 22392 2001 Latvia - Public expenditures review : education sector 73011 Macedonia, FYR 2008 Macedonia - FYR Macedonia public expenditure review 42155 Moldova 2014 Moldova - Public expenditure review : reforming local public 87268 finance for more efficient, equitable, and fiscally sustainable subnational spending 2013 Moldova - Public expenditure review : capital expenditures - 76310 making public investment work for competitiveness and inclusive growth in Moldova 82138 43 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS 2007 Moldova - Improving public expenditure efficiency for growth and 37933 poverty reduction 2003 Moldova - Public Economic Management Review 25423 Montenegro 2012 Montenegro - After the crisis : towards a smaller and more efficient 65909 government - public expenditure and institutional review 2008 Public expenditure and institutional review 46660 2006 Main Report 36533 Poland 2010 Poland - Mazowieckie public expenditure review : Local responses 52037 to the global economic crisis 2010 Analysis of social sectors and public wages 52536 2009 Better financing : stronger outcomes - a public sector expenditure 57305 review for the education sector in Poland 2003 Poland - Toward a fiscal framework for growth - a public 25033 expenditure and institutional review Romania 2010 Main Report 51191 2009 Romania - Public sector pay practices Romania : overall public 47755 sector trends and detailed analysis of local government contract employees 2006 Main Report 36363 2002 Romania - Building institutions for public expenditure management 24756 : reforms, efficiency and equity - a Public Expenditure and Institutions Review Russian 2011 Russia - Public expenditure review 58836 Federation 2011 Russian federation : social expenditure and fiscal federalism in 54392 Russia 2009 Russia environmental management system : directions for 51068 modernization 2008 Public spending in Russia for health care : issues and options 45127 2006 Russian Federation - Enhancing the impact of public support to 39213 agriculture and rural sectors 2005 Russia - Fiscal costs of structural reforms 30741 2001 Russia - towards improving the efficiency of public investment 22693 expenditures 2000 Russia : issues in public expenditure policy 67932 Serbia 2014 Serbia - Municipal public finance review : options for efficiency 87944 gains 2013 Serbia - Municipal finance and expenditure review 76855 2010 Serbia - Right-sizing the government wage bill 54056 2009 Serbia - Doing more with less : addressing the fiscal crisis by 48620 increasing public sector productivity 2008 Serbia : decentralization and local service delivery 42154 2003 Executive summary 23689 South Eastern 2010 Western Balkans - Public investment management study : main 53768 Europe and report Balkans 2007 Public expenditure policies in Southeast Europe 33400 Tajikistan 2014 Capital expenditures and public investment management 89184 Fiscal risks from state-owned enterprises 89183 Key issues in public finance management 88050 44 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS Tajikistan - Programmatic public expenditure review : capital 88052 expenditures and public investment management Tajikistan - Programmatic public expenditure review : fiscal risks 88051 from state-owned enterprises 2013 Government expenditures : size, composition and trends 77607 Review of public expenditures on education 89182 Review of public expenditures on health 89181 2012 Main Report 68418 2008 Synthesis note 43261 2008 Main Report 43280 2007 Tajikistan - Programmatic public expenditure review 39771 2005 Tajikistan : Public expenditure and institutional review 34891 Turkey 2014 Turkey - Public finance review : Turkey in transition - time for a 85104 fiscal policy pivot? 2012 Turkey - Transport sector expenditure review : synthesis report 62581 2006 Turkey - Public expenditure review 36764 2001 Turkey - Public expenditure and institutional review - reforming 22530 budgetary institutions for effective government 2001 Turkey - Forestry Sector Review 22458 Ukraine 2008 Ukraine - Improving intergovernmental fiscal relations and public 42450 health and education expenditure policy : selected issues 2006 Ukraine - Creating fiscal space for growth 36671 2003 Ukraine - Financing the environment : Ukraine's road to effective 68417 environmental management 2002 Ukraine - Review of the budget process : a public expenditure and 23356 institutional review 2001 Main Report 22677 Uzbekistan 2005 Uzbekistan - Public Expenditure Review 31014 Yemen, Rep. 2002 Macedonia - Public expenditure and institutional review 23349 MNA Algeria 2007 Main Report 36270 2002 Main Report 22591 Djibouti 2006 Djibouti - Public Expenditure Review (PER) - making public 34624 finances work for growth and poverty reduction Egypt, Arab 2014 Egypt - Medium-term macro-fiscal framework and debt 89840 Rep. sustainability 2012 Main Report 71249 2010 Infrastructure and economic growth in Egypt WPS5177 2009 Egypt - Linking funding to outputs : expenditures of the Ministry of 47547 Agriculture and Land Reclamation Iran, Islamic 2005 Iran - Report on public financial management, procurement, and 34777 Republic of expenditure systems in Iran 2005 Islamic Republic of Iran - Transport sector review and strategy 34600 note Jordan 2012 Hashemite kingdom of Jordan options for immediate fiscal 71979 adjustment and longer term consolidation Lebanon 2013 Lebanon - Economic and labor force impact of the change in the 79324 wage structure of the public sector 2010 Republic of Lebanon - Water sector : public expenditure review 52024 2008 Lebanon - Electricity sector public expenditure review 41421 45 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS 2005 Lebanon - Public expenditure review : reform priorities for fiscal 32857 adjustment, growth and poverty alleviation Libya 2009 Synthesis 45019 Middle East and 2014 A comprehensive review of the health financing systems in Jordan ACS9635 North Africa West Bank and 2007 Summary 38207 Gaza Yemen, Rep. 2006 Tracking basic education expenditure in Yemen : analyses of 38304 public resource management and teacher absenteeism LAC Argentina 2003 Argentina - Reforming policies and institutions for efficiency and 25991 equity of public expenditures Belize 2013 Belize - PPIAF SNTA municipal finance : report of RESP activity ACS3663 Bolivia 2011 Bolivia - Agriculture public expenditure review 59696 2004 Bolivia - Public expenditure management for fiscal sustainability 28519 and equitable and efficient public services Brazil 2008 Different paths to student learning : good practices and student 43590 performance - identifying success from municipal school systems in Brazil 2008 Main Report 40144 2007 Stylized facts and recommendations 36595 2007 Brazil - Governance in Brazil's unified health system : raising the 36601 quality of public spending and resource management 2002 Policy report 24413 2002 Main Report 22870 Colombia 2005 Colombia - Public expenditure review 25163 Costa Rica 2008 Costa Rica - Public expenditure review : enhancing the efficiency 40774 of expenditures 2002 Summary of issues and recommendations with executive 24300 summary Dominica 2005 Dominica - OECS fiscal issues : policies to achieve fiscal 27467 sustainability and improve efficiency and equity of public expenditures Dominican 2004 Dominican Republic - Public Expenditure Review : reforming 23852 Republic institutions for a more efficient public expenditure management Ecuador 2004 Main Report 28911 2000 Main document 19920 El Salvador 2010 Main Report 53500 2004 El Salvador - Public expenditure review 32856 Grenada 2004 Grenada - OECS fiscal issues : policies to achieve fiscal 28334 sustainability and improve efficiency and equity of public expenditure Guatemala 2013 Towards better expenditure quality : Guatemala public expenditure 78000 review 2012 Guatemala - Public expenditure review 67363 2000 Guatemala - Expenditure reform in a post-conflict country 19617 Guyana 2002 Guyana - Public expenditure review 20151 Haiti 2008 Haiti - Public Expenditure Management and Financial 40066 Accountability Review (PEMFAR) : improving efficiency of the fiscal system and investing in public capital to accelerate growth and reduce poverty 46 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS Honduras 2013 Public expenditures for decentralized governance in Honduras : 82662 towards restoring fiscal consolidation 2010 Main Report 64692 2007 Executive summary and main report 39251 2001 Honduras - Public expenditure management for poverty reduction 22070 and fiscal sustainability Jamaica 2005 Jamaica - Fiscal consolidation for growth and poverty reduction - a 29546 Public Expenditure Review Mexico 2012 Mexico - Social protection system in health and the transformation 76735 of state health systems Mexico's social protection system in health and the financial 76734 protection of citizens without social security 2009 Mexico - Agriculture and rural development public expenditure 51902 review 2006 Mexico - Water public expenditure review 36942 2005 Mexico - Infrastructure public expenditure review (IPER) 33483 2004 Core report 27894 2003 Mexico - State-level public expenditure review : the case of 25162 Veracruz-Llave Nicaragua 2010 Nicaragua - Selected issues in social sectors management : 71684 programmatic social NLTA implementation report 2008 Nicaragua - Public expenditure review 2001-2006 39807 2008 Nicaragua - Social protection public expenditure review 71687 2001 Nicaragua - Public expenditure review - improving the poverty 23095 focus of public spending Paraguay 2013 the quest for optimal tax and expenditure policies for shared 78194 prosperity 2006 Paraguay - Public expenditure review : main report 32797 Peru 2012 Peru - Public Expenditure Review for Peru : spending for results 62586 2010 Peru - The decentralization process and its links with public 52885 expenditure efficiency 2002 Peru - Restoring fiscal discipline for poverty reduction : a public 24286 expenditure review St. Kitts and 2003 Saint Kitts and Nevis - OECS fiscal issues : policies to achieve 25185 Nevis fiscal sustainability and improve efficiency and equity of public expenditures St. Vincent and 2005 Saint Vincent and the Grenadines - OECS fiscal issues : Policies 30885 the Grenadines to achieve fiscal sustainability and improve efficiency and equity of public expenditures Uruguay 2013 Uruguay - Public expenditure review : mitigating fiscal risks 68770 2001 Uruguay : maintaining social equity in a changing economy 21262 EAP Cambodia 2011 Cambodia - More efficient government spending for strong and 61694 inclusive growth : integrated fiduciary assessment and public expenditure review 2008 Cambodia - Reaching the people : Public Expenditure Tracking 52807 Survey (PETS) in health - budget year 2003-2004 2005 Cambodia - Public expenditure tracking survey (PETS) in primary 34911 education 2003 Cambodia - Enhancing service delivery through improved resource 25611 allocation and institutional reform : integrated fiduciary assessment and public expenditure review 47 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS China 2002 China - National development and sub-national finance : a review 22951 of provincial expenditures 2000 China - Managing public expenditures for better results : country 20342 economic memorandum Indonesia 2013 Indonesia - Spending more or spending better : improving 73050 education financing in Indonesia 2012 Indonesia - South Sulawesi Public Expenditure Analysis 2012 : 73504 Improving public service delivery and financial management in the gateway to Eastern Indonesia 2012 Southeast Sulawesi public expenditure analysis 2012 : public 73273 service delivery performance and development challenges in Bumi Haluoleo 2012 Investing in Indonesia's roads : improving efficiency and closing 73303 the financing gap 2012 Bantuan Langsung Tunai (BLT) temporary unconditional cash 67324 transfer Bantuan Siswa Miskin cash transfers for poor students 67319 History and evolution of social assistance in Indonesia 67479 Jamkesmas health service fee waiver 67312 JSLU, JSPACA, PKSA cash and in-kind transfers for at-risk youth, 67320 the disabled, and vulnerable elderly Main Report 67217 Program Keluarga Harapan (PKH) conditional cash transfer 67309 Public expenditure review summary 67299 Raskin subsidized rice delivery 67308 2012 Targeting poor and vulnerable households in Indonesia 67218 2011 Indonesia - Agriculture public expenditure review 2010 69346 2011 2011 North Sulawesi public expenditure analysis : sub-national 67534 public financial management and development in Bumi Nyiur Melambai 2009 Aceh growth diagnostic : identifying the binding constraints to 49568 growth in a post-conflict and post-disaster environment 2009 Indonesia - Investing in Indonesia's education at the district level : 47514 an analysis of regional public expenditure and financial management 2008 Investing in Indonesia's health : challenges and opportunities for 46314 future public spending - health public expenditure review 2008 2008 Service delivery and financial management in a new province : 46313 Gorontalo public expenditure analysis 2008 2007 Managing resources to build back and create a better future for 40822 Nias : Nias public expenditure analysis 2007 Indonesia public expenditure review 2007 - Spending for 38772 development : making the most of Indonesia's new opportunities 2007 Public financial managment in Aceh : measuring financial 40711 management performance in Aceh's local governments 2007 Main Report 38778 2006 Aceh public expenditure analysis : spending for reconstruction and 38417 poverty reduction 2005 Papua public expenditure analysis - overview report : regional 34046 finance and service delivery in Indonesia's most remote region 2003 Decentralizing Indonesia : A regional public expenditure review 26191 overview report 48 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS 2003 Indonesia - Selected fiscal issues in a new era 25437 2000 Indonesia - Public spending in a time of change 19845 Kiribati 2013 Summary of findings : Kiribati public expenditure analysis 78308 Lao PDR 2012 Government spending on health in Lao PDR : evidence and issues 76229 2011 Lao PDR - Public Expenditure Review (PER) 63200 2008 Lao PDR - Public expenditure tracking survey in primary education 39043 and primary health : making services reach poor people 2007 Lao PDR - Public expenditure review integrated fiduciary 39791 assessment Malaysia 2000 Malaysia - Malaysia public expenditures : managing the crisis ; 20371 challenging the future Mongolia 2009 Core report 43353 2006 Mongolia - Public financing of education : equity and efficiency 36979 implications 2002 Mongolia - Public expenditure and financial management review : 24439 bridging the public expenditure management gap Papua New 2004 Papua New Guinea - Public expenditure review and rationalization 31209 Guinea : overview of discussion paper Philippines 2013 Philippines - Second phase of the basic education public ACS2212 expenditure review : school based management - an empirical investigation 2013 Getting a grip on climate change in the Philippines : executive 78809 report Getting a grip on climate change in the Philippines : extended 78798 technical report 2012 Philippines - Basic education : public expenditure review 71272 2011 Philippines - Public expenditure review : strengthening public 55695 finance for more inclusive growth 2009 Philippines - Transport for growth : an institutional assessment of 47281 transport infrastructure 2007 Philippines : Agriculture public expenditure review 40493 2003 Philippines - Improving government performance : discipline, 24256 efficiency and equity in managing public resources (a public expenditure, procurement and financial management review) Samoa 2014 Samoa - Public expenditure review notes : taking stock of ACS7927 expenditure trends from FY2006-FY12 Solomon Islands 2011 Solomon Islands - Public expenditure review for the Solomon 68826 Islands Government Thailand 2012 Overview 67486 Timor-Leste 2013 Timor-Leste - Social assistance public expenditure and program 73484 performance report 2004 Timor-Leste - Public expenditure review 27886 2002 East Timor public administration : public expenditure management 27509 and accountability note Vietnam 2014 Vietnam fiscal transparency review : analysis and stakeholder 88273 feedback on state budget information in the public domain 2013 Vietnam - Agriculture public expenditure review ACS4599 2005 Cross sectoral issues 30035 2001 Viet Nam - Growing healthy : a review of Viet Nam's health sector 22210 2000 Main Report 21021 World Lithuania;World 2009 Lithuania - Social sectors public expenditure review 48604 49 APPENDIX B LIST OF PUBLIC EXPENDITURE KNOWLEDGE PRODUCTS World 2011 Forest sector public expenditure reviews : review and guidance 64491 note 2011 How do we improve public expenditure in agriculture? 66817 2009 Tanzania - Public expenditure review of the water sector 68621 50 APPENDIX C THE ROLE AND INFLUENCE OF PUBLIC EXPENDITURE REVIEWS ON DPO DESIGN Appendix C. The role and influence of Public Expenditure Reviews on DPO design (Table 2 of 2) Country, DPL, FY Was there a PER? If PER present and findings Were there Implementation If yes, PER findings used in used in DPO design, in associated Prior effective or not DPO? (Yes/No)? which Policy Areas of DPO Actions (PAs) (strong, If no PER, were findings on were PER findings used moderate, resource allocation from weak, none) other analytical work used in DPO? Albania Social Sector No recent PER. No links between 2006 PER No Reform, FY11 Extensive PER in 2006. and DPO (although the Not applicable DPL’s two Policy Areas PER 2006 covered public were: Social safety Net debt, the social insurance (SSN) Effectiveness; and system, health financing and Health Spending Efficiency health insurance institute and Equity. (HII). Although 2006 PER dated, The Program Document its findings remain relevant. (PD) does not discuss the adequacy of total However, the findings did expenditure allocations not factor into DPO design toward social transfer programs and health. The Project’s review documents do not discuss the composition of public expenditure. Benin PRSC 4-6, PER in 2004 focused on PER in 2004 recognized The PAs are series, FY07-10 social expenditure trends in that “the share of vague, given the weak to none the same sectors of focus in expenditures in five major seriousness of the the PRSC series. health programs is driven by issues discussed. the availability of foreign The Program However, the PRSC does funds rather than by national Document has no not contain any reference to priorities; some important discussion of PER. causes of child mortality and necessary morbidity are not budgetary addressed”. allocations. No analysis of A PRSC series focus: “to public support government efforts expenditures in toward achieving the MDGs critical MDG by improving access to related areas basic services, particularly where Benin is better efficiency of particularly expenditures in human lagging (maternal capital formation”. The mortality, child operation was explicitly mortality and aimed at improving MDG morbidity). outcomes. The 2010 IMF Ex- Post Assessment concludes that the 51 APPENDIX C THE ROLE AND INFLUENCE OF PUBLIC EXPENDITURE REVIEWS ON DPO DESIGN However, the PRSCs do not program could not follow up on the PER monitor poverty findings. reducing expenditures given the weakness of expenditure classification. Brazil State - Fiscal There was no PER or state Not Applicable Sustainability, FY09 specific social sector reports Not Applicable weak that could have guided the DPO. Cote d’Ivoire Post PER in 2008 Complex PER Policy Areas covered in the 5 of 7 PAs based Weak Reconstruction and relevant to the program. PER and included in the on PER Implementation Recovery, FY12 PER findings used in DPO DPO: recommendations. varied. I.e. design (see next two Health and Education These were: although PA columns). expenditures Adopt MTEF for (a) suggested Tax policy on cocoa/coffee health and increased pro- Budget execution education sectors poor Accountability of public Convert export expenditures, procurement taxes to ad the DPO Audit (of the cocoa sector) valorem rate for supported an cocoa; action to adopt Submit budget a MTEF in the execution laws; sector that was open code of not effective as ethics for public it was not procurement; binding. conduct audits of The result of four cocoa entities increasing expenditure on PER called for health and increasing pro- education was poor not achieved expenditures. In due to lack of particular, health institutional expenditures were depth in the very low by Sub- prior action Saharan African underpinning standards (below this result. 1 percent of GDP). Increasing the share of health and education expenditures was an expected result of the program. Indonesia Per in 2007. Policy Areas were based on Key associated Implementation Infrastructure DPL 4 both studies: Prior Actions (too moderate (series of 4), FY08-11 Also Infrastructure a) Infrastructure spending numerous to Electricity Framework for Policy and and investment at national mention all): pricing delayed Action (2004). and sub-national; a) Increase due to b) Allocation of fuel allocation to parliament subsidies (reducing national approval, but inefficient and pro-rich; infrastructure by moved forward 52 APPENDIX C THE ROLE AND INFLUENCE OF PUBLIC EXPENDITURE REVIEWS ON DPO DESIGN DPL design based on both allocate from consumption 30% over 2007 in 2010 during 2007 PER and the 2004 to connection) to expand level; last DPL, study. network; b) Publicize c) PFM in roads breakdown of rural d) Fiduciary (e-procurement subsidy by region electrification and audit) and customer target dropped e) Impediments to sub- category; national borrowing and c) Allocation to arrears infrastructure fund f) Incentives for PPPs (for PPP); d) Implement e- procurement; e) Assist water utilities (PDAMs) to clear arrears so they can resume borrowing. Latvia Safety Net and PER in 2007 before crisis, DPL used the 2007 PER proposal to shift from The Bank Social Sector Reform PER in 2010, (esp. for DPL) hospital financing to primary healthcare. Overall commissioned Program, FY10 with a focus on social sector relevance of PER limited as crisis downsized a PSIA reforms on a reduced budget envelope in 2009. (Employment budget due to crisis. and Welfare 2010 PER: Specifically commissioned to provide Impact of the Some findings from 2007 analytical underpinning for second DPL. The PER Financial Crisis used in DPL. supported social sector reforms undertaken since in Latvia). A 2009, but, as policy actions for second DPL were second study 2010 PER provided already designed during first DPL (as indicative by same thorough analysis of triggers) – there was little flexibility to incorporate authors in challenges in social new actions. 2012 that did protection, health and ex-post poverty education. Provided menu This could be the reason why some potentially simulations of policy options for medium important reform areas not included in the first found that term operation did not become reforms actions under ESSNS the second program. For example, the 2010 PER worked suggested reforms in Family State Benefits that reasonably were less targeted but constituted 37 percent of well. social protection budget as of 2010. The PER suggested a number of reforms in the program to direct resources from non-poor to the poor. However, this did not become part of the reform program. Mexico Green Sectoral PER in 2009, on This DPL does not have This DPL does not Not relevant to Growth, FY09 agriculture and rural Policy Areas related to PE. have Prior Actions this DPL. development; related to PE General PER in 2004. However, the EP DPL As in previous included 2004 PER column, the EP PER contents do not pertain findings/Policy areas on: DPL had several to DPL. Government Accounting; prior actions Tax Reform; related to 2004 Although no PE findings Conditional cash transfers PER findings. used in this DPL, the These were: Economic Policies (EP) DPL Implement approved 1 month after this accounting law to DPL includes a detailed establish accrual discussion on PE accounting; Management, and was 53 APPENDIX C THE ROLE AND INFLUENCE OF PUBLIC EXPENDITURE REVIEWS ON DPO DESIGN informed by several studies Tax reform for including the 2004 PER. basic foods and reducing subsidy for residential electricity; Taxes to enhance non-oil revenues Peru Programmatic PER (2002) Policy Areas were: Prior Actions Strong Fiscal Management PER (2007) a) Fiscal Management include: The PERs and and Competitiveness, Public Expenditure and (strengthening budget a) Enforcement of PEFA appear FY07-11 Fiscal Assessment (PEFA, reporting and planning)— fiscal rules; to have 2009). where most PAs were b) Eliminating informed the The first operation used the focused; sectoral and country and PER as a basis for its b) Results-based budgeting regional tax the Bank on design. The third operation (to enhance efficiency and exemptions; the most uses the PEFA to inform its promote higher impact of c) Linking budget important design (to inform the reform public spending—(higher presentation cycle aspects of process, and indicative efficiency evident in health with performance reform, and triggers) and nutrition programs); indicators and also provided c) Tax reforms (but little submission of with data and Overall, PERs and PEFA advance). sector analysis to used to inform design of the performance have proper series. indicators; baselines for d) Other PAs some of the related to fiscal indicators reforms Poland Energy PER in 2010 focusing on Not Applicable Efficiency and social sectors and public to this DPL. Renewable Energy wages. PER mostly Development, FY11 discusses forms of targeted social assistance. Not relevant for an energy sector DPL and did not factor into DPO design Romania Financial PER (2006) Consistency between 2010 Several Moderate Management and PER (2010) PER and DPL Policy areas associated Prior Social Protection DPL and Prior Actions. Policy Actions: 2 (series of 1-3, 2009- Both relevant. 2010 was in areas include: a) MTEF 11), FY11 middle of series and a) Improving FM by explicitly approved by emphasizes urgency due to introducing a MTEF; Parliament with global crisis. 2010 findings b) Public sector wage policy; binding used in DPL. c) Education reform expenditure limits; (allocation of teachers); b) Pay d) Health – allocation of adjustments made resources; in public sector e) Pension reform; according to laws; c) Schools to receive budgets by per-capita formula; d) Revised health provider payment mechanisms adopted; 54 APPENDIX C THE ROLE AND INFLUENCE OF PUBLIC EXPENDITURE REVIEWS ON DPO DESIGN e) Link/de-link pensions. St Lucia Economic No PER; A Policy Area relates to Prior Actions do weak and Social DPL, FY10 No PSIA improving the effectiveness not related to PE The Program Document and efficiency of social allocation, except Due partly to states that, “the Bank has safety nets (SSNs) by for a reference to long-term supported the program supporting transparent a Value Added reforms in a through analytic work in targeting mechanisms and Tax Bill. one-tranche public expenditure, prioritizing social spending operation. economic management, to vulnerable groups, Implementation poverty and social however, no expenditure is believed to protection”. However, there analysis. have improved is not enough evidence of after DPL analytical work being used closure. to support the operation. Tunisia Governance No PER No relevant Policy Areas in No Prior Actions Not relevant and Opportunity, DPO. related to FY11 Comprehensive Health expenditure Sector Study in May 2006, allocation. with important institutional reforms suggested. The only related PA is “institute a The DPL was prepared very national outreach rapidly. No obvious inclusion service policy to of findings. expand free access to care. No discussion on allocation Vietnam, PER in 2005 Policy Areas with There were 7 Very Weak. Programmatic PRSC 2013 PEFA recommendations in the Prior Actions for With 10 (series of PRSCs The PER was a major PER were: PRSC 6 and 10 progression of 6-10), FY10 exercise and very relevant Implementing the MTEF as indicative triggers series, the to the series. It suggested part of budget cycle; for PRSCs 7 to policy matrix 10 key reforms in the Balancing capital and 10. underwent medium term recurrent expenditure 7 of the 10 significant Eliminating off-budget indicative triggers changes. Most transactions were dropped. critical Implementing Treasury and indicative Budget Management triggers were Information System never State Audit of Vietnam incorporated (SAV) into the final Audits of SOEs policy matrix. Strengthen public investment management Strengthen public debt management Follow up PERs in 2008 and 2011. 55 APPENDIX D DPOS AND PUBLIC EXPENDITURE REVIEWS AT THE SUBNATIONAL LEVEL Appendix D. DPOs and Public Expenditure Reviews at the subnational level 1. As documented in this descriptive annex, subnational DPOs and PERs have taken place in a small number of large, federated countries durign the period under review (2000-13 for PERs and 2004-13 for DPOs). By far the largest number of subnational DPOs in this period took place in only one country: Brazil. 17PERs and DPOs generally tend to overlap in a very small number of countries. Given the important of subnational fiscal and development agenda, this may suggest a potential knowledge gap and an opportunity for the Bank to intensify knowledge engagement at the subnational levels, both as part of the knowledge and policy dialogue per se and as a contribution to the future DPO development. This very brief annex provides some very preliminary portfolio trends and figures that may raise questions and stimulate further discussion. 2. In the period under review (2004-14) there were a relatively limited number of subnational DPOs. In this period, the first set of subnational DPOs18 were provided to Pakistan in 2006, including an education and an irrigation policy credit to the Punjab region. There was a total of 35 DPOs at the subnational level between 2006 and 2013, equivalent to about 6 percent of all approved DPO operations in this period. On an annual basis, DPOs at the subnational level, are relatively rare, showing a a modest increase from one operation (2 percent of the total) in 2009 to six (10 percent) in 2014. 3. All 35 subnational operations were provided to the following five countries: Brazil, Pakistan, India, Nigeria, and Ghana. All are regionally large economies with significant Bank engagements. The large majority of these (17) were provided to Brazil. The trends and breakdown by country is provided in figure 1. 17In the previous period especially, there were subnational PERs in other countries (e.g., India, Pakistan, Russian Federation, Nigeria). 18 DPOs at the subnational level were identified by all policy lending that include a “subnational flag” as per business intelligence database. This includes only operations that lent directly to subnational governments, but not operations that were targeting decentralization reform via lending to central governments. The data here rely on BI report 2c.2.1, which was accessed on March 26, 2015. 57 APPENDIX D DPOS AND PUBLIC EXPENDITURE REVIEWS AT THE SUBNATIONAL LEVEL Figure D-1 . Trend of DPOs at subnational level in numbers and as share of total 10 100% 18 Subnational operations as share of total Number of subnational operations 16 Number of subnational operations 8 80% 14 6 60% 12 10 4 40% 8 2 20% 6 4 0 0% 2006 2007 2008 2009 2010 2011 2012 2013 2014 2 0 Number Share Brazil Pakistan India Nigeria Ghana 4. As too few of those DPOs were reviewed by the IEG, no clear conclusion can be drawn about the differences in outcome indicators at this time. 5. Similarly, relatively few PERs were produced at the subnational level. Between 2000 and 2014, IEG identified 38 subnational PERs out of a pool of 404, amounting to 9 percent of total.19 Though the number of reports produced varies, many more subnational PERs were produced in the latter half of the evaluation period, a trend reflecting the lending pattern of DPOs. The distribution of countries with subnational PERs is, however, much more balanced than with DPOs across the five regions. 6. Subnational PERs were found in 16 countries. Most prominent is Indonesia (12), followed by Ethiopia (4), and Pakistan (3). Of the five countries with subnational DPOs, only Brazil (2 PERs), Pakistan (3 PERs), and Nigeria (2 PERs) had a subnational public expenditure type knowledge product. This suggests that outside Brazil, where there has been a strong DPO and PER engagement, the integration of subnational PERs and DPOs is taking place in a very small number of countries. For example, while there were national PERs in Ghana, no national nor subnational PERs were conducted in India. Notable is also that no subnational DPOs were provided to Indonesia, China, or Mexico, despite their regional importance and the World Bank’s interest in their subnational expenditure composition. The trends and regional distribution of subnatiojnal PERs are shown in figure 2. As subnational coding was not available for this database, IEG instead judged content of the 19 PER by the document title. 58 APPENDIX D DPOS AND PUBLIC EXPENDITURE REVIEWS AT THE SUBNATIONAL LEVEL Figure D-2. Trend of PERs at subnational level in numbers and as share of total 8 80% 14 Subnational PERs as share of total PERs Number of subnational PERs 12 6 60% Number of subnational PERs 10 4 40% 8 6 2 20% 4 0 0% 2 0 Number Share SAR AFR ECA LAC EAP 59