30405 A User’s Guide to Poverty and Social Impact Analysis The World Bank Poverty Reduction Group (PRMPR) and Social Development Department (SDV) © 2003 The International Bank for Reconstruction and Development/The World Bank 1818 H Street, N.W. Washington, D.C. 20433, USA The findings, interpretations and conclusions expressed in this document are entirely those of the authors, and should not be attributed in any manner to the World Bank, to its affiliated organizations, or to members of its Board of Executive Directors or the countries they represent. For electronic copies of this document in English, French, Russian and Spanish, please visit the PSIA website at www.worldbank.org/psia. Contents Acknowledgments v Acronyms vi Purpose of the User’s Guide vii 1 Introduction 1 2 A Conceptual Framework for Understanding Poverty and Social Impacts 3 Impact of what: What is being analyzed? 3 Impact on what: What is the welfare measure being assessed? 3 Impact on whom: Whose welfare is being analyzed? 4 Impact how: How are impacts channeled? 4 Impact how: How do institutions affect outcomes? 6 Impact when: When do impacts materialize? 6 Impact if: What are the risks of an unexpected outcome? 7 3 Elements of Good Poverty and Social Impact Analysis 9 Element 1: Asking the right questions 9 Element 2: Identifying stakeholders 10 Element 3: Understanding transmission channels 12 Element 4: Assessing institutions 13 Element 5: Gathering data and information 14 Element 6: Analyzing impacts 18 Element 7: Contemplating enhancement and compensation measures 27 Element 8: Assessing risks 29 Element 9: Monitoring and evaluating impacts 31 Element 10: Fostering policy debate and feeding back into policy choice 34 4 Challenges and Operational Principles 39 Constraints 39 Principles 40 5 Possible Summary Matrix 42 6 Conclusions 45 iii A User’s Guide to Poverty and Social Impact Analysis Annex: Economic and Social Tools for Poverty and Social Impact Analysis 47 Bibliography 81 Boxes 1. Asking the Right Questions 10 2. Analyzing the Impact of Mine Closure in Russia: Stakeholder Analysis 11 3. Interest Groups and Collective Action 12 4. Decentralization in Indonesia: Institutional Analysis and Social Accountability 14 5. Illustrative Categorization of Selected Reforms according to Scale of Indirect Impacts 20 6. Impact of Public Expenditures in Indonesia: Average versus Marginal Benefit Incidence 23 7. Impact of Utility Pricing on the Poor in Armenia: Demand Analysis 24 8: Impact of Liberalization in Mexico: Supply-Side Analysis 25 9. Impact of Agricultural Subsidies and Tariffs in Turkey: Multimarket Modeling 26 10. Net Fiscal Incidence in the Philippines 27 11. Impact of the Indonesian Financial Crisis on the Poor: Partial Equilibrium Modeling and CGE Modeling with Micro-simulation 28 12. Labor Downsizing and the Design of Compensation Packages in Vietnam 29 13. M&E Tools for Promoting Accountability and Transparency during Policy Reform 33 14. Poverty and Social Impact Analysis of Cotton Reform in Chad 43 Tables 1. Data Collection Methods 15 2. Considerations in Choosing Impact Analysis Approaches 19 3. Planning M&E as Part of Poverty and Social Impact Analysis 34 4. A Summary Matrix for Poverty and Social Impact Analysis of Reform 44 iv Acknowledgments This User’s Guide is a collaborative product of the website since April 2001. We would like to acknowl- PREM Poverty Group (PRMPR) and ESSD’s Social edge the extensive comments received, among others, Development Department (SDV). It has been pre- from the ADB, Catholic Relief Services, Christian Aid, pared by a team comprised of Jehan Arulpragasam, DfID, GTZ, IMF, Ministrie van Buitenlandse Zaken Sabine Beddies, Sophie Brown, Aline Coudouel, Anis (The Netherlands), OXFAM, World Vision and World Dani, Andreas Groetschel, Sarah Hague, Sarah Keener, Learning. Timothy Kessler, Humberto Lopez, Mattias Lundberg, The team organized a wide range of consultations Jonathan Maack, Nayantara Mukerji, Stefano Pater- during the two years from the initial outline to the nostro and Sharon White. Joyce Chinsen and Nelly final version, including numerous meetings with non- Obias provided technical support to PSIA team governmental organizations, as well as multilateral through the entire production process. Cathy Sun- and bilateral partner organizations. The approach was shine helped to edit the final draft of the User’s Guide. refined through a series of learning events and a sem- The User’s Guide team received useful advice from inar series over these two years. The team also assisted John Page and Steen Jorgensen, and members of the country teams to field test the core approach, review- PREM Poverty and the Social Development Boards. ing the lessons learned during a workshop organized The report has also received substantive inputs from jointly with the IMF and DfID in October 2002. The the editors, Francois Bourguignon and Luiz Pereira da discussions and comments from participants at those Silva, and the authors of various chapters in the Toolkit events contributed to the development of the User’s for Evaluating the Poverty and Distributional Impact of Guide. Economic Policies. Ulrich Zachau and Stefan Koeberle Materials from this work have been posted on a new provided valuable advice, particularly on the interface website launched jointly by PRMPR and SDV of this analytical approach with emerging trends (www.worldbank.org/psia), which will continue to be within the World Bank on policy-based lending. updated with new tools and methods, and country Many others inside and outside the World Bank applications produced by the World Bank, countries provided valuable comments and feedback on the first and external partners. Suggestions and comments are draft, which has been posted on the Bank’s external most welcome at psia@worldbank.org. v Acronyms ADB Asian Development Bank BA Beneficiary assessment CGE Computable general equilibrium DFID Department for International Development (UK) GTZ Deutsche Gesellschaft für Technische Zusammenarbeit (Germany) IMF International Monetary Fund IO Input-output M&E Monitoring and evaluation NGO Nongovernmental organization PETS Public expenditure tracking survey PPA Participatory poverty assessment PREM Poverty Reduction and Economic Management Network (World Bank Group) PRS Poverty reduction strategy PRSP Poverty Reduction Strategy Paper PSIA Poverty and social impact analysis QSDS Quantitative service delivery survey SAM Social accounting matrices SDV Social Development Department (World Bank Group) SIA Social impact assessment SOCAT Social capital assessment tool vi Purpose of the User’s Guide Poverty and social impact analysis (PSIA) involves the and techniques, through the Toolkit for Evaluating analysis of the distributional impact of policy reforms the Poverty and Distributional Impact of Economic on the well-being of different stakeholder groups, with Policies and the Social Analysis Sourcebook, available a particular focus on the poor and vulnerable. PSIA is on the World Bank website. Additional guidance on a systematic analytic approach, not a separate product. economic and social analysis tools and methods is This User’s Guide introduces the main concepts under preparation. The Bank is also developing underlying PSIA, presents key elements of good practice guidance on issues, challenges, and tools that may be approaches to PSIA, and highlights some of the main con- of particular relevance in analyzing specific reforms. straints and operational principles for PSIA. It is intended A summary matrix and reform-specific notes will be for practitioners undertaking PSIA in developing coun- posted on an ongoing basis on the PSIA website. tries. It does not set out operational policy or guidance to More generally, the PSIA website presents resources World Bank staff. This User’s Guide highlights some of on economic and social tools and methods for PSIA, the key tools that practitioners may find useful to analyze country experience in undertaking PSIA for specific poverty and social impacts of policy reforms, but does not reforms, training events and material, and other aim to be comprehensive in its coverage. resources: http://www.worldbank.org/psia. Sugges- As a complement to this User’s Guide, the World tions and comments are most welcome Bank has also developed guidance on selected tools (psia@worldbank.org). vii 1 Introduction Poverty and social impact analysis (PSIA) refers to the pace/sequencing or institutional arrangements of the analysis of the distributional impact of policy reforms reform, or the introduction or strengthening of miti- on the well-being or welfare of different stakeholder gation measures. Finally, ex-post PSIA assesses the groups, with particular focus on the poor and vulner- actual distributional impacts of a completed reform, able.1 The adoption of the Poverty Reduction Strategy which helps analysts understand the likely impacts of Paper approach and of the Millennium Development future reforms. Goals has led to an increased need for more systematic PSIA is not new, and lessons can be drawn from analysis of the poverty and social implications of past experiences.3 Effective PSIA is undertaken early reforms. This User’s Guide is part of a comprehensive enough to inform the design of reforms, clearly sets response undertaken by the World Bank to address out the assumptions behind the analysis, addresses the those concerns.2 risks to policy implementation, considers all stake- The User’s Guide is intended for practitioners holders in the analysis, and promotes transparency undertaking PSIA in developing countries. Given the about expected impacts to strengthen local ownership. broad scope of policy issues, methods, and challenges Analysts have typically faced constraints in terms of involved, the User’s Guide does not specify minimum data, analysis, capacity, and time. Some of these con- standards for PSIA, but rather provides suggestions on straints can be addressed by building on earlier expe- how to approach the analysis. In advocating a multi- rience and by employing flexibility in the choice of disciplinary approach to PSIA, the User’s Guide pres- tools and methods. ents both economic and social analysis tools and The User’s Guide is organized as follows. Chapter 2 methods. While focusing on distributional impacts, introduces the main concepts underlying PSIA and PSIA also addresses issues of sustainability and risks to establishes the conceptual framework. Chapter 3 then policy reform that come with the poverty and social presents an approach to PSIA by reviewing 10 basic impacts of policy changes. elements underlying the sound analysis of the poverty PSIA includes ex-ante analysis of the likely impacts and social impacts of reforms. Chapter 4 considers of specific reforms, analysis during reform implemen- some of the major constraints often identified by PSIA tation, and ex-post analysis of completed reforms. practitioners, especially in developing countries, and Each of these has a specific utility. Ex-ante PSIA can provides basic operational principles for PSIA. Chap- inform the choice, design, and sequencing of alterna- ter 5 proposes a summary matrix that can be a useful tive policy options. During implementation, the mon- tool to capture and integrate the various elements of itoring of a reform and its impacts can lead to good PSIA. Finally, chapter 6 closes with brief conclu- refinement of the reform, a reconsideration of the sions. 1 A User’s Guide to Poverty and Social Impact Analysis Notes 1. This User’s Guide uses the terms “well-being” and “welfare” synonymously. 2. The PSIA website (http://www.worldbank.org/psia) presents guidance on the application of economic and social tools and methods for PSIA, country experience in undertaking PSIA for specific reforms, training events and material, and other resources. 3. The Bank has been engaged in this area for some time, especially in the context of projects. For eco- nomic literature on the topic, see, among others, Squire and van der Tak 1975; Timmer, Falcon, and Pearson 1983; and Gittinger 1985. For anthropological and sociological literature, see Finsterbusch, Ingersoll and Llewellyn 1990; Becker 1997; Goldman 2000; and Brinkerhoff and Crosby 2002. 2 2 A Conceptual Framework for Understanding Poverty and Social Impacts This chapter presents the main concepts underlying Tools for PSIA therefore must be able to address not poverty and social impact analysis. It addresses seven just major macroeconomic reforms, but also the key key areas: structural and sectoral policy changes with which countries are currently contending.2 ■ What is being analyzed? This shift from broad-based “stabilization and ■ What is the welfare measure being assessed? adjustment” suggests that PSIA should be undertaken ■ Whose welfare is being analyzed? on a reform-specific basis. Such an approach also ■ How are impacts channeled? makes the task of analyzing the impact of several ■ How do institutions affect outcomes? reforms more manageable. While it would be concep- ■ When do impacts materialize? tually preferable to assess the combined effect of a ■ What are the risks of an unexpected outcome? series of policy changes in a single analytical frame- work, few tools can accomplish this—and those that Impact of what: can tend to be complex and data-intensive. Therefore, What is being analyzed? it is often more practical to disaggregate expected overall impacts to individual reforms, and consider Poverty and social impact analysis focuses on the sequencing on a reform-specific basis. Consideration impact of policy change. The scope of the policy of the impacts of a “package” of reforms is still perti- debate in the development arena has now broadened nent, however. Where they cannot be analyzed in a sin- beyond macroeconomic stabilization and associated gle analytical framework, their combined effects on measures to also include specific structural and public various groups such as the poor may be most practi- expenditure reforms. This broader view is also implicit cally considered by independently assessing the impact in the poverty strategies of developing countries. In of each reform set on each group. However, such an fact, a review of fifteen Poverty Reduction Strategy approach will tend to lose interaction effects. Papers (PRSP) shows that poverty strategies com- monly focus on enhanced expenditure programs Impact on what: What is the welfare (especially in health, education, water and sanitation, measure being assessed? and roads and infrastructure); institutional reforms to improve governance (such as decentralization, civil service reform, and tax reform); and structural PSIA focuses on assessing distributional impacts on reforms (including trade reform, privatization, finan- welfare, or well-being, including both its income and cial sector reform, and agriculture sector reform).1 non-income dimensions. With poverty now recog- 3 A User’s Guide to Poverty and Social Impact Analysis nized as multidimensional (World Bank 2000a), devel- tion on protected commodity and labor markets can opment efforts are being targeted to address both derail trade liberalization. Similarly, vested interests income and non-income measures of welfare and within the public sector can also derail reforms. PSIA poverty, recently captured in part by the Millennium thus should identify and analyze the impact of policy Development Goals. Until recently, the income dimen- on other stakeholders, beyond the poor, who are sion of welfare was the main focus of poverty and dis- affected by or can influence reforms. tributional analysis, and economic tools were most often applied in analyzing the money-metric welfare Impact how: How are impacts measure.3 Now, however, non-income dimensions of channeled? welfare and poverty—such as human development and social development indicators addressing risk, Policy reforms can be expected to have an impact on vulnerability, and social capital4—are being given various stakeholders through five main transmission closer consideration. In undertaking PSIA, the analyst channels outlined below: employment; prices (pro- will need to choose appropriate indicators of welfare duction, consumption, and wages); access to goods and poverty based on the country and policy context. and services; assets; and transfers and taxes. Each pol- icy reform is likely to have impacts through more than Impact on whom: Whose welfare is one channel. For example, utility reforms might result being analyzed? in changes in prices and access, but might also have an impact on the fiscal stance of a country, and hence on PSIA is concerned with the distributional impacts of transfers and taxes. Further, different stakeholders are policy change on various groups, with a particular likely to be affected differently through these channels. focus on the welfare of the poor and those vulnerable For example, relative price changes will affect net con- to impoverishment. Depending on country circum- sumers and net producers differently, and even among stance, groups may be defined in terms of income these groups the impact may vary. For example, con- class, gender, ethnicity, age, geographic location, liveli- sumers will be affected differently depending on their hood, or other such criteria. In practice, however, consumption patterns or their ability to substitute household members do not always pool resources or goods. allocate benefits equally. When the impacts on differ- ent members within a household are likely to differ, it Employment is important to also analyze intra-household effects. The principal source of income for most households is PSIA is concerned with distributional impacts for employment. To the extent that a policy change affects two reasons. First, policy change can have a direct the structure of the labor market or the demand for impact on the welfare of the poor or other disadvan- labor, particularly in sectors that employ the poor taged groups. Understanding the impacts of policy (such as unskilled, rural off-farm, and agricultural change on these groups can inform the design of pol- labor), the welfare of low-income households will be icy. Second, the distributional impacts of a policy, even affected. There may be direct transmissions through among non-disadvantaged groups, are important for this channel in the case of certain policies: for exam- the effectiveness of that policy and its ultimate sus- ple, restructuring of a state enterprise may lead tainability. Even if a policy change results in overall directly to retrenchment of workers. In other cases welfare gains, it is likely that some groups may experi- transmission may be indirect. For instance, macro ence losses, at least in the short run. While losers may policies may stimulate faster growth, leading in turn to not necessarily be poor, reduction in their welfare may increased employment among the poor; an exchange not be acceptable for social or political economy rea- rate depreciation or trade liberalization may result in sons and may significantly affect the implementation contractions and layoffs in the nontradable sector. and sustainability of the reform. For instance, business Alternatively, some policies will have different impacts and union interests that fear the impact of competi- on formal labor markets and informal labor markets 4 A Conceptual Framework for Understanding Poverty and Social Impacts that employ many of the poor. For example, expendi- electricity grid, particularly among the poor, can also ture increase, reduction or switching may have differ- represent a welfare gain.5 In this regard, privatization ent impacts on formal sector employment and of service provision could either increase or decrease informal sector employment due to labor market seg- access relative to public sector provision.6 Lack of mentation (Agénor and Aizenman 1999). access to key infrastructure or services, either because they do not exist or because they are of poor quality, Prices (production, consumption, and wages) can limit the intended benefit of a policy. For example, Prices determine real household income. Prices in the restructuring a marketing board may be fiscally desir- markets for goods and services differentially affect the able but may eliminate key market services where real income of households to the extent that they con- alternatives do not exist. Structural or cultural norms sume or produce these products. How policy affects (such as restrictions on female mobility or female prices will have an important bearing on income and, property rights) may also impose higher transaction directly or indirectly, on non-income measures of wel- costs or create barriers to access. fare. For all households, but especially for small farm- ers and the self-employed, price changes will affect Assets both consumption and resource allocation decisions. Changes in the value of households’ assets will affect On the consumption side, policies that cause an income and non-income dimensions of welfare. increase in the prices of goods consumed by the poor Changes in asset values can be due to changes in their will have a direct negative effect on household welfare. levels or their returns. Assets themselves can be cate- These can include import tariffs on traded staples, or gorized into five classes, all of relevance to poor house- increased utility tariff rates. Consumer prices may be holds: physical (such as housing); natural (such as indirectly affected as well, for example through expan- land, water), human (such as education, skills); finan- sionary monetary policy that leads to general price cial (such as a savings account); and social (such as inflation. Producers will also be affected by policies membership in social networks that increase access to that cause relative price changes—particularly changes information or resources). Policy changes can have a to the prices of their outputs or inputs. Producer direct or indirect impact on these assets and their incomes are further affected by the difference between returns. For example, land reform may directly result farmgate and market prices, often conditioned by in an increase or decrease in land assets of the poor. transport costs and the degree to which private mar- Policy changes can also impact assets through indirect kets are efficient and competitive, rather than monop- channels. For example, inflationary policies will have a sonistic. Wage changes will affect net buyers and sellers negative wealth effect on those with monetary savings, of labor differently, and policies that change relative while participatory budgeting or community pro- prices will induce shifts in both demand and supply. grams may increase social capital. Pricing or trade changes could affect the natural resource assets of Access households or groups (such as by increasing or Well-being will be affected by access to goods and ser- decreasing deforestation or desertification) or even vices, whether through access to markets and service their human capital (such as by causing a deteriora- outlets or through improvements in the quality and tion in health conditions due to increased indoor air responsiveness of public or private service providers. pollution as a result of energy price changes). In many Policy can affect access directly by enhancing provi- cases, certain assets are also prerequisites to benefit sion of the infrastructure or services in question, or from a reform. For example, if farmers cannot reach a indirectly by removing constraints to access by partic- market due to lack of transport, the benefits of price ular households or groups. For example, improved liberalization are likely to be realized primarily by road infrastructure could dramatically enhance access middlemen and traders. It is important to take into to markets and services for groups in certain geo- account the legal and regulatory frameworks when graphic areas. A policy that expands connections to an analyzing this transmission channel; for example, 5 A User’s Guide to Poverty and Social Impact Analysis there are sometimes constraints on female land own- reform can affect institutions by changing organiza- ership. tional structures, roles, and responsibilities, or rules and incentives, as well as by altering market incen- Transfers and taxes tives—for example, by removing price distortions or Household welfare, finally, is affected by transfers to encouraging competition. These reforms in turn affect and from the household. These transfers can take the the behavior of economic agents and interest groups, form of private flows (such as gifts and remittances) or and thereby economic outcomes, including distribu- public flows (such as subsidies and taxes). Public tion and poverty reduction. finance has a direct impact on the welfare of specific Many reforms depend for their implementation on groups through transfers and tax policy. Public expen- institutional change. This may involve creating new diture programs may focus on granting additional organizations or changing rules and incentives to resources to particular groups through transfer poli- achieve new objectives through existing organizations cies, which may be in the form of subsidies or direct, (for example, improved cooperation among govern- targeted income transfer programs. Social protection ment agencies). Creation or modification of organiza- programs may be useful in protecting the poor against tional structures does not in itself guarantee the risk and vulnerability, depending on their targeting. institutional changes necessary for the reform to suc- Tax policy has direct distributional effects to the extent ceed.8 Changes in formal rules of the game often must that the resources or income of a household are taxed. be accompanied by changes in incentives in order to Regressive tax regimes disproportionately burden less alter the behavior of agents. Moreover, it is often well-off households. Subsidies may be captured by the assumed that institutions (including markets) func- non-poor or may simply be badly targeted. There may tion smoothly and according to formal rules. In prac- also be a conflict between strict progressivity and the tice, high transaction costs, ineffective enforcement, or political feasibility of policies (see Gelbach and Pritch- lack of competition or accountability can lead to sub- ett 2000). Poorer households may also be hurt in the optimal performance of government, market, or civil long run if the funds for public expenditure are bor- institutions. In some cases, institutional change rowed and must be repaid; they will suffer either from accompanying policy reform is not internalized by key any attempt to “inflate away” the debt or from implementing agents, and the behavior of these agents increased future taxes needed for repayment. In soci- can thus lead to perverse policy outcomes. eties with high gender inequities, the intra-household Understanding the impact of a policy reform impact of transfers may warrant special attention. requires an appreciation of the country’s organiza- tional structures and the institutional rules governing Impact how: How do institutions affect them. PSIA therefore depends upon careful organiza- outcomes? tional and institutional analysis of the formal and informal rules, the behaviors of key stakeholders who The impacts of policy reform on economic agents are can affect reform outcomes, and the underlying mediated through institutions. Institutions are the for- dynamics among them. This allows policymakers to mal and informal rules of the game in society; they are determine whether and how the existing rules and the shared understandings that allow organizations to informal practices will affect real costs and quality of interact.7 The impact of a policy reform is influenced goods and services for the poor and other stakeholders by the behavior of organizations. Organizations, in in the context of a specific policy change. turn, act in response to incentives created by the set of public, private, and civil society institutions whose Impact when: When do impacts rules mediate economic activity in the society (Ruther- materialize? ford 1994). These institutions include markets, legal systems, and the formal rules and informal behavior of A major challenge to PSIA is understanding that poli- implementing agencies, including government. Policy cies can affect different groups in very different ways. 6 A Conceptual Framework for Understanding Poverty and Social Impacts This is in large part because some of the economic and tions for the success of the policy reform.9 Assump- behavioral responses to a policy change can take time. tions must be made explicit as to how economic agents What is fixed in the short term may be variable in the and institutions are expected to act (for example, the longer term. Understanding and explaining how sign and magnitude of an elasticity) and how policy short-run losses may result in long-run gains for given impacts would be transmitted to households. A second groups, or how immediate gains may lead to eventual set of assumptions concerns conditions exogenous to losses, is one of the challenges inherent to PSIA. For the policy that need to be in place for the reform to instance, trade liberalization may cause employment achieve its intended impacts. In addition to questions losses in the nontradable sector in the short term. of direct relevance to a reform, risks from underlying However, increased efficiency may later result in eco- country conditions (for example, ethnic tensions) nomic growth, and some of the laid-off workers may need to be factored into the risk assessment. Clearly find jobs in the expanding tradable sector. In addition, identifying and articulating critical assumptions will some consumers may switch to cheaper nontraded serve to sharpen the rigor of the analysis, increase its goods, thereby increasing consumption of these. The transparency, and facilitate its validation (and if neces- combination of all these effects will determine the net sary, correction) by knowledgeable stakeholders. The impact on different groups over the long term. analysis will also permit the monitoring of, and hence To take another example, a policy that pursues or improve the understanding of, transmission channels results in an overvalued exchange rate will benefit some and impacts, with possible adjustments to the reform population groups in the short run (consumers and program over time. importers). But if the overvaluation proves unsustain- able in the long run and devaluation occurs, those same Notes groups will be negatively impacted. The net effect for these groups of having an initially overvalued exchange 1. Fifteen PRSPs were completed by end July 2002 and rate (a gain) followed by a devaluation (a loss) will included those for Albania, Bolivia, Burkina Faso, clearly depend on the magnitude of the deviations. The Guyana, Honduras, Malawi, Mauritania, Mozambique, international evidence suggests that sustained overval- Nicaragua, Niger, Tanzania, Uganda, Vietnam, Yemen, uations may lead to abrupt currency collapses (as in and Zambia. Seven of these strategies call for utility Mexico in 1994, East Asia in 1997, Brazil in 1999, and reforms; 5 for reforms of public sector pensions; 6 for Argentina in 2002) that are likely to generate net long- civil service reforms; 7 for fiscal decentralization; 11 for run losses. The issue becomes even more complex if reforms in the tax system (incl. VAT and other consump- one considers the impact on exporters. Unlike tion taxes); 11 for land reforms; 10 for trade reforms; and importers, exporters are harmed by an overvalued 6 for reforms of the macro-economic framework. exchange rate and are likely to benefit from devalua- 2. Of course, structural changes could have macro- tion. Eventually it will be necessary to consider both the economic effects. For instance, trade liberalization net effects across groups for a given time horizon and could have serious consequences for the fiscal deficit, the net effects over time for a given group. the current account deficit, and macroeconomic sta- bility. Understanding how these impacts affect the Impact if: What are the risks of an poor is critical to PSIA. unexpected outcome? 3. This User’s Guide lays out existing economic and social tools and approaches for distributional analysis The design of reforms is based on underlying assump- in order to give a broader picture of poverty to policy tions about the context and the behavioral response of analysts and decisionmakers. Insofar as the economic key institutional and human actors. If these assump- tools draw on existing examples of such analysis, appli- tions are not realized, reform outcomes are at risk. A cations focus mainly on income/expenditure measures crucial element of PSIA, therefore, is understanding of welfare. Increased attention to assessing the impacts and (publicly) articulating ex ante the key assump- of policy on non-income measures of welfare is an 7 A User’s Guide to Poverty and Social Impact Analysis important priority for future work. The social develop- 7. Organizations are purposive entities (such as ment tools described in this User’s Guide are more public agencies or firms) that have a formal structure focused on non-income dimensions of poverty, such as and seek to achieve certain objectives within the stakeholder interests, social capital, and vulnerability. opportunities and constraints afforded by the institu- 4. The World Bank’s Social Development Depart- tional framework of society (North 1990). ment has developed a new tool that provides data on 8. Formal changes in organizational structure are these indicators from Bank- and non-Bank sources for relatively easy to make but may take much longer to be country-level applications. institutionalized. In such cases, it is important to pay 5. To the extent that increasing access is viewed as a attention to the capacity and accountability of the reduction in transport and transaction costs, it is concerned agencies as well as the power relations effectively reducing the “price” of the good or service within them. Understanding these issues allows for the in question. mobilization of existing capacity and for the tailoring 6. Sometimes an increase in access may come at the of interventions to the institutional and organizational cost of a higher price (or, where there was previously contexts in which they will be implemented. no access at all, access may be granted at a price that is 9. Forecasting or simulating likely impacts of policy prohibitive for the poor). In urban Peru, liberalization by definition presupposes a view of likely causality and of telephone services led to greater access for the poor behavior. Depending on the analyst’s information base as well as lower prices. On the other hand, liberaliza- these can be empirically “estimated” based on the past, tion of electricity has led to greater access and reliabil- derived on the basis of theory, or assessed on the basis ity, but higher prices and lower overall consumption of knowledge of the country context and discussions (Torero and Pascó-Font 2001). with key stakeholders and experts. 8 3 Elements of Good Poverty and Social Impact Analysis Although there is no methodological template for ana- Element 1: Asking the right questions lyzing the poverty and social impacts of policy, it is possible to identify a number of elements that make The first step in the analysis of poverty and social for good-practice PSIA. This chapter outlines 10 key impacts is to identify the reforms that will be subject elements that those attempting to undertake or advise to analysis. This requires identification of the set of on PSIA will need to address: reforms included in the government’s agenda that are likely to have an impact on the distribution of income 1. Asking the right questions or assets. Ideally, if time and resources permit, PSIA 2. Identifying stakeholders should be carried out for each of these reforms. In 3. Understanding transmission channels practice, analyzing all the reforms in a development 4. Assessing institutions plan may not be realistic, so it will be necessary to fur- 5. Gathering data and information ther narrow down the reforms selected for analysis to 6. Analyzing impacts a manageable number. This selection process will 7. Contemplating enhancement and compensation inevitably be a matter of judgment at the country measures level, and will likely depend on factors such as: 8. Assessing risks 9. Monitoring and evaluating impacts ■ The expected size and direction of the poverty and 10. Fostering policy debate and feeding back into pol- social impacts icy choice. ■ The prominence of the issue in the government’s policy agenda While there is a logical sequence to these elements, this ■ The timing and urgency of the underlying policy or does not imply that they need to be undertaken in strict reform, and order or that all the steps will be feasible in every country. ■ The level of national debate surrounding the This chapter provides a broad overview of specific meth- reform. ods and tools that can be used to address each of these elements, pointing to the annex for further details and After selecting the reforms that will be subject to references; those methods and tools discussed in the PSIA, the second step is to formulate key questions for annex are presented in bold in the text. (Building country analysis. This requires an understanding of the under- capacity is presented in chapter 4 as one of the overarch- lying problems that the reform is intended to address ing principles for operationalizing PSIA, rather than in (see box 1). A focus on overly narrow questions, or this chapter as a discrete element of PSIA.) exclusively on short-term effects, may obscure issues 9 A User’s Guide to Poverty and Social Impact Analysis that could prove critical to the achievement of a par- Element 2: Identifying stakeholders ticular policy objective, or to informing policymakers and stakeholders of the tradeoffs inherent in a certain After asking the right questions and identifying the policy. A useful device is to conduct a problem diag- problem that requires solution, an early identification nosis by organizing the chain of cause-effect relation- of relevant stakeholders is important. Not only can ships, from policy objectives and policy actions to policy choices affect different stakeholders or eco- impacts, in the form of a hierarchical problem tree,1 in nomic agents in different ways, but these stakeholders order to formulate relevant research hypotheses. can also influence whether a policy is adopted and Identifying policy constraints is a key component of how it is implemented. the analytical process and can often prevent subse- Stakeholder analysis identifies people, groups, and quent missteps. Policy reforms are often implemented organizations that are important to take into account to remove constraints that stand in the way of achiev- when conducting PSIA.2 It identifies and analyzes those ing certain development goals. For instance, a country who are affected by the policy, as well as those who can may be unable to balance its budget because of unsus- potentially affect policy implementation. Identifying and tainable losses by state-owned enterprises. The prob- disaggregating the stakeholders in the first category— lem in this case will be to improve the overall fiscal beneficiaries and those who suffer adverse impacts—is balance as well as the performance of individual agen- central to the analysis of poverty and social impact of cies. For some objectives there may be multiple con- policy. They can be disaggregated by a large number of straints, some being more important than others. In characteristics such as household type, household size, such cases, it may be necessary to pursue more than ethnicity, gender, location, occupation, and so forth. For one policy reform, but also to be on the alert for inter- modeling work, stakeholder analysis can serve as an active effects that those reforms might have on each input into determining how best to disaggregate repre- other. In another example, a policymaker faced with sentative household groups or subgroups. Stakeholders inadequate public revenues may decide to raise taxes. in the second category—organized groups such as However, this will not be the appropriate response if unions, business associations, donors, and civil society the real problem is that expenditures are too high, organizations—may become sources of support or rather than that revenues are too low. In order to avoid opposition to policies. Analyzing such influential actors is inappropriate or mismatched policies, it is important essential to understand behavioral responses that condi- that the constraints on development objectives be tion impacts, and the likelihood of reform success. Box 2 made explicit—rather than assumed—at the begin- illustrates the use of stakeholder analysis to address the ning of the PSIA process. impact of mine closures in Russia. Box 1. Asking the Right Questions The analysis of a fiscal reform ideally includes an evaluation of between distribution and generation investments, were equally the short-term impact as well as the expected longer-term important to achieving a more sustainable energy sector. impact and the assumptions underlying the realization of long-term benefits. But beyond the dynamic impact of the Reform of the sugar sector in Guyana is being analyzed reform, the analyst should also consider whether structural because of its fiscal cost and the number of people affected by issues are affecting the country’s fiscal performance. the reform. The analysis is comparing the reform’s direct impacts on employment and indirect effects on municipal ser- In PSIA work in the Pakistan energy sector, the initial focus vices and dependents with the long-term employment and fis- was on an electricity tariff increase to cover costs that repre- cal losses that would likely occur if the sector were to continue sented a significant and chronic fiscal drain. Further problem in its current state, given the continuing decline in world sugar analysis revealed that questions about the increased costs of prices and the phasing out of preferential prices under the power generation and non-tariff charges, and the imbalance Lomé Accord. 10 Elements of Good Poverty and Social Impact Analysis Box 2. Analyzing the Impact of Mine Closure in Russia: Stakeholder Analysis In the early 1990s, the Russian coal industry was in a state of municipal and oblast-level governments were based par- crisis. A large number of economically inefficient mines were tially on the revenues that each could muster in the event of kept afloat by subsidies that reached $2.76 billion (more than mine closure. 1 percent of GDP) in 1994. Restructuring entailed closing 183 loss-making mines and downsizing the workforce (including The differences between stakeholder groups lay largely in their those involved in coal production, administration, social ser- analysis of the core problem. On the one hand, the Ministry of vices, and other auxiliary activities) from 900,000 in 1992 to Energy, regional governments, the labor union, and the mine 328,000 by end-2001. face workers advocated a narrow solution focused on preserv- ing the mining industry in some form. On the other hand, The Bank provided $1.3 billion in loans and played a major municipal governments, social service workers employed by role in helping the Russian government develop its strategy for the mines, and local businesses focused on the need to find new mitigation of the poverty and social impact of coal sector drivers of growth in mono-industry towns, as well as sources restructuring. The team carried out a stakeholder analysis of funding for services previously supplied by the mines. using structured interviews in Moscow, mine visits, and dis- Municipal governments did not have the revenue base to sup- cussions with union leaders. The analysis was designed to clar- port the schools and other services formerly provided by the ify the nature of the problem, identify the interests of various mines, and were hard hit by the closures. actors, and develop a solution for effective fund transfer using existing actors. An Interagency Coal Commission with representatives from municipalities, ministries, and government agencies helped The team grouped the stakeholders into several categories. discuss and plan reforms. The Ministry of the Treasury was Government ministries were not seen as neutral agents, and identified as a transparent channel through which social pro- their interests were explicitly identified. Similarly, the tection funds could be transferred directly to the workers, options of mine employees were differentiated by their pre- rather than moving funds through the Ministry of Energy and vious employment. Workers on the mine face, analytic and regional governments. The analysis of stakeholder interests administrative support workers, and workers in the schools was used to create a system of checks, balances, and independ- and hospitals previously funded by mine revenue would be ent assessments to ensure that all actors followed the rules laid impacted differently as mines closed. The interests of out in mine closure plans. Sources: Lockhart 2001; Haney and others 2003. A distinction should be made between stakeholders social tensions, key informant interviews may be who share multiple characteristics that enable them to needed to analyze the interests of stakeholders whose coalesce as a cohesive group (for example, labor support is critical to reform implementation, includ- unions) and those that are analytical categories rather ing those within government agencies, or interest than organic groups (for example, “the fourth income groups able to influence reform. Analyzing interests of quintile” or “the poor”). Stakeholder analysis goes stakeholders who are less organized may involve spe- beyond simply identifying groups to analyzing the cial surveys or focus groups. stated or unstated interests of actors in relation to a Stakeholder analysis contributes to an assessment policy, as well as the nature and degree of their organ- of the extent of country ownership of a particular pol- ization or ability to mobilize behind a common pur- icy in order to predict how different interests are likely pose (see box 3). To the extent that groups of the to influence government in general, and the policy second type are atomized or unorganized (such as process in particular. Ownership assessment reveals landless peasants, non-unionized workers, small busi- sources of potential resistance to policy change and nesses, consumers), they are less likely to be able to provides a rough estimate of the location and extent of easily voice their opposition to or support for a policy, pressure that government will face in adopting a pol- even if their support may be crucial to reform success.3 icy reform. This helps to assess government’s willing- While secondary resources such as social science ness to undertake and stick with the reform over time. research, news media reports, and advocacy literature Weak ownership can lead governments to abandon can help identify broad political economy issues and reforms midterm or produce distorted policies. For 11 A User’s Guide to Poverty and Social Impact Analysis Box 3. Interest Groups and Collective Action Estimating the influence of a particular group over decisions is such as consumers are typically less influential. Many develop- as much art as science. However, there are some useful criteria ment interventions are designed to reduce or eliminate rents for predicting the propensity of a group to lobby the govern- among a small group of privileged interests and increase the ment. The logic of collective action suggests that interests will overall welfare of the public. These are precisely the policies exert more pressure on policymakers or elected leaders when: that are most likely to be fought, making either tough political (a) the number of group members is small; (b) the benefits or decisions or a concerted communications strategy paramount. rents that would accrue to each member from the desired pol- However, if the impact is sufficiently large, public interest icy are very substantial and easy to perceive; and (c) members groups may emerge to advocate the interests of the less power- have the means, especially the financial resources and net- ful, or violent street protests may break out. For example, the works, to protect their interests. The behavioral premise is sim- Consumers Rights Commission of Pakistan was formed to ple: people fight harder when they have a large personal stake. advocate consumer interests on tariff reform, and this lobby- In contrast, the more diffused interests of unorganized groups ing is substituting for more violent forms of urban protest. example, some countries pursue bank deregulation sion channels that are going to dominate and require and privatization, but refuse to remove barriers to analysis will vary and will have distinct impacts on dif- entry because of entrenched interests, resulting in an ferent stakeholders, depending on the reform and the oligopolistic sector that charges high interest rates and country context. Impacts may differ along two key provides poor services. dimensions: first, they can be direct or indirect, and Factors that typically affect ownership can be ana- second, they can occur in the short or the long term. lyzed by looking at both the political economy of a Some policy reforms may have primarily direct country and its diversity (based on ethnic, religious, impacts, that is, impacts that result directly from linguistic, gender, and age differentials). By considering changes in the policy levers altered by the reform. For political economy, analysts can identify affected groups example, an increase in the value-added tax will trans- and assess their influence over government decision- late directly into lower purchasing power for a given makers. Taking stock of diversity is important because disposable income. Reforms may also have important reforms may polarize existing tensions in the short indirect impacts, that is, impacts resulting from the term, even while improving welfare in the long run. reform through channels other than the actual policy lever or action. Thus an increase in value-added tax Element 3: Understanding transmission rates will have a positive impact on the fiscal stance of channels the country; if this is translated into increased govern- ment expenditure, it will have impacts on various Once potential stakeholders have been identified, an groups of households through the goods, services, important early step in the PSIA process is to delineate transfers, and subsidies they receive. Such a stronger the channels by which the analyst expects a particular fiscal stance also will likely generate improved growth, policy change to impact various stakeholder groups.4 affecting household welfare. It is important to explicitly present the hypotheses and The second critical dimension relates to the timing assumptions underlying this analysis. These can then of impacts. Given that the nature of the impacts may be tested empirically through economic and social change over time, so will net impacts on various stake- analysis techniques. holders. To keep our earlier example of an increase in As discussed in chapter 2, the expected impact of a value-added tax rates, the direct impacts on purchasing policy change on the welfare of target groups and power will likely be felt in the very short term, while the other key stakeholders takes place through five main indirect impacts of improved service delivery and transmission channels: employment, prices (produc- higher growth will typically take more time to materi- tion, consumption, and wages), access to goods and alize. Stakeholders might therefore feel both negative services, assets, and transfers and taxes. The transmis- and positive impacts, but at different points in time. 12 Elements of Good Poverty and Social Impact Analysis Element 4: Assessing institutions also be useful to undertake concurrently an analysis of the constraints to private sector entry and participa- As discussed above, institutions affect the impact that tion. Quantitative or qualitative household surveys policies have on poverty and the welfare of different can also reveal who buys services, where, and at what households or groups. First, institutions mediate the price. Quantitative service delivery surveys and citizen transmission of certain policy impacts to people. Under- report cards can be applied to the analysis of the effec- standing social and market institutions helps to under- tiveness of state marketing agencies. Price analysis is stand impacts of a given policy change (such as always useful in ascertaining the competitiveness of a deregulation, privatization, or removal of an export tax). market and of market structure. Second, institutions are often the object of many types of policy reform. Privatization, civil service reform, decen- Analysis of implementing agencies tralization, and expenditure management reform are In judging the likely poverty impacts of reforms that examples of institutional reform that involve changes in involve a change in government responsibility, or cooper- the incentives and rules that govern public and private ation among government agencies or other implementing organizations. Third, many policy changes depend on agencies, the flow of decisionmaking, information, and particular organizations for their implementation. The resources within and among organizations needs to be incentives, performance, and capacity of these organiza- considered (see box 4). Two options for collecting this tions will be critical to the actual implementation of the kind of information are organizational mapping and the policy and thus its impact. Fourth, aside from well- institutional assessment tool. known barriers to entry faced by the poor, institution- Organizational mapping is a method that enhances specific intents of the reform may introduce new understanding of the internal behavior of organiza- transaction costs stemming from information asymme- tions by creating an inventory of the actors carrying try and bounded rationality that affect market behavior out reforms and explicitly revealing relationships or access to public services (Powers 2003). Two key areas among them. Organizational mapping has two com- of focus for PSIA are the analysis of market structure ponents: static mapping and process mapping. Static and the analysis of implementing agencies. mapping identifies ex ante the specific public actions associated with a policy reform, and the organizations Analysis of market structure (which may be outside government) responsible for Surveys among consumers and producers of goods implementing them. It maps out the relations among and services can be useful approaches to enhancing the implementing agencies and identifies those understanding of context-specific market structure. expected to support or obstruct the reform. The exer- Identifying the nature of the market (monopoly, cise is informed by earlier stakeholder analysis (see the monopsony, oligopoly, perfectly competitive, etc.) and section above on identifying stakeholders) of govern- what determines this market structure (natural ment and other organized actors. Process mapping monopoly, restrictions to entry, or collusion, for draws on work carried out to improve efficiency in the example) is a crucial first step toward understanding public and private sectors in industrialized countries the enabling conditions that would need to be created (Hunt 1996). It identifies current practices and norms for market reform to lead to improvements in per- in relevant organizations that cannot easily be gleaned formance and better outcomes for the poor. from documents or diagrams. It does so by tracing Enterprise (or trader) surveys can be useful for flows of critical resources, decisionmaking authority, understanding the nature of the market, the number and information in the current system. This helps cre- and types of economic agents, and market constraints, ate an understanding of the rules and incentives that as well as de jure and de facto barriers to entry and affect internal behavior and the extent to which organ- transaction costs. In the case of privatization or liber- izations pursue development objectives. Process map- alization, where an assumption is that market entry ping can help identify constraints to effective policy will lead to competition and price reduction, it might implementation at three levels: in organizational pro- 13 A User’s Guide to Poverty and Social Impact Analysis Box 4. Decentralization in Indonesia: Institutional Analysis and Social Accountability A research team led by Scott Guggenheim carried out an lage level. An existing government agency, the Department of institutional analysis of village-level governmental structures Community Development, acted as a partner and enforcing and traditional village decisionmaking bodies in Indonesia as agency. part of a decentralization project designed to address corrup- tion and top-down decisionmaking. The Kecamatan Devel- The KDP used transparency and social accountability to make opment Project (KDP) was committed to using local capacity the new institutional structure work. Existing village councils rather than developing a separate project implementation at the kecamatan (subdistrict) level, which were formal organ- unit. The analysis, conducted through focus groups and izations that had met once a year to feed into the government’s interviews with government officials, helped to identify the planning process, became the primary decisionmaking bodies. relative strength and capacity of existing systems, the flow of Decisions on proposals from villages were made in public money and information, and the location and nature of deci- meetings of the council, procurement forms were limited to sionmaking in the chain. The project changed the role and one page, expenditure information was kept on cash ledgers, authority of those structures, shifting the locus of power and information about the program was disseminated through within the system from regional governmental bodies to vil- posters, flyers, and radio broadcasts. Further, the KDP worked lage councils. Through the interview process, the team iden- with the Association of Independent Journalists to ensure tified the Village Infrastructure Project as a field-tested media coverage and gave small grants to reporters to build means to get money directly from central accounts to the vil- capacity for independent reporting. Source: National Management Consultants 2000. cedures, in the relationship between organizations, (a) roles; (b) knowledge and access to information; (c) and in the relationship with the authorizing environ- incentive structures; (d) receptivity to policy change; ment. Addressing them may require fine-tuning pro- (e) capacity; (f) resources or financial clout; and (g) cedures, recasting fundamental rules of operation, or scope to adapt to the new reform agenda. The advan- even replacing entire organizations. Process maps are tage of the institutional assessment tool is that it can constructed through in-depth, semi-structured inter- enable more systematic analysis of issues ranging from views with staff at all levels of the organization, focus- political incentives to administrative capacity at low ing particularly on those at the “front line” of cost. The disadvantage is that the tool relies on a desk delivering services. The main advantage of organiza- assessment, and lacks the interactive dimension of tional mapping is its ability to expose a problem area interviews with staff of the organizations that are being that may not be readily seen by relying directly on reformed. The tool is currently better suited for the stakeholders to describe their interests and constraints analysis of institutions with respect to investment (see box 4). A drawback is that it is more time-con- operations, but it could be used to assess institutions in suming, costly, and technically demanding than the context of the implementation of policy reform.6 guided questionnaires. Good process mapping needs to be used iteratively to test assumptions by monitor- Element 5: Gathering data and ing institutional performance over time. information The institutional assessment tool was designed to per- mit an institutional analysis of various components of Assessing data needs and available data and planning a project. The tool consists of questions that help the the phasing of future data collection efforts are an analyst structure thinking about the complex relation- important part of PSIA. Identification of data needs ships and processes within organizations upon which will benefit from the prior identification of policy reforms depend.5 The questions are used to evaluate issues, stakeholders, and likely transmission channels, the effectiveness of institutions, from performance as outlined above. Four discrete steps are suggested: incentives to their capacity to implement policy. They mapping out desirable data for PSIA; taking stock of address key issues of relevant organizations, including: available data and analysis; coping with PSIA data lim- 14 Elements of Good Poverty and Social Impact Analysis itations up front; and addressing PSIA data limitations The approach based on quantitative analysis, numeric today so they do not limit PSIA in the future. data, and close-ended data collection offers certain advantages. Analyzing the poverty and distributional Mapping desirable data for PSIA impacts of policy on welfare indicators will require link- Analysis of the poverty and social impacts of policy ing data at the macro or sectoral level (generally corre- can be extremely data-intensive. Specific data require- sponding to the level of policy intervention) to ments will, of course, depend on the nature of the disaggregated household-level data that capture the wel- reform being analyzed and the analytical tool or tech- fare measure of interest (usually an income/expenditure nique being employed. In approaching data and meth- aggregate, but possibly other welfare measures such as lit- ods, it is useful to distinguish among data collection eracy or infant mortality) and other behavioral variables instruments (close-ended or open-ended); data type (such as access). Close-ended surveys have generally been (numeric or non-numeric); and associated methods of used to collect such data. For analysis to be generalizable, data analysis (quantitative or qualitative). Tradition- data should be derived from a random sample. When the ally, analytical approaches have been either quantita- reform is expected to impact only a discrete group (for tive in nature and based on numeric data collected example, laid-off mine workers) or a geographic subre- using close-ended data collection methods, or qualita- gion, purposive sampling of just that group or subregion tive in nature and based largely on non-numeric data may be more appropriate and economical than a nation- collected using open-ended data collection methods. ally representative survey. Numeric data can be used to “Mixed methods” are increasingly being employed and undertake statistical and multivariate analysis to test are extremely useful for PSIA. hypotheses and determine relationships (see table 1). Table 1. Data Collection Methods Aspect Close-ended Open-ended Data collection • Structured, formal, predesigned • In-depth, open-ended, or semi-structured interviews, such as key informant interviews and case instrument questionnaires, such as living standards histories, focus group interviews, community interviews, mini-surveys. measurement study, social impact assess- • Ethnographic observation. ment surveya, willingness-to-pay survey, • Systematic (or directed) consultation, such as beneficiary assessment. client satisfaction survey, citizen report card. • Participatory data collection methods, such as participatory action research, participatory rural appraisal, participatory public expenditure review. • Focus group discussion. • Community and institutional surveys. • Written documents (for example, program records, process documentation, media reports). • Participatory visual exercises. Analytic method • Predominantly statistical analysis. • Inductive reasoning. • Deductive reasoning. • Interactive analytical process: research questions formulated, answered, and analyzed iteratively, e.g. in stakeholder analysis, participatory poverty assessment, scenario analysis. • Methods tailored to social context. Advantages • Findings can be generalized. • Able to analyze behavioral responses, explore new hypothesis, or recognize previously • Can quantitatively estimate size and undiscovered phenomena. distribution of impacts. • More effective in capturing intra-household features and non-income dimensions of poverty. • Explains statistical correlations. • Can identify particularly vulnerable subgroups. • Allows respondents to articulate their own views. Disadvantages • Results not available for long period of time. • Findings difficult to generalize, and difficult to aggregate and compare systematically. • Limited types of information can be gathered. • Fieldwork requires greater research skills than for quantitative enumeration. • Can sometimes be expensive and time-consuming. Note: This table is intended to provide an indicative distinction between these methods and not a comprehensive description of individual techniques. a. Social impact assessment adopts a more eclectic approach to data collection, choosing among open-ended, semi-structured, and close-ended instruments to fill information gaps for mixed-method analysis. Sources: Adapted from Carvalho and White 1997; Baker 2000; and World Bank 2002a. 15 A User’s Guide to Poverty and Social Impact Analysis Likewise, the approach based on qualitative analysis richer understanding of the impacts of policy on dif- and open-ended data collection has particular ferent subsets of the population, and to analyze strengths. A variety of open-ended data collection counter-intuitive results that might otherwise be dis- methods can be used to collect non-numeric informa- missed as spurious. And a successful mixture can elu- tion relevant to PSIA. Qualitative and contextual data cidate history, context, process, and identification of can be collected through participatory appraisals, asset transmission channels and differential impacts. While mapping, and structured interviewing of individuals, mixed methods can involve higher costs, requiring communities, or focus groups. This information can more complex skills and coordination with multidisci- be used to undertake stakeholder analysis (discussed plinary teams, the benefits in some cases outweigh the above), participatory poverty assessment, beneficiary costs. As the work of Amartya Sen and others demon- assessment, institutional analysis, and risk analysis strate, economics has contributed a great deal to, and (discussed below). Open-ended data collection meth- made liberal use of, qualitative analyses. ods such as those described in table 1 permit an inter- active analytical process—one in which research Taking stock of available data and analysis questions can be formulated, answered, and analyzed The first element of the stocktaking is to ascertain the iteratively in the field. The open-ended approach existence of key data. This will allow identification of allows subjects to articulate the research problem and data gaps that need to be filled or taken into account question. This interactive analytical process could when choosing an analytical approach. Household sur- enable quicker turnaround and a shorter time lapse vey data are generally pivotal to undertaking quantita- between questionnaire design and analysis than close- tive poverty and distributional analysis.10 An ended data collection methods and associated statisti- important consideration for poverty and social impact cal analyses.7 Open-ended data collection methods analysis is whether, in addition to a welfare (e.g. may also be undertaken using a random sample or a income/expenditure) aggregate, there is information in purposive sample and may also be quantified to tabu- the survey that provides the variable (or the computa- late and analyze information.8 tion of such a variable) related to the policy lever in In undertaking PSIA there is much benefit to mix- question—for example, household expenses on trans- ing and, where possible, matching elements of the port, or specifically public bus transport, if bus tariffs above approaches.9 This includes drawing on different are to be increased; or purchases of maize at subsidized types of data collected by different techniques for mul- prices, if the subsidy is to be removed. Other important tidisciplinary analysis. It is important to be aware that sources of data include sector studies—which may economic analysis is not limited to quantitative analy- include administrative data, household survey data, sis. Close-ended and/or open-ended data collection and qualitative information—and information on the techniques can be used to generate numeric and/or macroeconomic situation, including national accounts. non-numeric data, for analysis using quantitative In analyzing policy reform, it is very useful, where pos- and/or qualitative techniques and approaches. More- sible, to test the robustness of conclusions by matching over, analytical methods can be mixed sequentially or data from different sources. This is often referred to as in parallel over time. Mixed methods can leverage the “triangulation,” the practice of validating results benefits of both quantitative and qualitative analysis. among three different sources. For example, in Arme- Qualitative analysis can inform the design of close- nia three different sources were used to compile and ended questionnaires or the specification of an econo- compare information on consumption of, and expen- metric model and generate hypotheses to be tested diture on, utilities (using household survey data, utility further through quantitative research. Hypotheses accounts data, and focus groups). Similarly, for partic- generated by qualitative analyses can be tested for gen- ularly controversial issues, participants in discussion eralizability using quantitative approaches. The results groups may have an incentive to exaggerate or mini- of quantitative analysis can be further examined using mize certain impacts. Matching or triangulating results open-ended data collection methods to develop a is particularly important to validate such results. 16 Elements of Good Poverty and Social Impact Analysis Second, after identifying the availability of relevant survey data. In the interest of building national capac- primary data, ascertaining the existence of analysis and ity and enhancing ownership of the data and analysis, secondary data on the policy issue at hand is an obvi- where possible these data collection efforts should be ous next step. In many instances, burning policy issues undertaken through national institutions, such as the have been the subject of analysis and debate in the past; statistical agency, ministries, universities, or other it is useful to draw on whatever analysis already exists, research organizations. A national household survey is and whatever public debate has already occurred. Pro- a large undertaking; it can take months to plan and ject and program documentation, as well as data and implement such a survey and analyze the resulting analyses from other development agencies, are invalu- data. Where possible, it is useful to identify planned able. For sectoral reforms, information from existing household surveys that are to be fielded imminently sector analysis, including administrative, household and to add key questions relevant to the policy issue at survey, and qualitative data, can strengthen PSIA. Aca- hand. These questions can leverage a wealth of analyt- demic research and theses can also yield in-depth ical possibilities in the context of a full-fledged house- insights not normally available in official reports. hold survey. Third, it is useful to ascertain and build the capac- Alternatively, there are now several “off-the-shelf ” ity of local agencies involved in data collection and survey instruments that can be used to quickly collect, analysis (such as national statistical offices, ministries, enter, and analyze data (for example, the Core Welfare universities, research organizations, consulting firms, Indicator Questionnaire, or CWIQ, survey). Social NGOs, and so forth) to collect and analyze data. impact assessment surveys, based on purposive sam- pling, can often be turned around in a shorter time Coping with PSIA data limitations than a representative national household survey. Like- In many countries there are severe data limitations to wise, depending on the reform issue at hand, quantita- conducting poverty and social impact analysis. Some tive surveys can be employed using a purposive sample or many of the desired data outlined above may simply (for example, among workers of a firm that is to be not be available. In this case, policymakers and analysts downsized).11 When possible, use of mixed methods, will need to consider several options, outlined below. combining qualitative and quantitative analytical First, they can adapt the analytical approach to data approaches to triangulate results, helps to generate currently available. If the urgency of policy action richer and more robust findings. The use of data from severely limits the time available to gather further data, a non-representative sample to estimate parameters expeditious analysis using the limited available data may sometimes be required, and the “borrowing” of may be required. Some tools and approaches to parameters from other countries may also be needed. poverty and social impact analysis are far less data- Again, clearly stating assumptions (for example, that intensive than others. Adapting the analytical these elasticities apply to the population at hand) will approach to the available data, such as using time-use be important in these instances. Care should be taken data or focus group data to construct a simple house- when generalizing from such a purposive sample.12 hold model, might be the best course of action. While Third, policymakers can rethink the policy decision any analysis entails making assumptions, taking short- or the sequencing and pace of reform. One option is to cuts generally means making more assumptions in postpone the policy decision until adequate data can order to proceed. The analysis should be honest and be collected and appropriate analysis conducted. If transparent in stating these assumptions. Qualitative this course is taken, the costs of delaying reform (a techniques, such as individual, community, or focus policy decision in itself) will need to be considered. group interviews, can be used to validate assumptions Other possibilities are to pilot or phase the reform, so and inform the design of quantitative surveys. that progress can be monitored before a final decision A second option is to collect more data. If critical is made to implement a national program. data gaps have been identified, it may be useful to In the end, a tactical judgment will have to be gather the data needed—whether administrative or made as to how to proceed based on these consider- 17 A User’s Guide to Poverty and Social Impact Analysis ations. This judgment will be influenced by the time Considerations in choosing approaches to impact and resources at one’s disposal, which in turn will analysis depend critically on political and economic pressure In general, four factors will condition the choice of for action. In most cases, decisionmakers will not approach or tool to be used in analyzing the poverty want to embark on a major policy change without a and distributional consequences of a given reform: the sound understanding of the poverty and social importance of indirect impacts; data availability; time implications of a policy action, particularly if such availability; and capacity. For purposes of presenting a action is aimed at reducing poverty. In some simple typology, these four factors can effectively be instances, however, political or economic impera- collapsed into two dimensions. tives (as in a crisis situation) may lead policymakers The first is the importance of indirect impacts. As to take quick action. Where this happens, it will be noted above, policy changes may have direct and/or important to undertake PSIA as soon as feasible and indirect impacts, depending on the reform in question to consider measures to protect the poor from and the structure of the economy. A policy reform has adverse impacts and vulnerability to significant high indirect impacts if the net effect is transmitted risks (see section on compensatory measures, through several channels and markets, leads to behav- below). ioral changes at the household level, and/or has multi- ple round effects that may take time to work Addressing PSIA data limitations today so that they themselves through the economy. An example could do not limit future PSIA be a massive devaluation that immediately results in When circumstances dictate that a policy decision changes in relative prices, consumption, and power needs to be made without adequate data, it is impor- structures, but over time might be expected to lead to tant that steps be taken to improve the information shifts in the structure of employment and the econ- set over time. Since PSIA is necessarily a dynamic omy, changes in productivity, improved governance process of formulating and adjusting policy based on (by removing rent seeking), and possibly growth. increased knowledge, it would also be important to Second is the availability of data, time, and local put into place a strategy to gather the necessary data capacity. As discussed above, data availability and to enhance the basis for further and future (ex-ante domestic capacity for data collection and analysis will and ex-post) analysis of the poverty and social necessarily constrain the type of approach adopted. impacts of policy. Such a strategy can be designed in a The simple typology presented here collapses manner that builds national capacity for data collec- data/time/capacity into a single dimension. Over time, tion and analysis. Where possible, a strategy for data an objective of PSIA ought to be to improve the capac- collection should be linked to the timetable for policy ity of local practitioners and users. Wherever possible, formulation, or for policy review and reformulation. it is important that local partners—in the government In other words, the reason for developing a strategy or outside organizations, as appropriate—become for future data collection is not solely to permit ex- involved both in selecting tools for analysis and in post monitoring and evaluation of a current policy applying them. This engagement can be the basis for decision, but also to lay the groundwork for future ex- domestic capacity building, so that over time local ante analysis. Developing such a strategy is an integral analysts rather than international specialists conduct a part of PSIA. larger share of the analysis. Table 2 presents an indicative typology of how an Element 6: Analyzing impacts analyst may want to select an approach. It lays out a choice of tools based on the importance of indirect This section begins with general considerations in impacts for the reform in question, taking into consid- choosing approaches to impact analysis and then pro- eration constraints of data, time, and capacity.13 This vides an overview of several broad classes of methods table is only indicative, and the reality will vary for estimating impacts. depending on the country circumstances and the 18 Elements of Good Poverty and Social Impact Analysis Table 2. Considerations in Choosing Impact Analysis Approaches Data/Time/Local Capacity Availability Low Medium High Low • Beneficiary assessment • Social impact assessment • Poverty mapping • Participatory poverty assessment • Benefit incidence analysis • Social capital assessment tool Indirect impacts • Demand/supply analysis • Household models • Social impact assessment • Multimarket analysis • Social accounting matrices • Collect more data • Reduced form • Input/output models High • Use tools in adjacent cells in • Computable general equilibrium conjunction with assumptions • Macro-model + micro-simulation Note: The tools presented along the dimension of “Data/Time/Capacity Availability” are additive across rows. That is to say, any tool that can be used in the context of lower data/time/capacity can also be used with higher data/time/capacity, and certain tools, such as social impact assessment, can be applied to examine higher indirect impacts. reform in question. Choices will therefore have to be Once the relevance of indirect impacts has been made on a case-by-case basis. determined, the next consideration will be the avail- In contemplating the choice of tools, a helpful first ability of data, time, and capacity. Where these are in step is to consider whether the reform in question is short supply, the analysis might need to use simpler likely to have low or high indirect impacts. The answer tools and methods in the short term. In such cases, an will depend partly on the scale of the reform and its action plan to strengthen data and capacity should be importance to the economy, as well as the time hori- put in place for more robust analysis in the future. This zon. With regard to the latter, elasticities are typically way countries in the “low” data and capacity situation lower in the short run than in the long run. For could aim to improve their information base so they instance, a tax reform may have low indirect impacts have the option of adopting methods in the “medium” in the first year of implementation, but much larger and “high” columns, as appropriate. (See Annex for ones in subsequent years as agents adjust to the new data, time, and skill requirements for each tool.) tax rates. As another example, the indirect impact of PSIA can utilize various methods and tools, many utility reforms could be very low, in the case of of which require the combined skills of various disci- changes in tariffs paid only by a handful of rich con- plines (for example, macroeconomics, microeconom- sumers—or they could be very significant, as with the ics, social and political analysis). Where feasible, it is wholesale restructuring of the electricity sector in an advisable to integrate economic and social analyses in industrial country. Moreover, the impact of individual order to deepen the analysis. For instance, social reforms may be low, but if they are taken as a package impact assessments can be used to help define the the combined impact could be high. parameters and explanatory variables used in econo- While country circumstances and reform specifici- metric modeling, and conversely, an understanding of ties will ultimately determine the strength of indirect economic dynamics and constraints can strengthen impacts, it is possible to broadly classify specific the social analysis of a given policy. reforms as having lower or higher indirect impacts, The rest of this section briefly lays out the different based on the scale on which they are undertaken in social and economic tools for PSIA, and the reforms to most low-income countries. Box 5 provides an indica- which they are best applied. It first presents tools for tive breakdown. social analysis, which can be used in conjunction with 19 A User’s Guide to Poverty and Social Impact Analysis Box 5. Illustrative Categorization of Selected Reforms according to Scale of Indirect Impacts This categorization is indicative only: actual indirect impacts of a revenues; improvements in tax administration; cost recov- given reform will ultimately be driven by country circumstances, ery. including the scale and complexity of the policy adjustment. • Land reform: distribution to landless; changes in legal rights to own, exchange, and inherit land. Reforms with typically higher indirect impacts • Utility reform: restructuring of state-owned utilities; • Macroeconomic and fiscal reform: monetary policy increased private participation; full divestiture. reforms, affecting inflation and interest rates; broad exter- • Financial sector reform: privatization/closure of state nal policy, affecting balance of payments and reserves; and banks; promotion of financial institutions serving the broad fiscal policy, affecting fiscal deficits. poor. • Trade and exchange rate reform: reform of tariff and non- • Privatization: lease of assets; private management con- tariff barriers; exchange rate adjustments. tracts; full divestiture. • Agricultural reform: elimination of administered prices; • Civil service retrenchment: layoffs, reductions in the wage changes in domestic subsidies and taxes; abolition of mar- bill. keting boards. • Decentralization of public services. • Financial sector reform: liberalization of interest rates; alloca- • Social safety nets: changes in targeted cash/in-kind trans- tion of credit; lowering barriers to entry; regulatory reform. fers; benefits to needy groups (such as AIDS orphans); social insurance benefits. Reforms with typically lower indirect impacts • Pensions: scaling back pay-as-you-go public schemes; • Public finance reform: changes in allocation and level of increased private provision; introduction of social pensions public expenditures; changes in level and composition of (cash assistance for poorest pensioners). either direct or behavioral analysis methods and/or to impacts with behavioral analysis.14 These tools analyze inform the approaches for indirect impacts. It then how people are likely to be affected by reform, how reviews the two broad economic approaches to analyz- this impact will differ among groups (based on gender ing direct impacts: direct impact analysis and behavioral or ethnicity, for example), what coping mechanisms analysis. Finally, the section reviews complementary people have to deal with changes effected by reform, economic approaches to analyzing indirect impacts: and who is most likely to be vulnerable to a particular The first covers macroeconomic frameworks that aim at reform. In addition to the analysis of direct impacts, modeling the different impacts of policy interventions social analysis typically also includes an evaluation of on a variety of sectors or markets, but that leave open how different people are likely to respond to a reform the distributional implications of policy changes. These (behavioral response), and some of the institutional frameworks are either partial equilibrium analysis or constraints the reform may face during implementa- general equilibrium techniques. Then, the second group tion. In addition to demand and supply analyses, comprises tools that use as inputs the results of any of which are multidisciplinary tools typically carried out the macroeconomic frameworks, and assess the distrib- using a combination of qualitative and quantitative utional implications of policy changes: Tools linking techniques (presented below under “behavioral analy- microeconomic distribution or behavior to macroeco- sis”), three broad classes of methods fall within the nomic frameworks or models. Under each class of meth- repertoire of social analysis for policy reform: social ods, the discussion presents an overview of specific impact assessment, participatory poverty assessments, tools (referred to in bold text) that are discussed in and the social capital assessment tool. The choice greater detail in the annex (including their data require- among methods depends on the particular policy and ments and particular advantages and shortcomings). the time available for research. Social impact assessment (SIA) is used to assess how Social analysis the costs and benefits of reforms are distributed The first approach consists of several techniques of among different stakeholders and over time. It is par- social analysis that combine understanding of direct ticularly useful in understanding how the assets (phys- 20 Elements of Good Poverty and Social Impact Analysis ical, financial), capabilities (human, organizational), 2001). They are more relevant to broad-based economic and social relations (e.g. gender, exclusion) fiscal/expenditure and sectoral reforms with potential of stakeholders, and the institutional mechanisms impacts on livelihoods and vulnerability (Dulamdary through which policy actions are transmitted, affect and others 2001). BAs tend to use similar qualitative policy outcomes. Stakeholder analysis is a prerequisite data-gathering techniques, but they focus specifically for SIA. When reasonable national survey data exist, on consultation with those groups directly affected by a SIA uses a range of qualitative data collection tools specific intervention, project, or policy, and therefore (focus groups, semi-structured key informant inter- have not typically looked for national representativity. views, ethnographic field research, stakeholder work- They do not focus specifically on the poor. shops) to determine impacts, stakeholder preferences The social capital assessment tool (SOCAT) measures and priorities, and constraints on implementation. In social capital (institutions and networks, and their the absence of adequate quantitative data, SIA supple- underlying norms and values) at the level of house- ments qualitative, sociological impact analysis with holds, communities, and key organizations. It allows purposive surveys that capture direct impacts and analysts to identify how these social assets affect pro- behavioral responses to reform, or specific dimensions ductive behavior (for example, income generation and (such as time-use patterns) that affect reform out- risk management), and how this in turn responds to comes (the “low-low” cell in table 2). SIA can be used policy reform. For instance, well-functioning networks to examine the impacts of structural reforms such as with high levels of trust, such as among parent-teacher privatization of state-owned enterprises, agricultural associations or farmer associations, may facilitate policy reform, reform of basic services, utility reform, civil changes that call for collective action or cooperation. service reform, and fiscal policy. It is particularly rele- Alternatively, SOCAT data make it possible to assess vant for understanding the quality of impact on dif- whether certain policies strengthen or undermine social ferent groups, and examining how the poor cope with assets. The tool can be tailored to specific policies or reforms and access market opportunities. Given the used to give depth to other methods of data collection overlap of research methods, SIA is more cost-effective and analysis. A tailored version of the SOCAT survey when undertaken simultaneously with institutional was administered in Bosnia and Herzegovina, where analysis and social risk assessment. measurement of the level of social capital led to recom- Participatory poverty assessments (PPA) and beneficiary mendations for reform of the social welfare system, and assessments (BA) both rely on direct consultation of spe- improvements in service provision and the integration cific groups and field observation, using primarily qual- of returning refugees (World Bank 2002b). itative techniques (focus groups, key informant interviews, and a range of other tools classified under Direct impact analysis the broad label of participatory rural appraisal). Like Direct impact analysis is a simple assessment of who is poverty maps, PPAs have often been used before the directly affected by a policy change, and how much analysis of a specific policy reform to identify those they are affected. It assumes no behavioral response policies and issues of most relevance to the poor, and to from affected households or groups; that is, if prices understand the non-income dimensions of poverty and change, quantities do not adjust. Effectively all elastic- the processes through which reform actions filter down ities are assumed to be zero, including own-price elas- to the poor. PPAs tend to focus on information and ticities. This assumption is appropriate for assessing analysis at the national level by selecting a sample of short-term impacts, before economic agents have time regions for intensive research on poor people’s views, in to make adjustments. It otherwise represents a limita- order to understand poverty impacts through a series of tion of the approach. In particular it will tend to over- rapid assessment tools and structured task-based ana- state the impact on household welfare. The approach lytical exercises. They can be adapted for use in moni- can be used to analyze any type of policy change—for toring or seeking feedback on a particular policy and in example, a change in prices (such as a commodity designing pro-poor public policies (Norton and others price, tariff, wage, or exchange rate) or a change in 21 A User’s Guide to Poverty and Social Impact Analysis public finance policy (such as an expenditure program Applications also include planning of public invest- subsidy, tax, civil service or state-owned enterprise ments in education, health, and transport, and target- retrenchment). But it is best suited to reforms whose ing of direct social assistance and food aid to impacts are mainly short-term. Examples include the vulnerable populations. The method is most useful removal of a subsidy, a small-scale privatization, or a when constructed at a fine level of disaggregation, but single price change in a relatively isolated market. this requires very large data sets. Below are three examples of tools that fall within Tools to assess public service delivery allow analysts to this approach: incidence analysis, poverty mapping, measure the efficiency of public spending and the and tools to assess public service delivery. These range delivery performance through assessing leakages and in terms of data/time/capacity requirements from low their sources, captures of financial flows, and incen- to high, as shown in table 2, with poverty mapping by tives and accountability mechanisms at all stages of the far the most demanding. expenditure chain. This complements incidence analy- Incidence analysis estimates the distributional inci- sis, which relies on analyzing the cost of the services dence of a component of income or expenditure at the provided, irrespective of the service that actually household level. The analysis is an appropriate starting reaches the beneficiaries. Applications of these tools point where quantitative data are available (the “low- include the analysis of the efficiency and quality of medium” cell in table 2). A useful first step is to exam- health and education service delivery in Tanzania and ine key descriptive statistics for the country to see Uganda (Government of Tanzania, 1999 and 2001 and which households are “exposed” to the policy change. Reinikka, 2001). These tools, including Public Expendi- The most common application is in relation to tax and ture Tracking Surveys (PETS) and Quantitative Service expenditure reform; the technique has been used, for Delivery Surveys (QSDS), are described in more detail instance, to estimate the incidence of education in the section on monitoring and evaluation and in expenditure in Malawi. It can also be used for reforms box 13, and are presented in the Annex, under the that affect prices and consequently household “monitoring and evaluation” section. incomes, such as utility or agricultural reform. Appli- cations of this type include access to utility services in Behavioral analysis Guatemala (Foster and Araujo, 2001). There are two Behavioral analysis includes economic tools that go main types of incidence analysis relevant to the direct beyond direct impact analysis to recognize some impact analysis: simple incidence analysis and marginal behavioral responses among households and eco- incidence analysis. The first measures the incidence of nomic agents. Behavioral analysis includes methods average expenditure or tax, that is, it considers all that permit non-zero own-price and cross-price elas- expenditure or taxes. The second focuses on the distri- ticities. In other words, with a price or other policy butional incidence of the last or next unit of expendi- change, households may switch to consuming or pro- ture or tax (see box 6).15 ducing other goods and services and move along their Poverty maps are geographical profiles that show the respective demand or supply curves. The approach is, spatial distribution of poverty within a country, and however, limited to a purely “micro” focus. Namely, suggest where policies might have the greatest impact supply is not equated to demand in a market, markets on poverty reduction. Poverty maps can be used to do not clear, and prices are therefore not endogenous. illustrate outputs of most analytical tools. For Rather, households simply react to an exogenous pol- instance, a poverty map can be combined with maps icy shock based on behavioral specifications and that show the placement of primary health care facili- assumptions. If data, time, and capacity permit, ties to understand the access to health services by the behavioral analysis should always supplement simpler poor. The technique is particularly suited to reforms incidence analysis to more fully illuminate household with regionally differentiated impacts such as decen- responses to policy change. Some of the tools of tralization and agricultural reform, as in the case of behavioral analysis are behavioral incidence analysis, rice price changes in Madagascar (Mistiaen, 2002). demand/supply analysis, and household models. 22 Elements of Good Poverty and Social Impact Analysis Box 6. Impact of Public Expenditures in Indonesia: Average versus Marginal Benefit Incidence Average and marginal benefit incidence has been examined by A similar exercise was carried out for junior and senior sec- Lanjouw and others (2001) to assess how education and health ondary education and indicated that benefits of public spend- expenditures affect different income groups in Indonesia. Sta- ing for higher education levels become increasingly regressive. tic benefit incidence analysis entailed dividing groups into In health, per capita transfers on primary health care were expenditure quintiles and computing rates of utilization of the found to be rather evenly distributed across quintiles, while facilities for each group. For primary education, total govern- government spending on hospitals was highly regressive. ment outlays in 1998 amounted to nearly 8,000 billion rupiah (covering both routine and development expenditures). In that The authors also considered the marginal benefit incidence of year there were just over 25 million students enrolled in public public expenditures. In other words, they asked how a change in primary schools. Assuming uniform transfers, the government government spending would be felt across expenditure groups. thus transferred some 307,000 rupiah per public primary stu- First the incidence of changes in education and health provi- dent per year. sioning across two periods of approximately a decade each was analyzed. Second, the quintile-specific “marginal odds ratio” of The table below gives the incidence of government primary participation—defined as the incremental increase in the quin- education spending for each expenditure quintile. As can be tile-specific participation rate associated with an aggregate seen from the table, government expenditure has a pro-poor change in the program participation rate—was estimated on distribution, with an average per capita transfer of around the basis of survey data. This was compared with the “average 47,900 rupiah for the lowest quintile and 25,300 for the high- odds ratio”—the quintile-specific participation rate in a given est quintile. With practically universal enrollment, the pro- year relative to the participation rate for the population. On the poor bias is largely driven by the fact that poorer households basis of the historical analysis as well as the estimation results, tend to have more young children than other households (6.2 the evidence suggests that changes in public spending on pri- million primary school students in the lowest quintile, versus mary education would be even more strongly felt in the bottom 3.3 in the highest quintile). two quintiles than what static analysis would suggest. Expenditure Quintile 1 2 3 4 5 Total Population age 7–12 (millions) 6.8 6.2 5.4 4.8 3.8 27.0 Public school students (millions) 6.2 5.9 5.2 4.5 3.3 25.2 Average per capita transfer (rupiah) 47,898 45,324 40,004 34,375 25,270 38,574 Percent of total 24.0 23.5 20.7 17.8 13.1 100.0 Behavioral incidence analysis combines incidence marginal incidence analysis, the ex-ante behavioral marginal analysis, presented above, with econometric estimates evaluation of policy reforms, and the ex-post evaluation of of household behavior. It can be used to explain distri- assigned programs. butional changes arising from a policy change, and Demand and supply analyses estimate the responses of thereby addresses one of the shortcomings of inci- consumers and producers, respectively, to price dence analysis. Applications have included analysis of changes. Demand analysis can assess the willingness of the role of government policy (in relation to the pri- consumers at different income levels to pay for public vate sector) in expanding access to education in services like water and electricity. It has been used to Malaysia (Hammer, Nabi, and Cercone 1995); exami- assess the impact of higher electricity tariff rates in nation of the disincentive effects of food stamps on Armenia (box 7), and is being applied to the same labor supply in Sri Lanka (Sahn and Alderman 1995); issue in the Kyrgyz Republic. It has also been used to and study of the crowding out of private transfers by evaluate preferences and likely responses of water con- public funds in the Philippines (Cox and Jimenez sumers to tariffs and institutional reform such as pri- 1995) and South Africa (Jensen 1998). The Annex vatization in several African countries (Mozambique, present details on techniques for the ex-post behavioral Lesotho, Angola, and Zambia). Supply analysis is most 23 A User’s Guide to Poverty and Social Impact Analysis suited to analyzing agricultural reforms that affect the their equilibrium level.16 Thus prices are now endoge- poor in their role as producers, and has been used to nous. Partial equilibrium analysis is distinguished examine the impact of agricultural liberalization on from general equilibrium analysis (discussed below) in poor farmers in Mexico (box 8). Supply and demand that it does not include all production and consump- analyses are typically carried out using a combination tion accounts in an economy, and does not attempt to of qualitative and quantitative techniques. capture all markets and prices in an economy. Partial Household models are somewhat more complex, in equilibrium approaches (which include elasticities on that they analyze impacts by taking account of house- both the demand and supply sides) will allow for indi- holds as both consumers and producers. The models rect impacts that occur when changes in one market integrate producer, consumer and worker decisions affect other markets, but they will only capture these into a household problem, to reflect the fact that many changes to the extent that they include the relevant households, especially in rural areas, are simultane- markets.17 This is their biggest drawback relative to ously units of production and consumption. They are general equilibrium approaches. For this reason, par- particularly suited to addressing agricultural reforms, tial equilibrium analysis is better suited to analyzing but have been used in relation to large sets of reforms, sectoral reforms (such as agricultural marketing and including taxation. pricing and utility pricing reforms) that are less likely to have large impacts on macro aggregates. Partial Partial equilibrium analysis equilibrium techniques fall within the “high-medium” Partial equilibrium analysis goes a step beyond behav- category of table 2 in that they at least require house- ioral impact analysis in that it equates supply and hold survey data. Tools for partial equilibrium analysis demand in one or more markets so that prices clear at are multimarket models and reduced form techniques. Box 7. Impact of Utility Pricing on the Poor in Armenia: Demand Analysis A recent study (Lampietti and others 2001) uses multivariate policy tradeoff involved in raising tariffs, which can help cover welfare analysis to assess the poverty impact of raising tariffs costs but also threatens to reduce household consumption. in the electricity and water sectors in Armenia. It looks ex post at the impact of higher electricity prices (and an accompany- In both cases results from survey data are corroborated against ing expansion in social safety net provision) and ex ante at the predictions from multivariate models of household expen- increased water tariffs. The study estimates a demand function diture per head. The models include as explanatory variables to examine consumers’ responses to changes in prices, includ- demography, asset holding, and regional location; each is esti- ing through substitution of other forms of fuel for electricity. mated separately for rural and urban households. Possible supply side adjustments (to the cost and structure of production) are not taken into account. The electricity study finds that households cut their consump- tion and switched to wood and natural gas alternatives as a The analysis draws on two specially commissioned surveys, result of the rate increase. This effect was particularly marked undertaken over the course of the electricity reform: a quanti- for poor households. As a result, the reform has produced only tative household survey of water and electricity consumption a modest improvement in revenue. One policy implication is patterns (as well as of standard information on income and that future tariff rises are more closely aligned with likely con- demographics), and a qualitative consumer satisfaction survey sumer responses. Another is the need for action to mitigate based on focus group research concerned with attitudes poverty and environmental impacts. toward provision. For electricity, the data are matched with administrative statistics on payment and consumption. The results of the water analysis suggest that consumers are reluctant to pay significantly more for a service they deem The electricity study examines changes in consumption and unreliable. The authors suggest that reform should, therefore, payment behavior (pattern of arrears, and so forth) of poor and proceed in two stages—first enforcing payment from house- non-poor households following reform. The water analysis holds with reliable service, and then raising tariffs incremen- considers (a) how much extra non-poor and poor households tally to balance cost recovery with the need to maintain access would be willing to pay for an improved service, and (b) the of poor users. 24 Elements of Good Poverty and Social Impact Analysis Box 8. Impact of Liberalization in Mexico: Supply-Side Analysis Simple supply-side estimation can be used to examine the dif- had less access to and more problems with credit, and were less ferential impacts of policy change on welfare. López, Nash, and likely to use purchased inputs, such as seeds, fertilizer, pesti- Stanton (1995) use a household survey from Mexico to estimate cides, or to use a tractor for soil preparation. Their land was of the relationship between household assets and agricultural sup- lower quality on average, and their educational level lower, than ply response. At the time, the Mexican economy was becoming those with greater productive assets. This analysis predicted increasingly open – markets for inputs, outputs, and credit were that poor farmers would benefit less from liberalization. being liberalized. The study had two related goals. The first was to monitor the condition of Mexican farmers, especially the However, the results of the smaller “panel” study suggest that poor, and see how they had been affected by changes in policy conditions had improved both on average, and for the poorer and environment. The second goal is to understand the con- farm households in the sample. Cropping patterns are more straints facing the ability of poor households to adjust to the diverse, landholdings have increased, as has the use of pur- new regime and take advantage of new opportunities. chased inputs, and asset ownership has also improved mod- estly. They also find that among the poor, educational López and others (1995) construct a model showing that own attainment and off-farm income have declined. Although wealth affects both the level of output and the ability to respond López et al. (1995) do not speculate, this may be due to the to price changes. They test this model against a large “baseline” greater returns to on-farm labor brought about by liberaliza- dataset from 1991 and a smaller selected survey from 1993. tion, which reduce (relatively) the returns to off-farm income Using the baseline data, they find that farmers with fewer pro- and the educational investments necessary to enter the off- ductive capital assets (the “poor”) grew fewer crops on average, farm labor market. Source: López, Nash, and Stanton (1995). Multimarket models permit the combined estimation the economy, in varying degrees of aggregation. In of systems of supply and demand relationships, so that theory, a well-specified general equilibrium model can the analyst can see how policies in one sector impact capture indirect impacts of policy generated from all on other related sectors. Multimarket models repre- other markets. However, in practice, as with any eco- sent a simpler alternative to computable general equi- nomic estimation, it captures indirect impacts only librium models, and have been used, for example, to from those markets that are included in the model, examine the welfare impact of technical change in and results depend on the model specification and agriculture, increased exports, and input subsidies in parameters.18 While general equilibrium analysis can India (Binswanger and Quizon 1984, 1986) and agri- be used to analyze most types of policy reform, it is cultural subsidies and tariffs in Turkey (see box 9). most relevant to reforms with multiple and significant Reduced-form estimation can be used to simulate the indirect impacts on the economy through a number of impact of different policy variables on poverty and transmission channels. An exchange rate devaluation social outcomes. The approach is less data-intensive or alternative aggregate fiscal policies would be best than multimarket modeling. For instance, reduced assessed with a general equilibrium approach, data form techniques were used to study rural poverty in and capacity permitting. General equilibrium analysis, Zambia, taking advantage of household budget data, in capturing accounts from the entire economy, time-use information, and other sociological and requires not only household survey data but also com- anthropological data. prehensive and consistent national aggregate data. The computational and capacity requirements are also General equilibrium analysis generally high. Other drawbacks are that the technique General equilibrium analysis goes beyond partial equi- can be difficult to explain to policymakers, and results librium analysis in that it models all economic are sensitive to the assumptions on which a particular accounts in the economy and thus aims to present a model is based. The approach hence is presented in the comprehensive picture. What the methods in this cat- “high-high” cell in table 2. Specific tools for general egory have in common is a complete specification of equilibrium analysis are social accounting matrices 25 A User’s Guide to Poverty and Social Impact Analysis Box 9. Impact of Agricultural Subsidies and Tariffs in Turkey: Multimarket Modeling Hammer and Tan (1989) constructed a multimarket model of The model simulates the impact of changes in government pol- the agricultural sector in Turkey. Their model contains eight icy concerning direct intervention (subsidies and support separate agricultural markets, all of which are potential substi- prices) and tariffs. The results indicate that reducing export tutes for each other. Some of these are traded internationally. taxes leads to a broad-based increase in supply and exports, and Incomes in the rural areas are derived from agricultural prof- that the incidence of subsidies to fertilizer and feed grains is suf- its. The model also includes an explicit government account, ficiently skewed that they could be cut without damage to farm which taxes, provides subsidies, and intervenes directly in the incomes or export earnings. Also, import duties on milk prod- markets for selected outputs. Elasticities for supply and ucts are regressive. Imposing border prices (removing import demand were taken from published sources, and modified to tariffs and restrictions) leads to improved government finance satisfy theoretical restrictions and to conform to base data. and foreign exchange earnings. It also improves the incomes of Sensitivity analysis confirmed that the model was robust to middle and wealthy farm households, but at the risk of harming large changes in these and other assumptions. consumers—especially the poor—through higher prices. and input-output models and computable general and poverty outcomes are arrived at iteratively and equilibrium models. outside the macro-modeling exercise. In its simplest Social accounting matrices (SAM) and input-output form, the macroeconomic framework/model (such as (IO) models can be used for simple policy simulations any of those reviewed above) is solved to derive the (by selecting some accounts as exogenous, and leaving main equilibrium parameters (such as prices, wages, the others endogenous). For instance, in a SAM con- fiscal deficit, and so forth); these parameters are then taining agricultural production and transportation fed into a micro-model. There are several micro accounts, the impact of an exogenous change to agri- approaches that can be used to derive poverty and dis- culture can be simulated (leaving transport fixed) or tributional outcomes based on the parameters derived the other way round.19 SAMs have some serious limi- from the macroeconomic framework/model.21 The tations, including the facts that prices do not adjust to approaches presented below can be applied to a wide reflect changes in real activity, and results are highly variety of reforms. However, they are data- and skill- sensitive to which accounts are assumed to be endoge- intensive, and are located in the “high-high” cell in nous and which exogenous. table 2.22 The following specific techniques are Computable general equilibrium (CGE) models are com- described in more detail in the annex. pletely specified models of an economy (or a region). Linking macro-framework to a reduced form estimation is They vary in their complexity from the basic 1-2-3 a minimalist approach that simulates poverty impacts model (one country, two activities, three goods) to on the basis of various macroeconomic variables.23 versions with several activities and actors and hun- Tools have also been developed to examine how dreds of parameters. CGEs can be used in a number of changes in certain macro-variables—most particularly policy contexts, including public finance reform and growth rates—affect poverty, based on a country- macroeconomic stabilization.20 Box 10 illustrates the specific distribution. SimSIP and PovStat are tools of use of a CGE model to calculate the impact of fiscal this type.24 incidence in the Philippines. As well as being data- Linking macro-framework to behavioral analysis esti- intensive, CGEs—even simple ones—can be difficult mated for representative households has been done in the to build and understand. 1-2-3 PRSP model, which links the 1-2-3 model to a behavioral analysis of representative households Tools linking microeconomic distribution or behavior to (Devarajan and others 2001), and PAMS, which joins a macroeconomic frameworks or models labor-poverty module to a macro-consistency model The last class of tools and methods links microeco- (such as the Bank’s RMSM-X).25 The technique can be nomic behavior and/or distribution with a consistent used to simulate a wide range of policies, from labor macroeconomic framework or model. Distributional and wage policies to taxation, prices, and the alloca- 26 Elements of Good Poverty and Social Impact Analysis Box 10. Net Fiscal Incidence in the Philippines Ideally, one should be able to analyze the incidence of tax and of tax with the CGE model. The incidence reflects both actual expenditure policies simultaneously, that is, conduct a net fis- tax collections and the increased costs associated with each tax. cal incidence analysis. In practice, this type of analysis is diffi- The effective rates for indirect and direct taxes were aggregated cult to undertake because the data requirements are extensive. to obtain overall tax burden. Devarajan and Hossain (1998) completed one of the few exam- ples of this type of analysis in the Philippines. The net inci- For expenditures, the authors focused on health, education, dence of fiscal policy (indirect taxes, direct taxes, and and infrastructure spending. Nationwide incidence patterns expenditures) was estimated using a variety of data sources were derived from regional patterns of expenditures along and tools. with information on income distribution. To derive benefit incidence, the authors inferred the implicit subsidy on health, For both direct and indirect taxes, the authors calculate the education, and infrastructure for each income decile. Overall effective tax rate for each income decile defined as the change incidence of public expenditures in health, education, and in purchasing power of each income class. For direct taxes, they infrastructure was calculated as the weighted average of the calculated the effective tax rate using actual tax collection rates regional incidence, with the weights being the regional alloca- broken down by gross income. The family income and expen- tions of these expenditures. Total incidence of public expendi- diture survey was used to map income classes into deciles. For tures was calculated as benefits as a share of gross income. indirect taxes, a multisector CGE model was used to calculate the incidence of taxes. The effective tax rate for each type of tax The results indicate that tax incidence is fairly neutral. Expen- (such as VAT, import tariffs, excise taxes) was calculated indi- diture incidence is strongly progressive, as is the combined vidually. This was done by simulating the removal of each type incidence. tion and levels of government spending. Applications ter understanding of adverse impacts can also inform include the linking of a simple CGE model with a the design of appropriate compensation mechanisms, demand system for food to examine the impact of if needed. This component of the PSIA is informed by macroeconomic policy changes on food consumption the analysis and tools laid out in the previous section. and nutritional status in the Philippines (Orbeta and This analytical work can provide potential options to Alba 1998).26 limit the negative impacts on the welfare of the poor or A third technique is linking macro-framework to micro- other groups. In addition, finding the appropriate solu- simulation. A more disaggregated variant of the repre- tion, or set of solutions, also often necessitates substan- sentative household method above is to simulate tial discussion and debate by key stakeholders, in behavior at the level of the individual household. particular consultation with those affected to test Robillard, Bourguignon, and Robinson (2001) use this whether the proposed compensation measures can fea- approach to analyze the poverty impact of the Indone- sibly be implemented. In short, if the ex-ante poverty sian financial crisis (see box 11). Their household and social impact analysis shows that a proposed model is linked to a CGE through wages, and the sec- reform will have short-term adverse impacts on the liv- toral allocation of employment and prices. It is con- ing standards of the poor or other groups, it is critical strained to be consistent with the output of the CGE. that the analyst address the following considerations. Element 7: Contemplating enhancement Consider alternative design and compensation measures The design of reform may be improved by including enhancement or mitigation measures, or by different Poverty and social impact analysis is undertaken to sequencing of public actions. First, one may opt to maximize welfare gains, in particular for the poor, by proceed with the implementation of a reform as influencing the design of a policy reform. To the extent planned, but with a subsidization arrangement to pro- that there are losers from the reform, PSIA can inform tect the poor or others adversely affected by the policy. policy design leading to choices that minimize the For example, a water tariff increase associated with number of losers or the extent of adverse impacts. Bet- utility reform may be designed to protect those who 27 A User’s Guide to Poverty and Social Impact Analysis Box 11. Impact of the Indonesian Financial Crisis on the Poor: Partial Equilibrium Modeling and CGE Modeling with Micro-simulation General equilibrium models permit the analyst to examine items. Second, the results are exclusively nominal, in that the explicitly the indirect and second-round consequences of pol- welfare changes are due entirely to changes in the price of con- icy changes. These indirect consequences are often larger than sumption, and do not account for any concomitant change in the direct, immediate impact, and may have different distribu- income. Third, this analysis cannot control for other exoge- tional implications. General equilibrium models and partial nous events, such as the El Niño drought and resulting wide- equilibrium models may thus lead to significantly different spread forest fires. conclusions. Robillard, Bourguignon, and Robinson use a CGE model, con- A comparison of conclusions reached by two sets of nected to a micro-simulation model. The results are obtained researchers, examining the same event using different methods, in two steps. First, the CGE is run to derive a set of parameters reveals the differences between the models. Levinsohn, Berry, for prices, wages, and labor demand. These results are fed into and Friedman (1999) and Robillard, Bourguignon, and Robin- a micro-simulation model to estimate the effects on each of son (2001) both look at the impact of the Indonesian financial 10,000 households in the 1996 SUSENAS survey. In the micro- crisis on the poor—the former using partial equilibrium meth- simulation model, workers are divided into groups according ods, the latter using a CGE model with micro-simulation. to sex, residence, and skill. Individuals earn factor income from wage labor and enterprise profits, and households accrue The Levinsohn study used consumption data for nearly 60,000 profits and income to factors in proportion to their endow- households from the 1993 SUSENAS survey, together with ments. Labor supply is endogenous. The micro-simulation detailed information on price changes over the 1997–98 crisis model is constrained to conform to the aggregate levels pro- period, to compute household-specific cost-of-living changes. vided by the CGE model. It finds that the poorest urban households were hit hardest by the shock, experiencing a 10–30 percent increase in the cost of The Robillard team finds that poverty did increase during the living (depending on the method used to calculate the change). crisis, although not as severely as the previous results suggest. Rural households and wealthy urban households actually saw Also, the increase in poverty was due in equal parts to the cri- the cost of living fall. sis and to the drought. Comparing their micro-simulation results to those produced by the CGE alone, the authors find These results suggest that the poor are just as integrated into that the representative household model is likely to underesti- the economy as other classes, but have fewer opportunities to mate the impact of shocks on poverty. On the other hand, smooth consumption during a crisis. However, the methods ignoring both substitution and income effects, as Levinsohn, used have at least three serious drawbacks. First, the consump- Berry, and Friedman do, is likely to lead to overestimating the tion parameters are fixed, that is, no substitution is permitted increase in poverty, since it does not permit the household to between more expensive and less expensive consumption reallocate resources in response to the shock. consume relatively small quantities of water by incor- as barriers to entry in the domestic transport sector) porating a subsidy mechanism.27 Often contextual reduce the benefits of trade liberalization accruing to information and consultations are required to select intended beneficiaries, taking measures to address the most appropriate type of mechanism to best fit those constraints will be critical to achieving expected specific country circumstance and implementation welfare gains. Similarly, it will be essential to under- capacity. Alternatively, analysis of an electrical utility stand and address the factors that constrain the poor reform may determine access to be the main con- or other target groups from benefiting from market straint for the poor, resulting in the design of subsi- reforms—for example, lack of assets (land, credit, dized grid connection fees for targeted poor electricity grid connection) or of capabilities (price communities.28 In fiscal reform, key staple goods that information, market access). Micro-econometric make up the bulk of consumption for the poor may be analysis as well as qualitative analysis can assist in exempted from taxation.29 identifying the type of complementary measures that Second, the policy set may need to be expanded might be necessary. beyond the core policy measures (driven by the prob- Third, it is important to carefully consider sequenc- lem diagnosis) to include complementary measures. ing. For example, shutting down a commodity board For example, if “behind the border” bottlenecks (such can eliminate monopsony and subsidized inputs at the 28 Elements of Good Poverty and Social Impact Analysis same time. If critical inputs are likely to be unavailable calculate the cost of compensation, and consider it rel- or prohibitively expensive for vulnerable farmers in ative to the expected benefits of reform. In terms of certain locations, PSIA might suggest that the govern- costs, the compensation scheme itself (for example, a ment first take action to drop barriers to entry or large retrenchment or social program) will have fiscal encourage private merchants to pursue untapped costs that, depending on magnitude, can have indirect markets before it dismantles the commodity board. impacts on fiscal stability, prices, and the economy. Also, sustainability of the reform process can be Moreover, there is an opportunity cost, as any com- enhanced with quick wins among key stakeholders to pensation scheme will use resources that would other- build support for reform. For example, new resources wise have been spent elsewhere.30 for mining safety in Russia were used to persuade the unions of the need for reform. Consider delay or suspension If the findings of PSIA suggest that the short-to-long- Consider direct compensatory mechanisms term benefit of the best-designed policy intervention When adverse impacts of reform are unavoidable, con- does not exceed the short-term (or long-term) costs of siderations driving the decision to compensate losers mitigating or compensating the poor, or that other may be based on: (a) poverty grounds (especially if important groups might suffer irreversible losses, then some of the poor lose in the short run and the objec- consideration could be given to delaying the reform tive of the policy is poverty reduction); (b) equity (that is, resequencing) or abandoning or suspending grounds (especially if groups that have traditionally implementation of the policy. been the poorest and most vulnerable lose ground to those with greater economic security); or (c) political Element 8: Assessing risks economy grounds (especially if the losers have the capacity to organize and threaten either the sustain- Upon laying out the broad parameters of possible ability of reform or survival of the government). reform alternatives, it is important to consider the Careful consideration is required in the design of risk that some of the assumptions underlying the compensatory schemes—to ensure appropriate target- analysis may not be realized.31 This process may pro- ing of intended beneficiaries and cost effectiveness, vide further insight into policy choice and design, and to avoid perverse or distortionary incentive including sequencing. Risk analysis addresses the schemes that might compromise implementation of issue of what could go wrong to prevent a policy the intended policy (see box 12). It is also important to reform from delivering the intended poverty or social Box 12. Labor Downsizing and the Design of Compensation Packages in Vietnam The issue of labor downsizing and the design of compensation tive severance packages, based on the characteristics of indi- packages have been analyzed ex ante in the context of Vietnam vidual workers. by Martin Rama (2001). Proposed reforms included a major downsizing operation involving the liquidation, divestiture, The acceptance rate is defined as the fraction of the workers for or restructuring of approximately 6,000 state-owned enter- whom the separation package would exceed the present value prises, resulting in unemployment of roughly 5 percent of the of the estimated loss from job separation. Rama found that a Vietnamese labor force or 450,000 workers. In anticipation of formula based solely on earnings history had a consistently the massive layoffs a special compensation package was devel- higher acceptance rate for men, while women found a uniform oped which amounted to two months of salary per year of ser- lump-sum compensation more attractive. Based on these sim- vice plus a substantial cash training allowance. This package ulations, the government of Vietnam picked a separation pack- was a result of policy debates around simulations generated age that involved a sizeable lump-sum component in the form by Rama using DOSE (Downsizing Options Simulation Exer- of the training allowance in order to ensure that female work- cise). The simulation computed “acceptance rates” for alterna- ers would not be unduly penalized by the layoffs. 29 A User’s Guide to Poverty and Social Impact Analysis impacts. By addressing these questions explicitly, well as risks arising out of behavioral responses to the adjustments can be made to mitigate the risks (for proposed reforms. Obvious examples are political example, modifying the reform or introducing com- economy risks that may be latent but may become plementary measures). more acute when interest groups perceive reforms as Risk analysis can therefore help governments to a threat. Another example might be increased expo- anticipate—and avoid—major unintended conse- sure to exogenous market conditions in the absence of quences. The analysis should consider risks to the risk coping or insurance mechanisms to deal with reform program as well as risks emanating from its external competition or market failures. Risk assess- impacts. Part of the challenge is to identify explicitly in ment is based on the premise that risks become real- the analysis the assumptions that must be valid for a ity when assumptions turn out to be wrong. The policy to have its intended impact. This is a difficult likelihood of an assumption being invalid is, there- task and underscores the need to make operating fore, another way of judging the extent of risk. The assumptions explicit in monitoring the evolution of first step is to identify the assumptions—implicit and the policy reform and its evolving impacts (see the sec- explicit—about what should and should not happen tion on monitoring and evaluation, below). in order for a policy to achieve its goals. The next step There are four main types of risk in PSIA: is to make a judgment as to the likelihood that each assumption will hold, and its importance to policy. ■ Institutional risks. These include risks that assump- The more likely it is that an important assumption tions made regarding institutional performance will be invalid, the greater will be the need to alter the were incorrect. This could be due, for example, to policy. If assumptions are considered important but market or institutional failures in existence where more likely to be valid, there may be a need for a con- none was assumed (such as asymmetric informa- tingency plan. A variety of tools are available for risk tion or missing markets), or to the fact that key assessment. In particular, social risk assessment com- organizations involved perform in unexpected ways. pares data and indicators from the World Develop- ■ Political economy risks. This includes the risk that ment Indicators with external agencies to estimate the powerful interest groups may undermine reform likelihood and importance of risks to the reform pro- objectives by blocking implementation, capturing gram (see annex). benefits, or reversing reform actions. Sensitivity analysis is usually applied in the context ■ Exogenous risks. These include risk of shocks to the of quantitative economic models, and entails varying external environment such as a natural disaster or the magnitude of certain key parameters to judge their regional economic crisis that might have a bearing sensitivity to the model’s outcomes. Sensitivity analy- on the vulnerability of the poor. sis is especially important for parameters that are par- ■ Other country risks. These include the threat of an ticularly uncertain (as may be the case where these are increase in political instability or social tensions based on estimates from other countries) or where that could undermine effective implementation. risks are known (for example, droughts in the Sahel). One practical limitation of the approach is that it is There are three main methods available to conduct more often used to test sensitivity within a given a risk analysis: risk assessment, sensitivity analysis, and model than to assess alternate scenarios using different scenario analysis. The first and third are discussed in models, which is not always feasible.32 more detail in the annex. Scenario analysis is a tool for helping decisionmak- Risk assessment is an approach for systematically ers consider how policy impacts might vary in differ- identifying risks, and their importance to the reform ent plausible scenarios. Scenarios are based on a range at hand. It looks beyond vulnerability risks, which are of social, economic, political, or technological out- captured by the impact analysis, to include considera- comes that drive change in the country. In this way, tion of risks arising out of the sociopolitical and insti- unexpected risks can be highlighted, and contingency tutional context that could undermine the reform, as plans made.33 30 Elements of Good Poverty and Social Impact Analysis Element 9: Monitoring and evaluating line data), during, and after the reform. The evaluation impacts problem is particularly challenging in the case of econ- omy-wide policy reforms. As these reforms often apply When identifying and designing reform based on ex- to whole sectors or economies (unlike projects, which ante PSIA, it is important to consider setting up at an are restricted to a group or specific region), it is diffi- early stage systems for monitoring, social accountabil- cult to establish the counterfactual. Use of control ity, and ex-post evaluation of the impacts. In doing so, groups is possible only when the policy has been ini- some specific concerns should be borne in mind in the tially designed as a pilot or phased in so that those who context of reform-specific PSIA. This section outlines do not initially experience the reforms can serve as these issues.34 controls. The particular challenges of ex-post evalua- As noted above, good PSIA calls for monitoring tion for certain kinds of economy-wide reform require and evaluation (M&E), both to validate ex-ante particular foresight in setting up an evaluation frame- analyses and to influence the reformulation of policy. work ex ante. Given the challenges of ex-post evalua- Effective PSIA therefore implies a heavy demand on tion and the need for more rapid feedback on the data and information bases. In considering the infor- evolution and impact of policy, PSIA implies a special mation needs of PSIA, it is essential to build where role for monitoring for purely practical purposes. possible on existing systems of M&E. This should be Although monitoring cannot attribute causality, it can done with a view to developing a coherent national say something about whether, for whatever reasons, poverty monitoring system that brings together infor- assumptions are holding and expected impacts are mation bases, indicators, mechanisms for linking materializing. Monitoring can identify where “things M&E and policy decisionmaking, and so forth. This is are going well or going wrong,” as well as where sup- another area where capacity building is an embedded plementary interventions or changes in policy may be part of PSIA: the development or refinement of sys- needed to ensure that the desired impacts materialize. tems for monitoring, social accountability, and evalu- For example, reforms that affect service delivery may ation is most effective where it strengthens in-country benefit from participatory monitoring and evaluation capacity.35 that provides feedback from intended beneficiaries on Monitoring involves tracking the progress of quality of service delivery. Some methods for partici- processes and implementation (as measured by indi- patory M&E are described below. cators on inputs, outputs, and outcomes) associated with an intervention. This is done to ensure that Choosing indicators for PSIA agreed targets are met and the policy is on track. Eval- Several key criteria may be used to choose relevant uation analyzes how and why observed changes in indicators to monitor for PSIA. First, if impacts are indicators have occurred. Impact evaluation assesses transmitted through specific channels (for example, the extent to which a past intervention has contributed changes in producer prices, increases in sectoral to changes in outcomes or impacts for individuals, employment), these are obvious indicators to track. groups, households, and institutions. Second, if the conceptual framework underpinning the analysis hinges on specific assumptions (for Particular characteristics of M&E in the context example, that traders or firms will enter with liberal- of PSIA ization, that consumers or producers will substitute, M&E related to PSIA may be seen as a subset of a or even that certain elasticities will be of certain national poverty monitoring system, and as having magnitude), the validity of these assumptions hold- several characteristics. It is focused on monitoring ing over time can also be monitored. As discussed impacts of specific policy reforms with a view to vali- above, tracing impacts through transmission chan- dating policy analysis or informing policy adjustment nels and making all assumptions explicit in under- during the course of implementation. This ideally taking PSIA increases understanding of the requires information on key indicators before (base- theoretical premises on which the program is based. 31 A User’s Guide to Poverty and Social Impact Analysis In the context of M&E, the process of tracing Effective monitoring facilitates good evaluation through the theory-based transmission channels Understanding gained during the process of ex-ante also enables one to identify potential intermediate analysis in the course of PSIA and the identification of and process indicators that can be used to monitor indicators helps in designing a good evaluation. the implementation and outcomes of reform. Third, Process evaluation is important to understanding the given the importance of monitoring for adjusting “hows” and “whys” of policy reform. Process indica- policy in “real time,” some indicators for PSIA (such tors are usually timely and not costly to collect. Trac- as prices) should be chosen so that they can be ing transmission mechanisms prior to the reform tracked over a short time period (such as six helps in thinking through implicit assumptions and months). The purpose is to identify proxy or inter- highlights where potential constraints or risks may mediate indicators for outcomes or impacts that will arise. The process also helps to evaluate whether the gradually materialize. One way to do this is to trace expected impact of the reform is borne out in practice. through the critical assumptions or “theory” Where it is not, more in-depth analysis to explain through which it is believed the reform will influ- divergence can be conducted. When results confirm ence outcomes. Fourth, it is important to establish the assumptions, documenting the lessons learned can indicators to monitor key risks to reform (see the help in the design of similar reforms elsewhere or in preceding section on risk assessment). These might the future. cover reform-specific risks (regarding transmission The approach used in identifying indicators will mechanisms or institutions, for example) or broader ideally encompass both open-ended and close-ended risks arising from the sociopolitical context (such as methods, and as far as possible incorporate participa- the risk of elite capture). Fifth, when monitoring the tory methods. Open-ended methods examine the how impacts of a reform, it is important to ensure that and why of policy reform, and in the case of participa- impacts on gender or the environment are included, tory methods, promote ownership, accountability, and especially when they are expected to be significant. transparency. Close-ended methods, on the other Finally, the choice of indicators can be informed by hand, only touch on the how and why of changes, and the existing set of indicators already monitored in are primarily designed to assess the magnitude of the country, in the context of the existing national change. poverty monitoring system or of regular government reporting to its stakeholders. Building on existing M&E to promote social accountability and systems reduces costs and limits duplications. transparency In addition, indicators should satisfy a simple set of Monitoring and evaluation can also be implemented basic technical criteria true for all monitoring indica- to promote social accountability during the process tors. The ideal indicator will be: of reform, thereby leading to increased ownership and sustainability. There are various M&E tools avail- ■ Highly and unambiguously correlated with the able that if used appropriately can help to promote objective variable of interest (for instance, test social accountability. These include public expendi- scores accurately reflect literacy) ture tracking surveys (PETS), quantitative service ■ Sensitive to changes in the outcome or impact of delivery surveys (QSDS), citizen report cards, and interest participatory public expenditure reviews (see box ■ Timely, in that it can be collected in time to feed 13). Similarly, perception surveys that capture more back into policy adjustment qualitative information provide another means of ■ Relatively insensitive to other unrelated changes in pinpointing problems within service provider organ- the sector izations. Ideally, quantitative and perception surveys ■ Relatively difficult to manipulate, either by target can be used in tandem to provide critical informa- groups or by policymakers tion on the issues surrounding design and access to ■ Not too costly to monitor. policy reform. 32 Elements of Good Poverty and Social Impact Analysis Box 13. M&E Tools for Promoting Accountability and Transparency during Policy Reform Public expenditure tracking surveys (PETS), quantitative ser- work for a PPER in which civic groups influence stages of the vice delivery surveys (QSDS), participatory public expenditure budget process in a cyclic and iterative manner. The PPER reviews (PPER), and citizen report cards are useful tools for framework can also be applied to the participatory monitoring tracking public expenditure and monitoring reform effective- and evaluation of policy reforms covering all levels of indica- ness as it pertains to the expected outcomes, processes, and tors—input, output, outcome, impact—in a participatory impacts that will occur as a result of policy reform. manner. The system has four key stages: PETS and QSDS collect data through structured interviews ■ Formulation: how expenditure proposals are made, to and documentation from service providers. While a PETS which sectors, and in what amount traces money through an organization, a QSDS provides a ■ Analysis: review of the impact and implication of alterna- more robust analysis by pinpointing organizational weak- tive policy proposals and allocations nesses that can be addressed through reform. One output of ■ Expenditure tracking: identification of elusive bureau- these survey instruments is a case-specific diagnosis of public cratic channels through which funds flow, bottlenecks in service delivery, helping to identify weaknesses in implementa- the flow of resources, and other deficiencies of delivery tion capacity and suggesting where reform efforts should be systems concentrated. Data from PETS and QSDS can help provide ■ Performance evaluation: direct feedback from citizens (for answers to several kinds of questions, including: example, report cards) on quality of, access to, and satisfac- tion with public services. ■ How to strengthen the “voice” of service users ■ What kind of accountability mechanisms between different One-off engagement at any stage of the PPER cycle can be use- levels of government can improve service delivery ful, but participatory public expenditure systems only deliver ■ How to regulate private providers. when the feedback loop is institutionalized and space is given to external voice at each stage. Achieving that level of institu- Drawing on a number of successful cases and tested models tionalization requires the commitment of significant resources from around the globe, the World Bank has developed a frame- over the long term. A few key principles should be borne in mind in impact evaluation is data- and time-intensive rela- establishing an M&E system: tive to other forms of evaluation, and often can only be implemented after the reform has already been ■ Participatory monitoring and evaluation can help in place for some time. Therefore, the decision to promote ownership of reform. It can be used to do impact evaluations should be based on a need to identify output, outcome, process, and impact indi- fill knowledge gaps, or to apply lessons learned in cators that are meaningful to stakeholders. Reaching expanding reforms. agreement on key performance indicators can be ■ Involvement of national expertise in the implemen- challenging, and is much better dealt with prior to tation and setting up of an M&E system—relevant the reform. Agreement on standards to be achieved is ministries, the statistical office, the planning office, valuable both for policy managers and for affected private research agencies, universities, NGOs—not parties who are then more likely to accept the results only promotes ownership, but helps to build capac- of monitoring reports and use this to improve policy. ity for poverty analysis. In addition, follow-up public disclosure of informa- tion strengthens commitment to the reform. Planning and implementing M&E: activities related to ■ Accountability can be promoted by employing spe- each stage of the PSIA cific data collection tools designed to allow benefi- Where possible, monitoring and evaluation systems ciaries to monitor inputs and outputs of the for PSIA should be integrated within an existing reform, while also soliciting their views on the national poverty monitoring and evaluation system. effects of policy outcomes on their well-being. Building on existing resources reduces the cost of set- ■ Selectivity in choosing whether to conduct an ting up the system, and further strengthens existing impact evaluation or not is important because national capacity. 33 A User’s Guide to Poverty and Social Impact Analysis Before the reform, while analysts are still grappling on a six-month-plus cycle, depending on the reform). with the key questions and objectives of the PSIA, a Soon after implementation begins, perhaps after three to preliminary list of indicators and required tasks and six months, preliminary monitoring and evaluation of timeline for the M&E system can be identified. In par- processes can be conducted to see whether the theory of ticular, it will be important to ascertain the existing how the reform would work is being supported in prac- information base and gaps, including the availability tice. Do specified inputs and outputs appear to be lead- of relevant baseline data with regard to key indicators ing to outcomes or impact in the manner expected? If and welfare measures and the possible need to collect not, why not? At this time, midstream adjustments are baseline data (see table 3). made, as required, to ensure the reform is on track. Once some ex-ante analysis is completed and there In the post-reform or completion stage, roughly is an improved understanding of how the reform will three to six months after completion of the reform, operate, the preliminary list of indicators, particularly there could be—as a matter of good practice—a fol- intermediate and proxy indicators, can be refined; low-up assessment and an incidence analysis of basic these may include views and perceptions of those to be outcome indicators to identify early “losers” and “win- affected. An instrument can then be developed to be ners” and reasons for the patterns observed. This used in measuring the indicators. It is important that analysis, along with a more rigorous evaluation, improved understanding of the program and indica- should ideally be repeated as required to fill knowledge tors feed into the design of the quantitative evaluation. gaps in key policy areas, or to inform plans to further Once indicators have been identified, plans can be deepen or expand reforms or scale up pilots. made to collect any missing baseline data, ideally before implementation of the reform. Element 10: Fostering policy debate During the reform or implementation period, there and feeding back into policy choice could be a periodic collection of indicators (proxy/inter- mediate, every three to six months; some indicators, For low-income countries, PSIA has been conceptual- such as prices, every month; outcome/impact indicators ized as an integral part of the PRSP process and as an Table 3. Planning M&E as Part of Poverty and Social Impact Analysis Reform timeline PSIA timeline M&E processes timeline STAGE 1: Identify key reform issues, questions, outcomes, and Identify input, output, intermediate, outcome, and impact indicators. Prior to reform risks for investigation. Identify availability of baseline data. Trace out “theory” of how reform will lead to the desired Identify existing information sources and gaps. results on the ground. Specify required tasks/needs for covering gaps in M&E. Preliminary field visit for ex-ante analysis. Identify specific institution(s) to be responsible for M&E. Design ex-ante analysis. Begin to define process for M&E—periodicity for data collection; storage; maintenance, etc. Plan collection of baseline data, if such data do not exist. Conduct ex-ante analysis. Refine preliminary indicators with input from key stakeholders. Collect baseline data. Design instrument to be used in measuring indicators. STAGE 2: During 3–6 months after initial implementation (and periodically Process evaluation. implementation of up until completion of reforms): follow-up analysis. Social impact assessment. reform Preliminary incidence analysis. STAGE 3: Completion/post 3–6 months to 1 year after completion of reforms Process evaluation. implementation of reform (depending on outcomes of interest). Social impact assessment. Incidence analysis. 34 Elements of Good Poverty and Social Impact Analysis element of the dialogue on the country’s poverty given policy reform. Such initiatives are particularly reduction strategy. Fostering and drawing upon public relevant in the context of widespread uncertainty, sus- discussion of policy can be useful at various points of picion, and ignorance—or in countries where poor the PSIA process —for example, to help identify stake- and marginalized groups have no political voice. holders and their positions, to understand transmis- Establishing systems and forums for policy debate is sion channels, to validate technical impact analysis, or not only a valuable part of ex-ante PSIA, but is also to leverage social accountability. It is critical for PSIA important for its contribution to monitoring and to “close the loop” and ensure that the lessons learned social accountability during implementation of a from impact analysis, monitoring and evaluation, reform and ex post, as discussed above. social accountability, and public policy debate actually There may also be good reasons for a government inform and affect policy.36 to take a policy forum seriously. Elected leaders who rely on democratic legitimacy to bolster their popular- Fostering policy debate ity may find such a forum attractive, as may policy- Policy formulation is not simply a technical process; it makers who are genuinely uncertain about which is political as well. PSIA provides the technical param- policy reform path to take. From a leadership perspec- eters for evidence-based policymaking, laying out for tive, it may be sensible and more sustainable to pursue policymakers what is feasible and what are the likely a policy that rests on a social coalition or bargain than impacts of proposed policies and reform actions. The one that theory may dictate as first-best. accompanying debates determine what is likely to be Convening such policy forums among stakeholders, realistic in that political context, where the perceptions however, is not without risks. One is that implicit con- and interests of particular constituencies are invari- flict between major interests may become open hostil- ably weighed against the merits and demerits of the ity. A second risk is that political competition may reform. For that reason, the policy debate needs to override the possibility of constructive dialogue.37 Yet involve technocrats and researchers as well as parlia- another common risk is that overly high expectations mentarians, civil society, donors, and other key stake- of the forum will result in disappointment: people holders whose support is essential to the reform. may assume that public debate will lead to the adop- The process of policy debate, including among tion of a policy that simultaneously meets the needs of stakeholders, can be just as important as the analysis. all stakeholders, whereas in reality the typical process Numerous studies have concluded that policy is most of negotiation and compromise during policy formu- likely to be effective where there is broad ownership, lation often leads to policies that do not mirror all and policy debate among stakeholders is useful in stakeholder preferences. developing consensus and building ownership. One Managing the process of policy debate and discus- way to approach this is to disseminate information sion itself requires some planning, particularly in about the proposed reform and the results of the PSIA order to manage risks. In particular, once the decision to the public, especially to key stakeholders, and then is made to convene a forum, three concrete issues must to organize a policy forum where stakeholders can dis- be addressed: whom to invite, what to discuss, and cuss the tradeoffs involved. Such a policy forum can how to structure the dialogue. These decisions are best produce invaluable information. Insights gained made jointly by the PSIA team and the relevant min- through dialogue may be technical (for example, aca- istry or agency implementing the reform under con- demic research) or social (for example, the perspec- sideration. In the context of social accountability tives and concerns of groups that typically do not discussed above, the government may integrate these participate in the formal policy debate process). These debates within existing political processes (for exam- insights can either validate or revise previous hypothe- ple, by opening parliamentary debates to outside ses or analysis, including critical assumptions. Com- stakeholders), but may also consider setting up more municating policy impacts to stakeholders can also inclusive, long-term structures of policy debate—such enhance their understanding of the logic behind a as regular consultations, national workshops, or “town 35 A User’s Guide to Poverty and Social Impact Analysis hall meetings.”38 In many low-income countries, such organizations (see GTZ 1991). For a description of the structures were established during the poverty reduc- problem tree see http://europa.eu.int/comm/euro- tion strategy (PRS) consultation process. Building on peaid/evaluation/methods/PCM_Manual_ those structures may be an easy and viable way to sus- EN-march.pdf, and European Commission 2002. tain this policy dialogue. 2. To the extent that stakeholder analysis helps focus subsequent research on specific sets of actors, it Feeding back into policy choice increases the relevance of more complex analysis of Ensuring that lessons learned from the continuous poverty and social impacts while reducing time and monitoring and analysis of policy implementation cost. A more detailed discussion of stakeholders and feed back to the redesign and adjustment of policy is a their relevance to policies and programs is provided in major objective of PSIA. Sound ex-ante PSIA, as dis- the Social Analysis Sourcebook (World Bank 2002c). cussed, should lead to an explicit articulation of 3. The identification process disaggregates these expected impacts, transmission mechanisms, and actors in terms of social characteristics—such as cul- assumptions, and the establishment of a monitoring tural, structural, economic, political, or governmental. system for key indicators tracking the evolution of the 4. Doing this early on provides a basis for early val- reform program. Necessarily, ex-ante PSIA will not get idation of hypotheses, subsequent identification of everything right. Rather, monitoring and evaluation, data and information needs, and more rigorous analy- during and after policy implementation, is a critical sis of hypotheses in subsequent steps of PSIA. part of PSIA, with the objectives of (a) correcting 5. Some questions address issues of ownership and flawed policies, (b) making adjustments to improve commitment discussed in the previous section. In sit- policy choices, and/or (c) identifying constraints and uations where informant interviews are not feasible or opportunities for further public action to maximize where findings are not considered reliable, the institu- poverty-reducing impacts. tional assessment tool can be used to conduct or com- A critical step in the PSIA loop, therefore, is the plement stakeholder analysis. feedback of lessons from the monitoring of reforms 6. Toolkits for institutional assessments can be found during implementation and the subsequent evaluation at http://www1.worldbank.org/publicsector/toolkits.htm. of the poverty and social impacts of policy choice, so 7. This statement requires qualifiers. First, faster that the M&E can lead to appropriate adjustment of methods of close-ended data collection and analysis policy. Institutional setups are fundamental here. A are being developed (for example, the Core Welfare common pitfall is that units or systems charged with Indicator Questionnaire). Second, reliable open-ended M&E are not properly linked with the decisionmaking analysis requires time and care if quality is to be bodies responsible for policy formulation. The crucial ensured. final link in an effective PSIA process, then, is ensuring 8. These data collection instruments have often that the key body making decisions about a particular been employed using non-random samples, for exam- policy reform is accountable for and charged with the ple in ethnographic analysis. However, there is no rea- reporting of related M&E and the periodic reassess- son that they could not be used on random samples to ment of policy. Here again, building institutional generate statistically representative data. Likewise, capacity, by creating such linkages where they may not non-numeric data could be coded into numeric data. previously exist, is an important part of the PSIA 9. See Rao and Woolcock 2003 for examples of agenda. mixed methods. 10. Most countries have now undertaken at least Notes one national household survey, although at times the vintage and quality of data are an issue. Intra-house- 1. The problem tree is a tool that has been popular- hold data, when available, can permit distributional ized through its integration within the ZOPP method- analysis at the level of individual household members, ology championed by many European development a particular concern in considering the welfare of 36 Elements of Good Poverty and Social Impact Analysis women or other individuals who may be less powerful “tools to assess public service delivery,” later in this or privileged within the household. section. 11. In examining an economy-wide reform, such as 16. Behavioral impact analysis, in focusing on a rice tariff increase, it would obviously be preferable demand analysis and supply analysis separately, can to adopt a representative sample for any new survey, or arguably be seen as a “partial” equilibrium analysis. to adopt the same sample (or select a panel) from a The distinction drawn here is that since market household survey for which data already exist. Where demand and supply are not equated and do not clear, the reform is location-specific, or affects a specified it is not technically “equilibrium” analysis. population—for example, with the shutting down or 17. In general equilibrium terms, it also effectively privatization of a state mining company—a purposive assumes a closure. sample of those expected to be directly affected would 18. The standard caution and caveat with respect to be appropriate. economic modeling thus applies: great care should be 12. Using a non-representative sample to extrapo- taken in specifying the model and its parameters to late differentiated impacts of policies among groups country context and in making explicit the specific nationwide assumes that national distributional char- assumptions and limitations of simulations derived acteristics are identical to those of the non-representa- from such models. tive sample—a non-trivial assumption. 19. Supply is either perfectly elastic (if chosen to be 13. For the purpose of presenting this simple table, endogenous) and entirely demand driven, or perfectly indicative classifications of high, medium, and low are inelastic—that is, supply is constant. SAM-IO simula- used, whereas clearly this is a continuum in practice. tions also vary greatly depending on the assumptions While recognizing that time, data, and local capacity made about which accounts are exogenous and which are not perfectly correlated, they are deemed a close endogenous. enough match to collapse into a single dimension. For 20. Dervis, de Melo, and Robinson (1982) and example, when data is used as a proxy for this dimen- Shoven and Whalley (1992) provide good summaries. sion, “low” means that no nationally representative 21. It is also possible to run the micro-simulation household survey data exist; “medium” means that exercise not on the basis of parameters derived from a nationally representative household survey data exist; consistent macro-model, but on the basis of exoge- and “high” indicates the existence of nationally repre- nously assumed changes in parameters. Such an sentative household survey data along with other data, approach would not be so different from the simplest such as census data for poverty mapping, national form of “direct impact analysis” described earlier. accounts, and other data for computable general equi- 22. This is also an area where work is still ongoing librium models. and new tools and applications continue to be devel- 14. The Social Analysis Sourcebook (World Bank oped. 2002c) provides a more detailed description linking 23. This has been done, for example, by Agénor equity and social sustainability to development out- (2002), who has estimated such an equation—includ- comes. ing the relevant elasticities—on the basis of a cross- 15. Incidence analysis has drawbacks. First, it does country regression tailored to take as inputs the not explain why things are the way they are. Second, outputs of the RMSM-X model. One limitation to this whereas incidence may use public expenditure as the approach relates to the robustness of cross-country measure of the service’s benefit to the recipient, there estimates of these elasticities when applied to a may be no correlation between expenditure and national context. received (or perceived) value, or outcomes. Third, as 24. SimSIP has a module that looks at growth with many interpersonal welfare comparisons, the impacts, and is being expanded to include a module results of the analysis may vary depending on the that will accept as inputs the key aggregate wage and method and the dimension used to rank households. consumption variables generated from the 1-2-3 See Demery 2000 and van de Walle 1998. See also model. 37 A User’s Guide to Poverty and Social Impact Analysis 25. The separate labor and poverty module can nario analysis, see: www.gbn.org/public/gbnstory/ simulate the impact of policies on the labor market, downloads/gbn_mont_fleur.pdf (South Africa). income and expenditures, and related social welfare 34. This discussion deals with monitoring and eval- indicators. It permits the reallocation of labor in uation only as they relate to PSIA. It is not intended as response to changes in prices and wages. a comprehensive treatment of the issue. 26. Ianchovichina, Nicita, and Soloaga (2001) used 35. Building capacity in this context includes not a similar approach to examine the impact of NAFTA only the development of technical skills, but also on household welfare in Mexico. changes in incentives and demands for such informa- 27. PSIA conducted for the water sector in Africa tion among country stakeholders (including govern- has highlighted the importance of carefully evaluating ment) as well as improved understanding of what which mechanism is best suited to specific country constitutes a good information base and how that conditions (lifeline tariffs, coupon schemes, subsidized information base can be used for more creative analy- types of supply). This is often done by consulting con- sis and for immediate policy decisionmaking. sumers and key stakeholders such as utility personnel. 36. This discussion deals with policy feedback only 28. Subsidization choice would depend, at least in as it relates to PSIA. It is not intended as a compre- part, on institutional capacity and transaction costs of hensive treatment of the issue. delivering the subsidy. 37. Under some circumstances, there may be com- 29. It is worth noting that such exemptions may pelling political reasons to avoid public forums. Exam- introduce undesirable distortions into the tax and ples of situations in which a policy dialogue may be incentive scheme, and not only from an efficiency inadvisable are: (a) government commitment to the standpoint. To the extent that they allow non-poor policy is irreversible regardless of public reaction to producers to avoid taxes legally or facilitate tax eva- short-term costs; (b) an intransigent opposition party sion, exemptions that appear patently progressive can or social movement is expected to use the forum sim- limit progressive budgets that address social programs ply as a vehicle to embarrass the government; (c) rep- for the poor. resentatives of marginalized people are lacking, 30. The opportunity cost calculation is complicated meaning that the only organized interests likely to to the extent that the reform package as a whole might have a seat at the table are privileged social groups; or be conditional on the compensation mechanism. (d) open violence between participants is a serious 31. This discussion deals with risk analysis only as it possibility. In such cases, some other form of consulta- relates to PSIA. It is not intended as a comprehensive tion with stakeholders may be more appropriate than treatment of the issue. For further treatment of risks a public policy forum. see the Social Analysis Sourcebook (World Bank 2002c). 38. This is another area where good PSIA should 32. This is being done, for example, in Madagascar, consider capacity building as part of the agenda. At the where three different modeling approaches are being institutional level, capacity is required to organize used to assess the impact of a rice tariff on distribu- such forums and to open up space for policy discus- tion. sion. At the individual level, capacity is often required 33. For an operational discussion with examples, see for informed and effective participation and thus for Maack 2001. For in-depth case studies of applied sce- an informed debate. 38 4 Challenges and Operational Principles The previous chapter has presented a road map to line between simplifying reality to explain impacts conducting good PSIA. Practitioners should follow an and capturing context-specific institutions and approach to PSIA that is country- and context-spe- behavior. cific, dependent upon available data and capacity as Second, the extent and nature of reform impacts well as the reform issue in question. Key constraints may differ over time. For instance, the impact of tax and principles are briefly outlined below. reform might be limited to a single sector in the short run but might expand to other sectors over time as Constraints agents adjust to the new tax rates. Or a policy might result in short-term losses and gains among different Specific challenges that analysts may expect in practice groups, even when long-term effects are expected to include constraints on data, analysis, capacity, and time. be positive. Capturing these inter-temporal dimen- sions within distributional analysis is a complex Data and information constraints undertaking. In many instances the data and information required Third, rigorous analysis requires a comparison to to do a comprehensive analysis are not readily avail- be made between outcomes with and without reform able. Household survey data, which are particularly (the status quo itself being an alternative policy choice relevant to undertaking distributional analysis on a or counterfactual). This is very hard to do ex ante, national level, sometimes do not exist or are dated. when the analyst has to “simulate” what would happen Equally common, information sources that do exist, without a reform. It is also challenging in ex-post including survey data and sociological analyses, may analysis, as many factors will have changed during not address questions relevant to the reform at hand. reform implementation, masking reform-specific effects. Analytical constraints Finally, addressing these analytical challenges First, it is difficult to analyze the impact of macro- requires the right economic and social tools. Many economic and structural reforms at the microeco- useful tools exist, and this User’s Guide highlights nomic or household level. Policies have many direct some of the main ones. But more work is needed to and indirect effects at the microeconomic level, develop analytical methods that are better equipped to mediated through local institutions and behavior. meet the gaps and to develop more rigorous indicative It is often difficult to capture the complexity of survey tools and analysis, where adequate information reality in a model. The analyst has to walk a fine is otherwise lacking. 39 A User’s Guide to Poverty and Social Impact Analysis Capacity constraints Increase attention to ex-ante analysis Capacity constraints affect the choice of analytical It is important that ex-ante analysis of expected method. In poor countries, capacity to analyze policy poverty and social impacts underpin the design and is typically limited among government agencies, aca- choice of policies, particularly those that are expected demia, and civil society organizations. So while rigor- to have the greatest impacts in the short to medium ous analysis might call for complex tools and methods, term. This will help ensure that policies are conceived, local capacity may be suited to simpler approaches. designed, and implemented with a view to enhancing Over time, however, the capacity of in-country devel- poverty reduction and social objectives. opment agencies also requires strengthening, in terms of both core analytical expertise and resources allo- Build on earlier experience cated to PSIA. In practice, reforms often involve a series of measures over a long period of time. The ex-ante analysis of Time constraints future reforms can be informed by the analysis of ear- While the analyst may face difficult data and analytical lier reforms to ensure that past events and changes are challenges, the policymaker is often under pressure to considered. Where possible, ex-post and ex-ante make fast policy decisions and will not want to wait for analysis should be combined. a rigorous PSIA to be completed. In such circum- stances—such as in an economic crisis—arguments Use monitoring and evaluation to validate ex-ante for postponing policy decisions until there is adequate analysis analysis, debate, and consensus will need to be Ex-ante analysis cannot fully capture policy impacts. It weighed against the case for acting expediently. In is therefore important to track actual results through other cases, policymakers may want to time their monitoring and, where possible, ex-post evaluation. action for a particular moment in tune with the policy That way, midcourse corrections can be made to or political cycle. reforms that are not having their intended poverty or social impact. In many low-income countries, national Principles poverty monitoring capacity is being developed through PRS monitoring systems. Whenever possible, The challenges outlined above have often deterred pol- PSIA monitoring should be integrated within the PRS icy analysts and decisionmakers from undertaking ex- monitoring system. ante assessments of the poverty and social impacts of reform. While some have argued that “no analysis is bet- Maintain flexibility on tools and methods ter than bad analysis,” it is important to consider what It is important to tailor approaches to country capac- analysis is feasible, even where data and capacity are ity, reform issues, data availability, and time pressures. limited. The question, then, is how to approach poverty In some circumstances, some basic economic analysis and social impact analysis in the face of the various con- enriched with qualitative analysis may be appropriate, straints. Some basic principles for a good analysis of while in other cases econometric modeling may be the poverty and social impacts of reforms are as follows. most useful entry for PSIA. Understanding of impacts is enhanced when results from different analytical Promote country ownership techniques reinforce each other or highlight different If PSIA is to be an effective tool for policy, it needs to aspects of impacts. be country-owned. Ideally, countries should be responsible for the choice of reforms and for the Increase transparency in the links between policy and analysis. In undertaking the analysis, they can seek poverty external assistance from partners including the World There is much to be gained from laying out for public Bank, the United Nations, and bilateral donors. scrutiny the logic behind a policy choice—including 40 Challenges and Operational Principles Build national capacity expected losers and winners from reform, key assump- tions, and transmission mechanisms. It can help pro- Building national capacity is key to improving ana- mote national debate and acceptance of reform, and lytical rigor over time, in tandem with strengthened serve as a baseline against which to monitor progress. country ownership. Many low-income countries Moreover, it can highlight potential tradeoffs between have limited capacity and experience in areas of crit- the long-run benefits of reform, in terms of higher ical importance to PSIA. These areas include data growth and poverty reduction, and possible welfare collection systems, monitoring and evaluation sys- declines in the short run. tems, the capacity to conduct analysis and to trans- late data and analysis into policy, and the Strive to enhance gains and minimize losses, institutional structures and mechanisms for debate especially among the poor on such policy issues in the public domain. Building PSIA should give explicit consideration to such meas- national capacity in these areas must be a funda- ures as alternative policy choices and complementary or mental crosscutting aspect of PSIA. Development compensatory policies intended to enhance the benefits partners, including the Bank, have an important role to stakeholders, especially poor people, and minimize in strengthening national capacity and in filling ana- the losses they may experience as a result of reform. lytical gaps. PSIA approaches that foster “learning by This will strengthen the pro-poor impact of policies doing” should undergird development partners’ and improve their acceptability and sustainability. assistance to countries. 41 5 Possible Summary Matrix Chapter 3 presented a series of elements for good for PSIA (question 1) and the institutional mecha- PSIA. Pulling these elements together in a coherent, nisms through which the reform will be carried out strategic, and integrative fashion is what makes for (question 2). It then allows the analyst to summarize good poverty and social impact analysis. Invariably, as the anticipated impact of the policy reform on differ- discussed throughout this paper, a sensible approach ent stakeholders, as transmitted through the five chan- to PSIA is going to be country- and context-specific, nels discussed in chapter 2: employment, prices, access dependent upon available data and capacity as well as to goods and services, assets, and transfers and taxes the reform issue in question. Box 14 provides an (questions 3, 4, and 5). The analyst should explicitly example of this, describing the PSIA approach cur- recognize the stakeholders who are likely to gain from rently being used in Chad to address an ongoing the reform, those who are likely to lose, and those who reform issue in that country. are likely to have significant influence over the reform. The User’s Guide recognizes that the tools and tech- The matrix also calls for an explicit statement of the niques used for PSIA are likely to vary greatly across assumptions underlying the reform. Depending on the countries and reforms. However, regardless of the cho- country concerned, conclusions on likely policy sen methodology, there are some key components that impacts will draw on differing information bases and should be addressed in this kind of analysis. Table 4 tools. For example, in one country context the matrix presents an example of a summary matrix that captures may be filled out using informed reasoning based on and integrates these key components. In addition to secondary data and qualitative field research; in providing the analyst with a framework for considering another context, the conclusions may be based on and articulating key aspects of PSIA for a given reform, empirically simulated effects derived from modeling it offers a template for making explicit some of the techniques using data from a recent household survey results and assumptions underlying such analysis. The and existing social analysis. In either instance, the matrix itself can serve as a useful tool during the PSIA matrix calls for a description of the nature of the process. For instance, an analyst may wish to sketch out information base and analytical methodology. The the priors in each of the 10 elements of good PSIA matrix also calls for the analyst to specify key risks before even undertaking an analysis, and then return to associated with the reform, their likelihood and the matrix to validate or correct these hypotheses. expected magnitude (question 6). Finally, it proposes The matrix calls for the analyst to set out the reform to present the impact that the analysis has had on components and the reasons for selecting that reform national policy discussions (question 7). 42 Possible Summary Matrix Box 14. Poverty and Social Impact Analysis of Cotton Reform in Chad Cotton is a key crop in Chad, both for revenue generation and The aim of the scenario study is to identify and evaluate the for poverty reduction. Cotton accounted for 24 percent of total technical and economic efficiency of alternative scenarios for government revenues in 1997 and is the most important cash privatizing Cotontchad. The study examines options for priva- earner for about 300,000 rural farm families. However, weak tization (such as continued vertical integration, separate pri- organization and knowledge among farmers’ groups coupled vate ginneries, and so on) and assesses the risk posed by each. with structural inefficiencies in the sector have resulted in low yields and low farmer revenues. In parallel with this study, the PSIA assesses the impacts of the reforms on the welfare of farmers in the sector. The ex-ante quali- To address these inefficiencies, the government of Chad has tative component identifies relevant stakeholders (including farm- decided to privatize Cotontchad, the parastatal that currently ers, Cotontchad employees, microentrepreneurs), barriers faced by manages national cotton production, and strengthen farmers’ stakeholders under different reform scenarios, the strength of cur- groups. A key objective of the cotton sector reforms is to rent institutional structures, and the social risks of reform. improve farmer incomes. Several factors underscore the gov- ernment’s decision to proceed carefully in designing and The quantitative and qualitative analyses look at the compen- implementing the reforms: the possibility that yields will fall sation and enhancement measures necessary for reform suc- further if reform prompts farmers to return to subsistence cess and highlight farmer capacity, access to credit, input use, agriculture, the limited availability of information on rural and transport. Further work involves a “quasi-comparison poverty, and cotton farmers’ perceptions of the risks involved group” for different types of farmers—those who produce cot- with the reform. For these reasons, the government is carrying ton and those who do not or who have abandoned cotton—in out a poverty and social impact analysis to guide the reform. order to analyze the likely impact of the reforms on different groups and get a sense of the welfare impact on farmers who In order to analyze the likely poverty and social impacts of abandon cotton production. reform ex ante, the PSIA needs to do a problem analysis and clarify the assumptions on which the program is based. The The different scenarios for partial and complete privatization PSIA team, in consultation with the government and local and the ex-ante qualitative and quantitative work will be dis- counterparts, has identified pathways through which they cussed during a stakeholder forum. This public discussion is expect the reform to improve performance. By explaining the meant to increase the transparency of the reform and build causal links that tie program inputs to expected outputs, out- ownership by fostering policy debate. comes, and the ultimate goal of improving farmer income, the team has explicitly outlined the assumptions for each trans- In addition, there will be an ex-post impact evaluation of the mission channel of the reform so that they can be verified. reform. The ex-ante analysis will define key indicators to be monitored for policy feedback in ex-post analysis. The ex-post The PSIA that grew out of these discussions has three compo- analysis will employ quantitative methods of impact evalua- nents: (a) an economic scenario study of different options for tion, which attempt as far as possible to assess impact based on privatization; (b) ex-ante qualitative analysis, and a baseline what would have happened in the absence of reforms. This ex- quantitative survey; and (c) ex-post analysis that includes both post quantitative analysis will be applied to a panel data set, to qualitative and quantitative methods. estimate the impact on producer welfare. 43 Table 4. A Summary Matrix for Poverty and Social Impact Analysis of Reform Reform: 1. What reform was chosen (major components), and why? 2. What are the institutional mechanisms through which the reform will be carried out? 3. Which stakeholders are likely to be affected 4. Through which channels are the stakeholders affected? 5. What are the expected direction and order of magnitude of impact(s)? What by the reforms? Which stakeholders are likely Labor Prices Access to Assets Transfers are the underlying assumptions? to affect the reform and how? market goods and and taxes services Stakeholders affected (positively and negatively): Stakeholders with significant influence over the reform: Possible support or opposition: A User’s Guide to Poverty and Social Impact Analysis 44 What information basis and techniques were used to answer questions 3, 4, and 5? 6. What are the main risks that would change the expected impact of the reform? What are their likelihood and expected magnitude? Type/nature of risk Likelihood Expected magnitude Political economy risks: Exogenous shocks: Institutional risks: Other country risks: 7. What impact has the analysis had on national policy discussions? 6 Conclusions This User’s Guide to PSIA has provided an initial approaching PSIA. Furthermore, it has given a brief overview of the key considerations for practitioners overview of some of the tools and methods that contemplating the poverty and social impacts of policy might be used in undertaking analysis of poverty options with a view to informing policy choice and and social concerns associated with policy change. In design. It contends that ex-ante analysis of the likely so doing, it has attempted to draw upon tools used poverty and social impacts of a specific reform can be by economists and social scientists and to present undertaken more systematically than is typically done them in an integrated fashion. Applying these tools at present. It also takes the practical view that, for this to to the operational context using this multidiscipli- be possible, approaches and methods will need to be nary approach will lead to a richer, more integrated adapted to fit the context and circumstances at hand, understanding of policy impacts. Moreover, because and the limits to what is possible through ex-ante analy- of the marked differences between individual cases sis will need to be addressed through continual moni- in terms of reform issues, transmission channels, toring, analysis, and reevaluation of policy over time. and available data, the choice of tools and methods This User’s Guide has laid out 10 key elements to used for PSIA will vary substantially by type of be considered by analysts and policymakers in reform. 45 Annex: Economic and Social Tools for Poverty and Social Impact Analysis This annex presents information on a series of tools evaluate, what types of questions it can answer, and its and methods available for the analysis of poverty complementarity with other tools/methods, (2) its and social impacts of reforms. This annex presents key elements, (3) the requirements in terms of data, summary information on the tools, drawing in par- time, skills, software, and cost, (4) the limitations of ticular on the Toolkit for Evaluating the Poverty and the tool/method, and (5) references and country Distributional Impact of Economic Policies and the applications. Social Analysis Sourcebook, which provide more Note that some of the tools presented in this annex detailed information1. Additional guidance is cur- belong to more than one category. For instance, bene- rently under preparation on selected social and eco- ficiary assessment or participatory poverty assessment nomic tools. The World Bank is also developing can also be used as monitoring tools; while public guidance on issues, challenges, and tools that may be expenditure tracking or quantitative service delivery of particular relevance in analyzing specific reforms. surveys can also be used to analyze stakeholders and A summary matrix and reform-specific notes will be impacts. Also, note that some of the tools to analyze posted on an ongoing basis on the PSIA website. impacts categorized under “social” or “economic” This annex highlights some of the key tools for such actually use a mix of methods, as is the case for analysis, but does not aim to be comprehensive in its demand analysis. Moreover, some of the techniques coverage; updates on additional tools and methods presented can be used in carrying out more than one will be posted on an ongoing basis on the PSIA web- type of analysis. For instance, demand and supply site: http://www.worldbank.org/psia. These tools are analyses are components of partial equilibrium analy- organized following the User’s Guide elements, sis presented under “Multi-market models”, and both including stakeholder analysis, institutional assess- IMMPA and the Augmented CGE model with repre- ment, impact analysis, risk assessment and monitor- sentative household approach also fall within the “gen- ing. In practice, the analysis of poverty and social eral equilibrium models” category. impacts of reforms requires the combination of a vari- ety of complementary tools, both within and across categories. In addition, some tools have evolved to Note comprise the integrated application of both social and economic methods. 1. These are available at Each tool or method is presented within a sum- http://www.worldbank.org/psia and mary table. The table contains five components: (1) http://www.worldbank.org/socialanalysis what the tool/method is, what policy reforms it can sourcebook/ , respectively. 47 A User’s Guide to Poverty and Social Impact Analysis The tools and methods presented in the annex include the following. I. Identifying Stakeholders – Stakeholder analysis II. Assessing Institutions – Institutional analysis III.Analyzing Impacts – Social Tools – Social impact analysis – Beneficiary assessment – Participatory poverty assessment – Social capital assessment tool – Demand analysis: Consumer assessment IV. Analyzing Impacts – Economics Tools 1. Direct impact analysis tools – Benefit incidence analysis (Average and Marginal) – Tax incidence analysis – Poverty mapping 2. Behavioral models. – Ex-post behavioral marginal incidence analysis of public spending and social programs – Ex-ante behavioral marginal evaluation of policy reforms – Ex-post impact evaluation methods for assigned programs – Demand analysis: Estimating demand functions – Supply analysis – Household models 3. Partial equilibrium models – Partial equilibrium analysis: Multi-market models – Partial equilibrium analysis: Reduced-form estimation 4. General equilibrium models – Social Accounting Matrices – Computable General Equilibrium (CGE) Models 5. Tools linking microeconomic distribution or behavior to macroeconomic frameworks or models – PovStat – SimSip Poverty – 123 PRSP – Poverty Analysis Macroeconomic Simulator (PAMS) – Integrated macroeconomic model for poverty analysis (IMMPA) – Augmented CGE Model with Representative Household Approach V. Assessing Risks – Social Risk Assessment – Scenario Analysis VI. Monitoring and Evaluation – Public expenditure Tracking Survey (PETS) – Quantitative Service Delivery Survey (QSDS) – Citizen Report Card – Community Score Card 48 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Stakeholder Analysis What is it? Stakeholder analysis is a systematic methodology that uses qualitative data to determine the interests and influence of different groups in relation to a reform. What can it be used for? While stakeholder analysis can be carried out for any type of reform, it is particularly amenable to structural and sectoral reforms. Basic stakeholder analysis should precede reform design and should be consistently deepened as reform elements are finalized. What does it tell you? Stakeholder analysis assesses: (i) the extent to which reform may provoke political or social action; (ii) the level of ownership among different groups; (iii) differences in perception of the reform among different ethnic, religious or linguistic groups. Stakeholder analysis can be expanded into fuller political economy analysis that identifies affected groups and looks at: (i) their position vis-à-vis policy; (ii) their influence on government; (iii) the likelihood of their participation in coalitions to support change; (iv) strategies for overcoming opposition such as compensating losers or delaying implementation. Complementary tools: • Normally used in conjunction with social impact analysis. • Stakeholder analysis identifies groups to consider as categories for analysis. It is useful for the design of household surveys, microeconomic modeling and micro-macro linked models. Key Elements: Stakeholder analysis is iterative, and usually proceeds through the following sources of data to reach final conclusions: (i) background information on constraints to effective government policy- making; (ii) key informant interviews that identify specific stakeholders relevant to the sustainability of policy reform. Participants should be drawn from a diverse groups of interests in order to limit bias; (iii) verification of assumptions about stakeholder influence and interest through survey work and quantitative analysis of secondary data Requirements Data/information: Stakeholder interests are seldom explicitly spelled out in existing sources. The main sources of information are: (I) key informant interviews; (ii) secondary material such as newspaper articles, and social science research. Time: In cases where key informant interviews are already being carried out as part of other qualitative analysis, preparing an analytical piece on stakeholders can take as little as one additional staff week of effort. In cases where there is no significant qualitative work planned, a thorough exercise would likely involved a trip to the field and two to three staff weeks of effort. Analysis that is meant to predict the positions of key stakeholders in different reform scenarios is not a one-off piece of work and should grow out of the findings of other analytic work. Ensuring a complete and updated picture may require that specialists carry out the work over several calendar months. Skills: Sociological or anthropological training is helpful, as is a background in political science. Local knowledge, including contacts with local experts, is crucial. Those carrying out the analysis must also thoroughly understand the reform and the recent history in the sector. Supporting software: N/A Financial cost: When combined with other qualitative work, the incremental cost of stakeholder analysis can be as low as US$10,000. When no qualitative work is planned, costs can be up to US$25,000. Limitations: Stakeholder analysis relies on qualitative data and perceptions and preferences. The absence of statistical representative places greater onus on careful selection of respondents and interpretation of data. References and applications: • Bianchi and Kossoudji. (2001). • Brinkerhoff and Crosby (2002). • World Bank (2002e), Annex VII on Guyana. 49 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Institutional Analysis What is it? An analytical approach that uses qualitative methods to unpack the “black box” of decision-making and implementation processes What can it be used for? Useful for PSIA regardless of reform type, but particularly important for policy changes involving institutional reforms, such as decentralization of public services, utility reforms, land reforms, social safety net reforms. Useful for policy design and implementation. What does it tell you? Understanding of political economy and governance issues through analysis of the institutions that are involved in the design and implementation of reforms, and identification of dynamic processes, and potential constraints in this respect. Steps include: (i) Identification of government agencies, non-government organizations and firms that carry out the policy reform. (ii) Identification of their characteristics and dynamic relationships. Output: Understanding of the formal “rules of the game” (via static mapping, i.e.: functional organigram), and the informal rules that govern actual behavior in decision-making processes (via process mapping of crucial resource flows, e.g. money, information). Complementary tools: • Used in conjunction with Stakeholder Analysis, SIA, and demand analysis/customer assessments • PETS, Benefit Incidence Analysis Key Elements: Three types of information: (i) background information on key stakeholders, and organizational structures of relevant agencies; and (ii) in-depth interviews or focus groups with key informants from government agencies, non-government organizations and firms; (iii) triangulation and cross-referencing with other information to validate background information and key informant interviews. Requirements Data/information: Secondary material, including PERs, DPRs, IGRs, social/ political science research and in-country assessments of organizational structures and institutional settings. Primary data, that illustrates informal practices and identifies the dynamic processes within the policy design and implementation Time: A few weeks (4-5 person weeks) to a few months (2-3 person months for fieldwork, analysis and report): Can be completed quickly in combination with a Stakeholder Analysis to gain a brief overview of the formal and informal rules of the game. Institutional Analysis that aims to identify the dynamic processes within the policy design and implementation requires a more in-depth analysis, and may take a few months. Skills: Sociological, anthropological, and public policy training (incl. qualitative field research skills) are helpful. In depth knowledge of the country-context, reform area, policy design and implementation, and political economy (including interests and influences of key stake holders) is crucial. Supporting software: IPS Ltd.: http://www.ips-uk.com/ProcessMapping.htm - ProcessMap; Toolpack.com: http://www.toolpack.com/workflow.html; HPS Inc.: http://www.hps-inc.com/ithinkDemo.htm#; Triaster http://www.processnavigator.com/english/index.html; Ash House: http://www.ashhouse.co.uk/process.htm; Process Mapping: http://www.processmapping.com/; TSQ Europe: http://www.tqseurope.com/activemo.htm; Designtech: http://www.designtech.com/processmap.html; Financial cost: Depending on the depth of analysis, it can be low-cost if used in combination with Stakeholder Analysis, or adapted to SIA (US$ 25,000), but can be more costly if done more in depth (US$ 50,000). Limitations: Care should be taken in generalizing findings across different units of analysis and across regions with dissimilar informal institutions even within a country (e.g. panchayat institutions will vary enormously across different states within India). Resource and time requirements vary by the depth of analysis (incl. scope of geographical fieldwork done at local, provincial and/or national level) and reform complexity which may necessitate continuing the analysis during implementation. References and applications: • Brinkerhoff and Crosby (2002). • Hunt (1996). • North (1990). • Tymons and Jacobs (1997). 50 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Social Impact Analysis What is it? An analytical framework to identify the range of social impacts and responses to reform by people and institutions, including those that are vulnerable or poor. Often undertaken in an iterative manner, and includes relatively detailed information on social context for reform. What can it be used for? Can be used for many types of policy reforms. Has been used extensively for mining sector restructuring, parastatal privatization and agricultural reforms giving rise to significant social impacts. What does it tell you? Social, political context for reform, who is affected by the reform at what point in time, preferences and priorities of those affected by reform, constraints to implementation of reform, how people, institutions are likely to respond to reform including whether assumptions on how they will react or be affected by the reform are correct. Also provides insight into coping mechanisms and social risks, suggestion from stakeholders on most appropriate means to mitigate negative impact of reform and potential effectiveness in local context. Complementary tools: Used in conjunction with stakeholder analysis. Other tools such as institutional analysis and risk analysis complement and draw heavily on SIA. SIA can feed into assumptions for economic modeling. Key Elements: Characterized by use of mixed methods and direct consultation of those potentially affected that can include a wide range of data collection techniques: open-ended community discussion, key informant interviews, focus groups, quantitative survey, observation, ethnographic field research, PRA. Proper structuring of qualitative methods and interpretation of both qualitative and quantitative research requires sufficient knowledge of local customs and cultures and thus normally requires partnership with local consulting, NGO or research firms. Typically, SIA uses purposive surveys to collect quantitative information from a sample representative of a particular region or population groups relevant to a particular reform. This is particularly useful in situations when national household data do not exist or do not contain the specific information needed to assess reform impacts. Requirements Data/information: (1) The degree of diversity of the groups likely to be affected or to influence a reform (from the stakeholder analysis) based in part on detailed country level contextual information (cultural, ethnic, regulatory and institutional issues relevant to the reform or affected groups), typically from existing studies, press reports, and key informant interviews. This determines the sampling strategy for fieldwork. (2) Direct data on stakeholder perspectives, typically from field research. (3) Quantitative data typically on income, expenditures, behavioral responses, coping mechanisms or other variables relevant to the reform to compare with results from qualitative data. Typically, SIA uses purposes surveys to obtain quantitative information relevant to a particular reform expected to have disproportionate impacts on a specific region or known population groups. The sample will then be representative of that region but not nationally representative. This is particularly useful for situations when national household data do not exist or do not contain the specific information needed. Time: SIAs can vary greatly in length depending on the scale of research and the number of sample areas (which will be in part a function of the diversity or complexity of the groups involved and the size of the population affected). As this is typically combined with stakeholder analysis, a minimal time for both exercises is approximately 3 man months. Skills: Often requires either a team with mixed skills (in qualitative techniques and in quantitative data collection and analysis, and preferably with someone with sector knowledge), or two teams or individuals working together. The coordination, and iterative analysis of both qualitative or participatory data collection methods and quantitative analysis is paramount. Supporting software: N/A Financial cost: Varies according on the depth and purpose of analysis. A complete mixed methods SIA costs US$80-100,000. When national household survey data exist, a supplementary SIA for a specific reform would cost an average of US$35,000, excluding supervision time. May cost more where local capacity is low and needs to be supplemented by international consultants. Limitations: SIA is not the best instrument to use for broad based reforms where the transmission channels and groups affected are not well known. References and applications: • Finterbusch, Ingersoll and Llewellyn (1990). • Goldman (2000); Becker (1997). • World Bank (2002c) http://www.worldbank.org/socialanalysissourcebook/socialassess.htm. • Cernea and Kudat (1997) on the application to sectoral policy reforms including tariff issues. • Other applications: http://lnweb18.worldbank.org/ESSD/essdext.nsf/61ByDocName/CaseStudies 51 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Beneficiary Assessment What is it? A participatory assessment method and monitoring tool that incorporates direct consultation of those affected by and influencing reform. Similar to PPA, it relies primarily on qualitative research though with less emphasis on the use of visual techniques and on community follow-up to the research process. What can it be used for? Has traditionally been used to evaluate projects or sectoral reforms in the health, education, infrastructure, social protection and agricultural sectors, but can be adapted to assess or monitor the impact of some discrete policy interventions where transmission channels and affected groups are clearly defined. Can be used even for countries with limited capacity as an add-on to other economic tools. Used both to evaluate proposed reforms, to signal constraints to participation faced by target group, as well as to gain beneficiary feedback for ongoing reforms. What does it tell you? What is the beneficiary perspective on the problem being addressed by the reform, their perception of the proposed policy, and of any mitigatory measures being considered. Provides insights into the likely reception the reform will receive, as well as issues that may arise during implementation. Tends to reach down to the community-level, but not focused exclusively on the poor or the community. Complementary tools: • Used in conjunction with stakeholder analysis, and institutional analysis. Can also complement representative quantitative surveys. • Information on how different groups are likely to react to a proposed policy change can influence assumptions in macro and micro models, in terms of behavioral response (particularly where historical data is insufficient or lacking). Key Elements: Relies primarily on three data collection methods: (1) conversational interviews (2) focus group discussions, which in some cases have been combined with PRA tools; and (3) direct and participant observation. Although information collected may be qualitative in nature, also includes quantitative analysis of this beneficiary feedback. Requirements Data/information: Background information on stakeholders, on cultural, ethnic, or socioeconomic variations, and on the variables determining whether specific groups would be affected (such as type of access) is required to properly design a BA and its sampling strategy. Time: Generally within three to four months, from design to presentation of the final report. Skills: Sociological or anthropological training are helpful, but good listening skills are paramount. Good knowledge of the program, historical and cultural setting also important. Supporting software: N/A Financial cost: Average of US$40,000. Limitations: Tends to have a narrower focus than SIA or PPA, providing less contextual and historical background information, though also likely less resource intensive. References and applications: • Salmen (2002). • Salmen and Amelga (1998). • For summaries of specific country application of both BA and PPAs see: • http://www.worldbank.org/participation/beneficiaryassesment/beneficiary assessment.pdf 52 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Participatory Poverty Assessment What is it? An instrument for including the poor directly in discussions and debates on policies and priorities, and that relies primarily on qualitative, visual, participatory rural appraisal techniques. Uses data collection techniques similar to BA, though with a greater focus on consultation of the poor, and on a broader set of policy issues affecting the poor. What can it be used for? Can be adapted to the analysis or monitoring of many policy reforms. Has been used extensively in public expenditure reforms that require priority setting, or better understanding the reasons for accountability or low service use, or for institutionally complex reforms (such as land reform, liberalization of markets, labor market reforms) or for better targeting safety nets. Could also be used to monitor the local impact of macroeconomic policies such as devaluation. What does it tell you? In-depth analysis of the views of the poor and their political, social, and institutional context; policy priorities of the poor, multi-dimensional dynamic of poverty and of coping mechanisms; identification of constraints that could be overcome through public action to increase access to reform benefits, with a particular focus on constraints for the poor. Complementary tools: • Used in conjunction with stakeholder analysis. • Can also be used to complement institutional analysis, larger representative household surveys, or SOCAT. • Can be used together with poverty mapping, statistical analysis of household surveys, public expenditure tracking surveys, and benefit incidence analysis. Key Elements: PPAs (i) use a variety of flexible participatory methods that combine visual methods (mapping, matrices, diagrams) and verbal techniques (open-ended interviews, discussion groups) and (ii) emphasize exercises that facilitate information sharing, analysis and action, with a goal of giving communities more control over the research process. By their very nature, PPAs may create opportunities or expectations of follow-up at the community level, such as the development of community action plans, often supported by local government or NGOs. Requirements Data/information: Selecting the appropriate (purposive) sample areas for PPAs (typically from 40 - 60 sample communities) requires an adequate understanding of social, economic and poverty context of the various regions or areas of a country. PPAs focus on direct field research and therefore do not have other information pre-requisites. Time: From 5 to 9 months for research and analysis, assuming a research team of between 10 and 20 people. Skills: Skilled and experienced facilitators, who are able to listen and record information in as unbiased a manner as possible, and to manage expectations from the PPA at the community level. Supporting software: N/A Financial cost: From US$15,000 to US$200,000 depending on scale. Limitations: Not statistically representative. May raise expectations for follow-up or service improvements at the community level that local actors and/or the research team may not be able to provide. References and applications: • Robb (2002). • Norton et al (2001). • Salmen (1995). • For summaries of specific country application of both BA and PPAs: • http://www.worldbank.org/participation/beneficiaryassesment/beneficiary assessment.pdf 53 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Social Capital Assessment Tool (SOCAT) What is it? A set of integrated quantitative and qualitative measurement tools to investigate institutions, networks and norms that enable collective action. Has to be adapted to a specific research issue. Can be implemented in conjunction with other tools. What can it be used for? Primarily useful for reforms with low/medium indirect impacts. Agricultural reforms (changing subsidies/taxes), liberalizing markets, financial reforms (changing access to credit), labor market reforms (active labor market programs), utility reforms (access to services), decentralization, social safety net programs (changing public/private transfers). What does it tell you? Existence of institutions and networks affected by and/or involved in reform implementation. Which norms and values lead to policy adoption or resistance? The distribution of social assets and their role in income generation and risk management. What are the impacts of reforms on households with low social assets? Which adaptations in policy formulation and / or which mitigation measures are advisable? Complementary tools: Stakeholder analysis, institutional analysis, social impact analysis (SIA), beneficiary assessments (BA). Key Elements: Integrated application and analysis of quantitative and qualitative information (surveys, key informant interviews, focus groups) obtained at the level of households, communities, and organizations. Analysis builds on the understanding of solidarity, trust and cooperation, and conflict resolution (cognitive social capital), as well as organizations and their membership (structural social capital). Requirements Data/information: Use as standalone tool for social capital analysis, or use in conjunction with other surveys (e.g. LSMS, income/expenditure surveys) for analysis of links between poverty and social capital. Modules for integration in other surveys are available, so are sector specific questionnaires. Time: Typical application requires 3-4 months. Skills: Sociological or anthropological training are helpful, in particular a sound understanding of formal and informal institutions and networks. Good knowledge of the program and its setting is crucial. Supporting software: SOCAT Toolkit including interactive CD-ROM is available. Financial cost: Depends on sample size and local wage and transport costs for field team. Typical range for standalone SOCAT exercise would be US$50,000 to US$200,000. Costs can be substantially lower if used in conjunction with other data collection instruments. Limitations: Collects social capital data only at micro and meso levels. For analysis of links between social capital and poverty, combination with other survey is necessary. References and applications: • Grootaert and van Bastelaer (2002). • Additional information at: http://poverty.worldbank.org/library/topic.php?topic=4294 or at: http://iris.umd.edu/adass/proj/soccap.asp 54 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Demand Analysis: Consumer Assessment What is it? The adaptation and expansion of traditional demand analysis to a broader qualitative and quantitative research process that looks at consumer or client demand for different types of services (willingness to pay, ability to pay, preferences), probes qualitative and other factors driving demand and potential substitutes, obtains feedback on likely responses to potential changes tariffs or in service management (such as privatization), and explores ways in which to more effectively help the poor in terms of price and access based on local institutional context and past experience with programs targeted at the poor. (See also Table on Demand Analysis: Estimating Demand Functions) What can it be used for? Has been used in energy sector reforms and water sector reforms including privatization, but can also be applied to changes in cost recovery in other sectors such as health, education, or transport. What does it tell you? To shed light on how price increases affect different groups of consumers including the poor, specifically taking into account institutional factors that affect the transmission of these prices. Also, Consumer Assessment (CA) helps to project more realistic revenue/cost recovery levels, incorporate client perspectives and levels of satisfaction, and rank the service in question in terms of overall development priorities of different groups of clients. In its application in Africa CA has also outlined the viability of various options for reaching the poor given existing institutional and market constraints, and given their preferences. Complementary tools: • Can be used in conjunction with stakeholder analysis and institutional analysis. Elements of SOCAT can be integrated into CA. Can also complement nationally representative household surveys. • Feedback from CA can inform assumptions on elasticity or welfare impact on different groups in other economic models. In ECA, CA has been used to build standard demand models as well. Key Elements: Requires: (1) quantitative household surveys that include, but are not limited, to willingness and ability to pay, indicators of vulnerability or poverty, income, social capital and/or (2) can use existing LSMS or other household surveys and data from other utility or service providers on types of consumers and consumption or service levels; and (3) traditional focus group discussions, or focus group discussions using a variety of PRA (SARAR) visual aids. In some cases CA has also included (4) key informant interviews and (5) observation to triangulate information obtained from the various sources. In Africa CA has also been integrated into utilities’ financial models to project realistic cost recovery rates and tariffs. Requirements Data/information: Data on sources and services for different groups of consumers, coverage levels, consumption levels and tariffs, over time if available, from either utility data or direct research or existing surveys, and income distribution data by service type or customer grouping (though this is often collected during the research). Most effective as a decision tool if actual and projected costs of service provision under different scenarios are used in willingness to pay questions. Time: For CA generally six to eight months, with field work of two to three months total, though more disaggregated demand analysis (within peri-urban areas of a city) has taken longer. Skills: Requires quantitative skills (economist, social economist, or sector economist) in addition to skills in qualitative research (sociologist, anthropologist). Good knowledge of sector structure is essential. Supporting software: N/A Financial cost: For fieldwork from US$40,000 up to over US$100,000 excluding supervision of consultants. Limitations: Requires skill in triangulating information to provide assessment of client response to changes in tariff levels, and to distinguish potential biases in information provided. Also, effective qualitative work requires skilled facilitators. Willingness to pay questions can raise expectations of service improvements, and need therefore to be carefully linked to sector constraints and likely scenarios. References and applications: • Lampietti et al (2001) on utility pricing in Armenia • Sechaba Consultants (2002) on the water sector 55 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Benefit Incidence Analysis (Average and Marginal) What is it? Benefit incidence analysis estimates the impact of public transfers, taxes, subsidies, or policy changes that affect prices. BIA measures the distributional incidence of benefits for different groups of interest, for instance households at different income levels or in different regions. Average (or simple) BIA measures the incidence of all benefits - i.e. of the aggregate benefit. Marginal BIA estimates the incidence of the last (or the next) unit of benefit. (See also Table on Tax Incidence Analysis) What can it be used for? Benefit incidence analysis is most commonly used to examine the impact of public expenditures and public expenditure reforms. It is also applicable to other policy reforms, including reforms affecting prices that change household income or expenditure and tax reforms. It can be applied to direct transfers as well as to transfers obtained by consuming subsidized goods or services. What does it tell you? Benefit incidence tells us who benefits from services, transfers, or price changes. When estimating the size of benefits received by different groups, average BIA calculates the benefits received on average (i.e. on the basis of average unit costs); marginal BIA tells you who will benefit from a increase or decrease in benefit (i.e. the marginal change). These two might be very different – typically, additional beneficiaries are more likely to belong to groups not yet covered by the system (e.g. remote areas). Complementary tools: Simple or marginal BIA can be combined with information on household or individual behavior – see Tables on Behavioral Benefit Incidence Analysis, Social Impact Analysis and Beneficiary Assessment. These techniques explain distributional changes from a policy reform by taking into account the reactions households or individuals will have to the change. Key Elements: BIA proceeds as follows: (1) estimation of the value of the benefit: typically estimated as the cost of providing the service, transfer or subsidy. This can be quite difficult, with issues related to the inclusion of investment and administrative costs, and the treatment of cost recovery. Estimations are sometimes made at a regional level, to account for cost differences; (2) Identification of the users on the basis of household surveys; (3) Aggregation of users into groups of interest (commonly defined by income levels, region, urban/rural location, poor/non-poor, occupation, ethnicity, etc); (4) Accounting for household spending, in case of out-of-pocket expenditures to access the benefit. In case of financial transfers, the income groups can be defined pre- or post-transfers, which will yield different results. Requirements Data/information: (1) individual or household-level data from household surveys on welfare and on the use of service and receipt of public spending and (2) information on public expenditure to estimate the value of the benefits. For marginal BIA, panel data is ideal, although methods exist for cross-sectional data. Time: Analyzing household survey data can be time consuming, depending on how clean the data are, and how well managed the data entry process was. BIA can take between 4 to 8 weeks depending on the condition of the household survey data, and the accessibility of the unit cost of providing those services (usually obtained from government data). If a survey has to be undertaken first, then the timeframe extends significantly, to between 1 to 2 years. Skills: Good data handling skills, and experience with analyzing large scale household survey data sets. Experience with related statistical software packages (SPSS, SAS, STATA) Supporting software: SPSS, SAS, STATA. Financial cost: Costs of developing and using the tool can vary enormously, depending on whether a household survey already exists. If it does, the analysis can be done for around US$10,000. Limitations: Benefit incidence analysis does not take behavior into account, i.e. the likely change in demand from households that would result from policy changes. For methods which handle this, see Tables on “Ex-post behavioral marginal incidence analysis of public spending and social programs” and “Social Impact Analysis”. References and applications: • For an overview of the technique, see Demery (2003), Chapter 2 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Demery (2000) and van der Walle (1998) on the overall technique. • Castro-Leal, Dayton and Demery (1997) on a group of African countries. • Castro-Leal (1996) on South Africa. • Demery et al. (1995) on Ghana. • Devarajan and Hossain (1998) on benefit and tax incidence analysis in the Philippines. • Van der Walle (1992) and Lanjouw et al (2001) on Indonesia. • Van der Walle (2002c) on incidence of public transfers in Yemen. 56 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Tax Incidence Analysis What is it? Tax incidence analysis evaluates the distributional incidence of taxation – its incidence for various household groups (on the basis of income, geographic location, and other dimensions). The taxes have an effect on real income directly or via prices. (See also Table on Benefit Incidence Analysis). What can it be used for? Tax incidence analysis can be used to analyze the distributional impact of taxes or subsidies. It can also be used to analyze the distributional impact of other exogenous changes in prices, and publicly provided services. What does it tell you? The tool estimates the effect of changes in prices and incomes on the welfare of individuals or households. Most analysis is concerned with the share of taxes paid by different groups. Taxation is understood as a loss in real income. Complementary tools: Tax incidence analysis can be complemented by the analysis of the statutory incidence of taxation (i.e. the analysis of the rules which set who has to pay which taxes) and by the analysis of the functioning of the tax collection processes (see Tables on Institutional Analysis and Quantitative Service Delivery Surveys). As tax incidence analysis, benefit incidence analysis (simple and marginal) assesses the incidence of benefits, and behavioral BIA assesses distributional changes from change in benefits, taking into account reactions to the change. (See Tables on these two techniques). Key Elements: The technique (1) defines the groups of interest, typically in terms of income/consumption, geographic location, gender, ethnicity, age, socio-economic group, etc. and (2) calculates the taxes paid by each household groups. To do so, one needs to recognize that the statutory incidence of taxation (those who have to transfer the tax to the government) is not the same as the economic incidence of taxation (those whose real purchasing power declines because of the tax. The difference results from the fact that different statutory taxes are shifted among agents. Typically, one assumes that indirect taxes on goods are completely shifted to the consumer (i.e. the prices reflect the taxes paid by other categories), and that duty taxes are reflected in prices. Hence, taxation has impacts on various groups of households through the goods, services, transfers and subsidies that they receive. To quantify the tax paid, the technique either (a) estimates the taxes paid as the official tax rate times the pre-tax value of expenditure (if taxes can be assumed to be collected according to the letter of the law) or (b) estimates the “effective” tax rate for different categories of expenditure by dividing the tax base by the actual tax revenues and applies it to these categories. Requirements Data/information: The analysis requires information on tax/subsidy and their changes, and nationally representative household income or expenditure survey data (e.g. LSMS), including information on specific items to be taxed/subsidized. Time: One month, if the data are clean and include a calculated welfare variable (such as household expenditure, consumption or income). Skills: Familiarity with the tax system and market structure of the country. Econometric skills and expertise in the supporting software (see below). Supporting software: Any statistical software package can calculate point estimates easily (Stata, SPSS, etc). For variances, a matrix programming language (Gauss, Matlab, SAS IML) is useful. The software package DAD calculates concentration curves and other summary measures of incidence with standard errors. Financial cost: US$15,000 Limitations: Simple analysis of the incidence of taxes does not account for behavioral changes and hence only provides a first-order approximation of a tax’s true incidence. Furthermore, inaccuracy can come from the simple assumption of how statutory taxes translate into economic incidence. In addition, many indirect taxes are also levied on intermediate goods, and estimating the incidence of the tax on final consumer would require complex models. Finally, the method only focuses on the incidence of taxes and should be complemented by an analysis of the economic and administrative efficiency of the system. References and applications: • For an overview of the technique, see Sahn and Younger (2003), Chapter 1 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Alderman and del Ninno (1999) on the targeting of VAT exemptions in South Africa. • Ahmad and Stern (1984, 1987, 1990 and 1991) on alternative forms of taxation in India and Pakistan. • Chen et al. (2001) on Uganda. • Gibson (1998) on Papua New Guinea and the introduction of VAT. • Younger et al. (1999) on Madagascar. • Younger (1993) on Ghana. 57 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Poverty Mapping What is it? Technique to estimate geographically disaggregated welfare and inequality levels and changes, for small geographic areas, such as districts and sub-districts. This allows to take geographic heterogeneity into account. What can it be used for? The method can inform the targeting of public resources, and can simulating the geographic impact of policy reforms such as changes in trade barriers, decentralization, public expenditure, etc. Information disaggregated for small geographic areas is particularly important in the context of decentralization of public services. What does it tell you? The method provides an estimation of poverty/inequality distribution across a country’s sub-regions. It identifies poverty pockets, by giving satisfactorily precise estimates of poverty at levels of disaggregation far below that allowed by typical household surveys. Poverty and inequality estimates can then be represented on maps. These maps, on which other variables of interest can be applied, help assess the spatial impact of policies. Typically, the poverty maps can also include information on education, water, health, public services, agricultural production, etc. depending on the reform of interest. Complementary tools: A poverty map can be merged with other GIS (Geographic Information Systems) containing information on a variety of public actions. Social Impact Analysis and Participatory Poverty Assessments can help explain the geographic trends revealed in a poverty map. Key Elements: The method uses a household survey and a census. It imputes information on poverty and inequality in the census, using estimates from the household survey, through the construction of consumption-based welfare indicators for small geographic areas. In order to do so, (1) the variables common to the survey and the census are identified, (2) the survey is used to estimate a prediction model relating consumption to the variables which the two data sets have in common, (3) the parameter estimates can be applied to the census data to derive poverty statistics for each household in the census, and (4) poverty and inequality estimates can be calculated for small geographic areas and transposed into a GIS system. Requirements Data/information: A household survey and a population census, ideally from the same year. If different years are used the compatibility of the two sources showed be checked by comparing the estimates with basic poverty/inequality statistics in the sample data. In this case, the welfare estimates refer to the year of the census, whose explanatory variables form the basis of the predicted expenditure distribution Time: Depends on the quality of the survey and census data, minimum of two months; six months on average Skills: Good knowledge of poverty and inequality measurement. Good data handling skills, and experience with analyzing large scale household survey and census data sets. Experience with related statistical software packages (SPSS, SAS, STATA) Supporting software: SPSS, SAS, STATA and GIS software such as ARCView, purpose-written software produced by the World Bank (http://econ.worldbank.org/programs/poverty/topic/14460/). Financial cost: US$20-100,000 depending on level of specialized consultant, availability of counterpart contributions in terms of computational assistance, etc Limitations: Household variables do not capture some unobserved geographic effects (such as climate, quality of local administration etc). Hence, it may be desirable to complement the analysis using such additional data. Also, when using the technique to simulate the impact of reforms, behavioral changes are typically ignored. References and applications: • For an overview of the technique, see Lanjouw (2003), Chapter 4 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Elbers, Lanjouw and Lanjouw (2002) on the overall approach. • For purpose-written software and manual, as well as other country applications, see http://econ.worldbank.org/programs/poverty/topic/14460/. • Demombynes et al. (2002) on poverty in Ecuador, Madagascar and South Africa. • Elbers, Lanjouw, Mistiaen, Ozler and Simler (2002) on inequality in Ecuador, Madagascar and Mozambique. • Elbers, Lanjouw, Lanjouw and Leite (2002) on Brazil. • Mistiaen (2002) on the analysis of the impact of rice price changes in Madagascar. • Mistiaen et al. (2002) on health spending in Madagascar. 58 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Ex-post behavioral marginal incidence analysis of public spending and programs What is it? This type of analysis combines the analysis of the marginal incidence of benefits with the econometric modeling of household behavior. The analysis is ex-post, since it focuses on past interventions, drawing lessons for future ones. The methods allow to take the behavior of both the recipients of public spending or participants in the programs, and of the agents implementing them. Finally, the analysis is marginal since it focuses on the impact of increases or cuts in public spending and programs. What can it be used for? It can be used to explain distributional impacts of public finance or policy reform on individuals and households, allowing for behavioral responses. This applies to public spending and programs on education, health, and cash transfer programs. It can also be used in the analysis of other reforms, including land reform, pension reform, and micro-finance programs. What does it tell you? The analysis allows to estimate the distributional impacts of changes in public spending or programs, taking the behavioral responses or beneficiaries and implementing agencies into account. By examining actual change ex-post, these methods can also provide a reality check for the results of methods that attempt to approximate or predict changes ex-ante. Complementary tools: Ex-post Social Impact Analysis can complement these efforts, as can adaptations of tools such as the Quantitative Service Delivery Survey and Public Expenditure Tracking Surveys that use historical data (see Tables on these tools and techniques). Key Elements: The technique entails the econometric analysis of household data on welfare indicators and on receipt of the benefit under consideration and the modeling of household responses, such as changes in labor supply. Requirements Data/information: Behavioral marginal incidence can be done using: 1) single household survey cross-section with sufficient regional disaggregation and variance in participation; 2) two or more comparable household cross-sections; 3) Household level panel data, or 4) geographic level panel data for dynamic marginal incidence Time: A few weeks to a few months depending on the quality of the data. Skills: Econometric skills. Supporting software: EXCEL and STATA (or other micro-econometric and spreadsheet software) Financial cost: Costs of developing and using the tool can vary, depending on whether household surveys exist already. If they do, the analysis can be done for around US$10,000 Limitations: Behavioral benefit incidence analysis typically has more onerous data requirements than simple benefit incidence analysis to allow for behavioral modeling. References and applications: • For an overview of the technique, see van de Walle (2003), Chapter 3 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Lanjouw & Ravallion (1999) • van de Walle (1994) on Indonesia. • van de Walle (2002a) on rural roads. • van de Walle (2002b) on Viet Nam. • Ravallion (1999) 59 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Ex-ante behavioral marginal evaluation of policy reforms What is it? The techniques allow to estimate the situation that would result from changes in policies. The techniques allow for the analysis ex-ante, i.e. before a reform is implemented, of the distributional impacts of the reform. This analysis is marginal, because it aims at capturing changes from the existing situation (e.g. new policy, expansion, reduction of existing public actions). The analysis is also behavioral since the behaviors of various stakeholders are taken into account when defining the counterfactuals. What can it be used for? This type of analysis can be applied to types of transfer programs with expected impact on some dimension of household behavior (e.g. occupational choices, schooling, demand for various goods or services, etc.). This includes, among others, changes in taxes, expenditure, and targeted programs. It can also be used for any exogenous change in the environment of a household likely to trigger a non-negligible behavioral response (e.g. accessibility of various types of services, conditions on the labor market, producer and consumer prices). What does it tell you? It tells you what would be the likely distributional impacts of policies changes, taking the behaviors of various stakeholders into account. Complementary tools: Tools such as Stakeholder Analysis, Social Impact Analysis, and – in some cases – the Social Capital Assessment Tool can help analysts better understand the variables that are most likely to affect household behavior. Key Elements: The technique proceeds as follows: (1) identification of the policy reform to be analyzed; (2) identification of data set and information sources which contains the necessary information; (3) specification of the economic model which captures the mechanisms likely to affect the individual or household’s responses to the policy; (4) estimation of the model; (5) and simulation of the policy reform using the empirical estimate of the model. Requirements Data/information: Household surveys (+ specific surveys or questions depending on the issue of interest) Time: 6 months with experienced microeconomic modeler Skills: Micro-econometric modeling Supporting software: All software used in micro-econometrics - Stata, SAS, etc. Financial cost: Depends on the question being asked and the need for new data. If data is available, the cost can vary from US$10,000 to US$30,000. Limitations: The estimation of behavioral models that fit the policy to be evaluated or designed can be difficult, but can rely on simpler assumptions (accounting micro-simulation). Second, the approach relies on a structural model, which requires a set of assumptions. References and applications: • For an overview of the technique, see Bourguignon and Ferriera (2003), Chapter 6 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Atkinson and Bourguignon (1991) on tax-benefit models. • Attanasio, Meghir and Santiago (2002) on education choices in Mexico. • Bourguignon, Ferreira and Leite (2002) on conditional cash transfers in education in Brazil. • Blundell et al (2000) on tax credit in the U.K. • Younger (2002) on marginal benefit incidence and education in Peru. 60 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Ex-post impact evaluation methods for assigned programs What is it? Methods for assessing the counter-factual to evaluate the poverty impact of assigned programs What can it be used for? Any policy, program or shock that are assigned to some observational units but not others, and the units not assigned are largely unaffected. The units might be people, households, firms, communities, provinces or even countries. What does it tell you? It measures the impact, typically defined as difference between the value of the outcome with the program and its value under the counter-factual (what would have been the value of the indicator in the absence of the program). Complementary tools: The best evaluations often combine multiple methods: randomizing some aspects and using econometric methods to deal with the non-random elements, or by combining matching methods with longitudinal observations to try to eliminate matching errors with imperfect data. Complementary tools include Benefit Incidence Analysis, Social Impact Assessment and Demand Analysis, which can help policymakers track the impact of historical policy changes by combining household survey data with financial or service-provision data . Key Elements: The identification strategy establishes the assumptions under which observed outcomes for participants and non-participants can be used (often in combination with other data) to infer impact. If the program is randomly assigned across the population (every has the same chance, ex-ante, of being in the program) then the observed ex-post differences in outcomes are attributable to the program. This is not often the case, however, since programs tend to be purposively targeted to certain groups. In such cases, methods exist to estimate counterfactuals. Examples include propensity-score matching and “difference-in-difference” methods. Requirements Data/information: Data on relevant outcome indicators for those units who participate versus those who do not. Survey or census data covering participants and non-participants are essential. The data must include relevant outcome indicators and (depending on the identification strategy) other relevant covariates for either participation or outcomes. Time: Evaluation design should ideally begin even before the policy/program begins; it is often hard to do a good evaluation if one starts late. “Off-the-shelf” data are sometimes feasible, but it is more often the case that special-purpose data collection is needed and this needs advance planning. Skills: Sufficient knowledge of statistics/econometrics and quantitative data skills. Knowledge of microeconomics often helps. Good knowledge of the program and its setting is important. Supporting software: Standard statistical/econometric packages such as STATA are often sufficient. A number of special-purpose STATA routines are available for evaluation Financial cost: Varies enormously, mainly depending on current data availability. The marginal cost of the evaluation can be low in “data rich” settings and high in “data poor” settings where a lot of primary data collection is called for. Even in data rich settings, supplementary data collection is often required. Limitations: References and applications: • For an overview of the technique, see Ravallion (2003), Chapter 5 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Galasso et al. (2001) and Angrist et al. (2001) on randomized programs. • Van de Walle (2002a), Jalan and Ravallion (2003a and 2003b) on propensity-score matching. • Ravallion et al. (2001) on double-differences techniques. 61 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Demand Analysis: Estimating demand functions What is it? Partial equilibrium model that focuses on the level of demand for the commodities an individual, household or producer demands given the structure of relative prices faced, real income, and a set of individual characteristics. (See also Table on Demand Analysis: Consumer Assessment) What can it be used for? Can be used with a broad range of reforms for which the knowledge of consumer behavior is important. This simple technique, which focuses on a single good can be particularly useful for the analysis of changes in prices in which the good or service in question has few, if any, substitutes. This can include changes in tariffs, subsidies, and other prices. What does it tell you? How changes in income or in the price of a given good affect the demand of a particular group of consumers or producers. Complementary tools: Can be used in conjunction with stakeholder analysis. The analysis of a complete demand system is often used as the basis for more complex multi-market and computable general equilibrium models (see Tables on these two techniques). The most common complete demand systems are: Linear Expenditure System (LES); the Almost Ideal Demand System (AIDS) and the Generalized Almost Ideal Demand System (GAIDS) Demand analysis is also used to build household models, in combination with supply analysis. Key Elements: Methodologically there are two main approaches to estimate the parameters of a demand equation. One consists of specifying estimable single equation demand functions in a pragmatic fashion without recourse to economic theory, using reduced-form estimation. Alternatively one may wish to use the theory of demand to derive an estimable structural model which should provide guidance for the choice of variables to be included, functional forms, and restrictions on the parameters. This model, although usually difficult to estimate due to its typical high nonlinear nature, provides straightforward interpretations of the transmission channels. When demand analysis is used for complete models (see for instance Table on multi-market analysis or CGEs), complete systems of demand equations must be specified and estimated, which are able to take into account the mutual interdependence of large numbers of commodities in the choices made by consumers. Requirements Data/information: Requires household level consumption and income data, with sufficient variation in prices, either geographically or preferably over time. Time: 1 to 3 months. Skills: For reduced-form models, basic econometric skills may suffice. For structural models, advanced econometric and typically programming skills. Supporting software: Software for the analysis of household level data. Financial cost: US$5,000 for simple reduced form models; US$35,000 for detailed of especially complex models Limitations: Reduced form demand equations are simple to estimate, but the results depend on the choice of functional form and variables included in the equation. It also requires constancy in elasticities over all values of exogenous variables. Structural models, on the other hand, are affected by the theoretical underpinnings used to derive the estimable model, and can be extremely complex to estimate. References and applications: For the estimation of demand systems: • Stone (1954) on the Linear Expenditure System, • Deaton and Muellbauer (1986) on the Almost Ideal Demand System • Christensen et al. (1975) on the Transcendental Logarithmic System 62 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Supply analysis What is it? System of input and output equations used to determine supply responses to changing circumstances by producers (including households). Supply analysis takes into account changes in both output supply and input/factor demand. What can it be used for? Supply analysis can be used to determine the impact of changes in product and factor prices, in technology, and in access on factor demands (including labor), production, marketed output, aggregate supply, and incomes. For instance, it could be used to estimate the change in agricultural household production that could result from the liberalization of some markets (inputs, credit, outputs...). More generally, can be used to analyze the impact on production of the removal of barriers to access or other changes in markets. What does it tell you? Supply analysis is central to policy decisions in that it helps us understand the impact that alternative policy packages may have on the producers themselves. Through the changes it induces in commodity supply and in factor demand, the analysis of production response is an essential component of models that seek to explain market prices, wages and employment, external trade and government fiscal revenues. Complementary tools: Supply analysis can be combined with demand analysis to build household models. Institutional analysis and stakeholder analysis can help inform assumptions about constraints to changes in supplier behavior and the incentive structures within a market. PPA/BA techniques help understand inter-household relationships and how households are likely to respond. Key Elements: In studying supply response, it is important to distinguish between specific goods and broad sector aggregates, and between short-run and long-run responses. The basic theory of production is based on profit maximization with respect to output and input quantities. Maximization techniques will yield a set of input demand and output supply functions that are then used to solve for quantities. The impact of price changes on producers can be estimated for a single commodity, or for a system of commodities in the case where the firms/households produce multiple outputs. It is also important to distinguish between short-run and long-run outcomes. It is usually assumed that certain productive factors are fixed in the short run. In agriculture, for instance, the amount of land and the level of technology do not change within a cropping cycle. Labor, too, may be relatively slow to adjust. For this reason, it can be argued that the supply elasticity of agriculture is close to zero in the short run. In the medium- and long-term, fixed investments in productive technology come on-line, and supply can increase. Requirements Data/information: In the case of producing households, this requires household-level production data. For firm-level analysis, firm survey data is needed. Time: Between 1 and 3 months if the data is available Skills: Advance econometric techniques Supporting software: Advanced econometric software, such as SAS, STATA, etc Financial cost: US$10,000 to US$30,000 Limitations: Despite its different focus on short-run and long-run effects, supply analysis is a static tool. In addition, at the firm level many decisions are based on expectations that are difficult to model. References and applications: • López et al. (1995) on Mexico. 63 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Household Models What is it? Micro-econometric models that integrate producer, consumer and worker decisions into a household problem. These models reflect the fact that many households, especially in rural areas, are simultaneously units of production and consumption. What can it be used for? In the context of farm households, when markets are perfect the only link between production and consumption decisions is through the level of farm income from production. If there are market imperfections, policies that affect the prices of goods (factors) both produced (used) and consumed (sold) thus have complex implications for production and welfare. These models have been used to examine a wide range of policy reforms, such as price and marketing changes, market failures, and taxation. In addition, separate models can be constructed for different groups to examine structural differences in the impact policies have on these different groups. What does it tell you? The models allow to estimate the response of households to changes, in terms of their consumption, production, labor supply, and more generally any allocation of resources within the household. Complementary tools: • When production (labor) exceeds consumption (production needs), the household will be a net supplier of products (labor), and vice versa. In those circumstance, demand and supply analysis can be a complement to household models. • Also, if there are no market failures the household behaves as if production and consumption decisions were taken sequentially, in which case theory of production (i.e. supply analysis) and consumption (i.e. demand analysis) applies. • Social impact analysis and beneficiary assessment, which looks at household-level determinants of behavior, can provide information on household preferences and likely switching behavior in the event of a reform. Key Elements: If the household model is separable (i.e. production and consumption decisions can be assumed to be taken sequentially), the problem can be divided into two parts (demand and supply). If the model is not separable, the estimation of production and consumption must be done simultaneously. One possibility is proceeding with a reduced form approach. A second possibility is the calibration and simulation of a structural household model. Requirements Data/information: These models require integrated household surveys. Information is needed both on the demand side and the supply side. Ideally, the models would also account for the allocation of time within the household, which requires data on factors that do not usually appear on consumption or production surveys, such as allocation of time to child care, or other unremunerated work (e.g. time spent fetching water). Time: If an integrated household survey exists, a few months Skills: Advanced experience with household surveys and econometric skills. Supporting software: Statistical packages for the analysis of household data, including Stata, SPSS, and other software. Financial cost: US$10,000 - US$30,0000 Limitations: References and applications: • For an extensive review of these models see Sadoulet and de Janvry (1995). • Singh, Squire and Strauss (1986) on impact of price changes. • De Janvry et al. (1991) on household models for agricultural households. 64 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Partial Equilibrium Analysis: Multi-Market Models What is it? Multi-market models belong to the class of partial equilibrium models. They use partial equilibrium analysis of the impact of changes in prices and quantities in selected markets on household income and expenditure. They specify a system of demand and supply relationships for a few sectors of the economy, so that the impact of policies on one sector can be seen on other sectors in the economy. What can it be used for? Multi-market models are useful to analyze the poverty and distributional impact of policies that affect the prices and quantities of a small group of commodities. For example they can be used in estimating distributional impacts of the imposition or change in taxes, subsidies, quotas, tariffs on specific commodities; rise or fall in the price of imported or exported commodity. What does it tell you? Multi-market models allow the estimation of the impact of a policy or external shock on prices and output in a few sectors, and on household well-being. Complementary tools: • Stakeholder analysis can be useful to identify different groups of interest. • Multi-market models are not general equilibrium models, since they are not necessarily fully specified. If the policy reform is likely to have general equilibrium impacts, the analysis should be complemented by a CGE model. Key Elements: A multi-market model expands the traditional benefit incidence analysis to capture the induced substitution effects across selected goods in response to policy reform. The first step with this approach would be the identification of the market or markets where the policy reform under analysis is expected to have a direct effect. Household survey information would then be used to derive estimates of income, own-price and cross price elasticities of demand for the entire set of interlinked markets. Market closure (either price or quantity clearing) is imposed for each good in the system of equations. In short, multi-market models involve a system of equations, representing actors (producers, consumers, government), production or profit functions, factor and product markets, income accruing to the owners of productive inputs (including labor), and final consumption. Requirements Data/information: Multi-market models require (1) a disaggregated set of data on income or consumption distribution across households, (2) a complete parameterization for supply and demand functions in the market(s) directly affected by the policy reform, (3) a determination of the closures of the market(s) being modeled, (4) software to solve a system of potentially non-linear equations for the endogenous prices and quantities, and (5) a quantitative mapping of these endogenous variables into the income and consumption of households. Time: The required time to perform an analysis based on partial equilibrium models depends to a large extent on data availability and degree of sophistication of the econometric model. It could vary from about one week for a simple model to three months for very detailed models Skills: Familiarity with basic partial equilibrium modeling and micro-econometric estimation techniques Supporting software: Stata, SAS, GAMS Financial cost: US$5,000 for simple models; US$25,000 for detailed or especially complex models Limitations: These models are limited to selected markets, and hence ignore other interlinked markets by design. It is also prudent for the analysis to conduct sensitivity analysis of the results for different values of the parameters used in the model. References and applications: • For an overview of the technique, see Arulpragasam and Conway (2003), Chapter 12 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Binswanger and Quizon (1984, 1986) on agriculture in India. • Dorosh, del Ninno and Sahn (1995) on food aid in Mozambique. • Minot and Goletti (1998) on rice refom in Vietnam. 65 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Partial Equilibrium Analysis: reduced-form estimation What is it? Partial equilibrium model focusing on the effects of policy changes (including interest rate changes, taxes, etc.) or exogenous shocks (a change in import tariffs in another country, or a terms of trade shock) on a variable of interest, such as aggregate consumption or income. What can it be used for? Analysis of partial equilibrium on the basis of reduced-form estimation is one of the most common applications of econometric analysis, and can be used to examine a myriad of different outcomes. It can be applied to most policy changes or exogenous shocks. It is most useful for PSIA of policy reforms which have significant indirect effects. For example, simple tax incidence analysis (see Table on this method) can analyze the direct distributional impacts of tax changes, but does not capture the impact of tax changes on the overall economy and growth, thereby only providing a partial answer to the question of impact. Partial equilibrium analysis with reduced-form estimation can capture this indirect impact and provide a first approximation of the expected impact on aggregate incomes. What does it tell you? It can provide a good estimation of the impact that changes in a given policy or exogenous shocks have on a particular variable of interest. Once a model containing the policy variable and the variable of interest has been estimated, reduced-form models can be used to simulate the impact of policy alternatives. Complementary tools: Reduced-form estimation can be useful to understand the macroeconomic impact of a policy intervention on a selected variable of interest. There is often a need to complement the analysis by the use of household surveys to map these impacts into distributional changes. Stakeholder analysis can be useful to identify different groups of interest for the analysis. Key Elements: Reduced-form estimation assumes an underlying system of demand and supply equations but the model itself does not fully specify the whole array of economic and social interactions. Rather, the model is solved to derive a single estimating equation: an econometric model that relates the outcome and the policy variables or shock of interest. This can be done on the basis of two observations separated over time by a policy change. When using a single cross-sectional dataset, there must be significant variation across the sample population to estimate the equation. Analysis on aggregate units, such as cross-country regressions, should ideally be conducted on panels of cross-sectional and time-series data. Requirements Data/information: Reduced-form models require information on the variable of interest, and its hypothesized determinants. The specific data requirements depend on the unit of analysis, from household or individual level to country level. Time: The required time to perform analysis based on partial equilibrium model and reduced-form estimation depends to a large extent on the data availability and the degree of sophistication of the econometric model. It could vary from a week for a simple model to three months for very detailed models. Skills: Econometric skills Supporting software: Eviews, STATA, Gauss, etc. Financial cost: US$5,000 for simple models, US$25,000 for detailed, complex models. Limitations: The elasticities of the policy variable to the variable of interest are often based on cross-country regressions, and may differ from the elasticity in the country itself. References and applications: • Barro (1997) • Quah and Durlauf (1999) • Dollar and Kraay (2002) 66 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Social Accounting Matrices What is it? A social accounting matrix (SAM) is a technique related to national income accounting, providing a conceptual basis for examining growth and distributional issues within a single analytical framework. It can be seen as a tool for the organization of information in a single matrix of the interaction between production, income, consumption and capital accumulation. What can it be used for? SAMs can be used for some simple policy simulations What does it tell you? SAMs can be applied to the analysis of the interrelationships between structural features of an economy and the distribution of income and expenditure among household groups. Complementary tools: SAMS would complement and be complemented by the use of household surveys to map impacts into distributional changes. Stakeholder analysis can be useful to identify different groups of interest. Key Elements: A typical SAM contains entries for productive activities, commodities, factors, institutions, the capital account, and the “rest of the world.” An activity produces (and receives income from) commodities, buys commodities as production inputs, pays wages to labor, rents to capital, and taxes to the government. Factor income accrues to households as owners of the factors. The SAM can be constructed to distinguish household groups by, for example, sources of income. SAM techniques select some accounts as exogenous, and leave the remaining accounts endogenous. In part, this selection can be made with a sound theoretical basis, but it is often arbitrary. For example, if the SAM contains an account for agricultural production and one for transportation, an experiment can be run by imposing some exogenous change (a “shock”) to agriculture while leaving the transport sector fixed, or while allowing the transport sector to adjust endogenously as a result of the shock Requirements Data/information: The data sources for a SAM come from input-output tables, national income statistics, and a household survey with a labor module. Time: About three months for a moderately detailed SAM. Skills: Working with household datasets; strong knowledge of national accounts; use of Excel and maybe GAMS (for using dedicated software) Supporting software: Excel and GAMS-based dedicated software; STATA, SAS or SPSS for working with household datasets Financial cost: US$25,000 when the data is available. This does not include the cost of developing a new household survey. Limitations: SAM models have at least two major drawbacks. First, prices are fixed, and do not adjust to reflect changes in, say, real activity. As a result, supply is either perfectly elastic (if chosen to be endogenous) and entirely demand driven, or perfectly inelastic – that is, supply is constant. Second, the results of the simulations vary greatly depending on the assumptions made about which accounts are exogenous and which endogenous. References and applications: • For an overview of the technique, see Round (2003), Chapter 14 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Pyatt and Round (1985). • Powell and Round (2000). • Reinert and Roland-Holst (1997). • Sadoulet and de Janry (1995). • Tarp, Roland-Holst and Rand (2002). 67 A User’s Guide to Poverty and Social Impact Analysis Tool Name: CGE models What is it? CGE models are completely-specified models of an economy, or a region, including all production activities, factors and institutions. The models therefore include the modeling of all markets (in which agents’ decisions are price responsive and markets reconcile supply and demand decisions) and macroeconomic components, such as investment and savings, balance of payments, and government budget. What can it be used for? CGEs can be used to analyze the poverty and social impacts of a wide range of policies, including exogenous shocks (exchange rate, international prices, etc.), changes in taxation, subsidies and public expenditure (including changes in trade policies), and changes in the domestic economic and social structure (including technological changes, asset redistribution, human capital formation). What does it tell you? CGE models are best chosen for policy analysis when the socioeconomic structure, prices, and macroeconomic phenomena all prove important for the analysis. CGEs allow to take into account all the sectors of the economy as well as the macro-economy, and hence permit the explicit examination of both direct and indirect consequences of policies. This is particularly important for those policy reforms that are likely to play a large role in the economy and might have important impacts on other sectors and/or on the flow of foreign exchange or capital. Complementary tools: Other tools described here belong to this class of models, with an additional model to take distribution into account: the 1-2-3 PRSP, IMMPA and the Augmented CGE Model with Representative Household Approach. See their respective Tables. Key Elements: A CGE can be described by specifying the agents and their behavior, the rules that bring the different markets in equilibrium, and the macroeconomic characteristics. CGEs are based on SAMs (see Table on Social Accounting Matrices), and can be distinguished by the complexity and level of disaggregation of productive activities, factors and institutions, including households. Requirements Data/information: CGE models are data-intensive. They are constructed from combined national accounts and survey data. These are first compiled into a SAM, which is then used as the foundation of the CGE. Time: A few months to a year, depending on the existence of a SAM, or of another CGE model built to address a different question. Even these simple CGEs can be complex and time consuming. An alternative is to use a previously constructed CGE. For example, Ianchovichina et al. (2001) use a CGE model constructed by the Global Trade and Analysis Project (GTAP) to examine the impact of NAFTA on household welfare in Mexico. However, the use of a previously constructed simple CGE can limit the number of policy changes that can be simulated (in the previous example, the model was constructed to examine trade policy, and did not contained domestic taxes or public expenditure). Skills: Experienced modelers with substantial prior exposure to Computable General Equilibrium models are required Supporting software: Excel, Eviews, Gauss Financial cost: US$25-75,000 depending on existing data Limitations: The results of CGE simulations depend at least partly on the assumptions made in the model, such as the “closure” rules. These ensure that macroeconomic accounts (fiscal, trade, savings-investment) balance. Whether they are fixed exogenously or allowed to balance endogenously, and how they balance, can have a significant impact on the outcomes. In addition, the production accounts specified in most available CGEs are too aggregated to identify the impact of policy changes in one component of one account. Many CGEs have at most two agricultural activities, one each for tradable and non-tradable crops, or food crops and cash crops. References and applications: • Dervis et al. (1982) and Shoven and Whalley (1992) for summaries of CGE models use. • Ianchovichina, Nicita and Soloaga (2001). GTAP models at http://www.gtap.agecon.purdue.edu. 68 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: PovStat What is it? An Excel based software program which simulates the changes in poverty and inequality over time resulting from changes in output and employment growth. What can it be used for? PovStat may be used to simulate the poverty and inequality impact of policies affecting sector-level output and employment growth rates. What does it tell you? PovStat simulates poverty and inequality measures under alternative growth scenarios. Forecasts of varying levels of complexity may be computed depending on the availability of reliable data and the extent to which factors influencing poverty levels are incorporated. The simulations vary according to optional projection parameters. Complementary tools: Other software programs that provide poverty and inequality forecasts include SimSIP Poverty (see Table on SimSIP), and DAD (a software for distributive analysis). Social impact analysis and institutional analysis could complement this analysis by identifying constraints to market participation by certain groups which can affect poverty and inequality estimates. Key Elements: On the basis of household-level data, the software translates differential output and employment growth across sectors into differential growth in per capita income or consumption of households across those sectors. The tool simulates the impact of policies affecting output on poverty using the fact that poverty changes can be decomposed into two parts: a component related to the uniform growth of income, and a component due to changes in relative income. The simulations are made under the assumption either that the policy analyzed will be distribution neutral, or conversely assuming a specific quantifiable form for the distributional change. Changes in occupational distribution are accommodated through reweighing of sample households. Requirements Data/information: This program requires unit record household survey data. Also, a poverty line, survey year, and forecast horizon are parameters that must be provided by the user. Macroeconomic variables at the nationally aggregated or sectorally disaggregated level and growth rates of income, employment and population are also required. In addition, the user can input change in CPI and GDP deflator, change in relative price of food and shares of food in CPI, and changes in poverty line consumption bundle. This allows to generate different types of fore casts optional projection parameters such as employment shifts across sectors. The software can also be adapted for grouped data. Time: 1-2 days to format the household survey data, collate and check exogenous economic variables and enter everything into PovStat. Skills: Familiarity with Excel and appropriate household data handling software (such as Stata). Also, with PovCal if synthetic data from a grouped distribution are to be used Supporting software: Excel Financial cost: Limitations: PovStat does not capture second round effects. These may be captured by CGE models. References and applications: • For an overview of the technique, see Datt, Ramadas, van der Mensrugghe, Walker and Wodon (2003), Chapter 10 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Datt and Walker (2002). • Software available at http://www.worldbank.org/psia, section on Tools and Methods. 69 A User’s Guide to Poverty and Social Impact Analysis Tool Name: SimSIP Poverty What is it? SimSIP Poverty is a generic Excel based simulator, which allows to estimate the changes in poverty and inequality over time resulting from changes in output and employment growth. What can it be used for? This tool may be used to simulate the poverty and inequality impact of policies affecting sector-level output and employment growth. What does it tell you? It simulates poverty and inequality measures nationally and within sectors (urban and rural; agriculture, manufacturing and services). It may simulate the impact of various sectoral patterns of growth and population shifts between sectors on future poverty and inequality. Complementary tools: Other tools for poverty forecasts include PovStat (see Table on PovStat), and DAD (a software for distributive analysis) Social impact analysis and institutional analysis could complement this analysis by identifying constraints to market participation by certain groups which can affect poverty and inequality estimates. Key Elements: On the basis of existing information on group level household survey data (typically by deciles or quintiles), the software translates differential output and employment growth across sectors into differential growth in per capita income or consumption of households across those sectors. The tool simulates the impact of policies affecting output on poverty using the fact that poverty changes can be decomposed into two parts: a component related to the uniform growth of income, and a component due to changes in relative income. The simulations are made under the assumption either that the policy analyzed will be distribution neutral, or conversely assuming a specific quantifiable form for the distributional change. Changes in occupational distribution are accommodated through reweighing of sample households. Requirements Data/information: SimSIP Poverty uses grouped household data, typically groups by income: the mean income or consumption by group and the share of these groups are required. In addition, SimSIP Poverty requires macroeconomic data at a nationally aggregated or sectorally disaggregated level. This includes for example past or expected growth rates of output, employment and population by sector. Finally, the population size and growth, and a poverty line are necessary for calculating poverty incidence. Time: 1 day to gather the data on population shares and mean income/consumption by group, check the realism of scenarios, and enter the data into the software. Skills: Familiarity with Excel Supporting software: Excel Financial cost: Limitations: SimSIP does not capture second round effects. These may be captured by CGE models. References and applications: • For an overview of the technique, see Datt, Ramadas, van der Mensrugghe, Walker and Wodon (2003), Chapter 10 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Wodon et al. (2003). • Ramadas et al. (2002). • Software available at www.worldbank.org/simsip. 70 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: 123 PRSP What is it? 123PRSP (one country, two sector, and three goods) is a static computable general equilibrium (CGE) model. What can it be used for? 123PRSP can be used to analyze the impact of macroeconomic policy and external shocks on income distribution, employment and poverty. What does it tell you? It allows for a forecast of welfare measures and poverty outcomes consistent with a set of macroeconomic policies in the context of a very simple general equilibrium model. For a given set of macroeconomic policies, 123PRSP generates a set of wages, sector specific profits and relative prices that are mutually consistent. The projected changes in prices, wages and profits are then inputted into household data on wages, profits and commodity demand for representative groups, or segments of the distribution. In principle, 123PRSP can calculate the policy impact on each household in the sample so as to capture the effect on the entire distribution of income. For a given poverty line, 123PRSP can also compute the effect of different poverty measures. Complementary tools: Analysis of impacts on income distribution could be complemented by social impact analysis and institutional analysis, which look at variables that would affect household participation in growth. Scenario analysis, which helps policymakers assess the effects of major discontinuities on economic projections, could complement CGE models operating on a long time horizon. Key Elements: 123PRSP can be viewed as a middle ground between consistency models (such as RMSM-X), and more sophisticated approaches such us disaggregated computable general equilibrium models. The former are simple to estimate and use, but take the two most important determinants of poverty -economic growth and relative prices- as exogenous. The latter\ are useful to capture the poverty impacts of policies but are too data-intensive and difficult to master. One salient feature of 123PRSP is its modular approach; by linking several existing models together it can make use of individual modules which already exist. Further, if for data or other reasons a particular module is not available the rest of the framework can be implemented without it. Requirements Data/information: The 123PRSP model requires national accounts, a social accounting matrix (SAM), and some basic distributional data or a household survey. The model builds on an existing static aggregate model, such as the IMF’s Financial Programming Model (containing a consistent set of national accounts which are linked with fiscal balance of payments and monetary accounts). Macroeconomic policies are then fed into the “Get Real Module” or an alternative country specific model of long-run growth determination and into a trivariate VAR module of short run fluctuations. This trivariate module would require historical national account data. Both long-run and short-run projections would then feed into the 123 model to generate projections on changes in wages, profits and the prices of the three goods, which in turn are fed in a household data module to capture the effects of macroeconomic policies on poverty. Time: About three months if a household survey and the macro model are available Skills: Experienced modelers with expertise in financial programming and advanced time series econometrics. Supporting software: Eviews, Excel Financial cost: Without the cost of developing the macro model or the Household survey, about US$25,000 to set a new model. Limitations: As noted above, 123PRSP adopts several strategic simplifications in order to make the model user friendly. The cost of adopting this approach is that the causal chain from macroeconomic policies to poverty is in one direction only. The model in this regard does not capture the feedback effect of changes in the composition of demand (due to shifts in the distribution of income) on macroeconomic balances. References and applications: • For an overview of the technique, see Devarajan and Go (2003), Chapter 13 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. . 71 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Poverty Analysis Macroeconomic Simulator (PAMS) What is it? PAMS is an econometric model that links a macro-consistency model or macroeconomic framework to a labor-poverty module. What can it be used for? PAMS can be used to address the impact of macroeconomic policies and exogenous shocks (such as an exogenous rise or fall in output growth, or a change in the sectoral composition of output) on individual households. What does it tell you? PAMS can produce historical or counterfactual simulations of: (i) alternative growth scenarios with different assumptions for inflation, fiscal, and current account balances. These simulations allow for testing tradeoffs within a macro stabilization program. (ii) Different combinations of sectoral growth (agricultural or industrial, tradable or nontradable goods sectors) within a given aggregate GDP growth rate (iii) tax and budgetary transfer policies Complementary tools: Stakeholder analysis can be useful to identify groups to inform the selection process of micro categories. Social impact analysis and institutional analysis could help analysts identify constraints to market participation by certain groups which would affect poverty and inequality estimates. Key Elements: PAMS has three main components: (i) a standard aggregate macro-framework that can be taken from any macro-consistency model (for example, RMSM-X, 123) to project GDP, national accounts, the national budget, the balance of payments, price levels, etc. in aggregate consistent accounts; (ii) a labor market model breaking down labor categories by skill level and economic sectors whose production total is consistent with that of the macro framework. Individuals from the household surveys are grouped in representative groups of households defined by the labor category of the head of the household. For each labor category, labor demand depends on sectoral output and real wages. Wage income levels by economic sector and labor category can thus be determined. In addition, different income tax rates and different levels of budgetary transfers across labor categories can be added to wage income; and (iii) a model that uses the labor model results for each labor category to simulate the income growth for each individual inside its own group, assumed to be the average of its group. After projecting individual incomes, PAMS calculates the incidence of poverty and the inter-group inequality Requirements Data/information: The model requires national accounts (with a breakdown by sector) and household survey data with income/expenditure data by unit, and a wage and employment breakdown by sectors Time: With a macro model the time needed to build a PAMS would be about three months: (i) One month to select/extract categories of households from the household survey and match the economic sectors from the macro model. (ii) One month to link the macro model to the household survey data, and (iii) One month to run the macro and household module together and adjust. Skills: Knowledge of (i) National Accounts based macroeconomic models, (ii) of basic labor demand models and (iii) of the structure of household surveys is required Supporting software: Eviews, Excel Financial cost: US$25,000 when the data is available. This does not include the cost of developing a macro model or a new household survey Limitations: The main limitation is the lack of feedback of the micro model into the macro model. References and applications: • For an overview, see Pereira da Silva, Essama-Nssah and Samake (2003), Chapter 11 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Pereira da Silva, Essama-Nssah and Samake (2002). 72 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Integrated macroeconomic model for poverty analysis (IMMPA) What is it? IMMPA is a dynamic computable general equilibrium (CGE) model. What can it be used for? IMMPA can be used to analyze the impact of macroeconomic policy and external shocks on income distribution, employment, and poverty. What does it tell you? One of the main features of IMMPA is that it integrates the real and financial side of the economy; in this regard, IMMPA is useful to analyze both the impact of structural reforms (such as changes in tariffs or the composition of public expenditure) and the effects of short term stabilization policies (such as a cut in domestic credit or a rise in deposit interest rates). The detailed treatment of the labor market is key for the assessment of the poverty reduction impact of macroeconomic policies. Also it is useful to make the distinction between rural and urban sectors by completing separate projections on output and employment fluctuations for both areas, and therefore to study poverty in different geographical areas Complementary tools: IMMPA would complement and be complemented by the use of household surveys to map impacts into distributional changes. Stakeholder analysis can be useful to identify different groups of interest. Social impact analysis and institutional analysis could help analysts identify constraints to market participation by certain groups which would affect poverty and inequality estimates. Key Elements: The main distinguishing features of IMMPA from other CGE models are the following. First, IMMPA has a very detailed specification of the labor market, which is the main transmission channel of macroeconomic shocks and adjustment policies to economic activity, employment and relative prices. The labor market specification allows for a disaggregation at the urban and rural levels and in turn, for each of these areas, in the formal and informal sectors. Second, IMMPA links real and financial sectors through an explicit treatment of the financial system. Third, the model emphasizes the negative effect of external debt on private investment and therefore incorporates the possibility of debt overhang. Finally, IMMPA accounts explicitly for the channels through which various types of public investment outlays affect the economy. Requirements Data/information: The greatest drawback of any fully specified CGE model is the time and data required to construct it. The model must be constructed from combined national accounts and survey data. These are first compiled into a SAM, which is then used as the foundation of the model. IMMPA for example consists of 131 equations, more than 30 exogenous variables and more than 200 endogenous variables. Time: The process can take more than a year, and rarely less than a few months. Skills: Experienced modelers with substantial prior exposure to Computable General Equilibrium Models are required Supporting software: Eviews, Excel Financial cost: US$75,000 to develop the IMMPA general equilibrium model Limitations: CGE simulations depend to a large extent on the assumptions made in the model, especially those that are required to close the model. They are also data-intensive and difficult to master, something that could limit its usefulness under tight deadlines or capacity constraints. References and applications: • Agenor, Izquierdo, Fofack (2003). . 73 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Augmented CGE Model with Representative Household Approach What is it? This technique is based on a computable general equilibrium model with representative households that are linked to a household module. What can it be used for? Representative Household Models can be used to analyze the impact of macroeconomic policy and external shocks on income distribution, employment, and poverty What does it tell you? Representative household models allow for a forecast of welfare measures and poverty outcomes consistent with a set of macroeconomic policies in the context of a general equilibrium model Complementary tools: Key Elements: The key features of the Representative Household Approach are (i) a Computable General Equilibrium (CGE) model that incorporates markets for factors and commodities and their links to the rest of the economy, which generates equilibrium values for employment, wages and commodity prices as well as its “extended” functional distribution (i.e. labor differentiated by skill, education, gender, region, and sector of employment); and (ii) a mapping from the extended functional distribution into the “size” distribution (the distribution of income across different households). In this approach, the Representative Households that appear in the CGE (corresponding to aggregates or averages of groups of households) play a crucial role: the “size” distribution is generated by feeding data on the simulated outcomes for the Representative Household into a separate module that contains additional information about each household. Requirements Data/information: Representative Household Models require a social accounting matrix (SAM) and distributional data describing the Representative Household groups or, more specifically, a household survey Time: Only a few days to generate a base solution if data and skills are available. Between six months and a year to collect data and work with the simulations Skills: Experienced modelers with substantial prior exposure to Computable General Equilibrium models are required. Supporting software: Excel, Eviews, Gauss Financial cost: US$25-75,000 depending on the data that exists Limitations: In the absence of a CGE model to feed in the Representative Household module, it is data-intensive and difficult to master References and applications: • For an overview, see Logfren, Robinson and El-Said (2003), Chapter 15 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Robillard, Bourguignon and Robinson (2001) on Indonesia. • Coady and Harris (2001) on Mexico. • Lofgren et al. (2002). 74 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Social Risk Assessment What is it? Analytical approach that uses qualitative methods to identify and assess risks to, and from the policy reform, and to inform risk management strategies What can it be used for? Risk assessment is relevant to all reforms that have significant poverty and social impacts. Particularly useful for decentralization reforms; civil service retrenchment; land reform; financial sector reform (e.g. regulatory reform, privatization of SOE); labor market reform (e.g. minimum wage legislation); social safety nets; pension reforms; and agricultural reform (e.g. changing domestic subsidies and taxes, eliminating marketing boards). Social Risk Assessment follows an analysis of stakeholders, institutions, and social impacts, and feeds into M&E systems by establishing a baseline of objective risk indicators for country- and sector-level risk assessments What does it tell you? Helps to (a) identify risks that could undermine policy reform objectives; (b) inform analysis of alternatives in policy design, and inform design of complementary measures when it appears that a reform will have adverse impacts; (c) develop risk management strategies for the identified risks to, and from the policy reform. Risk assessment involves the following steps: (i) identification of assumptions – implicit or explicit – about what should (or should not) happen in order to for a policy to achieve its goals; (ii) assessing the likelihood that these assumptions will hold, and their importance to policy; (iii) assessing the impact of policy change should the assumptions prove invalid; (iv) informing risk management strategies to address important risks that are unavoidable. Complementary tools: Normally conducted after Stakeholder Analysis and Institutional Analysis, as a complement to impact analysis. Provides crucial insights for Scenario Analysis, and M&E systems Key Elements: (1) Identification of risks, (2) Assessment of the likelihood of occurrence and importance of each risk to the reform, and (3) Elaboration of adequate risk management strategies. Risks are identified from assumptions about transmission channels and likely impacts. This should cover country risks (e.g. conflict and violence, political instability, ethnic or religious tension); institutional risks (e.g.: weak governance or capacity, design complexity); political economy risks (e.g. capture of benefits, opposition or distortion by influential stakeholders); and exogenous risks (e.g. terms of trade, climate effects). Information about risks is gathered from (i) secondary literature, (ii) discussions with Bank staff and other partners; (iii) existing agencies that assess country risks; and (iv) questionnaires, in-depth interviews or focus groups with key informants from government agencies, non-government organizations and firms. This information is validated through triangulation and crosschecking among information obtained from these different sources. Requirements Data/information: Secondary material, including objective risk indicators, and risk assessments available from country databases, international risk rating agencies (e.g. EIU risk rating, ICRG, TI), and social science research, as well as from implementing agencies and partners. Primary data, that identifies the spectrum of risks to, and from the reform, illustrates their likelihood of occurrence and importance to the policy, and helps develop adequate risks management strategies. Time: Can be undertaken rapidly (2-4 person weeks) in country, depending on reform complexity. Skills: Sociological and anthropological training are helpful. It is crucial to have an in-depth knowledge of the country-context, reform area, country- and sector-level assessment of key assumptions regarding the reform, and objective country-level risk indicators. Supporting software: N/A Financial cost: Can be undertaken at relatively low-cost (US$16-25,000) Limitations: If poorly facilitated or done with contentious stakeholders, assessment can easily produce skewed perceptions of risks that are based, for instance, on dogma or political calculation, rather than reflection and deliberation. As findings are necessarily based on stakeholder understanding of complex issues, it is key to validate results through reiteration exercises. References and applications: • Beck et al (2002). • Kaufman and Kray (2000). • World Bank (2002c). • Economist Intelligence Unit Country Risks Ratings (http://www.eiu.com) • Transparency International Corruption Perception Index (http://www.transparency.org) • International Country Risk Guide ratings (http://www.prsgroup.com) 75 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Scenario Analysis What is it? Scenario analysis is a participatory exercise based on a facilitated process of brainstorming, rigorous data gathering to explore the issues raised in brainstorming and the creation of three to four plausible future situations (scenarios) in which a reform will play out. These scenarios are differentiated by plausible discontinuities (such as a change in government, a currency devaluation or a major shift in commodity or input prices), but take into account significant predictable factors (such as demographic trends). What can it be used for? Scenario analysis is forward-looking and is generally used to analyze “lumpy” investments or major changes in strategic direction. The process is particularly adapted to bringing the perspectives of different stakeholders together around contentious decisions. “Civic scenarios” have been used to bring leaders from different political groups together to lie out alternative paths during government transition in South Africa and the transition away from violence in Colombia and Guatemala. Scenarios have also been used to bring community leaders, environmentalists, politicians and transport specialists together to make long-term strategic plans for state-level transport investment in the United States. What does it tell you? Scenario analysis lets policy-makers: (i) “pre-test” the performance of a policy reform in different plausible situations, allowing for the creation of alternate plans; (ii) assess the level of ownership for a reform agenda among key stakeholders; (iii) get support for a reform agenda by including relevant stakeholders in discussions around scenarios to build a shared understanding of key issues in a reform. Complementary tools: Normally used in conjunction with economic models, which can serve as inputs to the scenario-building process, and stakeholder analysis, which helps determine key groups to consider in different scenarios. Key Elements: The elements of a complete scenario analysis are: (i) preliminary scenario workshop which brings together relevant stakeholders to brainstorm the key issues around a reform agenda; (ii) data collection wherein a researcher assembles relevant information around the issues identified in a workshop; (iii) scenario-building workshop where relevant stakeholders build alternate scenarios; (iv) dissemination process where scenarios are shortened to one-page briefing notes and shared with the public via newspapers, television and radio Requirements Data/information: Scenario analysis requires: (i) economic information, including standard economic projections; (ii) demographic information; (iii) sector- specific information relevant to the issues at hand; (iv) a basic profile of a country’s political economy and of ethnic, linguistic and religious divisions within a country. Time: When used to challenge analytic assumptions rather than to build support among stakeholders, the scenario exercise itself could be completed in three to four staff weeks. A participatory scenario exercise is usually carried out in two to three workshops lasting several days each. These workshops are usually spread over several calendar months in order to allow time for data collection and to accommodate the schedules of participants. Skills: An individual with strong facilitation skills and specific experience running scenario exercises. Research skills, including familiarity with economic and demographic trends. Supporting software: N/A Financial cost: A small exercise intended to ensure that the assumptions of policymakers are challenged would cost approximately US$10,000. A full exercise with participatory workshops designed to build support among stakeholders could cost as much as US$30,000. Limitations: Successful scenario analysis is based on the skill of facilitators and the choice of participants. Because the process is participatory and based on subjective understanding, it is best for strategic rather than tactical questions. References and applications: • Maack (2001). • Pruitt (2000). • Civic Scenarios: Kahane (1996) on South Africa, Kahane (1998) on Colombia. • Planning Scenarios: see experience of Utah at http://www.envisionutah.org/ 76 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Public Expenditure Tracking Survey (PETS) What is it? A technique to survey service-provides to assess the efficiency of public spending and the quality and quantity of services. What can it be used for? PETS can be used for the analysis of public expenditure management reforms, reforms to improve the efficiency of public expenditure, cross-cutting public sector reforms, anti-corruption, and service delivery reforms. What does it tell you? A PETS tracks the flow of resources through the various layers of government bureaucracy, down to the service facilities in order to determine how much of the originally allocated resources reach each level, and how long they take to get there. It can help identify the location and extent of impediments to resource flows (financial, staff, equipment). It can therefore evaluates the mechanisms and incentives responsible for public expenditure leakages, capture and deployment impediments. A PETS focuses on service provider behavior, incentives, and relationship between providers, policy-makers and users. Complementary tools: • A PETS can be cross-validated by a Quantitative Service Delivery Survey (QSDS) which assesses the efficiency of public spending at the level of service provider. • A PETS analysis can be linked upstream to public administration surveys, and downstream to household surveys. • Linking a PETS with household surveys would allow to include the demand for services or outcomes. • Benefit incidence analysis can be enhanced by using filter coefficients obtained from PETS/QSDS to deflate cost per user to take into account leakage or capture of funds. • Institutional and stakeholder analysis can help define the parameters of PETS surveys. • Citizen Report Cards can serve as a monitoring tool to verify the perceived effectiveness of public services for stakeholders. Key Elements: A PETS is typically implemented with the following steps: (1) Consultations with key stakeholders, including government agencies, donors and civil society organizations are carried out to: define the objectives of the survey, identify the key issues, determine the structure of resource flows and the institutional setup, review data availability, outline hypotheses and chose the appropriate survey tool. (2) Survey instruments are then constructed and implemented. The PETS deals with the fact that agents may have strong incentives to misreport data by using a multi-angular data collection strategy and carefully considering which sources and respondents have incentives to misreport, and identifying sources that tare the least contaminated by these incentives. Requirements Data/information: In addition to the PETS itself, uses public accounts sample data, preferably panel data, on government spending and information on outputs of service providers at ministerial, regional, local and service provider levels. Field testing of the survey is key to ensuring high quality results Time: Consultations, design, and pre-testing take several months. The survey itself takes 1-2 months, depending on sample size and data accessibility. Skills: Some prior experience of micro survey work and STATA required, and a detailed knowledge of the relevant institutional context. Microeconomics of provider behavior (incentives and organization theory). Supporting software: STATA Financial cost: US$60-100,000 plus design Limitations: Results suffer from data limitations, i.e. where service provision is not well recorded, or is in-kind. Respondents may have incentives to misreport information References and applications: • For an overview, see Dehn, Reinikka and Svensson (2003), Chapter 9 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Reinikka and Svensson (2002a) for an overview of the approach. • Reinikka (2001), Reinikka and Svensson (2003), Republic of Uganda (2000 and 2001) on Uganda. • Government of Tanzania (1999 and 2001) on education and health care in Tanzania. • Xiao and Canagarajah (2002) on Ghana. • Das et al. (2002) on Zambia. • World Bank (2001b) on Honduras. • See www.publicspending.org and http://econ.worldbank.org/programs/public_services/topic/tools/ for some of the tools available and their applications. 77 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Quantitative Service Delivery Survey (QSDS) What is it? A technique to survey the efficiency of service provision What can it be used for? A QSDS can be used for the analysis of public expenditure management reforms, service-delivery reforms, reforms to improve the efficiency of public expenditure, as well as cross-cutting public sector reforms. What does it tell you? A QSDS examines the efficiency of public spending and incentives, and various dimensions of services delivery in provider organizations, especially at the level of the service facility. It quantifies the factors affecting quality of service such as incentives, accountability mechanisms, and the relationship between agents and principals. Complementary tools: • A QSDS can be cross-validated by a Public Expenditure Tracking Survey (PETS) which tracks the flow of resources from the central level to the level of service provider. A QSDS analysis can also be linked upstream to public administration surveys, and downstream to household surveys • A QSDS analysis can also be linked upstream to public administration surveys, and downstream to household surveys. • Linking a QSDS with household surveys would allow to include the demand for services or outcomes. • Benefit incidence analysis can be enhanced by using filter coefficients obtained from PETS/QSDS to deflate cost per user to take into account leakage or capture of funds. • Institutional and stakeholder analysis can help define the parameters of PETS surveys. • Citizen Report Cards can serve as a monitoring tools to verify the perceived effectiveness of public services for stakeholders. Key Elements: The QSDS is typically implemented with the following steps: (1) Consultations with key stakeholders, including government agencies, donors and civil society organizations are carried out to: define the objectives of the survey, identify the key issues, determine the structure of resource flows and the institutional setup, review data availability, outline hypotheses and chose the appropriate survey tool. (2) Survey instruments are then constructed and implemented. The QSDS deals with the fact that agents may have strong incentives to misreport data by using a multi-angular data collection strategy and carefully considering which sources and respondents have incentives to misreport, and identifying sources that tare the least contaminated by these incentives. Requirements Data/information: In addition to the QSDS itself, uses public accounts sample data, preferably panel data, on government spending and information on outputs of service providers at ministerial, regional, local and service provider levels. Field testing of the survey is key to ensuring high quality results. Time: Consultations, design, and pre-testing take several months. The survey itself takes 1-2 months, depending on sample size and data accessibility. Skills: Some prior experience of micro survey work and STATA required, and a detailed knowledge of the relevant institutional context. Microeconomics of provider behavior (incentives and organization theory). Supporting software: STATA or other similar software Financial cost: US$60-100,000 plus design Limitations: Results suffer from data limitations, i.e. where service provision is not well recorded, or is in-kind. Respondents may have incentives to misreport information References and applications: • For an overview, see Dehn, Reinikka and Svensson (2003), Chapter 9 of the Toolkit for Evaluating the Poverty and Distributional Impact of Economic Policies. • Chaudhury and Hammer (2003) on Bangladesh. • Lindelow et al. (2003) on Uganda • See www.publicspending.org and http://econ.worldbank.org/programs/public_services/topic/tools/ for some of the tools available and their applications. 78 Annex: Economic and Social Tools for Poverty and Social Impact Analysis Tool Name: Citizen Report Card What is it? A participatory survey that solicits client feedback on the performance of public services. It combines qualitative and quantitative methods to collect useful demand side data that can help improve the performance of public services. Extensive media coverage and civil society advocacy allows the tool to be used for public accountability. What can it be used for? Citizen Report Cards are used in situations where demand side data, such as user perceptions on quality and satisfaction with public services, are absent. By collecting and aggregating user feedback they provide an avenue for citizens to signal public agencies and politicians on key reform areas, and also to create competition among state-owned monopolies. What does it tell you? Citizen Report Cards provide feedback from actual users of services regarding issues such as: (a) Availability of services, (b) Satisfaction with services, (c) Reliability/Quality of services and the indicators to measure these, (d) Responsiveness of service providers, (e) Hidden costs - corruption and support systems, (f) Willingness to pay, and (g) Quality of life. Complementary tools: • Can be used in conjunction with national service delivery and other household surveys. It is also being supported by the more qualitative community scorecard process. • Needs to be complemented with effective communications and publicity strategy to put information in the public domain and increase effectiveness. The data from citizen report cards can also be used in conjunction with Public Expenditure Tracking Surveys (example Sierra Leone) and Benefit Incidence Analysis. Key Elements: User-determined assessment criteria; quantitative feedback on service delivery quality; media involvement and broad public debate on process and survey results. Requirements Data/information: Being a survey itself, the only data/information required is for developing the sampling frame for the execution of the survey. For this basic demographic, economic and social data from recent household surveys would be needed to decide on the stratification. Time: Between 3-6 months from the initial scoping to the dissemination stage. In most countries an initial orientation workshop for different stakeholders is included. Skills: Usually the citizen report card is managed by a different agency from the on that actually executes the survey. For the latter, the norm has been to out source to a market research agency such as ORG-MARG (India) or the Social Weather Station (Philippines), which has adequate market research and statistical survey analysis skills. The managing organization is either an independent CSO with solid advocacy skills, networks and reputation (India), an international donor like the World Bank (Philippines), or a Government Department in charge of monitoring and independent review/oversight of public services. Supporting software: N/A Financial cost: Varies according on the depth and purpose of analysis. A full national survey in a moderately sized country would cost in the range of US$100,000. Limitations: The limitations include: (i) requires an agency with market research and data collection skills to conduct the survey, (ii) requires support of media, (iii) role of expectations in user perceptions needs to be factored, (iv) limitations in comparability across services, (v) cost considerations, (vi) large sample required for heterogeneous population and lesser used services, (vii) effort & time to stimulate action by service agencies & civil society, (viii) lack of predictability in how different players respond. References and applications: • World Bank. (2002d). • Public Affairs Center (2002). 79 A User’s Guide to Poverty and Social Impact Analysis Tool Name: Community Score Card What is it? A community based qualitative monitoring tool that draws on techniques of social audit, community monitoring and citizen report cards. The process is also an instrument for empowerment and accountability as it includes an interface meeting between service providers and the community that allows for immediate feedback. What can it be used for? The community scorecard is a tool for Participatory Public Expenditure Reviews. It is also used for local level monitoring and performance evaluation of services, projects and even government administrative units (like district assemblies) by the community themselves. The process allows for (a) tracking of inputs or expenditures (e.g. availability of drugs), (b) monitoring of the quality of services/projects, (c) generation of benchmark performance criteria that can be used in resource allocation and budget decisions, (d) comparison of performance across facilities/districts, (e) generating a direct feedback mechanism between providers and users, (f) building local capacity and (g) strengthening citizen voice and community empowerment. What does it tell you? The community scorecard provides information on (a) how inputs or expenditures match with entitlements/allocations at the local/ facility level, (b) the criteria used by the community and by providers themselves to assess their own performance, (c) how both the community and providers score themselves on these criteria, (d) anecdotal evidence on which these scores are based, and (e) how the assessments by the community and providers can be used to develop an action plan for improvements in the project/service. Complementary tools: • Can be used in conjunction with national service delivery surveys and the citizen report card survey. • Can form participatory component of public expenditure reviews, public expenditure tracking surveys, formal financial audits and benefit incidence analyses. Key Elements: Community-based, i.e. designed and executed, qualitative service assessment; professionally facilitated public discussion of results. Requirements Data/information: The input tracking scorecard requires supply side information on inputs and expenditures such as (a) Budgets/allocations to a facility/project, (b) Entitlements as specified under a policy/project document, (c)List of inventories at facility level, etc. At the community level, an existing social mapping, and basic demographic data is needed to divide the community into focus groups. If the process is to be conducted on a representative sample of communities across the nation/district then basic socio-economic data would be needed to select the sample frame. Time: The process itself requires only a few (sometimes a single) community gatherings. However the groundwork and orientation for collecting supply-side input/expenditure data, preparing the providers and community for the scorecard and for the interface meeting, as well as the secondary data analysis may require in the region of 3-6 weeks. Skills: The community scorecard process requires expert facilitation and so experience with participatory methods and a history of involvement with the community are usually pre-requisites for the process to run smoothly. Supporting software: N/A Financial cost: Financial costs of conducting the process in a single community are quite low - limited mainly to the time of the facilitating staff. If however done on a large scale with many communities involved, the costs would be higher. Overall cost ranges from US$30,000 to US$40,000, comparable to a beneficiary assessment. Limitations: The limitations of the community scorecard process include: (a) it depends a great deal on quality of facilitation, (b) input tracking dependent on availability of supply side data, (c) the interface meeting can get confrontational, (d) greater standardization of indicators needed when scaling up, (e) small sample size during gathering can bias results, (f) scoring not always applicable. References and applications: • Republic of Gambia (2002). • Information can be found at: http://www.worldbank.org/participation/spaccount.htm 80 Bibliography Note: The word processed describes informally repro- Alderman, H. and C. del Ninno. 1999. “Poverty issues duced works that may not be commonly available for zero rating VAT in South Africa.” Journal of through libraries. African Economies 8 (2): 182-208. Agénor, P. 2002. “Macroeconomic Adjustment and the Alwang, J., P. Siegel, and S. Jorgensen. 1996. “Seeking Poor: Analytical Issues and Cross-Country Evi- Guidelines for Poverty Reduction in Rural Zam- dence.” Staff Working Paper 2788. World Bank, bia.” World Development 24 (11): 1711–23. Washington, D.C. Angrist, J. E. Bettinger, E. Bloom, E. King and M. Kre- Agénor, P., and J. Aizenman. 1999. “Macroeconomic mer. 2001. “Vouchers for Private Schooling in Adjustment with Segmented Labor Markets.” Colombia: Evidence from a Randomized Natural Journal of Development Economics 58 (2): 277–96. Experiment”, NBER Working Paper 8343. Agénor, P., A. Izquierdo, and H. Fofack. 2003. Arulpragasam, J. and P. Conway. 2003. “Partial Equi- “IMMPA: A Quantitative Macroeconomic Frame- librium Multi-Market Analysis.” In F. Bour- work for the Analysis of Poverty Reduction Strate- guignon and L.A. Pereira da Silva, eds., Evaluating gies.” World Bank, Washington, D.C. Processed. the Poverty and Distributional Impact of Economic Policies (Techniques and Tools), Washington D.C.: Ahmad, E. and N. Stern. 1984. “The theory of reform World Bank and Indian indirect taxes.” Journal of Public Eco- nomics 25 (3): 259-98. Atkinson, A. and F. Bourguignon. 1991. “Tax-Benefit Models for Developing Countries: Lessons from Ahmad, E. and N. Stern. 1987. “Alternative sources of Developed Countries” in J. Khalilzadeh-Shirazi government revenue: Illustrations from India, and A. Shah, eds., Tax Policy in Developing Coun- 1979-80.” In Newbery, David and Nicholas Stern, tries, The World Bank, Washington, DC eds. The Theory of Taxation for Developing Coun- tries. Oxford: Oxford University Press. Attanasio, O., C. Meghir and A. Santiago. 2002. “Edu- cation Choices in Mexico: Using a Structural Ahmad, E. and N. Stern. 1990. “Tax reform and Model and a Randomized Experiment to Evaluate shadow prices for Pakistan.” Oxford Economic Progresa”, University College London, processed. Papers 42 (1): 135-59. Baker, J., 2000. Evaluating the Impact of Development Ahmad, E. and N. Stern. 1991. The Theory and Practice Projects on Poverty: A Handbook for Practitioners. of Tax Reform in Developing Countries. Cam- Washington, D.C.: World Bank. bridge: Cambridge University Press. 81 A User’s Guide to Poverty and Social Impact Analysis Barnum, H., and L. Squire. 1979. A Model of an Agri- Programs: the Case of Bolsa Escola”, World Bank cultural Household: Theory and Evidence. Balti- Policy Research Working Paper #2916, Washing- more: Johns Hopkins University Press, for the ton, DC. World Bank. Bourguignon, F. and F. H.G. Ferreira. 2003. “Ex-Ante Barro, R. 1997. Determinants of Economic Growth: A Evaluation of Policy Reforms using Behavioral Cross-Country Empirical Study. Cambridge: MIT Models.” In F. Bourguignon and L.A. Pereira da Press. Silva, eds., Evaluating the Poverty and Distribu- tional Impact of Economic Policies (Techniques and Beck, C., G. Clarke, A. Groff., P. Keefer, and P. Walsh. Tools), Washington D.C.: World Bank. 2002. “New Tools and New Tests in Comparative Political Economy: The Database of Political Insti- Bourguignon, F. and L.A. Pereira da Silva, eds. 2003. tutions”. World Bank. Washington, DC. Evaluating the Poverty and Distributional Impact of Economic Policies (Techniques and Tools), Wash- Becker, G. 1965. “A Theory of the Allocation of Time.” ington D.C.: World Bank Economic Journal 75: 493–517. Brinkerhoff, D., and B. L. Crosby. 2002. Managing Pol- Becker, H. 1997. Social Impact Assessment. London: icy Reform: Concepts and Tools for Decision-Makers University College Press. in Developing and Transition Countries. Bloom- Benjamin, D. 1992. “Household Composition, Labor field, Conn.: Kumarian Press. Markets, and Labor Demand: Testing for Separa- Carvalho, S., and H. White. 1997. “Combining the tion in Agricultural Household Models.” Econo- Quantitative and Qualitative Approaches to metrica 60: 287–322. Poverty Measurement and Analysis: The Practice Bianchi, R., and S. Kossoudji. 2001. “Interest Groups and the Potential.” Technical Paper 366. World and Organizations as Stakeholders.” Social Devel- Bank, Washington, D.C. opment Paper 35. World Bank, Washington, D.C. Castro-Leal, F. 1996. ‘Poverty and Inequality in the Binswanger, H., and J. Quizon. 1984. “Distributional Distribution of Public Education Spending in Consequences of Alternative Food Policies in India.” South Africa.’ PSP Discussion Paper Series 102, Discussion Paper 20. World Bank, Agriculture and World Bank, Poverty and Social Policy Depart- Rural Development Department, Washington, D.C. ment, Washington D.C. Binswanger, H. and J. Quizon 1986. “Modeling the Castro-Leal, F., J. Dayton and L. Demery. 1997. ‘Public Impact of Agricultural Growth and Government Social Spending in Africa: Do the Poor Benefit?’ Policy on Income Distribution in India.” World World Bank Research Observer Bank Economic Review 1: 103–48. Cernea, M. and A. Kudat, eds. 1997. Social Assessments Blundell, R., A. Duncan, J. McCrae and C. Meghir. for Better Development, Case Studies in Russia and 2000. “Evaluating In-Work Benefit Reforms: the Central Asia. ESSD Studies and Monograph Working Families Tax Credit in the UK”. Discus- Series, no. 16. World Bank: Washington, DC. sion Paper, Institute for Fiscal Studies, London. Chaudhury , N., and J. Hammer. 2003. “Ghost Doctors Bolt, R., and M. Fujimura. 2002. “Policy-based Lend- : Absenteeism in Bangladeshi Health Facilities.” ing and Poverty Reduction: An Overview of World Bank. Processed Processes, Assessment, and Options.” Working Chen, D., J. Matovu and R. Reinikka. 2001. “A quest for Paper Series 2. Asian Development Bank, Eco- revenue and tax incidence.” In R. Reinikka and P. nomics and Research Department, Manila. Collier, eds., Uganda’s Recovery: The Role of Farms, Bourguignon, F., F.H.G. Ferreira and P. Leite. 2002. Firms and Government. Washington, D.C.: World “Ex-ante Evaluation of Conditional Cash Transfer Bank. 82 Bibliography Christensen, L. R., D. W. Jorgensen, and L. J. Lau. 1975. Decaluwé, B., J.-C. Dumont, and L. Savard. 1999. “Transcendental Logarithmic Utility Functions.” “Measuring Poverty and Inequality in a Com- American Economic Review 65: 367–83. putable General Equilibrium Model.” Working Paper 9920. Université Laval, Centre de Recherche Coady, D., and R. Harris. 2001. “A Regional General en Économie et Finance Appliquées, Montreal. Equilibrium Analysis of the Welfare Impact of Cash Transfers: An Analysis of Progresa in Mex- Decaluwé, B., A. Patry, L. Savard, and E. Thorbecke. ico.” Trade and Macroeconomics Division Discus- 1999. “Poverty Analysis within a General Equilib- sion Paper 76. International Food Policy Research rium Framework.” Working Paper 9909. Univer- Institute. Washington, D.C. sité Laval, Centre de Recherche en Économie et Finance Appliquées, Montreal. Cornia, A., R. Jolly, and F. Stewart. 1987. Adjustment with a Human Face. Oxford: Clarendon Press. Dehn, J., R. Reinikka, and J. Svensson. 2003a. “Survey tools for Assessing Performance in Service Deliv- Cox, D., and E. Jimenez. 1995. “Private Transfers and ery.” In F. Bourguignon and L.A. Pereira da Silva, the Effectiveness of Public Income Redistribution eds., Evaluating the Poverty and Distributional in the Philippines.” In D. Van de Walle and K. Impact of Economic Policies (Techniques and Tools), Nead, eds., Public Spending and the Poor: Theory Washington D.C.: World Bank. and Evidence. Baltimore: Johns Hopkins Univer- sity Press for the World Bank. de Janvry, A., M. Fafchamps, and E. Sadoulet. 1991. “Peasant Household Behaviour with Missing Davidson, R., and J. Duclos. 2000. “Statistical Inference Markets: Some Paradoxes Explained.” Economic for Stochastic Dominance and for the Measure- Journal 101: 1400–17. ment of Poverty and Inequality.” Econometrica 68 (6): 1435–64. de Maio, L., F. Stewart, and R. van der Hoeven. 1999. “Computable General Equilibrium Models, Das, J., S. Dercon, J. Habyarimana, and P. Krishnan. Adjustment and the Poor in Africa.” World Devel- 2002. “Rules vs. Discretion: Public and Private opment 27: 453–75. Funding in Zambian Basic Education. Part I: Funding Equity.” World Bank, Washington, D.C. Demery, L. 2003. “Analyzing the Incidence of Public Spending.” In F. Bourguignon and L.A. Pereira da Datt, G., K. Ramadas, D. van der Mensbrugghe, T. Silva, eds., Evaluating the Poverty and Distribu- Walker, and Q. Wodon. 2003. “Predicting the tional Impact of Economic Policies (Techniques and effect of aggregate growth on poverty.” In F. Bour- Tools), Washington D.C.: World Bank. guignon and L.A. Pereira da Silva, eds., Evaluating the Poverty and Distributional Impact of Economic Demery, L. 2000. “Benefit Incidence: A Practitioner’s Policies (Techniques and Tools), Washington D.C.: Guide.” World Bank, Africa Region, Poverty and World Bank. Social Development Group, Washington, D.C. Datt, G. and T. Walker. 2002. PovStat 2.12, A Poverty Demery, L., S. Chao, R. Bernier and K. Mehra. 1995. Projection Toolkit, User’s Manual, World Bank, “The Incidence of Social Spending in Ghana”. PSP Washington, D.C. Processed. Discussion Papers Series 82. Poverty and Social Policy Department. World Bank, Washington, Davidson, R., and J. Duclos. 2000. “Statistical Inference D.C. for Stochastic Dominance and for the Measure- ment of Poverty and Inequality.” Econometrica 68 Demery, L., M. Ferroni, and C. Grootaert, eds. 1993. (6): 1435–64. Understanding the Social Effects of Policy Reform. Washington, D.C.: World Bank. Deaton, A., and J. Muellbauer. 1986. Economics and Consumer Behavior. New York: Cambridge Uni- Demombynes, G., C. Elbers, J. O. Lanjouw, P. Lanjouw, versity Press. J. A. Mistiaen, and B. Özler. 2002. ‘Producing an 83 A User’s Guide to Poverty and Social Impact Analysis Improved Geographic Profile of Poverty: Method- Easterly, W. 1999. “The Ghost of Financing Gap: Test- ology and Evidence from Three Developing ing the Growth Model Used in the International Countries’ Discussion Paper 2002/39, WIDER, Financial Institutions.” Journal of Development Helsinki. Economics 60: 423–38. Dervis, K., J. de Melo, and S. Robinson.1982. General Egamberdi, N., P. Gordon, A. Ikhamov, D. Kandiyoti, Equilibrium Models for Development Policy. New and J. Shoerberlein-Engel. 2000. “Uzbekistan York: Cambridge University Press. Agriculture Enterprise Restructuring and Devel- opment Program.” In A. Kudat, C. Keyder, and S. Devarajan, S. and D. S. Go. 2003. “The 123PRSP Peabody, eds., Social Assessment and Agricultural Model.” In F. Bourguignon and L.A. Pereira da Reform in Central Asia and Turkey. Washington, Silva, eds., Evaluating the Poverty and Distribu- D.C.: World Bank. tional Impact of Economic Policies (Techniques and Tools), Washington D.C.: World Bank. Elbers, C., J.O. Lanjouw, and P. Lanjouw. 2002. “Wel- fare in Villages and Towns: Micro level Estimation Devarajan, S., and D. Go (with F. Charlier, A. Dabalen, of Poverty and Inequality”, Policy Research Work- W. Easterly, H. Fofack, A. Izquierdo, and L. ing Paper 2911. World Bank, Washington, D.C. Koryukin). 2001. “A Macroeconomic Framework for Poverty Reduction Strategy Papers, with an Elbers, C., J.O. Lanjouw, P. Lanjouw, and P.G. Leite. Application to Zambia.” World Bank, Washington, 2002. “Poverty and Inequality in Brazil: New Esti- D.C. Processed. mates from Combined PPV-PNAD Data”. World Bank, Washington, D.C. Processed. Devarajan, S., W. Easterly, H. Fofack, D. Go, A. Izquierdo, C. Petersen, L. Pizzati, C. Scott, and L. Elbers, C., P. Lanjouw, J. A. Mistiaen, B. Özler, and K. Serven. 2000. “A Macroeconomic Framework for Simler. 2002. “Are Neighbours Equal? Estimating Poverty Reduction Strategies.” World Bank, Wash- Local Inequality in Three Developing Countries” ington, D.C. Processed. paper presented at the LSE/Cornell/WIDER con- ference on Spatial Distribution of Inequality, Lon- Devarajan, S., H. Ghanem, and K. Thierfelder. 1999. don School of Economics, London. “Labor Market Regulations, Trade Liberalization, and the Distribution of Income in Bangladesh.” European Commission. 2002. “Project Cycle Manage- Journal of Policy Reform 3 (1): 1–28. ment Handbook.” EuropeAid Cooperation Office, General Affairs, Evaluation. Brussels. Devarajan, S., and S. Hossain. 1998. “The Combined Incidence of Taxes and Public Expenditures in the Finsterbusch, K., J. Ingersoll, and L. Llewellyn. 1990. Philippines.” World Development 26: 963–77. Methods for Social Analysis in Developing Coun- tries. San Francisco: Westview Press. Dollar, D., and A. Kraay. 2002. “Growth Is Good for the Poor.” Journal of Economic Growth Vol 7, Number Foster, V., and C. Araujo. 2001. “Does Infrastructure 3, 195-225. Reform Work for the Poor? A Case Study from Guatemala.” World Bank, Washington, D.C. Dorosh, P., C. del Ninno and D. Sahn. 1995. “Poverty Processed. Alleviation in Mozambique: a multi-market analysis of the role of food aid”, Agicultural Eco- Galasso, E., M. Ravallion and A. Salvia. 2001. “Assisting nomics 13, 89-99. the Transition from Workfare to Work: A Ran- domized Experiment”. Policy Research Working Dulamdary, E., M. Shah, and R. Mearns, with B. Paper 2738, World Bank, Washington, D.C. Enkhbat and L. Ganzaya. 2001. “Mongolia: Partic- ipatory Living Standards Assessment.” National Gelbach, J., and L. Pritchett. 2000. “Indicator Targeting Statistics Office of Mongolia and The World Bank, in a Political Economy: Leakier Can Be Better.” Washington, D.C. Journal of Policy Reform 4 (2): 113–45. 84 Bibliography GTZ (Gesellschaft für Technische Zusammenarbeit). International Centres for Economics and Related 1991. Methods and Instruments for Project Plan- Disciplines, London. ning and Implementation. Eschborn, Germany. Hunt, D. 1996. Process Mapping: How to Reengineer Your Gibson, J. 1998. “Indirect tax reform and the poor in Business Processes. New York: John Wiley & Sons. Papua New Guinea.” Pacific Economic Bulletin 13 Ianchovichina, E., A. Nicita, and I. Soloaga. 2001. (2): 29-39. “Trade Reform and Household Welfare: The Case Gittinger, J. P. 1985. Economic Analysis of Agricultural of Mexico.” Working Paper 2667. World Bank, Projects. Washington, D.C.: World Bank. Washington, D.C. Goldman, L.R., ed. 2000. Social Impact Analysis: An Ironmonger, D. 1999. “An Overview of Time Use Sur- Applied Anthropology Manual. Oxford: Berg Press. veys.” Paper presented at the International Semi- nar on Time Use Studies, Ahmedabad, India, Government of Tanzania. 1999. “Tanzania Public December 7–10 . Expenditure Review: Health and Education Finan- cial Tracking Study. Final report, Vol. III.” Price Jabara, C., M. Lundberg, and A. Sireh Jallow. 1992. Waterhouse Coopers. Dar es Salaam. Processed. “Social Accounting Matrix for The Gambia.” Working Paper No. 20. Cornell University, Cornell Government of Tanzania. 2001. “ProPoor Expenditure Food and Nutrition Policy Program, Ithaca, N.Y. Tracking.” Research on Poverty Alleviation and Economic (REPOA) and Social Research Founda- Jalan, J. and M. Ravallion. 2003a. “Estimating Benefits tion to Tanzania PER Working Group. Dar es Incidence for Anti-poverty Program using Salaam. Processed. Propensity Score Matching”. Journal of Business and Economic Statistics, Volume 21, Issue 1. Grootaert, C., and T. van Bastelaer, eds. 2002. Under- standing and Measuring Social Capital: A Multidis- Jalan, J. and M. Ravallion. 2003b. “Does Piped Water ciplinary Tool for Practitioners. Washington, D.C.: Reduce Diarrhea for Children in Rural India?” World Bank. Journal of Econometrics, Volume 112, Issue 1. Hammer, J., and A. Tan. 1989. “A Multimarket Model Jalan, J. and M. Ravallion. 1999. “Are the Poor Less for Turkish Agriculture.” Working Paper Series Well Insured? Evidence on Vulnerability to 285. World Bank, Agriculture and Rural Develop- Income Risk in Rural China.” Journal of Develop- ment Department, Washington, D.C. ment Economics 58 (1): 61–81. Hammer, J., I. Nabi, and J. Cercone. 1995. “Distribu- Jalan, J., and M. Ravallion. 1997. “Spatial Poverty tional Effects of Social Sector Expenditures in Traps?” Policy Research Working Paper 1862. Malaysia.” In D. Van de Walle and K. Nead, eds., World Bank, Washington, D.C. Public Spending and the Poor: Theory and Evi- Jensen, R. 1998. “Public Transfers, Private Transfers, dence. Baltimore: Johns Hopkins University Press and the Crowding-Out Hypothesis: Evidence for the World Bank. from South Africa.” Research Working Paper Haney, M., M. Shkaratan, V. Kabalina, V. Paniotto, and R98–08. Harvard University, Kennedy School of C. Rughinis. 2003. “Mine Closure and Its Impact Government, Cambridge, Mass. on the Community: Five Years after Mine Closure Kahane, A. 1996. “The Mont Fleur Scenarios: What in Romania, Russia and Ukraine.” World Bank, will South Africa be like in the year 2002?” Deeper Washington, D.C. Forthcoming. News. Volume 7, Number 1. Emeryville, CA. Howes, S. 1993. “Mixed Dominance: A New Criterion Kahane, A. 1998. “Destino Colombia: A Scenario- for Poverty Analysis.” Working Paper DARP/2. Planning Process for the New Millenium”. Deeper London School of Economics, Suntory and Toyota News. Volume 9, Number 1. Emeryville, CA. 85 A User’s Guide to Poverty and Social Impact Analysis Kaufman, D. and A. Kraay. 2000. “Governance Matters Lofgren, H., S. Robinson and M. El-Said. 2003. II: Updated Indicators for 2000-01”.Working “Poverty and Inequality Analysis in a General Paper No 2772. World Bank, Washington, D.C. Equilibrium Framework: The Representative Household Approach.” In F. Bourguignon and L.A. Lampietti, J., A. Kolb, S. Gulyani, and V. Avenesyan. Pereira da Silva, eds., Evaluating the Poverty and 2001. “Utility Pricing and the Poor: Lessons from Distributional Impact of Economic Policies (Tech- Armenia.” Technical Paper 497. World Bank, niques and Tools), Washington D.C.: World Bank. Washington, D.C. López, R. 1986. “Structural Models of the Farm Lanjouw, P. 2003. “Estimating Geographically Disag- Household that Allow for Interdependent Utility gregated Welfare Levels and Changes.” In F. Bour- and Profit-Maximization Decisions.” In I. Singh, guignon and L.A. Pereira da Silva, eds., Evaluating L. Squire, and J. Strauss, eds., Agricultural House- the Poverty and Distributional Impact of Economic hold Models: Extensions, Applications, and Policy. Policies (Techniques and Tools), Washington D.C.: Baltimore: Johns Hopkins University Press for the World Bank. World Bank. Lanjouw, P., M. Pradhan, F. Saadah, H. Sayed, and R. López, R., J. Nash, and J. Stanton. 1995. “Adjustment and Sparrow. 2001. “Poverty, Education, and Health in Poverty in Mexican Agriculture: How Farmers’ Indonesia: Who Benefits from Public Spending?” Wealth Affects Supply Response.” Policy Research Policy Research Working Paper 2739. World Bank, Working Paper 1494. World Bank, Washington, D.C. Washington, D.C. Lundberg, M., M. Over, and P. Mujinja. 2000. “Sources Lanjouw, P. and M. Ravallion. 1999. “Benefit Incidence of Financial Assistance for Households Suffering and the Timing of Program Capture,” World Bank an Adult Death in Kagera.” Policy Research Work- Economic Review. 13 (2) : 257-274. ing Paper 2508. World Bank, Washington, D.C. Lanjouw, P., and M. Ravallion. 1995. “Poverty and Maack, J. 2001. “Scenario Analysis: A Tool for Task Household Size.” Economic Journal 105: 1415–34. Managers.” In R. Krueger, M. Casey, J. Donner, S. Levinsohn, J., S. Berry, and J. Friedman. 1999. “Impacts Kirsch, and J. Maack, “Social Analysis: Selected of the Indonesian Economic Crisis: Price Changes Tools and Techniques.” Social Development and the Poor.” Working Paper 7194. National Papers 36. World Bank, Social Development Bureau of Economic Research, Cambridge, Mass. Department, Washington, D.C. Lindelöw, M., R. Reinikka, and J. Svensson. 2003. Minot, N. and F. Goletti. 1998. “Export Liberalization “Health Care on the Frontline: Survey Evidence and Household Welfare: The Case of Rice in Viet- on Public and Private Providers in Uganda.” nam.” American Journal of Agricultural Economics Human Development Working Paper Series, 80 3: 738-749 World Bank, Africa Region,Washington D.C. Mistiaen, J.A. 2002. “Small Area Estimates of Welfare Forthcoming. Impacts: The Case of Food Price Changes in Lockhart, C. 2001. “Institutional Analysis: Russia Coal Madagascar”, World Bank. Washington D.C. Case Study.” World Bank, Social Development Processed Department, Washington, D.C. Processed. Mistiaen, J.A., Özler, B., Razafimanantena, T., and Lofgren, H., R. Harris, and S. Robinson with assistance Razafindravonona, J. 2002. “Putting Welfare on from M. Thomas and M. El-Said. 2002. “A Stan- the Map in Madagascar”. World Bank. Washington dard Computable General Equilibrium (CGE) D.C. Processed Model in GAMS.” Microcomputers in Policy Narayan, D., and P. Petesch. 2002. “An Empowering Research, Vol. 5. International Food Policy Approach to Poverty Reduction.” In Voices of the Research Institute. Washington, D.C. 86 Bibliography Poor: From Many Lands. New York: Oxford Uni- Papanek, G. 1994. The Social Impact of Program Lend- versity Press for the World Bank. ing. Manila: Asian Development Bank. National Management Consultants. 2000. “Second Pruitt, B. 2000. UNDP Civic Scenario/Civic Dialogue Annual Report of the Kecamatan Development Workshop. Antigua, Guatemala, Nov 8-10, 2000. Project.” Jakarta. Processed. New York: UNDP. North, D. 1990. Institutions, Institutional Change and Public Affairs Center. 2002. The State of Karnataka’s Economic Performance. New York: Cambridge Public Services: Benchmarks for the New Millen- University Press. nium. Bangalore, India: State of Karnataka. Norton, A., B. Bird, K. Brock, M. Kakande, and C. Pyatt, G., and J. Round, eds. 1985. Social Accounting Turk. 2001. A Rough Guide to PPAs: Participatory Matrices: A Basis for Planning. Washington, D.C.: Poverty Assessment—An Introduction to Theory World Bank. and Practice. London: ODI Publications. Quah, D., and S. Durlauf. 1999. “The New Empirics of Orbeta, A., and M. Alba. 1998. “Simulating the Impact Economic Growth.” In John Taylor and Michael of Macroeconomic Policy Changes on the Nutri- Woodford, ed., The Handbook of Macroeconomics. tional Status of Households.” Micro Impacts of Amsterdam: Elsevier Science Publishers. Macroeconomic and Adjustment Policies Radulescu, S., and M. Larionescu. 1999. “Social Assess- (MIMAP) Research Paper 21. International ment of Mining Restructuring in Romania.” Development Research Centre, Ottawa. World Bank, Washington, D.C. Processed. Papanek, G. 1994. The Social Impact of Program Lend- Rama, M. 2001. “The Gender Implications of Public ing. Manila: Asian Development Bank. Sector Downsizing: The Reform Program of Viet- Pereira da Silva, L.A., B. Essama-Nssah and I. Samaké. nam.” Policy Research Working Paper 2573. World 2002. “A Poverty Analysis Macroeconomic Simu- Bank, Washington, D.C. Republished in World lator (PAMS): Linking household surveys with Bank Research Observer 17 (2): 167–89. macro-models.” Working Paper 2888. World Ramadas, K. D. van der Mensbrugghe, and Q. Wodon Bank, Washington, D.C. (2002) SimSip Poverty: Poverty and Inequality Pereira da Silva, L.A., B. Essama-Nssah and I. Samaké. Comparisons Using Group Data, Washington DC: 2003. “Linking Aggregate Macro-Consistency Mod- World Bank. els to Household Surveys: A Poverty Analysis Macro- Ravallion, M. 1999. “Monitoring Targeting Perfor- economoic Simulator (PAMS).” In F. Bourguignon mance When Decentralized Allocations to the and L.A. Pereira da Silva, eds., Evaluating the Poverty Poor Are Unobserved.” Policy Research Working and Distributional Impact of Economic Policies (Tech- Paper 2080. World Bank, Washington, D.C. niques and Tools), Washington D.C.: World Bank. Ravallion, M. 1999. “Is More Targeting Consistent Powell M. and J. I. Round. 2000. ‘Structure and Link- with Less Spending?” International Tax and Public age in the Economy of Ghana: A SAM Approach’, Finance 6 : 411-19. in E Aryeetey, J Harrigan and M Nissanke, eds. Economic Reforms in Ghana: Miracle or Mirage, Ravallion, M. 2003. “Assessing the Poverty Impact of Oxford: James Currey Press an Assigned Program” In F. Bourguignon and L.A. Pereira da Silva, eds., Evaluating the Poverty and Powers, John. 2003. “The World Bank and Poverty and Distributional Impact of Economic Policies (Tech- Social Impact Analysis: Considering a Transaction niques and Tools), Washington D.C.: World Bank. Cost Analysis Approach.” World Bank, Social Development Department, Washington, D.C. Ravallion, M. E. Galasso, T. Lazo and E. Philipp. 2001. Forthcoming. “Do Workfare Participants Recover Quickly from 87 A User’s Guide to Poverty and Social Impact Analysis Retrenchment?” Policy Research Working Paper national Development Consultants Ltd. Kampala. 2672, World Bank. Washington, D.C. Processed. Rao, V., and M. Woolcock. 2003. “Integrating Qualita- Rickson, R., J. Western, and R. Burge. 1990. “Social tive and Quantitative Approaches in Program Impact Assessment: Knowledge and Develop- Evaluation.” In “Techniques and Tools for Evaluat- ment.” Environmental Impact Assessment Review ing the Poverty and Distributional Impact of Eco- 10. World Bank, Washington, D.C. nomic Policies.” Processed. World Bank, Robb, C. 2002. Can the Poor Influence Policy? Participa- Washington, D.C. tory Poverty Assessments in the Developing World. Reinert, K. A. and D. W. Roland-Holst (1997) “Social Washington, D.C.: World Bank. Accounting Matrices.” In J. F. Francois and K. A. Robillard, A.-S., F. Bourguignon, and S. Robinson. Reinert eds., Applied Methods for Trade Policy 2001. “Crisis and Income Distribution: A Micro- Analysis: A Handbook, Cambridge: Cambridge Macro Model for Indonesia.” International Food University Press Policy Research Institute, Washington, D.C. Reinikka, R. 2001. “Recovery in Service Delivery: Evi- Processed. dence from Schools and Health Centers.” In R. Robles, M., C. Siaens and Q. Wodon. 2003. “Poverty, Reinikka and P. Collier, eds., Uganda’s Recovery: Inequality and Growth in Paraguay: Simulations The Role of Farms, Firms and Government. World Using SimSIP Poverty.” Economia & Sociedad. Bank Regional and Sectoral Studies. Washington, Forthcoming. D.C.: World Bank Rosenbaum, P. and D. Rubin, 1983. “The central role of Reinikka, R., and J. Svensson. 2002a. “Explaining Leak- the propensity score in observational studies for age of Public Funds.” Discussion Paper 3227. Cen- causal effects.” Biometrika. 70 : 41-55 tre for Economic Policy Research, London. Round, Jeffery,. 2003. “Social Accounting Matrices and Reinikka, R. and J. Svensson. 2002b. “Working for SAM-based Multiplier Analysis.” In F. Bour- God? Evaluating Service Delivery of Religious guignon and L.A. Pereira da Silva, eds., Evaluating Not-for-Profit Health Care Providers in Uganda.” the Poverty and Distributional Impact of Economic World Bank, Development Research Group, Policies (Techniques and Tools), Washington D.C.: Washington, D.C. Processed. World Bank Reinikka, R., and J. Svensson. 2003. “The Power of Rutherford, M. 1994. Institutions in Economics: The Information: Evidence from an Information Old and the New Institutionalism. New York: Cam- Campaign to Reduce Capture.” World Bank. bridge University Press. Washington, D.C. Processed. Sadoulet E and A de Janvry. 1995. Quantitative Devel- Republic of Gambia. 2002. “Strategy for Poverty Alle- opment Policy Analysis. Baltimore: Johns Hopkins viation (SPAII) (PRSP).” Department of State for University Press. Finance and Economic Affairs, Strategy for Poverty Alleviation Coordinating Office. Banjul. Sahn, D., and H. Alderman. 1995. “Incentive Effects on Processed. Labor Supply of Sri Lanka’s Rice Subsidy.” In D. Van de Walle and K. Nead, eds., Public Spending Republic of Uganda. 2000. “Tracking the Flow of and and the Poor: Theory and Evidence. Baltimore: Accountability for UPE Funds.” International Johns Hopkins University Press for the World Development Consultants Ltd. Kampala. Bank. Processed. Sahn, D. E. and S. D. Younger. 2003. “Estimating the Republic of Uganda. 2001. “Study to Track Use of and Incidence of Indirect Taxes in Developing Coun- Accountability of UPE Capitation Grants.” Inter- 88 Bibliography tries.” In F. Bourguignon and L.A. Pereira da Silva, Tarp, F, D Roland-Holst and J Rand. 2002. “Trade and eds., Evaluating the Poverty and Distributional Income Growth in Vietnam: Estimates from a Impact of Economic Policies (Techniques and Tools), New Social Accounting Matrix.” Economic Systems Washington D.C.: World Bank Research 14 (2): 157-184. Salmen, Lawrence F. 2002. Beneficiary Assessment, An Thorbecke, E., and H. Jung. 1996. “A Multiplier Approach Described. Social Development Paper Decomposition Method to Analyze Poverty Alle- No. 10.World Bank, Social Development Depart- viation.” Journal of Development Economics 48 (2): ment, Washington, D.C. 279–300. Salmen, Lawrence F.1998. and Amelga, Misgana. Timmer, C. P., W. Falcon, and S. Pearson. 1983. Food Implementing Beneficiary Assessment in Education: Policy Analysis. Baltimore: Johns Hopkins Univer- A Guide for Practitioners (with examples from sity Press. Brazil). Social Development Paper No. 25. World Torero, M., and A. Pascó-Font. 2001. “The Social Bank, Social Development Department, Washing- Impact of Privatization and the Regulation of ton, D.C. Utilities in Peru.” Discussion Paper 2001/17. Salmen, Lawrence F. 1995. Participatory Poverty Assess- UNU-WIDER, Helsinki. ment, Incorporating Poor People’s Perspectives into Tymons, R.T., and Jacobs, R.A. 1997 “Multi-level Poverty Assessment Work. Social Development Process Mapping: A tool for Cross-Functional Paper No. 11. World Bank, Social Development Quality Analysis”, Production and Inventory Man- Department, Washington, D.C. agement Journal, 4th Quarter: 71-75. Sechaba Consultants. 2002. Ability and Willingness to van de Walle, D. 1992. “The Distribution of the Bene- Pay for Urban Water Supply. Maseru, Lesotho. fits from Social Services in Indonesia,1978-87.” Processed. Policy Research Working Paper 871. World Bank, Shoven, J., and J. Whalley. 1992. Applying General Washington, D.C. Equilibrium. New York: Cambridge University van de Walle, D. 1994.“The Distribution of Subsidies Press. through Public Health Services in Indonesia 1978- Singh, I., L. Squire, and J. Strauss. 1986. Agricultural 87.” World Bank Economic Review 8 (2): 279-309. Household Models: Extensions, Applications, and van de Walle, D. 1998. “Assessing the Welfare Impacts Policy. Baltimore: Johns Hopkins University Press. of Public Spending.” World Development 26 (3): Squire, L., and H. van der Tak. 1975. Economic Analy- 365–79. sis of Projects. Baltimore: Johns Hopkins Univer- van de Walle, D.. 2002a. “Choosing Rural Road Invest- sity Press for the World Bank. ments to Help Reduce Poverty.” World Develop- Stone, R. 1954. “Linear Expenditure Systems and ment 30 (4): 575–89. Demand Analysis: An Application to the Pattern van de Walle, D. 2002b, “The Static and Dynamic Inci- of British Demand.” Economic Journal 64: 511–27. dence of Viet Nam’s Public Safety Net,” Policy Strauss, J. 1984. “Joint Determination of Food Con- Research Working Paper 2791. World Bank, Wash- sumption and Production in Rural Sierra Leone: ington, D.C. Estimates of a Household-Firm Model.” Journal of van de Walle, D. 2002c. “Poverty and Transfers in Development Economics 14: 77–103. Yemen”. Middle East and North Africa Working Subramanian, S., and A. Deaton. 1996. “The Demand Paper 30, World Bank, Washington, DC. for Food and Calories.” Journal of Political Econ- van de Walle, D. 2003. “Behavioral Incidence Analysis omy 104: 133–62. of Public Spending and Social Programs.” In F. 89 A User’s Guide to Poverty and Social Impact Analysis Bourguignon and L.A. Pereira da Silva, eds., Eval- World Bank. 2002c. Social Analysis Sourcebook: Incorporat- uating the Poverty and Distributional Impact of ing Social Dimensions into Bank-Supported Projects. Economic Policies (Techniques and Tools), Wash- Social Development Department, Washington, D.C. ington D.C.: World Bank. World Bank. 2002d. Filipino Report Card on Pro-Poor Wodon, Q., K. Ramadas and D. van der Mens- Services: Summary, ESSD Unit, East Asia and brughghe. 2003. SimSIP Poverty Module, World Pacific Region, 2002. Washington D.C. Processed. Bank. Washington, D.C. World Bank. 2002e. “Program Document for a Poverty World Bank. 2000a. World Development Report Reduction Support Credit in the amount equiva- 2000/2001: Attacking Poverty. New York: Oxford lent to SDR 9.1 million to the Cooperative Repub- University Press. lic of Guyana.” Washington D.C. Processed. World Bank. 2000b. “Modeling Pensions Reform: The Xiao, Y., and S. Canagarajah. 2002. “Efficiency of Pub- World Bank’s Pension Reform Options Simula- lic Expenditure Distribution and Beyond: A tion Toolkit.” Human Development Network, Report on Ghana’s 2000 Public Expenditure Social Promotion, Washington, D.C. Tracking Survey in the Sectors of Primary Health and Education.” Africa Region Working Paper World Bank. 2001a. “Malawi Public Expenditures: Series 31. World Bank, Washington, D.C. Issues and Options.” Report 22440 MAI. Washing- ton, D.C. Yitzhaki, S., and J. Slemrod. 1991. “Welfare Domi- nance: An Application to Commodity Taxation.” World Bank. 2001b. “Honduras: Public Expenditure American Economic Review 81 (3): 480–96. Management for Poverty Reduction and Fiscal Sustainability.” Report 22070. Poverty Reduction Younger, S. 1993. “Estimating Tax Incidence in Ghana: and Economic Sector Management Unit, Wash- An Exercise using Household Data.” Cornell Food ington, D.C. and Nutrition Policy Program Working Paper 48. Ithaca. Processed. World Bank. 2002a. “Monitoring and Evaluation: Some Tools, Methods, and Approaches.” Opera- Younger, S., D.E. Sahn, S. Haggblade, and P.A. Dorosh. tions Evaluation Department, Washington, D.C. 1999. “Tax Incidence in Madagascar: An Analysis Using Household Data.” World Bank Economic World Bank. 2002b. “Bosnia and Herzegovina Local Review 13 : 303-331. Level Institutions and Social Capital Study: Find- ings and Recommendations.” Europe and Central Younger, S. 2002. “Benefits on the Margin: Observa- Asia, Environment and Socially Sustainable tions on Marginal Benefit Incidence”. Food and Development Department, Washington, D.C. Nutrition Policy Program. Cornell University. Processed. Ithaca, NY. Processed. 90