February 2016 I N S I D E Fink, Maskus, and Qian on The Economic Effects of Counterfeiting and Piracy: A Review and Implications for Developing Countries The World Bank Research Observer Brück et al. on Measuring Violent Conflict in Micro-level Surveys: Current Practices and Methodological Challenges T H E W O R L D B A N K ´ and Furst-Nichols on Buvinic Promoting Women’s Economic Research Empowerment: What Works? Cunningham and Villaseñor on Employer Voices, Employer Demands, and Implications for Observer Public Skills Development Policy Connecting the Labor and Education Sectors Gentilini on Revisiting the “Cash versus Food” Debate: Volume 31 • Number 1 • February 2016 New Evidence for an Old Puzzle? Volume 31, Issue 1 Lederman and Porto on The Price Is Not Always Right: On the Impacts of Commodity Prices on Households (and Countries) ISSN 0257-3032 (PRINT) 2 ISSN 1564-6971 (ONLINE) www.wbro.oxfordjournals.org Downloaded from https://academic.oup.com/wbro/issue/31/1 by Joint Bank-Fund library user on 06 October 2017 T H E WO R L D BA N K Research Observer EDITOR Emmanuel Jimenez, World Bank CO-EDITOR Luis Servén, World Bank EDITORIAL BOARD Harold Alderman, International Food Policy Research Institute Barry Eichengreen, University of California-Berkeley Marianne Fay, World Bank Jeffrey S. Hammer, Princeton University Ravi Kanbur, Cornell University Ana L. Revenga, World Bank Ann E. Harrison, University of Pennsylvania The World Bank Research Observer is intended for anyone who has a professional interest in development. Observer articles are written to be accessible to nonspecialist readers; con- tributors examine key issues in development economics, survey the literature and the lat- est World Bank research, and debate issues of development policy. Articles are reviewed by an editorial board drawn from across the Bank and the international community of econo- mists. Inconsistency with Bank policy is not grounds for rejection. The journal welcomes editorial comments and responses, which will be considered for pub- lication to the extent that space permits. On occasion the Observer considers unsolicited contributions. Any reader interested in preparing such an article is invited to submit a proposal of not more than two pages to the Editor. Please direct all editorial correspon- dence to the Editor, The World Bank Research Observer, 1818 H Street, NW, Washington, DC 20433, USA. The views and interpretations expressed in this journal are those of the authors and do not necessarily represent the views and policies of the World Bank or of its Executive Directors or the countries they represent. The World Bank does not guarantee the accuracy of data included in this publication and accepts no responsibility whatsoever for any consequences of their use. When maps are used, the boundaries, denominations, and other information do not imply on the part of the World Bank Group any judgment on the legal status of any territory or the endorsement or acceptance of such boundaries. For more information, please visit the Web sites of the Research Observer at www.wbro.oxfordjournals.org, the World Bank at www.worldbank.org, and Oxford University Press at www.oxfordjournals.org. Downloaded from https://academic.oup.com/wbro/issue/31/1 by Joint Bank-Fund library user on 06 October 2017 Subscriptions A subscription to The World Bank Research Observer (ISSN 0257-3032) comprises 2 issues. Prices include postage; for subscribers outside the Americas, issues are sent air freight. Annual Subscription Rate (Volume 31, 2 issues, 2016) Academic libraries Print edition and site-wide online access: US$240/£161/E240 Print edition only: US$221/£146/E221 Site-wide online access only: US$177/£118/E177 Corporate Print edition and site-wide online access: US$364/£242/E364 Print edition only: US$334/£222/E334 Site-wide online access only: US$269/£177/E269 Personal Print edition and individual online access: US$66/£45/E66 Please note: US$ rate applies to US & Canada, EurosE applies to Europe, UK£ applies to UK and Rest of World. Readers with mailing addresses in non-OECD countries and in socialist economies in transition are eligible to receive complimentary subscriptions on request by writing to the UK address below. There may be other subscription rates available; for a complete listing, please visit www.wbro.oxfordjournals.org/subscriptions. Full pre-payment in the correct currency is required for all orders. Payment should be in US dollars for orders being delivered to the USA or Canada; Euros for orders being delivered within Europe (excluding the UK); GBP sterling for orders being delivered elsewhere (i.e., not being delivered to USA, Canada, or Europe). All orders should be accompanied by full payment and sent to your nearest Oxford Journals office. Subscriptions are accepted for complete volumes only. Orders are regarded as firm, and payments are not refundable. Our prices include Standard Air as postage outside of the UK. Claims must be notified within four months of despatch/order date (whichever is later). Subscriptions in the EEC may be subject to European V AT. If registered, please supply details to avoid unnecessary charges. For subscriptions that include online versions, a proportion of the subscription price may be subject to UK V AT. Subscribers in Canada, please add GST to the prices quoted. Personal rate subscriptions are only available if payment is made by personal cheque or credit card, delivery is to a private address, and is for personal use only Back issues: The current year and two previous years’ issues are available from Oxford University Press. Previous volumes can be obtained from the Periodicals Service Company, 11 Main Street, Germantown, NY 12526, USA. E-mail: psc@periodicals.com. Tel: (518) 537-4700. Fax: (518) 537-5899. Contact information: Journals Customer Service Department, Oxford University Press, Great Clarendon Street, Oxford OX2 6DP , UK. E-mail: jnls.cust.serv@oup.com. Tel: þ 44 (0)1865 353907. Fax: þ 44 (0)1865 353485. In the Americas, please contact: Journals Customer Service Department, Oxford University Press, 2001 Evans Road, Cary, NC 27513, USA. E-mail: jnlorders@oup.com. Tel: (800) 852-7323 (toll-free in USA/Canada) or (919) 677-0977. Fax: (919) 677- 1714. In Japan, please contact: Journals Customer Service Department, Oxford University Press, 4-5-10-8F Shiba, Minato-ku, Tokyo, 108-8386, Japan. E-mail: custserv.jp@oup.com. Tel: þ 81 3 5444 5858. Fax: þ 81 3 3454 2929. Postal information: The World Bank Research Observer (ISSN 0257-3032) is published twice a year, in February and August, by Oxford University Press for the International Bank for Reconstruction and Development/THE WORLD BANK. Postmaster: send address changes to The World Bank Research Observer, Journals Customer Service Department, Oxford University Press, 2001 Evans Road, Cary, NC 27513-2009. Periodicals postage paid at Cary, NC and at additional mailing offices. Communications regarding original articles and editorial management should be addressed to The Editor, The World Bank Research Observer, The World Bank, 1818 H Street, NW, Washington, D.C. 20433, USA. Environmental and ethical policies: Oxford Journals, a division of Oxford University Press, is committed to working with the global community to bring the highest quality research to the widest possible audience. Oxford Journals will protect the environment by implementing environmentally friendly policies and practices wherever possible. Please see http://www. oxfordjournals.org/ethicalpolicies.html for further information on environmental and ethical policies. Digital Object Identifiers: For information on dois and to resolve them, please visit www.doi.org. Permissions: For information on how to request permissions to reproduce articles or information from this journal, please visit www.oxfordjournals.org/jnls/permissions. Advertising: Advertising, inserts, and artwork enquiries should be addressed to Advertising and Special Sales, Oxford Journals, Oxford University Press, Great Clarendon Street, Oxford, OX2 6DP , UK. Tel: þ 44 (0)1865 354767; Fax: þ 44(0)1865 353774; E-mail: jnlsadvertising@oup.com. Disclaimer: Statements of fact and opinion in the articles in The World Bank Research Observer are those of the respective authors and contributors and not of the International Bank for Reconstruction and Development/THE WORLD BANK or Oxford University Press. Neither Oxford University Press nor the International Bank for Reconstruction and Development/THE WORLD BANK make any representation, express or implied, in respect of the accuracy of the material in this journal and cannot accept any legal responsibility or liability for any errors or omissions that may be made. The reader should make her or his own evaluation as to the appropriateness or otherwise of any experimental technique described. Paper used: The World Bank Research Observer is printed on acid-free paper that meets the minimum requirements of ANSI Standard Z39.48-1984 (Permanence of Paper). Indexing and abstracting: The World Bank Research Observer is indexed and/or abstracted by ABI/INFORM, CAB Abstracts, Current Contents/Social and Behavioral Sciences, Journal of Economic Literature/EconLit, PAIS International, RePEc (Research in Economic Papers), Social Services Citation Index, and Wilson Business Abstracts. Copyright # 2016 The International Bank for Reconstruction and Development/THE WORLD BANK All rights reserved; no part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, or otherwise without prior written permission of the publisher or a license permitting restricted copying issued in the UK by the Copyright Licensing Agency Ltd, 90 Tottenham Court Road, London W1P 9HE, or in the USA by the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923. Typeset by Techset Composition Limited, Chennai, India; Printed by The Sheridan Press, USA Downloaded from https://academic.oup.com/wbro/issue/31/1 by Joint Bank-Fund library user on 06 October 2017 T H E WO R L D B A N K Research Observer Volume 31 † Number 1 † February 2016 The Economic Effects of Counterfeiting and Piracy: A Review and Implications for Developing Countries Carsten Fink, Keith E. Maskus, and Yi Qian 1 Measuring Violent Conflict in Micro-level Surveys: Current Practices and Methodological Challenges ¨ ck, Patricia Justino, Philip Verwimp, Alexandra Avdeenko, Tilman Bru and Andrew Tedesco 29 Promoting Women’s Economic Empowerment: What Works? ´ and Rebecca Furst-Nichols Mayra Buvinic 59 Employer Voices, Employer Demands, and Implications for Public Skills Development Policy Connecting the Labor and Education Sectors Wendy V. Cunningham and Paula Villasen ˜ or 102 Revisiting the “Cash versus Food” Debate: New Evidence for an Old Puzzle? Ugo Gentilini 135 The Price Is Not Always Right: On the Impacts of Commodity Prices on Households (and Countries) Daniel Lederman and Guido Porto 168 Downloaded from https://academic.oup.com/wbro/issue/31/1 by Joint Bank-Fund library user on 06 October 2017 The Economic Effects of Counterfeiting and Piracy: A Review and Implications for Developing Countries Carsten Fink, Keith E. Maskus, and Yi Qian Policy makers around the world recognize the potentially harmful consequences of trade- mark counterfeiting and copyright piracy. Indeed, many countries have recently initiated policy reforms to strengthen the enforcement of intellectual property rights (IPR). Further, minimum standards of enforcement have been incorporated in many international treaties, especially trade agreements. This emphasis on enforcement raises basic questions about the actual impacts of IP rights infringement, which differ across the types of IPR and economic sectors. We review the academic literature and other studies in the public domain to evaluate what has been learned about these socioeconomic effects, with an empha- sis on developing countries where possible. We also identify important gaps in our understand- ing of the consequences of counterfeiting and piracy and develop recommendations on how governments might collect data and conduct studies to better inform IPR enforcement policy. Counterfeiting, Piracy, Intellectual property rights . JEL codes: L14, O31, O34 Intellectual property rights (IPR) cover a broad array of legal rules affording individ- uals and enterprises, or their licensees, exclusive opportunities to make, copy, and sell products and technologies arising from their inventive and creative work. These rules range from patents on new inventions to trademarks and geographical indica- tions certifying the origin of goods. These rules also include copyright on creative and literary works, including software and digital transmissions. Because these rights are national in scope, how they are defined and protected varies considerably across countries for a variety of reasons (Maskus 2012). Despite their broad similarity as a means of supporting markets for knowledge, various forms of IPR embody different economic and social objectives. It follows The World Bank Research Observer # The Author 2015. Published by Oxford University Press on behalf of the International Bank for Reconstruction and Development / THE WORLD BANK. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com doi:10.1093/wbro/lkv010 Advance Access publication May 7, 2015 31:1–28 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 that their infringement, or actions taken by third parties in violation of exclusive rights, interferes in varying degrees with attaining these objectives. Producing an imitative good that infringes on one’s patent rights may diminish dynamic incen- tives to invest in R&D, for example, though it can also bolster market competition in a static sense. Moreover, there is an important international element: if IPR infringement weakens the profitability of largely foreign enterprises, why would governments choose to invest in costly enforcement activities? The particular forms of IPR infringement that have attracted the greatest global policy interest are counterfeiting, which is the illegal use of trademarks to sell unau- thorized goods, and piracy, which is the duplication and sale of copyrighted goods without the copyright holder’s permission. Policy makers around the world recog- nize the potentially harmful consequences of trademark counterfeiting and copy- right piracy. First, infringements of intellectual property (IP) rights undermine trust in rules-based systems that are at the core of modern market-based economies. Second, counterfeit products can confuse consumers and may be ineffective or even dangerous, as with adulterated medicines and foodstuffs or non-certified engine re- placement parts. Third, extensive counterfeiting can significantly deter the emer- gence of new products and firms, and piracy can block the development of creative industries, especially in poor countries where entry conditions are already weak. Over the past two decades, many governments—including those of developing countries—have initiated policy reforms to strengthen the enforcement of IPR (Maskus 2012). In addition, minimum standards of enforcement have also been incorporated in many international treaties, especially trade agreements, as IPR owners in wealthier economies push for stronger global protection. For example, the terms of the Trans-Pacific Partnership (TPP), while still under negotiation, embody extensive expectations for stronger border and domestic enforcement. However, instituting a policy framework for combating counterfeiting and piracy poses substantial challenges. The economic effects of infringements differ across dif- ferent IPR and economic sectors (Fink 2009), and it is important to understand these differences at the theoretical and empirical levels to set priorities for IPR en- forcement policy. Governments are invariably resource-constrained, and completely eradicating violations of IP law—similar to violations of other types of law—is out of reach for even the best-resourced states. This is especially true for developing countries, where many public goods are underprovided and enforcement challenges exist in many areas of law. Thus, in this article we review the academic literature and other studies in the public domain to evaluate what is known about the socioeconomic effects of coun- terfeiting and piracy.1 We also identify important gaps in the understanding of these effects and develop recommendations on how governments might collect data and conduct studies as inputs into the formulation of IPR enforcement policy. For the most part, we ignore differences and limitations in the effectiveness of IPR 2 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 enforcement measures, which are equally important but warrant a separate discus- sion. Where relevant, we draw inferences from these studies about important impli- cations for economic development and policy making in developing countries. In particular, we highlight the importance of collecting better and more consistent data, conducting microeconomic studies of the impacts of infringements on local firms, and raising consumer awareness of the potential externalities they impose on domestic producers and the potential risks consumers face with sub-standard products. Governments may also wish to develop an impact evaluation framework to assess their strategic investments in enforcement activities. This could be particu- larly important in light of growing external pressures for stronger efforts in the context of regional trade agreements. The paper is structured as follows. In the next section, we approach the topic from a theoretical perspective, first reviewing the different market failures under- lying trademark and copyright protection, and then exploring what happens when these rights are violated. The discussion concludes with brief suggestions for future theoretical research. In the next section we review existing empirical ev- idence, including aggregate estimates of levels of counterfeiting and piracy, studies of the effects of IPR infringements in particular industries, and anecdotal evidence from selected case studies. In the final section we explore what the findings of the literature imply for policymaking in developing countries, and develop recommen- dations for data collection and the conduct of future studies on counterfeiting and piracy. Theoretical Considerations In general, intellectual property rights aim to remedy the failure of markets to provide an efficient allocation of resources. The relevant market failures that give rise to copyright and trademark protection vary. Accordingly, the effects of trade- mark counterfeiting and copyright piracy vary and warrant separate treatments. Economics of Trademark Counterfeiting The primary welfare effects of counterfeiting depend crucially on whether consum- ers are deceived into believing that a fake good is produced by the owner of the trademark. The likelihood of this deception depends on product characteristics and the nature of distribution channels. For example, consumers are usually unable to ascertain whether a pharmaceutical product contains the desired chemical ingredi- ent, whereas most fashion shoppers can distinguish an original handbag from its fake clone. We discuss the implications of deceptive and non-deceptive counterfeit- ing in turn. Fink et al. 3 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 Deceptive Counterfeiting. The most relevant market failure giving rise to trademark protection arises from asymmetric information. For many goods, consumers do not have enough information to ascertain the quality of a potential purchase. A trade- mark guarantees that a product or a service originated with a particular producer. This is valuable information to consumers because it reduces both uncertainty and the costs of searching for particular quality levels. Indeed, producers compete by es- tablishing a reputation for different levels of quality. In turn, trademarks serve as an indication of product quality, a crucial support for functioning markets. Thus, trademarks, geographical indications, and similar rights enable high-quality pro- ducers to distinguish themselves in the market, supporting investments in improved product or service quality. Deceptive counterfeiters copy trademarks, logos, and designs to confuse consumers into believing they are buying the legitimate product. In the presence of information asymmetries, deceptive counterfeiting is virtually certain to reduce economic welfare (Grossman and Shapiro 1988a). Those who discover they bought a fake good realize less consumption value than the price they paid for it (Liu et al. 2005). Rational consumers, aware that fake goods are on the market but are indistinguishable from originals, are unwilling to pay the full price of a high-quality good. This problem un- dermines the incentive for producers to invest in higher quality and may destroy markets for high-quality goods (Akerlof 1970; Qian et al. 2013). For developing countries, where deceptive counterfeiting sometimes reaches high levels, at least two significant costs arise (Maskus 2012). First, extensive anecdotal evidence suggests that the likelihood of infringement deters market entry by local enterprises, thus limiting the growth of small and medium-sized enterprises. In this context, counterfeiting can be anti-competitive in a dynamic sense, even in poor countries. Second, for certain products, the consumption of deceptive fakes can generate negative externalities. Examples would be an increased risk of disease transmission in the case of pharmaceuticals, or environmental degradation in the case of indus- trial chemicals. Indeed, the distribution of counterfeit drugs poses a significant public health challenge in many developing countries, especially in Africa.2 The presence of negative externalities calls for public action against counterfeit prod- ucts, independent of any private incentive for enforcing trademark rights. Non-deceptive Counterfeiting. A more complex situation arises where consumers are perfectly able to ascertain the quality attributes of goods at the moment of purchase and therefore know whether they are buying a counterfeit. Because no information asymmetry prevails, trademarks, at first sight, seem irrelevant. However, for so- called status goods the display of the producer’s name confers prestige on the pur- chaser, yielding utility beyond that from the goods’ functional characteristics. Counterfeiting of status goods can reduce the prestige value of genuine versions. 4 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 Such value is real, as is manifested by the substantial marketing outlays of produc- ers of luxury goods and the prevalence of counterfeits. Grossman and Shapiro (1988b) analyze the effects of non-deceptive product counterfeiting. In their simplified model, status value depends negatively on the number of consumers who purchase a product bearing the same brand name, whether genuine or fake. In this setting, the social welfare consequences of product counterfeiting become theoretically ambiguous. Producers and consumers of the genuine product are made worse off by counterfeits because their presence reduces the status value, and therefore the price, of the genuine product. However, produc- ers and consumers of the counterfeit product are better off. In particular, consum- ers of fake products gain status value without paying the full price of the genuine product. The overall effect on social welfare is an empirical question. Strategic Responses to Counterfeiting. More recent analysis has identified further in- teresting features of markets with fake goods. For example, Qian (2014a) models the impacts of entry by counterfeiters on the subsequent responses of genuine pro- ducers. Because counterfeits are usually an inferior imitation of the authentic product, the theory adopts a vertical differentiation setup. In particular, the theory incorporates two layers of asymmetric information potentially generated by coun- terfeits: (1) counterfeiters fool buyers, and (2) buyers of counterfeits fool their peer consumers by pretending to have high-status goods. One key prediction is that entry by counterfeiters would induce a genuine producer to upgrade product quality and raise the product’s price if and only if the entrant’s quality is lower than a threshold level. In essence, genuine trademark owners must raise quality and price (and reduce sales) to attract demand in the market, which is consistent with evidence from China discussed in the next section. Note, however, that the possibili- ty that original producers may invest in higher quality versions to compete with counterfeiters does not imply higher economic welfare, or that the induced innova- tion is efficient. Qian et al. (2013) further show that competition from counterfeiting induces branded firms to invest more in improving searchable quality attributes such as appearance, and less in enhancing experiential attributes, such as functionality. Competition from counterfeit products can either lower the prices of genuine goods due to substitution in demand or raise their prices and production costs in order to move up-market. In addition, Qian (2014a) demonstrates how authentic producers could invest in “self-enforcement” strategies to limit the competition from counterfeit goods. Such strategies include conspicuous packaging of products and specially licensed company stores.3 Longer-term Welfare Considerations. Counterfeiting can encourage certain forms of innovation among genuine producers in an attempt to differentiate their products from imitations. This process is costly, however, perhaps especially in poor countries Fink et al. 5 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 where the margin of high-income consumers seeking high-quality genuine goods is relatively small. More fundamentally, under both deceptive and non-deceptive coun- terfeiting, producers of genuine products realize lower profits. Facing this problem, original trademark owners become less willing to invest in developing new goods, ultimately reducing the rate of new product development and lowering both entry of new firms and consumer welfare (Aghion et al. 2005; Scherer 1967). We con- clude that on balance, extensive counterfeiting is likely to harm consumers and impede growth in poor and emerging economies. Economics of Copyright Piracy The need for copyright protection comes from three characteristics associated with creative works such as books, music, films, and software. First, these products can be reproduced at low marginal cost; in the case of the Internet, reproduction occurs at virtually zero marginal cost. The second characteristic is that their consumption is non-rival; many people can enjoy them without diminishing availability for others. The third characteristic of such works is that the creative process may require costly sunk investments. If competitive market forces were to provide crea- tive works at marginal cost, actors, artists, authors, producers, and publishers would have little financial incentive to undertake those investments. The copyright offers a solution to this problem by affording producers of creative works the exclu- sive rights to authorize and control their reproduction. In turn, genuine copies may be sold at a profit, allowing for the financing of creative investments.4 In light of this essential economic rationale for protection, what are the implica- tions of copyright piracy? The basic answer is that high levels of copyright infringe- ments undermine the incentives for creating and licensing new works, ultimately lowering social welfare. Unauthorized copying tends to flourish where originals command a high price, copying is cheap (as with digital goods and internet trans- missions), and the likelihood of detection or meaningful punishment is low. In such circumstances, widespread copying can significantly diminish the incentives to develop and produce new creative and literary works. This barrier seems particular- ly relevant for indigenous music and software industries in developing countries with high rates of piracy, where copiers pervasively target local performers and emerging enterprises (Maskus 2000; Penna et al. 2004). Beyond this basic framework a number of theoretical nuances should be men- tioned. For example, theory suggests that piracy can raise or lower the price of origi- nal works (Besen and Kirby 1989; Takeyama 1997). If consumers who prefer to buy originals have sufficiently inelastic demand, it is possible for the prices they pay to be higher in the presence of pirated copies. The reason is that copying would induce original producers to concentrate their sales on this group, which pushes up 6 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 price. Overall sales of legitimate copies are likely to be lower, however, because copy- right owners would choose not to sell to the remaining market segment. An example is the model by Harbaugh and Khemka (2010), who point out that if copyright enforcement efforts are focused on larger users with higher valuations for software, such as public enterprises, universities, and government agencies, the original IP owners can charge higher-than-monopoly prices to those users. The re- maining users in the market then purchase pirated copies. When enforcement is spread more broadly throughout the market, however, original producers will sell more legitimate copies and prices will fall, even as the volume of pirated goods di- minishes. In this situation, it is possible to observe more genuine goods on the market and an increase in overall consumer benefits.5 Another reason prices may rise in the face of piracy is that unauthorized copying may induce copyright holders to add features and functionality to genuine copies to distinguish them in the marketplace. In turn, these features support higher prices, which consumers of originals are willing to pay, whereas pirated copies command lower prices (Varian 2000). An early variant of this concept is detailed by Liebowitz (1985), who presents empirical evidence that academic journals raised their prices after the introduction of photocopying because they had become more valuable to consumers. A similar process is underway in academic publishing in the digital age, with dramatic increases in subscription prices for paper copies (Maskus 2012). Finally, note that books, journals, music, and videos are “information goods” that can be shared across multiple uses through copying, rentals, and literal sharing. This fact affects strategies of the content producers and copyright owners, who must decide the formats, timing of releases, functionality, and prices of such goods. According to Varian (2000), copyright owners are likely to sell fewer copies at higher prices to manage the sharing problem. This strategy is likely to be profit- able when there are low transaction costs in sharing (e.g., movie rentals) and when sharing markets permits separation of higher-valuation and lower-valuation users. The former will receive goods faster at higher prices, and the latter will have delayed access at lower prices or will enter lower-quality and, perhaps, pirated markets. Network Effects. Piracy takes on interesting features in the presence of goods with network externalities, in which one consumer’s valuation of a creative work in- creases with the number of other consumers owning the same product. Computer programs are an example, as the ability to share files with other users increases the attractiveness of a particular software package. What happens if creative works subject to network externalities are pirated? Suppose that original and pirated copies are perfect substitutes, or at least highly in- teroperable as in the case of software. Consumers of original copies gain because the presence of pirated copies enlarges the network with which they can interact. In addi- tion, sonsumers of pirated copies will benefit from inexpensive access to the work. Fink et al. 7 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 The effects of copyright piracy on the producer of the creative work and overall welfare are ambiguous. As Conner and Rumelt (1991) and Takeyama (1994) show, it is theoretically possible for the copyright owner to reap higher profits from certain levels of piracy. The intuition is that the enhanced network value of the product may allow producers to charge a higher price for originals. Here, copyrights support profitable price discrimination. In theory, the copyright owner could achieve the same outcome by simply giving away a certain number of original works. In practice, this would not work because all consumers would expect a free legitimate copy. However, consumers differ in their willingness to use pirated copies, as determined, for example, by income levels. This difference allows the producer to segment the market and choose the profit-maximizing combination of price and network size.6 Effects on Tax Revenue and Employment In policy discussions on trademark counterfeiting and copyright piracy, effects on tax revenue and employment have gained in importance, especially in richer coun- tries where firms see their IPR infringed on abroad. From an economic perspective, these effects can be evaluated either in a shorter-term partial-equilibrium setting or a longer-term general-equilibrium setting. The overall partial-equilibrium employment effects are theoretically straightfor- ward and depend on how output shifts between genuine and illicit producers and the labor intensities of original and pirated copies. The same holds for tax revenues, which are nearly certain to fall because sales of pirated and counterfeit products occur in informal markets where taxes and import tariffs are usually not collected, and because sales of legitimate producers likely decline. No existing theoretical re- search examines the tradeoffs between employment in the formal and illicit infor- mal sectors, including the setting of wages, the existence and impact of social safety nets, and the nature and length of possible unemployment spells. The long-term general-equilibrium effects of IPR infringement are often ignored in policy discussions, but understanding them better is crucial. Again, direct im- pacts in affected sectors are surely negative. However, workers losing employment likely find other jobs, and governments facing a revenue shortfall likely adjust their tax structure to finance public spending. The key question is how workers and the efficiency of the tax system fare in the counterfactual equilibrium. No formal model- ing work seems to exist in the literature that would offer conceptual guidance. Impacts on Trade, Foreign Direct Investment, and Licensing The theoretical literature on the relationships between IPR protection and inter- national trade, FDI, and licensing is deep and sophisticated.7 These linkages 8 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 are important because they govern how policy reforms and enforcement affect market-based inward technology transfer (ITT) into developing economies. However, economic theory in this area focuses almost exclusively on IPR as technol- ogy protection, for example patents and trade secrets, and has ignored trademark counterfeiting and copyright piracy. Further, no models have considered the role of stronger enforcement as being distinct from basic legal reforms. Thus, a comprehensive review lies beyond the scope of this paper and we simply state the primary theoretical findings. Because multiple channels exist for transfer- ring technology, IPR in the recipient country has both scale effects and composition effects. Regarding scale, multinational enterprises (MNEs) seeking to deploy a new technology or sell a new good face a basic tradeoff when local IPR is enhanced (Maskus and Penubarti 1995). Because local firms face higher imitation costs, their outputs are reduced and MNEs sell more, whether through trade or FDI. At the same time, MNEs face less elastic demand from consumers who continue to buy their goods, thus permitting higher prices at reduced (monopolistic) volumes. This tradeoff between market expansion and market power is fundamental to the theory of intellec- tual property and ITT. This tradeoff extends to licensing as well: MNEs can expand the number of contracts for technology use and production rights but they can also raise royalty rates (Branstetter et al. 2006). The key point is that IPR reforms have ambigu- ous impacts on ITT, at least in theory, making it an empirical question. As for the composition of ITT, economic theory suggests that MNEs shift their em- phasis from exports at weak IPR levels to FDI and then licensing as protection im- proves (Markusen 2001; Javorcik 2004). This theory assumes (with little evidence) that imported high-technology goods are relatively difficult to imitate and thus carry their own protection. As IPR is strengthened, MNEs gain more confidence that local affiliate production cannot readily be copied, and become more willing to take advantage of cost reductions through FDI. However, technology remains inter- nal to the firm for purposes of control. At yet higher protection levels, MNEs become willing to license their technologies to local partners, thereby externalizing their knowledge assets. A related factor is that stronger patents and trademarks increase the confidence that contracts are enforceable, making arm’s-length licensing more attractive. One additional strategic response (among many) is worth mentioning here. Specifically, where MNEs fear losing their technological advantages to imitation and their trademark reputations to counterfeiting, they may fragment their produc- tion among multiple local plants at various locations to diminish information flows among them and gain more ability to deter infringement. This approach, which has been common among MNEs operating in China (Maskus 2000), is inefficient and likely reduces net technology transfer. For example, Sun et al. (2010) demonstrate theoretically that an MNE worried about imitation in an emerging market may limit the range of technologies it offers to deter entry by local imitators. Fink et al. 9 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 As mentioned, there are no theoretical studies focusing on FDI, licensing, and ITT with respect to counterfeiting and piracy, our focus here. However, many of the tradeoffs above surely apply. For example, reducing trademark infringement can lower the local selling costs of MNEs and expand their markets, whether through trade or FDI. Regional fragmentation of production is a costly reaction to counter- feiting, as is the need to engage in private enforcement. And, because of weak copy- rights, MNEs may deploy expensive software to deter copying or pay more for programs on the market, as noted above. Presumably, these factors are deterrents that companies take into account when deciding on investment locations. It is important to note that high-technology imports and FDI support localized learning and productivity gains in developing countries (Keller 2004; Branstetter et al. 2006), which are important for supporting productivity growth. To the extent that weak IPR diminishes such prospects, they have a negative dynamic impact on development. Recommendations for Future Theoretical Research While the economic literature offers useful theoretical guidance on the economic effects of counterfeiting and piracy, several issues deserve further exploration. We highlight three such issues here. First, as we described previously, the presence of (non-deceptive) counterfeit goods exerts an externality on consumers of originals. In their analysis, Grossman and Shapiro (1988b) assume that this externality is negative and takes the form of counterfeits diluting the exclusivity value of status goods. This assumption is appropriate for some classes of goods but not for others, and survey evidence suggests there is substantial heterogeneity in consumers’ decisions to purchase counterfeit versus authentic products Wilcox et al. (2009). Other types of externalities are conceivable, and it would be important to analyze how their presence affects consumers, the strategic reactions of producers, and overall welfare. Second, hardly any theoretical study has formally analyzed the short- and long- term employment effects of counterfeiting and piracy. It would be especially important to obtain conceptual guidance on how the different nature of formal and informal labor markets affects the welfare of different types of workers. Such guid- ance would seem especially important for developing countries, where informal employment often accounts for a larger share of total employment. Developing a better understanding of broader labor-market linkages may also be important for designing effective IP enforcement policies Fink (2009). Finally, it would be interesting to focus theoretical attention specifically on coun- terfeiting and piracy in terms of their potential impacts on technology transfer and localized learning. Little is known, for example, about how the enforceability of IPR affects relationships between MNEs and local suppliers and distributors. Nor is there 10 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 much analysis of how piracy and counterfeiting influence global franchising deci- sions, which often embody considerable knowhow regarding market organization. Review of Empirical Evidence Economic theory is helpful for rigorously examining the likely effects of trademark counterfeiting and copyright piracy. However, as we noted previously, some effects are theoretically ambiguous and require empirical investigation. In addition, even when the overall direction of effects is conceptually clear, empirical insights into the scale of effects are needed for policy makers to set priorities for public policy. Notwithstanding the need, generating credible empirical evidence poses a signifi- cant challenge. Counterfeiting and piracy, by their very nature, are illegal activities and thus largely escape official statistical recording. The basis for empirical research is thus weak, and studies must rely on indirect official data, selected information supplied by rights holders, and original surveys to gather evidence. In addition, welfare impacts such as producer and consumer surplus gains or losses are difficult to measure at both the individual and the aggregate societal level. Finally, the limited empirical evidence that is available comes mostly from developed countries. While this evidence can still inform policy makers in developing countries, there are special characteristics of poorer economies that caution against the direct applica- bility of this evidence—notably, different parameters of consumer demand and dif- ferent legal and institutional frameworks. In what follows we critically review relevant empirical studies, and focus on the extent to which the underlying methodologies produce credible results. We begin with a review of aggregate estimates of the level of counterfeiting and piracy. We then examine studies that have evaluated the welfare impact of these activities at the micro level, and review the studies that have specifically focused on the effects of Internet file sharing. We conclude by evaluating the advantages and disadvantages of alternative empirical approaches. Aggregate Estimates of Levels of Counterfeiting and Piracy We begin the review of aggregate estimates by considering efforts to ascertain the incidence of piracy and counterfeiting in international trade. Focusing on trade seems promising, because goods undergo processing when they cross borders. In particular, customs authorities investigate suspected infringements of IP rights, thereby generating indirect data on the incidence of pirated and counterfeit goods in different product categories and for different exporting and importing economies. The most comprehensive study attempting to quantify the importance of coun- terfeit and pirated goods in international trade was conducted by the OECD (2008). Fink et al. 11 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 This study estimated the value of pirated and counterfeit goods in international trade at U.S. $200 billion, or approximately 2% of global merchandise trade in 2005. In an update of this study, the OECD (2009) estimated the value of illicit goods in international trade as having grown to $250 billion in 2007 (largely re- flecting the growth in worldwide trade). These estimates do not include the value of Internet-related piracy of digital goods, which easily cross borders but are almost impossible to measure and, in any case, cannot be counted in any merchandise trade categories. The OECD (2008) report recognizes that its estimates are only “a crude indicator” of the value of counterfeit and pirated products in international trade. This caution is warranted. For instance, the OECD estimate employs data on interceptions and seizures by customs authorities in selected countries to estimate the proportion of counterfeit and pirated goods in different product groups and across exporting economies. However, these values only provide information about the relative inci- dence of pirated and counterfeit goods across product groups or across exporting economies. To arrive at their estimates, OECD staff use a value of the absolute inci- dence of counterfeit and pirated goods in one particular “fix-point” product group (e.g., apparel, leather articles, tobacco products). The value in question is not based on any hard data, but rather seems to reflect the best guess of OECD staff.8 Notwithstanding these methodological caveats, the evidence on the relative incidence of counterfeit and pirated goods across product groups is insightful. It shows, for example, that trade in illicit goods is concentrated in a small number of “sensitive” product categories; the top five product groups (at the two-digit Harmonized System level) account for more than three-quarters of all customs seizures. Industry Studies. Moving beyond trade, a large number of studies—often sponsored by affected industries—attempt to quantify the incidence of pirated and counterfeit goods at the level of specific sectors and to estimate their effects on certain economic performance variables. The methodological approaches and quality of the data in these studies vary substantially. A detailed review of each of these studies would go beyond the scope of this article, but it is worth highlighting several common issues. First, given the inherent difficulty of accurately measuring the prevailing levels of counterfeiting and piracy, the reliability of any underlying estimate is in many cases questionable. This difficulty applies to the results of original survey work. For example, industry associations in the major copyright-intensive industries publish annual surveys of piracy rates in major countries around the world. These question- naires may collect useful information about attitudes toward piracy and means of unauthorized copying, but the resulting estimates of illegitimate use need to be treated with caution. As a specific example, the Business Software Alliance (BSA) released their ninth study in May 2012, in which they argued that the global piracy rate for PC software 12 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 stood at around 42% in 2011, representing more than $60 billion (BSA 2012). This finding came from a study of more than 110 countries, with piracy rates ranging from approximately 19% (United States and Japan) to over 90% (Georgia and Zimbabwe). To compute these rates, the study relied on surveys of sampled con- sumers and businesses in a smaller set of 28 countries to determine the number of computers in use and the share of legitimately procured software.9 The study then estimated piracy rates for countries not in the sample surveys by means of a correla- tion between software usage and an “information development index” published by the International Telecommunications Union. The extent of bias in the estimates for these non-sampled countries arising from both the underlying approach and the application of the index is unknown. Regarding the latter, considerable variation is likely to exist in its applicability to countries of differing economic and social charac- teristics, as is the case for most such “development” indexes. Second, estimates of the economic effects of counterfeiting and piracy often do not account for demand responses—the possibility that not all consumers would switch to the original product if the counterfeit or pirated version were unavailable. Again, the BSA (2010) piracy estimates illustrate this point. Based on estimates of the volume of unlicensed software, the BSA computes its commercial value by applying a vector of prices for various kinds of computer programs including retail, licensed, and open-source. This total value is then taken as a measure of lost sales for legitimate software producers in each country. This approach assumes that each unit of pirated software would be fully replaced by a purchased version if copying were eliminated. That the BSA’s estimates of volumes and values foregone are likely substantially overstated emerges from a recent study estimating demand responses for one popular software product, Microsoft Office. Relying on original conjoint survey data from college students in Hong Kong, Leung (2013) estimated a discrete choice demand system for Microsoft Office from both legal and illegal sources; he concludes that the true gain from eliminating all sources of piracy is only 15% of the BSA’s estimated cost of piracy. Third, the counterfactual market equilibrium analyzed is often not clearly spelled out, nor does it seem realistic. In particular, studies provide estimates of the eco- nomic implications of certain levels of counterfeiting and piracy, and associate those levels with employment and tax losses without considering the longer-term general equilibrium responses outlined previously. Possible employment effects in the infor- mal sector are almost always ignored.10 Some studies attempt to consider cross-economy impacts, at least in an input- output framework. Consider, for example, a widely-cited calculation of U.S. copy- right industry sales, employment, and tax revenue losses (see Siwek 2007). The author employed industry-generated measures, such as those from BSA and LEK, of global sales losses in motion pictures, recorded music, software, and video games. Siwek considered the sum of these losses to be an underestimate because not all Fink et al. 13 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 countries are covered in all industry estimates. Making some adjustments for substi- tution between pirated and legitimate copies, and valuing all lost sales at legitimate prices, the study suggested a direct total loss of $25.6 billion for U.S. industries (in- cluding retailers). Siwek (2007) then applied “multipliers” available from the U.S. Department of Commerce, derived from its input-output model of the U.S. economy, to determine how this lost sales figure translates into total (direct plus indirect) losses in economic activity and total employment lost. These basic calculations resulted in a total sales loss for the U.S. economy of $58 billion per year, along with 373,000 jobs lost as a result of global piracy. All of that translated further into an earnings loss of $16.3 billion and reduced tax collections of $2.6 billion. These are large figures but must be kept in context. Although the multipliers account for cross-industry impacts in a static sense, the basic input-output approach again does not consider general-equilibrium linkages in the economy. If there really were $25.6 billion in reduced demand for U.S. copy- right goods, consumers would focus their expenditures elsewhere in the domestic and international economies, while workers would move to alternative employment and output and tax payments would rise there. Thus, Siwek’s (2007) study offers an assessment of the gross losses in sales and employment but greatly overstates the net losses. As we described previously, analysts would need to develop a general- equilibrium model for the latter calculations with endogenous changes in sectoral demands, output, and trade, while computing earnings losses at the difference between with-piracy and without-piracy wage rates. The United States International Trade Commission Analysis. Because of the methodo- logical shortcomings outlined above, we are skeptical that the results of the studies reviewed to this point offer useful guidance to policy makers. A more careful study was performed by the United States International Trade Commission (USITC: 2011); this project focused exclusively on the effects of IPR infringement in China, where concerns among major technology, fashion and content creators in the United States, the EU, and other rich countries are primarily focused. The USITC’s figures on gross sales lost due to Chinese infringement were taken largely from surveys of American firms. These firms claimed sizable damages, largely due to copyright piracy and trademark counterfeiting, though issues of in- digenous innovation and industrial espionage were also considered. Analysts at the USITC applied these reported reductions in demand to their well-regarded computa- tional general-equilibrium model. Such models are more flexible and more in- formative than simple computations with input-output tables, for they permit endogenous responses in prices, outputs, factor use, trade, and product demands. At any rate, the USITC model computed losses of nearly one million jobs in the United States in the sectors most affected by Chinese infringement. There was, 14 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 however, considerable uncertainty about this estimate depending on the scenarios considered. After having conducted this review, we observe that capturing the impacts of global infringement of IPR is a complex and uncertain problem that could use far more analytical and data attention. Moreover, there are no reliable computational general-equilibrium studies on how infringement affects the economies of develop- ing countries.11 This is not to say that the economy-wide effects are negligible and do not deserve the attention of policy makers in both developed and developing countries. We merely emphasize the challenge those policy makers face in address- ing counterfeiting and piracy problems with limited empirical guidance on offer. Micro-data Studies on the Effects of Counterfeiting and Piracy Although obtaining aggregate estimates of the levels of counterfeiting and piracy is important for understanding the scale of the problem, effective policy cannot be devised without considering the micro level of firms and consumers. Marketing Analyses and Consumer Surveys. Several marketing studies have been con- ducted to help understand consumer psychology and behavior that characterize the demand-side origin of counterfeiting. Although many of these are surveys or case studies, a few are experiments. Generally, experiments—whether in the field or in the lab—generate more credible evidence, but they also face methodological limita- tions. Field experiments that randomly assign people to control and treatment groups potentially have higher external validity because participants are real-world consumers making statements or decisions about the products of interest in (close to) real-world settings. However, compared with a lab experiment, in a field study it is more difficult to avoid interferences among participants and to exert tight control over the randomized treatment manipulations. Field experiments therefore generate results that are more susceptible to these “internal validity” concerns. As an early study in this literature strand, Bloch et al. (1993) examined the consumer’s role as an “accomplice” in the proliferation of product counterfeiting. These authors presented results from a field experiment showing that a surprisingly large proportion of adult consumers will select a counterfeit garment over the genuine good when there is a price advantage. The researchers also found that a significant price gap and negative attitudes toward large branded companies were the main factors driving counterfeit demand. In more recent work, Wilcox et al. (2009) conducted lab experiments and found that products with branded logos were more likely to be valued for their ability to help consumers gain social approval and status. In turn, this characteristic makes consumers more willing to buy such products from counterfeiters.12 Fink et al. 15 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 Looking further into the determinants of why consumers purchase non-deceptive counterfeit products, Vida (2007) reports on a consumer survey conducted in Slovenia. The results suggested that personal religiosity and education tend to reduce one’s willingness to buy fake goods. Such was not the case in Kwong et al. (2003), who studied consumer attitudes toward pirated CDs in Hong Kong. These researchers found that approximately 70% of the respondents stated that they bought pirated CDs and that they thought this generated high social benefit and low social cost. However, both studies concluded that gender matters, in that men are more willing to buy counterfeits than women. Bian and Veloutsou (2008) sur- veyed consumers in China and the United Kingdom, and found that perceived finan- cial, physical, and performance risks from consuming counterfeits were moderate. Still, consumer willingness to buy fakes varied with each product and each country, which makes it difficult to draw general conclusions. Finally, Casola et al. (2008) conducted three studies of consumer behavior in hypothetical situations. All three studies revealed that consumers were less willing to buy counterfeits when the victim was an individual rather than an organization. The average consumer would buy fakes if they cost about one-third of the genuine price. Finally, consumers’ willingness to buy counterfeit goods decreased if they were informed of the harmful impacts imposed on others. Thus, spreading aware- ness about the negative consequences of counterfeiting may be effective at deterring demand. Field Studies. Overall, studying consumer psychology is perhaps more useful for un- derstanding why there is a demand for counterfeits than for estimating the econom- ic impacts of counterfeiting. For the latter, it is more fruitful to collect field data over time to generate a sample panel on actual branded firms and their infringers. This is a challenging task because of the illicit nature of counterfeiting and the confiden- tiality concerns of both genuine and infringing companies. Nonetheless, a series of recent studies have been carried out in this direction. Qian (2008) offers the most comprehensive empirical investigation on the eco- nomic impacts of counterfeits. This study also stands out in that it is one of a rela- tively few studies that focus on a developing country. Qian collected original panel data from Chinese shoe companies from 1993 to 2004 (after government enforce- ment efforts were reduced in the footwear sector) through stratified random sam- pling to identify and measure the effects of competition from counterfeit goods on the prices, qualities, and other market outcomes of authentic footwear. The results show that brands experiencing less government enforcement action differentiated their products by introducing higher visible quality. These brands also engaged in forms of self-enforcement by establishing vertical relationships with licensed down- stream retailers (“company stores”) to control their sales, and setting higher prices 16 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 to signal their legitimacy in the marketplace. All these reactions helped reduce counterfeit sales in the period studied. It should be clearly noted that all the innovation and differentiation strategies mentioned above are costly. Although the authentic prices dropped initially under competitive pressure from counterfeits, the long-term effects on driving up authen- tic-product prices kicked in at various times for different firms. Larger firms with more human capital and research-and-development resources were faster in re- sponding and differentiating their products from counterfeits. Firms with more exports responded slower, perhaps because they were more diversified and less influ- enced by domestic counterfeits (Qian and Xie 2010). Field panel data of the kind used in the Chinese footwear study provide direct measures of how various economic variables change over time within each brand, but there are limitations that mean the results should be treated with caution. For example, even when field panel data are available for estimating how economic per- formance changes in response to counterfeiting, the underlying mechanism may be difficult to identify. Qian (2014b) explored the sales impacts of counterfeiting based on a combination of field data and lab experiments, and found that counterfeits have both a positive advertising effect (inducing more awareness of the brand) and a negative substitution effect on authentic products (reducing their sales). The ad- vertising effect dominates the substitution effect for high-end authentic product sales, and the substitution effect outweighs the advertising effect for low-end product sales. The advertising effect is found to be larger for fashion products and for brands that were less well-known at the time of infringement. Another impor- tant observation about consumer attitudes toward counterfeiting is that a high degree of income inequality seems to cause a greater demand for fake goods (Qian and Rucker 2013). Finally, several micro-data studies examine the economic effects of copyright piracy Notably, Hui and Png (2003) estimated the effects of physical CD piracy on the legitimate demand for recorded music in an econometric setting. These authors’ model expressly accounts for the demand linkages outlined in the previous section (sales of pirated copies could stimulate more demand for originals). Their econometric results showed that piracy had a negative net effect on the demand for legitimate music, though Hui and Png’s estimate of foregone sales by copyright holders was 58% lower than the music industry’s estimate (see International Federation of the Phonographic Industry 2003). The latter assumed a one-to-one substitution of pirated with legitimate purchases, whereas Hui and Png’s model allowed for a share of price-sensitive consumers to choose not to switch to legitimate goods in the coun- terfactual scenario. The Hui and Png analysis is limited, however, by an assumption that copyright holders do not adjust prices in response to lower piracy. That demand is responsive to prices charged for pirated and original copies also emerges from an experimental study of the music consumption behavior of Fink et al. 17 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 university students. Maffioletti and Ramello (2004) found that in general, students’ willingness to pay for a CD is lower than the market price of a legitimate copy. Thus, the hypothetical elimination of piracy would not expand sales of legitimate copies on a one-to-one basis at existing prices. At the same time, the study reveals that stu- dents’ willingness to pay for a pirated CD was substantially greater than its margin- al cost. This finding clearly points to the possibility that copyright holders might respond to stronger copyright enforcement by lowering their prices to capture a larger number of consumers.13 The Effects of File Sharing on the Internet One of the most frequently studied aspects of copyright infringement is how unau- thorized downloading and file sharing have affected sales of recorded music offered by major music publishers. It should be noted that this “end-user piracy” is some- times different in intent from commercial piracy. In particular, many people who download music for free and put music files on their computers in forms that can be readily shared by others rarely attempt to make money from these actions. Furthermore, some surveys suggest that many such users see little wrong with freely taking and sharing digital information products. For example, a recent survey of 1,607 people in the United Kingdom found that more than 80% admitted to having downloaded at least one file without authorization, and 47% said they did not think it was a crime (Bonnett 2010). In a survey of 1,000 U.S. college students in 2003, 69% said they had downloaded music, and 75% of those indicated they never paid for it (IPSOS Public Affairs 2003). Moreover, 76% responded that they would download even if they suspected the music file was illegal. These perceptions are among the reasons this form of unauthorized use is exceptionally difficult to control by copyright owners. Regardless of the motivation, downloading and file sharing through peer-to-peer (P2P) networks is common. The OECD (2005) calculated that one-third of Internet users in its member nations have downloaded files from P2P networks. Regarding the effects of downloading and file sharing, U.S. shipments of recorded music on CDs fell by 25% between 2000 and 2005 (Recording Industry Association of America 2006), and representatives of the music publishers blamed this trend on illegal downloads and P2P file sharing. Most recently, a global music industry asso- ciation noted a 7% drop in music sales in 2009, though it was unclear how much was due to unauthorized activity (International Federation of the Phonographic Industry 2010). That report claims that P2P piracy accounted for more than 20% of global Internet traffic, with higher shares in Latin America and Europe. This proliferation of file sharing has negatively affected authorized music sales. For example, Rob and Waldfogel (2006) collected data on albums obtained through downloading or purchase and used surveys among students at the University of 18 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 Pennsylvania to measure their valuation of this music. Using changes in Internet access as an instrumental variable, they found that downloading reduced the per- capita student expenditure on hit albums from 1999 to 2003 from $126 to $100, though it raised per-student welfare by $70. The most widely debated analysis surrounds the work of Oberholzer-Gee and Strumpf (2007), who suggest that downloading could, in theory, increase or decrease album sales due to the peering effects we outlined previously. These authors regressed the volume of sales by album on the number of downloads of songs on the album and other factors. To control for a possible simultaneity bias, they used as a primary instrumental variable the number of German secondary school children on holiday when they had more time to wait for downloads. Their results were striking; in a basic ordinary least squares regression, the coefficient of sales on downloads was 1.09 and highly significant, suggesting a large positive impact. However, after adding album fixed effects and first-stage instrumentation, they could find no evidence that the number of times an album was downloaded had a statistical impact on music sales, while the implied economic effects were small. This finding has been criticized on several grounds, primarily by Liebowitz (2005; 2006; 2007). In their more recent study, Oberholzer-Gee and Strumpf (2010) acknowledged that file sharing has reduced music sales, but by more modest amounts than generally supposed. Furthermore, they argue that offsetting increases in demand for complementary music services have increased incentives for the creation of new music, books, and movies. In other words, file sharing may be damaging to music publishers but not necessarily to artists themselves, especial- ly those who are younger and less well-known. However, this possibility deserves further study. Unfortunately, empirical studies on the effects of copyright piracy— whether physical or online—are almost exclusively confined to developed countries. To a good extent, this reflects the availability of data and the “home bias” of re- searchers in developed economies. However, there are reasons to believe that copy- right piracy is more widespread in poorer countries. For example, across a wide range of countries, Fink (2009) reported a correlation coefficient between software piracy rates and per capita GDP in 2004 of 2 0.89. This likely reflects poorer consumers turning more frequently to pirated goods, the greater importance of the informal economy as a source of employment, and less developed and more resource-constrained legal systems. Copyright-based industries exist in most developing countries. In some—such as the Philippines, Malaysia, and Mexico—these industries account for larger employ- ment shares than in developed economies.14 Copyright piracy is bound to affect the prospects of earning a livelihood from creative and artistic activities in poorer coun- tries. One study on the music industry in West Africa reported anecdotal evidence confirming both the high incidence of piracy and its negative effect on musicians’ Fink et al. 19 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 incomes (Penna et al 2004). However, this study also pointed to a large number of other barriers—from ineffective collection societies to tax policies—that render the recording activities unprofitable for most musicians. How a reduction in piracy levels alone would affect artists and those involved in the distribution of creative works remains an open question. Conclusions We conclude this review with some observations on what the literature suggests about the importance of policy initiatives for dealing with counterfeiting and piracy in developing nations. We then discuss the need for further work to improve data collection and analysis. Implications for Policymaking in Developing Countries. As we have noted, most of the studies we reviewed above offer evidence about the impacts of counterfeiting and piracy in more developed economies, with the notable exception of Qian (2008; 2014b). Thus, readers may question their applicability to developing countries and policy authorities. Indeed, there are important differences to note. First, the classic “intellectual property industries” (high-end fashion, digital content, publishing, software, and research-based pharmaceuticals, food products, and plant varieties) play a smaller role in the production structures of poor countries. Rather, local firms may be more focused on imitation and copying, while consumers may be more open to buying fake goods. In this context, the immediate gains from stronger IPR enforcement are likely to accrue to foreign content providers and multinational enterprises, most of which develop the bulk of their intellectual property in rich economies. It is thus not surprising that many developing countries face demands for more stringent IP enforcement in bilateral and regional free trade agreements they negotiate with developed countries.15 Notwithstanding the broader welfare gains that such agreements could offer, policymakers in developing countries rea- sonably wonder whether it makes sense to devote scarce development resources to IPR enforcement. However, it is important to point out that there is considerable and growing heterogeneity among developing countries. Rapidly emerging economies such as Brazil, Mexico, China, and Turkey see increasing interest among growing middle classes for higher quality goods protected by recognizable brands. There is also a rapid trend among innovative local firms toward brand development, investments in product variety and quality, and R&D expenditures in numerous industries, including software and digital content (Maskus 2012). In this regard, domestic interests in stronger IPR protection and enforcement are rapidly emerging and chal- lenging the political-economy equilibrium that favors weaker enforcement. These countries presumably can learn from the experience of developed countries. 20 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 Second, even poorer countries have significant interests in stronger enforcement for at least two reasons. One is that their citizens disproportionately suffer from adul- terated counterfeit medicines and ineffective or dangerous foods and beverages. Basic calculations suggest that this can be a major problem for sustaining public health in poor countries and that there is a positive payoff from working to reduce such risks (Maskus 2007). Another reason is that weak trademark enforcement can sharply diminish prospects for effective brand development, firm entry, and national market- ing success among potentially innovative companies in poor countries. Indeed, local counterfeiting often targets domestic enterprises at least as much as foreign brands (Mertha 2005). Similarly, extensive unauthorized copying of music, designs, and other creative activities limits opportunities for artists and artisans to build local markets, ultimately harming their ability to sell globally as well. Thus, public invest- ments in enforcement and creative infrastructures, perhaps in partnership with private entities, could help remove such barriers to entry. As a strategic matter, invari- ably scarce enforcement resources could focus on reducing deceptive counterfeiting (to deal with dangerous fakes) and educational campaigns on expanding consumer awareness of how unauthorized copying negatively affects local artists. Recommendations for Data Collection and Future Empirical Work. Policy makers in countries at different stages of development are interested in better understanding the economic effects of trademark counterfeiting and copyright piracy. So what di- rection should future analytical work take? As discussed previously, many studies exist that aim to quantify the extent of counterfeiting or piracy at the aggregate level, be it sectoral or economy-wide. Most of these studies have serious shortcom- ings, both in the data employed and in the adopted methodologies. More important, aggregate estimates of the incidence of piracy and counterfeiting by themselves offer policy makers little guidance on appropriate IP enforcement policies. More useful guidance emerges from micro studies that take into account the spe- cific characteristics of different industries and comprehensively assess the effects of illicit activities on prices, consumer demand, company revenues, innovative behav- ior, and overall welfare. Such studies are still scarce. Finding reliable and representa- tive data is challenging and creative approaches are needed to establish causality and derive credible estimates. Experimental studies offer one way forward, particu- larly in combination with broader empirical investigations. For example, research- ers could gather field panel data to analyze the overall impacts of counterfeiting, and supplement them with lab experiments to understand the underlying mecha- nisms (see Qian 2014a; Qian and Rucker 2013). The quality of any future empirical work will largely depend on the quality of the underlying data, however. Given the illegal nature of counterfeiting and piracy, col- lecting representative and systematic data on all relevant economic variables will Fink et al. 21 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 continue to be a challenge, and progress will only be gradual. Nonetheless, several promising avenues exist. For example, more could be done to improve and harmonize data collection in the course of IP enforcement activities. Customs statistics are an obvious place to start because they are generally the most direct source of information about the extent of piracy and counterfeiting flows that cross international borders. Many customs offices already compile and publish data on seizures of counterfeit and pirated goods. Although the World Customs Organization has developed interna- tional guidelines for reporting on seized products, actual reporting practices differ substantially across countries. For example, some countries report the number of seizures, whereas others focus on the quantity of seized goods, or the number of consignments. Substantial differences also exist regarding the valuation of seized goods (World Customs Organization 2009). From an analytical point of view, it would be important to achieve as much consis- tency as possible in the treatment of firms, products, and sectors. Data collection needs to be done consistently over time to track trends and analyze the impacts of changes in enforcement policy. Sampling interceptions data could be extended to detailed subcate- gories of goods to gain a better understanding of the scope of counterfeit trade. Beyond customs statistics, there is scope for collecting more comprehensive data associated with domestic law enforcement activities, notably through the judicial system. To our knowledge, no systemic effort exists at the international level to promote and harmonize the collection of such data. Intergovernmental organiza- tions can play a more active role in coordinating data collection initiatives, promot- ing the harmonization of reporting standards, and offering technical assistance to developing country governments. At the domestic level, governments may want to consider adopting an impact evaluation framework, especially when new IP enforcement policies and measures are introduced. Continuous data gathering would be an essential component of such a framework. Finding appropriate control groups against which enforcement activities could be evaluated may prove challenging but is possible in certain circumstances— especially when new initiatives are first implemented on a pilot basis. In addition, there is scope for conducting original surveys to support targeted ana- lytical work. Surveys of rights holders on key performance characteristics as they relate to counterfeiting activity could usefully complement firm-level data available through industry or official sources. Depending on the industry, rights holders may also possess valuable information on the nature of, and markets for, counterfeit and pirated goods that was gathered in the course of their own market intelligence and enforcement activities. Surveys of rights holders could take place within existing cooperation programs between the public and private sectors. Such firm surveys should be designed to collect as detailed and comprehensive data as possible while 22 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 guaranteeing the anonymity of respondents when submitting sensitive business information.16 In the longer term, as information from these surveys—especially across countries— becomes more standardized and comprehensive, econometric models could be de- veloped to estimate important market parameters about which limited information exists. Most prominently, the demand characteristics and substitution parameters between legitimate goods and counterfeit goods are important to estimate to gain an accurate picture of damages imposed on rights holders and the associated effects on employment and research-and-development spending. Such estimates need to be subjected to significant sensitivity and robustness analysis to ensure that they can be deployed with confidence. After these parameters are estimated, it would be feasible to develop computational general equilibrium models to assess the country-specific and international impacts of counterfeiting and piracy, particularly accounting for cross-industry effects and em- ployment channels. Models of this sort would be particularly useful for assessing the potential impacts of additional policy efforts to counteract illicit trade and production. It is important to emphasize, however, that such models necessarily rely on strong as- sumptions that could be misleading in this context. For example, employment in the counterfeiting and piracy sector is largely informal, and few models can manage the linkages between informal sector and formal sector employment. Still, with appropri- ate caveats, these models should be an improvement over existing estimates of economy-wide losses in employment and tax revenues because those estimates do not account for aggregate relationships in the economy. Notes Carsten Fink is the Chief Economist, World Intellectual Property Organization; carsten.fink@wipo.int. Keith Maskus is a Professor of Economics, University of Colorado Boulder. Yi Qian is an Associate Professor of Marketing and Behavioral Science, Sauder School of Business, University of British Columbia. The authors contributed equally and are listed in alphabetical order. Helpful comments from Intan Hamdan-Livramento, Sacha Wunsch-Vincent, and two anonymous referees are gratefully acknowledged. The opinions expressed herein are those of the authors and should not be attributed to any of the institutions with which they are affiliated. 1. The literature on counterfeiting and piracy is large and cannot be fully reviewed here. Instead, we focus on widely-cited studies of counterfeiting and piracy and offer an overview of the various ap- proaches that researchers have adopted. 2. See, for example, Newton et al. (2010). Counterfeit drugs are part of the broader challenge of substandard pharmaceuticals, which also extends to generic medicines. 3. Taylor (1993) analyzed such “masking” strategies, which are costly in welfare terms. Again, Maskus (2000; 2012) offers anecdotal evidence of such responses in developing countries. 4. Copyright protection is limited to strike a balance between incentives for creating new works and the desirability of making them widely available. Thus, copyright has a limited term and certain Fink et al. 23 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 exceptions and limitations to exclusive rights exist in areas in which society reaps special benefits from the rapid dissemination of creative works. 5. Maskus (2000) relates anecdotal evidence that this price moderation occurred in the software industry when Taiwan improved its enforcement activities in the 1990s. 6. Conner (1995) offers a broader set of circumstances in which it might be profitable for copyright owners to accommodate copying in the presence of network effects and quality differences. 7. Maskus (2012) offers a full review, while Arora et al. (2001) set out comprehensive models. 8. Another problem is that the analysis takes place at the highly aggregated two-digit Harmonized System categorization of traded goods. Within each such category, typically hundreds of goods subcate- gories may or may not be subject to much counterfeiting. The OECD (2008) analysis assumes that its average rates computed at the aggregate level apply to each subcategory. This list of methodological problems is not exhaustive. The OECD (2008) report describes several other important problems that likely bias the resulting estimates. 9. The difference between the total installed software and legitimately purchased software is the amount of piracy. Although this approach is sensible in that it relies on reported measures of legitimate usage in relation to overall computer capacity, both of which may be reliable, the BSA report does not indicate what the underlying sampling biases may be. This approach also ascribes some use of open- source software and freeware to piracy. 10. See, for example, Siwek (2007) and Frontier Economics (2009). The latter study considers the reemployment of workers “losing” their jobs as a result of counterfeiting and piracy but does not con- sider the effects of revenue-compensating tax policies. In addition, the assumption of a certain share of workers not finding reemployment implicitly assumes some form of historical shock that led to actual job losses. The latter assumption is difficult to reconcile with the more gradual evolution of counterfeit- ing and piracy activity observed in most sectors. 11. Maskus (2005) offers a simple partial-equilibrium model of the effects on employment and sales in several key sectors in Lebanon, using data for 1996. However, this approach cannot determine cross-industry equilibrium impacts and is only suggestive at best. 12. Bian and Moutinho (2009) find a similar result from surveying consumer attitudes in Scotland toward buying fake luxury watches. 13. This outcome was noted in survey evidence of the pricing strategy of legitimate software producers in Asian countries as copyright enforcement improved (Maskus 2000). 14. See WIPO (2013). 15. See Fink (2013) for a discussion of how trade agreements treat intellectual property rights and how they may affect economic welfare. 16. There are also recent attempts to employ surveys to estimate the incidence and characteristics of counterfeit and substandard drugs. For example, Bate et al. (forthcoming) studied 1,437 samples of Ciprofloxacin in 18 low-to-middle-income countries. These authors found that 9.88% of samples contained less than 80% of the correct active ingredient, and 41.5% of these failures are coun- terfeits. Counterfeit and substandard drugs tend to differ in two observable attributes: first, counterfeit drugs are more likely to mimic drugs registered with local drug safety regulators than substandard drugs. Second, after controlling for other factors, substandard drugs are on average cheaper than generics, whereas counterfeit drugs do not show any significant price difference. References Aghion, P., N. Bloom, R. Blundell, R. Griffith, and P . Howitt. 2005. “Competition and Innovation: An Inverted-U Relationship.” Quarterly Journal of Economics 120 (2): 701–28. Akerlof, G. A. 1970. “The Market for ‘Lemons’: Qualitative Uncertainty and the Market Mechanism.” Quarterly Journal of Economics 84 (3): 488 –500. 24 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 Anderson, E. T., Y. Qian, and D. Simester. 2010. “Spillover Effects of a Low-quality Entrant.” Northwestern University and Massachusetts Institute of Technology Working Paper. Arora, A., A. Fosfuri, and A. Gambardella. 2001. Markets for Technology: The Economics of Innovation and Corporate Strategy. Cambridge, MA: MIT Press. Bakos, Y., E. Brynjolfsson, and D. G. Lichtman. 1999. “Shared Information Goods.” Journal of Law and Economics 42 (1): 117 –55. Bate, R., G. Jin, and A. Mathur. “Falsified or Substandard? Assessing Price and Non-Price Signals of Drug Quality.” Journal of Economics & Management Strategy. In press. Besen, S. M., and S. N. Kirby. 1989. “Private Copying, Appropriability, and Optimal Copying Royalties.” Journal of Law and Economics 32 (2): 255–80. Bhattacharjee, S., R. D. Gopal, K. Lertwachara, and J. R. Marsden. 2007. “The Effect of P2P File- Sharing on Music Markets: A Survival Analysis of Albums on Ranking Charts.” Management Science 53 (9): 1359– 74. Bian, X., and L. Moutinho. 2009. “An Investigation of Determinants of Counterfeit Purchase Consideration.” Journal of Business Research 62 (3): 368–79. Bian, X., L. Moutinho, and C. Veloutsou. 2008. “A Cross-National Examination of Consumer Perceived Risk in the Context of Non-Deceptive Counterfeit Brands.” Journal of Consumer Behaviour 7 (1): 3–20. Blackburn, D. 2004. “Does File Sharing Affect Record Sales?” Working paper, Department of Economics, Harvard University. Bloch, P. H., R. F. Bush, and L. Campbell. (1993). “Consumer ‘Accomplices’ in Product Counterfeiting: A Demand Side Investigation.” Journal of Consumer Marketing 10 (4): 27 –36. Bonnett, T. (2010). “Four in Five People Download Illegally.” Sky News Online, March 19. Available at http://news.sky.com/home/technology/article/15576542. Branstetter, L., R. Fisman, and C. F. Foley. 2006. “Do Stronger Intellectual Property Rights Increase International Technology Transfer?” Quarterly Journal of Economics 121 (1): 321– 49. Business Software Alliance (BSA). 2012. “2011 BSA Global Software Piracy Study.” May. Available at http://globalstudy.bsa.org/2011/downloads/study_pdf/2011_BSA_Piracy_Study-Standard.pdf. Burnkrant, R. E., and A. Cousineau. 1975. “Informational and Normative Social Influence in Buyer Behavior.” Journal of Consumer Research 2 (3): 206 –14. Casola, L., S. Kemp, and A. Mackenzie. 2008. “Consumer Decisions in the Black Market for Stolen or Counterfeit Goods.” Journal of Economic Psychology 30 (2): 162–71. Conner, K. R. 1995. “Obtaining Strategic Advantage from Being Imitated: When Can Encouraging ‘Clones’ Pay?” Management Science 41 (2): 209– 25. Conner, K. R., and R. P. Rumelt. 1991. “Software Piracy: An Analysis of Protection Strategies.” Management Science 37 (2): 125 –39. Fink, C. 2009. “Enforcing Intellectual Property Rights: An Economic Perspective.” World Intellectual Property Organization WIPO/ACE/5/6, August 26. Available at http://www.wipo.int/edocs/mdocs/ enforcement/en/wipo_ace_5/wipo_ace_5_6.pdf. Fink, C. 2013. “Intellectual Property Rights: Economic Principles and Trade Rules.” In A. Lukauskas, R. M. Stern, and G. Zanini, eds., Handbook of Trade Policy for Development. Oxford: Oxford University Press. Frontier Economics. 2009. “The Impact of Counterfeiting on Governments and Consumers.” Report commissioned by BASCAP. May. Available at http://www.icc.se/policy/statements/2009/ BASCAP .pdf. Fink et al. 25 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 Grossman, G. M., and C. Shapiro. 1988a. “Counterfeit-Product Trade.” American Economic Review 78 (1): 59 –75. . 1988b. “Foreign Counterfeiting of Status Goods.” Quarterly Journal of Economics 103 (1): 79 –100. Harbaugh, R., and R. Khemka. 2010. “Does Copyright Enforcement Encourage Piracy?” Journal of Industrial Economics 58 (2): 306 –23. Hui, K.-L., and I. Png. 2003. “Piracy and the Legitimate Demand for Recorded Music.” Contributions to Economic Analysis and Policy 2 (1): Article 11. International Federation of the Phonographic Industry. 2003. The Recording Industry Commercial Piracy Report 2003. Available at http://www.ifpi.org/content/library/Piracy2003.pdf. . 2010. Recording Industries in Numbers 2010. Available at http://www.ifpi.org/content/ section_news/20100428.html. IPSOS Public Affairs. 2003. Internet Piracy on Campus. Research report. Javorcik, B. S. 2004. “The Composition of Foreign Direct Investment and Protection of Intellectual Property Rights: Evidence from Transition Economies.” European Economic Review 48 (1): 39– 62. Johnson, W. R. 1985. “The Economics of Copying.” Journal of Political Economy 93 (1): 158–74. Keller, W. 2004. “International Diffusion of Technology.” Journal of Economic Literature 42 (3): 752 –82. Kwong, K., O. H. M. Yau, J. S. Lee, L. Y. M. Sin, and A. C. B. Tse. 2003. “The Effects of Attitudinal and Demographic Factors on Intention to Buy Pirated CDs: The Case of Chinese Consumers.” Journal of Business Ethics 47 (3): 223– 35. Consulting, L. E. K. 2005. The Cost of Movie Piracy: An Analysis Prepared by LEK for the Motion Picture Association. Available at http://www.archive.org/details/MpaaPiracyReort. Leung, T. C. 2013. “What is the True Loss Due to Piracy? Evidence from Microsoft Office in Hong Kong.” Review of Economics and Statistics 95 (3): 1018–29. Liebowitz, S. J. 1985. “Copying and Indirect Appropriability: Photocopying of Journals.” Journal of Political Economy 93 (5): 945 –57. 2005. “Pitfalls in Measuring the Impact of File-Sharing on the Sound Recording Market.” CESIfo Economic Studies 51 (2 –3): 435–73. Liu, K., J.-A. Li, Y. Wu, and K. K. Lai. 2005. “Analysis of Monitoring and Limiting of Commercial Cheating: A Newsvendor Model.” Journal of the Operational Research Society 56 (7): 844–54. Maffioletti, A., and G. B. Ramello. 2004. “Should We Put Them in Jail? Copyright Infringement, Penalties and Consumer Behaviour: Insights from Experimental Data.” Review of Economic Research on Copyright Issues 1 (2): 81 – 95. Markusen, J. R. 2001. “Contracts, Intellectual Property Rights, and Multinational Investment in Developing Countries.” Journal of International Economics 53 (1): 189– 204. Maskus, K. E. 2000. Intellectual Property Rights in the Global Economy. Washington DC: Institute for International Economics. . 2005. “Strengthening Intellectual Property Rights in Lebanon.” In C. Fink, and K. E. Maskus, eds., Intellectual Property and Development: Lessons from Recent Economic Research. New York: World Bank and Oxford University Press. . 2007. “Lack of Intellectual Property Rights.” In B. Lomborg, ed., Solutions for the World’s Biggest Problems: Costs and Benefits. Cambridge: Cambridge University Press. . 2012. Private Rights and Public Problems: The Global Economics of Intellectual Property in the 21st Century. Washington, DC: Peterson Institute for International Economics. 26 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 Maskus, K. E., and M. Penubarti. 1995. “How Trade-Related are Intellectual Property Rights?” Journal of International Economics 39 (3/4): 227 –48. Mertha, A. C. 2005. The Politics of Piracy: Intellectual Property in Contemporary China. Ithaca, NY: Cornell University Press. Metro-Goldwyn-Mayer Studios Inc., et al. v. Grokster, Ltd. et al. 2005. 545 U.S. 913. Available at http://w2.eff.org/IP/P2P/MGM_v_Grokster/04-480.pdf. Newton, P. N., M. D. Green, and F. M. Ferna ´ ndez. 2010. “Impact of Poor-Quality Medicines in the ‘Developing’ World.” Trends in Pharmacological Sciences 31 (3–3): 99 –101. Oberholzer-Gee, F., and K. Strumpf. 2007. “The Effect of File-Sharing on Record Sales: An Empirical Analysis.” Journal of Political Economy 115 (1): 1– 42. . 2010. “File-Sharing and Copyright.” Working paper, Harvard Business School, Harvard University. Organisation for Co-operation and Economic Development. 2005. OECD Report on Digital Music: Opportunities and Challenges. Paris: OECD. Available at http://www.oecd.org/dataoecd/13/2/ 34995041.pdf. . 2008. The Economic Impact of Counterfeiting and Piracy. Paris: OECD. Available at http://www. oecd.org/document/4/0,3343,en_2649_34173_40876868_1_1_1_1,00.html. . 2009. “Magnitude of Counterfeiting and Piracy of Tangible Products: An Update.” Research report. Available at http://www.oecd.org/dataoecd/57/27/44088872.pdf. Peitz, M., and P. Waelbroeck. 2004. “The Effect of Internet Piracy on Music Sales: Cross-Section Evidence.” Review of Economic Research on Copyright Issues 1 (2): 71 –79. Penna, F. J., M. Thormann, and J. M. Finger. 2004. “The Africa Music Project.” In J. M. Finger, and P . Schuler, eds., Poor People’s Knowledge: Promoting Intellectual Property in Developing Countries. Oxford: Oxford University Press. Qian, Y. 2008. “Impacts of Entry by Counterfeiters.” Quarterly Journal of Economics 123 (4): 1577–1609. 2014a. “Brand Management and Strategies against Counterfeits.” Journal of Economics and Management Strategy 23 (2): 317 –343. 2014b. “Counterfeiters: Foes or Friends? How Counterfeits Affect Sales by Product Quality Tier.” Management Science 60 (10): 2381–2400. Qian, Y., Q. Gong, and Y. Chen. 2013. “Untangling Searchable and Experiential Quality Responses to Counterfeiting.” Working paper, Kellogg School of Management, Northwestern University. Qian, Y., and D. Rucker. 2010. “Income Inequality and Counterfeiting.” Working paper, Kellogg School of Management, Northwestern University. Qian, Y., and H. Xie. 2010. “Bayesian SEM with Random Changepoints in Analyzing the Timing of Responses to Counterfeits.” Working paper, Kellogg School of Management, Northwestern University. Recording Industry Association of America. 2006. “The Recording Industry Association of America’s 2005 Year-end Statistics.” Available at http://www.riaa.com. Rob, R., and J. Waldfogel. 2006. “Piracy on the High C’s: Music Downloading, Sales Displacement, and Social Welfare in a Sample of College Students.” Journal of Law and Economics 49 (1): 29 – 62. Scherer, F. M. 1967. “Market Structure and the Employment of Scientists and Engineers.” American Economic Review 57 (3): 524 –31. Siwek, S. E. 2007. “The True Cost of Copyright Industry Piracy to the U.S. Economy.” Institute for Policy Innovation Policy Report #189. Fink et al. 27 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 Sun, J., L. Debo, S. Kekre, and J. Sun. 2010. “Component-based Technology Transfer in the Presence of Potential Imitators.” Management Science 56 (3): 536–52. Takeyama, L. N. 1994. “The Welfare Implications of Unauthorized Reproduction of Intellectual Property in the Presence of Demand Network Externalities.” Journal of Industrial Economics 42 (2): 155 –66. Takayama, L. N. 1997. “The Intertemporal Consequences of Unauthorized Reproduction of Intellectual Property.” Journal of Law and Economics 40 (2): 511 –22. Taylor, M. S. 1994. “TRIPS, Trade, and Growth.” International Economic Review 35 (2): 361 –82. Tirole, J. 1988. The Theory of Industrial Organization. Cambridge, MA: MIT Press. United States International Trade Commission (USITC). 2011. China: Effects of Intellectual Property Infringement and Indigenous Innovation Policies on the U.S. Economy. Investigation No. 332–519, USITC Publication 4226. Varian, H. R. 2000. “Copying and Copyright.” Journal of Economic Perspectives 19 (2): 121 –38. Vida, I. 2007. “Determinants of Consumer Willingness to Purchase Non-deceptive Counterfeit Products.” Managing Global Transitions 5 (3): 253 –70. Wilcox, K., H. M. Kim, and S. Sen. 2009. “Why Do Consumers Buy Counterfeit Luxury Brands?” Journal of Marketing Research 46 (April): 247–59. World Intellectual Property Organization. 2013. “WIPO Studies on the Economic Contribution of the Copyright Industries.” Available at http://www.wipo.int/export/sites/www/copyright/en/ performance/pdf/economic_contribution_analysis_2013.pdf. World Customs Organization. 2009. “Customs and IPR Report 2009.” Available at http://www. wcoomd.org/files/1.%20Public%20files/PDFandDocuments/Reports/Extract_IPRReport2009.pdf. Zentner, A. 2006. “Measuring the Effect of Music Downloads on Music Purchases” Journal of Law and Economics 49 (1): 63– 90. 28 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/1/2461705/The-Economic-Effects-of-Counterfeiting-and-Piracy by Joint Bank-Fund Library user on 06 October 2017 Measuring Violent Conflict in Micro-level Surveys: Current Practices and Methodological Challenges ¨ ck, Patricia Justino, Philip Verwimp, Alexandra Avdeenko, Tilman Bru and Andrew Tedesco This paper reviews current practices and common challenges in the measurement of the causes, functioning, and consequences of violent conflict at the micro-level. We review ex- isting conflict- and violence-related survey questionnaires, with a particular focus on the World Bank’s Living Standard Measurement Surveys. We discuss methodological chal- lenges associated with empirical work in conflict-affected and fragile areas—such as opera- tionalizing a definition of conflict, using the appropriate units of analysis, deciding on the timing of the survey, dealing with data biases, and conducting surveys in an ethically sound manner under conditions of insecurity—and propose ways to improve the usefulness of existing surveys to analyze conflict processes at the micro-level. Violent conflict, house- holds, survey methods, questionnaire design. JEL codes: C81, D74, F52, O12 Introduction Analyzing the causes of violent conflict has preoccupied social scientists for a long time. The academic literature on conflict has traditionally had a strong macro per- spective, with a focus on understanding the rise of violence against state institu- tions, and between different ethnic groups (Horowitz 1985; Skaperdas 1992; Collier and Hoeffler 2004; Fearon and Laitin 2003). This literature has led to con- siderable advances in our knowledge on the complex causes of political violence. It has, however, offered a more limited understanding of the role of the causal mecha- nisms and micro-level dynamics that may shape the relationship between violent conflict and social, economic, and political outcomes. The World Bank Research Observer # The Author 2015. Published by Oxford University Press on behalf of the International Bank for Reconstruction and Development / THE WORLD BANK. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com doi:10.1093/wbro/lkv011 Advance Access publication August 18, 2015 31:29–58 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 These concerns have resulted in a growing new research agenda on the micro- level analysis of violent conflict.1 This emerging body of research has begun to shed light on some of the complex micro-level causes and consequences of violent conflict by generating important theoretical and empirical insights on a number of dimensions of violent conflict processes (see Justino 2012 for a comprehensive review). But despite considerable progress, we still have limited rigorous and com- parative evidence of how people live in contexts of violence and conflict, particularly regarding the following issues: what choices they make to secure lives and liveli- hoods; how institutional structures impact on and are affected by these decisions; or what policies work in establishing peace and supporting economic stability in areas and among populations affected by violent conflict. This lack of systematic understanding is explained to a large extent by the limited amount of information we currently have on how people live and survive in areas of violent conflict. Over the last three decades The World Bank and other institutions have developed highly sophisticated surveying techniques, most notably the Living Standards Measurement Studies (LSMS), to collect socio-economic data at the micro-level. These advances in data collection have led to a wealth of knowledge on how individuals, households, and communities live and adapt to a variety of shocks and life events, including price changes, sudden climatic shocks and loss of work, or illness, among others. The impact of political shocks and events such as violent pro- tests, communal riots, revolutions, civil wars, genocide, and international wars, is less-well understood. Several large household surveys conducted in conflict-affected countries such as Nigeria, Indonesia, Colombia, Rwanda, South Africa, Pakistan, and Liberia, only sporadically feature (a limited number of ) questions that capture the effects of violence and other forms of political instability and fragility. The absence of detailed information on how individuals, households, and commu- nities experience violent conflict means that researchers typically rely on crude proxies of conflict (often whether dwellings have been destroyed, the number of deaths in the household and whether the households has ever been displaced), which makes it hard to build a systematic and comparable understanding of how processes of violence and insecurity have affected different people and communities, the chan- nels through which violence may affect welfare and behavior at the micro-level, and how violent conflicts may transform societies, politics, and local economies. The objectives of this paper are to review current empirical research on conflict processes at the micro-level, discuss the methods used to empirically capture such processes, and suggest potential advances to current survey methodologies. The paper is organized as follows. In the following section, we review current survey practices in conflict-affected contexts, paying close attention to recent aca- demic literature that makes use of quasi-standardized institutional surveys, such as The World Bank’s LSMS. In the subsequent section we discuss recent studies based on surveys purposely designed to investigate conflict processes, and reflect on 30 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 common methodological challenges related to the design, implementation, and analysis of micro survey data in conflict-affected contexts. The final section con- cludes by discussing ways forward to improve methodologies for data collection at the micro-level in conflict-affected contexts. Current Empirical Approaches to Understanding Conflict at the Micro Level Social scientists have used a variety of empirical methods to study the impact of violent conflict on human welfare and behavior. Recent advances have been possible thanks to the wider availability of rigorous evidence from conflict-affected contexts. This new empirical research has developed in two broad directions (see Verwimp, Justino, and Bru ¨ ck, and Verwimp 2013). The most common ¨ ck 2009, and Justino, Bru direction has been the use of socio-economic datasets in conflict-affected regions that were not explicitly collected for the analysis of processes or consequences of violent conflict per se, but either contain a number of variables (often self-reported) that can be used as proxies for human exposure to violent conflict, or can be crea- tively merged with conflict event data. The second direction is based on data specifi- cally collected to identify the causes and functions of violent conflict at the micro-level. This is the ideal approach because it allows researchers to tailor the surveys to directly address important research questions about different aspects of conflict processes, their causes, and their consequences. It is, however, a less common approach due to the high costs of these surveys, the level of resources required, and the ethical and security constraints associated with doing primary research in areas of violence. In this section we review recent studies based on existing socio-economic surveys that were not conducted to analyze conflict processes, but have been used to generate important insights on conflict processes and its welfare outcomes. We focus on studies that have used the widely available LSMS conducted by the World Bank and partners, and national census data and the Demographic and Health Surveys (DHS). We also discuss these studies’ insights and propose ways in which the surveys could be improved in order to advance knowledge on conflict processes. We discuss the purposively designed survey approaches in the methodology section. Using Living Standards Measurement Surveys to Understand Conflict Processes The LSMS, implemented by the World Bank, aims to provide high quality data for policy makers to assess the effectiveness of interventions designed to improve the Bru ¨ ck et al. 31 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 living standards of individuals, households, and communities in countries where they are conducted. Micro-level empirical research on the effects of conflict and vio- lence has made use of several of these surveys. The main advantage of the LSMS is their comprehensive treatment of household welfare, and how they detail topics such as household demographics, income, health, labor, and education. However, these surveys are primarily designed to be conducted in peaceful contexts and often neglect to explicitly address violent con- flict as a category in its own right, even when they are implemented in countries affected by conflict.2 The LSMS are also designed to meet the needs of governments, who sometimes wish to avoid referring to the conflict in an effort to start afresh. As a result, questionnaires may focus on the experiences and the standards of living after, rather than during, the conflict. Some questions about conflict may also be politically sensitive and government officials may be apprehensive about including them in household surveys. For example, questions about the destruction or theft of assets that identify the perpetrators, especially if government forces are included in the list, may raise controversial or even legal issues for government administrations. Likewise, government-sponsored surveys may avoid addressing the conflict in for- merly rebel-held territories for fear of invoking distrust or upsetting a delicate peace settlement. As a result, some questions that are relevant for researchers may be left out in government-sponsored questionnaires. Several LSMS have nonetheless included conflict-related questions and yielded important insights for conflict research. In other cases, researchers have been able to match the information in these surveys to external conflict event datasets in order to design identification strategies that allow the causal analysis of conflict effects on individuals, households, and communities. For this paper we reviewed 24 of these surveys and analyzed their structure and content.3 In general, we found that including conflict questions in LSMS has been done in a piecemeal fashion, resulting in insights scattered across countries and categories rather than a system- atic and comparative approach to measuring conflict. But we also found good exam- ples of conflict-sensitive questions across these surveys that have resulted in important new knowledge on the impact of conflict on the lives and livelihoods of people in areas of violence. These studies have led to significant advances in our understanding of the causal impact of violent conflict on key aspects of people’s lives, including dynamics of poverty and food security, education and health, liveli- hood choices, labor supply decisions, intra-household decisions, and individual, household, and group behavior. We provide a summary of existing socio-economic surveys with conflict related questions in Appendix I, and in Appendix II summarize studies (and their main findings) that have used LSMS (and DHS) to understand conflict processes. 32 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Using other Standardized Household Surveys and National Census Data In addition to the LSMS, researchers have leveraged other standardized household surveys and national census data to study conflict processes. For example, Deininger (2003) conducted one of the first micro-level analyses of violent conflict and its consequences using data on communities and households included the 1999–2000 Uganda National Household Survey (UNHS) and the 1992 Uganda Integrated Household Survey (IHS). These surveys contain information on approximately 10,000 households and 1,000 communities. The surveys asked respondents questions related to the civil war, including information on victimization and motivations for participa- tion in the war. Czaika and Kis-Katos (2009) have studied the determinants of displace- ment in Aceh, Indonesia, using the Village Potential Census (PODES), which maps conflict-affected villages across the whole of Indonesia. The census itself includes ques- tions, posed to community leaders, on conflict events in particular communities. Rohner, Thoenig and Zilibotti (2012) have matched two waves of the Afrobarometer survey, a widely available socio-economic survey, to the ACLED dataset in order to in- vestigate the effect of the conflict in Northern Uganda on social capital.4,5 Verpoorten (2011) demonstrates how to use widely available census data to indi- rectly measure conflict mortality. Census data is generally quite comprehensive and includes mortality information on victims targeted by all combatant parties, those dying in both large and small events, those in both remote and accessible areas, as well as direct and indirect mortality levels. Census data has also been used in Weidmann (2009), who combined the Bosnian census with the ACLED conflict database to determine how conflict affects ethnic population concentration. Some researchers have uncovered important insights into the human capital effects of violent conflict through the creative use of historical datasets. One good example is Akbulut-Yuksel (2009), which shows how a unique data-set on city-level destruction in Germany caused by Allied Air Forces bombing during World War II can provide far- reaching insights when combined with a socio-economic panel. This is one of the first studies to capture the long-term human capital effects of violent conflict across several generations. Historical archives and surveys of survivors have been used recently by po- litical scientists to examine the long-term social and political legacies of the internal con- flict in Greece in the 1940s (Kalyvas 2006) and the Spanish Civil War (Balcells 2012). Using Demographic and Health Surveys (DHS) to Identify the Human Capital Effects of Violent Conflict Demographic and Health Surveys (DHS) are specialized surveys designed to monitor health, fertility, and mortality outcomes in several developing countries. Although Bru ¨ ck et al. 33 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 these surveys often lack information on conflict and violence—even when conducted in conflict-affected countries—several studies have leveraged them to analyze the demographic, health, and education effects of violent conflict. The methodologies followed are similar to those used with the LSMS. For instance, Bundervoet (2009) investigated the profile of victims of the 1993 killings in Burundi using the United Nations Population Fund (UNFPA) demographic survey conducted in Burundi in 2002. Verwimp and Van Bavel (2014) have used the same survey to study the effect of conflict in Burundi on the gender-gap in primary school completion. De Walque and Verwimp (2010) used Rwandan DHS data from 2000 and 1992 to estimate excess mortality in the 1994 genocide. De Walque (2004) made use of the DHS to assess the long-term impacts of the Cambodian genocide during the Khmer Rouge period. Schindler and Bru ¨ ck (2011) used DHS data to assess the effects of the geno- cide on (replacement) fertility in Rwanda. The potential of DHS in conflict research remains, however, largely underexploited. Responding to Challenges in the Use and Design of Standard Surveys Researchers have made use of existing socio-economic datasets collected for purposes other than conflict research by creatively using occasional questions related to conflict or by merging them with conflict event data. This approach makes good use of existing data and has provided valuable insights on the effect of conflict on socio-economic outcomes at the micro level. However, these studies face important shortcomings. Most notably, studies that rely on existing surveys that were not purposively designed to understand conflict processes—as is the case with most of the current literature— are generally based on an ad-hoc treatment of conflict, typically based on a limited number of self-reported indicators of household exposure to violence. These shortcomings can be mitigated through the use of high-quality event data- sets, as demonstrated in several studies discussed above. The main downside of this approach is that matching datasets may be difficult, either because names of loca- tions or identities of respondents cannot be reconstructed, or because researchers are not able to access that information for confidentiality reasons. However, given the many security, financial, and human resources trade-offs faced by researchers working on violent conflict at the micro-level, large institutionalized socio-economic surveys will often be the best option for gathering conflict research information. A question remains: Can these surveys be improved in ways that will advance research and knowledge on conflict processes at the micro-level? Below we suggest a number of guidelines to improve the sensitivity of future stan- dard socio-economic surveys conducted in conflict-affected contexts. We propose four ways in which these surveys could be adapted to capture more accurately 34 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 conflict-related events: (i) allow respondents to self-report on conflict events more comprehensively by including conflict-related scenarios in answer categories; (ii) record the timing of events; (iii) be sensitive to the type and intensity of violence; and (iv) include conflict questions across several survey sections and include a range of conflict-related choices in answer categories. We have used these guidelines elsewhere to construct the “Conflict Exposure Module” to be inserted in existing socio-economic surveys conducted in conflict-affected countries.6 Respondent Self-reporting Asking respondents to self-report on how conflict has affected them is a straightfor- ward way to learn more about conflict dynamics. Too few existing socio-economic surveys extend answer choices to give respondents the opportunity to explain how different dimensions of the conflict may have affected them. For example, the LSMS conducted in Tajikistan 2007 asked respondents why they did not work in the past 14 and 30 days. This question could be modified to provide a better understanding of conflict effects by including answer options such as “lack of security” or “disabil- ity due to violence”. The 2006 Iraq LSMS is one example where the effect of conflict on income is addressed by including answer options that point to “security” and “handicaps” as reasons for inability to work. These categories could be made more comprehensive by including a broader range of conflict-related scenarios, such as discrimination, crime, destruction of assets, disappearance of key markets, or mili- tary service. The use of self-reported information has experienced setbacks. Self-reported answers are highly subjective and may introduce biases because respondents may er- roneously recall the reasons why they made certain decisions. In situations as intense as conflict, respondents may construct a narrative that gives disproportionate weight to extraordinary and memorable experiences. Respondents may also be unable to dis- tinguish between ultimate and proximate causes. Despite these potential limitations, including conflict-related scenarios in the answer categories of questions routinely asked in large socio-economic surveys would be a cost-efficient way of adapting exist- ing surveys to better understanding conflict contexts. Some of these issues may be solved through econometric techniques such as the use of instrumental variables. Sensitivity to the Timing of Events In several of the studies we reviewed for this paper, carefully recorded information on when events occurred has allowed researchers to match socio-economic out- comes to conflict events. However, few socio-economic surveys in conflict-affected areas record time information systematically. Detailed time information would allow researchers to know whether events occurred before, after, or during a Bru ¨ ck et al. 35 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 conflict, and therefore capitalize on conflict event databases that provide a localized history of conflict events. Examples include time information on when household members left or joined their families, when income, asset, and food consumption losses occurred, when coping measures were introduced, when harm was inflicted, and when people were displaced or migrated. Researchers may also be able to construct localized conflict timelines by record- ing dates of major conflict events and allowing respondents to describe when an event occurred in reference to these timelines. This usually requires researchers to establish a local conflict event timeline beforehand, preferably with the aid of an initial community questionnaire. Respondents would then be able to remember whether specific events in their lives may have occurred in relation to, for instance, major attacks or other easily recalled events. The overarching goal of being time sensitive is to align respondents’ answers to major changes in the conflict’s dynam- ics, rather than produce a precise chronology. Existing socio-economic surveys can be adapted without adding substantial extra costs by introducing questions that capture the timing of events. Sensitivity to the Type and Intensity of Violence Micro-level surveys in conflict-affected areas could be more sensitive to different mo- dalities of violence that take place during violent conflicts. Each conflict creates its own hardships, which local populations feel acutely. Measuring how much people suffer is important for determining the sources of acute hardship and the type of violence borne by the population. For example, respondents in Angola suffered physical injuries from the widespread use of land mines, while Palestinians have suf- fered losses of income from the difficulty of moving through checkpoints. Further, Congolese families are ripped apart because of the widespread use of rape and sexual violence. Sensitivity to the type and intensity of violence requires answer categories to go beyond simple binary variables by including a range of responses at different levels of intensity. Comprehensiveness Overall, surveys could be more comprehensive in relation to conflict-related events by covering a broad range of channels whereby conflict may affect both lives and liveli- hoods. While LSMS are by their nature very comprehensive, conflict-sensitive ques- tions are often left out of many modules. Surveys that focus too narrowly on select categories may fail to account for the multi-dimensional impact of violent conflicts. In order to address this issue, socio-economic surveys could include conflict-related questions across their various sections on demographics, economic welfare, coping 36 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 activities, health, migration, education, perceptions of security, life satisfaction, and expectations.7 Not all existing socio-economic surveys will be able to follow these guidelines at all times given the number of financial, political, and logistical tradeoffs they may entail. However, more investment in the development of appropriate methodological systems may lead to considerable advances in gathering rigorous, systematic, and comparable evidence across different conflict-affected contexts. Better knowledge will, in turn, result in better and more effective policy interventions to provide physical and economic security to the millions of men, women, and children that continue to live in persistent cycles of violence and conflict. Recent Methodological Advances for Surveying in Conflict-affected Areas Purposely designed studies make up an emerging body of research on the causes and impacts of conflict at the micro level. These studies’ great advantage is that they identify and measure conflict directly within the survey questionnaire, allow- ing researchers to identify more precisely the complex causal mechanisms that shape the relationship between violent conflict and individual, household, and com- munity welfare and behavior. We review here examples from five types of purposely built surveys used in the literature: (i) ex-combatant surveys; (ii) genocide and atrocities surveys; (iii) surveys of displaced populations; (iv) post-conflict reconstruc- tion surveys; and (v) conflict surveys conducted among civilian populations. Ex-combatant Surveys Several surveys have been conducted to analyze the experiences of specific popula- tion groups living through violent conflict, notably former soldiers and members of rebel movements. Below we discuss four surveys that have developed novel and crea- tive instruments to capture processes of recruitment, armed group support, and combatant-civilian relations. The Surveys of War Affected Youth (SW AY) in Northern Uganda, conducted by Chris Blattman and Jeannie Annan in 2005 and 2006, have made an important contribution to research on the micro-level effects of violent conflict.8 The surveys implemented a novel sampling methodology—employing a technique which the authors define as “retrospective sampling”—that attempted to reconstruct the sample before the conflict took place. This helped the researchers correct for attri- tion bias resulting from conflict-related deaths or migration. The study also Bru ¨ ck et al. 37 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 creatively used culturally specific indicators to measure psychosocial well-being, such as “nightmares and insomnia” or “perceptions of haunting by spirits”. Although culturally-specific, introducing these questions has provided important directions for the collection of hard-to-quantify indicators in survey instruments. Humphreys and Weinstein (2004; 2008) conducted a novel survey on the motiva- tions of Sierra Leonean ex-combatants for joining and staying with armed groups, and their attitudes towards disarmament, demobilization, and reintegration. In par- ticular, the authors were able to reconstruct time periods within the conflict by asking respondents to recall their geographic location during active participation in armed groups at specific periods during the war. This allowed them to analyze both processes and motivations of recruitment across time. In order to assist recall, the authors constructed a timeline of well-known events and dates and placed individual answers within these different time periods. Another innovative aspect of this survey is the way in which sensitive questions were phrased. In order to avoid respondents feeling compromised by their answers, the survey focused on asking whether respon- dents observed potentially incriminating events (such as theft, rape, and assault), rather than asking whether respondents perpetrated them personally.9 Overall, con- trary to established wisdom that portrays ex-combatants in Sierra Leone as motivated by greed and the gains of looting, the survey finds evidence for an array of motiva- tions for joining and staying in armed groups, ranging from fear and forced recruit- ment, to access to protection and the provision of basic needs. Arjona and Kalyvas (2008) also studied individual motivations for joining armed groups in Colombia; their survey was conducted among 732 ex-combatants of a leftist guerrilla group and a right-wing paramilitary group. The sampling ap- proach followed in this survey is more challenging than the survey conducted in Sierra Leone due to security concerns in various sampling areas, and the reliance on sample frames drawn exclusively from a national demobilization and reintegra- tion program. These factors have prevented the authors from constructing a repre- sentative sample. However, the survey is unique in that it offers extensive information on individual motivations for joining armed groups, as well as on how different armed groups are organized and their relationship with civilian popula- tions, which has allowed a better understanding of how armed groups exercise control of different populations, and how local populations adjust their behavior in the presence of armed groups (see Arjona 2010).10 Mvukiyehe, Samii, and Taylor conducted over 3,000 interviews in 2007 among ex- combatants in Burundi, primarily focusing on armed group recruitment. This survey was implemented among combatants and non-combatants in order to identify how ex- periences of violence differed between the two groups. The survey covers personal expe- riences of violence, which have not been captured elsewhere, such as physical mistreatment, sexual abuse, or forced labor. This information is important for the design of policy interventions that consider the specific war experiences of combatants. 38 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Genocide and Atrocities Surveys Genocides are extreme violent events that produce enormous welfare impacts and pose unique challenges for researchers. In this section, we discuss two studies: the Genocide Transition Survey (2000) and the Darfur Refugee Questionnaire (2006). The Genocide Transition Survey (2000), conducted in Rwanda by Verwimp, is one of the first examples of the potential for panel survey-based research in conflict- affected areas.11 Verwimp tracked the fate of household members who had been interviewed in a nationwide agricultural survey prior to the 1994 genocide. In addition to important insights into the profiles of perpetrators (Verwimp 2005) and victims (Verwimp 2003b) of the 1994 genocide in Rwanda, this survey has demon- strated that tracking households and individuals is possible even under the difficult circumstances of such extreme violent events. By creating a systematic profile of genocide perpetrators, the survey presented a more nuanced portrayal of the geno- cide in Rwanda beyond the simple discourse about ethnic rivalry between Hutu and Tutsi. The study particularly showed how economic considerations and issues around land distribution may have affected the outbreak of the violence as much as pre-existing ethnic rivalries. The Darfur Refugee Questionnaire (DRQ) laid the foundations for the U.S. State Department to declare the killings in Darfur as genocide. The survey solicited a de- scription of violent acts from the victims surviving in refugee camps and linked them to their perpetrators. Details on this complex survey are provided in Totten and Markusen (2006). This survey has been critical in policy interventions aimed at the return and reintegration of displaced populations, and the design of political settlements that may avoid the re-ignition of the conflict. Surveys of Displaced Populations The welfare losses suffered by displaced persons are an important area of research in conflict studies. Two of the most prominent surveys of displaced populations are the Northern Uganda Livelihood Survey (NULS; 2007) and the Deininger, Iba ´n˜ ez, and Querubin (2004) study in Colombia. The NULS was conducted in 2007 (see Bjørkhaug et al. 2008). The survey is a follow-up of the 2005 Northern Uganda Internally Displaced Persons Profiling Study and the 2006 Lira District Early Recovery Needs Assessments conducted by Fafo in Norway. The carefully phrased questions in NULS are specific enough to capture important changes in people’s lives due to violent conflict, including motivations for migration, experience of violent crime and abduction, other forms of victimization and causes of health problems, as well as expectations of the future. Additionally, the survey identifies whether the person was a combatant, thus providing important Bru ¨ ck et al. 39 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 information on how combatants and displaced civilians may experience violence. Studies using the NULS 2007 include Bozzoli, Bru ¨ ck, and Muhumuza (2011 and 2012). Deininger, Iba´n ˜ ez, and Querubin (2004) use an unusually large survey conducted among 32,093 households applying for assistance from the Catholic Church in Colombia to investigate the decision to return after displacement. The survey contains unique information on the causes of displacement, household demographics, access to land, and labor market and education outcomes. This is one of the few surveys available that trace the movements of displaced people. Information was collected only if people requested assistance from the church, which may lead to some selection biases. However, this information has been used insightfully to examine the extent of asset losses and labor market prospects of displaced people (Iba ´n ˜ ez and Moya 2009), the determinants of displacement (Engel and Iba ´n ˜ ez 2007), and labor supply out- comes and wage changes for displaced people (Caldero ´n ´ n and Iba ˜ ez 2009). These two surveys have also been paramount in questioning some of the assump- tions underlying return policies for refugee and displaced populations. Notably, the two surveys have highlighted the fact that many population groups may not desire to return to original locations for a variety of reasons ranging from fear of violence, to the fact that they have, over the course of displacement, built new lives in their new locations that they do not want to abandon (Bozzoli et al. 2013). These find- ings are at odds with international efforts to relocate displaced and refugee people, and raise important implications in terms of reconciling the needs of displaced and host communities. Post-conflict Reconstruction Surveys Some surveys have been developed by international institutions operating in con- flict zones to assess the sustainability of post-conflict reconstruction. One example is the Standardized Monitoring and Assessment of Relief and Transitions (SMART) surveys, which provide a standardized methodology for measuring key statistics in the wake of complex emergencies (including conflict). The method focuses on basic indicators such as the nutritional status of children under 5 and mortality rates.12 The International Committee of the Red Cross (ICRC) and the Greenberg Research team conduct the “People on War Surveys” in a variety of conflict-affected countries (ICRC 2009).13 The ICRC funds the surveys in part to assess the perception of its own interventions. The surveys are standardized so that results can be compared across all participating countries. To account for country-specific contexts, the wording of some questions is modified where necessary. Mvukiyehe and Samii (2008 and 2009) have conducted a survey to evaluate peacekeeping operations in the Cote d’Ivoire. This survey captures the potential for 40 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 conflict re-escalation by reporting on events and circumstances that might warn of renewed conflict. The survey also investigates perceptions of security amongst popu- lations and repeated violence against civilians in different locations. Similar to the Humphreys and Weinstein ex-combatant survey discussed above, respondents were asked whether or not they witnessed or suspected “inter-ethnic fighting, presence of armed groups, or recruitment by armed groups in their localities” in relation to time periods constructed between well-known events, which has allowed research- ers to assess how early conflict signs have evolved across time. Conflict Surveys Conducted among Civilian Populations Several socio-economic household surveys have incorporated comprehensive com- ponents that capture the effect of various forms of violent conflict on civilian re- spondents. The Burundi Priority Household Panel (1998, 2007, and 2012) was designed to provide detailed information on the welfare effects of the civil war in Burundi by comparing households in villages affected by the war with households in non-affected areas (see Bundervoet et al. 2009). The survey features questions on violence and conflict at the individual, household, and community levels. The panel design, collected in three waves in 1998, 2007, and 2012, captures compara- ble data on welfare before and after incidences of violence. Special attention was given to tracking individuals who left the household since the first wave of the survey (Verwimp and Bundervoet 2009). The same team followed up the results of this survey with experimental behavioral games in conflict-affected and non-affect- ed areas in 2009 (Voors et al. 2012). This unique setup has allowed the researchers to link outcomes measured in the survey with behavioral data, thereby proving im- portant insights into how exposure to violent conflict may affect people’s fundamen- tal preferences and pro-social behavior. The Life in Kyrgyzstan Survey (LIK) interviewed 3,000 households annually over 3 years (2010 to 2012) to create a nationally representative panel (Bru ¨ ck et al. 2014). The LIK surveys cover a comprehensive list of topics, following similar proto- cols to the LSMS, with the additional inclusion of modules designed to elicit infor- mation about local tensions and conflicts, which has allowed researchers to better understand how local conflicts may affect development outcomes. The Maharashtra Household Longitudinal Survey (MHLS) was conducted in 2010 by the Institute of Development Studies (UK) among 1,089 households living in vio- lence-affected areas in the Indian state of Maharashtra (see Gupte, Justino, and Tranchant 2012). The same households were surveyed again in 2012. This unique panel study was designed to capture how individuals and households live in areas characterized by persistence communal violence in India. This survey has allowed Bru ¨ ck et al. 41 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 researchers to understand for the first time the intersection between economic vul- nerabilities and exposure to persistent forms of urban violence in slum areas in India. The Colombian Longitudinal Survey of Wealth, Income, Labor, and Land (ELCA) interviewed 10,000 households in rural and urban areas affected by conflict in Colombia. The first wave was conducted in 2010 and the second wave took place in 2013. This is the most comprehensive longitudinal survey ever conducted among conflict-affected populations. A novel aspect of the survey is the questionnaire modules on activities of armed groups in different neighborhoods (see Gafaro, Ibanez, and Justino, forthcoming), as well as detailed information on migration, re- cruitment, and local cooperation with armed groups. These surveys allow for the first time a comprehensive understanding of how violent conflict in its various mo- dalities may lead to specific changes in institutional arrangements and individual and group behavior that may explain the persistence of violence in some areas of Colombia, but not others, with profound implications for the ongoing peace process in Colombia, as well as conflict resolution interventions elsewhere in the world. Responding to Challenges in Purposely Designed Surveys Purposely designed surveys conducted in conflict-affected areas are the state-of-the- art of empirical conflict research at the micro-level. There are, however, only a limited number of these surveys due to a number of challenges. First, primary fieldwork re- search in conflict-affected countries is quite expensive due to a lack of infrastructure and difficulties in engaging with suitably qualified local field researchers. Second, security concerns can also pose difficulties to the research team and their respon- dents. In addition, researchers face a series of methodological challenges. Some of the most prevalent challenges include the following: (i) defining conflict at the micro- level; (ii) choosing the appropriate unit of analysis; (iii) identifying time dimensions in survey questionnaires; (iv) dealing with data biases (such as selection bias and recall error); and (v) addressing ethical and security issues associated with doing primary research in conflict-affected contexts. We discuss these common challenges below. Defining Conflict at the Micro-level One of the most important challenges to designing surveys in conflict-affected con- texts is to create and operationalize a definition of conflict that captures the complex impact of conflict on the lives of individuals, households, and communi- ties. Several authors have proposed more or less overlapping typologies of violent conflict (Gupta 1990; Singer and Small 1994; Sambanis 2001). These definitions are useful for understanding conflict as a macro-phenomenon, but are difficult to 42 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 uphold at the micro level because they are too far removed from the everyday distur- bances experienced by local populations (Justino, Bru ¨ ck, and Verwimp 2013). An additional difficulty in defining violent conflict from a micro-level perspective is to determine when a violent conflict starts and ends. In particular, lulls or spikes in violence may make the conflict feel as if it starts and stops rather than persists at a continuous intensity. Even after a conflict has subsided at the national level, the persistence of lower levels of violence and instability may continue to affect house- holds and their members. Likewise, as conflicts draw to a close, changes in the iden- tity of the belligerents may create new coping dilemmas for the population. Many individuals and groups living in conflict-affected areas find themselves, therefore, responding, acting, and being affected by stages in between conflict and peace. Macro-level concepts of time periods may miss these nuanced variations at the micro-level. As conflicts change frequently over place, time, and context, it is neces- sary to have a broad definition of conflict while also establishing observable charac- teristics that can be easily captured through empirical data collection. The Unit of Analysis The second methodological point in the design of surveys in conflict-affected con- texts is the choice of the appropriate unit of analysis for different types of ques- tions.14 Depending on the type of information sought, survey questions typically target individuals, household heads, or central figures in the community. The indi- vidual is the lowest level of analysis. The objective of using individual-level questions is to determine how individual decision-making—across gender, age and different socio-economic backgrounds—may respond to the impact of violence on liveli- hoods, well-being, and security. Individual-level surveys may also be able to capture specific individuals that may have been directly involved in the conflict, such as sol- diers, refugees, and displaced people. In household-level surveys, the head or another member of household responds on behalf of the household. Household-level questions allow the assessment of the impact of shocks on households and their reactions as collective decision makers. Questions can target changes in access to services, markets, investments, and land, which may affect the entire household even if only a few members are directly in- volved. Household-level questions can, in addition, be used to draw a broader picture about social relations and networks. Community-level questions may also be useful for uncovering the extent of the impact of violent conflict, especially when violent conflict events are concentrated in time and space. Community-level survey components may be able to generate a con- flict history that records the overall characteristics of localized events. This information can be used as a starting point for designing household and individual-level surveys and, crucially, to provide context when gathering time information. Community-level Bru ¨ ck et al. 43 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 analysis may also allow for a more accurate determination of deaths across the com- munity, for instance by examining listings of names in local administrative records. Moreover, knowledgeable members of communities often provide important qualita- tive and quantitative insights. Community-level questions are also useful for assessing migration flows, urgent needs of the community, and the impact of policy interven- tions, particularly reconstruction interventions in the post-conflict period. Time Dimension The timing of surveys is decisive for the quality of subsequent analysis. As with most socio-economic data, the quality of people’s responses about conflict processes tends to diminish as the time between the conflict and survey widens. Yet in many instanc- es, the intensity of conflict experiences may facilitate memory, and data collection in conflict-affected areas may be done well after a conflict has ended. However, much depends on the circumstances of the conflict and its long-term legacies. Researchers have used three main approaches to survey timing, including administering the survey while the conflict is ongoing, administering the survey ex post by asking re- spondents to assess before and after conditions, and making use of panel data. If the violent conflict is still ongoing at the time of the survey, researchers tend to use a 12-month reference period to elicit information on the short-term effects of violent conflict on individuals and households. This reference period has a number of advantages. First, its frequent use in other socio-economic surveys may allow for comparability. Second, this time period is useful for gathering economic data that may contain seasonal effects, such as any indicator linked to agricultural or climac- tic cycles. However, before employing a 12-month reference period, researchers should ensure that this is appropriate given the dynamics of the conflict. It may be that in the last 12 months, or in the period immediately before, a major conflict event significantly affected the respondents. In these cases, it may be better to refer to the conflict event specifically in the reference period. Researchers may employ variants of prompting devices such as “since your village was attacked,” “since the beginning of the conflict,” or “since armed fighting ceased in your area.” Surveys cannot always be conducted very close to conflict events. When conduct- ing a survey some time after a conflict, researchers have addressed the issue of tem- poral comparison by asking respondents to recall aspects of their lives before and after the conflict. Many questions asked in existing surveys address the problems of missing ex ante data by regularly using phrases such as “before the conflict” or “since the start of the conflict”. These types of questions can create further time var- iation by asking respondents to recall living standards at specific points during the conflict, usually demarcated by well-known events. Humphrey and Weinstein’s (2008) work in Sierra Leone provides a good example of this technique. Such ex post surveys may, however, introduce potentially severe biases, as respondents 44 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 may erroneously recall events, overestimate their levels of welfare before the conflict, or samples may exclude important sub-groups. These surveys must therefore pay particular attention to the design of reliable timelines of events and other mecha- nisms to elicit accurate information from respondents’ memories. Researchers may be able to collect longitudinal data if they are fortunate enough to have access to a survey that was done before, and reasonably close to, the onset of conflict. Panel datasets offer rich time variation and minimize many of the con- cerns about biases prevalent in other methods. However, follow-up surveys must be especially careful to control for attrition. People in conflict-affected areas tend to be highly mobile and subject to a high degree of mortality, making them difficult or im- possible to include in the follow-up survey. When these groups systematically differ from the overall population, excluding them will bias the sample. However, recent work discussed above in Burundi, Rwanda, Indonesia, India, and Colombia, among others, has shown that reliable longitudinal data can be effectively collected in con- flict-affected countries. Data-related Biases Research in conflict-affected areas takes place under unusual and often insecure cir- cumstances, adding extra difficulties for researchers attempting to create a represen- tative sample. This may even continue into the post-conflict period as governments may exclude researchers from sensitive areas, such as recently in Sri Lanka, Egypt, Syria, or Mali. Entire areas may be inaccessible, forcing researchers to rely on ex post surveys. Respondents may also self-censor answers to avoid any risk of retribution from insecure political authorities. Conflict researchers have employed several strate- gies to minimize such biases. Below we discuss some of the most prevalent biases in conflict contexts: selection biases and recall or response errors. Selection Biases Selection biases occur when individuals, households, or groups of households with particular characteristics cannot be sampled or interviewed. For example, declining economic activity during a conflict may result in entrepreneurial individuals migrat- ing out, changing the characteristics of the population left behind. Or combatants may target specific ethnic and social groups during the conflict, forcing targeted pop- ulations to migrate or leading to a large number of deaths among specific population groups. In addition, panel datasets may suffer from attrition bias, as discussed above. The SW AY surveys addressed this issue by asking close family members to respond to an “absentee survey” on behalf of the respondent when tracking was impossible. This and other methods can be employed to help researchers generate a sample that has an equal likelihood of including individuals or households who have died or Bru ¨ ck et al. 45 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 migrated out of a conflict-affected area. If respondents have died, tracking becomes impossible. Even in these circumstances researchers must take account of the poten- tial biases created by attrition (see Justino and Verwimp 2012). Recall and Response Errors The length of recall periods has been a topic of discussion in socio-economic, de- mographic, and epidemiological surveys for a long time (Deaton 2001). Exposure to conflict may sometimes aid recall due the intensity of the event. However, the reverse scenario is also possible when respondents may repress traumatic memo- ries or even refuse to talk about them. Training and sensitization of survey enu- merators can alert them to the potential for these biases. As discussed above, recall can be aided by using event timelines that stimulate the respondent’s memory and accurately situate personal events in time. Other forms of response errors may take place in surveys conducted in conflict-affected contexts. For in- stance, if the survey sponsor is viewed with distrust or suspicion, respondents may provide misleading answers to questions or even refuse to cooperate entirely. In some post-conflict situations, government-sponsored surveys may run this risk, especially in formerly rebel-held territories. This may create biases if questions about conflict are included, as responses from those most affected may be inaccu- rate or absent. In the extreme, lingering animosity towards the government might be so strong that discussing conflict could unsettle the fragile peace and put the security of survey workers into jeopardy. For example, in the LSMS survey in Guatemala in 2000, the authorities asked relatively few conflict questions, though the conflict had only recently ended, because they feared that the population in formal rebel territories would refuse to participate. In post-genocide Rwanda, the government does not allow researchers to ask questions about ethnicity in surveys, thus limiting the capacity of researchers to link ethnicity with other variables of in- terest such as poverty, displacement, or gender. Ethical and Security Considerations Conflict surveys risk asking questions that may do harm to respondents. Researchers have a duty to weigh important ethical considerations while designing and imple- menting surveys. Sensitive questions may evoke traumatic memories about suffering, remorse, victimization, or guilt, potentially “re-traumatizing” respondents and harming them psychologically. Some questions may also risk incriminating or inviting retribution upon a respondent. If answers inadvertently become public, responses that identify perpetrators, victims, or actions taken by former combatants are particu- larly susceptible to this risk. Several mechanisms have proven useful to address and minimize potential ethical risks. The first and simplest way is to avoid asking some of 46 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 these questions. Researchers should be self-critical about whether their questions are strictly necessary, potentially harmful, or if there are less risky ways of obtaining the same information. To limit the risk of harm, it is also generally good practice to ask about group behavior rather than asking for specific names of perpetrators. The training of enumerators and a local research team is a crucial aspect of doing high-quality surveys in sensitive and insecure areas. Ethics training should also provide comprehensive information about the risks that respondents face, and on adequate security responses.15 Proper training should also make enumerators aware of the emotional stresses associated with addressing sensitive issues sur- rounding conflict. A well-designed consent script is also important. Researchers should also be ready to respond should a harmful event occur, or should respon- dents become unduly distressed. Final Remarks and Further Advances in Conflict Data Collection We discussed in this paper how empirical research on conflict processes at the micro-level has flourished over the last few years due to the wider availability of good quality surveys conducted in conflict-affected countries. Some of the most comprehensive insights into the causes and consequences of violent conflict at the micro-level have been generated through purposely designed surveys. These surveys have been used to uncover unfolding process of conflict, rather than assessing con- flict as a one-off shock, because they are able to collect systematic information on the various channels whereby different forms of violence may affect individuals and households. Well-designed surveys also provide valuable disaggregated information on conflict processes across time and place. However, these surveys require a lot of resources: sample sizes are large, interviews sometimes last several hours, transpor- tation costs are high, and local expertise is crucial. Insecurity may also still be high in many of the areas being surveyed. One way of minimizing these costs is to rely on existing large socio-economic surveys conducted by The World Bank and other international institutions, which may either contain self-reported information on conflict exposure, or can be matched to conflict-event datasets. This approach is less ideal due to the lack of precise information on the complex facets of conflict in each case study, as well as difficulties in comparing information across countries. Empirical work based on existing socio-economic surveys has nonetheless resulted in considerable advances in our current knowledge on violent conflict at the micro-level due to the develop- ment of a wealth of creative research methods to analyze conflict processes among violence-affected populations and across time. Although information on conflict is sometimes limited, the use of existing socio-economic surveys has many advantages over the use of purposely designed surveys, notably its costs and ease of availability. We suggested potential solutions for how these surveys can be made more Bru ¨ ck et al. 47 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 comprehensive in a conflict-sensitive manner. We hope that these suggestions and the wide discussion on various survey methods offered by this paper will lead to further and better efforts to collect much needed information on the lives of individ- uals, households, and communities affected by violent conflicts across the world. Notes Tilman Bru ¨ ck is with IGZ - Leibniz Institute of Vegetable and Ornamental Crops, Großbeeren, Germany, ISDC - International Security and Development Center, Berlin, Germany, and the Households in Conflict Network. Correspondence may be sent to: tilman.brueck@isd-center.org. Patricia Justino is with the Institute of Development Studies, UK. Philip Verwimp is with Ecares, Centre Emile Bernheim and Solvay Brussels School of Economics and Management, Universite ´ Libre de Bruxelles, Belgium. Alexandra Avdeenko is with Mannheim University. Andrew Tedesco is an indepen- dent consultant. The authors would like to thank Raka Banerjee, Kristen Himelein, Gary Milante, and Diane Steele for their helpful comments and support. This paper is a shorter version of a sourcebook on conflict measurement using household surveys developed for the World Bank (Bru ¨ ck et al. 2013). 1. See www.microconflict.eu and www.hicn.org. 2. For example, the LSMS conducted in Timor-Leste in 2001 (after the country had experienced intense violent events in 1999) asks only two questions on war damage, focusing almost exclusively on damage to dwellings. The LSMS conducted in Tajikistan in 2003 shortly after the Tajik civil war asks only one question about war damage. 3. These include Azerbaijan (1995), four waves in Bosnia & Herzegovina (2001–2004), Guatemala (2000), Iraq (2006), Kosovo (2000), Nepal (1995/96, 2003/4, 2010), two waves in Peru (1991, 1994), Serbia (2002, 2003, 2007), Tajikistan (1999, 2003, 2007, 2009), Timor-Leste (2001, 2007), and Malawi (2004, 2010). For an excellent description of the design and use of the LSMS, refer to Deaton (2000). 4. The Armed Conflict Location and Event Dataset (ACLED), is a comprehensive dataset on political violence, including information on specific dates and locations of political violence, types of event, groups involved, fatalities, and changes in territorial control. 5. De Luca and Verpoorten (2011) have conducted a related study using the same datasets. 6. See http://www.ids.ac.uk/files/dmfile/Measuring_Conflict_in_Micro_Level_Surveys.pdf. 7. Concrete examples are provided in: http://www.ids.ac.uk/files/dmfile/Measuring_Conflict_in_ Micro_Level_Surveys.pdf. 8. See http://chrisblattman.com/projects/sway/ for a description of the project. 9. See also Fearon, Humphreys and Weinstein (2009), and the Sierra Leone PRIDE/JCTJ (2002). A useful website for the collection of information on different surveys is the Post-Conflict and Ex- Combatant Surveys, http://www.columbia.edu/~mh2245/XCSURVEYS/. 10. A similar methodology has been adopted by Guichaoua (2007) to examine motivations to join insurgent and incumbent groups in Nigeria. 11. See Verwimp (2003a) for a description of the survey. 12. The Centre for Research on the Epidemiology of Disasters compiles the Complex Emergencies Database (CE-DAT), which includes SMART survey data. 13. Afghanistan, Colombia, Democratic Republic of the Congo, Georgia, Haiti, Lebanon, Liberia, and the Philippines. 14. For a discussion of methodology in practice, see Green and Tony (2008), Verwimp and Bundervoet (2009), and Bundervoet et al. (2009). 15. Two American IRB-approved ethics training exercises can be found at https://www. citiprogram.org and http://phrp.nihtraining.com/. MICROCON, a large integrated program on conflict research funded under the EU’s 6th Framework, developed guidelines for the ethical review of all data collection efforts prior to field work. See www.microconflict.eu. 48 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Appendix I: Surveys with Conflict-related Questions by Regions Angola “Demographic and Health Survey (DHS).” 2006/7, 2011. Azerbaijan World Bank. 1995. “The Living Standards Measurement Study (LSMS) Azerbaijan, Household Survey.” Demographic and Health Survey (DHS), 2006. Bosnia & Herzegovina World Bank. 2001 – 2004. “LSMS Bosnia & Herzegovina, Household Survey.” Burundi Burundi Institute of Statistics. 1999. “The Burundi Priority Survey, October 1998 and March 1999.” Demographic and Health Survey. 2010. Samii, C., E. Mvukiyehe, and G. Taylor. 2007. “2007 National Survey Instruments, Burundi. Civilian Questionnaire – English Translation.” https:// files.nyu.edu/cds2083/public/burundisurvey/. Bundervoet, T., E. Nillesen, P. Verwimp, and M. Voors. 2009. “Integrating Conflict Questions in a Household Survey: An Example from Burundi.” HiCN Research Design Note 12. Households in Conflict Network. Chad Demographic and Health Survey. 2004. Colombia PROFAMILIA, Asociacio ´ n Pro-Bienestar de la Familia Colombiana and Macro International. 1995, 2000, 2005, 2010. Demographic and Health Survey (DHS) Colombia. Arjona, A.M., and S.N. Kalyvas. 2008. “Rebelling Against Rebellion: Comparing Insurgent and Counter-insurgent Recruitment.” http://www.crise.ox.ac.uk/copy/ mobilisation%20conference/arjona_kalyvas.pdf. See also https://www. kent.ac.uk/politics/carc/reading%20group/Arjona%20Kalyvas% 20Mobilisation%20for%20Poltiical%20Violence.pdf. Unpublished manuscript. Cote d’Ivoire Mvukiyehe, E., and C. Samii. 2008. “Laying a Foundation for Peace? A Quantitative Impact Evaluation of the United Nations Operation in Cote d’Ivoire.” http://www.columbia.edu/~cds81/docs/unoci/ics2008_report081218.pdf. Darfur The “Darfur Refugee Questionnaire” (DRQ) is published as Appendix I in: S. Totten and E. Markusen, eds. 2006. Genocide in Darfur: Investigating the Atrocities in the Sudan. Routledge, Taylor & Francis Group, New York. Democratic Republic Humphreys, M. 2007. “Democratic Republic of Congo,” (in French). http://www. of Congo columbia.edu/~mh2245/DRC/survey.pdf. Demographic and Health Survey (DHS). 2007. Eritrea Demographic and Health Survey (DHS). 2002. Gaza FAO survey, conducted in February-March 2009. http://www.apis.ps/documents. php?page=fs_reports. Guatemala World Bank. 2000. “LSMS Guatemala, Community Survey.” Demographic and Health Survey (DHS). 1995 – 1999 (3 surveys). Indonesia Barron, P., M. Humphreys, Y. Tajima, and J. Weinstein. “World Bank Aceh household and XC survey.” (ARLS) Household and XC Survey. World Bank. 2005. “GAM Reintegration Needs Assessment 2005. Survey Documentation.” See also: International Organization for Migration (IOM) prisoner survey. http://www.columbia.edu/~mh2245/XCSURVEYS/ ACEH_METHOD.pdf. Continued Bru ¨ ck et al. 49 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Appendix I: Continued Iraq World Bank. 2006. “LSMS Iraq, Household Survey– Individual Survey.” Fafo-Ais. 2004. “Iraq Living Conditions Survey 2004.” http://www.fafo.no/ais/ middeast/iraq/imira/Tabulation%20reports/english%20atlas.pdf. Fafo-Ais. “Iraqis in Jordan 2007, survey.” http://www.fafoarkiv.no/ais/middeast/ jordan/Iraqis_in_Jordan.htm, report: http://www.fafoarkiv.no/ais/middeast/ jordan/IJ.pdf. Kosovo World Bank. 2000. “LSMS Kosovo, Household Survey.” Liberia Taylor, G. 2007. “CHF International 2007. Ex-combatant Economic Reintegration Survey (Lofa County).” http://www.usip.org/sites/default/files/sr211.pdf. Pugel, J. 2006. “UNDP Liberia Ex-Combatant Survey Nr. 1. Field Guide for Enumerators and Supervisors. February-March 2006.” http://www.columbia. edu/~mh2245/XCSURVEYS/LIBERIA_FEB06_METHOD.pdf; survey accessible at: http://www.columbia.edu/~mh2245/XCSURVEYS/LIBERIA_FEB06.pdf (23/ 04/2010). (Results published in: Pugel, J. 2006: “What the Fighters Say: A Survey of Ex-combatants in Liberia.” UNDP Liberia.) Mvukiyehe, E., and C. Samii. 2008. Laying a Foundation for Peace in Liberia. December 23, 2008. Fearon, J.D., M. Humphreys, and J.M. Weinstein. 2009. “Can Development Aid Contribute to Social Cohesion after Civil War? Evidence from a Field Experiment in Post-Conflict Liberia.” American Economic Review: Papers & Proceedings 99 (2): 287 – 91. Data and Codebook available at Macartan Humphreys’ personal website. http://www.columbia.edu/~mh2245/. Demographic and Health Survey (DHS), 2007 –2011 (3 surveys). Malawi World Bank. 2004, 2010. “LSMS Malawi, Community Survey - Household Survey.” AfroBarometer 2005. “Attitudes to Democracy and Market in Malawi.” http://www.afrobarometer.org. Specifically, see http://afrobarometer.org/data/ malawi-round-3-codebook-2005. Demographic and Health Survey (DHS), 1996 –2010 (4 surveys). Mali AfroBarometer 2001. “ Attitudes to Democracy and Market in Mali.” http://afrobarometer.org/data/mali-round-1-codebook-2001. Mozambique Republic of Mozambique, Ministry of Agriculture. 2005. “National Agricultural Survey 2005, Small- and Medium-Sized Farms-Panel.” http://www.aec.msu.edu/ fs2/mozambique/survey/index.htm. (Though not directly related to conflict, this survey has a good section on “coping strategies”.) Nepal World Bank. 1996, 2003, 2005/6, 2010. “LSMS Nepal, Household Survey.” Samii, C., M. Gilligan, and K. Eck. 2009. Nepal Peacebuilding Survey: Study Design, December 10, 2009. http://microdata.worldbank.org/index.php/ citations/2205. Nigeria Guichaoua, Y. 2007. “Who joins ethnic militias? A survey of the Oodua People’s Congress in South-western Nigeria.” CRISE Working Paper. See http:// www.qeh.ox.ac.uk/publications/wps/crisewps. Continued 50 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Appendix I: Continued Papua New Guinea Household Income and Expenditure Survey 2008. Peru World Bank. 1991, 1994. “LSMS Peru, Household Survey.” Rwanda Rwandan Rural Labour and Death Survey, 2002. http://www.aec.msu.edu/fs2/ Rwanda/deathhistory_eng.pdf. ` re des Finances et de la Planification Institut National de la Statistique, Ministe ´ conomique, Kigali, Rwanda, 2006: Rwanda Demographic and Health Survey E 2005. Calverton: ORC Macro. Rwanda Prisoner Questionnaire. Discussed in: Straus, Scott (2007) “The Order of Genocide : The Dynamics of Genocide in Rwanda.” Genocide Studies and Prevention 2 (3): Article 6. Available at: http://scholarcommons.usf.edu/gsp/vol2/iss3/6. Davenport, C., and A. Stam. “Butare Survey.” http://www.columbia.edu/ ~mh2245/XCSURVEYS/BUTARE.pdf. Serbia World Bank. 2002, 2003, 2007. “LSMS Serbia, Household Survey (individual).” Sierra Leone Humphreys, M., and J. Weinstein. 2003. “Sierra Leone Ex-Combatant Survey #1.” http://www.columbia.edu/~mh2245/Survey.pdf. PRIDE/ International Center for Transitional Justice. 2002. “Ex-Combatant Views of the Truth and Reconciliation Commission and the Special Court in Sierra Leone.” http://www.ictj.org/images/content/0/9/090.pdf. Demographic and Health Survey (DHS), 2008. Sri Lanka Demographic and Health Survey (DHS), 2006/7. Tajikistan World Bank. 1999, 2007, 2009. LSMS Tajikistan, Household Survey (pop. point). Timor-Leste World Bank. 2001. “LSMS Timor-Leste, Household Survey (individual), Timor- Leste- Survey of Living Standards 2007 and Extension 2008.” Demographic and Health Survey (DHS), 2009/10. Uganda Blattman, C. 2005. “Uganda: Survey of War-affected Youth (SW AY), Household- Survey.” http://chrisblattman.com/data/sway/. Blattman, C. 2005/6. “Uganda: Survey of War-affected Youth (SW AY), Phase 1 (Males) Individual survey.” http://chrisblattman.com/data/sway/. Blattman, C. 2007. “Uganda: Survey of War-affected Youth (SW AY), Phase 2 (Females) Individual survey.” http://chrisblattman.com/data/sway/. Fafo AIS. 2007. “Northern Uganda Livelihood Survey.” 2005 Northern Uganda Internally Displaced Persons Profiling Study 2006 Lira District Early Recovery Needs Assessments. Demographic and Health Survey (DHS), 1995– 2011 (7 surveys). Vietnam Kalyvas, S.N., and M.A. Kocher. 2009. “The Dynamics of Violence in Vietnam: An Analysis of the Hamlet Evaluation System (HES).” Journal of Peace Research 46: 335–55. www.prio.no/misc/Download.aspx?file...Data%2FKK_appendix.doc. Note: Further information on DHS and LSMS can be found online at http://www.dhsprogram.com and http://go. worldbank.org/IFS9WG7EO0, respectively. Bru ¨ ck et al. 51 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Appendix II: Studies using LSMS and DHS in conflict-affected countries LSMS or DHS survey Academic work Selected Results DHS Liberia 1986 Hegre, H., G. Østby and † Shows how pre-existing absolute and relative C. Raleigh. 2009. welfare influence conflict events in the Liberian civil war. DHS Rwanda 1992 and De Walque, D. and † Finds that genocide-related mortality was 2000 P. Verwimp. 2010. highest among educated and urban families. DHS Rwanda 1992 and Schindler, K. and T. Bru ¨ ck. † Observes an impact of the genocide on marital 2010. 2011. status, the marriage market, and, consequently, on fertility. LSMS Nepal 1995/6 and Pivovarova, M and E. L. Swee. † Shows how endogeneity and self-selection 2003/4 2012. issues create biases in estimating the micro effects of conflict in Nepal. † Demonstrates that “low ability” individuals are most likely to be displaced, suffering from the direct effects of conflict and the adjustment costs of displacement. LSMS Tajikistan 1999 & Shemyakina, O. 2006. † Finds that exposure to conflict has a 2003 significant effect on female enrollment and schooling attainment, but no effect on males. LSMS Tajikistan 1999 & Shemyakina, O. 2009. † Studies the effect of conflict on marriage and 2003 reproductive behavior, finding that conflict postpones marriage among women of marriageable age. LSMS Tajikistan 1999 & Justino, P. and O. Shemyakina. † Studies the effect of remittances on the labor 2003 2008. supply, finding that remittances decrease labor participation rates, especially in conflict areas. DHS Cambodia 2000 De Walque, D. 2004. † Finds that excess mortality during the Khmer Rouge period was especially likely among adult males, especially with urban or educated backgrounds. LSMS Kosovo 2000 Bhaumik, G., I. Gang, and † Studies the relationship between ethnic M. S. Yun. 2005. conflict and economic disparity, showing that, despite an advantageous economic position, Serb rates of poverty were higher than Albanians. LSMS Kosovo 2000 Alva, M., E. Murrugarra, and † Examines the costs of conflict in education, P. Paci. 2002. showing that ethnic tension has harmed Albanian male youth educational attainment. Continued 52 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Appendix II: Continued LSMS Kosovo 2000 Douarin, L., J. Litchfield, and † Finds that exposure to conflict predicts the R. Sabates-Wheeler. 2010. livelihood choices of households, such as the take up of wage labor, reliance on remittances or social assistance, or entrepreneurial activities. LSMS Bosnia Herzegovina F. Kondylis. 2007. † Investigates conflict’s effects on the labor market 2001 – 4 (4 surveys) and finds that “able” workers are more likely to be displaced and unemployed after conflict. LSMS Bosnia Herzegovina E. L. Swee. 2009. † Finds that war intensity, particularly the 2001 military draft, adversely affects secondary, but not primary, schooling attainment. LSMS Bosnia Herzegovina Do, Q. T. and L. Iyer. 2009. † Finds no significant differences on mental 2001 health from people who experienced different levels of conflict intensity. LSMS Timor-Leste 2001 & Justino, P., M. Leone and † Studies impact of conflict on the level and 2007 P. Salardi. 2014. access to education of children and finds a substantial loss of human capital accumulation among boys. LSMS Nepal 2003/4 M. Hatlebakk. 2007. † Studies the effects of Maoist influence on data collection quality, finding only minor impacts such as the need for approval. LSMS Nepal 2003/4 DHS C. Valente. 2011. † Finds that abductions by Maoists and conflict Nepal 1996 – 2006 intensity increased the probability of early (3 surveys) marriage but that only abductions by Maoists had a negative effect on school attainment. LSMS Nepal 2003/4 Menon, N. and Y. van der † Finds that women’s likelihood of employment Meulen Rodgers. 2011. increased because of conflict. UNFPA- Burundi 2002 Bundervoet, T. 2009. † Finds that older, wealthier and better-educated males were more likely to be killed in the 1993 Burundi massacres. † Finds that communal pressure for land increased the likelihood of killings. 36 DHS surveys Østby, G. 2008. † Finds evidence that “horizontal inequalities” that coincide with identity divisions aggravate grievance and promote social cohesion, facilitating mobilization for conflict. Note: World Bank Living Standards Measurement Studies are available at the following website: http://econ. worldbank.org/WBSITE/EXTERNAL/EXTDEC/EXTRESEARCH/EXTLSMS/0, contentMDK:21588800~menuPK: 4196952~pagePK:64168445~piPK:64168309~theSitePK:3358997,00.html. Bru ¨ ck et al. 53 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 References Akbulut-Yuksel, M. 2009. “Children of war the long-run effects of large-scale physical destruction and warfare on children.” Households in Conflict Network Working Paper 62. Alva, S., E. Murrugarra, and P. Paci. 2002. “The hidden costs of ethnic conflict decomposing trends in educational outcomes of young Kosovars.” World Bank, Policy Research Working Paper 2880, Washington, DC. Annan, J., C. Blattman, and R. Horton. 2006. “The state of youth and youth protection in northern Uganda findings from the survey for war affected youth. A report for UNICEF Uganda. September 2006.” Arjona, A. M. 2010. “Social Order in Civil War.” PhD Dissertation, Yale University. Arjona, A. M., and S. N. Kalyvas. 2008. “Rebelling against rebellion: comparing insurgent and counter-insurgent recruitment.” Unpublished manuscript. Balcells, L. 2012. “Violence and displacement. Evidence from the Spanish civil war (1936– 1939).” UFAE and IAE Working Papers 896.12. Unitat de Fonaments de l’Ana ` mica (UAB) and ` lisi Econo ` mica (CSIC). ` lisi Econo Institut d’Ana Beegle, K., J. De Weerdt, and S. Dercon. 2008. “Migration and economic mobility in Tanzania: Evidence from a tracking survey.” The World Bank, Policy Research Working Paper Series 4798, Washington, DC. Bhaumik, S., I. Gang, and M. S. Yun. 2005. “Ethnic conflict and economic disparity: Serbians & Albanians in Kosovo.” William Davidson Institute Working Paper 808. Bjørkhaug, I., M. Bøa ˚ s, A. Hatløy, and K. M. Jennings. 2008. “Returning to uncertainty? Addressing vulnerabilities in northern Uganda.” http://www.fafo.no/nyhet/return2uncertanity.pdf. Blattman, C. 2009. “From violence to voting: War and political participation in Uganda.” The American Political Science Review 103 (2): 231 –47. Blattman, C., and J. Annan. 2007. “The consequences of child soldiering.” Households in Conflict Network Working Paper 22. ¨ ck, and T. Muhumuza. 2011. “Does War Influence Individual Expectations?” Bozzoli, C., T. Bru Economics Letters 113 (3): 288 –91. . 2012. “Movers or Stayers? Understanding the Drivers of IDP Camp Decongestion During Post- Conflict Recovery in Uganda”. German Institute for Economic Research (DIW Berlin), DIW Discussion Papers, 1197. ¨ ck, and N. Wald. 2013. “Self-Employment and Conflict in Colombia”. Journal of Bozzoli, C., T. Bru Conflict Resolution 57 (1): 117– 42. ¨ ck, T., D. Esenaliev, A. Kroeger, A. Kudebayeva, B. Mirkasimov, and S. Steiner. 2014. “Household Bru Survey Data for Research on Well-Being and Behavior in Central Asia”. Journal of Comparative Economics 42 (3): 819–35. ¨ ck, T., P Bru . Justino, P. Verwimp, and A. Tedesco. 2013. “Conflict Survey Sourcebook: Measuring Conflict Exposure in Micro-level Surveys”. World Bank, Washington, DC. http://siteresources. worldbank.org/INTSURAGRI/Resources/7420178-1294259038276/Measuring_Conflict_in_Micro_ Level_Surveys.pdf. Bundervoet, T. 2006. “Estimating poverty in Burundi.” Households in Conflict Network Working Paper 20. Bundervoet, T. 2009. “Livestock, Land and Political Power: The 1993 killings in Burundi.” Journal of Peace Research 46:357– 76. 54 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Bundervoet, T., E. Nillesen, P. Verwimp, and M. Voors. 2009. “Integrating conflict questions in a house- hold survey: An example from Burundi.” Households in Conflict Network Research Design Note 12. ´ n, V Caldero ´n ., and A. M. Iba ˜ ez. 2009. “Labor market effects of migration-related supply shocks: Evidence from internally displaced populations in Colombia.” MICROCON Research Working Papers 14. Collier, P., and A. Hoeffler. 2004. “Greed and Grievance in Civil War.” Oxford Economic Papers 56 (4): 563–95. Czaika, M., and K. Kis-Katos. (2009). “Civil conflict and displacement - village-level determinants of forced migration in Aceh.” Journal of Peace Research 46 (3): 399–418. D’Aoust, O., O. Sterck, and P. Verwimp. 2012. “Buying Peace: The Mirage of Rebels’ Reinsertion in Burundi.” Processed. Davenport, C., and A. C. Stam. 2009. “What really happened in Rwanda?” http://rwandarwaba nyarwanda.over-blog.com/article-christian-davenport-and-allan-c-stam-what-really-happened-in- rwanda-45022893.html. De Luca, G., and M. Verpoorten. 2011. “From vice to virtue? Civil war and social capital in Uganda” Households in Conflict Network Working Paper 111. De Walque, D. 2004. “The long-term legacy of the Khmer Rogue period in Cambodia.” World Bank Research Paper 3446, Washington, DC. De Walque, D., and P . Verwimp. 2010. “The demographic and socio-economic distribution of excess mortality during the 1994 genocide in Rwanda.” Journal of African Economies 19 (2): 141 –62. Deaton, A. 2000. The analysis of household surveys: A microeconometric approach to development policy. Baltimore, Maryland: The World Bank, John Hopkins University Press. Deaton, A. 2001. “Counting the world’s poor: Problems and possible solutions.” World Bank Research Observer 16(2): 125 –47. Deininger, K. 2003. “Causes and consequences of civil strife - micro-level evidence from Uganda.” The World Bank, Policy Research Working Paper Series 3045, Washington, DC. ´n Deininger, K., A. M. Iba ˜ ez, and P. Querubin. 2004. “Towards sustainable return policies for the dis- placed population: Why are some displaced households more willing to return than others?” Households in Conflict Network Working Paper 07. Do, Q. T., and L. Iyer. 2009. “Mental health in the aftermath of conflict.” Harvard Business School Working Paper 10 –040. Douarin, E., J. Litchfield, and R. Sabates-Wheeler. 2010. “Poverty, Livelihoods and War Legacies: the Case of Post-War Rural Kosovo.” MICROCON Research Working Paper 37. ´n Engel, S., and A. M. Iba ˜ ez. 2007. “Displacement due to violence in Colombia: A household-level anal- ysis.” Economic Development and Cultural Change 55 (2): 335–65. Fearon, J. D., M. Humphreys, and J. M. Weinstein. 2009. “Can development aid contribute to social co- hesion after civil war? Evidence from a field experiment in post-conflict Liberia.” American Economic Review: Papers & Proceedings 99 (2): 287–91. Fearon, J., and D. Laitin. 2003. “Ethnicity, insurgency and civil war.” American Political Science Review 97 (1): 75 –90. Food and Agriculture Organization. 1996. “Study on the impact of armed conflicts on the nutritional situation of children.” http://www.fao.org/docrep/005/w2357e/W2357E05.htm. Green, E. P., and O. O. Tony. 2008. “Village leaders as data collectors: Willing, capable, and rational.” Households in Conflict Network Research Design Note 9. Guichaoua, Y. 2007. “Who joins ethnic militias? A survey of the Oodua people’s congress in south- western Nigeria.” Crise Working Paper 44 (March). Bru ¨ ck et al. 55 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Gupta, D. K. 1990. The economics of political violence: The effect of political instability on economic growth. New York: Praeger. Gupte, J., P. Justino, and J. P. Tranchant. 2012. “Households amidst urban riots: The economic conse- quences of civil violence in India.” Households in Conflict Network Working Papers 126. Hatlebakk, M. 2007. “LSMS data quality in Maoist influenced areas of Nepal.” CMI WP 6. Hegre, H., G. Østby, and C. Raleigh. 2009. “Poverty and Civil War Events: A Disaggregated Study of Liberia.” Journal of Conflict Resolution 53:598 –623. Horowitz, D. 1985. Ethnic groups in conflict. Berkeley: University of California Press. Humphreys, M. 2008. “Community driven reconstruction in the Democratic Republic of Congo: Baseline report.” Unpublished report. http://www.columbia.edu/~mh2245/DRC/report.pdf. Humphreys, M., and J. M. Weinstein. 2007. “Demobilization and reintegration.” Journal of Conflict Resolution 51 (4): 531– 67. . 2004. “What the fighters say. A survey of ex-combatants in Sierra Leone June-August 2003.” http://www.columbia.edu/~mh2245/Report1_BW .pdf. . 2008. “Who fights? The determinants of participation in civil war.” American Journal of Political Science 52 (2): 436– 55. ´n Iba ˜ ez, A. M., and A. Moya. 2009. “Do conflicts create poverty traps? Asset losses and recovery for dis- placed households in Colombia.” MICROCON Research Working Papers 10. International Committee of the Red Cross. 2009. “Summary report: Afghanistan, Colombia, democrat- ic republic of Congo, Georgia, Haiti, Lebanon, Liberia and the Philippines. Opinion survey and in- depth research.” Justino, P., I. Cardona, C. Muller, and R. Mitchell. 2012. “Quantifying the Impact of Women’s Participation in Post-Conflict Economic Recovery.” Households in Conflict Network Working Paper 131. Justino, P. 2012. War and poverty: Handbook of the economics of peace and security. Oxford, UK: Oxford University Press. Justino, P ¨ ck, and P ., T. Bru . Verwimp, eds. 2013. A Micro-Level Perspective on the Dynamics of Conflict, Violence and Development. Oxford, UK: Oxford University Press. Justino, P., M. Leone, and P. Salardi. 2014. “Short and Long-Term Impact of Violence on Education: The Case of Timor Leste.” World Bank Economic Review 28:320 –53. Justino, P., and O. N. Shemyakina. 2008. “Private and public transfers as a coping strategy under armed conflict: The case of Tajikistan.” Working Paper. Justino, P., and P. Verwimp. 2013. “Poverty Dynamics, Violent Conflict and Convergence in Rwanda.” Review of Income and Wealth 59 (1): 66 –90. Kalyvas, S. N. 2006. The logic of violence in civil war. Cambridge: Cambridge University Press. Kalyvas, S. N., and M. A. Kocher. 2007. “How free is “Free-riding” In civil wars? Violence, insurgency, and the collective action problem.” World Politics 59 (2): 177 –216. . 2009. “The dynamics of violence in Vietnam: An analysis of the hamlet evaluation system (HES).” Journal of Peace Research 46:335 –55. Kondylis, F. 2007. “Conflict-induced displacement and labour market outcomes: evidence from post-war Bosnia and Herzegovina.” CEPDP , 777. Centre for Economic Performance, London School of Economics and Political Science, London, UK. Menon, N., and Y. van der Meulen Rodgers. 2011. “War and Women’s Work: Evidence from the Conflict in Nepal.” Households in Conflict Network Working Paper 104. Mvukiyehe, E., and C. Samii. 2008/9. “Laying a foundation for peace? A quantitative impact evalua- tion of United Nations operations in Cote d’Ivoire (with technical appendix).” UN Office of Internal Oversight Services, Basis of UN-OIOS report to the UN General Assembly A/63/713. 56 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Østby, G. 2008. “Polarization, Horizontal Inequalities and Violent Civil Conflict.” Journal of Peace Research 45 (2): 143 –16. Pivovarova, M., and E. L. Swee. 2012. “How Much Can Panel Data Help?” Households in Conflict Network Working Paper 116. Restrepo, J. A., M. Spagat, and J. Vargas. 2004. “The dynamics of the Colombian civil conflict: A new dataset. Homo oeconomicus.” Institute of SocioEconomics 21:396 –429. Rohner, D., M. Thoenig, and F. Zilibotti. 2012. “Seeds of Distrust: Conflict in Uganda” Households in Conflict Network Working Paper 112. Sambanis, N. 2001. “Do ethnic and nonethnic civil wars have the same causes? A theoretical and em- pirical inquiry ( part 1).” Journal of Conflict Resolution 45 (3): 259–82. ¨ ck. 2011. “The Effects of Conflict on Fertility in Rwanda.” World Bank, Policy Schindler, K., and T. Bru Research Working Paper 5715, Washington, DC. Shemyakina, O. 2006. “The Effect of Armed Conflict on Accumulation of Schooling: Results from Tajikistan.” Households in Conflict Network Working Paper 12. . 2009. “The marriage market and Tajik armed conflict.” Households in Conflict Network Working Papers 66. Singer, J. D., and M. Small. 1994. Correlates of war project: International and civil war data, 1815– 1992. Ann Arbor, Michigan, Inter-university Consortium for Political and Social Research. Skaperdas, S. 1992. “Cooperation, Conflict, and Power in the Absence of Property Rights.” American Economic Review 82 (4): 720 –39. Swee, E. L. 2009. “On war and schooling attainment: The case of Bosnia and Herzegovina.” Households in Conflict Network Working Paper 57. Totten, S., and E. Markusen. 2006. Genocide in Darfur: Investigating the atrocities in the Sudan. Taylor & Francis Group, LLC. United Nations High Commissioner for Refugees. 2010. “ Asylum levels and trends in industrialized countries 2009: Statistical overview of asylum applications lodged in Europe and selected non- European countries. Division of programme and support management.” Uppsala Conflict Data Program, International Peace Research Institute, Oslo. 2007. “UCDP/PRIO armed conflict dataset codebook.” http://www.pcr.uu.se/publications/UCDP_pub/UCDP_PRIO_ Codebook_v4-2007.pdf. Valente, C. 2011. “What Did the Maoists Ever Do for Us? Education and Marriage of Women Exposed to Civil Conflict in Nepal.” Households in Conflict Network Working Paper 105. Verpoorten, M. 2011. “Measure for Measure: How Well Do We Measure Micro-level Conflict Intensity?” LICOS Discussion Paper Series, Discussion Paper 275/2011. Verwimp, P. 2012. “Undernutrition, subsequent risk of mortality and civil war in Burundi.” Economics and Human Biology 10 (3): 221 –31. Verwimp, P. 2003a. Development and Genocide in Rwanda, Doctoral Dissertation, Economics Department, KU Leuven. Verwimp, P . 2003b. “Testing the double genocide thesis for central and southern Rwanda.” Journal of Conflict Resolution 47 (4): 423–42. . 2005. “An economic profile of peasant perpetrators of genocide: Micro-level evidence from Rwanda.” Journal of Development Economics 77:297 –323. Verwimp, P., and J. Van Bavel. 2014. “Schooling, Violent Conflict and Gender in Burundi.” World Bank Economic Review 28 (2): 384 –411. Verwimp, P ., P. Justino, and T. Bru¨ ck. 2009. “The analysis of conflict: A micro-level perspective.” Journal of Peace Research 46 (3): 307– 14. Bru ¨ ck et al. 57 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Verwimp, P., and T. Bundervoet. 2009. “Civil War and the Welfare of Extended Households: Evidence from Longitudinal Data from Burundi.” Households in Conflict Network Working Paper 70. Voors, M., E. Nillesen, P. Verwimp, E. Bulte, R. Lensink, and D. van Soest. 2012. “Does conflict affect preferences? Evidence from field experiments in Burundi.” American Economic Review 102 (2): 941 –64. World Bank. 2011. World Development Report on Conflict, Security and Development. Weidmann, N. B. 2009. “Violence and the Changing Ethnic Map: The Endogeneity of Territory and Conflict in Bosnia” Households in Conflict Network Working Paper 64. Weinstein, J. M. 2007. Inside rebellion: The politics of insurgent violence. New York: Cambridge University Press. World Health Organization. 2004. “Guidelines for conducting community survey surveys on injuries and violence.” http://whqlibdoc.who.int/publications/2004/9241546484.pdf. 58 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/29/2461776/Measuring-Violent-Conflict-in-Micro-level-Surveys by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Promoting Women’s Economic Empowerment: What Works? ´ and Rebecca Furst-Nichols Mayra Buvinic A review of rigorous evaluations of interventions that seek to empower women economical- ly shows that the same class of interventions has significantly different outcomes depend- ing on the client. Capital alone, as a small cash loan or grant, is not sufficient to grow women-owned subsistence-level firms. However, it can work if it is delivered in-kind to more successful women microentrepreneurs, and it should boost the performance of women’s larger-sized SMEs. Very poor women need a more intensive package of services than do less poor women to break out of subsistence production and grow their businesses. What works for young women does not necessarily work for adult women. Skills training, job search assistance, internships, and wage subsidies increase the employment levels of adult women but do not raise wages. However, similar interventions increase young women’s employability and earnings if social restrictions are not binding. Women who run subsistence-level firms face additional social constraints when compared to similar men, thus explaining the differences in the outcomes of some loans, grants, and training inter- ventions that favor men. Social constraints may also play a role in explaining women’s outcome gains that are short-lasting or emerge with a delay. The good news is that many of the additional constraints that women face can be overcome by simple, inexpensive adjustments in program design that lessen family and social pressures. These include pro- viding capital in-kind or transacted through the privacy of a mobile phone and providing secure savings accounts to nudge women to keep the money in the business rather than to divert it to non-business uses. JEL Codes: J16, J24, L25, L26, M53, O12 The observed disparity between the sexes in productivity and earnings is persistent and pervasive. The value added per worker is between 6 percent and 35 percent lower in female-owned than in male-owned firms; female-managed farms are 20 percent to 80 percent less productive than male-managed farms; in the workplace, The World Bank Research Observer # The Author 2014. Published by Oxford University Press on behalf of the International Bank for Reconstruction and Development / THE WORLD BANK. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com doi:10.1093/wbro/lku013 Advance Access publication November 20, 2014 31:59–101 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 females earn between 20 percent and 80 percent lower average wages than do males, depending on the country (World Bank 2012). This disparity is not because women are inferior entrepreneurs, farmers or wage workers; it is primarily the result of differences in the size of their businesses and farms, in the sectors in which they operate, and in human capital (health and education) and returns to this capital (World Bank 2012, 2013). A wide range of policies and programs—from strengthening economic rights for women under the law to providing women with greater access to quality child care and financial literacy—can potentially spur women’s economic advancement and reduce gender gaps in economic performance. This article reviews the empirical evi- dence on the effectiveness of actions that have direct, near-term impacts on women’s economic outcomes while recognizing the importance of complementary investments in women’s human capital and inclusive policies and legal frameworks over the long term. Short-term interventions that succeed in increasing women’s earnings are not to be underestimated. They benefit women and can have a transformative impact on society by fostering greater investments in child well-being, reduced household poverty, and enhanced aspirations for the next generation of girls and women (World Bank 2012). The interventions covered in this review include providing women with access to capital (loans and grants) and with savings accounts, business management training, on-the-job and skills training, and job vouchers. These are all increas- ingly commonplace interventions that are intended to boost the productivity and earnings of self-employed women, women farmers and women wage workers in developing countries. Table 1 summarizes the principal evaluation studies grouped by intervention type. As shown in the table, this review draws on credi- ble evaluations of the effectiveness of different interventions—evidence that is rigorous enough to ensure that the observed outcome was the result of the inter- vention rather than of selection effects or placement biases. The article updates and benefits from thematic reviews of this evidence commissioned by the Women’s Roadmap Project.1 The main question guiding the review is “what works” to increase women’s pro- ductivity and earnings as proxies for the more complex notion of economic empow- erment. This question is followed by the question of “for whom,” hypothesizing that the evidence would show differential outcomes for poor or very poor versus non- poor women (defined by income levels, type of occupation, or the nature and size of firms or farms) and for young (between the ages of 15 and 24, or older in post-con- flict settings) versus adult women (aged 25 to 64). The gender literature’s emphasis on documenting gender differences has often underplayed the equally important documentation of heterogeneity among women. 60 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Methodological Issues Interventions were assessed according to the strength of empirical evidence. They were judged as proven when the evidence was robust, with more than one credible evaluation yielding similar results across different settings or circumstances. Promising interventions were backed by positive findings from one credible study or its equivalent. Unproven interventions yielded no evidence of positive effects on pro- ductivity and earnings for the categories of women and settings evaluated. Measuring changes in the productivity and earnings of individual women is straightforward conceptually but much less so practically, especially for women in low-income settings who farm or are self-employed and have highly variable earnings (Rosenzweig 2012; World Bank 2013). In response, many of the studies reviewed used a variety of intermediate outcomes for women working in self- employment and farming, including growth in women’s physical and financial assets, improvements in the stability of women’s earnings, evidence of business cre- ation or start-ups, and increases in women’s leisure time. A majority of the evaluations reviewed rely on randomized control trials (RCTs) and naturally occurring experiments that yield rigorous evidence of cause-effect relationships but that are not usually very good at providing evidence on the “why,” the channels or mechanisms that explain why the intervention had a particular effect. This review pays special attention to possible channels that mediate observed outcomes. It also examines the timing of outcomes, when they appear and how long-lasting they are, and the cost-effectiveness of interventions. What the Evidence Says Access to Capital: Loans and Grants Poor and very poor women. The experimental evidence (table 1) overall shows that a small infusion of capital alone, as a loan or a grant, is not enough to grow subsis- tence-level, female-owned firms. Poor women who run subsistence-level enterprises (with average monthly revenues of US $80 to $100 at market exchange rates, profits of approximately $1 per day, and no paid employees) fail to benefit in terms of business profits or growth from micro loans or cash grants (of approximately $100 to $200 on average). This result was observed in Sri Lanka, where cash grants were given (de Mel, McKenzie, and Woodruff 2008, 2009, 2012), and in India, Bosnia and Herzegovina, Mongolia, the Philippines, and Thailand, where the intervention was microloans (Duflo et al. 2013; Augsburg et al. 2012; Attanasio et al. 2011; Karlan and Zinman 2010; Coleman 2006). In contrast, capital alone increased the business profits of male-owned micro- enterprises in both the Philippines and Sri Lanka – an effect that, in the case of ´ and Furst-Nichols Buvinic 61 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Sri Lanka, was still evident five years after the cash was disbursed (Karlan and Appel 2011; de Mel, McKenzie, and Woodruff 2012). The finding that micro loans or grants have a positive effect on the performance of male-owned firms, but not female-owned firms, may be because subsistence-level female firms operate in sectors that face more severe constraints to growth and because women face more pressures than men to spend some of the cash intended for the business for other purposes, or a combination of both. Duvendack et al. (2011) argue that the lack of effect of microloans is because many women microloan clients are “necessity” entrepreneurs who turn to self-em- ployment because there are no other jobs available and who would be better off in wage employment. De Mel, McKenzie, and Woodruff (2012) add that women face social and time constraints that limit the types of businesses they operate and the impact of capital alone on business profits and growth. Alternatively, women working in subsistence-level enterprises may face pressures to use portions of their revenue for other expenses and transfers rather than for business investments. Pressures can have internal or external origins. According to Fafchamps et al. (2014), an internal, present-day orientation rather than external pressures to share the cash seems to guide women’s choices, whereas men show a future orientation or more self-control. Blattman, Fiala, and Martinez (2014) also find young women to be more present-biased than young men, although this orien- tation did not prevent young Ugandan women from increasing their earnings in re- sponse to a large cash grant. An experiment in 26 rural communities in Western Kenya provides ex ante evidence for the external pressure hypothesis. Women in- vested less in a presumed business venture when investment income was visible compared to when it was hidden, and this tendency was more pronounced when close kin was present. This “kin tax” was applicable only to women; there was no effect for men (Jakiela and Ozier 2012). Further supporting evidence for external pressures that influence women’s but not men’s diversion of capital from firm to family comes from recent randomized evalua- tions of combined training and loan provision in Uganda, savings in Chile, and ATM cards in Kenya. Family proximity in Uganda had significant negative effects on the business investment decisions of married women, whereas men benefited from this proximity (Fiala 2014). In Chile, the take-up of personal savings accounts was partic- ularly high among microentrepreneurs (91 percent of whom were women) who were not heads of household and faced significant pressures to share resources with family or community (Kast and Pomeranz 2014). In Kenya, women with low household bargaining power did not use unsecured ATM cards for individual bank accounts, which reduced transaction costs, because of fear they would not be able to resist their husbands’ pressure to obtain some of those savings (Schaner 2014). By helping protect the privacy of individual choices, mobile money may provide women with sufficient autonomy and independence from social pressures to invest 62 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 cash in the business rather than in the extended family, as Aker et al. (2011) found in a randomized evaluation of a cash transfer program delivered to poor women via mobile phones in Niger. Positive crop outcomes were likely the result of the low cost of using the mobile phone to transfer cash and the greater privacy the mobile phone gave women to elect how to spend the transfer. The evidence also suggests that capital delivered in-kind (e.g., purchase of livestock or inventory) rather than as cash works for women-owned microenterprises with firms that are above subsistence level in size—that is, those with higher initial profits than other female-owned firms. Large, lasting gains in profit for female-owned micro- enterprises in urban Ghana were achieved when an in-kind transfer of capital (i.e., purchase of inventory) nudged the recipient women to keep a future business orienta- tion and not divert the money to more immediate uses (Fafchamps et al. 2014). Non-poor women. Access to finance may be enough to increase the growth and earnings of women-owned small firms in the larger, formal small and medium-sized enterprise (SME) segment (with 5 to 19 employees per firm), although rigorous evi- dence on the impact of access to financial services for SMEs by the firm owner’s sex is still lacking ( partly because of the comparatively few women-owned SMEs com- pared with women-owned microenterprises). However, there is sufficient evidence that in developing economies, capital constraints are one of the main reasons for the suboptimal size of female-owned firms compared with male-owned ones (Klapper and Parker 2011; Sabarwal and Terrell 2008) and, more generally, are a binding constraint to the growth of all small firms in developing countries, both female- and male-owned firms (Ayyagari, Demirguc-Kunt, and Maksimovic 2012). Young women. A comparatively large infusion of capital in the form of an uncondi- tional grant without training or other support increased the earnings of poor young adults (aged 16 to 35) in conflict-affected northern Uganda. This effect, still visible four years later, was stronger for young women (who started from lower incomes) than for young men (Blattman, Fiala, and Martinez 2014). It may be that this powerful effect of a large cash windfall on young women’s earnings would extend to adult women running subsistence enterprises, meaning that access to capital alone would grow these adult women’s earnings provided that it was a large enough amount and women were given time to show results. Timing of effects. Panel household survey data for Bangladesh covering a twenty- year period support the notion that time influences the outcomes we observe. These data show a beneficial effect, greater for females than for males, of 20-year cumula- tive microcredit borrowing on household per capita income and the reduction of extreme poverty (Khandker and Samad 2014). The authors speculate that past credit may affect current credit, leading to more risk taking over time. This could be ´ and Furst-Nichols Buvinic 63 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 particularly relevant for women, who may be more risk averse than men. Similarly, a two-month grace period versus immediate repayment requirements for poor urban women borrowers in Kolkata, India significantly raised long-run (three-year) business profits by encouraging risk taking (Field et al. 2014). Capital plus Training Poor and very poor women. Bandiera et al. (2013a) show that a large capital transfer (most often a cow or other livestock with an asset value of approximately $140), in- tensive asset-specific training, and regular follow-up visits by an asset specialist and BRAC program officer during a two-year period had a significant, transformative impact on the occupational choices of very poor women in Bangladesh. These women changed occupational choices from casual day labor to self-employment and increased earnings significantly. This positive effect was still evident four years later (Bandiera et al. 2013b). Although all of the women included were very poor, the effect was largest for women who had the highest initial relative earnings. Two evaluations of very similar pilots for the ultra-poor in rural India, one in West Bengal and one in Andhra Pradesh, yield findings in line with those of Bangladesh, although the simultaneous implementation of a government program for agricultural wage workers confounded the results from the Andhra Pradesh pilot (Banerjee et al. 2011; Morduch, Ravi, and Bauchet 2012; Bandiera et al. 2013a). Bandiera et al. (2013a) reviewed the available evidence for cost-effective- ness and found, for the five studies that had data on costs (including their own), that returns from these capital- and skills-intensive ultra-poor interventions were cost effective when compared to yields from savings accounts. The exceptions to these positive findings are randomized evaluations of ILO busi- ness training programs plus capital in Sri Lanka and Uganda. In urban Sri Lanka, the increased earnings of female business owners who received training and a US$130 one-time grant dissipated after two years (de Mel, McKenzie, and Woodruff 2014). In semi-urban Uganda, the delivery of capital (approximately US$200) plus training increased profits for male microentrepreneurs but not for female microen- trepreneurs. For men, the training helped ensure that the short-term results of the loans observed at six months lasted over nine months. The lack of results for women was attributed to family pressures: married women were drained by family demands for cash, whereas married men employed family members, thus benefiting from close family ties. Women who were physically separated from their families benefited from the loan plus training combination (Fiala 2014). Savings Poor and very poor women. The evidence on savings is less rich than that on loans and grants but more consistently shows positive economic outcomes for women. 64 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 It also shows that women’s demand for saving accounts is high. A review of nine randomized field experiments in countries covering different regions (with studies in Kenya, Philippines, Nepal and Guatemala reporting data disaggregated by sex) shows that savings are a promising way to improve rural women’s productivity (Knowles 2013). In both Ghana and Rwanda, differences between men and women in the existence of savings accounts explained the largest proportion of the gender wage gap for entrepreneurs (Gamberoni, Heath, and Nix 2013). For Ghana, where panel data were available, savings accounts were especially important to increase the profits of the smallest, subsistence-level women entrepreneurs, shifting business performance from losses to profits. The recent experimental evidence sheds light on reasons why savings may work especially well for women. In Western Kenya, women with access to savings ac- counts invested 45 percent more in their businesses and were less prone to sell busi- ness assets to address health emergencies (Dupas and Robinson 2009). These effects were sustained in a follow-up survey conducted three years later, showing that savings interventions can have lasting effects (Dupas and Robinson 2013). In rural Mindanao, Philippines, the decision-making ability and control over assets of less empowered women with below-median reported household decision-making power increased as result of commitment savings, where deposits could only be with- drawn after a savings goal was reached; there was no similar effect for men (Ashraf, Karlan, and Yin 2006, 2010). These savings likely provided women with greater legal and psychological control over funds. Commitment savings were particularly attractive to women who reported a present bias. The authors hypothesize that commitment savings are preferred by Filipino women who are responsible for household finances and recognize the temptation to spend or the risk of spousal control over funds. In urban Chile, however, there was particularly high take up of free, liquid savings accounts by microentrepreneurs (91 percent of them female) who were more exposed to family pressures than others. Liquid savings reduced reliance on short-term debt, improved consumption smoothing in the face of economic shock (indicating an advantage of liquid over commitment savings) and still protected re- sources from being shared with others (Kast and Pomeranz 2014). Women’s demand for savings accounts is suggested by the data. In urban Ghana, more than half of women microentrepreneurs use a susu collector (a vehicle for informal mobile savings), even when they must pay for the service (Fafchamps et al. 2014). In Western Kenya, women make use of savings accounts through village co- operatives far more than men do (Dupas and Robinson 2009). In Guatemala, female microfinance borrowers are significantly more likely than their male counterparts to open a commercial savings account (Atkinson et al. 2010). In the Philippines, women are more likely to take up commitment savings, although the difference com- pared with men is not statistically significant (Ashraf, Karlan, and Yin 2006). ´ and Furst-Nichols Buvinic 65 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 In Nepal, a majority (84 percent) of females in 19 urban slums opened simple savings accounts when offered access (Prina 2013). Business Training Poor and very poor women. Both microfinance organizations and banks usually offer business management training courses of varying length and quality as well as varying depth and breadth of business topics covered, which complicates the task of measuring their impacts. A systematic analysis by Woodruff and McKenzie (2013) of 20 evaluations of business training interventions that included female trainees and a complementary review by Bandiera et al. (2013a) of field experiments that combine business training and capital transfers consider this training heterogeneity and conclude that business training improves business practices for women micro- enterprise owners but has few measurable effects on business survivorship or profits. Evaluations across a wide spectrum of countries, including Bosnia and Herzegovina (Augsburg et al. 2012), Pakistan (Gine and Mansuri 2014), Peru (Valdivia 2011), Sri Lanka (de Mel, McKenzie, and Woodruff 2012), and Tanzania (Oppedal Berge, Bjorvatn, and Tungodden 2011), support these conclusions. The lack of positive training outcomes is not restricted to women but is more consistently observed for women than for men and is attributed to women’s lower- quality businesses and more binding external constraints, including greater house- hold duties and limited say in household decisions (Gine and Mansuri 2014; Oppedal Berge, Bjorvatn, and Tungodden 2011). In addition, attrition rates in busi- ness training programs are systematically higher for female than for male trainees, suggesting that these programs may not be designed to respond to the specific con- straints women in business face (Valdivia 2013; Woodruff and McKenzie 2013). Business training seems to have more of an impact on revenues and profits for business start-ups. Training helped women who were out of the labor force start enterprises more quickly in Sri Lanka. Getting these women to start businesses was easier than getting these businesses to grow (de Mel, McKenzie, and Woodruff 2014). Additionally, business training increases microloan repayment rates (Woodruff and McKenzie 2013). Non-poor women. Woodruff and McKenzie (2013) suggest that quality business training on its own may work better for large versus small firms. In the case of large firms, a small percentage increase in profits may be enough to justify the costs of business training, especially if the impacts last, because the costs of training often increase less than proportionally with the size of firms undergoing training. Xu and Zia (2012) similarly suggest that business training has better results with women- owned SMEs when compared with women-owned microenterprises. They advise 66 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 targeting motivated, self-selected women entrepreneurs to increase the impact of training programs. Timing of effects. Two- to three-day training programs are very common, though short courses are likely to have less impact than longer ones. Supporting the notion that high-quality training of reasonable duration can make a difference, a six-week, fully subsidized, high-quality basic business education for poor female microentre- preneurs in rural Mexico had noticeable effects, largely by improving accounting practices (Calderon, Cunha, and de Giorgi 2013). Similarly, a three-month intensive training delivered by specialized professionals increased business sales of women microentrepreneurs in Peru. Although in the short term only training plus custom- ized technical assistance to the firm showed an effect on sales, women who received training without technical assistance were able to catch up and show increased sales approximately two years after the training was first offered. Women had adopted the recommended business practices and made business adjustments, albeit at a slower pace than those who also received technical assistance. The busi- ness training appeared to be cost effective, unlike the customized advice to the firm, which cost twice as much as the training but whose effects were no longer visible in the second year (Valdivia 2011, 2013). Skills Training, On-the-Job Training and Job Vouchers Poor women. Evaluations of large-scale government programs in Latin America (three in Argentina and one in Mexico) that provided employment vouchers (around $100 –$150 per month) that function as wage subsidies and/or skills training to the unemployed poor in times of economic hardship find high uptake by females and positive impacts on their employment, but less clear effects on their wages (Galasso, Ravallion, and Salvia 2004; Jalan and Ravallion 2003; Galasso and Ravallion 2004; Revenga, Riboud, and Tan 1994). Evaluations of retraining and wage subsidy interventions in four transition econ- omies (Poland, Romania, Russia, and Slovakia) yield mixed effects: positive effects for women’s employment probabilities in Poland and Romania, no effects in Russia and mixed effects in Slovakia (Kluve, Lehmann, and Schmidt 1998; Benus et al. 2005; Lubyova and Ours 1999). Again, there is less support for the effect of these programs on wages. Finally, a retraining program in China also has mixed findings: a negative impact on employment probability in one city and a positive impact in another city, with no differences between the sexes and no discernible effects on wages in either loca- tion (Bidani, Goh, and O’Leary 2002). The program was better implemented in the second city, which may explain the observed city difference in employment outcomes (Todd 2013). ´ and Furst-Nichols Buvinic 67 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Todd’s (2013) review of the evaluation evidence argues that these programs helped workers find jobs more quickly, but at the expense of lower wages than workers may have obtained from a longer job search on their own. This analysis and a review on the same topic by Betcherman, Olivas, and Dar (2004) conclude that these large government programs tend to increase employment but not wages, and they raise the issue of possible negative ‘spillover’ effects on the jobs and wages of people who do not participate as well as the possibility that publicly provided training could crowd out training provided by private sector firms. Young women. In contrast to the above findings, youth labor training programs in Latin America that provide classroom training, internships and stipends for child care have documented success in promoting young women’s employment. Young women have flocked to these programs and have generally outperformed young men in obtaining results, as evidence has shown for programs in Peru (Nopo, Robles, and Saavedra 2007), Colombia (Attanasio, Kugler, and Meghir 2011), and Argentina (Galasso, Ravallion, and Salvia 2004). The exception is a youth employ- ment program in the Dominican Republic, which had a stronger impact on young men’s employment in formal sector jobs compared with young women’s but did reduce teen pregnancies among female trainees and increase their social and per- sonal competencies (Ibarrara ´ n et al. 2012). These programs’ overall success is related to the soft skills they offer in addition to technical skills, the quality of on-the-job training, and the commitment of firms to hire trainees. ‘Soft skills’ are personality traits, motivations and preferences that are valued in the labor market, and recent evidence shows that they matter and should have an important place as part of job training programs (Heckman and Kautz 2012). The comparatively greater success of these programs among young women versus young men could be because of systematic gender differences in performance by female trainees, because of firms’ preference for hiring female over male workers, or both (Katz 2013). However, young women’s outperformance could result from the fact that in the control groups, young men have an easier time finding jobs, whereas young women stagnate. Timing of effects. Recent evidence from Liberia, Uganda, and India replicates the positive outcomes of well-designed skills training programs for young women that are sustained in follow up measures, even in contexts where wage work is scarce. In Liberia, a demand-driven skills training program increased young women’s employ- ment and earnings immediately and in a follow-up measure six months after the classroom training ended (Adoho et al. 2014). Gains were particularly large for young women using business skills learned in self-employment. In Uganda, life skills and vocational training plus adolescent girls clubs increased the earnings of adolescent girls and delayed marriage, reduced fertility and decreased risky health 68 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 behaviors. The authors attribute the successful outcomes evident two years after the program to the combined effect of health and economic empowerment interven- tions (Bandiera et al. 2014). In India, a subsidized six-month long vocational train- ing program in stitching and tailoring for young women residing in poor slums of New Delhi increased employment, self-employment and earnings substantially over controls. Positive effects were sustained 18 months after the training ended and were attributed to an increase in skills rather than an increase in self-confidence or the positive effect of having a training certificate (Maitra and Mani 2014). Analyses showed that the programs in Liberia, Uganda, and India were all cost effective. In Jordan, however, the combination of vouchers and training increased the em- ployment of young women graduates in typically female occupations in the short term, but this effect disappeared in a follow-up measure 14 months later when the vouchers had expired. The program was not able to overcome social constraints that prevent firms in Jordan from hiring young women. The soft skills training com- ponent, however, led to sustained improvement in life outlook and reduced depres- sion in young women (Groh et al. 2012). Skills training programs can suffer from high female attrition rates, similar to business training programs. High attrition may be more pronounced in programs that do not pair skills training with job internships, vouchers, or stipends. More than 40 percent of female trainees did not complete the six-month training program on stitching and tailoring in New Delhi; significant barriers included the distance to the training center and lack of available child care support (Maitra and Mani 2014). One-third of participants in a three-month long apprenticeship program for poor vulnerable youth (orphans or school dropouts) aged 15 to 24 in Malawi dropped out. Female participants dropped out much more often because of external constraints, whereas their male counterparts did so to take advantage of unrelated job opportunities. The training was more costly to women, who had less access to financing, used more of their personal savings, and were treated worse than men during the apprenticeship (Cho et al. 2013). The target population in most of these studies is women and young women among the unemployed poor or first job seekers. Unfortunately, the studies do not disaggregate findings within different poverty levels or groups. Discussion: What Does Work Heterogeneity Affects Outcomes It is clear from the review of the evidence that for whom (the client) matters when assessing what works to promote women’s economic empowerment. Few evaluated interventions work for all women across different socioeconomic and age groups. ´ and Furst-Nichols Buvinic 69 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 The more rigorous evidence, which better describes the characteristics of beneficia- ries, often shows that the same class of interventions has significantly different out- comes depending on the client or beneficiary. This is the case with small infusions of capital, such as loans or grants, which work for the non-poor but not for the very poor if delivered as a stand-alone intervention. Capital alone, as a small loan or a grant, is not enough to grow women-owned subsistence-level firms. However, it can work if it is delivered in-kind (e.g., inventory) to more successful women microentre- preneurs with somewhat larger businesses, and it should boost the performance of women’s larger sized firms in the SME sector regardless of the way it is delivered. Something similar occurs with business management training. Business training improves business practices but has few measurable effects on the growth of women-owned subsistence-level firms. Its impact may be improved by targeting women running larger-sized firms as well as by increasing the quality and duration of the training. To see effects, sufficient time should be allowed before outcomes are measured. Further, the evidence shows that very poor women need a more intensive package of services than do less poor women (and poor men) to break out of subsis- tence production – on and off the farm, in agriculture and entrepreneurship – and to reap the benefits of higher productivity and earnings for themselves and their families. These interventions are expensive but can be cost effective. To move into profitable work, very poor women subsistence producers and entrepreneurs require a sufficiently large capital transfer paired with business training and follow-up tech- nical visits. There is no silver bullet to grow the productivity of subsistence produc- ers. This situation is particularly challenging because the delivery of bundled services is more expensive and requires more implementation capacity, and the very poor often reside in regions with few resources, frail institutions, and weak delivery systems. What works for young women does not necessarily work for adult women. Skills training, job search assistance, internships, and wage subsidies introduced to cope with economic shocks increase the employment levels of adult women but do not raise wages. However, similar skills training combined with on-the-job training or wage subsidies increase young women’s employability and earnings if social restric- tions against hiring young women are not binding. Different levels of constraints, labor market experience and opportunities between young and adult women are likely responsible for the noticeably better outcomes of these job programs among young women. Young women most likely face fewer social constraints than do their adult counterparts. They may also feel less pressure to take any job and may be more selective in training choices. Nevertheless, the better-performing programs with young women also offer a combination of services, including stipends or vouchers, which increase their costs, but these can still be cost effective. 70 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Interventions proven to increase young women’s economic opportunities are es- pecially attractive given the large potential benefits for those just entering the labor market, including potentially delaying early childbearing. Women Face Additional Constraints It is also clear that pre-existing gender differentials and inequalities influence the outcome of interventions. Women running subsistence-level firms face additional social constraints to improving their productivity and earnings when compared to similar men, thus explaining differences in the outcomes of some (loan and grant) interventions favoring men. Growing evidence strongly suggests that women running subsistence-level firms face external pressures to divest some of the windfall cash from loans or grants to relatives or household expenses. Thus, both in-kind capital and capital transacted through the privacy of a mobile phone have better outcomes because they nudge women to keep the money invested in the business. Men do not appear to face these same pressures. This same mechanism (i.e., external pressures) seems to account for the finding that savings accounts are a good vehicle for increasing women’s business earnings. Saving might elicit a greater sense of personal ownership among women compared with cash from a loan or a grant, which would make them less likely to divert savings intended for a business investment to household expenditures or to rela- tives. Further, savings may be especially beneficial to more risk-averse women, who may not be as comfortable as less risk-averse women with the repayment and inter- est requirements of a loan. In addition, the evidence shows that savings may partic- ularly benefit both the smallest, subsistence-level women entrepreneurs and women with little household decision-making power, which makes savings an inter- vention that can apply to women across the board, regardless of their poverty condi- tion and the social restrictions they face. Women’s comparative disadvantage is also evident in the evaluations of training programs. The low uptake and high attrition rates in many of the business and skills training programs reviewed suggest that attendance costs (both time and money) are higher for women, who face more social constraints to participation, than for men. High attrition rates affect these programs’ effectiveness and cost effec- tiveness as well as the validity of program evaluations (by reducing sample size and, therefore, the power of statistical tests). All indications are that many of these programs are not designed with women clients’ constraints in mind. Clever Design Adjustments are Available The good news is that many of women’s additional constraints can be overcome by simple, inexpensive adjustments in program design. This review has highlighted ´ and Furst-Nichols Buvinic 71 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 simple, clever design features that help women overcome family and social pressures as well as the mobility constraints that impede women from growing their earnings in poorer and/or more socially conservative settings. A telling example is to provide capital in-kind (e.g., by purchasing inventory or a physical asset) rather than in cash to nudge women to keep the money in the business rather than to divert it for household use or pass it on to relatives. Many of these design features give women a measure of autonomy in an oth- erwise restricted environment where they are not free to make work-related choices or travel alone. Perhaps most notable among them is the use of mobile money services to conduct financial transactions in private, receive reminders to save and obtain information on prices and jobs in real time without having to travel long distances. Financial services delivered through mobile phones can be less costly and may be more effective than other delivery mechanisms in increas- ing women’s business investments. However, security must be built into finan- cial products that use new technologies to reduce transaction costs. Unsecured accounts may not be attractive to women who may fear capture by more power- ful husbands or relatives. Secure accounts and commitment savings may be pre- ferred by women with family pressures or self-control problems. However, because savings liquidity may be useful to cope with economic or other shocks, the design of savings should strive to balance both the security and liquidity of accounts, and, in some cases, women may find uses for both types of accounts. Minor design adjustments in business and skills training programs and in class schedules and locations could go a long way toward increasing women’s uptake, retention, and training effectiveness. More generally, minor adjustments to the design of financial services and business and skills training to better accommo- date the needs and demands of women clients could improve performance and outcomes considerably. Market research could usefully test savings and loan products and training offerings demanded by different categories of women clients (by occupation and income levels, for instance), contributing to the design of products that are more appropriately tailored to women entrepreneurs and farmers. Female Autonomy Helps Interventions seem to work best when women micro and small producers and en- trepreneurs have economic independence or autonomy and few social restrictions. Female autonomy means that women have physical mobility and can make inde- pendent choices, both of which are requirements to succeed in business. Autonomy allows women to seek and benefit from productive services and resources directly, increasing the likelihood that they will be able to switch from low- to high-value production. Autonomy is either measured directly or is indicated by the fact 72 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 that women are unmarried, divorced or widowed heads of household or sole income-earners and, therefore, do not have a male partner who can control their choices. Sole-income earners were a significant proportion of the ultra-poor women in Bangladesh who transformed their occupations as result of the capital and train- ing provided by BRAC. Female headship was a characteristic trait of women farmers in Malawi and urban entrepreneurs in Uganda who successfully crossed over from lower-value to high-value production, (Dimova and Gang 2013; Campos et al. 2013). An alternative explanation of the above evidence is that these women entrepre- neurs and farmers do not necessarily lack autonomy, but they access fewer produc- tive services because their provision has traditionally been biased toward male clients, especially when there is an adult male present in the household. Autonomy or its absence seems to be much less of a concern for adult and young women in wage employment, however. This may be partly because women have higher status and are more independent in countries where the wage sector is more prominent and employs more women. However, it may also be that wage employ- ment in itself provides women with a measure of autonomy from the family, at least within the confines of the workplace. Time Can Influence Observed Outcomes Project impacts are not all necessarily linear or monotonic, and evaluation timeta- bles may not allow sufficient time for results to occur or to understand if effects are long-lasting or disappear with time. Evaluations that include repeated measures sometimes show positive employ- ment and income effects for women that disappear with time, as in the case of the combination of training and a cash grant to Sri Lankan female business owners and vocational training and job vouchers to Jordanian young women. Other times, they show effects that emerge with a delay, as in the case of two decades of microen- terprise lending in Bangladesh and business training for microentrepreneurs in Peru. Gender norms seem to play a role in influencing outcomes over time. Social restrictions specific to women can override project gains. However, project gains can also help to change gender norms, causing positive outcomes to emerge with a delay as, for instance, when interventions drive changes in men’s initial resistance to women working outside the home or in women’s own avoidance of business risks. Rigorous evaluations increasingly include repeated measures to assess the lasting impact of programs. This practice is to be encouraged, especially with the type of short-term economic empowerment interventions for women that are the focus of this article. ´ and Furst-Nichols Buvinic 73 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Knowledge Gaps There is very little knowledge on the characteristics and determinants of the success of working women—successful entrepreneurs and farmers who have “crossed over” to high-value production, young women who have successfully transitioned from school to good jobs in the labor market, and wage workers who have moved up to higher-paid jobs in factories or firms or into non-traditionally female occupations. This gap in knowledge is not surprising because there are comparatively few women to study. However, a systematic study of the trajectories of success among working women across sectors should yield valuable information for program and policy design. The results of this research could be useful to help identify the profiles of those women entrepreneurs, farmers or wage workers with the firms or farms that are most primed for growth or most ready to succeed in non-traditional occupations. The ability of programs to recruit and select adult and young women who are most ready to succeed in the sector would significantly increase their effectiveness. This review of the evidence has been limited, by definition, to literature that has been evaluated and published. Evaluations that yield no significant findings are seldom published, so this review contains less information than it should on lessons from failed programs. More importantly, it does not cover promising innovations for which there are no good evaluation data. Potentially effective interventions that are in need of evaluation data include job information for young women, firm certification programs for wage workers, and non-traditionally female skills training for women wage workers and entrepreneurs. Further evaluation is also needed for business associations and networks for entre- preneurs as well as mentors and role models for young women. Both of the latter programs are well known and frequently implemented, but we found little rigorous information on their impact on women’s productivity and earnings in developing countries. There is only one rigorous study of SEW A bank clients in India, which suggests that the presence of peers positively influences women’s business behavior and changes aspirations, especially among those exposed to more restrictive social norms (Field et al. 2014). These results suggest that business networks and mentors are useful and potentially transformative, but more data are needed about their workings and their labor market impacts. Finally, as this review has shown, there is a growing body of rigorous evidence on what works to empower women economically, but there is very little evidence on the financial and economic costs and benefits and even less evidence on the sustain- ability of interventions, both in terms of the lasting impacts of programs and the in- stitutional and political dimensions of sustainability. The gaps in knowledge are large and cut across the reviewed interventions. 74 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic Table 1. Principal Evaluation Studies by Intervention Type ´ and Furst-Nichols Study Features Sample Intervention Findings Access to Capital: Loans and Grants Aker et al. Mobile cash transfer Recipient households Use of mobile phones to distribute -Mobiles saved time and transport (2011) in rural Niger in 96 villages unconditional cash transfers in costs. targeted villages -Purchased on average .86 more types of food and non-food items. -Had more non-durable assets. -Grew .36 more types of crops and more female-produced cash crops. Attanasio et al. Group and individual 1,148 adult women from Opening of microfinance -Business ownership increased, (2011) lending in rural 40 communities branches and comparing group but profits and income did not. Mongolia liability loans, individual loans -Higher educated increased or having no branch business activity in services, whereas lower educated did so in agriculture. -Loans did not benefit poorest among marginal clients. -Without much human capital, it is difficult to put loans to good use. Augsburg et al. Microfinance and 995 poor women who were Microfinance loans -Borrowers started and expanded (2012) education in Bosnia originally rejected by their small businesses. and Herzegovina microfinance institution -Women increased earnings. -Young women showed greater present bias than young men. Continued 75 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 76 TABLE 1. Continued Study Features Sample Intervention Findings Blattman, Skilled self-employment 265 groups of average Large, unconditional cash grant -After four years, business assets Fiala, and in rural Uganda 22 members each, aged 16 to 35 in groups (equivalent of $382/ increased by 57%, work hours Martinez member) by 17%, and earnings by 38% (2014) compared to controls. -Half practiced a skilled trade. -Many formalized enterprises and hired labor. -Young women’s average incomes were 73% higher than controls compared with young men’s, The World Bank Research Observer, vol. 31, no. 1 (February 2016) which were 29% higher, though women started poorer. -Women’s work and earnings stagnated without the program. Coleman Microfinance in rural 445 households, mostly Credit Some savings, which may -Positive returns to capital for (2006) northeast Thailand small-scale farmers be used to provide loans men’s businesses; mixed impact on women’s businesses. -Women invest very little of smaller grants in business but as much, if not more, than men of the larger grant. -Women do not experience permanent increases in business income from grants, whereas men do. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic ´ and Furst-Nichols de Mel, Returns to capital in 385 microentrepreneurs Grants of $100 and $200 in cash -No significant impact on earnings McKenzie, and urban Sri Lanka (50% women) or in-kind for women, whereas impacts Woodruff are large and significant for (2008) men. -Returns vary with entrepreneurial ability and household wealth but not with risk aversion or uncertainty. de Mel, Gender, credit 405 low-capital microentrepreneurs Grants of $100 and $200 in cash -No significant impact on earnings McKenzie, and constraints, and (50% women) or in-kind for enterprises owned by Woodruff microenterprise returns women. (2009) in peri-urban Sri Lanka -Treatment impacts significant for larger, male-owned enterprises. de Mel, Long-term effects of one- 408 microentrepreneurs Grants of $100 or $200 in cash -Ten percentage point higher McKenzie, and time capital transfers to (50% women) or in-kind; five year follow-up survival rates and $8 – $12 Woodruff microenterprises in measure higher monthly profits for (2012) urban Sri Lanka male-owned businesses. -No short- or long-term impact of capital on female-owned businesses. Continued 77 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 78 TABLE 1. Continued Study Features Sample Intervention Findings Duflo et al. Microfinance in urban 52 slums selected to open a Microloans only, majority group -Consumption no different and (2013) India branch of a microfinance loans average businesses no more institution that lends only profitable, although an increase to women in profits at high end. -Men showed more future orientation or self-control. -Women had present-day orientation and external pressures to share their cash. Fafchamps Microenterprise growth 800 microentrepreneurs Cash or in-kind grants to -For women in subsistence The World Bank Research Observer, vol. 31, no. 1 (February 2016) et al. (2014) in urban Ghana microenterprises enterprise, no gain in profits from either treatment. -For women with larger businesses, in-kind grants increase profits, whereas cash grants do not. -For men, cash also has lower impact, but differences between cash and in-kind grants are less robust than for women. Jakiela and Familial connections 2,145 participants in Game where subjects could vary -Women invest 22.1% less when Ozier (2012) (kin) and investment 26 communities amount to be invested in investment income is in rural Kenya presumed business; conditions observable than when it is varied degree of privacy of hidden. investment choice - Effect is pronounced when relatives are present (equivalent of 4% “kin tax”). - No similar tendency to hide income among men. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic ´ and Furst-Nichols Karlan and Credit access in urban Poor men and women wage Individual credit with median -Positive impact of access to credit Zinman South Africa workers who would have loan size of $127 on clients’ retention of jobs (2010) otherwise been rejected (loans likely helped clients for loans smooth or avoid shocks that prevent them from getting to work). -Positive impact of access to credit on household incomes. -Loans increased household food consumption. -No significant difference in impact of credit assigned to men and women. Khandker and Microcredit in rural 2,600 households in 87 20 year panel survey measuring -Increased household non-land Samad (2014) Bangladesh villages (14.7% attrition impact of cumulative assets (by 0.25%). rate over total years) microcredit borrowing -Increased income and expenditure. -Increased labor supply of men (by 0.33%) and women (by 0.46%). -Reduced extreme poverty, especially for female borrowers, by 5 percentage points. Continued 79 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 TABLE 1. Continued 80 Study Features Sample Intervention Findings Capital plus Training Bandiera et al. Basic entrepreneurship 1,400 communities of ultra-poor Large capital transfer (value -After two years, very poor women (2013b) in rural Bangladesh women across 40 regions about $140), intensive asset- changed occupational choices specific training, and regular from casual day labor to self- follow up technical visits employment and increased earnings by 34%. -Effect was largest for women who had highest relative earnings at the start. -After four years, women’s earnings were up 38% relative to baseline, whereas controls remained unchanged. The World Bank Research Observer, vol. 31, no. 1 (February 2016) Reduction in casual wage labor was twice that of the second- year data. -Hours spent in self-employment were lower after four years, likely due to increased efficiency in livestock rearing activities because earnings per hour were 15% higher than at baseline. Bandiera et al. Women’s empowerment 4,888 adolescent girls in 100 Vocational and life skills training -Significant improvements in (2014) in rural and urban treatment communities entrepreneurial skills. Uganda -72% increase in likelihood of engaging in self-employment. -Gains in employment did not come at expense of school enrollment. -26% lower fertility rates in treatment and 6.9 percentage points less likely to be married after two years. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic ´ and Furst-Nichols Banerjee et al. Targeting the hard-core 1,000 households Large asset transfer with intensive -After 18 months, average (2011) poor in rural India training on managing the asset monthly per capita earnings of households who received the asset increased by 21%. -15% increase in household consumption. -Positive impacts on household assets and emotional well-being. de Mel, Business training and 624 women subsistence level ILO business training plus cash - Training changes women’s McKenzie, and cash grants to business owners and 628 grant of $130 conditional on business practices but has no Woodruff microenterprises in women potential business training completion impact on business profits, (2014) urban Sri Lanka owners sales, or capital stock. -Grant plus training increases business profitability in the first eight months, but impact dissipates in the second year. -For potential startups, business training speeds entry (without changing longer-term ownership rates) and increases profitability. Continued 81 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 TABLE 1. Continued 82 Study Features Sample Intervention Findings Fiala (2014) Loans, grants and 1,550 microentrepreneurs Random selection to receive -Six and nine months after training in semi-urban loans, cash grants, business intervention, men with access Uganda skills training or a combination to loans with training had 54% greater profits. -Loans only had some impact, but this disappeared at nine months. -No impact of unconditional grants. -No effects for women from any of the interventions. -Married women with family living nearby performed worse The World Bank Research Observer, vol. 31, no. 1 (February 2016) than those in the control group, whereas men benefitted from close family proximity. Morduch, Ravi, Anti-poverty program 3,485 ultra-poor women Large asset transfer with intensive -Three years after the program, and Bauchet in rural India training on managing the asset households with assets spent (2012) significantly less time in subsistence agricultural wage labor relative to controls. -Households spent more time in self-employment, where earnings were greater. -Simultaneous implementation of a government program increased monthly incomes of agricultural wage workers, confounding iden- tification of the effect of the pilot. -Monthly earnings increased by 66% over three years for both treatment and controls. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic ´ and Furst-Nichols Savings Ashraf, Karlan, Female empowerment in 4,000 adult women microfinance Individual commitment savings -Significant increase in average and Yin. the rural Philippines clients savings (of 81% over a year) (2006, 2010) compared to controls -Present-biased women were 15.8 percentage points more likely to take commitment savings. -Less empowered women showed largest gain in decision making ability through savings Atkinson et al. Incentives to save among 1,375 borrowers from 20 microfinance Introduction of new commercial -Prompting for savings at the time (2010) microfinance borrowers branches of Guatemala’s largest savings products to existing of loan payments doubles in Guatemala public sector bank borrowers, with no financial savings, whereas suggesting a incentives or penalties savings deposit equal to 10% of the loan repayment causes savings to double again. -Women are significantly more likely to take up the offer of a savings account. -Women are more likely to take up commitment savings. -Women’s accumulated net savings are significantly lower than men’s overall. Continued 83 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 84 TABLE 1. Continued Study Features Sample Intervention Findings Dupas and Savings constraints in 185 microentrepreneurs Individual commitment savings -Positive impact of savings on Robinson rural Kenya with interest-free account and business investment among (2009, 2013) high withdrawal fees women (45% increase). -Increase in women’s private expenditures (37% to 40% higher). The World Bank Research Observer, vol. 31, no. 1 (February 2016) -Some impact on making women less vulnerable to health shocks; women were less prone to sell businesses to address health emergencies. -No effect for men. -Positive effects of the savings sustained in a follow-up survey three years later. -Women made use of savings accounts far more than men did. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic ´ and Furst-Nichols Kast and Access to formal savings 3,500 low-income microfinance Providing free and easy access to -Savings and credit are used as Pomeranz in urban Chile institution clients (90% women) a formal savings account substitute inputs for (2014) consumption smoothing. Reducing barriers to savings reduces reliance on debt. -When cost of savings was reduced, demand for short- term credit decreases and overall consumption smoothing increases. -Consumption cutbacks associated with a negative income shock reduced by 44%. -Substantial improvements in subjective well-being. -Take-up is high among participants who are not head of the household, who have conflicts with partners over money, and who are expected to loan to social network. Continued 85 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 86 TABLE 1. Continued Study Features Sample Intervention Findings Prina (2013) Savings accounts in 1,118 female household heads aged Access to a fully liquid bank -After one year, access to savings urban Nepal 18 – 55 account with no opening, accounts increased monetary maintenance, or withdrawal fees assets by more than 50%. -Total assets (monetary and non- monetary) increased by 16%. -Positive effect on monetary assets The World Bank Research Observer, vol. 31, no. 1 (February 2016) was strongest for poorer households and those not linked to formal banking institutions prior to the intervention. -Lower transaction costs due to proximity to the bank and lack of fees may have improved take-up and usage. -Saving in accounts rather than cash reduces temptation to spend immediately. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic Schaner (2014) Transaction costs, 1,114 newly opened bank accounts Provision of ATM cards free of -Provision of ATM cards bargaining power, and owned by 749 married couples charge, when without ATM significantly increased overall ´ and Furst-Nichols savings account use in cards, bank accounts had $.78 account use among men and rural Kenya withdrawal fee married couples’ joint accounts. Rates of long-run account use increased by 70% (4.7 percentage points). -ATM cards had no impact on female-owned individual accounts. -Both men and women with low levels of bargaining power responded negatively to ATM, whereas both men and women with high levels of bargaining power responded positively (controlled for time inconsistency and financial literacy). ATM cards significantly increased the share of individuals reporting that both spouses make joint spending and savings decisions. Business Training Calderon, Business literacy in rural 875 female entrepreneurs (724 in 6 weeks of business literacy -24% increase in profits and 20% Cunha, and de Mexico follow-up) of small firms in poor state classes with two four-hour increase in revenues, largely Giorgi (2013) in Mexico; majority of sample with meetings per week; total of through improved accounting below secondary education 48 hours practices. -50% of non-attriting businesses had closed by the time of second follow-up survey 28 months after training. 87 Continued Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 88 TABLE 1. Continued Study Features Sample Intervention Findings Gine and Constraints to 4,100 microfinance clients in groups Eight full-time days of business -Improved business knowledge of Mansuri entrepreneurship in training and a lottery to access both male and female (2014) rural Pakistan business loans of USD $1,700 entrepreneurs, but little effect The World Bank Research Observer, vol. 31, no. 1 (February 2016) (7 times average loan size) on business outcomes for women (some effects for men). -Only impact for women is improved business knowledge. -Reduced business failure rates, improved business practices, and increased household expenditures by $40/year, but effects concentrated with male clients. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic ´ and Furst-Nichols Field et al. Peer support and 636 customers age 18 – 50 of India’s Two days of business counseling -Women who attended training (2014) entrepreneurship in largest women’s bank (SEW A) and assistance in identifying a alone used loans for home urban India medium-term financial goal, repair, whereas those invited with a random subsample with a friend used loans for invited to attend training with business purposes. a friend -After four months, those who came with a friend reported differences in business behavior (higher volume of business and more stated plans to increase revenues), whereas those who came alone experience no change. -Those invited with a friend report significantly higher household income and expenditures and were less likely to be housewives. -Impacts of peer training on business loans and labor supply are concentrated among women in social castes/ religions with more restrictive social norms. Continued 89 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 90 TABLE 1. Continued Study Features Sample Intervention Findings Oppedal Berge, Human and financial 644 microfinance clients Business training and a lottery -Improved business knowledge of Bjorvatn, and capital for to access business loans both male and female Tungodden microenterprise entrepreneurs, but increased (2011) development in profits of male-owned firms Tanzania only. -Lack of positive outcomes for women was attributed to their more binding external constraints (i.e., their greater The World Bank Research Observer, vol. 31, no. 1 (February 2016) household duties and limited say in household decisions) and their presumed reluctance to compete. Valdivia (2011) Training or technical 1,983 female microentrepreneurs 36 3-hour business training -The combination of training and assistance in urban sessions over 12 weeks; tailored business advice Peru halfadditionally offered resulted in an average 19% individualized technical increase in business sales. assistance -The sales of women who were exposed only to business training did not grow significantly compared to controls. -Having an expert visit the firm to offer customized advice cost twice as much as the training alone. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic ´ and Furst-Nichols Valdivia (2013) Business training plus 1600 female microentrepreneurs Same as above – follow-up 2 year -Training plus tailored TA reported in urban Peru measures after training and TA increased sales 7 to 10 months after the intervention. -Those that received training only ‘catch up’ 12 to 15 months later and increase business sales of more than 15% approximately 2 years after end training, especially larger firms. -Take up rate slightly above 50%. Travel time and child care needs affected retention. -Extra cost of TA not cost effective Skills Training, On-the-Job Training and Job Vouchers Adoho et al. Adolescent girls’ 1,989 young women aged 16 – 27 with Livelihood and life skills training -Increased employment by 47% (2014) employment program basic literacy and numeracy skills, not and assistance with job and earnings by 80%. in urban Liberia enrolled in school, and residing in or placement -Positive effects on empowerment near Monrovia measures, including access to money, self-confidence, and anxiety about future. -No net impact on fertility or sexual behavior. -Cost-benefit analysis: Budget cost of EPAG business development training is equivalent to three years’ increased income. Continued 91 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 92 TABLE 1. Continued Study Features Sample Intervention Findings Attanasio, Vocational training in Unemployed young people placed 3 months in-classroom training -11% increase in likelihood to be Kugler, and urban Colombia in lowest two deciles of income and 3 months of on-the-job in paid employment. Meghir distribution training; trainees receive daily -Wage and salary earnings (2011) stipend, including extra for increase by 18%. child care -Controlling for training institution fixed effects and pre- treatment characteristics: -Trainees work an average of 1.1 more days/month and 2.5 more hours/week than The World Bank Research Observer, vol. 31, no. 1 (February 2016) women in the control group. Benus et al. Retraining programs in Registered unemployed adults with Training and unemployment -Statistically significant effects on (2005) Russia and Romania income less than 50% of minimum benefits, including a public the likelihood of employment, wage, employed 6 of last 12 months service component, where local the likelihood of being or recent graduate; 45% with government and other eligible employed at least once, and university degree organizations propose public wage levels. projects and hire ALMP -Middle-aged had largest impacts. participants to work on them -Retraining increases the probability of employment and decreases the wage for females. -Program not beneficial for highly educated. Bidani, Goh, Job training in urban Adults Retraining and job search -Positive impact on employment and O’Leary China assistance administered by local probability in Wuhan. (2002) labor bureaus to promote labor -No effect on earning in market entry of people laid off by Shenyang. state-owned enterprises -Negative effect on employment probability in Shenyang. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic ´ and Furst-Nichols Cho et al. Constraints on women 759 poor vulnerable youth (orphans or Three-month vocational and -Training outcomes were better for (2013) for vocational training school dropouts) aged 15 to 24 entrepreneurship program male trainees. in urban Malawi -Women faced greater obstacles in undertaking and taking advantage of the training. -One-third of participants dropped out. Whereas males dropped out to take advantage of unrelated job opportunities, females dropped out more due to external constraints. -Training more costly to women, who had less access to financing and used more personal savings and were treated worse than men during apprenticeship. Galasso and Social protection in Heads of households with dependents Participants worked 20 hours per -26% of participants would have Ravallion urban Argentina who became unemployed as a result week at a private company and been unemployed and 23% (2004) of Argentina’s economic crisis in in exchange received direct would have been inactive 2003 income support without program. -Study found substantial leakage to ineligibles, but the program was still well targeted at the poor. Continued 93 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 94 TABLE 1. Continued Study Features Sample Intervention Findings Galasso, From workfare to work Adult beneficiaries of temporary Skill training and/or vouchers for -Voucher reduced probability of Ravallion, and in urban Argentina employment programs workfare participants to give to unemployment (despite fact Salvia (2004) prospective employers that few firms made use of the (18 month wage subsidy) voucher). -Private sector employment was 15% for voucher recipients compared to 9% for controls. The World Bank Research Observer, vol. 31, no. 1 (February 2016) -Women and younger workers had largest impacts. Groh et al. Training and wage Young women college graduates Awarded voucher to pay employer -Job voucher led to a 40 (2012) subsidies for female subsidy equivalent to minimum percentage point increase in youth in Jordan wage for up to 6 months, if employment in short run, but graduate is hired; invited to most employment was not attend 45 hours of employability formal. skills training designed to -Average effect was smaller and no provide them with soft skills longer statistically significant 14 months after the voucher period ended. -Soft skills component led to sustained improvement in life outlook and reduced depression. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic ´ and Furst-Nichols Ibarraran et al. Life skills, employability Men and women 16 – 29 years old from Vocational classroom and on-the- -7%– 12% increase in (2012) and training for youth lower socioeconomic strata, and those job training; soft skills employment for women only. in the Dominican meeting educational and skill -3 –7 hours per week increase in Republic requirements of participating firms weekly hours worked for women only. -Especially successful in countries with established private vocational training industry, significant wage employment, and high female mobility. Jalan and Anti-poverty program in Workers affiliated with a successful Up to six month ‘socially useful -Average gain in household Ravallion urban Argentina project and not receiving projects’ submitted by local income for participants was (2003) unemployment benefits from governments and non- $103 dollars, with greater another program governmental organizations gains for younger people. Kluve, Active labor market Unemployed workers offered program Three forms of training: publicly -Training/program increases Lehmann, and policies in Poland at their local labor office financed training and retraining, average employment Schmidt wage subsidies for workers in probability for women and men (1998) private or public firms, and over both short and medium public works; 2 – 3 month term. courses with unemployment -Non-training ALMP did not have benefits a positive benefit. Lubyova and Active labor market Adult unemployed workers Two ALMP programs providing -Workers who enter ALMP have a Ours (1999) programs in the Slovak retraining and counseling 150% increase in the exit rate Republic services, and wage subsidies at into a regular job. either socially purposeful jobs -Benefits of retraining were only (up to 2 years) or publicly useful observed for socially purposeful jobs (up to 6 months at public jobs. works-type job) Continued 95 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 96 TABLE 1. Continued Study Features Sample Intervention Findings Maitra and Learning and earning in Young women aged 18– 39, average Six-month vocational training -Employment increased by six Mani (2014) urban India age 22 program in stitching and percentage points. training for young women -Self-employment increased by four percentage points. -Earnings increased by 150% monthly. -Positive effects sustained The World Bank Research Observer, vol. 31, no. 1 (February 2016) 18 months after completion of training and were attributed to an increase in skills rather than in self-confidence or having a training certificate. -More than 40% of trainees did not complete six-month course due to distance to training center and lack of available child care support. -Cost-benefit analysis suggests that program is highly cost effective. -Using a second round of follow- up data 18 months after the program, all effects are sustained in medium term. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Buvinic ´ and Furst-Nichols Nopo, Robles, Gender and racial Technicians and professionals from Classroom training and -Positive employment impacts for and Saavedra discrimination in hiring middle and lower classes in Lima; 80% internships lasting three months women of 6% at 12 months (2007) in urban Peru with above high-school education focused on training females for and 15% at 18 months. traditionally male occupations; -Negative impacts for men. stipend, doubled for child care -After 18 months, beneficiary females generate 93% more labor income than their control counterparts. -Decrease in measures of occupational segregation. Revenga, Retraining in urban Youth and adults. Offered to 250,000 Short-term vocational education -Program trainees found jobs Riboud, and Mexico registered unemployed people age training more quickly. Tan (1994) 20 – 55, selected on basis of eligibility -Impacts mainly for trainees older index. than 25 with work experience. -Cost effective for women over 25 but not for younger women. 97 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Note Mayra Buvinic ´ (corresponding author) is Senior Fellow at the United Nations Foundation; email address: mayra.buvinic@gmail.com. Rebecca Furst-Nichols is Program Officer, United Nations Foundation; email address: rfurst.nichols@gmail.com. This work was supported by a generous grant from the ExxonMobil Foundation to the United Nations Foundation. The authors are indebted to Noa Gimmelli, ExxonMobil Foundation, and Emily Courey Pryor, United Nations Foundation, for their steadfast support. 1. A Roadmap for Promoting Women’s Economic Empowerment, a collaboration between the United Nations Foundation and the ExxonMobil Foundation, can be accessed at www.women econroadmap.org. References Adoho, F., S. Chakravarty, Jr.D. T. Korkoyah, M. Lundberg, and A. Tasneem. 2014. “The Impact of and Adolescent Girls Employment Program: The EPAG Project in Liberia.” Policy Research Working Paper 6832 World Bank, Policy Research Department, Washington, DC. Aker, J., R. Boumnijel, A. McClelland, and N. Tierney. 2011. “Zap It to Me: The Short-Term Impacts of a Mobile Cash Transfer Program.” CGD Working Paper 268. Center for Global Development, Washington, DC. Attanasio, O., B. Augsburg, R. De Haas, E. Fitzsimons, and H. Harmgart. 2011. “Group Lending or Individual Lending? Evidence from a Randomized Field Experiment in Mongolia.” Working Paper 136. European Bank for Reconstruction and Development, London, UK. Attanasio, O., A. Kugler, and C. Meghir. 2011. “Training Disadvantaged Youth in Latin America: Evidence from a Randomized Trial.” American Economic Journal: Applied Economics 3 (3) : 188 –220. Ashraf, N., D. Karlan, and W. Yin. 2006. “Female Empowerment in the Philippines.” Center Discussion Paper 949. Economic Growth Center, Yale University, New Haven, CT. . 2010. “Female Empowerment in the Philippines.” World Development 38 (3) : 333 –44. Atkinson, J., A. de Janvry, C. McIntosh, and E. Sadoulet. 2010. “Creating Incentives to Save Among Microfinance Borrowers: A Behavioral Experiment from Guatemala.” Processed. University of California, San Diego, California. Augsburg, B., R. de Haas, H. Harmgart, and C. Meghir. 2012. “Microfinance, Poverty and Education.” Working Paper 18538. National Bureau of Economic Research, Cambridge, MA. Ayyagari, M., A. Demirguc-Kunt, and V . Maksimovic. 2012. “Financing of Firms in Developing Countries: Lessons from Research.” Policy Research Working Paper 6036. World Bank, Policy Research Department, Washington, DC. Bandiera, O., N. Buehren, R. Burgess, M. Goldstein, S. Gulesci, I. Rasul, and M. Sulaiman. 2014. “Women’s Empowerment in Action: Evidence from a Randomized Control Trial in Africa.” International Growth Center Working Paper. London, England. Bandiera, O., R. Burgess, S. Gulesci, I. Rasul, and M. Sulaiman. 2013a. “Capital, Skills and the Economic Lives of the Poor: Recent Evidence from Field Experiments.” A Roadmap for Promoting Women’s Economic Empowerment. Available at: www.womeneconroadmap.org. Accessed November 2013. Bandiera, O., R. Burgess, N. Das, S. Gulesci, I. Rasul, and M. Sulaiman. 2013b. “Can Basic Entrepreneurship Transform the Economic Lives of the Poor?” IZA Discussion Papers 7386. Institute for the Study of Labor, Bonn, Germany. 98 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Banerjee, A., and E. Duflo. 2011. Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty. New York: PublicAffairs. Banerjee, A., E. Duflo, R. Chattopadhyay, and J. Shapiro. 2011. “Targeting the Hard-Core Poor: An Impact Assessment.” Innovations for Poverty Action, New Haven, CT. Benus, J., R. Catrinel Brinza, C. Cuica, I. Denisova, and M. Kartseva. 2005. “Retraining Programs in Russia and Romania: Impact Evaluation Study.” CEFIR Research Papers, Moscow, Russia. Betcherman, G., K. Olivas, and A. Dar. 2004. “Impacts of Active Labor Market Programs: New Evidence from Evaluations with Particular Attention to Developing and Transition Countries.” World Bank Social Protection Discussion Papers 402. World Bank, Washington, DC. Bidani, B., C. Goh, and C. J. O’Leary. 2002. “Has Training Helped Employ Xiagang in China? A Tale from Two Cities.” Journal of Chinese Economic and Business Studies 7 (1) : 77 –94. Blattman, C., N. Fiala, and S. Martinez. 2014. “Generating Skilled Self-Employment in Developing Countries: Experimental Evidence from Uganda.” Quarterly Journal of Economics 129 (2): 697 –752. Calderon, G., J. Cunha, and G. de Giorgi. 2013. “Business Literacy and Development: Evidence from a Randomized Controlled Trial in Rural Mexico.” Working Paper 19740. National Bureau of Economic Research, Cambridge, MA. Campos, F., M. Goldstein, L. McGorman, A. M. Munoz Boudet, and O. Pimhidzai. 2013. “Breaking the Metal Ceiling: Female Entrepreneurs Who Succeed in Male-Dominated Sectors in Uganda.” A Roadmap for Promoting Women’s Economic Empowerment. Available at: www.womeneconroad map.org. Accessed November 2013. Cho, Y., D. Kalomba, A. Mushfiq Mobarak, and V. Orozco. 2013. “Gender Differences in the Effects of Vocational Training: Constraints on Women and Drop-Out Behavior.” Policy Research Working Paper 6545. World Bank, Policy Research Department, Washington, DC. Coleman, B. 2006. “Microfinance in Northeast Thailand: Who Benefits and How Much?” World Development 34 (9) : 1612–38. de Mel, S., D. McKenzie, and C. Woodruff. 2008. “Returns to Capital in Microenterprises: Evidence from a Field Experiment.” Quarterly Journal of Economics 123 (4) : 1329–72. . 2009. “ Are Women More Credit Constrained? Experimental Evidence on Gender and Microenterprise Returns.” American Economic Journal: Applied Economics 1 (3) : 1–32. . 2012. “One-Time Transfers of Cash or Capital have Long-Lasting Effects on Microenterprises in Sri Lanka.” Science 335 (6071) : 962 –6. . 2014. “Business Training and Female Enterprise Start-Up, Growth, and Dynamics: Experimental Evidence from Sri Lanka.” Journal of Development Economics 106 : 199– 210. Dimova, R., and I. Gang. 2013. “Good and Efficient? Women’s Voice in Agriculture.” A Roadmap for Promoting Women’s Economic Empowerment. Available at: www.womeneconroadmap.org. Accessed November 2013. Duflo, E., A. Banerjee, R. Glennerster, and C. Kinnan. 2013. “The Miracle of Microfinance? Evidence from a Randomized Evaluation.” Working Paper 18950. National Bureau of Economic Research, Cambridge, MA. Duvendack, M., R. Palmer-Jones, J. Copestake, L. Hooper, Y. Loke, and N. Rao. 2011. “What is the Evidence of the Impact of Microfinance on the Well-Being of Poor People?” EPPI-Centre, Social Science Research Unit, Institute of Education, University of London. Dupas, P., and J. Robinson. 2009. “Savings Constraints and Microenterprise Development: Evidence from a Field Experiment in Kenya.” NBER Working Papers 14693. National Bureau of Economic Research, Cambridge, MA. . 2013. “Savings Constraints and Microenterprise Development: Evidence from a Field Experiment in Kenya.” American Economic Journal: Applied Economics 5 (1) : 163– 92. ´ and Furst-Nichols Buvinic 99 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Fafchamps, M., D. McKenzie, S. Quinn, and C. Woodruff. 2014. “Microenterprise Growth and the Flypaper Effect: Evidence from a Randomized Experiment in Ghana.” Journal of Development Economics 106 : 211 –26. Fiala, N. 2014. “Stimulating Microenterprise Growth: Results from a Loans, Grants and Training Experiment in Uganda.” Working Paper. German Institute for Economic Research, Berlin, Germany. Field, E., S. Jayachandran, R. Pande, and N. Rigol. 2014. “Friends at Work: Can Peer Support Stimulate Female Entrepreneurship?” Working Paper. Harvard Kennedy School, Cambridge, MA. Galasso, E., and M. Ravallion. 2004. “Social Protection in a Crisis: Argentina’s Plan Jefes y Jefas.” The World Bank Economic Review 18 (3) : 367 –99. Galasso, E., M. Ravallion, and A. Salvia. 2004. “Assisting the Transition from Workfare to Work: A Randomized Experiment.” Industrial and Labor Relations Review 58 (1) : 128 –42. Gamberoni, E., R. Heath, and E. Nix. 2013. “Subsistence, Transformational and Something in Between.” A Roadmap for Promoting Women’s Economic Empowerment. Available at: www. womeneconroadmap.org. Accessed November 2013. Gine, X., and G. Mansuri. 2014. “Money or Ideas? A Field Experiment on Constraints to Entrepreneurship in Rural Pakistan.” Policy Research Working Paper 6959. World Bank, Policy Research Department, Washington, DC. Groh, M., N. Krishnan, D. McKenzie, and T. Vishwanath. 2012. “Soft Skills or Hard Cash? The Impact of Training and Wage Subsidy Programs on Female Youth Employment in Jordan.” Policy Research Working Paper 6141. World Bank, Policy Research Department, Washington, DC. Heckman, J., and T. Kautz. 2012. “Hard Evidence on Soft Skills.” Labour Economics 19 (4) : 451 –64. ´ n, P Ibarrara ´a. 2012. “Life Skills, Employability and ., L. Ripani, B. Taboada, J. M. Villa, and B. Garcı Training for Disadvantaged Youth: Evidence from a Randomized Evaluation Design.” IZA Discussion Papers 6617. Institute for the Study of Labor, Bonn, Germany. Jakiela, P., and O. Ozier. 2012. “Does Africa Need a Rotten Kin Theorem? Experimental Evidence from Village Economies.” Policy Research Working Paper 6085. World Bank, Policy Research Department, Washington, DC. Jalan, J., and M. Ravallion. 2003. “Estimating the Benefit Incidence of an Anti-Poverty Program using Propensity Score Matching.” Journal of Business and Economic Statistics 21 (1) : 19 –30. Karlan, D., and J. Appel. 2011. More than Good Intentions: How a New Economics is Helping to Solve Global Poverty. New York: Dutton. Karlan, D., and J. Zinman. 2010. “Expanding Credit Access: Using Randomized Supply Decisions to Estimate the Impacts.” Review of Financial Studies 23 (1) : 433–64. Kast, F., and D. Pomeranz. 2014. “Saving More to Borrow Less: Experimental Evidence from Access to Formal Savings Accounts in Chile.” NBER Working Paper No. 20239. National Bureau of Economic Research, Cambridge, MA. Katz, E. 2013. “Identifying Research Gaps and Priorities for Women’s Economic Empowerment: Gender and Youth Employment.” A Roadmap for Promoting Women’s Economic Empowerment. Available at: www.womeneconroadmap.org. Accessed November 2013. Klapper, L., and S. Parker. 2011. “Gender and the Business Environment for New Firm Creation.” World Bank Research Observer 26 (2) : 237 –57. Kluve, J., H. Lehmann, and C. Schmidt. 1998. “Active Labor Market Policies in Poland: Human Capital Enhancement, Stigmatization or Benefit Churning.” Journal of Comparative Economics 27 (1) : 61 –89. Knowles, J. 2013. “Increasing the Productivity and Earnings of Rural Women: What Works?” A Roadmap for Promoting Women’s Economic Empowerment. Available at: www.womeneconroadmap. org. Accessed November 2013. 100 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Khandker, S., and H. Samad. 2014. “Dynamic Effects of Microcredit in Bangladesh.” Policy Research Working Paper 6821. World Bank, Policy Research Department, Washington, DC. Lubyova, M., and J. C. Van Ours. 1999. “Effects of Active Labor Market Programs on the Transition Rate from Unemployment into Regular Jobs in the Slovak Republic.” Journal of Comparative Economics 27 (1) : 90 –112. Maitra, P., and S. Mani. 2014. “Learning and Earning: Evidence from a Randomized Evaluation in India.” GCC Working Paper Series, GCC 14 – 05. Grand Challenges Canada, Philadelphia, PA. Morduch, J., S. Ravi, and J. Bauchet. 2012. “Failure vs. Displacement: Why an Innovative Anti-Poverty Program Showed No Net Impact.” Discussion Paper Series 32. PRIMCED, Tokyo, Japan. Nopo, H., M. Robles, and J. Saavedra. 2007. “Occupational Training to Reduce Gender Segregation: The Impacts of ProJoven.” Working Paper 623. Inter-American Development Bank, Washington, DC. Oppedal Berge, L., K. Bjorvatn, and B. Tungodden. 2011. “Human and Financial Capital for Microenterprise Development: Evidence from a Field and Lab Experiment.” CMI Working Papers 1. Chr. Michelsen Institute, Bergen, Norway. Prina, S. 2013. “Banking the Poor via Savings Accounts: Evidence from a Field Experiment.” Innovations for Poverty Action, New Haven, CT. Revenga, A., M. Riboud, and H. Tan. 1994. “The Impact of Mexico’s Retraining Program on Employment and Wages.” The World Bank Economic Review 8 (2) : 247–77. Rosenzweig, M. 2012. “Thinking Small: Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty: Review Essay.” Journal of Economic Literature 50 (1) : 115–27. Sabarwal, S., and K. Terrell. 2008. “Does Gender Matter for Firm Performance? Evidence from Eastern Europe and Central Asia.” Policy Research Working Paper 4705. World Bank, Policy Research Department, Washington, DC. Schaner, S. 2014. “The Cost of Convenience? Transaction Costs, Bargaining Power, and Savings Account Use in Kenya.” Innovations for Poverty Action, New Haven, CT. Todd, P. 2013. “How to Improve Women’s Employability and Quality of Work in Developing and Transition Economies.” A Roadmap for Promoting Women’s Economic Empowerment. Available at: www.womeneconroadmap.org. Accessed November 2013. Valdivia, M. 2011. “Training or Technical Assistance? A Field Experiment to Learn What Works to Increase Managerial Capital for Female Microentrepreneurs.” Processed. World Bank, Washington, DC. . 2013. “Business Training Plus for Female Entrepreneurship? Evidence from a Field Experiment in Peru.” GRADE Working Paper, Lima, Peru. Woodruff, C., and D. McKenzie. 2013. “Improving the Productivity and Earnings of Women-Owned and/or Managed Enterprises in Developing Countries: What Works?” A Roadmap for Promoting Women’s Economic Empowerment. Available at: www.womeneconroadmap.org. Accessed November 2013. World Bank. 2012. World Development Report 2012: Gender Equality and Development. World Bank, Washington, DC. . 2013. World Development Report 2013: Jobs. World Bank, Washington, DC. Xu, L., and B. Zia. 2012. “Financial Literacy around the World: An Overview of the Evidence with Practical Suggestions for the Way Forward.” Policy Research Working Paper 6107. World Bank, Policy Research Department, Washington, DC. ´ and Furst-Nichols Buvinic 101 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/59/2461696/Promoting-Women-s-Economic-Empowerment-What-Works by Sectoral Library Rm MC-C3-220 user on 06 October 2017 Employer Voices, Employer Demands, and Implications for Public Skills Development Policy Connecting the Labor and Education Sectors ˜ or Wendy V. Cunningham and Paula Villasen Educators believe that they are adequately preparing youth for the labor market while at the same time employers lament the students’ lack of skills. A possible source of the mismatch in perceptions is that employers and educators have different understandings of the types of skills valued in the labor market. Using economics and psychology literature to define four skills sets—socio-emotional, higher-order cognitive, basic cognitive, and technical—this paper reviews the literature that quantitatively measures employer skill demand, as reported in a preference survey. A sample of 27 studies reveals remarkable consistency across the world in the skills demanded by employers. While employers value all skill sets, there is a greater demand for socio-emotional skills and higher-order cognitive skills than for basic cognitive or technical skills. These results are robust across region, industry, occupation, and education level. Employers perceive that the greatest skills gaps are in socio-emotional and higher-order cognitive skills. These findings suggest the need to re-conceptualize the public sector’s role in preparing children for a future labor market. Namely, technical training is not equivalent to job training; instead, a broad range of skills, many of which are best taught long before labor market entry, should be included in school curricula from the earliest ages. The skills most demanded by employers—higher-order cognitive skills and socio-emotional skills—are largely learned or refined in adolescence, arguing for a general education well into secondary school until these skills are formed. Finally, the public sector can provide program- ming and incentives to non-school actors, namely parents and employers, to encourage them to invest in the skills development process. JEL codes: J23, J24 The World Bank Research Observer # The Author 2016. Published by Oxford University Press on behalf of the International Bank for Reconstruction and Development / THE WORLD BANK. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com doi:10.1093/wbro/lkv019 31:102–134 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Recent years have seen a rethinking of the nature of, and measurement of, skills related to labor market success, which increasingly draws on concepts developed in the psychology literature. Traditionally, the economics literature has focused on “years of schooling” as a proxy for these skills, but Hanushek and Woessman (2008) argue that this approach suffers from serious measurement error and should be discarded in favor of exploring the role that particular skills play in driving labor market outcomes.1 James Heckman and others parse the concept of “skills” into “cognitive” skills—such as knowledge, comprehension, and critical thinking—and “non-cognitive” skills, roughly defined as personality traits and socio-emotional behaviors.2 This distinction is increasingly relevant in the context of modern labor markets where jobs increasingly involve non-routine analytical tasks that require problem-solving, emotional intelligence, logic, and teamwork, rather than traditional routine manual tasks (Acemeglu and Autor 2011; Autor 2013; Aedo et al. 2013; World Bank 2015).3 At the same time, emerging evidence suggests that schools are not teaching the skills that labor markets need. A 37-country study reports that about one-third of the 35,000 employers interviewed have trouble finding workers with the right skills (Manpower 2010). Mourshed et al. (2012) interviewed nearly 3,000 employers and 1,000 education providers in nine countries, and reports that 42% of employ- ers believe that graduates are prepared for the labor market as compared to 72% of educators. While the recurring shortfall in cognitive skills as measured, for instance, by PISA scores, has been convincingly documented, an emerging body of literature suggests that we must cast the skills net wider. Shivpuri and Kim (2004), for instance, find that U.S. employers report that they most value interpersonal skills, followed by trustworthiness, leadership, and perseverance, while educators identify “knowledge” as the most important skill. In light of these emerging views, this paper systematically reviews the evidence on the skills demand profile of employers in developed and developing countries, with the objective of informing education and training policy.4 Specifically, the paper reviews 27 studies that explicitly ask employers to provide their ranking of the most important skills for their workforce, and to provide a ranking of the skills that are most lacking. The paper then utilizes a developmental psychology frame- work to recommend shifts to public policies and programs to bring the supply and demand for skills closer together. Skills for the Labor Market: Concepts and Definitions Following the literature on economics, as well as on behavioral, personality, and industrial and organizational (IO) psychology, we classify labor-market related skills into two groups: cognitive and socio-emotional.5 The American Psychological Cunningham and Villasen ˜ or 103 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Association (APA) defines cognitive skills as the “ability to understand complex ideas, to adapt effectively to the environment, to learn from experience, to engage in various forms of reasoning, to overcome obstacles [through deliberate thinking],” (Neisser et al. 1996). This definition includes intelligence, reasoning, information- processing, perception, memory, literacy, numeracy, and learning, among others. Within cognitive skills, we define three sub-sets for the purposes of this paper: basic cognitive, higher-order cognitive, and technical. Basic cognitive skills encompass fundamental academic knowledge and compre- hension, including literacy and math, and are strongly correlated with labor market success.6 Studies that use longitudinal data from the US to regress labor force vari- ables of young adults on cognitive test scores collected while the sample was in high school, find that a one standard deviation increase in a mathematics test score in 12th grade is correlated with 10– 15% higher annual earnings by the mid-twenties to early thirties (Mulligan 1999; Murnane et al. 2000; Lazear 2003). Similar results are found for the UK (McIntosh and Vignoles 2001) and Canada (Finnie and Meng 2001). Using literacy scores and labor force behaviors from the International Adult Literacy Survey (IALS), Hanushek and Zhang (2009) finds that a one stan- dard deviation increase in literacy scores increases earnings by 9.3% in a 13 country sample.7 A small set of papers find an impact of cognitive skills on wages in developing countries as diverse as Ghana (Glewwe 1996), Kenya (Knight and Sabot 1990), Pakistan (Alderman, Behrman, Ross and Sabot 1996), and South Africa (Moll 1998). These skills are the core of school curricula and are applicable across a wide range of professions. Higher-order cognitive skills encompass the capacity to deal with complex infor- mation processing and are aligned with the emerging non-routine job tasks noted above. These tasks include such abilities as critical thinking, application of knowl- edge, analysis, problem solving, evaluation, oral and written communication, and adaptive learning (Bloom et al. 1956). An emerging body of literature shows that these skills drive labor market outcomes across a wide-range of occupations (Herrnstein and Murray 1994; Hanushek and Woessmann 2008). For example, Gottfredson (1997) argues that reasoning, problem solving, and decision-making are among the most powerful predictors of overall job performance. These skills are often not included in traditional school curricula (Pina et al. 2012). Technical skills can be defined as the specific knowledge required to carry out an occupation, and are often equated with “job skills” in policy circles. Such skills might include the ability to repair a car’s muffler, being able to identify specific bac- teria under a microscope, or the know-how to sew dozens of shirts per hour. While the definition of technical skills overlaps with that of basic- and higher-order cogni- tive skills, they merit separate treatment in a policy-oriented paper in order to test the assumption that they alone are synonymous with job-skills, and to provide insights for refining job training policy.8 A review by Betcherman et al. (2007) of 104 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 job training programs across the world finds, at best, positive returns to technical training for women and, in some cases, youth. More commonly, technical training programs yield zero, or negative rates of return, the latter indicating that more valu- able skills would have been acquired if that person had spent her time working rather than in a training course.9 These skills may be included in the school curriculum, or may be taught through vocational and technical training programs or institutes. Socio-emotional skills, the second broad group of labor market-related skills, are behaviors, attitudes, and traits that are necessary complements to cognitive skills in the production process. These skills include such diverse concepts as grit to finish a job, working in teams, commitment, creativity, and honesty.10 A review of meta-evaluations, reported in Almlund et al. (2011), confirms a significant correlation between job perfor- mance and openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism.11 Individual studies find conditional correlations between socio-emotional skills and wages (Heckman and Rubenstein 2001; Kern et al. 2013; Mueller and Plug 2006; Heckman, Stixrud, and Urzu ´a 2006), professional success (Duckworth et al. 2007), absenteeism in the work place (Stormer and Fahr 2010), and a range of other labor force outcomes.12 In fact, these skills may be more important than cognitive skills for employment outcomes, especially among low-skilled occupations (Bowles, Gintis, and Osborne 2001; Mueller and Plug 2006; Carneiro, Crawford, and Goodman 2007; Kniesner and ter Weel 2008). These skills are not commonly a part of the standard school curriculum (Pina et al. 2012). While this evidence may convincingly signal that a range of cognitive and socio- emotional skills are important for labor market success, it is not likely to capture the employer perspective for a few reasons. First, supply-side surveys measure only a small set of pre-defined skills, thereby potentially missing the skills that employers most value. Second, the supply-side data measure the skills that the current labor force has, thereby not capturing skills that may not be available, but may be important for the production process. This points to a need to complement the existing literature with demand-side data measuring the skills that employers want and the extent to which these skills demands are met by the current education and training systems. Data This paper draws from the published literature across a range of fields. Each data point represents the statistical finding of a paper that summarizes raw data from em- ployer surveys, most of which were specialized surveys collected for the purpose of understanding employer skill needs. Using a large set of key words related to employer demand for skills, we search five databases to build our sample: Google Scholar,13 IDEAS-RePEc,14 EconLit,15 ERIC (Education Resources Information Center16), and PsyContent.17 In addition, Cunningham and Villasen ˜ or 105 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 we consult the publications of institutions dedicated to research in the education field such as the National Center on Education and the Economy (NCEE) and the International Association for the Evaluation of Educational Achievement (IAE). We follow a snowball approach by searching the citations of relevant papers we found in the demand-for-skills literature, using a similar approach to improve the key words list. We select the studies to be included in the sample based on four criteria. First, we limit the sample to studies published from 2000 –2015 in order to capture those skills that are likely relevant for today’s policymakers while providing a sufficiently large sample of studies from various countries to carry out the meta-analysis.18 Second, we only include studies that present results that summarize empirical data collected from employers, thereby excluding studies based on expert opinion or qualitative assessments. We make one exception by including an analysis of the US O*NET (Casner-Lotto and Barrington 2006), which tabulates data provided by workers. Third, we limit the sample to studies that ask employers to rank the demand for, or difficulty in acquiring all four of, our broad skills sets—basic, higher- order, technical, and socio-emotional—in order to provide a preference ranking across the skill sets. Studies that do not provide a ranking or do not disaggregate within these broad skills sets are included in the discussion but not in the statistical meta-evaluation (Bassi et al. 2012; Burnett and Jayaram 2012; IFC 2011). Fourth, we attempt to ensure a reasonable level of comparability in the structure of the questions across the sample. Our approach yields a sample of 27 studies drawn from data from dozens of coun- tries. The sample includes two global studies (9 countries in Mourshed et al. 2012, 40 countries in Andreasson 2009), three studies from Western Europe (all in the UK), five from Eastern Europe (Romania, Russia, Macedonia, Poland, and Kazakhstan), five from Latin America and the Caribbean (St. Kitts, Peru, Mexico, and two that survey employers in several LAC countries), six from South/East Asia and the Pacific (India, Pakistan, Indonesia, Vietnam, Tonga, Solomon Islands), four studies from the United States, and one each from Africa (Botswana) and the Middle East (Lebanon).19 About half the studies select a nationally representative sample, stratifying by firm size, industry, or other characteristics, while others use ad hoc sampling methods such as drawing a sample from another survey (World Bank 2012; Shivpuri and Kim 2004; and Beneitone et al. 2007) or over-sampling specific types of workers (Bassi et al. 2012 and CIDAC 2014 focus on youth-oriented indus- tries; Burrus et al. 2012 selects higher-skilled occupations). The sample principally draws from small, medium, and large firms operating in the formal sector, while a few include a sizeable number of micro-firms (Martin et al. 2008; Hamid et al. 2011; Rutkowski 2010). A wide range of industries are represented, though the manufacturing sector is over-represented. While most of the studies ask employers about their entire labor force, some ask employers to focus on certain groups, such 106 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 as youth (Mourshed et al. 2012; CBI 2012; Martin et al. 2007; Close 2012; IFC 2011; Shivpuri and Kim 2004) or new recruits (Casner-Lotto and Barrington 2013; Zemsky 1997). Due to our selection criteria, the questions of interest are similarly structured across the sample. Twenty four studies ask employers to identify the most important skills that they need in their employees, either by asking an open question or by pro- viding a list of pre-determined skills, and to rank the skills they listed in order of importance. We refer to the results from these studies as the “priority ranking” of the most important skills. Fifteen studies measure the most important skills gaps. Employers are asked which skills are most lacking and to rank them according to how large the perceived gap is. We refer to the results of these studies as the “gaps ranking”. Each study present rankings for all employers in its sample, and a few also disaggregate rankings by industry, occupation, or worker education level. The broad range of countries potentially represent different labor market charac- teristics and institutions. This could complicate comparisons if different results emerge in different kinds of labor markets. However, the findings of the studies in our sample are quite similar, suggesting robust patterns that cut across labor market structures. The skills reported in the studies in our sample are stated preferences rather than revealed preferences, but several studies suggest that there is a strong overlap between the two. Globally, employers are increasingly using headhunter firms and IO psychologists to design and administer skills tests for hiring and promotion (Salgado et al. 2001); the tests typically include the four broad skills sets that we have defined for this paper. Since employers pay IO psychologists to administer these tests, we may assume that the measured skills are valued by the employer. A comparison of a survey to assess employer skill demand and the hiring tests used by headhunters finds a significant overlap between the two surveys (CIDAC 2014). Coming at the question from a different angle, the US O*Net, which asks employees to assess the degree to which they use different skills in nearly 1,000 professions, reports the use of skills classified in all four of our broad skill sets (Burrus et al. 2013), thereby demonstrating that skills actually used on the job are quite similar to those measured in the employer demand surveys used for this paper. The skills that employers value, as reported in the studies in our sample, are closely matched to those that they attempt to detect when hiring, whether for low- or high-skilled workers. Referring back to the IO psychology literature, hiring tests are regularly administered to potential new low- and high-skilled workers (Salgado et al. 2001). Further, a review of the skills posted on a Ukrainian public job-search site reports that socio-emotional skills are required of low- and high-skilled jobs, while technical skills are only specified for more skilled jobs (World Bank forthcom- ing). This is consistent with Osterman (2001), who concludes that employers of low-skilled workers in the US particularly depend on socio-emotional skills, which Cunningham and Villasen ˜ or 107 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 are congruent with low-paid service jobs, given the very low technical demands of the job, while employers of higher-skilled workers put greater emphasis on estab- lishing technical skills, although employers seek IO psychology testing in all three skill sets (Anderson 2001). While socio-emotional and higher-order cognitive skills may be difficult to assess, particularly in low-skilled workers, this may be where first impressions in the interview and employer preference for those with work experi- ence play a role.20 Employer responses are qualitative and not quantitative, so while we know which skill is most important and which skills gap is more pressing, we do not know how much more important a skill is as compared to a skill ranked just below it (Rutkowski 2010). To address this, we not only present the skill that is rated as the most important for each study, we also consider the second-most important skill and the five most important skills (those ranked first through fifth). We report the first, second, and top five ranked skills gaps as well. When employers identify the most important skill, they may not consider skills that are crucial to the production process, but are abundant in the work force. Since these skills are not used as a selection mechanism when hiring, assigning wages, or awarding promotions, they may be inadvertently forgotten by those being surveyed. To ensure that employers are not only reporting skills gaps, several of the surveys differentiate between the most important skill and the most important skill gap. Results presented below find that the results are, indeed, different. Methodology To identify those skills most demanded by employers and the biggest skills gaps, we first map the skills identified in each study into one of our four broad skills sets, s ¼ fbasic cognitive, higher-order cognitive, technical, and socio-emotionalg, as defined in table 1.21,22 We then calculate the share of studies n ¼ 1, . . . ,N that identify s, for rank r PN n¼1 sn ¼ sjr;s : ð1Þ N We start with r ¼ 1 and then repeat the exercise for the second, third, fourth, and fifth priority rankings (r ¼ 1, . . . ,5), giving us a matrix of skill set-ranking cells.23 Finally, we calculate the frequency that each skill set is listed in the top five. Since employers can select any skill for any r, the same broad skill set s may appear several times in the top five ranking. We repeat the exercise with employer rankings of the biggest skills gap. 108 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Table 1. Classification of Skills Reported in the Sample Socio-emotional Higher-order cognitive Basic cognivitive Technical Adaptability Analysis skills Basic literacy Advanced IT Collaboration Critical thinking Numeracy Advanced vocational Commitment Decision-making Basic vocational Control emotions Entrepreneurship Business awareness Conscientiousness Foreign language Capable Cooperation Innovation Computer literacy Creativity Intellect Degree level Conflict aversion Language Degree subject Cultural diversity Learning processes Education level Customer awareness Listening skills Experience Customer care Manage risk Grades Dependability Oral communication Hands-on training Detail-oriented Organization Industry-based skills Efficiency Planning IT knowledge Emotional stability Problem-solving Job-specific skills Enterprising Strategic management Office administration Extraversion Time management Practical knowledge Flexibility Thinking skills Professional skills Hard worker Written-communications Score on employer test Honesty Statistical analysis Initiative STEM Independence Technical skills Integrity Theoretical training Leadership University attended Modesty Work experience Motivation Negotiating Negotiate conflict Networking Open to new ideas Perseverance Personal appearance Positive attitude Proactive Punctuality Professionalism Responsibility Self-confidence Self-management Social values Stress-management Teamwork Work ethic Cunningham and Villasen ˜ or 109 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Results are presented in three ways. First, the broad skills sets corresponding to the five most important skills listed in each of 24 studies are discussed. Second, the skill set that was ranked as most important is presented, as well as a discussion of the skills underlying those broader sets. Finally, the second-most important skill set is briefly noted. We also compare the priority skills demanded by employers in different regions, industries, occupations, and by worker education level. We repeat this exercise using data from the 15 studies that measure the greatest skills gaps: presentation of the five greatest skills gaps, the most important skills gap, the second-most important skill gap, and a comparative discussion by region and occupation.24 Results Skills Employers Most Value The top panel of table 2 shows that of the five skills ranked as most important, more than half can be classified as socio-emotional, another 29.7% as higher-order cogni- tive, and 12.7% as technical. Specifically, socio-emotional skills are named 60 times from a potential 118 responses. Higher-order cognitive skills are listed 35 times, and technical skills 15 times. Basic cognitive skills are only named eight times among the top five preferred skills in our 24-study sample. Limiting the responses to only the skill identified as the most important, 79% name socio-emotional skills (table 2, top panel, ranking “1”). Nineteen studies rank Table 2. Employer Ranking of Most Important Skills and Greatest Skills Gaps, Percentage Ranking Socio-emotional Higher-order cognitive Basic cognitive Technical Sample size Most important skill 1 79.2 16.7 0.0 4.2 24 2 41.7 37.5 12.5 8.3 24 3 45.8 29.2 8.3 16.7 24 4 45.8 33.3 4.2 16.7 24 5 40.9 31.8 9.1 18.2 22 Total 50.8 29.7 6.8 12.7 118 Greatest skill gap 1 20.0 46.7 0.0 33.3 15 2 53.3 26.7 0.0 20.0 15 3 53.3 33.3 6.7 6.7 15 4 60.0 26.7 0.0 13.3 15 5 21.4 57.1 7.1 14.3 14 Total 41.9 37.8 2.7 17.6 74 Source: Authors’ elaboration based on sample data. 110 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 a socio-emotional skill as their first priority, with particular emphasis on teamwork, honesty, and punctuality, but also listing work ethic, interpersonal skills, work atti- tude, integrity, life skills (negotiation, cultural diversity), and responsibility (table 3). Another four studies rank higher-order cognitive skills as the most impor- tant skill set, including communications, problem-solving, and critical thinking skills. One study identifies technical skills as the most important skill set, though the specified “job-related skills” may encompass a larger set of skills. No study names basic cognitive skills as the most important skill set. Considering the skill identified as the second most-important, socio-emotional skills again emerge the most frequently (table 2, top panel). More than 40% of the sample names a socio-emotional skill, listing many of the same skills that are ranked first. Another 37.5% lists a higher-order cognitive skill, adding time man- agement to the list of priority skills. Basic cognitive skills are named by 12.5% of the sample, with an emphasis on literacy, while 8% identify a technical skill as the second priority. Again, the technical skills named—practical technical skills and professional knowledge—can also encompass other broad skills sets. These results are consistent with two studies that survey employer preferences in several countries; namely that socio-emotional skills are most important. Mourshed, Farell, and Barton (2012) interview employers in nine countries and ask them to rate, on a scale from one to ten (low to high) the importance of 13 pre- determined skills; 80% rank work ethic or teamwork—both socio-emotional skills—as the top skills. The higher-order cognitive skill of language and oral com- munication is ranked third (72% of employers), and hands-on training in discipline (technical) is ranked fourth, with approximately 70% of employers citing it as very important. Andreasson (2009) finds similar results among business executives in 40 countries who identify that the skills most in demand in the next decade are “life skills”, defined as negotiating, networking, working with cultural diversity (48% of the sample), followed by problem solving and leadership. The equation (1) results for the 24-study sample may mask heterogeneity among the skills needs of employers in different contexts. Thus, we cut the data along differ- ent dimensions to explore if the aggregate results are reflected in sub-sets of the sample. Skills Preferences by Region Developed countries, with their more sophisticated jobs, may have a greater need for higher-order cognitive and socio-emotional skills compared to less-complex de- veloping country economies. However, different regions of the world show similar patterns in the broad skill sets that employers most value. Socio-emotional skills most frequently emerge among the five most important skills in each region. The only exception is the Middle East, where higher-order cognitive skills dominate in Cunningham and Villasen ˜ or 111 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 112 Table 3. Employer Skills Set Preferences Priority ranking Region/ Source The World Bank Research Observer, vol. 31, no. 1 (February 2016) country 1 2 3 4 5 World work ethic teamwork local language communication hands-on Mourshed et al. 2012 experience World life skills problem solving leadership critical thinking teamwork Andreasson 2009 LAC interpersonal social values technical learning processes n/r Beneitone et al. 2007 LAC critical thinking problem solving life skills Leadership communication Ogier 2009 Botswana hard work honesty punctuality Adaptable communication World Bank 2014 India integrity reliability teamwork willingness to learn entrepreneurship Blom and Saeki 2011 Indonesia thinking skills basic skills behavioral skills Computer n/r diGropello 2011 Kazakhstan independent time management communication Literacy analytical skills Sondergaard 2012 work Lebanon teamwork communication time management problem solving Independent work World Bank 2012 Macedonia responsibility literacy communications customer care motivation Rutkowski 2010 Mexico teamwork communication problem solving Innovation conflict resolution CIDAC 2014 Pakistan punctuality honesty commitment Reliability behavior Hamid et al. 2011 Peru teamwork interpersonal capable Proactive work under World Bank 2011 pressure Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Cunningham and Villasen Poland responsibility motivation teamwork advanced technical self-management Arnhold et al. 2011 skill Romania efficiency professional problem solving Cooperation Planning Balcar 2012 knowledge St Kitts honesty work ethic problem solving communications Teamwork Blom and Hobbs 2008 Solomon work attitude communication experience education level decision making Close 2012 ˜ or Islands Tonga honesty punctuality teamwork/ customer skills hard work TCCI 2010 communication Vietnam job-related skills practical technical punctuality drive/teamwork theoretical World Bank 2008 knowledge UK communication literacy numeracy enthusiasm teamwork Martin 2008 US interpersonal ethics/integrity leadership perseverance knowledge Shivpuri and Kim 2014 US problem solving fluid intelligence teamwork Innovation communication Burrus et al. 2013 US work ethic Communication teamwork critical thinking communication Casner-Lotto and Barrington 2006 US attitude communication industry-specific years of schooling test scores Zemsky 1997 Source: Data drawn from each paper cited in last column; “n/r” indicates that a ranking was not requested. 113 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 the single-country study from that region (World Bank 2012). Higher-order cogni- tive skills appear second-most frequently among the top five skills in Latin America and the Caribbean (LAC), Eastern Europe and developed countries, while technical skills are the second-most cited in the East Asia sample. Basic cognitive skills are rarely mentioned among the top five skills in any region. Socio-emotional skills are the top-rated skill set in most studies in LAC, South East Asia, Middle East/North Africa, Africa, Eastern Europe, and developed coun- tries (US and UK), and the specific type of skill is similar across studies. Five of the six Latin American surveys conclude that employers most value socio-emotional skills, specifically, inter-personal skills, teamwork, attitude, responsibility, honesty, and integrity (Beneitone et al. 2007, Bassi et al. 2012, World Bank 2011, Blom and Hobbs 2008).25 The two South East Asia studies give top ranking to socio- emotional skills, that is, punctuality, honesty, integrity, and reliability (Hamid et al. 2011, Blom and Saeki 2011). The sole Middle East/North Africa study reports that 70% of Lebanese employers most highly rate teamwork (World Bank 2012) while work ethic is the top rated skill by 70% of Botswanan employers, reported in the only African study (World Bank 2014). Three of the four Eastern European studies rank a socio-emotional skills as the priority skill, singling out responsibility and effi- ciency as the most desired skills in Romania, Macedonia, and Poland (Balcar 2012, Rutkowski 2010, Arnhold et al. 2011); a higher-order cognitive skill (independent work) is ranked first by nearly all employers in Kazahkstan (Sondergaard et al. 2012). Three of the five developed country studies (Shivpuri and Kim 2004; Casner-Lotto and Barrington 2006; Zemsky 1997) find that socio-emotional skills—work ethic, attitude and interpersonal skills—are the preferred skills; the other two countries name higher-order cognitive skills (Burrus et al. 2013, Martin 2008). The pattern is less uniform in East Asia and the Pacific. Two studies rank socio- emotional skills first, specifying work attitude, and honesty and punctuality (Close 2012; Tonga Chamber of Commerce 2010); this is similar to the skills named in the other regions. However, “job-related skills,” which we classify as technical, were pri- oritized in the Vietnam study (World Bank 2008), while cognitive skills (higher- order for managers, basic for workers) were ranked first in Indonesia (di Gropello 2011). Skills preferences are less uniform across regions when assessing the second-most important skill. While socio-emotional skills still dominate among LAC and South Asian employers, other skill sets emerge in the other regions. For example, of the four East Asia Pacific studies, each identifies a different skill set as second priority: basic cognitive in Indonesia (di Gropello 2011), technical in Vietnam (World Bank 2008), higher-order cognitive in the Solomon Islands (Close 2012), and socio-emotional in Tonga (Tonga Chamber of Commerce 2010). Similarly, in Eastern Europe technical skill emerges as a second priority in Romania (Balcar 2012) and basic cognitive 114 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 (literacy) is identified in Macedonia (Rutkowski 2010), while socio-emotional skills (motivation and time management) are ranked second in Poland (Arnhold et al. 2011) and Kazakhstan (Sondergaard 2012). In conclusion, in spite of the small number of studies per region, the evidence to date suggests that all four skills sets are important to employers regardless of region, with more weight placed on socio-emotional and higher-order cognitive skills. Most Important Skill by Industry: Manufacturing v. Services Given the different nature of job tasks in the manufacturing and service industries, we might expect to find a greater demand for technical skills in the former, and for socio-emotional skills in the latter. Three of our studies separately report the skills preference rankings for employers in the manufacturing sector and those in the service sector (World Bank 2008; diGropello 2011; diGropello 2008), allowing us to explore this assumption. Starting with the top five ranked skills, socio-emotional and higher-order cognitive skills are named most frequently in both the manufac- turing and service sectors. Socio-emotional skills were listed six times each (of a po- tential 18 mentions for each country-industry matrix) for the manufacturing and service sectors, with particular emphasis placed on independent work, teamwork, and creativity in both sectors. Higher-order cognitive skills—problem-solving and communications—were listed six times for the manufacturing sector, and five times in the service sector. Technical skills were named three times each by both sectors: computer skills, practical experience, and theoretical knowledge of the job. Turning to a comparison of the skill that is ranked r for each industry, the pre- ferred skill set in manufacturing is also the preferred skill set in services in each of the three countries in the sample. For example, Filipino manufacturing employers rank first the socio-emotional skill of independent work (15%). Service sector em- ployers also rank independent work as the most important skill. The pattern is re- peated for the second, third and fourth rankings in the countries in the sample. Although the industry-specific ranking of the most important skills within country are identical, the weight on each priority ranking reveals expected patterns: socio-emotional skills are more important in the service sector than the manufac- turing sector. For example, 54% of Indonesian manufacturing sector employers rate behavioral skills as very important (for their professional staff ), as compared to 72% of service firm employers (diGropello 2011). Higher-order communications skills are slightly more important in the service sector than the manufacturing sector, as observed in the Philippines (11.5% versus 10%) (diGropello 2010) and Vietnam (10% versus 8%) (World Bank 2008). There is not a global pattern in the most important skill in each industry across the three studies that provide industry-level disaggregation. For example, employers in the manufacturing sector prioritize socio-emotional skills in Latin America Cunningham and Villasen ˜ or 115 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 (Bassi et al. 2012), “practical” technical skills in Vietnam (World Bank 2008), and cognitive skills in Indonesia (diGropello 2011). Similar patterns emerge for service sector employers. We can conclude, with the caveats associated with a small sample, that within-country skills profiles dominate industry skills profiles. Most Important Skills by Occupation: Managers v. Workers Given the leadership and team management role of managers, we might expect employers to demand more higher-order cognitive and socio-emotional skills from their managers and more information-based skills (technical and basic cognitive) from their workers. Six studies report employer rankings of the most important skills for managers as compared to workers (Bassi et al. 2012; CIDAC 2014; diGropello 2011; World Bank 2014; Vasiliev et al. 2013; World Bank 2012). Socio-emotional skills strongly emerge among the top five skills that employers seek in both managers and workers, with higher-order cognitive skills also appearing frequently. Socio-emotional skills were listed ten times (of a potential 30 responses) for managers and 15 times for workers. Teamwork, honesty, hard work, punctuality, and leadership were named for both managers and workers, with slightly more emphasis placed on flexibility, inde- pendent work, and self-management for workers. Higher-order skills were listed ten times for managers and six times for workers. Employers prioritize analytical skills, problem-solving, decision-making, and communication among managers, with more emphasis placed on communication for workers. Technical and basic cognitive skills were named once each for managers and for workers. Employers across countries demand similar skills by occupation, though the range of preferred skills of workers is much broader than that of managers. In all six countries in the sample, employers sought managers with strong socio-emotional and higher-order cognitive skills, particularly problem-solving, teamwork, and leadership. These skills were also demanded of workers across countries, with team- work added to the list in Russia, Lebanon, and Mexico (Vasiliev et al. 2013; World Bank 2012; CIDAC 2014). However, a range of additional skills were sought in workers, such as basic cognitive skills (diGroppelo 2011), technical skills (Vasiliev et al. 2013), communications (World Bank 2012; CIDAC 2014), punctuality (World Bank 2014; World Bank 2012), and hard work (World Bank 2014). This small sample comparison suggests that there is a global set of skills for managers, while worker skill sets are somewhat country specific. Most Important Skill for the Less v. More Educated Workforce Since more educated workers are likely to hold more complex jobs, employers may require more higher-order cognitive skills and technical skills for these employees than for those who are less educated. Four country studies report employer demand 116 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 for skills for more educated compared to less educated employees (Burrus et al. 2013; Casner-Lotto and Barrington 2006; World Bank 2011, 2008), and find that employers overwhelmingly prioritize socio-emotional skills across education levels, while valuing higher-order cognitive skills in their more educated workers. Socio- emotional skills were listed 13 times (of a potential 20 responses) as one of the top five skills of less educated workers as compared to three mentions of a technical skill and one each of higher-order and basic cognitive skills. Similarly, socio-emotional skills were listed 10 times for more educated workers, and higher-order skills were listed five times. Technical skills were named twice as one of the top skills among more educated workers. Within country comparisons reveal that employers give a similar priority ranking to the broad skill sets sought in more and less educated workers, though the specific skill differs by education level in the four countries in the sample. For example, of the seven skills presented in the Peruvian survey, employers most demanded socio-emotional skills for more and less educated workers, naming interpersonal skills for workers with more than a tertiary education (17%) and teamwork for workers with less than a completed secondary education (23%; World Bank 2011). Cross-country comparisons show that regardless of education level, workers are expected to excel in socio-emotional skills. Employers of less educated workers in the four country sample all prioritize socio-emotional skills, naming work ethic, depend- ability, teamwork, and punctuality. The same skills are listed by employers of more edu- cated workers in all four countries. The only exceptions are that higher-order cognitive (communication) emerges for more educated workers in one country (the United States, as reported in Casner-Lotto and Barrington 2006), and Vietnam emphasizes technical skills for both its more and less educated workers, while it is barely men- tioned by employers in other countries. Partial Correlations The Indonesia study presents a particularly rich disaggregation of employer skills demands, allowing us to understand priority skills in finer segments of the labor market.26 The data show that the most important skills that employers seek in manag- ers are similar whether the manager is in the service or in the manufacturing sector. The only trend that is dampened when aggregating the data by occupation (such that industry is no longer broken out) is that the share of employers that identify socio- emotional skills as most important in their managers (64%) is overstated for manufac- turing managers (54%) and understated for service managers (72%); however, this skill is ranked as second-most important for managers regardless of sector. The most important skill for workers are, again, quite similar across industries with one excep- tion: socio-emotional skills are ranked second by service sector employers (38%) and third by manufacturing sector employers (25%; diGropello 2011). Cunningham and Villasen ˜ or 117 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Greatest Skill Gaps As noted above, it is possible that when employers cite the most important skill for the production process, they may actually be citing the greatest skill gap. A review of the skills gap literature shows that employers are not confusing these concepts, with different trends emerging in the gap analysis than those observed in the previ- ous exercise. The bottom panel of table 2 presents the greatest skills gaps, as calculated in equation (1), and shows that higher-order cognitive skills emerge more strongly in the gap analysis than in the priority ranking of the most important skills.27 More than 40% of the 15-study sample identifies a socio-emotional skill as one of the top five skill gaps, naming a lack of behavioral and inter-personal skills, whereas “work ethic,” which emerged strongly as one of the most important skills valued by employers, as discussed above, does not emerge as a priority skill gap (table 4). Nearly an equal share of studies, 38%, name a higher-order cognitive skill as one of the top five skill gaps. The specific skills listed are similar to those identified above by employers as the most important skills. Turning to the top-ranked skill gap, higher-order cognitive skills are named by nearly half of the employers, and technical skills, including professional skills, job- specific skills and work experience, are named by 33% of the sampled employers. While we classified all of these as “technical”, several likely include all four of our broad skills sets. For example, “professional skills” or “work experience” may include knowledge of specific equipment (technical), working with others (socio- emotional), the ability to resolve problems (higher-order cognitive), and basic math for operating the equipment (basic cognitive). The second-most important skills gap is dominated by socio-emotional skills, named by 53.3% of the sample (table 2, bottom panel). Behavioral rather than values-related skills (such as honesty, work ethic) emerge. Another 26% name higher-order cognitive skills as the second-most important skill gap, while one-fifth of the sample lists a technical skill. Basic cognitive skills are barely mentioned as an important skill gap; this may suggest that workers’ dominance of basic numeracy and literacy are adequate for employers in our sample, or that other skills gaps are more noticeable to employers. Most Pressing Skills Gaps by Region As discussed above, though the skills that employers most value did not differ by region, the different education and industry structures may lead to different skills gap profiles in more and less sophisticated economies. The expected trends emerge to some extent where technical skills gaps are most observed by Eastern European employers, while developed country employers note the absence of higher-order 118 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Cunningham and Villasen Table 4. Employer Skills Gaps ˜ or Country 1 2 3 4 5 Source Global problem solving communication work experience teamwork English Mourshed et al. 2012 Latin America critical thinking communication life skills STEM leadership Ogier 2009 Indonesia English computer thinking skills behavioral n/r diGropello 2011 Lebanon independent work computer Numeracy foreign language communication World Bank 2012 Macedonia responsibility motivation Communication customer orientation literacy Rutkowski 2010 Mexico technical knowledge leadership Innovation teamwork communication CIDAC 2014 Poland advanced technical skill responsibility motivation/commitment self-management problem solving Arnhold et al. 2011 Romania professional skills efficiency problem solving customer orientation planning Balcar 2012 Russia professional skills problem solving conscientiousness decision-making teamwork Vasiliev et al. 2013 Solomon Islands analytical skills work attitude decision making communication technical skills Close 2012 Tonga honesty punctuality customer awareness commitment problem solving TCCI 2010 United Kingdom foreign language customer awareness diversity self-management problem solving CBI 2012 United Kingdom planning customer awareness teamwork problem solving communication UKCES 2012 United Kingdom job-specific skills business awareness enterprising vocation-specific skills customer care Martin 2008 United States language communication leadership work ethic technical skills Casner-Lotto and Barrington 2006 Source: Data drawn from each paper cited in last column. 119 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 cognitive skills. In three (Romania, Russia, and Poland) of the four (Kazakhstan) studies of Eastern European employers (Balcar 2012; Vasiliev et al. 2013; Arnhold et al. 2011), the lack of technical skills was most lamented, followed by socio-emo- tional skills (efficiency, motivation, and responsibility). In contrast, employers in the US and UK largely identify a lack of higher-order cognitive skills (language, plan- ning), though one of the three UK studies identifies job-specific skills as the primary skill gap. Socio-emotional skills begin to emerge as the second-greatest skills gap in both Eastern Europe and developed country priority rakings. The results are too mixed in East Asia, and the sample is too small in Latin America, Africa, and the Middle East to report regional patterns. Most Important Skills Gaps: Managers v. Workers Employers in the five studies that disaggregate employer-perceived skills gaps by oc- cupation (UKCES 2012; diGropello 2011; Vasiliev et al. 2013; World Bank 2012; CIDAC 2014) list a higher-order cognitive skill, or socio-emotional skills nine times each (of 25 potential responses) as one of the top five skills gaps faced by their non- manager workers; employers name technical skills gaps four times. The employers emphasize socio-emotional skills as most lacking in their managers, naming them 10 times, compared to seven mentions of higher-order cognitive skills and technical skills gaps, which were mentioned five times. Although employers value similar skills in managers and workers, the top skills gaps differ by occupation. Similar to the aggregate measurement of the greatest skills gaps, the top-ranked skill gap among managers in three countries are primarily higher-order cognitive skills—decision-making, strategic management, and foreign language. Technical and socio-emotional skills are the most highly ranked skills gap in managers by one country each (Lebanon and Mexico, respectively). In contrast, a range of broad skills sets emerged as most lacking among workers. Employers in two countries named technical skills as most missing in their workers (UK and Mexico) while two others listed socio-emotional skills (conscientiousness and independent work). The fifth country named a higher-order cognitive skill (English language). The second-most important skills gaps for non-managerial workers reveal the same pattern: employers in two countries listing technical skills, and one country each listing higher-order cognitive, basic cognitive, and socio-emotional skills. The Skills Development Process and Policy for Developing the Skills Employers Demand Employer voices tell us that a broad range of skills are necessary for labor success. Some skills are taught after completing general education, particularly technical 120 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 training, while others are commonly acquired at a younger age, namely basic cogni- tive skills. A relatively large body of literature discusses appropriate methodologies for teaching these skills. However, there is much less guidance on the appropriate ages to learn socio-emotional and higher-order cognitive skills and the appropriate methodologies to teach them. The developmental psychology and education litera- ture provides a framework to guide these skill development processes. Just as with basic cognitive and technical skills, socio-emotional and higher-order cognitive skills are best taught at specific stages in the life-cycle. Neurological, biologi- cal, psychological, and social processes dictate that certain skills cannot be learned before certain ages (Guerra et al. 2014). For example, a toddler is me-centered and is not able to feel the genuine empathy that a primary school student displays. It is not for a lack of being taught to be empathetic, but instead the toddler is not neurological- ly or psychologically “ready”, and a toddlers’ social context—where she is still largely guided by caregivers—is not conducive to practicing, and thereby developing, this skill. Further, certain skills are the foundation for other skills (Cunha et al. 2005). For example, basic math—which is developmentally appropriate for primary school—is a foundation for secondary-school introduction to physics, just as impulse control is a foundation for the higher-order cognitive skill of problem solving. Heckman (2008) argues that most of the skills gaps at age 18 that help to explain adult outcomes are already present at age five, and that disadvantaged children are at a particular risk of falling behind early and not being able to catch up. Table 5 presents a rough representation of the appropriate period of the life-cycle during which one may acquire the skills that the employer surveys suggests. In the early years (age 0 –5), the most basic cognitive skills can be acquired. Also, some of the most important foundational socio-emotional skills are developed during this period, such as delayed gratification, impulse control, and working with others. During middle childhood (age 6– 11), learning really takes off with the ability to rapidly acquire basic cognitive skills—with some higher-order cognitive emerging, such as problem solving—and the child is in a context to develop more complex socio-emotional skills related to engaging and negotiating with others. During ado- lescence (12 –17), the foundations should already be built, the brain is neurologi- cally and psychologically ready, and the social context is appropriate to fully engage in higher-order cognitive development and complex socio-emotional development while still acquiring basic cognitive skills and beginning to develop technical inter- ests and skills. Once reaching early adulthood (18 –26), technical skills can be built on the foundation of the basic cognitive skills, higher-order cognitive skills, and socio-emotional skills learned earlier in life. A lot of skill development occurs outside the classroom, indicating that a wide range of age-relevant actors are best positioned to develop the young person’s skill sets (table 5). Drawing from the Bronfenbrenner ecological risk framework (1979), we see that a young person’s actors of influence broaden, and move away from the Cunningham and Villasen ˜ or 121 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Table 5. Skill Formation at Different Points of the Life-cycle Sample programs to Guide Actors to Build the Skills (for a list of evidence-based programs, Period Skill type Age-relevant Actors see Guerra et al. 2014) Early years † Basic cognitive Family, ECD programs Quality parenting (Nuevo (0 – 5) † Foundational Postnatal, Program on Socio- Cognitive Development, Early emotional Enrichment Program) Child-focused ECD (Perry Program, Head Start) Childhood † Basic cognitive Family, schools, peers Holistic curriculum, teaching (5 – 12) † Socio-emotional methodology, and monitoring † Foundational and evaluation system (KIPP , higher-order EPSIS, Ensen ˜ a Chile, RCCP) cognitive After-school/extra-school/ Adolescence † Basic cognitive Schools, peers, out-of-school programs/ (13 – 17) † Socio-emotional mentors, family activities (BBBS, Student † Higher-order Success Teams) cognitive † Initial technical Early (17– 29) and † Socio-emotional Higher education Apprenticeships (Jo ´ venes Middle (30 1 ) † Higher-order institutions, training programs) Adulthood cognitive institutes, work place, Experiential on-the-job training † Technical family Skills certification system, support systems for worker transition to firms where new learning can occur Source: Own elaboration based on Banerji et al. (2010) and Guerra, Modecki, and Cunningham (2014). nuclear family, as she ages. In early childhood, family and early childhood develop- ment programs are the age-relevant actors due to the biological forces of children being psychologically attached to a core, known family, and to practical issues related to a child’s independence. Thus, these are the actors responsible for develop- ing age-associated skills. During middle childhood, the school gains in importance, as do peers and other mentors, but the family still plays a dominant role. During adolescence, the family starts to fade as peers, educational institutions, and non- family mentors grow in importance, and finally, in young adulthood, socio- emotional skills are refined and shaped by higher education institutions (Robins et al. 2001), work environment, and the family (Roberts, Caspi, and Moffitt 2003).28 Technical skill development also continues into adulthood through on- the-job training; in fact, once reaching adulthood, firms are the primary source of ˜ or 2013). new skills acquisition for workers (Villasen 122 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 There are a multitude of methods for effectively teaching the appropriate skills by each actor at each life-cycle stage (table 5). For parents of young children, good family leave policies that allow parents to provide quality parenting and programs to enhance parental learning and encouragement of early stimulation and nutrition have been shown to foster greater acquisition of cognitive skills and socio- emotional skills in a child (Gertler et al. 2014; Kagitcibasi 1988). Child-centered ECD that focuses on improving personality traits and managing externalizing behaviors while also acquiring basic cognitive skills have shown positive results in employment, wages, and positive behaviors for more than 30 years after program participation (Schweinhart et al. 2005). Programs to help develop socio-emotional learning in middle childhood are largely designed for the school setting, and are complemented by parental engagement. CASEL (2013) recommends four prerequisites for developing socio-emotional skills in schools: emotionally supportive teachers, a positive school climate, socio-emotional learning in- tegrated into subject-based lessons, and free-standing lessons to teach socio-emotional skills. These may take place in the classroom via regular curriculum and instruction activities, or through complementary programs. However, the research finds that the most successful socio-emotional learning occurs when incorporated into the day-to-day curriculum and when connected to other school activities (Greenberg et al. 2003). For example, the US Knowledge is Power Program (KIPP) implemented in primary and sec- ondary schools sets expectations, requires behaviors grounded in good socio-economic skills, and works closely with each child and their families to ensure success (Angrist et al. 2010). Similar programs exist in developing countries as well (Heckman and Kautz 2012; Alfonso et al. 2012). Impact evidence shows these methods positively affect socio-emotional skill development, associated positive behaviors, and cognitive learning outcomes, especially in middle-childhood (CASEL 2013; 2015). Both middle childhood and adolescence are key periods for developing higher- order cognitive skills, and pedagogical methods can be powerful tools for fostering these skills. The most effective methodology for teaching problem-solving, creativi- ty, communications, and overall thinking skills is largely through student-centered, structured pedagogical methods (King et al. 1998) where the educator uses ques- tions, guided practice, team activities, student discussions, and peer tutoring, for example, to engage students to solve problems, test their ideas with other students, and express themselves verbally and in writing (Brookhart 2010). When the same material is taught through traditional didactic methods to a control group and in- teractive methods to a treatment group, the group receiving the interactive teaching has a statistically significant greater performance on language and math exams than the group receiving the didactic instruction (Brookhart 2010).29 While adolescents benefit from the school-based programs (CASEL 2015), their greater independence from the family sphere and initial transition to world of work implies that mentoring programs and technical training can also contribute to their Cunningham and Villasen ˜ or 123 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 socio-emotional and higher-order cognitive skills development process. Mentoring programs can take different forms such as after-school clubs, programs that pair model adults with children, or sports programs run by child development special- ists; the former two models have shown to increase cognitive and socio-emotional skills of participants relative to control groups (Tierney and Baldwin 2000; Boys & Girls Clubs of America 2004). As the school-to-work transition process begins, job training and apprenticeship programs should be augmented by socio-emotional and higher-order cognitive skills development. For example, such a program in the Dominican Republic has been shown to increase leadership, conflict management, self-esteem, interpersonal skills (for some groups), organization skills, empathy, and hard work, in comparison to a randomly selected control group that did not receive the intervention (Vezza et al. 2014; Ibarraran et al. 2012). Skills development in early adulthood is primarily focused on acquiring technical and higher-order cognitive skills; programs for socio-emotional skill development are quite rare. There is an increasing consensus that technical training models need to reform to better teach the range of skills that employers demand (OECD 2014; Fawcett 2014). Specifically, training should be delivered through structured experi- ence-based (rather than classroom) learning that reinforces all four broad skills sets by real-world practice in a supportive context. Although experiential learning is common practice in European technical training systems (Fawcett et al. 2014), em- ployers in other countries can find room for improvement. For example, Latin American executives identify “work-study programs that bring students the work place” as the primary way that firms can contribute to skills development, though few engage in the practice (Ogier 2009). The public sector can provide technical assis- tance and financial incentives to alleviate the costs of designing and implementing a training program while not affecting firm profitability. For example, Colombian law (Law 789 of 2002) provides tax breaks to firms to train their workers. Linked to this is the emerging recognition that instructors should be recruited from the private sector who can provide hands-on knowledge. Most Latin American executives in a small sample thought that the lack of teaching skill to teach in an applied manner is a bigger problem than educational infrastructure, curriculum, or teacher qualifica- tions in the subject matter (Ogier 2009). Conclusions The review confirms that there is a mismatch between the education sector’s per- ception of skills demand and that of the productive sector. While the education sector believes that it prepares students well for the labor market (Mourshed, Farrell, and Barton 2012; IFC 2011) by focusing on basic cognitive and technical skills development, a review of 24 studies that measure employer skills preferences 124 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 and 15 studies that measure key skill gaps find that employers have a different view of the most important skills for labor market success and where the principle skills gaps are. While employers around the world value all four skills sets, socio-emotional and higher-order cognitive skills are consistently prioritized. More than 50% of the top five skills identified by employers can be classified as socio-emotional skills, and 79% of the top-ranked priority skill are a socio-emotional skill. Another 30% of the top five skills can be classified as higher-order cognitive, with a top ranking in about 17% of the cases. Oral communication—a higher-order cognitive skill—ranks con- sistently very high, as do a small set of socio-emotional skills, namely ethics, punc- tuality, honesty, and teamwork. Technical skills, which are often assumed by policymakers to be equivalent to job skills, were ranked as third-most important in the aggregate estimate, but they emerge strongly for some groups. Specifically, Eastern European employers add technical skills to the list of priority skills sets, joining socio-emotional and higher- order cognitive skills. Employers in all other regions and in the US do not value technical skills as highly. This variable was difficult to analyze, though, since the classification of “technical skills” may have over-simplified employer preferences since all “job-related” responses were put in this skills set, even though many job related skills are socio-emotional or higher-order cognitive by nature. Technical skills seem to be complements to, not substitutes for, cognitive and socio-emotional skills. Basic cognitive skills were rarely prioritized, which may reflect that these skills are not needed or, more likely, that they are in sufficient supply that employers do not notice how important they are. This interpretation is supported by the lack of mention of basic cognitive skills among managers, who most certainly need these skills to perform their jobs well. The aggregate patterns hold up when dividing the sample by region, industry (manufacturing v. service firms), occupation (managers v. workers), and education level of the workforce; socio-emotional and higher-order cognitive skills emerge most strongly for each sub-group. And while the priority skills sets for managers and workers or for less and more educated workers are the same across countries, priority skills sets for manufacturing versus service sector differ across the three countries in the sub-sample. These results should be understood with a large margin of error given the very small size of the samples. Employers identify significant skills gaps. The aggregate analysis shows that socio-emotional skills were most cited among the top five skills gaps, but higher- order cognitive skills were the top-ranked skill gap. About 42% of the top five skills gaps can be classified as socio-emotional skills. However, they dominate the second through fourth priority rankings. Instead, higher-order cognitive skills are identified as the most important types of skills in nearly half of the sample. When Cunningham and Villasen ˜ or 125 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 disaggregating by region and occupation, slightly different trends emerge. Among regions, technical skills are named as the priority skill gap in Eastern Europe, but not named as the top skill gap by any study in any other region. Socio-emotional skills are particularly important in developing regions compared to the developed countries in the sample. Although socio-emotional and higher-order cognitive skills are consistently valued by employers, they often fall outside of school curriculum or teaching methods. When we bring employer preferences together with the skills formation process as understood by developmental psychologists, four key conclusions for edu- cation/skills development policy emerge. First, the skills development process neces- sarily begins at birth (or before) and continues throughout the life cycle. Certain skills that employers demand are formed in the toddler years and other skills can only be developed once the foundational skills are there. Waiting until school is completed to begin developing job-relevant skills is too late. Second, schools can play a much larger role in socio-emotional skill development by developing a teach- er’s personal skills, strengthening the school climate so it encourages positive be- haviors, weaving socio-emotional skills development and practice into pedagogical methods, and teaching these skills as part of the curriculum. Third, other actors should play a larger and more structured role in skills development. Certain skills are better taught by parents, mentors, or colleagues in the work place. This points to an education/skills development strategy and related programs to support the actors that are best suited to provide instruction to children at each age-appropriate stage. Fourth, the skills most demanded by employers—higher-order cognitive skills and socio-emotional skills—are largely taught (the former) or refined during ado- lescence, which argues for a secondary school general education until these skills are formed. Rather than the early tracking of youth into technical training (ranked third by employers), skills/education systems need to ensure that the foundational basic and higher-order cognitive and socio-emotional skills are there to allow for effective technical skill acquisition. While this paper presents evidence from the literature, more work needs to be done to definitively guide policy. First, the research needs to move beyond simple correlations and into marginal effects, namely, to generate parametric estimates that will better define the skills sets demanded by employers in different industries and occupations. Second, the question of revealed versus stated preferences merits greater research to determine the validity of employers preference surveys. While this paper presents evidence that these preference sets are very similar, by compar- ing employer survey skills to those used by for-profit headhunting firms and want ads, a direct analysis is necessary to understand the extent to which this gap may bias employer survey findings. Finally, the absence of consistency in definitions and measurement makes cross-country comparisons difficult. The research needs to move toward a common set of concepts to allow for cross-country learning. 126 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Many thanks to Monica Parra and Mariana Escalante for their contributions to the paper, and to Stephen Close, Margaret Grosh, Katia Herrera, William Maloney, Noel Muller, Jan Rutkowski, Fred Oswald, and Alexandria Valerio for their critical review and suggestions. Notes Wendy V . Cunningham is a Lead Economist at The World Bank; Wcunningham@worldbank.org. Paula Villasen ˜ or is the Social Promotion and Participation Coordinator for Upper Secondary Education at the Mexico Ministry of Education; paulavillasenor@gmail.com. 1. Hanushek and Woessman (2008) identify two sources of measurement error. First, there is sub- stantial heterogeneity in the skills acquired at each level of schooling across countries, regions within countries, and schools within regions. Second, much skill acquisition occurs outside of the classroom (Hanusheck 1979). 2. See Almlund et al. (2011) and Borghans et al. (2008) for reviews of this literature. 3. Autor, Levy, and Murnane (2003) illustrate this phenomenon by arguing that as firm technolo- gy improves, repetitive and predictable tasks are automated and workers performing routine tasks are substituted by computers, which themselves need to be complemented by workers who perform non- routine problem solving. 4. We intentionally include a broad geographical mix of countries to permit us to identify if there are global trends in skills demand or if skills demand and shortages differ by region or country. 5. Although the economics literature divides skills into cognitive and non-cognitive skills, psychol- ogists argue that the latter is a misnomer since they clearly involve cognition. With a nod to our psy- chology colleagues, this paper instead uses the term “socio-emotional skills.” For a light-hearted review of terminology, see http://www.npr.org/sections/ed/2015/05/28/404684712/non-academic- skills-are-key-to-success-but-what-should-we-call-them?utm_campaign=storyshare&utm_source=twitter. com&utm_medium=social. 6. The concept of “higher” and “lower” order (or, what we call “basic”) cognitive skills emerges from Bloom’s taxonomy of learning, where knowledge and comprehension are classified as “lower order” skills; built upon these are the higher-order skills encompassed in critical thinking (Bloom et al. 1956). 7. This study finds that the impact of school attainment on wages falls from 7.1–5.9% after con- trolling for literacy scores. 8. The psychology literature defines technical skills as a sub-set of cognitive skills (Almlund et al. 2011). 9. Tan and Nam (2012) review recent studies estimating the wage premium for technical training to general education and find higher returns to the latter. The results are merely suggestive, though, since the reviewed studies do not control for unobservable skills, which may be both correlated with higher wages, and with the choice of education path, thus potentially biasing the comparison. 10. The idea that socio-emotional skills are an important driver of economic success can be traced to Bowles and Gintis (1976), who explain that a measurable part of the variance in earnings among observationally equal individuals, particularly those with equal levels of education, are due to behavio- ral skills. 11. The socio-emotional skills that are most commonly measured and correlated in the economics literature are limited to personality trait facets based on the Goldberg Big 5 construct (Goldberg 1993), listed above. While this construct has been validated across cultures and time, it is limited in that it does not measure behaviors or attitudes. 12. See Almlund (2011) for a review of literature. Cunningham and Villasen ˜ or 127 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 13. This search produced papers from literature in economics, human resources psychology, in- dustrial sociology, management, and education. 14. Research Papers in Economics (IDEAS-RePEc) is the largest bibliographic database dedicated to Economics in which most of the papers are fully downloadable; see http://ideas.repec.org/. We searched the following JEL codes: J23 (Labor Demand) and J24 (Human Capital; Skills; Occupational Choice; Labor Productivity). 15. EconLit is the American Economic Association’s electronic bibliography; see www.aeaweb.org/ econlit/. 16. See www.eric.ed.gov; ERIC is one of the primary data base for education literature. 17. The website www.psycontent.com is the database of the American Psychological Association, specialized in psychology and psychiatric journals, and includes the following databases: PsyJOURNALS, PsycARTICLESw, PsycINFOw and PsyCOLLECTIONw. 18. The earlier literature is summarized in Cotton (1993). We include one paper from 1997 in order to balance out the samples that use US data. The recent papers only ask employers about skills demand for more educated workers; Zemsky (1997) provides information about the median worker. 19. A detailed description of each study sample frame is available from the authors. 20. Anecdotes (from the authors’ interviews) may illustrate the point. An owner of a bottling company in Tonga said that when interviewing, he first considers the body language when meeting the candidate, then assesses eye contact, and, once the person passes that phase, he asks about techni- cal skills. A psychologist working with youth in Jamaica tells of a young woman who, after several failed interviews, asked if the profane tattoo on her forehead may have been a factor in her lack of success. 21. Many surveys do not go into detail on technical skills, so “job-relevant skills”, computer skills, and “work experience” are included in this set. 22. The socio-emotional skills defined in the employer surveys differ from those in the economics literature. The standard research uses umbrella concepts that encompass a range of attitudes and be- haviors while employers indicate very specific attitudes and behaviors. For example, the “teamwork” skill that many employers define is not included in the standard literature, but teamwork concept is en- compassed in the extraversion and agreeableness umbrella concepts that have been well studied. Guerra, Modecki, and Cunningham (2014) map the 140 employer skills identified in this paper to the Goldberg Big 5 construct most commonly used in the economics literature. 23. Several studies only allow employers to priority rank the top five skills or skill gaps. Thus, we also limit r to five. 24. The data do not provide sufficient information to disaggregate skills gaps patterns by industry or by education level of the worker. 25. Ability to collaborate and cooperate with others, control emotions, and avoid negative reactions 26. The data in this review do not allow us to estimate partial correlations. However, the Indonesia study provides some insight into the question of partial versus simple correlations since it observes the prioritization of two dimensions simultaneously: industry and occupation. 27. Unlike in the sample which asks about the most important skills, the data on key skill gaps is disaggregated across skill level and type of industry in many of the studies. Since we do not know the share of the labor force in each of these categories, we cannot appropriately weight the responses. Thus, we make a very general assumption that the labor force is equally distributed across these cate- gories and we take a simple average across skill level or industry. 28. Contrary to decades of assumptions of the non-malleability of personality traits, psychologists are finding that personality change continues into adulthood once careers have been established that shape personality more profoundly than transitory early adulthood jobs (Roberts 1997), and as life changes such as marriage affect personality (Robins, Caspi, and Moffitt 2002). 29. Given the difficulty of directly measuring higher-order cognitive skills (see King et al. 1998 for a discussion), they are indirectly measured through performance on content tests that require the use of higher-order cognitive skills to score higher. 128 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 References *Indicates the reference is a data point for this paper. Acemoglu, D., and D. Autor. 2011. “Skills, Tasks, and Technologies: Implications for Employment and Earnings,” In O. Ashenfelter, and D. Card, eds., Handbook of Labor Economics, Volume 4. Amsterdam: Elsevier. Aedo, C., J. Hentschel, J. Luque, and M. Moreno. 2013. “From Occupations to Embedded Skills – A Cross-Country Comparison.” Policy Research Working Paper Series, No. 6560. Washington DC: World Bank. Aedo, C., and I. Walker. 2012. Skills for the 21st Century in Latin America and the Caribbean. Washington, DC: World Bank. Alderman, H., J. Behrman, D. Ross, and R. Sabot. 1996. “The Returns to Endogenous Human Capital in Pakistan’s Rural Wage Labour Market.” Oxford Bulletin of Economics and Statistics 58 (1): 29 –55. Alfonso, M., M. Bassi, and C. Borja. 2012. “Teaching Socio-emotional Skills in Latin American Schools: the Role of Non-Traditional Teachers.” Washington, DC: IDB. Almlund, M., A. Duckworth, J. Heckman, and T. Kautz. 2011. “Personality Psychology and Economics”. In E. Hanushek, ed., Handbook of the Economics of Education, Volume 4, 1–181. Amsterdam: North Holland. Anderson, N., D. S. Ones, H. K. Sinangil, and C. Viswesvaran. 2001. Handbook of Industrial, Work, and Organizational Psychology: Personnel Psychology, Volume 1. London: Sage Publications. *Andreasson, K. 2009. “Global education 20/20: What role for the private sector?” London: The Economist Intelligence Unit. Angrist, J., S. Dynarski, T. Kane, P. Pathak, and C. Walters. 2010. “Who Benefits from KIPP”. NBER Working Paper Series #15740, Cambridge, MA. *Arnhold, N., N. Kapil, I. Goldberg, M. Piatkowski, and J. Rutkowski. 2011. “Europe 2020 Poland: Fueling Growth and Competitiveness in Poland through Employment, Skills, and Innovation.” Working Paper 72815, World Bank, Washington, DC. Autor, D., F. Levy, and R. Murnane. 2003. “The Skill Content of Recent Technological Change: An Empirical Exploration.” Quarterly Journal of Economics 118 (4): 1279–1334. *Balcar, J. 2012. “The ‘Soft Five’ in Romania” The Romanian Economic Journal 15 (43): 23 –44. Banerji, A., W. Cunningham, A. Fiszbein, E. King, H. Patrinos, D. Robalino, and J. P . Tan. 2010. Stepping Up Skills for More Jobs and Greater Productivity. The World Bank, Washington, DC. Available at: http://www.skillsforemployment.org/wcmstest4/groups/skills/documents/skpcontent/mwdf/ mday/~edisp/fm11g_002234.pdf. ´ a, and J. Vargas. 2012. Unconnected: Skills, Education, and Employment in Bassi, M., M. Buso, S. Urzu Latin America. Inter-American Development Bank, Washington, DC. *Beneitone, P ´ , G. Siuf, and R. Wagenaar. 2007. Reflections ´ lez, M. Marty Maleta ., C. Esquetini, J. Gonza on and Outlook for Higher Education in Latin America: Final Report – Tuning Latin America Project 2004–2007. Tuning Project, Bilbao, Spain. Betcherman, G., M. Godfrey, S. Puerto, F. Rother, and A. Stavreska. 2007. “A Review of Interventions to Support Young Workers: Findings of the Youth Employment Inventory”. SP Discussion Paper No. 0715. The World Bank, Washington, DC. Bloom, B. S., M. D. Engelhart, E. J. Furst, W . H. Hill, and D. R. Krathwohl. 1956. Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. New York: David McKay Company. Cunningham and Villasen ˜ or 129 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 *Blom, A., and C. Hobbs. 2008. School and Work in the Eastern Caribbean: Does the Education System Adequately Prepare Youth for the Global Economy? The World Bank, Washington, DC. *Blom, A., and H. Saeki. 2011. “Employability and Skill Set of Newly Graduated Engineers in India.” Policy Research Working Paper 5640. The World Bank, Washington, DC. Bowles, S., and H. Gintis. 1976. Schooling in Capitalist America: Educational Reform and the Contradictions of Economic Life. Basic Books: New York. Bowles, S., H. Gintis, and M. Osborne. 2001. “Incentive-Enhancing Preferences: Personality, Behavior, and Earnings.” American Economic Review 91 (2): 155– 8. . 2001. “The Determinants of Earnings: A Behavioral Approach.” Journal of Economic Literature 39 (4): 1137– 76. Boys & Girls Clubs of America. 2004. Proven Results: A Compendium of Program Evaluations from Boys & Girls Clubs of America 1985-Present. Boys & Girls Clubs of America: Atlanta, GA. Available at: http://oms.bgca.net/Content/ProvenResultsEvaluationCompendium.pdf. Bronfenbrenner, U. 1979. The ecology of human development: Experiments by nature and design. Harvard University Press: Cambridge, MA. Brookhart, S. 2010. “How to Assess Higher-order Thinking Skills in Your Classroom.” ASCD: Alexandria, VA. Burnett, N., and S Jayaram. 2012. “Skills for employability in Africa and Asia.” Results for Development Institute: Washington, DC. *Burrus, J., T. Jackson, N. Xi, and J Steinberg. 2013. “Identifying the Most Important 21st Century Workforce Competencies: An Analysis of the Occupational Information Network (O*NET).” ETS Research Report ETS: Princeton, New Jersey. Carneiro, P., and J. Heckman. 2003. “Human Capital Policy”. In J. Heckman, A. Krueger, and B. Friedman (eds.), Inequality in America: What Role for Human Capital Policies? Cambridge, MA: MIT Press. Carneiro, P., C. Crawford, and A. Goodman. 2007. “The Impact of Early Cognitive and Noncognitive Skills on Later Outcomes.” CEE DP 92, Centre for the Economics of Education. London School of Economics. Collaborative for Academic, Social, and Emotional Learning. 2013. Effective Social and Emotional Learning Programs: Preschool and Elementary School Edition. Chicago, IL: CASEL. . 2015. Effective Social and Emotional Learning Programs: Middle and High School Edition. Chicago, IL: CASEL. *Casner-Lotto, J., and L Barrington. 2006. Are they really ready to work? Employers’ perspectives on the basic knowledge and applied skills of new entrants to the 21st century U.S. workforce. The Conference Board: New York, NY. Cawley, J., J. Heckman, and E. Vytlacil. 2001. “Three Observations on Wages and Measured Cognitive Ability.” Labour Economics 8:419–44. *Confederacy of British Industry. 2012. Learning to Grow: What Employers Need from Education and Skills. CBI: London, UK. *Centro de Investigacio´ n para el Desarrollo. 2014. “Professional Skills Survey: What employer look for - and do not find - in young professionals.” CIDAC: Mexico City, Mexico. *Close, S. 2012. Skills for Solomon Islands: Opening New Opportunities. The World Bank: Washington, DC. Cotton, K. 1993. “Developing Employability Skills.” Close-up #15, School Improvement Research Series, Northwest Regional Education Laboratory. 130 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Cunha, F., J. Heckman, L. Lochner, and D. Masterov. May 2005. “Interpreting the Evidence on Life Cycle Skills Formation.” NBER Working Paper No. 11331. Cunningham, W ´ , and D. Verner. 2008. Youth at Risk in Latin ., L. McGinnis, C. Tesliuc, R. Garcia-Verdu America and the Caribbean: Understanding the Causes, Realizing the Potential. The World Bank: Washington, DC. *Di Gropello, E. 2011. Indonesia Skill Report: Trends in Skill Demand, Gaps and Supply in Indonesia. The World Bank: Washington, DC. 2010. Skills for the Labor Market in the Philippines. The World Bank: Washington, DC. Duckworth, A., C. Peterson, M. Matthews, and D. Kelly. 2007. “Grit: Perseverance and Passion for Long Term Goals.” Journal of Personality and Social Psychology 92 (6): 1087–101. Fawcett, C., G. El Sawi, and C. Allison. 2014. TVET Models, Structures, and Policy Reform: Evidence from the Europe & Eurasia Region. USAID: Washington, DC. Finnie, R., and R. Meng. 2001. “Minorities, Cognitive Skills, and Incomes of Canadians”. Canadian Public Policy 28 (2): 257– 73. Gertler, P., J. Heckman, R. Pinto, A. Zanolini, C. Vermeersch, S. Walker, S. Chang-Lopez, and S. Grantham-McGregor. 2014. “Labor Market Returns to an Early Childhood Stimulation Intervention in Jamaica. Science 344 (6187): 998 –1001. Glewwe, Paul. (1996) “The Relevance of Standard Estimates of Rates of Return to Schooling for Education Policy: A Critical Assessment.” Journal of Development Economics 40 (2): 436 –82. Greenberg, M. T., R. P. Weissberg, M. U. O’Brien, J. E. Zins, L. Fredericks, H. Resnik, and M. J. Elias. 2003. “Enhancing school-based prevention and youth development through coordinated social, emotional, and academic learning.” American Psychologist 58 (6&7): 466–74. Goldberg, L. R. 1993. “The Structure of Phenotypic Personality Traits.” American Psychologist 48 (1): 26 –34. ´ lez, J., and R. Wagenar. 2008. Universities’ Contribution to the Bologna Process, 2nd edition. Gonza Tuning Project: Bilbao, Spain. Gottfredson, L. 1997. “Why g Matters - The Complexity of Everyday Life.” Intelligence 24 (1): 79 –132. Guerra, N., K. Modecki, and W. Cunningham. 2014. “Developing Social-Emotional Skills for the Labor Market: The PRACTICE Model.” Policy Research Working Paper #7123. World Bank, Washington, DC. *Hamid, I. S. Imaizumi, and A. Blom. 2011. Sindh Employer’s Survey 2010. South Asia Human Development Sector, Report No. 34. World Bank, Washington DC. Handel, M. 2012. “Trends in Job Skill Demands in OECD Countries”. OECD Social, Employment and Migration Working Papers, No. 143, OECD Publishing: Paris. Hanushek, E., and L. Woessman. 2008. “The Role of Cognitive Skills in Economic Development.” Journal of Economic Literature 46 (3): 607–68. Hanushek, E., and L. Zhang. 2009. “Quality-Consistent Estimates of International Schooling and skill Gradients.” Journal of Human Capital 3 (2): 107– 43. Heckman, J. J. 2008. Schools, Skills and Synapses. NBER Working Paper No. 14064. ´ a. 2006. “The Effects of Cognitive and Noncognitive Abilities on Heckman, J., J. Stixrud, and S. Urzu Labor Market Outcomes and Social Behavior.” Journal of Labor Economics 24 (3): 411–82. Heckman, J., and Y. Rubenstein. 2001. “The Importance of Noncognitive Skills: Lessons from the GED Testing Program.” American Economic Review 91 (2): 145 –9. Heckman, J., and T. Kautz. 2012. Hard Evidence on Soft Skills. NBER Working Paper No. 18581. Herrnstein, R., and C. Murray. 1994. The Bell Curve: Intelligence and Class Structure in American Life. Free Press: New York. Cunningham and Villasen ˜ or 131 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Ibarraran, P., L. Ripani, B. Taboada, J. Miguel Villa, and B. Garcia. 2012. “Life Skills, Employability and Training for Disadvantaged Youth: Evidence from a Randomized Evaluation Design.” IDB Working Paper Series No. IDB-WP-342. IDB: Washington DC. International Finance Corporation. 2011. Education for Employment: Realizing Arab Youth Potential. IFC: Washington DC. Kagitcibasi, C., D. Sunar, and S. Berkman. 1988. Comprehensive Preschool Education Project. Ottowa: IDRC. Kern, M., A. Duckworth, S. Urzu ´ a, R. Loeber, M. Stouthamer- Loeber, and D. Lynam. 2013. ”Do as You’re Told! Facets of Agreeableness and Early Adult Outcomes for Inner-City Boys.” Journal of Research in Personality 47 (6): 795–9. King, F. J., L. Goodson, and F. Rohani. 1998. “Higher Order Thinking Skills.” Center for Advancement of Learning and Assessment. Florida State University. Author: Florida. Available at: www.cala.fsu.ed. T. Kniesner, and B. ter Weel, eds. 2008. “Special Issue on Noncognitive Skills and Their Development.” Journal of Human Resources 43 (4): 729 –1059. Knight, J., and R. Sabot. 1990. Education, Productivity, and Inequality: The East African Natural Experiment. Oxford: Oxford University Press. Lazear, E. 2003. “Teacher Incentives.” Swedish Economic Policy Review 10 (3): 179 –214. Manpower. 2010. Supply/Demand: 2010 Talent Shortage Survey Results. Manpower Group: Milwaukee, WI. *Martin, R., F. Villeneuve-Smith, L. Marshall, and E. McKenzie. 2008. Employability skills examined: Shedding light on the views of stakeholders and employers. Learning Skills Network: London. McIntosh, S., and A. Vignoles. 2001. “Measuring and Assessing the Impact of Basic Skills on Labour Market Outcomes.” Oxford Economic Papers 53 (3): 435– 81. Moll, P. 1998. “Primary Schooling, Cognitive Skills, and Wages in South Africa.” Economica 65 (258): 263 –84. Montenegro, C., and H. Patrinos. 2015. “Comparable Estimates of Returns to Schooling Around the World”. Policy Research Working Paper #7020. World Bank, Washington, DC. *Mourshed, M., D. Farrell, and D. Barton. 2012. Education to Employment: Designing a System that Works. McKinsey Center for Government. McKinsey & Company: Washington, DC. Mulligan, C. 1999. “Galton versus the Human Capital Approach to Inheritance.” Journal of Political Economy 107 (6): S184–224. Mueller, G., and E. J. S. Plug. 2006. “Estimating the Effect of Personality on Male and Female Earnings”. Industrial and Labor Relations Review 601: 3–22. Murnane, R., J. Willett, Y. Duhaldeborde, and J. Taylor. 2000. “How Important Are the Cognitive Skills of Teenagers in Predicting Subsequent Earnings.” Journal of Policy Analysis and Management 19 (4): 547 –68. Neisser, U., G. Boodoo, T. Bouchard, A. Wade Boykin, N. Brody, S. Ceci, D. Halpern, J. Loehlin, R. Perloff, and R. Sternberg, and S. Urbina. 1996. “Intelligence: Knowns and Unknowns.” American Psychologist 51 (2): 77 –101. Organisation for Economic Co-operation and Development. 2014. Skills Beyond School: Synthesis Report. OECD Reviews of Vocational Education and Training. OECD Publishing: Paris. *Ogier, T. 2009. Skills to compete: Post-secondary education and business sustainability in Latin America. Economist Intelligence Unit: London, UK. Osterman, P . 2001. “Employers in the Low-Wage/Low-Skilled Labor Market.” In R. Kazis, and M. Miller (eds.), Low-wage Workers in the New Economy. Washington, DC: Urban Institute Press. 132 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Pina, P., T. Kotin, V. Hausman, and E. Macharia. 2012. Skills for employability: the informal economy: Results for Development Institute. Roberts, B. 1997. “Plaster or Plasticity: Are Work Experiences Associated with Personality Change in Women?” Journal of Psychology 65:205 –32. Roberts, B., A. Caspi, and T. Moffitt. 2003. “Work Experiences and Personality Development in Young Adulthood.” Journal of Personality and Social Psychology 84 (3): 582 –93. Robins, R., A. Caspi, and T. Moffitt. 2002. “It’s not just who you’re with, it’s who you are: Personality and relationship experiences across multiple relationships.” Journal of Personality. 70 (6): 925– 64. Robins, Richard, R. Chris Fraley, Brent Roberts, and Kali Trzesniewski. 2001. “A Longitudinal Study of Personality Change in Young Adulthood.” Journal of Personality 69 (4): 617– 40. *Rutkowski, J. 2010. “Demand for Skills in FYR of Macedonia.” Technical Note. World Bank, Washington, DC. Salgado, J., C. Viswesvaran, and D.S. Ones. 2001. “Predictors Used for Personnel Selection: An Overview of Constructs, Methods, and Techniques”. In N. Anderson, D. S. Ones, H.K. Sinangil, and C. Viswesvaran (eds.), Handbook of Industrial, Work, and Organizational Psychology: Personnel Psychology, Volume 1. London: Sage Publications. Schweinhart, L. J., J. Montie, Z. Xiang, W. Steven, C. R. Barnett, Belfield, and M. Nores. 2005. Lifetime Effects: The High/Scope Perry Preschool Study Through Age 40. High Scopes Press: Ypsilanti, MI. *Shivpuri, S., and B. Kim. 2004. “Do Employers and Colleges See Eye-to-Eye?” NACE Journal 65 (1): 37 –44. *Sondergaard, L., and M. Murthi. 2012. Skills, Not Just Diplomas: Managing Education for Results in Eastern Europe. Directions in Development Series. World Bank, Washington, DC. Stormer, S., and R. Fahr. 2010. Individual Determinants of Work Attendance: Evidence on the Role of Personality. IZA Discussion Paper No. 4927. Tan, J. P., and Y. J. Joy Nam. 2012. “Pre-employment Technical and Vocational Education and Training: Fostering Relevance, Effectiveness, and Efficiency”. In R. Almeida, J. Behrman, and D. Robalino ,(eds.), The Right Skills for the Job? World Bank: Washington, DC. Tierney, J., and J. Baldwin Grossman. 2000. Making a Difference: An Impact Study of Big Brothers Big Sisters. Public/Private Ventures: Philadelphia, PA. Available at: http://www.bbbs.org/site/ c.9iILI3NGKhK6F/b.5961035/k.A153/Big_impact8212proven_results.htm. *Tonga Chamber of Commerce & Industry. 2010. TCCI Annual Business Survey. Tonga Chamber of Commerce and Industry: Nuku-alofa, Tonga. *UK Commission for Knowledge and Skills. 2012. “UK Commission’s Employer Skills Survey 2011: UK Results.” Evidence Report 45. London, UK: UKCES. *Vasiliev, K., A. Lukiyanova, D. Chugunov, I. Maltseva, I. Shulga, J. J. Rutkowski, P. Cahu, M. P. Marie, P. Travkin;, S. Roshchin, and S. Nellemann. 2013. “Developing Skills for Innovative Growth in the Russian Federation”. World Bank, Washington, DC. Available at: http://documents.worldbank.org/ curated/en/2013/06/18004192/developing-skills-innovative-growth-russian-federation. ´a, G. Cruces, and J. Amendolaggine. 2014. “Youth and Employment Program, Vezza, E., B. Garcı Ministry of Labor, Dominican Republic: Impact Evaluation Study of the 2008–2009 Cohort.” Unpublished. World Bank, Washington, DC. ˜ or, P Villasen . 2013. “Analysis of the Skills Development Sector in Mexico.” Human Development Department, World Bank, Washington, DC. Mimeo. *World Bank. 2008. “Vietnam: Higher Education for Skills and Growth”. Human Development Department, EAP. The World Bank, Washington DC. Cunningham and Villasen ˜ or 133 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 . 2011. “Strengthening Skills and Employability in Peru.” Report No. 61699-PE. The World Bank, Washington DC. . 2012. “Lebanon, Good Jobs Needed: The Role of Macro, Investment, Education, Labor and Social Protection Policies (“Miles”)”. Human Development Department, MNA. World Bank, Washington DC. . 2014. “Botswana Labor Market Signals on Demand for Skills”. World Bank, Washington, DC. . 2015. “World Development Report 2016: Internet for Development.” World Bank, Washington DC. . Forthcoming. “Skills for a Modern Ukraine: Diagnostic and Policy Options.” World Bank, Washington DC. *Zemsky, R. 1997. “Skills and the Economy: An Employer Context for Understanding the School-to- Work Transition”. In A. M. Lesgold, M. J. Feuer, and A. Black, (eds.), Transitions in Work and Learning: Implications for Assessment, 34 –61. National Academy Press: Washington, DC. 134 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/102/2461930/Employer-Voices-Employer-Demands-and-Implications by International Monetary Fund user on 06 October 2017 Revisiting the “Cash versus Food” Debate: New Evidence for an Old Puzzle? Ugo Gentilini The longstanding “cash versus food” debate has received renewed attention in both research and practice. This paper reviews key issues shaping the debate and presents new evidence from randomized and quasi-experimental evaluations that deliberately compare cash and in-kind food transfers in ten developing counties. Findings show that relative effectiveness cannot be generalized: although some differences emerge in terms of food consumption and dietary diversity, average impacts tend to depend on context, specific objectives, their mea- surement, and program design. Costs for cash transfers and vouchers tend to be signifi- cantly lower relative to in-kind food. Yet the consistency and robustness of methods for efficiency analyses varies greatly. Social protection, Safety nets, Food security, Political economy, Cash transfers, In-kind transfers, Food aid, Vouchers, Food stamps, Impact evaluations, Cost-effectiveness. JEL codes: D610, H530, O120, O130, Q180 Introduction There is little doubt that cash transfers are amongst the most rigorously-evaluated interventions in development.1 The basic question that this paper addresses, however, is not whether cash transfers work in general, but whether and why they do so relative to in-kind food assistance. Indeed, the cash versus food debate is among the most longstanding, controversial, and polarizing social protection quan- daries. Consider the following quotes: [T]he big reason poor people are poor is because they don’t have enough money, and it shouldn’t come as a huge surprise that giving them money is a great way to reduce that problem – considerably more cost-effectively than paternalism [i.e. vouchers and in-kind transfers]. So let’s abandon The World Bank Research Observer # The Author 2015. Published by Oxford University Press on behalf of the International Bank for Reconstruction and Development / THE WORLD BANK. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com doi:10.1093/wbro/lkv012 Advance Access publication September 22, 2015 31:135–167 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 the huge welfare bureaucracy and just give money to those we should help out. (Kenny 2013) [T]housands of economics professors across the globe persuade millions of undergraduates that granting the poor distribution of benefits in-kind is less “efficient” than simply transferring to them cash (. . .). The econo- mist’s traditional, normative dictum on benefits in-kind may be analytical- ly elegant (. . .) but practically dead wrong. (Reinhardt 2013) These excerpts seem to posit two irreconcilable perspectives. But is there some truth in both? What does the cross-country evidence and experience tell us about trans- fers’ relative performance? Until recently, a key limiting factor in informing the debate has been the paucity of robust comparative evidence.2 As a result, the discus- sion on transfer selection has been largely based on inference—that is, based on findings from programs implemented in diverse contexts, relying on different evalu- ation methods, or designed for differing purposes. Such extrapolation and compari- son of findings could, at best, be suggestive of relative effectiveness. This paper reviews key findings emerging from a new generation of robust impact evaluations in ten developing countries, namely Bangladesh, Cambodia, the Democratic Republic of Congo, Ecuador, Ethiopia, Mexico, Niger, Sri Lanka, Uganda, and Yemen. These studies compare cash and food transfers under the same circumstances through randomized controlled trials (RCTs), quasi-experimen- tal methods, or regression analysis. The paper, which frames and discusses emerg- ing results within the broader issues that shape transfer selection debates, argues that the debate involves a range of factors that makes it more complex than often assumed. Moreover, it shows that, in terms of effectiveness, there seems to be no sys- tematic dominance of one transfer over the other, and that a number of information gaps persist. Overall, the paper argues that transfer selection may deserve further at- tention on a par with other program design features such as choices around condi- tionality and targeting methods. The analysis is conducted within certain parameters. First, the article examines the transfer debate within the universe of safety net programs, including interven- tions such as conditional cash transfers, school feeding, public works, or social pen- sions. In those programs, the size of transfers is generally devised to enable access to food and meet some essential needs. According to survey data from 69 countries, the average size of safety net transfers is about 23 percent of the poor’s income or consumption (World Bank 2014). Therefore, the vast body of literature on “cash grants”—which involve relatively large sums of money to jump-start entrepreneur- ship and investment—go beyond the scope of this paper. Similarly, in-kind transfers may encompass a vast array of items such as shelter, agricultural inputs, and subsi- dized health care. Yet we only consider a subset of those transfers, namely food com- modities; as such, we use the terms “in-kind” and “food” interchangeably. Also, we 136 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 regard food as a tout-court transfer, while de facto food rations often include a bundle of commodities of various monetary and nutritional values. The reminder of the paper is organized as follows: the next section sets out a framework examining key areas that influence transfer selection debates. The subse- quent section describes the compilation of evaluations and sets out their results. The next section provides a discussion of the implications from the analysis, while the last section concludes. Anatomy of the Debate The issue of transfer selection is relevant under a range of poverty-reduction debates, but it is particularly compelling for issues around food security. The role of social protection in food security is being increasingly examined, including through the lens of its three core pillars: how social protection can enhance food production and agricultural productivity (Tirivayi et al. 2013); how transfers can provide access to food (CFS 2012); and how social protection can, as Alderman (2014) put it, be more nutrition-sensitive. A fourth food security pillar, risk or stability, under- pins those three dimensions and it is central to the connections between social pro- tection and resilience (Davies et al. 2013). The interactions between social protection transfers and the abovementioned food security components are mediat- ed by a range of other factors (e.g., food security being only one factor in the nutri- tion equation), and occur under different timeframes (e.g., the likely timeframe for reducing malnutrition is measured more in terms of a generation, rather than of a few years). Three forms of transfers are considered in this paper, namely cash, food, and vouchers. Cash transfers provide people with money, while food transfers (or food aid) include the distribution of commodities. Food transfers can be procured inter- nationally (imported food) or local-sourced in the same country where the program is implemented. Vouchers are also known as stamps or near-cash transfers and can be used in predetermined locations, including selected shops, supermarkets, retail stores and fairs. Vouchers take two forms: on one hand, “value-based” vouchers provide access to commodities for a given monetary amount; on the other hand, vouchers can be “commodity-based”, or tied to a pre-defined quantity of given foods. Therefore, vouchers are a hybrid form of transfer that display features of both cash (value-based vouchers allow for some level of choice, although this is limited to the commodities available in the chosen location) and in-kind food (the imple- mentation of commodity-based vouchers can closely resemble that of public food distributions systems).3 Cash transfers can be considered the modality that provides beneficiaries (consumers) with largest choice while at the same time stimulating local markets. Also, vouchers can equally bolster local (food) markets, although the Gentilini 137 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 choice they provide is more restricted or none.4 Food transfers provide no choice, but can stimulate markets if procured locally, although at a different level than vouchers: indeed, the former tend to entail the engagement of a more limited number of upstream, larger producers, while the latter would directly involve a larger number of downstream, smaller actors.5 From this perspective, vouchers are more “local” in nature. Against this background, the reminder of the section briefly discusses a set of issues in the spheres of theory, political economy, and technical matters that will be useful when discussing the evaluations in the following section. Theory and Political Economy The microeconomic foundations of the debate tend to draw from neoclassical theory as laid out by Southworth (1945). The transfer debate received considerable empirical attention in the 1980s and mid-1990s, including the production of a vast body of literature examining the United States’ Supplemental Nutrition Assistance Program (SNAP; Basu 1996; Faminow 1995; Fraker et al. 1995; Devaney and Moffitt 1991; Levedahl 1991; Coate 1989; Blackorby and Donaldson 1988).6,7 Interestingly, an empirical regularity in the literature seems to be the presence of a “cash-out puzzle” rejecting the neoclassical hypothesis—that is, “inframarginal” cash and food transfers should hypothetically have an equal effect on food con- sumption.8 Indeed, “. . . virtually every study finds food stamps increase household nutrient availability at 2 to 10 times the rate of a like value of cash income,” (Barrett 2002). While there seems to be an overall difference in outcomes between cash and in-kind transfers, there remains great dispute as to why such difference persists.9 The transfer debate is not new. Historians found accounts of the quandary in 17th Century England, and even in ancient Edessa around 124 BC (Lindert 2005; Garnsey 1988). More recently, from the late-1990s and mid-2000s, the emergence of cash transfers on the global stage was in part defined by mounting criticism over the international food aid system, particularly in terms of transoceanic shipments and local monetization of commodities (Barrett and Maxwell 2006, 2005; Clay 2006; OECD 2005). Since the mid-2000s, the growing application of technology to the delivery has further propelled interest in cash as an efficient way of providing as- sistance (Vincent and Cull 2011). While the global food aid model has evolved dramatically toward more targeted, technology-supported, locally-procured and nutritionally-sound commodities, some lingering perceptions around food transfers seem hard to fully dispel.10 In part, the mixed performance of large-scale national public food distribution systems may have contributed to a view of food as an intrinsically second-best modality (GoI 2005). A related political dimension is that an appearance of a linkage to food may 138 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 be attractive not only to taxpayers, but also with food producers and possible lobbies. Also, in some contexts food transfers may still evoke the image of long truck convoys pouring onto famine-stricken areas, or of forces behind food distributions driven by commercial and geopolitical interests. More subtly, in-kind food may be somewhat more directly associated with the inability of countries or households to feed themselves, hence swiftly juxtaposing the issue of food provision with that of personal dignity. Instead, cash transfers tend to conjure just the opposite—images of freedom and independence—even if they are supposed to play the exact same role as food transfers (e.g., providing access to food). The latter observation raises perhaps the most fundamental philosophical dilemma in the debate, namely the trade-off between providing choice and ensuring (as much as possible) a positive externality. On one hand, a recurrent view of in- kind transfers considers them as inherently paternalistic. From such a standpoint, the arguments in favor of in-kind transfers revolve around an “over-provision” of goods (i.e., extramarginal transfers). This involves informational, principal-agent, or behavioral arguments that often provide the foundations for much of the debate around conditionalities (Fiszbein and Schady 2009). In broad terms, this suggests that the expected externalities of an in-kind transfer would be desirable when there is a private under-investment—say, in food consumption or nutrition—below an optimal social (or even private) level. On the other hand, the provision of in-kind transfers represents a vehicle to influence behaviors (e.g., consumption patterns). To some extent, paternalism can less judgmentally be considered an approach to address externalities in health, or the fact that parents make decisions for children (thus their welfare may be discounted), or the parents are misinformed about nutri- tional benefits (Das et al. 2005). Under this view, changing consumption via prices or modified preferences is a motivation for in-kind programs.11 Shifting gender control is a subset of this motivation. On the other hand, in-kind transfers do not allow people to maximize their utility through choice, that is, the magic of cash lies in its fungibility. Also, cash helps shift the balance of power. As Devarajan (2013) states, “. . . cash transfers have the potential to shift not just poverty-reducing poli- cies but also the balance of power between government and its citizens, in favor of the latter.” In other words, the mainstream argument in favor of cash is straightforward: cash provides choice and transfers power, hence making them “redefinitional”.12 This is an undisputable advantage, though it needs some qualifications: can the provision of choice be considered an objective per se? In that case, any cash transfer can be considered “effective” and would not require evaluations. Or would the same effect be achieved through in-kind transfers? An inframarginal in-kind transfer has the same income-transfer effect. As such it can free up disposable income. And can choice be exercised effectively when there are limited supplies of goods or they are Gentilini 139 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 unaffordable? This is intimately linked to technical issues around market function- ality, which will be discussed in the next section. Finally, an enlarged definition of “choice” may include the ability to choose what to receive in the first place. If the poor know best what they prefer, then they should be provided with choice on what to get (e.g., cash or food), not only on how to use a given transfer (i.e., cash). For example, in 2006 a combined cash and food transfer program was implemented in rural Malawi. The follow-up evaluation explored participants’ preferences over one or both transfers and found that “. . . most beneficiaries were very satisfied with re- ceiving both food and cash,” (Devereux 2008). Yet the redesign of the project in 2007 included only cash transfers. The point here is the need for a broader notion of “paternalism”—that is, one that is not associated with a given transfer modality, but instead based on the extent to which interventions involve and engage benefi- ciaries as key stakeholders throughout the life of the program. Ultimately, the tension between fungibility and paternalism seems ingrained in political economy and philosophical debates that go beyond the choice of transfers (Pritchett 2012, 2005; Lopez- Rodriguez 2011; Currie and Gahvari 2008). These debates nevertheless affect the selection of transfers, including through factors such as the contextual definition of what poor people “deserve”, the preferences ex- pressed in a given polity and social contract, public perceptions as rooted in histori- cal precedents, the influence of particular constituencies, and to some extent even the symbolic value carried by food in societies. Factors Affecting Choices The available transfer literature can be grouped, in addition to the earlier SNAP studies, into five broad “empirical buckets”. First, there is high-quality research con- ducted on individual cash-based programs (Evans et al. 2014; Baird et al. 2013; de Brauw et al. 2012; Barham and Maluccio 2009; Duflo 2003). Second, there are comprehensive reviews focusing on cash transfers, often in relation to sectors, themes, and regions (Alderman 2014; Davis 2013; Garcia and Moore 2012; Bailey and Harvey 2011; DFID 2011; Fiszbein and Schady 2009; Adato and Bassett 2008). Those two threads of literature are mirrored on the food front: in-kind food transfers have been carefully examined empirically (Kazianga et al. 2014; Dercon et al. 2012; Adelman et al. 2008a, 2008b; Gilligan and Hoddinott 2007; Yamano et al. 2005; Del Ninno and Dorosh 2003; Quisumbing 2003), as well as being subject of cross-country reviews and meta-analyses (Alderman and Bundy 2012; Webb et al. 2011; Del Ninno et al. 2007; Rogers and Coates 2002). Finally, publica- tions have documented and summarized evidence emerging from all those four sets of studies (Lentz al. 2013; Gentilini 2007). While recalling the general caveats on comparability, those studies have generally concluded that transfer appropriateness is context-specific and hinges on multiple 140 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 factors. These include program objectives, the functioning of markets, administra- tive capacity, seasonality, security, intra-household preferences, and community dy- namics. These factors largely shape the performance of transfers across time and space. Turning “needs” into “effective demand” is a key rationale for cash transfers. Yet this might be challenging in the presence of weakly integrated and competitive markets. In those contexts, price transmissions would not necessarily signal relative scarcities, and localized cash injections may result in price spikes leaving consumers or net buyers worse off.13 Ensuring that markets would respond to an increase in ef- fective demand is key to ensure that purchasing power is maintained and local mul- tipliers enabled. From this perspective, a basic level of market functioning is a prerequisite for the effective provision of cash transfers.14 The discussion on market analysis has important practical implications for program design, implementation, and efficiency. In some cases, prices may be par- ticularly volatile, with a certain degree of unpredictability in future trajectories. These could turn a program that was efficient in the design stage into a cost- inefficient program during implementation. Indeed, keeping purchasing power constant in the wake of sharp and protracted price increases may escalate costs due to extensive use of contingency funds, such as shown in Zambia (Harvey and Savage 2006). Price dynamics also have an influence on beneficiaries’ preferences, which tend to lean towards in-kind transfers when prices are higher.15 This was clearly docu- mented in the case of Ethiopia during the sudden increase in wheat prices in 2008 (Sabates-Wheeler and Devereux 2012). While it is difficult to generalize people’s preferences for a certain transfer modality, some general patterns can be discerned. As a natural consequence of price fluctuations, the preference for cash, vouchers, or in-kind food aid can vary by season. Yet gender also matters, and it is a generally observed pattern that women spend resources differently from men (Doss 2013). In a number of societies, however, women tend to prefer food, which they are more likely to control, while men may prefer cash transfers. However, anecdotal evidence shows that beyond the fact of who receives the transfer, the process of intrahouse- hold decision making also counts (e.g., men and women deciding in concert how to use household resources, even in contexts where women may be constrained in reaching markets, or face risks to access them, such as in refugee camps). As such, intra-household and gender-specific preferences are more complex and dynamic than often assumed. For instance, very few studies support the unitary model of household behavior, and even fewer, if any, are experimenting with the differential impacts (e.g., on nutrition) of cash versus food transfers as provided to men and women (Braido et al. 2012). New research is also shedding light on the intra-community effects of transfers, some of which are positive and empowering, while others generate undesired Gentilini 141 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 consequences in terms of social relations. For example, in Zimbabwe it was observed that, unlike food, cash transfers were not shared within the community, thus hin- dering informal mutual support and risk management mechanisms among members (MacAuslan and Riemenschneider 2011). Returning to the issue of price trends and real value, predictable price seasonality has been recognized as one of the key drivers of food insecurity in Africa. For example, in Malawi analysis on price trends over 20 years show mean inter-sea- sonal price fluctuations in the order of 60 percent (Ellis and Manda 2012). There are various emerging options to deal with marked seasonal price fluctuations; one way is index-linking cash transfers to food prices, such as in the Dowa Emergency Cash Transfer program in Malawi. In that context, the approach “. . . . protected household food security until prices started falling just before the next harvest, [al- though] it required a degree of administrative and budgetary flexibility that is in- conceivable for most governments and donor agencies,” (Devereux 2012). Another option is to deliver transfers half in cash and half in food. For example, in Swaziland a program was designed in 2007/08 where people were given a half ration of food (maize, beans, and oil) plus the equivalent in cash, each month for 6 months from November 2007 until the harvest of April 2008 (Devereux and Jere 2008). A third alternative is the distribution of cash and food transfers by season, with food provid- ed in the lean season and cash immediately after the harvest. That approach, for example, is implemented under the Productive Safety Net Program (PSNP) in Ethiopia. This is likely to have a number of advantages, although it may also entail considerable analytical planning and logistical coordination among modalities. More generally, cost-efficiency is one of the key factors that favor adopting cash- based transfers. Studies assessing relative costs showed that they tend to be higher for food transfers than for cash transfers and vouchers. This is not surprising, given the logistics that food programs normally entail (e.g., transport, storage, handling, etc.). Yet cost assessments seem not to adopt the same standard of rigor as impact assessments. Indeed, there are limited comprehensive protocols on cost analysis, with comparisons often being hampered by the lack of consistency and clarity around data quality, estimation strategies, and cost structures.16 For example, a review of 27 programs showed that administrative costs for cash transfers and vouchers vary from 3 to 37 percent of total cost (Meyer 2007). Clearly, such a range may have as much to do with different formulas and accounting methods as real differences in cost. Furthermore, a frequent practice in the literature is to convert the cost of food commodities based on market prices at the retail level (upon which the size of cash transfers are often determined). This may significantly overes- timate food costs, as they ignore the economies of scale from procuring large quan- tities of commodities at the producer-level. Preliminary calculations for a sample of countries have shown that those cost differentials could even offset the larger logis- tics costs of food programs, making in-kind transfers surprisingly more efficient 142 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 than cash transfer programs. For example, Creti (2011) estimated that in the West Bank and Gaza, the costs for vouchers would be from 2 to 2.5 times higher than for in-kind food commodities of similar energy and nutritional content. Similar results emerged for a voucher program in Kenya, but not in Bolivia (ACF 2012). Impact Evaluations This section presents new evidence emerging from a set of quantitative evaluations, most of which are taken from randomized and quasi-experimental studies. The next two subsections describe those studies and their results. Descriptive Features We examine 11 impact evaluations in 10 countries published over the period 2006 – 2013.17 These include the complete gamut of comparative experimental and quasi-experimental trials that, to our knowledge at the time of writing this paper, have been conducted on the matter in developing countries.18 The studies were fielded in various contexts and include responses to sudden natural disasters (Sri Lanka), slow onset crises (Niger) and man-made emergencies (Congo, Ecuador); others are implemented as part of long-term, institutionalized social protection systems (Bangladesh, Cambodia, Ethiopia and Mexico), or envisage developmental interventions in fragile contexts and regions (Yemen, Uganda). Public works were included in 3 cases, while conditional and unconditional pro- grams were present in 4 and 7 countries, respectively (4 countries had combined interventions). Some notable differences between countries emerge. For example, the size of monthly transfers range from $5 in Cambodia to $50 in Niger; similarly, the share of transfers in household expenditures varies from 2.5 percent (Cambodia) to 30 percent (Bangladesh). The composition of food rations also vary, ranging from one commodity (rice) in Cambodia to up to 11 commodities in Mexico. Programs could have durations as different as a one-off 3 months intervention in Sri Lanka to a full 12 months (for 4 consecutive years) in Bangladesh. Distribution frequency and delivery mechanisms also differ, while evaluation methods include RCTs, quasi- experimental techniques (difference-in-difference, propensity score matching), regressions, and combinations thereof. In particular, the Programa de Apoyo Alimentario (PAL) in Mexico provides un- conditional food transfers to most of the target households, while at the time of data collection (2004 –2005), cash transfers were provided to approximately 5 percent of beneficiaries living in villages so isolated that PROGRESA could not be imple- mented there. The transfer was of 150 pesos monthly, equivalent to $13, which is Gentilini 143 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 equal to the cost of the food basket in local markets. No adjustments to transfer amounts were made for family size or composition. The transfer is delivered bi- monthly through stores in the case of food, and biometric debit cards for cash trans- fers. The PAL experiment was carried out in 208 villages that were randomized into four treatment groups: an in-kind transfer plus educational classes (the standard PAL treatment); in-kind transfer without classes; a pure cash transfer of 150 pesos per month plus classes; and finally a control group. Cunha (2014) uses the experi- mental PAL data to estimate the impacts between the cash and food transfers using a difference-in-difference estimation strategy. Moreover, we completed Cunha’s study by examining the previous evaluation by Skoufias et al. (2008). Using the same estimation method and sample size, their analysis offered additional insights for poverty and labor market participation. In Niger, a large-scale public works program was implemented in 2011 over a six-month period, from April to September. The program included public works and unconditional transfers implemented in sequence for the same group of beneficia- ries. In some villages, registered households received cash, while in others they re- ceived food. In cash-receiving villages, each beneficiary was provided with 1,000 FCFA (or about $2) per day for a maximum of 25,000 FCFA per month. Food pay- ments consisted of commodities similar to those typically consumed in the region (see table 2 for a full list of food commodities provided in the ten countries). In this context, Hoddinott et al. (2014) assessed the relative impacts of cash versus food transfers through a randomized design using a single-difference evaluation ap- proach (no baseline data was available). The results are differentiated by pre-harvest (or “hunger”) season in July and the post-harvest season in October. The PSNP in Ethiopia is a widely studied flagship safety net program. Launched in 2005, the PSNP provides predictable transfers to about 7.5 million food insecure people for a period of 6 months each year. Transfers are delivered through two com- ponents: public works that provide temporary employment at a monthly wage of $16 (raised several times as a result of food prices), and an unconditional arm of direct support for households with limited working capacity. Sabates-Wheeler and Devereux (2012) compared the impacts of those cash and food interventions on food-gap and income using a single difference estimation strategy. In Uganda, a program was implemented by the World Food Programme (WFP) in the fragile Karamoja region to support early childhood development. The program is a conditional food or cash transfer, where the former were provided as take-home rations, while the latter included a cash transfer of about $12 per child for each 6-week cycle (Gilligan and Roy 2013). Food transfers were distributed through the WFP’s general food distribution program, and cash transfers were added electroni- cally to a card and retrieved from mobile money agent. Gilligan and Roy (2013) esti- mate the impacts of the program on child cognitive and non-cognitive development. 144 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 In Ecuador, a three-modality transfer program (featuring cash, food, and vouch- ers) was designed to address the food security and nutrition needs of Colombian ref- ugees and to support their integration into Ecuadorian communities. The value of the monthly transfers was standardized across all treatment arms at $40 per month per household. The cash was transferred monthly onto preprogrammed ATM cards. Cash transfer households were able to retrieve the cash in bundles of $10. Vouchers were given in denominations of $20, redeemable for a list of nutritionally approved foods at identified supermarkets. Hidrobo et al. (2014) use the experimental design of this program to assess the impacts of transfers using an Analysis of covariance (ANCOV A) approach. Aker (2013) evaluates the impacts of unconditional cash transfers and vouchers in the context of a complex emergency in the Democratic Republic of Congo. The program was part of the humanitarian response to internally displaced persons living in informal camps. Cash transfers and vouchers totaling $130 were provided over a seven-month period in three installments.19 Cash was directly deposited into an interest-free account at the office of a local cooperative located in the regional market center. Vouchers were used in fairs, including 122 vendors and 4 primary schools in the area. Impacts were estimated using a difference-in-difference with fixed effects. In Yemen, an emergency intervention to address seasonal food shocks was carried out in 136-village clusters that were randomly assigned to receive either a food or a cash transfer. Over the course of seven months, households received 3 food baskets. Over the same time period, households in the cash treatment group received 3 cash transfers equal to the local value of the food basket (about $50). Cash transfers were distributed through ID cards from the Yemen Post and Postal Savings Corporation, and food transfers were delivered through the Ministry of Education. Schwab (2013) compared the impacts of food and cash transfers using single difference, difference-in-difference, and ANCOVA estimation strategies. A pilot intervention was implemented in Cambodia as an extension to an existing food for education program. Food was provided through two modalities: early morning meals and take-home rations. The expansion included a cash scholarship to test the relative efficacy of the take-home rations and cash scholarships. Both transfers were conditional on a minimum attendance rate of 80 percent. The cash transfer was $60 per year and the take-home ration consisted of 10 kg of unfortified rice per month. Barker et al. (2014) estimated the comparative impacts using a difference-in-difference approach. In post-tsunami Sri Lanka, an emergency cash transfer program was designed to support the needs of affected villages as part of the broader food-based operation ran by WFP . The cash transfer pilot disbursed an average amount of US$2.44 per week. Food rations were specified for weekly amounts but in most cases transfers were given in bulk. Cash transfers were distributed by the Samurdhi Authority in Gentilini 145 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 two-week installments through bank accounts. Sharma (2006) compared the rela- tive impacts using a difference-in-difference estimation strategy. In Bangladesh, Ahmed et al. (2010) examined the efficacy of four different pro- grams utilizing food-only, cash-only, and combined cash and food transfers. For the purpose of this paper, we examine only the two “pure” cash and food transfer inter- ventions, namely the food-based Income Generating Vulnerable Group Development (IGVGD) and the cash-based Rural Maintenance Program (RMP). The IGVGD program exclusively targeted poor women, who receive a monthly food ration over a period of 24 months. The RMP also targeted divorced, widowed, separated, or aban- doned women with 4 years of employment to maintain rural roads. The RMP partici- pants received a monthly net salary of $30. Impacts of both programs were estimated using propensity score matching techniques. The specific features of the interventions are summarized in table 1, and describe the program types, modalities, transfer size, transfer as a percentage of household pre-program expenditures, frequency of distribution, household size, duration, delivery mechanisms, sample size, and evaluation method. Impacts The compilation of evaluation covers a number of dimensions. Table 2 lists 15 of them, including their availability across the examined countries. In most cases, cash interventions were conducted as a smaller-scale addition to preexisting large food-based programs, or as part of crisis responses. It is somewhat natural, there- fore, that the interventions tend to be generally food security-oriented. Indeed, about half of the indicators are food and nutrition-related, while indicators on poverty, income, and assets were collected in a more limited number of cases.20 Importantly, there is a rich body of literature and debate on the comparative performance of indicators and measurement techniques for food security and nutrition, particularly as they relate to access and utilization of food (Dary and Imhoff-Kunsch 2012; Barrett 2010; Wisemann et al. 2009; Webb et al. 2006). The selected studies show that, in absolute terms and across basically all indicators, cash, food, and vouchers are effective in meeting program objectives. They also rejected the myths of food being resold on markets or cash being spent on non- desirable items.21 The following subsections examine the relative impacts of cash and food transfers on food consumption, availability of calories, dietary diversity, poverty, and malnutrition. One of the most widely used indicators in the sample is food consumption. Adopted in 7 out of the 10 countries, the measure accounts for expenditures or value of food consumed at the household level. Figure 1 shows the difference in impacts on food consumption in Yemen, Cambodia, Mexico, Ecuador, Sri Lanka, and Bangladesh. Differences are expressed in percentage points (namely, average 146 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 Gentilini Table 1. Descriptive Statistics of Impact Evaluations Size as % Cash of Pre- Sample Program size Food program Frequency HH Years of Delivery Size Evaluation Name Country type* Modality (US$) basket HH exp. of Transfer Size Exposure data mechanism (HHs) Methods Reference 1 PAL Mexico CT, UT Cash, Food 13 F 11.5 Monthly 4.2 trial 1 year 2004– 2005 Biometric debit 5,028 DD Cunha (cash), cards (2014) bi-monthly (food) Zinder project Niger PW, UT Cash, Food 50 F2 11.5 Bi-weekly 7 6 months July to Mobile ATMs, 2,209 SD Hoddinott October smart cards et al., 2011 (2014) PSNP Ethiopia PW, UT Cash, Food 16.2 F3 N/A Monthly 5 6 months 2006– 2008 N/A 960 SD Sabates- per year Wheeler and Devereux (2012) Early Uganda CT Cash, Food 12 F4 N/A 6-week cycle 6.22 12 months October Mobile money 2,450 ANCOVA Gilligan and Childhood 2010 to cards (est.) Roy (2013) Development April 2012 Colombian Ecuador CT Cash, Food, 40 F5 16 Monthly 3.8 6 months April to ATM card 2,122 ANCOVA Hidrobo et al. refugees Vouchers November (2014) project 2011 IDPs project Democratic UT Cash, 18.57 V6 18.96 Bi-monthly 5.5 7 months September Bank accounts 252 Fixed effects Aker (2013) Republic of Vouchers 2011 to Congo March 2012 Unconditional Yemen UT Cash, Food 49 F7 N/A Bi-monthly 7.9 6 months 2011– 2012 ID card via 3,353 SD, ANCOVA, Schwab safety net Postal Savings DD, DDD (2013) Corporation Scholarship Cambodia CT Cash, Food 5 F8 2.5 Monthly 6 10 months August On-site manual 4,091 DD Barker et al. pilot program 2011 to distribution (2014) August 2012 Continued 147 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 148 Table 1. Continued Size as % Cash of Pre- Sample Program size Food program Frequency HH Years of Delivery Size Evaluation Name Country type* Modality (US$) basket HH exp. of Transfer Size Exposure data mechanism (HHs) Methods Reference CTPP Sri Lanka UT Cash, Food 9.8 F9 26.3 Bi-weekly 3.8 3 months November Samurdhi Bank 1,357 s DD Sharma (cash), 2005 to (2006) bi-monthly February (food) 2006 IGVGD, RMP Bangladesh UT, PW Cash, Food 19.7 F10 30 (cash) Bi-monthly 4.6 2– 4 years 2006 Public banks 1,200 PSM Ahmed et al. 15.5 (food) (cash), (2010) monthly (food) The World Bank Research Observer, vol. 31, no. 1 (February 2016) Note: 1 Seven basic items—enriched corn flour, rice, beans, dried pasta soup, biscuits, fortified milk powder, and vegetable oil—and two to four supplementary items (including canned sardines, canned tuna fish, dried lentils, chocolate, breakfast cereal, or corn starch. 2 3.5 kg of grain ( primarily maize in the first transfer period and sorghum in the second), 0.72 kg of pulses (cowpeas, red beans, or lentils), 0.14 kg of vegetable oil, and 0.035 kg of salt. 3 3 kgs of cereals, plus pulses and oils. 4 Food basket of approximately 1,200 calories, includes corn soy blend (“CSB” – highly fortified with iron among other nutrients), vitamin-A fortified oil, and sugar. 5 The food basket was valued according to regional market prices at $40 and included rice (24 kilograms), vegetable oil (4 liters), lentils (8kilograms), and canned sardines (8 cans of 0.425 kilograms) (voucher: The list of approved foods consists of cereals, tubers, fruits, vegetables, legumes, meats, fish, milk products, and eggs). 6 three food fairs, where participants could get palm oil, sugar, cassava flour, beans, rice, vegetable oil, dried fish, salt, potatoes and peanuts. 7 For an average household size of seven persons is 50 kg of wheat flour and 5.0 liters of vegetable oil. 8 10 kg of rice per month. 9 1.4 kg Rice, 1.4 kg Wheat flour, 0.42 kg Pulses, 0.14 kg Oil, 0.14 kg Sugar, 0.14 kg Corn soy blend. 10 up to 20 kilograms (kg) of wheat or 16 kg of rice per month. *CT ¼ conditional transfer, UT ¼ unconditional transfer, PW ¼ public works. Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 Table 2. Indicators Included in the Evaluations Mexico Niger Ethiopia Uganda Ecuador Congo Yemen Cambodia Sri Lanka Bangladesh Total consumption 3 3 3 3 Food consumption 3 3 3 3 3 3 3 Non-food consumption 3 3 3 3 3 3 Food gap 3 3 Food diversity 3 3 3 3 3 3 3 Caloric intake 3 3 3 3 Anthropometric 3 3 measures Income 3 3 3 Assets 3 Poverty headcount 3 3 Labor market 3 participation Sickness probability 3 Anemia 3 3 School dropout rates 3 Cognitive development 3 Figure 1. Relative Impacts of Food versus Cash Transfers on Food Expenditures (Difference in Percentage Points). impacts among food-receiving households minus those among cash-recipients), with negative values indicating the cases for which cash is more effective, and vice versa. Gentilini 149 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 The figure shows that only in Ecuador were impacts of food consumption larger for food-receiving beneficiaries, including relative to both cash and voucher trans- fers. In Yemen, Cambodia, Mexico, Sri Lanka, and Bangladesh the impacts on food consumption are higher for cash than for food-beneficiary households. In three cases—Yemen, Cambodia, and Sri Lanka—the difference is double digit. For Mexico and Ecuador, the difference in impacts is, however, not statistically significant.22 Measures for quantifying calorie intake may present additional information re- garding the difference in impacts on per capita food availability. Figure 2 sets out the impacts of food and cash transfers for programs in Sri Lanka, Yemen, Mexico, Ecuador, and Bangladesh. In contrast with food consumption measures, food transfers have a larger impact on calorie intake relative to cash in most contexts. In Ecuador, the larger effect on calories from food was mainly due to larger increases in cereal consumption (which represented 41 percent of households’ caloric intake). In Yemen, higher caloric con- sumption from food stemmed from the basket composition, including wheat and oil. In the case of Sri Lanka, the impact is negative but not significant. After disaggre- gating these impacts geographically, Sharma (2006) notes that the impacts only decline significantly for one sub-region. The reason for this spatial difference is that Figure 2. Relative Impacts of Food versus Cash Transfers on Per Capita Calorie Intake (Difference in Percentage Points). 150 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 the household baseline survey was conducted the week after Muslim and Hindu fes- tivities. Additionally, the negative effect can be explained by a change in diets, that is, a shift in consumption from highly caloric foods to diets of higher quality (e.g., eggs, meat). In the case of Bangladesh, cash transfers had a larger impact on food consumption. One possible explanation is that the size of the cash transfer was 70 percent higher than the food transfer. Ahmed et al. (2010) address this difference by comparing the change in marginal propensity to consume food, which shows consistent results. Indicators that measure dietary diversity can provide insightful information on the quality of consumption patterns, in addition to its quantity. Three indicators are used in the evaluations, namely the Dietary Diversity Index (DDI), Food Consumption Scores (FCS) and Household Dietary Diversity Score (HDDS).23 Difference for FCS is available for Cambodia, Niger (July and October), Ecuador, and Yemen. Results are mixed, with cash being more effective in three cases (Ecuador’s cash and vouchers arms and Yemen), and food in the other three (Cambodia and Niger, both seasons). In Ecuador, the larger increase in dietary diversity for vouchers was mainly due to larger increases in the number of days that participants consumed vegetables, eggs, milk, and dairy. Similar effects of transfers were noted for the DDI, which included the same sample of countries except Cambodia. One reason that the cash recipients had less diverse diets lies in their choice of using a significant proportion of their transfers to buy grains in bulk, the least expensive form of calories present on local markets. As pointed out by Hoddinott et al. (2014), such a purchasing strategy was a reflection of uncertainty regarding future food prices. The two studies in Congo and Ecuador also allowed for comparing the HDDS index among cash and voucher-receiving arms. In Congo, vouchers had a large impact on households’ dietary diversity (by 3.36 food groups), a 15 percent in- crease. Despite the comparatively lower impact of cash transfers, the difference in impact between cash transfers and vouchers is not significant. In Ecuador, the impact among cash and voucher recipients is fairly similar. The food gap measure indicates the decrease of months of food shortage. In the case of Ethiopia, a two-year exposure to food rations led to less months of food short- age compared to households participating in cash transfers ( public works). In Uganda, among cash- and food-treated households there was a reduction of 0.6 and 0.4 months of food insecurity. respectively. However, the difference is not statis- tically significant. Some countries present data on the relative impacts of transfers on short-term and long-term nutrition-related dimensions. In Mexico, both food and cash trans- fers increased the intake of micronutrient (iron) amongst children by 1.61 mg and 1.10 mg, respectively. However, the difference is not statistically significant. The same pattern holds for increases in zinc and vitamin C. Similarly, anemia Gentilini 151 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 prevalence was reduced by 2 percent in food-receiving households and 4 percent in the cash-receiving households. In Uganda, the prevalence of severe underweight was 3.8 percent lower in the cash group than in the food group, a result significant at the 10 percent confidence level. Cash transfers reduced anemia prevalence by about 10 percentage points for young children (also in this case at 10 percent confidence level). In this context, food transfers had no significant impact. Finally, in Cambodia neither treatment modality in the food-cash scholarship program had significant impacts on anthro- pometric indicators, possibly because of the small transfer size and short exposure to treatment. In some contexts cash and food transfers had an income multiplier effect on beneficiary households. In Ethiopia, PSNP food beneficiaries had a positive and significant income growth of 59.7 percent (at a 1 percent confidence level). In Cambodia, treatment households (both food and cash recipients) increased their net disposable liquidity by about $54 over the course of the year. In Bangladesh, both food and cash transfers increased income significantly: the cash-based RMP increased beneficiary income by 31.4 percent and the food-based IGVGD by 27.8 percent. Both food and cash transfers reduced poverty in Mexico and Bangladesh. Skoufias et al. (2008) showed the impact of cash and food transfers on the head- count ratio as measured by the food poverty line. Similarly, Ahmed et al. (2010) estimated the impacts of cash transfers from the RMP program and food transfers from the IGVGD on the extreme poverty headcount ratio. In both instances, food transfers had larger impacts, with a difference on 3.8 and 1.94 percentage points in Bangladesh and Mexico, respectively. The overall impacts of transfers on the poverty gap are larger. In the context of Mexico, food transfers decreased the poverty gap by 22.3 percent and cash transfers by 18.9 percent; moreover, the severity of poverty decreased by 27.8 percent and 22.97 percent, respectively. Skoufias et al. (2008) argue that the PAL transfer— equivalent to 11.5 percent of pre-transfer level household consumption—may have generated a multiplier effect that led to a reduction by 13 to 15 percent in the headcount poverty rate in two years. In the case of labor market participation, both food and cash transfer had a negative impact in agricultural activities, but a positive one in non-agricultural activities. Cash-receiving households increased non-agricultural activities by 7.1 percent and food-receiving households by 5.8 percent. Based on the results presented in this section, figure 3 summarizes impacts by transfers and indicators. On average, impacts tend to be balanced across modalities, with no systematic dominance of one transfer over the other. 152 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 Figure 3. Summary of Relative Impacts by Transfer Modality and Indicator. Note: Bars in light gray refer to percentage points, dark gray bars refer to changes in specific indicator values. Level of significance is indicated by the asterisks (* at the 90 percent level, ** at the 95 percent level, *** at the 99 percent level). Gentilini 153 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 Costs Most of the examined evaluations also conducted a cost analysis, including Mexico, Niger, Ecuador, Democratic Republic of Congo, Yemen, and Bangladesh. In Mexico, logistics costs of moving commodities from warehouses to villages have been esti- mated to be about 30 pesos per box: this is equivalent to nearly 20 percent of the wholesale cost of the transfer (150 Mexican pesos, or approximately USD 15). Since cash transfers were distributed in the same way as under Mexico’s Oportunidades program, based on such data Cuncha (2012) estimated that it costs 2.4 percent of the transfer amount to deliver cash to recipients. Therefore, food is about ten times more costly than cash transfers, although it is of higher value (indeed, the 150 pesos cash transfer could only purchase about 73 percent of the in-kind basket; see table 3). In Niger, food was about three times more expensive to implement than cash pay- ments. However, Hoddinott et al. (2014) excluded costs that were common to both the food and cash payments, such as expenses associated with implementing the public works, identifying the beneficiaries, program sensitization, identification of implementing partners and contract negotiations with partners selected to imple- ment this intervention.24 Some cash-specific costs were not considered, like fixed costs associated with setting up the cash delivery system. For example, each smart card used for the cash payments cost USD 6.00, while there were additional costs associated with computer programs needed to dispense payments through mobile ATMs. In the case of Ecuador, food costs were about four times higher than for cash and vouchers. The cost of physical materials associated with vouchers, such as printing, is trivial. However, significant staff costs are associated with supermarket selection, the negotiation of contracts with individual supermarkets, and voucher reconcilia- tion and payment. These staff costs account for nearly 90 percent of the cost of implementing the voucher component of the intervention. The cost of generating the debit cards was the main cost item in the cash transfer. The food transfer was significantly more expensive because of the cost of transporting the food to the Table 3. Costs per Total Transfer Country Food Cash Voucher Mexico $2.29 $0.31 Niger $12.91 $4 Ecuador $11.50 $3.03 $3.30 Dem. Rep. of Congo $11.34 $14.35 Yemen $10.37 $4.09 Source: Cunha (2014), Hidrobo et al. (2014), Hoddinott et al. (2014), Aker (2013), Schwab (2013). 154 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 distribution sites and the rental of storage facilities. Taking bulk items and repack- aging them for distribution is also very costly, accounting for approximately 30 percent of the cost of distributing the food ration (Hidrobo et al. 2014). Interestingly, Margolies and Hoddinott (2014) noted that food logistics costs also hinge on the location of food distribution points—that is, higher costs can be in part explained by delivering food directly to beneficiaries’ communities, such as for security reasons in Yemen; yet cash beneficiaries in Yemen had to collect the trans- fer at post offices, which meant higher transaction costs. In Ecuador, food distribu- tion sites were located farther than cash and voucher payment points, thus increasing private costs (time and money); in Uganda and Niger, there appears to be no difference in transaction costs since both transfers were distributed at the village-level. In general, there appears to be a trade-off between costs for the imple- menter and those for beneficiaries: as payment or distribution points get closer to beneficiaries, costs for the implementer get higher while the transaction costs for beneficiaries dwindle. In other words, programs that seem less expensive could be so because the cost of obtaining benefits had been shifted from the implementer to the beneficiary. The studies also raise the question of how to measure the value of food, whether at local market prices or procurement costs, with the latter being a more realistic cost (including for the different commodities that compose the basket). For two cases, Ecuador and Yemen, Margolies and Hoddinott (2014) estimated the full cost of cash and food transfers, with costs for food taken at the procurement level. In Ecuador, the procurement costs for food were higher than their local market value: indeed, accounting for the local procurement of most canned fish, rice, lentils and oil, and including the international procurement of some oil and lentils, it turns out that it cost USD 46.76 to provide a transfer that is locally valued at USD 40. This led to a total cost of providing food of USD 58.25 (USD 46.76 plus 11.46), which even exacerbated the cost differences, that is, total cost for cash is USD 42.99 while for vouchers it is USD 43.27 (the value of both voucher and cash transfer is, by defi- nition, USD 40). Indeed, the difference between food and cash is now USD 15.26 per transfer compared to USD 8.47 (i.e., USD 11.46 minus USD 2.99) when transfer values were excluded. In Yemen, food was more than twice as costly as cash transfers. Yet the difference becomes significantly smaller if additional cost items are included. For instance, households faced significant transaction costs (travel) for receiving the cash trans- fer. By factoring beneficiary cost to collect transfers, this raises the per-transfer cost (excluding the value of the transfer) of cash to USD 8.22. For food transfers, the ad- dition of beneficiary costs raises the per-transfer cost (excluding the transfer value) to $11.35 (Schwab 2013). Moreover, market conditions were such that it was possi- ble to procure for USD 39 a food basket locally valued at USD 49. Considering those Gentilini 155 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 factors, the cost difference between food and cash cost even reversed, with cash being USD 2.8 more expensive than food (figure 4). In the Democratic Republic of Congo, vouchers were more expensive than cash transfers. The cost breakdown shows that staff time represents the largest percent- age of costs for both interventions, followed by transport and voucher printing (for the voucher intervention) and account-opening fees (for the cash intervention). Yet since the account opening fees are a one-time, fixed cost, if the program were to continue cash transfers with existing beneficiaries, the cost per cash program recipi- ent would have only been USD 6 or 8 less expensive per program recipient (Aker 2013). In Bangladesh, Ahmed et al. (2010) considered the costs for delivering cash and food transfers. For cash, only the bank transaction cost was considered, thus gener- ating a very low cost for cash transfers (0.00115 per taka delivered). In the case of food, the analysis considered procurement costs as well as costs incurred at ports, losses, internal transport, storage, and handling, for a total cost of 1.2 taka per 1 taka transferred. Yet few studies have explicitly considered costs and impacts (or cost-effectiveness) of a program, hence quantifying the possible trade-offs between the two. As illus- trated by a maternal-child health program in Honduras, “. . . it cost 1.03 lempiras to deliver 1 lempira of income transfer in the form of a cash-like coupon, while it cost 5.69 lempiras to deliver the same income transfer in the form of food. However, Figure 4. Difference in Total Costs (USD) between Transfer Modalities, With and Without Procurement (Food Minus Cash Costs). Source: Margolies and Hoddinott (2014). 156 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 Table 4. Cost of Increasing a Given Indicator by 15 Percent in Ecuador Indicator Food Cash Voucher Consumption $10.78 $3.79 $3.81 Calories $10.78 $7.58 $4.50 Household dietary diversity score $28.75 $11.36 $8.25 Dietary diversity index $15.68 $3.25 $2.91 Food consumption score $17.25 $4.13 $3.09 Source: Hidrobo et al. (2014). the cash transfer had no effect on child’s calorie consumption nor on use of the health centers, while the food transfer increased both,” (Rogers and Coates 2002). In Bangladesh, Ahmed et al. (2010) estimated that the cost required to increase 100kcal to beneficiaries is roughly similar for cash and food transfers (USD 3.28 and USD 3.21, respectively), while the cost for increasing household income by 100 taka is lower for food (USD 0.68) than cash transfers (USD 1.27). In Ecuador it was estimated that food transfers cost twice as much as vouchers to boost calorie intake by 15 percent, and were 5 times as costly as a way of increasing dietary diversity and quality (Hidrobo et al. 2014). Table 4 details the results for Ecuador. In sum, cash and vouchers seem generally more efficient than food transfers. However, conclusions should be drawn with caution. Methods and approaches vary greatly, hindering intra- and cross-country comparisons. Most of the studies assess, for instance, food costs at local market prices instead of procurement costs. In other words, the availability of tools for transparent, comparable, and systematic cost analysis is limited. The development of such tools could help provide a more accu- rate and realistic account of comparative program performances. Discussion The evaluations reviewed in the previous section prompt two questions: first, are those studies helping to advance the debate meaningfully? The answer is yes, defi- nitely so, including by providing credible evidence, fresh perspectives and gearing the discussion toward results-oriented perspectives. In line with Hoddinott (2013), “Policy and interventions design discussion need to shift from their current ideologi- cal and political focus to one that emphasizes greater precision in interventions objectives, more nuanced understanding of context, and greater emphasis on costing.” The studies not only provide solid evidence based on counterfactuals, but also offer a platform for discussing a number of related social protection design and policy matters. However, are the reviewed evaluations conclusive about the debate? No, most probably not. Indeed, like any evaluation method, experiments have inherent Gentilini 157 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 advantages and limitations. While not all the studies were RCTs, findings from those methods are particularly relevant when interpreted in light of the very specific circumstances that define a given setting, although that may limit extrapolations and lessons for different scenarios (external validity). Importantly, our review showed that context and design matter. For example, the impacts tend to be associated with factors such as the marginal propensity to consume food, the duration of the program, the frequency of transfers, and per- ceived risks (e.g., of prices in the context of Niger), all of which affect purchasing and consumption behaviors. The composition of food baskets also plays an impor- tant role, including having a direct bearing on household dietary diversity and calorie availability. Therefore, a key issue that emerges from the evaluations is the need to interpret transfer selection as part of an organic decision-making frame- work. In particular, the choice of an optimal safety net should be guided by the spe- cific objectives pursued, the peculiar contexts where programs are implemented, the choice of key design parameters (e.g., targeting method, conditionality, transfer size, duration, etc.), and the selection of modalities (including based on technical issues such as the functioning of markets, etc.). Transfer performance is ultimately and largely a function of those factors, instead of an inherent superiority of one mo- dality over the other.25 Of course, a range of other elements help shape the program environment, such as the role of theory (including how that shapes the views of decision-makers), the avail- ability of local and international evidence, and political economy. Although concep- tually desirable, it will probably be impossible to fully separate technical and evidence-related issues from those of political economy. As such, real-world decision- making should perhaps consider political economy as an integral feature of the debate, and not necessarily a mere “non-technical” issue. Also, it is interesting to note that while only a few years ago the use of technology seemed to give an edge to cash transfers, the adoption of technological solutions (e.g., on payments, monitor- ing, verification of identity, etc.) now also underpins a range of voucher and food- based programs, hence making the field more leveled and overall transparent.26 The paper also shows that results on effectiveness (impacts) seem considerably more robust that those on efficiency. Devising robust and standardized tools and methods for identifying, collecting, and analyzing cost data should be a key priority for the transfer debate. In this regard, it would be important for cost calculations to be based on a more nuanced understanding of supply chains and agricultural markets. Indeed, implementation models can vary considerably, depending on the specific approaches and actors involved at different points in the supply chains. In the case of school feeding, for instance, Gelli and Suwa (2014) noted that “. . . dif- ferent approaches can even co-exist within the same country, where, for instance, programme implementation is owned by decentralised institutions (e.g. individual states in Brazil or India), or where agencies (. . .) are complementing the national 158 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 programmes (e.g. Ghana and Kenya), [or models] linking the provision of goods and services for school feeding to smallholder farmers and the community.” Those nuances need to be taken into account for credible cost analyses. Following Gelli et al. (2012), “. . . a holistic, system wide analysis of the goods, funds and informa- tion flows between the involved stakeholders and helps to identify the opportunities to achieve cost-effective and sustainable programs.” In a similar vein, the extensive logistics required for food-based programs—including procurement, transport, storage, and distribution—may posit particular risks for accountability, transparen- cy, and “leakages”. In India, for example, it was estimated that diversion and cor- ruption account for significant losses: in the early 2000s, about 58 percent of the food under the PDS program did not reach the intended beneficiaries (World Bank 2011). Such losses throughout the logistics chain should also be accounted for using comprehensive cost-effectiveness assessments of alternative modalities. Because of program objectives and rationale, the evaluations included a range of food security-related measures; more comparative research around a wider set of di- mensions, especially child malnutrition, would help advance research and practice. Value-based vouchers as a modality seem underexplored relative to their proven and possible performance, and so is the combinations of cash and food transfers. For instance, new evidence suggests considerable potential stemming from a mix of modalities, especially in protracted emergency contexts (Langendorf et al. 2014; Ruel and Alderman 2013). At the same time, attention should be paid to how ob- jectives and metrics are set. For example, if a program is designed to pursue food security objectives, studies often do not capture what might be the side-effects on housing, clothing, debt repayment, etc. In other words, “choosing what is impor- tant” in terms of a study’s focus may be subjective, hence leaving some other impor- tant dimensions unexplored. Conclusions This paper reviewed key issues and evidence in the cash versus food debate, includ- ing a new generation of comparative evaluations undertaken in ten developing countries. Those studies represent a refreshing evidence-based approach that help inform and navigate a longstanding quandary. The reviewed evidence shows that, in absolute terms, both modalities work. When compared to control groups, cash and food transfers (and vouchers when considered) bolstered improvements in a range of indicators such as food consump- tion, income, dietary diversity, poverty, and malnutrition. The paper also argues that, in relative terms, transfer modalities can lead to varied and mixed impacts over a range of dimensions. We observe a mild tendency of cash transfers to be more ef- fective than food in enhancing food consumption (in five studies out of seven), Gentilini 159 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 while food seems to outperform cash in increasing household caloric intake (in four evaluations out of six). However, overall effectiveness cannot be generalized and it depends not only on particular objectives, but also on the specific indicators used to measure those objectives. Differences in design and context explain part of the difference in performance between alternative transfer modalities. Indeed, the impact of cash and food trans- fers can also differ in programs with identical design and contexts. Therefore, since transfer modalities can alter the impact pathways of an intervention, transfer selec- tion should be considered as a key choice in safety net decision-making, similar to other program design features such as conditionality and targeting methods. Costs will be an increasingly key factor in gauging performance, hence calling for more rigorous and systematic cost-effectiveness analyses than many of those cur- rently available. In general, the reviewed studies show that cash transfers and vouchers tend to be more efficient than food-based interventions. Yet results should be interpreted with caution. Approaches for cost calculations are often not clearly described, rely on simplistic assumptions, and tend to be discretionary in the type of cost items considered. More standardized and robust approaches are required so that efficiency analyses match the higher standards of effectiveness offered by the examined impact evaluations. Notes Ugo Gentilini is a Senior Economist with the World Bank’s Social Protection & Labor Global Practice. He can be contacted at Ugentilini@worldbank.org. 1. See, for example, Andrew et al. (2014), Blattman and Niehaus (2014), Ozler (2013), Davis et al. (2012), and Independent Evaluation Group (2011). 2. A notable exception is the United States’ Supplemental Nutrition Assistance Program (SNAP), or ex-Food Stamp Program. As mentioned in the next section, SNAP has been subject to ample empiri- cal scrutiny in comparison to cash transfers. 3. In most cases, commodities are provided by private or commercial partners. Yet retailers could also be public or a mix of public-private actors, such as the case of fixed-price outlets in Syria (Omamo et al. 2010). 4. In practice, commodity-based vouchers can also provide some room for choice. There are in- stances where beneficiaries can choose between commodities so long as their weight does not exceed the limit established by the program (Levan Tchatchua, personal communication). 5. Initiatives such as WFP’s Purchase for Progress program deliberately procure food from small farmers, hence spurring local economic gains among well-identifiable actors similar to vouchers’. 6. The program is the world’s largest food voucher scheme. According to USDA data, about 1 every 7 Americans, or 47.4 million individuals, benefited from the SNAP voucher program from 2013 to 2014. See: http://www.fns.usda.gov/pd/34SNAPmonthly.htm. 7. During the same period, the debate was also examined in the context of famine relief as laid out in the influential analysis by Dreze and Sen (1989, p.97). 8. An in-kind transfer is defined as extramarginal (or inframarginal) when it is larger (smaller) than what households would have normally consumed in the absence of the transfer.When extramar- ginal, the theory predicts that, under certain conditions, in-kind transfers would be more effective in augmenting food consumption than an equal cash transfer. 160 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 9. Explanations include, among others, a “labeling” effect inducing a sense of moral obligation to use in-kind transfers for their intended food consumption purpose (Senauer and Young 1986); the role of gender and decision-making behaviors in multi-adult households (Breunig and Dasgupta 2005); or alterations in household budgeting and planning of monthly purchases induced by in-kind transfers (Wilde and Ranney 1996). 10. For instance, since 2000 locally-procured food in developing countries soared by 45 percent, while untargeted bilateral food aid (i.e., food aid sold on local markets) now represents only about 3 percent of total flows. The entire volume of international food aid itself amounts to less than 0.25 percent of total global food production (Barrett et al. 2012). Gradually, innovations such as the use of smart cards, mobile phones, biometric devices, and e-monitoring tools were also adopted in food and voucher programs (Omamo et al. 2010). 11. The political economy and distributional aspects of the debate were indirectly captured by James Tobin in his 1970 Henry Simons Lecture at the University of Chicago. Indeed, while Tobin pro- posed large-scale means tested cash transfers to reduce poverty without interfering with the market determination of relative prices, he also wished to pair these with “non-market egalitarian distribu- tions of commodities essential to life and citizenship” such as food stamps and other measures. 12. Paul Niehaus, keynote address at the event “Cash Transfers: The New Benchmark for Foreign Aid?” Center for Global Development, May 9, 2014. 13. For an interesting view on how recent high food prices may have ultimately benefited the poor, see Headey (2014). 14. Counter to this logic, in some cases markets are ( physically) brought to locations where a cash program is implemented. This is the case of fairs, for example, including areas where traders would not operate in the absence of the program. In such cases, cash-based transfers would be provided inde- pendently of the presence of working markets. The example of Congo discussed later in the paper falls under this category (Aker 2013). 15. Note that the issue of expressing and capturing preferences is notoriously difficult to handle, including a number of factors that may distort feedback such as who conducts the survey (e.g., founder of the program), expectations by beneficiaries, etc. 16. See some interesting initial guidance laid out in Ryckembusch et al. (2013), White et al. (2013), and Gelli et al. (2012). 17. Mexico benefited from two distinct evaluations by Cunha (2014) and Skoufias et al. (2008). 18. A new RCT is currently underway in Bangladesh, including compositions of cash, food, and nutrition training (Ahmed et al. 2013). Total sample size includes 5,000 households (4,000 of which are treated). Final results are expected to be released in late 2014. 19. Cash payments were made in three disbursements: September 2011 (US$90); November 2011 (US$20) and February 2012 (US$20). Program recipients had to travel to this market center to receive the cash transfer. Vouchers were provided in three installments, the first of which were com- modity-based, while the last two were value-based. 20. In the case of Niger, Hoddinott et al. (2014) do not report on labour market participation and assets, although their survey includes that information. 21. In Mexico, more than half of the cash transfer was spent on food, out of which a quarter was devoted to nutritious food items such as fruits and vegetables. In Niger, cash recipients reported spend- ing 70 percent of the transfer on food items, 10 percent on nonfood items, 9 percent on transfers to the households, saving 7 percent, and 3 percent to pay back loans. In Ecuador, cash beneficiaries used 83 percent of the transfers for food expenditures. The remainder was spent on nonfood expenditures (6.3 percent), shared with others outside the household (2.4 percent), and saved for later use (8.3 percent). In Congo, cash households used their transfers to purchase over six different categories of goods, health expenses, school fees, and debt reimbursement. In Yemen, cash households report spending 88 percent of their transfer on staple foods. Unlike food households, cash households report spending a portion of their transfers towards repaying debts (5 percent), transportation (2 percent), and near zero on qat (14 out of 10,500 YER). In Uganda, the average cash beneficiary spent Gentilini 161 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 53 percent of the transfer on food (41 percent on stables), while 23 percent was allocated for nonfood goods and 16 percent of the cash was saved. 22. Note that for Mexico, estimates are based on Skoufias et al. (2008) examining the same program and data as Cuncha (2012). Skoufias et al. show the consumption indicator in percentages, while Cunha uses currency. 23. The Dietary Diversity Index reflects the number of different foods or food groups consumed over a given reference period. The Food Consumption Scores index measures the frequency of con- sumption of 8 food groups during the seven days before the survey. The Household Dietary Diversity Score indicates the the number of food groups consumed in the previous seven days from 12 groups. It differs from the DDI in that frequency is measured across standardized food groups instead of individu- al food items. 24. Respondents reported a number of additional costs. For example, they indicated that on average it cost 480 FCFA (roughly 1 USD) to transport 100kg of cereals from the market to home, or otherwise 1,920 FCFA for the transfer period (four trips). This figure, however, does not take into account households’ pooling transport costs, which could significantly reduce the per-household cost. The average cost for obtaining the food transfers by beneficiaries, as reported in household surveys, was only 60 CFA per trip. 25. In terms of comparability, it should be noted that any transfer comparison requires some basic level of functionality in implementation conditions (e.g., in markets, delivery systems, etc.). This some- what obscures an important advantage of food transfers, which centers on its ability to operate in more extreme circumstances (e.g., absence of markets, etc.). 26. For example, WFP is currently supporting some 880,000 Syrian refugees in Lebanon through the use of electronic vouchers implemented in partnership with Mastercard. In Chhattisgarh, India, the PDS program distributes food by using portable technologies available among participating retail- ers and allowing people to buy commodities in their store of choice. References Action Contre la Faim (ACF). 2012. Meta-Evaluation of ACF Fresh Food Voucher Programmes. Paris. Adato, M., and L. Bassett. 2008. “What is the potential of cash transfers to strengthen families affected by HIV and AIDS? A review of the evidence on impacts and key policy debates.” IFPRI. Washington DC. Ahmed, A., A. Quisumbing, M. Nasreen, J. Hoddinott, and E. Bryan. 2010. “Comparing Food and Cash Transfers to the Ultra Poor in Bangladesh.” IFPRI. Washington, DC. Ahmed, A., J. Hoddinott, W. Quabili, S. Roy, F. Shaba, and E. Sraboni. 2013. “Which Form of Safety Net Transfer is Most Beneficial? Impacts on Income, Food Security, and Child Nutrition.” IFPRI, power- point presentation at Stakeholder Workshop (December 3). Dhaka. Aker, J. 2013. “Cash or Coupons? Testing the impacts of Cash versus Vouchers in the Democratic Republic of Congo.” Center for Global Development, Working Paper 320. Washington DC. Alderman, H. 2002. “Subsidies as Social Safety Nets: Effectiveness and Challenges.” World Bank, Social Protection Discussion Paper 0224. Washington DC. Alderman, H., and D. Bundy. 2012. “School Feeding Programs and Development: Are We Framing the Question Correctly?” World Bank Reserach Observer 27 (2) : 204 –21. Alderman, H. 2014 “Can Transfer Programs Be Made More Nutrition Sensitive?” Discussion Paper 1342. IFPRI, Washington, DC. Adelman, S., D. Gilligan, and K. Lehrer 2008a. “The Education and nutrition impacts of alternative food for education programs in Northern Uganda.” Paper Presented at the Conference Social Protection for the Poorest in Africa: Learning from Experience (Entebbe, 8–10 September). 162 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 . 2008b. “How Effective are Food for Education Programs? A Critical Assessment of the Evidence from Developing Countries.” Food Policy Review 8, IFPRI, Washington DC. Andrews, C., A. V. Lopez, and J. Baez. Forthcoming. “What Are We Learning on Safety Net Impacts? Reviewing Evidence from 2010–2013.” World Bank, Washington DC. Attanasio, O., E. Battistin, and A. Mesnard. 2012. “Food and Cash Transfers: Evidence from Colombia.” The Economic Journal 122 (559) : 92 –124. ¨ zler. 2011. “Cash or condition? Evidence from a cash transfer experi- Baird, S., C. McIntosh, and B. O ment.” The Quarterly Journal of Economics 126 (4) : 1709–53. Baird, S., F. H. Ferreira, B. Ozler, and M. Woolcock. 2013. “Relative effectiveness of conditional and un- conditional cash transfers for schooling outcomes in developing countries: a systematic review.” Campbell Systematic Reviews 9(8). Barham, T., and J. A. Maluccio. 2009. “Eradicating diseases: The effect of conditional cash transfers on vaccination coverage in rural Nicaragua.” Journal of Health Economics 28 (3) : 611 –21. Barker, M., D. Filmer, and J. Rigolini. 2014. “Evaluating Food versus Cash Assistance in Rural Cambodia.” World Bank, Washington, DC. Barrett, C. 2010. “Measuring Food Security.” Science 327 (5967) : 825–8. . 2002. “Food Security and Food Assistance Programs.” In B. Gardner, and G. Rausser, eds., Handbook of Agricultural Economics. Amsterdam: Elsevier. Barrett, C., and D. Maxwell. 2005. Food aid after fifty years: Recasting its role. New York: Routledge. Barrett, C., and D. G. Maxwell. 2006. “Towards a global food aid compact.” Food Policy 31 (2) : 105–18. Barrett, C., A. Binder, and J. Steets. 2012. Uniting on Food Assistance: The Case for Transatlantic Cooperation. London: Routledge. Basu, K. 1996. “Relief Programs: When It May Be Better to Give Food Instead of Cash.” World Development 24 (1) : 91 –6. Blackorby, C., and D. Donaldson. 1988. “Cash versus kind, self-selection, and efficient transfers.” American Economic Review 78:691– 700. Blattman, C., and P . Niehaus. 2014. “Show Them the Money: Why Giving Cash Helps Alleviate Poverty.” Foreign Affairs 93 (3) : 117–26. Braido, L., P. Olinto, and H. Perrone. 2012. “Gender bias in intrahousehold allocation: Evidence from an unintentional experiment.” Review of Economics and Statistics 94 (2) : 552 –65. Cash Learning Partnership (CaLP). 2013. “Cash Learning Partnership (CaLP) Strategy January 2013 – December 2014.” Oxford: CaLP . Committee for World Food Security (CFS). 2012. “Social Protection For Food Security: A Report by the High-Level Panel of Experts on Food Security and Nutrition.” High Level Panel of Experts Report 4. Rome. Clay, E. 2006. “Food Aid Tying is the Real Problem: A Response to the Barrett and Maxwell Proposal.” Food Policy 31:119 –22. Coate, S. 1989. “Cash versus Direct Food Relief.” Journal of Development Economics 30 (2) : 199 –224. Creti, P. 2011. “The Voucher Programme in the Gaza Strip: Mid-Term Review.” World Food Programme, Jerusalem; Oxfam GB, Oxford. Cunha, J. 2014. “Testing Paternalism: Cash versus In-kind Transfers.” American Economic Journal: Applied Economics 6 (2) : 195– 230. Currie, J., and F. Gahvari. 2008. “Transfers in Cash and In-Kind: Theory Meets the Data.” Journal of Economic Literature 46 (2) : 333–83. Gentilini 163 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 Dary, O., and B. Imhoff-Kunsch. 2012. “Measurement of food consumption to inform food fortification and other nutrition programs: An introduction to methods and their application.” Food and Nutrition Bulletin 33 (2) : 141S –5S. Das, J., D. Quy-Toan, and B. Ozler. 2005. “Reassessing Conditional Cash Transfer Programs.” World Bank Research Observer 20 (1) : 57 –80. Davies, M., C. Bene, A. Arnall, T. Tanner, A. J. Newsham, and C. Coirolo. 2013. “Promoting Resilient Livelihoods through Adaptive Social Protection: Lessons from 124 Programmes in South Asia.” Development Policy Review 31 (1) : 27 –58. Davis, B., M. Gaarder, S. Handa, and J. Yablonski. 2012. “Evaluating the impact of cash transfer pro- grammes in sub-Saharan Africa: an introduction to the special issue.” Journal of Development Effectiveness 4 (1) : 1 –8. De Brauw, A., D. Gilligan, J. Hoddinott, V. Moreira, and S. Roy. 2012. “The impact of Bolsa Familia on child, maternal, and household welfare.” IFPRI. Washington, DC. Del Ninno, C., and P. Dorosh. 2003. “Impacts of in-kind transfers on household food consumption: Evidence from targeted food programmes in Bangladesh.” Journal of Development Studies 40 (1) : 48 –78. Del Ninno, C., P. A. Dorosh, and K. Subbarao. 2007. “Food aid, domestic policy and food security: Contrasting experiences from South Asia and sub-Saharan Africa.” Food Policy 32 (4) : 413 –35. Dercon, S., A. Park, and A. Singh. 2012. School Meals as a Safety Net: An Evaluation of the Midday Meal Scheme in India. Econoomic Development and Cultural Change 62 (2) : 275– 306. Devaney, B., and R. Moffitt. 1991. “Dietary effects of the food stamp program.” American Journal of Agricultural Economics 73 (1) : 202– 11. Devarajan, S. 2013. “Let Them Eat Cash.” Future Development (blog), October 29. https://blogs. worldbank.org/futuredevelopment/let-them-eat-cash. Devereux, S. 2012. “Social Protection for Enhanced Food Security in Sub-Saharan Africa.” Working Paper 2012-010, UNDOP , New York. . 2008. Innovations in the Design and Delivery of Social Transfers: Lessons Learned from Malawi. Brighton, UK: Institute of Development Studies. Devereux, S., and P . Jere. 2008. “Choice, Dignity and Empowerment? Cash and Food Transfers in Swaziland. An evaluation of Save the Children’s Emergency Drought Response, 2007/08.” Institute of Development Studies and Save the Children. Brighton and Mbabane. Doss, C. 2013. “Intrahousehold Bargaining and Resource Allocation in Developing Countries.” World Bank Research Observer 28 (1) : 52 –78. Dreze, J., and A. Sen. 1989. Hunger and Public Action. Clarendon Press. Oxford. Duflo, E. 2003. “Grandmothers and Granddaughters: Old-Age Pensions and Intrahousehold Allocation in South Africa.” World Bank Economic Review 17 (1) : 1 –25. Ellis, F., and E. Manda. 2012. “Seasonal Food Crises and Policy Responses: A Narrative Account of Three Food Security Crises in Malawi.” Food Policy 40 (7) : 1407–17. Evans, D. K., S. Hausladen, K. Kosec, and N. Reese. 2014. Community-Based Conditional Cash Transfers in Tanzania: Results from a Randomized Trial. Washington, DC: World Bank. Faminow, M. D. 1995. “Issues in valuing food aid: The cash or in-kind controversy.” Food Policy 20 (1) : 3–10. Fiszbein, A., and N. Schady. 2009. Conditional Cash Transfers: Reducing Present and Future Poverty. Washington, DC: World Bank. Fraker, T., A. Martini, and J. Ohls. 1995. “The effect of food stamp cashout on food expenditures: An assessment of the findings from four demonstrations.” Journal of Human Resources 30 (4) : 633–649. 164 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 Garcia, M., and C. Moore. 2012. The Cash Dividend: The Rise of Cash Transfer Programs in Sub- Saharan Africa. World Bank, Washington, DC. Garnsey, P. 1988. Famine and Food Supply in Graeco-Roman World: Responses to Risk and Crisis. Cambridge, UK: Cambridge University Press. Gelli, A., A. Kretschmer, and L. Molinas. 2012. A comparison of supply chains for school food explor- ing operational trade-offs across implementation. London: PCD. Gelli, A., and Y. Suwa. 2014. “Investing in innovation: Trade-offs in the costs and cost-efficiency of school feeding using community based kitchens in Bangladesh.” Food and Nutrition Bulletin 35 (3) : 327–37. Gentilini, U. 2007 “Cash and Food Transfers: A Primer.” Occasional Paper 18, WFP, Rome. Gertler, P. J., S. Martinez, P. Premand, L. B. Rawlings, and C. M. Vermeersch. 2011. Impact evaluation in practice. Washington, DC: World Bank. Gilligan, D., and S. Roy. 2013. “Resources, stimulation, and cognition: How transfer programs and pre- school shape cognitive development in Uganda.” Paper presented at the Agricultural & Applied Economics Association’s 2013 AAEA & CAES Joint Annual Meeting, Washington, DC, August 4–6. Gilligan, D., and J. Hoddinott. 2007. “Is There Persistence in the Impact of Emergency Food Aid? Evidence on Consumption, Food Security, and Assets in Rural Ethiopia.” American Journal of Agricultural Economics 89 (2) : 225–42. Government of India. 2005. “Performance Evaluation of Targeted Public Distribution System (TPDS).” Planning Commission, New Delhi. Harvey, P., and K. Savage. 2006. “No Small Change. Oxfam GB Malawi and Zambia Emergency Cash Transfer Projects: a Synthesis of Key Learning,” London: Overseas Development Institute. Harvey, P., and S. Bailey. 2011. “Good practice review: Cash transfer programming in emergencies.” Humanitarian Practice Network 11. London: Overseas Development Institute. Headey, D. 2014. “Food Prices and Poverty Reduction in the Long Run.” IFPRI, Discussion Paper 1331. Washington, DC. Hidrobo, M., J. Hoddinott, A. Peterman, A. Margolies, and V. Moeira. 2014. “Cash, Food, or Vouchers? Evidence from a Randomized Experiment in Northern Ecuador.” Journal of Development Economics 107:144–56. Hoddinott, J. 2013. “Cash, Food, or Vouchers? Evidence from a Four-Country Experimental Study.” Presentation at IFPRI-WFP event, Washington, DC. Hoddinott, J., S. Sandstrom, and J. Upton. 2014. “The impact of cash and food transfers: Evidence from a randomized intervention in Niger.” Discussion Paper 1341, IFPRI, Washington, DC. Independent Evaluation Group (IEG). 2011. Evidence and Lessons Learned from Impact Evaluations on Social Safety Nets. Washington, DC: World Bank. Kazianga, H., D. de Walque, and H. Alderman. 2014. “School Feeding Programs, Intrahousehold Allocation and the Nutrition of Siblings: Evidence from a Randomized Trial in Rural Burkina Faso.” Journal of Development Economics 106:15 –34. Kenny, C. 2013. “For Fighting Poverty, Cash Is Surprisingly Effective.” Bloomberg Business Week, June 3. Available at: http://www.businessweek.com/articles/2013-06-03/for-fighting-poverty-cash-is- surprisingly-effective. Langendorf, C., T. Roederer, S. de Pee, D. Brown, S. Doyon, and A-A. Mamaty. 2014. “Preventing Acute Malnutrition among Young Children in Crises: A Prospective Intervention Study in Niger.” PLOS Medicine 11 (9) : 2 –15. Gentilini 165 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 ´ mez, and D. Maxwell. 2013. “On The Choice and Impacts of Innovative Lentz, E., C. Barrett, M. Go International Food Assistance Instruments.” World Development 49 (C) : 1– 8. Leroy, J., P. Gadsden, S. Rodriguez-Ramirez, and T. Gonzales de Cossio. 2010. “Cash and In-kind Transfers in Poor Rural Communities and Mexico Increase Household Fruit, Vegetable and Micronutrient Consumption but also Lead to Excess Energy Consumption.” American Society for Nutrition 140 (3) : 612–7. Levedahl, J. W. 1991. “The Effect of Food Stamps on Household Food Expenditures.” Washington, DC: U.S. Department of Agriculture. Lindert, P. 2005. Growing Public: Social Spending and Economic Growth Since the Eighteenth Century. Cambridge: Cambridge University Press. Lopez-Rodriguez, D. 2011. “The Political Economy of In-Kind Redistribution.” MPRA, Paper 44152, Munich. MacAuslan, I., and N. Riemenschneider. 2011. “Richer but resented: What do cash transfers do to social relations?” IDS Bulletin 42 (6) : 60 –6. Margolies, A., and J. Hoddinott. 2014. “Costing Alternative Transfer Modalities.” IFPRI, Working Paper 1375, Washington DC. Meyer, J. 2007. “The use of cash/vouchers in response to vulnerability and food insecurity.” WFP. Masuru and Rome. Moss, T. 2013. “Oil-to-Cash Lessons from Alaska – Or Why Anchorage Isn’t So Different from Accra”. CGD blog: Views from the Center, June 12. Available at: http://www.cgdev.org/blog/oil-cash-lessons- alaska%E2%80%94or-why-anchorage-isn%E2%80%99t-so-different-accra. Organisation for Economic Co-operation and Development. 2005. The Development Effectiveness of Food Aid: Does Tying Matter? Paris: OECD Publishing. Omano, S. W ¨ m. 2010. “Revolution: From Food Aid to Food Assistance. ., U. Gentilini, and S. Sandstro Innovations in Overcoming Hunger.” (Eds) WFP , Rome. Ozler, B. 2013. “Cash Transfers: Sorting Through the Hype.” Development Impact, August 22. Available at: https://blogs.worldbank.org/impactevaluations/cash-transfers-sorting-through-hype. Pritchett, L. 2012. “Impact Evaluation and Political Economy: What Does the “Conditional” in “Conditional Cash Transfers” Accomplish?” CGD blog: Views from the Center, January 12. Available at: http://www.cgdev.org/blog/impact-evaluation-and-political-economy-what-does-% E2%80%9Cconditional%E2%80%9D-%E2%80%9Cconditional-cash-transfers%E2%80%9D. . 2005. “A Lecture on the Political Economy of Targeted Safety Nets.” World Bank, Social Protection Discussion Paper 0501, Washington DC. Quisumbing, A. R. 2003. “Food aid and child nutrition in rural Ethiopia.” World Development 31 (7) : 1309–24. Reinhart, U. 2013. “On the Economics of Benefits In-Kind.” Princeton University. Available at: http:// www.princeton.edu/~reinhard/pdfs/100-NEXT_BENEFITS_IN_KIND.pdf. Rogers, B. L., and J. Coates. 2002. “Food-based safety nets and related programs.” World Bank, Social Protection Discussion Paper 223, Washington DC. Ruel, M., and H. Alderman 2013. “Nutrition-sensitive interventions and programmes: how can they help to accelerate progress in improving maternal and child nutrition?” The Lancet 382 (9891) : 536 –51. Ryckembusch, D., R. Frega, M. B. Da Silva, I. Sanogo, U. Gentilini, N. Grede, and L. Brown. 2013. “Enhancing nutrition: A new tool for ex-ante comparison of commodity-based vouchers and food transfers.” World Development 49 (C) : 58 –67. 166 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 Sabates-Wheeler, R., and S. Devereux. 2012. “Cash transfers and high food prices: Explaining out- comes on Ethiopia’s Productive Safety Net Programme.” Food Policy 35 (4) : 274–85. Schwab, B. 2013. “In the form of bread? A randomized comparison of cash and food transfers in Yemen.” Paper presented at the Agricultural & Applied Economics Association’s 2013 AAEA & CAES Joint Annual Meeting, August 4-6, Washington, DC. Senauer, B., and N. Young. 1986. “The impact of food stamps on food expenditures: Rejection of the traditional model.” American Journal of Agricultural Economics 68 (1) : 37 –43. Sharma, M. 2006. “An Assessment of the Effects of the Cash Transfer Pilot Project on Household Consumption Patterns in Tsunami-Affected Areas of Sri Lanka”. IFPRI, Washington, DC. Skoufias, E., M. Unar, and T. Gonzalez-Cossio. 2008. “The Impacts of Cash and In-Kind Transfers on Consumption and Labor Supply: Experimental Evidence from Rural Mexico.” World Bank, Policy Research Working Paper 4778, Washington, DC. Southworth, H. M. 1945. “The economics of public measures to subsidize food consumption.” Journal of Farm Economics 27 (1) : 38 –66. Economist, The. 2014. “Feeding expectations: Why food vouchers are a policy worth considering in developing countries.” Print edition, February 22 –28. Tirivayi, N., M. Knowles, and B. Davis. 2013. “The interaction between social protection and agricul- ture: A review of evidence.” Rome: Food and Agriculture Organization of the United Nations. United Kingdom government, Department of International Development (DFID). 2011. “DFID Cash Transfers Evidence Paper.” Policy Division, DFID, London. Vincent, K., and T. Cull. 2011. “Cell phones, electronic delivery systems and social cash transfers: Recent evidence and experiences from Africa.” International Social Security Review 64 (1) : 37 –51. Webb, P., B. Lorge Rogers, I. Rosenberg, N. Schlossman, C. Wanke, J. Bagriansky, and K. Sadleret al. 2011. Improving the Nutritional Quality of U.S. Food Aid: Recommendations for Changes to Products and Programs. Boston: Tufts University. Webb, P., J. Coates, E. Frongillo, B. Rogers, A. Swindale, and P. Bilinsky. 2006. “Measuring Household Food Security: Why It’s So Important and Yet So Difficult to Do.” Journal of Nutrition 136 (5) : 1404S–8S. White, P., A. Hodges, and M. Greenslade. 2013. “Guidance on measuring and maximising value for money in social transfer programmes: Toolkit and explanatory text”. London: DFID. Wisemann, D., L. Bassett, T. Benson, and J. Hoddinott. 2009. “Validation of the World Food Programme’s Food Consumption Score and Alternative Indicators of Household Food Security.” IFPRI, Discussion Paper 870. Washington, DC. World Bank. 2014. “The State of Social Safety Nets 2014.” Washington, DC. . 2011. “Social Protection for a Changing India.” New Delhi. Yamano, T., H. Alderman, and L. Christiaensen. 2005. “Child growth, shocks, and food aid in rural Ethiopia.” American Journal of Agricultural Economics 87 (2) : 273– 88. Gentilini 167 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/135/2600041/Revisiting-the-Cash-versus-Food-Debate-New by World Bank Publications user on 06 October 2017 The Price Is Not Always Right: On the Impacts of Commodity Prices on Households (and Countries) Daniel Lederman1 and Guido Porto2 This paper provides an overview of the impact that one-time changes in commodity and other prices have on household welfare. It begins with a collection of stylized facts related to commodities based on household survey data from Latin America and Africa. The data un- covers strong commodity dependence on both regions: households typically allocate a large fraction of their budget to commodities, and they often also depend on commodities to earn their income. This income and expenditure dependency suggests sizable impacts and adjust- ments following commodity price shocks. The article explores these effects with a review of the relevant literature. The authors study consumption and income responses, labor market responses, and spillovers across sectors. The paper provides evidence on the relative magni- tudes of various mechanisms through which commodity prices affect household (and na- tional) welfare in developing economies. Commodity price changes, Poverty and welfare impacts, Net consumers and net producers. JEL codes: I30, O13, Q17 Energy prices rose more than 140% between 2000 and 2013, according to the World Bank’s energy-price index, while the food-price index rose by more than 80%.3 These increases in commodity prices motivated researchers to assess the implications for households and workers in developing countries, as well as policy responses. An example of such research is the World Bank’s $4 billion contribution to a joint fund with J.P. Morgan, launched on June 21, 2011, to help developing countries invest in commodity-price hedging instruments.4 Yet commodity prices have stalled since 2012, and observers are once again concerned about the social and economic conse- quences of declining commodity prices.5 Part of the concern is about the volatility of commodity prices, as much as the medium-term trends, per se. The World Bank Research Observer # The Author 2015. Published by Oxford University Press on behalf of the International Bank for Reconstruction and Development / THE WORLD BANK. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com doi:10.1093/wbro/lkv013 Advance Access publication September 28, 2015 31:168–197 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 Whether it is the trends or the volatility of commodity prices, knowledge about the effects of commodity-price fluctuations on household welfare is useful for policy as much as it is useful for understanding economic behavior. It is also important to have a sense of the magnitudes of the impacts involved. The objective of this article is to put forth an economic framework for understand- ing how people in developing countries are affected by relatively permanent changes in commodity and other prices. The approach is to review the literature framed by a model that highlights issues relevant in the discussion of prices and welfare. The litera- ture review canvasses research from various branches, including shocks to exchange rates, financial crises, and trade reforms. In all these cases, the shocks manifest them- selves as price changes, and thus the findings are relevant for the discussion of commodity prices. We complement the literature review with a detailed analysis of household-survey data on commodity dependence in Latin America and Africa. Since we are interested in the impact of commodity prices on household welfare and behavior, our framework builds on agricultural household models. This frame- work analyzes how households choose consumption bundles and how they earn their living. In some cases, especially in rural areas, income is generated by agricul- tural production ( profits). In other cases, where the economy is more industrialized, wages and employment are the predominant sources of income, which can be affected by commodity prices. Since prices can affect both consumption and income, the simple framework opens the door to a discussion of how household welfare is affected by changes in commodity prices, both in the short- and long-run. The literature provides numerous empirical findings that can bring together various pieces of the puzzle illustrated by the framework. The paper is structured as follows. First, we review the literature on the first- order impacts of price changes. These are the impacts that would take place if households adjust neither the quantities of commodities that they consume nor change the sources of income they receive in response to the price change. The article derives the commonplace net-consumer-versus-net-producer result, which establishes that a price increase hurts net-consumers and benefits net-producers. Second, we discuss spillovers from commodity prices to other sectors. This refers to, for instance, whether households adjust all consumption choices in response to higher food prices, or whether only food expenditures are affected. We include a dis- cussion of scenarios where commodities are used as inputs into the production of other goods. Consequently, changes in commodity prices (such as oil) may affect the cost, and thus the price, of certain goods (such as transport). Finally, we also report evidence on spillovers in employment and wages; changes in food prices can affect wages and employment not only in the food sectors (primary, agro-manufacturers) but also in non-food sectors. Third, the article reviews how prices are transmitted to the local economy. Because commodities are traded in international markets, most of the shocks Lederman and Porto 169 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 associated with commodity prices are global. Consequently, a relevant question is to what extent do changes in international prices pass through to domestic markets? Fourth, the article explores available evidence on consumption and production responses. Here, we review how consumers adjust the quantity and quality of the consumption basket, as well as other behavioral responses associated with con- sumption. We also explore some of the patterns of adjustment in production, partic- ularly supply responses in agriculture. It is important to mention at the outset that there are at least five related litera- ture surveys. These include Winters, McCulloch, and McKay (2004) and Goldberg and Pavcnik (2004), both of which cover trade liberalization and poverty; Goldberg and Pavcnik (2007) and Harrison, McLaren, and McMillan (2011), which cover trade and the distribution of income; and Lederman (2013), which covers trade and inclusive growth. Our review complements this work by merging issues of poverty with issues of inequality. Furthermore, we have a somewhat narrower focus, as we are mainly interested in commodity prices, but we take a somewhat deeper look at some of these issues, all in the context of a general framework. The rest of this paper is organized as follows. The following section presents an overview of household commodity dependence in Latin America and Africa. Using various household income and expenditure surveys from Latin American and African countries, we estimate the average budget shares spent on commodity- related sectors (mostly food and energy), and the average income share derived from commodity-related sectors (agriculture, forestry, fishing, mining, food manu- factures, and energy). The data reveal strong commodity dependence on both regions. The next section introduces the analytical framework and reviews the liter- ature that measures those effects using data from developing countries. The impli- cations of price changes for net-consumers and net-producers are central to this analysis, with labor markets, various types of spillovers across markets, and imper- fect price transmission of border prices all playing fundamental supporting roles in the story. The subsequent section turns to departures from first-order effects and studies the role of household adjustments in production and consumption. This is followed by a simple example of the magnitudes of the different channels identified in the paper using Mexican data, while the final section concludes. Commodity Dependence: Stylized Facts This section characterizes the dependence on commodities of households in develop- ing countries. To study dependence on consumption, we focus on expenditure shares spent on items related to commodities, mainly food and energy. Households’ depen- dence on commodities on the income side is then measured by income shares in agriculture (including forestry and fishing), mining, food manufactures (beverages, 170 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 food, and tobacco), and energy. An important aspect of the analysis is related to poverty and inequality, because dependence on commodities either on the consump- tion or the income side can vary across quintiles of the income distribution during a given point in time. Regarding commodity-dependence on the consumption side, table 1 presents the average expenditure shares on commodities for eight Latin American countries and 19 African countries.6 In Latin America, there is significant cross-country variation in the average share spent on commodities, ranging from an average share of 71.3 percent in Bolivia to 31.7 percent in Mexico. A common feature in all these coun- tries is that the share of commodity expenditures decreases across quintiles. This is Table 1. Commodity Expenditure Shares, Latin America and Africa Quintiles p/c expenditures Total (1) (2) (3) (4) (5) Argentina 61.8 48.9 42.3 37.5 29.0 43.9 Bolivia 82.9 78.7 73.0 67.6 54.4 71.3 Colombia 80.1 69.1 61.7 55.6 41.6 61.6 El Salvador 65.2 59.6 54.6 49.4 39.0 53.5 Mexico 40.6 36.2 32.4 28.4 20.8 31.7 Nicaragua 63.5 60.9 57.3 54.4 41.6 55.5 Panama 60.0 49.0 43.9 37.4 28.3 43.7 Peru 60.1 50.6 42.6 35.8 26.1 43.0 Burundi 56.3 60.7 63.1 61.9 53.2 59.1 Benin 65.5 63.8 62.9 59.8 56.5 61.7 Burkina Faso 77.1 72.5 67.4 60.9 48.0 65.2 Coˆ te d’Ivoire 57.2 56.6 55.8 49.3 39.4 51.6 Cameroon 71.7 69.7 67.1 62.3 53.3 64.8 Ethiopia 84.0 79.7 77.0 70.5 55.5 73.3 Ghana 71.2 68.5 67.1 66.6 65.4 67.8 Gambia 67.6 71.3 71.0 70.9 68.6 69.9 Guinea Bissau 74.5 76.6 76.5 77.3 71.5 75.3 Kenya 85.0 82.7 80.7 76.6 67.8 78.5 Madagascar 89.6 90.5 90.7 90.6 88.1 89.9 Mali 79.5 74.4 73.1 67.5 68.8 72.7 Malawi 81.2 79.9 78.4 74.9 69.2 76.7 Nigeria 92.4 82.6 81.6 80.5 74.7 82.3 Rwanda 78.8 69.1 60.8 49.7 27.7 57.2 Senegal 63.7 63.2 62.7 59.8 54.7 60.8 Tanzania 90.0 88.4 85.8 82.9 74.9 84.4 Uganda 74.7 75.5 71.9 67.1 53.0 68.4 South Africa 63.6 55.7 46.7 32.1 15.6 42.7 Note: Authors’ own elaboration based on household surveys. Lederman and Porto 171 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 expected due to Engel’s Law: richer households typically spend a smaller share of their total expenditures on food. In Bolivia, for example, the average commodity share is 82.9 percent in the first quintile and 54.4 percent at the top quintile. In Mexico, the average share of the bottom quintile is 40.6 percent and it is 20.8 percent at the top. In Colombia, the poorest households spend, on average, 80.1 percent on commodities; the richest households spend 41.6 percent. Differences in the populations covered by the surveys across countries (i.e., urban versus rural populations) can clearly account for part of the differences in the average commodi- ty expenditure shares. Nevertheless, the data reveal a very high commodity depen- dence on the consumption side, especially for the poorest households. In Africa, there is also significant variation across countries, from a minimum average share of 42.7 percent in South Africa, to 84.4 percent in Tanzania, and a maximum of 89.9 percent in Madagascar. The commodity dependence on consumption seems higher in Africa than in Latin America, however. In most countries, commodity de- pendence in consumption is more prevalent at the bottom of the quintile distribu- tion. This means that Africa and Latin America are both highly dependent on commodities and, moreover, this dependence declines with household income. This pattern explains the literature’s concerns about the poverty impacts of commodity price hikes. Table 2 turns to dependence on the income side. The aim is to measure the share of household income derived from commodities, including income from employ- ment (wages) and self-employment (for instance, agricultural home production). Starting with Latin American data for 2004, Table 2 reports the total combined share of income derived from commodities, which varies widely across countries. This comparison, however, is unlikely to be very useful because of differences in cov- erage. An example of this is Argentina, which despite being a country with a rela- tively important agricultural sector, the household survey suggests that only 3.2 percent of household income is, on average, derived from commodities. This is clearly more a manifestation of the urban focus of the survey than of the economy. In the discussion that follows, it is thus convenient to focus more on within-country comparisons across quintiles. A regularity in most of our Latin American data is that lower quintiles exhibit higher shares of income derived from “commodities.” The differences are sometimes important. In Paraguay, for instance, while the bottom quintile earns 44.4 percent of income from commodities, the top quintile earns only 14.7 percent. We uncover similar patterns in the African data. The income commodity depen- dence in Africa is very high (higher than in Latin America) and varies across coun- tries. The lower quintiles also tend to show higher commodity dependence. In Ghana, for instance, the share of income derived from commodities is 71.4 in the first quintile, and 27.1 in the top quintile. Table 2 shows that in the data, most of the dependence from commodities in Africa is related to dependence on agriculture. 172 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 Table 2. Commodity Income Shares, Latin America and Africa Quintiles p/c income Total (1) (2) (3) (4) (5) Argentina 3.1 3.1 3.2 3.2 3.5 3.2 Bolivia 38.2 19.6 17.1 13.8 10.6 19.9 Brazil 22.3 14.8 10.1 7.8 5.5 12.1 Chile 17.7 15.6 13.4 10.8 9.0 13.3 Colombia 13.7 14.7 11.0 8.3 6.2 10.8 Costa Rica 26.0 23.9 19.1 13.1 8.9 18.2 Dominican Republic 22.0 16.1 12.7 12.4 7.4 14.1 Ecuador 44.1 33.9 24.5 17.5 11.5 26.3 El Salvador 19.6 17.1 10.9 7.8 5.4 12.2 Guatemala 25.2 23.0 20.6 14.3 11.1 18.8 Honduras 43.7 34.8 21.8 14.1 11.1 25.1 Mexico 24.1 14.0 11.0 7.1 6.4 12.5 Nicaragua 36.9 29.6 20.8 15.7 15.0 23.6 Panama 32.9 17.3 11.6 9.7 4.5 15.2 Paraguay 38.4 27.9 22.5 17.1 15.5 24.3 Peru 37.6 27.1 17.3 11.2 9.5 20.5 Uruguay 9.6 6.9 4.7 3.8 3.5 5.7 Venezuela 23.1 19.3 13.5 11.8 8.2 15.2 Burundi 54.0 57.9 60.8 59.6 51.7 56.8 Benin 60.6 58.3 56.9 54.1 52.2 56.4 Burkina Faso 73.9 69.4 64.3 57.3 43.7 61.7 Coˆ te d’Ivoire 57.1 56.5 55.5 49.0 38.6 51.3 Cameroon 68.4 66.8 64.3 59.9 51.7 62.2 Ethiopia 82.4 78.0 75.2 68.3 52.6 71.3 Ghana 67.5 65.2 64.3 63.8 62.5 64.7 Gambia 67.3 70.8 70.0 69.4 66.4 68.8 Guinea Bissau 68.4 72.6 72.0 73.0 67.3 70.7 Kenya 83.0 80.9 78.8 74.6 65.6 76.6 Madagascar 85.7 87.5 88.0 87.8 85.0 86.8 Mali 76.8 71.0 69.4 63.1 63.7 68.8 Malawi 58.8 61.0 60.9 59.2 55.6 59.1 Nigeria 88.5 75.3 75.0 74.8 69.6 76.6 Rwanda 77.7 67.3 59.1 47.9 26.4 55.7 Senegal 61.6 61.0 60.2 57.1 51.8 58.3 Tanzania 90.0 88.4 85.8 82.9 74.9 84.4 Uganda 71.2 73.0 69.3 64.3 49.5 65.5 South Africa 58.8 51.6 43.9 31.0 15.4 40.1 Note: Authors’ own elaboration based on household surveys. Since the data from both regions indicates that poor households are on average highly dependent on commodities on both the consumption and income sides, the distinction between net-consumers and net-producers will become particularly Lederman and Porto 173 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 relevant for the literature that attempts to assess the distributional and poverty impacts of commodity-price fluctuations. As a preliminary step in this direction, it is worth analyzing the evolution of con- sumption and income dependence on commodities during a period of time when global commodity prices rose. The purpose of the empirical exercise is to describe the changes in income shares observed in the data from 2004 to 2008. Transitions in and out of dependency can be studied with panel data, which is unavailable in our sample countries. We can, however, build pseudo-panels of households using repeated cross-sections with the method proposed by Dang, Lanjouw, Luoto, and McKenzie (2011). Since only the income surveys in Latin America are repeated cross-sections, we focus here solely on income commodity dependence for Latin America. Due to sampling and measurement issues, one can obtain a distorted view of over-time trends by looking just at the household expenditures and incomes between cross sections. What is needed is a methodology that removes the effect of changes in household samples on the average household consumption and income shares derived from commodities. To build the 2004 – 2008 pseudo-panel, we focus on households observed in 2004 and estimate commodity income shares for these households in 2008. This procedure requires the estimation of a regression model for commodity income shares separately for the 2004 and 2008 samples. We then use the estimates from the 2008 sample, together with the residuals from the 2004 regression to produce counter-factual panels. Intuitively, we predict average income shares using the ob- served covariates in 2004 and the estimated parameters for 2008.7 The results are reported in table 3. The first row reproduces the observed income shares in 2004 that were reported in table 2, while the second row shows the pre- dicted 2008 shares. A key result is that income dependence on commodities in- creased in Chile, El Salvador, Honduras, Peru, and Uruguay, which is expected given the rise in global commodity prices. However, dependency remained roughly constant in (urban) Argentina and in the Dominican Republic; Brazil and Mexico experienced a slight decline in commodity income dependence, while in Costa Rica, Ecuador, Panama, and Paraguay, the decline was large. This international heteroge- neity might be surprising given the commodity price boom, but differences are ex- pected due to various factors discussed below. It is also noteworthy that the dynamics of income dependence on commodities varies across quintiles within countries. In El Salvador, Honduras, and Uruguay, income dependence increased not only on average, but also for all quintiles. In Chile average dependence rose, but it decreased for lower quintiles and increased for higher quintiles. In contrast, in Peru, average dependence also rose, but it increased for lower quintiles and decreased for higher quintiles. In Argentina and the Dominican Republic, average dependence remained constant, but increased for low quintiles and declined for high quintiles in Argentina; it declined for low quintiles 174 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 Table 3. Commodity Income Shares Methodology Using Repeated Cross-sections, Latin America, 2004 – 2008 Quintiles p/c income Year (1) (2) (3) (4) (5) Total Argentina 2004 3.1 3.1 3.2 3.2 3.5 3.2 2008 4.2 3.8 3.2 3.0 2.5 3.3 Brazil 2004 22.3 14.8 10.1 7.8 5.5 12.1 2008 16.9 14.6 12.0 9.6 5.9 11.8 Chile 2003 17.7 15.6 13.4 10.8 9.0 13.3 2009 16.0 16.0 14.0 12.6 10.9 13.9 Costa Rica 2004 26.0 23.9 19.1 13.1 8.9 18.2 2008 18.9 20.3 17.5 13.6 8.1 15.7 Dominican Republic 2004 22.0 16.1 12.7 12.4 7.4 14.1 2008 17.2 13.9 11.4 9.5 6.7 11.7 Ecuador 2004 44.1 33.9 24.5 17.5 11.5 26.3 2008 32.3 28.6 24.4 19.8 12.3 23.5 El Salvador 2004 19.6 17.1 10.9 7.8 5.4 12.2 2008 28.3 22.4 17.9 14.0 8.0 18.1 Honduras 2004 43.7 34.8 21.8 14.1 11.1 25.1 2008 44.4 39.4 29.4 20.6 12.6 29.3 Mexico 2004 24.1 14.0 11.0 7.1 6.4 12.5 2008 18.9 13.9 11.1 9.4 7.0 12.1 Panama 2004 32.9 17.3 11.6 9.7 4.5 15.2 2009 20.7 14.6 10.9 9.4 4.2 12.0 Paraguay 2004 38.4 27.9 22.5 17.1 15.5 24.3 2008 33.0 26.9 22.9 19.1 14.7 23.3 Peru 2004 37.6 27.1 17.3 11.2 9.5 20.5 2008 39.1 29.4 20.7 13.1 7.7 22.0 Uruguay 2004 9.6 6.9 4.7 3.8 3.5 5.7 2008 14.9 14.5 12.8 11.3 8.6 12.4 Note: Authors’ own elaboration based on household surveys. and increased for high quintiles in the Dominican Republic. In Brazil, Ecuador, Mexico, and Paraguay, average dependence rose, declined for low quintiles, but increased for higher quintiles. Finally, in Costa Rica and Panama, dependence declined for the average as well as for all quintiles. These results uncover several observations that are important for the subsequent analysis. First, the data show that households adjust after price shocks. In fact, the commodity income shares changed between 2004 and 2008, when global prices rose to record levels. Second, these adjustments were heterogeneous, not only across countries but also across quintiles of the distribution of income within coun- tries. In what follows, the literature review aims to shed light on the impacts of Lederman and Porto 175 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 commodity price changes on household welfare and on household responses and adjustments in response to price signals. Price Changes and Household Welfare This section explores the impacts of price changes on household welfare. Our ana- lytical framework builds on standard agricultural household models, as in Singh, Squire, and Strauss (1986). Although this framework has influenced a large body of research spanning various fields of economics, the ongoing discussion will modify it as needed to illustrate key issues associated with the welfare impacts of commodity price changes. The framework analyzes the consequences of price variations from the viewpoint of households. Analyses of welfare changes resulting from such price signals tend to follow the approach pioneered by Dixit and Normal (1980), in which a household’s budget constraint is (at least initially) assumed to be fixed.8 Simply put, in equilibri- um, household expenditures (including savings) have to be financed with house- hold income (including transfers). In turn, total household expenditures are, by definition, the sum of expenditures across goods, evaluated at their corresponding prices and quantities (for a given level of household utility). However, the household’s decision about how much to spend depends on the prices of the consumed goods, on the level of utility, and on other household characteristics (such as household composition). A household’s income, on the other hand, comprises the sum of the wages of all working members of the household plus profits from business or other economic ac- tivities. Profits include, for instance, the net income from agricultural production or farm enterprises; they depend on prices, technical capacity, and household charac- teristics. Since profits are defined as sales net of purchases of inputs, some of the costs imposed by taxes on inputs or intermediate goods can be assessed by their impacts on profits. Income may also comprise transfers ( public or private), savings, and other unmeasured returns to factors of production, such as land holdings. This simple framework highlights the fact that household welfare depends on market equilibrium variables such as prices and wages (that affect household choices) and also on household endowments. For example, household consumption depends on the prices of consumer goods, and household income depends on the type of labor endowment (skilled, unskilled), the market wage rate, and the prices of goods produced by economic enterprises run by members of the household. It follows that changes in commodity prices affect welfare directly via consumption and production decisions, and that these impacts are heterogeneous insofar as they depend on household choices and endowments. In addition, there are short- run impacts when households do not adjust their consumption, employment, or 176 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 production choices; medium-run impacts refer to the welfare (or consumption or income) effects that take into account partial adjustments; and long-run impacts take into account income growth, investments, and other factors that are related to long-run choices. In other words, the time horizon associated with the impact of price changes on household welfare is commonly defined by the extent to which a household has been able to optimize its total consumption or income in light of the price changes. First-order Impacts Let us consider now the impacts of changes in the price of a commodity. To simplify, we follow Deaton (1989a) and assume that the principle of “separability” holds.9 Under this assumption, production decisions are independent of consumption deci- sions (utility maximization). This means that we can consider the income level of the household as exogenous (once optimal production decisions have been made) when utility maximization takes place as the household adjusts its consumption bundle. The separability assumption is not just a technicality: it allows us to simplify the welfare analysis, but it is valid only under certain economic conditions, such as perfect and complete markets for goods, credit, insurance, and so on.10 Prices affect both expenditures and income, even if they do so separately. On the consumption side, consumers are worse off if prices go up but are better off if prices go down. In a first-order approximation, these impacts can be measured with budget shares. That is, the size of the effect of a price change on a household directly depends on how important the good is in the household’s total expenditures. On the income side, there is also a direct impact on profits if the household produces the affected good, which depends on the share of income attributed to this good. In rural econo- mies, this source of income can account for a large fraction of total income. In more urbanized economies with more developed labor markets (as in many places in Latin America), the role of the direct production of (agricultural) goods tends to be less important. Comparisons of budget shares and income shares establish a key result in the lit- erature: a price increase hurts net-consumers (defined by the difference between ex- penditures and incomes associated with the commodity whose price changed), and benefits net-producers. The opposite is true for price declines: net-consumers are made better off and net-producers worse off. Further, the direction of welfare impacts carries over to the level of a national economy or any aggregate unit of analysis: a country that is a net exporter of an agricultural good will benefit, on average, from price increases, whether they come from global market fluctuations or from international policy reforms such as the international liberalization of agri- cultural trade policies or subsidies; but net-importers will probably be hurt by those Lederman and Porto 177 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 changes. In the context of the recent food crisis, for instance, this implies that coun- tries that are net-importers of food were in principle hurt, while net-exporters were the winners. Households, Net-Consumers, Net-Producers and Inequality The net-consumer/net-producer result was introduced by Deaton (1989a), who used data from the Thailand Socioeconomic Survey of 1981 –82 to explore the dis- tributional consequences of an export tax on rice across Thai households. He found that an increase in the price of rice (resulting from the elimination of the export tax) would benefit the average household. The average poor, as well as the average rich, would benefit little. The benefits for the poor are small because they tend to both consume and produce lots of rice, thus selling little. The average benefit to the rich is also small because while sellers are often large, there are few of them among the rich. The gains would be much higher in the middle of the income distribution, indicating that the middle class would gain the most from higher rice prices.11 The ideas introduced in Deaton’s work have been, and still are, extensively uti- lized in the literature. Early examples include the following authors: Deaton (1989b), who reviews applications for Cote d’Ivoire, Indonesia, and Morocco; Budd (1993), who investigates food prices and rural welfare in Cote d’Ivoire; Benjamin and Deaton (1993), who study cocoa and coffee in Cote d’Ivoire; Barret and Dorosh (1996), who look at rice prices in Madagascar; and Sahn and Sarris (1991), who examine structural adjustments in several Sub-Saharan African countries. Deaton (1997) provides an account of the early use of these techniques in distributional analysis of pricing policies. Recently, there has been a rebirth of this type of work, motivated by the food- price “crisis” associated with skyrocketing prices in global markets. Ivanic and Martin (2008) cover ten case studies using household survey data from Bolivia (2005), Cambodia (2003), Vietnam (1998 and 2004), Malawi (2004), Nicaragua (2001), Pakistan (1999), Peru (2003), and Zambia (1998). These authors find an overall poverty-increasing impact of higher food prices because in the sample, most poor households are net-consumers of food (both in urban and rural areas). Wodon et al. (2008) apply this method to a dozen West and Central African countries, including Burkina Faso (2003), Ghana (2005 – 2006), Democratic Republic of Congo (2005), Gabon (2005), Guinea (2002 –2003), Liberia (2007), Mali (2006), Niger (2005), Nigeria (2003 – 2004), Senegal (2006), Sierra Leona (2003), and Togo (2006). The authors focus on rice, wheat, maize, other cereals, milk, sugar, and vegetable oils and find that a 50 percent increase in prices for selected food items would cause an average increase in the share of the population in poverty of between 2.5 and 4.4 percentage points. The average impact would be 178 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 between 3.7 and 5.2 percentage points in urban areas, and between 2.2 and 4.1 points in rural areas. These authors also find significant heterogeneity across coun- tries, as expected. While the net-consumer/net-producer result is intuitive, it rests on two critical assumptions, namely that the first-order approximation is valid and that labor markets are unaffected by the price change (Deaton 1997). The first-order approxi- mation is analytically useful, but it is valid conceptually only for small price changes. We say “conceptually,” because in principle budget and income shares could be used to measure the distributional effects of both small and large price changes, but the approximation error will be larger when price changes are large. In these cases, it is crucial to incorporate supply and demand responses, and we discuss these below. The role of labor markets is discussed next. Labor Markets Price changes also affect wages; the mechanisms are simple. Consider, for instance, a small, open economy that takes commodity prices as given in international markets. Changes in relative product prices cause some sectors to expand and some others to contract. As a result, labor demand for different types of labor in different sectors can change, thereby affecting equilibrium wages.12 In the literature, these responses are captured by “wage-price” elasticities, which measure the proportional response of wages to price changes. These elasticities may vary across members of a household when they are endowed with different skills (unskilled, semi-skilled or skilled labor), or if they work in different sectors (industry premia). Clearly, if coun- tries differ in technologies, endowments, or labor regulations, the responses of equi- librium wages to prices can be heterogeneous across countries as well. The response of wages can generate first-order effects on household welfare. To account for these responses, the standard net-consumer/net-producer proposition needs to be modified. To illustrate, consider the extreme case where a farm house- hold consumes a product but does not produce it. Instead, the household earns income from selling labor to neighboring farms. By omitting wages, this household would be mistakenly classified as a net-consumer and could thus be presumably hurt by a price increase. But if wages respond positively to prices, the final welfare effect may not necessarily entail a loss. This type of wage response was studied in Ravallion (1990), who explored the conditions under which net consumers of food products in Bangladesh lose or gain in the face of increased food prices when rural wages adjust. Ravallion estimates low elasticities of agricultural wages to food prices and concludes that responses are unlikely to be strong enough to offset the short-run adverse distributional effects of higher food prices. The long-run estimates appear to be more favorable to the poor, Lederman and Porto 179 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 but it would nevertheless take around four years for any gains to materialize. Boyce and Ravallion (1991) take another look at this issue using newer data for Bangladesh. These authors set up a dynamic econometric model of agricultural wages and rice prices and find that increases in rice prices relative to the prices of manufactured goods have adverse effects on the real wages (measured per quantity of rice) in both the short- and long-run. In a study of informal export barriers and poverty in Moldova, Porto (2005) amends the first-order approximation used by Deaton (1989a) to allow wages to adjust. In this setup, exporters produce agro-manufactured goods (i.e., wines or apple juice) for international markets. These exporters purchase primary products from the farmers (grapes, apples) and manufacture exports using, among other inputs, labor. Farmers are endowed with land and labor and allocate time (net of leisure) to the “farm” labor market, the home (land) plot to produce primary inputs (grapes) for sale, or to work in nearby farms. In this framework, Porto derives addi- tional welfare impacts given by the direct impact on wages. Also, a given farm can either sell or purchase labor in the “farm-labor” market, and consequently, the esti- mation of the first-order welfare effects needs to include the additional impacts that arise due to changes in wage earnings or paid wages. Porto (2005) finds that in- creases in the prices of agro-manufactured exports such as wines (a major export sector in Moldova) have sizeable poverty-reducing impacts. Wages respond positively to export prices and this causes first-order gains that dominate both the consump- tion losses due to higher consumer prices and the profit losses due to higher wages paid to hired labor. Porto (2006) studies a case where households consume products and sell labor in the market, but do not produce farm products. This author’s model applies well to urban, middle-income economies where farm activities are not relevant (and are not captured by the available household surveys, either). Porto explores the distribu- tional consequences of MERCOSUR (a regional trade agreement between Argentina, Brazil, Paraguay, and Uruguay) in Argentina and finds welfare gains for the average poor and middle-income households (and negligible effects for the wealthiest house- holds). This is because on top of gains from price reductions due to tariff cuts, there are changes in wages that favor unskilled workers over skilled workers, and unskilled workers are concentrated at the bottom of the income distribution. Three recent studies are also worth mentioning. Ferreira, Fruttero, Leite, and Lucchetti (2011) examine the consequences of food price inflation in Brazil, a large food producer with a predominantly wage-earning agricultural labor force. These authors find large and negative consumption effects, which are also markedly re- gressive, as expected. However, they also uncover a positive and progressive income effect, particularly in rural areas. Thus, overall the Brazilian middle-income house- hold suffered larger proportional losses than the very poor or the richer households. Jacoby (2013) reaches a similar conclusion in his study of food prices in India. 180 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 Specifically, once the wage gains are accounted for, he finds that rural households across the income spectrum actually benefit from higher agricultural commodity prices. Finally, Nicita, Olarreaga, and Porto (2014) examine the presence of a pro-poor bias in the trade protection of the following six Sub-Saharan African (SSA) countries: Burkina Faso, Cameroon, Co ˆ te d’Ivoire, Ethiopia, Gambia, and Madagascar. These authors’ model includes consumption effects, agricultural income effects, and labor income effects; they find that trade policies tend to be biased in favor of poor house- holds because protection increases the agricultural prices of goods that are sold by African households, and this effect dominates the impacts on both expenditures and wages. Spillovers Broadly speaking, spillovers are the impacts of a change in market i on the activity in market j. There are two types of spillovers: 1) production linkages occur when the expansion of a sector affects upstream activities (backward linkages) or down- stream activities (forward linkages); 2) expenditure linkages occur when the in- crease in income due to the expansion of a sector raises the demand for outputs and thus the derived demand for inputs in other sectors. Given our focus on commodi- ties, we emphasize here spillovers in agriculture. The large body of literature on the topic uncovers two main regularities. First, the linkages are strong in rural areas because households earn a large share of their income from and spend a large share of their total expenditures on goods produced locally. Second, both production and expenditures spillovers are important (Mellor and Johnston 1984; Delgado 1996; Harriss 1987). Evidence on spillovers in agriculture in Latin America can be found in the regional study by De Ferranti et al. (2005). These authors report that while the share of the primary sector in GDP is 12 percent, there are massive spill- overs to the rest of the economy. In fact, the multiplier factor from the primary sector to national GDP is 0.22, so that, for instance, a one percent growth of the rural natural resource sector is associated with an increase of 0.22 percent increase in national GDP (much higher than the 0.12 percent increase to be expected from the sector’s share of GDP). At 0.28, the multiplier for the income of the poorest households is even higher. Note that spillovers can be interpreted as general equilibrium effects, but they are not necessarily the same. If there is some sort of labor immobility in general equilibri- um, for instance, then a change in the price of good i may have no effect on the wages paid in sector j. In such a scenario there are no spillovers, even in a general equilibrium setting. This distinction is useful to organize our discussion. Of the papers reviewed above, Ravallion (1990) estimates a partial equilibrium model and assumes no spillovers or general equilibrium effects from rice prices to non-agricultural wages. Lederman and Porto 181 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 An alternative interpretation is that labor markets are segmented. In contrast, Porto (2006) and Nicita (2009) take a general equilibrium approach and assume that a change in a given price pi affects economy-wide wages. This assumes that labor markets are integrated and workers are freely mobile across sectors; an increase in wages in one sector spills over to other sectors. Porto (2008) describes a variant of the spillover mechanism. Markets can be seg- mented so that wages can differ across sectors. However, sectors are related via forward and backward linkages, so an expansion of one sector may have implica- tions for other sectors. In such a setting, changes in corn prices in Mexico can affect the corn sector directly, as well as various rural services associated with corn pro- duction. For instance, higher corn prices may require more labor in corn planting, weeding, and harvesting (thus causing wages in the corn sector to increase), and may also require farm services (road graveling, farm-gate and fence repair, etc.). Porto finds that allowing for those spillovers can make a big difference in assessing the welfare impacts of higher corn prices. In particular, the average Mexican low- income household tends to be a net-consumer of corn and consequently is made worse off by an increase in corn prices. However, once spillovers from the corn sector to agricultural wages are allowed to take place, even low-income net-consumers stand to gain from higher corn prices. As should be clear from this review, the existence or lack of spillovers, and the nature of those spillovers, will be specific to the case under study. It is a priori diffi- cult to make generalizations. Different types of spillovers arise when other product markets, rather than labor markets, are affected. These spillovers are likely to take place in non-traded goods. As we have shown above, changes in commodity prices affect factor prices, includ- ing wages. If the wages earned in non-traded sectors are affected, then the cost of producing these goods will change, and this in turn will affect the equilibrium prices of these goods. As a result, there are additional welfare impacts on the con- sumption side. Notice that these are first-order impacts. Porto (2006) provides an example of how to estimate these impacts for the case of the countries that are members of the MERCOSUR customs union; he uses time series of prices to recover the elasticity of the price of non-traded goods with respect to the prices of traded products; import tariff cuts by MERCOSUR appear to cause the prices of non-traded goods to decline and households to benefit. It is noteworthy that these spillovers are similar to those highlighted by, for instance, Burstein, Neves, and Rebelo (2003) for the computation of the real exchange rate and the adjustment of prices to stabiliza- tion programs. These types of spillover can be particularly important for some commodity price shocks, especially on energy. Households do spend money directly on energy (elec- tricity, heating), but sizeable effects can be created by spillovers to other good prices. For instance, oil price shocks can affect households via expenditures on transport; 182 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 the direct impacts can be calculated using (exogenous) price changes or shocks and budget shares, as carried out above. To measure the indirect impacts we need to esti- mate the response of the prices of non-traded goods after commodity price shocks. This can be done with the “price shifting” model proposed by Coady and Newhouse (2006) and Coady et al. (2006). In this methodology, commodity price shocks are fully transmitted to non-traded prices using input-output tables to infer the share of different (commodity) inputs in the production cost structure. The welfare effects at the household level can then be approximated by multiplying the induced price change of non-traded goods and the budget shares from the surveys. This methodology has been applied extensively to study the distributional effects of fuel subsidies. Much of this work is summarized in Arze del Granado, Coady, and Gillingham (2012), who present cross-country welfare impacts of an increase in the retail price of fuel products by $0.25 per liter. These authors find substantial welfare effects of the total (direct and indirect) impact of such a price increase. On average, worldwide, household real income would decline by 5.4 percent; there is regional heterogeneity, as expected. The average welfare impact is the highest in Middle East and North Africa (7.4 percent), Africa (5.4 percent), Asia and Pacific (4.5 percent), and South and Central America (3.3 percent). A key finding of this line of work is that the spillovers—the indirect effects—are very important. On average, the indirect impact accounts for 61 percent of the total welfare effect. Price Pass-through So far, the review has focused on a generic analysis of price changes. Often these price changes, especially if they refer to commodities, are generated by external shocks. It is thus important to review the literature on the pass-through of interna- tional prices to the domestic economy, and furthermore to the household. There are various bodies of literature that are relevant here: international economics, interna- tional trade, and economic development. While this collection of theories and empirical analyses play different roles in our framework, we can learn from all three of them. Standard models of international trade and international economics assume com- petitive markets (and homogeneous goods) and frictionless trade. In this scenario, markets are integrated and the law of one price holds: domestic prices are equal to the international prices converted to the local currency. Any difference between these prices is due to transport and distribution costs as well as to trade policy. In this family of models, a proportional change in the exchange rate, in the international price, or in the tariff is fully transmitted to domestic prices. There is, however, strong evidence against this prediction. The papers reviewed by Rogoff (1996) consistently reject the law of one price for a variety of products and countries. Lederman and Porto 183 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 The second type of impact arises when distribution costs change. Distribution ac- tivities are mostly non-tradable services, which require factors of production such as labor and land. If factor prices react to changes in international prices or exchange rates, then distribution costs will themselves change, and this increase can feed into a larger-than-proportional increase in the domestic price of imported goods.13 There is evidence that this mechanism can be relevant in practice. In Argentina, for instance, Burstein, Neves, and Rebelo (2003) show that the distribution margin in both whole- sale and retail is around 60 percent. Using calibration methods to compute the impact of a stabilization program (i.e., they study a decline in the exchange rate), the authors find that the increase in the relative prices of non-tradable goods can be 60 percent higher in a model with distribution costs than in a model without such costs. With perfect pass-through, Burstein et al. (2003) assume that the prices of trad- able goods decrease in the same proportion as the decrease in the exchange rate, and this affects the prices of non-tradable goods. Without distribution costs, the authors calibrate an increase in the relative price of non-tradable goods of 15.4 percent. With distribution costs, the increase in the price of non-tradables is 25.7 percent. Another strand of literature is the work on exchange-rate pass-through (ERPT), which is defined as the percentage change in import prices resulting from a 1 percent change in the exchange rate. ERPT models are often associated with imper- fect competition in product markets (but they are consistent with perfect competition as well). Goldberg and Knetter (1997) identify two conditions for full pass-through: constant mark-ups, and constant marginal costs. These conditions are unlikely to hold in a given market, and as a result, the literature has found strong evidence of imperfect pass-through. In their review, Goldberg and Knetter (1997) conclude that we should expect a pass-through rate of around 60 percent. Because this paper includes measures of costs in the derivation of the pass-through rate, the results imply that around 40 percent of the change in the exchange rate is actually offset by changes in markups. In a comprehensive analysis of pass-through rates, Campa and Goldberg (2005) find large differences across developed countries. The estimated pass-through rate is 42 percent in the United States, 98 percent in France, 80 percent in Germany, and 46 percent in the United Kingdom. An interesting paper that attempts to incorporate the estimation of tariff-rate pass-through into the trade and poverty literature is that of Nicita (2009). This author argues that while imperfect price transmission has been established in the literature (as our review suggests), most of the work on the impacts of trade on poverty, and more generally on the distributional impacts of pricing policies, implic- itly assume full pass-through. Nicita attempts to fill this gap by estimating tariff-rate pass-through for Mexico; he finds lower pass-through rates for both manufacturing (27 percent) and agriculture (33 percent), and supports these findings by arguing that pass-through is likely to be smaller in a developing country like Mexico due to a lack of domestic transport infrastructure. 184 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 Another important strand of the literature is the work on spatial market integra- tion and the efficiency of arbitrage. In the presence of transport costs, prices across regions differ in the size of the transport costs but should move together due to arbi- trage. In practice, three scenarios can be observed: efficiency, characterized by price differentials that can be perfectly accounted for by transport costs; segmented effi- ciency, characterized by price differentials that are smaller than transport costs (so that markets are competitive but segmented and high transfer costs make trade and arbitrage not profitable); and inefficiency, characterized by price differentials that are larger than transfer costs and thus imply unexploited arbitrage profits (due to market power, for instance). This is relevant for our review because only if markets are efficient will perfect pass-through take place. There are various procedures to test for spatial market efficiency.14 The state-of- the-art method is the parity-bound model of Sexton, Kling, and Karman (1991), Baulch (1997), and Park et al. (2002), among others. The method relies on maximum likelihood methods to estimate a switching regression model for the three states, namely efficiency, segmented efficiency, and inefficiency. As expected, the evidence is mixed. Baulch (1997) finds that rice markets in the Philippines are spatially efficient and that differences in prices levels are due to transaction costs. Instead, Cirera and Arndt (2006) report inefficiencies in maize markets in Mozambique. Notably, these inefficiencies are somewhat reduced by market liberali- zation (but not by road rehabilitation), but tend to persist over time. This indicates imperfection in the pass-through rate of commodity shocks. Household Adjustment in Consumption and Production The discussion has followed the traditional first-order approach to assessing the dis- tributional effects of commodity price changes. A major concern with this approach is that it does not allow households to respond to the changes in the economic envi- ronment. In fact, even when wage adjustments are incorporated, the standard approach takes the net-consumer/net-producer position of the household as “exog- enous.” This is just a datum in the analysis that we can assess using survey data, as we did when we described the dependency of households in Latin America and Africa. The net-position of the household is, however, endogenous when house- holds respond. It is often argued that the first-order approximation can be accurate for small price changes. The “endogeneity” of a household choice in these cases is a second- order concern in the sense of being unimportant. This is true, and there are numer- ous examples where the framework works very well. The evaluation of the Doha Development Agenda is one instance, as explored by Hertel and Winters (2006) or Hoekman and Olarreaga (2007). For larger price changes, such as those that can Lederman and Porto 185 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 be expected from a persistent food-price crisis (or a collapse of global commodity prices), the approximation error grows larger. This section thus explores how large this error can actually become and discusses how the approximation can be im- proved by measuring household responses in consumption and production. A consumer will always lose from an increase in prices, but the losses can only be ameliorated by reducing purchases of the more expensive goods. A producer will instead always lose from a price decline. In this case, adjustments can mitigate the losses by shifting resources to more productive activities. Similarly, a farm that hires labor will lose from higher wages, even if labor is substituted with other production factors. Finally, a worker will lose from a reduction in wages, even if he decreases labor supply to enjoy more leisure. For a farm-household, or more generally in a model where the household partici- pates in multiple markets, adjustments can play a bigger role. In fact, Porto (2008) shows that it is possible for a household to change sides of the market (that is, to become a net-producer after a sufficiently high price increase) so that even an ex ante net-consumer can benefit from a price increase. In what follows, we illustrate various sources of adjustment and discuss attempts to quantify them. Consumption Responses We begin with consumption because research in this area has been prolific. To incorporate consumption responses, research needs to estimate a system of demand elasticities (own- and cross-price elasticities) to identify the pattern of substitution in consumption. The estimation of such a demand system is difficult, however. There is a long tradition in economics in estimating systems of demand equa- tions. The type of estimation method depends critically on the available data. In the early days when only national account data were available, estimation relied on time series of various consumption aggregates and price indices. The search for a utility-consistent empirical model of demand converged to the almost ideal demand system (AIDS) model of Deaton and Muellbauer (1980), which is now the leading framework for studying demand elasticities. The mass-production of household surveys in more recent years—a process in which the World Bank has been instrumental—has broadened the options for demand analysis. However, while expenditure surveys provide detailed accounts of what people consume, they collect little information on prices. In some cases, com- munity questionnaires collect price data at the village level, and this variation can be exploited to identify demand elasticities. For those cases in which the surveys collect information on quantities purchased as well as on expenditures, demand elasticities can be recovered from unit values—the ratio of expenditures over quanti- ty. Deaton (1987, 1988, 1990) developed a model of demand for quantity and 186 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 quality that allows for the extraction of the price signal embedded in the unit values at the level of a market (or cluster of households residing near each other, and thus shopping for consumption goods in the same markets), and these price signals identify the demand elasticities. Because the estimating equations are not necessarily derived from a utility framework (that is, the equations do not form a full-blown AIDS model), the estimated parameters cannot be used to recover the parameters of the utility func- tion. However, they do reveal how household consumption responds to price changes. Deaton’s unit values procedure has been used extensively to examine issues of tax reforms (see, for instance, Deaton (1989b), Deaton and Grimard (1992), and Deaton (1997) for case studies of Thailand, Pakistan, India, and Cote d’Ivoire).15 A nice application of Deaton’s methods to welfare analysis is the research conducted by Friedman and Levinsohn (2002), who studied consumption responses during the Indonesian financial crisis of 1997. These authors compared the welfare effects resulting from a first-order approximation (that is, without substitution effects) with the ones resulting from a second-order approximation (thus including substi- tution effects). However, they did not account for wages or, more generally, income responses. Two major findings are worth emphasizing in the context of the discus- sion about the magnitude of consumption responses to market signals. On the one hand, the losses estimated as first-order approximations for the urban poor were high at roughly 50 percent of the pre-crisis level of per capita expenditures. In rural areas, the losses of the poor were equivalent to nearly 40 percent of their pre-crisis real consumption. On the other hand, allowing for substitution effects in consump- tion made a big difference. The estimated losses were cut by half for the urban poor and by more than 40 percent overall. These results imply that first-order approxi- mations can be quite large when price shocks are large, such as those observed during the Indonesian crisis (when prices increased by 100 percent or more). In cases of economic crisis, when large price changes may be the norm rather than the exception, the results of Friedman and Levinsohn (2002) warn us about the perils of ignoring second-order terms in welfare analysis. In light of the stylized facts presented above, these adjustments need to be taken into account in the context of the recent surge of commodity prices. Farm Supply Responses We turn now to farm supply responses. As throughout all of our analysis, we are in- terested in responses at the household level, and especially for the poorest house- holds.16 Since these households are typically involved in agriculture, much of our review is concerned with agricultural farm responses. Conceptually, the issues are those that we discussed within consumption responses. Faced with changed relative prices, the question is whether there is any adjustment in Lederman and Porto 187 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 production and what factors prevent or facilitate that adjustment. It is perhaps not really surprising to learn that supply responses are somewhat feeble in developing countries. The body of literature is too large for a comprehensive review, but some of most fascinating recent examples include Cadot, Dutoit, and de Melo (2009), who ex- amined vanilla-market reforms in Madagascar, and McMillan, Welch, and Rodrik (2003), who studied cashews in Mozambique. The evidence on supply responses is not always so bleak, however, as two strands of literature illustrate. First, a common finding in empirical work is that sizeable responses can be expected when price changes are accompanied by appropriate complementary factors such as access to inputs, information, and credit (McKay, Morrissey, and Vaillant 1997), productive assets like animals and tools (Deininger and Olinto 2000), productive capital, education, and land quality (Lo ´ pez, Nash, and Stanton 1995; Heltberg and Tarp 2002). Second, there is a growing body of literature showing responses to marketing reforms under successful contract farming arrangements. Two examples illustrate the point: Brambilla and Porto (2011) estimate large adjustments in cotton production after changes in the outgrower contract environment in Zambia, and Maertens and Swinnen (2009) estimate similarly large responses in the vegetable sector in Senegal. Another way to explore these issues is to look at the performance of subsistence farmers relative to commercial farmers. Subsistence farmers produce for home con- sumption, whereas commercial farmers earn their income by selling their produce in markets. The evidence indicates that commercial farming is associated with higher consumption and incomes than subsistence farming. Cadot, Dutoit, and Olarreaga (2006) estimate an average loss in income of those in subsistence farming of 43 percent relative to commercial farmers that produce the same crop. Even with much smaller margins, what is puzzling is why subsistence farmers do not transform themselves into commercial farmers by selling their harvest on the market? This question is essentially about supply responses to market signals. Cadot, Dutoit, and Olarreaga (2010) argue that there are barriers to exit from subsistence agriculture, but the issue highlights explanations of why supply re- sponses to market signals might be sluggish. There are two major barriers here. One is risk: cash crops are riskier than food crops so that part of the difference in return is actually a risk premium (Fafchamps 1992; Rosenzweig and Binswanger 1993). The other barrier is related to missing markets: the differential rate of return is due in part to the shadow price of a good with missing markets such as for labor or food crops (de Janvry, Fafchamps, and Sadoulet 1991). Notice that missing markets can arise due to variable transaction and transportation costs, fixed trans- action costs (searching for partners, enforcing contracts with distant buyers, estab- lishing quality), or sunk costs of transacting. One example is Key, de Janvry, and Sadoulet (2000), who illustrate how supply responses are affected by transaction costs. Another interesting example is Cadot, Dutoit, and Olarreaga (2006), who 188 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 show that sunk costs can be particularly large, ranging from 124 to 153 percent of the market value of the average annual farm output in Madagascar. We end this section with a caveat. Positive and large supply responses to price changes are always an indication that economic incentives work. However, in the context of our analytical framework above, these supply responses comprise second- order effects. In consequence, they are likely to generate only small welfare impacts because of the opportunity cost of the resources used in the production of different goods. While this argument does not invalidate the search for supply responses in the data, it strongly argues for a careful interpretation of the findings. Quantifying The Mechanisms: The Case of Mexico In this section we use data from Mexico to quantify the different channels uncov- ered by our review of the literature. The data come from the 2006 Household Income and Expenditure National Survey, ENIGH (Encuesta Nacional de Ingresos y Gastos de los Hogares). The main objective is to illustrate how the mechanisms work. To do this, we work with a (hypothetical) increase of 20 percent in corn prices in rural Mexico. Our results are reported in table 4. We begin with the first-order consumption and production effects and the net- consumer-net-producer proposition. As discussed in our framework above, the con- sumption effects are calculated as the product of the budget share spent on corn and the 20 percent price increase. The production effects are the product of the income Table 4. Welfare Effects of Price Changes, Examples from Mexico Q1 Q2 Q3 Q4 Q5 All Net Producers Net Consumers First-order Effects Consumption effects 2 3.56 2 2.53 2 1.99 2 1.40 2 0.73 2 1.99 2 0.99 2 2.70 Production effects 2.25 1.86 1.65 1.49 1.12 1.60 2.78 0.77 Net effects 2 1.31 2 0.67 2 0.34 0.09 0.38 2 0.39 1.78 2 1.93 Segmented Labor Markets Income effects 5.86 4.84 4.28 3.87 2.90 4.16 7.22 1.99 Net effects 2.30 2.31 2.29 2.47 2.17 2.17 6.23 2 0.71 Labor Market Spillovers Income effects 8.67 7.21 6.45 5.86 4.78 6.31 10.77 3.14 Net effects 5.10 4.68 4.47 4.46 4.04 4.32 9.78 0.44 Second-order Effects Consumption effects 2 3.34 2 2.37 2 1.87 2 1.31 2 0.69 2 1.8 2 0.93 2 2.53 Production effects 2.48 2.05 1.81 1.64 1.23 1.76 3.05 0.84 Net effects 2 0.87 2 0.33 2 0.06 0.32 0.54 2 0.11 2.12 2 1.69 Note: Authors’ own calculations based on Porto (2008). Lederman and Porto 189 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 share of corn production and the 20 percent price increase. The net effect is the dif- ference between the production share and the consumption share. For the average Mexican household, an increase in corn prices brings only small welfare losses (equivalent to 0.39 percent of average national income). As expected, there is hetero- geneity across quintiles. The largest losses are at the first quintile (–1.31 percent), while quintiles 4 and 5 instead slightly benefit from higher prices. In fact, for these households, the average production effect is larger than the average consumption effect so that poorer households tend to be net consumers of corn, while richer households tend to be net-producers of corn. This illustrates the net-producer/ net-consumer proposition. We can also explore this directly by dividing households into net-producers (those that produce more corn than they consume, independent of the quintile) and net-consumers. The average net producer would enjoy gains equivalent to 1.78 percent of their initial (i.e., pre-price increase) average expendi- ture. Net-consumers would instead suffer a loss of 1.93 percent. To incorporate labor market responses, recall that the impacts depend on how wages respond to price changes, and this in turn depends on whether we think that labor markets are integrated or segmented, and upon whether there are spillovers and backward or forward linkages (Ravallion 1990; Porto 2005; Porto 2006). Continuing with the case of corn in Mexico, we provide two examples in table 4. In the second panel, we assume that labor markets are “segmented”, and thus we only allow wages in the agricultural sector to respond to corn prices. In this case, we use an agricultural wage elasticity with respect to corn prices of 0.40 (as estimated in Porto 2008). Consequently, an increase in corn prices of 20 percent translates into (roughly) 8 percent higher wages in agriculture. This, in turn, represents a wage- income effect on household welfare that depends on the share of total household income accounted for by agricultural wages. It should be pointed out that we only have information on wage income, and not on labor expenses. Thus, the results from this experiment may be biased upwards. The evidence in Porto (2005) for the Moldovan case suggests that these biases are, however, not very large. For the average Mexican household, allowing for wage effects induces positive overall welfare effects: higher corn prices would bring welfare increases of 2.17 percent. The average income effects (which includes both the direct production effects of 1.60 and the wage effect) would be 4.16 percent. Since the poor are also ag- ricultural workers, the welfare losses observed above for quintiles 1 to 3 are turned into average welfare gains due to the additional impacts on wages. In fact, the net effect in this (hypothetical) scenario would be roughly similar across the entire income distribution.17 The income gains of net-producers jump to 7.22 percent and their net gain becomes 6.23 percent—nearly 3.5 times higher than before. For net- consumers the income gains are more modest at around 1.99 percent, and these gains are not enough to offset the consumption losses. In the end, even with wage responses, net-consumers would be made worse off by the rise in corn prices. 190 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 In another example, in the third panel of table 4, we allow for spillovers from corn prices to the wages of self-employed individuals (in rural areas). The idea is that increases in agricultural prices may increase derived demand for labor in ser- vices, odd-jobs and, more generally, in local rural labor markets (not exclusively in agriculture). For the calculations, we use the same wage-price elasticity of 0.40 as before (Porto, 2008). The welfare effects of this expanded wage-income channel thus capture the increase in wages of 8 percent (20 percent price increase times the 0.4 wage-price elasticity) filtered through the share of wage income in total house- hold income. We estimate the following welfare impacts: the average national gain would be equivalent to 4.32 percent on initial income; poorer households benefit relatively more than richer households; the income gain of net-producers would be equivalent to 10.22 percent of their initial income, and the net gain would be 9.78 percent; net-consumers would also have an income gain of 3.44 percent and a net gain of 0.44 percent. As argued above, the net position of the household can be endogenous and it can thus change in response to a price shock. Consider, for example, an increase in the price of corn and let households be able to adjust. If consumption responses are suf- ficiently strong (because households can substitute easily to other food products) and/or if supply responses are large enough, then a net consumer can in principle become a net producer, and thus benefit from the increase in corn prices. The stan- dard argument against this caveat is that second-order adjustments of the type de- scribed here are often small and thus unlikely to change the implications of the first-order calculations. An example is shown in the fourth panel of table 4, where we estimate consumption substitutions and supply responses. To do this, we use an own-price consumption elasticity of corn of –0.61 (as estimated by Porto 2008) and we assume a unitary corn supply elasticity.18 To simplify the exposition, we ignore here the wage impacts. As the results show, allowing for consumption and production second-order effects does not, in fact, affect the results. The gains for net-producers are slightly larger and the losses for net-consumers are slightly smaller, but the welfare impacts are not affected much. Conclusions This paper provided an overview of the impacts of price changes on household welfare. We began by exploring commodity dependence in the data and by looking at expenditure and income dependence in the household surveys from Latin American and Africa. Our analysis suggests a strong dependence on commodity prices, a dependence that is biased towards the poor, especially on the consumption side. The paper then addressed the conceptual issues related to commodity depen- dence. It did so by laying out a theoretical framework, which described how Lederman and Porto 191 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 changes in commodity prices affect consumption, production, wages, and house- hold adjustments. Each of these aspects of the analysis of commodity dependence is illustrated with evidence from the existing literature. Our review tells the story of heterogeneous responses to commodity price changes. Most shocks to commodity prices are international in nature and the pass- through to domestic prices is imperfect (that is, changes in international prices are not translated one-to-one to domestic prices). The pass-through is also heteroge- neous across countries and depends on infrastructure, institutions, and market structure (the nature of imperfect competition in domestic markets). Faced with a price shock, households are affected on both the consumption and income sides. On the consumption side, higher prices cause welfare losses. In Latin America and particularly in Africa, higher food prices can cause large welfare losses, especially for the poor. On the income side, rising commodity prices can bring gains or losses, depending on which goods are produced and also on how wages respond to those price changes. The evidence shows that households tend to earn a significant share of their total income from commodities, so the consump- tion losses can be ameliorated, sometimes to a large extent. This is most likely to occur when wages and spillovers from commodities to other sectors are taken into account, particularly if wages respond positively to increases in labor demand in primary sectors, and if the spillovers to non-primary sectors are positive (via either production or expenditure linkages). In the end, the overall welfare impacts depend on the net-consumer-net-producer position of the household, after taking into account wage income. Net-consumers will be hurt by price increases, and net- producers will benefit. As expected, these impacts are heterogeneous across coun- tries. Some countries will gain and some others will lose. Also, in a country that gains from higher prices, there will be households that gain and households that lose out. Conversely, there will be winners and losers within countries that lose as an aggregate. Household adjustments can work as a cushion protecting against price changes. Given a price increase in commodities, consumption responses can ameliorate con- sumption losses and production responses can boost income gains. Typical adjustments under the separability assumption appear not to be enough to overturn the initial impacts derived from a static view of commodity dependence. Non-separability, which is likely to be a real feature in many developing countries, can work in both directions. Sometimes, market failures can shut down markets completely so that price changes can become irrelevant for many households. While this can work as a protective device, it can also hinder the realization of any efficiency gains and of expanded market opportunities. Sufficiently large price changes in the presence of market failures can generate discontinuities in household responses that can potentially multiply the typi- cally small responses found in the empirical literature—in the real world, the price is not always right. 192 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 We end with some caveats. First, due to space limitations, our review has focused on welfare impacts and we have neglected issues of insurance, coping mechanisms, and social protection policies. Second, the review centered the discussion on one- shot price shocks. In particular, we have not covered the literature on the effects of commodity price volatility. In addition, even in the case of one-shot price shocks, we should note that there might be hysteresis in the effects. In other words, a price shock may have long-term implications, even if the price change reverts at a later date (Baird, Friedman, and Schady 2011). These themes require careful consider- ation in future assessments of the developmental implications of price signals. Notes Laura Chioda, Louise Cord, Irene Brambilla, and Emily Sinnott provided insightful comments during the early stages of this research project. We are grateful to Emmanuel Jiminez and three anonymous referees for detailed comments on a previous version of this article. We are especially grateful to Cecilia Peluffo and German Bet for their excellent research assistance. All remaining errors are our responsibility. 1. World Bank. Email: dlederman@worldbank.org. 2. Universidad Nacional de La Plata, Calle 6 e 47 y 48, Oficina 521, La Plata 1900, Argentina. Email: guido.porto@depeco.econo.unlp.edu.ar. 3. Data taken from the Global Economic Monitor (GEM) Commodities, found at http:// databank.worldbank.org/data/databases/commodity-price-data. The data presented in the text was accessed on February 7, 2014. 4. World Bank, Press Release No:2011/559/EXT, Washington, DC. 5. See, for example, “Life after the Commodity Boom,” The Economist, March 29, 2014. 6. The Latin American surveys are from Socio-Economic Database for Latin America and the Caribbean (CEDLAS and The World Bank). See http://sedlac.econo.unlp.edu.ar/eng/statistics.php. The African surveys used in the analysis are as follows: Burundi (1998), Enque ˆ te Prioritaire; Benin (2003), Questionnaire des Indicateurs de Base du Bien-e ˆ tre; Burkina Faso (2003), Enque ˆ te Burkinabe sur les Conditions de Vie des Me ´ nages; Co ˆ te d’Ivoire (2002), Enque ˆ te Niveau de Vie Me ´ nages; Cameroon (2001), Deuxieme Enquete Camerounaise Aupre ` s des Me ´ nages; Ethiopia (2000), Household Income, Consumption and Expenditure Survey; Ghana (1998), Ghana Living Standards Survey; Gambia (1998), Household Economic Survey; Guine-Bissau (2002), Questiona ´ rio de Indicadores Ba ´ sicos de Bem-estar; Kenya (1997), Welfare Monitoring Survey III; Madagascar (2005), Enquete Pe ´ riodique Aupre ` s des Me ´ nages; Mali (2001), L’Enque ˆ te Malienne d’Evaluation de la Pauvrete; Malawi (2004), Integrated Household Survey; Nigeria (2004), Nigeria Living Standards Survey; Rwanda (1998), Enque ˆ te Integrale sur les Conditions de Vie des Me ´ nages; Senegal (2001), Enque ˆ te Senegalaise Aupre ` s des Me ´ nages; Tanzania (2000), Household Budget Survey; Uganda (2005), Uganda National Household Survey; South Africa (2000), Income and Expenditure Survey; Zambia (2003), Living Conditions Monitoring Survey III. See Nicita, Olarrega and Porto (2009) for a detailed description of the surveys. 7. Let si;t be the commodity income share of household i at time t. The cross-section model for t is si;t ¼ x0 i;t bt þ 1i;t . For t þ 1, the model for household j is sj;tþ1 ¼ x0 j;tþ1 btþ1 þ 1j;tþ1 , where we use j instead of i to highlight the fact that these are repeated cross-sections, not panels. We denote b b tþ1 the OLS estimates of btþ1 . From the empirical distribution of the residuals, let e 1i;tþ1 be the draw attached to household i. With these estimates, the prediction for the income commodity shares for household j at time t is e b tþ 1 þ e si;tþ1 ¼ x0 i;t b 1i;tþ1 . Note that, since we are taking averages, the residuals vanish (or are very small). Consequently, the lower and upper bounds in Dang, Lanjouw, Luoto, and McKenzie (2011) are very close in our experiment. Below, we report the lower boundary. Lederman and Porto 193 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 8. An alternative method is to start with the indirect utility function (see Deaton 1997). 9. See Barnum and Squire (1979) and Singh, Squire, and Strauss (1986) for a theoretical discussion of the principle of separability. Benjamin (1992) tests for separability in Indonesia and cannot reject it. 10. To further simplify the exposition, we also assume separability between consumption and leisure in utility. 11. Notice, however, that this analysis does not take into account the fiscal implications of elimi- nating the tax. 12. Notice that we are referring here to prices of tradable goods. For non-traded goods, the issues are more complex since they are endogenous. 13. Note that this is a type of spillover as those discussed above. 14. See, for instance, Fackler and Goodwin (2002). 15. In short, the difference is that compensated demand elasticities are needed for welfare analysis, and uncompensated elasticities are needed to analyze tax reforms. 16. We do not cover aggregate supply responses in our review. 17. We thank a referee for pointing this out. 18. The formulas for the calculations of the second-order effects are in Porto (2008). References Arze del Granado, F., D. Coady, and R. Gillingham. 2012. “The Unequal Benefits of Fuel Subsidies: A Review of Evidence for Developing Countries.” World Development 40 (11): 2234–48. Baird, S., J. Friedman, and N. Schady. 2011. “Aggregate Income Shocks and Infant Mortality in the Developing World.” Review of Economics and Statistics 93 (3): 847–56. Barnum, H., and L. Squire. 1979. “A model of an Agricultural Household. Theory and Evidence.” Ocassional Papers No. 27, World Bank, Washington, DC. Barrett, C., and P. A. Dorosh. 1996. “Farmers’ Welfare and Changing Food Prices: Non-parametric Evidence from Rice in Madagascar.” American Journal of Agricultural Economics 78 (3): 656– 69. Baulch, B. 1997. “Transfer Costs, Spatial Arbitrage, and Testing for Food Market Integration.” American Journal of Agricultural Economics 79 (2): 477–87. Benjamin, D. 1992. “Household Composition, Labor Markets, and Labor Demand: Testing for Separation in Agricultural Household Models.” Econometrica 60 (2): 287 –322. Benjamin, D., and A. Deaton. 1993. “Household Welfare and the Pricing of Cocoa and Coffee in Co ˆ te d’Ivoire: Lessons from the Living Standards Surveys.” The World Bank Economic Review 7: 293 –318. Boyce, J., and M. Ravallion. 1991. “A Dynamic Econometric Model of Agricultural Wage Determination in Bangladesh.” Oxford Bulletin of Economics and Statistics 53 (4): 361 –76. Brambilla, I., and G. Porto. 2011. “Market Structure, Outgrower Contracts and Farm Output. Evidence from Cotton Reforms in Zambia.” Oxford Economic Papers 63 (4): 740–66. Budd, J. W . 1993. “Changing Food Prices and Rural Welfare: a Nonparametric Examination of the Coˆ te d’Ivoire.” Economic Development and Cultural Change 41: 587 –603. Burstein, A., J. Neves, and S. Rebelo. 2003. “Distribution Costs and Real Exchange Rate Dynamics During Exchange-rate-based Stabilizations.” Journal of Monetary Economics 50: 1189–214. Cadot, O., L. Dutoit, and M. Olarreaga. 2006. “The Cost of Moving Out of Subsistence.” Mimeo, University of Lausanne, France. . 2010. “Barriers to Exit from Subsistence Agriculture.” In G. Porto, and B. Hoekman, eds., Trade Adjustment Costs in Developing Countries: Impacts, Determinants and Policy Responses. Washington, DC: CEPR and World Bank. 194 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 Cadot, O., L. Dutoit, and J. de Melo. 2009. “The Elimination of Madagascar’s Vanilla Marketing Board, 10 Years On.” Journal of African Economics 18 (3): 388 –430. Campa, J. M., and L. S. Goldberg. 2005. “Exchange Rate Pass-through into Import Prices.” The Review of Economics and Statistics 87(4): 679–90. Cirera, X., and C. Arndt. 2008. “Measuring the Impact of Road Rehabilitation on Spatial Market Efficiency in Maize Markets in Mozambique.” Agricultural Economics 39 (1): 17 –28. Coady, D., M. El-Said, R. Gillingham, K. Kpodar, P. Medas, and D. Newhouse. 2006. “The Magnitude and Distribution of Fuel Subsidies: Evidence from Bolivia, Ghana, Jordan, Mali, and Sri Lanka.” IMF Working Paper No. 06/247. Washington, DC: International Monetary Fund. Coady, D., and D. Newhouse. 2006. “Ghana: Evaluating the Fiscal and Social Costs of Increases in Domestic Fuel Prices.” In A. Coudouel, A. Dani, and S. Paternostro, eds., Poverty and Social Impact Analysis of Reforms, pp. 387–413. Washington, DC: World Bank. Dang, H., P. Lanjouw, J. Luoto, and D. McKenzie. 2011. “Using Repeated Cross-sections to Explore Movements In and Out of Poverty.” Policy Research Working Paper Series 5550, The World Bank, Washington, DC. De Ferranti, D., G. Perry, D. Lederman, A. Valdes, and W. Foster. 2005. Beyond the City: The Rural Contribution to Development in Latin America and the Caribbean. World Bank, Washington, DC. De Janvry, A., M. Fafchamps, and E. Sadoulet. 1991. “Peasant Household Behaviour with Missing Markets: Some Paradoxes Explained.” Economic Journal 101: 1400–17. Deaton, A. 1987. “Estimation of Own- and Cross-Price Elasticities From Household Survey Data.” Journal of Econometrics 36: 7–30. . 1988. “Quality, Quantity, and Spatial Variation of Price.” American Economic Review 78 (3): 418–30. . 1989a. “Rice Prices and Income Distribution in Thailand: A Non-Parametric Analysis.” Economic Journal 99 (395): 1–37. . 1989b. “Household Survey Data and Pricing Policies in Developing Countries.” The World Bank Economic Review 3: 183 –210. . 1990. “Price Elasticities from Survey Data.” Journal of Econometrics 44: 281–309. . 1997. The Analysis of Household Surveys. A Microeconometric Approach to Development Policy. Baltimore and London: John Hopkins University Press for the World Bank. Deaton, A., and J. Muellbauer. 1980. “An Almost Ideal Demand System.” American Economic Review 70: 312–36. Deaton, A., and F. Grimard. 1992. “Demand Analysis and Tax Reform in Pakistan.” Living Standards Measurement Study. Working Paper No. 85, The World Bank, Washington, DC. Deininger, K., and P . Olinto. 2000. “Why Liberalization Alone Has Not Improved Agricultural Productivity in Zambia: The Role of Asset Ownership and Working Capital Constraints.” World Bank Working Paper No. 2302, Washington, DC. Delgado, C. 1996. “Agricultural Transformation: The Key to Broad-based Growth and Poverty Alleviation in Africa.” In B. Ndulu, and N. van de Walle, eds., Agenda for Africa’s Economic Renewal. New Brunswick, NJ: Transaction Publishers. Dixit, A., and V. Norman. 1980. Theory of International Trade. A Dual, General Equilibrium Approach. Cambridge: Cambridge Economic Handbooks. Fackler, P. L., and B. K. Goodwin. 2002. “Spatial Price Analysis.” In B. L Gardner, and G. C. Rausser, eds., Handbook of Agricultural Economics. Amsterdam: Elsevier Science. Fafchamps, M. 1992. “Cash Crop Production, Food Price Volatility, and Rural Market Integration in the Third World.” American Journal of Agricultural Economics 74 (1): 90 –9. Lederman and Porto 195 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 Ferreira, F., A. Fruttero, P. Leite, and L. Lucchetti. 2011. “Rising Food Prices and Household Welfare Evidence from Brazil in 2008.” World Bank Policy Working Paper No. 5652, Washington, DC. Friedman, J., and J. Levinsohn. 2002. “The Distributional Impacts of Indonesia’s Financial Crisis on Household Welfare: A ‘Rapid Response’ Methodology.” World Bank Economic Review 16: 397 –423. Goldberg, P ., and M. Knetter. 1997. “Goods Prices and Exchange Rates: What Have We Learned?” Journal of Economic Literature 35: 1243– 72. Goldberg, P., and N. Pavcnik. 2004. “Trade, Inequality, and Poverty: What Do We Know? Evidence from Recent Trade Liberalization Episodes in Developing Countries.” Brookings Trade Forum 2004: 223 –69. . 2007. “Distributional Effects of Globalization in Developing Countries.” Journal of Economic Literature 45 (1): 39 –82. Harrison, A., J. McLaren, and M. McMillan. 2011. “Recent Perspectives on Trade and Inequality.” Policy Research Working Paper 5754, The World Bank, Washington, DC. Harriss, B. 1987. “Regional Growth Linkages from Agriculture.” Journal of Development Studies 23 (2): 275 –89. Heltberg, R., and F. Tarp. 2002. “Agriculture Supply Response and Poverty in Mozambique.” Food Policy 27 (2): 103 –24. Hertel, T., and A. L. Winters, eds. 2006. Poverty Impacts of the Doha Development Agenda. Washington, DC: Palgrave-McMillan for the World Bank. Hoekman, B., and M. Olarreaga, eds. 2007. Global Trade and Poor Nations: Poverty Impacts and Policy Implications of Liberalization. Washington, DC: Brookings Institution Press. Ivanic, M., and W. Martin. 2008. “Implications of Higher Global Food Prices for Poverty in Low- Income Countries.” World Bank Policy Research Working Paper Series No. 4594, Washington, DC. Jacoby, H. 2013. “Food Prices, Wages, and Welfare in Rural India.” World Bank Policy Research Working Paper No. 6412, Washington, DC. Key, N., E. Sadoulet, and A. de Janvry. 2000. “Transactions Costs and Agricultural Household Supply Response.” American Journal of Agricultural Economics 82: 245 –59. Lederman, D. 2013. “International Trade and Inclusive Growth: A Primer.” Indian Growth and Development Review 6 (1): 88 –112. ´ pez, R., J. Nash, and J. Stanton. 1995. “Adjustment and Poverty in Mexican Agriculture: How Lo Farmers’ Wealth Affects Supply Response.” World Bank Policy Research Working Paper No. 1494, Washington, DC. Maertens, M., and J. Swinnen. 2009. “Trade, Standards, and Poverty: Evidence from Senegal.” World Development 37 (1): 161 –78. McKay, A., O. Morrissey, and C. Vaillant. 1997. “Trade Liberalization and Agricultural Supply Response: Issues and Some Lessons.” European Journal of Development Research 9 (2): 129–47. McMillan, M., K. Welch, and D. Rodrik. 2003. “When Economic Reform Goes Wrong: Cashews in Mozambique.” Brookings Trade Forum 97 –151. Mellor, J., and B. Johnston. 1984. “The World Food Equation: Interrelations among Development, Employment and Food Consumption.” Journal of Economic Literature 22 (2): 524 –31. Nicita, A. 2009. “The Price Effect of Trade Liberalization: Measuring the Impacts on Household Welfare.” Journal of Development Economics 89 (1): 19 –27. Nicita, A., M. Olarrega, and G. Porto. 2009. “The Poverty Bias of Trade Policy in and Against Sub- Saharan Africa.” Mimeo, University of Geneva, Switzerland. . 2014. “Pro-Poor Trade Policy in Sub-Saharan Africa.” Journal of International Economics 92 (2): 252– 65. 196 The World Bank Research Observer, vol. 31, no. 1 (February 2016) Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 Park, A., H. Jin, S. Rozelle, and J. Huang. 2002. “Market Emergence and Transition Arbitrage, Transaction Costs and Autarky in China’s Grain Markets.” American Journal of Agricultural Economics 84 (1): 67 –82. Porto, G. 2005. “Informal Export Barriers and Poverty.” Journal of International Economics 66:447 –70. . 2006. “Using Survey Data to Assess the Distributional Effects of Trade Policy.” Journal of International Economics 70: 140 –60. . 2008. “Estimating Household Responses to Trade Reforms. Net Consumers and Net Producers in Rural Mexico.” Policy Research Working Paper No. 3695, World Bank, Washington, DC. Ravallion, M. 1990. “Rural Welfare Effects of Food Price Changes Under Induced Wage Responses: Theory and Evidence for Bangladesh.” Oxford Economic Papers 42: 574–85. Rogoff, K. 1996. “The Purchasing Power Par- ity Puzzle?” Journal of Economic Literature 34 (2): 647–68. Rosenzweig, M., and H. Binswanger. 1993. “Wealth, Weather Risk and the Profitability of Agricultural Investment.” Economic Journal 103: 56 –78. Sahn, D., and A. Sarris. 1991. “Structural Adjustment and the Welfare of Rural Smallholders: A Comparative Analysis from Sub-Saharan Africa.” World Bank Econonomic Review 5: 259 –89. Sexton, R., C. Kling, and H. Carman. 1991. “Market Integration, Efficiency of Arbitrage and Imperfect Competition: Methodology and an Application to US Celery.” American Journal of Agricultural Economics 7: 568 –80. Singh, I., L. Squire, and J. Strauss, eds. 1986. Agricultural Household Models: Extensions, Applications and Policy. Baltimore, MD: Johns Hopkins Press for the World Bank. Winters, A., N. McCulloch, and A. McKay. 2004. “Trade Liberalization and Poverty: The Evidence So Far.” Journal of Economic Literature 42: 72 –115. Wodon, Q., C. Tsimpo, P. Backiny-Yetna, G. Joseph, F. Adoho, and H. Coulombe. 2008. “Potential Impact of Higher Food Prices on Poverty: Summary Estimates for a Dozen West and Central African Countries.” World Bank Policy Research Working Paper Series No. 4745, Washington DC. Lederman and Porto 197 Downloaded from https://academic.oup.com/wbro/article-abstract/31/1/168/2461850/The-Price-Is-Not-Always-Right-On-the-Impacts-of by Joint Bank-Fund Library user on 06 October 2017 T H E W O R L D B A N K 1818 H Street NW Washington, DC 20433, USA World Wide Web: http://www.worldbank.org/ E-mail: researchobserver@worldbank.org Downloaded from https://academic.oup.com/wbro/issue/31/1 by Joint Bank-Fund library user on 06 October 2017