THE WORLD BANK 3 BSERVER i17526 USING COMPARISON GROUPS TO EVALUATE SOCIAL PROGRAMS Evaluating Social Policies: Principles and U.S. Experience Jean Baldwin Grossman Using Randomized Control Designs in Evaluating Social Sector Programs in Developing Countries John Newman, Laura Rawlings, and Paul Gertler AIDS and African Development Martha Ainsworth and Mead Over Privatization: Lessons from Market Economies Sunita Kikeri, John Nellis, and Mary Shirley The Costs and Benefits of Soil Conservation: The Farmer's Viewpoint Ernst Lutz, Stefano Pagiola, and Carlos Reiche WITH CUMULATIVE INDEX THE WORLD BANK RESEARCH OBSERVER VOLUME 9 NUMBER 2 JULY 1994 USING COMPARISON GROUPS TO EVALUATE SOCIAL PROGRAMS Evaluating Social Policies: Principles and U.S. Experience Jean Baldwin Grossman 159 Using Randomized Control Designs in Evaluating Social Sector Programs in Developing Countries John Newman, Laura Rawlings, and Paul Gertler 181 AIDS and African Development Martha Ainsworth and Mead Over 203 Privatization: Lessons from Market Economies Sunita Kikeri, John Nellis, and Mary Shirley 241 The Costs and Benefits of Soil Conservation: The Farmer's Viewpoint Ernst Lutz, Stefano Pagiola, and Carlos Reiche 273 Cumulative Index, 1986-1994 297 I USING COMPARISON GROUPS TO EVALUATE SOCIAL PROGRAMS Governments in the United States have pioneered in the systematic evaluation of social programs using the method of comparison groups. Such evaluations are often mandated at the time of project approval. Social scientists have provided policymakers with reliable techniques to assess the effectiveness of project designs and have helped to apply those techniques in specific instances. The two papers that follow discuss how and how not to evaluate the effectiveness of social programs. They show how the principles devel- oped and applied in the United States can be transformed and transferred to evaluations in developing countries. This discussion is not wholly speculative; as Newman, Rawlings, and Gertler make clear, there is already significant experience with systematic evaluation in developing countries. Both papers are concerned with practicalities as well as prin- ciples and discuss such questions as: What type of projects can be evalu- ated using comparison groups? What is the role of pilot projects in evaluation? Who should be compared with the project participants? When is a random draw of participants feasible and desirable? How large do participant and comparison groups need to be? How long should the project be monitored? Everyone is interested in getting value for money. These papers show how evaluation can help social sector programs to be successful the world over. EVALUATING SOCIAL POLICIES: PRINCIPLES AND U.S. EXPERIENCE jean Baldwin Grossman Invariably, studies, proposals, and plans for social programs contain a strong recommendation for evaluation and monitoring. Reliable informnation about what works and why is clearly vital for improving existing pr0ograms or design- ing future ones. Making such assessments requires effective methods of evalua- tion. Policymakers who use these evaluations need to know about the methods- the pitfalls to watch for and the relative advantages and disadvantages of differ- ent techniques in different situations. This article describes these evaluation methods and the experience accumulated in the United States in applying them in practice. Does a particular social program have the effect it is intended to have? If not, why not? Policymakcrs need answers to these questions if they are to make the most effective use of limited resources to advance so- cial goals. In developing countries in particular, governments and development institutions cannot afford to waste scarce resources on programs that do not achieve their goals. Ineffective programs should be modified to make them work better or be canceled altogether. Thus, evaluating or monitoring perfor- mance is often strongly recommended and may even be a requirement of a pro- gram's funding. But policymakers often know very little about how-and equally important, how not-to assess a program's effectiveness, and therefore cannot influence or adequately judge the quality of the evaluation methods proposed. Every newly initiated program is designed and fully expected to work. Most program designers, funders, and operators believe their programs do help. They can al- ways point to particular individuals or communities whose conditions im- proved. Anecdotal evidence is often very persuasive, especially if one is inclined The World Bank Research Observer, vol. 9, no. 2 (July 1954), pp. 159-3 O 1954 The International Bank for Reconstruction and Development/THE WORLD BANK 159 to believe a particular program is working-as a program's creator would be. But did the program cause the improvement? Would the situation of these individuals or communities have improved anyway, with help from some other source? Did the size or extent of the improvement justify the program's cost? An evaluation technique that can accurately address these issues arms policymakers with the information they need to determine whether a program is worth the cost. For example, evaluations of a nutrition program for pregnant women, infants, and children in the United States established that the program was indeed saving the country substantial amounts of money: "For every dollar spent on [the program], the associated saving in [government-funded medical costs] during the first sixty days [postpartum] ranged between $1.77 to $3.13" (Devaney, Bilheimer, and Shore 1992, Executive Summary). Similarly, an eval- uation of a residential education and training program for high school drop- outs in the United States found that, even though the program was relatively expensive, society more than recouped the costs through increased work effort by the participants, increased tax revenues, lower crime rates, and lower social service costs (Mallar and others 1982). Findings such as these from rigorous evaluations provide a reliable foundation on which to base decisions on wheth- er to continue, modify, or terminate a program. These decisions can save coun- tries or communities money that otherwise would be wasted on a mistargeted program or lost in closing down a program whose cost-effectiveness was not immediately apparent. A rigorous evaluation can also guide the modification needed to improve program delivery. For example, during a particularly deep and long recession, the U.S. government often provides unemployed workers with extra unemploy- ment compensation. An evaluation of such an emergency compensation pro- gram in the late 1970s (Brewster and others 1978) found that workers did not actually receive much of the money until after the economy began to recover. The rules for distributing the money were accordingly changed when a com- pensation scheme was next needed during the recession of the early 1980s. An evaluation of the modified program indicated that the new rules allowed pol- icymakers to deliver the aid more precisely when it was needed most. How does one set up an evaluation that will generate reliable and useful information? The answer can be informed by what has been learned elsewhere. For policymakers and evaluators who must weigh the advantages and disad- vantages of alternative techniques, U.S. experimentation in this area can pro- vide valuable guidance. Researchers in the United States have expanded and refined the theory of evaluation and have field-tested several different methods. This article draws on this accumulated experience to lay out some of the strat- egies that have been developed, describing why some have fallen out of favor and why the value of others is being questioned. The analysis goes-on to con- sider what conditions are conducive to each of the principal methods currently in use.1 160 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Evaluation Techniques A program's effect can be measured accurately only if one knows what would have happened without it. Because one obviously cannot observe the outcomes for the participants themselves had they not enrolled in the program, a proxy group of nonparticipants must be identified. Determining this hypo- thetical no-treatment (or counterfactual) state is the crux of designing an eval- uation because, under any strategy, a program's effects are ascertained by comparing the behavior of the treatment or participant group with the behav- ior of the selected counterfactual group. Indeed, determining the no-treatment state is so central to an evaluation that designs are categorized according to the way in which the counterfactual group is selected-nonrandom assignment (classified as a quasi-experimental design) and random assignment (classified as an experimental design). In the quasi- experimental category the two principal types of design are reflexive tech- niques, in which the postprogram behavior of participants is compared with their preprogram behavior, and matched comparisons, in which the postpro- gram behavior of the participants is compared with the behavior of a group of individuals who were similar to the participants before they enrolled in the program. In this article, as in much other research, the word "controls" is re- served to mean members of a randomized control group. All other counterfac- tual groups are called "comparison groups." Quasi-Experimental Designs This section discusses the design of the two major types of quasi-experimen- tal method-reflexive and matched comparison-with examples of evaluations that have used each method. For each, the way the counterfactual group is con- structed is pivotal. REFLEXIVE COMPARISON. In a reflexive comparison the participants serve as both the treatment and the comparison group. The counterfactual state is sur- mised using the preprogram behavior of the participants to infer what would have happened to them had they not joined the program. The strength of this methodology is that the socioeconomic and demographic characteristics of the group, previous experience, and the individuals' predisposition and innate abil- ities are the same both before and after the program. Consequently, observed changes in behavior from the pre- to postprogram period cannot be attributed to differences in these factors. Careful mathematical modeling is, however, re- quired to ensure that changes that would have occurred naturally are not at- tributed to the program. Generally, reflexive comparison group studies are time-series or panel studies that collect a large amount of data for several years both before and after the program to enable the researchers to understand how factors other than the program influence the outcome. jean Baldwin Grossman 161 A reflexive comparison was used to evaluate the effect of a water conserva- tion campaign in the United States. In 1972 a particular county declared a mor- atorium on new water hookups until alternative sources for water could be assessed. This moratorium lasted three years. Using monthly data from 1966 to 1976, researchers were able to estimate statistically that the moratorium re- duced water consumption by 15 percent (Maki, Hoffman, and Berk 1978). The finding has been used to justify the passage of similar regulations during other drought periods and in other counties. Because this technique requires a relatively long observation period before and after the program and is more dependent on statistical analysis and assump- tions than matched comparison or random assignment, it is the least used of the three methods for evaluating social policy. The technique is more common in research (such as psychological studies) where the key factors affecting the outcomes-such as an individual's self-esteem or resilience to adversity-are very difficult to measure accurately and large research samples are not feasible. MATCHED COMPARISON. In designing matched comparison groups, researchers identify a group of individuals whom the researchers judge to be comparable to the participant group in important dimensions but who do not receive pro- gram services. The researchers should match the two groups on factors that are known or believed to affect the key outcomes significantly. Such knowledge comes through previous experience or a theoretical understanding of the pro- cesses expected to underlie the intervention. The participant and comparison groups do not have to be similar with respect to characteristics that do not affect the outcomes of interest. For example, the researchers should draw on knowledge about factors that affect particular crops when selecting agricultur- al districts for comparison in a study of a program designed to affect agricul- tural productivity. The aggregate behavior of the comparison group is then assumed to indicate how the participants would have behaved had they not joined the program. Two principal types of matched comparison groups are prospective studies, in which comparison group members are selected at the same time as partici- pants are enrolling in the program, and retrospective studies, in which com- parisons are selected at a single point after the participant group has been enrolled. An example of a prospective study is the evaluation of the California Conservation Corps (CCC), a training and environmental conservation pro- gram for out-of-school youth (Wolf, Liederman, and Voith 1987). In this study selected participants who enrolled in the program during a twelve-month pe- riod were inducted into the research sample. During the same period, similar individuals who went to the CCC's single largest referral agency were designat- ed as comparison group members. Before the study began, researchers talked with referral staff to find out what kind of people were encouraged to enroll in the CcC. Then a brief survey was conducted at the referral agency and the ccc to determine how the people flowing through the two organizations 162 The World Bank Research Observer, vol. 9, no. 2 (July 1994) differed. On the basis of this survey, comparison group members were matched not only on age, race, and gender but on whether they had children, how re- ceptive they were to moving away from home, how much they enjoyed work- ing outdoors, and how much they enjoyed physical work. These were all important factors in corps members' decisions to join the program and might affect self-esteem, environmental awareness, and earnings-key outcomes of the study. An example of a retrospective study is the evaluation of the Job Corps, an education and training program for out-of-school youth (Mallar and others 1982). In this evaluation, Job Corps participants were compared with a group of individuals who had been surveyed by the Census Bureau for the Current Population Survey (cPs). The comparison group individuals (hereafter termed comparisons) were matched to the participants with respect to age, race, gen- der, poverty status, and education. All these factors were important in predict- ing future income. To minimize the risk that comparisons had enrolled in a Job Corps (information not given in the CPS), they were selected only from areas that were not served by Job Corps centers. The CcC and Job Corps studies are good examples of evaluations that have proved important in policy decisions. In particular, the Job Corps study found that the benefits to society outweighed the program's cost. Advocates used this finding to win political support for continued funding for the Job Corps, which had been slated for closure because of high costs. Similarly, advocates of the ccc have used the evaluation findings to secure funding during periods of bud- getary cutbacks. Matched comparison group methodology is the most common type of eval- uation used in the United States, but since the 1970s the flaws in straightfor- ward comparisons have been increasingly recognized. For example, individuals who do not participate in a program when given the opportunity may well dif- fer considerably from those who do, in such attributes as their motivation or their ability to benefit from the program-factors that would positively affect desired outcomes of the program. Because such factors are not easy to mea- sure, it is hard to ensure, even statistically, that they were similar for both groups. Thus, the relevant outcome (say, earnings) for the nonparticipants in this situation would be a poor proxy for the "would-have-been" outcome for the participants because the comparison group members are not comparable to the participants along important unobservable characteristics. This source of bias has come to be known as sample, or self-selection, bias. Studies evaluating the Salk (polio) vaccine and the Manpower Development and Training Act (MDTA), the employment and training program undertaken in the United States in the 1960s, illustrate some of the pitfalls of these quasi- experimental (nonrandom) comparison group methods. The Salk vaccine was evaluated using two different techniques, one quasi-experimental (a comparison of vaccine recipients with a nonrandom, naturally occurring group of individu- als who did not receive the vaccine) and the other experimental (a comparison jean Baldwin Grossman 163 of randomized groups). The estimate of the vaccine's effectiveness derived from the data from the randomized experiments was 14 percent higher than the esti- mate derived from the nonrandomly generated data. The nonrandom MDTA evaluation (Westat, Inc. 1984) compared the earnings of participants with those of nonparticipants. The estimated effect showed that the program decreased the income of trainees, even when the researchers controlled for differences in the two groups. To investigate this unexpected result, Director (1974) examined the earnings of participants and comparison group individuals before the program and found that, even before the program, the participants had earned less than the comparison group members. Thus Westat's estimate of a negative impact was likely incorrect. Reflexive evaluations that assume that a participant's situation before and after participating in the program would be similar may also not be valid. For example, if a string of bad luck with employment brought an individual to a training program, simply comparing the individual's situation before and after the program would overstate the effectiveness of a program because the par- ticipant's situation would have improved (on average) even in the absence of the program. Similarly, if the participant's situation were trending up or down, a before-and-after comparison would produce biased impact estimates. For ex- ample, a reflexive study of arthritis treatment found that the severity of the condition worsened over the life of the project. Randomized trials, however, showed that the treatment slowed the deterioration of the patient's condition (Deniston and Rosenstock 1972). This threat of trending is especially important when the program serves young people. For example, the Summer Training and Education Program, a federal pilot program, was a summer remediation and work-experience pro- gram for educationally and economically disadvantaged 14- and 15-year-olds. During the first summer of the program, participants lost half a grade in read- ing ability. If a before-and-after comparison had been used to judge the pro- gram's effectiveness, it would have been concluded that the extra instruction had harmed the participants. Fortunately, however, the evaluation was a ran- domized experiment. The control group members lost a full grade in reading- indicating that the true effect of the program was to raise test scores more than half a grade (Sipe, Grossman, and Milliner 1987). In the mid-1970s James Heckman salvaged the credibility of comparison group methodologies to some degree by developing a statistical solution enabling researchers to control theoretically for these unobservable differences (Heckman 1980). But many researchers felt uneasy about the statistical assumptions needed to correct for the biases and therefore searched for alternative solutions. Experimental Designs Using random assignment in social policy evaluation was one of the alter- natives found. In a random assignment evaluation, individuals who are eligible 164 The World Bank Research Observer, vol. 9, no. 2 (July 1994) for a program are identified and then randomly assigned to one of two groups: a treatment group, which is offered the program, or a control group, which is not. This methodology ensures that, before the program, the two groups are statistically equivalent, on average, with respect to all characteristics, observed and unobserved. Thus, if the average behavior of the two groups differs after the intervention, the difference can be confidently and causally linked to the program. Impact estimates based on random assignment evaluations are far more defensible than estimates based on nonrandom designs so long as the be- havior of all treatments and all controls is compared-including even those members of the treatment group who did not participate in the program, by chance or by choice and so long as researchers check and control for poten- tial sample bias arising from attrition.2 With these safeguards, those who do not like the results (including the sponsors of the research and the evaluators themselves) cannot attack the evaluation by claiming that the behavior of the controls does not accurately reflect what would have happened to the treat- ment group members if they had not received the intervention. One of the most recent and sophisticated examples of randomized social ex- periments is the evaluation (Bloom and others 1992) of the Job Training and Partnership Act (JTPA) program, a U.S. national training and employment pro- gram adopted in the 1980s. For this study, the first randomized evaluation of a national program, applicants were randomized after their training needs had been assessed so that a treatment and control comparison could be created for each type of service typically provided to JTPA participants-basic education, work experience, and on-the-job training (see Hotz 1992 for a detailed discus- sion of the evaluation design). An example from the 1970s is the Negative Income Tax (NIT) experiment, which was one of the first large randomized so- cial experiments conducted in the United States. For this study, low-income families were randomly assigned different guaranteed minimum incomes and different tax rates on earnings. Despite the lack of a "no-guaranteed-income" group, comparisons among the different treatment groups enabled researchers to develop unbiased estimates of the program's effect on participants' incentive to work and their general well-being (see Kershaw and Fair 1976 for details on the design and implementation of the evaluation). Both of these evaluations were multiyear, multisite, multimillion-dollar projects, and they had a significant effect on policy. The JTPA evaluation, which found few positive effects on participants' income, helped convince many in the U.S. government of the need to revamp the program. Although, for political reasons, a negative income tax was never enacted, subsequent re- vision of the Food Stamp program incorporated the taxing scheme of the NIT. In addition to these large experiments, many smaller randomized evalua- tions have been conducted. One of these assessed the effects of a culturally ap- propriate Mexican version of the children's television program "Sesame Street." The principal question in the Mexican study was whether the program increased knowledge among its viewers. The study found that children did Jean Baldwin Grossman 165 benefit, and "Plaza Sesamo" was consequently aired on the official government channel. Experimental versus Quasi-Experimental Designs Until the mid-1980s randomized and comparison group methodologies coex- isted without too much controversy. Statisticians and econometricians contin- ued to develop better ways of using comparison group data to estimate the effects of programs (Heckman and Robb 1985a, 1985b). However, the big draw- back of quasi-experimental methods-the need for statistical adjustment-re- mains. Evaluators identify members of a comparison group on the basis of judgments informed by theory and experience. They then attempt to control for any remaining differences through statistical techniques. The validity of the se- lection process remains an empirical question that can be resolved only after the data are collected. Because impact estimates derived from randomized experi- ments need no statistical adjustment, they are inherently less controversial and easier to defend. During the 1970s and early 1980s most evaluators recommend- ed the use of random assignment for evaluating major program initiatives but considered comparison group strategies acceptable for smaller evaluations, in view of advances that had been made in statistical modeling. But then controversy over a comparison group evaluation of the Compre- hensive Employment and Training Act (CETA) dealt a serious blow to research- ers' faith in statistical modeling. The U.S. government wanted to determine the effectiveness of the program, but politics dictated that no individual who de- sired services be denied them, which put a random assignment evaluation out of bounds. The government therefore funded a massive comparison group evaluation. Extensive data were collected on CETA participants nationally. The participants' behaviors were then compared with the behaviors of individuals with similar characteristics identified in other national surveys. Comparison group members were matched to participants with respect to gender, race, age, education, poverty status, and income history. Sophisticated matching algo- rithms were used to select the most comparable individuals. The studies found surprisingly few positive effects of the program (Bassi and others 1984; Dickerson, Johnson, and West 1984; Westat, Inc. 1984). But the sensitivity of the estimates to the statistical techniques used was disturbing. The evaluation community began to view this study and its results with suspicion. Fraker and Maynard (1984) wrote a seminal paper that showed how com- parison group methodologies, such as the one used in the CETA study, could lead to false negative estimates of impacts. Using data from a major random assignment demonstration program conducted in the United States, the Na- tional Supported Work (NSW) demonstration, the authors constructed compar- ison groups using various matching strategies, including those employed in the CETA evaluations. The authors calculated what the impact estimates would have been had the outcomes for the NSW treatment group been compared with 166 The World Bank Research Observer, vol. 9, no. 2 (July 1994) the outcomes for the matched comparison groups, instead of with those for the randomly selected NSW control group. They found that the estimates based on comparison groups did not come close to the actual impacts estimated using the randomized control group. Lalonde (1986) further damaged the case for comparison groups by showing that the various sophisticated econometric techniques developed to solve self- selection problems did not improve Fraker and Maynard's quasi-experimental estimates. Summarizing their findings, Lalonde and Maynard (1987) found that the statistical inference based on a comparison group methodology was incor- rect 40 percent of the time. They concluded the following: Nonexperimental procedures may not accurately estimate the true program impacts. In particular, there does not appear to be any for- mula (using nonexperimental methods) that researchers can confi- dently use to replicate the experimental results of the Supported Work Program. In addition, these studies suggest that recently de- veloped methods for constructing comparison groups are no more likely (and arguably less likely) than the econometric procedures to replicate the experimental estimates of the impact of training.... These findings are further evidence that the current skepticism sur- rounding the results of nonexperimental evaluations is justified (Lalonde and Maynard 1987, p. 226). These papers had a profound influence on evaluators' confidence in compar- ison group methodologies, ultimately undermining the CETA study so badly that most of the evaluation and policy communities ignored its results and dis- counted its conclusions. Thus the millions of dollars invested in the study were wasted because the findings lacked credibility. It was this research that led a government-appointed technical advisory panel to recommend that a randomized design evaluation replace the compar- ison group evaluation originally planned for the JTPA, the program that suc- ceeded CETA. The panel wrote that its recommendations are strongly conditioned by the judgment that it will not be possible to solve the problem of selection bias within the context of a quasi- experimental design, at least not in [a] short enough time frame to meet Congress' need for valid information to guide policy. Even though many authors studying employment and training programs have recognized the selection problem, no such study using a quasi- experimental design can be said to have controlled adequately for selection bias. The panel does not intend to set forth a counsel of despair. Rather, it is concerned that the past evaluations of CETA have consumed, and the contemplated evaluations of JTPA will con- sume, millions of dollars and much valuable time. It would be ex- tremely unfortunate if the analysis of [the originally planned JTPA evaluation design] would yield the same ambiguous conclusions as jean Baldwin Grossman 167 has the analysis of the [quasi-experimental] data base for CETA (Job Training Longitudinal Survey Research Advisory Panel 1985, p. 21). Nonetheless, the panel thought it "prudent to continue development work on the econometric front (p.22)." The "use of the quasi-experimental analysis de- sign in the long run may be the only way one can regularly measure nationally representative program outcomes and be able to do this without having to deny program treatment to some (p.22)," the panel wrote. Thus the panel persuaded the government to fund an evaluation of comparison group methods under the leadership of James Heckman and Joseph Hotz. Heckman and Hotz found the Lalonde-Fraker-Maynard argument flawed in that those researchers had used various econometric techniques without first testing to see if they were statistically appropriate. Using the same data, Heck- man and Hotz conducted tests of these techniques and found that estimates leading to incorrect statistical inferences were based on techniques or specifi- cations that could be rejected (Heckman, Hotz, and Dabos 1987). They con- cluded, "Given that a variety of robust nonexperimental methods have yet to be assessed and that better data on the selection process are being generated by a new JTPA evaluation study, which will shed important new light on the selection process, we are confident that reliable nonexperimental evaluation methods can and will be developed in the future for all subsidized employment and training programs (p.424)." Friedlander and Robins (1992) applied the quasi-experimental specification testing strategy to experimental data from the work-welfare demonstrations conducted in the United States during the late 1980s to see if the tests could successfully weed out the techniques that had led to incorrect conclusions in another (non-NSW) setting. They found that even when specification tests were incorporated, the reliability of the quasi-experimental results did not improve much. Thirty-six percent of the estimates that had passed the tests still led to the wrong statistical inference, and 56 percent of those that had failed the tests and had therefore been discarded were correct. The authors found that the most egregiously incorrect impact estimates were indeed discarded but that many correct estimates were thrown out at the same time. In defense of the Heckman-Hotz position, however, it should be noted that Friedlander and Robins used a data base that did not explicitly collect information that allowed extensive specification tests and modeling of the selection process. An important postscript to this discussion is that the effects found in the randomized JTPA evaluation echoed those found in the nonrandom CETA eval- uations, whose credibility had been so thoroughly undermined. JTPA appears to have a modest positive effect on earnings for adults eighteen months after enrollment, no effect on out-of-school young women, and a negative effect on out-of-school young men. The weak components of JTPA are basically the same components that were ineffective in CETA. Had the original CETA evaluation been a random assignment evaluation, policymakers would have had to con- tend with its findings and change the program. Instead, the earlier results were 168 The World Bank Research Observer, vol. 9, no. 2 (July 1994) simply ignored and the same ineffective (or worse) services were repeated for an additional ten years. Where does this discussion lead? I believe it leads to the conclusion that ran- dom assignment design is the evaluation strategy that yields the most reliable and defensible estimates. If the question at issue has profound implications for policy, it may pay to invest the time and resources necessary for random as- signment (see Heckman 1992 for a somewhat technical discussion of the rele- vant benefits of randomized versus quasi-experimental designs). But random assignment is not appropriate, or even possible, in many situations, as the dis- cussion that follows explains. Choosing an Evaluation Strategy Choosing an evaluation method depends on the kinds of questions being asked about a program, the number of participants the program will serve, the operational details that might make one or another type of evaluation unsuit- able, and the constraints on the time and resources available for the exercise. Key Questions In selecting a technique suited to assessing a particular program, policymak- ers must know which questions they most want answered. One evaluation technique may be more suited to tackling certain types of questions than an- other. For instance, both quasi-experimental and experimental strategies can investigate the effects of a program on individuals-such as the effectiveness of a particular retraining program in speeding workers' transition to new em- ployment. But if the principal questions concern the program's effects on the community, the providers of the service, or other aggregate organizations-for example, the likely effects on the local labor market of guaranteeing all young people a summer job-then experimental random assignment designs are not appropriate. Often, the intended scale of the program dictates which of these sets of ques- tions-individual or community-are the most important. For example, sup- pose that budget constraints allow only a fraction of the unemployed to receive retraining services. In such a case estimates of effects on individuals are likely to be of most interest, and any of the evaluation strategies could be appropri- ately applied. Or consider a farm subsidy program aimed at increasing the in- comes of farmers. The principal questions again are about the program's effects on individuals rather than about the feasibility of large-scale implemen- tation, and again any of the evaluation strategies would be suitable. But if the subsidy program is to serve every farmer who qualifies, and if issues such as the supply responses of the farmers are critical, then the evaluation options are more limited. To observe these aggregate or community-level responses to the Jean Baldwin Grossman 169 program, a trial demonstration of the program should simulate universal eligi- bility in certain sites (a technique known as "saturation"). Because random as- signment strategies require that some eligible individuals do not receive the treatment, demonstrations that saturate a site must be evaluated with quasi- experimental methods. For the same reason, quasi-experimental strategies must be used to evaluate existing programs that serve everyone who meets the eli- gibility criteria, such as Social Security in the United States or a national health service.3 Existing full-coverage programs can be evaluated by examining dif- ferent effects produced by different program intensities. Reflexive pre- and postprogram design can also be used to evaluate recently implemented pro- grams. But in both cases statistical analysis must be conducted to control for intervening factors. An example of a saturation demonstration is the Youth Incentive and Em- ployment Project, in which young people were guaranteed a job if they re- mained in school (Farkas and others 1982). The key questions were whether enough jobs could be created to fulfill this promise and whether the program increased graduation rates. Matched comparison was used here, pairing cities with similar labor market and youth characteristics. The study found that enough jobs could be created and that, during the program, earnings among black youth increased to nearly equal earnings of white youth in the partici- pating sites (a condition that did not exist before the program or in the com- parison sites). Another saturation demonstration, the Employment Opportunity Pilot Pro- gram (EOPP), was mounted by the U.S. government in the late 1970s to field- test an alternative welfare, employment, and training program. The EOPP pro- vided intensive job search assistance to all "employable" welfare recipients and all "employable" members of low-income households (Brown and others 1984). Individuals who could not find a job were given subsidized employment or training opportunities. One of the critical questions was whether an entitle- ment program such as EOPP was operationally feasible. How many jobs would be needed? How would the program affect the general labor market for low- wage jobs? To answer these questions, the demonstration consisted of ten sat- uration sites, in each of which sufficient job and training positions were to be funded to satisfy the full demand. Ten comparison sites were chosen that were judged to have populations, industrial structures, and labor market conditions similar to the EOPP sites. The original evaluation design also included an eleventh, random assign- ment, site. Information from this site would be used to gauge the validity of the estimated effects derived from the other sites. For budgetary and political reasons, this eleventh site was eliminated, but the attempt illustrates how mul- tiple strategies can be used to strengthen the overall quality of the evaluation. A potential problem with a matched site strategy is that the cities or sites that make up a matched pair may become dissimilar during the follow-up period. The longer the follow-up, the more likely the sites' situations are to 170 The World Bank Research Observer, vol. 9, no. 2 (July 1994) drift apart. For example, in selecting comparison sites in the EOPP demonstra- tion, changes in the unemployment rates from 1975 to 1978 (a preprogram pe- riod) were examined to determine whether the pairs' economies were likely to respond similarly to the improvement expected in national economic condi- tions (Brown and others 1984). But the economy of one of the comparison sites depended heavily on the auto industry, which was hit much harder by the re- cession of the early 1980s than by the recession of the mid-1970s. As a result, the comparability of this comparison site with its designated demonstration site was compromised. The researchers were thus unable to control effectively for local labor market conditions; instead, results were estimated both with and without this problem site. A slow phase-in of a program is conducive to mixed-mode evaluation. For example, consider a program that, because of financial or resource constraints, can be initiated in only a few sites and even then can serve only a small fraction of the individuals it is ultimately intended to serve. This situation opens the possibility of conducting a random assignment evaluation during the first few years of the program's operation to determine if it improves the lives of its participants. (For such an evaluation to be most useful, the limited early ver- sion of the program should be offered to the same set of individuals as those who will ultimately qualify for the program-not, however tempting, to those most in need, or to some other special group, because these would not be rep- resentative of the intended target population.) Once additional capacity is available and the program can be expanded to serve all intended recipients, more effects at the community level can be observed and analyzed through matched comparisons or some other quasi-experimental method. Number of Participants Some social programs are designed to affect individuals, others to affect communities or entire regions. Employment and training programs are geared primarily to individuals; health clinics affect communities; irrigation programs affect farms or regions; and most educational programs affect classes of young people. The number of potential program participants-millions of people, thousands of schools, hundreds of towns, a handful of regions-will affect the type of evaluation that is possible. When the number is small, matched com- parison or reflexive techniques are more likely to produce a counterfactual group more similar to the participant group than random assignment. Random assignment strategies are best when the number of potential participants is fair- ly large. The earlier discussion of matched comparison groups described how evalu- ators could carefully create a comparison group by selecting nonparticipants with identical sets of key characteristics, such as age and education (or rainfall and soil richness in an agricultural example). Random assignment obviously cannot exactly match individuals in this way; for example, the likelihood that Jean Baldwin Grossman 171 two randomly chosen individuals will be the same age is quite small. So, if key characteristics of a participant group and a comparison group can be matched exactly through matched comparison, why would one ever use random assign- ment? The reason is that although the characteristics explicitly matched by the researcher are the same, other, unobservable characteristics that might affect the outcome of the program are not. The question, then, is how to create similar groups using random selection. Two randomly chosen individuals are unlikely to be the same age, but the av- erage age of two randomly selected groups of a thousand people is likely to be quite close. In fact, the average of all characteristics (both observable and unobservable) of these two large groups-education level, IQ, healthiness, work history-is likely to be quite similar. If individuals are randomly assigned to groups, as the size of the groups increases, the average characteristics of the two groups converge. The two groups are unlikely ever to be identical, but statisticians can calculate how large the groups must be to be confident that any small observable difference is due to chance. Random assignment, thus, is a more reliable means of ensuring that, before the program begins, the control group is similar to the treatment group, on average, on all characteristics. Any differences between the two groups observed after the program can thus be attributed to the program. How large the groups of participants and controls must be for a useful eval- uation depends on how similar individuals are to one another naturally, at least with respect to the key outcome, and how much the program is expected to change this outcome (see Conlisk 1973 for a discussion of the underlying assumptions, specific formulas, and more general forms of the problem). If the outcome does not vary at all across the target population-for example, if all students leave school when they turn 16-only a very small control group would be needed to conclude confidently that students' increased educational attainment resulted from a successful program rather than from chance. The larger the underlying variation across people in the key outcome, the larger the treatment and control groups need to be. To continue the example, assume that equal numbers of students between the ages of 14 and 18 leave school ev- ery year and that the difference in the average age at the time they leave school is 16 for the controls and 17 for the participants. Given the same size sample as in the first example, it would be harder to be sure that the schooling differ- ence was not attributable to randomly selecting more of the older leavers into the treatment group than into the control group. Similarly, if the effect of a program is expected to be quite small, larger groups would be needed to be confident that any observed difference is not due to chance. For example, if a particular water treatment procedure was sup- posed to decrease the mortality rate by 1 percent, large groups of treatments and controls would be needed to conclude confidently that such a decrease was caused by the program; if the procedure was expected to decrease the mortality rate by 20 percentage points, much smaller groups of individuals would suffice. 172 The World Bank Research Observer, vol. 9, no. 2 (July 1994) The Long-Term Care Channeling demonstration (Kemper and others 1986) illustrates the relation between the expected effect and the size of the groups needed for a random assignment evaluation. This program attempted to enable the fragile elderly to stay in the community rather than have to go to a nursing home. It was expected that without the program 50 percent of the target pop- ulation would go into a nursing home; policymakers wanted the program to reduce this rate to 45 percent. To be confident that a difference of 5 percentage points was not due to chance, evaluators calculated that the treatment and con- trol groups would each have to consist of 1,715 individuals. The budget for the evaluation did not allow for such a large sample. Because they could fund only 1,000 individuals in each group, evaluators calculated that the rate would have to drop by 6.5 percentage points, to 43.5 percent, for them to conclude with confidence that the change was due to the program rather than to chance. In sum, programs affecting outcomes that are inherently less variable or that will have larger impacts require smaller samples. It is, nonetheless, rare for ran- domized social program evaluations to consist of fewer than 500 to 1,000 par- ticipants in each group. From this discussion, it should be clear that random assignment designs are not always feasible. If the number of group members needed to assess the im- pact is large and the potential recipients of the program are few, random as- signment may not be a good option. For some types of interventions, such as programs aimed at entire communities, random assignment evaluation would be too costly. In these cases, careful judgmental matching of communities is a more reasonable way to construct a comparison group, even though conclu- sions based on such comparisons would be less robust than those based on ran- domized groups. Operational Details Details of a program can and should affect the choice of the evaluation method and how it is carried out. In particular, the evaluation should be de- signed to minimize any risk that the study itself might compromise the pro- gram-by altering the program's delivery in some fundamental way, changing the type of individual who would be served, or changing the behavior of the members of the counterfactual group. ALTERING THE SERVICES DELIVERED. Sometimes, gathering information can it- self be construed as a service. An important feature of the Long-Term Care Channeling program was an assessment by a clinician of the client's physical health and well-being. Much of the baseline information needed to learn whom the program would serve and for whom it was most effective was also medical data. How could such information be collected from the control group mem- bers without inadvertently providing them with part of the program? It was decided that interviews of the controls by nonmedical personnel would not Jean Baldwin Grossman 173 constitute an assessment. But if interviewers (not clinicians) were to gather the baseline information for the control group, perhaps interviewers should also question participants to ensure comparable data. A medical interview with an interviewer is not the same thing as a medical interview with a clinician, how- ever, so this option was discarded. Similarly it was decided not to subject the participants to both the clinician's and the interviewer's questioning because this intense medical questioning would be exhausting for these fragile elderly and because it might be construed as a more intensive service. Ultimately, eval- uators opted to live with noncomparably collected data-participants were in- terviewed by clinicians, controls were questioned by interviewers. Less obviously, an evaluation can affect a program by overburdening pro- gram staff with research requirements. Evaluators should work closely with program staff to minimize the research burden on staff members. This is espe- cially true for randomization schemes. A good example of how random assign- ment can be incorporated into a complex selection process without changing the essential elements of the program can be seen in the evaluation of the Big Brother/Big Sister program, in which adult volunteers serve as mentors and friends to children who have only one parent. An essential element of the program is the careful matching of the adult with the child by trained social workers. Two alternative random assignment op- tions were presented to the program personnel. In the first option, caseworkers would select two young people who they deemed appropriate for a particular volunteer. Then "a flip of the coin" would decide which youngster was actually assigned to the volunteer. The nonassigned child would be part of the control group. The second option randomly assigned new applicants to either the treatment or the control group. Children in the treatment group would then be bumped to the top of the caseworkers' lists to be matched as soon as pos- sible, instead of having to wait the usual amount of time. The percentage of the treatment group that ultimately received matches would be higher in the first option than in the second, but the caseworkers' workloads would be twice what they were in the second. Both options preserved the matching element of the program. The design options were discussed with program staff, whose concerns were important in deciding which process was chosen. Finally, re- searchers decided to randomize the pool of new applicants before matching and so lessen the workload of an already overburdened staff. Then, before the process began, meetings were held at each site to explain and discuss the study and the randomization procedures and answer the staff's questions. Researchers should make the randomization process tolerable and compre- hensible to deliverers of the service because otherwise program staff may un- dermine the evaluation. Often, they are already overwhelmed with program requirements and resent the additional demands of the evaluation, which either entail more paperwork or oblige them to do their jobs slightly differently (for instance, to select two children per volunteer rather than one). Making the staff 174 The World Bank Research Observer, vol. 9, no. 2 (July 1994) part of the design process, providing funds to hire additional staff, or offsetting the extra work entailed can pay off handsomely in the long run. Researchers should also be sensitive to program operators' need to serve a certain number of special cases (either for political reasons or because the ap- plicant faces particular hardship). These exceptions can be made without seri- ously compromising the study, and they often buy evaluators the good will of the operators by indicating that the evaluation process is incorporating their point of view. A small number of participants can be allowed to circumvent the randomization process, by receiving services but being excluded from the research. However, program operators should be aware that evaluation results do not apply to these individuals. In sum, the process of randomization must be tailored to some degree to the needs of the operators. CHANGING THE TARGET POPULATION. The research requirements of a demon- stration can skew the selection of participants so that the population served by the pilot program differs from the intended recipients of the program itself. One reason is that individuals who are willing to participate in research may differ from individuals who are simply willing to receive services. For example, if extensive academic testing is needed to evaluate a youth education program, youngsters who might have participated but who are averse to being tested may fail to apply for the evaluated program. Similarly, if extensive recruiting is required to identify enough individuals to fill both the counterfactual and the treatment groups, the recruited individuals may differ from individuals who would come forward normally. This risk is minimized when there are more eligible individuals than the program can serve, so that no extra recruit- ment is needed. Making sure that the target population for the demonstration is as similar as possible to that for the program is vital because the evaluation will assess the effect of the program only on those who were served. For example, the Long Term Care demonstration told policymakers how case management af- fected the fragile elderly but not the healthy elderly.4 CONTAMINATING THE COUNTERFACTUAL GROUP. When designing an evalua- tion strategy, researchers must consider how the nontreatment group will react to the presence of the evaluation. In many quasi-experimental designs-for ex- ample, reflexive or matched site designs-the individuals in the comparison group are completely unaware of the assessment. When other designs are used, however, especially randomized selection, the nontreatment group is aware of the experiment and may be adversely affected by this information. Individuals who are randomly selected into a control group, for example, may feel that they were passed over for treatment because of a personal failing, and this sense of rejection could cause them to act differently than they would have otherwise. This behavior inaccurately reflects how participants would have behaved had the program not existed; in other words, the counterfactual group Jean Baldwin Grossman 175 is contaminated. Similarly, enriching the educational opportunities of one class in a school might well dishearten the unselected students, whose performance might suffer as a result. The changed performance would compromise the effectiveness of these students as members of a comparison group for the evaluation. A variant of this source of contamination is changed behavior of others in response to the selection process. If, for example, half the classes in a school were selected to receive additional resources, teachers might redirect their time to compensate nonparticipants for their lost opportunity. The resulting changed situation for the nonparticipants would damage their validity as com- parisons. A third source of contamination is the counterfactual group's unintended re- ceipt of program services. That occurred in the early random assignment eval- uation of the children's television program "Sesame Street," where some of the control children were found to have seen the program, thus compromising their value as a nontreatment group. Program staff can often inform evaluators about potential contamination. In the Channeling demonstration program, for example, operators pointed out that people living together could not be randomized into different groups-the wife could not be a treatment while the husband was a control. For one thing, some of the program's services, such as housekeeping, were received by the en- tire household, so that the control would be receiving some program services. For another, envy could affect the control's behavior. This valid concern was addressed by randomizing households: when individuals applied to the pro- gram, they provided the names of everyone they lived with, and if someone they lived with had been randomized earlier, the new applicant received what- ever designation-treatment or control-the previous household member had received. Constraints on Time and Resources Time and budget constraints also affect the choice of an evaluation strategy. All evaluation takes time, but random assignment and prospective comparison group designs (including prospective reflexive studies) take longer than retro- spective comparison group designs. In particular, comparison group designs that select currently enrolled participants or even program graduates and com- pare them with other similar youth do not have to allow time for building up a sample for treatment and for assessing postprogram effects. Typically, the sample buildup period lasts a year. Then time must be allocated to allow the last member of the treatment group enrolled to receive the intervention. Final- ly, time is usually allocated to investigate whether the participants reap post- program benefits. There is always a tension between wanting results quickly and wanting to investigate long-term effects. Generally, the behavior of the two groups is examined twelve to eighteen months after sample enrollment. 176 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Prospective studies are also often more costly than retrospective studies, as long as data for the retrospective studies are easily accessible. Prospective stud- ies usually entail telephone or in-person interviews. Baseline (preprogram) in- terviewing is cheaper than follow-up interviews because one knows where the respondents are; for follow-up interviews, respondents must be tracked down. A common mistake of inexperienced evaluators is that they do not make every effort to find respondents and persuade them to answer the research questions. Attrition of the sample through nonresponse can seriously compromise the generalizability of the results. Sample attrition damages both randomized and comparison group designs. Generally, the evaluation community heavily dis- counts results from experiments in which the response rates are less than 70 percent. Unless the follow-up period is very long (four or five years or more), good-quality social evaluations usually are expected to have response rates of 80 to 85 percent. Conclusions The general consensus at present is that random assignment is the evalua- tion technique that produces the most defensible results. Ashenfelter and Card (1985, p. 648) conclude that "randomized clinical trials are necessary to deter- mine program effects"; Barnow (1987, p. 190) says that "experiments appear to be the only method available at this time to overcome the limitations of non- experimental evaluations," and Manski and Garfinkel (1992) labeled random assignment the "new orthodoxy." But random assignment is not always feasi- ble or suitable, and when that is so, evaluators and policymakers must settle for techniques that are inherently more open to criticism. Because these quasi- experimental methods rely extensively on statistical assumptions (Cook and Campbell 1979), explicit efforts should be made to collect the data needed to test those assumptions rigorously. (Moffitt 1991 provides a nontechnical dis- cussion of the various estimation techniques and the tests appropriate for use with quasi-experimental data.) But the greater credibility of random assignment should not lull researchers into a false sense of security and lax methodology. Random assignment in its purest form (comparison of simple means) works only if there is no systematic sample attrition and the control group does not become contaminated. Thus, data must be collected and statistical methods need to be employed to adjust for any of these problems that may arise. Researchers should be careful in generalizing from experiments in general. Strictly speaking, estimated impacts describe only the effect that a particular program intervention, as delivered, had on the specific participants served. In drawing conclusions about the likely effectiveness of a program, it is vital to be clear about what intervention was actually delivered and thus what inter- vention was actually tested. The program "on the ground" could be quite jean Baldwin Grossman 177 different from the program described in the rhetoric of the policymakers. For example, the Supported Work program was developed to move individuals gradually from a very structured job environment with a lot of supervision and relatively lax work standards to one in which the participant was doing the job more or less independently and meeting standards that would be demanded in unsubsidized jobs. But because "graduated stress" was never actually docu- mented and related to outcomes, it would be incorrect to generalize from the Supported Work demonstration that graduated stress does or does not work for youth. To be able to generalize findings adequately to other settings, one must understand the mechanisms by which the services changed the participants' sit- uation compared with that of members of the counterfactual group, and the means by which these services changed the participants' decision process. Why did a program work or not work? Additional analysis should be conducted to explore the mechanisms through which the program had its effect. Heckman (1991) warns that it is all too easy for evaluators of randomized experiments to treat the program as a black box and fail to investigate in detail the social mechanisms by which it operates, because aggregate pronouncements of "effec- tiveness" or "ineffectiveness" can be made with relative ease. Knowledge of what works is built up slowly. An evaluation rarely answers all questions. Yet, by understanding how public policy changed behavior, policymakers may be able to improve future interventions. Notes Jean Grossman is Director of Research and Vice President at Public/Private Ventures in Phil- adelphia, Pennsylvania. 1. Several textbooks discuss these issues further; Cook and Campbell (1979) and Rossi and Freeman (1989) are but two examples. 2. Some sample loss may be unavoidable (or even desirable). For example, if one of the goals of a program is to extend the lives of elderly individuals, it might well turn out that living mem- bers of the treatment group are, on average, more impaired than living members of the control group because more of the most impaired controls die, while impaired members of the treatment group have a higher probability of living. A simple comparison of the impairment levels of the two groups-as is usual in random assignment evaluations-would thus be inappropriate. 3. Full-coverage programs can be assessed by the experimental method only if a government has the will to waive the full-coverage requirement for the period of the research, as the U.S. government did to permit a random assignment strategy for the JTPA evaluation described earlier. 4. Effects for different populations can sometimes be inferred statistically if the alternative population does not differ too much from the original population (see Conlisk 1973). References The word "processed" describes informally reproduced works that may not be commonly available through library systems. 178 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Ashenfelter, Orley, and David Card. 1985. "Using the Longitudinal Structure of Earnings to Es- timate the Effect of Training Programs." Review of Economics and Statistics 67:648-60. Barnow, Burt. 1987. "The Impact of CETA Programs on Earnings: A Review of the Literature." Journal of Human Resources 22:157-93. Bassi, Laurie J., Margaret C. Simms, Lynn C. Burbridge, and Charles L. Detsey. 1984. Measuring the Effect of CETA on Youth and the Economically Disadvantaged. Washington, D.C.: Urban Institute. Bloom, Howard, Larry Orr, George Cave, Stephan Bell, and Fred Doolittle. 1992. The National JTPA Study-Title Il-A Impacts on Earnings and Employment at 18 Month. Bethesda, Md.: Abt Associates. Brewster, J. Alan, Walter Corson, John Friedmann, Walter Nicholson, and Andrea Vayda. 1978. Follow-Up Study of Recipients of Federal Supplemental Benefits. Princeton, N.J.: Mathematica Policy Research, Inc. Brown, Randall, John Burghardt, Edward Cavin, and Rebecca Maynard. 1984. "The Employ- ment Opportunity Pilot Projects: Analytic Strategies and Estimates of Program Impacts." Princeton, N.J.: Mathematica Policy Research, Inc. Conlisk, John. 1973. "Choice of Response Function Form in Designing Subsidy Experiments." Econometrica 41(4):643-56. Cook, Thomas, and Donald Campbell. 1979. Quasi-Experimentation Design and Analysis Issues for Field Settings. Skokie, Ill.: Rand-McNally. Deniston, O., and I. Rosenstock. 1972. 'The Validity of Designs for Evaluating Health Services." Research report. University of Michigan, School of Public Health, Ann Arbor. Processed. Devaney, Barbara, Linda Bilheimer, and Jennifer Shore. 1992. "Medicaid Costs and Birth Out- comes: The Effect of Prenatal wIc Participation and the Use of Prenatal Care." Journal of Policy Analysis and Management 11 (4) :573-92. Dickerson, Katherine P., Terry R. Johnson, and Richard W. West. 1984. An Analysis of the Im- pact of CETA Programs on Participants' Earnings. Menlo Park, Calif.: SRI International. Director, S. 1974. "Evaluating the Impact of Manpower Training Programs." Ph.D. diss., North- western University, Evanston, Ill. Processed. Farkas, George, Randell Olsen, Ernest Stromsdorfer, L. Sharpe, Felicity Skidmore, D. Smith, and S. Merrill. 1982. Impacts from the Youth Incentive Entitlement Pilot Projects: Participation in Work and School over the Full Program Period. New York: Manpower Demonstration Re- search Corporation. Fraker, Thomas, and Rebecca Maynard. 1984. "An Assessment of Alternative Comparison Group Methodologies for Evaluating Employment and Training Programs." Princeton, N.J.: Mathematica Policy Research, Inc. Friedlander, Daniel, and Philip K. Robins. 1992. "Estimating the Effects of Employment and Training Programs: An Assessment of Some Nonexperimental Techniques." Manpower De- velopment and Research Corporation, New York. Heckman, James, J. 1980. "Sample Selection Bias as a Specification Error." In E. W. Stromsdor- fer and G. Farkas, eds., Evaluations Studies Review Annual. Vol. 5. Beverly Hills, Calif.: Sage. 1991. "Basic Knowledge-Not Black Box Evaluations." Focus 12(Summer): 234-35. 1992. "Randomization and Social Policy Evaluation." In Charles Manski and Irwin Garfinkel, eds., Evaluating Welfare and Training Programs. Cambridge, Mass.: Harvard Uni- versity Press. Heckman, James J., V. Joseph Hotz, and Marcelo Dabos. 1987. "Do We Need Experimental Data to Evaluate the Impact of Manpower Training On Earnings?" Evaluation Review 11(4):395-427. Heckman, James J., and Robert Robb. 1985a. "Alternative Methods for Evaluating the Impact of Interventions: An Overview." Journal of Econometrics 30:239-67. Jean Baldwin Grossman 179 . 1985b. "Alternative Methods for Evaluating the Impact of Interventions: An Overview." In James J. Heckman and B. Singer, eds., Longitudinal Analysis of Labor Market Data. Cambridge, U.K.: Cambridge University Press. Hotz, V. Joseph. 1992. "Designing an Evaluation of the JrPA." In Charles Manski and Irwin Garfinkel, eds., Evaluating Welfare and Training Programs. Cambridge, Mass.: Harvard Uni- versity Press. Job Training Longitudinal Survey Research Advisory Panel. 1985. "Recommendations of the Job Training Longitudinal Survey Research Panel." Prepared for the Office of Strategic Planning and Policy Development, Employment and Training Administration, U. S. Department of La- bor, Washington, D.C. Processed. Kemper, Peter, George Carcagno, Randall Brown, Robert Applebaum, and Judith Wooldridge. 1986. The Evaluation of the Long-Term Care Channeling Demonstration: Final Report. Princeton, N.J.: Mathematica Policy Research, Inc. Kershaw, David, and Jenifer Fair. 1976. The New Jersey Income-Maintenance Experiment. New York: Academic Press. Lalonde, Robert. 1986. "Evaluating the Econometric Evaluations of Training Programs with Ex- perimental Data." American Econometric Review 76(4):604-20. Lalonde, Robert, and Rebecca Maynard. 1987. "How Precise Are Evaluations of Employment and Training Programs: Evidence from a Field Experiment." Evaluation Review 11(4):212-27. Maki, J. E., D. Hoffman, and Robert Berk. 1978. "A Time Series Analysis of the Impact of a Water Conservation Campaign." Evaluation Quarterly 2(February):107-18. Mallar, Charles, Stuart Kerachshy, Craig Thornton, and David Long. 1982. "Evaluation of the Economic Impact of the Job Corps." Princeton, N.J.: Mathematica Policy Research, Inc. Manski, Charles, and Irwin Garfinkel, eds. 1992. Evaluating Welfare and Training Programs. Cambridge, Mass.: Harvard University Press. Moffitt, Robert. 1991. "Program Evaluation with Nonexperimental Data." Evaluation Review 15(3, June):291-314. Rossi, Peter, and Howard Freeman. 1989. Evaluation: A Systematic Approach. Newbury Park, Calif.: Sage. Sipe, Cynthia, Jean Baldwin Grossman, and Julita Milliner. 1987. The Summer Education and Training Program: A Report of the 1986 Experience. Philadelphia, Pa.: Public/Private Ventures. Westat, Inc. 1984. Summary of Net Impact Results of CETA. Rockville, Md. Wolf, Wendy, Sally Liederman, and Richard Voith. 1987. The California Conservation Corps: An Analysis of Short-term Impacts on Participants. Philadelphia, Pa.: Public/Private Ventures. 180 The World Bank Research Observer, vol. 9, no. 2 (July 1994) USING RANDOMIZED CONTROL DESIGNS IN EVALUATING SOCIAL SECTOR PROGRAMS IN DEVELOPING COUNTRIES John Newman Laura Rawlings Paul Gertler Seven case studies-from Bolivia, Colombia, Indonesia, Mexico, Nicaragua, Taiwan (China), and Turkey-demonstrate the feasibility of conducting rigor- ous impact evaluations in developing countries using randomized control de- signs. This experience, covering a wide variety of settings and social programs, offers lessons for task managers and policymakers interested in evaluating social sector investments. The main conclusions are: first, policymakers interested in assessing the effec- tiveness of a project ought to consider a randomized control design because such evaluations not only are feasible but also yield the most robust results. Second, the acute resource constraints common in developing countries that often make program rationing unavoidable also present opportunities for adopting random- ized control designs. Policymakers and program managers need to be alert to the opportunities for building randomized control designs into development pro- grams right from the start of the project cycle because they, more than academic researchers or evaluation experts, are in the best position to ensure that oppor- tunities for rigorous evaluations are exploited. D espite the importance of knowing whether social programs work as intended, evaluations of social sector investments are still uncommon in developing countries. This neglect of evaluation handicaps the de- velopment community's ability to demonstrate what has been achieved and so to win political support, design more effective projects, and set priorities for resource allocation. Today, as more money than ever is flowing to the social The World Bank Research Observer, vol. 9, no. 2 (July 1994), pp. 181-201 © 1994 The International Bank for Reconstruction and Development/THE WORLD BANK 181 sectors, governments and lending institutions are demanding value from that money. Evaluations can help make that happen by answering the critical ques- tion of how effective a particular social sector intervention is relative to other possible interventions. This article and the companion article by Grossman lay out the issues of which policymakers and task managers need to be aware to build successful evaluation designs into their projects. Grossman's article describes the advan- tages, disadvantages, and limitations of the three main types of evaluation strategy-two quasi-experimental (reflexive and matched comparison) and one experimental (randomized control) (see table 1)-and reviews their use in social sector programs in the United States. Examples can be found in devel- oping countries for each type of evaluation strategy discussed in Grossman's article.' Grossman expresses the view, generally shared by evaluation experts, that randomized control designs are the best evaluation strategy in technical terms but that in many situations it is not possible or appropriate to apply them. This article examines the use of randomized control designs in developing countries and reaches two main conclusions. First, whenever a project is of suf- ficient interest to policymakers to warrant an impact evaluation, program designers ought to consider a randomized control design because this method- ology yields the most robust results. Second, rigorous randomized control designs can often be built into a social sector program when acute resource constraints make rationing of services unavoidable. The second point is not new (Blum and Feachem 1983), but it may be salutary to remind policymakers and program managers that randomized control designs can often be built into Table 1. Evaluation Strategies Control group Type selection criteria Pros Cons Frequency of use None None Very cheap Nothing is Very common learned Reflexive Program Cheap Change in Occasional participants outcome may behavior before be due to other the intervention factors Matched Judgmental Better than Results may not Occasional comparison pairing random when be generalizable target population is small Random Random Statistical; Can be Rare inferences can be expensive drawn from result 182 The World Bank Research Observer, vol. 9, no. 2 (July 1994) a social sector program at relatively low cost. Program managers, rather than academic researchers or evaluation experts, are in the best position to ensure that the opportunities for rigorous evaluation are exploited. These opportunities present themselves whenever, for administrative or bud- getary reasons, the number of eligible candidates exceeds the number of par- ticipants that the program is capable of serving. In developing countries there may not be enough resources to provide the program to all potential beneficia- ries at once or even to all members of a high-priority group. Program managers frequently allocate scarce services by spreading resources evenly but thinly among eligible participants or by tightening the eligibility criteria until the number of people eligible matches the resources available. A common proce- dure is to rank each individual, community, or geographical area according to priorities set by the program, on the basis of such criteria as per capita income or the percentage of households with substandard housing. The cutoff point is then determined according to available funds. Tests are rarely done for the sta- tistical significance of the differences in the indicators used in the ranking. Thus, it is entirely possible that individuals or communities that are observa- tionally equivalent and equally eligible would be assigned different probabili- ties for receiving the program. If all potential beneficiaries are equally eligible, a random draw can be used to select among them, and those who are not selected can serve as controls for those who are. This procedure need not be incompatible with targeting, since eligibility can be restricted to members of a high-priority group. The element of randomization ensures both equity in the allocation process and equivalence in the treatment and control groups. Often, policymakers and program managers believe that conducting an impact evaluation of any type, especially one using a rigorous experimental design, would be too difficult or too costly in a developing country. In this arti- cle, we present a series of case studies that demonstrate that randomized con- trol designs have been used successfully in developing countries and that no insurmountable barriers of knowledge, experience, or cost stand in the way of conducting such evaluations. We also point out some of the design and imple- mentation issues that task managers may face when they try to implement rig- orous evaluations in developing countries and note that such evaluations are not always warranted. In some cases, after weighing what could be learned from an evaluation against the costs of carrying it out, it may make sense to decide not to conduct an evaluation. Most published impact evaluations pay little attention to costs-both the costs of carrying out the intervention and those of conducting the evaluation. Whether the evaluations themselves share this shortcoming or whether the published reports merely fail to provide the information, the outcome is a dearth of published data on the cost of evalua- tion. In the conclusion, we discuss some issues related to costs and provide some practical suggestions on setting up randomized control evaluations in developing countries. John Newman, Laura Rawlings, and Paul Gertler 183 Randomized Control Designs Work in Developing Countries This article presents seven success stories. The seven cases used randomized control design to evaluate the impact of social sector projects ranging from family planning to radio education and mass communication. Randomized control designs have been applied successfully in many diverse settings and programs in developing countries, although they have been used much less often than have other evaluation methodologies and much less often than they have been in industrial countries. Boruch, McSweeny, and Sonderstrom (1978) found that of 400 documented cases of randomized control designs in settings outside of laboratories, less than 5 percent were conducted in developing coun- tries. A review by Cuca and Pierce (1977) found that only twelve of ninety-six family planning program evaluations used randomized control design. Few impact evaluation studies of any type, but particularly those using ran- domized control designs, have been carried out in developing countries in recent years. This scarcity is reflected in the fact that few of our examples are drawn from the 1980s. This seeming reluctance to conduct evaluations some- times appears to stem from a sense that such studies are too expensive and too complicated to justify their use. The real problem, however, may be that eval- uations have been inappropriately applied. Policymakers and program manag- ers may have been discouraged by efforts to evaluate program impacts when the programs themselves were suffering from severe implementation prob- lems.2 An impact evaluation is not the appropriate tool for monitoring whether a program is functioning as it was designed to function. That is the purpose of a monitoring system, which provides inexpensive and timely information on the program and beneficiaries and on whether the program is being imple- mented as intended. To determine whether a program, properly implemented, has the desired effect requires an evaluation strategy that, in addition, collects data from an appropriate comparison group. Monitoring programs can be sim- ple and cheap-indeed, multilateral lending institutions are recommending that monitoring information be produced routinely in all projects that they finance. Evaluation is harder. The advantage of a technically sound impact evaluation is that it can pro- vide convincing evidence of program effectiveness for policymakers. That involves collecting information on a comparison group as well as the treatment group and applying a rigorous design to ensure that differences in outcomes result from the impact of the program rather than from measured or unmea- sured differences between the treatment and control groups. The technical soundness of the design can be instrumental in convincing policymakers of the reliability of a study's findings. The first two case studies, from Nicaragua and Turkey, illustrate how the use of a randomized control design convinced pol- icymakers of the effectiveness of new approaches to learning. The right design can also help policymakers choose among alternative program options, as illus- trated by the Colombia and Taiwan (China) case studies. 184 The World Bank Research Observer, vol. 9, no. 2 (July 1994) The implementation of an evaluation in a developing country can be as important as its design. A program manager setting out to conduct an impact evaluation in a developing country is also something of a pioneer. Typically, there are no consulting firms to call on to carry out the evaluation, as there are in the United States. Political support for the evaluation may be weak or absent. Further, many of the same factors that can make implementing a project difficult-the rapid turnover of staff, political change, sporadic inter- ruptions in cash flow-can make conducting an impact evaluation difficult. At the same time, the budgetary and administrative constraints in develop- ing countries that often make it impossible to reach all potential beneficiaries at once create opportunities for using randomization that are less often encountered in established market economies. The need to ration services and benefits means that a randomized control design can be built into a program's first implementation phases, as happened in the case of the education upgrad- ing program in Bolivia. Evaluating the first part of a phased-in program pre- sents an alternative to a pilot program, which may not accurately predict the effect of the full-fledged program because of differences in the way pilot and full programs are implemented, as illustrated by the experience with the "Ses- ame Street" program in Mexico. In addition, using a randomized control design in the first part of a program can build up valuable experience in con- ducting evaluations in developing countries, making it in many cases a more useful exercise than promoting expensive pilot programs. It is noteworthy that in six of the seven case studies, the programs delivered services to a community rather than directly to individuals, a common practice in developing countries. The experimental conditions required for a random- ized control group design are less likely to be contaminated in a society in which communities are relatively self-contained, as they tend to be in develop- ing countries. (See Grossman in this volume for a discussion of contamination of the control group.) Even when a program is delivered to communities, indicators at both the individual and community levels may be used to measure its impact. The indi- vidual comparisons provide more accurate measurements of the program's impact, but they are statistically more demanding. When programs directed at communities are evaluated using community-level variables, unbiased esti- mates of the impact of the availability of the program on measured community outcomes can be obtained without controlling statistically for the correlation between an individual's decision to participate in the program and the out- come. (See Grossman in this volume for a discussion of the problem of disen- tangling participation and treatment effects.) The use of communitywide averages combines the outcomes for individuals in the treatment community who choose not to participate in the program with those for individuals who do participate. Provided that a sufficiently large number of communities are included in the program and control groups, the measured differences in com- munity-level indicators between the program and control areas would yield John Newman, Laura Rawlings, and Paul Gertler 185 estimates of the expected effect of extending the program to similar, unserved communities. The community-level differences would not, however, yield esti- mates of the potential impact of extending the program benefits to all individ- uals or to a target group of individuals. A related problem is that most programs that require rationing are not assigned randomly to eligible communities, as they were in the Bolivia educa- tion upgrading project. Thus, differences in outcomes across communities may reflect a combination of the program's impact and an explicit or implicit allo- cation rule that may incorporate measured or unmeasured differences across communities. Failure to account for unmeasured differences that are related both to program allocation and to outcomes can yield biased estimates of a program's impact. In projects that require communities or individuals to apply for services, it is especially important that the evaluation be designed to analyze both the decision to apply for services and the impact of the project.3 For example, the Indonesia National Family Planning Coordination Board allo- cates more family planning resources to communities in which contraceptive prevalence is low. One study (Lerman and others 1989) reported a negative correlation between family planning program inputs and contraceptive preva- lence using least squares cross-section multivariate regressions. However, this result says more about the effect of past contraceptive choices on the way the government allocates program inputs than it does about the effect of those inputs on couples' contraceptive choices. Rosenzweig and Wolpin (1986) have pointed out that most of the economic studies that have attempted to evaluate social sector interventions have ignored this problem and have implicitly assumed that program managers randomly allocate programs across communities. They demonstrate that information over time on the spatial distribution of programs and program characteristics can be used to yield unbiased estimates of the effects of changes in local pro- grams on changes in local population characteristics. Working with changes eliminates the influence that unmeasured, fixed characteristics of the commu- nity could have on the outcome.4 Using repeated observations of program interventions and household outcomes in ex post matched comparisons is a promising approach that is worth pursuing. However, substantial improve- ments would have to be made in national information systems to generate and then link adequate information on program interventions (typically collected from community surveys, provider surveys, and administrative records) with household outcomes (obtained from household surveys) before any useful results could be realized. Even good national information systems are not yet designed so that these links can be made easily. The World Bank's Living Stan- dards Measurement Study is encouraging further efforts along these lines. Not all evaluations in developing countries will focus on the impact of expanding services to other groups or individuals. Evaluations have also been used to test the feasibility of introducing changes in the price of services deliv- ered, as is illustrated by the case from Indonesia. The Indonesian case also 186 The World Bank Research Observer, vol. 9, no. 2 (July 1994) underscores some of the political constraints that can be encountered when applying randomized control designs and the tradeoffs that must often be made between these political constraints and the reliability of the eval- uation design. Showing That Radio Education Works: The Radio Mathematics Project in Nicaragua This project used a randomized control evaluation design to assess and dem- onstrate the effectiveness of a new approach to learning-radio education. The positive findings of the evaluation led to the expansion of the radio education program to classrooms throughout Nicaragua and to further use of randomized control in evaluating the effectiveness of radio education compared with that of new textbooks. The Radio Mathematics Project was launched in 1974 by Stanford Univer- sity through the Ministry of Public Education, with the support of the U.S. Agency for International Development (USAID). The aim was to develop and implement a prototype system of radio-delivered mathematics instruc- tion for elementary school students. The project was implemented in four phases-research, pilot-level field tests, standardized tests, and the main field test. The first two years, 1974 and 1975, were dedicated to establishing the project, developing lessons, and conducting pilot tests of the program in first grade classrooms in California and in Masaya, Nicaragua. In 1976 and 1977 schools in the provinces of Masaya, Carazo, and Granada were randomly selected to receive the revised mathematics program. In 1978 the Province of Rio San Juan was also included in the project. School populations were categorized by grade and by rural and urban areas in each province so that the effect of the program on different groups within the population could be assessed. Within each province each qualifying school (any school with at least fifteen first graders) had an equal chance of being in the treatment group or in the control group. Each year, depending on the grade being evaluated, schools were chosen from the list of randomly assigned treat- ment and control groups using a three-step process. First, the number of classes to be chosen from each group was determined. Next, a list of eligible classes was drawn up for each cell in each category (for example, rural control schools in Masaya). Finally, the appropriate number of classes was selected from each list. From 1975 to 1978, this process generated a total of 145 control classes and 257 treatment classes for the evaluation. The radio education program for the first through fourth grades consisted of an hour of mathematics instruction daily throughout the school year, divided into radio instruction and teacher-assisted exercises. During the period when the program was being fully implemented (1976-78), project personnel john Newman, Laura Rawlings, and Paul Gertler 187 administered tests to students in the control and treatment groups both before and after the program aired. Quantitative evaluations showed statistically significant improvements on mathematics tests for students in the first through third grades who received the radio education program (Friend, Searle, and Suppes 1980). For the first grade the mean correct score on the tests was 65.5 percent for the treatment classes, but only 38.8 percent for the control classes. In the second grade the scores were 66.1 and 58.4 percent, and in the third grade, 51.7 and 43.2 percent. For all three grades these differences in scores were statistically significant at the 99 percent level of confidence, that is, it is 99 percent likely that the differences between the control and treatment groups could be attributed to the treatment rather than to chance. Scores for fourth graders were not statis- tically different for the treatment and control classes, but this grade was tested during a period of revolutionary turmoil, when many schools dismissed children before the daily broadcast of the fourth grade lesson-an extreme example of how failure to implement a project as planned precludes meaning- ful evaluation. Qualitative evaluations based on classroom observation and weekly tests also constituted an important part of the overall evaluation. These activities allowed teaching methods to be assessed and refined rapidly and provided valuable feedback to teachers. The qualitative evaluations found students to be attentive and able to keep pace with the worksheets and to learn new skills. Teachers reported satisfaction with the program, which they said reduced their workload and introduced students to new concepts. Explicit efforts were made to build political support for the evaluation. Two advisory committees, with representatives from the Ministry of Public Educa- tion and participating schools, were established to explain the objectives of the program and the evaluation. Briefing sessions were conducted to explain the use of randomized control design and to reassure teachers that the program, not their teaching, was being evaluated. Each eligible school had the same probability of being selected to receive the program, lessening the chance of any school or individual developing feelings of animosity toward a program that had "rejected" them, which could have influenced the results. This success led to a second evaluation using a randomized control design, which confirmed the greater effectiveness of the radio education program in increasing children's learning ability compared with a program that provided additional textbooks (Jamison and others 1981). Since this trial run in Nicaragua, the number of interactive radio-based education programs in devel- oping countries has grown steadily. During the 1980s radio mathematics pro- grams were introduced in Bolivia, Costa Rica, the Dominican Republic, Ecuador, Guatemala, Honduras, Lesotho, Nepal, and Thailand. Interactive radio instruction programs in science, health, Spanish, English as a Second Language (ESL), and teacher training have also spread across the developing world (USAID 1990). 188 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Testing for Lasting Effects: Early Childhood Education in Turkey In 1982 a pilot program headed by the Psychology Department at Bogazici University in Istanbul, Turkey, was initiated to test whether educating lower- income mothers of three- and five-year-olds improves the children's learning abilities. Because the beneficial effects of early-childhood interventions pro- vided directly to children had often been found to dissipate with time, the pro- gram managers hoped that, by educating mothers instead of children, the program would have a lasting effect on children's cognitive abilities. The hypothesis was that the mothers' training would constitute a permanent change in the children's environment. This program was evaluated twice: once at the time of the project, to assess immediate effects; and again nine years later, to find out whether the effects were lasting-an ambitious follow-up program. A series of assessments, tests, and interviews were used to establish a base- line for the project. Three categories of mothers were then selected to receive training: those whose children were attending an educational preschool, those whose children were attending a custodial daycare center, and those who were caring for their children at home. Treatment and control groups were estab- lished through random selection. The treatment group began a two-year, two- part training program that consisted of a cognitive development program for children, implemented through a series of exercises completed by mother and child working together, and an enrichment program that educated mothers about their children's health and education needs. An initial impact assessment was conducted at the end of the two-year train- ing program. The children of mothers who had gone through the program scored significantly higher in measures of IQ, analytical training, and classifi- cation tasks than children in the control group. They also had higher grades, most notably in Turkish and mathematics. Because the initial evaluation showed such positive results, a revised version of the program was extended to other areas of the country, with the support of nongovernment organizations and private industry. A television version of the enrichment training for mothers was also developed and broadcast in a series of eleven short programs. A long-term impact evaluation was recently completed for 217 of the origi- nal 255 participants in the training program-a follow-up rate of 85 percent (Kagitcibasi, Sunar, and Bekman 1993). The evaluation included interviews with the children, now twelve to fifteen years old, and their parents. The results of this study confirmed the hypothesis that changing the environment in which children learn can lead to sustainable improvements in education. One of the most striking long-term impacts of the training is the much higher school retention rates for the children whose mothers participated in the pro- gram: 86 percent, compared with 67 percent for the children of mothers in the control group. Throughout the first five years of primary school, the academic performance and vocabulary test scores of children whose mothers had John Newman, Laura Rawlings, and Paul Gertler 189 received the training were consistently superior to those of children whose mothers had not. In addition, both the children and the mothers who had ben- efited from the training program had significantly different scores for answers on questions that demonstrated self-confidence, attitudes toward academics, and expectations about educational achievement. Testing Alternative Service Delivery Modes: The Taichung Family Planning Program in Taiwan, China In 1962 the Taiwan Provincial Health Department began what was at the time the largest intensive family planning program ever carried out in a city the size of Taichung, which had a population of 325,000. The decision to extend the program to the entire city was prompted by the results of a series of surveys in 1961-62 that revealed a strong demand for family planning ser- vices and a readiness to use a new form of birth control, the intrauterine device (IUD). Information services and supplies were offered for a wide variety of con- traceptive methods. Program officials chose to test the effectiveness of different combinations of services and information by randomly assigning treatments by lin, a neighbor- hood unit averaging twenty households. In all, some 36,000 married couples of childbearing age (couples in which the wife was between the ages of twenty and thirty-nine) were included in control and treatment groups. Four types of treatment were designed, ranging from more intensive and more costly to less intensive and less costly: * Treatment 1: Everything, husband and wife. Personal visits to husbands and wives by trained health workers providing family planning informa- tion and services; mailings to newlyweds and couples with at least two children detailing family planning methods and benefits and identifying the location of clinics; and neighborhood family planning meetings offering in- formation about family planning. * Treatment 2: Everything, wife only. Same as treatment 1, but without the visits to the husband by the health workers. * Treatment 3: Mailings. Only informational mailings, as detailed in treat- ment 1. * No treatment. In addition, the city was divided into three "density" sectors, which differed, insofar as possible, only in the proportion of lins receiving more intensive or less intensive treatments. The density variation was introduced to determine to what extent the beneficiary population could be depended on to spread the desired innovation and to establish how many households within a given area needed to be contacted to stimulate diffusion of the innovation. Differences among the three density sectors were minimized by constructing sectors that were as similar as possible on the basis of measurable characteristics such as 190 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Table 2. Cumulative Acceptance Rates per 100 Married Women Aged 20-39 for All Methods of Birth Control in Taichung (percentage) Density sector Treatment Heavy Medium Light All sectors Treatment 1 20 12 14 17 Treatment 2 18 14 14 17 Treatment 3 8 7 8 8 Nothing 9 7 7 8 Total 14 9 8 11 Source: Authors' calculations, from Freedman and Takeshita (1969). fertility, occupational composition, and education. In the sector designated to receive high-density treatment (928 lins), half the couples were randomly cho- sen to receive an "everything" treatment (treatment 1 or 2). In the sector des- ignated to receive low-density treatment (730 lins), only 20 percent of the couples received an "everything" treatment. In the medium-density sector, 34 percent of couples received an "everything" treatment. Each lin within each sector had the same probability of receiving a treatment because the treatments were allocated randomly by lin. However, the probability of being selected into each treatment category varied according to the treatment density to which the sector was assigned (Freedman and Takeshita 1969). During the experimental period of the program from February 1963 to March 1964, the contraceptive acceptance rate was significantly higher in the high-density sector than in the medium- or low-density sectors (table 2). The variation between medium- and low-density sectors was slight. The experiment suggested that the marginal effect of approaching the husbands (treatment 1) in addition to the wives (treatment 2) was negligible and that the mail cam- paign (treatment 3) was largely ineffective. In 1964, elements of the Taichung program that were considered the most promising-notably house visits by fieldworkers-were replicated throughout Taiwan, and greater emphasis was placed on the availability of IUDs as a method of family planning. Targeting and Random Assignment: The Cognitive Abilities of Malnourished Children in Colombia A pilot program in Cali, Colombia, in 1971-75 was designed to determine what levels of education, nutrition, and health services for preschool children and parents from low-income families would reduce malnutrition and whether these actions could produce improvements in children's intellectual functioning (McKay and others 1978). Medical practitioners had long asserted that inade- quate nutrition impairs a child's cognitive development, perhaps permanently, John Newman, Laura Rawlings, and Paul Gertler 191 but these claims had never been systematically investigated. This case shows that, when program services are to be phased in, a randomized control design can be used even in a program that aims eventually to cover all eligible participants. Random assignment is used simply to determine which groups or individuals receive the program first. This case also shows that achieving an efficient ran- domized control design may require that the target group be identified first. The program was run by the staff of the Human Ecology Research Station, with the support of the Colombian Ministry of Education, the Ford Founda- tion, the National Institute for Child Health and Human Development, and a number of private industries in Colombia. The first step was a multiphase screening survey to identify a target group of malnourished children from among households with four-year-old children. The survey identified general nutritional levels, gathered demographic data, and screened for malnutrition. The 333 malnourished children identified through this process were classified into twenty sectors by neighborhood. Each sector of thirteen to nineteen chil- dren was randomly assigned to one of four treatment groups that differed only in the duration of the treatments, which were staggered over time. Two other groups of children of the same age were formed to allow for qualitative com- parisons with the treatment groups. One group consisted of children from high-income families living in Cali, and the other of children from low-income families who exhibited no signs of malnutrition but who lived in the same neighborhoods and participated in the screening process that had identified the children who qualified for the program. The children in the treatment groups participated in six hours of health- and nutrition-related and educational activities a day, five days a week. The nutri- tional component provided 75 percent of recommended daily protein and cal- orie intake, along with mineral and vitamin supplements. Health care services included daily observations of all children and immediate pediatric attention as warranted. The educational component focused on developing cognitive processes and language, social, and psychomotor skills. Because one of the objectives of the study was to assess how long such a program should last, time-sequencing of treatments formed a crucial part of the pilot program. A randomly selected subgroup from the larger pool of mal- nourished children was assigned to treatment 4, the longest treatment period of 4,170 hours. Over staggered eight-month periods, other randomly selected subgroups received treatments 3, 2, and 1; the last was the shortest, lasting only 990 hours. The children's development was traced over the forty-four months of the program by measuring each child's cognitive ability at equally spaced intervals five times during the study period. The tests measured such indicators of cognitive ability as use of language, spatial relations, quantitative concepts, logical thinking, and manual dexterity and motor control. One problem, how- ever, is that different tests were administered at each measurement point, mak- ing it difficult to compare the test results. 192 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Because children were assigned randomly to the four treatment groups, dif- ferences among the groups could be attributed to differences in the duration of the program. Children who received the longest treatment showed the greatest gains. For children eight years old, results on the Stanford-Binet intelligence tests-reported as mental age minus chronological age-were as follows for the different groups: treatment 1, -15 months; treatment 2, -11 months; treat- ment 3, -9 months; and treatment 4, -5 months. The treatment groups differed from one another in the expected direction-the longer the treatment, the greater the gain-but the differences between adjacent treatment groups were not statistically significant. (It should be noted, however, that the sample sizes were small.) Even with the maximum treatment, none of the groups ever reached the average level of ability shown by children from the nonrandomly selected high socioeconomic group, who had a mental age minus chronological age of +10 months as measured by the Stanford-Binet test. No member of the target group was denied treatment, a factor that facili- tated acceptance of the randomized control design, particularly in the sensitive case of a study of the effects of malnutrition on intellectual development. Testing the Whole Program: The Impact of "Sesame Street" in Mexico A new version of the children's television program "Sesame Street," in Span- ish and adapted to Latin American culture, was introduced in Mexico in 1971. Policymakers in the communications and education fields were interested in exploring the effect of the program on children's cognitive skills. The evalua- tion was designed to assess the effectiveness of the entire program, rather than the relative effectiveness of different strategies, as in some of the other case studies. This case illustrates some of the problems that can occur in moving from a pilot program to broader implementation of the project. A randomized control design was applied to a pilot test carried out in day- care centers serving low-income families in Mexico City in 1971. Two hundred and twenty-one children three to five years old from three daycare centers were divided by age and gender and then randomly assigned to treatment or control groups. Children in the treatment group watched "Plaza Sesamo" for fifty min- utes a day five days a week for six months. Children in the control group watched cartoons. To make sure that children in the control group did not watch "Plaza Sesamo" at home in the evening (it was shown again at 6:00 P.M.), children in that group were kept at the daycare centers until 7:00 P.M.; children in the treatment group left earlier (Hoole 1978). Nine cognitive development tests were administered to the randomly selected control and experimental groups before and after the pilot program began. Statistically significant differences were found for four of the nine cognitive tests administered after the program. The greatest differences were John Newman, Laura Rawlings, and Paul Gertler 193 in the tests of letters and words, general knowledge, and numbers-topics most closely related to the objectives of "Plaza Sesamo." (Differences after the pro- gram in adjusted test mean scores for four- and five-year-olds in the experi- mental group and those in the control group were 7.3 and 4.8 in general knowledge; 4.5 and 5.1 for letters and words; and 7.8 and 6.2 in numbers, all significant at the 99 percent confidence level.) The encouraging results of this pilot test prompted a larger field test. Con- trol and treatment groups were randomly selected from lower- and middle- class preschool children in daycare centers in urban and rural areas. The impact of "Plaza Sesamo" was not as clear in the broad field test, which used a slightly different methodology (the tests were revised, and a rural component was added). The field test was also ultimately less rigorous because of the larger number of dropouts and contamination that occurred because some chil- dren had watched an earlier version of "Plaza Sesamo" at home. However, the evaluators suggested that the difference between the pilot test and the field experiment resulted less from the difference in methodology than from impor- tant differences in the social environments in which the children watched the program (Diaz-Guerrero and others 1976). Essentially, they hypothesized that the presence of a greater number of adults in the laboratory-type setting of the pilot project created a slightly different environment that was more conducive to learning. Because the laboratory-type setting was not replicated when the program was expanded, the nature of the intervention changed. Although the results of the field test were less conclusive, the results of the pilot test helped to generate broad interest in "Sesame Street," not only in Mexico but throughout Latin America. Assigning Services by Lottery: Educational Investments in the El Chaco Region of Bolivia This case study and the malnourished children project in Colombia both illustrate that the targeting of project interventions does not have to rule out the use of randomized control designs for evaluation. When resources are limited, it may be preferable to group individuals or communities on the basis of some rough classification criteria, treat them as observationally equivalent, and conduct a lottery to distribute limited resources, rather than spend funds on more costly information collection activities to target services more narrowly. This approach was followed in a pilot program recently introduced in the El Chaco region of Bolivia to upgrade physical facilities and teacher training in rural public schools (Coa 1992). The program is one of several activities financed by the Social Investment Fund (SIF)-an institution set up by the Government of Bolivia to finance education and health projects in low- income areas-and is also supported by the World Bank and Kreditanstalt fur Wiederaufbau. 194 The World Bank Research Observer, vol. 9, no. 2 (July 1994) To direct interventions to the neediest cases, project managers assigned schools in the region to one of three priority groups on the basis of community characteristics and assessments of the current state of their infrastructure. Rec- ognizing that funds spent making finer distinctions among schools could be better spent on program activities, project managers made no attempt to mea- sure subtle differences among the schools or to rank them in order of priority. All eight schools in the highest priority group were upgraded under the project. The next highest priority group contained 120 schools, but funds were available to upgrade only 54 of them. These schools were selected randomly. This group is of particular interest to policymakers because schools in this cat- egory are the hardest hit by current budget stringencies. Because the allocation rule assigned services to all the schools in the top pri- ority group, the effect of the intervention on that group will be measured using a reflexive comparison design (see the Grossman article for a discussion of this type of design). For the medium-priority group, conditions are right for using a randomized control group design. Baseline information for the evaluation was collected between May and June 1993 using household, community, and school facility questionnaires. A follow-up survey will be conducted one year later, after the project interventions have been completed.5 Combining Randomized Control and Matched Comparisons: The Indonesia Resource Mobilization Study Sometimes evaluations combine randomized control group and matched comparison designs, as in the Indonesia Resource Mobilization Study. This study was designed in 1991 to ascertain the potential impact of user fees on health system revenues, health care utilization, patient's choice of medical care, provider services, and health outcomes and to assess the willingness of patients to pay for improvements in the health care system. The Resource Mobilization Study is one component of the Third Health Project, a set of health care initiatives implemented by the government in the provinces of Kalimantan Timur and Nusa Tenggara Barat to increase the availability and improve the quality of medical services primarily through resource investments (such as new facilities, additional personnel, and more drugs and other medical supplies; see Indonesia 1992). The project has so far been funded by a World Bank loan, but unless the government finds other sources of financing once the loan is expended, the improvements in health care services will not be sustained. For that reason, the government wanted to take advantage of the opportunity presented by the health project to experiment with increases in user fees in two provinces before extending the scheme nationwide. The increases were likely to be less unpopular if they came at the same time as an overall improvement in the quality of services under the health project. John Newman, Laura Rawlings, and Paul Gertler 195 The interaction between the evaluation team and government policymakers led to several important and practical compromises in the design of the exper- iment. The government initially planned to increase fees uniformly across the two provinces, while expanding mechanisms to exempt the poor from having to pay the new fees. The evaluation team argued for delaying some of the fee increases so that experimental control and treatment groups could be studied. Random assignment of the fee increases at the individual level was clearly not practical because health care services are priced at the provider level. Applying the fee increase at the facility level would be difficult as well, because health care prices are set at the district level. Although local officials were eager to increase fees to generate additional revenue, they were reluctant to set different prices at different facilities within the same district for fear of political back- lash. In the end, differences in fees were applied only at the district level, in six districts randomly selected from among the twelve in the two provinces. Fees were one and a half times higher than prevailing rates. Price variations were also introduced among levels of care (such as hospital and health center or inpatient and outpatient care). The small size of the sample of districts subjected to fee increases created sta- tistical problems, so a matched comparison was introduced to strengthen the evaluation design. Treatment and comparison villages were matched not directly on a village-by-village basis, but by comparing the distribution of socioeconomic characteristics of treatment and control villages as groups. First, 110 treatment villages were selected randomly from among the six randomly selected districts. Next, the same number of control villages was selected randomly from among the randomly selected control districts, and the distribution of their socioeco- nomic characteristics (income level, family size, access to medical care, and other data from national household surveys) was compared with that of the treatment villages. The control village that was the least similar to the treatment villages was dropped in favor of another randomly selected replacement village drawn from the control district, and the process was repeated until the comparability of the two groups could no longer be improved. This iterative process-made possible by the availability of national survey and census data on household- and village-level socioeconomic characteristics-substantially improved the fit of the match in one of the two provinces.6 Baseline information was collected in 1991 on the matched treatment and control villages using household, community, and health provider question- naires. Follow-up surveys of the same households and providers were con- ducted in 1993, some eighteen to twenty months after the fee increases. Results of the analysis are expected in the summer of 1994. The collection of data both before and after the fee increases is intended to isolate the effect of the policy reforms from other factors that may have influenced people's use of medical services over time. The control-and-treatment-group design controls for other influences, such as changes in weather, morbidity pattern, and income, that 196 The World Bank Research Observer, vol. 9, no. 2 (July 1994) cannot be controlled for in a reflexive design, which tests the same group before and after the intervention. Conclusion These cases demonstrate the feasibility of conducting impact evaluations using randomized control designs in developing countries. They also demon- strate that there is no single blueprint for conducting evaluations. The evalua- tion designs explored in this article were tailored to the question of interest in the social sector project or treatment being evaluated. Such evaluations are most effective when they seek to answer a clear question of interest to policy- makers and when the intervention itself can be precisely defined and measured: Can radio education improve learning? What is the most effective level of intensity in the provision of family planning services? How will people react to an increase in prices for medical services? More effort needs to be devoted to collecting and reporting information on the costs of carrying out specific interventions. Having that information would allow the outcomes of different kinds of interventions to be expressed in terms of how much they cost to implement rather than in terms of outcome indica- tors that are not directly comparable. Because initial conditions and service delivery levels are often very poor in developing countries, an impact evalua- tion might easily find a sizable absolute improvement in the outcome indicators for given inputs. But it is important to remember that the relevant factor in deciding resource allocation is the opportunity cost of investing in one project rather than another; that is, the expected gains from investing in one project compared with the expected gains from investing in another. More effort also needs to be devoted to collecting information on the costs of conducting evaluation studies. The critical question in deciding whether to conduct an evaluation is whether the expected value of the information obtained is greater than the cost of collecting it. Again, the relevant cost is the opportunity cost of using the funds. If the project to be evaluated is only one of a group of projects expected to have high returns with low risk, the oppor- tunity cost of financing an impact evaluation instead of investing in another project might be high. If the level of uncertainty about what can be gained from the project is appreciable, however, spending the money on an evaluation study probably makes sense. The opportunity cost of investing in a project with low returns can be considerable when other investments could yield higher returns. In addition to concern about the costs of conducting impact evaluations, policymakers and program managers need to be aware of some of the issues involved in setting up an impact evaluation study within a project. Some of the decisions made early on in the design of a project can make an impact evalu- ation easier or harder to conduct later: Who is eligible to participate in the John Newman, Laura Rawlings, and Paul Gertler 197 project? How are the project activities rationed among eligible beneficiaries if resources do not permit delivery of services to all who are eligible? How is the project being phased in? Policymakers and program managers should be alert to opportunities for introducing randomization into program implementation, thus building in possibilities for generating randomized control designs. Ran- domization can be used to allocate a limited number of spaces among equally eligible potential participants, as in the radio education project in Nicaragua. The education upgrading project in Bolivia shows that such opportunistic ran- domization need not be incompatible with targeting interventions to high- priority groups. Randomization may also be built into the plans for expanding a program: the last groups of participants to receive the program's benefits can serve as controls for the first groups. This approach is particularly appropriate in situations where it is ethically untenable to generate a control group that will be denied access to the program altogether. The Colombia case study of the malnourished children project is a good example of the use of this type of randomized control design. In developing countries, the task of organizing an impact evaluation usually falls on program managers. It is rare to find either government agencies that have the capacity to conduct evaluations or local consulting firms that can be contracted to do the work. One way around some of these problems is for pro- gram managers to establish a small evaluation unit, preferably within the project unit. Household data collection can usually be subcontracted from a national statistical institute or a private company. Data on the internal opera- tion of the project, including cost data and monitoring indicators, should be collected as part of the project's management information system. The evalu- ation unit should ensure that data on households, which will provide informa- tion on the outcomes, can be easily linked with the data on project inputs. Freeing personnel in the evaluation unit from direct data collection tasks allows them to concentrate on analyzing the data and bringing the results to the attention of program managers. For some tasks, such as designing the evaluation and analyzing the data, the evaluation unit may need to call on consultants or technical assistance from lending institutions.7 As the Indonesia case illustrates, there are often tradeoffs in the evaluation design that need to be analyzed by experts. The evaluation unit will also require support in addressing some of the conceptual issues involved in analyzing the data, particularly if the evaluation design relies on statistically controlling for differences between participants and nonpartici- pants in measuring impacts. The wide availability of powerful and cheap microcomputers and of user-friendly statistical software makes the task of pro- cessing the data much easier and cheaper than in the past. By demonstrating a project's benefits, impact evaluations can also help to build political support for a project. Impact evaluations can also identify the best ways to carry out particular kinds of interventions and provide convincing evidence for changing or eliminating unsuccessful programs or components, 198 The World Bank Research Observer, vol. 9, no. 2 (July 1994) thereby improving the cost-effectiveness of project interventions. As the devel- opment community embarks on a major increase in social sector spending, it should reconsider the role that impact evaluations can play in ensuring the con- tinual improvement of the quality of social sector investments. Only policy- makers have the power to draw together all the parties involved in a planned intervention, allowing them to debate the merits of conducting an evaluation and of how best to proceed should they decide that evaluation is warranted. Policymakers and program managers need to be aware of the tradeoffs and fea- sibility of the various evaluation options before they can make an informed judgment. Notes John Newman is senior economist in the World Bank's Human Resources Division for Mex- ico and Latin America. Laura Rawlings is a consultant to the Poverty and Human Resources Division of the World Bank's Policy Research Department. Paul Gertler is senior economist at the RAND Corporation. 1. The most common form of evaluation in developing countries, as in industrial countries, is the matched comparison study. Examples of influential matched comparisons conducted in developing countries include those of television-based educational reform in El Salvador (Mayo, Hornick, and McAnany 1976), the Dacca family planning project in Pakistan, the Rajastan ap- plied nutrition program in India (UNESCO 1984), and the Matlab family planning project in Bang- ladesh (Nag 1992; Balk and others 1988). A recent matched comparison is Revenga, Riboud, and Tan (1994) on employment programs in Mexico. 2. Berg (1987) and Binnendjik (1989) discuss some common concerns voiced about impact evaluation studies. 3. For further discussion of the problems involved in disentangling participation and impact, see the Grossman paper in this volume, Heckman (1992), and Manski and Garfinkel (1992). 4. Programs in Indonesia have been the subject of several evaluations that statistically control for the nonrandom placement of programs. Pitt, Rosenzweig, and Gibbons (1993) evaluated the impact of health and education programs on illness rates and school enrollment; Frankenberg (1993) evaluated the impact of health infrastructure on infant mortality; and Gertler and Molyneaux (1994) evaluated the impact of family planning programs on contraceptive preva- lence and fertility. 5. The cost of collecting the data for the baseline and follow-up surveys in the El Chaco area is roughly US$300,000, about 0.4 percent of the total SIF budget of $74.5 million as of May 1993. 6. Both the sample size and the size of the fee increases were selected to obtain a statistical power of more than 80 percent. Power calculations used the national household survey data on health care utilization. 7. For practical information on conducting evaluations, see the "Program Evaluation Kit" put out by Sage Publications, Newbury Park, California, in 1987, which includes books on designing and implementing evaluations. Hoole (1978); Dennis and Boruch (1989); North (1988); and Free- man, Rossi, and Wright (1980) provide useful sources for exploring the developing-country con- text. For general information on evaluations, Evaluation Review may be consulted. For information on evaluation designs, the classic work by Campbell and Stanley (1963) is recom- mended. Fitz-Gibbon and Lyons Morris (1987) also provide practical information on designing evaluations. Rieken and Boruch (1984) provide further discussion of experimental designs in evaluating social programs. John Newman, Laura Rawlings, and Paul Gertler 199 References The word "processed" describes informally reproduced works that may not be commonly available through library systems. Balk, Deborah, Khodezatul Faiz, Ubaidur Rob, J. Chakraborty, and George Simmons. 1988. "An Analysis of Costs and Cost-Effectiveness of the Family Planning-Health Services Project in Matlab, Bangladesh." International Center for Diarrheal Research, Bangladesh. Processed. Berg, Alan. 1987. Malnutrition: What Can be Done? Lessons from World Bank Experience. Bal- timore, Md.: Johns Hopkins University Press. Binnendjik, Annette L. 1989. "Donor Agency Experience with the Monitoring and Evaluation of Development Projects." Evaluation Review 13(3):206-22. Blum, Deborah, and Richard Feachem. 1983. "Measuring the Impact of Water Supply and San- itation Investments on Diarrhoeal Diseases: Problems of Methodology." International Journal of Epidemiology 12(3):357-65. Boruch, Robert, John McSweeny, and John Soderstrom. 1978. "Randomized Field Experiments for Program Planning, Development, and Evaluation: An Illustrative Bibliography." Evalua- tion Quarterly 2(4):655-95. Campbell, Donald, and Julian Stanley. 1963. Experimental and Quasi-Experimental Designs for Research. Chicago, Ill.: Rand McNally. Coa, Ramiro. 1992. "Diseno para la Evaluaci6n de Impacto de las Intervenciones FIS." Fondo de Inversi6n Social, La Paz, Bolivia. Processed. Cuca, Roberto, and Catherine Pierce. 1977. Experiments in Family Planning: Lessons from the Developing World. Baltimore, Md.: Johns Hopkins University Press. Dennis, Michael, and Robert Boruch. 1989. "Randomized Experiments for Planning and Testing Projects in Developing Countries: Threshold Conditions." Evaluation Review 13(3):292-309. Diaz-Guerrero, R., Isabel Reyes-Lagunes, Donald Witzke, and Wayne Holtzman, 1976. "Plaza Sesamo in Mexico: An Evaluation." Journal of Communication Spring:145-54. Fitz-Gibbon, Carol Taylor, and Lynn Lyons Morris. 1987. How to Design a Program Evaluation. Newbury Park, Calif.: Sage Publications. Frankenberg, Elizabeth. 1993. "The Effect of Access to Health Care on Infant Mortality in Indonesia: 1980-87." Dorothy S. Thomas Award Paper presented at 1993 Population Associ- ation of America (PAA) meetings, Cincinnati, Ohio. Processed. Freedman, Ronald, and John Y. Takeshita. 1969. Family Planning in Taiwan: An Experiment in Social Change. Princeton, N.J.: Princeton University Press. Freeman, Howard, Peter Rossi, and Sonia Wright, 1980. Evaluating Social Projects in Developing Countries. Paris: Organization for Economic Cooperation and Development (OECD). Friend, Jamesine, Barbara Searle, and Patrick Suppes, eds. 1980. Radio Mathematics in Nicara- gua. Stanford, Calif.: Stanford University Press. Gertler, Paul, and John Molyneaux. 1994. "How Economic Development and Family Planning Programs Combined to Reduce Indonesian Fertility." Demography 21(1):33-64. Heckman, James J. 1992. "Randomization and Social Policy Evaluation." In Charles Manski and Irwin Garfinkel, eds., Evaluating Welfare and Training Programs. Cambridge, Mass.: Harvard University Press. Hoole, Francis W. 1978. Evaluation Research and Development Activities. Beverly Hills, Calif.: Sage Publications. Indonesia, Ministry of Health. 1992. "Health Care Resource Needs and Mobilization in KalTim and NTB: Interim Results, Health Project III." WD-6281-1-MoH/RI. Jakarta. Jamison, Dean, Barbara Searle, Klaus Galda, and Stephen P. Heyneman. 1981. "Improving Ele- mentary Mathematics Education in Nicaragua: An Experimental Study of the Impact of Text- books and Radio on Achievement." Journal of Educational Psychology 73(4):556-67. 200 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Kagitcibasi, Cigdem, Diane Sunar, and Sevda Bekman. 1993. "Long-Term Effects of Early Inter- vention." Department of Education, Bogadzdi University, Istanbul, Turkey. Processed. Lerman, Charles, John Molyneaux, Soetedjo Moeljodihardjo, and Sahala Pandjaitan. 1989. "The Correlation between Family Planning Program Inputs and Contraceptive Use in Indonesia." Studies in Family Planning 20(1):26-37. Manski, Charles, and Irwin Garfinkel, eds. 1992. Evaluating Welfare and Training Programs. Cambridge, Mass.: Harvard University Press. Mayo, J. K., R. C. Hornick, and E. G. McAnany. 1976. Educational Reform with Television: The El Salvador Experience. Palo Alto, Calif.: Stanford University Press. McKay, H., A. McKay, L. Siniestra, H. Gomez, and P. Lloreda. 1978. "Improving Cognitive Ability in Chronically Deprived Children." Science 200(21):270-78 Nag, Moni. 1992. "Family Planning Success Stories in Bangladesh and India." Policy Research Working Paper 1041. World Bank, Population and Human Resources Department, Washing- ton, D.C. Processed. North, W. Haven. 1988. Evaluation in Developing Countries: A Step in Dialogue. Paris: OECD. Pitt, Mark M., Mark R. Rozenzweig, and Donna M. Gibbons. 1993. "The Determinants and Consequences of the Placement of Government Programs in Indonesia: 1980-86." The World Bank Economic Review 7(3):319-48. Revenga, Ana, Michelle Riboud, and Hong Tan. 1994. "The Impact of Mexico's Retraining Pro- gram on Employment and Wages." The World Bank Economic Review 8(2):247-77. Rieken, Henry, and Robert Boruch. 1984. Social Experimentation: A Method for Planning and Evaluating Social Intervention. New York: Academy Press. Rosenzweig, M., and K. Wolpin. 1986. "Evaluating the Effects of Optimally Distributed Public Programs: Child Health and Family Planning Interventions." American Economic Review 76(3):470-82. Rossi, Peter, Howard Freeman, and Sonia Wright. 1979. Evaluation: A Systematic Approach. Newbury Park, Calif.: Sage Publications. UNESCO (United Nations Educational, Scientific and Cultural Organization. 1984. Project Evalu- ation: Problems of Methodology. Paris. USAID (U.S. Agency for International Development). 1990. Interactive Radio Instruction: Con- fronting Crisis in Basic Education. AID Science and Technology in Development Series. Washington, D.C. John Newman, Laura Rawlings, and Paul Gertler 201 AIDS and African Development Martha Ainsworth Mead Over Because AIDS affects primarily the most productive age group and is fatal and widespread, it will have a larger impact on African development than other more common diseases. Infection rates are higher in urban than in rural areas, and studies suggest that they are highest among urban high-income, skilled men and their partners. Macroeconomic models show that the greater the infection rate among educated workers and the greater the propensity to finance medical care out of savings, the more detrimental is the impact of AIDS on the growth of per capita income. Regardless of the macroeconomic effect, most households and businesses directly affected by AIDS will be economically worse off, at least in the short run. It is not clear, however, what effect AIDS will have on poverty in Sub-Saharan Africa or on income inequality. Governments need to assess the po- tential economic impact of AIDS, implement cost-effective programs to mitigate the impact, and target prevention programs to the economic sectors most sensi- tive to HIV infection. T he World Health Organization (WHO) estimates that by mid-1993 more than 8 million African adults were infected with the human immunodeficiency virus (HIV), the virus that causes AIDS, and an estimat- ed 1.2 million adults had died of AIDS (WHO/GPA 1993).1 Africa accounts for more than 60 percent of the 13 million cumulative cases of adult HIV infection worldwide. Other, more prevalent diseases in Sub-Saharan Africa claim more lives than AIDS and also have a negative impact on economic development-for example, there are probably 110 million cases of malaria in Sub-Saharan Africa each year (World Bank, forthcoming). Why, then, should policymakers be par- ticularly concerned about the economic impact of AIDS? Four characteristics of AIDS and its epidemiology give it an economic impact greater than that of more prevalent diseases.2 First, with current medical tech- nology, AIDS is always fatal. There is no cure and no vaccine. In contrast, fewer The World Bank Research Observer, vol. 9, no. 2 (July 1994), pp. 203-40 C 1994 The International Bank for Reconstruction and Development/THE WORLD BANK 203 than 20 percent of malaria episodes in infants and fewer than 1 percent in adults end in death (Ghana Health Assessment Project Team 1981). Second, AIDS in Africa is affecting primarily prime-aged adults in their eco- nomically most productive years, who acquire HIV mainly through heterosex- ual intercourse, and young children, who acquire it from their mothers at birth. The skills and experience lost through adult deaths from AIDS represent a huge setback to efforts to raise the productivity of the work force. And because adults are often responsible for the care of both young and elderly dependents, increased adult deaths will leave large numbers of surviving family members with lower incomes, poorer health, and reduced prospects. AIDS is also a grow- ing cause of child mortality, which threatens to erase past improvements in child health. Third, in many African countries, HIV and AIDS are already widespread and their economic and social effects are already being felt. Before the AIDS epi- demic, adults in their economically most productive years were dying at a rate of about five to six per thousand persons a year, or eight times the rate in in- dustrial countries. In many African countries, particularly in urban areas, the baseline adult mortality rate has doubled or tripled, and AIDS is now the major cause of adult deaths. And those who have died of AIDS represent only a frac- tion of those who are infected. The high prevalence (percentage of people in- fected) of HIV in Africa and the long incubation period for AIDS imply that disability and death from HIV and AIDS will hold back African development well into the next century, even if prevention efforts are successful. The need is thus urgent to invest in programs to prevent and slow the spread of HIV and to mitigate the economic and social impact of AIDS.3 Fourth, unlike other major causes of premature death in Africa, AIDS has not spared the elite. The degree to which AIDS will affect economic growth will depend largely on the severity with which it strikes educated Africans. Although the channels through which AIDS will affect economic growth are clear, the likely severity of that impact is still not well documented-because of a dearth of studies measuring current impacts and uncertainties about the future spread of the disease. This article provides an overview of the likely im- pact of the AIDS epidemic on African economic development. The first half ex- plores the aggregate economic effect of AIDS-its demographic impact and its effect on growth in per capita income. Though there are numerous predictive models employing a wide range of assumptions, there is little empirical infor- mation to inform the choice of assumptions. In particular, the aggregate eco- nomic impact of AIDS will depend on the behavioral responses of economic agents to demographic shocks. The second half of the article reviews what is known about the coping behavior of households, firms, and government in such areas as medical care, labor and productivity, and education. This under- standing is important for improving aggregate models and for exploring the distributional implications of AIDS. 204 The World Bank Research Observer, vol. 9, no. 2 (July 1994) The Aggregate Effect on the Economy The size of the AIDS epidemic, now and in the future, and its potential effects on the growth rate of the population and the health of the labor force are the starting point for any discussion of the economic impact of AIDS. Four steps are involved in such projections: * Estimating the current rate of HIV infection in the adult population as a whole and within economically important categories * Projecting the path of HIV infection in each of these population groups * Computing the demographic consequences of the path of HIV infection * Estimating the economic consequences of these demographic changes. Estimating the current rate of HIV infection with the data at hand is difficult. Projecting infection rates is even more difficult, requiring detailed knowledge about the sexual behavior of different epidemiological and demographic groups in the country. Modeling these behaviors is important because of what it can tell us about the cost-effective targeting of interventions to these different groups (Over and Piot 1993; Bos and Bulatao 1992; Anderson and others 1991). This approach implies a unidirectional causal link from HIV infection to eco- nomic consequences. It does not allow for rational economic agents to respond to the perceived growing threat of AIDS-and to the price changes it sends rip- pling through markets-by changing their behavior and thereby changing the HIV incidence rate (percentage rate of new infections among uninfected peo- ple). Some authors have taken the first step toward developing such a model by simulating the impact on the future epidemic of specific changes in behavior (Bulatao 1991; Way and Stanecki 1991; Armstrong and Bos 1992). A few au- thors are beginning to model human sexual behavior (Philippson and Posner 1993; Kremer 1994). No one, however, has yet produced a model that allows feedback in both directions. An approach that allows behavioral responses to affect the future spread of Hiv would be theoretically satisfying, but the long period between a change in sexual behavior and the demographic consequences of that behavior-as well as the subsequent lag between the demographic changes and their economic consequences-suggests that the gains in understanding from such an approach would be apparent only in projections covering many decades. For medium- term projections, a model that treats the epidemiology and demographic con- sequences of AIDS as unaffected by the economic projections is likely to be suf- ficiently accurate. HIV Prevalence in the General Population If we had good data on seroprevalence (the percentage of people found to be HIV-positive through blood tests) in the adult population as a whole in each African country by time of infection, we could predict when these people Martha Ainsworth and Mead Over 205 would come down with AIDS. But systematic national seroprevalence surveys have been conducted only in Rwanda (1986), Uganda (1988), and Cote d'Ivoire (1989), and for none of these countries do we have a series of annual surveys. Rather, the available data consist of information on conveniently sampled sub- groups of the population, such as pregnant women and blood donors. We can plot estimates of the percentage of "low-risk" adults infected with HIV in 1993 in rural and urban areas of the fourteen African countries with the highest in- fection rates, using the U.S. Census Bureau data set on HIV seroprevalence rates (figure 1).4 For rates in the population at large, we can then extrapolate from these data. Urban or rural residency is the economically relevant variable most often collected in studies of HIV prevalence. For many African countries, data on ru- ral and urban levels of infection can be used to estimate the national infection rate. One approach is to calculate a weighted average of the urban and rural low-risk prevalence rates, using the urban and rural proportions of the adult Figure 1. Rural and Urban Rates of HIVInfection Among Low-Risk Adults in Fourteen African Countries, circa 1993 Rural adult prevalence (percent) 18 - Zambia 16 - 14- Zimbabwe 12 - Tanzania REGRESSION 100 Central l wan African Republic \ Rwan 8- 6- ~~~~Kn~a. Botswana 6- Burkina Faso /Uganda Cameroon Zaire Ce d'Ivoire 2 - Burundi 20% OF URBAN 0 0 5 10 15 20 25 30 35 Urban adult prevalence (percent) Source: U.S. Census Bureau (1993). 206 The World Bank Research Observer, vol. 9, no. 2 (July 1994) population as the weights. Applying this approach to the forty-seven Sub- Saharan African countries in the U.S. Bureau of the Census data set on sero- prevalence rates yields estimates of 3.4 million urban and from 1.9 to 4.3 mil- lion rural HIv-infected adults in mid-1993.5 Assuming that about one million HIV-infected persons have died since the beginning of the epidemic, the larger total of 7.7 million infected adults is consistent with the 8 million cumulative infections predicted for mid-1993 by WHO on the basis of its confidential data-suggesting that WHO's data may not differ much from the Census Bu- reau data base. Two frequency distributions of the national adult seroprevalence rate for the forty-seven Sub-Saharan countries for which data are available are presented in figure 2. One assumes conservatively that rural prevalence rates are one-fifth of urban rates; the other uses the rural prevalence rates in the data set or esti- mates them from a regression of the prevalence rates in figure 1. Weighting each country's estimated prevalence rate by its population yields seropreva- Figure 2. Frequency Distribution of National Adult Infection Rates for HIV-1, 1993 Number of Sub-Saharan countries 35- 30- 30 Rural calculated as 20% of urban E Actual or estimated rural 26 25- 20 15 10 10 6 5 4 4 4 4 0 0 0 to 4 4 to 8 8 to 12 More thanl12 Percentage of adults infected with HIV Source: Authors' calculations based on U.S. Bureau of Census data; see text. Martha Ainsworth and Mead Over 207 lence rates for the adult population of Africa of 2.5 for the conservative case and 3.6 percent for the other case. The Current Rate of HIV Infection by Economic Group A full understanding of the economic impact of AIDS requires information not only on its average prevalence but also on whether prevalence differs for different economic groups. It is often stated that HIV is a disease of poverty.6 The most frequently cited evidence to support this assertion is the greater es- timated number of HIV-infected adults in rural areas of Sub-Saharan Africa than in urban areas, where almost all adults with at least a secondary school education reside. But much less certainty can be attached to rural than to urban infection rates. Estimates of the number of rural adults infected with HIV bracket the more accurately estimated urban number. For the purpose of un- derstanding the epidemic's economic impact, however, the number of infected people is less important than the percentage infected in each economic group. And even the larger estimate of the number of rural adult infections-some 27 percent larger than the urban number-corresponds to a rural infection rate only half the urban rate. By its disproportionate effect on the urban population, HIV presents a sharp contrast to other causes of death. A comparison of adult mortality in seven Sub-Saharan countries in the 1970s and 1980s (before AIDS could have signifi- cantly affected the patterns) found higher urban life expectancies at age fif- teen-by as many as six years-in every case (Timaeus 1993, table 6-3). Beyond the simple but compelling correlation between HIV infection and ur- ban residence, the evidence on the correlation of HIV infection with socioeco- nomic status is incomplete. For the purpose of predicting the economic impact of current infection patterns, it would be sufficient to have a multivariate anal- ysis of infection rates by education level, while holding constant age, gender, and ethnic group. But most epidemiological studies either omit socioeconomic indicators altogether or apply only bivariate analysis. The bivariate analyses beg the question because any measured correlation between HIV infection and education might be due to the fact that, for example, older individuals are more likely to be both HIV infected and educated. Those that do apply multi- variate analysis include among the explanatory variables such "risk factors" as marital status, number of sex partners, history of sexually transmitted diseases, occupation, and amount of travel. The inclusion of the "risk factors" is un- helpful, because they are outcomes of the same set of decisions that could lead to HIV infection. Their inclusion in the analysis serves only to mask the rela- tionship between infection and the true independent variables.7 Suppose that changing sexual partners is positively related to socioeconomic status for men and negatively related to status for unattached women because of differences in social roles. If this were the case, one would predict, after con- trolling for other determinants of casual sexual activity such as age and 208 The World Bank Research Observer, vol. 9, no. 2 (July 1994) ethnicity, that men with higher status who are unaware of the risk of HIV in- fection would have a higher rate of HIV infection than similar men of lower status. Furthermore, the spouses of these men would share their risk-resulting in higher infection rates for women whose husbands have higher status. Unat- tached women, particularly prostitutes, would have lower infection rates at higher status. A few studies seem to confirm this hypothesized relation between socioeco- nomic status and sexual behavior or HIV infection. Preliminary results from multicountry surveys sponsored by WHO during the late 1980s show a mono- tonically increasing relationship between male schooling and the number of ca- sual or commercial partners over the past twelve months for the Central African Republic, C6te d'Ivoire, Kenya, and Togo (Cleland and others 1992; Carael and others 1990), although no multivariate analysis of the data has yet been published. In urban Rwanda in 1987 the rate of HIV infection in women attending prenatal clinics was not correlated with their own socioeconomic sta- tus, but rose with the education and income of their husband or partner (Allen and others 1991). Infection rates also differed by partner's occupation, ranging from 9 percent for women whose husbands or partners were farmers to 30 per- cent for women whose husbands were in the private sector or government (ta- ble 1). Applying a multivariate analysis to the data, the Rwandan study found that the odds of HIV infection were 1.96 times higher among women with high- income partners (more than 10,000 Rwandan francs a month) than for women with lower-income partners.8 Education, occupation, and age had no statisti- cally significant effect. However, these results are biased because they include as covariates many behaviors that are themselves determined by socioeconomic status-such as number of partners and presence of other sexually transmitted diseases. Thus, the results should be interpreted as the effect of income, hold- ing constant other behaviors also determined by income. An ongoing study of a population sample from the Rakai District of Uganda finds a strong, statistically significant, and positive association between educa- tion and HIV infection for both women and men, based on a bivariate analysis. Men and women with secondary educations have infection rates (20 and 41 percent) more than twice as high as men and women with no education (8 and 14 percent; Serwadda and others 1992). Multivariate analysis of the data is confounded by the inclusion of risk factors as though they were independent variables. Studies of workers in a textile factory and a large bank in Kinshasa, Zaire, in 1987 found that the risk of HIV infection rose with socioeconomic status (Ryder and others 1990). Since the studies controlled for area of resi- dence (urban), gender (almost all workers were men), and travel pattern (em- ployers of the same enterprise are more likely to share travel patterns than subjects chosen in a random survey or in other kinds of convenience surveys), the absence of multiple regression analysis is less restrictive in their interpreta- tion. Seroprevalence rates were higher among male employees of the bank (5.8 percent) than among male employees of the textile factory (2.8 percent). The World Bank Research Observer, vol. 9, no. 2 (July 1994) 209 Table 1. Percentage of Women Who Are HIV-Positive, by Socioeconomic Status of Their Male Partners, Kigali, Rwanda, 1987 Socioeconomic status of partner Percentage of HIV-positive women Education 0-4 years 18 5-7 years 32 8 years or more 34 Monthly income (Rwandan francs) None 22 1-9,999 25 10,000 or more 35 Occupation Farmer 9 Military 22 Private sector 32 Civil service 38 Note: Total number of respondents, 1,350. Source: Allen and others (1991). Seroprevalence rates varied from 2.8 percent for manual workers to 4.5 percent for foremen and 5.3 percent for executives in the textile factory (Ndilu and oth- ers 1988) and from 4.2 percent of manual laborers to 6.8 percent of high-in- come employees in the bank (Ryder and others 1990). A study of adults in Monze District, Zambia, that controlled for age, gender, and urban or rural residence also found that skilled workers and traders were more likely to be infected (Buve and Foster 1993). Data on the infection rate of unattached women come largely from studies of prostitutes. The few studies that have collected information on the socio- economic status of their subjects uniformly find that prostitutes that charge more for their services have fewer sexual contacts and, presumably as a result, lower infection rates (Kreiss and others 1986; Denis and others 1987, cited in Padian 1988). Some observers predict that the gap in seroprevalence between rich and poor, urban and rural, will eventually narrow. There is evidence, for example, that many urbanites return to their village of origin when they come down with AIDS, lowering urban prevalence and raising rural prevalence even in the absence of any behavioral change (Buve and Foster 1993). But if the most im- portant cause of differences in prevalence in urban and rural areas and by in- come is the higher frequency of unprotected sex with new partners among urban high-income males and low-income single females, and if this behavior 210 The World Bank Research Observer, vol. 9, no. 2 (July 1994) continues, epidemiological models predict that the gap in prevalence will continue (Over and Piot 1993). The gap will also continue if both groups change their behavior by the same amount (for example, by increasing the fre- quency of condom use). Only if groups that currently engage in riskier behav- ior systematically lower their risk while low-risk groups do not, will the gap narrow. Such an outcome is a possible result of greater responsiveness by bet- ter-educated men and by prostitutes to the fear of HIV infection, but it is by no means assured. The study of sexual behavior as it affects the rate of new HIV infection (the incidence rate) will require fully developed structural models that characterize the simultaneous determination of epidemiological risk fac- tors, sexual behavior, and infection. There is a need for new data and an even greater need for new analyses of existing data that include socioeconomic as well as epidemiological variables. Projecting Infection Rates Given the estimates of current HIV infection rates in Sub-Saharan countries derived above, the next task is to project infection rates. Because the median incubation period of HIV infection is as long as ten years, current patterns of infection can be used to predict mortality patterns five to ten years in the fu- ture. However, over longer periods, differences in the structure and parameters of alternative models used to project the progression of infection rates in an African country lead to divergent outcomes. (The models have been discussed in several surveys and so are not described here; United Nations/WHO 1991; Stoto 1993; Stover 1993a; Chin and Sato 1994.) At an upper extreme of a range of assumptions about sexual behavior, Anderson and his coauthors have shown that national rates of adult HIV infec- tion could rise to as high as 40 percent over fifty years (Anderson 1988; Ander- son and others 1991). Other analysts, using more modest assumptions about sexual behavior, have projected ceilings of 10 to 20 percent (Bulatao 1991; Bon- gaarts 1990). Data from WHO-supported national surveys of sexual behavior in the late 1980s suggest that even the lower rates of change in sexual partners assumed by Bulatao exceed those observed in Africa (Cleland and others 1992). The release of recently collected WHO data on behavior should lead to substan- tial new understanding of the possible future trajectories of infection rates in African countries. In the meantime, it is important to perform sensitivity anal- ysis on estimates of impact with respect to the national seroprevalence rate and its distribution by socioeconomic class. Demographic Consequences of HIV Infection Compared with projecting infection rates, projecting the demographic con- sequences of infection is straightforward, requiring only assumptions about the Martha Ainsworth and Mead Over 211 effect of HIV infection on mortality and fertility. Given these assumptions, de- mographic models routinely compute predictions about the demographic variables that influence a country's economic growth: the size and health of the labor force and its age distribution. If infection data were available by socio- economic group or sector of employment, it would be useful to extend the models to predict the size and health of these subsets of the labor force. IMPACT ON MORTALITY AND FERTILITY. The demographic impact of an HIV epidemic depends on how it affects mortality and fertility. Stover (1993a, b) notes that all epidemiologic and demographic models agree that the higher death rates due to AIDS would eventually reduce population growth to zero if the HIV infection rate among adults rose high enough. He also derives a rela- tionship between the prevalence rate and the proportion of the work force that is sick in any given year (Stover 1993b). Both relations are approximately linear over the range of interest, and both share a critical dependence on the length of the "incubation period" of HIV infection-the time from HIV infection to AIDS. The importance of the incubation period is sufficient to warrant a review of what is known about its length. The impact of HIV on mortality depends on the distribution of the lag be- tween infection and AIDS and between AIDS and death. The time profile of HIV incubation is relatively well known for homosexual and bisexual men in indus- trial countries (Moss and Bachetti 1989; Rutherford and others 1990). A lon- gitudinal medical study of a cohort of homosexual and bisexual men in San Francisco, launched in the 1970s before the advent of the AIDS epidemic, has drawn blood from participants every six months. Researchers were able to test for HIV and to determine within a six-month interval when each participant became infected. In this ongoing study, the median time from infection to de- velopment of AIDS is ten to eleven years, with an almost constant hazard rate of converting from infection to AIDS of about 9 percent a year. No one knows whether this distribution of incubation time is applicable to epidemics among other population groups and in developing countries (Ryder and Mgerwa 1994). Indirect methods suggest that the time from HIV infection to AIDS is shorter in Sub-Saharan Africa. Among 101 HIV-positive but asymp- tomatic hospital employees in Kinshasa, 31 percent developed AIDS-related ill- ness or AIDS over a period of two years, much higher than the 18 to 22 percent that would have been predicted on the basis of the San Francisco data (Ryder and Mgerwa 1994, p. 271). Researchers working in Kagera Region, Tanzania, measured the HIV prevalence rate in 1989 and the incidence rate between 1989 and 1991. Applying the model of a stable epidemic, which they admit may not be applicable in Kagera, the researchers tentatively estimate the average dura- tion of a case of HIV infection as the ratio of prevalence to incidence. For the portion of Kagera Region where prevalence and incidence were highest, and therefore most accurately estimated, the prevalence rate was 14 percent and the incidence rate 1.8 percent, implying an average duration of infection of seven 212 The World Bank Research Observer, vol. 9, no. 2 (July 1994) years (Killewo and others 1993). Stover (1993a, b) suggests that the mean in- cubation period might be as low as five years or as high as ten years in Sub- Saharan Africa. Indeed, it could vary from country to country. The other important variable affecting the demographic consequences of HIV infection is the fertility rate. An elementary conclusion of the demographic modeling is that HIV infection will not sufficiently shorten the lives of a large enough number of women to reduce national birth rates unless infection lowers age-specific fertility rates.9 By the time most Hlv-infected women die, they have already given birth to several children (Stover 1993a, b). Behavioral responses to the AIDS epidemic can affect individual fertility, however. For example, HIV- positive couples may decide to avoid pregnancy rather than risk giving birth to children who will be infected with HIV or who may soon be orphaned. Other behaviors related to reducing the spread of HIV-such as delayed marriage, less sexual activity, and greater use of condoms-may also reduce fertility (Cald- well and others 1993). Conversely, uninfected parents might respond to the higher mortality rate of children who will be infected with HIV as adults by having more children, to guarantee a certain number of surviving offspring for their security in old age. There is no good evidence about which of these effects might dominate. It is inherently difficult to deduce how fertility is responding to the epidemic. Comparing fertility rates of Hlv-infected and uninfected women fundamentally confounds the effect of HIV infection with differences in the characteristics of these women that led to their different HIV status and that also affect their fer- tility. For instance, prostitutes may be more likely to be infected than other women and, because of infertility caused by other sexually transmitted diseas- es, to have fewer children regardless of their HIV status. Even among prosti- tutes who are pregnant, the probability of having another child is likely to be less than that of other women, and prostitutes are more likely to be Hlv-infect- ed than are average pregnant women. Thus, without holding constant these confounding characteristics, one cannot deduce the effect of HIV infection on fertility by comparing the fertility behavior of women subsequent to their hav- ing a child and being informed of their HIV status. Two studies in Zaire have nevertheless attempted to gauge the impact of HIV infection and counseling on fertility. A study followed 553 women for thirty-six months after delivery at Mama Yemo Hospital in Kinshasa, Zaire, and found that HIV-positive women had lower annual pregnancy rates than HIV-negative women (245 per 1,000 compared with 316 per 1,000) and that fer- tility was lowest among symptomatic HIV-positive women (Ryder and others 1991). A second study at the same hospital examined the impact of an HIV- counseling and testing program on 364 postpartum women and found, after a twelve-month follow-up, that 6 percent of HIV-positive and 8 percent of HIV- negative women had become pregnant (Heyward and others 1993). For the rea- sons just mentioned, these studies are difficult to interpret, but they suggest Martha Ainsworth and Mead Over 213 that HIV-positive women are unresponsive to counseling and testing programs designed to prevent births. Because of the lack of evidence and the possibility that fertility could be af- fected in either direction, most demographers have assumed that fertility will slowly decline through the coming decades just as it was projected to do in the absence of the AIDS epidemic. None of the demographic models takes into ac- count the possible effect of AIDS-related behavioral changes on fertility. PROJECTING DEMOGRAPHIC EFFECTS. The uncertainty about the incubation pe- riod in Africa translates into some uncertainty about the effect of a sustained prevalence of HIV infection on the growth, health, and age distribution of the labor force. Assuming that fertility rates follow the path they would have taken without HIV infection and that the incubation period is ten years, Stover (1993a, b) calculates that it would take a sustained prevalence rate of 48 per- cent of the adult population to halt population growth in an African country that would otherwise be growing at a rapid 3 percent a year. If the incubation period is only five years, a 30-percent seroprevalence rate is sufficient to bring population growth to zero. Furthermore, the relationship between these values is a linear function of the prevalence rate as depicted in figure 3 (Stover 1994, personal communication). Consider the cases of a 2 and a 4 percent annual base growth rate of the population (figure 3). For the country that was growing at 4 percent before the onset of the AIDS epidemic, an incubation period of five years means that the population growth rate would fall to zero if the national seroprevalence rate rose to 40 percent (the intersection of the solid curve and a zero rate of popu- lation growth on the left axis of figure 3). If the incubation period is ten years (the dashed curve), the population would stop growing only if the adult prev- alence rate exceeded 50 percent. Such extreme prevalence rates have been observed in high-risk groups in urban areas but are unlikely to prevail across an entire country. With an underlying population growth rate of 2 percent a year (right axis), however, the sustained national seroprevalence rate among adults that would reduce population growth to zero drops to 32 percent for an incubation period of ten years and to 20 percent for an incubation period of five years. Two lessons can be drawn from this exercise. First, zero or negative popu- lation growth is not a likely outcome for an entire country in Africa, although it could occur in areas where infection rates are extremely high if there were no offsetting in-migration. Second, the impact of a change in the sustained prevalence rate on population growth can be estimated, regardless of current levels of prevalence or population growth, if the incubation period is also known. Every ten-percentage-point increase in the prevalence rate will, accord- ing to figure 3, reduce population growth by between 0.6 and 1 percentage point a year, depending on the incubation period. 214 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Figure 3. Impact of a Sustained HIV Infection Rate on the Population Growth Rate Population growth starting at 4 percent Population growth starting at 2 percent \'.2 Ten-year incubation 2------------------------------ 0 Five-year incubation 1 .-- < --------- -: -1 0 10 20 30 40 50 Percentage of adults infected Note: Assumes fertility rate is unchanged by the HrV epidemic. Source: Authors' derivation based on Stover (1993a, b, c). Impact on Growth of per Capita Income Despite different approaches and assumptions, the various demographic- epidemiologic models agree that the AIDS epidemic will lead to a steep rise in mortality rates among economically active adults and among young children. Life expectancy will decline. Assuming that the slow decline in fertility is not affected by the epidemic and that seroprevalence rates stay below 30 percent among adults, population growth rates will be slowed but will stabilize at pos- itive levels nationally. In addition to slowing the growth of the population, and therefore of the labor force, the AIDS epidemic has a second direct and immediate effect on na- tional output: it makes people sick. With a survival rate of about one year for Africans who contract AIDS, Stover (1993b) shows that in a steady-state epi- demic, the portion of the HIV-infected adult population that will be sick with AIDS is determined largely by the incubation period. An incubation period of Martha Ainsworth and Mead Over 215 five years means that 14 percent of the HIV-infected population would be sick in any year; the rate falls to 9 percent with an incubation period of ten years. Every ten-percentage-point increase in the prevalence of HIV infection increases the share of the adult labor force that is sick during the year by 0.9 to 1.4 per- centage points. At these slower population growth rates, up to 4.2 percent of the adult work force will be suffering from AIDS. Higher mortality caused by AIDS is a tragedy to those who die and to their families. But if the epidemic slows the growth of the gross domestic product (GDP) by less than it slows the growth of the population and the labor force, the negative economic effects of AIDS will be partially offset by increased growth in per capita income. Under what conditions will a high death rate from AIDS result in higher average welfare among those who survive? If national output is produced by two factors of production, workers and capital, then the growth rate of output is simply a weighted average of the growth rates of these two productive factors. In a world of constant returns to scale, the weights are the output elasticities of the two factors and they sum to one. In this simple world, the impact of an HIV epidemic with a sustained na- tional prevalence of, for example, 10 percent is the weighted average of its ef- fects on the growth of capital and healthy labor. When HIV prevalence is rising, the growth of effective labor will temporarily fall even lower as the share of workers who are sick rises and their efficiency falls. But as soon as the prevalence rate levels off, the rate of sickness will also level off, and the only effect of the epidemic on the growth of effective labor will be through its effect on the number of workers. From figure 3, every ten- percentage-point increase in the epidemic will slow the growth of the work force by 0.6 to 1.0 percentage points a year. How likely is it that a sustained HIV prevalence rate of 10 percent will slow the growth of capital by as much as 0.6 to 1.0 percentage points a year and thereby slow the rate of per capita income growth?10 The HIV epidemic will slow capital accumulation primarily by inducing a partial reallocation of pri- vate and public resources from savings and investment to medical treatment of AIDS patients, social support payments for orphans, and, increasingly, pro- grams to prevent the spread of HIV (see the following section on coping with AIDS). Following Cuddington (1992) and Over (1992), the savings consumed by AIDS treatment during the year are determined as follows, National Proportion savings Number Proportion Proportion Annual of of HIV+ treatment (1) diverted = of x of workers x workers cost ofx treatment by AIDS workers HIV+ with AIDS AIDS cost from treatment savings The magnitude of this diversion of savings thus depends partly on epidemi- ological factors-the HIV prevalence rate and the proportion of HIV-positive workers who are sick-and partly on two economic factors-the annual cost of treating an AIDS sufferer and the way that cost is financed. Annual medical 216 The World Bank Research Observer, vol. 9, no. 2 (July 1994) costs and the way they are financed differ both within and among developing countries (Over 1992; Over and others 1988). Cost and financing can also be influenced by public policy, as countries and individuals decide whether to buy expensive antivirals that achieve only limited additional healthy life. (Over and Piot 1993 provide estimates of the cost-effectiveness of alternative ways of treating AIDS in a developing country.) One of the most obvious effects of the AIDS epidemic will be increased pri- vate and public expenditure on medical care. There is virtually no representa- tive empirical evidence on how households finance AIDS medical costs or on other effects of the epidemic on savings behavior. Households may finance part or all of medical costs out of savings. According to anecdotal evidence, some households have sold land or productive assets to finance medical care (Ankrah and others 1992; Anarfi 1992, cited in Caldwell and others 1993). We do not know to what extent this behavior differs from that associated with hospital- ization of adults for other causes. Many households presumably finance med- ical care from transfers from other households (Anarfi 1992 cited in Caldwell and others 1993), but these transfers must also be financed. Households with little or no savings may reallocate their existing medical budget from other household members to the person with AIDS (McGrath and others 1993) or re- duce the consumption of other household members. But even if medical costs are financed from savings, anticipatory behavior by households may result in increased savings that offset spending on medical care. For example, at least one anthropological study has documented attempts by infected adults to build a house or otherwise provide for family members who will survive them (Ankrah and others 1993). The prospect of contracting HIV may induce unaffected individuals to save more against the day when they may need to spend more on medical care if they become infected or to support themselves in the event that their children die prematurely of AIDS. The next section of the paper reviews evidence on the magnitude of AIDS treatment costs and suggests the empirical regularity that these costs average from two to four times per capita income. Substituting the ratio of average medical treatment costs for AIDS to per capita income into equation 1 and di- viding by total national capital stock gives an expression for the percentage reduction in capital stock each year due to AIDS treatment costs: Average Proportion medical costs Proportion Annual of HIV+ x of HIV+ for AIDS x of treatment percentage workers workers treatment cost from percentage workers wihAD reduction with AIDS GDP savings (2) in capital per capita due to = x 100 AIDS Capital-output ratio treatment Note that the denominator in equation 2 is the capital-output ratio, which also commonly ranges between two and four. If the ratio of average medical Martha Ainsworth and Mead Over 217 treatment costs for AIDS to per capita income and the capital-output ratio can- cel out in the typical country, the impact of HIV prevalence on capital stock will be determined by the product of the three remaining proportions in the numerator. An HIV prevalence rate of 10 percent of the population, some 10 percent of them sick with AIDS in a given year, and a reallocation from savings of about half of AIDS treatment costs would result in a reduction in the growth rate of capital of 0.5 percentage point a year (0.1 x 0.1 x 0.5 x 100). The impact on the rate of growth of capital is of the same order of magni- tude as the reduction in the growth rate of labor that such an epidemic would produce. That means that the direction and size of the epidemic's impact on per capita income will be determined by the exact values of the parameters in the two equations above, which will differ from country to country. Sensitivity analysis of the two equations with respect to two epidemiological parameters (the prevalence rate and incubation period of the epidemic) and three economic ones (the capital-output ratio, the ratio of AIDS treatment cost to per capita income, and the proportion of AIDS treatment costs financed from savings) re- veals that the direction of the epidemic's impact on per capita income is deter- mined by the economic parameters, while the size of the impact is determined by the epidemiologic ones. If more than half of treatment costs are financed from savings and the ratio of the cost of treating an AIDS case to per capita income is more than twice the capital-output ratio, the epidemic will reduce the growth rate of per capita income. The magnitude of that reduction will range from 0.1 percentage point a year if the national prevalence rate is 10 per- cent and the incubation period is ten years to 0.8 percentage point a year when the prevalence rate is 30 percent and the incubation period only five years.11 Studies have simulated the macroeconomic impact of AIDS under alternative assumptions about the share of treatment costs financed from savings, the productivity of workers with AIDS, and the impact of AIDS on population growth (Cuddington 1993; Cuddington and Hancock 1992; Over 1992; Kambou, Devarajan, and Over 1992). Under the assumptions that population growth slows by 0.7 percentage points, that treatment costs are financed en- tirely from savings, and that a worker with AIDS is half as productive as a healthy worker, Cuddington (1993) predicts that GDP growth in Tanzania would slow by 0.8 percentage points and that growth in per capita GDP would slow by 0.1 percentage points. For Malawi, under similar assumptions on treatment costs and productivity but assuming a larger slowdown in popula- tion growth of 1.2 percentage points, Cuddington and Hancock (1992) predict that AIDS would reduce GDP growth by 1.5 percentage points and per capita income by 0.3 points. (To get the growth rate of GDP per capita to increase, the authors had to assume that AIDS treatment costs had no effect on national saving and that people with AIDS were as productive as healthy members of the work force until death. Both of these assumptions are clearly implausible.) These models ignore existing labor market rigidities, such as a high mini- mum wage or restrictions on firing workers, that would be expected to slow 218 The World Bank Research Observer, vol. 9, no. 2 (July 1994) an economy's adaptation to the epidemic. They predict a smaller impact from AIDS than would be the case were these rigidities also considered. These simu- lations also assume that the work force is homogeneous in terms of education and productivity. Over (1992) adds realism to the models by incorporating two sectors (a high- productivity urban sector and a low-productivity rural sector) and three types of labor (the uneducated, those with primary schooling, and those with more than primary schooling). He applied the model to thirty Sub-Saharan countries for which complete data were available, under alternative assumptions about the share of AIDS treatment costs financed from savings and the relation be- tween the socioeconomic status of the work force and the incidence of HIV/ AIDS.12 The model does not consider the lower productivity of workers with AIDS. Results for the ten countries with the highest levels of HIV infection show that the negative impact of AIDS on average growth in per capita income in- creases with the share of AIDS treatment costs financed from savings and with the skill of infected workers. For example, if each group of workers has twice the risk of contracting HIV of the skill group below it and if AIDS treatment costs are financed entirely from savings, growth in per capita income will be 0.41 percentage points lower each year. The AIDS epidemic increases growth in per capita income only if the less-skilled have a rate of infection twice that of the next higher-skill group (gradient of 0.5) and if treatment costs are not fi- nanced out of savings. Neither assumption seems plausible. These models cannot hope to capture the complexity of all the adjustments that will occur as individuals, households, and firms cope with AIDS. However, they are useful in demonstrating how certain characteristics of the AIDS epi- demic are likely to affect the growth in per capita income. Under the most plausible assumptions, AIDS reduces the rate of growth in per capita income below what it would be in the absence of AIDS, despite slower population growth. Though the magnitude of the reductions seems small, for many coun- tries in Sub-Saharan Africa even a small drop means that already negative per capita growth rates will fall even more. That will happen if, as some of these models assume, AIDS is killing a higher percentage of the most skilled workers and if increased medical costs are financed out of savings. When slower pop- ulation growth is the result of a higher mortality rate among more productive adults, the potential economic benefits of the slower population growth will probably not be realized. We turn next to an exploration of how African populations cope with AIDS, looking at that behavior in more detail than is possible in aggregate models. The models described above were based on assumptions about the determi- nants of the economic impact of a given number and distribution of AIDS cases-the share of medical care financed from savings, the impact of AIDS on the productivity of individual workers and the work force, the substitutability of capital and labor in African production processes-rather than on empirical Martha Ainsworth and Mead Over 219 investigation of coping behavior. The next section examines what is known about responses to the AIDS epidemic, both to improve estimates of macroeco- nomic impacts and to explore the distributional implications of illness and deaths caused by AIDS. The focus is on the coping behavior of private (house- holds and firms) and public agents, which will affect rates of capital formation (through expenditure decisions) and the productivity of labor. Coping with the Epidemic Learning how households cope with AIDS is important because households are the fundamental economic decisionmaking unit in African economies, ac- counting for most agricultural production, informal self-employment, and con- sumption decisions. Illness and death from HIV and AIDS will affect resource allocation, production, consumption, savings, investment, and, above all, the well-being of patients and their surviving family members. When someone is stricken with AIDS, healthy members of the household begin to reallocate their time, reevaluate their production decisions, and alter their consumption and investment to improve the outcomes within this new constraint. In the absence of large-scale social welfare programs in Sub-Saharan countries, most house- holds rely on their own endowments and assistance from the extended family and neighbors to cope with the effects of AIDS. Policymakers in countries hard hit by the AIDS epidemic also face tough de- cisions about allocating scarce public resources for treating the disease in an environment where many households already live in poverty. Several important research questions remain to be answered before the economic impact on households and the public sector can be properly assessed and the appropriate policy response determined: How quickly and effectively can households recov- er from the impact of AIDS through traditional sources of support? How have disability and deaths from AIDS affected the living standards of households that send assistance to other households? How is AIDS likely to alter the distribution of income? How important will AIDS become relative to other causes of poverty in Sub-Saharan Africa, and is AIDS-induced poverty any more amenable to re- duction than poverty from other causes? Health Care Costs Estimating health care costs for AIDS is important for several reasons. Total expenditures (public and private) are needed to estimate the overall decline in national savings likely to accompany the AIDS epidemic; these estimates can then be fed into the models on aggregate economic growth discussed above. They are important for ministry of health planning and budgeting because, to the extent that publicly provided health care services are highly subsidized or free, most of the additional spending associated with meeting increased de- 220 The World Bank Research Observer, vol. 9, no. 2 (July 1994) mand for health care will come out of the public budget. And they are critical for evaluating alternative treatment strategies and identifying treatments that provide the greatest benefits, given a country's overall constraints. Estimating the costs of treating AIDS sufferers is not easy, especially in the African context. Information on household spending on treatment often comes from survivors, who may not be aware of all costs. Estimating the effects of the AIDS epidemic on public expenditure requires information on the likely de- mand effect and the cost per unit of demand. Neither is known. For example, although there are rough estimates of the number of people infected in Sub- Saharan Africa, little is known about how many of them are seeking medical treatment for opportunistic illnesses associated with AIDS or for terminal care. Many of these opportunistic infections were already widely prevalent in Africa before the onset of the AIDS epidemic, making the connection with AIDS more difficult to identify. The selection of treatment also complicates estimation. Ideal treatments based on Western protocols are unlikely to be suitable or af- fordable in Africa, but what will take their place is not clear. And as the epi- demic spreads, the amount spent per person treated is likely to drop as resources are stretched to their limits. That means that current levels of public spending on patients and their survivors are probably not realistic estimates of what costs will be when the number of AIDS cases rises to five or more times current levels. Most evidence on how much households spend on medical care for AIDS pa- tients comes from their survivors. These figures underestimate spending be- cause survivors are not usually aware of all AIDS-related expenditures and because they may not have recognized the connection between some illnesses and AIDS. Davachi and others (1988) estimated that a single twenty-five-day episode of inpatient treatment for a pediatric AIDS case at Mama Yemo Hos- pital in Kinshasa, Zaire, costs households $90, three times the average monthly income. A subsequent study in the same hospital found that HIV-positive pa- tients had spent $109 on medical treatment prior to admission-twice the amount spent by HlV-negative patients (Hassig and others 1990). In Kagera Re- gion, Tanzania, 88 percent of a sample of adults who died of AIDS in 1990-91 were reported by surviving household members to have received medical care before death, compared with 77 percent of adults who died of other causes, and to have spent an average of $65 for drugs and medical care, compared with $47 for adults dying of other causes (Mujinja and Over, 1993). A study of fewer than forty symptomatic AIDS patients in Ghana found median monthly spend- ing on treatment of $25 (Anarfi 1992, cited in Caldwell and others 1993). The impact of AIDS on the demand for medical care cannot be measured simply by the number of terminally ill AIDS patients admitted or treated at health facilities. Far more common will be increased consultations for oppor- tunistic infections already prevalent in the population, such as tuberculosis, pneumonia, and malaria. Which cases have an AIDS-related cause will not usu- ally be known. For example, about 30 percent of adult admissions and 43 per- Martha Ainsworth and Mead Over 221 cent of the adult bed days in a 250-bed hospital in southern Zambia in 1991 were for HIV-related illnesses (Buve and others 1992). The large share of bed days was due mainly to the long hospital stays of HIV-positive patients with tuberculosis (44.5 days); HIV-positive patients without tuberculosis spent about the same amount of time in hospital as HIV-negative patients (10.5-11 days). In July 1992, almost one-third of 1,489 admissions to Bukoba Regional Hos- pital in northwestern Tanzania tested HIV-positive; 57.4 percent of patients ad- mitted for tuberculosis treatment and about one-third of malaria patients were found to be infected with HIV (Kwesigabo and others 1993). It is not easy to distinguish malaria patients who are HIV-infected but not yet immune- compromised from patients who represent additional cases of malaria caused by AIDS-related failure of the immune system. This makes it all the more dif- ficult to predict the impact of the epidemic on private medical expenditures or on the demand for publicly provided medical care. In considering treatment costs, the hypothetical costs of ideal treatment should be distinguished from the costs of treatment actually provided (Hanson 1992; Over and others 1988). The costs of "ideal" treatment for AIDS and re- lated illnesses will vary according to the level of the health system providing the treatment and the availability of resources. Pallangyo and Laing (1990) es- timate that lifetime drug and nursing costs for a typical HIV-positive adult in Tanzania would amount to $290, assuming inpatient or outpatient care for sev- enteen episodes of opportunistic illness and full availability of the drugs rec- ommended in current treatment protocols (table 2). If only 60 percent of the drugs on the treatment protocol are available, costs drop by 10 percent (World Bank 1992). Total spending by the Tanzanian health sector for treating AIDS Table 2. Estimates of Total Direct Treatment Costs per AIDS Case (U.S. dollars) Country and year Low Mean High Kenya (1992) 938 Rwanda (1988-90) 358 South Africa (1991) 1,850 11,800 Tanzania (1991) adults, 290 children, 195 Tanzania (1987-88) 104 631 Zaire (1987-88) 132 1,585 Zimbabwe (1991) 64 614 2,574 Note: Low and high costs are presented because private expenditure and access to the most expensive public care often depend partly on what patients are willing and able to spend. Sources: For Kenya: Family Health International/AIDSTECH (1993, table 7, p. 26); for Rwanda: Shepard and Bail (1991); for South Africa: Broomberg and others (1993); for Tanzania: Pallangyo and Laing (1990); for Tanzania and Zaire: Over and others (1988); for Zimbabwe: Whiteside (1991, cited in Whiteside and FitzSimmons 1992). 222 The World Bank Research Observer, vol. 9, no. 2 (July 1994) patients under these assumptions would constitute 40 to 50 percent of recurrent costs in the 1991 health budget (World Bank 1992, p. 65). Some countries are exploring alternative models of care, such as home-based care, which would reduce the burden on costly inpatient facilities. A program launched in Kabale District, Uganda, in 1991 provides basic medical care, counseling, and homemaking services for households with AIDS patients. After two years the program, which was offered by all health care agencies in the area, had reached an estimated 15 percent of AIDS sufferers at an average cost per home visit of $0.40 and an average cost per AIDS case of $53 (Masheija and others 1993). What shares were paid by the public sector and what by house- holds or other private agencies is not known. Nor is it clear whether the capital costs necessary to run the program were included. A home-based care program in Monze, Zambia, was estimated to cost $11 (at the 1992 exchange rate of $1.51 per Zambian pound) per visit if vehicle depreciation is included or half that amount if not (Foster 1993a). The lifetime cost per patient for home-based care (five visits) came to $54 ($26 excluding depreciation), or the equivalent of eighteen days in a hospital. Although home-based care may not provide the same services as outpatient or inpatient hospital service, Foster (1993a) notes that home-based care is usually offered in addition to some hospitalization, not as a complete substitute. Home-based care is likely to cost more per patient in rural than in urban areas, because of the greater distances. The time cost of home-based care to households already burdened by the loss of a productive adult is not included in these estimates. Even if the cost to the public health system is less, home-based care may merely shift the costs of treatment back to the household. This concern points to an urgent need to measure carefully the costs and benefits of alternative models of health care from the perspective of both fam- ilies and the health system. But no study has yet measured actual public or total expenditures (private and public) on AIDS treatment in an African country. De- riving them for the public sector requires information on the number of pa- tients, the percentage who seek care in each type of public or private facility, how often they seek it, and the resources spent for treatment. Because this in- formation is generally not available, most studies have attempted to calculate an actual treatment cost per patient in the public sector, without asserting how many patients might be involved. These costs need to be considered in the con- text of average public expenditures on health care in Sub-Saharan countries to- day, which range from $1 to $30 per person, with an average in most countries of $5 or less per person (World Bank, forthcoming). Lifetime treatment costs of hospitalized AIDS patients in Rwanda were esti- mated at $358 in 1988-90, 91 percent of it for inpatient care (Shepard and Bail 1991).13 The daily costs of hospitalization at Mama Yemo Hospital in Kin- shasa, Zaire, in 1988 were estimated at $5.36 for HIV-positive patients and $4.65 for HIV-negative patients (Hassig and others 1990). In Kenya the daily cost of providing inpatient care drugs, tests, radiology, and overhead-ranges Martha Ainsworth and Mead Over 223 from $33 in a private hospital to $11 in government hospitals and $4 in mission hospitals (Family Health International/AIDSTECH 1993, table 6, p. 23). These estimates seem to assume that the daily costs of AIDS treatment are similar to those for other illnesses requiring hospitalization; the amount was not calcu- lated on the basis of AIDS-specific opportunistic infections. Other studies have estimated total (public and private) actual costs per case. Private expenditures for treatment depend in part on how much patients are willing and able to spend, so these studies often give low and high estimates of total expenditure per case (Over and others 1988). For example, assuming that every symptomatic HIV-positive adult seeks some modern sector treat- ment, if only at a local dispensary or health post, the estimated total treatment cost per case of HIV in 1987-88 was between $132 and $1,585 in Zaire and between $104 and $631 in Tanzania, where private sector options are fewer and incomes lower (table 2) (Over and others 1988). Costs per case suggest that if every AIDS patient received modern treatment, overall expenditure would be enormous. Multiplying public and private costs per AIDS case by the number of estimated cases for five countries in Sub-Saharan Africa in the 1990s yields results ranging from the equivalent of 23 percent of public health spending for Kenya to 66.5 percent for Rwanda (figure 4). Of course, the direct costs of treating AIDS in the public sector will likely never reach these levels. Not all AIDS patients will seek care, and as the numbers of AIDS patients rise, lower-cost treatment options will be selected. The challenge for the health sector is to define policies that will guide the allocation of avail- able resources in order to offer limited palliative care for AIDS patients while still providing effective health care to other patients, most of whom can be cured. AIDS will have other effects on the health care system as well. As people come down with AIDS and the demand for scarce medical care rises, patients with more treatable conditions may not receive the care they need. A random sample of more than 200 admissions to Mama Yemo Hospital in Kinshasa, Zaire, in 1988 revealed that half were HIV-positive (Hassig and others 1990). Since the hospital was operating at full capacity before the AIDS epidemic, this means that a substantial number of people with other treatable illnesses were being crowded out by AIDS patients. The study points to an unexpectedly high mortality rate among non-AIDS patients as evidence that only the most desper- ately ill patients without AIDS are being admitted. In Kenya, an estimated 40 percent of hospital beds are thought to be occupied by HIV-positive patients (Family Health International/AIDSTECH 1993, p. 29). Governments are also trying to prevent the spread of HIV, and the success of these efforts will influence the future impact of AIDS on the economy. Mann and others (1992) estimate that spending on AIDS prevention by national AIDS programs in Sub-Saharan Africa in 1991 amounted to only 7 cents per capita. Total prevention costs for the region were estimated at $36.9 million. As re- ported by country program managers, prevention spending ranges from less than one cent per capita in Nigeria to 66 cents in Congo (table 3). Though 224 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Figure 4. Potential AIDS Treatment Costs as a Sbare of Total and Public Healtb Expenditures, 1990 Percentage of health spending 70 - 65.5 6 AIDS spending as a share of total health spending 60 - AIDS spending as a share of total public health spedn 50 - 40.6 40-777 35.3 30 - 26.5 23.0 2. 20SR 10 31 3.f 0 Zimbabwe Kenya Malawi Tanzania Rwanda Note: Derived from a regression of AIDS treatment costs as a function of GNP per capita. Source: Ainsworth and Over (1994, Figure 4, p. 568). these amounts seem low, they need to be viewed in the context of average pub- lic health spending in Sub-Saharan Africa of $5 per person per year. A study of donor-financed AIDS prevention and control programs in the late 1980s in sixteen Sub-Saharan countries estimated an average first-year program budget per HIV-positive person of $20 (Over and Piot 1993, tables 20-22). The range of programmed spending was wide-from $2 per HIV-positive person in Ugan- da to $31,370 in Ethiopia-though the range of actual annual expenditure was smaller (table 3). Assuming that the potential annual benefits of an AIDS pre- vention program are roughly proportional to the number of new cases that might be averted that year-in turn, roughly proportional to the number of infectious adults currently in the population-the logic for spending 100 times more per HIV-positive adult in one country than another is not apparent. De- cisions about how much spending is "enough" in a given country should depend on the stage of the epidemic and on the cost-effectiveness of the chosen interventions. Martha Ainsworth and Mead Over 225 Table 3. Annual Expenditure on Prevention by National AIDS Programs, Selected African Countries, 1991 (U.S. dollars) Total program resources Amount per HIV Country mobilized (millions) Amount per capita positive person Cameroon 2.5 0.21 58.53 Congo 1.5 0.66 32.16 C6te d'Ivoire 2.2 0.19 5.46 Ethiopia 2.8 0.06 19.92 Nigeria 0.1 0.00 0.41 Rwanda 0.8 0.12 3.08 Senegal 0.9 0.13 185.62 Tanzania 1.0 0.04 1.72 Uganda 1.2 0.06 2.00 Zambia 5.5 0.65 10.66 Weighted average 0.07 6.52 Note: Includes total resources mobilized from domestic and international sources, minus expenditure on patient care and research. Average is weighted by national population. Source: Cameron and Shepard (1992, table 11.2, pp. 480-81). Number of HIV positive persons are authors' estimates for 1993, as presented in figures 1 and 2. Labor and Productivity The burden of caring for AIDS sufferers, whether in the home or in the hos- pital, usually falls on women, who would otherwise be engaged in farming or other productive work (Lado 1992, cited in Foster 1993b; Ankrah and others 1992). Over the longer run, as it becomes apparent that the AIDS patient will die, the household may change its mix of productive activities. After a house- hold member dies, the household loses entirely the person's time and skills and may also lose access to land, housing, and other assets inherited by others. One field study of 129 households-20 of them affected by AIDS directly or indirectly-described the process by which AIDS affects a typical rural Ugandan household over time, providing insights into household coping strategies (Bar- nett and Blaikie 1992). The authors trace the process through which the death of an adult leads to reduced remittances, farm inputs, and family labor; lower agricultural production; shifts from cash crops to subsistence crops; reduced schooling; and higher levels of child malnutrition. The sample was too small to detect a quantifiable impact of AIDS on agricultural production-still the most important economic activity in many African countries-as distinct from the continuous coping of all farm households to changes in the environment. No study has documented farmers' supply response to AIDS in terms of out- put or labor inputs. Most evidence is anecdotal. Gillespie (1989) postulated that the sensitivity of agricultural zones to the loss of labor from AIDS depends 226 The World Bank Research Observer, vol. 9, no. 2 (July 1994) on five factors: the seasonality of the demand for labor, the degree of labor specialization by gender and age, the interdependence of labor inputs, econo- mies of scale in labor, and the substitutability of labor-saving technologies. He also predicted that affected families would shift from labor-intensive crops, such as legumes, to tubers, such as potatoes, sweet potatoes, and cassava. For example, if faced with a rising rate of loss of productive adults, farmers in Zambia might well switch from maize production, which requires some S50 hours of labor per hectare annually, two-thirds of it during December and Jan- uary, to cassava, which requires only 448 hours per hectare, spread out more evenly over the year (Marter 1978, cited in Foster 1993b). If households shift production from export to subsistence crops, export revenues will also suffer. But such switches from more to less labor-intensive crops and from cash to subsistence crops, though often predicted, have yet to be documented. Nor has it been shown that crop switches in response to death of a household member from AIDS are any more or less significant than those in response to drought, a change in prices, or other factors. Productivity in nonagricultural activities is likely to suffer as well. Workers who fall ill due to AIDS will be less productive on the job and will be absent more often. Absenteeism is also likely to increase among healthy workers, as they take time off to attend funerals and care for those who are ill. Observers in South Africa, Zambia, and Zimbabwe have speculated that absenteeism and fatigue on the job due to AIDS may be more costly than AIDS deaths (PANOS Institute 1992, p. 75). Job turnover and training and recruitment costs will rise as well. For example, among approximately 1,700 employees of Barclay's Bank in Zambia, the mortality rate rose steadily from 0.4 percent per year in 1987 to 2.2 percent in 1992, with adults younger than forty-six accounting for 86 percent of the deaths during this six-year period (Keembe 1993). The mortality rate among 6,000 employees of twenty-one companies in Lusaka and copper- belt towns in Zambia rose eightfold, from 0.24 to 2.1 percent a year during 1987-93 (Baggaley and others 1993). About half of the twenty-one firms report- ed that AIDS had affected their productivity, and 19 percent reported that it had affected their recruitment. AIDS-related deaths will also reduce a firm's stock of skilled labor. Certain types of scarce, highly skilled local labor may have to be replaced by expatri- ates, raising the costs of production. In 1986, for example, Kenya produced only 174 engineering graduates and Zimbabwe only 52 (PANOS Institute 1992, p.71). Industries that rely on skilled workers will be particularly vulnerable. Only 6 percent of Zambia's labor force is employed in the mining sector- which accounted for 85 percent of the value of Zambia's exports in 1988 (Foster 1993b)-but one-third of these workers are highly skilled. In Kenya, about 10 percent of the mining labor force can be classified as highly skilled, and infection rates are estimated to be 12 percent (Family Health International/ AIDSTECH 1993, annex A-8, and table 4).14 The impact on firms of the deaths of skilled workers and managers will depend on how difficult and costly it is Martha Ainsworth and Mead Over 227 for firms to replace workers at various grades, a subject about which little is known. Larger outlays for health, unemployment, funeral, and death benefits will raise the costs of production or reduce the productivity of labor. The annual hospital bill for the Uganda Railway Corporation has risen to $77,300, and the costs per hospitalized patient have risen dramatically from $69 in 1988 to $300 in 1992 (PANOS Institute 1993, p. 76). Firms will cope with the reduced pro- ductivity and loss of trained staff through their production and investment de- cisions. They may choose less labor-intensive technology, substituting capital for labor. They may recruit and train more workers than needed for a specific job, in anticipation of some losses to AIDS. Although initially all these effects may be seen as costs to firms, ultimately workers are likely to bear the brunt of them, through lower wages and less liberal absentee policies. Survivors: Caring for Orphans In hard-hit countries, the AIDS epidemic will create large numbers of survi- vors-widows, widowers, elderly parents, and orphaned children.15 Women in Sub-Saharan Africa are particularly vulnerable to HIV infection and to the eco- nomic impact of AIDS. HIV is more easily transmitted from men to women than from women to men. A woman's lower economic status, lower education level, and dependency on her husband may give her little influence in determining whether a condom is used during intercourse and little bargaining power over her husband's extramarital affairs. Throughout Africa, women's status is re- lated to their childbearing capacity. Thus, perversely, practicing safe sex or ab- stinence may limit a woman's exposure to AIDS at the cost of lowering her status. Customary inheritance practices often leave nothing to the wife, and where inheritance laws exist, they are often difficult to enforce. Among the Tonga of Zambia, as in many other groups, the husband's relatives customar- ily claim all of the household's possessions upon his death (Foster 1993b). The widow can be left without shelter or the means to earn a living. Suspicion that widows are infected by HIV may limit their prospects for remarriage locally, prompting their migration to areas where they are unknown. Because of the way HIV spreads through families, many children who lose one parent to AIDS will eventually lose the other. And because child fostering is common in many African societies even for children whose parents are alive, both orphaned and other children in the household will be affected by the AIDS epidemic. For example, in C6te d'Ivoire in the mid-1980s, one child in five un- der age fifteen whose parents were alive was living away from them (Ainsworth 1992). Preliminary results of a study in Kagera Region, Tanzania, show that about one child in five with two living parents lives with only one of them and that about 12 percent live with other relatives (Ainsworth and others 1993). The number rises to one in three or four for children who have lost one parent and are living with someone other than the remaining parent. This practice 228 The World Bank Research Observer, vol. 9, no. 2 (July 1994) means that both orphans and other resident children will be affected by the loss of adults. Further, since the AIDS epidemic tends to cluster in communities, survivors will also tend to be clustered, straining traditional mechanisms com- munities have for coping with adult death. New groups and communities will be thrown into poverty. International attention has focused on the plight of AIDS orphans. WHO (1992) predicts that by the year 2000, more than 10 million HIV-negative chil- dren worldwide under age ten will have lost their mothers to AIDS-some 90 percent of them in Sub-Saharan Africa.16 UNICEF predicts that 3 to 5 million maternal orphans will be created by the AIDS epidemic in ten Central and East African countries by 1999, amounting to 6 to 11 percent of children under the age of fifteen (Preble 1990; UNICEF 1990, 1991). The accuracy of both of these predictions rests on their assumptions about AIDS mortality among women, the number of children left behind, and the mortality and age structure of the sur- viving children. Neither study predicts the number of children who will be or- phaned by the death of their fathers or provides separate estimates for those who will lose both parents. Sample surveys have attempted to estimate the numbers of orphans in areas hard-hit by the epidemic.17 In Masaka District, Uganda, a survey of 5,022 chil- dren under fifteen years old found that 10 percent had lost one or both parents (Kamali and others 1992). Fifteen percent of the surviving parents of orphans were infected with HIV, compared with 5 percent of the parents of nonor- phaned children. Within one year of initial interviews, twenty-six children had lost one or both parents; in twenty-four cases the remaining parent was HIV- positive. A survey of 1,781 children from 586 randomly selected households in Manicaland, Zimbabwe, found that 6.9 percent of children under fifteen had lost one or both parents (G. Foster and others 1992). In interpreting these findings, it is important to keep in mind that many if not most orphans in Sub-Saharan Africa have lost their parents to causes other than AIDS. High baseline levels of adult mortality in African countries result in many orphaned children even without AIDS, and families and communities have established coping mechanisms for absorbing them. Actual counts and es- timates based on census data for four East African countries show that roughly 2 percent of children under age fifteen were motherless and 3.5 to 7 percent had lost their father (table 4). In three of the countries-Kenya, Malawi, and Uganda-the censuses predate the onset of the AIDS epidemic. Using Malawi's figure for two-parent orphans of 0.5 percent of all children under fifteen, it can be estimated that 5 to 8.5 percent of all children under the age of fifteen were orphans (having lost one or both parents) before the onset of the AIDS epidem- ic. Intercensal population growth rates can be compared with the number of children enumerated in hard-hit districts to estimate the additional orphaning caused by AIDS. Ainsworth and Over (1994, p. 575) estimate this figure for Uganda and Kagera Region of Tanzania. Martha Ainsworth and Mead Over 229 Table 4. Actual and Estimated Numbers of Orphans under Age Fifteen in Selected East African Countries Maternal orphans Paternal orphans Two-parent orphans Total orphansa Country Year Number Percent Number Percent Number Percent Number Percent Kenya 1969 100,837 1.91 374,563 7.08 26,465 0,50 c 449,3 Malawi 1977 49,355 2.03 85,096 3.51 12,421 0.51 122,030 5.03 Tanzania 1988 200,985 1.96 401t70 3,91 57,6 02S33 8.--S153 Kagera Region 1988 18,125 2.92 *2 .. 7,-- US 466 7.5 Uganda 1969 106,203 2.44 203,869 4.68 21,779 0 3 6.62 Masaka and Rakai 1969 6,717 2.31 12,467 4.28 17457 GM 6. Note: Shaded areas are estimates. For Kenya, Tanzania, and Uganda, it was assumed that, absent AIDS, 0.5 percent of children under fifteen are two-parent orphans. For Tanzania, it was further assumed that the proportion of paternal to maternal orphans was two to one and that AIDS caused higher rates of orphanhood only in three districts in the hard-hit Kagera Region (Ainsworth and Rwegarulira 1992). a. Number of maternal and paternal orphans minus two-parent orphans. Source: Ainsworth and Over (1994, table 5, p. 574), based on census data. Though there are many orphans in Sub-Saharan Africa, there are few or- phanages. In 1990 in the hard-hit Kagera Region of Tanzania, orphanages were operating at only 57 percent of capacity, and nationwide only 1,083 children were residing in children's homes, most of them run by the private sector (Ainsworth and Rwegarulira 1992). Largely because extended families are will- ing to absorb orphaned children, there has been no rush to build orphanages in response to the AIDS epidemic. The annual cost of the publicly financed chil- dren's home in Dar es Salaam was estimated at $649 to $928 per child per year-several times per capita income (Ainsworth and Rwegarulira 1992). Pri- vate sector orphanages may spend several times this amount. (Koda and Over 1993 estimated the annual cost of one privately operated orphanage in north- west Tanzania at $3,370 per child, compared to $165 to subsidize care in a fos- ter home.) These efforts are clearly not replicable on a large scale. However, children living in households affected by AIDS-whether orphans or not-may be disadvantaged in many respects, and donors and government agencies are already preparing assistance programs specifically for them. But we have yet to see any published analysis of expenditures by public agencies in Africa concerned with child welfare or of the effect of the AIDS epidemic on the level of expenditure or its uses. Few studies have examined the cost-effec- tiveness of programs for orphans as a poverty-reduction strategy. No study has assessed the efficacy of using orphanhood in targeting assistance to the neediest children. There are many needy children in countries hard-hit by AIDS-includ- ing children orphaned from other causes and children in impoverished families. The AIDS epidemic will add to the numbers of children in distress, although it is not known by how much. Policymakers need to identify the most vulnerable children, whether orphaned or not. 230 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Education The education sector will experience both demand and supply shocks as a result of the AIDS epidemic.18 The demand for schooling may be lower than it would have been in the absence of AIDS for two reasons. First, the cohort entering school may be smaller than it would otherwise have been. The worst- case scenario in a World Bank study (1992b) of the AIDS epidemic in Tanzania finds that by 2020 the epidemic will have reduced the cohort of primary-school-aged children by 22 percent and that of secondary-school-aged children by 14 percent over enrollments in the absence of AIDS. Second, adult deaths from AIDS may result in lower investments in children's schooling, reducing future productivity and the household's stock of human capital. When an adult falls ill or dies, children may be removed from school because they are needed at home or because the family has fewer resources to pay for schooling (Katabaro 1993). Preliminary results from the Kagera study suggest that children who have lost a father or both parents are less likely to enroll in school than are other children (Ainsworth and Koda 1993). All chil- dren, whether orphans or not, in households that have experienced the recent death of a female adult have markedly lower enrollment rates and, once en- rolled, spend fewer hours in school. AIDS is thus likely to lower school enroll- ments as children substitute for the loss of female labor. In countries like Uganda, where households finance perhaps 70 percent of the costs of primary schooling, the death of adults will probably substantially reduce the house- hold's ability to pay school fees-more so than in Tanzania, where fees are lower. In the longer run, as AIDS lowers life expectancies, the impact on the demand for education will depend on whether households perceive their own children as being able to avoid the higher risk of mortality in the surrounding population or of being equally exposed to it. If households feel that their chil- dren are less vulnerable, demand for education will remain unchanged or may even increase, to the extent that households believe that AIDS will result in a shortage of skilled labor. If households believe that their children share the new higher mortality risk, shorter anticipated lifetimes would reduce the demand for education. Policymakers need information on whether policies targeted to individual needy children will be as effective as systemic reforms to improve the quality of schooling. Particularly in countries where enrollment rates were in decline even before the AIDS epidemic, it will be important to identify the causes of the decline and to determine whether interventions directed to a specific group of children are likely to be cost-effective. Tanzanian census data for 1988, for example, show that children who have lost their mother are less likely to enroll in school than are children whose mother is alive (Ainsworth and Rwegarulira 1992). But the gap between wealthier and poorer districts within a region is sometimes even larger. Studies are needed comparing the impact of adult deaths on school enrollment with that of other factors, such as low incomes. Martha Ainsworth and Mead Over 231 In the short run, there may be pressure to exempt children from hard-hit families from paying school fees. Schooling is already underfunded throughout Africa, and the revenues from school fees can often make a big difference at the margin in the quality of education. In Uganda, the major expense is not the official school fees, but the fees paid by parent associations to top off the salaries of underpaid teachers. Exempting orphans would place a larger finan- cial burden on other children's families, many of them poor. On the supply side, AIDS will affect the number of teachers needed in the work force, their turnover and training costs, and the efficiency of the school- ing system (Shaeffer 1993). There are also important externalities to changing the behavior of teachers, who can influence children to change their behavior in ways that reduce their chances of HIV infection. Thus, policies that help pre- vent the spread of HIV infection to teachers and that prolong the effectiveness of HIV-infected teachers are high priorities. It is important to note as well that poor working conditions and inadequate remuneration have already created large-scale teacher attrition in many countries (Mburugu 1993). AIDS will con- tribute to these problems, but preventing AIDS will be only one of the solutions. Conclusion Because AIDS affects primarily the most productive age group and is fatal and widespread, it will have a larger impact on African development than other, more common diseases. Further, the long incubation period between infection with HIV and development of full-blown AIDS makes it certain that, absent a cure, AIDS will affect African development prospects for the foresee- able future. Models of the impact of HIV/AIDS on population growth generally assume that only mortality is affected by the epidemic, although behavioral responses to the epidemic could also result in fertility changes. Depending on the incu- bation period and the underlying population growth rate in the absence of AIDS, these models generally predict that population growth will remain posi- tive as long as the nationwide HIV infection rate among adults remains less than 30 percent. There are areas within countries where rates that high prevail, but to date no African country has experienced such a high level of infection nationally. The impact of AIDS on growth of per capita income will depend on its effects on the growth rate of healthy labor and capital and on the socioeconomic distribution of HIV infection. Since the overwhelming share of the population in most countries is rural, the majority of those infected are poor and unskilled. However, infection rates are higher in urban areas, where most of the educated live. Furthermore, a handful of studies in Sub-Saharan Africa suggest that prevalence in urban areas is higher among higher-income, higher-skilled men and their partners. Macroeconomic models show that the higher the infection 232 The World Bank Research Observer, vol. 9, no. 2 (July 1994) rate among the more educated workers and the higher the propensity of med- ical care to be financed out of savings, the more detrimental the disease will be to growth of per capita income. While the literature on macroeconomic impact has explored the sensitivity of results to alternative assumptions on epidemiological parameters and the re- sponses of economic agents, little empirical information is available to guide the choice of the most likely scenario in a given country. Studies are needed that document the coping behavior of economic agents-households, firms, and the public sector-to inform the macroeconomic models and to elucidate the distributional implications of AIDS. Regardless of the macroeconomic ef- fect, most households and businesses directly affected by deaths from AIDS will be economically worse off, at least in the short run. It is not clear, however, to what extent AIDS will affect poverty in Sub-Saharan Africa or whether, by disproportionately impoverishing the more educated, it will actually reduce the inequality of income distribution in some areas. Sub-Saharan Africa has the lowest levels of per capita income of any devel- oping region, and economic performance faltered in the late 1980s, exposing many underlying economic problems. These countries face a large number of development problems in every sector, of which AIDS is but one. The potential macroeconomic impact of AIDS may be significant, particularly when economic indicators are already so low, and skilled labor is already scarce. Understand- ing how much AIDS will affect African development and choosing appropriate policies to reduce poverty and promote economic growth depend on accurate measures of the coping processes of economic agents. Governments need to as- sess more accurately the potential economic impact of AIDS, identify and im- plernent programs to mitigate that impact, and target cost-effective prevention programs to the economically important sectors that are most sensitive to HIV infection, to reduce the economic impact in the long term. Notes Martha Ainsworth and Mead Over are with the Policy Research Department of the World Bankc. This article draws on a background paper prepared for the Symposium on AIDS and De- velopment on May 11, 1993, in Abidjan, C6te d'lvoire, sponsored by the African Development Bank. It benefits from the work of many individuals within and outside the Bank and the au- thors' ongoing collaboration with Phare Mujinja, Godlike Koda, George Lwihula, and Innocent Semali in a research project on the household-level consequences of AIDS deaths. The suggestions of Dorothy Black, Nicholas Dodd, Emmanuel Jimenez, Wendy Roseberry, and Richard Sabot on earlier versions are gratefully acknowledged. Mead Over is grateful to the Commission on Behavioral and Social Sciences and Education of the National Academy of Sciences for support during his work on this paper. The opinions expressed are those of the authors and do not nec- essarily reflect policy of the World Bank or its members. 1. Hiv refers to HIv-1. A second strain of HIV, HIV-2, is also spreading rapidly in Africa but is thought to be less pathogenic. 2. The line of argument and examples in this section are taken from Ainsworth and Over (1992). Martha Ainsworth and Mead Over 233 3. Comparing the cost-effectiveness of alternative policies to mitigate the economic impact of AIDS as in Over and Piot (1993) would be a logical extension of this discussion, but is beyond the scope of this paper. 4. The most comprehensive, publicly available data on seroprevalence rates is the data base maintained by the U.S. Bureau of Census. The data base includes the results of thousands of individual studies of varying quality. Four HIV prevalence estimates for each country are selected as most representative of the epidemic in the year of reference for four specific groups of adults: "low-risk urban," "high-risk urban," "low-risk rural," "high-risk rural." The December 1993 edition of the data base contains estimates of the mid-1993 seroprevalence of "low-risk urban" adults for forty-seven of the fifty-seven Sub-Saharan countries and of "low-risk rural" for twen- ty-three. S. Data for twenty-three African countries show a positive correlation of 0.7 between the rural low-risk rate and the urban low-risk rate, with the urban rate always higher. But as figure 1 shows there is considerable noise in that relationship. Rural seroprevalence surveys are infre- quent and generally unrepresentative. Two studies from the Mwanza Region of Tanzania (Kigadye and others 1993; Borgdorff and others 1993) find that women attending antenatal clin- ics had statistically significant lower infection rates up through age thirty-four, when most in- fections occur, than women of the same age in the general population. But medical researchers often target rural seroprevalence studies to regions where infection is feared to be high. Because of this uncertainty about rural prevalence rates, we compute two estimates of national preva- lence, one using the actual rural prevalence rate or, if that is unavailable, the rate estimated from the regression line in figure 1, and a second estimate of rural prevalence as one-fifth urban prev- alence in each country (after Chin and Sato 1994). The two approaches yield infection rates for rural African adults of 3.1 and 1.3 percent, which correspond to the two different estimates of the number of infected rural adults presented in the text. 6. For example, a recent UNDP study on the economic impact of AIDS in Asia states that HIV could be called "the poverty virus" in Asia because of its disproportionate effect on the poor (Bloom and Lyons 1993, p. vi). However, Solon and Barrozo (1993) in that volume explicitly argue that in the Philippines the impact of AIDS will be large because of the disproportionately high risk of infection of the overseas contract workers who have higher-than-average incomes. 7. In the context of a full structural model of sexual behavior, these risk factors could be in- cluded, provided that instrumental variables could be identified for each. Such a model would be useful for predicting the future path of new infections, but is unnecessary for predicting the economic impact of current patterns of infection. 8. That is, the coefficient on the odds ratio [p/(1 - p)] was 1.96, where p is the probability of being infected. 9. For a sufficiently short incubation period of AIDS, this conclusion would be altered. 10. The growth rate of the capital stock is defined as the ratio of investment (new capital) to the existing stock of capital less a constant percentage for depreciation. 11. If, as hypothesized in many modern growth models, returns to scale are increasing rather than constant, the range of impacts is wider. 12. The latter assumption, the "socioeconomic gradient" of the epidemic, is measured in terms of the ratio of infection rates among more-skilled and less-skilled workers. A "gradient" of 0.5 means that risk falls with the skill level of the worker, 1.0 implies that the three groups are equally at risk, and 2.0 that risk rises with the skill level of the worker. With a value of 0.5, each higher-skill group has half as large a risk as the next lowest skill group. 13. Multiplying this amount by the 1,849 confirmed hospitalized AIDS cases in Rwanda in 1991 yields $0.6 million, or 4.6 percent of the public health budget. However, this is clearly a low estimate, since it is not known what percentage of AIDS patients seek inpatient care and how much care went to patients with opportunistic illnesses but who did not satisfy the WHo/Bangui criterion (Ryder and Mgerwa 1994, p. 272) of a positive blood test. 14. Only 0.1 percent of the labor force is in the mining sector, but at Ksh573 million in 1990 it represented a disproportionate share of exports (AIDSTECH 1992, appendix A-9). Agriculture, the largest export, accounted for KshlO.9 billion; manufacturing for Ksh6.9 million. 234 The World Bank Research Observer, vol. 9, no. 2 (July 1994) 15. Portions of this section are excerpted from Ainsworth and Over (1992). 16. Note, however, that this is the cumulative number of children orphaned since the begin- ning of the AIDS epidemic. By the end of the 1990s, many of the orphans predicted by WHO will be ten years or older, and some will be adults with children of their own. Further, the WHO predictions do not take into account child mortality due to HIV infection. They also assume that all HIV-negative orphans survive-the model assumes no source of child mortality other than AIDS (Burton 1992, personal communication). 17. Enumerations of orphans in Rakai and Masaka districts of Uganda in 1989 estimated 25,634 orphans in Rakai (12.8 percent of children under eighteen) and 22,051 orphans in Masaka (4.95 percent). Twenty-three percent of the orphans in Rakai had lost both parents. A 1991 enu- meration of orphans in the Kagera Region of Tanzania counted 35,291 children under the age of fifteen who had lost one or both parents. Neither sample surveys nor enumerations have been able to assess the share of orphans due to AIDS, however. 18. Portions of this section are excerpted from Ainsworth and Over (1992). References Ainsworth, Martha. 1992. Economic Aspects of Child Fostering in C6te d'lvoire. LSMS Working Paper 92. Washington, D.C.: World Bank. Ainsworth, Martha, and Godlike Koda. 1993. "The Impact of Fatal Adult Illness on Child Schooling." In World Bank and University of Dar es Salaam. The Economic Impact of Fatal Adult Illness in Sub-Saharan Africa. Report of a workshop in Bukoba, Tanzania. Ainsworth, Martha, and Mead Over. 1992. "The Economic Impact of AIDS: Shocks, Responses, and Outcomes." AFTPN Technical Working Paper 1. World Bank, Population, Health, and Nu- trition Division, Africa Technical Department, Washington, D.C. 1. 1994. "The Economic Impact of AIDS on Africa." In M. Essex, Souleymane Mboup, Phyllis J. Kanki, and M. R. Kalengayi, eds., AIDS in Africa. New York: Raven Press. Ainsworth, Martha, and A. A. Rwegarulira. 1992. "Coping with the AIDS Epidemic in Tanzania: Survivor Assistance." AFTPN Technical Working Paper 3. World Bank, Population, Health, and Nutrition Division, Africa Technical Department, Washington, D.C. Processed Ainsworth, Martha, Godlike Koda, George Lwihula, Phare Mujinja, Mead Over, and Innocent Semali. 1993. "Characteristics of the KHDS Sample of Households and Survivors." In World Bank and University of Dar es Salaam. The Economic Impact of Fatal Adult Illness in Sub- Saharan Africa. Report of a workshop in Bukoba, Tanzania. Allen, Susan, C. Lindan, A. Serufilira, P. van de Perre, A. Rundle, F. Nsengumuremyi, M. Carael, J. Schwalbe, and S. Hulley. 1991. "Human Immunodeficiency Virus Infection in Urban Rwan- da: Demographic and Behavioral Correlates in a Representative Sample of Childbearing Women." Journal of the American Medical Association 266(12):1657-63. Anarfi, J. K. 1992. "Report on a Pilot Study on the Coping Strategies of Households with AIDS Sufferers-June 1992." University of Ghana, Institute for Statistical, Social, and Economic Re- search, Legon, Ghana. Anderson, R. M. 1988. "The Impact of the Spread of HIV on Population Growth and Age Struc- ture in Developing Countries." In Alan Fleming, Manuel Garbalo, David W. FitzSimmons, Michael R. Bailey, and Jonathan Mann, eds., The Global Impact of AIDS. New York: Alan R. Liss, Inc. Anderson, R. M., R. M. May, M. C. Boily, G. P. Garnett, and J. T. Rowley. 1991. "The Spread of HIV-1 in Africa: Sexual Contact Patterns and the Predicted Demographic Impact of AIDS." Nature 352:581-89. Ankrah, E. M., D. Schumann, J. McGrath, A. Gessa, S. Nkumbi, M. Lubega, S. Kyomuhendo, and S. Wangalwa. 1992. "Stress and Coping among Rural Families in Uganda." Paper pre- sented at the 8th International Conference on AIDS/STDs, Amsterdam, July 19-24. Abstract PoD 5009. Processed. Martha Ainsworth and Mead Over 235 Ankrah, E., D. Schumann, J. McGrath, S. Nkumbi, M. Lubega, and A. Misanya-Gessa. 1993. "Family Provision of Care to Persons with AIDS in Urban and Rural Uganda." Abstracts of the 9th International Conference on AIDS, Berlin. June. Armstrong, Jill, and Eduard Bos. 1992. "The Demographic, Economic, and Social Impacts of AIDS." In Jonathan Mann, Daniel J. M. Tarantola, and Thomas W. Netter, eds., AIDS in the World. Cambridge, Mass.: Harvard University Press. Baggaley, R., D. Childngwa, P. Godfrey-Faussett, and J. Porter. 1993. "Impact of HIV on Zam- bian Businesses." Abstracts of the 8th International Conference on AIDS in Africa, Marrakesh, December. Barnett, Tony, and Piers Blaikie. 1992. AIDS in Africa: Its Present and Future Impact. London: The Guilford Press. Bloom, David, and Joyce V. Lyons. 1993. Economic Implications of AIDS in Asia. New Delhi: United Nations Development Programme. Bongaarts, John. 1990. "A Model of the Spread of HIV Infection and the Demographic Impact of AIDS." Statistics in Medicine 8:103-20. Borgdorff, Martien, Longin Barongo, Ellen van Jaarsveld, Arnoud Klokke, Kesheni Senkoro, James Newell, Angus Nicoll, Frank Mosha, Heiner Grosskurth, Ronals Swai, Henri van As- ten, Johan Velema, Richard Hayes, Lex Muller, and Joas Rugemalila. 1993. "Sentinel Surveil- lance for HIv-1 Infection: How Representative are Blood Donors, Outpatients with Fever, Anaemia, or Sexually Transmitted Diseases, and Antenatal Clinic Attenders in Mwanza Re- gion, Tanzania?" AIDS 7:567-72. Bos, Eduard, and Rodolfo A. Bulatao. 1992. "The Demographic Impact of AIDS in Sub-Saharan Africa: Short- and Long-Term Projections." International Journal of Forecasting 8:367-84. Broomberg, Jonathan. 1993. "Current Research on the Economic Impact of HIV/AIDS: A Review of the International and South African Literature." In Sholto Cross and Alan Whiteside, eds., Fac- ing up to AIDS: The Socio-Economic Impact in Southern Africa. New York: St. Martin's Press. Broomberg, Jonathan, M. Steinberg, M. Masobe, and G. Behr. 1993. "The Economic Impact of AIDS in South Africa." In Sholto Cross and Alan Whiteside, eds., Facing up to AIDS: The Socio- Economic Impact in Southern Africa. New York: St. Martin's Press. Bulatao, Rodolfo A. 1991. "Projecting the Demographic Impact of the Hiv Epidemic Using Stan- dard Parameters." In United Nations/World Health Organization, "The AIDS Epidemic and Its Demographic Consequences." Proceedings of the UN/WHO Workshop on Modeling the Demographic Impact of the AIDS Epidemic in Pattern 11 Countries: Progress to Date and Poli- cies for the Future. New York: United Nations. Burton, Tony. 1992. Personal Communication. World Health Organization (WHO), Geneva. Buve, A. M. A., and S. D. Foster. 1993. "HIV Infection in a Rural District in Zambia: Prevalence and Socioeconomic Factors Associated with Seropositivity." Abstracts of the 9th International Conference on AIDS, Berlin, June. Buve, A. M. A., C. Mbwili, E. Mungo, N. Tihon-Tollenaere, and M. Zeko. 1992. "Mortality among Female Nurses in the Face of the AIDS Epidemic: A Pilot Study Conducted in Two Hospitals in the Southern Province of Zambia." Institute of Tropical Medicine, Antwerp, Bel- gium. Processed. Caldwell, J., P. Caldwell, E. M. Ankrah, J. K. Anarfi, D. K. Agyeman, K. Awusabo-Asare, and I. 0. Orubuloye. 1993. "African Families and AIDS: Context Reactions and Potential Interven- tions." Health Transition Review 3(Supplementary issue):1-16. Cameron, Charles, and Julia Shepard. 1992. "The Cost of AIDS Care and Prevention." In Jonathan Mann, Daniel J. M. Tarantola, and Thomas W. Netter, eds., AIDS in the World. Cambridge, Mass.: Harvard University Press. Carael M., J. Cleland, J. C. Deheneffe, and L. Adeokun. 1990. "Research on Sexual Behavior That Transmits HIV: The GPA/WHO Collaborative Surveys. Preliminary Findings." Internation- al Union for the Scientific Study of Population: Seminar on Anthropological Studies Relevant to the Sexual Transmission of HIV, Sonderberg, Denmark. Processed. 236 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Chin, J., and P. A. Sato. 1994. "Estimates and Projections of the HIV/AIDS Epidemic in Sub- Saharan Africa." In M. Essex, Souleymane Mboup, Phyllis J. Kanki, and M. R. Kalengayi, eds., AIDS in Africa. New York: Raven Press. Cleland, John, M. Carael, J. C. Deheneffe, and B. Ferry. 1992. "Sexual Behavior in the Face of Risk: Preliminary Results from First AIDs-Related Surveys." Health Transition Review 2(Sup- plement) :185-204. Cuddington, John T. 1992. "The Macroeconomic Impact of AIDS in a Dualistic, Labor-Surplus Economy with an Application to Tanzania." Georgetown University, School of Foreign Ser- vice, Economics Department. Processed. . 1993. "Modeling the Macroeconomic Effects of AIDS with an Application to Tanzania." The World Bank Economic Review 7(2):173-89. Cuddington, John T., and John Hancock. 1992. "Assessing the Impact of AIDS on the Growth Path of the Malawian Economy." Department of Economics Working Paper 92-07. George- town University, Washington, D.C. Processed. Davachi, F., P. Baudoux, K. Ndoko, B. N'galy, and J. Mann. 1988. "The Economic Impact on Families of Children with AIDS in Kinshasa, Zaire." In Alan Fleming, Manuel Carbalo, David W. FitzSimmons, Michael R. Bailey, and Jonathan Mann, eds., The Global Impact of AIDS. New York: Alan R. Liss, Inc. Denis, F., F. Barin, G. Gershy-Damet, J. L. Rey, M. Lhuillier, M. Mounier, G. Leonard, A. Sangare, A. Goudeau, and S. M'Boup. 1987. "Prevalence of Human T-lymphotropic Retrovi- ruses Type III (HIV) and Type IV in Ivory Coast." Lancet 1(8530):408-11. Family Health International/AIDSTECH. 1993. "An Assessment of the Economic Impact of AIDS in Kenya." Paper prepared for the United States Agency for International Development (USAID), Washington, D.C. Foster, G., R. Shakespeare, and F. Chinemana. 1992. "Orphans in Zimbabwe-A Descriptive and Enumerative Survey." Poster presented at the 8th International Conference on AIDS, Am- sterdam, July 19-24. Poster Abstracts, vol. 2, PoD5158. Foster, S. D. 1993a. "Cost and Burden of AIDS on the Zambian Health Care System: Policies to Mitigate the Impact on Health Services." London School of Hygiene and Tropical Medicine and John Snow, Inc., London. Processed. . 1993b. "The Economic Impact of AIDS in Zambia." World Bank, Africa Technical De- partment, Poverty and Human Resources Division, Washington, D.C. Processed. Ghana Health Assessment Project Team (Report prepared by R. Morrow, P. Smith, and K. Nimo.) 1981. "A Quantitative Method of Assessing the Health Impact of Different Diseases in Less Developed Countries." International Journal of Epidemiology 10(1):73-80. Gillespie, Stuart. 1989. "Potential Impact of AIDS on Farming Systems: A Case Study of Rwan- da." Land Use Policy 4:301-12. Hanson, Kara. 1992. "AIDS: What Does Economics Have to Offer?" Health Policy and Planning 7(4):315-28. Hassig, Susan E., J. Perriens, E. Baende, M. Kahotwa, K. Bishagara, N. Kinkela, and B. Kapita. 1990. "An Analysis of the Economic Impact of HIV Infection among Patients at Mama Yemo Hospital, Kinshasa, Zaire." AIDS 4:883-87. Heyward, W. L., V. L. Batter, M. Malulu, N. Mbuyi, L. Mbu, M. E. St. Louis, M. Kamenga, and R. W. Ryder. 1993. "Impact of HIV Counselling and Testing among Child-Bearing Women in Kinshasa, Zaire." AIDS 7:1633-37. Kamali, A., J. F. Kengeya-Kayondo, S. S. Malamba, A. J. Nunn, J. A. Seeley, H. U. Wagner, and D. W. Mulder. 1992. "Orphans and HIV-1 Infection in a Rural Population Cohort in S.W. Uganda." Poster presented at the 8th International Conference on AIDS, Amsterdam, July 19-24. Poster Abstracts, vol. 2, PoD5159. Kambou, G., S. Devarajan, and Mead Over. 1992. "The Economic Impact of AIDS in an African Country: Simulations with a General Equilibrium Model of Cameroon." Journal of African Economies 1(1):109-30. Martha Ainsworth and Mead Over 237 Katabaro, J. K. 1993. "The Impact of HIV/AIDS on Education in Tanzania: A Case of Bujunango- ma Community in Bukoba Rural District, Kagera Region." Paper presented at the AIDS and Education Seminar at the International Institute for Educational Planning, Paris, December 8-10. Processed. Keembe, A. L. 1993. "Presentation by A. L. Keembe, Personnel Director, Barclay's Bank of Zambia, Ltd." Presented at the Employer's Response to HIV/AIDS Workshop, Kitwe, Zambia. Processed. Killewo, J. Z. J., C. J. Comoro, J. Lugalla, and G. Kwesigabo, eds. 1993. "Systemic Interventions and their Evaluation against HIV/AIDS in Kagera Region, Tanzania." Proceedings of a work- shop held in Bukoba Town, Kagera, Tanzania, 10-11 May. Muhimbili University College of Health Sciences, Department of Epidemiology and Biostatistics. Processed. Koda, Godlike, and Mead Over. 1993. "Cost of Survivor Assistance Programs in Northwestern Tanzania." Poster presented at the 8th International Conference on AIDS in Africa, Marrakesh, December. Kremer, Michael. 1994. "Can Having Fewer Partners Increase Prevalence of AIDS?" Cambridge, Mass.: Department of Economics. Processed. Kreiss, Joan K., Davy Koech, Frank A. Plummer, King K. Holmes, Marilyn Lightfoote, Peter Piot, Allan R. Ronald, J.O. Ndinya-Achola, Lourdes J. D'Costa, Pacita Roberts, Elizabeth Ngugi, and Thomas C. Quinn. 1986. "AIDS Virus Infection in Nairobi Prostitutes: Spread of the Epidemic to East Africa." New England Journal of Medicine 314:414-18. Kwesigabo, G., J. Killewo, A. Sandstrom, S. Winani, G. Biberfield, and F. Mhalu. 1993. "Prev- alence of Hiv-1 Infection among Hospitalized Patients in Bukoba Hospital, Tanzania." Ab- stract MOP 57. Abstracts of the 8th International Conference on AIDS in Africa, Marrakesh, December. Lado, Cleophas. 1992. "Female Labour Participation in Agricultural Production and the Impli- cations for Nutrition and Health in Rural Africa." Social Science and Medicine 34:7, 789-807. "Life Insurance: Testing the HIV Limit." 1992. AIDS Analysis Africa 2(2, March/April). Rivonia, South Africa. Mann, Jonathan, Daniel J. M. Tarantola, and Thomas W. Netter, eds. 1992. "Funding the Glo- bal AIDS Strategy." In AIDS in the World. Cambridge, Mass.: Harvard University Press. Marter, Alan. 1978. "Cassava or Maize: A Comparative Study of the Economics of Production and Market Potential of Cassava and Maize in Zambia." University of Zambia, Institute for African Studies, Rural Development Studies Bureau, July. Processed. Masheija, C., J. Kipp, G. Hyakahuma, F. V. Sonnenburg, and T. Rehle. 1993. "Home-Based Care in Kabale District, Uganda." Abstract MOP 005. Abstracts of the 8th International Con- ference on AIDS in Africa, Marrakesh, December. Mburugu, Edward K. 1993. "HIV/AIDS Spread and Possible Impacts in Kenyan Universities." Pa- per presented at the AIDS and Education Seminar at the International Institute for Educational Planning, Paris, December 8-10. Processed. McGrath, J. W., E. M. Ankrah, D. A. Schumann, S. Nkumbi, and M. Lubega. 1993. "AIDS and the Urban Family: Its Impact on Kampala, Uganda." AIDS Care 5(1):55-70. Moss, A. R., and P. Bachetti. 1989. "Natural History of HIV Infection." AIDS 3:55-61. Mugisha, S. K. 1993. "The Impact of Fatal Adult Illness on the Education System in Kagera." In World Bank and University of Dar es Salaam, "The Economic Impact of Fatal Adult Illness in Sub-Saharan Africa." Report of a workshop in Bukoba, Tanzania. Processed. Mujinja, Phare, and Mead Over. 1993. "The Impact of AIDS on Health Care Utilization and Ex- penditure by the Fatally Ill in Northwest Tanzania." Abstract WS-D23-1, 9th International Conference on AIDS, Berlin. June. Ndilu, M., D. Sequeira, S. Hassig, R. Kanibak, R. Colebunders, M. Kashamuka, and R. Ryder. 1988. "Medical, Social, and Economic Impact of HIV Infection in a Large African Factory." Poster presented at the 4th International Conference on AIDS, Stockholm, Abstract 9583. 238 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Nkowane, Benjamin M. 1988. "The Impact of Human Immunodeficiency Virus Infection and AIDS on a Primary Industry: Mining (A Case Study of Zambia)." In Alan Fleming, Manuel Carbalo, David W. FitzSimmons, Michael R. Bailey, and Jonathan Mann, eds., The Global Impact of AIDS. New York: Alan R. Liss, Inc. Over, Mead. 1992. "The Macroeconomic Impact of AIDS in Sub-Saharan Africa." AFTPN Tech- nical Working Paper 3. World Bank, Population, Health and Nutrition Division, Africa Tech- nical Department, Washington, D.C. Processed. Over, Mead, and Peter Piot. 1993. "HIV Infection and Sexually Transmitted Diseases." In D. T. Jamison, H. Mosely, and J.-L. Bobadilla, eds., Disease Control Priorities in Developing Coun- tries. New York: Oxford University Press. Over, Mead, S. Bertozzi, J. Chin, B. N'Galy, and K. Nyamuryekung'e. 1988. "The Direct and Indirect Cost of HIV Infection in Developing Countries: The Cases of Zaire and Tanzania." In Alan Fleming, Manuel Carbalo, David W. FitzSimmons, Michael R. Bailey, and Jonathan Mann, eds., The Global Impact of AIDS. New York: Alan R. Liss, Inc. Padian, Nancy. 1988. "Prostitute Women and AIDS: Epidemiology." AIDS 2:413-19. Pallangyo, K., and Richard Laing. 1990. "Alternative Approaches to Managing the Opportunistic Illnesses of HIV-Infected Persons: Costs and Burden on the Tanzanian Health Care System." Background paper for the Tanzania AIDS assessment. World Bank, Washington, D.C. Processed. PANOS Institute. 1992. The Hidden Cost of AIDS. Washington, D.C. Philippson, Tomas J., and Richard A. Posner. 1993. Private Choices and Public Health. Cam- bridge, Mass.: Harvard University Press. Preble, Elizabeth A. 1990. "Impact of HIV/AIDS on African Children." Social Science and Medicine 31(6):671-80. Rutherford, G. W., A. R. Lifson, N. A. Hessol, W. W. Darrow, P. M. O'Malley, S. P. Buch- binder, J. L. Barnhart, T. W. Bodecker, L. Cannon, and L. S. Doll. 1990. "Course of HIV-1 Infection in a Cohort of Homosexual and Bisexual Men: An 11-Year Follow-Up Study." Brit- ish Medical Journal 301(6762):1183-88. Ryder, R. W., M. Ndilu, S. E. Hassig, M. Kamenga, D. Sequeira, M. Kashamuka, H. Francis, F. Behets, R. L. Colebunders, A. Dopagne, R. Kambale, and W. L. Heyward. 1990. "Hetero- sexual Transmission of HIv-1 among Employees and their Spouses at Two Large Businesses in Zaire." AIDS 4:725-32. Ryder, R. W., V. L. Batter, M. Nsuami, N. Badi, L. Mondele, B. Matela, M. Utshudi, and W. L. Heyward. 1991. "Fertility Rates in 238 HIV-1-Seropositive Women in Zaire Followed for Three Years Postpartum." AIDS S:lS21-27. Ryder, R. W., and R. D. Mgerwa. 1994. "The Clinical Definition and Diagnosis of AIDS in African Adults." In M. Essex, Souleymane Mboup, Phyllis J. Kanki, and M. R. Kalengayi, eds., AIDS in Africa. New York: Raven Press. Serwadda, David, Maria J. Waver, Stanley D. Musgrave, Nelson K. Sewankambo, Jonathan E. Kaplan, and Ronald H. Gray. 1992. "HIV Risk Factors in Three Geographic Strata of Rural Rakai District, Uganda.' AIDS 6:983-89. Shaeffer, Sheldon. 1993. "Impact of HIV/AIDS on Education." Background paper for the Expert Seminar on the Impact of HIV/AIDS on Education, International Institute for Education Plan- ning, Paris, December 8-10. Processed. Shepard, Donald S., and Richard N. Bail. 1991. "Costs of Care for Persons with AIDS in Rwanda." Paper prepared for the World Health Organization, Global Program on AIDS, October 31. Processed. Solon, Orville, and Angelica 0. Barrozo. 1993. "Overseas Contract Workers and the Economic Consequences of HIV and AIDS in the Philippines." In David Bloom, and Joyce V. Lyons. Eco- nomic Implications of AIDS in Asia. New Delhi: United Nations Development Program. Stoto, Michael A. 1993. "Models of the Demographic Effect of AIDS." In Karen A. Foote, Kenneth H. Hill, and Linda G. Martin, eds., Demographic Change in Sub-Saharan Africa. Washington, D.C.: National Academy Press. Martha Ainsworth and Mead Over 239 Stover, John. 1993a. "The Impact of HIV/AIDS on Population Growth in Africa." Paper prepared for the African Population Advisory Committee. World Bank, Washington, D.C., May. Pro- cessed Stover, John. 1993b. "The Impact of HIV/AIDS on Adult and Child Mortality in the Developing World." Paper presented at the International Union for the Scientific Study of Population (IUSSP) Seminar on AIDS Impact and Prevention in the Developing World: The Contribution of Demography and Social Science. Veyrier-du-Lac, France, December 5-9. Processed. Stover, John. 1994. Personal Communication. Futures Group, Glastonbury, Conn. Timaeus, Ian. 1993. "Adult Mortality." In Karen A. Foote, Kenneth H. Hill, and Linda G. Martin, eds., Demographic Change in Sub-Saharan Africa. Washington, D.C.: National Acad- emy Press. UNICEF. 1990. Children and AIDS: An Impending Calamity. New York. . 1991. "Report on a Meeting about AIDS and Orphans in Africa." Florence, Italy, June 14-15. Processed. United Nations/World Health Organization (WHO). 1991. "The AIDS Epidemic and Its Demo- graphic Consequences." Proceedings of the UN/WHO Workshop on Modeling the Demographic Impact of the AIDS Epidemic in Pattern II Countries: Progress to Date and Policies for the Future. New York. Way, Peter, and Karen Stanecki. 1991. "The Demographic Impact of an AIDS Epidemic on an African Country: Application of the iwgAIDS Model." Center for International Research Staff Paper 58, U.S. Bureau of the Census. Whiteside, Alan. 1991. "HIV Infection and AIDS in Zimbabwe: An Assessment." Southern African Foundation for Economic Research. Durban, South Africa. Processed. Whiteside, Alan, and David FitzSimmons. 1992. "The AIDS Epidemic: Economic, Political, and Security Implications." Conflict Studies 251. Research Institute for the Study of Conflict and Terrorism. Durban, South Africa. Processed. World Bank. 1992. Tanzania: AIDS Assessment and Planning Study. Washington, D.C. . Forthcoming. Better Health in Africa. Development in Practice series. Washington, D.C. World Health Organization (WHO)/Global Programme on AIDS. 1992. "Current and Future Di- mensions of the HIV/AIDS Pandemic: A Capsule Summary." WHo/GPA/REs/sFI/92.1, Geneva. . 1993. "The HIV/AIDS Pandemic: 1993 Overview." WHO/GPA/CNP/EVA/93.1, Geneva. 240 The World Bank Research Observer, vol. 9, no. 2 (July 1994) PRIVATIZATION: LESSONS FROM MARKET ECONOMIES Sunita Kikeri John Nellis Mary Shirley In the past decade governments all over the world have begun privatizing state enterprises-indeed, it is becoming quite hard to find a country without a pro- gram of privatization under way or at least on the policy agenda. This striking reversal of the push to expand state ownership in the 1960s and 1970s results from generally poor performance of state enterprises and a disappointing record of past reform efforts that fell short of ownership change. This article examines the objectives of privatization and the strategies for achieving them, documenting recent trends and reviewing the experience with privatizing state-owned commercial, manufacturing, and service enterprises in both competitive and noncompetitive markets. The authors analyze the various tactics that can be or have been employed, in relation to scope, pace, sequencing, and methods of implementation. The evidence shows that privatization produc- es benefits of efficiency and innovation-if done right. The lessons of experience discussed in the article offer guidance on how to realize the promise of privatiza- tion while minimizing the risks and costs. P rivatization is on the rise. Since the big expansion in state ownership in the 1960s and 1970s, the number of state-owned enterprises (SOEs) worldwide has been shrinking at an accelerating pace: more than 15,000 SOEs, at a conservative estimate, have been privatized, most of them since 1990.1 Years of disappointing SOE performance and a history of halfhearted and ineffectual attempts at reform have impelled many governments, including those of Chile, the former Czechoslovakia, Germany, Mexico, and Russia, to launch privatization programs. Rhetoric still exceeds action in many countries, but a substantial number that previously depended heavily on SOEs to fulfill their development objectives-such as Zambia and just about every ex-socialist The World Bank Research Observer, vol. 9, no. 2 (July 1994), pp. 241-72 © 1994 The International Bank for Reconstruction and Development/THE WORLD BANK 241 country in Eastern and Central Europe are now considering divesting most of their SOEs, including public utilities and enterprises formerly classified as strategic, such as airlines, ports, and railways and petrochemical, steel, cement, and even defense industries. This article reviews the experience with privatizing state-owned commercial, manufacturing, and service enterprises, operating in both competitive and non- competitive markets, touching tangentially on the special case of privatization of banks and other financial institutions. The article is based on a wide variety of sources, including authors' field work, World Bank and International Fi- nance Corporation (IFC) reports, a recently completed World Bank research project (Galal and others forthcoming), and external publications on privati- zation. Privatization is defined here as the transfer of majority ownership of SOEs to the private sector by the sale of ongoing concerns or of assets following liquida- tion. Methods of privatizing management but not ownership-through man- agement contracts, leases, and concessions-are discussed (less exhaustively) as alternatives to outright sale, but the analysis focuses on lessons derived from transfer of ownership. The article does not examine mechanisms, such as de- regulation and new private sector entry, or private sector financing through "build-operate-transfer"arrangements, that are sometimes included in broader discussions of privatization. Why Privatize? History and Evidence At one time or another since 1945 most countries in the world have attempt- ed to use SOEs to achieve their economic and social objectives. Developing countries typically relied more on SOEs than industrial economies did in the hope that they would balance or replace a weak or ideologically unacceptable private sector; would invest more and also produce a capital surplus to finance investment; and would transfer technology to strategic firms in mining, tele- communications, transport, and heavy industry. By the early 1980s SOEs ac- counted, on average, for 17 percent of gross domestic product (GDP) in Sub- Saharan Africa (in a thirteen-country sample; see Nellis 1986), 12 percent in Latin America, and a much more modest 3 percent in Asia (but this excludes China, India, and Myanmar), compared with 10 percent of GDP in mixed econ- omies worldwide (Short 1984). In Eastern Europe and Central Asia, SOEs uni- formly accounted for the bulk-as high as 90 percent-of all productive activities. Growth and Performance of SOEs Some SOEs performed, and still perform, well: the national electricity com- pany in France, the major steel producer in the Republic of Korea, the airline 242 The World Bank Research Ohserver, vol. 9, no. 2 (July 1994) in Ethiopia, a publicly owned fertilizer company in Indonesia, and many SOEs in Singapore are a few of many well-run, profitable, efficient state-owned en- tities. But the good performers tend to be the exceptions. Evidence from a wide range of countries shows that many SOEs have been economically inefficient and have incurred heavy financial losses. For example, World Bank estimates show SOE losses between 1989 and 1991 reaching 9 percent of GDP in Argenti- na, 8 percent in Yugoslavia, and more than 5 percent, on average, in a sample of Sub-Saharan African countries. In Turkey the marginal efficiency of SOE capital is half that of the private sector. Average return on capital employed between 1985 and 1991 was 12 percent for SOEs versus 20 percent for the pri- vate sector. Labor productivity in 1990 was 38 percent lower in SOEs overall than it was in the private sector. In the 1980s about half of Tanzania's more than 350 SOEs incurred losses that, in one year at least, put the entire sector in deficit. In 1991 about 30 percent of all SOEs in China incurred losses, and the consolidated government and enterprise deficit was in the range of 8 percent of GDP (McKinnon 1994; Yusuf and Hua 1992). In some countries, SOEs have become an unsustainable burden on the budget and the banking system, absorbing scarce public resources. Government trans- fers and subsidies to SOEs amounted to more than 3 percent of GDP in Mexico in 1982, 4 percent of gross national product (GNP) in Turkey in 1990, and 9 percent of GDP in Poland, according to World Bank estimates. In Ghana the annual average outflow from government to fourteen core SOEs in the late 1980s consistently exceeded the meager inflows-in the form of dividends and taxes-from the firms to the state. Arguably, these transfers and subsidies do not matter if the money spent alleviates a market failure or lays the base for future returns, but in some instances the costs of SOEs may exceed the costs of market failures, while investments in SOEs have not consistently produced cost- covering streams of income. The financial performance of nine key SOEs (tele- communications, postal services, airlines, railways, transport, power, cement, iron and steel, and textiles) in five West African countries (Benin, Ghana, Guin- ea, Nigeria, and Senegal) has been persistently poor, with annual government transfers and overdrafts to these sectors ranging from 8 to 14 percent of GDP, an indication that earlier efforts to reform these sectors have failed. Recent World Bank reviews of power and telecommunications sectors found steady deteriorations in the performance of many SOEs since the 1970s (World Bank 1993; Wellenius and others 1989; and Ambrose, Hennemeyer, and Chapon 1990). The problem is not simply that SOEs have yielded a disappointing rate of return on the capital invested in them. Overextended and poorly performing SOEs have also slowed the growth of the private sector in many developing countries. How does this happen? First, our field studies found that govern- ments often block the entry of private firms that would compete with SOEs. For example, Peruvian legislation at one time prohibited private firms from producing their own electricity, forcing them to buy from the often unreliable Sunita Kikeri, John Nellis, and Mary Shirley 243 public grid. Niger gave a state distribution company a monopoly on many food imports; when it was lifted, there was a rush of new private entrants-despite falling prices. In India entire sectors and subsectors of industry (energy, ship- yards, nonferrous metals, telecommunications equipment, mining, and steel plants) were statutory monopolies in 1988, reserved for SOEs. Second, government credit directed to capital-intensive SOEs often crowds private firms out of credit markets. Before privatization and reform in Guinea, SOEs-which contributed only 25 percent of GDP-absorbed 90 percent of for- mal credit from domestic banks. This imbalance might merely reflect the more capital-intensive character of state enterprises, but in fact the credit is often allocated by government directive and used to cover current account deficits. Some banking systems in developing countries are now insolvent because of the nonperforming debt of state enterprises. Third, even when government eliminates explicit guarantees for credit taken on by SOEs, bankers tend to assume that there is an implicit government guar- antee, a perception that affects their lending, to the disadvantage of the private sector. For example, in Russia, the rapidly growing private sector is finding it hard to obtain access to financing because credit allocation is still dominated by directed state credits to SOEs. And fourth, inefficient provision of critical inputs by badly managed public utilities has increased the costs of business to private firms and limited the potential for expansion, particularly for smaller firms (World Bank 1991). Still, even if one accepts the view that SOEs, on average, function poorly and impede private sector development, is privatization the solution? Or can the deficiencies of SOEs be remedied without changing ownership? In theory a state-owned firm should operate as efficiently as a private firm, if both function in a competitive market setting and if their behavior is governed by the same rules and incentives; and in fact, comparison of the performance of public with private enterprises (mostly based on experience in industrial countries) has not consistently provided compelling evidence that ownership matters. Many ana- lysts have concluded that competition and regulation are more important than ownership in determining economic performance (Vickers and Yarrow 1988; Hemming and Mansoor 1988), and that therefore policymakers should focus primarily on making markets work well. (It is less often acknowledged that well-functioning markets eliminate much of the justification for government ownership.) A common operational interpretation of this reasoning is that gov- ernments should privatize only if markets function reasonably well and there are no significant market failures. But much experience of the past two decades shows that, in practice, markets and public ownership are linked in ways that can reduce competition, even without a significant market failure. To illustrate: because losses from state enterprises can affect the public bud- get and debt, governments often intervene to protect high-cost SOEs from com- petition, to give them subsidies or privileged access to finance, or otherwise to tilt what should be a level playing field. To the extent that the high costs of 244 The World Bank Research Observer, vol. 9, no. 2 (July 1994) SOEs are the result of burdens imposed by government (which is often the case), this sense of obligation to protect them from competition is intensified. These burdens may be social (maintaining employment or developing backward re- gions), political (providing jobs and power to certain groups), or rent-seeking in nature. In this way state ownership, created to overcome or correct market failure, can sometimes aggravate or perpetuate it. In such cases privatization could well be used to enhance competition be- cause it reduces the government's direct stake in protecting particular indus- tries.2 The odds of succeeding are much improved when privatization is part of a more general liberalization process: in other words, when privatization is not delayed in the (often unfounded) hope that government will construct a well-functioning market but is used to help create that market through enlarg- ing the number of actors who have a direct stake in seeing markets work well. This role for privatization is regarded as crucial by Czech reformers, who con- sider privatization's principal function to be "opening economic space"; priva- tization also served this purpose in Chile and Mexico, where it was treated as an integral part of a broader liberalization program. In addition privatization and the closely associated process of liquidation may eliminate a government failure more costly to economic welfare than the market failure that the SOE was created to correct (as appears to have happened in some of the liquidations and sales in Guinea; see Suzuki 1991). Moreover, recent evidence shows gains from privatization even if market structures do not change. Chile offers a striking example. Infrastructure mo- nopolies in electricity generation and distribution and in telecommunications were efficiently run and effectively regulated as SOEs. Their privatization none- theless produced substantial welfare gains, primarily because private owners overcame a constraint on investment that the government had imposed on the SOEs, and secondarily because the new private management operated the com- panies more efficiently (Galal and others forthcoming). Finally, a focus on market failures alone ignores the findings from principal- agent literature, which imply that a change in ownership per se can affect eco- nomic performance, all other things being equal. By reducing the number of principals to a single owner whose one overriding objective is to maximize profits, privatization greatly simplifies the principal-agent problem and creates the potential for efficiency gains. In the Chilean case cited above, the profit- maximizing private owners responded to excess demand with new investment. In contrast, government's diverse principals with multiple objectives had long regarded investment in telecommunications as low priority and would have been unlikely to make the investment if the enterprise had remained public.3 Efforts at Reform In the past twenty years most countries in which SOEs were major economic and social actors-including most developing countries-have adopted reform Sunita Kikeri, John Nellis, and Mary Shirley 245 programs, short of ownership transfer, to try to rectify the causes of poor SOE performance. These reforms aimed to expose SOEs to domestic and external competition and level the playing field by eliminating easy access to credit from the budget and banking system; by freeing managers of SOEs from noncommer- cial goals and government interference in day-to-day decisionmaking; and by developing institutional mechanisms, such as contract plans and performance evaluation systems, to hold managers accountable for results. In response to these reforms, some SOEs have indeed performed better (see Galal 1991; Shirley and Nellis 1991). But two related problems persist. First, the reforms are technically and politically difficult to implement. Often, well- designed programs have fallen short in implementation. Second, performance does improve when the full reform package is put in place, but the necessary steps are numerous and hard to coordinate, and entire reform programs have seldom been enacted. For instance, governments often discontinue budget sup- port but continue to guarantee borrowing from the banking sector, and pay and employment reforms have yielded little without increased managerial au- tonomy and accountability. Both these problems could apply equally to privatization, which also requires technical competence, political will, and comprehensiveness. But a third and more important problem of SOE reform is sustainability. Improvements in SOE performance, unlike those arising out of privatization, are relatively easy to re- verse, whether in industrial countries such as Japan and New Zealand or in developing countries. In Senegal, for example, SOE performance overall has re- mained poor, despite reforms dating back to 1977. "Th-e effect of a slight finan- cial improvement achieved in the early 1980s soon dissipated, and total losses have continued to climb. The Senegalese SOEs that signed contracts pledging better performance (in return for greater autonomy) did perform better than those without contracts, but the device failed to impose crucial financial disci- pline. Although far from valueless, the performance contract has proved an in- adequate-because unenduring-tool in many other countries (see Nellis 1989). The Senegalese case illustrates the common problem: faced with a financial crisis or pushed by external forces, governments will initiate fundamental and far-reaching reforms in SOEs that give management the mandate and the power to run their companies in a commercial manner. As the reforms are enacted, economic and financial performance does improve-although usually at the ex- pense of some previously highly regarded political objectives. As the crisis fades, however, or the external pressure weakens, government ownership facilitates the revival of noncommercial objectives, which in turn often leads to renewed poor performance. So performance of SOEs can be improved without changing ownership, but these improvements are exceedingly difficult to sustain. Our investigations found that several SOEs, judged in the World Bank's World Development Report 1983 to be well on the road to enduring perfor- mance improvements thanks to exemplary government reform efforts (fertilizer SOEs in Turkey, manufacturing SOEs in Pakistan, and the Senegalese bus 246 The World Bank Research Observer, vol. 9, no. 2 (July 1994) company), either have not improved as expected or have even substantially de- teriorated. Bangladesh, for example, attempted to reform industrial SOEs in the 1980s by increasing managerial autonomy, restructuring the enterprises, and undertaking employment and wage changes. Yet, SOE performance deteriorat- ed throughout the reform period, and net transfers from the state to SOEs in- creased from 0.8 percent of GDP in 1986 to 3.2 percent of GDP in 1989. China launched a restructuring program in the 1980s to stem losses and im- prove the efficiency of SOEs by introducing bankruptcy legislation and compe- tition from nonstate enterprise. The reforms led to rapidly growing private, "collective," and town and village enterprise sectors (the share of national SOEs in industrial production dropped from close to 70 percent in 1986 to 53 percent in 1990), and the introduction of a "production responsibility system" im- proved performance in at least some industrial SOEs. Total factor productivity for SOEs rose at a respectable 3 percent a year between 1984 and 1988 (although this was well below the 6 percent rate of nonstate enterprises). Nonetheless, close to 30 percent of all SOEs still incur losses that absorb a sixth of the gov- ernment's budgetary expenditures; such losses rose substantially between 1987 and 1990 (Yusuf and Hua 1992). The Turn toward Privatization The high costs and poor performance of SOEs and the modest and fleeting results of reform efforts have turned many governments toward privatization. Other reasons include shifting development theory and ideologies in the face of mounting SOE losses, the collapse of communism in Eastern Europe and the Soviet Union, and some successes of early privatizers such as the United King- dom. Fiscal crises have also led some governments to privatize as a way to raise revenues and stem losses, especially in the face of increasing public debt. More governments have opted for privatization because of their inability to finance needed investment in their SOEs than because of their expectations of efficiency gains. Finally, some of the initial reasons for state ownership have disappeared. Technology and growth have introduced competition into traditionally mo- nopolistic activities such as telecommunications or the generation of electricity; the emergence of a dynamic private sector has weakened the argument that state ownership is needed as a substitute for frail or nonexistent domestic en- trepreneurship in many developing countries; and increasing capacity of gov- ernments, at least in the middle-income developing countries, to curb the excesses of multinational companies has weakened the case for SOEs as defend- ers of sovereignty. Between 1980 and 1992 more than 15,000 enterprises were privatized world- wide (figure 1), more than 11,000 of these in the former East Germany. Among the more than 3,800 sales in countries that borrow from the World Bank, the former socialist economies in Eastern Europe and Central Asia and countries in Latin America together account for 85 percent (figure 2).4 Sunita Kikeri, John Nellis, and Mary Shirley 247 Figure 1. Percentage of SOEs Privatized Worldwide, bv Region, 1980-92 Germany (71) OECD (excluding Germany) (1) Middle East and North Africa (0.4) Sub-Saharan Africa (3) '~'c' ~ ______Eastern Europe and Central Asia (18) South Asia (0.5) East Asia and the Pacific (0.7) Latin America and the Caribbean (6) Source: World Bank, Country Economics Department; Sader (1993); Privatisation International Yearbook (various years); Im, Jalali, and Saghir (1993); Saghir (1993). Privatization has substantially reduced the size of the SOE sector in several countries. Between 1984 and 1990 Mexico privatized more than 20 percent of total SOE assets by selling or liquidating more than 400 of its 1,155 SOEs (in tele- communications, airlines, sugar, mining, manufacturing, banking, and other services) and merging or transferring to municipalities a further 400. It has con- tinued the divestiture process in the 1990s. Chile, in privatizing all but 23 of its 524 SOEs since 1973, reduced state ownership of producing assets from 39 per- cent of GDP in 1973 to 12 percent in 1989. Argentina and Venezuela significantly downsized their SOE sectors by selling their telephone companies and airlines. They are now privatizing other utilities and large industrial SOEs, and Argentina in particular is well on its way to eliminating virtually all of its SOE sector, in- cluding industries under the Ministry of Defense (Alexander and Corti 1993). But in most developing countries, particularly the low-income countries, the change has been modest. Most often it is the small, low-value firms in industry and services that have been sold. In Guinea, for example, seventy of ninety- eight privatizations carried out in the mid-1980s dealt with virtually defunct retail outlets and small, nonoperating enterprises. Tunisia partially or fully privatized thirty-nine SOEs between 1988 and 1992; in every case the firm di- vested was a small, usually money-losing operation, functioning in a competi- tive market. The revenue generated by all thirty-nine sales equaled slightly 248 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Figure 2. Percentage of SOBs Privatized in Countries 7hat Borrow from the World Bank, by Region, 1980-92 Eastern Europe and Central Asia (64) ____South Asia (2) East Asia and the Pacific (2) Sub-Saharan Latin America and Africa (9) - - - - -the Caribbean (21) Middle East and North Africa (1) Source: World Bank, Countrv Economics Department; Sader (1993); Privatisation International Yearbook (various years); Im, Jalali, and Saghir (1993); Saghir (1993). more than 1 percent of the SOE sector's 1987 book value (authors' field work). This emphasis on the small is changing; in the past five years alone, thirty-five very large SOEs with a combined sales value of more than $22 billion have been privatized in fourteen developing countries (table 1)V The Impact of Privatizati'on As noted, economists and others have long been concerned with the rela- tionship between ownership and efficiency. The 1970s and the early 1980s wit- nessed an outpouring of articles and studies comparing the efficiency of similar types of production or service under private and public ownership (Borcherd- ing, Pommerehne, and Scheider 1982; Domberger and Piggott 1986; see also Yarrow 1988, p. 374, table a.1 for a listing of empirical studies). Most of the cases were drawn from industrial countries and focused on municipal services, airlines and other transport services, electric utilities, insurance, hospitals, and housing. The studies were partial, in that they concentrated mostly on costs. Most found that private production was indeed cheaper than public, but they tended to conclude-as one study put it-that "it is not so much the difference in... ownership but the lack of competition which leads to the often observed Sunita Kikeri, jobn Nelis, and Mary Sbirley 249 Table 1. Recent Privatization Transactions Valued at US$100 Million or more, 1988-92 Gross transaction Date of value (millions Economy Enterprise sale of US$) Sector Mexico Bancomer 10/91 2,550 Banking Mexico Bancamex 9/91 2,300 Banking Korea, Republic of Korea Electric Power 6/89 2,100 Power Venezuela CANTV 11/90 1,885 Telecommunications Mexico TELMEX 12/90 1,760 Telecommunications Brazil Usiminas 12/91 1,430 Steel Mexico Mexicana de Cobre 10/88 1,360 Mining Argentina ENTEL 11/90 1,244 Telecommunications Mexico Banca Serfin 1/92 909 Banking Mexico Multibanco Comermex 2/92 872 Banking Malaysia Telekom Malaysia 10/90 861 Telecommunications Brazil Copesul 5/92 839 Petrochemical Mexico Cananea 9/90 475 Mining Argentina Somisa 10/92 404 Steel Philippines Philippines Airlines 1/92 368 Airline Mexico Aerovias de Mexico 11/88 339 Airline Philippines Nonoc 10/90 325 Mining Taiwan (China) China Steel 4/89 285 Steel Argentina Aerolinas Argentinas 4/90 285 Airline Mexico Banca Cremi 6/91 248 Banking Mexico Multibanco de Mercantil 6/91 204 Banking Mexico Banpais 6/91 182 Banking Mexico Sicartsa 1 11/91 170 Steel Chile Compafiia de Telefonos 1/88 170 Telecommunications Mexico Sidermex North 11/91 145 Steel Venezuela VIASA 9/91 145 Airline Mexico Mexicana de Aviacion 6/89 140 Airline Brazil Aracruz 5/88 130 Pulp and paper Turkey Petkim 6/90 125 Petrochemical Peru Hierro Peru 12/92 120 Mining Poland Kwidzyn 8/92 120 Pulp and paper Hungary Tungsram 5/89 110 Electric equipment Mexico Nikko Hotel 10/88 110 Hotel Mexico Terefaltos Mexicanos 1/88 106 Chemical Colombia Papelcol 8/90 100 Pulp and paper Source: Privatisation International Yearbook and World Bank data. less efficient production in public firms." This study reasoned that "given suf- ficient competition between public and private producers (and no discrimina- tive regulations and subsidies), the differences in unit cost turn out to be insignificant"(Borcherding, Pommerehne, and Scheider 1982: 136). As of the late 1980s few attempts to examine the difference between public and private production had been undertaken in developing countries. A study comparing the performance of fifty-three public and twenty-four private 250 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Tunisian enterprises (in similar sectors) found that the productive efficiency of public enterprises was not detectably different from that of private firms (Svejnar and Hariga forthcoming). A survey of the scant material from cases in developing countries offered a guarded conclusion: "There is no evidence of a statistically significant kind to suggest that public enterprises in LDCs have a lower level of technical efficiency than private firms operating at the same scale of operation. [But] on a less formal level the tendency . . . seems to be pointing in that direction"(Millward 1988: 157). Thus, past studies undertaken in industrial economies largely attributed su- perior efficiency in private over public firms to market structure rather than to ownership, while the few studies of developing countries revealed marginal ef- ficiency differences between public and private firms. However, more recent ev- idence, which compares SOE performance before and after privatization, shows considerable economic benefits resulting from properly structured privatiza- tion, even when the privatized firms were monopolies. A recent World Bank research project, in contrast to earlier partial studies, examined the effects of divestiture on all important actors. It also assessed the long-run impact of privatization by isolating its effects on firm behavior from concurrent changes (such as changes in macroeconomic policy, technology, de- mand structure, or the regulatory framework) and answering the counterfac- tual-what would have happened to performance if ownership had not changed? The study found that privatization significantly improved domestic and international welfare in eleven of twelve divestitures analyzed, where nine of the twelve were noncompetitive (Galal and others forthcoming). Productiv- ity rose in nine cases and stayed the same in the other three. Relaxation of the investment constraint and diversification into previously forbidden products and markets resulted in massive expansion in most cases. Workers as a class were judged to be no worse off after sale (layoffs did occur, but affected work- ers received generous compensation) and in three cases were significantly better off (thanks especially to share purchase schemes for workers who retained their jobs). Buyers most often made money, but the other stakeholders also gained in the process. In most cases consumers benefited or were no worse off. Other studies, less rigorous but with broader coverage, also found improve- ments after privatization. A study of forty-one firms fully or partially priva- tized between 1981 and 1989 by public share offerings in fifteen countries (primarily industrial, but including Chile, Jamaica, and Mexico) found sub- stantial efficiency gains (Megginson, Nash, and van Randenborgh 1992). The privatized firms increased returns on sales, assets, and equity; raised internal efficiency; improved their capital structure; increased investments; and margin- ally increased their work forces as a result of faster growth. In some other cases-for example, Argentina, Chile, Niger, and Tunisia-dynamic expansion and investment in modernization by the new owners raised employment levels in several of the firms investigated (authors' field studies). Similar results were found in an analysis of sixty-two privatized petrochemical and auto parts firms Sunita Kikeri, John Nellis, and Mary Shirley 251 in Mexico, where privatization also helped reduce management numbers but allowed the remaining managers to be paid at more competitive rates (Perez Escamilla 1988). Most clear-cut success stories come from high- or middle-income countries. The privatization process is harder to launch in least developed settings, and the chances of failure are greater. Still, positive results have been obtained. In Bangladesh privatized textile companies were more profitable than public tex- tile mills. This was partly attributable to debt write-offs, but greater attention to cost containment and more aggressive marketing also helped (Lorch 1988). Privatization revived a near-dead textile company in Niger, which now oper- ates profitably at close to full capacity with a larger work force; the company exports much of its production and has won a large domestic market share against imports (authors' field study). In Swaziland a development finance cor- poration, which had been closed before sale, became profitable in its second year of private ownership; a privatized agro-industrial SOE in Mozambique di- versified its products, began servicing its debts, and increased production five- fold (IFC field studies). Evidently, privatization can yield positive results even in difficult settings. Revenues from some sales have been large; gross proceeds from the sale of about 700 SOEs in five Latin American countries exceeded $18.3 billion during the past ten years, or 8 percent of their total outstanding foreign debt as of 1990 (authors' estimates). But net revenues have often been modest or even negative because of the small size of transactions and the need to pay debt, delinquent taxes, and transaction fees from the proceeds-and because many sales have been on credit. Privatization in many cases has reduced the overall financial burden of SOEs and at the same time has increased government income, because taxes paid by privatized firms have sometimes exceeded those taxes or dividends previously paid by SOEs. In Mexico government transfers and subsidies to SOEs declined by 50 percent between 1982 and 1988; the stabilization program after the 1982 shock was certainly the most important cause, but privatizations helped lock in these reductions. In a particularly dramatic case, Malaysia sold a sports lot- tery in 1985; by 1989 revenues from levies on the privatized lottery were three times greater (in real terms) than revenue from the former SOE (Galal and oth- ers forthcoming). In Chile the net annual flow of funds from the electricity dis- tribution company ENERSIS declined after privatization because the government no longer received dividends, but taxes increased as performance gradually im- proved (Galal and others forthcoming). Consumers can gain from privatization. The World Bank research project cited above (Galal and others forthcoming) found that, quite apart from any benefits from improved service, consumers for the most part gained or re- mained unaffected by privatization. In the five instances in which consumers did lose, the (generally small) losses were attributable mostly to prices moving closer to their scarcity values. The study concluded, for example, that consum- 252 The World Bank Research Observer, vol. 9, no. 2 (July 1994) ers in Chile were unaffected by the sale of ENERSIS; paying consumers were better off, but this gain was balanced by losses to those who had hitherto been getting the service free through illegal connections. (If only those who had pre- viously been paying had been considered legitimate consumers, the findings would have shown substantial increases in consumer welfare.) In the aggregate, telecommunications consumers in the United Kingdom are better off. Consum- ers of long-distance service did better in all years, while the position of con- sumers of local and other services remained more or less unchanged. Increased competition and technological changes played a part, but new managers with full operational autonomy and an effective regulatory framework also made a difference. The quality of service deteriorated in the early years after privati- zation (partly because of increased demand), but subsequent improvements now make the service better than it was before privatization. Privatization Objectives and Strategy Once a government has decided to privatize, how should it go about the process? Rapidly accumulating experience offers several lessons. Defining Objectives Enhancing efficiency should be the primary goal. Governments may feel pres- sure to design privatization programs to minimize losers and maximize winners or to maximize revenues so that losers can be compensated. But designing pro- grams instead to maximize efficiency, although politically more difficult, will bring to the economy more sustained gains, which can then be distributed to potential losers. Privatization does the most good when it is used to increase competition and prevent monopolistic behavior. The high risks in developing countries often lead prospective investors to argue for special privileges and protection in exchange for private investment, but this can lead to perverse re- sults. A steel mill in Togo, for example, was leased to a private owner but con- tinued to enjoy heavy protection. Although the leaseholder made money, overall economic welfare suffered. The correct economic decision would have been to liquidate the steel mill, but the government feared the political conse- quences of closure. Opportunities for competition exist even in sectors once regarded as naturally monopolistic. For example, the water and electricity gen- eration companies in the United Kingdom were divided into ten and sixteen smaller units, respectively, before being sold. The companies compete directly, and the performance of a water company in one geographical area can be com- pared with those in other regions to encourage "yardstick competition," under which performance indexes are collected, adjusted for regional differences, and then compared. A company whose performance falls below its historical posi- tion in relation to the average might risk losing its franchise. Sunita Kikeri, John Nellis, and Mary Shirley 253 Maximizing short-term government revenues should not be the primary con- sideration. Selling SOEs as monopolies to maximize the selling price can be good for the budget but bad for the economy. In privatizing infrastructure SOEs, a monopoly concession in some activities may be unavoidable because of economies of scale. But society will benefit if the government first deregulates and establishes adequate tariff regulation, and then privatizes-even if that means a lower sale price. To maximize the selling price and short-term revenue to government, Jamaica appears to have given away too much in the transac- tion, granting its privatized telephone company a twenty-five-year concession on local and competitive international services that guaranteed a rate of return exceeding industry norms and providing few incentives to reduce costs. By con- trast, other telecommunications privatizations in the region have entailed both shorter monopoly periods and lower rates of return. In the recent telecommu- nications sales in Argentina, Mexico, and Venezuela, nonbasic services were opened up to varying levels of competition. Private purchasers obtained seven- to ten-year concessions in local services, but the deals were combined with in- centives to encourage expansion and reduce costs. For example, in Mexico the privatized TELMEX pays a corporate tax rate of 10 percent if it meets the prom- ised investment schedule; 29 percent if it does not. In this instance, as in many other infrastructure sales, greater weight was given to follow-on investments than to price. Privatization has been used successfully to develop capital markets in Chile, France, Jamaica, Japan, Nigeria, and the United Kingdom (table 2). Yet, over- emphasizing its role in developing capital markets can cause problems. Weak Table 2. Privatization and Capital Market Development Number of Proceeds of Proceeds as a enterprises sale through percentage of Number sold through stock exchange stock market of new Country stock exchange (millions of US$)a capitalizationb shareholders Canadac 2 812 0.4 n.a. Chile (since 1985) 14 894 9.3 63,316d France 14 5,148 3.0 5,000,000 Jamaica 3 121 12.6 30,000 Japan 1 75,600 1.7 1,670,000 Nigeria 16 27 2.0 400,000 Trinidad and Tobago 2 7 2.5 n.a. Tunisia 2 9 n.a. n.a. United Kingdom 14 51,721 n.a. 7,400,000 n.a. Not available. a. Includes share auctions and public offerings. b. Market capitalization in year of last public offering. c. Includes only federal ("Crown") enterprises. d. Does not include shareholders in ENDESA. Source: For Chile, France, Jamaica, and the United Kingdom: Bouin and Michalet (1991); for Canada, Nigeria, Trinidad and Tobago, and Tunisia: World Bank data; for Japan: Takano (1992). 254 The World Bank Research Observer, vol. 9, no. 2 (July 1994) absorptive capacity creates delays and crowds out new private share issues; the sale of 24 percent of Telekom Malaysia in September 1990, for example, aimed to raise M$2.35 billion-nearly as much as the M$3 billion the Kuala Lumpur stock exchange had raised in total in 1989. Selling poorly performing enter- prises without adequate information and prudential regulation may exploit small, first-time investors, end up in government bailouts, or undermine the credibility of future sales. In most developing countries the kind of preparation poorly performing SOEs need for their shares to sell on the stock market re- quires time and resources better spent on activities benefiting a larger part of society. Finally, capital market transactions involve potentially large pricing discounts (see below) that can result in a political backlash. Short-run distributional goals, although they cannot be ignored, should not be pursued at the cost of managerial competence. Many governments (Chile, France, Jamaica, Nigeria, and the United Kingdom, for example) see wide distribution of shares as an important objective of privatization, even though share owner- ship tends to reconcentrate over time.6 But dispersed ownership may come at the expense of management performance. In mature stock markets, the threat of corporate takeovers and bankruptcy, influential financial journalism, and the active participation of nonexecutive directors can to some extent provide man- agement discipline. Most developing countries lack these, and the managements and corporate governance of most SOEs are usually in need of a shakeup. One way both to improve corporate governance and to spread ownership is to reserve core shareholdings (at least temporarily) for strategic investors with a significant stake in the company, as has been done in Chile and France and in a few innovative privatizations in Tunisia. Skilled owners or investors are thus placed in charge of the assets, although this tactic could create entrenched interests. In all twenty-eight of the privatizations conducted between December 1989 and May 1993 in Sri Lanka, a majority shareholding was sold on tender to a corporate investor, 10 percent was gifted to the employees of the privatized company, and 30 percent was reserved for subsequent sale to the public through the stock exchange (as of May 1993, nine public offerings had been concluded, and another six were in process; see Jayasinghe 1993). When priva- tizing state-owned banks, the governments of Mexico and Venezuela attempted to resolve the potential problem of entrenched interests by giving incentives to core investors, but requiring them to divest a portion of their shares to the pub- lic and employees after an initial period. Another solution is to involve institutional investors-life insurance, pen- sion, and provident funds-where trade buyers are absent. Although they tend to be more passive than trade owners, often exerting limited influence in direct corporate governance, institutional investors are preferable to a large number of dispersed shareholders precisely because of their size. Because institutional investors are large, they have economies of scale in collecting information on companies, and their decisions to sell or buy can affect share prices. Thus, they are more informed than individual investors and have more incentive to Sunita Kikeri, John Nellis, and Mary Shirley 255 monitor performance and keep management up to the mark-and manage- ment has more incentive to listen. Private pension funds in Chile do this well: they now hold more than half of all the corporate bonds issued by privatized companies and a significant percentage of their equity as well. Findings of re- search on the effects of institutional investors are, however, ambiguous: some studies, for instance, argue that the ease with which institutional investors can move their shares from firm to firm leads them to focus on short-term profits (see, for example, Black 1992; Porter 1992). Conditions for Success Two factors are paramount in determining the strategy and influencing the outcomes of privatization for economic productivity and consumer welfare. The first is country conditions, in particular the extent to which free entry is allowed by the macroeconomic and regulatory policy framework, markets are competitive, resources are allocated on the basis of marginal returns, and the regulatory and supervisory institutions are effective. The second is the nature of the market into which the enterprise is divested, that is, whether the market is competitive or potentially competitive as opposed to noncompetitive (see table 3). (A potentially competitive market is defined as one in which relax- ation of legal or regulatory barriers to entry would reasonably be expected to produce, in the short to medium term, either domestic or foreign competitors.) COUNTRY CONDITIONS. Privatization of both competitive and noncompetitive SOEs is easier to launch and more likely to yield financial and economic benefits in countries that encourage entry and free trade, offer a stable climate for Table 3. Privatization: A Framework for Decisionmaking Enterprise conditions Country conditions Competitive Noncom petitive High capacity to regulate; Decision Decision market friendly * Sell * Ensure or install appropriate regulatory environment * Then consider sale Low capacity to regulate; Decision Decision market unfriendly * Sell, with attention to * Consider privatization of competitive conditions management arrangements * Install market-friendly policy framework * Install appropriate regulatory environment * Then consider sale 256 The World Bank Research Observer, vol. 9, no. 2 (July 1994) investment, and have a relatively well-developed institutional and regulatory capacity. The advantages of creating the right policy and institutional frame- work are illustrated by successful privatizers such as Chile and Mexico, which began macroeconomic and public sector reforms well before privatization. Chile, for example, reduced tariffs to 10 percent across the board, eliminated its budget deficit through tax reforms and expenditure cutbacks, and intro- duced a stabilization program that reduced inflation from 380 percent in 1974 to 38 percent in 1979 (Corbo 1993). Such reforms attract private investors and ensure that privatization expands competition and productive efficiency, rather than simply transferring rents from SOEs to private owners. In low-income countries, privatization of both competitive and noncompet- itive SOEs is a challenge. In addition to the usual social and political obstacles, such countries often face macroeconomic distortions; thin capital markets; lim- ited interest from foreign investors; and weak legal, regulatory, and institution- al capacity. Poorly conceived and badly managed privatization in such settings has compounded the problem. Terms and conditions of sale granting excessive sweeteners such as new monopoly rights or overly generous tax and duty ex- emptions, have reduced competition and efficiency, as in Guinea, for example (see Suzuki 1991). In other instances, privatized enterprises have gone bank- rupt, although closures do not necessarily indicate that the policy of privatiza- tion is misguided. Many of these firms had not been viable when they were state owned but had been kept alive by government protection and subsidies. Few developing countries can afford to subsidize, at the expense of all of their citizens, the relatively small number of people in unproductive SOE jobs. The demise of loss-making firms (public or private) can free assets for more pro- ductive use, eliminate a burden on the economy, and allow investments and job creation elsewhere. The distributional effects of privatization in low-income countries are yet to be thoroughly analyzed, but a concentration of assets in the hands of a small elite is a legitimate concern. Sales without competitive bidding, at predeter- mined concessionary prices or into protected markets, have helped enrich small groups of well-connected individuals. The examples cited earlier show that privatization in low-income countries can yield significant economic gains in those (unfortunately few) cases where it is well managed, but the fact remains that the weaker the economy and the governing institutions, the more difficult it will be for privatization to yield benefits. MARKET CONDITIONS. Privatizing enterprises that produce tradables-in com- petitive or potentially competitive sectors such as industry, agriculture, and re- tail operations-is easier because regulation is not required. If privatization also expands competition-a likely outcome because, for reasons cited earlier, governments find it hard to maintain a level playing field for their SOEs-then solid and rapid economic benefits will result (as long as there are no economy- wide distortions that hinder competition). In countries where markets prevent Sunita Kikeri, John Nellis, and Mary Shirley 257 monopolistic behavior and provide incentives for efficiency, successful privati- zation may require little more than adequate attention to transparency. Even in countries where markets are weak, privatization may still be beneficial if it reduces the fiscal burden of SOE subsidies and exposes fully the costs of the distortions. In such circumstances, rapid privatization of competitive firms, ac- companied by liberalized trade and prices and strengthened institutional and regulatory framework, should be feasible. Privatizing SOEs that operate as monopolies, such as power, water, or tele- communications, is harder in any setting: the enterprises are larger, the stakes higher, foreign investment issues more sensitive, and regulatory questions more salient. Nonetheless, the research cited above shows that privatizing such en- terprises has yielded benefits in Chile, Mexico, and the United Kingdom (Galal and others forthcoming). A regulatory framework that sets out the tariff re- gime, establishes goals for universal service, develops targets for minimizing costs, and has credible regulatory institutions that can supervise the established procedures and resolve conflicts is essential to clarify the rules of the game and create a stable and predictable operating environment for private investors. Such a framework also helps allay fears about equity and political opposition, allowing decisionmakers to erect mechanisms to defend transparency, compe- tition, and the public interest. Chile's well-developed regulatory framework en- sured that sale of its power companies increased efficiency without harming consumer interests. Tariffs were structured so that large consumers with high demand at peak periods, who cause the system to expand, would pay a higher price than consumers who do not cause this expansion; these would pay a price equivalent to the short-run marginal cost. Suppliers to large consumers have to compete in this segment of the market. Privatizing utilities and other natural monopolies is especially difficult in low-income countries, where private investors are few, capital markets thin, and the institutional and regulatory framework inadequate. In such instances, privatizing management is often a step in the right direction. Even where mo- nopolies remain state owned, competitive activities can be separated out, and services provided in-house can be competitively bid for, as in Venezuela, where a private company collects tariffs for the public water company. What, How Much, and How Fast to Sell Most divesting governments-including Chile, Jamaica, Mexico, the Philippines, Poland, Togo, Tunisia, and the United Kingdom-began by selling small and medium-size firms that operated in competitive markets. Such sales require little prior restructuring, entail minimal political risk, are more easily absorbed by local private investors, and help prepare for subsequent sales of large and complex SOEs. The firms can and should be privatized quickly in or- der to put the assets to productive use. The Mexicans see this as the main les- son of their successful experience. They recommend starting with the small 258 The World Bank Research Observer, vol. 9, no. 2 (July 1994) firms to learn how to do it, to educate the public, and to minimize risks-in the words of Pedro Aspe, Minister of Finance in Mexico in 1991: "If one makes a mistake selling a night-club or a bicycle factory . . . it is not as tragic as if these mistakes are made while selling the largest commercial bank in the coun- try, the telephone company, or a major airline"(Aspe 1991: 8). But there is no universal strategy. The choice of where to begin depends on investor interest, government capacity, and which sectors and SOEs most need new investments and improved efficiency. Some countries choose to privatize public utilities first, despite the complexities involved. The window of political opportunity may be open but briefly, and the most important cases may best be tackled before conditions change. Large privatizations give the policy in- stant credibility and send clear signals of commitment to financial markets and investors. The potential economic and financial benefits of privatizing the util- ities first may make it worth the risks: privatizing badly managed utilities that provide critical goods and services can help accelerate modernization and growth. Latin American countries (Argentina in particular) have taken this course with good results. Privatizing Management A big advantage of sales over divestiture methods that do not privatize own- ership is that sales transfer property rights to profit-oriented owners who have an incentive to improve performance. But outright sales may not always be fi- nancially or politically feasible, particularly in low-income countries or in en- terprises, such as railways, water, and power, that require a big investment in modernization. In such instances, one option is to bring in private managers and allow the SOE to operate much like a private firm, even if the assets remain state owned. Management contracts have worked well in sectors such as hotels, airlines, and agriculture, where contract negotiation and monitoring are routine and where there is an ample supply of experienced managers. But management con- tracts have disadvantages. Typically, contractors do not assume risk, and loss- es are borne by the state. Flat fee-for-service arrangements, payable regardless of performance, provide little incentive to improve efficiency and maintain the value of assets (Hegstad and Newport 1987). Properly drawn contracts can minimize these problems, but that-particularly in low-income countries- usually requires strengthening government's capacity to monitor and enforce contractual obligations. Once again, the dilemma surfaces: in low-income countries, the very factors that block privatization also impede alternative solutions. Leases overcome some of these drawbacks because the private party, which pays the government a fee to use the assets, assumes commercial risk and has more incentives to reduce costs and maintain the value of the assets. Lease arrangements have been widely used in Africa in sectors that have difficulty Sunita Kikeri, john Nellis, and Mary Shirley 259 attracting private investors: water supply in C6te d'Ivoire and Guinea, power in C6te d'Ivoire, road transport in Niger, port management in Nigeria, and mining operations in Guinea (Triche 1990). The gains have been substantial. In C6te d'Ivoire, for example, the leased water company improved technical efficiency, new connections, and billing and collection of receivables-and re- duced the number of highly paid expatriate staff members by 70 percent. Concessions, in which a private operator is responsible for capital expendi- tures and investments as well as existing assets, are more desirable for the gov- ernment but less feasible than leases because private financing (or willingness to finance) tends to fall short of the needed investment, particularly in sectors or countries where political and economic risks are high. Concessions have been successfully used in the recent telecommunication and railway privatiza- tions in Argentina. Analysis of the experience with private management arrangements empha- sizes that governments should avoid interfering in management and should concentrate instead on accountability. SOE management can be held account- able by various mechanisms: business plans, properly staffed and empowered boards of directors, contract plans and performance agreements, and perfor- mance evaluation and incentive systems. All are costly and difficult to install; none are foolproof methods of improving performance. Managers could be given incentives to improve operations and enhance the long-term value of the assets by linking fees to performance, encouraging equity investments, or al- lowing managers to purchase some or all of the assets or shares when the con- tract or lease expires. (Obviously, this last option must not link market value at the end of the lease period with purchase price, or the lessee would have an incentive to run down the value of the enterprise.) Finally, private management arrangements are likely to work best when they are a step toward full privati- zation, because it is change in ownership that checks government interference and brings in badly needed investment capital. Full or Partial Sale? Selling minority shares has sometimes worked well, particularly when com- petition is introduced; when managerial control is transferred to competent core investors; when government's voting rights are limited; and when the mi- nority share offering is a prelude to a majority share offering. In Malaysia, par- tial sale of the port operations significantly improved behavior and enhanced productivity (Galal and others forthcoming). In Indonesia the public offering of 27 percent of equity of a state-owned cement company reportedly changed operational behavior for the better. Management reported that the need to meet listing requirements on the stock exchange led the company to hire pri- vate accountants, which improved transparency; managers also claimed that, to maintain share values, they implemented cost-cutting measures that had pre- viously been avoided (authors' field notes, October 1992). In Japan the sale of 260 The World Bank Research Observer, vol. 9, no. 2 (July 1994) 33 percent of shares of Nippon Telephone and Telegraph (NTT) to 1.6 million small shareholders (with a further government commitment to sell up to 67 percent of total shares), combined with increased competition and the ap- pointment of a new chief executive officer from the private sector, resulted in higher profits and improved service for NTT despite reduced long-distance tar- iffs and staff cuts of 20 percent over five years (Takano 1992). But the NTT case also illustrates the limits of partial privatization. Most shares remain state owned, and enactment of NTT's Corporation Law has al- lowed the government to remain involved in company operations-in some ar- eas, even more than it could when NTT was a state-owned company. For example, the regulatory ministry can approve or reject appointments to the se- nior executive corps, even after such appointments have been approved by the shareholders' meeting; the ministry also has the authority to approve or reject NTT's business plan (Takano 1992). Implementation Once the commitment to privatize has been made and a strategy planned, the details of implementation present their own challenges to policymakers. The principal issues are the preparations needed for sale, pricing and valuation, financing, and managing the process. Preparing for Sale Experience shows that small and medium-size competitive SOEs can and should be sold rapidly through competitive bidding without much restructur- ing. But selling large enterprises requires considerable preparation, for exam- ple, legally transforming public entities into corporate forms that permit private shares (such as joint stock companies), breaking up large firms and mo- nopolies into viable and nonviable units, separating competitive from noncom- petitive activities, and identifying peripheral assets (such as real estate holdings, sports teams, and restaurants) that can be sold as separate concerns. Many such actions have been taken in the former East Germany and elsewhere in Eastern Europe and Central Asia, as well as in Argentina (railways, steel) and Mexico (steel). Successful large privatizations-the utilities in Chile, Mexico, New Zealand, Venezuela, and the United Kingdom are examples-have also entailed the ap- pointment of new managers, often from the private sector, with different atti- tudes and approaches and increased autonomy to prepare for privatization. SOEs are typically encumbered by large debts. Private buyers are usually re- luctant to take on the debt, and governments are rarely willing to discount the sale price by the amount of the debt; debt write-downs are standard practice of divesting governments the world over. Such debt relief should occur only Sunita Kikeri, John Nellis, and Mary Shirley 261 when ownership and management change hands, and not before, otherwise there is a real risk that arrears will simply recur. The extent of the write-off varies. The governments of Argentina and Venezuela assumed debts of $930 million and $471 million, respectively, upon the sale of their telephone companies. In Germany government assumes, on average, 70 percent of the old debts of the companies sold. Many SOEs have significant potential environmental liabilities, an issue par- ticularly germane to Eastern Europe and Central Asia. Inappropriately dis- posed waste can be cleaned up before sale or by the purchaser as a condition of sale. Preparation for privatization can also involve downsizing the labor force, as occurred in Argentina, Japan, Mexico, New Zealand, Tunisia, the United Kingdom, and elsewhere. Large-scale labor shedding is best handled by the state, particularly for large firms or when liquidation precedes sale of assets. Private investors would rather avoid contentious and highly visible labor dis- putes; and the government is better able to design social safety net measures such as retraining, severance pay, and unemployment insurance. Some governments, in the interest of speed, have sold large firms with their labor force intact. This strategy has worked well in telecommunications and other high-growth industries able to absorb excess labor. In some instances governments have limited the latitude for private owners to make labor deci- sions, but such restraints have reduced investor interest (as in Pakistan, where there were twelve-month restrictions on layoffs) and have invited demands for subsidies to cover costs (as in the case of jute mills in Bangladesh). Moreover, restrictions may not be easy to enforce. As noted, in Germany the selling agen- cy, the Treuhandanstalt, places employment maintenance clauses in the sales contracts with stiff monetary penalties for noncompliance. But these are hard to enforce because the monitoring party-the Treuhand itself-is working its way out of existence as fast as possible, and the threat of repossession lacks credibility. Labor opposition to privatization is common, understandable, and some- times intense-but it has been muted where employees understood that the al- ternative was liquidation. Campaigns to explain the costs and benefits of privatization were critical in Japan, New Zealand, Tunisia, and Venezuela- indeed, in just about every country where divestiture has succeeded. Attractive severance packages have weakened opposition and created a social safety net. In Tunisia, generous severance packages, beyond those provided by law, in- duced so many voluntary departures (90 percent of all redundancies) that there was little need for outright dismissals. Employee ownership schemes are useful too, although employee ownership at too high a proportion can lead to diffi- culties in employment and wage restructuring and make it hard to attract buy- ers. In Chile it has been argued that workers' shares helped raise the price paid for SOEs because the buyers regarded the workers' stake as reducing the risks of renationalization (see Luders 1990). The governments of Argentina, Chile, 262 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Jamaica, Nigeria, Pakistan, Poland, Sri Lanka, and Venezuela, among others, reserved 5 to 20 percent of shares for employees at reduced prices and on easy credit terms. The benefits of the schemes were thought to outweigh the costs in lost revenue, which were usually low in the first place. Some argue that government will get a better price for SOEs if they are phys- ically rehabilitated before sale. But there are many reasons why most large new investments for enterprises should be left to new private owners once a decision has been made to privatize the firm. Getting the private sector to finance and manage investments and take the risk is a major reason for privatizing in the first place. There is little evidence that governments recover the costs of phys- ical restructuring in the form of higher sales prices. And governments have found that restructuring can delay the privatization process. An exception to the rule might be long-planned and carefully vetted expansions, especially for well-run SOEs, that would be costly to delay. For example, Chile systematically planned new electricity generation projects to come on stream every few years. It would have been wasteful to delay these projects while privatizing the elec- tricity company, and Chile did not do so. Pricing and Valuation The best way to determine the sale price is to let the market decide through competitive bidding, particularly for small and medium-size firms-as has been illustrated in Chile, the former East Germany, Mexico, and Tunisia. Even for larger enterprises, market-based pricing is preferable to technical methods, such as net asset value, net present value of discounted cash flow earnings trends (price-earnings ratios), dividend yields, or a combination of these meth- ods. Bidders, however, must be carefully prequalified and the regulatory envi- ronment must provide incentives for modernization. Valuation is difficult in the best of circumstances but doubly so in develop- ing countries where macroeconomic changes are taking place; where assets have not been traded before; and where information is weak, comparables few, and the market thin. An independent valuation helps set a floor price and as- sure a fair process and thus provides political cover, but technical methods sel- dom determine the market price for an enterprise. Overvaluation and unrealistic price expectations can delay the process. Many governments base asking prices on historical book value-on the grounds that they wish to re- cover at least what they put in-but this has often led to valuations of eroded assets that bear little resemblance to what any buyer will offer. Conversely, if book value has not been adjusted for inflation, it may grossly underestimate market value. In the sale of the ALUNASA aluminum mill in Costa Rica, for example, the company was priced on the basis of book value at $52 million, despite persistent heavy losses. There were no takers at this price. The govern- ment then used a comparable mill in Venezuela, valued at about $8 million, as Sunita Kikeri, John Nellis, and Mary Shirley 263 a reference price. ALUNASA was finally sold for $4 million, about 7.5 percent of book value. The rational economic solution for firms persistently showing losses or in a state of negative net worth is to accept any positive price offered, to give them away, or even to induce someone to take them over. Economically, it makes sense that "whenever social welfare is higher under private operations than un- der public operation .. . government should be willing to pay the private sector to take over the enterprise. This might happen, for instance, if the enterprise is loss-making under government operation but becomes viable under private operation without large deleterious welfare effects on consumers or work- ers"(Jones, Tandon, and Vogelsang 1991: 17). Although it is not common prac- tice, the Treuhand has sold dozens of firms at the symbolic price of one deutsche mark. In these cases, according to Treuhand officials, "we are not selling companies; we are buying management and technology." Overpricing shares in a public offering is also a recipe for failure. In two Turkish offerings, where shares appear to have been overpriced, investors lost a total of LT450 billion (May 1991 prices) and grew wary of participating in future public offerings. In the end the Turkish government repurchased the shares to maintain the price and then resold them at later dates in smaller tranches. This was costly and set a bad precedent for the remaining public of- ferings. To avoid such problems, discounts on privatization sales in general have been much higher than the traditional after-market premium of 10 to 15 percent in other flotations (table 4). Gains to small investors should be viewed as a mea- sure of success rather than as a financial loss to governments, because in such sales distributing ownership is more important than raising revenues. Prices also have to be low enough to foster demand and ensure a full subscription. To offset potential political and financial costs, some countries sell shares in tranches. France and the United Kingdom typically started with smaller share offerings and higher discounts; as commitment was demonstrated and the confidence of the private sector increased, larger percentages were offered and discounts declined. "Clawback clauses," used in the sale of the twelve re- gional electricity distribution companies in the United Kingdom, entitle the government to a share in any gains that an enterprise might make through sub- sequent sales of land, buildings, or other property that might be undervalued in the original sale of the enterprise. Argentina, Malaysia, and Mexico are examples of developing countries that have sold in tranches. A case in point is the privatization of Mexico's TELMEX (telecommunications) where the Mexican government restructured its capital so that only 40 percent of the stock had voting rights and then sold controlling shares in December 1990 by selling 20.4 percent of the company (that is, 51 percent of the voting stock) for $1.7 billion; in May 1991 a further 16.5 per- cent was sold for about $2.4 billion, and in May 1992 another 4.7 percent was sold for $1.4 billion. The implicit market value of the company jumped by 78 percent in five months after the first sale and more than doubled in the year 264 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Table 4. Prices for Selected Public Share Offers, 1979-87 Subscription First day price closing price Premium Date of (local (local or discount Country Enterprise offer currency) currency) (percent) France Compagnie Generale 5/87 FF290 FF323 11.4 d'Electricite Paribus 1/87 FF405 FF480 18.5 Saint-Gobain 11/86 FF310 FF369 10.0 Sogenal 3/87 FF125 FF225 80.0 Jamaica National Commercial Bank 12/86 J$2.95 J$4.94 67.5 Caribbean Cement Co. 6/87 J$2.00 J$1.55 -22.5 Philippines Philippine National Bank n.a. -P170 P225 50.0 United Kingdom British Airways 2/87 125p 169p 35.2 British Gas 12/86 135p 148p 9.3 British Petroleum 11/79 363p 367p 1.1 British Telecom 12/84 130p 173p 33.1 Rolls Royce 5/87 170p 232p 36.3 n.a. Not available. p. Pence. Source: For France: World Bank data; for Jamaica: Leeds (1987); for the Philippines: World Bank data; for the United Kingdom: Vickers and Yarrow (1988). after the second sale; by selling in tranches, the government was able to reap part of these gains (Galal and others forthcoming). Another tactic for winning political support without too large a fiscal sacri- fice is to offer discounts to small investors and to ask institutional investors to pay higher prices, either fixed or by tender. Tender methods are more appro- priate for sales of shares to well-informed financial institutions and trade buy- ers than to small investors; the former are better able to assess bid strategy and price the investment opportunity. The tender method was extensively applied in the public offerings in the United Kingdom and to all twenty-eight privati- zations in Sri Lanka. Financing Government decisions often compound financing constraints brought about by weak financial systems. A surprising number of governments have put SOEs on the market at the same time that they have crowded out their SOE shares by issuing high-yield, low-risk, tax-free government bonds. Some governments have further narrowed the market by excluding the participation of foreigners or minority groups. For political and social reasons, governments everywhere are reluctant to cede control over assets, especially those deemed strategic, to foreigners. In the Ssnita Kikeri, John Nellis, and Mary Shirley 265 past many states restricted external participation in privatization, thus narrow- ing the range of financing options (especially for large SOEs) and blocking an important source of new capital, markets, management, and technology. Many countries, such as Chile, India, and Mexico, recently have begun to relax such restrictions and instead set out to attract foreign investors by creating a stable economic and regulatory environment and guaranteeing convertibility and profit repatriation. Political sensitivities can be allayed by reserving a "golden share" for gov- ernment-that is, stipulating, in a general privatization law or in a particular sales agreement, that government retain one nonvoting special share that gives it the power to reject subsequent sale or major capital or physical restructuring of the firm. New Zealand used a variant of this arrangement in some of its privatizations, and the device is being adopted in developing countries (Argentina) and ex-socialist countries (Russia). Another device is to combine the sale of a controlling interest to a foreign investor with widespread distri- bution of the remaining shares to citizens and employees. In Indonesia, New Zealand, and Togo, for instance, SOEs have been sold to foreigners with the stipulation that a certain amount of shares gradually be floated to small inves- tors through the stock market. Foreign direct investment has been increasing since the mid-1980s. In forty developing countries it rose from 0.44 percent of GDP in 1984 to 1.15 percent in 1990. Nonetheless, in 1990 it accounted for only 10 percent of all private investment in those countries (Pfefferman and Madarassy 1992). This implies that the bulk of SOEs will have to be sold to domestic investors. Yet, govern- ments have sometimes limited the domestic market for privatized enterprises by excluding (or favoring) certain ethnic groups. In East and South Asia commer- cially oriented and relatively wealthy groups of minorities are excluded, while in East Africa citizens of Asian origin are hindered or excluded. In Kenya, where privatization has been debated since the early 1980s, few sales have taken place, in part because of the sensitivities involved in dividing national assets between African citizens and those of Asian origin (the likely buyers of SOEs in any unrestricted sale). Such restrictions lead to costly delays and limit the entry of groups with the necessary capital, skills, and experience to provide jobs and opportunities for the majority. Mechanisms-such as the reservation or subse- quent sale of a portion of shares for certain groups-need to be developed to mitigate political concerns and safeguard the interests of the majority, while at the same time tapping the expertise and resources of the minorities. Regulating weak financial systems is clearly a legitimate function of govern- ment, but it should not preclude institutional investors from performing a cat- alytic function in financing privatization. In turn, privatization can help strengthen and diversify financial markets. After Chile encountered problems created by poor banking practices during its first sales, it was reluctant to allow banks or even private pension funds to invest in SOE shares. The government solved the problem by creating a special commission to classify the risk of these 266 The World Bank Research Observer, vol. 9, no. 2 (July 1994) investments-very conservatively-and by limiting the amount of high-risk shares pension funds could hold. Sales for cash are better than accepting debt, even if this means selling at a lower price. As the experiences of Mexico and Venezuela show, outright sale cleanly severs the link between enterprise and state. Cash sales also provide the liquidity to pay enterprise liabilities and severance pay. Nevertheless, many countries have resorted to government-financed sales for debt, because finan- cial systems are not deep enough, SOEs are not sufficiently attractive, and pre- ferred buyers do not have enough cash. Selling on the installment plan is one option, but lowering the price, selling in tranches, or even giving small assets away may be better ways to ease financing constraints than lavish use of debt. Highly leveraged sales, regardless of whether the seller (government) or the banks are the source of credit, are risky. In Chile many privatized firms failed between 1974 and 1984, partly because of the large debts owed to government. The initial terms were attractive. Buyers had to pay 10 to 20 percent down, with one year's grace on further payments. After that, however, they faced a short (five to seven years) repayment period, at a real interest rate of 8 to 12 percent. The firms had a very thin equity cushion when the recession hit in the early 1980s; seven of every ten privatized companies went into bankruptcy and reverted to state hands when their controlling banks were nationalized. In the second round of sales, the chastened government gave no credit (except to the smallest investors and employees), and bidders had to prove their solvency (Hachette and Luders 1988). Debt-equity swaps can ease financing constraints and help improve a coun- try's investment climate. In a swap the debtholder who wants to buy the en- terprise exchanges debt-worth a fraction of its face value in the secondary market-for equity. Usually, the debt trades at a rate better than the secondary market price but still well below the face value. Swaps have been a way for heavily indebted countries such as Argentina, Chile, and the Philippines to bring foreign investors, including commercial banks, into transactions that might not have been concluded without their participation (Sung 1991). In Argentina swaps in privatizations reduced the face value of outstanding com- mercial bank debt by 20 percent. Critics of swaps argue with some justification that governments may be bet- ter off selling the enterprise and using the proceeds of the sale to repay or re- purchase the debt on the secondary market, as Mexico has done. That way they might capture more of the discount and expand the participation of local investors. This approach is appealing, but it has its limits: while Mexico's par- ticipation in the Brady plan put it in a good position to attract investors, a large debt overhang may deter foreign or domestic investors from buying SOEs, particularly large companies that require new investment. Thus, in heavily in- debted countries, debt-equity swaps can be useful-although paradoxically, the more successful debt-equity swaps are, the more their usefulness declines. As a country buys its debt back, it becomes a better credit risk; the secondary Sunita Kikeri, John Nellis, and Mary Shirley 267 market discount on the debt drops, and this reduces the incentive for the in- vestor. (This was one reason why the volume of Chile's swaps dropped from $1.9 billion in 1989 to $0.7 billion in 1990.) Swapping public debt for private equity raises issues, such as the effect on inflation and government borrowing requirements, that go beyond the scope of this article. Managing Privatization Transparency, essential in every privatization, requires competitive bidding procedures, clear selection criteria for evaluating bids, disclosure of purchase price and buyer, well-defined institutional responsibilities, and adequate mon- itoring of the program. A lack of transparency can lead to suspicions of unfair dealing (founded or unfounded) and to a popular outcry that can threaten not only privatization but reform in general. But zeal for transparency should not become an excuse for inaction. For small competitive firms, all that is normally needed is light supervision and re- view of transactions; Mexico and the former East Germany, for example, di- vested hundreds of enterprises in this way. Small business units can be divested even more quickly, as the experience of the former Czechoslovakia, Germany, Hungary, Poland, and Russia shows. But the larger and more visible the trans- action and the less competitive the market for the enterprise's goods or servic- es, the greater the importance of transparency. Both transparency and speed are served by centralizing policy responsibili- ties for privatization in a strong unit created for the purpose rather than in cabinet commissions and sector ministries. The latter tend to create delays, be- cause of vested interests, and make the process less transparent. The privatiza- tion authority should answer to the top of the political hierarchy, have a clear mandate and authority, and have a small but highly competent staff. In the second privatization phase in Mexico, a unit of seven people in the Ministry of Finance, reporting directly to an interministerial commission of key minis- ters and freed of public sector rules and regulations, divested hundreds of en- terprises within a few years. In the Philippines the Asset Privatization Trust, headed by a qualified private businessman and staffed by a small group of ex- perienced private individuals paid at private sector rates, disposed of more than 150 nonperforming assets in two years. Although decisionmaking is best centralized, the process can be accelerated and the workload of the unit in charge reduced by decentralizing implementa- tion. Specific tasks can be delegated to banks and financial institutions (as was done in France, Mexico, and Nigeria), to international and local business con- sultancies (as in Argentina and Venezuela), to holding companies (as in Egypt and the Philippines), and to SOE managers themselves (as in the former Czechoslovakia, Hungary, Russia, Tunisia, and Turkey). The privatization au- thority must supervise these implementing agencies and provide a clear time- table for privatization; otherwise, the risk of inaction or abuse is great. Clear 268 The World Bank Research Ohserver, vol. 9, no. 2 (July 1994) standards of accountability are also necessary to minimize fraud and ensure transparency. Government capacity to handle the privatization process is scarce; time and money have to be spent obtaining the right technical, financial, and legal skills. Small privatizations can and should be handled locally to the extent possible (as in Mexico), but skills may need to be imported for large transactions and in the least developed countries, where institutional weakness has substantially delayed privatization. The lesson that ownership matters has been heeded: increasingly, govern- ments worldwide are moving to tap private management and finance and to privatize. The evidence indicates that-as long as it is done right-privatiza- tion can and does bring efficiency and needed innovation. The lessons drawn from the experience with privatization offer some guidance on how to realize the potential while minimizing the costs. Notes Sunita Kikeri and John Nellis are with the Private Sector Development Department and Mary Shirley with the Policy Research Department at the World Bank. 1. This is a conservative estimate of the number-difficult to calculate-of SOEs privatized as of mid-1993. The net reduction in SOEs worldwide is even harder to determine. Some states have been creating SOEs, while others have been divesting (Burundi, for example); and there are cases (Ghana, for example) where privatized firms ran into difficulties and have found their way back into the state's portfolio for a perhaps temporary, perhaps indefinite stay. 2. Lobbying by the private sector would persist, of course. Moreover, it must be admitted that some firms in every economy seem "too big to fail" and that in some countries the public-private distinction is quite blurred and a policy of "first privatize the private sector" should be followed. Nonetheless, the sentence footnoted is more often correct than not. 3. Under other circumstances there may be reasons why the private owner may not invest in a privatized utility, as cited by Sappington and Stiglitz (1987). See also Levy and Spiller (1994) on problems of commitment and private investment in utilities. 4. These numbers would be much higher if they included the very large number of completed "small" privatizations of shops, microenterprises, and kiosks in the retail and services sectors. As of mid-1993 an estimated 80,000 such firms had been privatized in Poland alone, 7,000 in Czechoslovakia, 1,100 in Hungary, 13,000 in the former East Germany, and 200,000 in Russia. The numbers also predate the major privatizations in Argentina. Also excluded are the mass restitutions of property to former owners in Chile and the privatization of small enterprises (mostly jute mills) in Bangladesh. 5. Unless otherwise designated, dollar figures represent U.S. dollars throughout. 6. Share ownership reconcentrates despite the mechanisms used to attract and retain small shareholders (such as bonuses or matching shares, pricing discounts, and reduced taxes on div- idends). In the United Kingdom ownership quickly reconcentrated. Similarly, at the time of the public issue of the Malaysian International Shipping Corporation, there were approximately 60,000 shareholders; the number fell to fewer than 5,000 after a brief round of secondary trading. Similar patterns have emerged in other countries divesting through their stock markets (see Adam and Cavendish 1990). Sunita Kikeri, John Nellis, and Mary Shirley 269 References The word "processed"describes informally reproduced works that may not be commonly available through library systems. Adam, C. S., and W. P. Cavendish. 1990. "Can Privatization Succeed? Economic Structure and Programme Design in Eight Commonwealth Countries." Oxford University. Processed. Alexander, Myrna, and Carlos Corti. 1993. "Argentina's Privatization Program: Experience, Is- sues, and Lessons." CFS Discussion Paper 103. World Bank, Cofinancing and Financial Advi- sory Services Department, Washington, D.C. Processed. Ambrose, William W., Paul R. Hennemeyer, and Jean-Paul Chapon. 1990. Privatizing Telecom- munications Systems: Business Opportunities in Developing Countries. IFC Discussion Paper 10. Washington, D.C.: World Bank. Aspe, Pedro. 1991. "Thoughts on the Structural Transformation of Mexico: The Case of Priva- tization of Public Sector Enterprises." Address to the Los Angeles World Affairs Council, Los Angeles, Calif. Processed. Black, Bernard S. 1992. "The Value of Institutional Investor Monitoring: The Empirical Evi- dence." UCLA Law Review 39(4):895. Borcherding, T. E., W. W. Pommerehne, and F. Scheider. 1982. "Comparing the Efficiency of Private and Public Production: The Evidence from Five Countries."Zeitschrift fur Nation- alokonomie, Supplement 2:127-56 Bouin, O., and C. A. Michalet. 1991. Rebalancing the Public and Private Sectors: Developing Country Experience. Paris: OECD. Corbo, Vittorio. 1993. "Economic Reforms in Chile: An Overview." Paper presented at Eastern Economic Association Meeting, Washington, D.C. Processed. Domberger, Simon, and John Piggott. 1986. "Privatization Policies and Public Enterprise: A Sur- vey." Economic Record (Australia) 62(June):145-62. Galal, Ahmed. 1991. Public Enterprise Reform: Lessons from the Past and Issues for the Future. World Bank Discussion Paper 119. Washington, D.C. . Forthcoming. "Divesting Electricity Distribution Companies: An Ex-Post Analysis of Chilectra Metropolitana, Chile." World Bank, Policy Research Department, Washington, D.C. Processed. Galal, Ahmed, Leroy Jones, Pankaj Tandon, and Ingo Vogelsang. Forthcoming. Welfare Conse- quences of Selling Public Enterprises: Case Studies from Chile, Malaysia, Mexico, and the Unit- ed Kingdom. New York: Oxford University Press. Hachette, Dominique, and Rolf Luders. 1988. "Privatization in Argentina and Chile: Lessons from a Comparison." Latin America and the Caribbean Region Discussion Paper 18. World Bank, Washington, D.C. Processed. Hegstad, Sven Olaf, and Ian Newport. 1987. Management Contracts: Main Features and Design Issues. World Bank Technical Paper 65. Washington, D.C. Hemming, Richard, and Ali M. Mansoor. 1988. Privatization and Public Enterprises. International Monetary Fund Occasional Paper 56. Washington, D.C. Im, J. Soo, Robert Jalali, and Jamal Saghir. 1993. "Privatization in the Republics of the Former Soviet Union: Framework and Initial Results." World Bank, IFC and Legal Department, Wash- ington, D.C. Processed. Jayasinghe, Tissa. 1993. Personal communication. Government of Sri Lanka, Ministry of Fi- nance, Colombo. Jones, Leroy P., Pankaj Tandon, and Ingo Vogelsang. 1991. "Selling Public Enterprises: A Cost- Benefit Methodology."In Ravi Ramamurti and Raymond Vernon, eds., Privatization and Con- trol of State-Owned Enterprises. EDI Development Studies. Washington, D.C.: World Bank. Leeds, Roger S. 1987. Privatization in Jamaica: Two Case Studies. Cambridge, Mass.: Harvard University, Center for Business and Government. 270 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Levy, Brian, and Pablo T. Spiller. 1994. "Regulation, Institutions, and Commitment in Telecom- munications: A Comparative Analysis of Five Country Studies." In Proceedings of the World Bank Annual Conference on Development Economics 1993. Washington, D.C. Lorch, Klaus. 1988. The Privatization Transaction and Its Longer-Term Effects: A Case Study of the Textile Industry in Bangladesh. Cambridge, Mass.: Harvard University, Center for Business and Government. Luders, Rolf J. 1990. "Chile's Massive SOE Divestiture Program: 1975-1990." Paper presented at the Conference on Privatization and Ownership Changes in East and Central Europe, World Bank, Washington, D.C. Processed. McKinnon, Ronald I. 1994. "Gradual versus Rapid Liberalization in Socialist Economies: The Problem of Macroeconomic Control." In Proceedings of the World Bank Annual Conference on Development Economics 1993. Washington, D.C. Megginson, William L., Robert C. Nash, and Matthias van Randenborgh. 1992. "Efficiency Gains from Privatization: An International Empirical Analysis." University of Georgia, De- partment of Banking and Finance, Athens, Ga. Processed. Millward, Robert. 1988. "Measured Sources of Inefficiency in the Performance of Public and Private Enterprises in LDCs." In Paul Cook and Colin Kirkpatrick, eds., Privatisation in Less Developed Countries. Sussex, U.K.: Wheatsheaf Books. Nellis, John R. 1986. Public Enterprises in Sub-Saharan Africa. World Bank Discussion Paper 1. Washington, D.C.: World Bank. . 1989. "Contract Plans and Public Enterprise Performance." Discussion Paper 48. World Bank, Country Economics Department, Washington, D.C. Processed. Perez Escamilla, Juan Ricardo Costas. 1988. "La privatizaci6n en Mexico 1982-1988. Efectos de la Venta de Paraestatales." ETAM. Mexico City. Processed. Pfefferman, Guy P., and Andrea Madarassy. 1992. Trends in Private Investment in Developing Countries, 1992 Edition. IFC Discussion Paper 14. Washington, D.C.: World Bank. Porter, Michael. 1992. Capital Choices: Changing the Way America Invests in Industry. Washing- ton, D.C.: Council on Competitiveness and Harvard Business School. Privatisation International Yearbook. Various years. London: Privatisation International Ltd. Sader, Frank. 1993. "Privatizations and Foreign Investment in the Developing World: 1988-1992." World Bank, International Economics Department, Washington, D.C. Processed. Saghir, Jamal. 1993. "Privatization in Tunisia." World Bank, IFC, CFS Discussion Paper 101. Washington, D.C. Processed. Sappington, David E. M., and Joseph E. Stiglitz. 1987. "Privatization, Information, and Incen- tives." NBER Working Paper 2196. National Bureau of Economic Research, Cambridge, Mass. Processed. Shirley, Mary, and John Nellis. 1991. Public Enterprise Reform: The Lessons of Experience. EDI Development Studies. Washington, D.C.: World Bank. Short, P. T. 1984. "The Role of Public Enterprises: An International Statistical Comparison." In Robert H. Floyd, Clive S. Gray, and R. P. Short, eds., Public Enterprise in Mixed Economies: Some Macroeconomic Aspects. Washington, D.C.: International Monetary Fund. Sung, Woonki. 1991. "Role of Debt Equity Swaps in Privatization." World Bank, Cofinancing Advisory Services Department, Washington, D.C. Processed. Suzuki, Hiroaki. 1991. "Lessons Learned from the Guinea Privatization Program." World Bank, Africa Technical Department, Washington, D.C. Processed. Svejnar, Jan, and M. Hariga. Forthcoming. "Private Ownership, Export Orientation and Enter- prise Productivity in a Developing Economy: Evidence from Tunisia." Journal of Development Economics. Takano, Yoshiro. 1992. "Nippon Telegraph and Telephone Privatization Study: Experience of Japan and Lessons for Developing Countries." World Bank, Cofinancing Financial Adivsory Services Department, Washington, D.C. Processed. Sunita Kikeri, John Nellis, and Mary Shirley 271 Triche, Thelma A. 1990. "Private Participation in the Delivery of Guinea's Water Supply Servic- es." WPs 477. World Bank, Infrastructure and Urban Development Department, Washington, D.C. Processed. Vickers, John, and George Yarrow. 1988. Privatization: An Economic Analysis. Cambridge, Mass.: MIT Press. Wellenius, Bjorn, Peter A. Stern, Timothy E. Nulty, and Richard D. Stern, eds. 1989. Restruc- turing and Managing the Telecommunications Sector. A World Bank Symposium. Washington, D.C.: World Bank. World Bank. 1983. World Development Report 1983. New York: Oxford University Press. . 1991. Developing the Private Sector: The World Bank's Experience and Approach. Wash- ington, D.C. - . 1993. The Bank's Role in the Electric Power Sector: Policies for Effective Institutional, Regulatory, and Financial Reform. A World Bank Policy Paper. Washington, D.C. Yarrow, George K. 1988. "Privatization in Theory and Practice." Economic Policy 2:324-77. Yusuf, Shahid, and Xiaofeng Hua. 1992. "China's State Sector." World Bank, East Asia and Pa- cific Country Department, Washington, D.C. Processed. 272 The World Bank Research Observer, vol. 9, no. 2 (July 1994) THE COSTS AND BENEFITS OF SOIL CONSERVATION: THE FARMERS' VIEWPOINT Ernst Lutz Stefano Pagiola Carlos Reiche Most countries in Central America and the Caribbean depend heavily on agri- culture; efforts to sustain and improve the sector's productivity are therefore cru- cial to the region's economic development and to the welfare of its people. Land degradation is thought to pose a severe threat to the sustainability of agricultural production. Yet despite long-standing concern about this threat and dramatic claims of environmental damage, surprisingly little empirical analysis has been done on the causes and severity of land degradation problems in the region and on how best to tackle them. Meanwhile, many of the conservation programs de- signed to address the problems have fallen short of expectations. Often farmers have not adopted the recommended conservation practices or have abandoned them once the project ended. The research presented in this article attempts to bridge the empirical gap, us- ing cost-benefit analysis to investigate the nature and severity of the soil degra- dation problem and to assess the cost-effectiveness of proposed solutions. Because soil degradation problems tend to be site-specific, the analysis is rooted in case studies, and because conservation programs stand or fall on the partici- pation of farmers, the study's main focus is on the profitability of the measures and the deterrents to their adoption from the farmers' point of view. oil degradation can be defined as a reduction in the land's actual or potential uses (Blaikie and Brookfield 1987). Many cultivation practices tend to degrade soil over time. For example, cultivation can expose soil to water and wind erosion, repeated tillage can weaken soil structure, crop produc- tion can remove nutrients, and use of machinery can compact the soil. Central America's mountains and heavy rainfall make much of the region particularly The World Bank Research Observer, vol. 9, no. 2 (July 1994), pp. 273-95 © 1994 The International Bank for Reconstruction and Development/THE WORLD BANK 273 vulnerable to degradation-a problem exacerbated by population pressures that have opened to farming new areas only marginally suited to agriculture. Soil degradation, in turn, affects productivity. As soil is degraded, crop yields decline or the levels of inputs (and hence costs) needed to restore productivity rise. Despite long-standing concern about these problems, surprisingly little hard evidence exists on their magnitude. The degradation figures quoted in the lit- erature are often extrapolated from very limited data and may exaggerate the problem because they often consider "moved soil" as "lost soil," even though much of it may have been deposited on other agricultural land. For instance, in a recent assessment of the extent of human-induced soil degradation, the International Soil References and Information Centre (ISRIC) estimated that 56 percent of the land in Central America had experienced moderate degradation, implying that productivity has been substantially reduced, and that 41 percent had experienced strong degradation, implying that agricultural use has become impossible (Oldeman, Hakkeling, and Sombroek 1990). Aggregate measures such as these, however, often have a weak empirical basis. Few studies have directly measured erosion rates and the factors that influence them, and these studies have generally been scattered and unsystematic. Even less effort has been devoted to studying other forms of land degradation, such as depletion of nutrients, damage to physical and chemical properties of soil, or reductions in its capacity to retain moisture. Table 1 presents the available estimates of erosion rates in Central American countries. The data were obtained in a variety of ways and are therefore not always strictly comparable, but they do give some idea of the great diversity of erosion rates present within the region. Predictions abound of catastrophic effects on agricultural productivity aris- ing from soil degradation. Evidence on the magnitude of these effects, by con- trast, is hard to find-in fact, in many cases claims of declines in productivity are made with no evidence at all (see Biot, Lambert, and Perkin 1992 for some African examples). Leonard (1987, p. 130), for example, simply asserts that a "pattern of extensive land use leading to soil loss or decline in fertility is ap- parent" in the Caribbean areas of Central America. Speaking of the highland areas, he points to "increasing reports of localized desertification in areas of western Honduras and Costa Rica." He also mentions that cotton yields are "reportedly declining" where severe erosion has been experienced. But nowhere does he provide any indication of the size or rate of fertility loss. More gener- ally, the assumption that fertility must be declining rapidly is usually left im- plicit from statements about high rates of erosion. But erosion rates, even where they are significant, may have very little effect on productivity under certain conditions. Erosion rates in the Tierra Blanca area in Costa Rica's Cartago Province, for example, are extremely high, but the effect on productivity is minor because soils in that region are deep (up to 1 meter in some places) and contain a high percentage of organic matter. More- over, the subsoil is itself productive, although less so than the topsoil. The 274 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Table 1. Empirical Evidence on Soil Erosion in Central America and the Caribbean Average annual rate Rainfall Slope Farming of erosion per hectare Country and area Source (millimeters) (percent) system Metric tons Millimeters Dominican Republic Taveras Hartshorn (1981) - - - 275 - North Central Altieri (1990) - 36 Various 24-69 - South West Veloz (1988) - 30 Various 2-1,254 - El Salvador Metapan Flores (1979) 1,895 - Corn 137 9 CTA (1956) 1,724 30 Corn, beans 230 15 Haiti Camp-Perrin CUNARD (1991) 2,000 30 Hedges 4-45 - Papaye Grosjean (1987) 1,214 25 Grasshedge 8 - Honduras Tatumbla, Morazan Welchez (1991) 2,000 45 Corn, beans 42 3 Tatumbla, MorazAn SAnchez (1991) 900-1,500 15-40 - 18-30 - Nicaragua Cristo Rey PCEO (1981) 1,700 30-40 Cotton 40 - Panama Cuenca del Canal Soto (1981) 1,200 35 Rice 153 - Cuenca del Canal Soto (1981) 1,200 35 Corn 137 - Cuenca del Canal Soto (1981) 1,200 35 Rice 118 - Cocle VAsquez (1991) 1,937 - Rice, corn, 340 17 cassava, beans Chiriqui Oster (1981) 1,500-2,800 - Pasture 35 5 Chiriqui Oster (1981) 1,500-2,800 - Coffee 77 11 Chiriqui Oster (1981) 1,500-2,800 - - 183 27 - Not available. Note: Figures rounded. Chiriqui region in Panama provides a similar example. Conversely, areas with shallow soils or unfavorable subsoils, such as the Turrubares area in Costa Rica, can be very sensitive to even limited rates of erosion. The same phenom- enon is true of other forms of soil degradation. The effect of nutrient loss on productivity, for example, depends on the initial stock of nutrients and on their rate of regeneration. Because of the different effects of soil degradation on productivity, a specific soil conservation technique-particularly an expensive one-may not be worth- Ernst Lutz, Stefano Pagiola, and Carlos Reiche 275 while from either a farmer's perspective or society's. Degradation can be slowed or arrested by a large range of methods, including cultural practices such as contour plowing and minimum tillage; vegetative practices such as grass strips, strip-cropping, and vegetative barriers; and mechanical measures such as ter- races and cutoff drains. Implementing any of these techniques can be costly, either directly in investment requirements or indirectly in forgone production, and some measures are better suited than others to specific conditions. The crit- ical question facing farmers-and society as a whole-is whether the benefits of a given conservation measure or set of measures are worth the costs. Conceptual Issues The problem of soil degradation and conservation can be examined from two perspectives: that of society as a whole and that of individual farmers. From the standpoint of society, all the costs and benefits of a given activity must be con- sidered. Agricultural production that leads to siltation of reservoirs, for exam- ple, represents a real cost to society that should be considered together with the value of the output obtained and any effects on fertility. In addition, to measure the true opportunity cost of the resources used in and obtained from agricultural production, their valuation should be adjusted for any distortions resulting from policy interventions or market failures. Farmers, however, are likely to consider only the costs and benefits that actually accrue to them from the decisions they make about how to use their resources. They value these costs and benefits at the prices they actually face, with no attempt to adjust for distortions. This article examines the returns to investment in conservation measures mainly from the farmers' point of view for two reasons. First, decisions about land use are ultimately made by the farmers themselves and not by social plan- ners or government agencies. Farmers decide how to use their land in light of their own objectives, production possibilities, and constraints, not on the basis of any theory of the social good. Understanding the incentives (and disincen- tives) individual farmers face is necessary, therefore, to understand patterns of resource use and to formulate appropriate responses to problems. Second, land use problems generally depend heavily on site-specific biophysical characteris- tics, which can vary significantly even within small areas (Pagiola 1993). Anal- ysis at the farm level is the most apt to incorporate site-specific effects. A farm-level approach also places the emphasis firmly on the effects of deg- radation on farm productivity. In developing countries, where substantial num- bers of people still depend directly on agricultural production, the effect of degradation on yields is often critical. This is not to belittle the importance, in some situations, of off-farm effects of soil degradation, such as siltation of res- ervoirs and waterways. But even where such off-farm effects are the primary concern, considering them first at the farm level is appropriate because that is where the conservation measures would have to be implemented.1 276 The World Bank Research Observer, vol. 9, no. 2 (July 1994) In making their land use decisions, farm households need to consider both the agroecological and the economic characteristics of the environment in which they operate. In addition, they often face numerous constraints, such as tenure and liquidity problems, the need to meet consumption requirements, and the need to compensate for missing or incomplete markets. Moreover, many farm decisions are made in the context of considerable risk and uncer- tainty about weather, pests, fluctuating market demand, and so on. A complete analysis of land use decisions, therefore, requires that one look at the issue in the context of overall decisionmaking of the household (Singh, Squire, and Strauss 1986; Reardon and Vosti 1992). The farm household's problem can be formulated as one of maximizing the utility of consumption over time, subject to a budget constraint imposed by its returns from agriculture over time and any returns from nonfarm activities, and subject to any other constraints it might face. Singh, Squire, and Strauss (1986) show that if markets exist for all goods and services, the problem of maximization is separable, in the sense that production decisions are made in- dependently of consumption decisions. Even when production decisions are not separable, however, they can be analyzed independently as long as the "prices"' of goods for which markets are missing are interpreted as shadow prices that reflect the farm household's perception of the severity of the constraints they face (De Janvry, Fafchamps, and Sadoulet 1990). The household's problem, then, can be summarized as one of maximizing the present value of the stream of expected net returns to agricultural produc- tion (Pagiola 1993). In practice, data are generally not available to estimate complex maximization models. But for empirical analysis, the model can be reformulated to fit a cost-benefit analysis framework. The household's choice can be viewed as selecting between two or more alternative cropping systems. For example, the choice might be between retaining the traditional cultivation system, in which conservation measures are limited to contour plowing, or re- placing it with a new system, which conserves more soil by using terraces or reduced-tillage techniques. Each system is characterized by distinct production functions and soil conservation functions, and each generates a different opti- mal path. From the household's perspective, the problem is whether returns under the optimal path of the new, more conserving system are sufficiently greater than returns under the optimal path of the current, more degrading sys- tem to justify the cost of switching. Basically, it would be in the farm household's financial interest to adopt the new system if the net present value of the incremental returns from switching were positive (NPV > 0). This formulation is equivalent to a standard cost- benefit analysis formulation and lends itself particularly well to empirical anal- ysis, because suitable data are often available. Observing practices in use al- lows time paths of yields and use of inputs to be constructed; these are then used to project costs and revenues over time. The method can also be used if the only data available are on total costs and revenues in each period, and it Ernst Lutz, Stefano Pagiola, and Carlos Reiche 277 also lends itself well to incorporating lumpy investments and other discontinu- ities in cropping practices (Walker 1982; Taylor and others 1986).2 The discussion so far has assumed that the only constraints on behavior are those imposed by the properties of the biophysical system. The NPV > 0 crite- rion is thus a necessary but not a sufficient criterion for adopting a new pro- duction system. Even if the NPV estimate is positive, other factors might prevent a household from adopting a new system. In principle, these other con- straints could be built into the optimization framework. The effect of tenure insecurity might be included, for example, by limiting the length of the time horizon. In practice, however, the profitability of a system is generally easier to compute if one first assumes that no constraints hold and then verifies whether any specific constraints are binding. The cost-benefit calculations themselves often provide insight into whether particular constraints are likely to prove binding. The length of time it takes for an investment to be repaid, for example, can indicate whether tenure issues are likely to pose problems. If the investment is repaid quickly, insecurity of tenure is unlikely to affect adop- tion. Of course, if adopting a new production system is unprofitable for the farm household, the question of whether other constraints might prevent its adoption does not arise. Methodology Cost-benefit analysis techniques provide a coherent framework for integrat- ing information on the biophysical and economic environments faced by farm- ers. Variants of these techniques have been used to examine soil conservation cases in the Dominican Republic (Veloz and others 1985), India (Magrath 1989), and Kenya (Pagiola 1992). Other simple techniques, such as calculating the value of lost nutrients (Repetto and Cruz 1991), can only roughly indicate the severity of the problem; they cannot provide guidance in selecting the best response. The basic principles of the analysis are straightforward. First, the effects of continued erosion (or other types of soil degradation) on productivity are es- timated for the time horizon of interest. These are then used to estimate returns at each point in time. Second, the calculations are repeated under the condi- tions that would be experienced if a specific conservation measure were adopt- ed. The returns to the investment in this measure are then obtained by taking the difference between the streams of discounted costs and benefits in the cases with and without conservation. This method estimates the returns to the spe- cific conservation measures being examined, not to conservation per se. A find- ing that certain conservation practices are not profitable does not mean that no conservation measure is profitable-often, numerous measures designed to reduce degradation rates are already being practiced, implying that farmers consider them profitable. 278 The World Bank Research Observer, vol. 9, no. 2 (July 1994) As was argued in the previous section, when the analysis is carried out at the farm level using prices actually faced by farmers, a positive NPV estimate for a given conservation measure can be interpreted as showing that adoption of that measure would profit the farmer. Farmers should, in principle, be will- ing to adopt the measure voluntarily. But, as with all cost-benefit analysis, oth- er, unexamined options might be preferable. The analysis can be repeated for each known option, and the most profitable among them found. For this article and the larger work from which it is drawn, the availability of data dictated the choice both of the sites studied (see table 2) and of the aspects of the problem analyzed: erosion and mechanical methods of conser- vation.3 Research efforts have mainly focused on problems arising from ero- sion, to the neglect of other forms of soil degradation, and most conservation projects in the past have tended to emphasize mechanical conservation struc- tures. Consequently the case studies do not present a comprehensive overview of soil conservation problems and practices in the region; they do, however, illustrate the wide diversity of conditions encountered, help explain farmers' behavior, and indicate appropriate policy responses. Our country studies, except for Haiti, were conducted by local practitioners; in most cases, teams were composed of economists, agronomists, and soil sci- entists from relevant government agencies. This collaborative and participatory approach to the research drew on local knowledge and expertise and also de- veloped local analytical capacity. Data on the nature and rate of degradation caused by current practices, on the effects of degradation on future productivity, and on the effects of conser- vation practices are very scarce. Various methods for estimating the required relationships were chosen, depending on the nature of the available data. Econometric techniques were sometimes used to estimate the effect on yield of certain observed conditions (such as the presence or absence of certain conser- vation measures). For our purposes, estimating a time trend of yields with and without a given conservation measure was usually sufficient, although disen- tangling the effect of soil degradation on productivity is very difficult (Capalbo and Antle 1988), and even the limited objective encountered problems such as bias in sample selection when nonconserved and conserved fields were com- pared. In addition, many of the case studies had to rely on farmer recall for data and were not able to control fully for other sources of variation in yield, such as weather. In other cases, simple models of the physical environment- such as the Universal Soil Loss Equation (USLE) and, in the Haiti study, the Soil Changes under Agroforestry (SCUAF) model-were employed, using a mixture of experimental and observational data. A modeling approach is more flexible because it allows parameter values to be drawn from a variety of data sources. But it requires detailed qualitative and quantitative knowledge of the biophys- ical environment; building and validating a complete and realistic model are complex endeavors. Even calibrating existing models is far from easy. Ernst Lutz, Stefano Pagiola, and Carlos Reiche 279 Table 2. Case Study Areas Country and area Biophysical environment Degradation problem Conservation measures proposed Costa Rica Barva area, Heredia Province Important coffee-producing region. Soil loss affects nutrients available Diversion ditches. Relatively deep soil, but vulnerable to to coffee. erosion because of topography. Tierra Blanca-San Juan Important vegetable-producing area. Because of deep soils, decline in Diversion ditches are Chicoa, Cartago Province Deep volcanic soils. yield is not significant, but erosion recommended, but interfere with washes away seed and fertilizer prevalent cultivation practices. and exposes rocks. Turrubares, Central Pacific Previously used for pasture, now Very high rates of erosion. Soils Diversion ditches or terraces. region converted to production of coco yam are thin and vulnerable to erosion. for export. Dominican Republic El Naranial subwatershed, Subsistence agriculture. Steep slopes, High rates of erosion. Diversion ditches at 10-meter Peravia Province soils of moderate natural fertility. intervals, live barriers, and contour cropping. Guatemala Patzite, Department of Quiche Small-farm area. Strongly undulating Heavily affected by soil erosion. Terraces with a protected topography; soils of medium depth embankment. and fertility. Haiti Maissade watershed, Hilly area. Generally less degraded and Erosion. Ramp pay (indigenous technique in Central Plateau region more productive than most other hilly which crop residue is placed along regions of Haiti. the contour and held in place by stakes); hedgerows along the contour; and contour rock walls. Honduras Tatumbla, Department of Subsistence agriculture predominant. Susceptible to water erosion, Diversion ditches with live Francisco Morazan Thin topsoil, low in organic material especially in the high areas. barriers. and many nutrients. Yorito, Department of Yoro Subsistence agriculture, still largely Cleared plots vulnerable to Diversion ditches with live forested. Shallow, easily erodible soils, erosion. barriers. of medium to low natural fertility. Nicaragua Santa Lucia valley, watershed Subtropical foothills. Moderately deep High risk of erosion due to steep Manually constructed diversion of Malacatoya River soils. One of the most productive areas slopes, scarce vegetation cover, ditches with stone barriers. in the country. and intense precipitation. Deforestation on upper slopes. Ianana Coc& Province Subsistence agriculture using slash-and- Rapid decline in yield on cleared Combination of erosion burn techniques, with plots cultivated plots; deforestation. prevention measures (planting on one year in every five. Shallow soils, the contour, live and dead generally low in organic matter and barriers, diversion ditches) and nutrients, on steep slopes. improved cultivation practices. Source: Case studies in Lutz, Pagiola, and Reiche (1994). Obtaining the required economic data was generally less problematic. Crop production budgets, used to estimate returns, were the main requirement and were generally widely available, although rarely at the degree of disaggregation needed. Fortunately, preliminary budgets built from available secondary data were easy to confirm, supplement, and correct during fieldwork. The most im- portant task was to ensure that the crop production budgets accurately reflect- ed practices and prices in the area. Inputs provided by the households themselves, such as family labor, were priced at their cost in the nearest mar- ket. Output and input prices used in the analysis were meant to represent long- run real price trends. Assessing the discount rate is crucial, given the intertem- poral nature of the problem, but beset by controversy. Here, because the anal- ysis examines the profitability of conservation from the farm household's viewpoint, the appropriate discount rate to use should be the farmers' cost of borrowing or their rate of time preference. However, little empirical evidence exists on either (Pender 1992). Therefore, and to facilitate comparability of re- sults across study sites, a common real discount rate of 20 percent was used in each case study. In addition, the internal rate of return (IRR) was computed in each case. If the appropriate discount rate, assuming it were known, is smaller than the IRR, the proposed conservation measures would be profitable. Effects of Degradation on Productivity The estimated losses in productivity vary considerably across the case stud- ies; table 3 presents findings for some of the crops analyzed. In several cases, the data point to rapid rates of decline in yield. In the Maissade watershed of Haiti, for example, yields of corn and sorghum were estimated to decline by as much as 60 percent during a decade. In the Tatumbla region in Honduras, corn yields declined almost 50 percent in ten years if no conservation measures were used. Elsewhere, estimated declines were minor. Coffee yields in the Bar- va region of Costa Rica, for example, were estimated to decline by slightly more than 10 percent in ten years, and there is reason to believe that the actual rate is lower. In Costa Rica's Tierra Blanca region, declines in potato yield caused by erosion were easily countered by small increases in fertilizer use; in- deed, potato production has been steadily increasing despite high rates of ero- sion. The effects of degradation can also vary significantly across crops, even in the same area, as shown by the data from El Naranjal in the Dominican Republic. If no conservation measures were adopted, returns to agricultural produc- tion would gradually decline in each of the cases studied. Eventually, produc- tion would become uneconomic and cease although exactly when would vary, depending on the rate of decline in yield, the cost of production, and the price of the output. (Because farmers are likely to adjust their production prac- tices as yields decline, the time before production becomes unprofitable is likely 282 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Table 3. Estimates of the Impact of Soil Degradation on Productivity of Selected Crops in the Case Study Areas Projected Production, as percentage of initial yield, after shutdown Country and area Crop 10 years 20 years 30 years 40 years 50 years year Costa Rica Barva Coffee 89 78 67 56 46 20 Turrubares Coco yam 0 0 0 0 0 4 Dominican Republic El Naranjal Pigeon peas 58 16 0 0 0 Peanuts 100 100 100 100 100 16a Beans 77 53 30 0 0 Guatemala Patzite Corn 0 0 0 0 0 job Haiti Maissade Corn, sorghum 41 22 10 1 0 25 Honduras Tatumbla Corn 53 39 39 39 39 8 Yorito Corn 82 65 47 41 41 11 Note: Projected year for production shutdown is in the absence of conservation measures. a. The sixteen-year shutdown period applies to the pigeon peas-beans-peanuts intercrop system. Because the three crops are cultivated together, peanut cultivation is assumed to cease when the yields of the other crops make production uneconomic. b. Corn can be produced in years one through nine, but decline in yield is so rapid that it reaches zero in year ten. Source: Case studies in Lutz, Pagiola, and Reiche (1994). to be overestimated.) The very high rates of decline experienced in Turrubares mean that the production of coco yam would shut down in four years if no conservation measures were adopted. By contrast, in Tierra Blanca the produc- tion of potatoes would remain profitable more or less indefinitely even without conservation measures. Not all the damage caused by soil degradation takes the form of losses in yield. In Tierra Blanca, for example, the effects of degradation are reflected primarily in higher costs arising from the need to apply higher rates of fertil- izer, from the lower efficiency of fertilizer (because some washes away), and from the labor required to remove stones that accumulate on fields as soil erodes. In Panama's Cocle Province, agricultural production on a given plot can be sustained only for a very short time if no conservation measures are used. There, the costs of degradation are reflected primarily in the need to clear new plots of land. These examples, together with the diversity of effects on yield, reinforce the need for site-specific information to understand degradation problems and Ernst Lutz, Stefano Pagiola, and Carlos Reiche 283 devise effective ways of helping farmers respond to them. These case studies, however, are by no means a random sample of degradation conditions in the region; they are drawn from sites for which data were available and therefore primarily from areas where degradation problems were serious enough to war- rant data collection. Consequently, they probably represent high-case scenarios on the degree and rate of degradation in the region. The estimated effects on yields of conservation practices were likewise var- ied. In some instances, yields were expected to recover once conservation mea- sures were established-partly because soil regenerates after the processes of degradation are halted, partly because fertilizers are used more efficiently, and partly because improved cultivation practices are sometimes introduced togeth- er with conservation. In the Tatumbla area of Honduras, for example, if diver- sion ditches are built and improved planting practices are adopted, annual corn yields are estimated to increase about 145 kilograms per hectare. Elsewhere, conservation measures might slow but not halt the decline in yield. In the Tur- rubares area of Costa Rica, for example, it was estimated that diversion ditches would halve the rate of decline in yield; the much more expensive terraces, on the other hand, were expected to reduce the rate of decline by 90 percent. Again, the diversity of conditions is evident. As well as reducing soil loss and hence the rate of decline in yield, conser- vation measures can affect yields by encouraging the retention of moisture and stimulating improvements in the soil's physical structure (English, Tiffen, and Mortimore 1994; Shaxson and others 1989). In Haiti's Maissade area, land treated with diversion ditches and other conservation structures was found to produce an average of 51 percent more corn and 28 percent more sorghum than did plots without conservation structures in 1988 (a year of poorly timed rainfall) and an average of 22 percent more corn and 32 percent more sorghum than did plots without conservation structures in 1989 (a more normal year). In dry areas, therefore, soil conservation can often reduce the risk of crop fail- ure by improving moisture retention. But in turning over some of the cultivated area to use as diversion ditches, terraces, or hedges, these conservation measures can also adversely affect pro- duction. Physical structures, in particular, can reduce the available area for cul- tivation by more than 10 percent. Construction of cutoff drains in Tierra Blanca, for example, reduced the effective cultivation area by about 14 percent, while terrace construction in the Patzite region of Guatemala led to a 15 percent reduction. Further, terracing often entails movements of earth that bring unpro- ductive soil to the surface. In Tierra Blanca, the few diversion ditches that had been constructed with subsidies had the additional disadvantage of interfering with the prevailing production practices, which rely heavily on mechanical equipment. Such drawbacks clearly, and often heavily, influence the ultimate profitability of these conservation measures. Because some of the productivity estimates are based on weak or incomplete data, extensive sensitivity analyses were incorporated into each case study. The 284 The World Bank Research Observer, vol. 9, no. 2 (July 1994) results are robust to changes in the estimated effects on yield in several cases, but are affected significantly in other cases by changes in assumed rates of de- cline in yield. In such instances, the premium to additional research would be high. In the Santa Lucia case study in Nicaragua, data were insufficient to es- timate the effects of degradation on productivity. Simulation analysis was used, therefore, to examine returns to the proposed conservation measures (diversion ditches constructed manually with stone barriers) under a range of assumptions about the effect of degradation and conservation on yield. The simulations show that the proposed conservation measures are likely to be profitable only if they lead to substantially improved yields. Farm-Level Returns to Soil Conservation Measures Effects on yield are not the only factors to consider in analyzing the costs and benefits of investing in a given conservation measure. Table 4 summarizes the results of a full economic analysis of each case study where data were suf- ficient to allow adequate assessment. The most profitable conservation measures were found in Maissade in Haiti, and Turrubares in Costa Rica. The conservation measure used in Maissade is an indigenous technique, known as ramp pay, which consists of crop stubble laid out along the contour, supported by stakes, and covered with soil. It is cheap to construct and very effective in halting erosion. Moreover, without conservation measures, yield would decline particularly rapidly in that area. Conservation measures in Turrubares consist of expensive terraces, but they protect highly profitable export crop production from extremely rapid rates of yield decline. Rates of return to the proposed conservation measures were also estimated to be high in the Tatumbla area of Honduras, where yields decline rapidly if no conservation measures are taken. The least profitable conservation measures studied were found in Barva and Tierra Blanca, Costa Rica. The Tierra Blanca case is particularly interesting, because rates of erosion are very high. But the region's deep volcanic soils mean that degradation has very little effect on productivity. In fact, production would actually be higher without the proposed conservation measures-diver- sion ditches-because their construction would reduce the effective cultivated area and, by interfering with current production practices, would increase the costs of production. Unsurprisingly, the farmers in the area have little interest in building the ditches. In Maissade in Haiti, Turrubares in Costa Rica, and Patzite in Guatemala, data were available to examine the returns to different forms of conservation. In Maissade, the indigenous ramp pay conservation technique is clearly supe- rior to rock walls, which are more expensive and lack the agronomic advan- tages of ramp pay. In Turrubares the choice is less clear: terraces slow erosion much more effectively than diversion ditches, but they are also more expensive Ernst Lutz, Stefano Pagiola, and Carlos Reiche 285 Table 4. Estimated Returns to Investments in Conservation in the Case Study Areas Internal Net Present Rate of Return Conservation Value (NPV) (IRR) Years to Country and area measure Crop (US$) (percent) break even Costa Rica Barva Diversion ditches Coffee -920 < 0 > 100 Tierra Blanca Diversion ditches Potatoes -3,440 < 0 > 100 Turrubares Diversion ditches Coco yam 1,110 84.2 2 Turrubares Terraces Coco yam 4,140 60.2 3 Dominican Republic El Naranjal Diversion ditches Pigeon peas, peanuts, beans -132 16.9 > 100 Guatemala Patzite Terraces Corn -156 16.5 > 100 Haiti Maissade Ramp pay Corn, sorghum 1,180 -a 0 Rock walls Corn, sorghum 956 _-a 1 Honduras Tatumbla Diversion ditches Corn 909 56.5 4 Yorito Diversion ditches Corn 83 21.9 18 Panama Cocle Terraces Rice, corn, yuca beans 34 27.2 8 Note: Net present value is computed over fifty years, using a 20 percent real discount rate. a. Undefined, because net returns are positive from year one onward. Source: Case studies in Lutz, Pagiola, and Reiche (1994). to construct and entail a greater reduction in effective cultivated area. The tradeoff between effectiveness and cost is fairly easy to make in this case be- cause the greater effectiveness of terraces more than compensates for their ad- ditional cost. But greater effectiveness does not always equate with higher profitability. In Patzite, for example, a combination of diversion ditches and live barriers appears to be substantially more profitable than terraces, even if much less effective. This situation appears to be more representative of condi- tions encountered in Central America: in analyses of twenty conservation tech- niques in Mexico, for example, McIntire (1994) also found that cultivation and cropping practices, including vegetative barriers, were superior to structural measures in terms of profitability. Only when crop production is very profit- able but extremely vulnerable to degradation (as in the case of Turrubares) are expensive conservation measures likely to be justified. Unfortunately, data were insufficient to examine differences in returns with- in the study areas. Evidence from Kenya (Pagiola 1992) suggests that returns 286 The World Bank Research Observer, vol. 9, no. 2 (July 1994) to conservation can vary considerably even within narrowly defined agroeco- logical zones. Farmers on different slopes, for example, experience different rates of erosion. They also face different costs of conservation; the optimal spacing of terraces and diversion ditches, for example, is a function of the slope. Whether these differences are significant in any given instance is an em- pirical matter. In each case, adoption rates appeared to correlate well with the estimated profitability of conservation. The profitability of ramp pay is confirmed by its widespread adoption in Maissade. Conservation measures were also adopted at high rates in the Tatumbla region of Honduras and the Turrubares region of Costa Rica; not surprisingly, the rates were very low in Tierra Blanca. Adoption rates were also low in Yorito, Honduras; there, the studies estimated the conservation measures to be marginally profitable, but the estimates are based on particularly weak data and are fairly sensitive to changes in assump- tions. Thus, it may be perfectly rational for farmers not to adopt the proposed conservation measures. In some cases-in Tierra Blanca, for instance-degra- dation simply is not a significant problem for productivity. In others, the costs of the proposed conservation measures are too high relative to their benefits. The case of Patzite in Guatemala illustrates this best: although degradation is relatively rapid and, if left untreated, will make production uneconomic within a decade, the proposed terraces are very expensive and take much of the land out of cultivation. Again, this is not to say that all conservation measures are unprofitable. Visits to Tierra Blanca show, for example, that although farmers have not built diversion ditches without subsidies, they do plant along contours and, on steeper slopes, construct temporary embankments on their fields. (The effects of these measures are implicit in the estimates of degradation and of impact on productivity for the "without conservation" case.) Obstacles to Adopting Conservation Measures Profitability of conservation practices is a necessary but not always sufficient condition for their adoption. Factors other than strict cost-benefit consider- ations also play a role (Van Kooten, Weisensel, and Chinthammit 1990; Murray 1994). Some of these factors are reflected in the cost-benefit analysis to the extent that they affect the prices faced by farmers. The effects of imper- fect factor markets, for example, are reflected in higher prices for inputs, which affect the profitability of production activities. Most often, however, institu- tional issues (such as land tenure and access to credit) and the conservation ethic of farmers must be considered together with the results of the cost- benefit analysis. The analysis carried out for the case studies does not always provide conclusive evidence on these, but it does provide some insights. It has often been argued that insecure property rights discourage farmers from undertaking long-term investments, such as investments in soil conservation, Ernst Lutz, Stefano Pagiola, and Carlos Reiche 287 because they may not themselves be able to reap the benefits (Ervin 1986; Wacht- er 1992). To make tenure more secure, numerous efforts have been made to pro- vide farmers with legal title to their land. The U.S. Agency for International Development (USAID), for example, has funded titling projects in several coun- tries, including El Salvador and Honduras. But equating land titles with secure tenure and thus with increased investment is too simplistic. Unless numerous im- provements are made to the legal system and governmental institutions, most farmers find land titles too costly to obtain or enforce, and unless access to credit is improved for farmers holding titles, the desired effect on investment may not materialize. Tenure insecurity may not be as significant a factor in adopting conservation measures as is sometimes thought, however. Table 4 shows that in most of the case studies, profitable conservation measures had relatively short payback periods. Where long payback periods were forecast, the measures were either unprofitable or only marginally profitable and were thus unlikely to be adopted even in the absence of tenure problems. Other evidence from the case studies also reinforces this conclusion. About 80 percent of the farmers in the Tatum- bla area in Honduras own land by occupation-that is, they do not have any legal titles-yet most have adopted the recommended conservation measures. In the Patzite region in Guatemala, where only 10 percent of farmers have title to their land, erosion is a significant problem, but farmers have been relatively slow to adopt conservation measures. At first sight, this might appear to be evidence for the importance of titling. But, in our view, negative profitability of the recommended conservation measures is more likely to account for low adoption rates than tenure insecurity or lack of land titles. Another often-cited obstacle to adoption is the lack of capital markets. If credit markets fail, adoption of conservation will be limited by the farmers' ability to finance the required investments (Pender 1992). The research for this project did not bring to light any direct evidence on the functioning of capital markets in the areas studied. The estimated rates of return for investments in conservation measures shown in table 4 give some indication of the maximum rates of interest that could be supported before the investments become unprofitable. Several of the estimated rates of return are encouragingly high.4 Conclusions and Policy Implications Whether conservation measures are profitable for the farmers is an empirical and site-specific issue. Returns to conservation depend on the specific agroeco- logical conditions faced, on the technologies used, and on the prices of inputs used and outputs produced. Hard data on the actual extent of soil degradation and its effects on productivity remain extrerrely scarce despite several decades of soil conservation efforts (Lal 1988; Walling 1988). More systematic research is needed on soil degradation and its consequences-and there is considerable 288 The World Bank Research Observer, vol. 9, no. 2 (July 1994) scope for collaboration on such research, since all countries within Central America include a large number of different agroecological regions, and many agroecological regions are found in more than one country. Regional organi- zations such as CATIE (Centro Agron6mico de Investigaci6n y Ensefianza) have an obvious coordinating role to play. The payoff is likely to be high, because the approach to soil conservation would be more targeted, with efforts con- centrated where they are needed most. The results of the case studies show that conservation is profitable in some instances but not in others. In view of the small number of cases studied and the weak data available, broad lessons must be drawn with care. It does seem safe to say, however, that except when high-value crops are planted on very fragile soils (such as the coco yam in Turrubares), expensive mechanical struc- tures are unlikely to be profitable for the farmers. Conservation measures are particularly likely to be profitable either when they are cheap and simple or when they allow farmers to adopt improved practices. Generally, the farmers' decision to invest in conservation is based on normal considerations of benefit and cost: they tend to adopt conservation measures when it is in their interest to do so, unless some constraint is present. Cases in which returns to conservation were estimated to be low or negative correlated well with low adoption rates. A full examination of the role of government policy in conservation requires a broader analysis than that undertaken here; in particular, off-site effects of degradation would have to be explicitly included, and allowance made for dis- tortions in observed price signals resulting from government policies or market failures. Nevertheless, several important points emerge from this analysis. Subsidies Advocates of soil conservation often argue that subsidies are indispensable to induce farmers to adopt conservation measures. But such statements often assume that conservation is inherently desirable whether or not there is con- crete evidence that the benefits outweigh the costs. The results presented here show that this may be far from the case; frequently, the benefits of specific con- servation techniques (such as mechanical structures) do not justify their costs. Unless there are important off-farm effects or the price signals received by farmers are significantly distorted, subsidies to induce adoption would there- fore not increase economic efficiency. When off-farm effects are present, the rationale for intervention is potent, because the farmers' estimation of returns to conservation will pay inadequate attention to its social benefits. In the Santa Lucia Milpas Altas watershed in Guatemala, for example, a USAID project uses subsidies (so-called pago social) to induce farmers to build terraces and thus reduce flooding in the historic town of Antigua. In the same watershed, farmers who do not receive subsidies generally use less costly conservation methods such as vegetative barriers and Ernst Lutz, Stefano Pagiola, and Carlos Reiche 289 live fences. Although these measures are profitable to the farmers, they may not be enough to control floods. The effect of price distortions is more difficult to establish; the many factors that affect the profitability of a given conservation measure and their compli- cated interactions make it hard to predict whether a distortion encourages or discourages conservation. It has sometimes been suggested that typical policy distortions in developing countries tend to encourage degradation (Panayotou 1993), but the empirical basis to substantiate this point is weak. The best way to deal with policy distortions or market failures is to attempt to eradicate the distortions themselves; subsidies should be used only in the rare instances when such direct action is virtually impossible. Whatever the justification, the use of subsidies encounters several difficul- ties. First, the divergence between social and private returns to conservation must be established, so that intervention can be directed where it will be most effective. Subsidies are often used where no off-farm effects are present-wast- ing scarce budgetary resources in areas where they are not justified by any so- cial benefits. In Costa Rica, for example, the soil conservation service (SENACSA) subsidizes half the cost of establishing conservation measures on the fields of small farmers, irrespective of location. Subsidies are also provided in areas such as Turrubares, where individual farmers already have sufficient in- centive to conserve purely on productivity grounds. Conversely, subsidies are not always provided when off-farm effects are present. More commonly, sub- sidies are provided to construct, but not maintain, the conservation measures, so farmers sometimes allow them to decay. In Nicaragua, for example, terraces were built on fields in the Lake Xolotlan watershed above Managua in an ef- fort to reduce flooding in the city and sedimentation in its reservoirs. These terraces were built at no cost to farmers, but because they interfered with cul- tivation practices and resulted in no net benefits to the farmers, most were soon destroyed. Similar experiences have occurred in the Tierra Blanca area of Costa Rica. The second problem in using subsidies, then, is the difficulty of designing appropriate incentive structures for the farmers so that social objectives are met. The case of the Lake Xolotlan watershed illustrates a situation in which subsidies are insufficient to overcome the divergence between private and social returns to conservation. The El Naranjal watershed in the Dominican Republic provides another example. There, the USAID-funded Management of Natural Resources Project (MARENA) provided subsidized credit to participating farm- ers. Consequently, adoption rates were initially very high, even though the ev- idence suggests that the measures were unprofitable from the farmers' perspective. In 1985 more than 90 percent of the area's farms practiced soil conservation. Five years later only half of these farms continued to do so. Sub- sidies can persuade farmers to modify their behavior only as long as they con- tinue to be paid. In contrast, MARENA's successor, which tied conservation to access to irrigation, seems to have stimulated considerable use of conservation 290 The World Bank Research Observer, vol. 9, no. 2 (July 1994) techniques even though no subsidies were offered-in fact, the cost of partic- ipation was quite high. Although sufficient data were not available to analyze the new practices fully, they appear to be highly profitable. Another risk in designing subsidization schemes is that of creating perverse incentives for farmers. In Costa Rica, for example, a reforestation credit system unintentionally encouraged farmers to deforest their land so that they might qualify for the credit. The expectation that subsidies will be forthcoming to fund conservation efforts may also encourage farmers to delay conservation, even when such measures are privately profitable, in the hope that the govern- ment will bear part of their cost. Even when subsidies are justified, then, they must be designed with great care. Land Tenure, Research, and Extension Governments should also ensure that constraints such as insecure tenure do not prevent farmers from adopting conservation measures. But such efforts also require substantiating research if they are to be effective. Too often the existence of tenure problems and the effectiveness of titling as a solution are simply taken as given. Governments already do some research on soil conservation and provide, through extension services, some assistance to farmers who undertake conser- vation work. However, research in experiment stations has tended to favor technical efficiency (including structural measures such as terraces) over prof- itability for farmers. Further, government extension work is often ineffective. In many cases, nongovernmental organizations, such as Vecinos Mundiales in Central America (Lopez and Pio Camey 1994) have proved to be more effective than extension services at presenting a range of conservation options to farm- ers and delivering related technical assistance. Because of the wide variety of conditions that farmers face, government extension services should also pro- vide, explain, and demonstrate to farmers the corresponding variety of options available rather than, as has often happened in the past, pushing broadly for adoption of specific techniques. And governments may find it both innovative and effective to decentralize decisionmaking and channel budgetary resources for soil conservation to the local level to allow communities to participate in the decisionmaking and to contract assistance from those from whom the greatest contributions can be expected. Research is not likely to produce a "breakthrough technology" that will solve all conservation problems. Improvements are likely to be more marginal. But, alone or in combination with others, improved techniques can significantly af- fect productivity. Modifications in the ramp pay technique used in Haiti are an example. Here, the traditional practice of gathering crop stubble along the con- tour was improved by more exact placement and by covering the structure with upslope soil, thus discouraging rat infestations and encouraging infiltration of the surface flow. These changes made the practice much more effective in halt- Ernst Lutz, Stefano Pagiola, and Carlos Reiche 291 ing degradation and more acceptable to farmers. Similar improvements in tech- niques arising from research have been successful in West Africa (Reij 1992). The conflict between conservation and production noted in many of the case studies often affects the returns to conservation very significantly. Attempts to develop practices that reduce or eliminate this conflict-"overlap technolo- gies," in the terminology of Reardon and Vosti (1992)-should be especially encouraged. And to make the research truly useful, it should be carried out primarily on the farm and in close consultation with farmers. Notes Ernst Lutz is with the Land, Water, and Natural Habitats Division of the World Bank's En- vironment Department; Stefano Pagiola is at the Food Research Institute, Stanford University; and Carlos Reiche is at the Centro Agron6mico de Investigaci6n y Ensefianza (CATIE), Costa Rica. The results presented in this article are summarized from a broader study (Lutz, Pagiola, and Reiche 1994). The data used in the case studies were collected and analyzed by Mauricio Cuesta and Hector Manuel Melo Abreu (Costa Rica); Jose Abel Hernandez (Dominican Repub- lic); Jose Bueno Alferez, Jose Roberto Hernandez Navas, and Rafael Lazo Melendez (El Salva- dor); Luis Eduardo Barrientos, Saul Adolfo Lima, and Pedro Antonio Rosado (Guatemala); Jon L. Jickling and Thomas A. White (Haiti); Jose Wilfredo Andino, Carlos Awad Ramirez, Gabino L6pez Vargas, Gilberto Palma, Augustin Pio Camey, and Antonio Valdez (Honduras); Danilo Antonio Montalban and Miguel Obando Espinoza (Nicaragua); and Cesar Isaza, Julio Santa- maria, and Tomas Vasquez (Panama). The authors are indebted to Hans Binswanger, John English, John McIntire, Augusta Molnar, Alfredo Sfeir-Younis, Stephen Vosti, and several mem- bers of the Editorial Committee for valuble comments and suggestions. 1. Off-farm effects and another important land degradation problem-the inappropriate use of common-property lands-are outside the scope of this research. (Off-farm effects were dis- cussed by Magrath and Arens 1989; for common-property issues, see Bromley 1992.) 2. Combined investments from households in a village or watershed area are sometimes re- quired to manage land degradation problems effectively. For an analysis of such problems in the same area as the Haiti case study, see White and Runge (1992). 3. In addition to the sites listed in table 2, research was carried out at other sites in several of the countries listed and at several sites in El Salvador. Data on these sites-in particular, on the effects of degradation on yields-were insufficient to allow a full analysis of the returns to con- servation measures. 4. Even when rates of return to investment in conservation are high, conservation might not be undertaken if even higher rates of return can be obtained from off-farm opportunities. South- gate (1992), for example, argues that high returns to urban employment in Ecuador encourage farmers to depreciate their land assets and then move to urban areas. Similarly, Schneider and others (1993) argue that perceptions of limitless land resources in the Amazon prompt farmers to "mine" their soils and then move on. References The word "processed" describes informally reproduced works that may not be commonly available through library systems. Altieri, M. 1990. "Stabilizing Hillside Farming Systems in the Sierra of the Dominican Republic." In Research Needs and Applications to Reduce Erosion and Sedimentation in Tropical Steep Lands. Proceedings of the Fiji Symposium. IAHS-AISH Publications 192. June. 292 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Biot, Yvan, Robert Lambert, and Scott Perkin. 1992. "What's the Problem? An Essay on Land Degradation, Science and Development in Sub-Saharan Africa." Discussion Paper 222. School of Development Studies, University of East Anglia, Norwich. Processed. Blaikie, Piers, and Harold Brookfield. 1987. "Defining and Debating the Problem." In Piers Blaikie and Harold Brookfield, eds., Land Degradation and Society. London: Methuen & Co. Bromley, Daniel W., ed. 1992. Making the Commons Work: Theory, Practice, and Policy. San Francisco: ics Press. Capalbo, John M., and Susan M. Antle. 1988. Agricultural Productivity: Measurement and Ex- planation. Washington: Resources for the Future. CTA (Centro de Tecnologia Agricola). 1956. Informe Interno. San Salvador, El Salvador. CUNARD. 1991. "Soil Erosion Estimation From the Cropping Systems Trial at the Brutus Site, 1990." Associates in Rural Development/Project Sove Te, Burlington, Vermont. Processed. De Janvry, Alain, Marcel Fafchamps, and Elisabeth Sadoulet. 1990. "Peasant Household Behav- ior with Missing Markets: Some Paradoxes Explained." University of California, Department of Agricultural and Resource Economics, Berkeley. Processed. English, John, Mary Tiffen, and Michael Mortimore. 1994. Land Resource Management in the Machakos District, Kenya 1930-1990. World Bank Environment Paper 5. Washington, D.C. Ervin, David E. 1986. "Constraints to Practicing Soil Conservation: Land Tenure Relationships." In Stephen B. Lovejoy and Ted L. Napier, eds., Conserving Soil: Insights from Socioeconomic Research. Ankeny, Iowa: Soil Conservation Society of America. Flores Zelaya, 0. 1976-81. "Informes anuales de parcelas para investigaci6n de! control de la erosi6n y escorrentia superficial en el Distrito Forestal de Metapan (seis informes)." Servicio de Ordenaci6n de Cuencas Hidrograficas y Conservaci6n de Suelos, San Salvador, El Salvador. Processed. Grosjean, M. 1987. "Rapport d'Ouverture: La Recherche de Conservation de Sol a Papaye. Hinche/Plateau Central. Project Agrosilvicol." Port-au-Prince, Haiti: Helvetas. Processed. Harsthorn, G., G. Antonini, R. Dubois, D. Harcharik, S. Heckadon, H. Newton, C. Quezada, J. Shores, and G. Staples. 1981. "Perfil ambiental de la Repuiblica Dominicana: Un estudio de campo." USAID Contract AID/SOD/PDC-C-0247. JRB Associates, Va. Lal, Rattan C. 1988. "Preface." In Rattan C. Lal, ed., Soil Erosion Research Methods. Ankeny, Iowa: Soil and Water Conservation Society. Leonard, H. Jeffrey. 1987. Natural Resources and Development in Central America: A Regional Environmental Profile. International Institute for Environment and Development. New Brunswick, N.J.: Transaction Books. L6pez, Vargas Gabino, and Augustin Pio Camey. 1994. "Practical Experiences and Lessons Learned by Vecinos Mundiales from Soil Conservation Work in Rural Communities of Hon- duras." In Ernst Lutz, Stefano Pagiola, and Carlos Reiche, eds., Economic and Institutional Analyses of Soil Conservation Projects in Central America and the Caribbean. World Bank En- vironment Paper 8. Washington, D.C. Lutz, Ernst, Stefano Pagiola, and Carlos Reiche, eds. 1994. Economic and Institutional Analyses of Soil Conservation Projects in Central America and the Caribbean. World Bank Environment Paper 8. Washington, D.C. Magrath, William B., 1989. "Economic Analysis of Soil Conservation Technologies." Environ- ment Department Divisional Working Paper 1989-4. World Bank, Policy Research Division, Washington, D.C. Processed Magrath, William B., and Peter Arens. 1989. "The Costs of Soil Erosion on Java: A Natural Resource Accounting Approach." Environment Department Working Paper 18. World Bank, Washington, D.C. Processed. McIntire, John. 1994. "A Review of the Soil Conservation Sector in Mexico." In Ernst Lutz, Stefano Pagiola, and Carlos Reiche, eds., Economic and Institutional Analyses of Soil Conser- vation Projects in Central America and the Caribbean. World Bank Environment Paper 8. Washington, D.C. Ernst Lutz, Stefano Pagiola, and Carlos Reiche 293 Murray, Gerald. 1994. "Technoeconomic, Organizational, and Ideational Factors as Determi- nants of Soil Conservation in the Dominican Republic." In Ernst Lutz, Stefano Pagiola, and Carlos Reiche, eds., Economic and Institutional Analyses of Soil Conservation Projects in Cen- tral America and the Caribbean. World Bank Environment Paper 8. Washington, D.C. Oldeman, L. R., R. T. A. Hakkeling, and W. G. Sombroek. 1990. World Map of the Status of Human-Induced Soil Degradation: An Explanatory Note. 2d ed. rev. Wageningen: International Soil Reference and Information Centre. Oster, R. 1981. "Conservaci6n de suelos en tierras altas de Chiriqui." Informe T&cnico, David, Panama. Processed. Pagiola, Stefano. 1992. "Quantifying the Returns to Soil Conservation in Kenya." Paper present- ed at the Annual Meetings of the Soil and Water Conservation Society, Baltimore, Maryland, August 10-12, 1992. Processed. . 1993. "Soil Conservation and the Sustainability of Agricultural Production." Ph.D. Diss., Food Research Institute, Stanford University. Processed. Panayotou, Theodore. 1993. Green Markets: The Economics of Sustainable Development. ICEG Sector Studies 7. San Francisco: ics Press. PCEO. ("Proyecto de Control de erosi6n de Occidente.") 1981. Internal report. Irena, Le6n, Nicaragua. Processed. Pender, John L. 1992. "Credit Rationing and Farmers' Irrigation Investments in Rural South In- dia: Theory and Evidence." Ph.D. Diss., Food Research Institute, Stanford University. Pro- cessed. Reardon, Thomas, and Stephen A. Vosti. 1992. "Issues in the Analysis of the Effects of Policy on Conservation and Productivity at the Household Level in Developing Countries." Quar- terly Journal of International Agriculture 31(4, October-December):380-96. Reij, Chris. 1992. "Building on Traditions: The Improvement of Indigenous swc Techniques in the West African Sahel." Paper presented at the International Symposium on Soil and Water Conservation, Soil and Water Conservation Society, Honolulu, Hawaii. Processed. Repetto, Robert, and Wilfrido Cruz. 1991. Accounts Overdue: Natural Resource Depreciation in Costa Rica. Washington, D.C.: World Resources Institute. Sanchez, Hector. 1991. Personal communication from Hector Sanchez, DIMA-SPS. Tegucigalpa, Honduras. Schneider, Robert, Gunars Platais, David Rosenblatt, and Maryla Webb. 1993. "Sustainability, Yield Loss and Imediatismo: Choice of Technique at the Frontier." LATEN Dissemination Note 1. World Bank, Latin America Technical Department. Washington, D.C. Processed. Shaxson, T. F., N. W. Hudson, D. W. Sanders, E. Roose, and W. C. Moldenhauer. 1989. "Land Husbandry: A Framework for Soil and Water Conservation." Soil and Water Conservation Society, Ankeny, Iowa. Processed. Singh, Inderjit, Lyn Squire, and John A. Strauss, eds. 1986. Agricultural Household Models: Ex- tensions, Applications, and Policy. Baltimore: Johns Hopkins University Press. Soto, J. G. 1981. "Evaluaci6n experimental de la erosi6n hidrica en el suelo de la subcuenca de Rio Indio, bajo el sistema tradicional de manejo." Thesis, Facultad de Agronomia, Univer- sidad de Panama. Southgate, Douglas. 1992. "The Rationality of Land Degradation in Latin America: Some Les- sons from the Ecuadorian Andes." Paper presented at the International Symposium on Soil and Water Conservation, Soil and Water Conservation Society, Honolulu, Hawaii. Taylor, Daniel B., Douglas L. Young, David J. Walker, and Edgar L. Michalson. 1986. "Farm Level Economics of Soil Conservation in the Palouse Area of the Northwest: Comment." American Journal of Agricultural Economics 68(2, May) :364-65. Van Kooten, G. C., Ward P. Weisensel, and Duangdao Chinthammit. 1990. "Valuing Trade- Offs between Net Returns and Stewardship Practices: The Case of Soil Conservation in Saskatchewan." American Journal of Agricultural Economics 72(1, February): 104-13. 294 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Vasquez, Tomas, and Julio Santamaria. 1994. "Economic and Institutional Analysis of Soil Con- servation at the Farm Level in Cocle, Panama" in Ernst Lutz, Stefano Pagiola, and Carlos Reiche, eds., Economic and Institutional Analyses of Soil Conservation Projects in Central America and the Caribbean. World Bank Environment Paper 8, Washington, D.C. Veloz, Alberto, Douglas Southgate, Fred Hitzhusen, and Robert Macgregor. 1985. "The Eco- nomics of Erosion Control in a Subtropical Watershed: A Dominican Case." Land Economics 61(1, May):145-55. Wachter, Daniel. 1992. "Land Titling for Land Conservation in Developing Countries." Envi- ronment Department Divisional Working Paper No. 1992-28. World Bank, Washington, D.C. Processed. Walker, David J. 1982. "A Damage Function to Evaluate Erosion Control Economics." Ameri- can Journal of Agricultural Economics 64(4, November):145-55. Walling, D. E. 1988. "Measuring Sediment Yield from River Basins." In Rattan C. Lal, ed., Soil Erosion Methods. Ankeny, Iowa: Soil and Water Conservation Society. Welchez, L. 1991. Personal communication from Welchez of LUPE/SRN project, Francisco Morazan, Honduras. White, Thomas A., and C. Ford Runge. 1992. "Common Property and Collective Action: Co- operative Watershed Management in Haiti." Working Paper P92-3. Center for International Food and Agricultural Policy, University of Minnesota, St. Paul. Processed. Ernst Lutz, Stefano Pagiola, and Carlos Reiche 295 WORLD BANK RESEARCH OBSERVER CUMULATIVE INDEX, 1986-94 Authors Ahamed, Liaquat, "Stabilization Policies in Developing Countries," 1 (1, January 1986): 79-110 Ahmad, Ehtisham, "Social Security and the Poor: Choices for Developing Countries," 6 (1, January 1991): 105-27 Ainsworth, Martha, and Mead Over, "AIDS and African Development," 9 (2, July 1994): 203-40 Albrecht, Douglas, and Adrian Ziderman, "Student Loans: An Effective Instrument for Cost Recovery in Higher Education?" 8 (1, January 1993): 71-90 Ayub, Mahmood A., and Sven 0. Hegstad, "Management of Public Industrial Enter- prises," 2 (1, January 1987): 79-101 Bale, Malcolm D. (See Ulrich Koester) Barnes, Douglas F., Keith Openshaw, Kirk R. Smith, and Robert van der Plas, "The Design and Diffusion of Improved Cooking Stoves," 8 (2, July 1993): 119-41 Baumol, William J., and Kyu Sik Lee, "Contestable Markets, Trade, and Develop- ment," 6 (1, January 1991): 1-17 Bell, Clive, "Reforming Property Rights in Land and Tenancy," 5 (2, July 1990): 143-66 Besley, Timothy, "How Do Market Failures Justify Interventions in Rural Credit Mar- kets?" 9 (1, January 1994): 27-47 Bhagwati, Jagdish N., "Export-Promoting Trade Strategy: Issues and Evidence," 3 (1, January 1988): 27-57 Binswanger, Hans, "Agricultural Mechanization: A Comparative Historical Perspec- tive," 1 (1, January 1986): 27-56 Binswanger, Hans, and Prabhu Pingali, "Technological Priorities for Farming in Sub- Saharan Africa," 3 (1, January 1988): 81-98 Binswanger, Hans, and Joachim von Braun, "Technological Change and Commercial- ization in Agriculture: The Effect on the Poor," 6 (1, January 1991): 57-80 Birdsall, Nancy, "Economic Analyses of Rapid Population Growth," 4 (1, January 1989): 23-50 Borrell, Brent, and Ronald C. Duncan, "A Survey of the Costs of World Sugar Poli- cies," 7 (2, July 1992): 171-94 The World Bank Research Observer, vol. 9, no. 2 (July 1994), pp. 297-308 © 1994 The International Bank for Reconstruction and Development/THE WORLD BANK 297 Caprio, Gerard, Jr., and Ross Levine, "Reforming Finance in Transitional Socialist Economies," 9 (1, January 1994): 1-24 Carmichael, Jeffrey, "The Debt Crisis: Where Do We Stand after Seven Years?" 4 (2, July 1989): 121-42 Claessens, Stijn, "Alternative Forms of External Finance: A Survey," 8 (1, January 1993): 91-117 Corbo, Vittorio, and Jaime de Melo, "Lessons from the Southern Cone Policy Re- forms," 2 (2, July 1987): 111-42 Corden, Max, "Macroeconomic Adjustment in Developing Countries," 4 (1, January 1989): 51-64 , "The Relevance for Developing Countries of Recent Developments in Macro- economic Theory," 2 (2, July 1987): 171-88 Coricelli, Fabrizio, and Timothy D. Lane, "Wage Controls during the Transition from Central Planning to a Market Economy," 8 (2, July 1993): 195-210 Cox, Donald, and Emmanuel Jimenez, "Achieving Social Objectives through Private Transfers: A Review," 5 (2, July 1990): 205-18 Cuddington, John, "Commodity Export Booms in Developing Countries," 4 (2, July 1989): 143-65 de Haan, Cornelis, (See Dina L. Umali) de Melo, Jaime (See Vittorio Corbo; Carl B. Hamilton) Duncan, Ronald C. (See Brent Borrell; Donald 0. Mitchell) Easterly, William R., and Klaus Schmidt-Hebbel, "Fiscal Deficits and Macroeconomic Performance in Developing Countries," 8 (2, July 1993): 211-37 Easterly, William (See Stanley Fischer) Edwards, Sebastian, "Exchange Rate Misalignment in Developing Countries," 4 (1, January 1989): 3-21 Eichengreen, Barry, and Richard Portes, "The Interwar Debt Crisis and Its Aftermath," 5 (1, January 1990): 69-94 Elkan, Walter, "Entrepreneurs and Entrepreneurship in Africa," 3 (2, July 1988): 171-88 Eskeland, Gunnar S., and Emmanuel Jimenez, "Policy Instruments for Pollution Con- trol in Developing Countries," 7 (2, July 1992): 145-69 Feder, Gershon, and Raymond Noronha, "Land Rights Systems and Agricultural De- velopment in Sub-Saharan Africa," 2 (2, July 1987): 143-69 Feder, Gershon, and Roger Slade, "The Impact of Agricultural Extension: The Train- ing and Visit System in India," 1 (2, July 1986): 139-61 Feder, Gershon (See Monika Huppi; Dina L. Umali) 298 The World Bank Research7 Observer, vol. 9, no. 2 (July 1994) Fields, Gary S., "Changes in Poverty and Inequality in Developing Countries," 4 (2, July 1989): 167-8S Findlay, Ronald (See Stanislaw Wellisz) Finger, J. Michael, "Dumping and Antidumping: The Rhetoric and Reality of Protec- tion in Industrial Countries," 7 (2, July 1992): 121-43 Fischer, Stanley, "Issues in Medium-Term Macroeconomic Adjustment," 1 (2, July 1986): 163-82 Fischer, Stanley, and William Easterly, "The Economics of the Government Budget Constraint," 5 (2, July 1990): 127-42 Fitzgerald, Bruce, and Terry Monson, "Preferential Credit and Insurance as Means to Promote Exports," 4 (1, January 1989): 89-114 Gertler, Paul, (See Ernst Lutz) Glaessner, Thomas (See J. Luis Guasch) Goto, Junichi, "The Multifibre Arrangement and Its Effects on Developing Countries," 4 (2, July 1989): 203-27 Greenaway, David, and Chris Milner, "South-South Trade: Theory, Evidence, and Pol- icy," 5 (1, January 1990): 47-68 Gross, David J. (See Stephen K. Mayo) Grossman, Jean Baldwin, "Evaluating Social Policies: Principles and U.S. Experience," 9 (2, July 1994): 159-80 Guasch, J. Luis, and Thomas Glaessner, "Using Auctions to Allocate and Price Long- Term Credit" 8 (2, July 1993): 169-94 Haggard, Stephan, and Steven B. Webb, "What Do We Know about the Political Econ- omy of Economic Policy Reform?" 8 (2, July 1993): 143-68 Hamilton, Carl B., Jaime de Melo, and L. Alan Walters, "Who Wins and Who Loses from Voluntary Export Restraints? The Case of Footwear," 7 (1, January 1992): 17-35 Hammer, Jeffrey S., "The Economics of Malaria Control," 8 (1, January 1993): 1-22 Havrylyshyn, Oli, "Trade Policy and Productivity Gains in Developing Countries: A Survey of the Literature," 5 (1, January 1990): 1-24 Hegstad, Sven 0. (See Mahmood A. Ayub) Hughes Hallett, A. J., "The Impact of EC-92 on Trade in Developing Countries," 9 (.1, January 1994): 121-46 Huppi, Monika, and Gershon Feder, "The Role of Groups and Credit Cooperatives in Rural Lending," 5 (2, July 1990): 187-204 Jaffee, Steven, and Jitendra Srivastava, "The Roles of the Private and Public Sectors in Enhancing the Performance of Seed Systems," 9(1, January 1994): 97-117 Cumulative Index, 1 986-94 299 Jimenez, Emmanuel, "The Public Subsidization of Education and Health in Developing Countries: A Review of Equity and Efficiency," 1 (1, January 1986): 111-29 Jimenez, Emmanuel, Marlaine E. Lockheed, and Vicente Paqueo, "The Relative Effi- ciency of Private and Public Schools in Developing Countries," 6 (2, July 1991): 205-18 Jimenez, Emmanuel (See Donald Cox; Gunnar S. Eskeland) Kannappan, Subbiah, "Urban Labor Markets and Development," 3 (2, July 1988): 189-206 Khan, Mohsin S., "Macroeconomic Adjustment in Developing Countries: A Policy Per- spective," 2 (1, January 1987): 23-42 Kiguel, Miguel A., and Nissan Liviatan, "When Do Heterodox Stabilization Programs Work? Lessons from Experience," 7 (1, January 1992): 35-59 Kikeri, Sunita, John Nellis, and Mary Shirley, "Privatization: Lessons from Market Economies," 9 (2, July 1994): 241-72 Koester, Ulrich, and Malcolm D. Bale, "The Common Agricultural Policy: A Review of Its Operation and Effects on Developing Countries," 5 (1, January 1990): 95-121 Kravis, Irving B. "The Three Faces of the International Comparison Project," 1 (1, Jan- uary 1986): 3-26 Krueger, Anne O., "Aid in the Development Process," 1 (1, January 1986): 57-78 Lal, Deepak, and Sarath Rajapatirana, "Foreign Trade Regimes and Economic Growth in Developing Countries," 2 (2, July 1987): 189-217 Lane, Timothy D. (See Fabrizio Coricelli) Lee, Kyu Sik (See William J. Baumol) Levin, Henry M., "A Benefit-Cost Analysis of Nutritional Programs for Anemia Re- duction," 1 (2, July 1986): 219-45 Levine, Ross, (See Gerard Caprio Jr.) Lindauer, David L., Oey Astra Meesook, and Parita Suebsaeng, "Government Wage Policy in Africa: Some Findings and Policy Issues," 3 (1, January 1988): 1-25 Lindauer, David L., and Ann D. Velenchik, "Government Spending in Developing Countries: Trends, Causes, and Consequences," 7 (1, January 1992): 59-79 Liviatan, Nissan (See Miguel A. Kiguel) Lockheed, Marlaine E. (See Emmanuel Jimenez) Lutz, Ernst, Stefano Pagiola, and Carlos Reiche, "The Costs and Benefits of Soil Con- servation: The Farmer's Viewpoint," 9 (2, July 1994): 273-95 Malpezzi, Stephen (See Stephen K. Mayo) Markandya, Anil, and David W. Pearce, "Development, the Environment, and the So- cial Rate of Discount," 6 (2, July 1991): 137-52 300 The World Bank Researcb Observer, vol. 9, no. 2 (July 1994) Mayo, Stephen K., Stephen Malpezzi, and David J. Gross, "Shelter Strategies for the Urban Poor in Developing Countries," 1 (2, July 1986): 183-203 McCleary, William, "The Earmarking of Government Revenue: A Review of Some World Bank Experience," 6 (1, January 1991): 81-104 McLure, Charles E., Jr., "A Simpler Consumption-Based Alternative to the Income Tax for Socialist Economies in Transition," 7 (2, July 1992): 221-37 McNelis, Paul D., "Indexation and Stabilization: Theory and Experience," 3 (2, July 1988): 157-69 Meesook, Oey Astra (See David L. Lindauer) Milner, Chris (See David Greenaway) Mintz, Jack M., and Jesus Seade, "Cash Flow or Income? The Choice of Base for Com- pany Taxation," 6 (2, July 1991): 177-90 Mitchell, Donald O., and Ronald C. Duncan, "Market Behavior of Grains Exporters," 2 (1, January 1987): 3-21 Mitra, Pradeep, "The Coordinated Reform of Tariffs and Indirect Taxes," 7 (2, July 1992): 195-218 Monson, Terry (See Bruce Fitzgerald) Musgrove, Philip, "Feeding Latin America's Children," 8 (1, January 1993): 23-45 Mussa, Michael, "Macroeconomic Policy and Trade Liberalization: Some Guidelines," 2 (1, January 1987): 61-77 Myers, Robert J., "Incomplete Markets and Commodity-Linked Finance in Developing Countries," 7 (1, January 1992): 79-95 Nash, John (See Vinod Thomas) Nellis, John (See Sunita Kikeri) Nelson, Joan M., "Organized Labor, Politics, and Labor Market Flexibility in Devel- oping Countries," 6 (1, January 1991): 37-56 Newbery, David M., "Charging for Roads," 3 (2, July 1988): 119-38 Newman, John, Laura Rawlings, and Paul Gertler, "Using Randomized Control De- signs in Evaluating Social Sector Programs in Developing Countries," 9 (2, July 1994): 181-201 Noronha, Raymond (See Gershon Feder) Openshaw, Keith (See Douglas F. Barnes) Over, Mead (See Martha Ainsworth) Pagiola, Stefano (See Ernst Lutz) Paqueo, Vicente (See Emmanuel Jimenez) Pearce, David W. (See Anil Markandya) Cumulative Index, 1986-94 301 Pingali, Prabhu (See Hans Binswanger) Pohl, Gerhard, and Piritta Sorsa, "Is European Integration Bad News for Developing Countries? A Comment on Hughes Hallett," 9 (1, January 1994): 147-55. Portes, Richard (See Barry Eichengreen) Psacharopoulos, George, "Education and Development: A Review," 3 (1, January 1988): 99-116 Psacharopoulos, George, and Zafiris Tzannatos, "Female Labor Force Participation: An International Perspective," 4 (2, July 1989): 187-201 Rajapatirana, Sarath (See Deepak Lal) Ravallion, Martin, "On 'Hunger and Public Action': A Review Article on the Book by Jean Dreze and Amartya Sen," 7 (1, January 1992): 1-17 , "Reaching the Rural Poor through Public Employment: Arguments, Evidence, and Lessons from South Asia," 6 (2, July 1991): 153-75 Rawlings, Laura, (See John Newman) Reiche, Carlos (See Ernst Lutz) Schmidt-Hebbel (See William Easterly) Seade, Jesus (See Jack M. Mintz) Serven, Luis, and Andres Solimano, "Private Investment and Macroeconomic Adjust- ment: A Survey," 7 (1, January 1992): 95-114 Shirley, Mary (See Sunita Kikeri) Shoup, Carl, "The Value-Added Tax and Developing Countries," 3 (2, July 1988): 139-56 Slade, Roger (See Gershon Feder) Smith, Kirk R. (See Douglas F. Barnes) Solimano, Andres, "Inflation and the Costs of Stabilization: Historical and Recent Ex- periences and Policy Lessons," 5 (2, July 1990): 167-85 Solimano, Andres (See Luis Serven) Sorsa, Piritta (See Gerhard Pohl) Srinivasan, T. N., "The Costs and Benefits of Being a Small, Remote, Island, Land- locked, or Ministate Economy," 1 (2, July 1986): 205-18 Srivastava, Jitendra (See Steven Jaffee) Stiglitz, Joseph E., "Some Theoretical Aspects of Agricultural Policies," 2 (1, January 1987): 43-60 Suebsaeng, Parita (See David L. Lindauer) Summers, Lawrence H., and Vinod Thomas, "Recent Lessons of Development," 8 (2, July 1993): 241-54 302 The World Bank Research Observer, vol. 9, no. 2 (July 1994) Tait, Alan A., "A Not-So-Simple Alternative to the Income Tax for Socialist Econo- mies in Transition: A Comment on McLure," 7 (2, July 1992): 239-48 Takacs, Wendy E., "Options for Dismantling Trade Restrictions in Developing Coun- tries," 5 (1, January 1990): 25-46 Thomas, Vinod, and John Nash, "Reform of Trade Policy: Recent Evidence from The- ory and Practice," 6 (2, July 1991): 219-40 Thomas, Vinod (See Lawrence H. Summers) Tzannatos, Zafiris (See George Psacharopoulos) Umali, Dina L., Gershon Feder, and Cornelis de Haan, "Animal Health Services: Find- ing the Balance between Public and Private Delivery," 9 (1, January 1994): 71-96 van der Plas, Robert (See Douglas F. Barnes) Velenchik, Ann D. (See David L. Lindauer) von Braun, Joachim (See Hans Binswanger) Wade, Robert, "The Management of Common Property Resources: Finding a Coop- erative Solution," 2 (2, July 1987): 219-34 Wakeman-Linn, John, "The Market for Developing Country Debt: The Nature and Importance of Its Shortcomings," 6 (2, July 1991): 191-203 Walters, L. Alan (See Carl B. Hamilton) Warr, Peter G., "Export Processing Zones: The Economics of Enclave Manufacturing," 4 (1, January 1989): 65-88 Webb, Steven B. (See Stephan Haggard) Wellisz, Stanislaw, and Ronald Findlay, "The State and the Invisible Hand," 3 (1, Jan- uary 1988): 59-80 Winters, L. Alan (See Carl B. Hamilton) Wood, Adrian, "How Much Does Trade with the South Affect Workers in the North?" 6 (1, January 1991): 19-36 The World Bank Water Demand Research Team, "The Demand for Water in Rural Areas: Determinants and Policy Implications," 8 (1, January 1993): 47-70 Yaron, Jacob, "What Makes Rural Finance Institutions Successful?" 9 (1, January 1994): 49-70 Ziderman, Adrian (See Douglas Albrecht) Titles "Achieving Social Objectives through Private Transfers: A Review," Donald Cox and Emmanuel Jimenez, 5 (2, July 1990): 205-18 "Agricultural Mechanization: A Comparative Historical Perspective," Hans Binswanger, 1 (1, January 1986): 27-56 Cumulative Index, 1986-94 303 "Aid in the Development Process," Anne 0. Krueger, 1 (1, January 1986): 57-78 "AIDS and African Development," Martha Ainsworth and Mead Over, 9 (2, July 1994): 203-40 "Alternative Forms of External Finance: A Survey," Stijn Claessens, 8 (1, January 1993): 91-117 "Animal Health Services: Finding the Balance between Public and Private Delivery," Dina L. Umali, Gershon Feder, and Cornelis de Haan, 9 (1, January 1994): 71-96. "A Benefit-Cost Analysis of Nutritional Programs for Anemia Reduction," Henry M. Levin, 1 (2, July 1986): 219-45 "Cash Flow or Income? The Choice of Base for Company Taxation," Jack M. Mintz and Jesus Seade, 6 (2, July 1991): 177-90 "Changes in Poverty and Inequality in Developing Countries," Gary S. Fields, 4 (2, July 1989): 167-85 "Charging for Roads," David M. Newbery, 3 (2, July 1988): 119-38 "Commodity Export Booms in Developing Countries," John Cuddington, 4 (2, July 1989): 143-65 "The Common Agricultural Policy: A Review of Its Operation and Effects on Devel- oping Countries," Ulrich Koester and Malcolm D. Bale, 5 (1, January 1990): 95-121 "Contestable Markets, Trade, and Development," William J. Baumol and Kyu Sik Lee, 6 (1, January 1991): 1-17 "The Coordinated Reform of Tariffs and Indirect Taxes," Pradeep Mitra, 7 (2, July 1992): 195-218 "The Costs and Benefits of Being a Small, Remote, Island, Landlocked, or Ministate Economy," T. N. Srinivasan, 1 (2, July 1986): 205-18 "The Costs and Benefits of Soil Conservation: The Farmer's Viewpoint," Ernst Lutz, Stefano Pagiola, and Carlos Reiche, 9 (2, July 1994): 273-94 "The Debt Crisis: Where Do We Stand after Seven Years?" Jeffrey Carmichael, 4 (2, July 1989): 121-42 "The Demand for Water in Rural Areas: Determinants and Policy Implications," The World Bank Water Demand Research Team, 8 (1, January 1993): 47-70 "The Design and Diffusion of Improved Cooking Stoves," Douglas F. Barnes, Keith Openshaw, Kirk R. Smith, and Robert van der Plas, 8 (2, July 1993): 119-41 "Development, the Environment, and the Social Rate of Discount," Anil Markandya and David W. Pearce, 6 (2, July 1991): 137-52 "Dumping and Antidumping: The Rhetoric and the Reality of Protection in Industrial Countries," J. Michael Finger, 7 (2, July 1992): 121-43 "The Earmarking of Government Revenue: A Review of Some World Bank Experi- ence," William McCleary, 6 (1, January 1991): 81-104 304 The World Bank Research Observer, vol. 9, no. 2 (July 1994) "Economic Analyses of Rapid Population Growth," Nancy Birdsall, 4 (1, January 1989): 23-50 "The Economics of Malaria Control," Jeffrey S. Hammer, 8 (1, January 1993): 1-22 "The Economics of the Government Budget Constraint," Stanley Fischer and William Easterly, 5 (2, July 1990): 127-42 "Education and Development: A Review," George Psacharopoulos, 3 (1, January 1988): 99-116 "Entrepreneurs and Entrepreneurship in Africa," Walter Elkan, 3 (2, July 1988): 171-88 "Evaluating Social Policies: Principles and U.S. Experience," Jean Baldwin Grossman, 9 (2, July 1994): 159-80 "Exchange Rate Misalignment in Developing Countries," Sebastian Edwards, 4 (1, January 1989): 3-21 "Export Processing Zones: The Economics of Enclave Manufacturing," Peter G. Warr, 4 (1, January 1989): 65-88 "Export-Promoting Trade Strategy: Issues and Evidence," Jagdish N. Bhagwati, 3 (1, January 1988): 27-57 "Feeding Latin America's Children," Philip Musgrove, 8 (1, January 1993): 23-45 "Female Labor Force Participation: An International Perspective," George Psacharopoulos and Zafiris Tzannatos, 4 (2, July 1989): 187-201 "Fiscal Deficits and Macroeconomic Performance in Developing Countries," William Easterly and Klaus Schmidt-Hebbel, 8 (2, July 1993): 211-37 "Foreign Trade Regimes and Economic Growth in Developing Countries," Deepak Lal and Sarath Rajapatirana, 2 (2, July 1987): 189-217 "Government Spending in Developing Countries: Trends, Causes, and Consequences," David L. Lindauer and Ann D. Velenchik, 7 (1, January 1992): 59-79 "Government Wage Policy in Africa: Some Findings and Policy Issues," David L. Lindauer, Oey Astra Meesook, and Parita Suebsaeng, 3 (1, January 1988): 1-25 "How Do Market Failures Justify Interventions in Rural Credit Markets?" Timothy Besley, 9 (1, January 1994): 27-47. "How Much Does Trade with the South Affect Workers in the North?" Adrian Wood, 6 (1, January 1991): 19-36 "The Impact of Agricultural Extension: The Training and Visit System in India," Gershon Feder and Roger Slade, 1 (2, July 1986): 139-61 "The Impact of EC-92 on Trade in Developing Countries," A. J. Hughes Hallett, 9 (1, January 1994): 121-46. "Incomplete Markets and Commodity-Linked Finance in Developing Countries," Robert J. Myers, 7 (1, January 1992): 79-95 Cumulative Index, 1986-94 305 "Indexation and Stabilization: Theory and Experience," Paul D. McNelis, 3 (2, July 1988): 157-69 "Inflation and the Costs of Stabilization: Historical and Recent Experiences and Policy Lessons," Andres Solimano, 5 (2, July 1990): 167-85 "The Interwar Debt Crisis and Its Aftermath," Barry Eichengreen and Richard Portes, 5 (1, January 1990): 69-94 "Is European Integration Bad News for Developing Countries? A Comment on Hughes Hallett," Gerhard Pohl and Piritta Sorsa, 9 (1, January 1994): 147-55. "Issues in Medium-Term Macroeconomic Adjustment," Stanley Fischer, 1 (2, July 1986): 163-82 "Land Rights Systems and Agricultural Development in Sub-Saharan Africa," Gershon Feder and Raymond Noronha, 2 (2, July 1987): 143-69 "Lessons from the Southern Cone Policy Reforms," Vittorio Corbo and Jaime de Melo, 2 (2, July 1987): 111-42 "Macroeconomic Adjustment in Developing Countries," W. Max Corden, 4 (1, Janu- ary 1989): 51-64 "Macroeconomic Adjustment in Developing Countries: A Policy Perspective," Mohsin S. Khan, 2 (1, January 1987): 23-42 "Macroeconomic Policy and Trade Liberalization: Some Guidelines," Michael Mussa, 2 (1, January 1987): 61-77 "The Management of Common Property Resources: Finding a Cooperative Solution," Robert Wade, 2 (2, July 1987): 219-34 "Management of Public Industrial Enterprises," Mahmood A. Ayub and Sven 0. Hegstad, 2 (1, January 1987): 79-101 "Market Behavior of Grains Exporters," Donald 0. Mitchell and Ronald C. Duncan, 2 (1, January 1987): 3-21 "The Market for Developing Country Debt: The Nature and Importance of Its Short- comings," John Wakeman-Linn, 6 (2, July 1991): 191-203 "The Multifibre Arrangement and Its Effects on Developing Countries," Junichi Goto, 4 (2, July 1989): 203-27 "A Not-So-Simple Alternative to the Income Tax for Socialist Economies in Transi- tion: A Comment on McLure," Alan A. Tait, 7 (2, July 1992): 239-48 "On 'Hunger and Public Action': A Review Article on the Book by Jean Dreze and Amartya Sen," Martin Ravallion, 7 (1, January 1992): 1-17 "Options for Dismantling Trade Restrictions in Developing Countries," Wendy E. Takacs, 5 (1, January 1990): 25-46 "Organized Labor, Politics, and Labor Market Flexibility in Developing Countries," Joan M. Nelson, 6 (1, January 1991): 37-56 306 The World Bank Research Observer, vol.9, no.2 (July 1994) "Policy Instruments for Pollution Control in Developing Countries," Gunnar S. Eskeland and Emmanuel Jimenez, 7 (2, July 1992): 145-69 "Preferential Credit and Insurance as Means to Promote Exports," Bruce Fitzgerald and Terry Monson, 4 (1, January 1989): 89-114 "Private Investment and Macroeconomic Adjustment: A Survey," Luis Serven and Andres Solimano, 7 (1, January 1992): 95-114 "Privatization: Lessons from Market Economies," Sunita Kikeri, John Nellis, and Mary Shirley, 9 (2, July 1994): 241-72 "The Public Subsidization of Education and Health in Developing Countries: A Review of Equity and Efficiency," Emmanuel Jimenez, 1 (1, January 1986): 111-29 "Reaching the Rural Poor through Public Employment: Arguments, Evidence, and Les- sons from South Asia," Martin Ravallion, 6 (2, July 1991): 153-75 "Recent Lessons of Development," Lawrence H. Summers and Vinod Thomas, 8 (2, July 1993): 241-54 "Reform of Trade Policy: Recent Evidence from Theory and Practice," Vinod Thomas and John Nash, 6 (2, July 1991): 219-40 "Reforming Finance in Transitional Socialist Economies," Gerard Caprio, Jr. and Ross Levine, 9 (1, January 1994): 1-24 "Reforming Property Rights in Land and Tenancy," Clive Bell, 5 (2, July 1990): 143-66 "The Relative Efficiency of Private and Public Schools in Developing Countries," Em- manuel Jimenez, Marlaine E. Lockheed, and Vicente Paqueo, 6 (2, July 1991): 205-18 "The Relevance for Developing Countries of Recent Developments in Macroeconomic Theory," W. Max Corden, 2 (2, July 1987): 171-88 "The Role of Groups and Credit Cooperatives in Rural Lending," Monika Huppi and Gershon Feder, 5 (2, July 1990): 187-204 "The Roles of the Private and Public Sectors in Enhancing the Performance of Seed Systems," Steven Jaffee and Jitendra Srivastava, 9 (1, January 1994): 97-117 "Shelter Strategies for the Urban Poor in Developing Countries," Stephen K. Mayo, Stephen Malpezzi, and David J. Gross, 1 (2, July 1986): 183-203 "A Simpler Consumption-Based Alternative to the Income Tax for Socialist Economies in Transition," Charles E. McLure, Jr., 7 (2, July 1992): 221-37 "Social Security and the Poor: Choices for Developing Countries," Ehtisham Ahmad, 6 (1, January 1991): 105-27 "Some Theoretical Aspects of Agricultural Policies," Joseph E. Stiglitz, 2 (1, January 1987): 43-60 "South-South Trade: Theory, Evidence, and Policy," David Greenaway and Chris Milner, 5 (1, January 1990): 47-68 Cumulative Index, 1986-94 307 "Stabilization Policies in Developing Countries," Liaquat Ahamed, 1 (1, January 1986): 79-110 "The State and the Invisible Hand," Stanislaw Wellisz and Ronald Findlay, 3 (1, Jan- uary 1988): 59-80 "Student Loans: An Effective Instrument for Cost Recovery in Higher Education?" Douglas Albrecht and Adrian Ziderman, 8 (1, January 1993): 71-90 "A Survey of the Costs of World Sugar Policies," Brent Borrell and Ronald C. Duncan, 7 (2, July 1992): 171-94 "Technological Change and Commercialization in Agriculture: The Effect on the Poor," Hans Binswanger and Joachim von Braun, 6 (1, January 1991): 57-80 "Technological Priorities for Farming in Sub-Saharan Africa," Hans Binswanger and Prabhu Pingali, 3 (1, January 1988): 81-98 "The Three Faces of the International Comparison Project," Irving B. Kravis, 1 (1, Jan- uary 1986): 3-26 "Trade Policy and Productivity Gains in Developing Countries: A Survey of the Liter- ature," Oli Havrylyshyn, 5 (1, January 1990): 1-24 "Urban Labor Markets and Development," Subbiah Kannappan, 3 (2, July 1988): 189-206 "Using Auctions to Allocate and Price Long-Term Credit," J. Luis Guasch and Thomas Glaessner, 8 (2, July 1993): 169-94 "Using Randomized Control Designs in Evaluating Social Sector Programs in Devel- oping Countries," John Newman, Laura Rawlings, and Paul Gertler, 9 (2, July 1994): 181-201 "The Value-Added Tax and Developing Countries," Carl Shoup, 3 (2, July 1988): 139-56 "Wage Controls during the Transition from Central Planning to a Market Economy," Coricelli, Fabrizio, and Timothy D. Lane, 8 (2, July 1993): 195-210 "What Do We Know about the Political Economy of Economic Policy Reform," Stephan Haggard and Steven B. Webb, 8 (2, July 1993): 143-68 "What Makes Rural Finance Institutions Successful?" Jacob Yaron, 9 (1, January 1994): 49-70 "When Do Heterodox Stabilization Programs Work? Lessons from Experience," Miguel A. Kiguel and Nissan Liviatan, 7 (1, January 1992): 35-59 "Who Wins and Who Loses from Voluntary Export Restraints? The Case of Foot- wear," Carl B. Hamilton, Jaime de Melo, and L. Alan Winters, 7 (1, January 1992): 17-35 308 TheWorldBankResearch Observer, vol.9,no.2 (July1994) -~~~ p World Development Report 1994: Infrastructure for Development This seventeenth edition of the World Development Report focuses on essential services-transport, telecommunications, water and sanitation, power and gas, and major water works. The Report examines the link between infrastructure and development and explores ways in which developing countries can improve both the quantity and quality of infrastructure services. The Report identifies inadequate incentives for providers as a cause of poor past performance. It recommends modifying incentives, the key to improvement, through the application of three principles: * Commercial Management of Infrastructure * Increased Competition * Expanded User Involvement World Development Indicators 1994 is also included-statistical tables provid- ing instant access to the most comprehensive and current data available on social and economic development in over 200 economies. 254 pages. Stock No. 60991 (English). Hardcover, $37.95. Stock No. 60992 (English), Stock No. 12534 (French), Stock No. 12535 (Spanish). Paperback, $19.95. Available in September: Stock No. 12539(Arabic), Stock No. 12537 (Chinese), Stock No. 12536 (German), Stock No. 12538 (Japanese), Stock No. 12541 (Portuguese), Stock No. 12540 (Russian). Paperback, $19.95. World Development Indicators 1994 - Data on Diskette. Each package includes a user's guide to the *STARS* retrieval system and a 3-1/2 double-density diskette for PCs with a hard disk, at least 512K RAM, and MS-DOS version of 2.1 or higher. Stock No. 12577. $70.00. New From the World Banzk Adjustment in Africa: Reforms, Results, and the Road Ahead A World Bank Policy Research Report To reverse the economic decline that began in the 1970s, many Sub-Saharan countries initiated programs to pave the way for long-term development and prosperity by restructuring their economies. This report addresses just how much those countries changed their policies, the extent to which their policy reforms restored growth, and the future for adjust- ment. The study brings together a wealth of data to assess reform in the macroeconomic framework, trade liberalization, market and price deregu- lation, public enterprise privatization, and improvements in financial and public sector management. The report recognizes that adjustment can work in Africa but that it cannot work miracles in reducing poverty or ensuring sustained, equitable growth. African adjustments must go hand in hand with long-term develop- ment efforts to invest more in human capital and infrastructure, expand institutional capacity, and provide better governance. Publishedfor the World Bank by Oxford University Press 1994. 304 pages. ISBN 0- 19-520994-X (English), ISBN 0-8213-2530-2 (French). $19.95 Also available: Adjustment in Africa: A Summary 1994. 24 pages. ISBN 0-8213-2795-X (English), ISBN 0-8213-2796-8 (French). $6.95 To order, please fill out the coupon that follows. World Bank Publications Order Coupon CUSTOMERS IN THE UNITED STATES: CUSTOMERS OUTSIDE THE UNITED STATES: Complete this coupon and return to Contact your local World Bank Publications distributor The World Bank for information on prices in local currency and payment Box 7247-8619 terms. (See opposite page for a complete list of Philadelphia, PA 19170-8619 distributors.) If no distributor is listed for your country, U.S.A. use this order form and return it to the U.S. address. Charge by credit card by calling (202) 473-1155 or Orders that are sent to the U.S. address from fax this completed order coupon to (202) 676-0581. countries with distributors will be returned to the customer. Quantity Title Stock Number Price Total Subtotal US$ * If a purchase order is used, actual postage will Postage and handling* US$ be charged. If payment is by check or credit card, postage and handling charges are Total US$ US$5.00 per order. For air mail delivery outside the U.S., include US$8.00 for the first item and US$6.00 for each additional item. CHECK METHOD OF PAYMENT ] Enclosed is my check payable to The World Bank. O Charge my O VISA O MasterCard [I American Express Credit card account number Expiration date Signature [O Bill me. (Institutional customers only. Purchase order must be included.) PLEASE PRINT CLEARLY Name Firm Address City State Postal code Country Telephone 1048 Distributors of World Bank Publications Prices and terms vary by country ARGENTINA CHINA GREECE Carlos Hirsch, SRL China Financial & Economic Papasotiriou S.A. Galeria Guemes Publishing House 35, Stournara Str. Florida 165, 4th Floor-Ofc. 453/ 8, Da Fo Si Dong Jie 106 82 Athens 465 Beijing 1333 Buenos Aires HONG KONG, MACAO COLOMBIA Asia 2000 Ltd. Oficina del Libro Intemacional Infoenlace Ltda. 46-48 Wyndham Street Alberti 40 Apartado Aereo 34270 Winning Centre 1082 Buenos Aires Bogota D.E. 7th Floor Central Hong Kong AUSTRALIA, COTE D'IVOIRE PAPUA NEW GUINEA, Centre d'Edition et de HUNGARY FIJI, SOLOMON ISLANDS, Diffusion Foundation for Market VANUATU, AND Africaines (CEDA) Economy WESTERN SAMOA 04 B.P. 541 Dombovari Ut 17-19 D.A. Information Services Abidjan 04 Plateau H-1117 Budapest 648 Whitehorse Road Mitcham 3132 CYPRUS INDIA Victoria Center of Applied Research Allied Publishers Private Ltd. Cyprus College 751 Mount Road AUSTRIA 6, Diogenes Street, Engomi Madras - 600 002 Gerold and Co. P.O. Box 2006 Graben 31 Nicosia Branch offices: A-1011 Wien 15 J.N. Heredia Marg DENMARK Ballard Estate BANGLADESH SamfundsLitteratur Bombay - 400 038 Micro Industries Development Rosenoerns Alle 11 Assistance Society (MIDAS) DK-1970 Frederiksberg C 13/14 Asaf Ali Road House 5, Road 16 New Delhi - 110 002 Dhanmondi R/Area DOMINICAN REPUBLIC Dhaka 1209 Editora Taller, C. por A. 17 Chittaranjan Avenue Restauraci6n e Isabel la Calcutta - 700 072 Branch offices: Cat6lica 309 Pine View, 1st Floor Apartado de Correos 2190 Z-1 Jayadeva Hostel Building 100 Agrabad Commercial Santo Domingo 5th Main Road, Gandhinagar Area Bangalore - 560 009 Chittagong 4100 EGYPT, ARAB REPUBLIC OF Al Ahram 3-5-1129 Kachiguda 76, K.D.A. Avenue Al Galaa Street Cross Road Kulna 9100 Cairo Hyderabad - 500 027 BELGIUM The Middle East Observer Prarthana Flats, 1st Floor Jean De Lannoy 41, Sherif Street Thakore Baug, Navrangpura Av. du Roi 202 Cairo Ahmedabad - 380 009 1060 Brussels FINLAND Patiala House BRAZIL Akateeminen Kirjakauppa 16-A Ashok Marg Publicacoes Tecnical P.O. Box 128 Lucknow - 226 001 hntemationael Ltda. SF-00101 Helsinki 10 Rua Pexoto Gomide, 209 81 Hill Road 01409 Sao Paulo, SP FRANCE Nagpur 440 010 World Bank Publications CANADA 66, avenue d'Iena INDONESIA Le Diffuseur 75116 Paris Pt. Indira Limited 151A Boul. de Mortagne Jalan Borobudur 20 Boucherville, Qu6bec GERMANY P.O. Box 181 J4B 5E6 UNO-Verlag Jakarta 10320 Poppelsdorfer Allee 55 Renouf Publishing Co. 53115 Bonn IRAN 1294 Algoma Road Kowkab Publishers Ottawa, Ontario KlB 3W8 P.O. Box 19575-511 Tehran IRELAND NIGERIA For subscription orders: Government Supplies Agency University Press Limited International Subscription Service 4-5 Harcourt Road Three Crowns Building Jericho P.O. Box 41095 Dublin 2 Private Mail Bag 5095 Craighall lbadan Johannesburg 2024 ISRAEL Yozmot Literature Ltd. NORWAY SPAIN P.O. Box 56055 Narvesen Information Center Mundi-Prensa Libros, S.A. Tel Aviv 61560 Book Department Castello 37 P.O. Box 6125 Etterstad 28001 Madrid ITALY N-0602 Oslo 6 Licosa Commissionaria Sansoni Libreria Intemacional AEDOS SPA PAKISTAN Consell de Cent, 391 Via Duca Di Calabria, 1/1 Mirza Book Agency 08009 Barcelona Casella Postale 552 65, Shahrah-e-Quaid-e-Azam 50125 Firenze P.O. Box No. 729 SRI LANKA & THE MALDIVES Lahore 54000 Lake House Bookshop JAMAICA P.O. Box 244 Ian Randle Publishers Ltd. PERU 100, Sir Chittampalam A. 206 Old Hope Road Editorial Desarrollo SA Gardiner Mawatha Kingston 6 Apartado 3824 Colombo 2 Lirna 1 JAPAN SWEDEN Eastern Book Service PHILIPPINES For single titles: Hongo 3-Chome, Bunkyo-ku 113 International Book Center Fritzes Fackboksforetaget Tokyo Suite 1703, Cityland 10 Regeringsgatan 12, Box 16356 Condominium Tower 1 S-103 27 Stockholm KENYA Ayala Avenue, H.V. dela Africa Book Service (E.A.) Ltd. Costa Extension For subscription orders: Quaran House, Mfangano St. Makati, Metro Manila Wennergren-Williams AB P.O. Box 45245 P. 0. Box 1305 Nairobi POLAND S-171 25 Solna International Publishing Service KOREA, REPUBLIC OF Ul. Piekna 31/37 SWITZERLAND Pan Korea Book Corporation 00-677 Warszawa For single titles: P.O. Box 101, Kwangwhamun Librairie Payot Seoul For subscription orders: Case postale 3212 IPS Journals CH 1002 Lausanne Korean Stock Book Centre Ul. Okrezna 3 P.O. Box 34 02-916 Warszawa For subscription orders: Yeoeida Librairie Payot Seoul PORTUGAL Service des Abonnements Livraria Portugal Case postale 3312 MALAYSIA Rua Do Carmo 70-74 CH 1002 Lausanne University of Malaya Cooperative 1200 Lisbon Bookshop, Limited THAILAND P.O. Box 1127, Jalan PantaiBaru SAUDI ARABIA, QATAR Central Department Store 59700 Kuala Lumpur Jarir Book Store 306 Silom Road P.O. Box 3196 Bangkok MEXICO Riyadh 11471 INFOTEC TRINIDAD & TOBAGO, Apartado Postal 22-860 SINGAPORE, TAIWAN, Systematics Studies Unit 14060 Tlalpan, Mexico D.F. MYANMAR,BRUNEI #9 Watts Street Gower Asia Pacific Pte Ltd. Curepe NETHERLANDS Golden Wheel Building Trinidad, West Indies De Lindeboom/InOr-Publikaties 41, Kallang Pudding, #04-03 P.O. Box 202 Singapore 1334 UNITED KINGDOM 7480 AE Haaksbergen Microinfo Ltd. SOUTH AFRICA, BOTSWANA P.O. Box 3 NEW ZEALAND For single titles: Alton, Hampshire GU34 2PG EBSCO NZ Ltd. Oxford University Press England Private Mail Bag 99914 Southern Africa New Market P.O. Box 1141 ZIMBABWE Auckland Cape Town 8000 Longman Zimbabwe (Pte.) Ltd. P.O. Box ST 125 Southerton Harare