WPS4993 Policy Research Working Paper 4993 Impact Evaluation Series No. 31 Evaluating the Impact of Egyptian Social Fund for Development Programs Hala Abou-Ali Hesham El-Azony Heba El-Laithy Jonathan Haughton Shahidur R. Khandker The World Bank Development Research Group Sustainable Rural and Urban Development Team July 2009 Policy Research Working Paper 4993 Abstract The Egyptian Social Fund for Development was illiteracy, health and potable water programs have lowered established in 1991 with a mandate to reduce poverty. household spending on health, sanitation interventions Since its inception, it has disbursed about $2.5 billion, have cut household spending on sanitation and lowered of which nearly two-fifths was devoted to supporting poverty, and road projects have reduced household microcredit and financing community development transportation costs by 20 percent. Microcredit is and infrastructure. This paper investigates the size of the associated with higher household expenditures in impact of the Fund's interventions, whether the benefits metropolitan areas and urban Upper Egypt, but not have been commensurate with the costs, and whether elsewhere. the programs have been targeted successfully to the poor. The Social Fund for Development's road projects The core of the impact evaluation applies propensity- generate benefits that, by some estimates, exceed the score matching to data from the 2004/2005 national costs, as do health and potable water interventions; this is Household Income, Expenditure and Consumption less evident for interventions in education and sanitation. Survey. The Fund argues that its mission is primarily social, and The authors find that Egypt's Social Fund for so should not be judged using a cost-benefit analysis. The Development programs have had clear and measurable Fund support for microcredit is strongly pro-poor; the effects, in the expected direction, for all of the programs other programs analyzed have a more modest pro-poor considered: educational interventions have reduced orientation. This paper--a product of the a product of the Sustainable Rural and Urban Development Team, Development Research Group--is part of a larger effort in the department to understand the cost-effectiveness of rural development interventions such as social fund of Egypt. Policy Research Working Papers are also posted on the Web at http://econ.worldbank.org. The author may be contacted at skhandker@worldbank.org. The Impact Evaluation Series has been established in recognition of the importance of impact evaluation studies for World Bank operations and for development in general. The series serves as a vehicle for the dissemination of findings of those studies. Papers in this series are part of the Bank's Policy Research Working Paper Series. The papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors. They do not necessarily represent the views of the International Bank for Reconstruction and Development/World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank or the governments they represent. Produced by the Research Support Team Evaluating the Impact of Egyptian Social Fund for Development Programs Hala Abou-Ali Cairo University Hesham El-Azony Egyptian Social Fund for Development Heba El-Laithy Cairo University Jonathan Haughton Suffolk University, Boston Shahidur R. Khandker 1 World Bank, Washington DC Disclaimer: The findings and opinions in this paper do not necessarily reflect those of the institutions with which the authors are affiliated. 1 Please send correspondence to skhandker@worldbank.org . Introduction Social funds have been defined as "agencies that finance small projects in several sectors targeted to benefit a country's poor and vulnerable groups based on a participatory manner of demand" (Jorgensen and Van Domelen 1999). After the first social fund was set up in Bolivia in 1987, the idea attracted immediate support, and by May 2001 the World Bank had provided financing to more than 98 social funds in 58 countries. This is the context in which the Egyptian Social Fund for Development (SFD) was established in 1991, when the government sought to widen the social safety net in the wake of an economic reform and social adjustment program. Run by a semi-autonomous agency that reports to the office of the Prime Minister, the SFD is one of the three main safety net programs in Egypt, along with the system of subsidies of basic foodstuffs, and the Ministry of Social Solidarity's cash transfer program. The mandate of the SFD is to (i) reduce poverty by supporting community-level initiatives, (ii) increase employment opportunities, and (iii) encourage small-enterprise development. In doing this, it is tasked with mobilizing national and international resources, and cooperating with governmental bodies, NGOs, and community and private sector groups. Since its inception, the SFD had disbursed a total of LE 11.4 billion as of the end of the first quarter of 2008, equivalent to approximately US$2.5 billion; this represents about US$2.36 per person per year 2, or about 0.15 percent of GDP. An estimated 32% of Egyptians are potential beneficiaries from SFD projects, in the sense that they live in villages or wards where there is at least one SFD intervention. Of its total spending, the SFD devoted 62 percent to supporting the development of small enterprises, 9 percent to microcredit, and the remaining 28 percent to financing community infrastructure. The SFD acts as a source of funding for projects, which are implemented by other agencies. The focus of this paper is on SFD's activities in community infrastructure and microcredit. Broadly, the question of interest is whether the SFD has had an impact on living standards. More specifically, our purpose is to measure the effects of SFD interventions on the immediate outputs (such as the number of households connected to sewers), on the short-term direct outcomes (for instance, the under-five mortality rate), and on the long-term impact on poverty, illiteracy, and employment. 2 Source for population information: UN World Population Prospects: The 2006 Revision. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 2 of 36 We also address a number of other interesting issues, such as whether SFD interventions are well targeted to the poor ­ a subject explored in more detail in ESFD (2006) ­ or cost effective (see Rawlings et al. 2004, p.v), and whether the benefits of SFD interventions exceed the costs. The cost-benefit analysis should not, however, be viewed as an evaluation of the SFD, which was tasked with pursuing a number of social objectives rather than ensuring that benefits exceeded costs. The usual justification for undertaking an impact evaluation applies here: as a matter of public policy we would like to know how effective the SFD has been, and hence if its efforts should be expanded or reduced, or its programs reoriented or redesigned. There is a dearth of such evaluations; a recent multi- donor review mission wrote that "SFD needs to ... above all ­ deepen its attention to practical policy analysis and impact assessment." In what follows we first set out the essential features of the SFD's activities in microcredit and community infrastructure (comprising public works, and community development), outline the methodology used to measure the impact of these programs, discuss the viability of the data, present the results, weigh the costs against the benefits of SFD programs, and evaluate the extent to which the interventions were targeted to the poor. The Social Fund for Development The main programs of the SFD are set out in Table 1. In this paper we seek to measure the impact of interventions in education, health, potable water, sanitation, road maintenance, and microcredit. Collectively these represent disbursements of LE 3 billion over the period 1991 ­ 2007, or just over a quarter of all SFD spending. The remaining spending was either on minor programs or administration, or went to the Small Enterprise Development Organization, which we are not in a position to evaluate. The Community Development Program is intended to promote projects that alleviate poverty. It is targeted geographically to relatively poor parts of the country, where it funds literacy classes, supports primary health care through the renovation of health units and training, and finances small environmental projects, including the digging of latrines, garbage collection, and tree planting. In targeting poor areas, the SFD uses its network of 27 regional offices, which help identify local needs, mobilize community participation, and monitor the progress of projects. The Community Development Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 3 of 36 Program activities work with long-established partners that include the Ministry of Health and Population, and the General Authority for Literacy and Adult Education. The Public Works Program has provided substantial funding for the extension of public potable water networks, sewage treatment and connections, and the construction and maintenance of paved roads. It also finances the restoration and construction of some public buildings such as health units and youth centers, and environmental projects that include covering watercourses, stabilizing river banks, and collecting garbage. This program works closely with existing government units both for project identification and implementation, although much of the actual work on the projects is done by local private contractors. Table 1: Principal Programs and Activities of the Social Fund for Development, 1991-2007 Main Program Main activity / Activities, 1991 ­ 2007 Evaluated in Disbursements, this paper? 1991-2007 Community Health Renovated health units (840) Yes Development LE 390 million Health awareness campaigns Training of health personnel Education Illiteracy classes (1.5 million people enrolled) Yes LE 200 million Single classroom schools Community schools Training of teachers for literacy classes Other Public buildings (rehabilitation; youth and community No LE 210 million centers) Environment (covering waterways, weeding, river bank protection) Public Works Potable Water Public networks (8,480 km) Yes LE 620 million Ground tanks (375) Artesian wells (440) Shelter houses (200) Sanitation Public networks (459) Yes LE 220 million Treatment plants (16) Pumping stations (38) House connections (19,500) Roads Construction and maintenance of rural roads (2,500 km) Yes LE 500 million Other NGO capacity building No LE 660 million Environment (latrines, garbage collection, draining, pest spraying, tree planting) Training Microcredit LE 1,062 Loans (575,000) Yes million Small LE 7,100 Support for small enterprises, including loans and technical No Enterprises million support. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 4 of 36 The micro-credit program seeks to encourage the supply of loans to low-income households. It provides working capital to some financial intermediaries, such as Community Development Associations, Productive Family Associations, and the Principal Bank for Development and Agricultural Credit; in other cases it gives grants to financial institutions, on condition that they commit to more microlending. The government has been increasing the pressure on the SFD to operate in a way that is financially self- sustaining ­ for instance, by charging fees, or putting more emphasis on microlending at appropriate interest rates. It recently refused a loan from the World Bank that would have provided funds to the SFD, because it was concerned that the return on the spending would not be sufficient to justify the cost of repaying the loan. On the other hand, the government might be more inclined to help the SFD obtain such funding if it found that SFD projects have a strong impact, which returns us again to the importance of evaluating the effects of SFD spending. Measuring Impacts In seeking to measure the impact of SFD programs, we first need to clarify what impacts need to be measured, and then set out an appropriate methodology whereby these impacts may be identified and quantified. There are two broad types of program effect that are of interest to us: short-term outcomes, and long-term impacts. The variables that we use to measure these effects are set out in more detail in Table 2, for each of the six sectors of interest. So, for instance, in evaluating projects under the education program, we might be interested in the proportion of participants in literacy classes who obtained diplomas (an outcome), and the effect on the illiteracy rate or employment rate (impacts). We would like to think that SFD interventions would have unambiguously positive effects: more education would reduce the illiteracy rate; health projects should reduce mortality rates and poverty, as should more potable water and improved sanitation; better roads, and more microcredit, should increase incomes and lower poverty. However, experience from other evaluations of social funds suggests that improvements such as these cannot be taken for granted. For instance, Rawlings et al. (2004), in a comparative study of social funds in six countries, found that while interventions led to greater access to and use of basic services, tangible improvements in welfare were sometimes elusive. For instance, educational projects did not raise test Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 5 of 36 scores or primary school enrollment rates in rural Bolivia, although they did raise enrollment rates in Nicaragua and Zambia. Similarly, spending on primary health centers reduced infant mortality in Bolivia, but had small or no discernible effects on health outcomes in other countries. And sanitation projects led to health gains in Peru and Bolivia but not in Honduras. Table 2. Measures of Inputs, Outputs, Outcomes, and Impacts Inputs Outputs Outcomes Impacts (short-term) (long-run) Education Money # of class rooms established # (percentage) of the Illiteracy rate # of individuals trained to teach # of beneficiaries participants getting the Employment rate # of nursery classes intervention Health Renovation of health clinics # of beneficiaries Rate of immunization # of health personal trained # of health clinics renovated Under 5 mortality rate # of health awareness campaigns as percentage of total Average expenditure on health clinics medicine # of mothers that received pre-natal care Potable water Length of pipe lines constructed / # of household connected as # under 5 mortality rate rehabilitated a % of total of household Incidence of diarrhea # of elevated or ground tanks in the community Incidence of illnesses due to Expenditure on water water related disease procurement Expenditure on medicine Sewerage Length of sewerage network # of household connected Mortality connection # of waste water treatment plant Expenditure on sanitation/ Morbidity septic tanks Diarrhea Expenditure on health Roads Length of rural roads constructed/ Transportation expenses Employment rehabilitated reduction Farm income/ non farm Mode of transport income Travel time to reach Under 5 mortality clinic/school School enrolment rate Microcredit # of NGOs receiving loans from Average amount loans Diversification of income Poverty the SFD extended (received) by Average income Amount of money received from SFD NGOs Employment level the SFD # of beneficiaries receiving Diversification of loans employment Where projects have no discernible overall impact, the explanation is typically that the money was poorly spent (e.g. on a shiny new school to replace an adequate older building, or on a road to nowhere), or crowded out existing services (e.g. a new health center might change where one goes for care, but may not raise the level of care appreciably). The simplest approach to measuring the effects of a program is to compare the outcomes for those who were covered by the program with those who were not. Unfortunately, this does not work, because of the problem of endogenous program placement. For example, the rate of gastroenteritis is 6.9% in villages with an SFD health program but just 2.7% for communities without one; yet it would not be credible to claim that an SFD health program actually causes more gastroenteritis. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 6 of 36 The difficulty is that the SFD has, for some of its programs, targeted poorer areas, where many of the outcomes of interest (literacy rates, mortality rates, poverty rates, expenditure levels) are inherently worse. This also shows up in Table 3, which shows expenditure levels and poverty rates for villages with and without the program interventions of interest. Thus we see that household expenditure per capita levels are lower in localities with an SFD program; and the poverty rate is 30% in villages where the SFD had a program to provide potable water, compared with a poverty rate of 18% elsewhere in Egypt. Table 3. Poverty Rates and Expenditure Levels for Areas With, and Without, SFD Programs Without any SFD With at least one All program SFD program Per capita annual expenditure 2,680 2,292 2,556 Per capita annual food expenditure 1,165 1,076 1,127 Headcount poverty rate (%): Overall 18.6 21.6 19.6 If there was an SFD intervention in Education 19.8 15.0 19.6 Health 19.5 22.1 19.6 Microcredit 18.8 23.1 19.6 Paving roads 19.5 21.8 19.6 Potable water 18.2 30.2 19.6 Waste water 19.6 12.8 19.6 Ranked by expenditure per capita Poorest 40% Middle 40% Top 20% % having any SFD intervention Urban 10.8 10.4 7.6 Rural 9.9 9.9 10.5 Source: Authors' calculations based on HIECS and community survey 2004/2005. There are a number of possible solutions to the problem of endogenous program placement. In most of what follows we use propensity score matching, but we check the robustness of our results using a common impact model. Some further explanation of these approaches is in order. Propensity Score Matching Even if the placement of SFD programs ("treatments") has not been done randomly, it may be possible to measure the impact of the program by using matched comparisons. The idea is to match each participant, in this case a village, with an otherwise identical non-participant (the comparator) ­ based on observed pre-treatment or exogenous characteristics ­ and then to measure the average difference in the outcome variable between them. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 7 of 36 Units that cannot be matched are discarded ­ this is central to good matching ­ because they "cannot support causal inferences about missing potential outcomes" (Diamond 2005, p.9). The hope is that this allows one to mimic the effects of randomization. Of course, the resulting measure of impact is only compelling to the extent that one believes that the matching has been done well and the treatment assignment is ignorable; in other words, we know that the treatment was not assigned randomly, but we believe that we may proceed as if it were. To match treatment with non-treatment units it is helpful to create a summary measure of similarity in the form of a propensity score. Let p(Xi) be the probability that unit i be assigned to the treatment group, conditional on Xi, and define p( X i ) Pr(Ti = 1 | X i ) = E (Ti | X i ). (1) This probability of participation ­ the propensity score ­ can be estimated using an assignment model. Given survey and census information, we first pool the two samples (i.e. the participants and non- participants) and estimate a probit model of program participation as a function of pre-treatment and exogenous variables that might influence participation. The computation of propensity scores is only the first step in the process. Rosenbaum and Rubin (1983) prove that treatment cases may be matched with comparison cases using just the propensity score rather than the entire set of pre-determined covariates Xi. In other words, to find the non-participant that is most closely matched to the participant, one only needs to find the non-participants with the propensity scores closest to that of the participant. They also show that G TT = E p ( X ) [Gi |T =1, p ( X ) | Ti = 1], (2) where Gi |T =1, p ( x ) is the difference between the treatment outcome YiT for treated unit i and the (control) outcome for the non-treated unit closest in propensity score to i. In other words, the average treatment effect (the gain for the treated, here denoted by GTT) may be obtained by computing the expected value of the difference in the outcome variable between each treated household and the perfectly matched comparison household as matched using the propensity score. Perfect matching is not possible in reality, so in practice one needs to compute (3) Yi - Y j , 1 1 Gi |T =1 = ^ N iN Ji jJ i where Yi is the observed outcome for the ith individual who is treated and Ji is the set of comparators for i, and N is the set of units for which the set of comparators is non-empty (the "common support", discussed Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 8 of 36 in more detail below). With nearest neighbor matching one chooses the m closest comparators; we follow the common practice of using m=1. As a robustness check, we also use kernel matching, which puts more weight on closer comparators than those that are more distant. Dehejia and Wahba (2002) argue that the choice of matching mechanism is not as crucial as the proper estimation of the propensity scores, but this is not a settled issue. In practice, the plausibility of propensity score matching depends on ensuring "common support" and "balancing." Only in the area of common support is it possible to make comparisons that allow us to make inferences about causality (Rubin and Waterman 2006), so our comparisons need to be confined to this area, and an impact evaluation is not possible unless there is an area of common support (Imbens 2004, p.7). The area of common support occurs where the densities of the estimated propensity scores for participants and for non-participants overlap, as shown in Fig. 1. Above point B there are no comparators for borrowers, so matching is not possible in this zone; below point A there are no borrowers that need to be matched. In all the results reported below there was an acceptably broad region of common support. 3 Region of common support 2 Density for non- participants 1 Density for participants 0 0 A .2 .4 .6 B .8 1 Fig. 1. Illustrating the Region of Common Support. In addition, for propensity score matching to work, the treatment and comparison groups need to be "balanced." A treated unit and matched comparator might both have essentially the same propensity scores, but this does not guarantee that they are similar in a relevant way: one village might be well educated but ageing, while another (with a similar propensity score) might be poorly educated but young. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 9 of 36 It is not necessary for every individual match to be close, but it is important for the distributions of covariates for the treated and the comparators to be similar, and this is what is meant by balance. More formally, in order to verify balance we need to check whether p( X | T = 1) = p( X | T = 0) ^ ^ (4) where p gives the empirical (rather than population) density of the data. ^ In line with common practice, the algorithm we use is to estimate a propensity score, match treated with non-treated units, and check for balance; if balance is not achieved, we revise the assignment model, and repeat this process until (hopefully) balance is achieved. In checking for balance, we divide the data into strata, based on the estimated propensity score, and then use a series of t-tests or chi-square tests to check that, within each stratum, the values of each covariate (age, literacy rates, and so on) are on average the same. This allows one to "check the adequacy of the statistical model" of assignment (Imbens 2004, p.18). Impact Models There is another way to deal with the problem of endogenous program placement. It may be possible to assume that SFD programs have been assigned randomly, conditional on some observable variables, X, that might include measures such as village size, or literacy rates, or the structure of the local economy. This would constitute partial randomization, and is a maintained assumption rather than a testable proposition. If we are willing to accept the conditional exogeneity of program placement, it may be possible to estimate the impact of a treatment such as an SFD program using a parametric model with controls. Suppose that we may assume YiT = T + X i T + iT , i = 1,..., n (5a) YiC = C + X i C + iC , i = 1,..., n (5b) where the error terms are normally distributed with zero means and constant variances, and the superscripts refer to the treatment group (T) and the comparison group (C). These two equations are often estimated together in the form of a switching regression using the pooled data from both the treatment and comparison samples, giving Yi = C + ( T - C )Ti + X i C + X i ( T - C )Ti + iT , i = 1,..., n (6) Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 10 of 36 where Ti takes on the values of one or zero and the error term now takes the form i = Ti ( iT - iC ) + iC . If we may assume that the error term (the "latent effects") has zero mean conditional on the X covariates and treatment ­ a reasonable assumption if there is even partial randomization ­ then we have E ( iT | X , T = t ) = E ( iC | X , T = t ) = 0, t = 0,1, and we can get consistent estimates of the average treatment effects by applying ordinary least squares (OLS) to Eqn. 6, and noting that G ATE = E [ T - C + X i ( T - C )]. (7) If we are also willing to assume (more problematically) that T=C then we obtain the common-impact model,where the average treatment effect reduces to T - C . In what we report below we mainly emphasize the results of propensity score matching, but we include the results of the common-impact model both as a robustness check, and as legitimate estimates in their own right. Choosing a Counterfactual In all impact evaluations one is trying to compare the actual impact with a counterfactual, where the latter represents our view of what would have happened in the absence of the project. In the case of SFD projects, a number of possible comparisons are possible, because of the presence of non-SFD interventions by ministries and NGOs. The situation is illustrated in Figure 2, and suggests the following useful comparisons: (i) A vs. B. Any given village either has an SFD intervention or it does not. This comparison of A vs. B provides one possible baseline. The implicit assumption here is that in the absence of an SFD intervention, there is still a substantial probability that there would have been some other intervention ­ most villages without an SFD project have an intervention by some other organization. If, in this comparison, SFD villages (A) perform better than non-SFD villages (B), then it is largely because the SFD projects are more effective than non-SFD projects. This is a relatively high bar, and probably provides a lower bound to the impact of the SFD. (ii) D vs. F. This compares villages that have just an SFD project with villages that have no interventions at all. It is the relevant comparison if, in the absence of an SFD project, there would have been no other project. In that sense it measures the pure effect of the SFD. However, it implicitly assumes that, in the absence of an SFD project, there would have be no comparable Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 11 of 36 project so, for instance, if the SFD had not financed a project to pave the local road, then the road would have remained unpaved. This is an extreme assumption, and is not entirely plausible, because other organizations ­ NGOs, ministries, local self-help groups ­ might well have stepped in if the SFD were not present. Thus the comparison between D and F may be expected to provide an upper bound to the effect of the SFD. (iii) C vs. E. In this case the comparison measures the incremental impact of SFD interventions, given that there are already other interventions. If there are synergies between SFD projects and other interventions ­ for instance, an SFD sewerage project might work best if some other entity is already at work providing piped water ­ then this comparison should pick up this effect. Is there an SFD intervention? Yes: A No: B With other "SFD only": With other No interventions D interventions: interventions ("mixed"): C E at all: F Figure 2. Creating a Counterfactual for SFD interventions In the results reported below, we measure the impact using all three comparisons. But before doing this, some discussion of the data is warranted. Data Considerations Most SFD programs operate at the level of a village (or sub-village), and this is the level at which the analysis is conducted. Although the basic unit of observation is the village, many of the variables ­ per capita income and expenditure levels, for instance ­ are based on household-level data. The main source of such data is the 2004/2005 edition of the national Household Income, Expenditure and Consumption Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 12 of 36 Survey (HIECS), conducted by the Central Agency for Public Mobilization and Statistics (CAPMAS), and which interviewed 47,095 households between July 1, 2004 and June 30, 2005. The sampling is stratified (by governorate), and clustered, with 40 households being interviewed in every sampled village or ward (the primary sampling units). The questionnaire collected basic socioeconomic data, including enough information to establish consumption-based measures of poverty. Three changes were made to the survey to ensure that the data could be used for an impact evaluation. First, a module was added to the survey instrument that included questions about SFD activities and community organizations and initiatives. This supplementary module was administered to all of the 12,000 households surveyed in the fourth quarter of the HIECS survey (i.e. from April-June 2005) ­ each of the four rounds of the survey are nationally representative ­ as well as to all 10,000 sampled households living in areas with SFD interventions. This module gathered information on illiteracy and health programs, the extent of SFD interventions, and the providers and impact of microcredit. Second, an additional 3,760 households in 94 primary sampling units (PSUs) were surveyed in areas where the SFD was active, in order to ensure enough coverage of areas in which the SFD operated; the sampling followed the HIECS protocol, and the questionnaire included the supplementary module, but omitted some of the irrelevant parts of the HIECS questionnaire. The intention was to ensure that strictly comparable survey data would be available for at least 80 treatment villages (PSUs) for each of the seven sub-sectors (education, health, potable water, wastewater, road maintenance, the environment, and microcredit) covered by SFD interventions. The third modification was that a questionnaire on microcredit was administered to a randomly-selected sample drawn from all present borrowers under the SFD program, within PSUs in the HIECS master sample. It should be noted that panel data were not available, so powerful techniques of impact evaluation such as double differencing were not feasible. This is a major, if inescapable, limitation of the current study. The household-level data were supplemented with information from a community survey that was integrated with the HIECS survey, and provided information on the availability, accessibility, and quality of water and sewerage systems, health posts, and primary schools at the level of individual villages and wards (i.e. PSUs). It also sought to measure perceptions of the SFD interventions, and the extent of community participation in the projects. The community survey was also administered in the 94 areas Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 13 of 36 covered by the supplemental survey. At the community level, information was also available from the 2000 census, and this was an important source of pre-determined variables for use in estimating the propensity score equation. Results 1: The Example of Potable Water To illustrate the approach that we take to measuring the impacts of SFD programs, we first present in some detail the results for potable water projects, comparing villages that have both SFD and other ("non- SFD") potable water interventions with villages that only have non-SFD interventions; this is the C vs. E comparison, to use our terminology from the previous section. This allows us to discuss both the methodological and practical issues involved in our estimations. A complete summary of the results for the other SFD programs and comparisons is given in the subsequent section of the paper. Propensity Score Matching. The first step here is to estimate a propensity score equation that satisfies the balancing property. Our procedure was to estimate a probit assignment model, check for balance, and then adjust the model until balance was achieved. The model whose estimates are shown in Table 4 is "balanced", and it is also broadly reasonable: SFD potable water projects were more likely to occur in areas where families were larger, the poverty rate higher, and access to sewers was lower. The region of common support stretches from 0.012 to 0.725, which includes all the SFD project areas and 1,044 of the 1,089 non-SFD areas, and should provide an adequate basis for subsequent matching. Table 4. Estimate of Propensity Score Equation, Potable Water Projects, Comparing "Mixed" SFD and Other Interventions with Non-SFD Interventions Coefficient Standard Error Dependent variable: =1 if village has SFD piped water project, =0 otherwise Households with access to a sewage system in 2004/05 (%) -0.169 0.013 Average household size in 2004/05 0.251 0.007 Female-headed households in 2004/05 (% of all households) 0.577 0.051 PSU had one or more training programs (yes=1) in 2004/05 -0.304 0.014 Illiteracy rate in 2004/05 (%) -0.243 0.007 Headcount poverty rate in 2000 (%) 0.0015 0.0003 Female-headed households in 2000 (% of all households) 0.033 0.001 Intercept -2.548 0.040 Notes. Probit equation; pseudo R2 = 0.082. Data come from 1,291 PSUs (villages and wards), of which 202 have SFD and other interventions, and 1,089 have non-SFD interventions only. The regression was weighted by population size. The region of common support is [0.012, 0.725] and includes all the PSUs with SFD and other interventions and 1,044 of the PSUs with non- SFD interventions. Source: Authors' calculations based on HIECS and community survey 2004/2005. In Tables 5-7 we present a number of estimates of the effect of SFD interventions on the headcount poverty rate, which is one of the long-term impacts of most interest. The first measure is based on a regression equation in which the headcount poverty rate in a village is the dependent variable, and the Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 14 of 36 presence of an SFD intervention is captured as a binary covariate. The regression is confined to the sample of PSUs (villages and wards) in which there are non-SFD projects to supply potable water, and so the measure of impact here should pick up the incremental effect of SFD interventions, given that there are already other interventions. The regression results in Table 5 use only those observations that are in the region of common support, and they weight the observations by the inverse of the propensity scores, as suggested by Imbens (2004, p.16). By this measure, the presence of an SFD piped water project reduces the poverty rate by 0.8 percentage points, an estimate that is significant at the 1% level. Table 5. Estimates of Regression Showing Impact of SFD projects on the Headcount Poverty Rate, Comparing PSUs Having "Mixed" SFD and Other Interventions with PSUs Having only Non-SFD Interventions Coefficient p-value Dependent variable: Headcount poverty rate (%) SFD piped water project in place (1=yes, 0=no) -0.81 0.00 Households with access to a sewage system in 2004/05 (%) -1.53 0.00 Average household size in 2004/05 0.26 0.07 Female-headed households in 2004/05 (% of all households) 1.65 0.08 PSU had one or more training programs (yes=1) in 2004/05 -0.16 0.64 Illiteracy rate in 2004/05 (% illiterate among those older than 10) -0.45 0.01 Headcount poverty rate in 2000 (%) 0.07 0.00 Female-headed households in 2000 (% of all households) 0.06 0.02 Intercept 1.08 0.17 Notes. Observations are weighted by the square root of the inverse of the propensity score (if SFD project is in place) or by square root of the inverse of 1-propensity score (if SFD project is not in place). Adjusted R2 = 0.168. Data come from 1,290 PSUs (villages and wards) for which data were available and fall within the region of common support (as determined by the propensity score equation ­ see Table 4). Source: Authors' calculations based on HIECS and community survey 2004/2005. The results of the propensity score matching are shown in Table 6. The upper panel uses nearest- neighbor matching, and also finds that the poverty rate is 0.8 percentage points lower in villages with SFD projects than in the comparison villages, although the difference is not quite statistically significant. Kernel matching finds that the under-5 mortality rate is about 0.4 percentage points lower in the treatment villages, but here again the difference is not quite statistically significant. Table 7 displays the results of estimating straightforward impact equations. These use observations from the full data set, and not just for villages in the area of common support, and they include a large number of other variables (which reduces the sample size somewhat, because some values are missing). The results of the common-impact model ­ given in the middle columns of Table 7 ­ show that the presence of a SFD piped water project is associated with a 0.5 percentage point reduction in the poverty rate, after controlling for other influences, but the result is not statistically significant. The general-impact model is based on the same variables as used in the common-impact model, but also includes interactions between Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 15 of 36 the presence of the SFD project and these variables; it finds an implausibly large reduction (of 2.2 percentage points) in the poverty rate. Table 6. Results of Propensity Score Matching to Determine the Impact of SFD projects on the Headcount Poverty Rate, Comparing PSUs Having "Mixed" SFD and Other Interventions with PSUs Having only Non- SFD Interventions Number Headcount Analytical Standard Bootstrapped Standard of PSUs Poverty rate (%) Error / (t-statistic) Error / (t-statistic) Nearest neighbor matching Matched treated cases 202 4.07 Matched comparison cases 170 4.83 Treated ­ Comparison -0.76 0.56 (t=-1.37) 0.53 (t=-1.43) Kernel matching Matched treated cases 202 4.08 Matched comparison cases 1,044 4.48 Treated ­ Comparison -0.39 n.a. 0.32 (t=-1.22) Notes: Comparisons use propensity score based on equation in Table 4, and confined to the region of common support, where the propensity score is between 0.020 and 0.725. "Treated cases" refers to PSUs where there are both SFD and non-SFD potable water projects, and these constitute 16% of all 1,246 PSUs in the region of common support. "Comparison cases" refers to PSUs where there are only non-SFD potable water projects. Kernel matching uses Epanechnikov kernel functions with a bandwidth of 0.06. Bootstrapped results are based on 100 draws. Source: Authors' calculations based on HIECS and community survey 2004/2005. A more complete set of estimates of the effects of potable water provision are shown in Table 8; in each case, we are comparing villages where there are both SFD and non-SFD water projects with villages where there are only non-SFD water projects. Surprisingly, the presence of an SFD water supply project does not appear to increase the number of households that have access to piped water, suggesting that for this type of project, the SFD substitutes for, rather than supplements, other potable water projects. On the other hand, when there is an SFD project, households spend less on water than they otherwise would; one possibility is that the SFD may not require as much cost recovery as other suppliers of water. The effects of SFD water projects on health are mixed; they appear to reduce spending on health care, and may reduce the child mortality rate, but the incidence of illness, especially diarrhea and renal disease but perhaps also malaria and bilharzia, actually rises. This merits further investigation; the presence of additional water, if poorly treated or disposed of improperly, could indeed increase the incidence of water-borne disease; if this is the case, then the provision of potable water would need to include provisions that deal with water quality and disposal. Results 2: Summarizing the Impacts of SFD Projects The measured impacts of SFD projects are summarized in Tables 9-14. In each case we only present the relevant measures of impact, and not the complete (and voluminous) regression and other details. The Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 16 of 36 Table 7. Estimates of Impact Equations for SFD Potable Water Projects (In the Presence of Other Potable Water Interventions) on the Headcount Poverty Rate Common-impact model General impact model Coefficient p-value Coefficient p-value Dependent variable: Headcount Poverty Rate (%) SFD piped water project in place (1=yes, 0=no) -0.48 0.13 -19.82 0.04 Average value of durable goods -0.65 0.00 -0.60 0.00 Children as proportion of household members -3.71 0.20 -5.95 0.06 Adult males as proportion of household members -1.36 0.68 -5.48 0.12 Adult females as proportion of household members -5.03 0.16 -6.95 0.07 Proportion of households with access to sewerage (%) -0.83 0.03 -0.73 0.07 Average household size 0.33 0.04 0.40 0.03 Female-headed households (% of households) 1.34 0.18 0.96 0.39 Proportion of children who are working (%) -5.56 0.00 -4.89 0.00 Illiteracy rate (%) 0.89 0.35 0.49 0.65 Prevailing economic activity in PSU is services (yes=1) 0.97 0.12 0.64 0.32 PSU has one or more training programs (yes=1) 0.42 0.16 0.45 0.14 PSU has an illiteracy eradication program (yes=1) -0.59 0.71 0.03 0.88 2000: Employment rate (%) 0.13 0.01 0.02 0.01 2000: Illiteracy rate (%) -0.03 0.17 -0.03 0.18 2000: Headcount poverty rate (%) 0.05 0.00 0.06 0.00 2000: Per capita expenditure 0.001 0.00 0.001 0.02 2000: Per capita food expenditure -0.004 0.02 -0.002 0.15 2000: Female-headed households (% of households) 0.05 0.08 0.01 0.78 2000: Access to piped water (% of households) 0.002 0.62 -0.002 0.64 2000: Access to sanitation (% of households) 0.002 0.65 0.002 0.67 Intercept 8.17 0.01 9.72 0.00 Interaction terms (of variables with presence of SFD piped No Yes water project): Adjusted R2 0.25 0.26 Number of observations (PSUs) 1,267 1,267 Notes: Treatment effect for interactive equation, averaged over all observations: -2.21. Coefficient shown here has been multiplied by 106. Source: Authors' calculations based on HIECS and community survey 2004/2005. Table 8. Estimates of Effects of SFD Potable Water Interventions (in the Presence of Other Potable Water Interventions) Propensity Score Matching Model Impact Model Regression Nearest Kernel Common General Neighbor Matching Impact Impact Outputs Households with access to piped water (%) -0.02 -0.12 -0.02 -0.02 -0.03 Average household spending on potable water -1.02** -1.92* -1.39** -1.21* -1.57 Outcomes (short-term) Under-5 mortality rate (per thousand) -5.94 4.85 -4.07 -5.55 -35.19 Prevalence of diarrhea (%) 2.63** 2.93** 2.37** 2.49** 3.43 Prevalence of malaria (%) 0.04* -0.00 0.04 0.04 0.11 Prevalence of renal disease (%) 0.42** 0.46** 0.46** 0.54** 0.15 Prevalence of bilharzia (schistosomiasis) (%) 0.12* 0.16 0.01 0.14* -0.16 Per capita expenditure on health care -12.13** -17.78* -12.71** -6.34 22.44 Impact (long-run) Poverty rate (%) -0.81** -0.76 -0.39 -0.48 -2.21 Notes: * denotes significant at 10% level, ** denotes significant at 5% level. Significance levels for General Impact Model are not shown here. Source: Authors' calculations based on HIECS and community survey 2004/2005. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 17 of 36 computations begin by estimating a propensity score equation, and then match the treated with the control villages (or households, in the case of the microcredit results) in the area of common support. As discussed above, the "Regression" columns report the results of estimating the impact equation in the area of common support, weighting the observations with the inverse of the propensity score. The single largest component of the SFD interventions in education has been the provision of literacy classes, which have enrolled almost 1.5 million people. It may be seen from Table 9 that this has had a measurable effect in reducing the proportion of people who are illiterate (by about 0.04; the mean value is 0.29) and providing literacy certificates (up by about 0.08; the mean value is 0.37). These results are robust, in the sense that they apply to all three comparisons. However, the efforts have not had any clear effect on the employment rate, and they are associated with higher, rather than lower, poverty rates, which is unexpected. One possibility is that SFD educational projects equip people to leave their villages and find better jobs elsewhere; this could actually increase the local poverty rate, if this process allows the less-poor to leave. Table 9. Summary of Effects of SFD Interventions in Education SFD intervention vs. no SFD SFD intervention only vs. no SFD intervention with others vs. intervention [A vs. B] intervention at all [D vs. F] other interventions only [C vs. E] Mean Regres- Nearest Kernel Regres- Nearest Kernel Regres- Nearest Kernel sion Neighbor sion Neighbor sion Neighbor Outcome: % who got 0.37 0.08*** 0.01 0.07** 0.08** 0.02 0.09* 0.08** 0.05 0.10 certificate (2.72) (0.18) (2.01) (1.99) (0.25) (1.93) (1.96) (0.53) (1.62) Impacts: Illiteracy rate 0.29 -0.04*** -0.02 -0.01 -0.04*** -0.02 -0.02 -0.03** -0.02 -0.004 (-4.8) (-0.08) (-1.08) (-3.55) (-0.76) (-1.10) (-2.20) (-0.19) (-0.19) Employment 0.95 -0.005 -0.01 -0.003 0.002 -0.0004 0.003 -0.01** -0.01 -0.01 rate (-1.52) (-0.81) (-0.53) (0.58) (-0.05) (0.63) (-2.38) (-0.59) (-1.11) Poverty rate 4.1 0.04* 1.40** 0.95** 0.26 0.71 0.63 0.51 1.08 1.22 (1.71) (2.34) (2.39) (0.88) (0.99) (1.30) (1.45) (1.21) (1.60) Notes: Numbers in parentheses are t-statistics, * significant at 10% level, ** significant at 5% level, *** significant at 1% level. % of participants who obtained a certificate. All results are derived by first estimating a propensity score equation, and then confining the regression, nearest-neighbor matching, or kernel matching, to the region of common support. Source: Authors' calculations based on HIECS and community survey 2004/2005. SFD health projects mainly consist of renovating health centers, training personnel, and conducting health awareness campaigns. The clearest and most consistent effect of this spending is that it reduces annual spending on health, by about LE 9 per household (the mean spending is LE 51). There is also a clear and strong effect on the under-5 mortality rate, which now stands at 19.9 per thousand, but is reduced by about 6 per thousand as a result of SFD interventions. However, the effect of health interventions on the poverty rate is less clear. When SFD interventions occur in addition to other interventions (the C vs. E comparison), they appear to lower the poverty rate, Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 18 of 36 but if one compares areas that just have SFD health projects with areas that have no health projects, the SFD areas are poorer. This is not compelling, and suggests that our propensity score matching may not be controlling adequately for other influences in these cases. Table 10. Summary of Effects of SFD Interventions in Health SFD intervention vs. no SFD SFD intervention only vs. no SFD intervention with others vs. intervention [A vs. B] intervention at all [D vs. F] other interventions only [C vs. E] Mean Regres- Nearest Kernel Regres- Nearest Kernel Regres- Nearest Kernel sion Neighbor sion Neighbor sion Neighbor Under-5 19.9 -6.19* -7.29 -5.92 -4.41 -16.52 -4.32 -8.71* -11.75 -9.25* mortality rate (-1.92) (-0.96) (-1.55) (-1.00) (-0.74) (-0.90) (-1.84) (-0.97) (-1.65) Expenditure on 50.9 -8.69*** -3.79 -10.18 -10.00*** -8.91* -12.70*** -7.82** -0.79 -6.88* Health (-3.75) (-0.88) (-3.25) (-3.32) (-1.77) (0.63) (-2.17) (-0.12) (-1.69) Poverty Rate 4.5 0.15 -0.20 0.44 2.16*** 3.43*** 2.68*** -1.57*** -2.07* -1.48*** (0.59) (0.28) (0.86) (6.37) (3.03) (3.26) (-4.37) (1.95) (-2.85) Notes: Numbers in parentheses are t-statistics, * significant at 10% level, ** significant at 5% level, *** significant at 1% level. All results are derived by first estimating a propensity score equation, and then confining the regression, nearest-neighbor matching, or kernel matching, to the region of common support. Source: Authors' calculations based on HIECS and community survey 2004/2005. The SFD has spent LE 620 million on providing potable water, including a public network of 8,480 kilometers. As Table 11 shows, this has led to sharp drops in spending on water and on health care. It is noteworthy that the reduction in health spending here is twice as large as the reduction associated with SFD health projects. There is also some modest evidence in Table 11 that the provision of potable water has led to a lower child mortality rate. The effects on poverty of providing potable water are more mixed; where SFD projects are in addition to other interventions, they help reduce poverty, but not otherwise. Table 11. Summary of Effects of SFD Interventions in Potable Water Provision SFD intervention vs. no SFD SFD intervention only vs. no SFD intervention with others vs. intervention [A vs. B] intervention at all [D vs. F] other interventions only [C vs. E] Mean Regres- Nearest Kernel Regres- Nearest Kernel Regres- Nearest Kernel sion Neighbor sion Neighbor sion Neighbor Expenditure on 17.2 -1.53*** -1.72* -1.42*** -2.83*** -0.90 -1.76* -1.02** -1.92* -1.39** water (-3.61) (-1.88) (-2.65) (-3.48) (-0.64) (-1.70) (-2.03) (-1.88) (-2.36) Under-5 19.9 -3.87 -0.93 -4.19 2.30 2.59 1.71 -5.94* 4.85 -4.07 mortality rate (-1.25) (-0.11) (-1.10) (0.39) (0.16) (0.24) (-1.16) (0.84) (-0.84) Expenditure on 50.9 -15.36*** -23.87*** -13.85** -21.42*** -16.22* -17.00*** -12.13*** -17.78** -12.72*** Health (-4.16) (-3.63) (-4.05) (-3.47) (-1.76) (-3.31) (-2.69) (-2.20) (-3.87) Poverty Rate 3.9 -0.36* -0.57 -0.15 0.64 0.20 0.40 -0.81*** -0.76 -0.40 (-1.66) (-1.15) (-0.41) (1.61) (0.23) (0.53) (-3.22) (-1.4) (-1.06) Notes: Numbers in parentheses are t-statistics, * significant at 10% level, ** significant at 5% level, *** significant at 1% level. All results are derived by first estimating a propensity score equation, and then confining the regression, nearest-neighbor matching, or kernel matching, to the region of common support. Source: Authors' calculations based on HIECS and community survey 2004/2005. The sanitation projects of the SFD consist mainly of constructing public networks and providing house connections. These activities have had the expected effects ­ increasing access to sewerage networks, halving household spending on sanitation, and lowering the poverty rate. These effects are especially Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 19 of 36 clear where SFD projects are undertaken in conjunction with other interventions (comparing C vs. E); when SFD-only areas are compared with areas lacking any project, the effects of SFD projects are either not statistically significant or counterintuitive (in the case of renal disease). It is curious that sanitation projects are not associated with lower household spending on health; one might have expected better sanitation facilities to improve health and so reduce such spending. Better health might then be associated with higher productivity and incomes, and a lower poverty rate. Table 12. Summary of Effects of SFD Interventions in Sanitation SFD intervention vs. no SFD SFD intervention only vs. no SFD intervention with others vs. intervention [A vs. B] intervention at all [D vs. F] other interventions only [C vs. E] Mean Regres- Nearest Kernel Regres- Nearest Kernel Regres- Nearest Kernel sion Neighbor sion Neighbor sion Neighbor Access: sewer- 0.52 0.04 0.12 0.11 0.01 0.11 0.07 0.06* 0.27* 0.16* age network (1.57) (1.02) (1.47) (0.25) (0.70) (0.66) (1.81) (1.94) (1.80) Spending on 4.41 -1.47** -2.26 -1.82 0.003 -0.048 -0.53 -2.30*** -1.60 -3.07*** sanitation (-2.49) (-1.50) (-1.41) (0.00) (-0.01) (-0.29) (-2.81) (-0.67) (-3.28) Prevalence: renal 0.54 0.22* 0.11 0.20 0.99*** 0.71 0.88** -0.39** -0.23 -0.44*** disease (1.96) (0.30) (0.82) (6.59) (0.94) (2.17) (-2.45) (-0.44) (-4.01) Prevalence: 0.09 -0.04 0.003 -0.04 -0.09 -0.51 -0.11*** 0.09 0.19 0.03 bilharzia (-0.62) (0.03) (-0.50) (-1.57) (-1.21) (-4.52) (0.83) (1.03) (0.17) Spending on 99.1 0.79 24.25 9.83 3.20 19.96 10.09 6.59 8.72 9.55 health (0.14) (1.30) (0.66) (0.42) (0.82) (0.63) (0.74) (0.22) (0.40) Poverty rate 3.6 -0.80*** -1.86 -1.21* -0.15 -1.43 -0.97 -1.07*** -0.91 -1.54* (-2.62) (-1.52) (-1.66) (-0.34) (-1.02) (-0.86) (-2.57) (-0.64) (-1.80) Notes: Numbers in parentheses are t-statistics, * significant at 10% level, ** significant at 5% level, *** significant at 1% level. All results are derived by first estimating a propensity score equation, and then confining the regression, nearest-neighbor matching, or kernel matching, to the region of common support. Source: Authors' calculations based on HIECS and community survey 2004/2005. Over the period 1991-2007, the SFD spent over LE 500 million on building and maintaining rural roads. As expected, these projects were associated with substantial reductions in household spending on transportation, by about LE 13 (compared to a mean of LE 66). The economic effects of these projects are quite strong: they raised the employment rate, and they increased farm income by almost half, presumably by lowering the cost of acquiring inputs and of delivering farm output to market. Only in the case where SFD-only areas are compared with no-project areas do roads also raise non-farm income; in the other comparisons non-farm income fell, perhaps as a result of greater competition from outsiders as areas become more accessible, or possibly because the increased attractiveness of farming led some people to cut back on non-farm activities. More surprisingly, there was no clear impact of the road projects on the poverty rate. A plausible explanation is that opening up villages to the outside world may raise incomes on average, but expose some vulnerable members of society to new economic pressures. There is an interesting pattern that runs through Tables 9 to 13: as a general rule, SFD interventions have a stronger measurable effect when the comparison is between areas where there are SFD interventions alongside other projects (case C) and areas where there are only other projects (case E). A reasonable Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 20 of 36 interpretation is that the SFD has the strongest impact when its efforts complement other projects, rather than when it operates alone. It is not entirely clear why this should be so, but it is possible that there are threshold effects at work: for instance, providing potable water to half a village may be helpful, but if the SFD then steps in and supplements such efforts, thereby ensuring everyone has potable water, it may be that only then are the effects on health appreciable. Table 13. Summary of Effects of SFD Interventions in Road Maintenance and Improvement SFD intervention vs. no SFD SFD intervention only vs. no SFD intervention with others vs. intervention [A vs. B] intervention at all [D vs. F] other interventions only [C vs. E] Mean Regres- Nearest Kernel Regres- Nearest Kernel Regres- Nearest Kernel sion Neighbor sion Neighbor sion Neighbor Spending on 65.8 -12.94*** -5.93* -11.57*** 2.92 -4.40 1.34 -17.79** -12.41** -16.68*** transportation (-6.76) (-1.90) (-4.20) (0.86) (-0.60) (0.25) (-7.78) (-2.38) (-6.32) % road utilization 0.15 -0.001 -0.006 0.01 -0.30** 0.06 0.002 0.008 0.005 0.01 (-0.10) (-0.28) (0.95) (-2.08) (0.20) (0.14) (0.82) (0.21) (0.74) Employment rate 0.95 0.01*** 0.01 0.01*** -0.01 -0.01 -0.002 0.02*** 0.01 0.02*** (2.99) (1.32) (2.70) (-1.38) (-0.85) (-0.20) (4.17) (1.58) (2.90) Farm-income 68.5 27.8*** 29.7*** 20.9*** 36.2*** 35.3 37.9** 23.8*** 14.0 15.2* (6.68) (2.93) (2.84) (4.85) (1.56) (2.30) (4.72) (1.01) (1.82) Non-farm 54.6 -10.0* -6.1 -11.4** 22.48** 35.62 16.06 -20.4*** -19.8** -22.1*** income -1.89 -0.65 -2.08 2.12 1.62 0.93 -3.48 -2.22 -3.63 Under-5 19.9 -3.68 -16.22 -4.80 -1.38 2.74 -1.54 -4.97 -13.08 -5.81 mortality rate (-1.17) (-1.44) (-1.20) (-0.24) (0.27) (-0.24) (-1.31) (-1.07) (-1.21) Enrollment rate, 0.91 0.01 0.02 0.01 -0.002 -0.02 -0.004 0.01 0.01 0.01 school (1.24) (1.21) (0.92) (-0.16) (-0.56) (-0.18) (1.42) (0.39) (1.02) Poverty rate 3.8 0.01 -1.07 -0.15 0.66* 1.79 0.84 -0.36 -0.68 -0.38 (0.05) (-1.46) (-0.33) (1.67) (1.29) (0.96) (-1.23) (-0.79) (-0.77) Notes: Numbers in parentheses are t-statistics, * significant at 10% level, ** significant at 5% level, *** significant at 1% level. All coefficients shown here should be multiplied by 1,000. All results are derived by first estimating a propensity score equation, and then confining the regression, nearest-neighbor matching, or kernel matching, to the region of common support. Source: Authors' calculations based on HIECS and community survey 2004/2005. Microcredit The SFD does not lend directly to households, but since 1991 it has spent LE 1,062 million to support microcredit, channeling both working capital and grants to financial intermediaries that in turn do provide microcredit. SFD support for microcredit has gone to productive family associations (33%), NGOs (31%), and the PBDAC (20%), as well as to village banks (9%), and occasionally to commercial banks (2%) or other lenders (5%). However, it is not possible to link individual loans with the activities of the SFD, so the approach that we take in this section is to estimate the impact of microcredit in general. Our working assumption is that if microcredit has an impact, then SFD support for microcredit has an impact. This is not a trivial assumption; if SFD support for microlending simply displaces other support for microlending, then SFD Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 21 of 36 interventions in this area might have no net effect. Unfortunately, there is no way to determine whether this is the case. In the Egyptian context, microcredit consists of extending modest amounts of credit to individuals. Microloans may not exceed LE 5,000 (about US$900), and most are much smaller: 48% of the microloans made in 2005 were for no more than LE 1,000, and only 7% were for more than LE 3,000. The expectation is that such loans provide a way for households to generate income, and in the case of poor households, even rise out of poverty. In 2005, 44% of those who took out microloans worked in agriculture, and a further 30% were traders. Fully 55% of such loans went to women, and 42% of borrowers said they were illiterate. On the other hand, most of the loans are short-term; in 2005, 70% of microloans had a duration of less than a year, and 22% were repayable over 1-2 years. When asked (in the 2004/05 HIECS survey) whether microloans increased their income, 80% of households said yes, while the remaining 20% said no. Our analysis of the impact of microcredit is disaggregated to the level of five regions ­ metropolitan areas, urban and rural Lower Egypt, and urban and rural Upper Egypt. This is done because of the belief that the effects of microcredit may vary substantially, depending on the underlying economic structure. We measure the impacts using propensity score matching; separate propensity score equations are estimated for each region, using household-level data ­ unlike our earlier estimates, which are based on village-level aggregates ­ and the models adapted until they were balanced. We employ the propensity scores to match "treated" (i.e. borrower) individuals in the region of common support with comparators, using kernel matching, nearest-neighbor matching, and a regression model. The results of the kernel matching are shown in Table 14, and in almost all cases were substantially the same as those found using the other two techniques; the two exceptions are noted in Table 14. One clear pattern does emerge from Table 14: microlending appears to lead to substantially higher levels of non-farm income per capita, especially in urban areas. It is also associated with widespread self- employment and, more modestly, with lower literacy rates. Most of the other effects are not consistent from one region to the next. The sharpest distinction is between the metropolitan areas and urban Lower Egypt on the one hand, and the rest of the country on the other. In the former, microcredit is associated with higher levels of expenditure, food expenditure, and Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 22 of 36 income, and lower levels of poverty. Elsewhere, microcredit comes with lower expenditure and income per capita and higher poverty rates. Table 14. Summary of the Effects of Microcredit Lower Egypt: Lower Egypt: Upper Egypt: Upper Egypt: Metropolitan Urban Rural Urban Rural Coeff- Coeff- Coeff- Coeff- Coeff- icient t-stat icient t-stat icient t-stat icient t-stat icient t-stat Farm income per capita 398.5 5.99 -61.8 1.82 -141.4 -3.73 118.8 2.80 -257.1 -7.05 Non-farm income per capita 896.9 4.70 573.9 4.44 113.5 2.93 604.5 5.02 121.9 3.79 ln(expenditure per capita) 0.10 3.00 0.10 2.42 -0.01 -0.71 -0.25 -8.91 -0.14 -9.10 ln(income per capita) 0.12 2.96 -0.03 -0.66 -0.04 -2.25 -0.21 -6.70 -0.11 -6.65 ln(food expenditure per capita) 0.21 6.49 0.18 4.22 -0.01 -0.64 -0.11 -3.19 -0.09 -5.34 Food as share of expenditure 0.05 5.67 0.04 3.40 0.001 0.25 0.06 10.40 0.02 5.95 Unemployment rate (in area) 0.03 1.85 0.01 0.33 0.04 5.67 -0.01 -0.93 0.01 1.92 % in area working for a wage 1.97 0.85 12.07 2.95 18.08 11.35 -3.82 -1.36 20.36 15.36 % in area self-employed 38.02 10.29 32.98 7.35 16.80 11.60 33.21 14.63 11.69 8.79 Illiteracy rate (in area) -0.03 -2.19 -0.04 -1.10 -0.08 -7.48 -0.02 -1.06 -0.09 -7.80 Poverty gap rate (P1) -0.59 -6.58 -0.76 -2.65 0.55 2.00 4.37 5.76 4.52 8.38 Headcount poverty rate (P0) -3.47 -4.24 -4.17 -2.29 3.15 2.34 16.75 6.55 16.99 8.93 Memo items: # of households in sample 2961 2514 7772 2558 6675 Number of treatment cases 235 116 659 269 827 Region of common support: Lr bound propensity score 0.01 0.01 0.01 0.02 0.05 Upr bound propensity score 0.33 0.13 0.46 0.29 0.30 Mean values of variables: Farm income per capita 55.37 124.09 735.54 120.99 709.71 Non-farm income per capita 955.29 887.50 354.76 722.19 245.28 ln(expenditure per capita) 8.15 7.87 7.68 7.80 7.43 ln(income per capita) 8.28 8.06 7.85 7.96 7.61 ln(food expend. per capita) 7.27 7.06 6.97 6.97 6.75 Food as share of expenditure 0.43 0.46 0.50 0.45 0.52 Unemployment rate (in area) 0.05 0.06 0.04 0.06 0.03 % in area working for a wage 76.15 61.07 45.46 62.81 41.45 % in area self-employed 20.92 31.02 40.37 31.57 42.79 Illiteracy rate (in area) 0.18 0.22 0.33 0.24 0.46 Poverty gap rate (P1) 0.59 1.22 1.65 3.26 6.88 Headcount poverty rate (P0) 4.36 7.32 11.94 16.30 34.19 Notes: Results shown here are based on propensity score matching using kernel matching. Only treatment cases in the area of common support are matched. Nearest-neighbor matching and regression model do not find a statistically significant effect here. Coefficients highlighted in bright yellow are statistically significant at the 5% level or higher; those highlighted in pale yellow are statistically significant at the 10% level. Source: Authors' calculations based on HIECS and community survey 2004/2005. On explanation for this surprising result is that matching techniques work well when the variables in the propensity score equation capture, and hence control for, all the relevant factors that distinguish one Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 23 of 36 borrower from another. Inevitably, there are also unobservable influences that cannot be captured in survey data, and it is entirely possible that we have not been able to control satisfactorily for the fact that borrowers of microcredit have little education and are inherently poor, especially in the less-affluent regions of the country. Costs Compared with Benefits Even if a program has an impact, it may not be worth undertaking if it is too costly. The results of a first pass at measuring the cost effectiveness of SFD programs (except for microcredit) are shown in Table 15. The procedure that we followed involves measuring the total impacts of each program, using only the statistically significant coefficients from Tables 8-13, and dividing into the total cost. The numbers are in Egyptian pounds, and are for 2004. For instance, in order to measure the cost per one fewer illiterate person, using the regression method and comparing A vs. B, we divided the cost of the program (LE 200m) by (-0.038 × population in A × adults as a share of the population). This gives a value of LE 1,842, which is the cost of pulling one person out of illiteracy, on the (not very reasonable) assumption that the eradication of illiteracy was the only goal of SFD educational spending. The available information on the cost of each program is approximate, and represents a raw total of spending from 1991 to 2007; in the absence of annual numbers it was not possible to inflate earlier figures to reflect inflation, or to cut off the spending totals in 2004, the year for which the household and community survey data are available. Nonetheless, the spending totals do reflect the correct orders of magnitude involved. As with the measures of the impacts themselves, the estimates of cost-effectiveness are not completely consistent. For instance, the provision of potable water or sanitation is associated with a reduction in poverty, at a cost (per one fewer poor) that averages somewhere between LE 24,684 (kernel estimates of A vs. B for sanitation) and LE 63,161 (regression estimates of A vs. B for potable water). Yet road schemes and literacy programs are associated with more poverty ­ either the econometric results are not adequately controlling for program placement, or roads and reading allow the less-poor to leave, reducing the poverty rate of those who remain ­ and the effects of health interventions on poverty are either positive or negative, depending on the comparison that one favors. The cost-effectiveness analysis, however interesting, is incomplete, because it does not try to balance benefits with costs, and it does not provide a mechanism for adding up the multiple benefits for any given Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 24 of 36 program, such as roads, or sanitation. The solution to undertake a cost-benefit analysis, which puts a monetary value on the benefits generated by the programs, and compares these values with the costs. Table 15. Cost Effectiveness of Social Fund for Development Programs (in LE per outcome) A vs. B D vs. F C vs. E Regres- Nearest Kernel Regres- Nearest Kernel Regres- Nearest Kernel sion Neigh- sion Neigh- sion Neigh- bor bor bor Education (LE 200m) Per certificate obtained 909 1,000 1,866 1,555 1,667 Per one less illiterate 1,842 3,682 5,385 Per one more job -9,723 Per one fewer poor -89,952 -2,508 -3,683 Health (LE 390m) Per one less U-5 death 183,176 196,170 184,762 Per LE1 saved on health spending 13.1 33.7 37.8 26.5 21.8 24.8 Per one fewer poor 15,594 9,814 12,568 -10,866 -8,241 -11,525 Potable water (LE 620m) Per LE1 saved on water 146.2 130.4 157.9 250.6 403.4 322.5 170.3 236.6 Per one less U-5 death 551,786 Per LE1 saved on health spending 14.6 9.4 16.2 33.2 43.8 41.8 27.0 18.4 25.8 Per one fewer poor 63,161 40,458 Sanitation (LE 220m) Per person getting access to sanitation 24,399 5,741 9,946 Per LE1 saved on sanitation spending 203.6 678.9 508.0 Per person fewer with renal disease -1,387 -374 -421 4,045 3,541 Per person fewer with bilharzia 3,371 Per one fewer poor 37,552 24,684 145,668 101,268 Roads (LE 500m) Per LE1 of transport spending saved 27.8 60.7 31.1 47.3 67.8 50.5 Road utilization 0.002 Per one more job 59,903 54,457 87,674 93,519 Per LE1 of additional farm income 12.9 12.1 17.2 17.3 16.6 35.4 Per LE1 of additional non-farm inc -36 -32 28 -41 -43 -38 Per one fewer poor -95,188 Source: Authors' calculations based on HIECS and community survey 2004/2005. There are two challenges in undertaking the cost-benefit analysis: first, one has to measure the total impact of the programs, which we have done above; and second, it is necessary to value each output. For the latter, we drew on secondary sources, using the values that are set out in Table 16. We suppose that the benefit of pulling someone out of illiteracy is the avoided cost of schooling per child that completes fifth grade. This treats literacy as a right that needs to be provided, rather than valuing it solely in terms of labor market outcomes. We value an additional job by measuring the economic rent that accrues when someone who was not working now has full-time employment. This may be measured as the wage Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 25 of 36 actually received minus the opportunity cost of supplying that labor (see Boardman et al. 2006, chapter 11). To measure the wage appropriate to the type of jobs that SFD interventions are likely to help create, we used the private sector monthly wages received by those with an elementary school education or less in 2004 (Said 2007, Table A2), adjusted to the prices of 2004; in the absence of further information, we assume that the labor rent constitutes half of this total, and we also assume that the jobs created last for, on average, five years. This gives a total value per job created of LE 10,700 (in 2004). This is substantially less than the "cost of creating a job," which Ghoneim (2006) puts at LE 40,000 in the relatively labor-intensive garment sector and MEMRI (2002) pegs at LE 50,000 in agriculture. Some SFD programs reduce poverty. One way to put a value on this is to measure the increase in expenditure that would be required to raise a poor person, whose expenditure is at the mean level for the poor, out of poverty. The World Bank (2007) shows that the annual poverty deficit per poor person is LE 266 per year, or LE 1,333 over a period of five years, which we take to be the effective life of an SFD poverty-reducing intervention. Table 16: Benefits per unit of output produced by SFD projects Impact to be valued LE Comments and Sources Per one less illiterate 1,157 The benefit is that we save the cost of educating a child to the fifth grade. The cost per fifth-grade completer was LE 911 in 1998-99 (USAID 2006, based on Zaalouk 2006); this was grossed up by 27% to reflect consumer price inflation between 1998-99 and 2004 (World Bank, World Development Indicators, online, accessed March 7, 2009). Per one more job 10,700 Measures the economic rent obtained by working; assumed to be half of the wage earned in private sector employment by those with an elementary school education or less in 2006 (adjusted to the prices of 2004); the wage data come from Said (2007). Per one fewer poor 1,333 The poverty line is LE 1,423 per person per year. Using this line, a poverty rate of 19.6%, and data on income distribution in 2004 from PovcalNet, we fitted a quadratic approximation to the Lorenz curve (see Datt 1998) and estimated the average expenditure of the poor to be LE 1,175. We assume that when someone is brought out of poverty, their spending rises from LE 1,175 to LE 1,423 (i.e. by LE 248), and maintains this improvement for five years. Per one less U-5 death 796,000 Boardman et al. (2006, chapter 15) conclude that a value of US$4 m ± USD$2 m, in 2002 prices, is appropriate for the US; Viscusi and Aldy (2003) in their meta- analysis use a value of US$6.7m (in 2000 prices). We assume that the value of a statistical life in Egypt is the same multiple of GDP/capita as in the US. Per LE1 saved on medicines (or water, 5 When a project saves LE 1 in spending, we assume that this lasts for five years. or sanitation, or health, or transport) Per person less with renal disease 75,222 The Disability-Adjusted Life Year (DALY) of nephritis is 0.091 and of nephrosis is 0.098, which means that one less case of renal disease saves the equivalent of 0.0945 of the value of a life (Mathers et al. 2001). Per person less with bilharzia 4,776 The Disability-Adjusted Life Year (DALY) of a bilharzia infection is 0.006, which means it is equivalent to 0.006 of the value of a life (Mathers et al. 2001). Per LE1 of additional farm income, or 5 We assume that this effect lasts for five years. additional non-farm income Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 26 of 36 Where SFD interventions reduce mortality, it is necessary to use a value of a statistical life. To the best of our knowledge, this has not been estimated in the context of Egypt. Based on their review of studies undertaken in the United States, Boardman et al. (2006) believe that in the United States a statistical life may be valued at US$4m ± 2m. We adjust this number to reflect the lower levels of income in Egypt than in the United States, giving a value of LE 796,000 in 2004 prices. Sanitation projects also reduce the incidence of disease, and these too need to be valued. One approach is to estimate the Disability-Adjusted Life Years (DALYs); for each medical condition, the DALY measures the estimated reduction in life expectancy plus a measure of the handicap faced by sufferers. For an incident of bilharzia, the DALY is 0.006 (Mathers et al. 2001), which means that each case of bilharzia that is avoided leads to a saving equivalent to 0.006 of the value of a life. The DALY for renal disease is 0.0945. The combined impact of these effects is shown in Table 17. Consider, for example, the regression estimates of the effects of SFD road projects, comparing areas with SFD projects (A) with areas that do not have SFD projects (B). The total benefit of these projects come from savings in transport costs (LE 90m), job creation (LE 334m), higher farm income (LE 193m), adjusted for lower non-farm income (-LE 70m), for a net total benefit of LE 548 m. This exceeds the cost of the road projects (LE 500m), indicating that these projects appear to be economically worthwhile. Since some of these results are sensitive to the valuation of a statistical life, we also report the totals under the assumption that a statistical life has one- fifth of the valuation discussed above. The most consistent results are for SFD road projects, where the measured benefits are generally positive, and in the case of the A vs. B comparison, are arguably higher than the costs. The net benefits are not as large in the D vs. F and C vs. E comparisons, where they typically do not exceed half of the cost of the program. For SFD potable water projects, the benefits are also generally positive, but in only one case (the regression estimate based on C vs. E, assuming a high value of a statistical life) do the benefits exceed the cost. The SFD health projects generate modest benefits in raising people out of poverty and reducing private spending on health care, but by some of the econometric measures they lower child mortality. If the value that we use for a statistical life is appropriate, then the main benefit of these health projects occurs by reducing child deaths. The benefits of the education projects are rather modest, relative to the cost; and the effects of the sanitation projects cannot be determined with much reliability; they are driven by the effects on renal disease, which in turn are highly sensitive to the value put on a statistical life; in this case some of the Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 27 of 36 econometric estimates have an unexpected sign, implying, if taken at face value, that some of the projects may have made some health outcomes worse. Table 17. Cost-Benefit Analysis of Social Fund for Development Programs A vs. B D vs. F C vs. E Regres- Nearest Kernel Regres- Nearest Kernel Regres- Nearest Kernel sion Neigh- sion Neigh- sion Neigh- bor bor bor Education (LE 200m) Per one less illiterate (LE 1,157) 126 63 43 Per one more job (LE 10,700) -220 Per one fewer poor (LE 248/yr × 5) -3 -106 -72 Total 123 -106 -72 63 -177 Total, with lower value of life 123 -106 -72 63 -177 Health (LE 390m) Per one less U-5 death (LE 0.796m) 1,695 1,583 1,680 Per LE1 saved on health spending (× 5) 149 58 52 74 89 79 Per one fewer poor (LE 248/yr × 5) 33 53 41 -48 -63 -45 Total 1,844 91 104 115 1,624 -63 1,714 Total, with lower value of life 319 91 104 115 200 -63 202 Potable water (LE 620m) Per LE1 saved on water (× 5) 21 24 20 12 8 10 18 13 Per one less U-5 death (LE 0.796m) 894 Per LE1 saved on health spending (× 5) 212 330 191 93 71 74 115 168 120 Per one fewer poor (LE 248/yr × 5) 13 20 Total 247 354 211 106 71 82 1,039 186 133 Total, with lower value of life 247 354 211 106 71 82 234 186 133 Sanitation (LE 220m) Per LE1 saved on sanitation spending (× 5) 5 2 2 Per person fewer with renal disease (0.0945 × life) -11,928 -44,269 -39,271 4,091 4,674 Per person fewer with bilharzia (0.006 × life) 312 Per one fewer poor (LE 248/yr × 5) 8 12 2 3 Total -11.915 12 -44,269 -38,960 4,094 4.679 Total, with lower value of life -1,180 12 -4,427 -3,896 413 472 Roads (LE 500m) Per LE1 of transport spending saved (× 5) 90 41 80 53 37 50 Per one more job (LE 10,700) 334 367 228 214 Per LE1 of additional farm income (× 5) 193 207 145 144 151 71 Per LE1 of additional non-farm inc (× 5) -70 -79 90 -61 -59 -66 Per one fewer poor (LE 248/yr × 5) -7 Total 548 248 514 227 151 291 -22 198 Total, with lower value of life 548 248 514 227 151 291 -22 198 Source: Authors' calculations based on HIECS and community survey 2004/2005. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 28 of 36 This cost-benefit exercise rests on fairly strong assumptions, and so is somewhat tenuous; better measurement might help alleviate some of the problems. However, even these rudimentary numbers are useful enough to help concentrate minds for the future: the SFD education and sanitation projects do not look compelling, and even the other projects (health, potable water, and roads) have benefits that do not generally exceed the costs. There are certainly other worthwhile justifications for SFD projects ­ equity, the satisfaction of human rights, political imperatives ­ but on strictly economic terms none of them look particularly compelling based on the available evidence. Targeting Even when the costs of a program exceed its benefits, the program might be justifiable on distributional grounds; for example, although minimum wage laws are inefficient, they may be one of the more effective ways of channeling resources to the poor (Gramlich 1997). This naturally leads us to ask whether the SFD programs were successful in targeting the poor. During the first phase of its work (1992- 1996) the SFD functioned primarily as an emergency fund and safety net to support implementation of the Economic Reform and Structural Adjustment Program (ESAP), and it sought to target its assistance geographically based on poverty and unemployment rates at the level of the 27 governorates, broken down where possible into urban and rural divisions.3 The SFD has since transitioned into a permanent development organization with the twin goals of supporting and developing micro- and small enterprises, and enhancing human capital for the poor, and so retains its original focus on combating poverty. A poverty-targeting unit has now been established within the SFD to help it identify more clearly those areas of the country where interventions are likely to have the greatest impact on the poor. The first question to ask is how effectively the SFD has targeted its spending at the governorate level. Given its stated commitment to targeting the poor, one might expect SFD spending to go disproportionately to governorates with a higher level of poverty. We measure this visually in Figure 3: the horizontal axis shows the cumulative percentage of the population, obtained by sorting governorates from those with the highest to those with the lowest level of poverty. The vertical axis shows cumulative percentages; the smooth regular curve tracks the incidence of poverty and shows, for instance, that 80% of the poor in Egypt live in the poorest half of the governorates (as measured by population). If SFD spending were distributed to governorates strictly in proportion to population, then it would follow the diagonal in Figure 3. This is close to being the case for microcredit projects (Figure 3, bottom panel). 3 As of 2004 there were 27 governorates; two more (Helwan, and 6th of October) were created in April 2008. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 29 of 36 On the other hand, spending on health projects comes close to tracking the number of poor people by governorate, at least in the governorates with the highest incidence of poverty. It is clear from Figure 3 that despite SFD claims that its programs are allocated largely based on the incidence of poverty at the governorate level, all of them fall short of allocating spending (at the governorate level) in proportion to the number of poor. 100 Cumulative % of poverty, etc 90 Poverty 80 70 Community Development 60 50 40 Education 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 Cumulative % of population (by governorate, from poorest to least poor) 100 Cumulative % of poverty, etc. 90 Poverty 80 70 Public Works 60 Health 50 40 30 Microcredit 20 10 0 0 10 20 30 40 50 60 70 80 90 100 Cumulative % of population (by governorate, from poorest to least poor) Figure 3. The incidence of poverty, and of SFD spending, at the governorate level Source: Authors' calculations based on HIECS and community survey 2004/2005. Even if SFD programs are not allocated to governorates in proportion to poverty, they could be well targeted to the poor at the village or ward level ­ for instance, by going disproportionately to poor villages, even in rich governorates. With the exception of microcredit, we do not have household-level Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 30 of 36 information on the targeting of SFD programs, but we do have information from the HIECS 2004/2005 at the village level, and so can determine whether SFD spending is directed mainly towards poor villages (even if this is no guarantee that the poor, rather than the non-poor, are the ultimate beneficiaries). Figure 4 displays the relevant information; it shows the cumulative percentage of the population from the poorest to the richest villages and wards on the horizontal axis, while the vertical axis shows the cumulative percentage of SFD spending on the various programs; for each program there is a different "concentration curves". Although it is not strictly comparable, since it is based on household-level data, Figure 4 also shows a Lorenz curve ­ the heavy line to the lower right ­ which tracks the distribution of per capita income in 2004. 100 Cumulative % of SFD spending 90 80 Roads 70 Microcredit 60 50 Education Income Distribution (Lorenz Curve) 40 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 Cumulative % of population (poor to rich) 100 Cumulative % of SFD spending 90 80 Potable Water 70 Health 60 Environment 50 40 Waste Water Income Distribution (Lorenz Curve) 30 20 10 0 0 10 20 30 40 50 60 70 80 90 100 Cumulative % of population (poor to rich) Figure 4. Distribution of Spending on SFD Programs, Based on Village/Ward-Level Data, 2004-05. Source: Authors' calculations based on HIECS and community survey 2004/2005. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 31 of 36 The thin diagonal in Figure 4 shows the line of "perfect equality" and serves as a useful reference. For two of the SFD programs ­ education, and waste water ­ the cumulative percentage of spending is below the line of perfect equality. This means, for instance, that the poorest 20% of villages (by population) receive less than 20% of SFD spending on education, or on waste water treatment. Spending on these programs thus favors the better-off over the poor. 4 All the other SFD programs have concentration curves above the line of perfect equality, meaning that spending is geared more towards the poor than the rich (at the village level). Spending on potable water projects, and to a lesser extent on road and health interventions, is relatively strongly geared toward poorer villages. While these effects are clear, they are not overwhelming; the poorest 30% of the population lives in villages that received 43% of SFD potable water spending, and 35% of SFD health and road spending. These reflect a pro-poor bias, but not a strong one. Table 18 provides some further information on SFD targeting. For each SFD program, it shows the headcount poverty rate in villages with, and without, an SFD intervention. For instance, the poverty rate in villages and wards with SFD road projects was 21.8%, compared to 19.5% in areas without an SFD road project, a difference that is statistically significant (t=-5.03); the national poverty rate in 2004-05 was 19.6%. The numbers confirm that SFD education and wastewater treatment projects were not well targeted to villages with higher poverty rates, while road, health, and (especially) potable water projects were more successfully targeted to the poor. The poverty rate in villages and wards that received any SFD intervention was 21.6%, which is significantly higher than the 18.6% poverty rate in areas with no SFD project. The numbers in Table 18 allow us to check the robustness of these findings. Let us suppose that there are six basic needs, as follows: (a) Access to drinking water on the property; (b) Having a toilet other than a simple pit latrine; (c) Primary-age children are enrolled in school; (d) The household head has at least three years of education and is employed; or at least one in every three household members is employed; (e) There are no more than three persons per room (excluding bathrooms); and 4Public spending is "regressive" if it represents a smaller percentage of household income (or spending) for the poor than for the rich; this would be reflected in a concentration curve below the Lorenz curve, which is not the case even for SFD spending on education or waste water treatment. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 32 of 36 (f) The dwelling is not improvised, and the walls are made of brick (or better). Based on the HIECS 2004/2005 data, 6.2% of the Egyptian population met all of these basic needs, while 14.2% had three or more unmet basic needs. Defining the poor as those with at least three unmet basic needs, we see that only SFD health and potable water projects were targeted to villages with disproportionately high levels of this type of poverty. Table 18. Poverty Rates in Areas Served by SFD Projects Headcount poverty rate (%) t-statistic for % of households with at least 3 difference in unmet basic needs No SFD With SFD poverty rates No SFD With SFD project project project project Education 19.8 15.0 14.2 14.5 10.1 Wastewater treatment 19.6 12.8 9.9 14.2 13.2 Environment 19.5 20.1 -1.7 14.2 13.8 Roads 19.5 21.8 -5.0 14.3 13.2 Health 19.5 22.1 -6.0 14.2 14.7 Microcredit 18.8 23.1 -18.1 14.1 14.9 Potable water 18.2 30.2 -38.6 13.5 19.5 Microcredit* 45.6 Any intervention 18.6 21.6 -15.6 14.2 14.3 Note: * Based on household-level data. All other figures are based on village/ward-level data from HIECS 2004/2005. Source: Authors' calculations based on HIECS and community survey 2004/2005. These numbers likely understate the extent to which SFD programs reach poor households if, as is plausible (although not certain), the actual beneficiaries of SFD programs in a given village are disproportionately poor. In the case of microcredit it is possible to identify the households that avail of the credit, using the HIECS 2005/2005 household data. The relevant numbers are set out in Table 19, and they show that the poorest 10% of the population account for 24.9% of all microcredit loans; furthermore, the headcount poverty rate among microcredit users was 45.6%, or far above the national rate of 19.6% (in 2004-05). Without further information on the size of loans, we cannot tell what proportion of the value of microcredit goes to the poorest households, but the indications are that microcredit is very highly targeted to the poor. However, it must be noted that these numbers refer to all microcredit, and not specifically to SFD-supported microcredit, which individuals cannot identify separately, mainly because SFD support for microcredit operates indirectly via NGOs and financial intermediaries. Conclusions In our examination of the Egyptian Social Fund for Development, we have focused on three key questions: How large is the impact of the SFD interventions? Have the benefits been commensurate with the costs? And have the programs been targeted successfully to the poor? Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 33 of 36 Table 19. Percentage of direct beneficiaries of microcredit lending, by decile, 2004-05. Urban Areas Rural Areas All Egypt Deciles Poorest 23.8 25.2 24.9 2 9.9 18.1 16.2 3 8.9 15.6 14.1 4 7.8 11.8 10.8 5 6.1 9.1 8.4 6 7.8 6.5 6.8 7 11.3 5.8 7.1 8 8.0 3.7 4.7 9 8.9 3.0 4.4 Richest 7.7 1.3 2.8 Memo: Poverty rate, beneficiaries 33.1 49.4 45.6 Poverty rate, population 10.1 26.8 19.6 Source: Authors' calculations based on HIECS and community survey 2004/2005. We have sought to measure the impact of six separate SFD programs; in every case there are several measurable outcomes, each estimated in at least three distinct ways for three types of comparisons. This gives well over 200 measures of impact. Inevitably, with such a large body of estimates, some of the results do not appear to be entirely consistent, and a few are either surprising or implausible. That said, the general thrust of the estimates is that SFD programs have had clear and measurable effects, in the expected direction, for all of the programs considered here. Educational interventions have reduced illiteracy, but have had no clear effect on employment; health programs have brought down household spending on health by almost a fifth, and markedly reduced under-5 mortality; potable water provision has lowered household spending both on water and health, and has also reduced under-5 mortality; sanitation interventions have cut household spending on sanitation and lowered poverty; and road projects are associated with a 20 percent drop in household transportation costs, and a higher employment rate. As a general rule, SFD interventions have had the strongest identifiable impact when they have complemented other projects, rather than operating alone. The SFD supports microcredit, but does not lend directly to households. Our analysis of the impact of microlending in general finds that it is associated with higher levels of non-farm income per capita. In the metropolitan areas and in urban Upper Egypt, microcredit goes together with higher household expenditures and less poverty, although this association is not evident elsewhere. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 34 of 36 The SFD has a mandate to reduce poverty, increase employment opportunities, and encourage small- enterprise development. This focus on the social dimension means that the SFD has not traditionally needed to trade off costs with economic benefits, and so it cannot be judged by this measure. An economic cost-benefit analysis does have a role, however, if only in identifying the most easily-justifiable SFD activities. We find that SFD road projects generate benefits that, by some estimates, exceed the costs. This is also true of health and potable water interventions, at least if the value put on a statistical life is not too low. The strictly economic benefits of educational projects are more modest relative to the costs; and the benefits from sanitation projects are highly sensitive to the econometric measures used to quantify their impact, and so the measures are unreliable. It should be emphasized that the valuation of unit benefits is based on secondary sources, and so requires strong assumptions that are difficulty to verify independently. Even where SFD interventions may not pass a cost-benefit test, they may well be justified on other grounds ­ as an efficient way to achieve social equity or to further human rights, or to satisfy political needs. With the exception of microcredit, it is not possible to associate program benefits with individual households, so in most cases we have only been able to measure the effectiveness with which the SFD has targeted its interventions to poor areas, whether poor governorates, or poor villages/wards. The SFD interventions that we have analyzed go to areas that have a higher headcount poverty rate than the national average (21.6% vs. 19.6%), and a lower level of average per capita annual household expenditure (LE 2,292 vs. LE 2,556). In this broad sense, SFD interventions are pro-poor, although at first sight only modestly so. A household-level analysis of targeting was only possible for microcredit, and in this case the poverty rate among borrowers was 45.6%, or more than twice the national average. Projects related to roads, health, and (especially) potable water were also targeted to poor areas, but spending on education and on wastewater treatment went disproportionately to areas with lower-than- average village-level poverty rates. If, within these villages, the beneficiaries were generally poor, then even these interventions may have been pro-poor. There is unfinished business. The data were not sufficiently rich to allow us to do a complete cost-benefit analysis of the SFD microcredit programs, or of the impact, costs, or benefits of the largest single SFD program, which provides support to small and medium enterprises. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 35 of 36 References Aldy, Joseph E. and W. Kip Viscusi. 2006. Adjusting the Value of a Statistical Life for Age and Cohort Effects. Discussion Paper 06-19, Resources for the Future, Washington DC. Boardman, Anthony, David Greenberg, Aidan Vining and David Weimer. 2006. Cost-Benefit Analysis: Concepts and Practice, 3rd edition, Pearson/Prentice Hall, Upper Saddle River, NJ. Datt, Gaurav. 1998. Computational Tools for Poverty Measurement and Analysis. Food Consumption and Nutrition Division Discussion Paper No. 50, International Food Policy Research Institute, Washington DC. Dehejia, Rajeev and Sadek Wahba. 2002. "Propensity Score-Matching Methods for Nonexperimental Causal Studies," Review of Economics and Statistics, 84(1) 151-161. DeStefano, Joseph. 2006. Meeting EFA: Egypt Community Schools. Equip2 Case Study, United States Agency for International Development, Washington DC. Diamond, Alexis. 2005. Reliable Estimation of Average and Quantile Causal Effects in Non-Experimental Settings. Working draft, Harvard University, Cambridge MA. Egyptian Social Fund for Development (ESFD). 2006. Social, Economic and Environmental Impact Evaluation of Projects of the Egyptian Social Fund for Development. Draft, August. el-Jesri, Manal. 2008. A Portrait of Poverty. Egypt Today, March. Ghoneim, Ahmed Farouk. 2006. "Every Thread Counts," Al-Ahram Weekly On-line, No. 813, September. Gramlich, Edward. 1997. A Guide to Benefit-Cost Analysis, 2nd edition. Waveland Press. Imbens, Guido. 2004. "Nonparametric Estimation of Average Treatment Effects under Exogeneity: A Review," Review of Economics and Statistics, 86(1): 4-29. Jorgensen, Steen Lau, and Julie van Domelen. 1999. Helping the Poor Manage Risk Better: The Role of Social Funds, presented at Inter-American Development Bank Conference on Social Protection and Poverty, February 4-5, Washington DC. Mathers, Colin D., Alan D. Lopez, and Christopher J. L. Murray. 2001. The Burden of Disease and Mortality by Condition: Data, Methods, and Results for 2001. In A.D. Lopez, C. D. Mathers, M. Ezzati, C.J.L. Murray, and D.T. Jamison (eds.), Global Burdens of Disease and Risk Factors, Oxford University Press, NY. MEMRI (Middle East Media Research Institute). 2002. Inquiry and Analysis ­ No. 81. January. Rawlings, Laura B., Lynne Sherburne-Benz, and Julie van Domelen. 2004. Evaluating Social Funds: A Cross- Country Analysis of Community Investments, World Bank, Washington DC. Rosenbaum, P. and D. Rubin. 1983. "The Central Role of the Propensity Score in Observational Studies for Causal Effects," Biometrika, 70(1): 41-55. Rubin, Donald and Richard Waterman. 2006. "Estimating the Causal Effects of Marketing Interventions Using Propensity Score Methodology," Statistical Science, 21(2): 206-222. Said, Mona. 2007. The Fall and Rise of Earnings and Inequality in Egypt: New Evidence from the ELMPS, 2006. Working Paper 0708, The American University in Cairo. Viscusi, W. Kip, and Joseph E. Aldy. 2003. "The Value of a Statistical life: A Critical Review of Market Estimates Throughout the World," The Journal of Risk and Uncertainty, 27(1): 5-76. WHO (World Health Organization). 2009. Global Burden of Disease (GBD). Available at http://www.who.int/healthinfo/global_burden_disease/en/index.html [Accessed March 9, 2009] Zaalouk, Malak. 2006. The Pedagogy of Empowerment: Community Schools as a Social Movement in Egypt. American University in Cairo Press. Impact Evaluation: Egyptian Social Fund for Development Version of June 29, 2009 Page 36 of 36