S P D I S C U S S I O N P A P E R NO. 0815 44671 Beyond DALYs: Developing Indicators to Assess the Impact of Public Health Interventions on the Lives of People with Disabilities Daniel Mont and Mitchell Loeb May 2008 Beyond DALYs: Developing Indicators to Assess the Impact of Public Health Interventions on the Lives of People with Disabilities Daniel Mont Disability & Development Team HDNSP The World Bank Mitchell Loeb Office of Analysis and Epidemiology National Center for Health Statistics Centers for Disease Control & Prevention May 2008 Acknowledgements We would like to thank Barbara Altman, Linda Bilheimer, Jeanine Braithwaite, Jed Friedman, Scott Grosse, Don Lollar, Diane Makuc, Jennifer Madans, Kelechi Ohiri, Pia Rockhold, Julian Schweitzer, and Sándor Sipos for their comments on earlier drafts. Abstract Two indicators ­ the Activity Limitation Score (ALS) and the Participation Restriction Score (PRS) ­ are presented in this paper for use in assessing the impact of public health interventions on the lives of disabled people. They address a gap in the Disability- Adjusted Life Years (DALY) approach which is not sensitive to changes in people's functional status resulting from interventions that do not change an underlying medical diagnosis. Household data from Zambia are used to explore the potential usefulness of these measures. JEL Classification: C8 - Data Collection and Data Estimation Methodology; Computer Programs, I10 - Health General, J14 - Economics of the Edlerly; Economics of the Handicapped Keywords: Disability, Indicators, Health, DALY Measurement The findings, interpretations, and conclusions expressed herein are those of the authors, and do not necessarily represent or reflect the views of the National Center for Health Statistics, Centers for Disease Control and Prevention, the International Bank for Reconstruction and Development / The World Bank and its affiliated organizations, or those of the Executive Directors of The World Bank or the governments they represent. i ACRONYMS ALS Activity Limitation Score DALYs Disability-Adjusted Life Years ICF WHO's International Classification of Functioning, Disability and Health PRS Participation Restriction Score SEAs Standard Enumeration Areas UN United Nations WHO World Health Organization ii Introduction Disability and poverty are intricately linked [1, 2, 3, 4, 5]. Roughly 10-12 percent of the world's population has a disability [6] and they are among the poorest of the poor [5]. Including people with disabilities in economic development activities, therefore, is key to achieving the UN Millennium Development Goals [7] of ending poverty and hunger, providing universal education, and improving health outcomes. Recently, as evidenced by the passage of the UN Convention on the Rights of Persons with Disabilities [8], more attention is being focused on interventions designed to be more inclusive and have an impact on disabled people's lives. In order to evaluate these interventions it is necessary to have indicators capable of assessing the impact of policies and programs on the lives of disabled people. Disability-Adjusted Life Years (DALYs) are a frequently used indicator for assessing the relative effect of public health interventions for disabled people. The introduction of DALYs, as opposed to only using mortality, was an advance in that it incorporates a measure of the impact of living with a disability [9]. With the implementation of DALYs, conditions which were non-fatal but were at the root of significant disabilities rose in importance in the field of public health, such as onchocersiasis (river blindness). While an extensive literature evaluates the approach embodied in DALYs, [10,11,12], this paper focuses on a single aspect ­ the inability of DALYs to record improvements in the functioning of disabled people by means that are non-curative, and thus do not remove a person's underlying medical diagnosis [10]. DALYs do not reflect the change in people's functional status or well-being if they receive rehabilitation services, assistive devices, accommodations, or live in a society that has become more open and accessible to individuals with functional limitations. DALYs only reflect the presence of a medical condition that is associated with certain functional limitations. Therefore, public health interventions that mitigate the effects of a health condition but do not "cure" it, get no credit. This paper proposes two measures for assessing the impact of interventions on the lives of disabled people, the Activity Limitation Score (ALS) and the Participation restriction Score (PRS). These measures are closely linked to the WHO's International Classification of Functioning, Disability and Health (ICF) and the social model of disability [13]. We believe these measures could become important tools in monitoring the implementation of the recently ratified UN Convention on the Rights of Persons with Disabilities. The structure of this paper is as follows: After briefly describing the social model of disability and the ICF, we present a series of indicators for capturing the functional status of individuals. Then, using household survey data from Zambia, we explore the usefulness of this measure as it relates to economic development outcomes. 1 The computer software programme SPSS (Release 13.0/15.0) was used for data entry and data analyses. Analyses comprise univariate, bivariate and multivariate techniques. The significance of observed associations and/or differences between variables was tested using the Student's t-test, the Pearson correlation coefficient (r), and linear regression. A difference was considered to be statistically significant if p < 0.05. 1. Developing Indicators Consistent with the ICF and the Social Model of Disability A key goal of economic development is to create the same opportunities for all persons to participate in economic and social life, regardless of their personal characteristics. According to Amartya Sen's capabilities approach [14, 15, 16, 17], poverty is based on a standard of living described by the capability to conduct various "functionings." These functionings are defined as the attainment of states of being that are fundamental to living an acceptable quality of life. They include things such as being well-sheltered, being well-nourished, being able to move about freely, and being able to form and maintain a family. As such, they are not input based, like income and consumption measures. They are output based. Does a person have the capability to combine the resources at their disposal to live a complete and dignified life? This relates directly to the idea of "participation" in the social model of disability. 1.01 The ICF and the Social Model of Disability According to the ICF and the social model, disability arises out of the interaction between functional limitations and an unaccommodating environment [18, 19, 20]. In the ICF people are not identified as having a disability based upon a medical condition, as in DALYS, but rather are classified according to a detailed description of their functioning within various domains: Body Function and Structure, Activities, and Participation. Body structure and function is the domain most closely related to the medical model as it refers to the physiological and psychological functions of body systems. Body structures are defined by the ICF as "anatomic parts of the body such as organs, limbs and their components." This domain relates to very specific capabilities, for example being able to lift one's arm over one's head or produce articulate speech sounds. Activities pertain to a wide range of deliberate actions performed by an individual. They are basic deliberate actions undertaken in order to accomplish a task, such as walking or climbing stairs. Participation refers to activities that are integral to economic and social life and the social roles that accomplish that life, such as being able to attend school or hold a job. The dividing line between activities and participation is not distinct. Rather, various actions fall on a spectrum from basic activities to more complex participation. 2 Disability in the ICF and in the social model arises out of activity limitations and restrictions placed upon participation that emerge from the interaction between functional limitations and an unaccommodating environment. For example, a given level of impairment in the body function domain will not necessarily translate into an activity or participation limitation if the environment accommodates a person's different functional status. And environment refers to not only the physical environment but the cultural and political environment, as well. This model is captured in Figure 1 below. The end result of disability comes from participation restrictions that result from the interaction of many factors. Consider a person who had poliomyelitis as a child. Polio is a particular health condition that tends to be associated with certain body function limitations. According to a medical model of disability such as the one on which the DALYs are based, that is where the story ends. But what is the impact of polio on someone's life? How does it affect their ability to undertake a basic activity like walking? The relationship between a body function associated with polio ­ maybe paralyzed legs ­ and the ability to walk is based not only on the particular manifestation of the disease but on environmental factors, for example good roads and curb cuts. Moreover, how does a particular limitation in walking affect a person's ability to participate in society? Accessible transportation and the availability of assistive devices will lessen the link between a limitation in walking and a restriction in employment. So will less stigma and prejudice or the enactment of anti-discrimination laws. So, too, will the availability of a quality education which will hinge on parental resources and the quality and accessibility of the local school system. Therefore, knowing only that a person has had polio tells you very little about their degree of disability, the quality of their life, or their ability to participate in productive activities. 3 1.02 Previous Summary Measures of Functional Status How can we capture the complex nature of a human being's functional status with a summary measure? One example is a tally of activity limitations that encapsulates limitations (without the use of assistive technology) across a broad spectrum of functional domains such as that employed in a recent study of disability in Zambia by Eide and Loeb [21]. In that study, a summary activity limitation score was constructed based on the responses to questions on 43 activities across the nine different functional domains laid out by the ICF [22]. Individual activities were scored on a 5 point scale from (0) no difficulty (1) mild (2) moderate (3) severe difficulty to (4) unable to carry out or perform the activity. Based on this approach, the overall score for an individual could range from zero to 172, with a score of zero indicating no limitation. A frequency distribution of that indicator showed a very broad range of functional limitations (range zero - 162) which is positively skewed, with most people having mild or moderate limitations. A summary participation restriction score was calculated based on the same 43 activities; however, the participation aspect measured performance in the individual's current environment i.e. functioning with the use of assistive technology. This score based on participation restrictions had similar properties to the activity limitation score (possible range zero to 172; survey range zero ­ 161) with the majority of respondents reporting mild to moderate restrictions. Unlike DALYs, these measures can pick up the effects of a public health or other policy interventions, which improves a person's ability to carry out the activities needed to be a member of a community's economic and social life. A person's medical diagnosis can remain the same, but their ability to function in society ­ and thus improve their quality of life ­ can be measured. The summary measures reported in the study by Eide and Loeb [21], however, have some drawbacks. First, they weight all 43 activities equally. Activities as different as learning, making friends, moving, seeing, toileting, eating and drinking, and shopping are treated exactly the same. Some of these are very basic activities, but some are more complex. Some are highly dependent on the environment, but others less so. Moreover, an argument can be made that the same activity is counted more than once, since the activities are not completely distinct or independent. For example, the activity of hearing is included as well as that of understanding others and producing messages. Obviously, difficulty hearing can lead to limitations in communication. While there is certainly more to hearing than communication, this still raises the question as to whether different activity limitations are truly additive. 1.03 New Approach to the Measurement of Functioning: the Activity Limitation Score (ALS) and the Participation Restriction Score (PRS) The Activity Limitation Score (ALS) and Participation Restriction Score (PRS) attempt to draw upon the advantages of the approach taken in Eide and Loeb while at the same time minimizing the problems inherent in that approach. 4 Like the measures described above, the ALS and PRS were also modeled closely on the ICF Checklist [22] which was prepared as an information collection tool by the WHO. The ICF offers several possible ways to operationalize the Activity and Participation lists [13 Annex 3, page 234]. While the ALS and PRS measure the same concepts as Eide and Loeb, activity limitations and participation restrictions, the approach adopted for their measurement is different. In the approach taken by Eide and Loeb all 43 activities listed were considered as both activity and participation domains and were aggregated with equal weights. The alternative used here involves separating the nine domains into two dimensions based on whether they refer to basic activities (sensory experiences, basic learning & applying knowledge, communication and mobility) or activities that are more complex (self-care1, domestic life, interpersonal behaviors, major life activities and community, social & civic life). (See Appendix for a full listing of activity items under each domain.) This approach, delineating basic and complex activities, embraces the conceptualization of functioning as outlined by the UN Washington Group on Disability Statistics [23, 24]. Basic activities, as reflected in the ALS, are measured without the benefit of assistive devices of any kind (whether technical or personal). The complex activities in the PRS, on the other hand, are measured according to how the individual performs in their usual environment (where they normally live, work or play and with the benefit of assistance). Each individual item is scored on a 5 point scale from (0) no difficulty (1) mild (2) moderate (3) severe difficulty to (4) unable to carry out or perform the activity. Certain higher order activities included in the original work by Eide and Loeb [21] were omitted from the ALS as they involve the use of assistive technology: communicating using devices, using transportation to move around and driving a vehicle. Thus the 43 activities considered by Eide and Loeb were reduced to 40; 18 basic activities and 22 complex activities. The approach used then was to create two summary scores, one for each dimension (activity and participation) as the sum of domains which in turn are the sums of individual activity items (under each domain) weighted in order to equalize domain effects. The ALS and PRS were scaled from 0 through 100. Weights were constructed for each domain so that the maximum score for each of the 4 ALS domains was 25 and the maximum score for each of the 5 PRS domains was 20. For example, Sensory Experiences, which is an ALS domain, comprises two activities seeing and hearing. The raw activity limitation score would range from 0 to 8. So each observation was multiplied by 3.125 in order to equalize effects. High scores represent less capacity to perform basic functions and more difficulty in carrying out complex activities. 1Self-care tasks actually fall on the border between basic and complex activities. The analyses presented here included self-care among the basic activities i.e. as part of the ALS; however, the results were very similar when self-care was included as a complex activity, as part of the PRS. 5 The advantages of the ALS and PRS score to other indicators referred to above, are that they: 1. Allow for the variability in functional capacity of people with the same medical diagnoses; 2. Can measure improvements in functionality that result from interventions (such as rehabilitation services) that do not change an underlying diagnosis; 3. Separate out activity limitations from participation restrictions, which also allows the analyst to examine the relationship between these two dimensions; 4. Reduce the degree of counting particular activity limitations more than once; 5. Weight different domains equally, instead of by the number of individual items within a domain. The ALS and PRS still implicitly weight the various functional domains. In fact, they give each domain an equal weight, making mobility equivalent to sensory, for example. The development of an appropriate weighting scheme would have to empirically examine the importance of each activity domain on outcomes of interest, such as employment or poverty. That goes beyond the scope of this paper. However, to gain some insight into the relative importance of each domain, analyses were conducted not just using the overall ALS and PRS, but also using the scores from the individual domains. As shown later, the results by individual domain were similar to the results using the overall ALS and PRS scores. The rest of this paper uses household survey data from Zambia to examine the potential of the ALS and PRS as indicators for, or measures of the impact of, public health interventions. In particular, it examines whether disability ­ as identified by these measures ­ is consistent with previous studies on the prevalence of disability. More importantly, it examines the relationship between the ALS, the PRS, and their components. Zambia was selected because of the high quality of the data available. Disability data in developing countries is often of poor quality [6]. The survey in Zambia represents one of the only data sources that has extensive data on functioning consistent with the WHO's International Classification of Functioning, Disability and Health and also has some data on socio-economic indicators and the receipt of services. 1.04 Testing the Performance of the ALS and the PRS Data for constructing the ALS is generally more available, and easier to make internationally comparable. An important test of the PRS will be to show its relation to standard consumption-based measures of poverty, although there are strong reasons to believe that a poverty line for disabled people should be set higher than for non-disabled people because of the extra costs associated with achieving the same life outcomes [25, 26, 27]. A third issue for investigation is how well the ALS and PRS can measure the effects of public health interventions for people with disabilities. Finally, we will discuss how the correlation between the two measures can be used as an indicator of the inclusiveness of the society. 6 The Zambian survey was implemented in 2005/2006 and undertaken after extensive consultations with key stakeholders, including disabled peoples' organizations [20]. It included both a set of core living conditions indicators, and a detailed assessment of activity limitations faced by people with disabilities. The sample covered 350 Standard Enumeration Areas (SEAs) in the 9 provinces. Data were collected from 5751 non- institutionalized private households in the rural and urban areas of Zambia. The sample is nationally and regionally representative and is expected to yield reliable estimates at provincial, location and national levels. The sample selected for analysis includes those individuals identified as having a disability according to screening procedures based on the work of the UN Washington Group for Disability Statistics [23] and as described in Eide & Loeb [21]. The screening questions used were the six UN Washington Group on Disability Measurement Census Questions that asked for scaled responses to basic activity questions involving walking, seeing, hearing, cognition, communication, and self care. The sample included 1222 individuals 15 to 44 years of age inclusive at the time of the interview; 58% were male and 42% were female. The age range chosen for the analyses presented here was selected in order to facilitate analysis of the participation indicators. A key part of the analysis is to determine the impact of activity limitations on people's ability to participate. Older people are less likely to participate in key life activities, such as work or receiving training, for reasons related as much to social roles as functional ability. The desire was to abstract from this. Future work may use a wider age range and incorporate non-parametric regressions of ALS and PRS on years of age to contextualize these measures. That goes beyond the scope of demonstrating the reasoning behind the ALS and PRS and their potential usefulness. 1.05 Relationship between the ALS and the PRS As mentioned above, both the ALS and PRS were designed as scores with potential range from 0 to 100. Using the Zambian data the ALS ranged from 0 to 91.4 (mean= 16.8, SD=12.0) and the PRS ranged from 0 to 100.0 (mean=15.0, SD=20.9). As seen in Figure 2, both scores were positively skewed, indicating more individuals with lesser degrees of either activity limitation or participation restriction and fewer individuals with severe limitations/restrictions. Scores of 0 were recorded for both ALS and PRS which is indicative of the ability of the screening instrument and the activity and participation scores to identify persons with mild functional limitations ­ who experience very little or no restrictions in their daily activities or their ability to participate. A second feature of these distributions is that they reflect the entire range of limitations / restrictions from the very mild to the very severe ­ placing the burden on the researcher to determine where to set a potential cut-off for the determination of disability. Disability, as depicted here, is a measure of functional ability ­ both a person's physical capacity to carry out certain activities without the aid of assistive technology and their actual performance in their current environment, with assistance. As such they are subject 7 Figure 2: Distribution of the Activity Limitation and Participation Restriction Scores among persons 15-44 years of age, Zambian disability survey ALS distribution PRS distribution to change over time based on the individual's own adaptability, the availability of assistive technology and the accommodations of the environment. Furthermore the measures are independent of whatever medical diagnosis or condition the individual may have. 8 The ALS and PRS are strongly correlated (Pearson correlation coefficient (r) = 0.55, p < 0.001). This correlation should provide an indication of the effect of the environment (physical, social, legislative) on an individual's ability to turn their activity limitation into social participation. A more hostile environment ­ or the lack of assistive technology ­ is expected to lead to a higher correlation between the two indicators. Neither gender nor urban/rural residence is associated with either the ALS or the PRS. Women and men have similar scores; however, PRS scores are lower (though not significantly so; p = 0.06) for women. This may be a reflection of the ability of women to adapt their environment to accommodate the difficulties they face or something more specific about the type of limitations they face. More research is needed to corroborate or refute these findings. Age however appears to be negatively correlated with both scores indicating that younger respondents experience more difficulties than do older respondents. Normally, one would depict the aging population as weaker or more vulnerable in many respects and thus more susceptible to disease, chronic disabling conditions and poorer outcomes in terms of rehabilitation. However, it is important to recall that the population analyzed here is between the ages of 15 and 44 years inclusive. What is probably at work in this sample, is a reflection of the fact that certain types of disabilities that begin developing in middle age tend to be less severe, (e.g. extreme risk taking behavior and the results thereof are trademarks of youth, congenital conditions can sometimes be significant, as are the impacts of malnutrition or severe disease on young children) and that the older population may have had the benefit of becoming accustomed to and adapting to the limitations they acquired when younger. A series of linear regression analyses were carried out to provide insight into the relationship between the ALS and the PRS (overall and PRS major life activities sub- score). In addition, regressions were run using the ALS sub-scores for each component domain. This relaxed the assumption implicit in the main regression that each ALS domain has an equal effect either on the PRS or on a sub-component of the PRS. In addition the usefulness of the PRS (and PRS sub-scores) in predicting poverty was also assessed. Each of these regression analyses adjusted for age, gender and urban/rural location. With PRS as the dependent variable, it is apparent that the ALS and each ALS component (both together and independently) are correlated with social participation ­ as measured by the overall PRS (See Table 1). The same results are shown when only looking at major life activities ­ indeed we observe a similar general trend in terms of size of predictive contribution (See Table 2). The ALS leaning sub-score is in all cases the most powerful predictor, followed by the mobility, communication, and sensory sub-scores. 9 Table 1: Linear regression results: Dependent Variable is overall PRS, persons 15- 44 years of age in Zambian disability survey Model (1) Model (2) B p-value B p-value Intercept 5.07 0.06 5.31 0.02 Age in years -0.09 0.16 -0.08 0.16 Sex -1.75 0.11 -0.91 0.32 Urban/Rural -2.17 0.05 -0.89 0.34 ALS 0.95 <0.001 ALS sensory -0.38 <0.001 ALS learn 1.97 <0.001 ALS communication 0.53 <0.001 ALS mobility 1.17 <0.001 Unstandardized beta coefficient Table 2: Linear regression results: Dependent variable is PRS for Major Life Activities, persons 15-44 years of age in Zambian disability survey Model (1) Model (2) B p-value B p-value Intercept 1.13 0.11 1.47 0.02 Age in Years -0.03 0.08 -0.02 0.16 Sex -0.39 0.19 -0.20 0.44 Urban/Rural -0.25 0.40 0.01 0.97 ALS 0.21 <0.001 ALS sensory -0.07 0.02 ALS learn 0.47 <0.001 ALS communication 0.09 <0.001 ALS mobility 0.14 <0.001 Unstandardized beta coefficient Surprisingly, the sensory sub-score is negatively correlated with the dependent variable. This finding is consistent with some preliminary analyses reported from Vietnam's Household Living Standards Survey, which found a positive correlation between poverty and functional limitations in every domain except vision, where there was a negative correlation [28]. Two reasons might account for this finding. First, sensory difficulties are seldom alone in determining complex participation restrictions - i.e. these complex participation variables are measured on multiple domains. For example, vision difficulties can affect people's ability to walk. Hearing difficulties might impact a person's communication ability. Therefore, the component of sensory difficulties that are independent of the other domains may not have a particularly negative impact on participation. This also implies a certain degree of lack of independence between domains of the ALS. The ALS and 10 PRS presented here contain less counting of the same activity more than once than previous incarnations of the measure. Another reason could be a result of differences in self-reporting. People with more income and/or higher education may have a greater need for their eyesight because of the particular demands their level of social participation places upon them. They may therefore be more inclined to report mild or moderate difficulties seeing. Alternatively, in less techno-dependent cultures or rural societies where the demands placed on the individual are less, there may be less incentive to self-report mild or moderate seeing difficulties. This could have some impact on the resultant negative correlation observed in Table 2. 1.06 Relationship between PRS and Poverty We used a derived asset index [27] to assess the relationship between the PRS and poverty. The asset index is a composite consumption based indicator composed of: eight specific items a household might possess: the source of water for the household, type of toilet facility, source of power for cooking and lighting, and a composite measure of housing standard. A high asset index is indicative of greater material wealth. In Table 3 we observe that the PRS (Model 1) is negatively and significantly correlated with the assets index (taking into account the moderating effects of demographic background variables) ­ the greater the participation restrictions experienced the poorer the person's household. As might be expected in this set of regressions the effect of urban/rural location is much more prominent. When the PRS is broken down into its components (Model 2), we find no significant correlation between the asset index and the self-care and domestic activity components, and a small but statistically significant positive correlation between the life activities component. There was also observed a statistically significant negative relationship between the PRS interpersonal relationships component and the asset index. These are difficult to interpret, however, because of the complex nature of the interaction of various participation restrictions. These restrictions might also influence factors like living independently or with extended families which will influence the assets in the disabled person's household. More detailed work is needed in this area, but initial indications do show a link between the PRS and outcomes associated with poverty. 11 Table 3: Linear regression results: Dependent variable is Assets Index (a measure of poverty), persons 15-44 years of age in Zambian disability survey Model (1) Model (2) B p-value B p-value Intercept -0.44 0.61 -0.36 0.68 Age in years -0.05 0.02 -0.05 0.01 Sex 0.13 0.72 0.11 0.76 Urban/Rural 5.97 <0.001 5.94 <0.001 PRS -0.04 <0.001 PRS self-care -0.10 0.13 PRS domestic -0.01 0.90 PRS interpersonal -0.13 0.01 PRS life activities 0.11 0.04 PRS social participation -0.09 0.08 Unstandardized beta coefficient 1.07 Using the ALS and the PRS to assess the effects of public health interventions ­ A key motivation behind the development of the ALS and PRS was to determine if they are capable of recording effects of public health interventions that do not change an underlying medical diagnosis. To that end, we examined the differences in ALS and PRS between people receiving various types of services. These services could include physical rehabilitation or occupational therapy, or training in how to communicate or use assistive devices, for example. These could be correlated with decreases in the ALS or PRS, but have no impact on DALYs. Individuals identified in the survey as having a disability were asked whether they needed a variety of services. Those who indicated that they needed a service were subsequently asked whether or not they had received the service. These responses were then analyzed with respect to their respective ALS and PRS scores in the table below. Results are presented as mean ALS or PRS for those receiving or not receiving the service in question among those who indicated that they needed the service. Table 4 indicates that receiving a service reduced the mean ALS scores (i.e. lower ALS = improved capacity). Recalling that ALS is measured without the benefit of accommodations, it might seem unlikely that the provision of assistive device services for example, would improve, or lower, the mean ALS score. However, other services ­ such as surgery ­ could potentially have an impact on the ALS. Moreover, the impact of service provision may have wider ranging implications to an individual's capacity as measured by the ALS than the mere provision of the service would imply. This theory would need to be addressed in future research. 12 Table 4: Mean ALS and PRS by receipt of Health Interventions among persons 15- 44 years of age who needed the service, Zambian disability survey Mean ALS Mean PRS Received the service: Yes No p-value Yes No p-value Rehabilitation service 15.4 17.2 0.03 12.0 16.0 0.003 Assistive device service 14.8 17.1 0.01 8.7 15.9 <0.001 Counseling service 12.7 17.2 <0.001 9.1 15.5 <0.001 Health service 15.9 18.0 0.007 14.3 15.9 ns Results presented for PRS show substantially larger effects ­ with the exception of health services. Mean PRS scores are significantly lower among those who received the service (except for health services) ­ indicating that service provision significantly improves social participation, which we know is linked with reductions in poverty. In other words, the effect of health interventions and other interventions can be ascertained by use of these indicators of participation. To truly gage the effectiveness of these interventions, it is important to model who receives them ­ and thus control for self-selection or endogeneity. The results in Table 3 about the relationship between disability and poverty suffer from the same problem. Poverty could cause a restriction in participation just as much as restricted participation could lead to poverty. In the case of the effect of services, we used a standard two-stage Heckman technique [29] to correct for possible selection bias, using urban/rural as the identifying variable, since it is correlated with availability of services but not necessarily with the effectiveness of those services. The results remained essentially unchanged as can be seen in Table 5, which reports the second stage of the estimation. For example, receiving rehabilitative services ­ controlling for age, gender, and selection bias ­ reduces the participation score by over 4 points. Receiving assistive devices reduces the PRS by over 5 points, and counseling by over 8. Regular health services still have no significant impact on participation. The Mills Ratio variable ­ lambda ­ is not significant in any of the regressions, suggesting selection bias may not be a significant issue. A more sophisticated attempt to deal with the endogeneity problem goes beyond the scope of this paper. Our purpose is to demonstrate how the ALS and PRS could potentially be used to fill a gap in evaluating public health programs that is not addressed by DALYs, mortality, or any other available health measure. 13 Table 5: Second Stage Regression Results for Effect of Services on PRS Rehabilitative Assistive Counseling Health Services Devices B p- B p- B p- B p- value value value value Intercept -24.34 0.42 10.55 0.17 14.03 0.08 -64.94 0.10 Received Services -3.85 0.02 -4.83 0.01 -7.69 0.001 0.57 0.73 Age -0.26 0.03 -0.11 0.22 -0.18 0.12 -0.51 0.01 Male -4.69 0.02 -2.72 0.08 -4.61 0.01 -3.07 0.03 Lambda 24.43 0.11 7.64 0.09 8.50 0.12 253.89 0.03 Unstandardized beta coefficient Say, for example, we had a demonstration project looking at providing physical rehabilitation services to victims of traffic accidents with correctly constructed treatment and control groups. The ALS and PRS could be used in a variety of ways. 1) Participation in the program could be correlated with the ALS to see how accessible the program was. Controlling for other factors, if people with higher ALS scores used the program less, that would be a sign that there were barriers preventing them from accessing services 2) Changes in their ALS could be a measure of how effective the program was at targeting improvements at the activity level. Medical assessments often focus solely on changes in body function, for example in the case of surgery. 3) Changes in the PRS will determine how effective the program is at integrating people into society. 2.0 Conclusions The ALS and PRS indicators presented in this paper address a fundamental gap in what is measured by DALYs, by being sensitive to changes in people's functional status resulting from interventions that do not change an underlying medical diagnosis. Moreover, by examining the relationship between the ALS and PRS it is possible to explore the extent to which environmental conditions help translate functional limitations into disabilities, within the social model context. A country with a stronger correlation between the two indicators has a less accommodating environment. Similarly, changes in the correlation between the ALS and PRS over time within a country could be an indicator of either an improvement in the accessibility of the environment or a worsening of the situation, as in the case of fragile states, or crisis situations like Hurricane Katrina. This could be an important approach to measuring the impact of various programs and policies. Data from Zambia demonstrated how the ALS and PRS can depict the range of functioning within a country, and that there are currently significant correlations between 14 the ALS, PRS, and poverty in Zambia as is likely the case in many countries especially developing countries. More research is needed to refine and better understand the associations among these and other indicators. More detailed investigations need to address the appropriate weighting of the various sub-components of the ALS and PRS, taking into account those sub- components influence on outcome variables of interest. Also, the unexpected negative correlation between the sensory ALS and the PRS are suggestive of the complicated relationships across and within functional domains and the nature of self-reporting. However, we believe this is an important step towards developing a public health indicator that can be used to supplement some of the deficiencies of DALYs. 15 16 References 1. Braithwaite, J. and D. Mont, "Disability and Poverty: A Survey of World Bank Poverty Assessments and Implications, " SP Discussion Paper, February 2008 2. Elwan, A., "Poverty and Disability: A Survey of the Literature," SP Discussion Paper No. 9932, The World Bank, December 1999 3. Hoogeveen, J.G., "Measuring Welfare for Small but Vulnerable Groups: Poverty and Disability in Uganda," Journal of African Economies, 14:4, pp. 603-631, August 2005. 4. Yeo, R. and K. Moore, "Including Disabled People in Poverty Reduction Work: Nothing About Us, Without Us," World Development, 31:3, pp571-590, 2003 5. Department for International Development, UK, Disability, Poverty and Development, Issues Note, 2000 6. Mont, D. "Measuring Disability Prevalence," SP Discussion Paper No. 0706, The World Bank, March 2007 7. UN Millennium Development Goals Website, http://www.un.org/millenniumgoals/ 8. UN Convention Website, http://www.un.org/disabilities/convention/index.shtml 9. Mont, D., "Measuring Health and Disability," The Lancet, 369, pp.1658-63, May 2007b 10. World Bank. World Bank Development Report 1993: Investing in Health. New York: Oxford University Press, 1993 11. Fox-Rushby, J., "Disability Adjusted Life Years for Decision-Making? An Overview of the Literature," London: Office of Health Economics, 2002 12. Gold, M.R., Stevenson, D., Fryback, D.G., "HALYs and QALYs and DALYs, Oh My: Similarities and Differences in Summary Measures of Population Health," Annual Review of Public Health 2002; 23: pp. 115-34 13. International Classification of Functioning, Disability and Health: ICF, World Health Organization, Geneva, 2001. ICF Website, http://www.who.int/classifications/icf/site/onlinebrowser/../index.cfm . 14. Sen, A. K., The Living Standard," Oxford Economic Papers, 36, pp.74-90, 1984. 15. Sen, A.K. Commodities and Capabilities, North Holland, Amsterdam, 1985. 16. Sen, A.K., "Capability and Well-Being," in M. Nussbaum and A.K. Sen, eds. The Quality of Life, pp. 30-53, Clarendon Press, Oxford, 1993. 17. Sen, A.K., Development as Freedom, Oxford University Press, Oxford, 1999. 18. Hughes B, Paterson K. The social model of disability and the disappearing body: towards a sociology of impairment. Disabil Soc, 1997; 12: pp. 325­40. 19. Shakespeare T, Watson N. Defending the social model. Disabil Soc 1997; 12: pp. 293­300. 20. Altman, D. "Sociology and Disability," in Albrecht, G., M. Bury, eds. Handbook of Disability Studies, New York: Russell Sage, 2002 21. Eide, A.H. and M. Loeb (eds), "Living Conditions among People with Activity Limitations in Zambia: A National Representative Study", SINTEF Health Research, August 2006, http://www.sintef.no/lc 22. ICF Website, ICF Checklist, http://www.who.int/classifications/icf/site/checklist/icf-checklist.pdf 17 23. UN Washington Group on Disability Statistics (WG) Website, http://www.cdc.gov/nchs/citygroup.htm 24. Altman, BM, Rasch EK, Madans JH. Disability Measurement Matrix: A tool for the coordination of measurement purpose and instrument development, in B Altman & S Barnartt eds. International views on Disability measures: Moving towards comparative measurement. Research in Social Science and Disability, 4: pp. 263-284. Elsevier Ltd. Oxford, 2006. 25. Kuklys, W., Amartya Sen's Capability Approach: Theoretical Insights and Empirical Applications, Studies in Choice and Welfare, Springer-Verlag, 2005 26. Tibble, M. "Review of the existing research on the extra costs of disability." Department for Work and Pensions Working Paper. No. 21. 2005 27. Filmer D, Pritchett LH., Estimating wealth effects without expenditure data - or tears: an application to educational enrolments in states of India. Demography; 38:1, pp. 115­32, February 2001. 28. Phong, N., "Experience Operationalizing a Survey with a Disability Module: Vietnam Case," presented at Workshop on Strengthening Disability Measurement in South Asian Countries, Bangkok, Thailand, April 2-4, 2008 29. Heckman, J. Sample Selection Bias as Specification Error, Econometrica, 47:1, January 1979. 18 Appendix: Domains and activities included in the ALS and PRS ACTIVITY LIMITATION SCORE PARTICIPATION RESTRICTION SCORE How difficult is it for you to perform this activity without any Do you have any difficulty performing this activity in your kind of assistance at all? current environment? (Without the use of any assistive devices ­ either technical or (Current environment: where you live, work, and play etc for the personal). majority of your time ­ and with the use of any assistive devices, either technical or personal.) SENSORY EXPERIENCES SELF CARE a. watching/looking/seeing a. washing oneself b. listening/hearing b. care of body parts, teeth, nails and hair BASIC LEARNING & APPLYING KNOWLEDGE c. toileting a. learning to read/write/count/calculate d. dressing and undressing b. acquiring skills (manipulating tools, painting, carving etc.) e. eating and drinking c. thinking/concentrating DOMESTIC LIFE d. reading/writing/counting/calculating a. shopping (getting goods and services) e. solving problems b. preparing meals (cooking) COMMUNICATION c. doing housework (washing/cleaning) a. understanding others (spoken, written or sign language) d. taking care of personal objects (mending/repairing) b. producing messages (spoken, written or sign language) e. taking care of others c. communicating directly with others INTERPERSONAL BEHAVIOURS MOBILITY a. making friends and maintaining friendships a. staying in one body position b. interacting with persons in authority (officials, village chiefs) b. changing a body position (sitting/standing/bending/lying) c. interacting with strangers c. transferring oneself (moving from one surface to another) d. creating and maintaining family relationships d. lifting/carrying/moving/handling objects e. making and maintaining intimate relationships e. fine hand use (picking up/grasping/manipulating/releasing) MAJOR LIFE AREAS f. hand & arm use (pulling/pushing/reaching/throwing/catching) a. going to school and studying (education) g. walking b. getting and keeping a job (work & employment) h. moving around (crawling/climbing/running/jumping) c. handling income and payments (economic life) COMMUNITY, SOCIAL AND CIVIC LIFE a. clubs/organisations (community life) b. recreation/leisure (sports/play/crafts/hobbies/arts/culture) c. religious/spiritual activities d. political life and citizenship 19 Social Protection Discussion Paper Series Titles No. Title 0815 Beyond DALYs: Developing Indicators to Assess the Impact of Public Health Interventions on the Lives of People with Disabilities by Daniel Mont and Mitchell Loeb, May 2008 0814 Enforcement of Labor Regulation and Firm Size by Rita Almeida and Pedro Carneiro, May 2008 (online only) 0813 Labor Markets Lending and Analytical Work at the World Bank: FY2002- 2007 by Milan Vodopivec, Jean Fares and Michael Justesen, May 2008 0812 Risk and Vulnerability Analysis in the World Bank Analytic Work: FY2000- 2007 by Valerie Kozel, Pierre Fallavier and Reena Badiani, May 2008 0811 Pension Lending and Analytical Work at the World Bank: FY2002-2007 by Richard Hinz, Melike Egelmelzer and Sergei Biletsky, May 2008 (online only) 0810 Social Safety Nets Lending and Analytical Work at the World Bank: FY2002-2007 by Margaret Grosh and Annamaria Milazzo, May 2008 0809 Social Funds as an Instrument of Social Protection: An Analysis of Lending Trends - FY2000-2007 by, Samantha De Silva and June Wei Sum, May 2008 0808 Disability Activities at the World Bank: FY2002-2007 by Jeanine Braithwaite, Richard Carroll, and Karen Peffley, May 2008 0807 Migration, Labor Markets, and Integration of Migrants: An Overview for Europe by Rainer Münz, April 2008 (online only) 0806 Is the Window of Opportunity Closing for Brazilian Youth? Labor Market Trends and Business Cycle Effects by Michael Justesen, April 2008 0805 Disability and Poverty: A Survey of World Bank Poverty Assessments and Implications by Jeanine Braithwaite and Daniel Mont, February 2008 0804 Poverty Traps and Social Protection by Christopher B. Barrett, Michael R. Carter and Munenobu Ikegami, February 2008 0803 Live Longer, Work Longer: Making It Happen in the Labor Market by Milan Vodopivec and Primoz Dolenc, February 2008 (online only) 0802 Disability in Kazakhstan: An Evaluation of Official Data by Ai-Gul S. Seitenova and Charles M. Becker, February 2008 (online only) 0801 Disability Insurance with Pre-funding and Private Participation: The Chilean Model by Estelle James, Augusto Iglesias and Alejandra Cox Edwards, January 2008 0719 The Life-Course Perspective and Social Policies: An Issues Note by A.L. Bovenberg, November 2007 0718 Social Safety Nets and Targeted Social Assistance: Lessons from the European Experience by Chris de Neubourg, Julie Castonguay and Keetie Roelen, November 2007 (online only) 0717 Informality and Social Protection: Preliminary Results from Pilot Surveys in Bulgaria and Colombia by Franco Peracchi, Valeria Perotti and Stefano Scarpetta, October 2007 (online only) 0716 How Labor Market Policies can Combine Workers' Protection with Job Creation: A Partial Review of Some Key Issues and Policy Options by Gaëlle Pierre and Stefano Scarpetta, October 2007 (online only) 0715 A Review of Interventions to Support Young Workers: Findings of the Youth Employment Inventory by Gordon Betcherman, Martin Godfrey, Susana Puerto, Friederike Rother, and Antoneta Stavreska, October 2007 0714 Performance of Social Safety Net Programs in Uttar Pradesh by Mohamed Ihsan Ajwad, October 2007 0713 Are All Labor Regulations Equal? Assessing the Effects of Job Security, Labor Dispute and Contract Labor Laws in India by Ahmad Ahsan and Carmen Pagés, June 2007 0712 Convention on the Rights of Persons with Disabilities: Its Implementation and Relevance for the World Bank by Katherine Guernsey, Marco Nicoli and Alberto Ninio, June 2007 0711 Reaching the Poor and Vulnerable: Targeting Strategies for Social Funds and other Community-Driven Programs by Julie Van Domelen, May 2007 0710 The Macedonia Community Development Project: Empowerment through Targeting and Institution Building by Caroline Mascarell, May 2007 0709 The Nuts and Bolts of Brazil's Bolsa Família Program: Implementing Conditional Cash Transfers in a Decentralized Context by Kathy Lindert, Anja Linder, Jason Hobbs and Bénédicte de la Brière, May 2007 (online only) 0708 Globalization and Employment Conditions Study by Drusilla K. Brown, April 2007 0707 The Kosovo Pension Reform: Achievements and Lessons by John Gubbels, David Snelbecker and Lena Zezulin, April 2007 (online only) 0706 Measuring Disability Prevalence by Daniel Mont, March 2007 0705 Social Safety Nets in World Bank Lending and Analytic Work: FY2002- 2006 by Annamaria Milazzo and Margaret Grosh, March 2007 (online only) 0704 Child Labor and Youth Employment: Ethiopia Country Study by Lorenzo Guarcello and Furio Rosati, March 2007 0703 Aging and Demographic Change in European Societies: Main Trends and Alternative Policy Options by Rainer Muenz, March 2007 (online only) 0702 Seasonal Migration and Early Childhood Development by Karen Macours and Renos Vakis, March 2007 0701 The Social Assimilation of Immigrants by Domenico de Palo, Riccardo Faini and Alessandra Venturini, February 2007 (online only) 0616 Pension Systems in Latin America: Concepts and Measurements of Coverage by Rafael Rofman and Leonardo Lucchetti, November 2006 (online only). Also available in Spanish. 0615 Labor Market Outcomes of Natives and Immigrants: Evidence from the ECHP by Franco Peracchi and Domenico Depalo, November 2006 (online only) 0614 The Relative Merits of Skilled and Unskilled Migration, Temporary and Permanent Labor Migration, and Portability of Social Security Benefits by Johannes Koettl under guidance of and with input from Robert Holzmann and Stefano Scarpetta, November 2006 (online only) 0613 The Limited Job Prospects of Displaced Workers: Evidence from Two Cities in China by Gordon Betcherman and Niels-Hugo Blunch, October 2006 0612 Unemployment Insurance in Chile: A New Model of Income Support for Unemployed Workers by Germán Acevedo, Patricio Eskenazi and Carmen Pagés, October 2006 0611 Evaluating Social Fund Impact: A Toolkit for Task Teams and Social Fund Managers by Sarah Adam, October 2006 0610 Risk and Vulnerability Considerations in Poverty Analysis: Recent Advances and Future Directions by Carlo Cafiero and Renos Vakis, October 2006 0609 Comparing Individual Retirement Accounts in Asia: Singapore, Thailand, Hong Kong and PRC by Yasue Pai, September 2006 (online only) 0608 Pension System Reforms by Anita M. Schwarz, September 2006 (online only) 0607 Youth Labor Market in Burkina Faso: Recent Trends by Daniel Parent, July 2006 0606 Youth in the Labor Market and the Transition from School to Work in Tanzania by Florence Kondylis and Marco Manacorda, July 2006 0605 Redistributing Income to the Poor and the Rich: Public Transfers in Latin America and the Caribbean by Kathy Lindert, Emmanuel Skoufias and Joseph Shapiro, August 2006 (online only) 0604 Uninsured Risk and Asset Protection: Can Conditional Cash Transfer Programs Serve as Safety Nets? by Alain de Janvry, Elisabeth Sadoulet, Pantelis Solomon and Renos Vakis, June 2006 0603 Examining Conditional Cash Transfer Programs: A Role for Increased Social Inclusion? by Bénédicte de la Brière and Laura B. Rawlings, June 2006 (online only) 0602 Civil-service Pension Schemes Around the World by Robert Palacios and Edward Whitehouse, May 2006 (online only) 0601 Social Pensions Part I: Their Role in the Overall Pension System by Robert Palacios and Oleksiy Sluchynsky, May 2006 (online only) To view Social Protection Discussion papers published prior to 2006, please visit www.worldbank.org/sp. Summary Findings Two indicators -- the Activity Limitation Score (ALS) and the Participation Restriction Score (PRS) are presented in this paper for use in assessing the impact of public health interventions on the lives of disabled people. They address a gap in the Disability-Adjusted Life Years (DALY) approach which is not sensitive to changes in people s functional status resulting from interventions that do not change an underlying medical diagnosis. Household data from Zambia are used to explore the potential usefulness of these measures. HUMAN DEVELOPMENT NETWORK About this series... Social Protection Discussion Papers are published to communicate the results of The World Bank's work to the development community with the least possible delay. The typescript manuscript of this paper therefore has not been prepared in accordance with the procedures appropriate to formally edited texts. The findings, interpretations, and conclusions expressed herein are those of the author(s), and do not necessarily reflect the views of the International Bank for Reconstruction and Development / The World Bank and its affiliated organizations, or those of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. For free copies of this paper, please contact the Social Protection Advisory Service, The World Bank, 1818 H Street, N.W., Room G7-703, Washington, D.C. 20433 USA. Telephone: (202) 458-5267, Fax: (202) 614-0471, E-mail: socialprotection@worldbank.org or visit the Social Protection website at www.worldbank.org/sp.