WORLD BANK SOCIAL PROTECTION POLICY NOTE November 2018 No. 16 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward Rashiel B. Velarde1 This Note chronicles the Philippine’s experience in undertaking a national household targeting system— Listahanan—which has had two rounds of implementation since developed in 2007. Besides the main operational features of the system, the data used, enhancements made in the 2015 round, the Note also highlights the process as well as key decisions that were made in selecting the final technical design of both rounds that allow the system to identify and locate poor households across the Philippines. It offers some recommendations on how the Philippine targeting system can maintain its functionality and relevance as it matures. This Note contributes to the body of global knowledge on establishing effective targeting systems that other countries can use as a resource. I. Introduction The Government of the Philippines (GoP) has made significant progress in developing the social protection sector over the last decade, which has been one of the main strategies to address poverty and vulnerability of Filipinos. One that is hailed as a pivotal development has been the introduction of a national household targeting system—popularly known an Listahanan. The Listahanan is a tool developed by the Department of Social Welfare and Development (DSWD) that allows it to identify poor Filipinos—the main clients of the agency’s social assistance programs—using a standard and internationally accepted method, applying what has been done at the time in model Latin American countries. The Listahanan enabled the creation of a registry of poor Filipinos across the country that was used to identify poor beneficiaries of the GoP’s largest social protection programs—the Pantawid Pamilyang Pilipino Program conditional cash transfer (CCT) and Philhealth universal health care (UHC) program—and their subsequent massive expansion. More formally known as the National Household Targeting System for Poverty Reduction (NHTS-PR), the Listahanan was adopted as the Government’s primary mechanism for targeting poor households for social protection programs in March 2010 through Executive Order 867 (EO 867). EO 867 maintained the DSWD as its implementing agency and the repository of the Listahanan registry. It further allowed the DSWD to update this registry every four (4) years. Now the registry contains records of 15 million households nationwide (or about 75 percent of the population), of which 5.2 million households had been classified as “poor” and prioritized for various Government assistance programs. Besides the Pantawid Pamilya CCT program and the UHC, many other programs have used the Listahanan since its adoption in 2010. The significance of establishing the Listahanan and its achievements extend far beyond the country. The Philippine Social Protection Grounded in empirical studies, the experience of Listahanan is as a leading “good practice example” for a Note series aims to disseminate experiences, good practices, and key growing number of countries in East Asia and the Pacific as well as in other regions that aspire to introduce findings from the Philippines on the such registries. This Note documents the main features, processes, and key decision points involved in the topics related to social protection. It design and continuous enhancement of the Philippine household targeting system as a contribution to also aims to broaden the dialogue on the body of global knowledge on targeting systems. social protection and stimulate public engagement in moving forward the policy agenda. The views expressed here are those of the authors and do 1 Rashiel B. Velarde, Economist, Social Protection & Jobs, World Bank Manila. This Policy Note was prepared with excellent not necessarily reflect those of the Research Assistance of Angelo Gabrielle Santos. The Author thanks the National Household Targeting Office (NHTO) of the World Bank. Department of Social Welfare and Development (DSWD) for their review and inputs in completing this Policy Note. Any other remaining errors are the Author’s alone. www.worldbank.org 1 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward II. THE DEVELOPMENT OF THE PHILIPPINES’ FIRST HOUSEHOLD TARGETING SYSTEM The Listahanan, or the Philippine Government’s national household targeting system for poverty reduction (NHTS-PR), is a proxy means test (PMT)-based targeting system. As such, it uses a uniform set of proxy or alternative indicators to assess one’s economic welfare instead of directly asking for a household’s or individual’s income or consumption. Proxy Means Test (PMT) is a statistical tool or model that aggregates multiple proxy indicators into a single score that represents the individual or household’s welfare level. The score is used to determine eligibility to receive benefits from public programs. In the case of the Philippines, the score represents the household’s per capita income. Household composition, socio-economic characteristics, assets, housing conditions and tenure status, education, access to basic services, and regional variables are commonly used indicators in PMT models. Developing a PMT-based targeting system was a decision taken by the Philippine Government considering experience in other countries. The appropriateness of such a system for the Philippines was acknowledged mainly for the following reasons. • First, PMT models are widely considered as a straightforward, practical, and reliable way of measuring economic wellbeing or poverty level. In countries like the Philippines where micro-datasets abound and are easily accessible, developing a PMT does not pose difficulties. With the use of proxy variables that are available in regular household surveys and censuses, the government can predict the welfare level of a household and easily rank this side by side existing poverty standards. • Second, while a PMT-based targeting system offers objectivity and uniformity, it still gives the Government enough flexibility to identify various groups whom it can prioritize for specific forms of assistance. PMTs are based on a scientific (statistical) process of using multiple welfare indicators that have standard definitions and measurements. It thus offers an opportunity to minimize discretionary and subjective, often politicized, means of identifying specific groups within the population, specifically the poor. At the same time, PMTs allow the use of different cut-offs to identify specific sub-groups of the poor (food poor, chronic poor) or vulnerable groups (“just above” the poverty threshold) who may qualify for specific targeted assistance. • Third, the performance of similar PMT-based targeting systems in Colombia (SISBEN), Mexico (Oportunidades), Chile (Ficha de Informaćion Social), and other Latin American countries provided real case examples for the Philippines and encouraged the Government to pilot a similar system in 2007. There is evidence that PMT has worked well in these countries with large informal sectors and where actual incomes are not well documented and difficult to verify—a similar case to the Philippines. The informal sector accounts for nearly half of total employment in the Philippines in 2006 and systems are not in place to cross-validate reported incomes. While PMT-based household targeting was first introduced in the Philippines through the Listahanan, PMT models are not entirely new to the Philippines. Since 2005, the National Statistical Coordination Board (NSCB, now Philippine Statistical Authority or PSA), with assistance from the World Bank, has been producing poverty maps or municipal poverty estimates using PMT models. The first poverty maps capitalized on the 2000 Census of Population and Housing and 2000 merged FIES and LFS surveys—officially called the 2000 Small Area Poverty Estimates (2000 SAPE). Since then, poverty maps were also produced for 2003, 2006, 2009, and 2012 using more updated household surveys and censuses. The SAPEs, specifically those in 2000 and 2003, were used to guide the formulation of the Philippines’ first PMT models for household targeting, as detailed below. To align with the Philippines’ official poverty estimation methodology, the Listahanan’s PMT predicts income as a welfare measure. The Philippines is one of the remaining few countries that uses an income-based poverty estimation methodology, although there has been a longstanding debate within Government and the statistical community to move towards using consumption or expenditure. As an analytical exercise, the initial PMT models for Listahanan were formulated to predict both per capita income and per capita expenditure (or more accurately income and expenditure per household member, respectively). These models turned out to be very similar, mainly because income and expenditure data in the household surveys are highly correlated (coefficient of 0.9). In the end, the Government opted to keep the income models to be consistent with its official poverty methodology. In finally determining who is poor, Listahanan’s PMT results are compared with official poverty thresholds. As the PMT only predicts income, it cannot identify the poor by itself. It needs to be measured against a standard cut-off point that defines who is poor or not. The Philippine Government separately releases poverty thresholds for every province which are used to generate official poverty estimates. Predicted per capita income for each household based on the PMT is compared against the applicable poverty threshold for the province it belongs to determine if the household falls below (poor) or above it (not poor). 2 WORLD BANK SOCIAL PROTECTION POLICY NOTE The First PMT Models and Data Sources The first PMT models were developed in 2007 under the leadership of the World Bank and in collaboration with local academics. At the time, DSWD has just started to implement its internal reforms, which is supported by a World Bank grant. Part of the assistance is to enhance the internal capacity of DSWD staff to execute its ambitious reforms, which include designing an effective household targeting system that will enable it design more direct household-based assistance programs. International practitioners and experts in targeting systems were hired through the WB grant to assist DSWD in developing its first PMT models for household targeting and in designing the strategy for its subsequent nationwide roll-out. Local academics were engaged to provide guidance on the country context, available poverty assessments, data sources, and technical inputs in designing the PMT models for households targeting. Like in countries with PMT-based household targeting systems, the first PMT models for the Philippines contained a combination of household- and individual-level indicators (Figure 1). Socio-economic indicators that are used in PMT models generally include household demographics, education and occupation of household members, housing conditions, access to basic services, asset ownership, and location. These indicators are relatively easy to collect because they are mostly observable to the interviewer or can be easily verifiable by any third party observer, and thus difficult to be misreported by the interview respondent. These indicators are available in the Philippine FIES and LFS. Figure 1. The First PMT model specification log (y)u = au + bu Xhu + cu Zhi u + u log (per capita income) = structural parameter + household-specific indicators + individual-specific indicators + estimation errors where u = urban or rural h = household identifier i = individual identifier in a household In 2007, available micro-datasets for the Philippines are the Family Income and Expenditure Survey (FIES) and the Labor Force Survey (LFS) of 2003. The Philippines has been producing the triennial FIES and the quarterly LFS since the 1950s. The FIES is used mainly for official poverty estimation while the LFS for employment statistics. Both surveys are publicly accessible for a small fee either in separate forms or in one merged/combined file. The FIES would have about 38,000 to 42,000 household sample at any given survey year while the LFS has about the same number of sample households at any given survey quarter. These two micro-datasets were chosen as the main data sources to formulate the Philippine PMT models because, combined, they hold the largest collection of qualitative and quantitative household- and individual-level information that can be used to construct models to predict income (or expenditure) and poverty status of individuals or households. However, until the recent past, it takes 18-24 months after the FIES surveys were completed for the full data to be publicly available due to capacity issues in the statistical agencies. Although a FIES 2006 survey was already completed when the PMT formulation started in 2007, it took another 18 months (June 2008) for the dataset to be publicly available. Third-party household datasets, particularly those from the Community-Based Monitoring System (CBMS), were also explored. However, available data proved to be dated, sparse, and varied in municipalities where they exist, and therefore cannot be used as a standard data source for the PMT. The first PMT models of Listananan were based on official urban-rural poverty thresholds of 2003. Poverty thresholds do not directly figure in the PMT models. Even so, they are used in guiding the formulation of the PMT by way of estimating consequent ex-ante model precision in estimating overall poverty and classifying poor (and non-poor) households, which is the ultimate use of the Philippine PMT models. The Philippine’s official methodology calls for separate urban and rural poverty threshold in every province. However, up until 2012, this full set of provincial poverty thresholds with urban-rural disaggregation were not publicly released in Government websites. As a result, the poverty thresholds used to classify poor households during the course of developing the first PMT models for the Philippines depended on what was available at that time. Ex-ante model precision in classifying poor and non-poor households of the first PMT models in 2007 were thus based on available 2003 national urban and rural poverty thresholds. Over the course of the first Listahanan implementation, more disaggregated poverty thresholds (provincial average) were used accordingly as they become available. The Listahanan classifies a household as “poor” if the predicted income by the PMT model is below the official provincial poverty threshold. The household is considered “not poor” otherwise. www.worldbank.org 3 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward The first PMT models were guided by official poverty maps, which they intended to complement in turn. The Listahanan was not designed to duplicate, replace, nor dispute existing official poverty estimates. Government official poverty estimates are available only every three years and are meaningful only at the provincial level. Meanwhile, poverty maps (SAPE) that are used at the municipal/city-level are released with long delays. Listahanan is intended to complement available poverty numbers by drilling further down the community and household-level. Experience from other countries implementing PMT-based targeting systems has shown that they work more effectively when household (or individual) assessment is complemented with geographic selection of poor areas based on poverty maps.2 This strategy was made possible in the Philippines with the combination of official provincial poverty estimates and municipal/city poverty maps. When the targeting system was being piloted in 2007, NSCB (now PSA) has just released its first municipal/city-level poverty estimates using the results of the 2003 national household surveys and 2000 census. The 2003 SAPE was useful in developing the Listahanan in two ways. First, the PMT model for municipal/ city poverty estimates yielded a high goodness-of-fit coefficient in explaining variations in income (R2 of 71 percent). It thus provided a good starting point to develop the PMT for household targeting. The first PMT models for Listahanan did, in fact, incorporate several indicators used in the 2000 SAPE. Second, municipal/city-level poverty estimates in the 2003 SAPE were used to design a more strategic plan for the Listahanan enumeration process. It was used as basis in identifying priority areas (municipalities and cities) for data collection to maximize coverage of the poor, estimating the number of households to be assessed in these areas, and planning the deployment of enumerators for the household data collection. The models were chosen to balance precision with pragmatism. As the first of its kind in the Philippines, it was evidently important for the models to be accurate in predicting household welfare. At the same time, it is crucial to demonstrate that investing in a PMT-based targeting system is a viable and practical use of Government resources since household surveys/assessments are not exactly cheap. It was therefore a conscious effort to keep the first PMT models simple so that the accompanying implementation is practicable for DSWD to implement. With these in mind, a parsimonious set of 17 socio-economic indicators represented by 39 variables (including sub-categories per indicator) figured in the first PMT models (Table 1). These are the indicators that were scientifically tested to correlate highly with household welfare as represented by income based on existing national household surveys in the Philippines. It is notable that most of them also figured in the Philippine 2000 and 2003 SAPE and in other country PMT models.3 Two separate models were formulated to reflect the dichotomy between urban and rural poverty in the Philippines. Much like in Latin American country examples, poverty in the Philippines is mostly concentrated in rural areas where living conditions are notably different than in urban areas. One particular distinction between rural and urban poverty is the reliance of rural poor households on farm/agricultural entrepreneurial incomes while urban poor households are spread out in low-skill wage employment in the industry and services sectors. Urban poor households also have significantly better access to services (e.g., electricity) than those in rural areas. The distinction between urban and rural living conditions is also reflected in the cost of living (prices) households face in these areas, and therefore also in the poverty thresholds used to determine household poverty. A single universal model was initially formulated with urban/rural indicator as one of the correlates. However, accuracy was poor (producing high misclassification errors, as defined in the next paragraph) and the model was generating a significantly reduced number of poor nationwide. This initial model was recalibrated by introducing separate models for urban and rural areas in order to better capture the structural distinction between living conditions of the poor in rural and urban settings and to, consequently, maximize the coverage of the poor in Listahanan. Table 1. Household- and individual-specific indicators in the first PMT models Household-specific Indicators Housing conditions Access to services 1. Make of roof 6. Main source of water supply 2. Make of walls 7. Type of toilet facility 8. Access to electricity Ownership of assets Other HH characteristics 3. Ownership of house 9. Agricultural household 4. Refrigerator 10. Availability of domestic help 5. Washing Machine 11. Regional location Individual-specific indicators 12. Marital status of the Household (HH) Head 15. Number of family members (family size) 13. Gender of the HH Head 16. Age of family members 14. Level of education of HH Head and wife 17. Education of family members 2 Grosh, M. et al. (2008); Castañeda, et al. (2005); Coady, Grosh and Hoddinott (2004). 3 National Statistical Coordination Board and The World Bank (November 2005), “Estimation of Local Poverty in the Philippines”; (September 2008), “2003 Intercensal Small Area Poverty Estimates”; Castañeda, et al. (2005). 4 WORLD BANK SOCIAL PROTECTION POLICY NOTE Final models were chosen based on those that generated acceptable exclusion error rates. As a statistical model, by construction the PMT models have intrinsic errors,εh, in estimating the true value of per capita income of households (Figure 1 above). The formula is designed to be right “on average” or “at the mean” and so it will not always correctly predict income. In doing so, they also have consequent errors in predicting the true poverty status of households upon applying the poverty thresholds. Provincial poverty thresholds were used to distinguish poor from non-poor households as these were what was shared publicly by the NSCB at the time.4 Exclusion error refers to the share of true poor households that were incorrectly predicted as ‘not poor’ by the PMT models relative to the number of true poor households (Table 2). Inclusion error refers to the share of true non-poor households that were incorrectly predicted as ‘poor’ by the PMT relative to the total number of PMT-predicted ‘poor’ households. The first PMT models were finally chosen based on those that produced errors acceptable to the Government, with bias for minimizing exclusion error over inclusion error. The Government deemed it more acceptable and easier to weed out misclassified ‘not poor’ households by means of additional but relatively easy public verification mechanisms rather than for misclassified ‘poor’ households—who are usually disempowered and voiceless—to publicly contest the results of the PMT during community validation. Table 2. First PMT Exclusion and Inclusion Errors Defined Predicted Poverty Status based on PMT results Not Poor Poor Total Not Poor A B C True Poverty Status based on Poor D E F actual Incomes Total G H I Accuracy Exclusion Error Inclusion Error = (A+E)/I = D/F = B/H The First Listahanan Registry The Philippines’ first household targeting exercise resulted in Listahanan 1—a registry of more than 50 million individuals from 10.9 million households across 80 provinces in the country. Listahanan 1 became one of the Philippines’ largest household databases when it was finally launched publicly in 2011, second to the population census. It covered 58 percent of the 18.5 million households in the Philippines in 2010.5 Out of the 10.9 million households enumerated, 5.2 million were classified by the first PMT models as ‘poor.’ Listahanan results were notably higher than the Government’s estimate of 4.7 million poor households in 2006 and 4.9 million in 2009. This was mainly due to the PMT models’ downward bias in estimating incomes and, therefore, in classifying more households as ‘poor.’6 Note also that Government official poverty figures are survey-based and are, themselves, estimates of the actual number of poor population and are likewise subject to errors. The results were found to be generally consistent with those of the 2003 SAPE. In 890 of the 1,640 municipalities reached by the Listahanan with high coverage—that is, in areas where Listahanan assessed at least 80 percent of all households—the average poverty incidence based on the 2003 SAPE is 52 percent while the average share of poor households over those assessed by Listahanan is 51 percent (Figure 2). In areas where Listahanan covered less than 80 percent of the household population, or where the household population may not be sufficiently represented, it is expected that Listahanan will generate an overestimate of the true poverty rate. This is because the Listahanan is designed to purposively cover areas where poor households are presumed to concentrate, for instance, in ‘pockets of poverty’ as will be discussed later in this Section. For this reason, poverty estimates in these municipalities were not computed based on the Listahanan and grayed-out in the maps below. The resulting database of poor households was a product of a four-stage implementation cycle of the Listahanan (Box 1). Listahanan’s operational cycle repeats every 4 years as mandated in Executive Order 867 of 2010. Since it was piloted in 2007, the cycle has only been repeated twice to date. The first national database was officially launched in October 2011 at the National Statistics Month celebration, reflecting the statistical community’s support to this new system. 4 Official poverty figures however use poverty thresholds separate for urban and rural barangays in every province. 5 2010 Census of Population and Housing in www.psa.gov.ph. 6 Later in 2011, the Government revised its poverty estimation methodology which resulted in a downward estimate of 3.7 million poor households from the original estimate of 4.7 million poor households in 2006, and 3.9 million poor households in 2009 from its original estimate of 4.9 million. www.worldbank.org 5 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward Figure 2. Municipal/City Poverty Estimates based on 2003 SAPE and Listahanan 1 (a) 2003 SAPE, % of population (b) Listahanan 1 Poverty Estimates, % of households Stage 1: In the Preparatory Stage, the Listahanan’s Household Assessment Form was developed. It was fashioned after the official household survey instruments used to formulate the PMT. All information contained in the Listahanan 1 database were collected through a nationwide house-to-house survey aided by the Household Assessment Form (HAF). The HAF is a 2-page printed questionnaire that contains all the indicators required to generate PMT results plus some additional indicators that are useful to DSWD for profiling households. As much as possible, it is important to maintain only the PMT-relevant indicators or questions in the HAF to keep it efficient and cost-effective. Questions in the HAF were crafted based on how they were framed in the national household surveys (FIES and LFS) to ensure consistency of PMT results ex-ante — that is at the time of development using the FIES and LFS — and ex-post — that is after Listahanan household assessments were conducted. The HAF was pretested and the data collection strategy piloted prior to the Listahanan’s nationwide roll-out. It was a conscious effort to ensure that the HAF not only captures accurately the household information that are needed for the PMT but also that the household interviews are done efficiently and in a standard manner. Additional questions were included in the HAF besides those in the PMT as long as the household interview is kept within 30 minutes. During the pre-tests, household interviews lasted to an average of 20 minutes. Upon completion of the Listahanan 1, household interviews averaged 25 minutes to complete, with the length of the interview depending mainly on the number of household members.7 The pretests revealed that correctly identifying household members and the ‘primary occupation’ for each working household member were the most difficult items to fill-out in the HAF. Part of the pilot process is to capture these weaknesses to address them in personnel training and adjust the implementation design accordingly. The pilot phase was also important in testing the data collection strategy and the review process for quality control of completed HAFs. This was done by area supervisors by means of call backs and re-interviews to check for errors and validity of collected information. The pilot stage for the first PMT as well as the associated house-to-house data collection strategy began in March 2007 when targeting was still under the Pantawid Pamilya National Project Management Office (Pantawid Pamilya NPMO). 7 Household size ranged from 1 to 30 members in Listahanan 1. 6 WORLD BANK SOCIAL PROTECTION POLICY NOTE Box 1. The Listahanan’s 4-year Operational Cycle 1. Preparation Stage • Formulation/review of the PMT • Testing and preparation of survey instruments • Systems development/updating • Planning of data collection strategies • Hiring and training of field workers 4. Data Sharing Stage 2. Data Collection Stage • Prepare data for PUF • Field enumeration • Generate reports, profits • Encoding • Dissemination • Initial PMT processing 3. Validation Stage • Posting of preliminary list of poor at the community • Receipt of community feedback • HH assessment for ODA and complaints • Final PMT processing Stage 1: Preparation. It is at this initial stage of the Listahanan process that the PMT models, instruments, and systems are designed and tested. This is also when data collection strategies are planned and coordinated with field offices as well as with the local government units. With all these processes involved, this takes the most amount of time in the targeting process, and for a good reason. Stage 2: Data collection and analysis. The collection of data is done through household assessment. In conducting household assessment, all households in the identified areas were assessed through house-to-house interview using the household assessment form (HAF), a two-page questionnaire with 39 observable and verifiable variables. All information indicated in the household assessment forms are encoded to a web application and subjected to Proxy Means Test (PMT). Classification of households as poor or non-poor. After incomes are estimated using the PMT, the estimated income of a household is compared to the poverty threshold of the province to determine if it is poor or non-poor. Households with estimated income that fall below the poverty threshold of their province are classified as poor. A preliminary list of poor households is then generated, which will be subjected to validation. Stage 3: Community Validation. Validation is the process of community-vetting of the Listahanan’s list of poor households. At this stage, the preliminary results of the PMT models were shared with the community for them to challenge or correct. Specifically, the preliminary list of poor households is posted in every municipality and barangay to get feedback on: a) validity of poor non-poor classification with local perception and b) identify inclusion or exclusions from the list. It therefore allows for feedback, queries, and requests to be received from everyone in the barangay. All feedback are received and recorded in the Complaints Form. A Local Verification Committee (LVC)—which consisted of the Municipal/ City Social Welfare Officer (chair), the Municipal/City Planning and Development Officer, and three other members who represent the CSO and/or NGO—was created at the municipal level to address or act to all complaints and appeals received during the validation period. The validation process is given 30 days in the community and completed nationwide in 30 days between September 2010 and January 2011. Stage 4: Dissemination. This involved informing the LGUs and individuals of the process surrounding the NHTS-PR and providing options to apply and present complaints in the validation process. This included designing flyers and posters, and using a variety of media (e.g., radio, TV) to inform communities about the validation process. The strategy and tools were designed by the NHTS-PR social marketing team at DSWD central office. The tools were distributed to the NHTS regional focal persons, who implemented the communications campaign. Source of basic information: NHTS-PR Operations Manual (July 2012 and May 2015 versions). www.worldbank.org 7 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward In April 2008, the Government decided to roll out the targeting system on a nationwide scale. The decision was prompted by two reasons. First was the Government’s realization to act swiftly in protecting poor households during the global food and fuel crises in 2008 and the ensuing financial crisis then. These all happened when the Philippines’ poverty and human development outcomes were already trailing. Second was brought about by encouraging results of the Listahanan in identifying poor beneficiaries of the newly-established conditional cash transfer (CCT) program, the Pantawid Pamilyang Pilipino Program—the program that first used the Listahanan results. Joint DSWD and World Bank field missions to observe the Listahanan implementation gathered positive feedback from the community, as well as interest from local government units for partnerships. Listahanan has since then rolled out across the country to build the database of potential beneficiaries of social assistance programs. The World Bank provided continued technical support and advice to DSWD throughout the rapid scale-up, as well as linkages with other country practitioners to learn from their experiences in implementing national targeting systems. The nationwide data collection followed a carefully planned approach that was based on a combination of targeting methods. The Listahanan 1 adopted a two-step targeting approach to carefully plan its nationwide roll-out to collect household information and locate the poor. Using a combination approaches in the data collection strategy for targeting has been shown to be a cost-effective approach and yields higher accuracy of results, and thus considered an international best practice.8 Colombia, Mexico, Cost Rica combine household assessment mechanisms with a certain degree of geographic targeting. For Listahanan, the first step involved geographic pre-selection of priority areas to be covered. The second step involved household assessments, which was done with the aid of the HAF. Listahanan’s PMT models were applied to the data collected from each household using the HAF then compared to applicable poverty thresholds to determine if the household enumerated is poor or not. Geographic targeting was done to ensure that areas with the highest concentration of poor households are prioritized for the household assessment. This step involved identifying areas with a high concentration or density of poor households so Listahanan can prioritize their registration. The priority geographic targets included (a) the 20 poorest provinces based on the results of the 2006 Family Income and Expenditure Survey (FIES) of the National Statistics Office (NSO), (b) municipalities with poverty rate ≥ 60% based on the 2003 Small Area Estimates (SAEs) of the National Statistical Coordination Board (NSCB),9 and (c) “pockets of poverty” in highly urbanized cities as defined by the Housing and Land Use Regulatory Board (HLURB) and the Presidential Commission of Urban Poor (PCUP) (Box 2). A total of 389 municipalities and pockets of poverty in 45 cities were identified as priority areas based on these criteria. In these pre-selected priority areas, all households were assessed (census) using the HAF. Outside of these priority areas, households were enumerated on-demand, that is, only as they wish. Stage 2: Based on this two-step process, a three-phased implementation was carried out to roll-out the Listahanan nationwide (Table 3). After the successful pilot and initial roll-out of the targeting system under the stewardship of the Pantawid Pamilya NPMO, in April 2009, a dedicated National Household Targeting Office (NHTO) was created within DSWD to plan and oversee the completion of the Listahanan’s massive operation. A phased roll-out strategy allowed time to make further refinements in the nuts and bolts of the Listahanan. As implementation rolls, operational guidelines were constantly improved and made more thorough, concepts and standards clarified (e.g., in identifying POPs), the accuracy and cost-effectiveness of the HAF further tested, the operational strategy for data collection refined, and the information systems that support the entire operation continuously strengthened. On-demand applications (ODA) for household assessment were undertaken in more than half of cities and municipalities nationwide. ODA provides opportunity for households within the target areas who were not assessed during Stage 2 regular enumeration to “apply” to be assessed under the Listahanan and be included in its database. These assessments were undertaken between September and November 2010. It generated a total of 1,212,528 households that were additionally assessed on-demand in urban barangays in 920 municipalities and cities across the country (Table 4). 8 Coady, Grosh and Hoddinott (2004). 9 The NSO and NSCB were merged together with the Bureau of labor and Employment Statistics (BLES) and Bureau of Agricultural Statistics into the Philippines Statistics Authority (PSA) in 2013. 8 WORLD BANK SOCIAL PROTECTION POLICY NOTE Box 2. Listahanan Process for Highly Urbanized Cities, Component cities and Pockets of Poverty Highly urbanized and component cities were identified following the classification of the Housing and Land Use Regulatory Board (HLURB). Highly urbanized cities (HUCs) are cities are those with a minimum population of two hundred thousand (200,000) inhabitants, as certified by the then National Statistics Office (now PSA), and with the latest annual income of at least fifty million Philippine pesos (P50,000,000.00) based on 1991 constant prices, as certified by the City Treasurer. Independent component cities (ICCs) are cities whose charters prohibit their voters from voting for provincial elective officials. Independent component cities shall be independent of the province. Component cities (CCs) are cities that do not meet the requirements for HUCs nor ICCs. In HUCs, ICCs, or CCs, the Listahanan adopted a combination of enumeration strategies, in recognition of the fact that high income inequality exist in these areas, for time efficiency and cost-effectiveness in covering poor households in the assessment. In areas within a municipality or barangay where clusters of poor households reside, i.e., “pockets of poverty” (POPs), all households were interviewed (also referred to as a saturation or a census). Outside the POPs, households are interviewed only as they wish, or on-demand. There had been two opportunities opened for on-demand application during the first round of Listahanan: first was at Stage 2, which happened at the community right after enumeration in POPs were completed; second was at Stage 3 during the community validation process. DSWD, in coordination with Local Government Units (LGUs), adopted a uniform process in determining POPs using a standard list of 10 indicators. The process starts with the LGUs proposing areas as POPs based on their own assessment. The LGU then provides data on each of the indicators for their proposed POPs. LGUs need to present a complete set of information to DSWD before any area can be considered a POP. DSWD then calculates a score for each of the LGU-proposed POPs by applying a uniform set of weights to each indicator. Scores are presented back to the LGU for consultation before they are provided the final list of POPs in the municipality or barangay. 10 Indicators Used in Identifying Pockets of Poverty Share of households in the area who do not have access to Share of malnourished children in the area (>60%) water (>60%) Share of households in the area who do not have access to Area is inaccessible through the usual means of transport or electricity (>60%) requires a great effort on the part of the traveler to navigate through passageways/thoroughfares leading to the area Share of households in the area who have dwellings made of The area is vulnerable to any of the following hazards: light and/or salvaged materials (>60%) Flood Volcanic eruption Earthquake Tsunami Typhoons Share of households in the area who are considered ‘informal The area does not have any of the following facilities: settlers’ (>30%) Schools (Primary, HS) Health facility *an ‘informal settler’ is one who has built and resided in Day Cay Center/ Recreational a building/house on a lot he does not own or without the Pre-school consent of the lot owner School participation rate in the area (<90%) The area does not have access to garbage/waste collection Source: NHTS-PR Guidelines on Pockets of Poverty. www.worldbank.org 9 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward Table 3. Listahanan 1 Nationwide Data Collection Phases Implementation Target Areas Enumeration Strategy Municipalities/ Cities Covered Phase (a) 20 poorest provinces based on the results of the FIES 2006; (a) 253 1 (b) municipalities with poverty rate ≥60%; and Census (b) 83 (April – August 2009) (c) “pockets of poverty” (POPs) in highly (c) 42 cities urbanized cities. 2 (a) municipalities with poverty incidence between (a) Census (a) 102 (September 2009 – ≥50% and <60%; and (b) Census in POPs; on- (b) 83 cities April 2010) (b) component cities demand outside the POPs 3 Census in POPs; on-demand (September 2009 – Municipalities/cities with poverty incidence <50% 918 municipalities outside the POPs August 2010) 1,508 muns and 133 cities in 80 TOTAL provinces across 17 regions* *Including areas enumerated during the Preparatory Stage and initial roll-out between 2007 and 2009 (160 municipalities and cities in 38 provinces across the 17 regions). Table 4. Results of Listahanan 1 Stage 2 On-demand application Applications Households Coverage Region received assessed Municipalities Cities Total I 84,722 48,253 104 9 113 II 61,269 43,696 79 3 82 III 187,315 127,072 112 13 125 IV-A 254,220 202,433 117 12 129 IV-B 28,309 18,278 20 1 21 V 123,575 89,003 62 5 67 VI 96,915 64,509 96 14 110 VII 86,856 56,643 91 10 101 VIII 20,141 20,502 0 5 5 IX 47,603 19,279 13 3 16 X 39,375 22,236 32 5 37 XI 12,551 10,526 29 6 35 XII 52,749 30,811 28 5 33 CAR 21,727 17,713 23 1 24 CARAGA 6,674 5,897 3 2 5 NCR 144,444 436,124* 0 17 17 TOTAL 1,267,445 1,212,528 809 111 920 *NCR accomplishment is based on results of ODA and additional households to complete census in identified POPs. 10 WORLD BANK SOCIAL PROTECTION POLICY NOTE Several quality control measures were put in place to ensure data is valid and accurately collected. While no system is immune to errors and fraud, strong monitoring and oversight mechanisms were embedded into the Listahanan operational design to minimize them. Field supervisors (also called Area Supervisors) conduct random visits during the enumeration as well as re-interviews of enumerated households on a daily basis to make sure Enumerators actually visit assigned enumeration areas and follow standard procedures in data collection. Staff from the NHTO and its field units at the regional offices of DSWD, the National Household Targeting Units (NHTUs), also conduct unannounced field visits or “spot checks” to monitor compliance of field teams to standard procedures in data collection. Automated checks for consistency and duplicates were also added once completed HAFs are entered into the database to make sure responses were valid and households were not repeatedly assessed. Enumerators and Supervisors were paid on the basis of turning out clean reviewed HAFs. These measures ensure that Listahanan contains valid and reliable data of households. Stage 3: Initial results of the PMT based on information collected in Stage 2 undergo community validation. Community validation is a feedback mechanism that was put in place to, first, compare the PMT results with the community perception of who is poor and, second, to capture remaining mistakes in information collected from households that may have led to misclassifying them as poor or not poor. As mentioned above (para. 14), exclusion and inclusion errors are intrinsic in PMT-based targeting systems because by their very nature, statistical models have estimation errors. These are ex-ante errors and are measured pre-implementation. However, actual implementation of the PMT-based targeting system itself can present errors due to human mistakes at any stage in the operational cycle. Implementation may work both ways to the ex-post or final errors of the targeting system. Implementation may worsen final errors if carefully planned operational procedures are not adhered to. But what implementation should strive is to reduce these errors. Community validation was built into the Listahanan to address this. Community validations were conducted in all barangays across the country regardless of the enumeration strategy that was applied during data collection (Stage 2). Validation followed a standard set of rules in receiving and addressing complaints, appeals, or feedback from the community. The purpose of this feedback system is to establish a transparent mechanism to validate the initial results of the poverty classification as determined by the PMT and assess possible sources of errors of misclassification, perceived or otherwise. During validation, the preliminary list of poor households was posted for 30 days in conspicuous places in the barangay, commonly at barangay halls, street corners, or public markets. This opens up the preliminary results of the PMT to be challenged or vetted by members of the community or barangay. Local DSWD offices or LGUs provided space and logistics support in receiving complaints, appeals, or other forms of feedback. Every feedback was received and recorded by NHTO personnel of DSWD in individual Complaints Form for a period of 10 days, then these are consolidated in the Consolidated Complaints Form. Addressing all forms of feedback follow a standard procedure and resolved at the appropriate level with a quick turn-around time. Simple requests for information are immediately addressed by NHTO Area Supervisors and Coordinators on-site during the validation. Meanwhile, a Local Verification Committee (LVC) decided on all other complaints and appeals in the Consolidated Complaints Form that may require assessing or re-assessing a specific household. The LVC is a 5-member committee that was usually chaired by the Municipal Social Welfare Officer and include the Municipal/City Development Planning Officer and 3 others from either the CSO and/or NGO as members of the committee. They were given 5 days after the receipt of the last complaint/appeal to decide on all cases. Box 3. Types of feedback received during community validation Following are the types of feedback received during community validation: • A household should not be included in the list of poor households. • A household is poor and should be included in the list of poor households. • A household was not visited/interviewed during the enumeration. • The enumerator entered wrong information in the household assessment form. • The preliminary list of poor households is incomplete and does not represent the actual number of poor households in the community. www.worldbank.org 11 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward In receiving appeals and complaints, the community validation opened the second round of on-demand applications. Unlike the first round of ODA which was launched only in urban barangays where parts were purposively not enumerated, a total of 1,441,874 complaints and appeals across the country were received during the community validations, of which a majority (80 percent) required visiting the household to assess or re-assess it (Table 5). This gave another opportunity for other members of the community who were missed in Stage 2 to be assessed and other households to be re-assessed to correct for any mistakes in information recorded about them during Stage 2 household assessments. DSWD field teams were given 5 – 14 days to complete these additional household assessments. At the national level, the entire validation process—including undertaking additional household assessments—was completed in 5 months, from September 2010 to Jan 2011. Table 5. Results of Listahanan 1 Community Validation Coverage Accomplishment Complaints Region Households Corrected received Municipalities Cities Total assessed entries I 47,273 12 9 21 27,804 12,543 II 44,202 37 3 40 38,127 1,253 III 47,804 113 13 126 30,348 14,223 IV-A 139,904 126 12 138 123,017 15,386 IV-B 62,187 60 2 62 44,346 14,093 V 105,352 99 7 106 75,246 8,186 VI 119,224 112 16 128 68,098 40,920 VII 65,075 109 16 125 48,375 6,927 VIII 123,745 125 7 132 85,679 16,353 IX 153,606 54 5 59 136,547 11,025 X 79,419 65 9 64 52,419 23,619 XI 61,345 41 6 47 44,172 20,949 XII 51,368 43 5 48 40,389 3,465 ARMM 140,503 105 2 107 119,520 12,276 CAR 37,939 51 2 53 30,680 1,679 CARAGA 32,649 43 6 49 62,441 11,032 NCR 50,278 0 17 17 62,246 846 TOTAL 1,441,874 1,346 137 1,483 1,089,454 214,793 Stage 4: Validated results of the PMT models identified 5.2 million Filipino households as poor out of 10.9 million households assessed throughout the country. Results were disseminated publicly in October 2011 and in various other fora at the national and local levels. National and regional publications called “Profile of the Poor” were prepared using the database and shared with potential partners and data users of the Listahanan. As of December 2017, Listahanan 1 has been used by 59 national programs— including two of the largest ones, the CCT program and subsidized health insurance (Philhealth)—and has been shared with 1,095 local government units, 56 non-government organizations, 34 legislators, and 15 academic institutions to design their own programs and research. Data use is guided by the Listahanan’s Data Sharing Protocol to ensure that information is not misused, i.e., for politics. 12 WORLD BANK SOCIAL PROTECTION POLICY NOTE After the launch of validated results, ‘special validations’ continued to accommodate additional requests for assessment. In 2013, these special validations were undertaken in disaster areas to help assess those who were affected by massive flooding caused by monsoon rains (Habagat) and destructive typhoon Pablo in August and December 2012, which were reported to have affected several provinces in southern Philippines. A total of 951,132 households were assessed by Listahanan for this purpose, of which 464,894 (49 percent) were classified by the PMT models as ‘poor’ (Table 6). In addition, some 464,000 households were assessed as potential beneficiaries of DSWD’s and LGU’s other assistance programs, of which 156,363 were found to be poor. LGUs requested for special validation due to reported high under-coverage during the enumeration stage and exclusion rates among the poor as in Zamboanga del Norte, Capiz and Catanduanes. Technically, these special validations meant that the database did not close after it was launched in 2011 as designed. Special validations have a specific purpose and, for some, undertaken at a specific time when household conditions are not at their normal levels. Hence, results of special validations may be extremely biased. Households assessed through special validations by the NHTO were saved in a separate database and do not form part of the official Listahanan database. Table 6. Results of ‘Special validations’ Reason Assessed Households Poor Households 2013 LGU requests 294,651 84,688 Habagat 271,132 138,179 Typhoon Pablo 216,535 170,350 Sub-total 782,318 393,217 2014 DSWD Social Pension 116,368 38,451 DSWD MCCT 52,446 33,224 168,814 71,675 TOTAL 951,132 464,894 III. ADJUSTMENTS IN LISTAHANAN 2 A review of the Listahanan 1 provided useful lessons to improve in succeeding rounds of targeting. Executive Order 867 of 2010 allows DSWD to update the Listahanan every four years. For its next round, the experience from Listahanan 1 pointed to key enhancements in the following key areas. Experience from the first round of a massive and nationwide household targeting highlighted several operational aspects that required special attention for the succeeding rounds of targeting. These include improving the design and performance of the PMT models, reducing the period for data collection, using the latest available technology (as tablets and mobile applications), improving the management information system (MIS), strengthening training of enumerators and other field staff, and adopting effective strategic communication practices, which all contribute to increasing the accuracy of Listahanan in identifying the poor. Accuracy of PMT models. The first PMT models were reviewed for robustness as soon as the detailed 2006 and 2009 FIES-LFS datasets became available in 2008 and 2011, respectively. Both reviews found that (i) the same set of indicators remain highly correlated and significant in predicting incomes, but that (ii) the 2003 weights are increasingly underestimating incomes over time, especially of households in the upper 60 percent. In correcting (ii) by merely updating the weights of the same set of indicators using more recent data, the analyses found that (iii) this reduces the consequent inclusion errors but increases exclusion errors, as expected. Additional measures from the operational side are thus needed post-estimation if both inclusion and exclusion errors are to be reduced at the same time. While these measures were being explored, DSWD decided to continue using the first PMT models based on 2003 data to maintain current levels of exclusion errors and for consistency in policy throughout the Listahanan 1 implementation, and re-open the discussion when the cycle for next Listahanan round starts. www.worldbank.org 13 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward Prolonged period of data collection. Data collection for Listahanan 1 was effectively completed over 3 years—from the pilot stage in March 2007 until phase 3 of the nationwide enumeration was completed in August 2010—and for a good reason since all aspects of the operation were built from scratch and needed careful testing. This introduced some inconsistencies in the PMT formulation, as poverty thresholds used to predict household welfare status changed over the course of implementation. More specifically, national urban and rural poverty thresholds for 2007 were used for those covered during the pilot stage while more disaggregated provincial poverty thresholds for 2006, which were released in 2008, were used for those covered in the massive enumeration. The inconsistency was, however, a result of availability of information and one that is outside of DSWD’s control. Cost-efficiency. Twice conducting on-demand applications in Listahanan 1 proved to be administratively and financially costly for DSWD. Many places were visited twice by the field teams in two separate occasions, first during the Stage 2 ODA process and second during the Stage 3 community validation process. This resulted in re-contracting a great number of Area Supervisors and Enumerators to go back to the same barangays where additional households requested to be assessed. Stage 2 ODA cost DSWD Php39 million or Php31 per household assessed. The diffused and sporadic nature of response to accommodate additional ODA—1.2 million on-demand assessments in 920 cities and municipalities throughout the country—in a limited time also strained heavily the administrative capacity of staff. Stronger collaboration with local experts and executives. While guarding its independence from local politics was at the heart of putting up a centrally-managed targeting system as the Listahanan, experience from the first round showed that support from the LGUs remains important to the success of the Listahanan. Weak coordination with LGUs caused delays in the implementation, for instance, in data collection without a clear knowledge of political boundaries and effectively of households that belong to a specific locality. Experience from Listahanan 1 was that LGUs were, in general, ready to extend assistance when called upon, especially in terms of local knowledge about their constituency (residents in the outskirts), logistics (transportation), and security to field teams. Meanwhile, interest of local academics and experts on the PMT formulation were aroused because of proven early successes of programs that used the Listahanan’s first PMT models. For the first time, shared interest in household targeting is now more openly discussed between local policy-makers and academics. IV. PREPARATION FOR LISTAHANAN 2 Development of the second PMT models started in 2012, shortly after the Listahanan 1 results were publicly launched in October 2011. Data sharing completes the 4-stage cycle of the Listahanan and triggers the start of the next cycle. In June 2012, the DSWD, with technical assistance from the World Bank, initiated the review of the first PMT models. The main motivation for the review is the fact that the first PMT models used the 2003 FIES and LFS data and produced parameters that are already 10 years old by the time the second targeting process is due in 2013/2014. Since the determinants of household income (or consumption) can change over time, reviewing and updating the PMT model when a new household survey is available is warranted. The review assessed whether the indicators, parameters, and overall model specification is still applicable or they already require adjustments for the second round of targeting.10 Since 2007, several national household datasets became available for the second cycle of Listahanan. In 2012, the most recent datasets that came out were the mid-decade 2007 Census of Population (Popcen) and the linked 2009 FIES-LFS. The 2007 Popcen collects not only household- and individual-level information but also barangay-level indicators. Data from the 2010 Population of Census and Housing (2010 CPH) was still not available at the time the new PMT models were being formulated. The Second PMT Models The second PMT models were developed with local academics at the helm. With the increasing interest from local academics and experts to engage in Listahanan and build stronger local ownership, leadership in reviewing and updating of the PMT models for the second round of household targeting was transitioned to local academics. The World Bank provided guidance and technical inputs throughout the process based on the experience from Listahanan 1 as well as from other countries implementing similar PMT-based targeting systems.11 10 The Listahanan update was implemented in 2015 because of delays in securing its budget. 11 The new PMT models for Listahanan 2 underwent peer review from World Bank staff and international academics on 12 August 2014. Comments were received from Damien Echevin, Ph.D., (Professor at Sherbrooke University, Canada), Matt Wai-Poi, Ph.D., (Senior Economist, Poverty Global Practice, World Bank Jakarta), Michael Lokshin, Ph.D., (Manager, Development Economics and Chief Economist Office, World Bank DC), and Rashiel Velarde (Economist, Social Protection and Labor Global Practice, World Bank Manila). 14 WORLD BANK SOCIAL PROTECTION POLICY NOTE The formulation of the new PMT models took a more sophisticated approach with a view to drastically reducing both exclusion and inclusion errors. To achieve this, the new models introduced five (5) major changes in its design (Table 7). Taken together, these changes generated significant reductions in the final ex-ante exclusion and inclusion errors to 8 and 13 percent, respectively, from ex-ante exclusion and inclusion errors of 18 and 45 percent, respectively, in the 2003 models. Each of these major changes are further explained next. Increased number of correlates to increase predictive power. Studies, particularly Mapa, Balisacan and Briones (2008), Balisacan (2005) and Balisacan and Pernia (2003), suggest that community (or Barangay) characteristics such as presence of infrastructure (road network, electricity, telephone, and water system) and the presence and number of business establishments in the Barangay are factors that explain the households’ per capita income in that Barangay. Incorporating these variables to the PMT models will likely increase the models goodness-of-fit and the increase the likelihood of having lower errors within sample (Figure 3). These barangay characteristic variables are available in the Census of Population and Housing (CPH). For this particular undertaking, the data from the 2007 CPH was incorporated in the models. Apart from community-level indicators, additional household- and individual-specific indicators taken from the 2009 FIES and LFS were also added into the models. All these meant a significant increase in the number of correlates used to increase the predictive power of the new PMT models. Again, the additional indicators that figured in the models were chosen so that they are observable or easily verifiable (Table 8). Table 7. Summary of Enhancements to the PMT Models Listahanan 1 Listahanan 2 Household-level variables from the Labor Force Household-level variables from the LFS and FIES Survey (LFS) and Family Income and Expenditure + community-level variables from the Census of Survey (FIES); Population; Explanatory variables Aggregate occupations used in the model based on More detailed occupations used based on 4-digit 2-digit occupational codes occupational codes Reference data 2003 FIES-LFS 2009 FIES-LFS; 2007 CPH 1 Model for Urban areas 1 Model for NCR Sub-models 1 Model for Rural areas 1 Model for the Rest of the Philippines (ROP) 2 layers for both NCR and ROP models: Layer 1 – predicts per capita income of households 1 layer for both Urban and Rural models to predict and minimizes exclusion error; Layers per capita income of households and balance the exclusion and inclusion errors Layer 2 – predicts misclassification of real non- poor households as poor to minimizes inclusion error Reference population All poor households in the official household Bottom 40 population in the official household to estimate the PMT surveys (LFS and FIES) surveys (LFS and FIES) Basis for identifying Point estimate of the predicted per capita income Lower bound of the 95% predicted interval of per poor households versus the official poverty threshold capita income versus the official poverty threshold Figure 3. The second PMT model specification www.worldbank.org 15 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward Changing the PMT correlates can mitigate “gaming” of the Listahanan. Because of the simplicity of the first PMT models, it would be easy for people to learn household characteristics that can easily be associated with poverty. For example, the number of household members (household size) is one indicator that figured heavily in the first PMT models. Program implementers and members of the community observed that this was so because mainly those with big families in their communities received targeted Government assistance like the Pantawid Pamilya conditional cash transfer (CCT) program, which used the Listahanan to select poor households. Others were able to recall appliances that were asked during the house-to-house interviews. This, however, is not uncommon. Experience of other countries that have undertaken updates of their registries showed that, after completing the massive nationwide operation and the resulting list published, people started guessing about indicators used in categorizing households. This created incentives for some to modify previously reported information to change the result. Table 8. Community-, Household- and individual-specific indicators in the second PMT models Barangay-level Indicators* 1. Presence of town city hall/provincial capitol in the 9. Number of auto repair shop, vulcanizing shop, electronic Barangay repair shop, or other repair shops 2. Presence of high school 10. Poblacion/City District indicator 3. Presence of street patterns 11. Presence of cemetery 4. Number of recreational establishments 12. Availability landline telephone system or calling station 5. Number of commercial establishments 13. Availability of cellular phone signal 6. Number of hotel dormitory, motel or other lodging 14. Number of banking institutions/pawnshops financing places in the barangay and investment 7. Number of establishments offering personal services like 15. Number of recreational establishments OUTSIDE the restaurants, cafeteria, etc barangay but within 2kms 8. Share of population 10 yrs old and above who are 16. Number of households dwelling in private land which farmers, farm laborers, fishermen, loggers, and forest they do not own except in danger areas product gatherers (>50%) Household-specific Indicators Ownership of assets Housing conditions 16. Ownership of house and lot* 19. Make of roof 17. Number of the following appliances owned*: 20.Make of walls • Refrigerator/s 21. Building type* • Washing Machine/s • Telephone/s or cellphone/s • TV set/s Access to services • Radio/s 22. Main source of water supply • VTR/VHS/VCD/DVD 23. Type of toilet facility • Stereo or CD player/s • Microwave oven/s 24. Access to electricity • Sala set/s • Dining set/s Other HH Characteristics 25. Household type* 26. Number of HHs in the housing unit* 27. Agricultural household 16. Number of the following vehicles owned*: 28. Availability of domestic help • Car/jeep 29. Regional location • Motorcycle/tricycle 30. Urban location* Individual-specific indicators 31. Marital status of the HH Head 34. Age of family members 32. Gender of the HH Head 35. Education of family members 33. Number of family members (family size) 36. Occupation of working family members *Indicators not in the first PMT models. 16 WORLD BANK SOCIAL PROTECTION POLICY NOTE The NCR and ROP models. The international best practice in designing PMTs is to use a single model or standard consistently applied to everyone, but flexible enough for adjustments to be made especially with reference to price differences in urban and rural areas and on applicable location-adjusted price or cost of living indices. In the Philippine case, this was indirectly followed by using a single model that is applied to all municipalities and cities, with the exception of the National Capital Region (NCR). Based on the experience from Listahanan 1 (which applied separate models for urban and rural barangays) and available poverty diagnostics for the Philippines, the profile of all-urban NCR vary considerably from those of other regions or in other urban areas in the country. Being the most economically active region, it enjoys the lowest poverty rate among the 17 regions in the country with an estimate incidence of 2.6 percent in 2015 (equivalent to about 76,500 poor households). Access to services, which is a key determinant of poverty in many places, is relatively easier in NCR and so does asset ownership. Several models were explored, including separate urban/rural models as in the first PMT models and island group models (separate for Luzon, Visayas, and Mindanao), but the NCR/ ROP models produced the highest accuracy. The 2-stage stepwise approach to minimizing errors. The first stage PMT models for NCR and ROP predict per capita income of households using (i) restricted model estimation using households in the bottom 40 percent of the income distribution as the reference population (instead of all households), and (ii) the lower bound of the 95% interval estimate of per capita income (instead of a point estimate). These two changes in the formulation of the first stage PMT models tend to bias the estimate of per capita downward and is tantamount to casting a larger net to minimize misclassifying the poor. The objective of the first stage models is therefore to minimize exclusion error. However, in doing so it has the counter-effect of predicting true non-poor as poor, thus generating high inclusion errors. Inclusion error increased to 51 percent for ROP and to 75 percent for NCR from 24 and 37 percent for ROP and NCR, respectively, applying the first PMT parameters to the latest household data sources. The second stage models were introduced to predict the likelihood that a household was erroneously classified by the first stage model as poor (and is in fact not poor). The objective of the second layer of models is therefore to minimize inclusion errors after applying the first stage PMT models. Unlike the first stage models that are linear models and use ordinary least squares estimation methodology, the second stage models apply a non-linear (logarithmic) estimation methodology. While a two-stage model introduces some complexity in the processes of identifying poor households, this is balanced by using indicators that are observable and easy to collect (implement) for both models and a flexible software platform. As with the first PMT models, the final consideration in selecting the new PMT models was based on the assessment of consequent ex-ante misclassification errors. Different model specification such as urban/rural, provincial, and regional disaggregation, have been tried. These models do not yield significant increase in model accuracy and reductions in inclusion and exclusion errors. The trade- off between parsimony and goodness-of-fit was thoughtfully considered. However, the National Technical Advisory Group (NTAG)— with representatives coming from government statistical agencies and the academe—placed more weight on model accuracy over parsimony in choosing the final 2009 PMT models. The flexibility of Listahanan’s PMT models will now be put to use in identifying vulnerable households. After one round of Listahanan, the Government recognized the utility and credibility of the system in enabling Government reach those who need help the most. The Listahanan was designed to be flexible enough not only to distinguish the poor from those who are not poor, but also those who are at the brink of falling into poverty. This flexibility is embedded in the Listahanan’s PMT models that predict incomes, which can be compared with pre-determined cut-offs that are used to identify who is poor, food poor, not poor or near-poor. In 2014, in its continuous search in making Listahanan more relevant, DSWD initiated a study to define the cut-off point that it can use to uniformly identify those who are not poor but highly vulnerable of falling into poverty, ie, the ‘near-poor.’ The objective is to incorporate this measure in the results of the new PMT models for Listahanan 2 so that appropriate interventions can be extended to vulnerable households when needed. Three cut-off points are currently being explored and may be used depending on the resources available to the Government—10 percent, 28 percent, and 37 percent above the official poverty thresholds. The cut-off points were derived based on the analysis from the panel of households in the 2004, 2008, and 2010 Annual Poverty Indicators Survey of the PSA. The technical note for the new PMT models has undergone technical reviews. While DSWD already received the endorsement of its own technical advisors (NTAG) on the new PMT models in June 2013, it remained open to review and scrutiny of other external partners. The new PMT models were presented to local academics for their review and inputs at the National Statistics Forum in October 2013. It also underwent additional expert reviews from targeting specialists and practitioners from the World Bank in August 2014. The final version of the Technical Note incorporating inputs and recommendations from partners was produced in October 2014. Manuals and instruments for field implementers were updated to reflect the operations that will support the 2009 PMT models. www.worldbank.org 17 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward V. DATA COLLECTION FOR LISTAHANAN 2 Listahanan 2 resulted in a database of 15.1 million households assessed in 80 provinces across the country. This covers three- quarters of the 20.3 million Filipino households in 2015.12 Out of the 15.1 million households enumerated, 5.1 million were classified by the 2009 PMT models as ‘poor’ (Table 9). Table 9. Overall accomplishment of the Listahanan 2 Households Identified Poor Share Assessed Households to poor Cordillera Administrative Region (CAR) 293,547 64,327 1.3% National Capital Region (NCR) 1,208,779 96,495 1.9% REGION I Ilocos Region 881,443 165,235 3.2% REGION II Cagayan Valley 633,661 135,824 2.7% REGION III Central Luzon 1,379,247 244,593 4.8% REGION IV-A CALABARZON 1,346,258 202,279 4.0% REGION IV-B MIMAROPA 569,316 207,863 4.1% REGION V Bicol Region 1,074,005 369,395 7.2% REGION VI Western Visayas 1,303,860 452,436 8.8% REGION VI Central Visayas 1,087,290 525,561 10.3% REGION VIII Eastern Visayas 718,492 330,945 6.5% REGION IX Zamboanga Peninsula 704,872 364,723 7.1% REGION X Northern Mindanao 870,198 485,539 9.5% REGION XI Davao Region 836,567 280,325 5.5% REGION XII SOCCSKSARGEN 800,653 358,132 7.0% REGION XIII CARAGA 494,498 259,858 5.1% ARMM 925,957 573,446 11.2% Urban 5,072,995 1,195,073 23.4% Rural 10,055,648 3,921,903 76.6% PHILIPPINES 15,128,643 5,116,976 100.0% As with Listahanan 1, instruments for Listahanan 2 were crafted based on official household surveys and census used in formulating the new PMT models. Two instruments were developed to capture community and household indicators in the new PMT models: the Barangay/Community Characteristics Form (BCCF), which bears barangay-level questions that were patterned after the 2007 Popcen, and the Family Assessment Form (FAF), which bears household- and individual-specific questions that were patterned after those in the 2009 FIES-LFS (Annexes 2 and 3). Questions in the Listahanan 2 instruments were framed exactly as they were in the 2009 FIES-LFS and 2007 Popcen. The new instruments are 5 pages long in total—a 1-page BCCF and a 4-page FAF. 12 Based on 2010 census-based population projections of the PSA in https://psa.gov.ph/sites/default/files/attachments/hsd/pressrelease/Table1_8.pdf and average Filipino household size of 5 members in the 2012 FIES. 18 WORLD BANK SOCIAL PROTECTION POLICY NOTE A census of households in rural barangays. The decision was borne out of two main reasons. First, after the dissemination of Listahanan 1, LGUs became more aware and acknowledged the reliability and utility of this database of poor households. In the years following its release, a number of LGUs requested additional assessments in their locality (Table 6 on Special Validations) to identify poor households among their constituents and ensure they are listed in the Government’s database of poor. It is anticipated that this will happen again in the second round of Listahanan, as more and more LGUs appreciate its relevance. Second, a lesson learned from the Listahanan 1 implementation was the high administrative and financial cost related to undertaking on-demand household assessments twice (at Stage 2 and 3). To minimize the administrative and financial costs, a full census was carried out in all rural barangays. In these areas, ODA no longer applied. ODA was implemented only in urban barangays where the same enumeration strategy as in Listahanan 1 was retained, that is, a combination of census in identified pockets-of-poverty and on-demand application elsewhere. Period for household data collection is shortened. In an effort to address concerns on data consistency and the cost of prolonged data collection, DSWD planned on completing household data collection for the second round of Listahanan within a year despite the decision to do a census in all rural barangays. Several changes were made in the operational design, including those below, to ensure that this goal is met. Meanwhile, data collection for Barangay Characteristics started well in advance in September 2014 and completed in February 2015 for all 42,026 barangays nationwide. Staff complement at the central and field offices of the NHTO was reinforced. At the NHTO central office, at least additional 13 regular staff were hired to support project coordination with the field offices, MIS enhancements, data processing, and overall logistics of the assessment. At the NHTUs in DSWD regional offices, a total of about 175 staff were hired to support day-to-day ground-level coordination and supervision. NHTO and NHTU staff were complemented by an army of about 39,000 field workers made up of enumerators, area supervisors, encoders, and verifiers who were deployed both for the barangay and household data collection. Use of video materials for staff training. While this may sound trivial, consistency and uniformity are central to the reliability of a targeting system like Listahanan. More so when data is collected by 39,000 personnel. In Listahanan 1, personnel trainings were done in a cascaded manner and undertaken only in face-to-face formats. About 10 NHTO central office staff acted as the resource persons to train NHTU staff at the regional offices (which constrained the schedule for nationwide trainings), who then trained area coordinators and supervisors who then trained the enumerators. Loss of key concepts happened at every stage, which resulted in so much time spent in supervision and correcting mistakes at the point of enumeration or data coding. Video training materials for Listahanan 2 were produced and distributed to field offices to reduce the heavy demands from NHTO staff in doing the trainings and be able to conduct simultaneous trainings. More importantly, it came as a form of quality control and easy reference material for field staff—making sure that standard concepts are adhered to. Paper-based enumeration was complemented by computer-aided data collection. FAFs were prepared in two formats—one in the usual paper-based instrument and another in an android application format. Some 13,000 android tablets and 4,500 laptops were purchased to aid in data collection and its supervision. The tablets were equipped with a specialized electronic FAF (e-FAF) application for Enumerators that mirrors the paper-form of the FAF. The tablets were deployed mainly in urban barangays where internet connection is relatively more reliable for data submission. Laptops were also equipped with specialized applications such as the FAF Data Management System for Area Supervisors and Area Coordinators. This application allows supervisors to monitor Enumerators’ daily completion of household assessments and keep track of the enumeration schedule. Laptops also allowed for faster reviews and spot check of FAFs by the Area Supervisors. Validation routines were embedded in the tablet and laptop applications to automate some data consistency checks for immediate detection of mistakes. On average, it took 30 minutes to accomplish an e-FAF for an average household with 5 members. This has considerably reduced the time for data collection (33 minutes) and encoding (19 minutes) using the paper-FAFs by 43 percent.13 Meanwhile, paper-FAFs were mainly deployed in rural barangays where electricity for charging and internet connectivity are problematic. Submission and encoding for these paper-based instruments followed the same procedure as in Listahanan 1. Paper-based enumeration started in May 2015 while tablet-aided enumeration began in July 2015. Enumeration of 15.1 million households nationwide was completed in October 2015. 13 Figures based on the pretests of the paper-FAF and e-FAF undertaken in July 2014 in an island municipality. www.worldbank.org 19 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward On-demand application streamlined with community validation. Unlike in Listahanan 1, Listahanan 2 accommodated on-demand applications for assessment only when it reached Stage 3 or Community Validation. This was done to minimize additional administrative and financial costs to field staff, especially with the shortened implementation period for the whole 4-stage cycle of Listahanan 2. Community Validation for this round of Listahanan followed the same procedures as the first round, which was carried out in all barangays and given 30 days to complete.14 An innovation introduced in Listahanan 2 is the use of laptops during the community validation process. This allowed field staff to immediately respond to basic information queries and corrections received in community assemblies. It also enabled them to check up-front if a requesting household was already enumerated but categorized as ‘not poor’ by the PMT, thereby reducing time spent in unnecessary reassessments and reviews/deliberations by the LVC. Altogether, these efforts reduced household data collection from 17 months down to 6 months. While on the whole Listahanan 2 may have taken longer than expected, this was mainly due to delays in securing the budget for the nationwide enumeration, hiring and training of field staff, and procuring equipment at the Preparatory Stage of the cycle (Table 10). New information was also collected, such as those at the level of the barangay and additional 2 pages of information from the households, which added to the overall time spent for Stage 2 data collection. Table 10. Timeline to complete the cycle for Listahanan 1 & 2 Number of months completed Stage Listahanan 1 Listahanan 2 1 – Preparation 25 months (March 2007 – March 2009) 35 months (June 2012 – April 2015)* BCCF: 5 months (October 2014 – February 2015) 2 – Data Collection 17 months (April 2009 – August 2010) FAF: 6 months (May 2015 – October 2015) 6 months: (September 2015 – February 2016; 3 – Validation 5 months (September 2010 – January 2011**) Dec. 2017***) 4 – Final Database June 2011; October 2011 February 2016; March 2016 & Dissemination 52 months 45 months + 12 months for 4Ps Full cycle (March 2007 – June 2011) (June 2012 – February 2016) *Preparation stage took longer than planned for Listahanan 2 due to budget constraints to finance hiring/training of field personnel and the nationwide household data collection. **Does not include “special validations” in 2013 and 2014 for the typhoons. ***Includes extended ODA for 4Ps but not “special validation” in 2018 for TRAIN Law. VI. LESSONS AND WAYS FORWARD FOR SUCCEEDING LISTAHANAN ROUNDS The Philippines’ experience in introducing a household targeting system for Government-wide use has generally been a story of success. The Philippines has decades of experience in implementing and experimentation in designing programs that would more directly and impactfully change the country’s history of growth-poverty conundrum. Multiple programs have been introduced over many decades across various agencies and programs but none has been as effective as when the Listahanan was introduced. Harnessing the experience of other countries in developing better targeted programs and customizing those to suit the Philippine context has evidently benefited the design of the Listahanan and its performance. And this is evidenced by the targeting success of the CCT program, Pantawid Pamilya Pilipino Program, the first program that used the Listahanan, in reaching the poorest households. More and more national agencies, local governments, and academic institutions using Listahanan for program design and research is also a testament to its credibility and reliability. 14 Community Validation in Listahanan 1 and 2 were both guided by DSWD Memorandum Circular No. 3 of 2010, “Guidelines for Validating and Finalizing the List of Poor Households.” 20 WORLD BANK SOCIAL PROTECTION POLICY NOTE Beyond its main intent as a targeting system, Listahanan has also set a global example on the use of strategic communications to increase awareness and build stronger support for social protection systems. Strategic communications served an important role in addressing misconceptions regarding the technical nature of a targeting system such as the Listahanan. Continuous assistance from the World Bank in this regard augmented available Government resources to support the Communications Unit of the NHTO, which is manned by one staff at the central office who develops and oversees the execution of the multi-year communication plan. The Bank provided support by hiring two (2) communications specialists who assisted in continuously updating the communication plan to support the requirements of the Operations Unit, and in developing effective information materials (i.e., Information Tool Kit, Profile of the Poor publication, etc.) for specific target audiences and used mainly for the community validation, launch, and continued dissemination of the Listahanan 2 results. Additionally, audio-video materials aided the massive nationwide training of 39,000 field staff prior to their deployment for household data collection in May 2015. Beyond the Philippines, the Listahanan is seen not only as a good country example in establishing a functioning targeting system but also recognized to be at the forefront in using strategic communications in implementing social protection systems. The experience in developing Listahanan’s strategic communication plan and its outputs has also been presented (twice) as the first case study in the WB’s recently-established community of practice on Communications, Social Marketing and Outreach (COSMO COP). Furthermore, thanks in part to strong and strategic communications, Listahanan has been widely utilized by both government and non-government agencies for programs that benefit the poor. While Listahanan 2 database is yet to be widely utilized , the Listahanan 1 database has been used by 59 national agencies, including the conditional cash transfer program of DSWD, social pension for poor elderly, and the PhilHealth subsidized health insurance. It has also been shared with 1,095 LGUs, 56 CSOs, 34 legislators, and 15 universities and research institutions. Moving forward, it is important to keep a simple PMT model that can be easily operationalized. While PMT models are technically statistical models—and for that they need to meet a certain standard of technical acceptability—they are just one component of the whole machinery of a targeting system. An out-of-sample robustness check of the Listahanan 2 models showed that the level of sophistication introduced in the new models have not been able to achieve lower ex-post errors compared to the old PMT models.15 The results of the validation exercise showed that the trade-off between exclusion and inclusion errors cannot be dealt with only by recalibrating the PMT models. Instead, the whole targeting operation needs to be mobilized to address these errors during implementation and once PMT models have been finalized based on ex-ante errors. This can be done through methodical validation exercises that intend to filter and minimize inclusion errors. The overall intent for a PMT model is to be used to identify households or individuals who will benefit from specific programs. PMT models that support targeting systems, thus, need to be practical and able to easily integrate into the targeting operations. Listahanan’s PMT model needs to maintain simplicity and parsimony in order to maintain, in turn, its functionality as a targeting system that is practical and pragmatic to use in designing programs. Vertical and horizontal coordination during implementation cannot be overemphasized. The first round of Listahanan over 2007 to 2011 has mainly been an experimental (pilot) period for a system that is the first of its kind in Philippines. In that sense, DSWD implementation of the first Listahanan has been gradual, kept at low-key, and generally isolated from other surveys or data collection efforts that are being undertaken at the time. This allowed DSWD some level of liberty in doing parallel data collection operations on the ground. The case of Listahanan 2 faced several challenges that mounted to horizontal and vertical difficulties in coordination. Unlike the first round, Listahanan 2 used a longer instrument, reached out to more households, and undertaken on a massive scale over a shorter period of time. It was also done at the same time as two other nationwide data collection efforts were being undertaken by Government—the mid-decade population census (by the Philippine Statistical Authority) and the Social Welfare and Development Indicators (also by the DSWD). Consequently, there were reports of interviewee fatigue, large numbers of unassessed households including current Pantawid beneficiaries (due to transfers of residence, refusals, and lack of qualified respondent), and smaller turn-out during the validation phase. Succeeding Listahanan implementations will benefit from avoiding these other massive surveys and censuses in the future. 15 Findings from a robustness check of the current PMT models were presented to DSWD Officials and the NHTO on 03 September 2018. The analysis used out-of-sample national household data from the Family Income and Expenditure Survey (FIES) 2015, Labor Force Survey (LFS) Jan 2016, and the Form 5 (Barangay community characteristics) of the Census of Population and Housing (CPH) 2010. www.worldbank.org 21 The Philippines’ Targeting System for the Poor: Successes, lessons and ways forward To stay relevant, Listahanan operations need to catch up with technological innovations to adapt to the increasing need for faster response. After two rounds of implementation, the Listahanan system is mature enough to undergo big and necessary changes in order to adapt to the changing needs of the clients it serves. More and more Government agencies and partner institutions—donors, civic groups, and local governments—recognize the utility of the Listahanan to reach the poorest communities and populations. This, in turn, increases the demand for better and faster updating of the Listahanan registry. The system has to adapt to the need for more dynamic information, wider use of computer-assisted or mobile-based technologies for faster data collection and processing, more secure data storage and sharing facilities and protocols that balance protecting information and faster sharing in times of actual need. In addition, the Listahanan has to start planning to link up with the Philippines national identification system, which is another new and big reform that is expected to drive big changes in the landscape of service delivery in the country. References: Castaneda, T., et al. (2005). “Designing and Implementing Household Targeting Systems: Lessons from Latin American and The United States.” World Bank Social Protection Discussion Paper Series. The World Bank, Washington DC. Coady, D., M. Grosh, and J. Hoddinott (2004). “Targeting of Transfers in Developing Countries, a Review of Lessons and Experience.” World Bank Regional and Sectoral Studies. The World Bank, Washington DC. DSWD (January 2015). Listahanan – Talaan ng Pamilyang Nangagailangan Operations Manual. DSWD (July 2012). National Household Targeting System for Poverty Reduction Operations Manual. DSWD (March 2013). Social Welfare and Development Reform Project Midterm Report. Fernandez, L. (January 2012). “Technical Note on Estimations of Proxy Means Test (PMT) Model of the National Household Targeting System for Poverty Reduction (NHTS-PR) with the Latest Household Surveys: FIES-LFS 2009 in the Philippines.” Internal document prepared for DSWD. Grosh, M. C. del Ninno, E. Tesliuc, A. Ouerghi (2008). “For Protection & Promotion, the Design and Implementation of Effective Safety Nets.” The World Bank, Washington DC. Mapa, D. and M. Albis (July 2014). “New Proxy Means Test (PMT) Models: Improving Targeting of the Poor for Social Protection. Final Technical Note.” Unpublished manuscript prepared for DSWD. Velarde, R. (03 September 2018). “2015 PMT Validation Exercise,” presentation to DSWD. 22 WORLD BANK SOCIAL PROTECTION POLICY NOTE www.worldbank.org 23