Measuring Food Consumed Away from Home in Vietnam Pop quiz: • What did you eat outside of the home this past week? The Project • Can you recall how many different food items you ate, To understand how to capture best FAFH, together with and how much of each? the World Bank’s Living Standards Measurement Study and responding to the UN Statistical Commission’s call • What about drinks and snacks? for improved guidelines on data collection, we designed an experiment that tested different methods of collecting It’s harder than you think, isn’t FAFH consumption. We ran the experiment in Hanoi, it? It turns out that this tricky Vietnam, in collaboration with the General Statistics measurement also affects our Office of Vietnam (GSO), which collects the national household survey that is used to track poverty (the understanding of poverty. But Vietnam Household Living Standards Survey, or VHLSS). some solutions are on the way. We tested four different approaches: • A gold standard (expensive and impractical, but one that gave us a benchmark). We provided a personal food diary to each adult member of the household which helped them track, item-by-item While it may be obvious that the type and quantity and day-by-day, consumption of all FAFH for seven of food individuals consume in general can be an days; trained enumerators provided directions and important metric of individual and family welfare, check-ins to ensure compliance. looking specifically at food individuals eat outside their home may not seem particularly important. But for • The existing one-line approach, which is what the policy makers, accurate knowledge about the amount GSO currently implements, asking one household of food consumed away from home (FAFH) can be informant (through a single question) to recall FAFH essential to accurately understand trends in poverty away from home for everyone in the household. and inform policy accordingly. • Individual recall, designed to collect FAFH Consumption patterns are rapidly changing across the information directly from each adult in the household, developing world, with prepared and packaged meals, and and through a more detailed FAFH module (not meals consumed outside the home, taking an ever-growing just a single question!). By asking every individual share of the households’ food budget. And with rising separately about their own FAFH consumption rather incomes, urbanization, and women entering the labor force, than relying on a single household informant – and consumption of FAFH – and figuring out how to accurately asking separately about various meal events – we measure it – is becoming more important in poverty hypothesized this would be more accurate than the measurement work. Earlier work in Peru, for instance, single household informant, but less intensive than a found that the impact of excluding explicit measures of food diary. FAFH can be substantial – poverty rates were 16% lower when FAFH is accounted compared to when it’s not. • Targeting a household informant and drawing from behavioral science and survey methodology But how well do we capture measures of FAFH in general literatures, we developed a simple worksheet across the world? Not well, it turns out. Many nationally tool to help one informant keep track total FAFH representative household surveys collect very limited consumption for everyone in the house for one week. information on FAFH, in part due to the challenges involved This approach also enabled us to implement in a first – food consumed at home is likely to be more carefully visit a run-down of the module, allowing the informant tracked than FAFH, which can come from multiple sources, to know the information that was going to be collected in multiple locations, and at multiple times. After all, can you later. Furthermore, the two visits (one at the start of easily recall every snack you consumed in the last week outside the recall period and one at the end) made the recall your home? period more salient for the informant. The Results Using the personal diary as the “gold standard benchmark,” we compared FAFH data collected across the different approaches. The existing approach is the least accurate. The worksheet and bounding variation worked best. Unsurprisingly, the single household informants Making FAFH salient by having a visible object like a simple in the first arm, when asked in one question without worksheet helped individuals take notes during the week. In any behavioral intervention to recall all household addition, creating “bounds” for our measurement over seven consumption, underestimated FAFH by 33% relative days by having enumerators visit households at the start and to the gold standard. After all, that’s a lot of information finish of the measurement period also helped with precision. about a household to aggregate in a single question. Combined, using behavioral insights led to underestimating FAFH consumption by only 11%, a level that is actually not statistically different from the golden standard - a remarkable improvement from the existing methodology. MEAN PER CAPITA FAFH (in thousands of VND) 9000 Individual recall is more reliable, but people still make mistakes. 8000 When we get more granular and ask each individual in the household to make their own recollections, 7000 FAFH measurement improves – it deviates by 22% from the “gold standard.” 6000 5000 One-line Individual HH bounding Diary recall recall and salience (gold standard) Treatment arms Poor Policy Implications While the implementation of a better-designed module tested seem promising in drastically improving FAFH combined with the application of behavioral techniques – measurement. So part of the challenge is to do a cost benefit in this case, salience through a worksheet and bounding analysis ex ante as to “how much mismeasurement” are we combined – dramatically increased reliability of the willing to accept. measurement of FAFH, we recognize the factors at play that policy makers face when deciding what techniques Ultimately, the value of accurate food away from home data to employ to improve measurement, including time is essential for policy targeting, welfare tracking, and more. commitment and cost. In Understanding consumption patterns this project, rough estimates affects policymaking in critical areas of suggest that the individual Understanding consumption population welfare including poverty, recall treatment was 7% patterns affects policymaking food security, malnutrition, and non- more expensive than the status quo; the bounding and in critical areas of population communicable diseases. This work highlights the inaccuracy associated salience treatment was 33% welfare including poverty, food with collecting FAFH consumption as expensive; and the diary, or security, malnutrition, and from a single question in a survey, “benchmark” treatment, was and the high potential value of using twice more expensive. Time non-communicable diseases. behaviorally-informed approaches spent by enumerators visiting to identify cost-effective alternatives homes and gathering data that can better inform policy making over time. When also increases with more visits and reminders. At the considering approaches to measuring food away from home, end, the feasibility and costs of any of these options – or policy makers should consider their own context, costs, and variations of them - will depend on the field structure and needs to decide on improved methods of data collection. protocols that are already in place. But both new options This study provides useful insights into ways to go about it. About eMBeD The Mind, Behavior, and Development Unit (eMBeD), the World Bank’s behavioral science team in the Poverty and Equity Global Practice, works closely with project teams, governments, and other partners to diagnose, design, and evaluate behaviorally informed interventions. By collaborating with a worldwide network of scientists and practitioners, the eMBeD team provides answers to important economic and social questions, and contributes to the global effort to eliminate poverty and enhance equity. Stay Connected eMBeD@worldbank.org #embed_wb worldbank.org/embed bit.ly/eMBeDNews REFERENCES: Farfán, G., Genoni, M. E., & Vakis, R. (2017). You are what (and where) you eat: Capturing food away from home in welfare measures. Food Policy, 72, 146-156.