69142 Kementerian Koordinator Bidang Kesejahteraan Rakyat Indonesia’s PNPM Generasi Program Final Impact Evaluation Report June 2011 PN PM SUPPORT FACI LI TY DARI RAKYAT AMERIKA THE WORLD BANK, JAKARTA OFFICE Indonesia Stock Exchange Building Tower II/12th Floor Jl. Jend. Sudirman Kav. 52-53 Jakarta 12910 Tel: (6221) 5299-3000 Fax: (6221) 5299-3111 Website: www.worldbank.org/id Printed in March 2012 This volume is a product of staff of the World Bank. The findings, interpretations, and conclusions expressed herein do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement of acceptance of such boundaries. Indonesia’s PNPM Generasi Program Final Impact Evaluation Report June 2011 Benjamin A. Olken, M.I.T. Department of Economics Junko Onishi, The World Bank Susan Wong, The World Bank Table of Contents Acknowledgments v Glossary vii Executive Summary 1 Ringkasan Eksekutif 7 Introduction 15 1.1 Background 16 1.2 The Generasi project 18 1.3 Experimental Design 22 1.4 Survey Design and Implementation 24 Evaluation Methodology 29 2.1 Regression Specifications 30 2.2 Balance Tests 32 Main Results 35 3.1 Direct Benefits of Generasi Funds 36 3.2 Program Impact on Main Targeted Indicators 37 3.3 Program Impact on Long-Term Final Outcomes 38 3.4 Impact on Non-Targeted Indicators 39 Where Were Generasi’s Effects Largest? 43 4.1 Heterogeneity by Area Characteristics 44 4.2 Heterogeneity Over Time 46 4.3 Heterogeneity Across Poor vs. Non-poor Individuals 47 How And Why Did The Generasi Project Work? 51 5.1 Changes in Provider Quantities 52 5.2 Changes in Provider Quality 53 5.3 Changes in Provider Effort 53 5.4 Changes in Community Effort 53 5.5 Services and Prices from Providers 54 Discussion 57 Policy Implications And Conclusion 61 References 64 Annexes 67 Annex I. Randomization and Implementation of Generasi in 2007 68 ii Indonesia’s PNPM Generasi Program List of Figures Figure 1. Village funding allocations, 2007 and 2008 19 Figure 2. Timeline of project and surveys 25 List of Tables Table 1. Performance metrics and weights 21 Table 2. Questionnaire modules and sample size 26 Table 3. Generasi implementation and randomization results 69 Table 4. Baseline regressions, 12 main indicators * 70 Table 5. Baseline regressions, long-term final outcomes 71 Table 6. Direct benefits 72 Table 7. Direct benefits, incentivized vs. non-incentivized 73 Table 8. Direct benefits, provincial breakdown 75 Table 9. Program impact on main targeted indicators 77 Table 10. Program impact on main targeted indicators, incentivized vs. non-incentivized 79 Table 11. Program impact on main targeted indicators, provincial breakdown 81 Table 12. Program impact on longer term outcomes 84 Table 13. Program impact on longer term outcomes, incentivized vs. non-incentivized 86 Table 14. Program impact on longer term outcomes, provincial breakdown 89 Table 15. Program impact on non-targeted indicators 92 Table 16. Program impact on main targeted indicators, interactions with pre-period subdistrict level variables 95 Table 17. Program impact on main targeted indicators, interactions with pre-period subdistrict-level variables, incentivized vs. non-incentivized 97 Table 18. Program impact on longer-term outcomes, interactions with pre-period subdistrict level variables 100 Table 20. Program impact on longer-term outcomes, pre-period subdistrict average per capita consumption interactions 106 Table 21. Program impact on main targeted indicators, pre-period village access interactions 109 Table 22. Program impact on longer-term outcomes, pre-period village access interactions 111 Table 23. Program impact on main targeted indicators, excludes 49 newly added Generasi subdistricts in Year 2 113 Table 24. Program impact on main targeted indicators, comparing incentivized vs. non-incentivized, excludes 49 newly added Generasi subdistricts in Year 2 115 Table 25. Program impact on longer-term outcomes, excludes 49 newly added Generasi subdistricts in Year 2 117 Table 26. Direct benefits, per-capita consumption quintile breakdown 119 Table 27. Program impact on main targeted indicators, per-capita consumption quintile breakdown. 121 Table 29. Results for service provider quantities 126 Table 30. Results for service provider quantities, incentivized vs. non-incentivized 128 Final Impact Evaluation Report iii Table 31. Results for service provider quality (health and education infrastructure availability) 130 Table 32. Results for service provider quality (health and education infrastructure availability), incentivized vs. non-incentivized 132 Table 33. Results for service provider level of effort 134 Table 34. Results for service provider level of effort, incentivized vs. non-incentivized 135 Table 35. Results for community efforts at service provision, monitoring, and participation 137 Table 36. Results for community efforts at service provision, monitoring, and participation, incentivized vs. non-incentivized 140 Table 37. Service prices and supply 144 Table 38. Service prices and supply, incentivized vs. non-incentivized 149 List of Boxes Box 1. Generasi Program Target Indicators 17 iv Indonesia’s PNPM Generasi Program Acknowledgments The authors wish to thank the members of the PNPM Generasi Team, including Sadwanto Purnomo, Gerda Gulo, Juliana Wilson, Yulia Herawati, Gregorius Endarso, Gregorius Pattinasarany, Joey Neggers, Lina Marliani, Scott Guggenheim, Robert Wrobel, John Victor Bottini, Threesia Mariana Siregar, Sentot Surya Satria, Christine Panjaitan, Soenoe Widjajanti, Suhartini B. Rianto, and Erni Yanti Siregar. Special thanks go to Yulia Herawati, Gregorius Endarso, Joey Neggers, and Lina Marliani for their excellent and tireless support in survey preparation, oversight, and data preparation and analysis. The government of Indonesia—through the Ministry of Planning (Bappenas), the Coordinating Ministry for Economy and Social Welfare (Menkokesra), and the Ministry of Home Affairs—has provided tremendous support to the program and its evaluations over the past three years. Special thanks to Sujana Royat (Menkokesra); Prasetijono Widjojo, Endah Murniningtyas, Pungky Sumadi, and Vivi Yulaswati (Bappenas); and Ayip Muflich, Eko Sri Haryanto, and Bito Wikantosa (Ministry of Home Affairs) for their generous support of the PNPM Generasi program. The University of Gadjah Mada (UGM), Center for Public Policy Studies, implemented the field surveys over the last three rounds. SMERU, the Indonesian independent research organization, collaborated with the program to provide complementary qualitative studies in 2007 and 2009. In total, some 860 enumerators and researchers from these two Indonesian organizations contributed to this tremendous data collection effort. The authors are grateful to both institutions for their diligent work over the past three years. This final evaluation report drew from the 2007 baseline survey report written by Robert Sparrow, Jossy Moeis, Arie Damayanti, and Yulia Herawati. This report benefited from the comments of peer reviewers, including Ana-Maria Arriagada, Emanuela Galasso, Lisa Hannigan, Jack Molyneaux, Rebekah Pinto, Joppe Jaitze De Ree, Emmanuel Skoufias, and Sudarno Sumarto. The team is also grateful for the collaboration with the Program Keluarga Harapan evaluation team from the World Bank, consisting of Vivi Alatas, Jon Jellema, and Edgar Janz. Editorial assistance for this report was provided by Robert Livernash. Anju Sachdeva and Elizabeth Acul provided valuable administrative support. Financial support for the overall PNPM Generasi program and the evaluation series has come from the government of Indonesia; the World Bank Decentralization Support Facility; the Netherlands Embassy; the PNPM Support Facility, which consists of donors from Australia, the United Kingdom, the Netherlands, and Denmark; and the Spanish Impact Evaluation Fund. Final Impact Evaluation Report v The findings, interpretations, and conclusions expressed in this paper are entirely those of the authors and should not be attributed in any manner to the World Bank, to its affiliated organizations, or to members of its Board of Executive Directors, or the countries they represent. The World Bank does not guarantee the accuracy of the data included in this publication and accepts no responsibility for any consequence of their use. vi Indonesia’s PNPM Generasi Program Glossary ANC Antenatal care ARI Acute respiratory infection Bappenas Ministry of Planning Buku KIA Mother and child health book CCT Conditional cash transfers CDD Community-driven development GIS Geographical information system Gotong royong Semi-volunteer public labor service at the village or community level KDP Subdistrict (Kecamatan) Development Project KIA Kartu Insentif Anak (Child Incentive Health Card) Menkokesra Coordinating Ministry for Economic and Social Welfare MIT Massachusetts Institute of Technology NTT Nusa Tenggara Timur province PKH Program Keluarga Harapan (Hopeful Family Program) PNC Postnatal care PNPM Program Nasional Pemberdayaan Masyarakat (National Program for Community Empowerment) PNPM Generasi Sehat dan Cerdas PNPM Healthy and Smart Generation PNPM-Rural Program Nasional Pemberdayaan Masyarakat Perdesaan (National Program for Community Empowerment in Rural Areas) PNPM-Urban Program Nasional Pemberdayaan Masyarakat Perkotaan (National Program for Community Empowerment in Urban Areas) PODES Village potential statistics Posyandu Village integrated health post (monthly community weighing post) Puskesmas Community health center SUSENAS National Socioeconomic Survey SD Sekolah Dasar (primary school) SD Standard deviations SMP Sekolah Menengah Pertama (junior secondary school) SPADA Support for Poor and Disadvantaged Areas Project UGM Gadjah Mada University Notes: All $ = U.S. dollars, unless otherwise noted Final Impact Evaluation Report vii Executive Summary Executive Summary ndonesia has made remarkable strides in key human development indicators over the past few decades. I Primary school enrollment is close to universal for both boys and girls, and the child mortality rate has declined rapidly. Nevertheless, infant mortality, child malnutrition, maternal mortality, junior secondary school enrollment, and educational learning quality have all remained problematic in Indonesia compared to other countries in the region. Furthermore, achievements in these indicators reveal large geographical disparities, with poorer outcomes in rural and remote provinces and districts. Improving access to basic quality health and education services is a key component of an overall poverty reduction strategy for Indonesia. In 2007, the government of Indonesia launched two large-scale pilots of programs designed to tackle these issues: (1) conditional cash transfers (CCTs) to households, known as the Hopeful Family Program (Keluarga Harapan Program or PKH), and (2) an incentivized community block grant program, known as the National Community Empowerment Program—Healthy and Smart Generation (Program Nasional Pemberdayaan Masyarakat—Generasi Sehat dan Cerdas, or PNPM Generasi). These two complementary pilot projects began in six provinces and are designed to target the same health and education indicators. They are consistent with both the Indonesian government’s priorities and the Millennium Development Goals: to reduce poverty, maternal mortality, and child mortality, and to ensure universal coverage of basic education. PKH focused more on supply-side ready areas, predominantly urban and in Java, while PNPM Generasi operated in rural areas. This study reports on the final evaluation of the incentivized community block grant program, PNPM Generasi. A separate report has been prepared by the World Bank on the results of the PKH program. The Generasi project began in mid-2007 in rural areas of five Indonesian provinces selected by the government: West Java, East Java, North Sulawesi, Gorontalo, and Nusa Tenggara Timur. The project builds on the Indonesian government’s existing community-driven development program, known as the National Community Empowerment Program in Rural Areas (PNPM-Rural) or formerly, the Kecamatan Development Program (KDP). The Ministry of Home Affairs Community Development Department implements both Generasi and PNPM-Rural. In 2007, Generasi covered 1,605 villages in 129 subdistricts, with a total budget of $20 million. By 2009, Generasi covered just over 2,000 villages in 164 subdistricts in the same five provinces with a total annual budget of $40 million. The program operates as follows: each year, villages receive a block grant. With the assistance of trained program facilitators and service delivery workers, villagers undertake a social mapping and participatory planning exercise to decide how best to use the block grant funds to reach 12 education and health targets related to maternal and child health behavior and education behavior. These 12 targets relate to prenatal and postnatal care, child immunizations, and primary and junior secondary school enrollment and attendance. To give communities incentives to focus on the most effective policies, the government bases the size of the village’s Generasi block grant for the subsequent year partly on the village’s performance on each of the targeted indicators. The Generasi project thereby takes the idea of performance incentives from conditional cash transfer programs and applies it in a way that allows communities the flexibility to address supply constraints, demand constraints, or some combination. To the best of our knowledge, the Generasi project is the first health and education program worldwide that combines community block grants with explicit performance bonuses for communities. To allow for a rigorous, randomized evaluation of Generasi, the government of Indonesia incorporated random assignment into the selection of Generasi locations. Within the districts selected by the government 2 Indonesia’s PNPM Generasi Program Executive Summary for the program, entire subdistricts (kecamatan) were randomly allocated to either receive Generasi or to be in a control group. Each Generasi location was further randomly allocated to one of two versions of the program: one “incentivized” treatment with the pay-for-performance component (treatment A) described above, and a second, otherwise identical “non-incentivized” treatment without the pay-for-performance incentives (treatment B). With over 2,100 villages randomized to receive either the incentivized or non- incentivized version of the Generasi program (plus over 1,000 villages in control subdistricts), and over 1.8 million target beneficiaries of the program in treatment areas, to the best of our knowledge this represents one of the largest randomized social experiments conducted in the world to date. This document describes the findings from the three-wave evaluation series carried out from 2007 to 2010. The baseline survey took place from June 2007 to August 2007. The second wave was conducted from October 2008 to January 2009, after 15 to 18 months of Generasi implementation. The third and final evaluation survey was implemented from October 2009 to January 2010 after 27 to 30 months of project implementation. Over 45,000 household members, village heads, and school and health facility staff were respondents for the third and final round of survey. The evaluation series also included a qualitative component. To the extent possible, the authors have incorporated findings from the complementary qualitative study in 12 villages in two provinces; this qualitative component was conducted in 2007 and 2009. The qualitative study—using focus group discussions, in-depth key informant interviews, and direct observation—provided deeper insights into processes, causal chains, and villagers’ values, motivations, and reactions. The main findings of the Generasi impact evaluation are as follows: 1. After 30 months of program implementation, Generasi had a statistically significant positive impact on average across the 12 indicators it was designed to address. The strongest improvements among the health indicators were in the frequency of weight checks for young children. The program also increased the number of iron sachets pregnant mothers received through antenatal care visits. These improvements were supported by dramatic increases in mothers and children participating in village health post (posyandu) activities to receive the targeted maternal, neonatal, and child health services. Education indicators also saw improvements in the final evaluation, reversing the zero or negative impact found at the interim evaluation. The improvement in education indicators was most notable in the increased school participation rate among the primary school-age group. 2. The main long-term impact was a decrease in malnutrition. The latest Wave III survey shows that childhood malnutrition1 was reduced by 2.2 percentage points, about a 10 percent reduction from the control level. This reduction in malnutrition was strongest in areas with a higher malnutrition rate prior to project implementation, most notably in the Nusa Tenggara Timur (NTT) Province, where underweight rates were reduced by 8.8 percentage points, a 20 percent decline compared to control areas; severe underweight rates were reduced by 5.5 percentage points, a 33 percent decline; and severe stunting was reduced by 6.6 percentage points, a 21 percent decline compared to control areas. Surprisingly, in Java, there was a negative impact on stunting and severe stunting which needs to be explored further. Although reductions in infant and child mortalities were observed in the interim evaluation, the same levels of reduction in mortality were not sustained in the final 1 Childhood malnutrition was measured by weight-for-age of children under three. Final Impact Evaluation Report 3 Executive Summary evaluation. In terms of the longer-term education learning outcomes, the program did not improve childhood test scores as yet. 3. Making grants conditional on performance improves program effectiveness in health but not in education. On average, the incentivized group outperformed the non-incentivized group in improving health indicators, particularly in increasing antenatal care services. On net, between 50- 75% of the total impact of the block grant program on health indicators can be attributed to the performance incentives. However, for education indicators the incentivized group did not appear to perform better than the non-incentivized group. There may be several reasons for this. The data shows two results: first, the impact of incentives became weaker over time; and second, the positive impacts on education only occurred in Year Two of the program, probably due to time lags in implementation of the education interventions. Thus, by Year Two, the incentives were less strong just when the program was beginning to have impacts on education. Another factor may have been that gains in health were easier to attain than gains in education. Baseline levels for health indicators were lower than for education, making it perhaps easier to make gains in health. In addition, education targets may have been more difficult to achieve since those targets involved more people and involved school enrollment and attendance every day, as opposed to once-a-month for health targets, with fewer villagers involved. The qualitative report also suggests that communities favored giving school assistance directly to the greatest number of students, rather than out-of- school children, and that motivation may have dampened any effects from the incentives. Lastly, the qualitative report indicates that the incentives rules were sometimes difficult for communities to understand. 4. Generasi had the greatest impact in areas with low baseline health and education indicators. Areas with lower pre-project health and education indicators have more room for improvements. The greater impacts in areas with lower baseline indicators appear more prominently in the final evaluation survey than the interim results, with stronger improvements found in education indicators in these areas. On average, the program was about twice as effective in areas at the 10th percentile of service provision (very low health and education status) at baseline as it was on average. However, these improvements in health and education indicators in areas with low baseline coverage did not appear to have resulted in improving long-term health and education outcomes in these areas outside of malnutrition. Furthermore, the greater impacts observed in health and education indicators were not simply correlated with pre-project levels of poverty, but instead were driven by the level of health and education indicators in the area. Policy Implications and Conclusion The evaluation results point to several relevant policy implications and conclusions: 1. Generasi is most effective in areas with low health and education status. The impact evaluation found that Generasi impacts are stronger in areas where health and education indicators are low. This suggests that future expansion of Generasi implementation should prioritize areas where these indicators are lagging behind and not necessarily in areas identified as poor. 4 Indonesia’s PNPM Generasi Program Executive Summary 2. Community incentives had mixed results—health responded more favorably than education. As a result of the two-year project implementation, health indicators responded positively to community incentives, but education indicators saw no positive or negative response to community incentives. Learning from this experience, the government may wish to see how community incentive interventions can work in other lagging areas, e.g., water and sanitation access. The policy implications are that poverty programs may wish to experiment more with embedding incentives into the designs; however, the interventions and incentives will need to be monitored and evaluated over time. One possibility is that the conditionalities may work less well over time, as there may be more “gaming” of the system as the program progresses and the rules become more familiar. Alternatively, the program may work better over time as it continues to incentivize communities to work harder toward the specified targets. In addition, qualitative evidence suggests that simplifying the incentive scheme may make it much easier for communities to understand. 3. The government’s existing national community-driven development architecture and network (PNPM) was useful as a platform for other forms of local assistance. Generasi was started as an experiment in adapting the community participatory planning and block grant process to focus on specific education and health targets that were not being addressed sufficiently in the existing community program. This project has illustrated the flexibility and adaptability of this community model once the architecture and machinery are established. It also serves as a possible vehicle for improving health and education indicators in supply-deficient areas, where the traditional household conditional cash transfer model may not be as effective due to supply constraints. 4. The project should regularly review the appropriateness of the targets. Target indicators must be relevant to communities, yet reflect development priorities of the government. Although it is important not to overload the project with too many target indicators, Generasi should regularly review its 12 target indicators and assess if existing ones should be replaced or added. For example, school participation for children ages 7–12 has now reached nearly universal coverage at 98.5 percent and higher, and the program may wish to add other targets to capture other priority areas that are lagging, such as education learning achievement, early childhood development, or water and sanitation. 5. A follow-up evaluation may be needed in the future to examine the longer term sustainability of interventions and impacts. The final round of evaluation took place after 2.5 to 3 years of project implementation. This three-year evaluation series was useful in providing empirical evidence to inform project implementation and learn lessons for the next phase. Should the program continue in the original treatment sites and should the original control sites remain as such, the government may wish to consider the possibility of another evaluation in a few years to examine if the impacts of this program are indeed sustainable over time and if additional progress can be made on learning and health outcomes. Final Impact Evaluation Report 5 6 Indonesia’s PNPM Generasi Program Ringkasan Eksekutif Final Impact Evaluation Report 7 Executive Summary ndonesia telah membuat kemajuan yang besar terhadap indikator utama pembangunan manusia I dalam beberapa dekade terakhir. Angka partisipasi sekolah dasar hampir mencapai cakupan universal untuk anak laki-laki maupun perempuan, dan angka kematian anak menurun dengan pesat. Namun demikian, angka kematian bayi, angka kurang gizi pada anak, angka kematian ibu, angka pendaftaran sekolah menengah pertama, dan kualitas pembelajaran pendidikan masih menjadi masalah di Indonesia dibandingkan dengan negara-negara lain di kawasan regional. Selanjutnya, pencapaian indikator-indikator ini menyingkapkan adanya kesenjangan geografis yang besar di mana hasil yang lebih rendah dialami oleh provinsi dan kabupaten perdesaan yang terpencil. Peningkatan akses terhadap pelayanan kesehatan dan pendidikan dasar yang berkualitas merupakan komponen utama dari strategi pengentasaan kemiskinan secara keseluruhan bagi Indonesia. Pada tahun 2007, Pemerintah Indonesia meluncurkan dua program percontohan berskala besar yang dirancang untuk mengatasi persoalan-persoalan tersebut: (1) bantuan langsung bersyarat (conditional cash transfer atau CCT) kepada rumah tangga, yang dikenal sebagai Program Keluarga Harapan (PKH), dan (2) program bantuan langsung masyarakat ‘berinsentif ”, yang dikenal sebagai Program Nasional Pemberdayaan Masyarakat—Generasi Sehat dan Cerdas, atau PNPM Generasi. Kedua proyek percontohan yang saling melengkapi ini mulai dilaksanakan di enam provinsi dan dirancang untuk mencapai indikator kesehatan dan pendidikan yang sama. Proyek-proyek tersebut sejalan dengan prioritas Pemerintah Indonesia dan Tujuan Pembangunan Milenium: mengurangi angka kemiskinan, kematian ibu, dan kematian anak, dan memastikan cakupan pendidikan dasar yang bersifat universal. PKH lebih berfokus pada daerah- daerah yang siap memberikan pelayanan (supply-side), terutama di daerah-daerah perkotaan dan Pulau Jawa sedangkan PNPM Generasi melakukan kegiatan di daerah-daerah perdesaan. Studi ini melaporkan evaluasi akhir terhadap program bantuan langsung masyarakat “berinsentif ”, PNPM Generasi. Sebuah laporan terpisah telah disusun oleh Bank Dunia mengenai hasil program PKH. Proyek Generasi dimulai pada pertengahan tahun 2007 di daerah perdesaan di lima provinsi di Indonesia yang dipilih oleh pemerintah: Jawa Barat, Jawa Timur, Sulawesi Utara, Gorontalo, dan Nusa Tenggara Timur. Proyek ini memanfaatkan program pembangunan berbasis masyarakat Pemerintah Indonesia yang dikenal sebagai Program Nasional Pemberdayaan Masyarakat Perdesaan (PNPM- Mandiri Perdesaan) atau yang sebelumnya disebut Program Pengembangan Kecamatan (PPK). Ditjen Pemberdayaan Masyarakat dan Desa (PMD) Kementerian Dalam Negeri adalah pelaksana PNPM Generasi maupun PNPM Mandiri Perdesaan. Pada tahun 2007, PNPM Generasi mencakup 1.605 desa di 129 kecamatan, dengan total anggaran sebesar AS$20 juta. Pada tahun 2009, PNPM Generasi mencakup lebih dari 2.000 desa di 164 kecamatan di kelima provinsi yang sama dengan total anggaran tahunan sebesar AS$40 juta. Program ini melakukan kegiatan sebagai berikut: setiap tahun, desa menerima dana hibah (block grants). Dengan bantuan dari fasilitator program yang terlatih dan petugas pemberi pelayanan, warga desa melakukan pemetaan sosial dan perencanaan partisipatif untuk menentukan cara terbaik dalam memanfaatkan dana hibah untuk mencapai 12 target pelayanan pendidikan dan kesehatan yang berkaitan dengan perilaku kesehatan ibu dan anak serta perilaku pendidikan. Ke-12 target tersebut berkaitan dengan pemeriksaan kehamilan dan perawatan nifas, imunisasi anak serta angka pendaftaran dan tingkat kehadiran siswa sekolah dasar dan sekolah menengah pertama. Agar masyarakat terdorong untuk berfokus pada kebijakan-kebijakan yang paling efektif maka pemerintah mendasarkan ukuran hibah PNPM Generasi bagi setiap desa untuk tahun berikutnya pada kinerja desa atas setiap indikator yang ditargetkan. Dengan demikian, PNPM Generasi selangkah lebih maju daripada gagasan pemberian 8 Indonesia’s PNPM Generasi Program Executive Summary insentif kinerja dalam program-program transfer tunai bersyarat yang lain dan menerapkannya dengan cara yang memberikan fleksibilitas kepada masyarakat untuk mengatasi kendala dari sisi penyelenggara pelayanan (supply constraints), kendala dari sisi pemanfaat pelayanan (demand constraints), atau pun keduanya. Sepanjang yang kami ketahui, PNPM Generasi adalah program pelayanan kesehatan dan pendidikan pertama di dunia yang menggabungkan pemberian bantuan langsung kepada masyarakat dengan insentif kinerja yang eksplisit bagi masyarakat. Agar evaluasi terhadap PNPM Generasi dapat dilakukan secara saksama dan acak (random) maka Pemerintah Indonesia menentukan lokasi program secara acak (random assignment). Di kabupaten- kabupaten yang dipilih oleh pemerintah untuk melaksanakan program, seluruh kecamatan secara acak dialokasikan untuk mendapatkan bantuan PNPM Generasi atau masuk dalam kelompok kontrol. Setiap lokasi PNPM Generasi selanjutnya dialokasikan secara acak kepada salah satu dari dua versi program: pertama, perlakuan “berinsentif ” (incentivized) dengan komponen pembayaran atas kinerja (treatment A) yang diuraikan di atas, dan kedua, perlakuan serupa “non-berinsentif ” (non-incentivized) yaitu tanpa insentif pembayaran atas kinerja (treatment B). Dengan lebih dari 2,100 desa yang dialokasikan secara acak untuk menerima program Generasi yang “berinsentif ” (incentivized) atau “non-berinsentif ” (non- incentivized) (dan lebih dari 1,000 desa yang masuk dalam kelompok kontrol), lalu dengan lebih dari 1.8 juta pemanfaat program, ini merupakan salah satu bentuk eksperimen sosial secara acak terbesar di dunia yg telah dilakukan sampai saat ini. Dokumen ini menguraikan temuan-temuan dari evaluasi yang dilaksanakan dalam tiga gelombang dari tahun 2007 sampai 2010. Survei dasar (baseline survey) berlangsung dari bulan Juni 2007 sampai Agustus 2007. Survei kedua diadakan dari bulan Oktober 2008 sampai Januari 2009 setelah pelaksanaan PNPM Generasi selama 15 sampai 18 bulan. Survei ketiga dan terakhir dilaksanakan dari bulan Oktober 2009 sampai Januari 2010 setelah pelaksanaan proyek selama 27 sampai 30 bulan. Lebih dari 45.000 anggota rumah tangga, kepala desa serta petugas sekolah dan sarana pelayanan kesehatan menjadi responden dalam survei putaran ketiga dan terakhir ini. Rangkaian evaluasi ini juga mencakup komponen kualitatif. Sedapat mungkin, penulis telah mencantumkan temuan-temuan dari studi kualitatif pendukung di 12 desa di dua provinsi; komponen kualitatif ini dilaksanakan pada tahun 2007 dan 2009. Studi kualitatif—yang menggunakan diskusi kelompok fokus, wawancara mendalam dengan narasumber kunci, dan observasi langsung—memberikan pemahaman yang lebih dalam mengenai proses, rantai penyebab serta nilai-nilai, motivasi dan reaksi warga masyarakat. Temuan-temuan utama dari evaluasi terhadap dampak PNPM Generasi adalah sebagai berikut: 1. Setelah 30 bulan pelaksanaan program, PNPM Generasi rata-rata mempunyai dampak positif yang signifikan secara statistik terhadap 12 indikator yang menjadi tujuan dari rancangan program. Peningkatan terbesar di antara indikator-indikator kesehatan adalah pada frekuensi pemeriksaan berat badan anak. Program juga meningkatkan pemberian jumlah tablet zat besi kepada ibu hamil pada saat kunjungan pemeriksaan kehamilan. Peningkatan-peningkatan tersebut didukung oleh kenaikan yang dramatis pada jumlah ibu dan anak yang mengikuti kegiatan Posyandu untuk mendapatkan pelayanan kesehatan ibu, bayi baru lahir dan anak. Indikator pendidikan juga mengalami peningkatan pada saat dilakukan evaluasi akhir, dibandingkan dengan yang ditemukan Final Impact Evaluation Report 9 Executive Summary dalam evaluasi sementara (interim) dengan dampak nihil atau negatif. Peningkatan indikator pendidikan yang paling menonjol terdapat pada kenaikan angka partisipasi sekolah dari kelompok usia sekolah dasar. 2. Dampak jangka panjang utama adalah penurunan angka kurang gizi. Hasil dari survei Gelombang III yang terakhir memperlihatkan bahwa angka kurang gizi pada anak2 menurun sebanyak 2,2 persen, yang berarti penurunan sekitar 10 persen dari tingkat kontrol. Penurunan angka kurang gizi terbesar terjadi di daerah-daerah dengan angka kurang gizi yang lebih tinggi sebelum pelaksanaan proyek, terutama di Provinsi Nusa Tenggara Timur (NTT), di mana angka berat badan kurang menurun sebanyak 8,8 persen, yang berarti penurunan 20% dibandingkan dengan daerah-daerah kontrol; angka berat badan sangat kurang menurun sebanyak 5,5 persen, yang berarti penurunan 33 persen dibandingkan dengan daerah-daerah kontrol; dan angka tubuh sangat pendek (severe stunting) menurun sebanyak 6,6 persen, yang berarti penurunan 21 persen dibandingkan dengan daerah-daerah kontrol. Yang mengejutkan, di Pulau Jawa, terdapat dampak negatif terhadap angka tubuh pendek dan sangat pendek yang perlu dikaji lebih jauh. Meskipun terjadi penurunan angka kematian bayi dan anak berdasarkan hasil evaluasi sementara, tingkat penurunan angka kematian tersebut mengalami perubahan pada evaluasi akhir. Dalam hal hasil pembelajaran pendidikan jangka panjang, program belum berhasil meningkatkan nilai hasil ujian siswa. 3. Pemberian hibah bersyarat kinerja meningkatkan efektivitas program di sektor kesehatan tetapi tidak di sektor pendidikan. Secara rata-rata, kelompok berinsentif memiliki kinerja yang lebih baik daripada kelompok non-berinsentif dalam meningkatkan indikator kesehatan, terutama dalam memperluas pelayanan pemeriksaan kehamilan. Secara netto, antara 50-75% total dampak program pemberian hibah terhadap indikator kesehatan dihasilkan oleh insentif atas kinerja. Namun untuk indikator pendidikan, kelompok berinsentif tampaknya tidak mempunyai kinerja yang lebih baik daripada kelompok non-berinsentif. Hal ini mungkin disebabkan oleh beberapa alasan. Data yang ada memperlihatkan dua hasil: pertama, dampak insentif semakin melemah dari waktu ke waktu; dan kedua, dampak positif terhadap pendidikan hanya terjadi pada Tahun Kedua program, yang mungkin disebabkan oleh penundaan dalam pelaksanaan intervensi pendidikan. Jadi, pada Tahun Kedua, insentif justru melemah ketika program baru mulai memberikan dampaknya terhadap sektor pendidikan. Faktor lainnya adalah keberhasilan di sektor kesehatan lebih mudah dicapai daripada di sektor pendidikan. Patokan dasar untuk indikator kesehatan lebih rendah daripada untuk indikator pendidikan, sehingga indikator kesehatan mungkin lebih mudah untuk ditingkatkan. Selain itu, target pendidikan lebih sulit dicapai mengingat target tersebut melibatkan lebih banyak orang dan meliputi angka pendaftaran dan kehadiran siswa di sekolah setiap hari, sedangkan target kesehatan hanya sebulan sekali dan melibatkan lebih sedikit warga masyarakat. Laporan kualitatif juga menunjukkan bahwa masyarakat lebih suka jika bantuan sekolah diberikan langsung kepada sebanyak mungkin siswa dan bukan kepada anak di luar sekolah dan bahwa motivasi seperti ini mungkin telah melemahkan pengaruh dari insentif. Terakhir, laporan kualitatif menunjukkan bahwa peraturan pemberian insentif kadang-kadang sulit dipahami oleh masyarakat. 2 Angka kurang gizi pada anak diukur menurut berat badan berbanding usia anak di bawah tiga tahun. 10 Indonesia’s PNPM Generasi Program Executive Summary 4. PNPM Generasi mempunyai dampak terbesar di daerah-daerah dengan indikator dasar kesehatan dan pendidikan yang rendah. Daerah-daerah dengan indikator kesehatan dan pendidikan pra-proyek yang lebih rendah mempunyai ruang lebih luas untuk peningkatan. Dampak yang lebih besar di daerah-daerah dengan indikator dasar yang lebih rendah tampak lebih menonjol pada hasil survei evaluasi akhir ketimbang hasil survei evaluasi sementara, dan peningkatan yang lebih besar terlihat pada indikator kesehatan di daerah-daerah tersebut. Secara rata-rata, efektivitas program dua kali lebih besar di daerah-daerah yang termasuk dalam kelompok persentil ke-10 dari pemberian pelayanan (status kesehatan dan pendidikan yang sangat rendah) sebagai patokan dasar. Namun, peningkatan indikator kesehatan dan pendidikan di daerah-daerah dengan cakupan dasar yang rendah tampaknya belum mendorong peningkatan hasil-hasil kesehatan dan pendidikan jangka panjang di daerah-daerah tersebut, kecuali angka kurang gizi. Selanjutnya, dampak lebih besar yang terlihat pada indikator kesehatan dan pendidikan bukan hanya berkorelasi dengan tingkat kemiskinan pra-proyek melainkan juga dihasilkan oleh tingkat indikator kesehatan dan pendidikan di daerah tersebut. Implikasi Kebijakan dan Kesimpulan Evaluasi ini menghasilkan beberapa implikasi kebijakan dan kesimpulan yang relevan: 1. PNPM Generasi mengalami efektivitas terbesar di daerah-daerah dengan status kesehatan dan pendidikan yang rendah. Evaluasi dampak ini menemukan bahwa dampak PNPM Generasi lebih kuat di daerah-daerah di mana terdapat indikator kesehatan dan pendidikan yang rendah. Hal ini menunjukkan bahwa perluasan implementasi PNPM Generasi di masa mendatang hendaknya mengutamakan daerah-daerah dengan indikator yang masih rendah dan tidak harus selalu di daerah- daerah yang diidentifikasi sebagai daerah miskin. 2. Insentif kepada masyarakat mendatangkan hasil yang beragam—kesehatan merespons dengan lebih baik daripada pendidikan. Sebagai hasil dari pelaksanaan proyek selama dua tahun, indikator kesehatan merespons dengan positif terhadap insentif masyarakat, tetapi indikator pendidikan memberikan respons negatif. Dengan belajar dari pengalaman ini, pemerintah mungkin ingin melihat bagaimana intervensi insentif masyarakat dapat berhasil di bidang-bidang lain yang masih tertinggal, misalnya akses terhadap air bersih dan sanitasi. Sebagai implikasi kebijakan, program pengentasan kemiskinan mungkin ingin memperluas ujicobanya dengan memasukkan insentif ke dalam rancangan-rancangannya; meskipun demikian, intervensi dan insentif tersebut perlu dipantau dan dievaluasi dari waktu ke waktu. Salah satu kemungkinan yang bisa terjadi adalah kondisionalitas dapat menjadi kurang efektif seraya waktu berlalu karena mungkin ada lebih banyak “permainan” sistem seraya program berjalan dan peraturan lebih dikenal. Kemungkinan lain adalah, program dapat menjadi lebih efektif seraya waktu berlalu karena program terus mendorong masyarakat untuk bekerja lebih keras dalam rangka mencapai target yang ditetapkan. Selain itu, bukti kualitatif memperlihatkan bahwa jika skema insentif disederhanakan maka masyarakat akan jauh lebih mudah memahaminya. Final Impact Evaluation Report 11 Executive Summary 3. Arsitektur dan jaringan Program Nasional Pemberdayaan Masyarakat (PNPM) dapat digunakan sebagai platform untuk bentuk bantuan lokal lainnya. PNPM Generasi dimulai sebagai program ujicoba dalam menyesuaikan proses perencanaan partisipatif masyarakat dan pemberian hibah yang berfokus pada target pendidikan dan kesehatan tertentu yang belum secara memadai dicakup dalam program-program masyarakat yang ada. Proyek ini menggambarkan fleksibilitas dan adaptabilitas model program masyarakat setelah arsitektur dan mekanismenya ditetapkan. Proyek ini juga berfungsi sebagai sarana yang memungkinkan peningkatan indikator kesehatan dan pendidikan di daerah-daerah yang kurang terlayani (supply deficient) di mana model bantuan langsung bersyarat tradisional kepada rumah tangga mungkin kurang efektif akibat adanya keterbatasan penyelenggara pelayanan (supply constraints). 4. Proyek perlu secara teratur meninjau kesesuaian target. Indikator-indikator target harus relevan dengan masyarakat namun tetap mencerminkan prioritas pembangunan pemerintah. Meskipun penting untuk tidak terlalu membebani proyek dengan terlalu banyak indikator target, PNPM Generasi hendaknya secara teratur meninjau ke-12 indikator targetnya dan mengkaji apakah indikator-indikator yang ada perlu diganti atau ditambahkan. Misalnya, angka partisipasi sekolah untuk anak usia 7-12 tahun sekarang hampir mencapai cakupan universal sebesar 98,5 persen atau lebih tinggi, dan program mungkin ingin menambahkan target lain yang mencakup bidang-bidang prioritas lain yang masih tertinggal, seperti pencapaian pembelajaran pendidikan, pengembangan anak usia dini, atau air bersih dan sanitasi. 5. Sebuah evaluasi lanjutan mungkin dibutuhkan di masa mendatang untuk memeriksa keberlanjutan intervensi dan dampak program dalam jangka yang lebih panjang. Evaluasi gelombang terakhir berlangsung setelah 2,5 sampai 3 tahun pelaksanaan proyek. Rangkaian evaluasi selama tiga tahun ini berguna untuk menyediakan bukti empiris yang akan mempengaruhi pelaksanaan proyek dan memetik pelajaran bagi tahap selanjutnya. Seandainya program masih berlanjut di lokasi perlakuan semula sehingga lokasi kontrolnya pun tidak berubah maka pemerintah mungkin ingin mempertimbangkan kemungkinan evaluasi lain yang akan dilaksanakan dalam beberapa tahun mendatang untuk memeriksa apakah dampak program ini memang berkelanjutan dari waktu ke waktu dan apakah kemajuan tambahan dapat dicapai sehubungan dengan hasil pendidikan dan kesehatan. 12 Indonesia’s PNPM Generasi Program Executive Summary Final Impact Evaluation Report 13 Introduction Introduction 1.1 Background ver the past decades, Indonesia has made remarkable strides in key human development O indicators. Primary school enrollment is close to universal for both boys and girls and the child mortality rate has declined rapidly (World Bank 2006; World Bank 2008). Nevertheless, infant and maternal mortality, child malnutrition, junior secondary school enrollment, school transition rates, and learning outcomes are lower in Indonesia than in other countries in the region (World Bank 2006; World Bank 2008). Furthermore, there are substantial geographical disparities in these outcomes, with poorer outcomes in rural and remote provinces and districts. Improving the health and education of children is considered critical to economic development and forms an important component of the Millennium Development Goals. Faced with these challenges, many developing countries have sought to stimulate demand for maternal and child health services and education through conditional cash transfer programs. For example, Mexico’s Progresa program (Gertler 2004; Schultz 2004; Rawlings and Rubio 2005) links cash payments to behaviors such as immunization, growth monitoring, school enrollment, and school attendance. However, these types of demand-side interventions may be inappropriate in many developing world contexts, where beneficiaries do not have adequate access to health and education services (Schubert and Slater 2006; Lagarde, Haines, and Palmer 2007). In such environments, programs that address both the supply- and demand-side constraints directly may be more appropriate. In 2007, the government of Indonesia launched two large-scale pilots of programs designed to tackle these issues: (1) conditional cash transfers to households, and (2) an incentivized community block grant program. These two pilot projects began in six provinces and were designed to achieve the same objectives and goals. These goals are consistent with the Indonesian government’s priorities and the Millennium Development Goals: to reduce poverty, maternal mortality, and child mortality, as well as ensure universal coverage of basic education. The Household CCT—the Keluarga Harapan Program (PKH)—applies the traditional CCT design with quarterly cash transfers to poor individual households identified through statistical means. CCT recipient households receive regular cash transfers through the post office as long as they meet the requirements of using specified health and education services. When it began, PKH focused primarily upon more supply- side-ready urban areas, primarily in Java. The Incentivized Community Block Grant Program, known as PNPM Generasi, differs from the Household CCT in that block grants are allocated to communities rather than to individual targeted households. Unlike PKH, PNPM Generasi focuses primarily on rural areas. This pilot program builds upon an existing Indonesian government community program known as the National Community Empowerment Program in Rural Areas (PNPM-Rural). Under PNPM Generasi, over 1,600 rural villages received an annual block grant during the first year. Each village can use the grant for any activity that supported one of 12 indicators of health and education service delivery (such as prenatal and postnatal 16 Indonesia’s PNPM Generasi Program Introduction care, childbirth assisted by trained personnel, immunization, school enrollment, and school attendance). To give communities incentives to focus on the most effective policies, the government bases the size of the village’s Generasi block grant for the subsequent year partly on the village’s performance on each of the 12 targeted health and education indicators. The Generasi project thus takes the idea of performance incentives from conditional cash transfer programs and applies it in a way that allows communities the flexibility to address supply constraints, demand constraints, or some combination. To the best of our knowledge, the Generasi project is the first health and education program worldwide that combines community block grants with explicit performance bonuses for communities. To allow for a rigorous, randomized evaluation of Generasi, the government of Indonesia incorporated random assignment into the selection of Generasi locations. Unlike current evaluations of conditional cash transfer programs, which cannot separately identify the impact of the incentives from the impact of the additional cash provided (Gertler 2004), the Generasi evaluation was designed to separate out these two effects. Specifically, each Generasi location was further randomly allocated to one of two versions of the program: (1) an “incentivized” treatment with the pay-for-performance component (treatment A) described above; and (2) an otherwise identical “non-incentivized” treatment without the pay-for- performance incentives (treatment B). This study focuses on the Generasi program. It describes the three waves of evaluation surveys conducted between 2007 and 2010. To the extent possible, findings from the accompanying qualitative study were also incorporated into this report to provide greater understanding of possible causal effects, processes, and villagers’ perspectives. The authors will have two other forthcoming papers discussing in greater detail the issues of incentives, cost-effectiveness, and project operations.3 Box 1. Generasi Program Target Indicators Health Indicators 1. Four prenatal care visits 2. Taking iron tablets during pregnancy 3. Delivery assisted by a trained professional 4. Two postnatal care visits 5. Complete childhood immunizations 6. Adequate monthly weight increases for infants 7. Monthly weighing for children under three and biannually for children under five 8. Vitamin A twice a year for children under five Education Indicators 9. Primary school enrollment of children 6-to-12 years old 10. Minimum attendance rate of 85 percent for primary school-aged children 11. Junior secondary school enrollment of children 13-to-15 years old 12. Minimum attendance rate of 85 percent for junior secondary school-aged children 3 Olken, B.A., J. Onishi, and S. Wong. 2011. “Should Aid Reward Performance? Evidence from a field experiment on health and education in Indonesia” (forthcoming), Also, ““A Community Approach to Achieving Health and Education Outcomes: A Pilot in Indonesia” (forthcoming) Final Impact Evaluation Report 17 Introduction 1.2 The Generasi project This section describes the Generasi project, the Indonesian community block grant program that is the focus of this study. PNPM Generasi—known in full as the National Community Empowerment Program–Healthy and Smart Generation (Program National Pemberdayaan Masyarakat–Generasi Sehat dan Cerdas) — began in mid-2007 in rural areas of five Indonesian provinces selected by the government: West Java, East Java, North Sulawesi, Gorontalo, and Nusa Tenggara Timur.4 In 2007, the project covered 1,650 villages in 129 subdistricts, with a total budget of $20 million. In the project’s second year, which began in mid-2008, the project expanded to cover a total of 2,150 villages in 176 subdistricts, with a total budget of approximately $38 million. The third year, in 2009, was in 164 subdistricts, with a planned expansion to one other province, West Nusa Tenggara in 2010. The Generasi project focuses on 12 indicators of maternal and child health behavior and educational behavior (see Box 1). These indicators are in line with the Ministry of Health priorities and protocols and the government’s constitutional obligation of ensuring nine years of basic education for all Indonesian children. These indicators were chosen by the government of Indonesia to be as similar as possible to the conditions for the individual household conditional cash transfer program being piloted at the same time as Generasi (but in different locations). They are in the same spirit as the conditions used by conditional cash transfer programs in other countries, such as Progresa in Mexico (Levy 2006). These 12 indicators respond to those seeking health and educational services that are within the direct control of villagers— such as the number of children who receive immunization, prenatal and postnatal care, and the number of children enrolled and attending school—rather than long-term outcomes, such as test scores or infant mortality. In Generasi, all participating villages receive a block grant each year to improve education and maternal and child health in their villages. These village block grants ranged from an average of $8,500 in 2007 up to an average of $18,200 in 2009. Block grants are usable for a wide variety of purposes, including, but not limited to, hiring extra midwives for the village, subsidizing the costs of prenatal and postnatal care, providing supplementary feeding, hiring extra teachers, opening a branch school in the village (kelas jauh or satellite classrooms, or sekolah terbuka or formal part-time junior secondary schooling), providing scholarships or school supplies, providing transportation funds for health care or school attendance, improving health or school buildings, or even rehabilitating a road to improve access to health and education facilities during the rainy season. To decide on the allocation of the funds within a village, trained facilitators help each village elect an eleven-member village management team, as well as select local facilitators and volunteers. Through social mapping and in-depth discussion groups, villagers identify problems and bottlenecks in reaching the 12 indicators. Inter-village meetings and consultation workshops with local health and education service providers allow community leaders to obtain information, technical assistance, and support from the local health and education offices as well as to coordinate the use of Generasi funds with other health and education interventions in the area. Following these discussions, the elected management team makes the final Generasi budget allocation. 4 An initial test of the Generasi concept was run in three villages in Gorontalo province from 2006 to 2008. Those villages are not included in the main Generasi project or analysis. 18 Indonesia’s PNPM Generasi Program Introduction In 2007 and 2008, communities used the block grant funds for education (56 percent of expenditure) and health (44 percent) activities (see Figure 1). Communities chose to use most of the funds for “individual goods” such as school materials, equipment and uniforms, and school financial assistance. On the health side, the majority of expenditures were used for supplementary feeding activities and financial assistance for pregnant mothers to use services (e.g., midwife fees, transport, etc.) Figure 1. Village funding allocations, 2007 and 2008 Education Activities 56% of Expenditures Financial Training & incentives for behavior change teachers (4%) communication Incentives for (1%) Training & behavior health workers (4%) 3% %) change communication (3%) In ure Infrastructu %) (8% ies and Facilities ipment (10%) equip Supplementary Financial feeding activities (41%) assistance and Infrastructure (16%) school fees School materials, (31%) equipment, and uniforms (56%) Financial assistance for pregnant mothers to use services (27%) Source: Generasi Project management information system data Performance incentives are a critical (and unique) element of the Generasi approach. The size of a village’s block grant depends in part on its performance on the 12 targeted indicators. The purpose of the performance bonus is to increase the village’s effort at achieving the targeted indicators (Holmstrom 1979), both by encouraging a more effective allocation of Generasi funds and by stimulating village outreach efforts to encourage mothers and children to obtain appropriate health care and increase educational enrollment and attendance. The performance bonus is structured as a relative competition among villages within the same subdistrict (kecamatan). By making the performance bonuses relative to other villages in the subdistrict, the government sought to minimize the impact of unobserved differences in the capabilities of different areas on the performance bonuses (Lazear and Rosen 1981; Mookherjee 1984; Gibbons and Murphy 1990). The fixed allocation to each subdistrict also ensures that the performance bonus system would not result in an unequal geographic distribution of funds.5 5 As discussed by Gibbons and Murphy (1990) and others, one potential pitfall of relative performance incentives is that agents may have an incentive to either sabotage or collude with other agents. With an average of 12 villages per subdistrict, in this case villages face a much greater return from increasing their own performance than from sabotaging that of other villagers. Nevertheless, this possibility remains, and therefore makes the equilibrium implications of the incentives an important empirical question. Final Impact Evaluation Report 19 Introduction The specific rule for allocating Generasi funds to villages within the subdistrict is as follows. The size of overall Generasi allocation for the entire subdistrict is predetermined by the subdistrict’s population and poverty level.6 Within a subdistrict, in year 1 of the project funds are divided among villages in proportion to the number of target beneficiaries in each village (that is, the number of children of varying ages and the expected number of pregnant women). Starting in year 2 of project implementation, 80 percent of the subdistrict’s funds continue to be divided among villages in proportion to the number of target beneficiaries; the remaining 20 percent of the subdistrict’s funds form a performance bonus pool, to be divided among villages based on their performance on the 12 Generasi indicators.7 The performance bonus pool is allocated to villages in proportion to a weighted sum of each village’s performance above a predicted minimum achievement level. Specifically, each village’s share of the performance bonus pool is determined by: ShareOfBonusv = Pv / (Σ Pj) where Pv=Σ[ wi X (yvi - mvi)] In this formula, yvi represents village v’s performance on indicator i, wi represents the weight for indicator i, mvi represents the predicted minimum achievement level for village v and indicator i, and Pv is the total number of bonus “points” earned by village v. The weights for each indicator, wi, are shown in Table 1, and were set by the government to be approximately proportional to the marginal cost of having an additional individual complete that indicator. Generasi uses performance relative to a constant predicted minimum attainment level, rather than improvements over an actual baseline, to avoid the ratchet effect (Weitzman 1980); the minimums, mvi, are determined based on historical national datasets.8 6 In 2007 the average block grant for each subdistrict was $103,000 per subdistrict; in 2008, the average block grant was raised to $178,000 per subdistrict, and for 2009, it was $206,000 per subdistrict. A subdistrict contains roughly between 15,000 and 50,000 individuals and 10 to 20 villages. 7 Starting in year 2, for allocating the non-incentivized portion of the block grant (i.e., 80 percent of the subdistrict allocation in incentivized areas and 100 percent of the subdistrict allocation in non-incentivized areas), the number of target beneficiaries is weighted depending on a village’s access to facilities. This calculation is identical in both incentivized and non-incentivized areas. 8 For each of the 12 Generasi indicators i, the project set the predicted minimum attainment level, mvi, in village v to be equal to 70 percent of the average achievement level for villages with similar levels of access to health and education providers and numbers of beneficiaries. These minimum achievement levels were estimated by combining data on levels of each indicator from the 2004 SUSENAS household survey and 2003 PODES census of villages. For all health indicators except monthly weighing, access to providers was divided into three categories: (1) having a midwife practicing in the village, (2) not having a midwife in the village but having a midwife practicing within 4km from the center of the village, or (3) not having a midwife practicing within 4km of the village center. For middle school, access was divided into three categories: (1) having a middle school located in the village or within 4km of the village center, (2) having a middle school located between 5 and 9km of the village center, or (3) having a middle school located 10km or more from the village center. For monthly weighing and primary school, all villages were assumed to have the same level of access, since weighing of children is always conducted in the village at monthly posyandu meetings and since virtually all villages in Indonesia have a primary school. 20 Indonesia’s PNPM Generasi Program Introduction Table 1. Performance metrics and weights Weight per measured Potential times per Potential points per Performance metric achievement person per year person per year 1. Prenatal care visit 12 4 48 2. Iron tablets (30 pill packet) 7 3 21 3. Childbirth assisted by trained professional 100 1 100 4. Postnatal care visit 25 2 50 5. Immunization 4 12 48 6. Monthly weight increases 4 12 48 7. Monthly weighing 2 12 24 8. Vitamin A pill 10 2 20 9. Primary enrollment 25 1 25 10. Monthly primary attendance >= 85% 2 12 24 11. Middle school enrollment 50 1 50 12. Monthly middle school attendance >= 85% 5 12 60 Source: PNPM Generasi Operational Manual As noted previously, two versions of the Generasi project are being run to separate the impact of the performance bonuses from the overall impact of having additional financial resources available for health and education: the program with performance bonuses described above (referred to as “treatment A”), and an identical program without performance bonuses (referred to as “treatment B”). Treatment B is identical to treatment A except that in treatment B, there is no performance bonus pool; instead, in all years, 100 percent of funds are divided among villages in proportion to the number of target beneficiaries in each village. In all other respects, the two versions of the program are identical. Even the village’s annual points score Pv is also calculated in treatment B areas; the only difference is that in treatment B villages the points are used simply as an end-of-year monitoring and evaluation tool, and have no relationship to the allocation of funds. Within a given subdistrict, all villages participate in the same treatment of the program; that is, either all villages received treatment A, or all villages received treatment B. The Generasi project design builds on the Indonesian government’s existing community-driven development program, known as the National Community Empowerment Program (PNPM), which, along with its predecessor programs (Kecamatan Development Project), have funded over $2 billion in local infrastructure and microcredit programs in some 61,000 Indonesian villages over the past decade. The Generasi project is implemented by the government of Indonesia’s Ministry of Home Affairs, and is funded through government of Indonesia resources and in part with loans from the World Bank and grants from several bilateral donors. Technical assistance and evaluations have been supported by a multi- donor trust fund with contributions from the World Bank, Netherlands Embassy, Australia, United Kingdom, and the Danish Embassy, and the World-Bank-managed Spanish Impact Evaluation Fund. Final Impact Evaluation Report 21 Introduction 1.3 Experimental Design In order to evaluate the overall impact of Generasi, as well as to separately identify the impact of Generasi’s performance incentives, Generasi locations were selected by lottery to form a randomized, controlled field experiment. The use of randomized evaluation techniques is considered the gold standard for impact evaluation of clinical and public health interventions (Gordis 2004), as well as development programs more generally (Duflo, Glennerster, and Kremer 2007). It has formed the basis of a number of high- profile social policy experiments in the United States (see Newhouse 1993; Kling, Liebman, and Katz 2007) and internationally (see Gertler 2004; Miguel and Kremer 2004; Schultz 2004; Skoufias 2005). The Generasi randomization was conducted at the subdistrict (kecamatan) level, so that all villages within the subdistrict either received the same treatment of Generasi (treatment A or treatment B) or were in the control group. Randomizing at the subdistrict level is important since many health and education services, such as community health centers (puskesmas) and junior secondary schools, provide services to multiple villages within a subdistrict. Increased demand for services from one village within a subdistrict could potentially therefore crowd out the services provided to other villages within the same subdistrict; alternatively, an effort by one village to improve service provision at the community health center could also benefit other villages in the same subdistrict. By randomizing at the subdistrict level, so that all villages in the subdistrict receive the same treatment status, the evaluation design ensures that we capture the total net effect of the program, since any within-subdistrict spillovers would also be captured in other treatment villages.9 This type of cluster-randomized design is common in program evaluations where there might be local spillovers from the treatment (Miguel and Kremer 2004; Olken 2007). The Generasi locations were selected through the following procedure. First, 300 target subdistricts were identified, targeting poor, rural areas that had an existing community-driven development infrastructure.10,11 Each subdistrict was then randomly assigned by computer into one of three equal-sized groups: treatment A, incentivized (100 subdistricts); treatment B, non-incentivized (100 subdistricts); or control (100 subdistricts). Within a subdistrict, all villages received the same treatment. The randomization was stratified by district (kabupaten), to ensure a balanced randomization across the 20 different districts in the study. The tests for balance confirm that the three groups of subdistricts appear similar on pre-period characteristics (World Bank 2008). Note that a total of 36 out of the 300 subdistricts should not have 9 Spillovers to other subdistricts are much less likely to be a problem, since the health service providers (Subdistrict Health Centers and midwifes), primary schools, and junior secondary schools that are the focus of this survey primarily provide services within a single subdistrict. Nevertheless, by using GIS information on the location of service providers, we will be able to test empirically for the presence of these cross-subdistrict spillovers. 10 To identify the 300 target subdistricts, we began by eliminating the wealthiest 20 percent of districts (kabupaten) within the five target provinces identified by the government, determined by the district’s poverty rate, malnutrition rate, and junior secondary school transition rate. Districts where the PNPM program was not scheduled to operate in 2007 were also ineligible. Twenty districts were randomly selected from the remaining eligible districts, stratified by island group. Within the 20 selected districts, subdistricts were eligible for Generasi if they had previously received the PNPM program or were considered less than 67 percent urban by the Central Statistics Office. 11 Since Generasi is implemented through the national PNPM program, it could only be implemented in districts that were already included in the PNPM program. Prior experience with PNPM at the subdistrict level also simplified Generasi implementation, since the relevant legal structures for disbursing Generasi funds had already been established in these locations. 22 Indonesia’s PNPM Generasi Program Introduction been included in the randomization, as they were ineligible for Generasi because they had been selected (prior to the randomization) to receive other programs or had had prior implementation problems with previous PNPM programs. Since the eligibility decision was made on the basis of lists determined prior to the randomization, and since we obtained those lists for treatment and control areas, we excluded ineligible subdistricts in both treatment and control groups from our main analysis.12 The Generasi program was phased in over two years. In phasing in the program in the first year (2007), the government for budgetary reasons prioritized those locations that had previously participated in the PNPM rural infrastructure program (denoted group P), since those locations already had the legal infrastructure for distributing PNPM funds and it was easier to re-budget other monies to fund Generasi in those areas. After all group P subdistricts randomized to receive the program had been funded, the government held another lottery to select which group NP subdistricts would begin receiving the program in 2007 and which would begin in 2008.13 By year two of the program (2008), 95 percent of eligible subdistricts—172 out of the 181 eligible subdistricts randomized to receive Generasi—were receiving the program. Of the remaining nine subdistricts, seven received the regular PNPM program instead of Generasi and two received Generasi in year one only.14 Since the randomization results were followed 99 percent of the time in Wave I and 95 percent of the time in Wave II, in the analysis below, we use the original randomization results all 264 eligible subdistricts as the basis of the analysis, and interpret the results as intent-to-treat estimates (Imbens and Angrist, 1994). An important consideration for the analysis is the potential for differential provision of other programs in the pure control groups. The main potential avenue through which this might occur is other PNPM programs. Specifically, to ensure a fair allocation of funds, the Ministry of Home Affairs decided that no subdistrict would receive both the Generasi program and other PNPM programs, which typically fund local infrastructure (roads, bridges, etc.) and microcredit. In 2007, 17 (out of 83) eligible control subdistricts received other PNPM programs, as did two treatment subdistricts in the non-priority (NP) areas; in 2008, 31 (out of 83) eligible control subdistricts received other PNPM programs, as did the seven eligible subdistricts that should have been receiving Generasi in 2008 but received regular PNPM-rural instead. Since regular PNPM programs tend to focus on basic infrastructure, not health and education, it is unlikely that the differential provision of other PNPM programs in control areas will have substantial 12 The determination that these subdisticts would be ineligible had been made prior to the randomization, but was not communicated to the study team, which is why they were included in the randomization. Subdistricts were deemed ineligible if they had been allocated to receive the urban poverty program (UPP), conflict area poverty program (SPADA), or if they had had a previous problem with PNPM implementation. We subsequently obtained the pre-randomization lists used to make this determination, and use these pre-randomization lists to restrict our sample (in both treatment and control areas) to those subdistricts that would actually be eligible for the program. Nevertheless, data collection surveys were conducted in all 300 subdistricts that were initially included in the randomization, regardless of the final eligibility, so as a robustness check we can alternatively estimate intent-to-treat effects using the full 300 subdistricts from the original randomization. 13 Specifically, in 2007 all 105 eligible group P subdistricts were funded. In group NP, in 2007 Generasi was funded in 22 eligible subdistricts. Of these 22 subdistricts, 21 were chosen randomly by computer, stratified by province, in a second lottery among Group NP locations. Group P status was determined prior to randomization. All but 7 of the remaining NP subdistricts were added in 2008. 14 We do not know why these 7 subdistricts received regular PNPM rather than Generasi. We therefore include them in the treatment group as if they had received the program, and interpret the resulting estimates as intent-to-treat estimates. Likewise, we include the 2 subdistricts that received the program in year 1 but not in year 2 in the treatment group throughout. Final Impact Evaluation Report 23 Introduction impacts on the Generasi evaluation results. Nevertheless, in interpreting the results, it is important to recognize that some portion of the eligible “pure control” subdistricts received PNPM. 1.4 Survey Design and Implementation The main data for the impact analysis is from a set of surveys of households, village officials, health service providers, and school officials. A detailed list of the contents of each survey module, as well as the sample size for each module, can be found in Table 2. Three waves of the survey were planned as part of the evaluation series. Wave I, the baseline round, was conducted from June to August 2007 prior to Generasi implementation. Wave II, the first follow- up survey round, was conducted from October to December 2008. Wave III, a longer term follow-up survey round, was conducted from October 2009 to January 2010. These surveys were designed by the World Bank and the government of Indonesia and were conducted by the Center for Population and Policy Studies of the University of Gadjah Mada, Yogyakarta, Indonesia. The final evaluation is based on data collected through the Wave I, Wave II, and Wave III surveys, which were funded by the World Bank (through the Decentralization Support Facility, PNPM multidonor trust fund (PSF), and the WB- managed Spanish Impact Evaluation Fund). The sample for the surveys covers each of the 300 subdistricts that were included in the original Generasi randomization. In each subdistrict, eight villages were randomly selected (unless the subdistrict had fewer than eight villages, in which case all were selected). This resulted in a total of 2,313 villages sampled in each of the three survey waves. The sampling design for the household component of the Generasi surveys was chosen to ensure adequate coverage in the key Generasi demographic groups: mothers who recently were pregnant or gave birth, children under age 3, and children of school age. Within each village, one hamlet (dusun) was randomly selected, and a list of all households was obtained from the head of the hamlet. Five households were randomly sampled from that list to be interviewed. These households were stratified so that two selected households had at least one child under age 2, two selected households had a child under age 15 but no children under age 2, and one household had no children under age 15. For some of the analysis (e.g., for examining how the incentives affect the differential targeting of Generasi benefits and increments in service provision), it is useful to have baseline and follow-up characteristics for the same individuals. Therefore, in the follow-up surveys, in half of the randomly selected villages (four villages out of the eight villages sampled in every subdistrict), the same households sampled in Wave I were contacted again in subsequent waves to form an individual level panel. Teams tracked and re-interviewed migrated or split households who provided information for any of the married women or children modules, as long as they were within the same subdistrict. In panel areas, 95 percent of target households were able to be re-interviewed in Wave 2 and 98 percent of target households were able to be re-interviewed in Wave 3. In the other half of villages, a new cross-section of households was drawn in each survey wave. The combination of panel households and non-panel households allows us to investigate heterogeneous treatment effects based on pre-period income levels and other characteristics, while at the same time ensuring that sufficient respondents with recent births and young children are enrolled in the survey sample in every round. 24 Indonesia’s PNPM Generasi Program Introduction Figure 2. Timeline of project and surveys Jun-Aug Oct-Dec Oct-Dec 2007 2008 2009 Tratment Fund distribution & monitoring Fund distribution & monitoring Group A Conditional Conditional Villages bonuses bonuses Wave Wave Wave Tratment I Fund distribution & monitoring II Fund distribution & monitoring III Group B Survey Survey Survey Villages n=35k n=35k n=35k hh=12k hh=12k hh=12k Comparison Villages At baseline, groups are Interim Final statistically identical Results Results Source: authors Health facilities and schools were also contacted again to form a panel. For midwives, a randomly selected 75 percent of the midwife sample was re-contacted to form a panel, and 25 percent of the midwives were newly sampled in each wave to ensure the sample captures potential in-migration of midwives in response to Generasi. Data from these surveys were supplemented with detailed administrative data from the Generasi project’s internal management information system. This included detailed budget allocations for the block grants, performance data on the 12 Generasi indicators, and data on participation levels in Generasi village meetings. Additionally, the Indonesian research organization, SMERU, conducted an accompanying qualitative study in 12 villages to probe deeper into the “whys and hows” of community and service provider processes and motivations. This qualitative work allowed the study team to explore in greater depth some of the decision-making and implementation issues behind the quantitative results. Final Impact Evaluation Report 25 Introduction Table 2. Questionnaire modules and sample size Module Contents Sample Size Panel/Non- (Wave III) Panel (Waves II/III) Household core Household roster, deaths in previous 12 months, 12,306 (Respondent: female migration, water/sanitation, receipt of government household head or poverty programs, participation in non-formal education, spouse of a male consumption, assets, economic shocks, health insurance, household head) morbidity, outpatient care use, social capital, knowledge and participation in PNPM/KDP activities Married women Fertility history, use of health services during pregnancy, 11,140 age 16–49 inspection of Generasi coupons (Wave II), family planning, health and education knowledge Children age 6–15 School enrollment, attendance, grade repetition, cost of 9,779 (Respondent: schooling, scholarships, child labor 50% panel, 50% mother of the child) non-panel Children age < 3 Growth monitoring (posyandu), immunization records, 6,708 (Respondent: inspection of the Generasi coupons (Wave II), motor mother of the child) development (Wave III), breastfeeding and nutritional intake, weight measurement, height measurement (Waves I & III) Home-based tests Test of math and reading skills administered at home 7,687 for math and (Respondent: (separate test for ages 6–12 and ages 13–15) reading tests, children age 6–15) (Waves I & III) 7,336 for anthro- pometric tests Village Demography of the village, hamlet information, access 2,315 100% panel characteristics to health services and schools, economic shocks, access to (Respondent: Village media, community participation, daily laborer wage rate, Head) development projects in the village (Waves II & III) Community health Head of facility background, coverage area, budget, 300 100% panel center (Puskesmas) staff roster, time allocation of head doctor and midwife coordinator, service hours, services provided, fee schedule, number of patients per service during the previous month, medical and vaccine stock, data on village health post, participation in Generasi (Waves II & III), direct observation regarding cleanliness Village midwives Personal background, location of duty, condition of 1,177 75% panel, 25% facility, time allocation, income, services provided, fee non-panel schedule (public and private), experiences during past three deliveries, number of patients seen per service during the previous month, equipment and tools, medical supplies and stock, village health post management, participation in Generasi (Waves II & III), structure of subsidies received Principal background, principal time allocation, teacher 1,197 50% panel 50% Primary school roster, school facilities, teaching hours, enrollment records, non-panel (Waves II & III) attendance records, official test scores, scholarships, fees, budget, participation in Generasi (Waves II&III), direct observation of classrooms, including random check on classroom attendance 26 Indonesia’s PNPM Generasi Program Introduction Module Contents Sample Size Panel/Non- (Wave III) Panel (Waves II/III) Junior secondary Same questionnaire as for primary school 867 66% panel, 33% school non-panel Village health post Respondent characteristics, health post characteristics, 50% panel 50% cadre (Waves II & service providers, cadre roster, tools and equipment, 2,397 non-panel III) participation in Generasi Sources: Survey questionnaires, UGM reports for Waves I, II and III surveys. Over 45,000 household members, village heads, and school and health facility staff were respondents for the Generasi final Wave III survey. Final Impact Evaluation Report 27 Evaluation Methodology Evaluation Methodology 2.1 Regression Specifications ince the Generasi program was designed as a randomized experiment, the evaluation is econometrically S straightforward: essentially, we compare outcomes in those subdistricts randomized to be treatments with those subdistricts randomized to be control areas, controlling for the level of the outcome at baseline. In implementing our analysis, we restrict attention to the 264 “eligible” subdistricts, as discussed in Section 1.3 above, and use the randomization results combined with the government’s prioritization rule to construct our treatment variables. Specifically, analyzing Wave II data (corresponding to the first treatment year), we define the GENERASI variable to be a dummy that takes value 1 if the subdistrict was randomized to receive GENERASI and either (a) it was in the priority area (group P), or (b) was in the non-priority area and selected in the additional lottery to receive the program in 2007. In analyzing Wave III data, we define the GENERASI variable to be a dummy that takes value 1 if the subdistrict was randomized to receive Generasi. We define the GENERASI_INCENTIVES variable to be a dummy that takes value 1 if the GENERASI variable is 1 and if the subdistrict was randomized to be in the incentivized version of the program. GENERASI_INCENTIVES thus captures the additional effect of the incentives above and beyond the main effect of having the program, and is the key variable of interest in the paper. Note that by defining the variables in this way, we are exploiting only the variation in program exposure due to the lottery. These variables capture the intent-to-treat effect of the program, and since the lottery results were very closely followed—they predict true program implementation in 99 percent of subdistricts in 2007 and 95 percent of subdistricts in 2008—they will be very close to the true effect of the treatment on the treated (Imbens and Angrist, 1994). In running the regressions, we take advantage of the baseline data by controlling for the average level of the outcome variable in the subdistrict in the baseline survey. Since we also have individual-specific panel data for half our sample, we include the pre-period value for those who have it, as well as a dummy variable that corresponds to having non-missing pre-period values. Since households came from one of three different samples (those with a child under age 2, those with a child age 2–15 but not in the first group, and all others), we include dummies for those three sample types, interacted with whether a household came from a panel or non-panel village. Finally, since many of the indicators for children vary naturally as the child ages, for all child-level variables we include age dummies. To examine the overall impact of Generasi treatment, for each indicator of interest, we estimate the following regressions on the 264 subdistricts that remain after we drop the list of ineligible subdistricts: Wave II data: ypdsi2 = àd + â1 GENERASI đ2 + ã1ypdsi1 + ã21{ypdsi1≠missing} + ã3 yđ1+ SAMPLEpdsi + àp x Ps + åpdsi Wave III data: ypdsi3 = àd + â1 GENERASI đ 3+ ã1ypdsi1 + ã21{ypdsi1≠missing} + ã3 yđ1+ SAMPLEpdsi + àp x Ps + åpdsi Wave II and III combined average effect: ypdsi = àd + â1 GENERASI dst + ã1t ypdsi1 + ã2t1{ypdsi1≠missing} + ã3t yđ1+ ðt SAMPLEpdsi + àp x Ps + åpdsi 30 Indonesia’s PNPM Generasi Program Evaluation Methodology where p is a person, d is a district, s is a subdistrict, t is the survey wave (1 = baseline, 2 = interim survey, 3 = final survey), y pdsit is the outcome in Wave t, á d is a district fixed effect, y pdsi1 is the baseline value for individual i (assuming that this is a panel household, and 0 if it is not a panel household), 1ypdsi1 mis sin g  is s 1 is the average baseline value for the subdistrict, SAMPLE are a dummy for being a panel household, y d dummies for how the household was sampled interacted with being a panel or cross-section household, and á p  Ps are province-specific dummies for being in the previous-KDP sample. Standard errors are clustered at the subdistrict level. Note that in the final equation for computing the average effect over Wave II and Wave III, all control variables (e.g., district FE, sample controls, baseline values, etc) are to be fully interacted with wave dummies, to capture the fact that there may be differential trends in different parts of the country. To examine the additional impact of the incentives (treatment A compared to treatment B), we estimate the same regression, but with an additional variable that captures the additional effect of incentives: Wave II data: ypdsi2 = àd + â1 GENERASI pds2 + â1 GENERASI_INCENTIVESpds2 + ã1ypdsi1 + ã21{ypdsi1≠missing} + ã3 ypds1+ SAMPLEpdsi + àp x Ps + åpdsi Wave III data:\ ypdsi3 = àd + â1 GENERASI pds3 + â1 GENERASI_INCENTIVESpds3 + ã1ypdsi1 + ã21{ypdsi1≠missing} + ã3 ypds1+ SAMPLEpdsi + àp x Ps + åpdsi Wave II and III combined average effect: ypdsit = àd + â1 GENERASI pdst + â1 GENERASI_INCENTIVESpdst + ã1t ypdsi1 + ã2t1{ypdsi1≠missing} + ã3t ypds1+ ðt SAMPLEpdsi + àp x Ps + åpdsit Using the estimates from this regression we can also calculate the total impact of the treatment A program by adding the coefficients on GENERASI_INCENTIVES and GENERASI. We also examine a wide variety of additional specifications as robustness tests; these specifications are discussed in more detail in Section 3. Since we have a large number of indicators, in order to calculate joint significance we will calculate average standardized effects for each family of indicators, following Kling, Liebman, and Katz (2007). Specifically, for each indicator i, define σ2i to be the variance of i. We then estimate (1) for each indicator, but run the regressions jointly, clustering the standard errors by subdistrict to allow for arbitrary correlation among the errors within subdistricts, both between and across indicators. We then define the average standardized effect as. âi Σ i ói Final Impact Evaluation Report 31 Evaluation Methodology Note that all of the analysis presented here (regression specifications, outcome variables, and aggregate effects) follows an analysis plan that was finalized in April 2009 for the Wave II data (before we examined any of the Wave II data) and in January 22, 2010 (before we examined any of the Wave III data).15 This hypothesis document was registered with the Abdul Latif Jameel Poverty Action Lab at MIT and is available on request. 2.2 Balance Tests This section examines the balance of key child health and education indicators using data from the interviews of mothers in the baseline household surveys and the same estimation procedure shown in equation (1). We examine all of the 12 major indicators that are the focus of the program (these indicators are discussed in more detail in Section 3 below). The results are shown in Table 4. Column (1) shows the mean of each variable in the control group. Column (2) shows the “Generasi Effect”, i.e., the difference between Generasi project areas and controls. Since this is a balance check, one would expect no significant differences between treatment and controls. Column (3) and column (4) show the coefficients showing the incentivized and non-incentivized area, respectively, with column (5) showing the additional effect of the incentives (i.e., the difference between Treatment A and Treatment B). Column (6) shows the total number of observations. Looking across columns (2) through (5), we find that of the 48 coefficients estimated, 5 are statistically significant at the 10 percent level or higher, which is precisely what would be predicted by random chance. Similarly, 3 of 48 coefficients are statistically significant at the 5 percent level or higher, which is also what one would predict based on random chance. These results confirm that the randomization was indeed carried out properly and that the treatment and control groups are balanced. The final rows of Table 4 consider the average standardized effects, computed via equation (3). We report average standardized effects for all 12 of the main indicators, and then separately report average standardized effects for the 8 health indicators and 4 education indicators. One of the 16 coefficients is statistically significant at the 10 percent level, once again consistent with what one would expect based on random chance. This confirms that the sample is indeed balanced. Table 5 reports the same baseline regressions for the long-term health indicators, neonatal mortality, infant mortality, acute disease, malnourishment, and severe malnourishment. Two of the 20 coefficients are statistically significant at the 10 percent level, once again consistent with a balanced sample and random chance. None of the average standardized effects show any differences. Thus, along a wide variety of measures, the sample appears balanced at baseline. 15 The only Wave II variables we examine that were not in the original Wave II analysis plan are some additional variables related to exploring the negative education result: these variables are separately marked in the table as “Additional Education Indicators.” 32 Indonesia’s PNPM Generasi Program Evaluation Methodology Final Impact Evaluation Report 33 Main Results Main Results n this section, we will describe the main results of Generasi during the past two years. We will report I the results in terms of what types of support did beneficiaries actually receive, and the impact of the program on the 12 main target indicators, long-term final outcomes, and non-targeted indicators. The tables are organized as follows: each row reports the results for a different variable. Column (1) of Table 6 (and all subsequent tables) shows the mean level of the variable in the baseline survey (2007). Column (2) shows the mean level of the variable in the control group in the Wave II (2008) survey and column (3) reports the coefficient on the GENERASI variable from estimating equation (1), and is interpretable as the average impact of the Generasi on the variable for Wave II.16 Columns (4) and (5) show the control means and average impact of Generasi respectively for Wave III (2009). Column (6) shows the Wave II and III combined average treatment effects. Average standardized effects overall and individually for health and education, computed using equation (3), are shown at the bottom of the table. For the tables showing incentivized (Versi A) versus non-incentivized (Versi B), the first three columns show the results of Wave II. Column (3) indicates the additional effects from the incentivized version. Wave III results are illustrated in the same way for Columns (4), (5), and (6). The final three columns show the average effects across the two waves of surveys. For regional breakdowns, we show results for Java (West and East Java), the additional Generasi effects in NTT, and the additional Generasi effects in Sulawesi/Gorontalo as compared with Java. The results are shown across Wave II, Wave III, and average for the two waves. 3.1 Direct Benefits of Generasi Funds The main idea behind Generasi is for communities and local service providers to come together and propose activities to best attain the 12 main health and education indicators. Communities must be able to design appropriate village-level activities that will address common and shared hurdles to accessing these health and education services. This section explores the types and quantities of direct benefits received by children under three years of age, school-aged children, and pregnant mothers. Both Wave II and Wave III surveys found that assistance was indeed reaching beneficiaries in Generasi areas. In general, Generasi significantly increased the provision of scholarships, school uniforms, school supplies, transport subsidies to attend school, intensive supplementary feeding at school, and other school support. For health, beneficiaries received assistance for supplementary feeding at village health posts, antenatal/postnatal care, and subsidies for delivery assistance (Table 6). These activities are consistent with the types of assistance the project was reporting through its internal reporting system. In terms of incentivized versus non-incentivized areas, there was not much difference overall in spending and activity patterns. Incentivized areas experienced some more education activities (in the form of scholarships and transport subsidies) (Table 7). 16 As described above, all regressions include district fixed effects, Group P interacted with province fixed effects, dummy variables for how the household was sampled, and (for child indicators) age dummies. The main regressions also include the average baseline value of the variable in the subdistrict and, for panel respondents, that individual’s baseline value. 36 Indonesia’s PNPM Generasi Program Main Results Turning to regional breakdowns, Table 8 shows the Generasi effect for Java (columns 2, 6, and 9), and then the incremental additional Generasi effect for NTT (columns 3, 7, and 10) and Sulawesi (columns 4, 8, and 11). To find the total Generasi effect for NTT or Sulawesi, we add the effect for Java and the incremental effect. Average direct benefits for education were substantially higher in NTT and Sulawesi than Java in both waves, particularly for uniforms, other school supplies, transport subsidies (Sulawesi) and other school support (NTT), and supplemental feeding at school (NTT). The higher impacts may have reflected the fact that the Generasi subsidy was higher on a per-capita basis off-Java than on-Java, so that the block grant could provide goods to a greater percentage of households. For health, the differences were more in composition, with more supplementary feeding activities at village health posts in NTT than in Java, and fewer subsidies for childbirth in NTT than in Java. This may reflect the greater need for nutrition supplements outside of Java, or perhaps difficulties in midwife availability in NTT (Table 8). There were no major differences in the impact of incentives by province. 3.2 Program Impact on Main Targeted Indicators This section describes the impact on the 12 main indicators during the two years of project implementation. To examine this, for each the 12 indicators provided to the villagers for improvement (shown in Box 1), we examine the impact of Generasi on an analogous indicator from the household survey. We first examine the average standardized effect, which, as discussed above, is a statistically efficient way of pooling all of the effects to maximize our statistical power given that we don’t necessarily have the power to detect effects on indicators one-by-one. It is important to note that while Generasi may have affected the average of the 12 indicators, this does not mean that it affected all of them individually. Conversely, given that we have weak statistical power on some of our indicators, it is possible that Generasi is affecting more than just the indicators that are individually statistically significant. We find that Generasi improved both health and education indicators on average over the two-year implementation period, with particularly significant results in Wave III. Specifically, the average standardized effects shown in Table 9 indicate that the Generasi program succeeded in improving target indicators, particularly in Wave III. On average, in Wave III, the average Generasi impact on the 12 indicators was 0.041 standard deviations. The effect was comparably sized in health and education, at 0.038 standard deviations for health and 0.046 standard deviations for education. For education, the positive impacts detected in Wave III represent a reversal from Wave II, which indicated generally negative impacts on all education indicators. For health indicators, the strongest impact was for participation in growth monitoring (weight checks), with children attending on average 0.15 more growth monitoring sessions over the previous three months, a 6.8 percent increase over the average level in control areas. Over the two-year implementation period, pregnant women on average received 0.08 more sachets of iron tablets (containing 30 tablets in one sachet) during pregnancy, a 4.7 percent increase over the control level. In the second year of the program (Wave III), there was also a substantial reduction in malnutrition (with a 2.2 percentage point reduction in the share of children more than 2 standard deviations below normal on weight-for-age), representing a 9.6 percent reduction over the average level in control areas. The improvements in health indicators in Final Impact Evaluation Report 37 Main Results both waves were predominantly found in treatment A (subdistricts with the incentivized design) areas, suggesting that incentives work well for health indicators (Table 10). On net, between 50-75% of the total impact of the block grant program on health indicators can be attributed to the performance incentives. 17 Impacts on both prenatal visits and immunizations were higher in incentivized Generasi areas than in non-incentivized Generasi areas. Generasi had positive impacts on education indicators in Wave III, an encouraging reversal from Wave II where the impacts were negative (Table 9). In Wave III, Generasi increased school participation of 7–12 year olds by 0.8 percentage points. Since the control group had a participation rate of 98.5 percent, this implies that Generasi resulted in almost all 7–12 year old children being enrolled in school. The increase in school participation in Wave III came predominantly in NTT, where 7–12 year old enrollments increased by 3.8 percentage points, a 4 percent increase over control areas. Interestingly, community incentives did not have clear impacts on education indicators as no significant difference was detected between treatment A and B (Table 10). NTT also saw a 10.1 percentage point increase in the percentage of age 13–15 year olds enrolled in junior secondary school, a 19 percent increase over control areas, although this was not statistically significant.. Since the overall participation rate for 13–15 year olds did not change, this implies that Generasi’s impact for this age group was increasing enrollments in junior secondary instead of primary school for this group. Consistent with this, the junior secondary gross enrollment rate (the share of children enrolled in jr. secondary divided by the number of children of jr. secondary school age) rose by 18 percentage points—a 29 percent increase compared to control areas—in NTT in Wave III. 3.3 Program Impact on Long-Term Final Outcomes This section describes the project’s impact on long-term health and education outcomes. On health, neonatal and infant mortality and morbidity of childhood illnesses (diarrhea, acute respiratory infection, and malnutrition) were measured. Measurements of malnutrition included underweight (defined as <-2SD weight-for-age), severe underweight (defined as <-3SD weight-for-age), stunting (defined as <-2SD height-for-age), severe stunting (defined as <-3SD height-for-age), wasting (defined as <-2SD weight-for-height), and severe wasting (defined as <-3SD weight-for-height) all for children under three. The final outcome for education was measured by multi-grade home-based tests in math and Indonesian administered at baseline and Wave III. The tests were drawn from the standard question database of the Indonesian Ministry of Education. On the health side, the project had large impacts on reducing neonatal and infant mortality in Wave II, largely found in Sulawesi; however, these impacts did not persist in Wave III. Specifically, in Wave II, Generasi led to six neonatal deaths avoided per 1,000 live births in the previous two years and nine infant deaths avoided per 1,000 live births in the previous one year (Table 12). However, these impacts had disappeared in Wave III. For all age groups, no statistically significant impact on mortality was detected 17 The impact of incentives is discussed in more detail in a separate paper by the authors, “Should Aid Reward Performance? Evidence from a field experiment on health and education in Indonesia” (forthcoming) 38 Indonesia’s PNPM Generasi Program Main Results for Wave III. Compared to Wave II, the mean mortality in control areas had considerably reduced, possibly annulling the reduction in mortality in the treatment areas. The large reductions in mortality attributable to the project in Wave II were mainly detected in Sulawesi (Table 14), which were not sustained in Wave III. However, in Wave III Generasi did lead to a substantial reduction in malnutrition (weight-for-age), with a 2.2 percentage-point reduction overall. (Table 12) This represents a 10 percent decline in malnutrition rates compared to control kecamatans. When broken down by province, NTT saw large and statistically significant reductions in underweight (both underweight and severe underweight). In Wave III, in NTT province, underweight rates were reduced by 8.8 percentage points, a 20 percent decline compared to control areas; severe underweight rates were reduced by 5.5 percentage points, a 33 percent decline; and severe stunting was reduced by 6.6 percentage points, a 21 percent decline compared to control areas. The reduction in stunting suggests that in NTT, Generasi was able to address long-term childhood malnutrition. In Java however, surprisingly stunting worsened in Generasi areas relative to control areas, though some of that worsening was due to a reduction in stunting in the control areas. Severe stunting also worsened significantly in Java, but that was not driven by reduced severe stunting in the control areas (Table 14). There is no obvious explanation for these Java findings and they need to be explored further. The project does not appear to have had an impact on learning of school-aged children, measured by math or Indonesian tests administered separately for children in the 7–12 and 13–15 age groups. This finding was consistent in all models with different interaction terms. Given the nature of the multi-grade tests, the tests showed that children of higher grades on average scored higher than children of lower grades, implying that this multi-grade test was successful in assessing the levels of learning achievements by grade. However, the fact that none of the tests could detect the project’s impact on the age-adjusted standardized test scores suggests either that the project had no impact on improving children’s learning, or that tests were not sensitive enough to capture the changes in children’s learning due to the project. Given that the program only affected school participation in Wave III, and that enrollment increased by only a few percentage points, it is not surprising that impacts were not found on test scores at this stage. 3.4 Impact on Non-Targeted Indicators As with all incentive programs, an important question in this context is the problem of multi-tasking (Holmstom and Milgrom 1991). While Generasi encouraged communities to focus on 12 targeted indicators for maternal/child health and primary and junior secondary education, there are a wide variety of other aspects of health and education not covered by the indicators. To the extent that Generasi supports general community mobilization on health and education and increased service provision, one could find positive spillovers on non-targeted indicators. However, it is also possible that communities diverted attention from non-targeted indicators toward targeted indicators, resulting in negative spillovers. Which dominates is ultimately an empirical question. We explore spillovers on a wide range of non-targeted health and education indicators. In general, spillovers were positive for health throughout the two years of implementation. For education, we found negative effects for education in Wave II (consistent with the impact on the main education indicators in Wave II discussed above), while none was found for education in Wave III (Table 15). Final Impact Evaluation Report 39 Main Results Specifically, for health, on average across all 11 health indicators we examined, we found positive impacts on non-targeted indicators, with an average improvement of 0.0362 standard deviations (Table 15). Measured in terms of average standardized effects, this is almost identical in magnitude to the 0.0357 average impact on the 8 targeted health indicators shown in Table 9. For example, statistically significant improvements were found in non-targeted health indicators such as increased facility-based deliveries rather than deliveries at home (the program incentivized deliveries by midwives, but these often take place at home instead of at a facility), the quality of services at the village health post measured by the share of standardized services performed (the program incentivized quantity only, not quality), and improved parental knowledge in parenting practices. The fact that these measures of health service quality increased suggests that the quality and quantity of health services appear to be complements in this context, rather than substitutes. This is consistent with a model where the key driver is facilitating patients’ access to trained providers, rather than encouraging a given set of providers to see more patients in a given amount of time. In education, spillovers were negative in Wave II (-0.082 standard deviations, significant) and zero in Wave III (0.004, not significant), consistent with the main effects of the program in Table 9. In Wave II, the effects are driven by school-aged children appearing to have increased paid work and household work (respectively by 11 minutes and 40 minutes, Table 15), consistent with the lower enrollment rates for these children shown in Wave II. These negative effects on work were not found in Wave III (column 5). In sum, Generasi had a statistically significant positive impact on average on the 12 indicators it was designed to address. The strongest improvements among the health indicators were in the frequency of weight checks for young children and the number of iron sachets pregnant mothers received through antenatal care visits. The main long-term impact was a decrease in child malnutrition measured by weight-for-age under three, which was reduced by the second year by 2.2 percentage points, a 9.6 percent reduction from the control groups. Although reductions in infant and child mortality were observed in the interim evaluation, the same levels of reduction in mortality were not sustained in the final evaluation. Education indicators also saw improvements in the final evaluation, reversing the zero or negative impact found during the interim evaluation. The improvement in education indicators was most notable in the increased school participation rate among the primary school-age group. There is, however, no evidence that the program improves test scores as yet. 40 Indonesia’s PNPM Generasi Program Main Results Final Impact Evaluation Report 41 Where Were Generasi’s Effects Largest? Where Were Generasi’s Effects Largest? 4.1 Heterogeneity by Area Characteristics his section explores the types of environments in which Generasi is most effective. We explore T treatment effect heterogeneity along three dimensions—(1) baseline levels of the target indicators, (2) baseline per-capita expenditure levels, and (3) baseline levels of access to providers—all of which were pre- specified in the analysis plan before we observed any of the data from the program. This section explores each of these dimensions of treatment effect heterogeneity in turn. Baseline Level of Target Indicators An important question is whether Generasi works better or worse in the areas where there is the lowest level of performance on the baseline indicators. On the one hand, one might expect Generasi to work best in these areas, since there is the most room to improve. On the other hand, one might expect the program to perform worse in these areas, if the low level of baseline service provision is indicative of structural problems that make achieving the indicators difficult. To examine this question, we re-estimate the impact of Generasi, including interaction terms. Specifically, for each indicator, we include an interaction term, interacting the Generasi dummies with the subdistrict- level average of that indicator at baseline.18 Table 16 shows the results for the main targeted indicators, with the interaction coefficients shown in columns 4, 8, and 11. A negative coefficient on the interaction implies a larger impact in subdistricts with lower baseline values. To aid the interpretation of these results, we also compute the net Generasi impact at the 10th percentile of the baseline distribution for the indicator (e.g., those with very low health/education status); this is shown in columns 5, 9, and 12 and can be compared to the mean impact of Generasi shown in Table 9 to gauge the magnitude of the treatment effect heterogeneity. In general, Table 16 shows that Generasi was more effective in subdistricts with lower baseline for both health and education indicators. Examining the average standardized effects, in Wave III, for example, the average standardized effect at the 10th percentile at baseline was 0.0771 standard deviations overall. By comparison, the comparable average Generasi effect shown in Table 9 was 0.0407, so the program was about twice as effective in locations with the 10th percentile of baseline indicators as it was on average. The treatment effect heterogeneity is greater for education than for health: in Wave III, Generasi’s impact on education was 2.4 times greater at the 10th percentile than at the mean (0.109 standard deivations compared with 0.0463), whereas for health Generasi’s impact was only 1.6 times greater at the 10th percentile than at the mean (0.0609 standard deviations compared with 0.0380). Similar treatment effect heterogeneity is also seen in Wave II as well, although given that the program only improved health in Wave II, the treatment effects are only found for health as well. Looking at the indicators one by one, several have particularly noticeable heterogeneity in Wave III. In particular, at the 10th percentile Generasi reduced malnourishment by 6.2 percentage points (2.8 times the average effect) and increased school participation among children in the 7–12 age group by 1.9 percentage points (2.3 times the average affect). 18 Note that all models discussed above include the subdistrict-level average of the indicator at baseline as a control; the difference here is that now we also add an interaction with the Generasi dummy to examine treatment effect heterogeneity. 44 Indonesia’s PNPM Generasi Program Where Were Generasi’s Effects Largest? Table 17 examines the heterogeneity in difference between the incentivized and non-incentivized versions of the program. The results show that, for health, there is a negative interaction between treatment A and baseline levels, implying that the incentives were more effective in low baseline areas. For example, over the two years of the program, the additional impact of the incentives (i.e., treatment A compared with treatment B) on health indicators was twice as large at the 10th percentile as on the average (0.0638 standard deviations at the 10th percentile vs. an average effect of 0.0316 shown in Table 10). In terms of long-term health outcomes, larger levels of reduction in malnutrition (in terms of underweight) were also found in areas with a higher baseline malnutrition rate, as discussed above. Interestingly, Table 18 shows that this pattern holds only for malnourishment (weight for age < 2 standard deviations), but not for other metrics of malnourishment. In fact, for other metrics—severe malnourishment (weight for age < 3 standard deviations) and wasting (weight for height < 2 standard deviations)—the pattern actually seems to be the opposite. Reductions in neonatal and infant mortality were also larger in subdistricts with higher mortality rates. In fact, the reductions in mortality rates were found predominantly in areas with high baseline mortality rates. Baseline-Level Subdistrict Average Per Capita Consumption In order to assess how Generasi works in relatively better-off subdistricts and relatively poor subdistricts, the baseline level of subdistrict average per capita consumption was included in the model as an interaction term. The results are shown in Table 19 and Table 20. In general, no differences in the impact on health indicators were observed between better-off and poorer subdisticts. The only health indicator that differed in terms of impact between better-off and poorer subdistricts was the impact on the numbers of sachets of iron tablets pregnant mothers received, with the impact being stronger in better-off subdistricts than poorer ones (Table 19). For education, the negative impact on education in Wave II was larger in poorer subdistricts; in Wave III, the positive impacts were also found to be larger in poorer subdistricts. With regard to long-term outcomes, the impact on stunting and severe stunting, measures of chronic malnutrition, was also observed to be stronger in poorer subdistricts than in relatively better-off subdistricts (Table 20). Nevertheless, when averaged, no difference in impacts on long-term health outcomes was observed between better-off and poorer subdistricts. Village Access The effects of baseline levels of village access to health facilities and schools were important for the government to consider in the project design, given the disparities among villages in access to basic services within a subdistrict. The government’s concern was that pre-existing disparities may work against villages with worse access to these services. To investigate this issue, in Table 21 for each indicator, we interact Generasi with a measure of access to services at baseline. For health services, access was defined as a dummy variable taking a value of 1 if there is a midwife or other health facility in the village at baseline and 0 otherwise. For education, access is defined as a dummy variable taking a value of 1 if there is an age-appropriate school (primary for 7–12 year olds; junior secondary for 13–15 year olds) in the village at baseline and 0 otherwise. In general, Final Impact Evaluation Report 45 Where Were Generasi’s Effects Largest? access to health facilities does not seem to matter in Generasi’s ability to improve health indicators (Table 21). For education, there is some evidence that Generasi had a larger impact in villages with schools at baseline as compared to villages without a school. In Wave II, villages with a preexisting primary school improved school participation among 7–12 years old by 1.0 percentage points (column 4), and in Wave III villages with a pre-existing junior secondary school improved school participation among 13–15 year olds by 6.6 percentage points (column 7) compared to their respective controls. However, across all indicators, the average standardized effects on the interaction of baseline access and Generasi were not statistically significant for education or for health in either survey wave. With respect to the long-term indicators (Table 22), the only significant interactions were for acute malnutrition (measured by wasting and severe wasting), with the estimates suggesting that Generasi actually led to worse outcomes in areas without access to a health facility or a midwife in the village at baseline. However, as with the main indicators, across all indicators, the average standardized effects on the interaction of baseline access and Generasi were not statistically significant for education or for health long-term outcomes in either survey wave. On net, the results in this section point strongly to baseline levels of service provision as the key determinant of where Generasi was more effective. We find robust evidence that the program was more effective in areas where baseline service levels were lower. On average, the program was about twice as effective in areas at the 10th percentile of service provision at baseline as it was on overage. This suggests that these low-performing areas, where there is the greatest room for improvement, are the most promising areas for targeting future expansion of the program. 4.2 Heterogeneity Over Time To examine how the Generasi program changes as communities gain experience from the program, we re-ran the main analysis on a subsample of subdistricts. Specifically, we restrict attention to the 129 subdistricts that started the program in year 1 and the subdistricts that remained controls over the entire period. We drop the 49 subdistricts that were controls in Wave II but started treatment in Wave III. This allows us to examine precisely how the impact of the program changed as communities gained more experience with the program. Since whether a subdistrict started Generasi in Wave III is random (conditional on being in group P or NP), once we condition on a group P/NP dummy, dropping these 49 subdistricts from all waves does not affect the validity of the results. The results on the 12 main indicators are presented in Table 23. This table presents the results for this new subsample, but adds an additional column that tests statistically whether the Generasi impact in Wave II (after 1 year of the program) is different from the Generasi impact in Wave III (after 2 years of the program). The results confirm that the program, overall, was statistically significantly more effective in the second year of the program, driven by improvements of 0.15 standard deviations in education. Overall, the point estimates suggest that health also improved in the second year (by 0.033 standard deviations), though this is not statistically significant. Looking at indicators one-by-one, we see statistically significant improvements in malnourishment, weight checks, age 7–12 school participation rates, age 13–15 participation rates, and age 13–15 gross attendance rates. Thus, this analysis confirms that overall, the Generasi program became more effective in its second year. 46 Indonesia’s PNPM Generasi Program Where Were Generasi’s Effects Largest? Table 24 repeats the same exercise, comparing the incentivized and non-incentivized versions of the program. The final column in the table shows how the additional effect of the incentives—the difference between Versi A and Versi B—changed over the two years of program implementation. While the average standardized effects are negative (0.016 standard deviations), they are not statistically significant, so we cannot conclude statistically that the incentives became less effective over time. Table 25 examines the changes over time in the final outcomes. Consistent with the results above, we see a statistically significant improvement in Generasi’s ability to combat malnutrition from the first to second year of the program. On mortality, as discussed above, Generasi had less of an impact on mortality in the second year of the program than in the first. The difference is not statistically significant for 0–28 day mortality, but is statistically significant for the 0–12 month indicators. 4.3 Heterogeneity Across Poor vs. Non-poor Individuals A unique feature of Generasi is that communities conduct their own needs assessment and targeting of beneficiaries. As part of the government’s poverty reduction program, it is very important to understand whether and how communities are able to target their resources toward the poorest in their communities. This section examines the program’s impact separately among the poor, defined as the bottom two quintiles of per-capita consumption in our data, and the non-poor, defined as the top three quintiles of per-capita consumption in our data. Table 26 begins by examining the targeting of the direct benefits provided by Generasi, such as scholarships, uniforms, and health support. We re-estimate the effect of Generasi shown in Table 6 separately by income quintile. Columns 2, 6, and 9 report Generasi’s impact on the top three quintiles (in Wave II, Wave II, and on average, respectively); columns 3, 7, and 10 report Generasi’s impact on the bottom two quintiles; and columns 4, 8, and 11 show the difference between Generasi’s impact on the top three quintiles relative to the bottom three quintiles. A negative coefficient in columns 4, 8, and 11 indicates that the program had a larger impact on the poor than on the non-poor, i.e., that the program exhibited differential targeting toward the poor. Table 26 shows that Generasi was more successful in targeting direct benefits to the poor in Wave III than in Wave II. 19 In Wave III, communities were statistically significantly able to target the bottom two quintiles in providing school uniforms, other school supplies, and other school support. Interestingly, 19 Note that the estimates of Generasi’s impact in this section are slightly different than the corresponding estimates in Sections 3.1 through 3.3. The reason that the overall estimated Generasi effect in Sections 3.1 through 3.3 will not be exactly a weighted average of the Generasi impact on the bottom two and top three quintiles estimated here is that, in the estimates here, we restrict attention to panel households only, since we only have information on consumption at baseline for panel households, whereas in Section 3, we examine all households. By construction, panel households in Wave II and Wave III will have slightly older children and mothers, than those in the repeated cross-section sample, and Generasi’s impact could be slightly different for these different groups. The key emphasis in this section is therefore on the difference in Generasi’s impact between those with higher and lower consumption at baseline, rather than the average level of Generasi’s impact on this more selected subgroup. Final Impact Evaluation Report 47 Where Were Generasi’s Effects Largest? the targeting seems stronger for benefits that are provided in-kind (such as uniforms, school supplies, or supplementary feeding) than for benefits that are provided as cash or subsidies (such as scholarships, transport subsidies, or subsidies for childbirth), consistent with the idea that in-kind benefits are a way of achieving self-targeting (Nichols and Zeckhauser 1982; Currie and Gahvari 2008). With respect to improvements on target indicators, in general, there was little difference in impact among the relatively better-off (top three quintiles) and the poor (bottom two quintiles) households in both health and education indicators in Wave II and health indicators in Wave III (Table 27). However, in Wave III, the impact on education was largely seen among the poor households, though the difference in average standardized effects is not statistically significant. Specifically, among the poorest two quintiles, primary school participation and attendance among 7–12 year-olds increased by 2 percentage points, a 1.8 percent increase in participation and 2.3 percent increase in attendance, compared to control areas, and age-appropriate junior secondary school participation among 13–15 year-olds improved by 7.5 percentage points, an 11 percent increase compared to control areas (column 8), whereas there were no impacts among the relatively better-off households (column 7). No notable differences were found on long-term health outcomes between the poor and the relatively better-off households (Table 28). 48 Indonesia’s PNPM Generasi Program Where Were Generasi’s Effects Largest? Final Impact Evaluation Report 49 How And Why Did The Generasi Project Work? How And Why Did The Generasi Project Work? his section explores the various channels through which Generasi could have impacts on basic health T and education services using data from the provider surveys. We look at some of the potential causal chains to see why Generasi might have had the effects that it did. We first examine changes on the provider supply side: (a) the quantity of education and health providers; (b) the inputs used by providers such as facilities and medical supplies; and (c) the level of effort provided by education and health providers. We then examine community efforts at service provision, outreach, monitoring, and participation in various education and health programs. Overall, we find that Generasi had the greatest impact on community effort, which is intuitive given the project’s emphasis on community mobilization as a centerpiece of its strategy. The impact on community effort was greatest in relation to the increase of cadres working at village health posts, and participation in meetings about health education. On the education side, there was an increase in the number of junior school committee meetings with teachers. Households in Generasi areas felt that both health and education services had improved over the past two years. In terms of service provision overall, there were no statistically significant impacts. There was a slight decrease in health provider inputs due to a decrease in completing all antenatal care service items at all times in private, and to some extent, public practice. In terms of incentivized versus non-incentivized areas, the only difference overall for these variables was in provider effort. Midwives in incentivized areas spent more hours working during the three days prior to the survey date. For all other variables, there was no overall significant difference between the incentivized and non-incentivized areas. 5.1 Changes in Provider Quantities We first examine Generasi’s impact on the quantity of providers (Table 29 and Table 30) We looked at changes in providers at the village level. For health, we examined the presence of midwives, the number of active village health posts, and full-time health personnel. For education, we examined the presence of primary schools and junior secondary schools in the village and the number of teachers at primary school and junior secondary school. On the health side, Generasi had no impact on provider quantities on average, with the only increase coming from an increase in the number of health personnel at the health facility of 1.57 people (6.8 percent). None of the other individual health effects were statistically significant, and there was no difference between incentivized and non-incentivized areas overall. On the education side, however, there was a significant increase in quantities. On average, all of the quantity variables we examined increased by 0.04 standard deviations over the two years we examine. Specifically, Generasi led to a 3.2 percentage point increase in the probability a junior secondary school was located in the village, representing about a 6.5 percent increase over the baseline level. This could have been due to the provision of additional class extension facilities (klas jauh) for junior secondary schools under Generasi. The number of teachers at junior secondary, including teachers receiving honoraria (guru honor), also increased, by about 0.5 teachers on average (about a 2.2 percent increase). However, there were no effects detected on the primary school side. The improvement in the presence of junior secondary schools came equally from incentivized and non-incentivized areas, and appears slightly more pronounced in Wave II. 52 Indonesia’s PNPM Generasi Program How And Why Did The Generasi Project Work? 5.2 Changes in Provider Quality This section describes Generasi’s impact on the inputs providers use. Specifically, we examine the quality of the midwife’s facility (access to water and electricity), the availability of medical supplies and tools, the number of school classrooms, the condition of the school building, and the presence of latrines at school, all using data from the provider interviews. Overall, for both waves, we found, if anything, a negative impact on these variables (Table 31 and Table 32). There was a decrease for health (average of -0.05 standard deviations) caused primarily from a decrease in the antenatal care provision of all service items. One hypothesis is that this decrease may be due to the midwives doing more under Generasi and therefore spending less time with each patient. However, it is worth noting that the data on quality of antenatal care from household surveys actually showed an increase in quality, so on net our results on quality of care are inconclusive. Otherwise, the project did not increase the infrastructure, tools or drug supply for midwives or vaccination stocks at subdistrict health facilities (puskesmas), nor were there any changes in infrastructure at schools. When comparing the impact of treatment A and treatment B for the third wave and overall for the two years, there were no significant differences. 5.3 Changes in Provider Effort This section examines the third component of supply: levels of provider effort. Using data from the provider surveys, we examine time use of midwives for outreach, hours spent for public and private services, and participation in village health post activities. For teachers, we examine teacher absence, and teacher teaching behavior. Overall, we found no significant impact on average over the two waves of the survey (Table 33 and Table 34). The most striking difference is in Table 34, which examines the impact of the incentives. Specifically, we find that across the two survey waves, we find an increase in the average standardized effect for provider effort of 0.046 standard deviations, driven by an increase of 0.066 standard deviations in health. Within health, the impacts seem to be an increase hours worked by midwives (increase of 1.724 hours worked over the previous 3 days), and specifically, an increase of 0.83 hours over the previous three days providing public services. There were no effects on education. 5.4 Changes in Community Effort This section explores the impact of Generasi on community effort. We examine three types of community effort: (1) community effort at direct service provision, such as the number of active village health post sessions and the number of cadres at the village health post; (2) community effort at outreach, such as health sweepings to make sure everyone in the village obtains service and school committee meetings with parents; and (3) community effort at monitoring, such as the number of school committee meetings. We also examine spillovers of Generasi to other types of community activities, such as semi-volunteer public labor service (gotong royong), government service, and other community groups. Final Impact Evaluation Report 53 How And Why Did The Generasi Project Work? Overall, we find that in Wave III, Generasi increased community effort, particularly on the number of cadres at village health posts, participation in meetings about health education, and number of junior secondary school committee meetings with teachers during the year. On average, Generasi had positive impacts on community effort (an increase of .06 standard deviations, significant at the one percent level) (Table 35 and Table 36). In the area of community effort at direct service provision, we found that Generasi had an impact on increasing the number of cadres over the past few years. The average treatment effect across the two waves was 0.26 people; for Wave III, the number of village health post cadres increased by 0.32 persons or about 6.6 percent of the control group mean. The increase in number of cadres occurred in both Treatment A and Treatment B areas. There were no other changes in relation to number of active village health posts or the frequency of village health post meetings. Turning to community effort at outreach, we found no effects of Generasi—either treatment A or treatment B—on any of our metrics of community outreach in Waves II or III. Specifically, we found no impact on the number of village health post sweepings, where the village health post cadres go door-to- door to make sure all households are receiving services, and we found no impact on the number of school committee meetings with parents for either primary or junior secondary schools. For community effort at monitoring, over the past two waves, we see some improvement in the number of junior secondary school committee meetings with teachers during the year (an increase of 0.44 meetings). This increase was primarily due to more meetings during the past year. There were no other effects detected for number of primary or junior school committee members or primary school committee meetings. In terms of community participation in health or education programs, overall there was a marked increase in meeting participation related to health education and the proportion of children under three years of age with health cards (KIA). In Wave III, we also asked if households felt that health or education services had improved over the last two years. Respondents in Generasi areas felt strongly that both services had improved. We also sought to explore whether or not Generasi would have spillover effects to other types of community activities. We looked at participation in gotong royong, women’s participation in women’s and government groups, and general participation in social groups. Overall we found no spillover effects for these activities, except for participation in gotong royong. For the two years, the average increase was 2.7 additional hours per household in the past 12 months on gotong royong (a 7.2 percent increase over the baseline level). 5.5 Services and Prices from Providers An important question is the extent to which the demand increase spurred by Generasi results in an increase in the utility of target beneficiaries, or is instead largely absorbed in the form of higher prices charged by providers. To the extent that supply is inelastic, one might expect a substantial share of the benefits to be absorbed in the form of higher prices; to the extent that it is elastic, one would expect a greater quantity increase and a smaller price effect. 54 Indonesia’s PNPM Generasi Program How And Why Did The Generasi Project Work? To investigate this, we examine data on prices and quantities from providers (Table 37). We focus on provider data for this analysis, rather than household data, since we can best obtain information on quantities and prices for a well-defined set of services from the provider data. The use of provider quantity data also provides a cross-check on the household-level data analyzed above. We examine data from midwives, health centers, village health posts, and schools, and examine child births attended by midwives at government and private practice, antenatal care, post-natal care, family planning, and school enrollments On quantities, the story for service provision is consistent across the two waves of the survey: Generasi significantly increased maternal and child health services offered at the village health post. Specifically, the number of children weighed at last visit increased on average over the past two years by 9.97 children (11.2 children for Wave III); the number of children with nutritional supplements increased by 15.2 children on average over the past two years (16.3 children in Wave III); the number of children immunized increased by 2.7 children on average over the past two years and by 2.4 children in Wave III; the number of children receiving Vitamin A increased by 10.2 children on average during the past two years (11.7 children in Wave III). For mothers, increased posyandu services translated into more pregnant mothers receiving antenatal care (an increase of 1.3 on average over the past two years); and more mothers receiving iron pills (an increase of 1.8 on average over the past two years) (Table 37). The accompanying qualitative study found that communities especially appreciated the services at the village health post level. In Wave III, we also saw improvements in antenatal care visits, an increase of two antenatal care visits in the last month. The number of childbirths at government practice also increased in the last month by 1.8 births. Related to fees, the data from midwives show increases in fees charged for delivery services. In private practice, fees charged by midwives for childbirth increased by an average of Rp. 16,899 over the two years, an increase of about 5 percent over the baseline level. The total number of services delivered in private practice, however, did not change substantially. For births at government practice over the past two years, fees increased by Rps 21,973, an increase of 15 percent over the baseline level. The number of births at government practice in last month increased on average by 0.947 births (1,78 births by Wave III). On average, there was a 10 percent increase in the total number of births assisted by midwives and a 5 percent increase in prices. To the extent that Generasi can be exclusively thought of as a demand shock, this suggests a supply elasticity of midwife services of about 2, suggesting that supply is relatively elastic and that most of the benefits accrued to the beneficiaries, not the providers. For other services considered, there were no increases in fees, and in fact we observed a decrease in village health post fees and junior secondary school fees associated with Generasi. Final Impact Evaluation Report 55 Discussion Discussion his document describes the findings from the three-wave evaluation series carried out from the period T of 2007 to 2010. The third and final evaluation survey was implemented in October 2009 to January 2010 after 27 to 30 months of project implementation. The main findings of the Generasi impact evaluation are as follows: 1. After 30 months of program implementation, Generasi had a statistically significant positive impact on average across the 12 indicators it was meant to address. The strongest improvements among the health indicators were in the frequency of weight checks for young children. The program also increased the number of iron sachets pregnant mothers received through antenatal care visits. These improvements were supported by dramatic increases in mothers and children participating in village health post activities to receive the targeted maternal, neonatal, and child health services. Education indicators also saw improvements in the final evaluation, reversing the zero or negative impact found at the interim evaluation. The improvement in education indicators was most notable in the increased school participation rate among the primary school-age group. 2. The main long-term impact was a decrease in malnutrition. The latest Wave III survey shows that childhood malnutrition was reduced by 2.2 percentage points, about a 10 percent reduction from the control level. This reduction in malnutrition was strongest in areas with a higher malnutrition rate prior to project implementation, most notably in the Nusa Tenggara Timur (NTT) Province where severe underweight rates were reduced by 5.5 percentage points, a 33 percent decline and severe stunting was reduced by 6.6 percentage points, a 21 percent decline compared to control areas. In Java, however, there was actually a negative impact on stunting and severe stunting which needs to be explored further. Although reductions in infant and child mortality were observed in the interim evaluation, the same levels of reduction in mortality were not sustained in the final evaluation. In terms of the longer term education learning outcomes, the program did not improve childhood test scores as yet. 3. Making grants conditional upon performance improves program effectiveness in health but not in education. On average, the incentivized group outperformed the non-incentivized group in improving health indicators, particularly in increasing antenatal care services and improving coverage of childhood immunization. On net, between 50-75% of the total impact of the block grant program on health indicators can be attributed to the performance incentives. However, for education indicators the incentivized group did not appear to perform better than the non- incentivized group. There may be several reasons for this. The data show two results: (1) the impact of incentives became weaker over time; and (2) the positive impacts on education only occurred in Year Two of the program, probably due to time lags in implementation. Thus, incentives were not real factors in Year One implementation and by Year Two, the incentives were less strong just when education indicators were improving. Another factor may have been that school enrollment and attendance obstacles are much harder problems to overcome than for health. Baseline levels for health indicators were lower than for education, making it perhaps easier to make gains in health. In addition, education targets may have been more difficult to achieve since those targets involved more people and involved school enrollment and attendance every day, as opposed to once a month for health targets, with fewer villagers involved. Also, the qualitative report suggests that communities favored giving school assistance directly to the greatest number of students, rather than difficult- 58 Indonesia’s PNPM Generasi Program Discussion to-reach, out-of school children, and that motivation may have dampened any effects from the incentives. The qualitative study furthermore reports continued issues of teacher absenteeism, job opportunities for teenagers, and lack of proper school facilities, which discourage communities from sending their children to school. Lastly, the qualitative report indicates that the incentives rules were oftentimes difficult for communities to understand. 4. Generasi had the greatest impact in areas with low baseline health and education indicators. We find robust evidence that the program was more effective in areas where baseline service levels were lower. On average, the program was about twice as effective in areas at the 10th percentile of service provision (very low health and education coverage) at baseline as it was on average. Areas with lower pre-project health and education indicators imply that they have more room for improvement. The greater impacts in areas with a lower baseline appear more prominently in the final evaluation survey than the interim results, with stronger improvements found in education indicators in these areas. These improvements in health and education indicators in areas with low baseline coverage however, did not appear to have resulted in improving long-term health and education outcomes in these areas. Furthermore, the greater impacts observed in health and education indicators were not simply correlated with pre-project levels of poverty, but instead were driven by the level of health and education indicators in the area. Final Impact Evaluation Report 59 60 Indonesia’s PNPM Generasi Program Policy Implications And Conclusion Final Impact Evaluation Report 61 Policy Implications And Conclusion Based upon these findings, some of the policy implications and conclusions are: 1. Generasi is most effective in areas with low health and education coverage. The impact evaluation found that Generasi impacts are stronger in areas where health and education indicators are low. This suggests that future expansion of Generasi implementation should prioritize areas where these indicators are lagging behind and not necessarily in areas identified as poor. 2. Community incentives had mixed results—incentives may be successful in other contexts. As a result of the two-year project implementation, health indicators responded strongly and positively to community incentives, but education indicators saw no positive or negative response to community incentives. Learning from this experience, the government may wish to see how community incentive interventions can work in other lagging areas, such as water and sanitation access. The policy implications are that poverty programs may wish to experiment more with embedding incentives into the designs; however, the interventions and incentives will need to be monitored and evaluated over time. One possibility is that the conditionalities may work less well over time, as there may be more “gaming” of the system as the program progresses and the rules become more familiar. Alternatively, the program may work better over time as it continues to incentivize communities to work harder toward the specified targets. Also, qualitative evidence suggests that simplifying the incentive scheme may make it much easier for communities to understand. 3. The project was able to build upon the government’s existing national community-driven development architecture and network, which began in 1998. In the past, the CDD program was used as a post-disaster and post-conflict assistance vehicle to deliver resources quickly. There is also a pilot under way to see how the program can be used to improve the environment. Generasi was started as an experiment in adapting the existing KDP/PNPM-Rural process of community participatory planning and block grants to focus upon specific education and health targets, which thus far were not being addressed sufficiently in the current design. The project added a focus on specific health and education targets, rather than a general open menu of activities, as has been the case in KDP. Generasi also experimented with adding performance incentives to the block grant model by rewarding those villages that were more successful in reaching the targets. When the government of Indonesia decided in 2007 to move from an unconditional cash transfer scheme to a conditional transfer scheme, they opted to try two different approaches, one the traditional individual household approach as proven successful in many countries of Latin America, and the other, an incentivized community block grant program, taking into consideration the success and architecture already in place under KDP. Unlike in Mexico and other countries, it was not clear that Indonesia had the administrative capacity and supply-side services to make an individual CCT program work in certain areas of the country. Thus, Generasi provides one unique example of how an established government program can be adapted to address certain education and health targets using a community approach. Building upon an already existing national program, which covered most of the poorest areas of the country, also facilitated a much faster start-up of the pilot. 4. The project should regularly review the appropriateness of the targets. Target indicators must be relevant to communities yet reflect development priorities of the government. Although it is 62 Indonesia’s PNPM Generasi Program Policy Implications And Conclusion important not to overload the project with too many target indicators, Generasi should regularly review its 12 target indicators and assess if existing ones should be replaced or added to. The education indicators in particular should be monitored carefully in the future. While the most recent evaluation round found positive standardized effects for education, Generasi needs to monitor carefully the project’s progress on the education front within the broader context of a changing national education environment. Primary level enrollment rates are already over 98.5 percent, thus focusing on improvements in the quality of nine-year basic education in Indonesia in terms of student learning is vital. With increased funding in the education sector, the project needs to ensure that its relatively small amount of funds are used strategically and in areas where it has comparative advantage, such as community participatory planning, targeting, and implementation. The accompanying qualitative study reports numerous other education assistance programs occurring at the village level. The government may wish to address other priority areas as well such as early childhood development, education learning achievement, or water and sanitation by adding to the existing Generasi target indicators. 5. A follow-up evaluation may be needed in the future to examine the longer term sustainability of interventions and impacts. The final round of evaluation took place after 2.5 to 3 years of project implementation. Building the evaluation into the design of the program from the outset was critical to learn lessons from the program for possible expansion in the future. To allow for a rigorous, randomized evaluation of Generasi, the government incorporated random assignment into the selection of the locations. Each location was further randomly allocated to an incentivized versus non-incentivized treatment allowing for comparison of effects. As this is a pilot program, it was important that the evaluation prove robust and provide empirical evidence as to whether the intervention was having its desired impact. Discussions from the earliest stages included evaluation of the design. The government has been highly supportive of the evaluation efforts throughout the three years. Should the program continue in the original treatment and control sites, it would be worthwhile for the government to consider the possibility of another evaluation in a few years time to explore if the impacts of this program are indeed sustainable over time and if additional progress can be made on longer time learning and health outcomes. Final Impact Evaluation Report 63 Policy Implications And Conclusion References Bjorkman, A., and P. Svensson. 2009. “Power to the People: Evidence from a Randomized Experiment of a Community Based Monitoring Project in Uganda.” Quarterly Journal of Economics 124(2): 735- 769. Currie, J., and F. Gahvari. 2008. “Transfers in Cash and In-Kind: Theory Meets the Data.” Journal of Economic Literature 46(2): 333–83. Duflo, E., R. Glennerster, and M. Kremer. 2007. “Using Randomization in Development Economics Research: A Toolkit.” In T. P. Schultz and J. Strauss, eds. Handbook of Development Economics. North Holland: Elsevier Science Ltd. Vol. 4: 3895–62. Gertler, P. 2004. “Do Conditional Cash Transfers Improve Child Health? Evidence from PROGRESA’s Control Randomized Experiment.” American Economic Review (Papers Proceedings) 94 (2): 336–341. Paul Do Conditional Cash Transfers Improve Child Health? Evidence from PRO Review (Papers and) 94(2): 336-341. Gibbons, R., and K. J. Murphy. 1990. “Relative performance evaluation for chief executive officers.” Industrial and Labor Relations Review 43 (Special Issue): 30–51. Gordis, L. 2004. Epidemiology. Philadelphia: W. B. Saunders Company. Holmstrom, B.1979. “Moral Hazard and Observability.” Bell Journal of Economics 10(1): 74-91. Holmstrom, B. and P. Milgrom. 1991. “Multitask principal-agent analysis: Incentive contracts, asset ownership, and job design.” Journal of Law, Economics, and Organization 7: 24–52. Imbens, G., and J. Angrist. 1994. “Identification and Estimation of Local Average Treatment Effects.” Econometrica 61 (2): 467–476. Kling, J.R., J.B.Liebman, and L.F. Katz. 2007. “Experimental Analysis of Neighborhood Effects.” Econometrica 75 (1): 83–119. Lagarde, M., A. Haines, and N. Palmer. 2007. “Conditional Cash Transfers for Improving Uptake of Health Interventions in Low- and Middle-Income Countries: A Systematic Review.” Journal of the American Medical Association 298(16):1900–1910. Lazear, E.P., and S. Rosen. 1981. “Rank-order tournaments as optimum labor contracts.” Journal of Political Economy 89: 841–864. Levy, S. 2006. Progress Against Poverty: Sustaining Mexico’s Progresa-Oportunidades Program. Washington, DC: Brookings Institution Press. Miguel, E., and M. Kremer. 2004. “Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities.” Econometrica 72: 159–217. Ministry of Home Affairs, Government of Indonesia. 2007. Petunjuk Teknis Operasional PNPM Generasi Sehat dan Cerdas (Versi A). Jakarta, Indonesia. 64 Indonesia’s PNPM Generasi Program Policy Implications And Conclusion Mookherjee, D. 1984. “Optimal Incentive Schemes with Many Agents.” Review of Economic Studies 51: 433–446. Newhouse J.P., et al. 1993. Free For All? Lessons from the Rand Health Insurance Experiment. Cambridge, MA: Harvard University Press. Nichols, A.L., and R. Zeckhauser. 1982. “Targeting Transfers Through Restrictions on Recipients.” American Economic Review 72: 372–277. Olken, B. A. 2007. “Monitoring Corruption: Evidence from a Field Experiment in Indonesia.” Quarterly Journal of Economics, 115 (2): 200–249. Olken, B.A., J. Onishi, and S. Wong. 2011. “Should Aid Reward Performance? Evidence from a field experiment on health and education in Indonesia” (forthcoming) --------------------------------------. 2011. “A Community Approach to Achieving Health and Education Outcomes: A Pilot in Indonesia” (forthcoming) Rahayu, S., N. Toyamah, S. Hutagalung, M. Rosfadhila, and M. Syukri. 2008. “Qualitative Baseline Study for PNPM Generasi and PKH: The Availability and Use of Maternal and Child Health Services and Basic Education Services in the Provinces of West Java and East Nusa Tenggara.” Jakarta, Indonesia: SMERU Research Institute. Rawlings, L., and G. Rubio. 2005. “Evaluating the Impact of Conditional Cash Transfer programs.” The World Bank Research Observer 20(1): 29–55. Schubert, B., and R. Slater. 2006. “Social Cash Transfers in Low-Income African Countries: Conditional or Unconditional?” Development Policy Review 24(5): 571–578 Schultz, T. P. 2004. “School Subsidies for the Poor: Evaluating the Mexican Progresa Poverty Program.” Journal of Development Economics 74 (1): 199–250. Skoufias, E. 2005. “PROGRESA and Its Impacts on the Welfare of Rural Households in Mexico.” IFPRI Research Report No. 139. Washington, DC: International Food Policy Research Institute. Sparrow, R., J. Moeis, A. Damayanti, Y. Herawati. 2008. “Conditional Cash Transfers in Indonesia: Baseline Survey Report for Program Keluarga Harapan and PNPM Generasi.” Jakarta, Indonesia: World Bank. Weitzman, M.L. 1980. “The ‘Ratchet Principle’ and Performance Incentives.” Bell Journal of Economics (The RAND Corporation) 11(1): 302–308. World Bank. 2008. Investing in Indonesia’s Health: Challenges and Opportunities for Future Public Spending. Jakarta, Indonesia, and Washington, DC: World Bank. -----------------. 2006. Making the New Indonesia Work for the Poor. Jakarta, Indonesia and Washington, DC: World Bank. Final Impact Evaluation Report 65 Annexes Annexes Annex I. Randomization and Implementation of Generasi in 2007 The primary reason that some (not all) 200 subdistricts received Generasi was that the 2007 supplemental budget allocation for Generasi was cut by the national parliament, so that in 2007, funding was only available for 129 Generasi subdistricts.20 In choosing which 129 of the 200 randomly selected subdistricts should be funded, the Ministry of Home Affairs prioritized those locations that had already participated in the PNPM program, since those locations already had the legal infrastructure for distributing PNPM program funds and it was easier to re-budget other monies to fund Generasi in those areas. The final allocation of Generasi is shown in Table 3. The 300 Generasi sample subdistricts are subdivided into two groups: the 170 subdistricts that had received the PNPM program in previous years (denoted group P, the prioritized group), and the 130 subdistricts that had not previously received the PNPM program in previous years (denoted group NP, the non-prioritized group).21 In Group P, Generasi was funded according to the randomization results in a total of 106 subdistricts in 2007, or 92 percent of the Group P subdistricts that had been chosen according to the randomization.22 The 2008 allocation for Group P was similar.23 In Group NP, Generasi was funded in 23 subdistricts, or 27 percent of the Group NP subdistricts that had been chosen according to the randomization. Of these 23 subdistricts, 21 were chosen randomly by computer, stratified by province, in a second lottery among Group NP locations; the remaining 2 subdistricts were chosen by the ministry. In 2008, additional funding became available, and a total of 71 (84 percent) of the 85 Group NP subdistricts randomly selected for Generasi received the program. 20 Funding for 108 subdistricts came from World Bank loans; funding for the remaining 21 subdistricts came from a grant from the Dutch government. Both funding sources were channeled through the government budget, and were implemented identically in the field. 21 The randomization results are statistically unrelated to whether a subdistrict is in Group P or Group NP. Specifically, the p-value from a Fisher exact test of the two-way relationship between the three randomization categories (incentivized, non- incentivized, control) and a group P dummy is 0.739. 22 Four categories of subdistricts were deemed ineligible for Generasi in 2007: (1) they had been identified as “problematic” PNPM subdistricts, i.e., there were allegations of improper use of PNPM funds; (2) they had been identified as eligible for the urban version of PNPM; (3) they had been identified as eligible for the SPADA (conflict areas) version of PNPM; or (4) they were the one subdistrict where a three-village Generasi pilot was being run from 2006 to 2008. Which subdistricts fell into which categories were determined based on information obtained prior to the date of the randomization, and is available for all subdistricts regardless of the results of the randomization. 23 The only difference between 2007 and 2008 in Group P is that two Group P subdistricts funded in 2007 were identified as “problematic” and were dropped from 2008, and one subdistrict that had previously been identified as “problematic” resolved its financial problems and was allowed to resume. 68 Indonesia’s PNPM Generasi Program Annexes Table 3. Generasi implementation and randomization results Randomization results Incentivized Non-incentivized Control Total Generasi Generasi P NP P NP P NP Total subdistricts 60 40 55 45 55 45 300 Received Generasi in: 2007 57 11 49 12 0 0 129 2008 55 35 50 36 0 0 176 Since the share of subdistricts randomly selected to Generasi that were subsequently funded is much higher in Group P, and since Group P/NP status is predetermined with respect to the randomization (it depends only on whether a subdistrict had received the PNPM program in previous years), we can improve the statistical power beyond intent-to-treat estimates by incorporating this information into the analysis. Final Impact Evaluation Report 69 Annexes Table 4. Baseline regressions, 12 main indicators * Total Total Total Versi A Control Number Indicator Generasi Year Versi A Versi B additional mean observations 1 Effect impact impact effect (1) (2) (3) (4) (5) (6) Number prenatal visits 7.808 -0.255 -0.356 -0.159 -0.198 3708 (0.130) (0.229) (0.262) (0.262) (0.257) Delivery by trained 0.691 -0.020 -0.018 -0.022 0.003 2810 midwife (0.016) (0.025) (0.029) (0.030) (0.031) Number of postnatal visits 3.012 0.073 -0.023 0.166 -0.189 2810 (0.111) (0.176) (0.195) (0.209) (0.198) Iron tablet sachets 1.591 -0.048 -0.074 -0.023 -0.051 3676 (0.038) (0.064) (0.070) (0.076) (0.072) Percent of immunization 0.680 -0.0378** -0.0387* -0.0369* -0.002 3316 (0.011) (0.019) (0.021) (0.022) (0.023) Number of weight checks 2.140 -0.066 -0.089 -0.042 -0.046 4045 (0.033) (0.056) (0.060) (0.068) (0.064) Number Vitamin A 1.521 0.028 0.094 -0.036 0.1303** 2284 supplements (0.044) (0.053) (0.061) (0.062) (0.065) Percent malnourished 0.173 0.015 0.022 0.007 0.015 3977 (0.011) (0.013) (0.016) (0.015) (0.016) SD age gross enrollment 0.950 0.000 0.006 -0.006 0.012 5137 (0.005) (0.007) (0.009) (0.008) (0.008) SMP age gross enrollment 0.825 0.009 0.017 0.001 0.017 1754 (0.016) (0.023) (0.027) (0.027) (0.027) SD age gross attendance 0.910 -0.016 -0.006 -0.0280** 0.022 4397 (0.007) (0.012) (0.014) (0.014) (0.016) SMP age gross attendance 0.752 0.020 0.037 0.004 0.032 1511 (0.019) (0.027) (0.031) (0.032) (0.033) Average standardized effect -0.023 -0.016 -0.030 0.013 (0.020) (0.023) (0.024) (0.023) Average standardized effect -0.036 -0.0426* -0.030 -0.013 health (0.023) (0.024) (0.028) (0.026) Average standardized effect 0.004 0.036 -0.029 0.065 education (0.042) (0.047) (0.049) (0.048) *Note that these are in the baseline, so we predict no effect. 70 Indonesia’s PNPM Generasi Program Annexes Table 5. Baseline regressions, long-term final outcomes Total Total Total Versi A Control Number Indicator Generasi Year Versi A Versi B additional mean observations 1 Effect impact impact effect (1) (2) (3) (4) (5) (6) Mortality 0–28 days 0.014 -0.003 -0.007 0.001 -0.008 2847 (births in past 18 months) (0.004) (0.005) (0.006) (0.005) (0.006) Mortality 0–11 months 0.024 -0.0088* -0.0137** -0.004 -0.010 3508 (of births in past 24 months) (0.005) (0.005) (0.006) (0.006) (0.006) Diarrhea or ARI 0.348 -0.013 -0.025 0.000 -0.025 4083 (0.013) (0.017) (0.019) (0.021) (0.022) Malnourished 0.173 0.015 0.022 0.007 0.015 3977 (< -2 SD deviations) (0.011) (0.013) (0.016) (0.015) (0.016) Severe malnourished 0.046 0.009 0.004 0.014 -0.009 3977 (< -3 SD deviations) (0.006) (0.009) (0.010) (0.011) (0.010) Average standardized effect 0.007 0.027 -0.012 0.039 health (0.021) (0.025) (0.026) (0.027) Average standardized effect -0.018 -0.005 -0.027 0.019 health excluding mortality (0.027) (0.019) (0.033) (0.033) Note: average standardized effect rows are always defined so that positive is an improvement (i.e., lower mortality, lower malnourishment) Final Impact Evaluation Report 71 Annexes Table 6. Direct benefits Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Baseline Control Control Indicator Treatment Treatment Treatment Mean Mean Mean Effect Effect Effect (1) (2) (3) (4) (5) (6) Received scholarship 0.025 0.0120** 0.041 0.0144** 0.0132*** [0.0048] (0.005) [0.0067] (0.006) (0.004) Received uniform 0.013 0.0963*** 0.011 0.0771*** 0.0861*** [0.0036] (0.013) [0.0036] (0.009) (0.009) Received other school supplies 0.008 0.0585*** 0.011 0.0614*** 0.0600*** [0.0027] (0.009) [0.0036] (0.008) (0.007) Received transport subsidy 0.000 0.0096*** 0.000 0.0063*** 0.0079*** [0.0000] (0.003) [0.0000] (0.001) (0.002) Received other school support 0.000 0.000 0.001 0.0068** 0.0036** [0.0000] (0.000) [0.0011] (0.003) (0.001) Received supplementary 0.000 0.0045** 0.005 0.004 0.0045* feeding at school [0.0000] (0.002) [0.0023] (0.004) (0.002) Received supp. feeding at 0.469 0.1548*** 0.459 0.1900*** 0.1746*** posyandu [0.0171] (0.024) [0.0168] (0.021) (0.018) Received intensive supp. 0.027 0.0162** 0.047 0.0216*** 0.0192*** feeding at school [0.0055] (0.007) [0.0071] (0.008) (0.006) Received health subs. for pre/ 0.005 0.0307*** 0.007 0.0320*** 0.0314*** post-natal care [0.0023] (0.006) [0.0028] (0.005) (0.005) Received health subsidy for 0.039 0.1135*** 0.046 0.1117*** 0.1125*** childbirth [0.0078] (0.014) [0.0084] (0.014) (0.011) Average standardized effect 0.3194*** 0.3009*** 0.3327*** (0.030) (0.022) (0.024) Average standardized effect 0.2978*** 0.2905*** 0.3398*** health (0.028) (0.026) (0.032) Average standardized effect 0.3409*** 0.3113*** 0.3284*** educ. (0.046) (0.034) (0.032) All outcomes are dummy variables, (no baseline values in 1.1 tables b/c not have Wave I info) Average standardized effect total, health and education for average exclude supplementary feeding at posyandu and intensive supplementary feeding at school *statistically significant at 10 percent level; **statistically significant at 5 percent level; ***statistically significant at 1 percent level 72 Indonesia’s PNPM Generasi Program Annexes Table 7. Direct benefits, incentivized vs. non-incentivized Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Received 0.0162** 0.008 0.009 0.0208** 0.009 0.012 0.0187*** 0.008 0.0108* scholarship (0.007) (0.006) (0.008) (0.009) (0.007) (0.009) (0.005) (0.005) (0.006) Received 0.1095*** 0.0828*** 0.027 0.0824*** 0.0723*** 0.010 0.0948*** 0.0777*** 0.017 uniform (0.019) (0.012) (0.018) (0.013) (0.010) (0.015) (0.013) (0.008) (0.012) Received other 0.0634*** 0.0535*** 0.010 0.0701*** 0.0534*** 0.017 0.0670*** 0.0533*** 0.014 school supplies (0.012) (0.009) (0.012) (0.012) (0.010) (0.015) (0.010) (0.007) (0.011) Received transport 0.0143*** 0.0049* 0.009 0.0078*** 0.0050*** 0.003 0.0108*** 0.0051*** 0.0056* subsidy (0.005) (0.003) (0.006) (0.002) (0.002) (0.003) (0.003) (0.002) (0.003) Received other 0.000 0.000 0.001 0.0072** 0.0063* 0.001 0.0039** 0.0033* 0.001 school support (0.000) (0.000) (0.000) (0.003) (0.003) (0.004) (0.002) (0.002) (0.002) Received supp. feeding at 0.005 0.0041** 0.001 0.006 0.003 0.003 0.005 0.004 0.002 school (0.003) (0.002) (0.004) (0.006) (0.005) (0.007) (0.004) (0.003) (0.004) Received supp. feeding at 0.1533*** 0.1563*** -0.003 0.1745*** 0.2044*** -0.030 0.1647*** 0.1843*** -0.020 posyandu (0.028) (0.027) (0.028) (0.025) (0.022) (0.023) (0.022) (0.019) (0.019) Received intensive supp. 0.008 0.0252** -0.018 0.0242** 0.0191** 0.005 0.0173*** 0.0212*** -0.004 feeding at School (0.007) (0.011) (0.011) (0.010) (0.009) (0.010) (0.007) (0.007) (0.007) Received health 0.0343*** 0.0270*** 0.007 0.0273*** 0.0364*** -0.009 0.0304*** 0.0323*** -0.002 subsidy for pre/postnatal (0.008) (0.007) (0.009) (0.006) (0.007) (0.009) (0.006) (0.006) (0.007) care Received health subsidy for 0.1010*** 0.1273*** -0.026 0.0974*** 0.1249*** -0.028 0.0991*** 0.1260*** -0.027 childbirth (0.017) (0.017) (0.019) (0.016) (0.020) (0.023) (0.012) (0.015) (0.016) Final Impact Evaluation Report 73 Annexes Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Average standardized 0.3394*** 0.2995*** 0.040 0.3076*** 0.2950*** 0.013 0.3526*** 0.3140*** 0.039 effect (0.041) (0.030) (0.040) (0.031) (0.028) (0.039) (0.032) (0.026) (0.035) Average standardized 0.2847*** 0.3122*** -0.028 0.2657*** 0.3136*** -0.048 0.3179*** 0.3620*** -0.044 effect health (0.037) (0.031) (0.039) (0.031) (0.035) (0.042) (0.039) (0.040) (0.047) Average standardized 0.3940*** 0.2867*** 0.1073* 0.3495*** 0.2764*** 0.073 0.3734*** 0.2852*** 0.0883* effect educ. (0.063) (0.041) (0.060) (0.049) (0.041) (0.059) (0.045) (0.032) (0.046) All outcomes are dummy variables Average standardized effect total, health and education for average exclude received supplementary feeding at posyandu and received intensive supplementary feeding at school 74 Indonesia’s PNPM Generasi Program Table 8. Direct benefits, provincial breakdown   Wave II Wave III AVERAGE Additional Additional Additional Additional Additional Additional Java Java Java Control NTT Sulawesi Control NTT Sulawesi NTT Sulawesi Indicator Generasi Generasi Generasi Mean Generasi Generasi Mean Generasi Generasi Generasi Generasi Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) Received 0.025 0.0142** -0.010 0.003 0.041 0.0146** 0.004 -0.008 0.0144*** -0.003 -0.003 scholarship [0.0048] (0.006) (0.014) (0.014) [0.0067] (0.006) (0.021) (0.014) (0.005) (0.013) (0.011) Received uniform 0.013 0.0698*** 0.0647** 0.0611* 0.011 0.0423*** 0.0936*** 0.0782*** 0.0550*** 0.0803*** 0.0702*** [0.0036] (0.016) (0.032) (0.033) [0.0036] (0.007) (0.024) (0.026) (0.009) (0.023) (0.024) Received other 0.008 0.0389*** 0.0392** 0.0614** 0.011 0.0396*** 0.0643*** 0.0398* 0.0393*** 0.0516*** 0.0495** school supplies [0.0027] (0.011) (0.019) (0.031) [0.0036] (0.007) (0.023) (0.021) (0.006) (0.016) (0.022) Received transport 0.000 0.0048* 0.008 0.0176* 0.000 0.0049*** 0.000 0.010 0.0049*** 0.004 0.0132** subsidy [0.0000] (0.003) (0.007) (0.010) [0.0000] (0.002) (0.002) (0.006) (0.002) (0.004) (0.006) Received other 0.000 0.000 0.000 0.000 0.001 0.000 0.0287*** 0.001 0.000 0.0142*** 0.000 school support [0.0000] (0.000) (0.000) (0.000) [0.0011] (0.002) (0.009) (0.005) (0.001) (0.005) (0.003) Received supplementary 0.000 0.001 0.0148*** -0.001 0.005 0.000 0.016 0.001 0.001 0.0152* 0.000 feeding at school [0.0000] (0.001) (0.006) (0.001) [0.0023] (0.001) (0.015) (0.002) (0.001) (0.009) (0.001) Received supp. 0.469 0.1106*** 0.0953* 0.1351* 0.459 0.1631*** 0.0955** 0.036 0.1406*** 0.0932** 0.078 feeding at posyandu [0.0171] (0.029) (0.055) (0.070) [0.0168] (0.027) (0.045) (0.059) (0.022) (0.039) (0.054) Received intensive supp. feeding at 0.027 0.013 0.025 -0.0214* 0.047 0.0210* 0.000 0.004 0.0176** 0.011 -0.007 school [0.0055] (0.009) (0.020) (0.012) [0.0071] (0.011) (0.020) (0.015) (0.008) (0.015) (0.010) Final Impact Evaluation Report Annexes 75 76 Annexes   Wave II Wave III AVERAGE Additional Additional Additional Additional Additional Additional Java Java Java Control NTT Sulawesi Control NTT Sulawesi NTT Sulawesi Indicator Generasi Generasi Generasi Mean Generasi Generasi Mean Generasi Generasi Generasi Generasi Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) Received health Indonesia’s PNPM Generasi Program subs. for pre/post- 0.005 0.0209** 0.026 0.0300** 0.007 0.0305*** 0.008 -0.001 0.0257*** 0.017 0.014 natal care [0.0023] (0.008) (0.016) (0.015) [0.0028] (0.006) (0.014) (0.013) (0.006) (0.012) (0.012) Received health subsidy for 0.039 0.1301*** -0.0694** 0.002 0.046 0.1362*** -0.0822*** -0.039 0.1332*** -0.0758*** -0.020 childbirth [0.0078] (0.019) (0.027) (0.040) [0.0084] (0.020) (0.027) (0.037) (0.015) (0.022) (0.030) Average 0.2484*** 0.1613** 0.2001** 0.2354*** 0.1976*** 0.1130* 0.2550*** 0.2013*** 0.1729*** standardized effect (0.036) (0.067) (0.079) (0.023) (0.062) (0.058) (0.026) (0.059) (0.059) Average standardized effect 0.2544*** 0.1141* 0.111 0.2885*** 0.019 -0.018 0.3314*** 0.017 0.026 health (0.036) (0.067) (0.075) (0.032) (0.070) (0.063) (0.039) (0.079) (0.089) Average standardized effect 0.2424*** 0.2085** 0.2898** 0.1823*** 0.3762*** 0.2444** 0.2092*** 0.3119*** 0.2611*** educ. (0.057) (0.102) (0.129) (0.029) (0.091) (0.099) (0.031) (0.078) (0.098) Average standardized effect total, health and education for average exclude received supplementary feeding at posyandu and received intensive supplementary feeding at school Table 9. Program impact on main targeted indicators Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Treatment Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Effect (1) (2) (3) (4) (5) (6) Main 12 indicators Number prenatal visits 7.447 7.464 0.032 7.639 0.061 0.047 [4.2935] [4.1639] (0.189) [4.2297] (0.169) (0.128) Delivery by trained midwife 0.670 0.755 0.037 0.780 0.001 0.018 [0.4705] [0.4303] (0.023) [0.4144] (0.019) (0.017) Number of postnatal visits 1.720 1.737 -0.121 1.634 -0.033 -0.075 [2.4477] [2.4079] (0.112) [2.4597] (0.109) (0.081) Iron tablet sachets 1.588 1.977 0.090 1.741 0.060 0.0746* [1.2554] [1.4426] (0.072) [1.2748] (0.053) (0.045) Percent of immunization 0.653 0.693 0.018 0.756 0.002 0.010 [0.3664] [0.3441] (0.015) [0.2858] (0.013) (0.010) Number of weight checks 2.126 2.192 0.1161*** 2.262 0.1872*** 0.1529*** [1.1895] [1.1718] (0.043) [1.1204] (0.047) (0.035) Number vitamin A supplements 1.529 1.560 -0.002 1.454 0.041 0.019 [1.1370] [1.0089] (0.045) [0.9520] (0.044) (0.033) Percent malnourished 0.168 0.199 -0.003 0.228 -0.0218* -0.013 [0.3739] [0.3995] (0.013) [0.4199] (0.013) (0.009) Age 7–12 participation rate 0.948 0.982 0.001 0.985 0.0082** 0.005 [0.2221] [0.1334] (0.005) [0.1207] (0.004) (0.003) Age 13–15 participation rate 0.822 0.906 -0.0424** 0.874 0.016 -0.010 [0.3827] [0.2928] (0.018) [0.3327] (0.015) (0.013) Age 7–12 gross attendance 0.904 0.956 0.002 0.960 0.003 0.003 [0.2773] [0.1568] (0.005) [0.1463] (0.005) (0.004) Age 13–15 gross attendance 0.768 0.884 -0.0529*** 0.860 0.020 -0.012 Additional Education Indicators [0.4125] [0.3022] (0.019) [0.3364] (0.016) (0.014) Age 13–15 conditional attendance 0.958 0.976 -0.0115** 0.984 0.005 -0.002 [0.1752] [0.1036] (0.005) [0.0832] (0.005) (0.004) Final Impact Evaluation Report Annexes 77 78 Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Treatment Annexes Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Effect (1) (2) (3) (4) (5) (6) Age 13–15 enrolled in SMP 0.593 0.672 -0.007 0.707 0.0397* 0.019 [0.4915] [0.4704] (0.028) [0.4561] (0.022) (0.019) Age 7–12 enrolled in SD 0.886 0.905 0.000 0.917 0.002 0.001 [0.3182] [0.2930] (0.008) [0.2757] (0.007) (0.006) Age 13–15 enrolled other than SMP 0.229 0.234 -0.038 0.167 -0.023 -0.0296* Indonesia’s PNPM Generasi Program [0.4205] [0.4240] (0.025) [0.3735] (0.019) (0.017) Kecamatan SD gross enrollment 0.931 0.943 -0.009 0.956 -0.010 -0.010 [0.1125] [0.1495] (0.014) [0.1496] (0.012) (0.010) Kecamatan SMP gross enrollment 0.869 0.997 -0.044 1.001 0.009 -0.015 [0.4593] [0.6691] (0.053) [0.6426] (0.049) (0.039) Average standardized effect -0.004 0.0407** 0.022 (0.019) (0.016) (0.014) Average standardized effect health 0.033 0.0380* 0.0357** (0.020) (0.020) (0.016) Average standardized effect educ. -0.0764** 0.0463* -0.005 (0.036) (0.025) (0.024) Table 10. Program impact on main targeted indicators, incentivized vs. non-incentivized Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Main 12 indicators Number prenatal visits 0.333 -0.280 0.6129*** 0.156 -0.025 0.181 0.235 -0.136 0.3706** (0.233) (0.200) (0.220) (0.192) (0.188) (0.173) (0.155) (0.139) (0.147) Delivery by trained 0.034 0.040 -0.005 0.011 -0.008 0.019 0.023 0.014 0.009 midwife (0.026) (0.026) (0.025) (0.021) (0.023) (0.021) (0.019) (0.019) (0.018) Number of postnatal -0.169 -0.065 -0.104 -0.034 -0.031 -0.003 -0.099 -0.051 -0.048 visits (0.140) (0.120) (0.140) (0.129) (0.124) (0.129) (0.099) (0.091) (0.101) Iron tablet sachets 0.129 0.050 0.078 0.076 0.045 0.031 0.1008* 0.049 0.052 (0.084) (0.081) (0.081) (0.058) (0.065) (0.063) (0.052) (0.052) (0.051) Percent of 0.025 0.010 0.015 0.010 -0.007 0.017 0.018 0.002 0.016 immunization (0.018) (0.018) (0.018) (0.015) (0.015) (0.014) (0.012) (0.011) (0.011) Number of weight 0.1633*** 0.068 0.0958* 0.1747*** 0.1983*** -0.024 0.1672*** 0.1394*** 0.028 checks (0.052) (0.049) (0.054) (0.055) (0.052) (0.051) (0.041) (0.039) (0.040) Number Vitamin A -0.008 0.005 -0.013 0.0847* 0.002 0.083 0.038 0.000 0.037 supplements (0.052) (0.055) (0.058) (0.048) (0.054) (0.053) (0.035) (0.040) (0.038) Percent malnourished -0.016 0.011 -0.0265* -0.017 -0.0262* 0.009 -0.016 -0.009 -0.006 (0.016) (0.015) (0.016) (0.014) (0.015) (0.016) (0.011) (0.011) (0.013) Age 7–12 -0.001 0.003 -0.004 0.005 0.0108*** -0.006 0.003 0.0075** -0.005 participation rate (0.005) (0.006) (0.006) (0.005) (0.004) (0.005) (0.004) (0.004) (0.004) Age 13–15 -0.0343* -0.0504** 0.016 0.020 0.013 0.007 -0.004 -0.015 0.011 participation rate (0.020) (0.023) (0.024) (0.017) (0.016) (0.014) (0.014) (0.016) (0.014) Age 7–12 gross 0.001 0.002 -0.001 0.003 0.004 -0.001 0.002 0.003 -0.001 Final Impact Evaluation Report attendance Annexes 79 80 Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Annexes Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (0.005) (0.005) (0.006) (0.007) (0.006) (0.006) (0.005) (0.004) (0.005) Age 13–15 gross -0.0405* -0.0651*** 0.025 0.025 0.016 0.010 -0.005 -0.020 0.015 attendance Additional Education (0.021) (0.024) (0.025) (0.018) (0.017) (0.015) (0.015) (0.016) (0.015) Indicators Age 13–15 -0.009 -0.0141** 0.005 0.008 0.003 0.005 0.000 -0.005 0.005 Indonesia’s PNPM Generasi Program conditional attendance (0.006) (0.006) (0.007) (0.006) (0.006) (0.005) (0.004) (0.004) (0.004) Age 13–15 enrolled 0.011 -0.024 0.035 0.0454* 0.035 0.011 0.030 0.010 0.020 in SMP (0.029) (0.034) (0.031) (0.026) (0.025) (0.024) (0.021) (0.023) (0.021) Age 7–12 enrolled -0.002 0.002 -0.004 0.007 -0.003 0.010 0.003 -0.001 0.004 in SD (0.010) (0.010) (0.010) (0.009) (0.008) (0.010) (0.007) (0.007) (0.008) Age 13–15 enrolled -0.048 -0.028 -0.020 -0.023 -0.024 0.001 -0.0332* -0.026 -0.007 other than SMP (0.030) (0.028) (0.029) (0.022) (0.022) (0.022) (0.020) (0.019) (0.020) Kecamatan SD gross -0.014 -0.005 -0.009 -0.005 -0.015 0.010 -0.008 -0.011 0.002 enrollment (0.017) (0.016) (0.016) (0.015) (0.014) (0.014) (0.012) (0.012) (0.012) Kecamatan SMP gross -0.016 -0.073 0.057 -0.023 0.039 -0.061 -0.022 -0.009 -0.012 enrollment (0.067) (0.056) (0.061) (0.055) (0.055) (0.046) (0.045) (0.042) (0.039) Average standardized 0.014 -0.022 0.036 0.0502** 0.0320* 0.018 0.0345** 0.010 0.025 effect (0.023) (0.021) (0.024) (0.020) (0.018) (0.019) (0.016) (0.016) (0.016) Average standardized 0.0526** 0.012 0.0406* 0.0515** 0.026 0.026 0.0516*** 0.020 0.0316* effect health (0.024) (0.023) (0.024) (0.023) (0.022) (0.022) (0.018) (0.018) (0.018) Average standardized -0.063 -0.0899** 0.027 0.048 0.0449* 0.003 0.000 -0.011 0.011 effect educ. (0.039) (0.045) (0.045) (0.029) (0.027) (0.027) (0.026) (0.029) (0.026) Table 11. Program impact on main targeted indicators, provincial breakdown     Wave II Wave III AVERAGE Additional Additional Additional Additional Additional Additional Java Java Java Baseline Control NTT Sulawesi Control NTT Sulawesi NTT Sulawesi Indicator Generasi Generasi Generasi Mean Mean Generasi Generasi Mean Generasi Generasi Generasi Generasi Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Main 12 indicators Number 7.447 7.464 0.274 -1.1338** 0.106 7.639 0.057 0.029 -0.011 0.164 -0.5534* 0.031 prenatal visits [4.2935] [0.1415] (0.226) (0.448) (0.564) [0.1449] (0.193) (0.493) (0.445) (0.150) (0.328) (0.382) Delivery by 0.670 0.755 0.0474* -0.096 0.083 0.780 0.021 -0.073 -0.027 0.0337* -0.0840* 0.021 trained midwife [0.4705] [0.0173] (0.024) (0.065) (0.054) [0.0164] (0.021) (0.060) (0.043) (0.018) (0.050) (0.039) Number of 1.720 1.737 -0.219 0.060 0.5783** 1.634 -0.138 0.278 0.268 -0.177 0.164 0.4096** postnatal visits [2.4477] [0.0969] (0.153) (0.239) (0.290) [0.0975] (0.148) (0.256) (0.249) (0.112) (0.177) (0.196) Iron tablet 1.588 1.977 0.133 -0.079 -0.178 1.741 0.085 -0.066 -0.068 0.1088* -0.072 -0.119 sachets [1.2554] [0.0493] (0.099) (0.151) (0.182) [0.0440] (0.067) (0.140) (0.141) (0.060) (0.103) (0.118) Percent of 0.653 0.693 -0.006 0.053 0.084 0.756 -0.010 0.036 0.029 -0.008 0.0454* 0.056 immunization [0.3664] [0.0127] (0.016) (0.035) (0.059) [0.0107] (0.014) (0.042) (0.040) (0.011) (0.025) (0.040) Number of 2.126 2.192 0.077 0.040 0.206 2.262 0.1480** 0.102 0.111 0.1140*** 0.066 0.157 weight checks [1.1895] [0.0402] (0.055) (0.087) (0.152) [0.0382] (0.057) (0.124) (0.135) (0.043) (0.079) (0.114) Number vitamin 1.529 1.560 0.047 -0.2363** 0.022 1.454 0.055 -0.083 0.029 0.050 -0.1618* 0.026 A supplements [1.1370] [0.0435] (0.054) (0.108) (0.134) [0.0412] (0.057) (0.115) (0.114) (0.040) (0.087) (0.078) Percent 0.168 0.199 0.007 -0.032 -0.012 0.228 -0.003 -0.0849*** -0.004 0.002 -0.0568** -0.008 malnourished [0.3739] [0.0138] (0.015) (0.035) (0.039) [0.0144] (0.015) (0.030) (0.039) (0.010) (0.023) (0.029) Age 7–12 participation 0.948 0.982 -0.001 0.014 -0.010 0.985 -0.004 0.0415*** 0.009 -0.003 0.0283*** 0.000 rate [0.2221] [0.0050] (0.004) (0.012) (0.015) [0.0046] (0.004) (0.008) (0.013) (0.003) (0.007) (0.011) Final Impact Evaluation Report Annexes 81 82 Annexes     Wave II Wave III AVERAGE Additional Additional Additional Additional Additional Additional Java Java Java Baseline Control NTT Sulawesi Control NTT Sulawesi NTT Sulawesi Indicator Generasi Generasi Generasi Mean Mean Generasi Generasi Mean Generasi Generasi Generasi Generasi Effect Effect Effect Effect Effect Effect Effect Effect Effect Age 13–15 participation 0.822 0.906 -0.027 -0.030 -0.052 0.874 0.023 -0.001 -0.037 0.001 -0.014 -0.043 rate [0.3827] [0.0187] (0.023) (0.040) (0.055) [0.0223] (0.018) (0.038) (0.047) (0.016) (0.033) (0.038) Indonesia’s PNPM Generasi Program Age 7–12 gross 0.904 0.956 0.000 0.014 -0.019 0.960 -0.0101* 0.0473*** 0.009 -0.006 0.0313*** -0.004 attendance [0.2773] [0.0059] (0.005) (0.011) (0.014) [0.0056] (0.006) (0.013) (0.016) (0.004) (0.008) (0.013) Age 13–15 gross 0.768 0.884 -0.035 -0.034 -0.062 0.860 0.030 -0.008 -0.048 0.001 -0.020 -0.053 attendance Additional Education [0.4125] [0.0193] (0.024) (0.041) (0.060) [0.0226] (0.019) (0.041) (0.046) (0.017) (0.034) (0.039) Indicators Age 13–15 conditional 0.958 0.976 -0.0111* 0.005 -0.013 0.984 0.008 -0.010 -0.007 0.000 -0.003 -0.009 attendance [0.1752] [0.0070] (0.006) (0.012) (0.017) [0.0060] (0.008) (0.011) (0.014) (0.005) (0.009) (0.011) Age 13–15 0.593 0.672 -0.022 0.055 0.014 0.707 0.014 0.097 0.015 -0.002 0.078 0.014 enrolled in SMP [0.4915] [0.0301] (0.035) (0.066) (0.079) [0.0306] (0.027) (0.060) (0.059) (0.023) (0.050) (0.052) Age 7–12 0.886 0.905 -0.004 0.028 -0.027 0.917 -0.003 0.0310* -0.021 -0.003 0.0293** -0.024 enrolled in SD [0.3182] [0.0109] (0.010) (0.019) (0.025) [0.0106] (0.009) (0.016) (0.020) (0.008) (0.013) (0.019) Age 13–15 enrolled other 0.229 0.234 -0.008 -0.078 -0.067 0.167 -0.001 -0.067 -0.036 -0.004 -0.0718* -0.049 than SMP [0.4205] [0.0271] (0.032) (0.056) (0.073) [0.0251] (0.022) (0.048) (0.054) (0.020) (0.039) (0.052)     Wave II Wave III AVERAGE Additional Additional Additional Additional Additional Additional Java Java Java Baseline Control NTT Sulawesi Control NTT Sulawesi NTT Sulawesi Indicator Generasi Generasi Generasi Mean Mean Generasi Generasi Mean Generasi Generasi Generasi Generasi Effect Effect Effect Effect Effect Effect Effect Effect Effect Kecamatan SD 0.931 0.943 -0.002 -0.017 -0.027 0.956 -0.004 -0.023 -0.010 -0.003 -0.020 -0.018 gross enrollment [0.1125] [0.0165] (0.018) (0.040) (0.034) [0.0165] (0.016) (0.027) (0.038) (0.013) (0.025) (0.030) Kecamatan SMP 0.869 0.997 -0.070 0.058 0.094 1.001 -0.049 0.2294** 0.098 -0.058 0.1444* 0.096 gross enrollment [0.4593] [0.0768] (0.071) (0.133) (0.138) [0.0742] (0.072) (0.098) (0.110) (0.054) (0.087) (0.091) Average standardized 0.003 -0.044 0.018 0.025 0.061 0.006 0.016 0.010 0.013 effect (0.022) (0.043) (0.061) (0.017) (0.046) (0.051) -0.015 -0.035 -0.044 Average standardized 0.032 -0.062 0.0989* 0.028 0.028 0.028 0.0303* -0.018 0.061 effect health (0.024) (0.047) (0.056) (0.022) (0.057) (0.056) -0.018 -0.041 -0.046 Average standardized -0.055 -0.008 -0.143 0.019 0.1294** -0.038 -0.012 0.067 -0.083 effect educ. (0.043) (0.078) (0.114) (0.030) (0.061) (0.071) -0.029 -0.058 -0.073 Final Impact Evaluation Report Annexes 83 84 Table 12. Program impact on longer term outcomes     Wave II (2008) Wave III (2009) AVERAGE Annexes Generasi Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Treatment Effect (1) (2) (3) (4) (5) (6) Malnourished (0–3 years) 0.168 0.199 -0.003 0.228 -0.0218* -0.013 [0.0059] [0.3995] (0.013) [0.4199] (0.013) (0.009) Severely malnourished (0–3 years) 0.047 0.057 -0.006 0.069 -0.015 -0.0105* [0.0033] [0.2317] (0.007) [0.2543] (0.009) (0.006) Wasting (0–3 years) 0.124 0.199 -0.001 -0.001 Indonesia’s PNPM Generasi Program [0.0055] [0.3995] (0.015) (0.015) Severe wasting (0–3 years) 0.048 0.089 0.003 0.003 [0.0036] [0.2849] (0.010) (0.010) Stunting (0–3 years) 0.383 0.358 0.0302* 0.0302* [0.0080] [0.4797] (0.017) (0.017) Severe stunting (0–3 years) 0.206 0.211 0.006 0.006 [0.0067] [0.4083] (0.017) (0.017) Diarrhea or ARI 0.355 0.327 -0.007 0.305 0.000 -0.004 [0.0075] [0.4694] (0.018) [0.4605] (0.020) (0.013) Mortality 0–28 days (births in past 18 months) 0.013 0.013 -0.0061** 0.008 -0.001 -0.003 [0.0021] [0.1129] (0.003) [0.0888] (0.004) (0.003) Mortality 0–12 months (births in past 24 0.024 0.035 -0.0087** 0.011 0.003 -0.003 months) [0.0026] [0.1842] (0.004) [0.1061] (0.005) (0.003) Mortality 0–12 months (births in past 12-24 0.028 0.055 -0.0146** 0.011 0.006 -0.005 months) [0.0042] [0.2279] (0.006) [0.1060] (0.009) (0.005) Mortality 0–12 months (births in past 21 0.023 0.030 -0.0112*** 0.011 0.003 -0.004 months) [0.0027] [0.1700] (0.004) [0.1043] (0.005) (0.004) Mortality 0–12 months (births in past 12-21 0.027 0.047 -0.0205*** 0.010 0.006 -0.008 months) [0.0047] [0.2130] (0.007) [0.1012] (0.009) (0.006) Home-based Bahasa test 7–12 years -0.039 -0.013 -0.023 -0.023 (age-adjusted Z-score) [0.0188] [1.0563] (0.041) (0.041)     Wave II (2008) Wave III (2009) AVERAGE Generasi Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Treatment Effect (1) (2) (3) (4) (5) (6) Home-based math test 7–12 years -0.032 -0.063 -0.012 -0.012 (age-adjusted Z-score) [0.0191] [1.0481] (0.043) (0.043) Home-based total test 7–12 years -0.044 -0.033 -0.015 -0.015 (age-adjusted Z-score) [0.0192] [1.0457] (0.042) (0.042) Home-based Bahasa test 13–15 years -0.008 -0.068 0.065 0.065 (age-adjusted Z-score) [0.0322] [1.1176] (0.068) (0.068) Home-based math test 13–15 years -0.001 0.004 0.044 0.044 (age-adjusted Z-score) [0.0320] [1.0395] (0.062) (0.062) Home-based total test 13–15 years -0.005 -0.014 0.053 0.053 (age-adjusted Z-score) [0.0321] [1.0401] (0.066) (0.066) Average standardized effect 0.0491*** -0.002 0.016 (0.019) (0.019) (0.016) Average standardized effect health 0.0491*** -0.011 0.016 (0.019) (0.022) (0.014) Average standardized effect educ. 0.017 0.017 (0.036) (0.036) Final Impact Evaluation Report Annexes 85 86 Table 13. Program impact on longer term outcomes, incentivized vs. non-incentivized   Wave II Wave III AVERAGE Annexes Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Malnourished -0.016 0.011 -0.0265* -0.017 -0.0262* 0.009 -0.016 -0.009 -0.006 (0–3 years) (0.016) (0.015) (0.016) (0.014) (0.015) (0.016) (0.011) (0.011) (0.013) Severely Indonesia’s PNPM Generasi Program malnourished -0.007 -0.005 -0.003 -0.016 -0.014 -0.002 -0.012 -0.009 -0.002 (0–3 years) (0.009) (0.008) (0.009) (0.010) (0.010) (0.009) (0.007) (0.007) (0.007) Wasting (0–3 -0.005 0.003 -0.008 -0.005 0.003 -0.008 years) (0.017) (0.016) (0.015) (0.017) (0.016) (0.015) Severe wasting 0.000 0.006 -0.006 0.000 0.006 -0.006 (0–3 years) (0.011) (0.012) (0.012) (0.011) (0.012) (0.012) Stunting (0–3 0.0335* 0.027 0.006 0.0335* 0.027 0.006 years) (0.020) (0.020) (0.021) (0.020) (0.020) (0.021) Severe stunting -0.007 0.019 -0.026 -0.007 0.019 -0.026 (0–3 years) (0.019) (0.019) (0.018) (0.019) (0.019) (0.018) Diarrhea or ARI -0.026 0.012 -0.038 0.003 -0.003 0.006 -0.010 0.003 -0.013 (0.023) (0.020) (0.024) (0.023) (0.022) (0.019) (0.017) (0.015) (0.016) Mortality 0–28 days (births in -0.0063* -0.006 0.000 0.006 -0.0076* 0.0138*** 0.000 -0.0073*** 0.0076** past 18 months) (0.003) (0.004) (0.003) (0.005) (0.004) (0.004) (0.003) (0.003) (0.003) Mortality 0–12 months (births in -0.006 -0.0112** 0.005 0.0115** -0.004 0.0155*** 0.003 -0.0078** 0.0105*** past 24 months) (0.005) (0.005) (0.005) (0.006) (0.005) (0.006) (0.004) (0.004) (0.004)   Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Mortality 0–12 months (births -0.012 -0.0182** 0.007 0.014 -0.001 0.014 0.000 -0.010 0.010 in past 12-24 months) (0.007) (0.007) (0.007) (0.011) (0.010) (0.011) (0.006) (0.006) (0.007) Mortality 0–12 months (births in -0.0116** -0.0108** -0.001 0.0114* -0.005 0.0165*** 0.000 -0.0083** 0.0086** past 21 months) (0.005) (0.005) (0.005) (0.006) (0.005) (0.005) (0.004) (0.004) (0.004) Mortality 0–12 months (births -0.0216*** -0.0192** -0.002 0.015 -0.002 0.017 -0.005 -0.0114* 0.007 in past 12–21 months) (0.007) (0.008) (0.007) (0.012) (0.010) (0.012) (0.007) (0.007) (0.007) Home-based Bahasa test 7–12 -0.048 -0.001 -0.046 -0.048 -0.001 -0.046 years (age-adjusted (0.048) (0.044) (0.044) (0.048) (0.044) (0.044) Z-score) Home-based math -0.027 0.001 -0.028 -0.027 0.001 -0.028 test 7–12 years (age-adjusted (0.049) (0.049) (0.048) (0.049) (0.049) (0.048) Z-score) Home-based total -0.042 0.010 -0.052 -0.042 0.010 -0.052 test 7–12 years (age-adjusted (0.049) (0.047) (0.046) (0.049) (0.047) (0.046) Z-score) Home-based Bahasa test 13–15 0.034 0.094 -0.060 0.034 0.094 -0.060 yrs. Final Impact Evaluation Report (age-adjusted (0.071) (0.078) (0.061) (0.071) (0.078) (0.061) Annexes Z-score) 87 88   Wave II Wave III AVERAGE Versi A Versi B Versi A Annexes Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Home-based math -0.002 0.085 -0.087 -0.002 0.085 -0.087 test 13–15 years (age-adjusted (0.068) (0.071) (0.063) (0.068) (0.071) (0.063) Z-score) Home-based total Indonesia’s PNPM Generasi Program 0.012 0.088 -0.076 0.012 0.088 -0.076 test 13–15 years (age-adjusted (0.071) (0.076) (0.064) (0.071) (0.076) (0.064) Z-score) Average 0.0598*** 0.0381* 0.022 -0.031 0.024 -0.0549** 6.6-0.001 0.0326* -0.0332* standardized effect (0.022) (0.020) (0.021) (0.022) (0.021) (0.022) (0.019) (0.018) (0.017) Average standardized effect 0.0598*** 0.0381* 0.022 -0.040 0.015 -0.0546** 0.005 0.0273* -0.022 health (0.022) (0.020) (0.021) (0.027) (0.024) (0.026) (0.018) (0.016) (0.018) Average standardized effect -0.012 0.043 -0.056 -0.012 0.043 -0.056 educ. (0.039) (0.042) (0.037) (0.039) (0.042) (0.037) Table 14. Program impact on longer term outcomes, provincial breakdown     Wave II Wave III AVERAGE Additional Additional Additional Additional Additional Additional Java Java Java Baseline Control NTT Sulawesi Control NTT Sulawesi NTT Sulawesi Indicator Generasi Generasi Generasi Mean Mean Generasi Generasi Mean Generasi Generasi Generasi Generasi Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Malnourished 0.168 0.199 0.007 -0.032 -0.012 0.228 -0.003 -0.0849*** -0.004 0.002 -0.0568** -0.008 (0–3 years) [0.0059] [0.0138] (0.015) (0.035) (0.039) [0.0144] (0.015) (0.030) (0.039) (0.010) (0.023) (0.029) Severely malnourished 0.047 0.057 0.002 -0.008 -0.0408* 0.069 0.000 -0.0543** -0.021 0.001 -0.0299* -0.030 (0–3 years) [0.0033] [0.0080] (0.007) (0.020) (0.023) [0.0087] (0.009) (0.028) (0.028) (0.006) (0.017) (0.021) Stunting (0–3 0.383 0.358 0.0512** -0.0711* -0.038 0.0512** -0.0711* -0.038 years) [0.0080] [0.0174] (0.022) (0.041) (0.049) (0.022) (0.041) (0.049) Severe stunting 0.206 0.211 0.0360* -0.0989*** -0.056 0.0360* -0.0989*** -0.056 (0–3 years) [0.0067] [0.0148] (0.022) (0.038) (0.048) (0.022) (0.038) (0.048) Wasting (0–3 0.124 0.199 -0.013 0.004 0.0706* -0.013 0.004 0.0706* years) [0.0055] [0.0145] (0.017) (0.040) (0.040) (0.017) (0.040) (0.040) Severe wasting 0.048 0.089 -0.003 0.007 0.030 -0.003 0.007 0.030 (0–3 years) [0.0036] [0.0103] (0.013) (0.028) (0.025) (0.013) (0.028) (0.025) Diarrhea or 0.355 0.327 -0.014 0.029 -0.005 0.305 0.001 -0.035 0.039 -0.006 -0.002 0.019 ARI [0.0075] [0.0161] (0.024) (0.042) (0.048) [0.0157] (0.025) (0.055) (0.049) (0.016) (0.036) (0.036) Mortality 0–28 days (births 0.013 0.013 -0.003 -0.013 -0.002 0.008 0.001 -0.008 0.002 -0.001 -0.011 0.001 in past 18 months) [0.0021] [0.0045] (0.004) (0.008) (0.009) [0.0035] (0.005) (0.013) (0.009) (0.003) (0.007) (0.007) Mortality Final Impact Evaluation Report 0–12 months Annexes 0.024 0.035 -0.003 -0.006 -0.0352** 0.011 0.002 0.001 0.004 0.000 -0.003 -0.014 (births past 24 months) 89 90     Wave II Wave III AVERAGE Additional Additional Additional Additional Additional Additional Java Java Java Baseline Control NTT Sulawesi Control NTT Sulawesi NTT Sulawesi Annexes Indicator Generasi Generasi Generasi Mean Mean Generasi Generasi Mean Generasi Generasi Generasi Generasi Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) [0.0026] [0.0065] (0.004) (0.012) (0.017) [0.0038] (0.004) (0.015) (0.019) (0.003) (0.010) (0.014) Mortality 0–12 months (births 0.028 0.055 -0.006 -0.013 -0.038 0.011 0.006 -0.010 0.018 -0.001 -0.011 -0.011 past 12-24 months) Indonesia’s PNPM Generasi Program [0.0042] [0.0119] (0.005) (0.020) (0.025) [0.0056] (0.005) (0.028) (0.029) (0.004) (0.017) (0.021) Mortality 0–12 months 0.023 0.030 -0.004 -0.008 -0.0371** 0.011 0.003 0.002 -0.002 -0.001 -0.004 -0.018 (births past 21 months) [0.0027] [0.0064] (0.004) (0.012) (0.016) [0.0039] (0.005) (0.016) (0.018) (0.003) (0.011) (0.013) Mortality 0–12 months (births 0.027 0.047 -0.0104* -0.019 -0.0407** 0.010 0.009 -0.012 0.000 -0.002 -0.014 -0.020 past 12-21 months) [0.0047] [0.0129] (0.006) (0.023) (0.021) [0.0059] (0.007) (0.030) (0.030) (0.005) (0.019) (0.019) Home-based Bahasa test -0.039 -0.013 -0.050 0.097 0.038 -0.050 0.097 0.038 7–12 years (age-adjusted [0.0188] [0.0436] (0.051) (0.105) (0.107) (0.051) (0.105) (0.107) Z-score) Home-based math test 7–12 -0.032 -0.063 -0.015 -0.061 0.117 -0.015 -0.061 0.117 years (age-adjusted [0.0191] [0.0453] (0.053) (0.114) (0.113) (0.053) (0.114) (0.113) Z-score) Home-based total test 7–12 -0.044 -0.033 -0.029 0.002 0.096 -0.029 0.002 0.096 years     Wave II Wave III AVERAGE Additional Additional Additional Additional Additional Additional Java Java Java Baseline Control NTT Sulawesi Control NTT Sulawesi NTT Sulawesi Indicator Generasi Generasi Generasi Mean Mean Generasi Generasi Mean Generasi Generasi Generasi Generasi Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) (age-adjusted [0.0192] [0.0453] (0.054) (0.104) (0.101) (0.054) (0.104) (0.101) Z-score) Home-based Bahasa test -0.008 -0.068 0.075 0.110 -0.200 0.075 0.110 -0.200 13–15 years (age-adjusted [0.0322] [0.0790] (0.086) (0.185) (0.143) (0.086) (0.185) (0.143) Z-score) Home-based math test -0.001 0.004 0.025 0.054 0.050 0.025 0.054 0.050 13–15 years (age-adjusted [0.0320] [0.0754] (0.076) (0.140) (0.202) (0.076) (0.140) (0.202) Z-score) Home-based total test 13– -0.005 -0.014 0.043 0.088 -0.047 0.043 0.088 -0.047 15 years (age-adjusted [0.0321] [0.0755] (0.082) (0.161) (0.201) (0.082) (0.161) (0.201) Z-score) Average standardized 0.018 0.050 0.1344** -0.015 0.069 -0.010 -0.003 0.065 0.033 effect (0.020) (0.051) (0.062) (0.021) (0.053) (0.049) (0.020) (0.043) (0.041) Average standardized 0.018 0.050 0.1344** -0.025 0.081 -0.021 -0.007 0.0752* 0.044 effect health (0.020) (0.051) (0.062) (0.017) (0.069) (0.069) (0.013) (0.044) (0.049) Average standardized 0.007 0.045 0.012 0.007 0.045 0.012 Final Impact Evaluation Report effect educ. Annexes (0.046) (0.090) (0.100) (0.046) (0.090) (0.100) 91 92 Table 15. Program impact on non-targeted indicators     Wave II (2008) Wave III (2009) AVERAGE Annexes Generasi Generasi Average Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Treatment Effect (1) (2) (3) (4) (5) (6) Quality of prenatal care services 0.546 0.577 0.011 0.600 0.012 0.011 [0.244] [0.2465] (0.011) [0.2415] (0.011) (0.008) Facility-based vs home deliveries 0.397 0.441 0.031 0.473 0.0366* 0.0337** [0.490] [0.4969] (0.020) [0.4996] (0.021) (0.017) Indonesia’s PNPM Generasi Program Use of family planning 0.528 0.532 0.001 0.566 -0.0232** -0.012 [0.499] [0.4992] (0.014) [0.4958] (0.012) (0.010) Use of health services curative care . 0.587 -0.005 0.573 -0.006 -0.005 [.] [0.4411] (0.016) [0.4526] (0.018) (0.012) Quality of posyandu . 0.330 0.0576*** 0.378 0.016 0.0361** [.] [0.4706] (0.022) [0.4853] (0.019) (0.015) Initiation of breastfeeding 0.575 0.632 0.016 0.646 0.027 0.022 [0.494] [0.4826] (0.019) [0.4784] (0.019) (0.015) Exclusive breastfeeding 0.472 0.401 0.019 0.405 0.000 0.009 [0.499] [0.4904] (0.022) [0.4912] (0.020) (0.015) Mother’s knowledge . 0.602 0.006 0.581 0.0167** 0.0117** [.] [0.1879] (0.008) [0.1814] (0.006) (0.005) Age 7–15 hours wage work 0.431 0.156 0.1794* 0.245 0.005 0.084 [3.869] [1.4714] (0.099) [2.9462] (0.067) (0.057) Age 7–15 hours household work 3.915 3.287 0.6686*** 2.885 -0.155 0.219 [6.761] [5.6108] (0.223) [5.6129] (0.175) (0.141) Age 7–15 wage work dummy 0.031 0.021 0.006 0.028 0.000 0.003 [0.174] [0.1428] (0.005) [0.1646] (0.005) (0.004) Age 7–15 household work dummy 0.728 0.673 0.008 0.671 -0.008 -0.001 [0.445] [0.4693] (0.014) [0.4702] (0.017) (0.012) Gross high school enrollment . 0.383 -0.027 0.414 -0.038 -0.033 [.] [0.4873] (0.037) [0.4936] (0.029) (0.027) Dropout rates 0.026 0.011 0.0069* 0.013 -0.001 0.002     Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Treatment Effect (1) (2) (3) (4) (5) (6) [0.158] [0.1026] (0.004) [0.1114] (0.003) (0.003) SD to SMP transition 0.885 0.976 -0.005 0.948 0.013 0.005 [0.319] [0.1534] (0.011) [0.2234] (0.017) (0.011) SD to SMP transition alt. def. 0.869 0.954 0.000 0.948 0.003 0.002 [0.338] [0.2118] (0.014) [0.2234] (0.017) (0.012) Number of hours attended school 17.112 25.882 -0.7041** 24.459 -0.006 -0.323 [13.674] [7.9529] (0.310) [8.3715] (0.398) (0.281) Number hrs school (enroll only) 18.401 26.776 -0.5339** 25.453 -0.351 -0.4337* [13.316] [6.4391] (0.235) [6.8991] (0.411) (0.257) Numbers attend Paket A . 0.001 -0.001 -0.001 [.] [0.0348] (0.001) (0.001) Numbers attend Paket B . 0.002 0.000 0.000 [.] [0.0425] (0.001) (0.001) Numbers attend Paket C . 0.000 0.000 0.000 [.] [0.0000] (0.001) (0.001) Fertility rate 0.344 0.326 -0.009 0.313 0.007 -0.001 [0.475] [0.4691] (0.010) [0.4640] (0.009) (0.007) Number migrate out village 12 . 13.858 -0.173 -0.173 mons [.] [28.5038] (1.463) (1.463) Number HH indivs. migrate out 0.069 0.049 -0.011 0.045 0.000 -0.005 kec 12 mons [0.314] [0.2539] (0.010) [0.2327] (0.012) (0.008) Excess prenatal visits 0.737 0.760 -0.006 0.763 0.024 0.009 [0.440] [0.4274] (0.018) [0.4255] (0.017) (0.013) Excess vitamin A 0.043 0.057 -0.003 0.059 0.007 0.002 [0.204] [0.2314] (0.009) [0.2366] (0.009) (0.007) Final Impact Evaluation Report Annexes Distance to SMP (km) 11.439 2.883 -0.045 2.529 0.039 0.002 93 94     Wave II (2008) Wave III (2009) AVERAGE Annexes Generasi Generasi Average Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Treatment Effect (1) (2) (3) (4) (5) (6) [20.544] [2.7078] (0.198) [3.2733] (0.175) (0.145) Time spent oneway to SMP (hr) 0.397 0.375 0.0377** 0.408 -0.004 0.014 [0.350] [0.2929] (0.018) [0.3342] (0.017) (0.014) Transportation cost oneway to 777.527 867.051 87.750 863.590 -110.974 -23.434 SMP Indonesia’s PNPM Generasi Program [1421.307] [1662.9639] (110.026) [1786.8816] (139.109) (93.318) Woman role in child decisions 0.773 0.757 -0.001 0.749 0.002 0.001 dummy [0.419] [0.4292] (0.015) [0.4341] (0.015) (0.011) Woman role in purchase decisions 0.289 0.294 -0.003 0.270 -0.016 -0.010 dummy [0.453] [0.4557] (0.018) [0.4440] (0.015) (0.012) Average standardized effect -0.022 0.019 0.005 (0.018) (0.012) (0.010) Average standardized effect health 0.0392** 0.0339* 0.0362*** (0.018) (0.018) (0.014) Average standardized effect educ. -0.0827*** 0.004 -0.0268** (0.030) (0.015) (0.014) Table 16. Program impact on main targeted indicators, interactions with pre-period subdistrict level variables     Wave II Wave III AVERAGE Interaction Generasi Interaction Generasi Interaction Generasi Baseline Control Generasi Control Generasi Generasi Indicator with Pre- at 10th with Pre- at 10th with Pre- at 10th Mean Mean Effect Mean Effect Effect Period Level Percentile Period Level Percentile Period Level Percentile (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Main 12 indicators Number prenatal visits 7.447 7.464 -0.939 0.134 -0.329 7.639 0.777 -0.096 0.340 -0.115 0.022 -0.015 [4.2935] [4.1639] (0.787) (0.107) (0.336) [4.2297] (0.552) (0.069) (0.269) (0.482) (0.064) (0.216) Delivery by trained 0.670 0.755 0.055 -0.028 0.049 0.780 0.002 -0.002 0.002 0.030 -0.018 0.026 midwife [0.4705] [0.4303] (0.054) (0.062) (0.043) [0.4144] (0.055) (0.065) (0.042) (0.043) (0.049) (0.034) Number of postnatal 1.720 1.737 0.034 -0.094 -0.004 1.634 0.215 -0.149 0.155 0.120 -0.118 0.073 visits [2.4477] [2.4079] (0.188) (0.107) (0.155) [2.4597] (0.186) (0.114) (0.150) (0.143) (0.083) (0.116) Iron tablet sachets 1.588 1.977 0.3564* -0.169 0.1875* 1.741 -0.068 0.081 0.014 0.159 -0.054 0.1053* [1.2554] [1.4426] (0.186) (0.104) (0.099) [1.2748] (0.179) (0.111) (0.080) (0.126) (0.075) (0.063) Percent of immunization 0.653 0.693 0.1087** -0.1425** 0.0567** 0.756 -0.027 0.044 -0.012 0.052 -0.065 0.029 [0.3664] [0.3441] (0.051) (0.070) (0.027) [0.2858] (0.051) (0.069) (0.027) (0.039) (0.054) (0.021) Number of weight checks 2.126 2.192 0.286 -0.081 0.1750** 2.262 0.259 -0.034 0.2121** 0.2788** -0.060 0.1965*** [1.1895] [1.1718] (0.192) (0.084) (0.083) [1.1204] (0.205) (0.092) (0.086) (0.141) (0.061) (0.062) Number Vitamin A 1.529 1.560 0.040 -0.028 0.012 1.454 0.065 -0.016 0.049 0.053 -0.022 0.030 supplements [1.1370] [1.0089] (0.142) (0.093) (0.062) [0.9520] (0.163) (0.106) (0.067) (0.113) (0.073) (0.048) Percent malnourished 0.168 0.199 0.031 -0.1798* -0.029 0.228 0.022 -0.2521** -0.0619*** 0.0253* -0.2114*** -0.0452*** [0.3739] [0.3995] (0.020) (0.097) (0.021) [0.4199] (0.021) (0.106) (0.022) (0.015) (0.070) (0.015) Age 7–12 participation 0.948 0.982 0.065 -0.067 0.006 0.985 0.1467** -0.1455* 0.0194** 0.1035* -0.1036* 0.0129** rate [0.2221] [0.1334] (0.074) (0.076) (0.009) [0.1207] (0.073) (0.075) (0.008) (0.056) (0.058) (0.006) Age 13–15 participation 0.822 0.906 0.015 -0.069 -0.027 0.874 0.066 -0.061 0.029 0.049 -0.072 0.006 rate [0.3827] [0.2928] (0.081) (0.096) (0.028) [0.3327] (0.079) (0.092) (0.027) (0.063) (0.074) (0.022) Age 7–12 gross 0.904 0.956 0.045 -0.048 0.008 0.960 0.1002** -0.1072** 0.0177** 0.0682*** -0.0730** 0.0120** attendance [0.2773] [0.1568] (0.032) (0.035) (0.006) [0.1463] (0.042) (0.046) (0.008) (0.026) (0.029) (0.005) Age 13–15 gross Final Impact Evaluation Report 0.768 0.884 -0.031 -0.029 -0.045 0.860 0.077 -0.076 0.039 0.033 -0.060 0.003 attendance Annexes 95 96     Wave II Wave III AVERAGE Interaction Generasi Interaction Generasi Interaction Generasi Annexes Baseline Control Generasi Control Generasi Generasi Indicator with Pre- at 10th with Pre- at 10th with Pre- at 10th Mean Mean Effect Mean Effect Effect Period Level Percentile Period Level Percentile Period Level Percentile Additional Education [0.4125] [0.3022] (0.071) (0.088) (0.030) [0.3364] (0.063) (0.078) (0.027) (0.051) (0.063) (0.022) Indicators Age 13–15 conditional 0.958 0.976 -0.099 0.092 -0.0222** 0.984 -0.004 0.009 0.004 -0.045 0.045 -0.008 attendance [0.1752] [0.1036] (0.071) (0.074) (0.011) [0.0832] (0.043) (0.045) (0.007) (0.045) (0.047) (0.007) Age 13–15 enrolled in 0.593 0.672 -0.019 0.021 -0.013 0.707 0.1228** -0.1408* 0.0826** 0.063 -0.072 0.042 SMP Indonesia’s PNPM Generasi Program [0.4915] [0.4704] (0.074) (0.105) (0.047) [0.4561] (0.058) (0.085) (0.037) (0.055) (0.077) (0.035) Age 7–12 enrolled in SD 0.886 0.905 -0.042 0.048 -0.005 0.917 -0.046 0.054 -0.004 -0.044 0.051 -0.005 [0.3182] [0.2930] (0.090) (0.100) (0.015) [0.2757] (0.086) (0.097) (0.013) (0.066) (0.074) (0.011) Age 13–15 enrolled other 0.229 0.234 -0.048 0.045 -0.048 0.167 0.042 -0.2947** 0.042 -0.001 -0.131 -0.001 than SMP [0.4205] [0.4240] (0.032) (0.108) (0.032) [0.3735] (0.027) (0.116) (0.027) (0.021) (0.088) (0.021) Kecamatan SD gross 0.931 0.943 -0.032 0.003 -0.030 0.956 0.033 -0.006 0.029 0.004 -0.002 0.003 enrollment [0.1125] [0.1495] (0.035) (0.004) (0.032) [0.1496] (0.031) (0.004) (0.028) (0.026) (0.003) (0.024) Kecamatan SMP gross 0.869 0.997 0.078 -0.191 0.014 1.001 -0.009 0.027 0.001 0.032 -0.073 0.008 enrollment [0.4593] [0.6691] (0.113) (0.131) (0.076) [0.6426] (0.080) (0.110) (0.054) (0.070) (0.088) (0.048) Average standardized 0.113 -0.1858** 0.034 0.1954*** -0.2082*** 0.0771*** 0.1637*** -0.2048*** 0.0618*** effect (0.082) (0.083) (0.030) (0.071) (0.077) (0.027) (0.056) (0.058) (0.021) Average standardized 0.085 -0.1419*** 0.0698** 0.045 -0.062 0.0609* 0.0746* -0.1123*** 0.0694*** effect health (0.054) (0.048) (0.031) (0.055) (0.057) (0.033) (0.041) (0.038) (0.025) Average standardized 0.168 -0.274 -0.039 0.4963*** -0.5003** 0.1094** 0.3421** -0.3896** 0.047 effect educ. (0.205) (0.224) (0.056) (0.185) (0.203) (0.043) (0.145) (0.159) (0.038) Table 17. Program impact on main targeted indicators, interactions with pre-period subdistrict-level variables, incentivized vs. non-incentivized   Wave II Wave III AVERAGE Generasi Generasi Generasi Generasi Generasi Versi A Generasi Generasi Versi A Generasi Generasi Versi A Versi A Versi A Versi A Versi A Versi B Additonal Versi A Versi B Additonal Versi A Versi B Additonal Additional Additional Additional Indicator Total Effect Total Effect Effect Total Effect Total Effect Effect Total Effect Total Effect Effect Effect * Effect * Effect * * Pre-Period * Pre-Period at 10th * Pre-Period * Pre-Period at 10th * Pre-Period * Pre-Period at 10th Pre-Period Pre-Period Pre-Period Level Level Percentile Level Level Percentile Level Level Percentile Level Level Level (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Main 12 indicators Number prenatal 0.167 0.115 0.052 0.490 -0.045 -0.1456* 0.101 -0.095 0.065 -0.015 0.080 0.158 visits (0.131) (0.117) (0.133) (0.365) (0.080) (0.082) (0.083) (0.284) (0.080) (0.071) (0.082) (0.253) Delivery by trained -0.088 0.042 -0.1296* 0.050 -0.056 0.049 -0.105 0.065 -0.074 0.040 -0.1143** 0.0588* midwife (0.074) (0.070) (0.071) (0.046) (0.071) (0.076) (0.068) (0.042) (0.055) (0.056) (0.053) (0.034) Number of postnatal -0.141 -0.039 -0.102 0.012 -0.2502* -0.061 -0.189 0.221 -0.1904** -0.043 -0.147 0.123 visits (0.126) (0.130) (0.142) (0.193) (0.137) (0.128) (0.136) (0.182) (0.094) (0.101) (0.103) (0.142) Iron tablet sachets -0.142 -0.206 0.064 0.044 0.009 0.160 -0.151 0.116 -0.082 -0.020 -0.061 0.086 (0.126) (0.130) (0.152) (0.111) (0.124) (0.143) (0.153) (0.097) (0.093) (0.093) (0.111) (0.074) Percent of -0.1884** -0.086 -0.102 0.041 0.016 0.074 -0.057 0.033 -0.102 -0.021 -0.080 0.0376* immunization (0.085) (0.078) (0.087) (0.029) (0.079) (0.073) (0.066) (0.025) (0.062) (0.057) (0.056) (0.020) Number of weight -0.071 -0.086 0.016 0.083 -0.065 0.000 -0.065 0.022 -0.069 -0.043 -0.025 0.045 checks (0.098) (0.099) (0.106) (0.108) (0.110) (0.115) (0.129) (0.120) (0.069) (0.086) (0.094) (0.093) Number Vitamin A -0.030 -0.024 -0.007 -0.008 -0.001 -0.044 0.043 0.060 -0.013 -0.037 0.024 0.026 supplements (0.128) (0.115) (0.154) (0.096) (0.115) (0.130) (0.133) (0.085) (0.085) (0.093) (0.106) (0.065) Percent malnourished -0.2564** -0.100 -0.156 -0.0481* -0.2677** -0.2400** -0.028 0.006 -0.2591*** -0.1657** -0.093 -0.020 (0.129) (0.113) (0.138) (0.027) (0.132) (0.116) (0.128) (0.027) (0.095) (0.078) (0.100) (0.021) Age 7–12 -0.042 -0.087 0.045 -0.007 -0.114 -0.1800** 0.066 -0.011 -0.074 -0.129 0.055 -0.009 participation rate (0.090) (0.106) (0.127) (0.012) (0.094) (0.081) (0.098) (0.010) (0.066) (0.080) (0.091) (0.009) Final Impact Evaluation Report Annexes 97 98   Wave II Wave III AVERAGE Generasi Generasi Generasi Annexes Generasi Generasi Versi A Generasi Generasi Versi A Generasi Generasi Versi A Versi A Versi A Versi A Versi A Versi B Additonal Versi A Versi B Additonal Versi A Versi B Additonal Additional Additional Additional Indicator Total Effect Total Effect Effect Total Effect Total Effect Effect Total Effect Total Effect Effect Effect * Effect * Effect * * Pre-Period * Pre-Period at 10th * Pre-Period * Pre-Period at 10th * Pre-Period * Pre-Period at 10th Pre-Period Pre-Period Pre-Period Level Level Percentile Level Level Percentile Level Level Percentile Level Level Level (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Age 13–15 -0.063 -0.079 0.016 0.013 -0.006 -0.115 0.110 -0.021 -0.036 -0.100 0.065 -0.006 participation rate (0.120) (0.121) (0.149) (0.044) (0.109) (0.098) (0.101) (0.028) (0.085) (0.090) (0.098) (0.028) Indonesia’s PNPM Generasi Program Age 7–12 gross -0.051 -0.045 -0.006 0.000 -0.1064** -0.1085** 0.002 -0.001 -0.0738** -0.0721** -0.002 -0.001 attendance (0.039) (0.039) (0.033) (0.006) (0.050) (0.049) (0.034) (0.007) (0.032) (0.031) (0.027) (0.005) Age 13–15 gross -0.052 -0.033 -0.019 0.031 -0.022 -0.110 0.087 -0.016 -0.037 -0.077 0.040 0.004 attendance Additional Education (0.111) (0.109) (0.133) (0.048) (0.108) (0.078) (0.099) (0.032) (0.080) (0.073) (0.087) (0.029) Indicators Age 13–15 conditional 0.156 0.013 0.142 -0.013 0.034 -0.016 0.050 -0.002 0.092 -0.004 0.0962* -0.008 attendance (0.097) (0.063) (0.107) (0.015) (0.055) (0.048) (0.051) (0.008) (0.056) (0.041) (0.052) (0.008) Age 13–15 enrolled -0.166 0.151 -0.3171** 0.1389** -0.1762* -0.111 -0.065 0.032 -0.1595* -0.005 -0.154 0.072 in SMP (0.121) (0.134) (0.154) (0.061) (0.104) (0.098) (0.109) (0.047) (0.084) (0.095) (0.099) (0.044) Age 7–12 enrolled 0.2202** -0.1891* 0.4093*** -0.0509*** 0.056 0.053 0.003 0.009 0.127 -0.053 0.1799* -0.017 in SD (0.106) (0.113) (0.113) (0.014) (0.111) (0.126) (0.134) (0.017) (0.081) (0.098) (0.104) (0.013) Age 13–15 enrolled -0.004 0.078 -0.082 0.000 -0.3051** -0.2881** -0.017 0.001 -0.153 -0.117 -0.036 -0.001 other than SMP (0.149) (0.125) (0.168) (0.046) (0.147) (0.119) (0.124) (0.033) (0.111) (0.094) (0.104) (0.028) Kecamatan SD gross 0.006 -0.001 0.007 -0.054 0.038 0.026 0.013 0.012 -0.001 -0.003 0.003 -0.015 enrollment (0.006) (0.005) (0.007) (0.051) (0.043) (0.036) (0.049) (0.044) (0.005) (0.004) (0.006) (0.038)   Wave II Wave III AVERAGE Generasi Generasi Generasi Generasi Generasi Versi A Generasi Generasi Versi A Generasi Generasi Versi A Versi A Versi A Versi A Versi A Versi B Additonal Versi A Versi B Additonal Versi A Versi B Additonal Additional Additional Additional Indicator Total Effect Total Effect Effect Total Effect Total Effect Effect Total Effect Total Effect Effect Effect * Effect * Effect * * Pre-Period * Pre-Period at 10th * Pre-Period * Pre-Period at 10th * Pre-Period * Pre-Period at 10th Pre-Period Pre-Period Pre-Period Level Level Percentile Level Level Percentile Level Level Percentile Level Level Level (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Kecamatan SMP gross -0.303 -0.087 -0.216 0.126 0.041 -0.054 0.095 0.009 -0.1912* 0.044 -0.2355* 0.060 enrollment (0.197) (0.141) (0.223) (0.101) (0.086) (0.100) (0.094) (0.052) (0.108) (0.112) (0.132) (0.054) Average standardized -0.2140** -0.157 -0.057 0.056 -0.1919** -0.2225*** 0.031 0.025 -0.2055*** -0.1957** -0.010 0.037 effect (0.096) (0.112) (0.133) (0.042) (0.091) (0.085) (0.088) (0.033) (0.065) (0.079) (0.087) (0.029) Average standardized -0.1975*** -0.078 -0.1193* 0.0706* -0.106 -0.020 -0.086 0.061 -0.1615*** -0.059 -0.1022** 0.0638** effect health (0.061) (0.056) (0.068) (0.037) (0.066) (0.066) (0.064) (0.039) (0.047) (0.045) (0.051) (0.031) Average standardized -0.247 -0.314 0.067 0.026 -0.363 -0.6274*** 0.264 -0.048 -0.2934* -0.4685** 0.175 -0.017 effect educ. (0.253) (0.312) (0.369) (0.086) (0.244) (0.219) (0.235) (0.050) (0.174) (0.221) (0.241) (0.051) Final Impact Evaluation Report Annexes 99 100 Table 18. Program impact on longer-term outcomes, interactions with pre-period subdistrict level variables   Wave II Wave III AVERAGE Annexes Interaction Interaction Interaction Generasi Generasi Generasi Baseline Control Generasi with Pre- Control Generasi with Pre- Generasi with Pre- Indicator at 10th at 10th at 10th Mean Mean Effect Period Mean Effect Period Effect Period Percentile Percentile Percentile Level Level Level (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Malnourished 0.168 0.199 0.031 -0.1798* -0.029 0.228 0.022 -0.2521** -0.0619*** 0.0253* -0.2114*** -0.0452*** (0–3 years) [0.0059] [0.3995] (0.020) (0.097) (0.021) [0.4199] (0.021) (0.106) (0.022) (0.015) (0.070) (0.015) Indonesia’s PNPM Generasi Program Severely malnourished 0.047 0.057 0.000 -0.118 -0.015 0.069 -0.0296*** 0.3097** 0.011 -0.0147** 0.084 -0.004 (0–3 years) [0.0033] [0.2317] (0.009) (0.097) (0.011) [0.2543] (0.011) (0.155) (0.017) (0.007) (0.095) (0.011) Stunting (0–3 0.383 0.358 0.067 -0.092 0.005 0.067 -0.092 0.005 years) [0.0080] [0.4797] (0.041) (0.090) (0.028) (0.041) (0.090) (0.028) Severe stunting (0–3 0.206 0.211 0.014 -0.038 -0.003 0.014 -0.038 -0.003 years) [0.0067] [0.4083] (0.027) (0.099) (0.029) (0.027) (0.099) (0.029) Wasting (0-3 0.124 0.199 -0.038 0.3040** 0.0428* -0.038 0.3040** 0.0428* years) [0.0055] [0.3995] (0.024) (0.148) (0.025) (0.024) (0.148) (0.025) Severe wasting 0.048 0.089 -0.005 0.154 0.016 -0.005 0.154 0.016 (0-3 years) [0.0036] [0.2849] (0.014) (0.148) (0.015) (0.014) (0.148) (0.015) Diarrhea or 0.355 0.327 0.024 -0.088 -0.030 0.305 0.007 -0.019 -0.005 0.016 -0.055 -0.018 ARI [0.0075] [0.4694] (0.036) (0.097) (0.034) [0.4605] (0.046) (0.130) (0.043) (0.028) (0.078) (0.027) Mortality 0–28 days 0.013 0.013 -0.003 -0.218 -0.0201** 0.008 0.001 -0.107 -0.008 -0.001 -0.1634* -0.0138** (births past 18 months) [0.0021] [0.1129] (0.003) (0.138) (0.010) [0.0888] (0.004) (0.113) (0.008) (0.003) (0.089) (0.006)   Wave II Wave III AVERAGE Interaction Interaction Interaction Generasi Generasi Generasi Baseline Control Generasi with Pre- Control Generasi with Pre- Generasi with Pre- Indicator at 10th at 10th at 10th Mean Mean Effect Period Mean Effect Period Effect Period Percentile Percentile Percentile Level Level Level Mortality 0–12 months 0.024 0.035 -0.005 -0.147 -0.017 0.011 0.0085* -0.173 -0.006 0.002 -0.156 -0.011 (births past 24 months) [0.0026] [0.1842] (0.005) (0.148) (0.011) [0.1061] (0.005) (0.110) (0.009) (0.004) (0.101) (0.007) Mortality 0–12 months (births 0.028 0.055 -0.0132** -0.030 -0.018 0.011 0.006 0.014 0.008 -0.004 -0.017 -0.007 past 12-24 months) [0.0042] [0.2279] (0.006) (0.090) (0.013) [0.1060] (0.009) (0.089) (0.013) (0.005) (0.075) (0.011) Mortality 0–12 months 0.023 0.030 -0.008 -0.132 -0.0187** 0.011 0.0079* -0.1879** -0.008 0.000 -0.1566** -0.0129** (births past 21 months) [0.0027] [0.1700] (0.005) (0.128) (0.009) [0.1043] (0.005) (0.094) (0.008) (0.004) (0.079) (0.006) Mortality 0–12 months (births 0.027 0.047 -0.0174*** -0.069 -0.0273** 0.010 0.003 0.094 0.017 -0.008 -0.010 -0.009 past 12-21 months) [0.0047] [0.2130] (0.007) (0.069) (0.011) [0.1012] (0.010) (0.065) (0.011) (0.006) (0.053) (0.008) Home-based Bahasa test -0.039 -0.013 -0.023 0.010 -0.029 -0.023 0.010 -0.029 7–12 years (age-adjusted [0.0188] [1.0563] (0.040) (0.091) (0.071) (0.040) (0.091) (0.071) Z-score) Home-based math test -0.032 -0.063 -0.013 -0.046 0.010 -0.013 -0.046 0.010 7–12 years (age-adjusted [0.0191] [1.0481] (0.043) (0.115) (0.075) (0.043) (0.115) (0.075) Z-score) Final Impact Evaluation Report Annexes 101 102   Wave II Wave III AVERAGE Interaction Interaction Interaction Generasi Generasi Generasi Annexes Baseline Control Generasi with Pre- Control Generasi with Pre- Generasi with Pre- Indicator at 10th at 10th at 10th Mean Mean Effect Period Mean Effect Period Effect Period Percentile Percentile Percentile Level Level Level Home-based total test 7–12 -0.044 -0.033 -0.015 -0.014 -0.007 -0.015 -0.014 -0.007 years (age-adjusted [0.0192] [1.0457] (0.042) (0.117) (0.083) (0.042) (0.117) (0.083) Z-score) Home-based Indonesia’s PNPM Generasi Program Bahasa test -0.008 -0.068 0.065 0.018 0.051 0.065 0.018 0.051 13–15 years (age-adjusted [0.0322] [1.1176] (0.069) (0.129) (0.139) (0.069) (0.129) (0.139) Z-score) Home-based math test -0.001 0.004 0.044 0.068 -0.005 0.044 0.068 -0.005 13–15 years (age-adjusted [0.0320] [1.0395] (0.062) (0.096) (0.089) (0.062) (0.096) (0.089) Z-score) Home-based total test -0.005 -0.014 0.051 0.086 -0.003 0.051 0.086 -0.003 13–15 years (age-adjusted [0.0321] [1.0401] (0.066) (0.106) (0.097) (0.066) (0.106) (0.097) Z-score) Average standardized 0.010 -0.6669* 0.1059*** -0.008 -0.121 0.002 0.006 -0.229 0.031 effect (0.022) (0.343) (0.035) (0.020) (0.180) (0.028) (0.017) (0.156) (0.025) Average standardized 0.010 -0.6669* 0.1059*** -0.020 -0.191 -0.003 0.001 -0.353 0.041 effect health (0.022) (0.343) (0.035) (0.024) (0.270) (0.032) (0.017) (0.235) (0.026) Average standardized 0.017 0.019 0.011 0.017 0.019 0.011 effect educ. (0.036) (0.068) (0.057) (0.036) (0.068) (0.057) Table 19. Program impact on main targeted indicators, pre-period subdistrict average per capita consumption interactions   Wave II Wave III AVERAGE Interaction Interaction Interaction Baseline Control Generasi with Pre-Period Control Generasi with Pre-Period Generasi with Pre-Period Indicator Mean Mean Effect Kecamatan log Mean Effect Kecamatan log Effect Kecamatan log average PC cons average PC cons average PC cons (1) (2) (3) (4) (5) (6) (7) (8) (9) Main 12 indicators Number prenatal visits 7.447 7.464 -15.4392** 1.2312** 7.639 0.672 -0.050 -8.764 0.701 [4.2935] [0.1415] (7.658) (7.658) [0.1449] (8.387) (8.387) (5.463) (5.463) Delivery by trained 0.670 0.755 0.136 -0.008 0.780 -0.682 0.054 -0.312 0.026 midwife [0.4705] [0.0173] (0.911) (0.911) [0.0164] (0.914) (0.914) (0.656) (0.656) Number of postnatal 1.720 1.737 0.276 -0.031 1.634 7.987 -0.638 3.799 -0.308 visits [2.4477] [0.0969] (4.628) (4.628) [0.0975] (5.254) (5.254) (3.278) (3.278) Iron tablet sachets 1.588 1.977 -3.133 0.256 1.741 -4.6722** 0.3766** -3.6077** 0.2925** [1.2554] [0.0493] (2.785) (2.785) [0.0440] (2.331) (2.331) (1.772) (1.772) Percent of immunization 0.653 0.693 -0.405 0.034 0.756 -0.273 0.022 -0.310 0.025 [0.3664] [0.0127] (0.592) (0.592) [0.0107] (0.624) (0.624) (0.402) (0.402) Number of weight checks 2.126 2.192 0.479 -0.029 2.262 1.623 -0.114 0.757 -0.048 [1.1895] [0.0402] (1.801) (1.801) [0.0382] (2.394) (2.394) (1.635) (1.635) Number Vitamin A 1.529 1.560 -1.049 0.084 1.454 -0.537 0.046 -0.735 0.060 supplements [1.1370] [0.0435] (1.912) (1.912) [0.0412] (2.153) (2.153) (1.367) (1.367) Percent malnourished 0.168 0.199 0.111 -0.009 0.228 -0.835 0.065 -0.274 0.021 [0.3739] [0.0138] (0.581) (0.581) [0.0144] (0.576) (0.576) (0.436) (0.436) Age 7–12 participation 0.948 0.982 0.153 -0.012 0.985 0.3575** -0.0278** 0.2672* -0.0208* rate [0.2221] [0.0050] (0.224) (0.224) [0.0046] (0.160) (0.160) (0.151) (0.151) Age 13–15 participation 0.822 0.906 -1.7009*** 0.1323*** 0.874 0.765 -0.060 -0.343 0.027 rate Final Impact Evaluation Report Annexes [0.3827] [0.0187] (0.606) (0.606) [0.0223] (0.674) (0.674) (0.526) (0.526) 103 104   Wave II Wave III AVERAGE Interaction Interaction Interaction Annexes Baseline Control Generasi with Pre-Period Control Generasi with Pre-Period Generasi with Pre-Period Indicator Mean Mean Effect Kecamatan log Mean Effect Kecamatan log Effect Kecamatan log average PC cons average PC cons average PC cons (1) (2) (3) (4) (5) (6) (7) (8) (9) Age 7–12 gross 0.904 0.956 0.113 -0.009 0.960 0.350 -0.028 0.208 -0.016 attendance [0.2773] [0.0059] (0.210) (0.210) [0.0056] (0.234) (0.234) (0.167) (0.167) Age 13–15 gross Indonesia’s PNPM Generasi Program 0.768 0.884 -1.4114** 0.1085** 0.860 0.520 -0.040 -0.282 0.022 attendance Additional Education [0.4125] [0.0193] (0.651) (0.651) [0.0226] (0.730) (0.730) (0.554) (0.554) Indicators Age 13–15 conditional 0.958 0.976 0.092 -0.008 0.984 -0.381 0.031 -0.072 0.006 attendance [0.1752] [0.0070] (0.223) (0.223) [0.0060] (0.275) (0.275) (0.165) (0.165) Age 13–15 enrolled in 0.593 0.672 -1.664 0.132 0.707 2.1561* -0.1683* 0.283 -0.021 SMP [0.4915] [0.0301] (1.151) (1.151) [0.0306] (1.120) (1.120) (0.925) (0.925) Age 7–12 enrolled in SD 0.886 0.905 0.560 -0.045 0.917 0.7243** -0.0575** 0.6507** -0.0517** [0.3182] [0.0109] (0.348) (0.348) [0.0106] (0.321) (0.321) (0.268) (0.268) Age 13–15 enrolled other 0.229 0.234 0.163 -0.016 0.167 -0.810 0.063 -0.286 0.020 than SMP [0.4205] [0.0271] (1.048) (1.048) [0.0251] (1.000) (1.000) (0.819) (0.819) Kecamatan SD gross 0.931 0.943 0.466 -0.038 0.956 -0.018 0.001 0.258 -0.021 enrollment [0.1125] [0.0165] (0.623) (0.623) [0.0165] (0.503) (0.503) (0.443) (0.443) Kecamatan SMP gross 0.869 0.997 -3.392 0.266 1.001 3.4494* -0.273 -0.402 0.031 enrollment [0.4593] [0.0768] (2.350) (2.350) [0.0742] (2.070) (2.070) (1.622) (1.622)   Wave II Wave III AVERAGE Interaction Interaction Interaction Baseline Control Generasi with Pre-Period Control Generasi with Pre-Period Generasi with Pre-Period Indicator Mean Mean Effect Kecamatan log Mean Effect Kecamatan log Effect Kecamatan log average PC cons average PC cons average PC cons (1) (2) (3) (4) (5) (6) (7) (8) (9) Average standardized -1.3668* 0.1086* 0.695 -0.052 -0.271 0.023 effect (0.744) (0.059) (0.748) (0.059) (0.557) (0.044) Average standardized -0.941 0.077 -0.001 0.003 -0.534 0.045 effect health (0.835) (0.066) (0.938) (0.074) (0.667) (0.053) Average standardized -2.2190* 0.1712* 2.0857* -0.1623* 0.257 -0.021 effect educ. (1.269) (0.100) (1.091) (0.086) (0.979) (0.077) Final Impact Evaluation Report Annexes 105 106 Table 20. Program impact on longer-term outcomes, pre-period subdistrict average per capita consumption interactions     Wave II Wave III AVERAGE Annexes Interaction Interaction Interaction with Pre-Period with Pre-Period with Pre-Period Baseline Control Generasi Control Generasi Generasi Indicator Kecamatan log Kecamatan log Kecamatan log Mean Mean Effect Mean Effect Effect average PC average PC average PC cons cons cons (1) (2) (3) (4) (5) (6) (7) (8) (9) Malnourished (0–3 years) 0.168 0.199 0.111 -0.009 0.228 -0.835 0.065 -0.274 0.021 [0.0059] [0.0138] (0.581) (0.581) [0.0144] (0.576) (0.576) (0.436) (0.436) Indonesia’s PNPM Generasi Program Severely malnourished (0–3 0.047 0.057 -0.071 0.005 0.069 -0.711 0.055 -0.324 0.025 years) [0.0033] [0.0080] (0.318) (0.318) [0.0087] (0.446) (0.446) (0.286) (0.286) Stunting (0–3 years) 0.383 0.358 -1.4516* 0.1178** -1.4516* 0.1178** [0.0080] [0.0174] (0.748) (0.748) (0.748) (0.748) Severe stunting (0–3 years) 0.206 0.211 -1.5380** 0.1227** -1.5380** 0.1227** [0.0067] [0.0148] (0.762) (0.762) (0.762) (0.762) Wasting (0–3 years) 0.124 0.199 0.105 -0.008 0.105 -0.008 [0.0055] [0.0145] (0.801) (0.801) (0.801) (0.801) Severe wasting (0–3 years) 0.048 0.089 0.035 -0.003 0.035 -0.003 [0.0036] [0.0103] (0.564) (0.564) (0.564) (0.564) Diarrhea or ARI 0.355 0.327 -0.155 0.012 0.305 0.136 -0.011 -0.024 0.002 [0.0075] [0.0161] (0.717) (0.717) [0.0157] (0.946) (0.946) (0.570) (0.570) Mortality 0–28 days (births 0.013 0.013 0.096 -0.008 0.008 0.042 -0.003 0.097 -0.008 past 18 months) [0.0021] [0.0045] (0.125) (0.125) [0.0035] (0.234) (0.234) (0.132) (0.132) Mortality 0–12 months (births 0.024 0.035 -0.110 0.008 0.011 0.275 -0.022 0.047 -0.004 past 24 months) [0.0026] [0.0065] (0.192) (0.192) [0.0038] (0.259) (0.259) (0.168) (0.168) Mortality 0–12 months (births 0.028 0.055 -0.191 0.014 0.011 0.015 -0.001 -0.097 0.007 past 12-24 months) [0.0042] [0.0119] (0.346) (0.346) [0.0056] (0.489) (0.489) (0.287) (0.287)     Wave II Wave III AVERAGE Interaction Interaction Interaction with Pre-Period with Pre-Period with Pre-Period Baseline Control Generasi Control Generasi Generasi Indicator Kecamatan log Kecamatan log Kecamatan log Mean Mean Effect Mean Effect Effect average PC average PC average PC cons cons cons Mortality 0–12 months (births 0.023 0.030 -0.056 0.004 0.011 0.117 -0.009 0.017 -0.002 past 21 months) [0.0027] [0.0064] (0.198) (0.198) [0.0039] (0.269) (0.269) (0.169) (0.169) Mortality 0–12 months (births 0.027 0.047 -0.102 0.006 0.010 -0.434 0.035 -0.192 0.015 past 12-21 months) [0.0047] [0.0129] (0.433) (0.433) [0.0059] (0.541) (0.541) (0.317) (0.317) Home-based Bahasa test 7–12 -0.039 -0.013 0.476 -0.040 0.476 -0.040 years (age-adjusted Z-score) [0.0188] [0.0436] (2.068) (2.068) (2.068) (2.068) Home-based math test 7–12 -0.032 -0.063 0.602 -0.049 0.602 -0.049 years (age-adjusted Z-score) [0.0191] [0.0453] (2.313) (2.313) (2.313) (2.313) Home-based total test 7–12 -0.044 -0.033 0.001 -0.001 0.001 -0.001 years (age-adjusted Z-score) [0.0192] [0.0453] (1.955) (1.955) (1.955) (1.955) Home-based Bahasa test -0.008 -0.068 2.959 -0.230 2.959 -0.230 13–15 years (age-adjusted Z-score) [0.0322] [0.0790] (3.512) (3.512) (3.512) (3.512) Home-based math test 13–15 -0.001 0.004 -0.800 0.067 -0.800 0.067 years (age-adjusted Z-score) [0.0320] [0.0754] (2.618) (2.618) (2.618) (2.618) Home-based total test 13–15 -0.005 -0.014 -0.062 0.009 -0.062 0.009 years Final Impact Evaluation Report Annexes 107 108     Wave II Wave III AVERAGE Interaction Interaction Interaction Annexes with Pre-Period with Pre-Period with Pre-Period Baseline Control Generasi Control Generasi Generasi Indicator Kecamatan log Kecamatan log Kecamatan log Mean Mean Effect Mean Effect Effect average PC average PC average PC cons cons cons (age-adjusted Z-score) [0.0321] [0.0755] (2.988) (2.988) (2.988) (2.988) Average standardized effect 0.245 -0.015 0.629 -0.050 0.578 -0.045 (0.894) (0.071) (0.977) (0.077) (0.796) (0.063) Average standardized effect Indonesia’s PNPM Generasi Program 0.245 -0.015 0.706 -0.057 0.630 -0.049 health (0.894) (0.071) (1.161) (0.092) (0.727) (0.057) Average standardized effect 0.474 -0.036 0.474 -0.036 educ. (1.656) (0.131) (1.656) (0.131) Table 21. Program impact on main targeted indicators, pre-period village access interactions     Wave II Wave III AVERAGE Interaction with Interaction with Interaction with Baseline Control Generasi Control Generasi Generasi Indicator Pre-Period Village Pre-Period Village Pre-Period Village Mean Mean Effect Mean Effect Effect Access Access Access (1) (2) (3) (4) (5) (6) (7) (8) (9) Main 12 indicators Number prenatal visits 7.447 7.464 -0.382 0.485 7.639 -0.010 0.092 -0.145 0.229 [4.2935] [4.1639] (0.397) (0.399) [4.2297] (0.407) (0.446) (0.291) (0.302) Delivery by trained 0.670 0.755 0.004 0.037 0.780 0.020 -0.023 0.011 0.008 midwife [0.4705] [0.4303] (0.052) (0.050) [0.4144] (0.053) (0.055) (0.042) (0.040) Number of postnatal visits 1.720 1.737 -0.110 -0.017 1.634 0.247 -0.325 0.101 -0.205 [2.4477] [2.4079] (0.235) (0.252) [2.4597] (0.218) (0.226) (0.175) (0.183) Iron tablet sachets 1.588 1.977 0.151 -0.067 1.741 0.182 -0.144 0.1749* -0.116 [1.2554] [1.4426] (0.130) (0.147) [1.2748] (0.131) (0.143) (0.094) (0.103) Percent of immunization 0.653 0.693 0.040 -0.026 0.756 0.013 -0.014 0.0379* -0.033 [0.3664] [0.3441] (0.033) (0.033) [0.2858] (0.028) (0.031) (0.022) (0.023) Number of weight checks 2.126 2.192 0.1821** -0.083 2.262 0.2031* -0.013 0.2111*** -0.068 [1.1895] [1.1718] (0.081) (0.088) [1.1204] (0.113) (0.119) (0.065) (0.070) Number Vitamin A 1.529 1.560 0.136 -0.1730* 1.454 0.1988* -0.197 0.1582** -0.1748** supplements [1.1370] [1.0089] (0.095) (0.100) [0.9520] (0.109) (0.122) (0.076) (0.082) Percent malnourished 0.168 0.199 -0.018 0.020 0.228 0.032 -0.063 -0.001 -0.014 [0.3739] [0.3995] (0.032) (0.034) [0.4199] (0.035) (0.039) (0.023) (0.026) Age 7–12 participation 0.948 0.982 -0.003 0.0126* 0.985 0.0091* -0.002 0.004 0.004 rate [0.2221] [0.1334] (0.006) (0.008) [0.1207] (0.005) (0.008) (0.004) (0.006) Age 13–15 participation 0.822 0.906 -0.034 -0.024 0.874 -0.002 0.052 -0.016 0.017 rate [0.3827] [0.2928] (0.021) (0.030) [0.3327] (0.019) (0.032) (0.016) (0.023) Final Impact Evaluation Report Age 7–12 gross attendance 0.904 0.956 -0.003 0.014 0.960 0.006 -0.008 0.002 0.003 Annexes [0.2773] [0.1568] (0.006) (0.009) [0.1463] (0.007) (0.011) (0.005) (0.007) 109 110     Wave II Wave III AVERAGE Interaction with Interaction with Interaction with Annexes Baseline Control Generasi Control Generasi Generasi Indicator Pre-Period Village Pre-Period Village Pre-Period Village Mean Mean Effect Mean Effect Effect Access Access Access (1) (2) (3) (4) (5) (6) (7) (8) (9) Age 13–15 gross 0.768 0.884 -0.0511** -0.006 0.860 -0.010 0.0758** -0.0271* 0.036 attendance Additional Education [0.4125] [0.3022] (0.022) (0.032) [0.3364] (0.020) (0.035) (0.016) (0.025) Indicators Age 13–15 conditional Indonesia’s PNPM Generasi Program 0.958 0.976 -0.0146** 0.008 0.984 -0.006 0.020 -0.0091** 0.0131* attendance [0.1752] [0.1036] (0.007) (0.011) [0.0832] (0.006) (0.012) (0.005) (0.008) Age 13–15 enrolled in 0.593 0.672 0.026 -0.0991** 0.707 0.026 0.032 0.027 -0.029 SMP [0.4915] [0.4704] (0.031) (0.047) [0.4561] (0.027) (0.043) (0.022) (0.032) Age 7–12 enrolled in SD 0.886 0.905 -0.003 0.010 0.917 0.000 0.008 0.000 0.004 [0.3182] [0.2930] (0.010) (0.015) [0.2757] (0.009) (0.015) (0.008) (0.010) Age 13–15 enrolled other 0.229 0.234 -0.0635** 0.0796** 0.167 -0.028 0.021 -0.0443** 0.0488* than SMP [0.4205] [0.4240] (0.029) (0.040) [0.3735] (0.024) (0.038) (0.020) (0.028) Average standardized effect 0.009 -0.012 0.0571* -0.004 0.043 -0.015 (0.035) (0.039) (0.032) (0.034) (0.026) (0.028) Average standardized effect 0.054 -0.027 0.0782* -0.048 0.0766** -0.049 health (0.047) (0.049) (0.043) (0.044) (0.034) (0.035) Average standardized effect -0.0815* 0.017 0.015 0.084 -0.025 0.054 educ. (0.042) (0.057) (0.034) (0.057) (0.030) (0.042) Table 22. Program impact on longer-term outcomes, pre-period village access interactions     Wave II Wave III AVERAGE Interaction with Interaction with Interaction with Baseline Control Generasi Control Generasi Generasi Indicator Pre-Period Pre-Period Pre-Period Mean Mean Effect Mean Effect Effect Village Access Village Access Village Access (1) (2) (3) (4) (5) (6) (7) (8) (9) Malnourished (0–3 years) 0.168 0.199 -0.018 0.020 0.228 0.032 -0.063 -0.001 -0.014 [0.0059] [0.3995] (0.032) (0.034) [0.4199] (0.035) (0.039) (0.023) (0.026) Severely malnourished (0–3 years) 0.047 0.057 0.007 -0.016 0.069 -0.003 -0.015 0.002 -0.015 [0.0033] [0.2317] (0.016) (0.017) [0.2543] (0.024) (0.026) (0.013) (0.014) Stunting (0–3 years) 0.383 0.358 -0.015 0.054 -0.015 0.054 [0.0080] [0.4797] (0.043) (0.047) (0.043) (0.047) Severe stunting (0–3 years) 0.206 0.211 0.007 -0.002 0.007 -0.002 [0.0067] [0.4083] (0.039) (0.043) (0.039) (0.043) Wasting (0–3 years) 0.124 0.199 0.0926*** -0.1139*** 0.0926*** -0.1139*** [0.0055] [0.3995] (0.034) (0.038) (0.034) (0.038) Severe wasting (0–3 years) 0.048 0.089 0.0532** -0.0613** 0.0532** -0.0613** [0.0036] [0.2849] (0.025) (0.027) (0.025) (0.027) Diarrhea or ARI 0.355 0.327 -0.014 0.010 0.305 -0.038 0.049 -0.028 0.032 [0.0075] [0.4694] (0.035) (0.038) [0.4605] (0.044) (0.046) (0.027) (0.028) Mortality 0–28 days (births past 18 0.013 0.013 0.000 -0.007 0.008 0.000 -0.001 0.002 -0.007 months) [0.0021] [0.1129] (0.003) (0.005) [0.0888] (0.014) (0.015) (0.006) (0.007) Mortality 0–12 months (births past 0.024 0.035 -0.012 0.004 0.011 -0.005 0.010 -0.006 0.004 24 months) [0.0026] [0.1842] (0.011) (0.011) [0.1061] (0.018) (0.019) (0.010) (0.010) Mortality 0–12 months (births past 0.028 0.055 -0.010 -0.007 0.011 0.004 0.002 -0.003 -0.003 12-24 months) [0.0042] [0.2279] (0.017) (0.017) [0.1060] (0.026) (0.029) (0.013) (0.014) Mortality 0–12 months (births past 0.023 0.030 -0.013 0.002 0.011 -0.008 0.012 -0.006 0.003 21 months) [0.0027] [0.1700] (0.009) (0.009) [0.1043] (0.020) (0.021) (0.009) (0.010) Mortality 0–12 months (births past 0.027 0.047 -0.009 -0.016 0.010 0.001 0.006 -0.001 -0.010 Final Impact Evaluation Report 12–21 months) Annexes [0.0047] [0.2130] (0.013) (0.015) [0.1012] (0.034) (0.037) (0.016) (0.017) 111 112     Wave II Wave III AVERAGE Interaction with Interaction with Interaction with Annexes Baseline Control Generasi Control Generasi Generasi Indicator Pre-Period Pre-Period Pre-Period Mean Mean Effect Mean Effect Effect Village Access Village Access Village Access Home-based Bahasa test 7–12 years -0.039 -0.013 -0.013 -0.017 -0.013 -0.017 (age-adjusted Z-score) [0.0188] [1.0563] (0.049) (0.074) (0.049) (0.074) Home-based math test 7–12 years -0.032 -0.063 -0.038 0.082 -0.038 0.082 (age-adjusted Z-score) [0.0191] [1.0481] (0.051) (0.080) (0.051) (0.080) Home-based total test 7–12 years -0.044 -0.033 -0.033 0.062 -0.033 0.062 (age-adjusted Z-score) [0.0192] [1.0457] (0.049) (0.079) (0.049) (0.079) Indonesia’s PNPM Generasi Program Home-based Bahasa test 13–15 -0.008 -0.068 0.104 -0.085 0.104 -0.085 years (age-adjusted Z-score) [0.0322] [1.1176] (0.074) (0.110) (0.074) (0.110) Home-based math test 13–15 years -0.001 0.004 0.061 -0.037 0.061 -0.037 (age-adjusted Z-score) [0.0320] [1.0395] (0.078) (0.112) (0.078) (0.112) Home-based total test 13–15 years -0.005 -0.014 0.082 -0.064 0.082 -0.064 (age-adjusted Z-score) [0.0321] [1.0401] (0.081) (0.115) (0.081) (0.115) Average standardized effect 0.036 0.018 -0.007 0.007 -0.003 0.025 (0.036) (0.039) (0.051) (0.057) (0.028) (0.033) Average standardized effect health 0.036 0.018 -0.022 0.014 -0.017 0.041 (0.036) (0.039) (0.074) (0.080) (0.037) (0.038) Average standardized effect educ. 0.024 -0.008 0.024 -0.008 (0.043) (0.062) (0.043) (0.062)   Table 23. Program impact on main targeted indicators, excludes 49 newly added Generasi subdistricts in Year 2 Wave II (2008) Wave III (2009) AVERAGE Generasi Wave Generasi Generasi Average Treatment Indicator Baseline Mean Control Mean Control Mean III – Generasi Treatment Effect Treatment Effect Effect Wave II Main 12 indicators Number prenatal visits 7.444 7.464 -0.022 7.639 0.025 0.000 0.047 [4.3634] [4.1639] (0.200) [4.2297] (0.184) (0.138) (0.231) Delivery by trained 0.666 0.755 0.023 0.780 0.012 0.018 -0.011 midwife [0.4716] [0.4303] (0.024) [0.4144] (0.022) (0.019) (0.026) Number of postnatal visits 1.731 1.737 -0.087 1.634 0.020 -0.036 0.107 [2.4677] [2.4079] (0.118) [2.4597] (0.112) (0.086) (0.149) Iron tablet sachets 1.589 1.977 0.103 1.741 0.1150** 0.1087** 0.012 [1.2581] [1.4426] (0.074) [1.2748] (0.057) (0.049) (0.076) Percent of immunization 0.655 0.693 0.021 0.756 0.004 0.013 -0.017 [0.3653] [0.3441] (0.016) [0.2858] (0.015) (0.011) (0.018) Number of weight checks 2.145 2.192 0.1064** 2.262 0.1998*** 0.1513*** 0.0934* [1.1772] [1.1718] (0.045) [1.1204] (0.053) (0.038) (0.055) Number Vitamin A 1.522 1.560 -0.019 1.454 0.057 0.015 0.076 supplements [1.1460] [1.0089] (0.047) [0.9520] (0.051) (0.036) (0.066) Percent malnourished 0.177 0.199 0.003 0.228 -0.0366*** -0.016 -0.0398* [0.3813] [0.3995] (0.014) [0.4199] (0.013) (0.010) (0.021) Age 7–12 participation 0.949 0.982 0.001 0.985 0.0141*** 0.0075** 0.0133* rate [0.2211] [0.1334] (0.005) [0.1207] (0.004) (0.004) (0.007) Age 13–15 participation 0.817 0.906 -0.0455*** 0.874 0.021 -0.011 0.0661*** rate [0.3865] [0.2928] (0.017) [0.3327] (0.016) (0.014) (0.023) Age 7–12 gross attendance 0.906 0.956 0.000 0.960 0.007 0.003 0.007 [0.2758] [0.1568] (0.005) [0.1463] (0.006) (0.004) (0.008) Age 13–15 gross 0.759 0.884 -0.0562*** 0.860 0.0289* -0.012 0.0851*** Final Impact Evaluation Report attendance Annexes 113 114 Annexes Wave II (2008) Wave III (2009) AVERAGE Generasi Wave Generasi Generasi Average Treatment Indicator Baseline Mean Control Mean Control Mean III – Generasi Treatment Effect Treatment Effect Effect Wave II Additional Education [0.4184] [0.3022] (0.018) [0.3364] (0.017) (0.015) (0.024) Indicators Age 13–15 conditional 0.952 0.976 -0.0103* 0.984 0.009 0.000 0.0195** attendance [0.1888] [0.1036] (0.005) [0.0832] (0.006) (0.004) (0.009) Indonesia’s PNPM Generasi Program Age 13–15 enrolled in 0.591 0.672 0.002 0.707 0.030 0.017 0.029 SMP [0.4918] [0.4704] (0.028) [0.4561] (0.025) (0.021) (0.034) Age 7–12 enrolled in SD 0.885 0.905 0.003 0.917 0.007 0.005 0.004 [0.3189] [0.2930] (0.009) [0.2757] (0.008) (0.007) (0.012) Age 13–15 enrolled other 0.227 0.234 -0.0509** 0.167 -0.011 -0.0298* 0.040 than SMP [0.4187] [0.4240] (0.025) [0.3735] (0.020) (0.018) (0.031) Kecamatan SD gross 0.931 0.943 -0.007 0.956 -0.008 -0.008 -0.001 enrollment [0.1147] [0.1495] (0.015) [0.1496] (0.014) (0.012) (0.019) Kecamatan SMP gross 0.872 0.997 -0.049 1.001 0.006 -0.021 0.054 enrollment [0.4644] [0.6691] (0.053) [0.6426] (0.057) (0.043) (0.072) Average standardized effect -0.011 0.0614*** 0.0269* 0.0725*** (0.019) (0.018) (0.015) (0.021) Average standardized effect 0.026 0.0567*** 0.0408** 0.033 health (0.021) (0.021) (0.017) (0.026) Average standardized effect -0.0857** 0.0708** -0.001 0.1524*** educ. (0.034) (0.029) (0.027) (0.030) Table 24. Program impact on main targeted indicators, comparing incentivized vs. non-incentivized, excludes 49 newly added Generasi subdis- tricts in Year 2 Wave II Wave III Wave III – Wave II Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Effect Treatment Effect Additional Effect Effect Effect Effect Effect Effect Effect Wave III-Wave II Wave III-Wave II Wave III-Wave II Main 12 indicators Number prenatal visits 0.300 -0.3559* 0.6556*** 0.165 -0.115 0.280 -0.135 0.241 -0.376 (0.239) (0.212) (0.217) (0.209) (0.215) (0.211) (0.268) (0.265) (0.267) Delivery by trained midwife 0.022 0.025 -0.003 0.023 0.001 0.022 0.001 -0.024 0.025 (0.028) (0.027) (0.025) (0.025) (0.027) (0.027) (0.030) (0.030) (0.031) Number of postnatal visits -0.133 -0.035 -0.098 0.034 0.006 0.028 0.166 0.041 0.126 (0.143) (0.128) (0.139) (0.141) (0.133) (0.157) (0.173) (0.173) (0.175) Iron tablet sachets 0.140 0.064 0.076 0.1356** 0.094 0.042 -0.005 0.030 -0.035 (0.086) (0.083) (0.081) (0.062) (0.073) (0.073) (0.088) (0.088) (0.089) Percent of immunization 0.028 0.012 0.016 0.017 -0.009 0.025 -0.011 -0.021 0.010 (0.019) (0.018) (0.018) (0.016) (0.017) (0.016) (0.021) (0.021) (0.021) Number of weight checks 0.1533*** 0.058 0.0952* 0.1717*** 0.2270*** -0.055 0.018 0.1689*** -0.1505** (0.053) (0.050) (0.054) (0.060) (0.060) (0.056) (0.064) (0.063) (0.063) Number Vitamin A -0.026 -0.012 -0.014 0.088 0.028 0.060 0.114 0.040 0.074 supplements (0.055) (0.055) (0.058) (0.054) (0.065) (0.064) (0.076) (0.079) (0.081) Percent malnourished -0.010 0.017 -0.0263* -0.0374** -0.0358** -0.002 -0.028 -0.0525** 0.025 (0.017) (0.015) (0.016) (0.016) (0.017) (0.019) (0.024) (0.024) (0.025) Age 7–12 participation rate -0.001 0.003 -0.004 0.0088* 0.0192*** -0.0104** 0.010 0.0162** -0.006 (0.005) (0.006) (0.006) (0.005) (0.004) (0.005) (0.008) (0.008) (0.007) Age 13–15 participation -0.0376* -0.0535** 0.016 0.024 0.017 0.007 0.0616** 0.0707** -0.009 rate (0.019) (0.022) (0.024) (0.018) (0.019) (0.016) (0.026) (0.028) (0.028) Age 7–12 gross attendance -0.001 0.000 -0.001 0.005 0.008 -0.003 0.006 0.008 -0.002 (0.006) (0.006) (0.006) (0.008) (0.007) (0.008) (0.009) (0.010) (0.009) Age 13–15 gross attendance -0.0443** -0.0680*** 0.024 0.0359* 0.022 0.014 0.0803*** 0.0902*** -0.010 Additional Education Final Impact Evaluation Report (0.020) (0.023) (0.024) (0.019) (0.020) (0.017) (0.027) (0.029) (0.029) Indicators Annexes 115 116 Wave II Wave III Wave III – Wave II Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Annexes Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Effect Treatment Effect Additional Effect Effect Effect Effect Effect Effect Effect Wave III-Wave II Wave III-Wave II Wave III-Wave II Age 13–15 conditional -0.008 -0.0128* 0.005 0.0134** 0.005 0.008 0.0214** 0.0178* 0.004 attendance (0.006) (0.007) (0.007) (0.006) (0.007) (0.005) (0.009) (0.010) (0.010) Age 13–15 enrolled in SMP 0.020 -0.018 0.038 0.0473* 0.014 0.033 0.027 0.032 -0.005 (0.030) (0.034) (0.031) (0.028) (0.030) (0.028) (0.040) (0.040) (0.042) Age 7–12 enrolled in SD 0.000 0.005 -0.004 0.003 0.011 -0.008 0.002 0.006 -0.004 Indonesia’s PNPM Generasi Program (0.010) (0.010) (0.010) (0.010) (0.010) (0.012) (0.014) (0.014) (0.014) Age 13–15 enrolled other -0.0609** -0.041 -0.020 -0.020 -0.002 -0.018 0.041 0.038 0.003 than SMP (0.029) (0.029) (0.029) (0.023) (0.024) (0.026) (0.036) (0.036) (0.037) Kecamatan SD gross -0.012 -0.002 -0.010 -0.018 0.001 -0.019 -0.006 0.003 -0.009 enrollment (0.018) (0.017) (0.016) (0.017) (0.016) (0.017) (0.022) (0.022) (0.022) Kecamatan SMP gross -0.018 -0.080 0.062 0.026 -0.015 0.040 0.043 0.066 -0.022 enrollment (0.065) (0.058) (0.063) (0.063) (0.061) (0.052) (0.085) (0.078) (0.076) Average standardized effect 0.007 -0.030 0.036 0.0716*** 0.0515** 0.020 0.0647** 0.0809*** -0.016 (0.023) (0.021) (0.023) (0.021) (0.021) (0.021) (0.027) (0.023) (0.027) Average standardized effect 0.0468* 0.004 0.0426* 0.0734*** 0.040 0.033 0.029 0.038 -0.009 health (0.025) (0.024) (0.024) (0.024) (0.025) (0.025) (0.030) (0.028) (0.029) Average standardized effect -0.0736** -0.0976** 0.024 0.0679** 0.0737** -0.006 0.1385* 0.1678*** -0.031 educ. (0.037) (0.043) (0.044) (0.031) (0.034) (0.030) (0.075) (0.035) (0.045) Table 25. Program impact on longer-term outcomes, excludes 49 newly added Generasi subdistricts in Year 2     Wave II (2008) Wave III (2009) AVERAGE Generasi Wave Generasi Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect III – Generasi Treatment Effect Wave II Malnourished (0–3 years) 0.177 0.199 0.003 0.228 -0.0366*** -0.016 -0.0398* [0.0068] [0.3995] (0.014) [0.4199] (0.013) (0.010) (0.021) Severely malnourished (0–3 0.050 0.057 -0.007 0.069 -0.0235** -0.0150** -0.016 years) [0.0039] [0.2317] (0.008) [0.2543] (0.010) (0.007) (0.013) Wasting (0–3 years) 0.395 0.358 0.019 0.019 [0.0091] [0.4797] (0.020) (0.020) Severe wasting (0–3 years) 0.214 0.211 0.000 0.000 [0.0076] [0.4083] (0.019) (0.019) Stunting (0–3 years) 0.128 0.199 -0.001 -0.001 [0.0063] [0.3995] (0.016) (0.016) Severe stunting (0–3 years) 0.049 0.089 0.004 0.004 [0.0041] [0.2849] (0.012) (0.012) Diarrhea or ARI 0.346 0.327 -0.002 0.305 0.007 0.002 0.009 [0.0084] [0.4694] (0.018) [0.4605] (0.022) (0.014) (0.024) Mortality 0–28 days (births in 0.011 0.013 -0.0067* 0.008 -0.004 -0.0054* 0.003 past 18 months) [0.0023] [0.1129] (0.003) [0.0888] (0.005) (0.003) (0.006) Mortality 0–12 months 0.023 0.035 -0.0102** 0.011 0.002 -0.005 0.0124* (births in past 24 months) [0.0028] [0.1842] (0.005) [0.1061] (0.006) (0.004) (0.007) Mortality 0–12 months 0.026 0.055 -0.0169** 0.011 0.005 -0.008 0.0218* (births in past 12–24 months) [0.0047] [0.2279] (0.007) [0.1060] (0.009) (0.006) (0.011) Mortality 0–12 months 0.022 0.030 -0.0126*** 0.011 0.001 -0.006 0.0134* (births in past 21 months) [0.0029] [0.1700] (0.005) [0.1043] (0.006) (0.004) (0.008) Mortality 0–12 months 0.025 0.047 -0.0230*** 0.010 0.006 -0.0103* 0.0294** (births in past 12–21 months) Final Impact Evaluation Report [0.0052] [0.2130] (0.007) [0.1012] (0.009) (0.006) (0.012) Annexes 117 118     Wave II (2008) Wave III (2009) AVERAGE Generasi Wave Annexes Generasi Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect III – Generasi Treatment Effect Wave II Home-based Bahasa test 7–12 -0.047 -0.013 -0.019 -0.019 years (age-adjusted Z-score) [0.0210] [1.0563] (0.050) (0.050) Home-based math test 7–12 -0.007 -0.063 0.016 0.016 years Indonesia’s PNPM Generasi Program (age-adjusted Z-score) [0.0213] [1.0481] (0.047) (0.047) Home-based total test 7–12 -0.033 -0.033 0.010 0.010 years (age-adjusted Z-score) [0.0212] [1.0457] (0.049) (0.049) Home-based Bahasa test 0.019 -0.068 0.078 0.078 13–15 years (age-adjusted Z-score) [0.0345] [1.1176] (0.074) (0.074) Home-based math test 13–15 0.003 0.004 0.074 0.074 years (age-adjusted Z-score) [0.0352] [1.0395] (0.065) (0.065) Home-based total test 13–15 0.012 -0.014 0.079 0.079 years (age-adjusted Z-score) [0.0347] [1.0401] (0.070) (0.070) Average standardized effect 0.0513** 0.013 0.0308* -0.031 (0.020) (0.021) (0.018) (0.031) Average standardized effect 0.0513** 0.002 0.0276* -0.031 health (0.020) (0.023) (0.016) (0.031) Average standardized effect 0.037 0.037 educ. (0.041) (0.041) Table 26. Direct benefits, per-capita consumption quintile breakdown   Wave II Wave III AVERAGE Generasi Top Generasi Top Generasi Top Generasi Generasi Generasi Generasi Generasi Generasi Control 3 Quintile Control 3 Quintile 3 Quintile Indicator for Top 3 for Bottom for Top 3 for Bottom for Top 3 for Bottom Mean Additional Mean Additional Additional Quintiles 2 Quintiles Quintiles 2 Quintiles Quintiles 2 Quintiles Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) Received 0.024 0.010 0.002 0.008 0.035 0.0156** 0.0223* -0.007 0.0121** 0.014 -0.002 scholarship [0.0042] (0.008) (0.009) (0.010) [0.0050] (0.008) (0.013) (0.015) (0.006) (0.008) (0.010) Received 0.005 0.0745*** 0.1046*** -0.030 0.015 0.0391*** 0.0982*** -0.0591*** 0.0578*** 0.0989*** -0.0412*** uniform [0.0019] (0.018) (0.019) (0.022) [0.0033] (0.010) (0.015) (0.016) (0.010) (0.014) (0.015) Received other 0.010 0.0518*** 0.0630*** -0.011 0.010 0.0313*** 0.0783*** -0.0470*** 0.0424*** 0.0697*** -0.0273** school supplies [0.0027] (0.015) (0.013) (0.017) [0.0027] (0.009) (0.013) (0.015) (0.009) (0.010) (0.012) Received 0.000 0.0096*** 0.0079* 0.002 0.000 0.0058*** 0.0062*** 0.000 0.0076*** 0.0069*** 0.001 transport subsidy [0.0000] (0.004) (0.004) (0.005) [0.0000] (0.002) (0.002) (0.003) (0.002) (0.002) (0.003) Received other 0.000 0.000 0.000 0.000 0.004 -0.003 0.0134** -0.0159** -0.001 0.0066** -0.0072** school support [0.0000] 0.000 0.000 0.000 [0.0017] (0.004) (0.006) (0.007) (0.002) (0.003) (0.003) Received supplementary 0.000 0.0058* 0.0035** 0.002 0.001 0.001 0.013 -0.012 0.0033* 0.0083* -0.005 feeding at school [0.0000] (0.003) (0.002) (0.003) [0.0008] (0.003) (0.009) (0.009) (0.002) (0.004) (0.005) Received supplementary 0.512 0.1488*** 0.1907*** -0.042 0.433 0.1935*** 0.2365*** -0.043 0.1780*** 0.2146*** -0.037 feeding at posyandu [0.0174] (0.038) (0.038) (0.042) [0.0162] (0.030) (0.034) (0.040) (0.025) (0.029) (0.030) Received intensive supp. 0.033 0.023 0.0269* -0.004 0.042 0.012 0.0440*** -0.0323* 0.018 0.0352*** -0.017 feeding at school [0.0062] (0.015) (0.015) (0.019) [0.0065] (0.016) (0.013) (0.018) (0.011) (0.010) (0.013) Final Impact Evaluation Report Annexes 119 120   Wave II Wave III AVERAGE Annexes Generasi Top Generasi Top Generasi Top Generasi Generasi Generasi Generasi Generasi Generasi Control 3 Quintile Control 3 Quintile 3 Quintile Indicator for Top 3 for Bottom for Top 3 for Bottom for Top 3 for Bottom Mean Additional Mean Additional Additional Quintiles 2 Quintiles Quintiles 2 Quintiles Quintiles 2 Quintiles Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) Received health subs. for pre/ 0.002 0.0236** 0.0342*** -0.011 0.011 0.0287*** 0.0336* -0.005 0.0222*** 0.0372*** -0.015 post-natal care [0.0023] (0.011) (0.009) (0.014) [0.0063] (0.010) (0.018) (0.020) (0.008) (0.010) (0.011) Indonesia’s PNPM Generasi Program Received health subsidy for 0.008 0.1387*** 0.1148*** 0.024 0.032 0.1107*** 0.0671** 0.044 0.1267*** 0.0940*** 0.033 childbirth [0.0059] (0.031) (0.030) (0.035) [0.0141] (0.037) (0.027) (0.044) (0.025) (0.021) (0.026) Average standardized 0.4796*** 0.5443*** -0.0647 0.2284*** 0.3623*** -0.1339** 0.2999*** 0.4012*** -0.1013** effect (0.069) (0.057) (0.078) (0.033) (0.044) (0.052) (0.032) (0.035) (0.040) Average standardized 0.5054*** 0.5182*** -0.0128 0.2741*** 0.3272*** -0.053 0.3519*** 0.3989*** -0.0471 effect health (0.091) (0.077) (0.105) (0.056) (0.058) (0.075) (0.050) (0.047) (0.058) Average standardized 0.4473*** 0.5770*** -0.1297 0.1826*** 0.3974*** -0.2148*** 0.2479*** 0.4035*** -0.1556*** effect educ. Table 27. Program impact on main targeted indicators, per-capita consumption quintile breakdown.     Wave II Wave III AVERAGE Generasi Generasi Generasi Generasi Generasi Top 3 Generasi Generasi Top 3 Generasi Generasi Top 3 Baseline Control Control Indicator for Top 3 for Bottom Quintile for Top 3 for Bottom Quintile for Top 3 for Bottom Quintile Mean Mean Mean Quintiles 2 Quintiles Additional Quintiles 2 Quintiles Additional Quintiles 2 Quintiles Additional Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Main 12 indicators Number 7.473 7.636 0.028 0.412 -0.384 7.071 0.056 -0.638 0.694 0.027 -0.059 0.086 prenatal visits [4.3010] [0.2399] (0.385) (0.455) (0.547) [0.3065] (0.539) (0.501) (0.710) (0.317) (0.333) (0.421) Delivery by trained 0.663 0.742 -0.008 -0.075 0.067 0.687 0.004 -0.041 0.045 -0.004 -0.059 0.055 midwife [0.4727] [0.0278] (0.044) (0.049) (0.060) [0.0364] (0.064) (0.062) (0.092) (0.038) (0.044) (0.055) Number of 1.726 1.944 -0.230 -0.350 0.119 1.669 -0.002 -0.5225* 0.521 -0.155 -0.4116** 0.257 postnatal visits [2.4409] [0.1731] (0.261) (0.255) (0.341) [0.1956] (0.357) (0.302) (0.471) (0.212) (0.195) (0.279) Iron tablet 1.608 1.939 0.129 0.137 -0.009 1.556 -0.017 -0.064 0.047 0.063 0.050 0.012 sachets [1.2519] [0.0820] (0.138) (0.165) (0.172) [0.0879] (0.150) (0.145) (0.198) (0.105) (0.113) (0.135) Percent of 0.654 0.699 0.039 0.024 0.015 0.765 -0.0667** -0.055 -0.012 -0.002 -0.004 0.002 immunization [0.3649] [0.0185] (0.028) (0.031) (0.038) [0.0221] (0.033) (0.039) (0.049) (0.023) (0.026) (0.030) Number of 2.119 2.189 0.1453* 0.1872** -0.042 2.026 0.3356*** 0.3657*** -0.030 0.2517*** 0.2595*** -0.008 weight checks [1.1843] [0.0462] (0.085) (0.080) (0.104) [0.0566] (0.092) (0.098) (0.127) (0.065) (0.069) (0.083) Number Vitamin A 1.560 1.745 0.071 -0.031 0.102 1.732 -0.1928* -0.158 -0.035 0.006 -0.098 0.104 supplements [1.1635] [0.0539] (0.085) (0.096) (0.111) [0.0880] (0.111) (0.145) (0.186) (0.066) (0.082) (0.092) Percent 0.168 0.222 -0.007 0.000 -0.008 0.296 0.013 -0.055 0.068 -0.004 -0.019 0.016 malnourished [0.3742] [0.0165] (0.025) (0.028) (0.038) [0.0207] (0.033) (0.037) (0.055) (0.021) (0.023) (0.032) Final Impact Evaluation Report Annexes 121 122     Wave II Wave III AVERAGE Generasi Generasi Generasi Annexes Generasi Generasi Top 3 Generasi Generasi Top 3 Generasi Generasi Top 3 Baseline Control Control Indicator for Top 3 for Bottom Quintile for Top 3 for Bottom Quintile for Top 3 for Bottom Quintile Mean Mean Mean Quintiles 2 Quintiles Additional Quintiles 2 Quintiles Additional Quintiles 2 Quintiles Additional Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Age 7–12 participation 0.950 0.976 -0.005 0.010 -0.015 0.971 -0.003 0.0177** -0.0206* -0.003 0.0134* -0.0163* rate [0.2171] [0.0053] (0.007) (0.008) (0.010) [0.0057] (0.007) (0.009) (0.012) (0.006) (0.007) (0.008) Indonesia’s PNPM Generasi Program Age 13–15 participation 0.806 0.908 -0.019 -0.044 0.025 0.889 0.016 0.007 0.010 -0.002 -0.012 0.010 rate [0.3957] [0.0156] (0.026) (0.030) (0.037) [0.0154] (0.025) (0.025) (0.034) (0.018) (0.022) (0.026) Age 7–12 gross 0.900 0.953 -0.006 0.006 -0.012 0.944 -0.013 0.0222* -0.0354** -0.008 0.0134* -0.0217** attendance [0.2832] [0.0058] (0.008) (0.008) (0.011) [0.0065] (0.009) (0.012) (0.015) (0.007) (0.008) (0.010) Age 13–15 gross 0.749 0.887 -0.043 -0.0585* 0.016 0.862 0.019 0.009 0.009 -0.012 -0.017 0.005 attendance Additional Education [0.4248] [0.0157] (0.027) (0.032) (0.039) [0.0160] (0.027) (0.026) (0.037) (0.020) (0.024) (0.028) Indicators Age 13–15 conditional 0.958 0.977 -0.0255*** -0.0180* -0.008 0.970 0.006 0.003 0.003 -0.008 -0.005 -0.003 attendance [0.1746] [0.0041] (0.008) (0.011) (0.012) [0.0062] (0.011) (0.008) (0.014) (0.007) (0.007) (0.009) Age 13–15 enrolled in 0.593 0.651 0.007 -0.011 0.018 0.668 -0.004 0.0759** -0.079 0.002 0.039 -0.037 SMP [0.4916] [0.0256] (0.042) (0.047) (0.057) [0.0231] (0.036) (0.035) (0.051) (0.028) (0.032) (0.039) Age 7–12 0.893 0.878 0.013 -0.006 0.019 0.901 -0.009 0.011 -0.020 0.004 0.001 0.004 enrolled in SD [0.3096] [0.0111] (0.017) (0.015) (0.020) [0.0101] (0.015) (0.014) (0.022) (0.012) (0.011) (0.015)     Wave II Wave III AVERAGE Generasi Generasi Generasi Generasi Generasi Top 3 Generasi Generasi Top 3 Generasi Generasi Top 3 Baseline Control Control Indicator for Top 3 for Bottom Quintile for Top 3 for Bottom Quintile for Top 3 for Bottom Quintile Mean Mean Mean Quintiles 2 Quintiles Additional Quintiles 2 Quintiles Additional Quintiles 2 Quintiles Additional Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Age 13–15 enrolled other 0.213 0.257 -0.039 -0.027 -0.012 0.221 0.012 -0.0703** 0.0827* -0.015 -0.0474* 0.032 than SMP [0.4098] [0.0235] (0.040) (0.040) (0.052) [0.0204] (0.033) (0.033) (0.047) (0.027) (0.028) (0.037) Average standardized 0.003 -0.014 0.017 -0.011 -0.011 0.000 0.008 -0.004 0.012 effect (0.030) (0.038) (0.045) (0.036) (0.033) (0.048) (0.024) (0.028) (0.033) Average standardized 0.040 0.011 0.029 -0.019 -0.050 0.031 0.025 -0.015 0.039 effect health (0.035) (0.045) (0.053) (0.044) (0.047) (0.063) (0.029) (0.035) (0.040) Average standardized -0.070 -0.063 -0.007 0.006 0.068 -0.062 -0.027 0.016 -0.043 effect educ. (0.050) (0.058) (0.068) (0.043) (0.049) (0.065) (0.035) (0.044) (0.050) Final Impact Evaluation Report Annexes 123 124 Table 28. Program impact on longer-term outcomes, per-capita consumption quintile breakdown     Wave II Wave III AVERAGE Generasi Generasi Generasi Annexes Generasi Generasi Generasi Generasi Top 3 Generasi Top 3 Generasi Top 3 Baseline Control for Control for for Indicator for Top 3 Quintile for Top 3 Quintile for Top 3 Quintile Mean Mean Bottom 2 Mean Bottom 2 Bottom 2 Quintiles Additional Quintiles Additional Quintiles Additional Quintiles Quintiles Quintiles Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12) Malnourished (0–3 years) 0.168 0.222 -0.007 0.000 -0.008 0.296 0.013 -0.055 0.068 -0.004 -0.019 0.016 [0.0085] [0.0165] (0.025) (0.028) (0.038) [0.0207] (0.033) (0.037) (0.055) (0.021) (0.023) (0.032) Severely malnourished 0.045 0.063 0.010 -0.004 0.014 0.097 -0.0390** -0.016 -0.023 -0.012 -0.012 0.000 Indonesia’s PNPM Generasi Program (0–3 years) [0.0047] [0.0096] (0.013) (0.016) (0.019) [0.0134] (0.019) (0.026) (0.031) (0.012) (0.015) (0.017) Stunting (0–3 years) 0.392 0.424 0.0764** 0.042 0.034 0.0764** 0.042 0.034 [0.0116] [0.0235] (0.037) (0.041) (0.056) (0.037) (0.041) (0.056) Severe stunting (0–3 0.209 0.246 -0.008 0.040 -0.048 -0.008 0.040 -0.048 years) [0.0097] [0.0205] (0.034) (0.035) (0.047) (0.034) (0.035) (0.047) Wasting (0–3 years) 0.110 0.188 -0.022 0.020 -0.042 -0.022 0.020 -0.042 [0.0075] [0.0186] (0.036) (0.034) (0.049) (0.036) (0.034) (0.049) Severe wasting (0-3 years) 0.046 0.081 -0.022 0.018 -0.039 -0.022 0.018 -0.039 [0.0050] [0.0130] (0.024) (0.026) (0.037) (0.024) (0.026) (0.037) Diarrhea or ARI 0.352 0.361 -0.027 0.020 -0.047 0.338 -0.042 0.022 -0.064 -0.023 0.012 -0.035 [0.0107] [0.0189] (0.037) (0.034) (0.048) [0.0212] (0.043) (0.038) (0.054) (0.026) (0.027) (0.034) Mortality 0–28 days 0.011 0.000 0.000 0.005 -0.006 0.019 0.0244** -0.006 0.030 0.009 0.001 0.007 (births in past 18 months) [0.0028] [0.0000] (0.001) (0.005) (0.005) [0.0108] (0.012) (0.023) (0.024) (0.006) (0.011) (0.011) Mortality 0–12 months 0.022 0.000 0.003 0.001 0.002 0.045 -0.014 0.004 -0.017 -0.003 0.001 -0.004 (births in past 24 months) [0.0036] [0.0000] (0.003) (0.003) (0.005) [0.0140] (0.021) (0.024) (0.033) (0.008) (0.009) (0.011) Mortality 0–12 months (births past 12–24 0.025 0.000 0.003 -0.003 0.006 0.055 -0.016 -0.037 0.021 -0.004 -0.012 0.008 months) [0.0058] [0.0000] (0.003) (0.003) (0.006) [0.0219] (0.027) (0.041) (0.051) (0.007) (0.011) (0.012) Mortality 0–12 months 0.023 0.000 0.004 0.002 0.002 0.048 -0.012 0.001 -0.013 0.000 0.000 0.000 (births in past 21 months) [0.0038] [0.0000] (0.004) (0.003) (0.006) [0.0157] (0.025) (0.027) (0.038) (0.009) (0.011) (0.013)     Wave II Wave III AVERAGE Generasi Generasi Generasi Generasi Generasi Generasi Generasi Top 3 Generasi Top 3 Generasi Top 3 Baseline Control for Control for for Indicator for Top 3 Quintile for Top 3 Quintile for Top 3 Quintile Mean Mean Bottom 2 Mean Bottom 2 Bottom 2 Quintiles Additional Quintiles Additional Quintiles Additional Quintiles Quintiles Quintiles Effect Effect Effect Mortality 0–12 months (births past 12–21 0.027 0.000 0.005 -0.003 0.007 0.066 -0.013 -0.055 0.042 -0.001 -0.016 0.016 months) [0.0068] [0.0000] (0.004) (0.003) (0.007) [0.0286] (0.033) (0.050) (0.065) (0.008) (0.012) (0.014) Home-based Bahasa test -0.025 -0.001 -0.030 -0.011 -0.018 -0.030 -0.011 -0.018 7–12 years (age-adjusted Z-score) [0.0263] [0.0360] (0.071) (0.072) (0.099) (0.071) (0.072) (0.099) Home-based math test -0.042 0.036 -0.036 -0.057 0.021 -0.036 -0.057 0.021 7–12 years (age-adjusted Z-score) [0.0278] [0.0369] (0.070) (0.071) (0.099) (0.070) (0.071) (0.099) Home-based total test -0.042 -0.001 -0.023 -0.042 0.018 -0.023 -0.042 0.018 7–12 years (age-adjusted Z-score) [0.0270] [0.0386] (0.075) (0.071) (0.105) (0.075) (0.071) (0.105) Home-based Bahasa test -0.045 -0.093 0.065 -0.003 0.067 0.065 -0.003 0.067 13–15 years (age-adjusted Z-score) [0.0480] [0.0610] (0.111) (0.102) (0.133) (0.111) (0.102) (0.133) Home-based math test 0.023 -0.077 0.077 0.054 0.023 0.077 0.054 0.023 13–15 years (age-adjusted Z-score) [0.0470] [0.0556] (0.105) (0.090) (0.134) (0.105) (0.090) (0.134) Home-based total test -0.011 -0.100 0.073 0.041 0.033 0.073 0.041 0.033 13–15 years (age-adjusted Z-score) [0.0472] [0.0588] (0.113) (0.095) (0.138) (0.113) (0.095) (0.138) Average standardized -0.027 -0.014 -0.013 0.020 0.018 0.002 0.011 0.005 0.006 effect (0.038) (0.038) (0.065) (0.037) (0.049) (0.062) (0.030) (0.033) (0.042) Average standardized -0.027 -0.014 -0.013 0.020 0.029 -0.009 0.007 0.009 -0.003 effect health (0.038) (0.038) (0.065) (0.048) (0.068) (0.086) (0.032) (0.042) (0.051) Average standardized Final Impact Evaluation Report 0.018 -0.004 0.022 0.018 -0.004 0.022 effect educ. Annexes   (0.061) (0.054) (0.078) (0.061) (0.054) (0.078) 125 126 Table 29. Results for service provider quantities     Wave II (2008) Wave III (2009) AVERAGE Annexes Generasi Generasi Average Treatment Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Effect (1) (2) (3) (4) (5) (6) Midwife/Polindes in village 0.783 0.806 -0.002 0.829 -0.015 -0.009 [0.4125] [0.3955] (0.018) [0.3767] (0.015) (0.012) Number of active posyandu in village 4.516 4.222 -0.047 4.323 0.161 0.066 [3.5031] [2.6797] (0.121) [2.9473] (0.162) (0.134) SD located in village 0.991 0.992 0.002 0.992 -0.002 0.000 [0.0966] [0.0884] (0.003) [0.0882] (0.003) (0.002) Indonesia’s PNPM Generasi Program SMP located in village 0.457 0.465 0.023 0.472 0.0388*** 0.0315** [0.4983] [0.4992] (0.016) [0.4996] (0.014) (0.013) Number of teachers at SD (includes guru . 10.445 0.218 10.818 0.058 0.132 honor) [.] [3.1180] (0.251) [2.9402] (0.238) (0.215) Number of teachers at SMP (includes guru 22.568 21.949 0.304 22.490 0.673 0.5049* honor) [11.6755] [10.7522] (0.218) [10.5056] (0.485) (0.305) Number of full-time health personnel (excl. 23.317 24.220 1.363 25.085 1.7583* 1.5765** admin/support staff) [9.4449] [9.0609] (0.857) [8.9955] (0.969) (0.753) Number of all full/ part-time health personnel 24.657 26.098 1.9976** 26.378 2.3317** 2.1785*** (excl. admin/ support staff) [9.4721] [8.6960] (0.962) [8.8856] (0.907) (0.825) Number of full-time midwives 9.290 10.122 0.102 10.366 0.345 0.233 [4.5503] [4.1675] (0.362) [4.3504] (0.354) (0.285) Number of all full-time and part-time 9.691 10.585 0.123 10.756 0.6246* 0.394 midwives [4.4807] [4.0611] (0.443) [4.2848] (0.321) (0.340) Total full-time midwife to pop. ratio (using 0.000 0.000 0.000 0.000 0.000 0.000 Wave I pusk pop.) [0.0002] [0.0002] 0.000 [0.0002] 0.000 0.000 Total full/part-time midwife to pop. ratio 0.000 0.000 0.000 0.000 0.000 0.000 (using Wave I pusk pop.) [0.0002] [0.0002] 0.000 [0.0002] 0.000 0.000     Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Treatment Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Effect (1) (2) (3) (4) (5) (6) Number of teachers at SD (only full-time) . 6.616 0.161 7.043 0.060 0.107 [.] [3.0317] (0.251) [2.8390] (0.224) (0.210) Number of teachers at SMP (only full-time) 12.512 12.537 0.052 14.013 0.174 0.118 [11.4388] [11.2403] (0.334) [11.4466] (0.617) (0.433) Average standardized effect 0.025 0.026 0.026 (0.020) (0.021) (0.017) Average standardized effect health -0.011 0.008 0.000 (0.031) (0.033) (0.028) Average standardized effect educ. 0.0403* 0.0400 0.0403* (0.0232) (0.0249) (0.0206) Final Impact Evaluation Report Annexes 127 128 Table 30. Results for service provider quantities, incentivized vs. non-incentivized   Wave II Wave III AVERAGE Annexes Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Midwife/Polindes in village 0.005 -0.009 0.015 -0.010 -0.020 0.010 -0.003 -0.015 0.012 (0.021) (0.022) (0.022) (0.017) (0.018) (0.020) (0.015) (0.015) (0.016) Number of active posyandu in -0.095 0.001 -0.096 0.127 0.195 -0.069 0.025 0.106 -0.081 Indonesia’s PNPM Generasi Program village (0.124) (0.147) (0.126) (0.179) (0.176) (0.148) (0.140) (0.152) (0.120) SD located in village 0.002 0.003 -0.001 -0.004 0.001 -0.0046* -0.002 0.002 -0.003 (0.003) (0.004) (0.004) (0.003) (0.003) (0.003) (0.003) (0.003) (0.003) SMP located in village 0.020 0.026 -0.007 0.0374** 0.0401** -0.003 0.0293* 0.0338** -0.005 (0.018) (0.019) (0.019) (0.017) (0.016) (0.017) (0.016) (0.015) (0.016) Number of teachers at SD (includes 0.334 0.101 0.233 0.235 -0.112 0.347 0.284 -0.017 0.301 guru honor) (0.325) (0.283) (0.347) (0.299) (0.267) (0.304) (0.279) (0.245) (0.299) Number of teachers at SMP 0.081 0.5262** -0.4451* 0.509 0.834 -0.325 0.317 0.6909** -0.374 (includes guru honor) (0.252) (0.255) (0.256) (0.574) (0.544) (0.556) (0.364) (0.345) (0.364) Midwife/Polindes in village 1.7542* 0.966 0.789 1.9655* 1.556 0.409 1.8599** 1.295 0.565 (1.050) (1.026) (1.166) (1.036) (1.346) (1.422) (0.871) (1.005) (1.129) Number of all full/part-time health personnel (excl. admin/support 2.6115** 1.379 1.233 2.8899*** 1.789 1.101 2.7589*** 1.605 1.154 staff) (1.196) (1.105) (1.265) (0.973) (1.255) (1.328) (0.992) (1.025) (1.163) Number of full-time midwives 0.587 -0.382 0.9691* 0.431 0.262 0.169 0.485 -0.013 0.498 (0.436) (0.461) (0.544) (0.435) (0.431) (0.493) (0.352) (0.373) (0.451) Number of all full-time and part- 0.654 -0.414 1.0677** 1.1355** 0.130 1.0060** 0.9121* -0.118 1.0296** time midwives (0.610) (0.352) (0.492) (0.438) (0.339) (0.449) (0.484) (0.294) (0.437)   Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Total full-time midwife to pop. 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 ratio (using Wave I pusk pop.) 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 Total full/part-time midwife to pop. 0.000 -0.0000** 0.0000** 0.0000* 0.000 0.0000** 0.000 0.000 0.0000** ratio (using Wave I pusk pop.) 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 0.000 Number of teachers at SD (only 0.181 0.141 0.040 0.012 0.106 -0.095 0.087 0.126 -0.039 full-time) (0.315) (0.291) (0.342) (0.263) (0.273) (0.294) (0.258) (0.256) (0.295) Number of teachers at SD (only -0.177 0.279 -0.456 0.009 0.336 -0.328 -0.073 0.307 -0.380 full-time) (0.382) (0.366) (0.336) (0.706) (0.686) (0.641) (0.497) (0.478) (0.448) Average standardized effect 0.026 0.024 0.001 0.029 0.023 0.006 0.028 0.024 0.004 (0.023) (0.024) (0.024) (0.024) (0.024) (0.024) (0.021) (0.020) (0.022) Average standardized effect health -0.011 -0.011 0.001 0.008 0.007 0.002 0.001 -0.001 0.001 (0.035) (0.037) (0.035) (0.037) (0.038) (0.036) (0.031) (0.033) (0.029) Average standardized effect educ. 0.044 0.042 0.002 0.039 0.031 0.008 0.041 0.036 0.005 (0.028) (0.028) (0.030) (0.030) (0.029) (0.031) (0.026) (0.025) (0.028) Final Impact Evaluation Report Annexes 129 130 Table 31. Results for service provider quality (health and education infrastructure availability)     Wave II (2008) Wave III (2009) AVERAGE Annexes Generasi Generasi Average Treatment Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Effect (1) (2) (3) (4) (5) (6) Midwives: Has access to water 0.714 0.785 0.023 0.833 -0.030 -0.006 [0.4525] [0.4121] (0.021) [0.3737] (0.025) (0.017) Has access to electricity 0.957 0.995 0.001 0.983 0.004 0.003 [0.2030] [0.0716] (0.012) [0.1284] (0.010) (0.008) Indonesia’s PNPM Generasi Program Percentage of drugs in stock 0.610 0.791 -0.014 0.836 -0.008 -0.011 [0.2649] [0.2189] (0.017) [0.2022] (0.016) (0.013) Percentage of tools available 0.624 0.665 0.004 0.687 0.005 0.005 [0.2556] [0.2146] (0.013) [0.1933] (0.013) (0.010) Oxytocin in stock . 0.905 0.024 0.946 -0.011 0.005 [.] [0.2936] (0.018) [0.2267] (0.018) (0.013) Proportion of last 3 deliveries using partograph . 0.933 -0.012 -0.012 [.] [0.2113] (0.015) (0.015) Antenatal care service items “always do” (public) . 0.592 -0.0275* -0.0275* [.] [0.2295] (0.016) (0.016) Antental care service items “always do” (private) . 0.602 -0.0400*** -0.0400*** Schools: [.] [0.2102] (0.013) (0.013) Number of classrooms (SD) . 5.872 -0.059 6.157 -0.097 -0.079 [.] [1.3070] (0.126) [1.4065] (0.113) (0.104) Number of classrooms (SMP) 8.629 8.931 0.119 9.490 0.099 0.108 [5.8766] [5.9331] (0.234) [6.1713] (0.318) (0.238) Condition of school building (SD, scale 0–1) . 0.869 -0.003 0.906 -0.019 -0.012 [.] [0.1912] (0.014) [0.1577] (0.012) (0.009) Condition of school building (SMP, scale 0–1) 0.903 0.916 0.000 0.939 0.004 0.002 [0.1577] [0.1391] (0.013) [0.1225] (0.009) (0.008) Has student latrine (SD) . 0.799 0.010 0.870 0.006 0.008 [.] [0.4015] (0.028) [0.3364] (0.023) (0.019) Has student latrine (SMP) 0.929 0.923 0.023 0.943 -0.011 0.005 Puskesmas: [0.2564] [0.2676] (0.020) [0.2332] (0.017) (0.014)     Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Treatment Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Effect (1) (2) (3) (4) (5) (6) Stock out any vaccine last 2 months 0.437 0.085 -0.005 0.146 -0.024 -0.015 [0.4969] [0.2811] (0.041) [0.3556] (0.045) (0.031) Average standardized effect 0.015 -0.0476** -0.0323* (0.024) (0.023) (0.019) Average standardized effect health 0.020 -0.0621** -0.0497** (0.031) (0.029) (0.024) Average standardized effect educ. 0.011 -0.028 -0.009 (0.035) (0.034) (0.027) Average standardized effect SMP. 0.035 0.001 0.016 (0.039) (0.037) (0.028) Average standardized effect SD. -0.012 -0.057 -0.035 (0.053) (0.050) (0.041) Final Impact Evaluation Report Annexes 131 132 Table 32. Results for service provider quality (health and education infrastructure availability), incentivized vs. non-incentivized   Wave II Wave III AVERAGE Annexes Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Midwives: Has access to water -0.002 0.0466* -0.049 -0.037 -0.022 -0.014 -0.020 0.008 -0.028 (0.028) (0.024) (0.031) (0.029) (0.029) (0.031) (0.022) (0.021) (0.025) Indonesia’s PNPM Generasi Program Has access to electricity 0.007 -0.005 0.012 -0.004 0.012 -0.016 0.000 0.005 -0.005 (0.017) (0.015) (0.019) (0.015) (0.010) (0.015) (0.012) (0.009) (0.013) Percentage of drugs in stock -0.030 0.001 -0.031 0.002 -0.017 0.019 -0.012 -0.010 -0.002 (0.020) (0.021) (0.023) (0.017) (0.019) (0.016) (0.015) (0.016) (0.015) Percentage of tools available 0.012 -0.004 0.015 -0.011 0.020 -0.0308** -0.001 0.010 -0.012 (0.015) (0.017) (0.017) (0.015) (0.015) (0.016) (0.012) (0.012) (0.013) Oxytocin in stock 0.026 0.022 0.003 0.000 -0.022 0.022 0.012 -0.002 0.015 (0.022) (0.020) (0.020) (0.019) (0.022) (0.020) (0.014) (0.015) (0.014) Proportion of last 3 deliveries using -0.013 -0.012 -0.001 -0.013 -0.012 -0.001 partograph (0.017) (0.017) (0.017) (0.017) (0.017) (0.017) Antenatal care service items “always do” -0.020 -0.0345* 0.014 -0.020 -0.0345* 0.014 (public) (0.019) (0.018) (0.018) (0.019) (0.018) (0.018) Antental care service items “always do” -0.0484*** -0.0318** -0.017 -0.0484*** -0.0318** -0.017 (private) Schools: (0.016) (0.016) (0.018) (0.016) (0.016) (0.018) Number of classrooms (SD) 0.029 -0.148 0.177 -0.169 -0.027 -0.142 -0.085 -0.074 -0.011 (0.161) (0.138) (0.164) (0.134) (0.137) (0.150) (0.130) (0.121) (0.142) Number of classrooms (SMP) 0.224 0.016 0.209 0.009 0.187 -0.178 0.098 0.119 -0.021 (0.247) (0.280) (0.243) (0.350) (0.365) (0.329) (0.258) (0.276) (0.242) Condition of school building (SD, scale 0-1) -0.018 0.012 -0.0301* -0.0255* -0.013 -0.012 -0.0217** -0.002 -0.020 (0.017) (0.016) (0.018) (0.013) (0.015) (0.014) (0.011) (0.011) (0.012)   Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect Condition of school building (SMP, scale 0.000 -0.001 0.001 0.000 0.007 -0.007 0.000 0.004 -0.004 0-1) (0.012) (0.017) (0.015) (0.010) (0.010) (0.010) (0.008) (0.010) (0.008) Has student latrine (SD) 0.020 0.001 0.019 0.013 -0.001 0.014 0.016 0.000 0.016 (0.033) (0.034) (0.037) (0.025) (0.027) (0.025) (0.021) (0.024) (0.024) Has student latrine (SMP) 0.031 0.015 0.016 -0.022 0.000 -0.022 0.001 0.008 -0.006 Puskesmas: (0.023) (0.025) (0.028) (0.020) (0.019) (0.020) (0.016) (0.017) (0.018) Stock out any vaccine last 2 months 0.009 -0.018 0.026 -0.024 -0.024 0.000 -0.010 -0.021 0.011 (0.051) (0.049) (0.055) (0.051) (0.053) (0.051) (0.036) (0.037) (0.040) Average standardized effect 0.016 0.015 0.001 -0.0625** -0.033 -0.030 -0.0413* -0.023 -0.018 (0.028) (0.031) (0.034) (0.027) (0.027) (0.029) (0.022) (0.024) (0.026) Average standardized effect health 0.009 0.031 -0.023 -0.0681** -0.0556* -0.013 -0.0568** -0.043 -0.014 (0.039) (0.040) (0.049) (0.034) (0.033) (0.035) (0.029) (0.029) (0.032) Average standardized effect educ. 0.022 0.001 0.020 -0.055 -0.002 -0.053 -0.021 0.002 -0.023 (0.040) (0.043) (0.045) (0.039) (0.038) (0.039) (0.031) (0.032) (0.034) Average standardized effect SMP. 0.018 0.018 0.018 0.030 0.030 0.030 0.026 0.026 0.026 (0.050) (0.050) (0.050) (0.043) (0.043) (0.043) (0.034) (0.034) (0.034) Average standardized effect SD. -0.016 -0.016 -0.016 -0.035 -0.035 -0.035 -0.022 -0.022 -0.022 (0.061) (0.061) (0.061) (0.060) (0.060) (0.060) (0.050) (0.050) (0.050) Final Impact Evaluation Report Annexes 133 Annexes Table 33. Results for service provider level of effort     Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Baseline Control Control Indicator Treatment Treatment Treatment Mean Mean Mean Effect Effect Effect (1) (2) (3) (4) (5) (6) Midwives: Hours spent in outreach over past 3 days 3.165 2.498 0.362 3.211 0.057 0.198 [4.4875] [3.9634] (0.312) [5.4667] (0.351) (0.255) Hours spent providing public services over 13.548 12.389 -0.282 12.827 0.544 0.162 past 3 days [10.0559] [7.7669] (0.482) [6.8272] (0.516) (0.367) Hours spent providing private services over 10.805 9.995 -0.128 10.022 0.741 0.340 past 3 days [12.5048] [11.1686] (0.702) [8.5484] (0.589) (0.455) Total hours spent working over past 3 days 27.518 24.882 -0.119 26.060 1.277 0.631 [15.7132] [13.2966] (0.877) [12.2635] (0.832) (0.628) Number of posyandus attended in past 4.166 3.754 0.123 3.665 -0.052 0.028 month [3.3213] [3.1314] (0.223) [3.1183] (0.201) (0.162) Number of hours midwife spends per 3.039 2.722 0.159 2.744 0.011 0.079 posyandu Teachers: [1.6932] [1.6643] (0.108) [2.2397] (0.121) (0.087) Percent present at time of interview (SD) . 0.864 0.011 0.874 0.004 0.011 [.] [0.1656] (0.013) [0.1440] (0.009) (0.013) Percent present at time of interview (SMP) . 0.880 -0.004 0.898 -0.012 -0.008 [.] [0.1499] (0.012) [0.1346] (0.010) (0.008) Percent teaching at time of class observation . 0.665 -0.028 0.657 -0.008 -0.017 (SD) [.] [0.4728] (0.034) [0.4753] (0.036) (0.025) Percent teaching at time of class observation . 0.607 -0.060 0.540 0.031 -0.011 (SMP) Puskesmas: [.] [0.4895] (0.038) [0.4994] (0.043) (0.029) Minutes wait at recent health visits 25.201 28.877 3.066 27.795 1.790 2.366 [23.7360] [27.1099] (3.636) [23.1787] (3.630) (2.698) Percent of providers present at time of . 0.695 0.0901*** 0.812 -0.0431* 0.018 observation [.] [0.2378] (0.032) [0.2059] (0.026) (0.020) Average standardized effect 0.003 0.032 0.018 (0.023) (0.024) (0.017) Average standardized effect health 0.027 0.042 0.033 (0.033) (0.034) (0.024) Average standardized effect educ. -0.034 0.016 -0.006 (0.037) (0.036) (0.025) 134 Indonesia’s PNPM Generasi Program Table 34. Results for service provider level of effort, incentivized vs. non-incentivized   Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Midwives: Hours spent in outreach over 0.7961* -0.074 0.8700** 0.076 0.038 0.038 0.391 0.007 0.383 past 3 days (0.410) (0.337) (0.425) (0.389) (0.419) (0.400) (0.299) (0.305) (0.327) Hours spent providing public 0.536 -1.1020* 1.6380** 0.675 0.417 0.257 0.579 -0.248 0.8272* services over past 3 days (0.608) (0.594) (0.721) (0.619) (0.567) (0.585) (0.460) (0.419) (0.487) Hours spent providing private 0.212 -0.469 0.681 0.894 0.591 0.304 0.570 0.112 0.458 services over past 3 days (0.832) (0.826) (0.886) (0.674) (0.669) (0.644) (0.525) (0.524) (0.524) Total hours spent working over 1.477 -1.7182* 3.1956*** 1.6276* 0.936 0.692 1.5004** -0.224 1.7246** past 3 days (1.047) (1.039) (1.154) (0.951) (0.932) (0.884) (0.712) (0.728) (0.723) Number of posyandus attended 0.189 0.059 0.130 -0.162 0.053 -0.215 -0.009 0.064 -0.073 in past month (0.332) (0.227) (0.348) (0.247) (0.268) (0.324) (0.241) (0.195) (0.294) Number of hours midwife 0.137 0.181 -0.044 0.110 -0.083 0.192 0.127 0.032 0.095 spends per posyandu Teachers: (0.130) (0.120) (0.127) (0.152) (0.133) (0.153) (0.111) (0.095) (0.111) Percent present at time of 0.006 0.016 -0.010 0.000 0.008 -0.009 0.006 0.016 -0.010 interview (SD) (0.016) (0.015) (0.017) (0.011) (0.011) (0.012) (0.016) (0.015) (0.017) Percent present at time of 0.001 -0.010 0.011 -0.008 -0.015 0.007 -0.004 -0.013 0.009 interview (SMP) Final Impact Evaluation Report Annexes 135 136   Wave II Wave III AVERAGE Versi A Versi B Versi A Annexes Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) (0.015) (0.014) (0.014) (0.012) (0.012) (0.013) (0.010) (0.010) (0.010) Percent teaching at time of class -0.006 -0.050 0.044 -0.003 -0.012 0.009 -0.005 -0.028 0.023 observation (SD) (0.038) (0.042) (0.042) (0.040) (0.041) (0.038) (0.028) (0.029) (0.028) Indonesia’s PNPM Generasi Program Percent teaching at time of class -0.069 -0.052 -0.018 0.039 0.024 0.015 -0.010 -0.011 0.002 observation (SMP) Puskesmas: (0.044) (0.047) (0.049) (0.049) (0.048) (0.044) (0.033) (0.033) (0.032) Minutes wait at recent health 0.778 6.035 -5.257 2.409 1.281 1.128 1.696 3.042 -1.345 visits (3.637) (4.685) (3.953) (4.269) (4.224) (4.400) (3.033) (3.302) (3.320) Percent of providers present at 0.0714** 0.1090*** -0.038 -0.009 -0.0757** 0.0667** 0.030 0.006 0.024 time of observation (0.036) (0.039) (0.035) (0.029) (0.030) (0.030) (0.022) (0.023) (0.024) Average standardized effect 0.045 -0.040 0.0846*** 0.043 0.021 0.021 0.0409* -0.005 0.0463* (0.029) (0.028) (0.032) (0.028) (0.028) (0.030) (0.022) (0.020) (0.024) Average standardized effect 0.0892** -0.036 0.1250*** 0.056 0.030 0.026 0.0665** 0.000 0.0662** health (0.044) (0.038) (0.048) (0.040) (0.039) (0.040) (0.031) (0.028) (0.034) Average standardized effect -0.022 -0.046 0.024 0.023 0.009 0.014 0.002 -0.014 0.016 educ. (0.043) (0.044) (0.047) (0.041) (0.042) (0.042) (0.030) (0.029) (0.030) Table 35. Results for community efforts at service provision, monitoring, and participation     Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Treatment Effect (1) (2) (3) (4) (5) (6) Community effort at direct service provision: Number of posyandus in village 4.519 4.222 -0.045 4.323 0.162 0.067 [3.5043] [2.6797] (0.121) [2.9473] (0.161) (0.133) Number of posyandu meetings in past year at . 11.800 0.039 11.809 -0.087 -0.029 selected posyandus [.] [1.5199] (0.094) [1.9054] (0.089) (0.067) Number of kader at selected posyandus . 4.750 0.186 4.803 0.3241** 0.2604** Community effort at outreach [.] [1.8781] (0.115) [2.0707] (0.133) (0.104) Number of sweepings at selected posyandus in last . 5.965 -0.128 6.025 -0.377 -0.262 year [.] [5.9648] (0.333) [6.5037] (0.311) (0.228) Number of SD school committee meetings with . 2.217 -0.003 2.385 -0.063 -0.035 parents in past year [.] [2.5568] (0.119) [3.0552] (0.168) (0.109) Number of SMP school committee meetings with 2.309 2.178 -0.045 2.223 0.212 0.095 parents in past year Community effort at monitoring [1.9728] [1.5352] (0.097) [1.4214] (0.158) (0.106) Number of SD school committee members . 9.049 0.134 8.457 0.098 0.114 [.] [4.7201] (0.328) [3.6531] (0.328) (0.263) Number of SMP school committee members 8.259 8.944 -1.133 7.387 0.215 -0.404 [4.7625] [13.8010] (0.926) [3.9533] (0.298) (0.463) Number of SD school committee meetings with . 4.112 -0.245 4.174 -0.108 -0.171 teachers in past year [.] [5.2974] (0.311) [5.2103] (0.288) (0.230) Number of SMP school committee meetings with 4.476 3.860 0.303 3.462 0.5528* 0.4384** teachers in year Participation in health / education programs: [5.4650] [4.5890] (0.340) [3.3086] (0.291) (0.222) Participation in meetings about health education . 0.281 0.1093*** 0.323 0.000 0.1093*** Final Impact Evaluation Report [.] [0.4497] (0.036) [0.4680] 0.000 (0.036) Annexes 137 138     Wave II (2008) Wave III (2009) AVERAGE Annexes Generasi Generasi Average Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Treatment Effect (1) (2) (3) (4) (5) (6) Proportion of kids under 3 who own buku kupon . 0.167 0.1146*** 0.1146*** [.] [0.3892] (0.023) (0.023) Proportion of moms who own buku kupon . 0.455 0.1611*** 0.1611*** [.] [0.5222] (0.038) (0.038) Proportion of kids under 3 who use buku kupon . 0.083 0.0608*** 0.0608*** Indonesia’s PNPM Generasi Program [.] [0.2887] (0.017) (0.017) Proportion of moms who use buku kupon . 0.200 0.0782*** 0.0782*** [.] [0.4216] (0.021) (0.021) Proportion of kids under 3 with KIA 0.385 0.522 0.1226*** 0.575 0.0870*** 0.1041*** [0.4867] [0.4998] (0.018) [0.4947] (0.020) (0.015) Proportion of kids under 3 with KMS 0.573 0.474 -0.023 0.454 -0.004 -0.013 [0.4947] [0.4996] (0.023) [0.4981] (0.024) (0.017) Proportion of HH that think health services . 0.601 0.0413*** 0.0413*** improved over last 2 years [.] [0.4898] (0.013) (0.013) Proportion of HH that think education improved . 0.593 0.0419*** 0.0419*** over last 2 years Spillovers to other types of community activities [.] [0.4914] (0.013) (0.013) Participation in gotong royong (hours worked per 37.679 28.489 3.533 21.180 2.031 2.7145* household) [105.0800] [76.9129] (2.169) [48.8922] (2.060) (1.620) Women’s participation in women’s groups (number . 4.045 -0.234 -0.234 of meetings) [.] [6.7041] (0.265) (0.265) Women’s participation in government groups 0.099 0.073 0.023 0.127 -0.008 0.006 (number of meetings) [1.4825] [0.7677] (0.020) [1.1630] (0.036) (0.021) HH respondent’s participation in social groups 12.899 11.222 0.293 10.287 0.193 0.239 (number of meetings) [14.8305] [13.3580] (0.393) [11.9432] (0.426) (0.315)     Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Indicator Baseline Mean Control Mean Control Mean Treatment Effect Treatment Effect Treatment Effect (1) (2) (3) (4) (5) (6) Participation in general election 2009 . 0.969 0.003 0.003 [.] [0.1737] (0.004) (0.004) Average standardized effect 0.0607*** 0.0445*** 0.0619*** (0.014) (0.015) (0.011) Average standardized effect Community Effort 0.002 0.0453* 0.021 (0.021) (0.024) (0.017) Average standardized effect health/educ. 0.1951*** 0.0810*** 0.1565***   (0.031) (0.017) (0.022) Average standardized effect other community. 0.0314* 0.006 0.010   (0.017) (0.020) (0.015) Final Impact Evaluation Report Annexes 139 140 Table 36. Results for community efforts at service provision, monitoring, and participation, incentivized vs. non-incentivized   Wave II Wave III AVERAGE Annexes Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Community effort at direct service provision: Number of posyandus in -0.092 0.004 -0.096 0.128 0.196 -0.068 0.027 0.107 -0.080 village Indonesia’s PNPM Generasi Program (0.124) (0.147) (0.126) (0.178) (0.176) (0.148) (0.140) (0.151) (0.120) Number of posyandu meetings in past year at -0.003 0.082 -0.084 -0.112 -0.063 -0.049 -0.061 0.002 -0.063 selected posyandus (0.102) (0.111) (0.102) (0.112) (0.091) (0.100) (0.079) (0.076) (0.078) Number of kader at selected 0.174 0.197 -0.023 0.2890** 0.3577** -0.069 0.2349** 0.2854** -0.051 posyandus Community effort at outreach (0.113) (0.153) (0.138) (0.139) (0.171) (0.165) (0.105) (0.139) (0.133) Number of sweepings at selected posyandus in last -0.296 0.042 -0.338 -0.127 -0.6155* 0.4888* -0.186 -0.337 0.150 year (0.394) (0.377) (0.389) (0.342) (0.346) (0.295) (0.266) (0.257) (0.257) Number of SD school committee meetings with 0.066 -0.070 0.136 0.002 -0.125 0.126 0.031 -0.099 0.130 parents in past year (0.133) (0.133) (0.121) (0.181) (0.182) (0.137) (0.117) (0.119) (0.093) Number of SMP school committee meetings with -0.121 0.032 -0.153 0.213 0.210 0.003 0.066 0.125 -0.060 parents in past year Community effort at (0.113) (0.118) (0.126) (0.147) (0.223) (0.207) (0.103) (0.147) (0.140) monitoring Number of SD school 0.7613* -0.503 1.2638*** -0.003 0.195 -0.198 0.317 -0.085 0.401 committee members (0.392) (0.410) (0.478) (0.334) (0.402) (0.344) (0.287) (0.314) (0.297)   Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect Number of SMP school -0.845 -1.421 0.577 0.199 0.231 -0.032 -0.296 -0.511 0.215 committee members (0.993) (0.934) (0.539) (0.332) (0.332) (0.291) (0.498) (0.475) (0.297) Number of SD school committee meetings with -0.124 -0.367 0.243 -0.121 -0.096 -0.025 -0.129 -0.213 0.084 teachers in past year (0.358) (0.357) (0.354) (0.316) (0.319) (0.268) (0.255) (0.252) (0.211) Number of SMP school committee meetings with 0.477 0.132 0.345 0.530 0.5755* -0.045 0.4957* 0.381 0.115 teachers in year Participation in health / (0.424) (0.394) (0.455) (0.342) (0.346) (0.364) (0.262) (0.258) (0.269) education programs: Participation in meetings 0.1046** 0.1136*** -0.009 -0.018 0.000 -0.018 0.1020*** 0.1160*** -0.014 about health education (0.040) (0.036) (0.025) (0.023) 0.000 (0.023) (0.037) (0.036) (0.017) Proportion of kids under 3 0.1295*** 0.1014*** 0.028 0.1295*** 0.1014*** 0.028 who own buku kupon (0.026) (0.024) (0.018) (0.026) (0.024) (0.018) Proportion of moms who 0.1802*** 0.1439*** 0.0363* 0.1802*** 0.1439*** 0.0363* own buku kupon (0.039) (0.039) (0.022) (0.039) (0.039) (0.022) Proportion of kids under 3 0.0736*** 0.0498*** 0.0239* 0.0736*** 0.0498*** 0.0239* who use buku kupon (0.020) (0.018) (0.013) (0.020) (0.018) (0.013) Proportion of moms who 0.0812*** 0.0754*** 0.006 0.0812*** 0.0754*** 0.006 use buku kupon (0.022) (0.023) (0.014) (0.022) (0.023) (0.014) Final Impact Evaluation Report Annexes 141 142   Wave II Wave III AVERAGE Versi A Versi B Versi A Annexes Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect Proportion of kids under 3 0.1215*** 0.1237*** -0.002 0.0837*** 0.0899*** -0.006 0.1020*** 0.1061*** -0.004 with KIA (0.020) (0.024) (0.024) (0.023) (0.024) (0.024) (0.017) (0.019) (0.020) Proportion of kids under 3 -0.001 -0.0455* 0.0447* 0.018 -0.025 0.0433* 0.009 -0.0348* 0.0437** with KMS Indonesia’s PNPM Generasi Program (0.026) (0.026) (0.026) (0.026) (0.028) (0.026) (0.019) (0.020) (0.020) Proportion of HH that think health services 0.0284* 0.0537*** -0.025 0.0284* 0.0537*** -0.025 improved over last 2 years (0.015) (0.015) (0.016) (0.015) (0.015) (0.016) Proportion of HH that think education improved 0.0300** 0.0533*** -0.023 0.0300** 0.0533*** -0.023 over last 2 years Spillovers to other types of (0.015) (0.015) (0.014) (0.015) (0.015) (0.014) community activities Participation in gotong royong (hours worked per -0.971 8.1457** -9.1170*** 1.468 2.570 -1.102 0.512 4.8860** -4.3744** household) (2.287) (3.221) (3.497) (2.055) (2.660) (2.406) (1.668) (2.191) (2.195) Women’s participation in women’s groups (number of -0.140 -0.328 0.188 -0.140 -0.328 0.188 meetings) (0.314) (0.310) (0.330) (0.314) (0.310) (0.330) Women’s participation in government groups 0.026 0.020 0.006 -0.015 -0.002 -0.013 0.004 0.009 -0.005 (number of meetings) (0.024) (0.022) (0.023) (0.038) (0.043) (0.036) (0.023) (0.026) (0.024)   Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect HH respondent’s participation in social 0.676 -0.093 0.769 0.238 0.149 0.089 0.429 0.054 0.375 groups (number of meetings) (0.442) (0.474) (0.480) (0.482) (0.515) (0.522) (0.341) (0.390) (0.379) Participation in general 0.006 -0.001 0.0065* 0.006 -0.001 0.0065* election 2009 (0.005) (0.005) (0.004) (0.005) (0.005) (0.004) Average standardized effect 0.0727*** 0.0494*** 0.023 0.0430*** 0.0460** -0.003 0.0683*** 0.0561*** 0.012 (0.016) (0.017) (0.016) (0.015) (0.019) (0.018) (0.012) (0.014) (0.013) Average standardized effect 0.014 -0.009 0.023 0.0431* 0.048 -0.004 0.026 0.017 0.010 Community Effort (0.022) (0.025) (0.023) (0.025) (0.031) (0.029) (0.018) (0.022) (0.019) Average standardized effect 0.2224*** 0.1702*** 0.0522* 0.0754*** 0.0864*** -0.011 0.1682*** 0.1464*** 0.022 health/educ.   (0.035) (0.034) (0.029) (0.020) (0.020) (0.020) (0.025) (0.024) (0.020) Average standardized effect 0.021 0.0427** -0.022 0.010 0.003 0.008 0.012 0.009 0.003 other community. (0.018) (0.021) (0.020) (0.021) (0.024) (0.022) (0.016) (0.018) (0.017) Final Impact Evaluation Report Annexes 143 144 Table 37. Service prices and supply     Wave II (2008) Wave III (2009) AVERAGE Annexes Generasi Generasi Average Baseline Control Control Indicator Treatment Treatment Treatment Mean Mean Mean Effect Effect Effect (1) (2) (3) (4) (5) (6) Midwife: Fee charged for child birth at private practice 292400.000 340500.000 16066.1377** 368800.000 17587.1797*** 16899.4766*** [1.411e+05] [1.390e+05] (6261.651) [1.434e+05] (6628.806) (5303.498) Number of child births at private practice in Indonesia’s PNPM Generasi Program 2.969 2.591 -0.175 3.054 -0.102 -0.136 last month [4.1740] [3.2213] (0.216) [3.5011] (0.191) (0.156) Fee charged for child birth at gov’t practice 139000.000 186500.000 24515.3672* 191800.000 19871.049 21973.5293** [1.501e+05] [1.662e+05] (12536.684) [1.697e+05] (12102.598) (9206.056) Number of child births at gov’t practice in last 6.930 3.123 -0.012 1.881 1.7795*** 0.9475** month [30.2697] [8.7605] (0.790) [5.0591] (0.630) (0.476) Fee charged for child birth (avg. of private & 248100.000 307800.000 18618.7813** 345600.000 6761.731 12257.9414* gov’t ) [1.485e+05] [1.561e+05] (8483.630) [1.493e+05] (9786.132) (6971.444) Total number of child births in last month 9.898 5.701 -0.214 4.935 1.6894** 0.806 [30.7320] [9.2988] (0.820) [6.6173] (0.661) (0.507) Fee paid by mother for normal childbirth 1121000 1284000.000 2471000.000 1638000.000 222100.000 1314000 [1.874e+06] [2.048e+06] (2475000.000) [2.496e+06] (368900.000) -1216000 Fee charged for ANC at private practice 12308.043 14642.106 94.402 15367.052 1895.917 1093.801 [8025.0337] [7399.2935] (481.915) [7969.7622] (1249.632) (745.351) Number of ANC visits at private practice in 3.235 4.128 -0.8038** 4.292 0.012 -0.366 last month [8.2073] [8.3705] (0.372) [5.4555] (0.342) (0.265) Fee charged for ANC at gov’t practice 2617.852 2715.972 76.135 2585.617 -72.753 -5.753 [5884.7715] [4145.5962] (335.065) [3570.6465] (227.364) (212.145) Number of ANC visit at gov’t practice in last 12.456 6.967 0.209 7.070 1.8859* 1.109 month [36.1089] [13.2213] (1.364) [14.4124] (1.016) (0.830)     Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Baseline Control Control Indicator Treatment Treatment Treatment Mean Mean Mean Effect Effect Effect (1) (2) (3) (4) (5) (6) Fee charged for ANC visit (avg. of private & 6036.449 8927.499 -597.858 9290.846 805.702 160.272 gov’t) [7471.1411] [7710.5264] (518.962) [7764.8867] (740.525) (496.521) Total number of ANC visits in last month 15.691 11.095 -0.639 11.362 1.8755* 0.709 [38.4889] [15.4716] (1.463) [15.6925] (1.100) (0.902) Fee paid by mother for ANC visit 17338.510 19562.107 1000.604 19058.764 2195.936 1593.669 [28614.9609] [30578.2129] (1822.869) [20377.2188] (2129.526) (1472.720) Fee charged for family planning visit at private . 14292.106 355.477 15142.046 -300.867 -9.668 practice [.] [8826.5283] (297.993) [4660.4409] (428.997) (286.598) Number of family planning visits at private . 34.320 0.446 40.158 3.099 1.911 practice in last month [.] [38.2805] (3.249) [46.5810] (3.032) (2.352) Fee charged for family planning visit at gov’t . 8073.427 -193.441 7965.517 -1102.9316** -696.751 practice [.] [13691.1299] (643.838) [7618.8853] (528.205) (450.579) Number of family planning visits at gov’t . 15.958 1.236 19.593 2.440 1.874 practice in last month [.] [31.5673] (4.713) [40.9436] (3.830) (3.061) Fee charged for family planning visit (avg. . 12658.750 -56.084 13135.080 -113.114 -86.965 private & gov’t) [.] [9376.9297] (343.456) [5726.6289] (453.843) (323.458) Total number of family planning visits in last . 41.764 -0.415 51.978 4.946 2.480 month [.] [42.9055] (4.476) [59.1947] (4.336) (3.243) Fee paid by mother for family planning visit 13734.921 15526.718 -11.637 15591.503 79.395 36.643 Puskesmas: [4013.8237] [4580.6084] (320.938) [4289.8433] (366.192) (252.678) Final Impact Evaluation Report Annexes 145 146     Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Annexes Baseline Control Control Indicator Treatment Treatment Treatment Mean Mean Mean Effect Effect Effect (1) (2) (3) (4) (5) (6) Normal child birth at Puskesmas—fee 140000.000 183900.000 17226.805 185100.000 -20460.000 -3594.634 charged by midwife [1.295e+05] [1.675e+05] (21378.611) [1.332e+05] (18333.529) (14617.567) Normal child birth at Puskesmas—quantity by 45.023 43.808 -12.138 39.444 -1.987 -6.532 midwife Indonesia’s PNPM Generasi Program Posyandu: [63.1350] [84.1984] (14.966) [44.3125] (5.291) (7.006) Posyandu—Fee for visit . 213.692 -40.261 284.013 -79.3349* -61.2154* [.] [923.0052] (42.215) [1312.7780] (45.851) (31.522) Posyandu—quantity of kids weighed at last . 41.055 8.5895*** 40.282 11.1555*** 9.9782*** meeting where service was offered [.] [26.5202] (1.592) [26.7517] (1.693) (1.414) Posyandu—quantity of kids with nutritional . 34.831 13.9572*** 33.472 16.3303*** 15.2437*** supplement at last meeting where service was offered [.] [28.4862] (1.716) [29.6807] (1.819) (1.465) Posyandu—quantity of kids immunized at last . 11.874 3.2217*** 13.174 2.3748** 2.7589*** meeting where service was offered [.] [16.0090] (0.969) [19.8409] (1.082) (0.805) Posyandu—quantity of moms receiving ANC . 4.546 1.9556*** 5.332 0.830 1.3495*** visits at last meeting where service was offered [.] [9.0516] (0.582) [15.1231] (0.714) (0.513) Posyandu—quantity of moms receiving iron . 4.804 2.2504*** 5.374 1.4837** 1.8366*** pills at last meeting where service was offered [.] [10.3386] (0.664) [15.6095] (0.752) (0.552) Posyandu—quantity of kids receiving Vitamin . 44.068 8.3930*** 42.750 11.6894*** 10.1789*** A at last meeting where service was offered [.] [38.4256] (2.450) [36.3574] (2.517) (2.069) Posyandu—quantity of moms receiving family . 2.829 4.406 3.510 0.228 2.159 planning pills at last meeting where service was offered [.] [8.0230] (3.329) [15.3820] (0.642) (1.573)     Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Baseline Control Control Indicator Treatment Treatment Treatment Mean Mean Mean Effect Effect Effect (1) (2) (3) (4) (5) (6) Posyandu—quantity of moms receiving family . 2.807 -0.071 3.356 -0.010 -0.038 planning injections at last meeting where service was [.] [9.4971] (0.540) [14.8391] (0.674) (0.460) offered Schools: . 33958.297 -9852.431 121200.000 -61690.000 -37800.000 SD—annual cost of school for TA 07/08 [.] [1.518e+05] (13107.263) [1.617e+06] (73027.086) (39400.621) . 153.551 -6.819 163.867 19.498 7.370 SD—number of students enrolled at TA [.] [66.3133] (6.459) [74.7577] (17.038) (10.296) 07/08 . 153.973 -5.098 164.463 -1.378 -3.092 SD—number of students enrolled in TA [.] [66.3197] (6.342) [73.5988] (6.041) (5.589) 08/09 16713.465 28026.555 1645.832 13575.885 -181.847 661.746 SD—cost of school from parents for previous [94179.7656] [1.355e+05] (3728.477) [46820.0430] (3605.460) (2771.815) semester 66477.492 774300.000 -326500.000 215900.000 -7811.033 -152900.000 SMP—annual cost of school for TA 07/08 [2.483e+05] [5.773e+06] (322400.000) [9.901e+05] (42954.406) (159800.000) 282.371 301.457 -16.0561** 307.338 14.084 0.354 SMP—number of students enrolled at TA [246.3422] [235.5591] (7.963) [237.4138] (13.593) (8.961) 07/08 303.265 311.362 -8.080 317.172 8.738 1.079 SMP—number of students enrolled in TA [248.2967] [243.6746] (7.897) [244.3427] (11.981) (7.957) 08/09 69681.758 111200.000 -4346.149 101100.000 -3.321e+04* -2.051e+04* SMP—cost of school from parents for [3.988e+05] [2.252e+05] (10311.505) [2.577e+05] (17276.186) (10843.114) previous semester Average standardized effect fees -0.075 -0.012 -0.044 Final Impact Evaluation Report (0.059) (0.025) (0.032) Annexes 147 148     Wave II (2008) Wave III (2009) AVERAGE Generasi Generasi Average Annexes Baseline Control Control Indicator Treatment Treatment Treatment Mean Mean Mean Effect Effect Effect (1) (2) (3) (4) (5) (6) Average standardized effect health fees -0.106 -0.025 -0.066 (0.075) (0.030) (0.041) Average standardized effect education fees 0.032 0.034 0.033 (0.034) (0.027) (0.021) Average standardized effect midwife fees -0.119 -0.047 -0.0838* Indonesia’s PNPM Generasi Program (0.087) (0.035) (0.046) Average standardized effect puskesmas fees -0.103 0.153 0.024 (0.118) (0.127) (0.089) Average standardized effect school fees 0.032 0.034 0.033 (0.034) (0.027) (0.021) Average standardized effect posyandu fees 0.044 0.0608* 0.0543* (0.046) (0.035) (0.028) Average standardized effect quantities 0.0727** 0.1374*** 0.0936*** (0.032) (0.032) (0.024) Average standardized effect health quantities 0.1041*** 0.1497*** 0.1108*** (0.036) (0.035) (0.024) Average standardized effect education -0.069 0.082 0.016 quantities (0.051) (0.072) (0.054) Average standardized effect midwife quantities -0.021 0.1290** 0.039 (0.039) (0.056) (0.031) Average standardized effect puskesmas -0.145 -0.045 -0.099 quantities (0.169) (0.114) (0.100) Average standardized effect school quantities -0.069 0.082 0.016 (0.051) (0.072) (0.054) Average standardized effect posyandu -0.1185*** -0.0470*** -0.0838*** quantities   (0.087) (0.035) (0.046) Table 38. Service prices and supply, incentivized vs. non-incentivized Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Midwife: Fee charged for child 13145.7207* 18949.5234*** -5803.804 15794.7549** 19427.3438*** -3632.588 14665.2598** 19153.0586*** -4487.8 birth at private practice (7860.749) (7129.304) (8374.794) (7438.127) (7461.506) (6807.363) (6225.63) (5971.29) (6077.35) Number of child births at private practice in last -0.099 -0.250 0.151 -0.147 -0.060 -0.087 -0.13 -0.141 0.011 month (0.262) (0.222) (0.221) (0.225) (0.221) (0.230) (0.19) (0.17) (0.17) Fee charged for child 27426.9473* 21694.695 5732.252 24515.9727* 15477.616 9038.356 25920.7715** 18199.309 7721.463 birth at gov’t practice (14805.148) (14775.625) (15701.604) (13554.463) (14328.847) (13974.700) (10565.90) (11117.17) (11567.04) Number of child births at gov’t practice in last -0.519 0.488 -1.007 2.2665*** 1.3062* 0.960 1.0211* 0.876 0.146 month (0.903) (1.127) (1.294) (0.856) (0.742) (0.985) (0.59) (0.63) (0.78) Fee charged for child birth (avg. of private & 24158.8730*** 12906.093 11252.780 5414.223 8192.665 -2778.443 13714.8086* 10731.815 2982.993 gov’t ) (9221.418) (10614.481) (10432.307) (10509.979) (11734.011) (10529.023) (7756.79) (8272.30) (7924.44) Total number of child -0.642 0.209 -0.851 2.2279** 1.167 1.061 0.942 0.673 0.269 births in last month (0.930) (1.154) (1.307) (0.885) (0.789) (1.030) (0.63) (0.67) (0.81) Fee paid by mother for 6824000.000 -2402000.000 9226000.000 172200.000 274800.000 -102600.000 3201000 -734800 3935000 normal child birth (5776000.000) (2476000.000) (7487000.000) (432400.000) (419900.000) (425700.000) (2670000.00) (1077000.00) (3328000.00) Fee charged for ANC at -410.655 587.682 -998.336 1130.766 2666.979 -1536.213 432.59 1750.901 -1318.311 private practice (565.747) (573.075) (619.275) (769.402) (1866.353) (1382.672) (503.76) (1118.55) (895.15) Final Impact Evaluation Report Annexes 149 150 Wave II Wave III AVERAGE Versi A Versi B Versi A Annexes Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Number of ANC visits at private practice in last -0.8657** -0.7424* -0.123 -0.275 0.291 -0.566 -0.5600* -0.176 -0.384 month (0.410) (0.411) (0.348) (0.397) (0.420) (0.452) (0.31) (0.31) (0.33) Fee charged for ANC at -129.658 262.123 -391.781 -202.630 51.090 -253.720 -165.59 143.027 -308.616 Indonesia’s PNPM Generasi Program gov’t practice (332.287) (437.960) (403.126) (267.801) (252.958) (255.069) (228.27) (249.74) (223.42) Number of ANC visit at gov’t practice in last 0.543 -0.121 0.664 2.4392** 1.344 1.095 1.57 0.655 0.915 month (1.694) (1.412) (1.516) (1.174) (1.251) (1.328) (1.00) (0.96) (1.05) Fee charged for ANC visit -694.384 -503.060 -191.325 225.993 1382.486 -1156.493 -220.777 537.137 -757.914 (avg. of private & gov’t) (592.016) (590.519) (567.468) (691.955) (951.437) (775.923) (512.89) (622.24) (567.94) Total number of ANC -0.336 -0.938 0.602 2.1737* 1.583 0.591 1.009 0.414 0.595 visits in last month (1.818) (1.522) (1.642) (1.295) (1.350) (1.466) (1.08) (1.06) (1.16) Fee paid by mother for 795.175 1217.752 -422.578 2080.765 2320.727 -239.961 1440.938 1757.107 -316.17 ANC visit (2474.196) (2437.362) (3297.154) (2005.891) (3074.491) (2896.810) (1653.35) (2002.57) (2177.08) Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Fee charged for family planning visit at private 443.170 269.185 173.985 -195.500 -404.951 209.452 88.617 -106.59 195.207 practice (303.750) (434.276) (452.555) (646.361) (377.320) (615.865) (401.82) (317.64) (440.66) Number of family planning visits at private -1.690 2.532 -4.222 3.101 3.098 0.003 1.073 2.744 -1.671 practice in last month (4.000) (3.996) (4.591) (3.509) (3.575) (3.653) (2.80) (2.82) (3.07) Fee charged for family planning visit at gov’t 120.100 -472.234 592.334 -975.589 -1218.6964** 243.108 -497.614 -876.0026* 378.389 practice (640.245) (845.715) (785.829) (631.466) (585.901) (601.473) (511.61) (527.08) (517.52) Number of family planning visits at gov’t -4.402 6.739 -11.142 3.170 1.728 1.442 0.001 3.702 -3.701 practice in last month (6.340) (8.590) (11.731) (4.282) (4.883) (5.070) (3.69) (4.64) (5.73) Fee charged for family planning visit (avg. 109.798 -219.277 329.076 60.558 -280.506 341.064 83.459 -252.777 336.236 private & gov’t) (327.849) (481.775) (454.455) (630.211) (429.640) (575.610) (406.04) (370.29) (429.10) Total number of family planning visits in last -5.691 4.822 -10.512 5.658 4.259 1.399 0.688 4.232 -3.544 month (5.509) (7.157) (8.987) (4.913) (5.307) (5.427) (3.78) (4.41) (5.04) Fee paid by mother for -39.187 16.081 -55.267 -127.257 337.242 -464.499 -96.493 186.736 -283.23 family planning visit (387.521) (403.092) (460.853) (417.303) (466.121) (491.966) (298.92) (327.16) (369.16) Puskesmas: Normal child birth at Puskesmas—fee charged 10381.356 24633.846 -14250.000 -23920.000 -16660.000 -7267.054 -8440.709 1689.567 -10130 by midwife Final Impact Evaluation Report (25297.057) (23783.584) (23969.926) (20233.248) (21722.717) (20108.373) (16350.92) (16862.83) (15587.50) Annexes 151 152 Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Annexes Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Normal child birth at Puskesmas— quantity by -17.323 -7.153 -10.169 -0.835 -3.125 2.290 -7.914 -5.183 -2.731 midwife (15.961) (16.190) (11.741) (5.734) (5.864) (4.705) (7.44) (7.64) (5.60) Posyandu: Indonesia’s PNPM Generasi Program Posyandu— Fee for visit -40.939 -39.572 -1.367 -63.328 -94.7143* 31.386 -52.124 -70.1740** 18.05 (48.052) (42.952) (34.401) (48.136) (50.187) (35.767) (33.52) (35.01) (26.95) Posyandu — quantity of kids weighed at last 8.8299*** 8.3422*** 0.488 12.9883*** 9.4164*** 3.572 11.1293*** 8.8449*** 2.285 meeting where service was offered (1.933) (2.121) (2.511) (2.285) (1.797) (2.332) (1.88) (1.62) (2.09) Posyandu– quantity of kids with nutritional 13.6680*** 14.2499*** -0.582 17.3945*** 15.3175*** 2.077 15.7339*** 14.7638*** 0.97 supplement at last meeting where (2.163) (2.102) (2.534) (2.348) (2.049) (2.485) (1.92) (1.73) (2.17) service was offered Posyandu– quantity of kids immunized at last 4.1219*** 2.3301* 1.792 3.1876** 1.594 1.593 3.6087*** 1.9315* 1.677 meeting where service was offered (1.178) (1.229) (1.467) (1.243) (1.242) (1.228) (0.93) (1.01) (1.09) Posyandu– quantity of moms receiving ANC 2.3455*** 1.5625* 0.783 1.030 0.638 0.392 1.6319*** 1.0728* 0.559 visits at last meeting where (0.605) (0.840) (0.904) (0.808) (0.749) (0.621) (0.56) (0.61) (0.57) service was offered Posyandu– quantity of 2.7863*** 1.7083* 1.078 1.346 1.6175* -0.272 1.9839*** 1.6911** 0.293 moms receiving iron pills at last meeting where (0.755) (0.936) (1.076) (0.827) (0.839) (0.716) -0.60 -0.68 -0.65 service was offered Posyandu– quantity of 10.7649*** 5.9622** 4.803 14.9044*** 8.6119*** 6.2926* 13.0215*** 7.3714*** 5.6501** kids receiving Vitamin A at last meeting where (3.027) (2.844) (3.292) (3.475) (2.583) (3.471) (2.70) (2.26) (2.81) service was offered Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Posyandu– quantity of moms receiving family 8.889 -0.088 8.976 -0.152 0.591 -0.742 3.823 0.531 3.292 planning pills at last meeting where (6.799) (1.329) (7.232) (0.709) (0.696) (0.575) (3.02) (0.68) (3.03) service was offered Posyandu– quantity of moms receiving family 0.194 -0.336 0.529 -0.146 0.121 -0.267 -0.007 -0.068 0.061 planning injections at last meeting (0.651) (0.683) (0.791) (0.707) (0.767) (0.604) (0.52) (0.53) (0.51) where service was offered Schools: SD - Annual cost of -5621.315 -14140.000 8522.981 -33510.000 -88600.000 55090.492 -19520 -55740 36220.6328* school for TA 07/08 (15757.546) (13487.735) (13230.781) (64832.281) (83692.430) (35470.367) (34984.65) (45762.17) (21465.90) SD – Number of students -5.854 -7.797 1.943 19.844 19.167 0.677 7.906 6.845 1.061 enrolled at TA 07/08 (7.882) (7.551) (8.437) (19.346) (19.326) (18.255) (11.93) (12.01) (12.21) SD– Number of students -4.766 -5.435 0.669 -3.482 0.631 -4.113 -4.186 -2.019 -2.167 enrolled in TA 08/09 (7.723) (7.392) (8.221) (6.916) (7.455) (7.844) (6.70) (6.73) (7.44) SD–Cost of school from parents for previous 4550.471 -1250.907 5801.377 2134.465 -2254.846 4389.310 3235.942 -1760.189 4996.131 semester (5483.050) (3507.645) (5384.107) (3358.656) (4392.255) (3235.125) (3195.87) (3119.93) (3058.62) SMP– annual cost of -275600.000 -377700.000 102000.000 -11710.000 -4015.936 -7690.910 -134800 -170800 35951.957 school for TA 07/08 (325000.000) (344400.000) (178300.000) (44060.379) (51037.434) (41428.430) (157000.00) (172100.00) (79981.49) SMP – Number of students enrolled at TA -8.024 -24.1228** 16.0990* 12.939 15.217 -2.277 2.925 -2.208 5.133 07/08 (7.132) (10.920) (9.573) (16.254) (15.313) (16.031) (10.21) (10.69) (10.77) Final Impact Evaluation Report Annexes 153 154 Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Annexes Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) SMP – Number of students enrolled in TA -4.368 -11.807 7.440 5.770 11.663 -5.893 0.816 1.339 -0.523 08/09 (7.277) (10.496) (8.843) (14.353) (13.192) (13.610) (9.24) (9.19) (9.27) SMP–Cost of school Indonesia’s PNPM Generasi Program from parents for previous -616.899 -8422.964 7806.065 -3.161e+04* -3.473e+04* 3118.445 -17890 -2.315e+04* 5261.384 semester (12157.008) (12587.983) (13621.094) (17972.979) (18929.580) (13038.866) (11693.67) (12305.38) (10260.56) 13145.7207* 18949.5234*** -5803.804 15794.7549** 19427.3438*** -3632.588 14665.2598** 19153.0586*** -4487.8 Average standardized -0.189 0.037 -0.213 -0.005 -0.023 0.022 -0.085 -0.003 -0.078 effect fees (0.131) (0.059) (0.169) (0.025) (0.031) (0.028) (0.063) (0.031) (0.077) Average standardized -0.229 0.033 -0.261 -0.009 -0.043 0.034 -0.111 -0.018 -0.093 effect health fees (0.168) (0.075) (0.217) (0.031) (0.039) (0.036) (0.080) (0.040) (0.099) Average standardized -0.049 0.055 -0.044 0.007 0.044 -0.020 0.003 0.0464** -0.0276* effect education fees (0.047) (0.033) (0.036) (0.020) (0.032) (0.020) (0.017) (0.023) (0.015) Average standardized -0.265 0.047 -0.312 -0.029 -0.066 0.037 -0.138 -0.025 -0.113 effect midwife fees (0.196) (0.088) (0.252) (0.035) (0.045) (0.040) (0.092) (0.046) (0.115) Average standardized -0.062 -0.147 0.085 0.179 0.125 0.054 0.056 -0.011 0.067 effect puskesmas fees (0.139) (0.131) (0.132) (0.140) (0.150) (0.139) (0.100) (0.103) (0.095) Average standardized -0.049 0.055 -0.044 0.007 0.044 -0.020 0.003 0.0464** -0.0276* effect school fees (0.047) (0.033) (0.036) (0.020) (0.032) (0.020) (0.017) (0.023) (0.015) Average standardized 0.045 0.043 0.002 0.049 0.0726* -0.024 0.046 0.0623** -0.016 effect posyandu fees (0.052) (0.047) (0.037) (0.037) (0.038) (0.027) (0.030) (0.031) (0.024) Wave II Wave III AVERAGE Versi A Versi B Versi A Versi A Versi B Versi A Versi A Versi B Versi A Average Average Average Indicator Treatment Treatment Additional Treatment Treatment Additional Treatment Treatment Additional Effect Effect Effect Effect Effect Effect Effect Effect Effect (1) (2) (3) (4) (5) (6) (7) (8) (9) Average standardized 0.1007** 0.044 0.057 0.1564*** 0.1191*** 0.037 0.1086*** 0.0789*** 0.030 effect quantities (0.049) (0.035) (0.058) (0.038) (0.040) (0.043) (0.030) (0.030) (0.035) Average standardized 0.1346** 0.0732* 0.061 0.1751*** 0.1251*** 0.050 0.1291*** 0.0929*** 0.036 effect health quantities (0.057) (0.038) (0.066) (0.040) (0.043) (0.047) (0.031) (0.030) (0.036) Average standardized effect education -0.052 -0.086 0.034 0.072 0.092 -0.020 0.017 0.016 0.001 quantities (0.061) (0.059) (0.064) (0.083) (0.084) (0.083) (0.063) (0.063) (0.067) Average standardized -0.043 0.000 -0.042 0.1541** 0.105 0.050 0.038 0.039 -0.002 effect midwife quantities (0.046) (0.050) (0.056) (0.067) (0.070) (0.080) (0.037) (0.040) (0.046) Average standardized effect puskesmas -0.207 -0.086 -0.122 -0.019 -0.071 0.052 -0.119 -0.078 -0.041 quantities (0.180) (0.182) (0.132) (0.123) (0.126) (0.101) (0.106) (0.109) (0.080) Average standardized -0.052 -0.086 0.034 0.072 0.092 -0.020 0.017 0.016 0.001 effect school quantities (0.061) (0.059) (0.064) (0.083) (0.084) (0.083) (0.063) (0.063) (0.067) Average standardized -0.2652*** 0.0465*** -0.312 -0.0294*** -0.0660*** 0.037 -0.1375*** -0.0248*** -0.1127* effect posyandu quantities (0.196) (0.088) (0.252) (0.035) (0.045) (0.040) (0.092) (0.046) (0.115) Final Impact Evaluation Report Annexes 155 Annexes 156 Indonesia’s PNPM Generasi Program SEKRETARIAT POKJA PENGENDALI Kementerian Koordinator Bidang Kesejahteraan Rakyat Jl. Medan Merdeka Barat No. 3, Jakarta Pusat 10110 Indonesia Telp/Fax : (021) 345 9077 Email : info@pnpm-mandiri.org Website : www.pnpm-mandiri.org PNPM SUPPORT FACILITY Jalan Diponegoro No.72 Menteng, Jakarta Pusat, 10310 Indonesia Phone: (62 21) 3148175 Fax: (62 21) 31903090