Lao PDR: Early Childhood Education Project IDA Credit Number 5370-LA and IDA Grant Number H910-LA Implementation Support Review, May 14-24, 2018 Aide-Memoire I. INTRODUCTION An implementation support mission for the Early Childhood Education Project (ECE project) was carried out in the period May 14–24, 2018. The objectives of the mission were to: (i) take stock of the implementation progress; (ii) present and discuss the results from the midline impact evaluation survey and the associated results framework indicators; and (iii) review procurement plan implementation, safeguards compliance, and financial management. The mission consisted of Mr. Plamen Danchev (Senior Education Specialist and Task Team Leader), Dr. Sally Brinkman (Senior Early Childhood Development Research Advisor), Mr. Phouvanh Phommalangsy (Education Specialist), Ms. Achariya Kohtbantau (Education Consultant), Ms. Emiko Naka (Education Consultant), Ms. Siriphone Vanitsaveth (Senior Financial Management Specialist), Ms. Saysi Sayaseng (Financial Management Consultant), Mr. Latharo Lor (Procurement Specialist), Ms. Waraporn Hirunwatsiri (Senior Environmental Specialist), Ms. Chansouk Insouvanh (Social Safeguards Consultant) and Ms. Boualamphan Phouthavisouk (Team Assistant). The team wishes to thank the Ministry of Education and Sports (MOES) and Development Partners (DPs) for their guidance and valuable insights. Special appreciation goes to H.E. Assoc. Prof. Dr Khamphay Sisavanh, Vice Minister, Ms. Khampaseuth Kitignavong, Deputy Director General of Department of Planning (DOP), the staff of the Education Sector Plan Coordinating Unit (ECU) and all MOES departments who contributed to the achievement of the mission’s objectives. The aide- memoire was discussed at the wrap-up meeting on May 24, 2018 with Ms. Khampaseuth Kitignavong as the chair and the final version of the aide-memoire was endorsed by the Bank management. II. PROJECT DATA Table 1: Key Project Data Project Data USD (millions) Original Project Amount 28 Total Disbursement 14.9 Disbursement in FY 3.61 Closing Date 31-Dec-2020 III. IMPLEMENTATION PROGRESS AND KEY FINDINGS Progress towards achieving the Project Development Objective (PDO) The PDO of the project is to increase coverage and improve the quality of early childhood education services for 3 to 5-year-olds in target districts. The accomplishment of the PDO is measured through: 1 This amount does not include WA# G35 in the total amount of USD 182,829.99, received by the Bank’s WFA on May 30, 2018, as we ll as a new WA of USD 4.8M to be received by WFA by end June 2018. As such, actual disbursement in FY18 alone will likely range between USD 3.77M (13.5% of the total grant and credit amount) and USD 8.57M (30.6% of the total grant and credit amount) 1 (i) Enrollment rates in ECE programs. The end targets are overachieved at mid-point by a large margin for both children aged 3-5 years and for age 5 years. It is important to note that the data used for these indicators is based on actual attendance rates, rather than mere enrollments as per school registers. These results confirm the full uptake and positive community reception of the ECE programs. Considering that all planned access interventions (new constructions and teacher trainings) will be completed well before the original closing date, this outcome is ringfenced from any future falters. Increasing access to ECE programs is a key objective of the Lao PDR Education Sector Development Plan for the period 2016-2020 and the Bank’s Country Partnership Framework (2017-2021). The access provided through the project has been based on low-cost interventions, reinforcing the project’s efficiency. This outcome contributes to the quantitative targets of Lao PDR’s Sustainable Development Goal (SDG) 4.22. (ii) Improved children stimulation at home. The midline target is partially met and there are strong indications for further gains. The community awareness campaign (CAC) uptake exceeds twice the expected levels, while the impact on stimulation at midline detects behavior change within just twelve months of CAC implementation. Prospects are further boosted by (i) the project’s work to enhance the content of CAC by adopting materials and approaches developed for the Social and Behavior Change Communication (SBCC) modality, (ii) the significant number of upcoming refresher trainings, and (iii) the increased availability of age appropriate toys and books in the communities (toy libraries). From this point until the new closing date of December 2020, the impact can only grow and the end targets are highly likely to be met. Improving parenting practices is a key objective of the Lao PDR Education Sector Development Plan for the period 2016-2020 and is at the heart of the cross sectoral efforts of the Bank and the Government of Lao PDR to improve nutrition and child development outcomes through SBCC. The CAC is based on a low-cost design, confirming the project’s efficiency. This outcome is an indirect but important contribution for meeting the qualitative aspect of Lao PDR’s SDG 4.2. (iii) Improved overall child development outcomes. The midline target is overachieved for all development domains, and significantly exceeds the targets for emergent literacy and numeracy. In view of the ongoing classroom observations and the upcoming refresher trainings for the teachers, this indicator is on track to meet or overachieve its endline target, considering the extended implementation time until December 2020. Increasing quality to ECE programs is a key objective of the Lao PDR Education Sector Development Plan for the period 2016-2020 and the Bank’s Country Partnership Framework (2017-2021). The low-cost access to ECE programs under the project has brought about significant gains in child development, proving the project’s efficiency. This outcome directly contributes to the qualitative targets of Lao PDR’s SDG 4.2. (iv) Quality support provided to teachers. The midline target for the share of teachers who received feedback based on classroom observations at least once annually is overachieved by a large margin. In fact, most project teachers have received two or three support visits by the district education officers. The routine for implementation has been developed and resources are available to support greater number of classroom observations until the new project closing date. Considering the project extension by 17 months to December 2020 (approved in May 2018), the impressive achievements so far, the solutions found for past implementation hurdles, the implementation routine gained at local level, the lack of underlying factors that may cause an “eclipse” of child development outcomes until the project closing, the high relevance of the PDO and the 2 SDG 4.2: By 2030 ensure that all girls and boys have access to quality early childhood development, care and pre-primary education so that they are ready for primary education. 2 excellent efficiency of the project interventions, the progress towards achieving the PDOs is on track and expectations are high for exceeding all targets. Implementation Progress Presently all project activities are under full implementation, adhering to their implementation schedules. The independent process monitoring survey results suggest significant gains in implementation quantity and quality, driven by (i) the extensive capacity building efforts to improve fidelity of program implementation at the community level, and (ii) the timely release of project resources to communities (toy libraries and their regular replenishments, teaching and learning aids, playgrounds, village operating grants, stipends for community playgroup caregivers and toy librarians, among others). Past issues of delayed transfers to villages and suboptimal coordination across the central, provincial, district and community tiers have been effectively resolved. The most eloquent evidence for the implementation progress is the overachievement or full achievement of the intermediate targets for 12 out of the 14 intermediate outcome indicators (IOI), with two indicators’ intermediate targets partially achieved. Many IOIs have already fully met their end-of-project targets as well. Details on the indicators are presented in the results framework in Annex 1. While stunting rates are not measured at the PDO or IOI levels, it is noteworthy that stunting prevalence among the target children aged 4 has dropped from 53 to 41 percent in the project communities). While attribution is yet to be fully analyzed, this impressive catch-up in growth was achieved within two years (time between baseline and midline impact evaluation measurement). Considering the very good pace of implementation and the accomplished results, the overall implementation progress is strong and significant. Component 1: Increasing Coverage of Early Childhood Education in Target Districts. All planned 73 community child development group (CCDG) facilities have been built and operationalized on time (1,825 new places created). Operating grants, remuneration of CCDG caregivers and teaching aids have been timely provided. The overall child development outcomes of the children enrolled in the CCDGs outperform the multi-age teaching preprimary programs (MAT, see Component 2). All 250 originally planned preprimary classrooms were constructed on time (6,250 places). Following the first project restructuring (approved May 2017), 300 new preprimary classrooms (7,500 places) were added to the component, and at the time of this aide-memoire, all new communities have been trained on the principles of community-based contracting, and construction works have been progressing on schedule. Completion is set for the end of the 2018 calendar year (two years before the new closing date). The playgrounds for the already built CCDG facilities, MAT and preprimary classrooms have been delivered to the communities, while the installation schedule for the remainder is aligned with the construction timeline for the 300 additional preprimary classrooms. The delivery of the CCDG program will continue until the new project closing date to assess its impact over the originally planned three-year implementation period. Based on the very encouraging overall child development results in the CCDGs, MOES is considering the implications for a national scale up after the project closing. Component 2: Improving Quality of Early Childhood Education Services. All subcomponents and activities are implemented with sufficient fidelity. The Multi Age Teaching (MAT) pilot has benefitted from a series of refresher trainings in a bid to strengthen the teachers’ capacity to deliver the preprimary curriculum to children of different abilities and ages (3-5) in one classroom. The midline results show strong gains in child development for children aged 3 and 4, albeit lower compared to the other piloted modality – the CCDGs. It appears that the greater attention given to 3- and 4-year olds has been at the expense of the 5-year old children. 3 The latter outcome suggests that a better alternative to MAT might be the implementation of CCDGs for the 3 and 4-year olds, followed by a preprimary school readiness program for the 5-year olds – an important midline finding to inform MOES decisions for future ECE policies in the country. The National School Meals Program (NSMP) implementation has continued to be satisfactory. The stunting rates reduction detected by the midline impact evaluation survey has occurred predominantly in the communities that implemented NSMP in kindergartens and preprimary classes. The first independent monitoring report by the WFP (recruited by the project to conduct NSMP monitoring and support visits) suggests significant community mobilization, in-kind support and ownership over the NSMP program. The MOES decision of January 29, 2018 to transition NSMP from ECE project to government funding in July 2019 assures the sustainability and budget affordability of the NSMP. With respect to the Community Awareness Campaign (CAC), the twelve months of implementation have been effective in encouraging greater enrollment into ECE programs. CAC impact on behavior change for better health, nutrition and hygiene practices and greater child stimulation at home is detectable, but understandably modest, given the short implementation period and the need to change perceptions and behaviors. Community uptake and attendance of CAC sessions has exceeded twice the midline and endline targets, which holds a promise for greater impact towards the new closing date. The Disability Screening (DS) pilot’s first phases have been completed, synergies have been identified with Humanity and Inclusion (former Handicap International) both in terms of design and geographical coverage, and coordinated actions and exchange of experience have been agreed upon. Component 3: Project management, capacity development and monitoring and evaluation The project’s monitoring and evaluation (M&E) arrangements and implementation have undergone complete reversal during the review period. With the completion of procurement and the deployment of independent survey and monitoring agents to conduct process monitoring for all project activities (WFP for the NSMP, and Indochina RL for all other project activities), reliable and detailed information has been collected and used for informing project implementation course. The arrangements with the independent monitoring firms provide for government capacity building and knowledge and skills transfer, which strengthens overall sustainability. Capacity development programs have been incorporated into the project’s annual operational plan, with focused knowledge and experience exchange visits carried out in March 2018 (“Every Child Ready, Every Child Learning” in Auckland, New Zealand), and planned for August 2018 (executive workshop on education development in Beijing, China), and November 2018 (Study visit to Cambodia on disability screening, school based management, and learning assessments). The English language program for MOES staff has continued successfully, with no dropouts. Regarding project management, the recruitment of a project coordinator (to strengthen the project implementation unit’s capacity) has progressed well, currently at the stage of condu cting interviews with the shortlisted candidates. This was a key action agreed during the last implementation support mission in December 2018. The project management unit managed to complete all agreed actions from the previous mission, with two actions pending MOES final approval and one action in progress with extended timeline. MOES teams also resolved past implementation hurdles that allowed the project to move forward and achieve excellent results. Procurement. Procurement work has progressed based on the Banks’ approval of the updated procurement plan (pending MOES internal approval procedures). The mission encouraged the project to continue the selection processes for all consultants so that procedures can reach the contract signing stage by the time MOES endorses the updated procurement plan. The mission also advised the ECE 4 project to proceed with the selection of the ECE project coordinator using the pool of longlisted candidates from the GPE II Project expressions of interest (both GPE II and ECE projects planned to recruit project coordinators). The procurement packages for the remaining new preprimary classroom construction will be added to the next updated procurement plan, with the ECE project procurement team joining the engineer teams to deliver procurement training to the communities. Financial Management (FM). Albeit with a delay, the ECE project implementation unit has undertaken actions to address the issue of long outstanding unliquidated advances. The diminishing liquidity of the designated account has been resolved by amending the disbursement ceiling from fixed to variable, based on six-month disbursement forecasts. However, future delays in advances’ liquidation (no later than 30 days) may cause the same liquidity issues to reoccur. The MOES leadership has committed to follow closely financial management performance, as it requires smooth coordination between the MOES Department of Finance divisions for accounting and inspection, the project management unit, the various activity implementing units at the MOES and the provincial and district teams. It was agreed that by July 1, 2018 all advances older than 30 days will be liquidated and that continued attention on this aspect will be paid beyond the agreed deadline. Safeguards. The mission conducted social and environment safeguards field visits to 11 communities (four CCDG communities (Component 1), three MAT communities (Component 2), and four communities that built pre-primary classrooms (Component1)). No compliance issues were observed. IV. NEXT STEPS AND AGREED ACTIONS The World Bank mission team and MOES agreed that the next implementation support review of the project will be conducted in the period October 15-26, 2018. The following actions have been agreed upon: Table 2: Summary of Agreed Actions Actions Responsible Due Date MOES 1 MOES to liquidate all outstanding advances. July 1, 2018 (ECU, DOF) MOES to update bidding documents/technical MOES 2 specifications for next round of procurement of toy (ECU, August 31, 2018 library replenishment items to attach greater weight to quality DECE) of the toys. September 28, 3 MOES to appoint ECE project coordinator MOES 2018 MOES to update the National School Meals Program manual to reflect the new funding modality and organize awareness MOES September 28, 4 raising workshops with all concerned provincial, district and (ECU, IEC) 2018 village representatives. MOES to enrich the Community Awareness Campaign MOES 5 modules using the Social and Behavior Change (ECU, October 31, 2018 Communication concept. DECE) 5 Table 3: Status of Agreements from the previous AM Actions Responsible Due Date Status In progress, Clear long outstanding advances and refund December deadline 1 unused advances to the project bank ECU, IU 2017 extended to July account 1, 2018 Deliver the Toy Library boxes to all 270 MOES 2 January 2018 Completed CAC villages. (ECU IUs) Project steering committee meeting to discuss the NSMP measures and the MOES 3 January 2018 Completed considerations for potential extension of the (ECU) project closing date. Conduct training workshops for VEDC (5 MOES 4 January 2018 Completed teams, 9 locations). (CBC/ECU) Deliver CAC refresher training, combined MOES 5 with Toy Libraries and the VEDC grant January 2018 Completed (ECU IUs) guidelines. Disburse NSMP Semester 2 funds to VEDC MOES 6 January 2018 Completed accounts. (ECU. IEC, DOF) Village-level VEDC Grant Guideline MOES 7 January 2018 Completed training. (CAC) MOES, 8 Complete midline IE data collection. January 2018 Completed Indochina RL Update the procurement plan and submit to 9 ECU January 2018 Completed the Bank through STEP for review Launch the DESB monitoring and support MOES 10 February 2018 Completed visits to CCDG communities. (ECU, IUs) Deliver CCDG refresher training to all 146 MOES 11 February 2018 Completed CCDG caregivers. (ECU and IUs) Deliver MAT refresher training to all 62 MOES 12 February 2018 Completed MAT teachers. (ECU, IUs) Launch the DESB monitoring and support MOES 13 February 2018 Completed visits to CAC, MAT, and TL communities. (ECU IUs) ECU, Launch NSMP Independent monitoring 14 World Food February 2018 Completed visits- Round 1. Program Start of construction of pre-primary MOES 15 March 2018 Completed classrooms. (CBC/ECU) Delivery and installation of playground MOES 16 April 2018 Completed equipment to schools. (CBC/ECU) Completed, Develop a concept and a script for CCDG MOES 17 April 2018 pending MOES instructional videos. (TA, ECU, IUs) decision Develop a concept and a script for CAC Completed, MOES 18 instructional videos and video materials in April 2018 pending MOES (TA, ECU, IUs) line with the CAC modules. decision Annexes: Annex 1: Results Framework Annex 2: List of officials met and composition of WB team 6 Annex I. Results Framework (Target values) Actual values Actual values at the time of the Aide Memoire (May 2018, or two months prior to the July 2018 measurement date) Project Development Objective Indicators Baseline Cumulative Target Values (July 2014, Unit of End target Data Source/ Responsible for Indicator Name unless YR1 YR2 YR3 YR4 YR 5 Frequency Measure YR 6 Methodology Data Collection otherwise July 2015 July 2016 July 2017 July 2018 July 2019 stated) December 2020 Percentage of children aged 3-5 (25%) enrolled in ECE programs in CCDG 12% (F: 25%) (50%) IE Data Impact Percentage (2015-16) NA NA NA ECU and MAT target villages F:50% 78.3% (F: 50%) collections Evaluation (disaggregated by gender) F:50% Percentage of children aged 5 enrolled (23%) EMIS and in ECE programs in all target villages 11% (F;50%) IE Data DESB, EMIS and Percentage (2015-16) NA NA NA (48%) Impact except villages under the school lunch F:50% 66.7% Collections ECU Evaluation program (disaggregated by gender) F:50% Percentage of children aged 3-5 in (65%) IE Data Impact target CAC villages who were Percentage (2015-16) NA 55% NA NA (75%) ECU 62.3% Collections Evaluation stimulated at home Percentage of children aged 3-5 in CCDG and MAT target villages with (42%) IE Data Impact Percentage (2015-16) NA 37% NA NA (47%) ECU an overall child development score 42.9% Collections Evaluation above 0.6 Percentage of CCDG and MAT teachers receiving at least once (60%) DPPE M&E DPPE M&E annually performance feedback from Percentage (2015-16) 0 0 0 (100%) (100%) Yearly 82% Network/ECU Network/ECU ECE officers based on classroom observations Intermediate Outcome Indicators End target Unit of Baseline YR1 YR2 YR3 YR4 YR 5 Indicator Name YR 6 Frequency Methodology Data Collection Measure (2013-14) July 2015 July 2016 July 2017 July 2018 July 2019 December 2020 (46,200) Number of beneficiaries (number) of 0 28,819 27,870 33,080 (F: 50%) (51,700) (51,700) DPPE M&E DPPE M&E which female (%) (Bank Core Number Yearly F: 0% F: 50% F: 50% F: 44% 48,213 (F: 50%) (F: 50%) Network/ECU Network/ECU indicator) F:46% Number of VEDC members trained in (810) DPPE M&E DPPE M&E Number 0 0 0 810 (810) (810) Yearly Community Awareness Campaign 810 network/ECU Network/ECU 7 End target Unit of Baseline YR1 YR2 YR3 YR4 YR 5 Indicator Name YR 6 Frequency Methodology Data Collection Measure (2013-14) July 2015 July 2016 July 2017 July 2018 July 2019 December 2020 Number of VHVs trained in (270) DPPE M&E DPPE M&E Number 0 0 0 270 (270) (270) Yearly Community Awareness Campaign 270 network/ECU Network/ECU Number of community members (4590) External having received CAC sessions on all Number 0 0 0 4590 (4590) (4590) Yearly ECU 10,821 monitoring seven CAC modules Number of fully furnished pre- DPPE M&E primary classrooms with water and (350) DPPE M&E Number 0 76 120 250 (550) (550) Yearly network/ECU sanitation facilities constructed in 372 Network/ECU /EMIS target villages Number of CCDG shelters (73) DPPE M&E Number 0 0 0 73 (73) (73) Yearly DPPE constructed 73 Network/ECU Number of ethnic group pre-primary (200) DPPE M&E DPPE M&E Number 0 200 200 200 (200) (200) Yearly teachers receiving pre-service training 200 network/ECU Network/ECU Number of pre-primary MAT teachers (62) DPPE M&E DPPE M&E Number 0 0 0 62 (62) (62) Yearly receiving in-service training 62 network/ECU Network/ECU Number of CCDG caregivers (146) DPPE M&E DPPE M&E receiving annual capacity building Number 0 0 0 146 (146) (146) Yearly 146 network/ECU Network/ECU support Number of children aged 3 to 5 years (200) DPPE M&E DPPE M&E Number 0 0 0 0 (300) (300) Yearly receiving disability screening 211 network/ECU Network/ECU Number of preprimary and primary (25,200) students receiving meals prepared at Number 0 28,619 27,670 26,430 (24,000) (24,000) Yearly NSMP NSMP 25,111 school (270) 270 established Number of CAC villages with External Number 0 0 0 0 254 (270) (270) Yearly ECU established and used Toy Libraries monitoring actively used and promoted Percentage of CCDG/MAT (85%) External classrooms with educative toys and Percentage 0 0 0 0 (100) (100) Yearly ECU 88.7% Monitoring books accessible for children Percentage of target villages (except (85%) External school lunch program) with Percentage 0 0 0 0 (100) (100) Yearly ECU 87.9% monitoring playgrounds 8 Midline measurement for the PDO indicators Analyses are based on 5869 children aged 3-5 years at baseline, and 4779 at midline. Percentage of children aged 3-5 enrolled in ECE programs in CCDG and MAT target villages (disaggregated by gender) Baseline: Of the 2076 children in CCDG and MAT target villages, 353 (12.0%) had attended ECE at baseline. This included 189 boys and 164 girls. Midline: Of the 1731 children in CCDG and MAT target villages, 1355 (78.3%) had attended ECE at midline. This included 685 boys and 670 girls. Percentage of children aged 5 enrolled in ECE programs (CCDG, MAT, KG + CAC, and No ECE + CAC) (disaggregated by gender) Baseline: Of the 1286 5-year olds in IE villages, 11% had attended ECE at midline. This included 244 boys and 232 girls. Midline: Of the 1578 5-year olds in IE villages, 1053 (66.7%) had attended ECE at midline. This included 524 boys and 529 girls. Percentage of children aged 3-5 in target CAC villages who were stimulated at home (activities include reading book, telling story, singing/dancing, drawing, playing, and outdoor activities) Baseline: Of the 4179 children in CAC target villages, 2236 (53.5%) participated in at least one of the six home stimulation activities in the past week. Midline: Of the 3434 children in CAC target villages, 2139 (62.3%) participated in at least one of the six home stimulation activities in the past week. Percentage of children aged 3-5 in CCDG and MAT target villages with an overall child development score above 0.6 Baseline: Of the 2076 in CCDG and MAT target villages, 823 (39.6%) had an overall child development score of 0.6 or above. Midline: Of the 1731 in CCDG and MAT target villages, 742 (42.9%) had an overall child development score of 0.6 or above. Percentage of CCDG and MAT teachers receiving at least once annually performance feedback from ECE officers based on classroom observations: How many times has a DESB Officer come to observe your CCDG/MAT and provide feedback? CCDG teachers MAT teachers 0 26 (18.4%) 17 (27.4%) 1 36 (25.5%) 23 (37.1%) 2 50 (35.5%) 20 (32.3%) 3 21(14.9%) 1 (1.6%) 4 1 (0.7%) 0 (0.0%) 5 3 (2.1%) 0 (0.0%) 6 3 (2.1%) 1 (1.6%) 7 0 (0.0%) 0 (0.0%) 8 1(0.7%) 0 (0.0%) 9 Midline measurement of the intermediate outcome indicators (IOIs) Number of community members having received CAC sessions on all seven CAC modules How many village members have consistently attended the full suite of CAC sessions conducted to date? 1-25 members 54 (20.0%) villages 26-50 members 116 (43.0%) villages 51-75 members 44 (16.3%) villages 76-100 members 31 (11.5%) villages Missing/don’t know 25 (9.3%) villages Based on totalled reports, 10,821 community members have consistently attended CAC sessions conducted to date. Number of CAC villages with established and used Toy Libraries Has your community used the Toy Library? 254 (94.1%) responded yes, 16 (5.9%) responded no Have you promoted the Toy Library in your community? 186 (73.2%) responded yes, 68 (26.8%) responded no Methods of promotion were reported as follows: Disseminated toy box through village meeting and explained use of the toy box to everyone (n=69) Advertised through the village's speaker that the village has received a toy box which can be borrowed free of charge (n=24) Called villagers for meeting and informed them that the village has received a toy box for children which can be borrowed (n=83) Held training for parents and told them that their kids can play (n=1) Through arranging community awareness meeting (n=5) Held village council meeting and assigned someone to be responsible for informing villagers that they can come and play (n=1) Asked parents to have a meeting at school and trained them on taking kids to play with the toys (n=1) Disseminated information to each Unit's head and asked them to tell the villagers in each Unit (n=1) Percentage of CCDG/MAT classrooms with educative toys/books accessible for children Results based on CCDG/MAT teacher report: Have you received any resources for your CCDG? 126 (89.4%) CCDG teachers responded yes, 15 (10.6%) responded no Have you received any resources for your MAT? 54 (87.1%) MAT teachers responded yes, 8 (12.9%) responded no Total CCDG and MAT combined: 180 (88.7%) yes, 23 (11.3%) no 10 Annex I Midline Impact Evaluation Results Lao PDR Early Childhood Education Project Overview 1. Aims 2. Intervention modalities 3. Research Design 4. Instruments 5. Preliminary results at Midline – Attendance – Stunting – Child development 11 1) Overall Aims • The project is expected to improve educational equity, (since ECE has been shown to have larger effects on the most disadvantaged populations). • This is to be achieved through increased access to Early Childhood Education (ECE) programs 2) Intervention Modalities CCDG (Community Child Development Groups) Community based playgroups, including training of locally recruited “teachers”, a purpose built hut and grants provided to communities to sustain implementation MAT (Multi-age Teaching) Local pre-primary teachers are trained in the delivery of curriculum to children of age 3-5 in one class CAC (Community Awareness Campaign) and Toy Libraries A training course provided to the Village Education Development Committees (VEDC), which then undertake community awareness training of parents in their village. Toy library made available for each community. 12 3) Research Design 376 villages 135 villages 119 villages 122 villages Study One Study Two Study Three CCDGs, MAT and CAC Existing KG and CAC No ECE and CAC Existing KG Existing KG No ECE + No ECE No CCDG + CAC MAT + CAC + CAC + No CAC CAC CAC 73 villages 62 villages 61 villages 58 villages 74 villages 48 villages Data collection • Baseline Nov 2015 - March 2016 • Midine Nov-Dec 2017 • 5 provinces, 14 districts, 376 villages • 7,355 households, 7,520 children • Panel survey, however those children who were 7 years or older were replaced with other children 2-6.11 years old at midline 13 Sample details Quota Actual completed Province District HH Caregiver Children Caregiver/ Name Name Vill HH Children Vill Old New Old New Old New 2015 2017 2015 2017 2015 2017 Phongsaly Khoua 25 491 500 25 464 29 374 126 470 30 La 15 298 300 15 269 29 261 39 269 31 Namor 39 773 780 39 666 108 640 140 612 168 Oudomxay Beng 18 360 360 18 303 53 255 105 293 67 Houn 12 238 240 12 234 4 225 15 233 7 Parkbeng 6 120 120 6 102 18 68 52 90 30 Viengxay 36 691 720 36 565 123 447 273 543 177 Houa Meoung 42 829 840 42 664 125 681 153 554 280 Xam Tai 36 692 720 36 565 135 588 134 500 222 Houaphan Sobbao 39 763 780 39 609 143 625 149 518 256 Meoung at 41 798 820 41 623 168 648 172 562 258 Kouan 30 566 600 30 497 76 462 139 454 147 Xaiyabouly Saysathan 20 400 400 20 365 35 340 60 346 54 Borlikhamxay Saychamphone 17 339 340 17 312 25 315 26 314 27 Total 376 7358 7520 376 7309 7512 7512 4) Instruments Impact Evaluation instruments • Village Head • Household • Primary Caregiver • Child (direct assessment and caregiver response) Monitoring instruments • DESB • VEDC • CCDG/MAT teacher • Toy Librarian/ Toy audit and replenishment 14 Child Development Instruments • Direct Assessment – Head shoulders knees and toes – 5 piece jigsaw puzzle – Math concepts, including producing sets – Number knowledge and counting – Spatial awareness – Comprehension – Letter knowledge and – Early reading Requires some resources, takes training and practice to administer, more time consuming than parent/teacher report, but can confirm child’s skills. Child Development Instruments • Caregiver interview (generally parent) – Communication (6 questions) – Approaches to learning and perseverance (10 questions) – Concepts and math (9 questions) – Literacy reading and writing (12 questions) – Cultural knowledge (5 items) – Social and emotional (13 items) – Less time consuming, relies on parent’s honesty and understanding of the questions, and knowledge of the child’s skills. 15 Improvement: increasing ECE attendance % EVER Attended % CURRENTLY attending 100 100 90 90 80 80 70 70 60 60 50 50 CCDG 40 40 MAT 30 30 20 20 10 10 0 0 Baseline Midline Baseline Midline Baseline Midline Baseline Midline Baseline Midline Baseline Midline 3 years 4 years 5 years 3 years 4 years 5 years Improvement: increasing ECE attendance 90 80 70 60 50 40 Baseline Midline 30 20 10 0 KG + CAC KG Control No ECE + CAC No ECE Control % of children ever attended ECE 16 Improvement: lowering stunting rates 60 Stunting Baseline Stunting Midline Stunting Midline 50 Wasting Baseline Wasting Midline Wasting Midline 40 Underweight Baseline Underweight Midline 30 20 10 0 KG + CAC KG Control No ECE + CAC No ECE Control Total sample 3 to <5 years - Stunting by Age month 70.0% 52.8% stunted at baseline (n=4072) 60.0% 44.6% stunted at midline (n=2549) Drop seems to be for the 4 year olds 50.0% 40.0% Stunting 30.0% Baseline Nov 2015-March 2016 20.0% Midine Nov-Dec 2017 10.0% 0.0% 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 17 Stunting by age (yr) by intervention arm 70.0% 60.0% 50.0% 40.0% 30.0% Baseline (n-4072) Midline (n= 2549) 20.0% 10.0% 0.0% 3.00 4.00 3.00 4.00 3.00 4.00 3.00 4.00 3.00 4.00 3.00 4.00 age_yr age_yr age_yr age_yr age_yr age_yr CCDG MAT KG_CAC KG_CONTROL NoECE_CAC NoECE_PureContr Improvement: Numeracy skills 0.7 0.6 0.5 3 years Baseline 3 years Midline 0.4 3 years Midline 4 years Baseline 0.3 4 years Midline 4 years Midline 5 years Baseline 0.2 5 years Midline 0.1 0 CCDG MAT KG + CAC KG No CAC No ECE + No ECE No CAC CAC 18 Improvement: Literacy skills 0.4 0.35 0.3 3 years Baseline 0.25 3 years Midline 3 years Midline 0.2 4 years Baseline 4 years Midline 0.15 4 years Midline 5 years Baseline 5 years Midline 0.1 0.05 0 CCDG MAT KG + CAC KG No CAC No ECE + No ECE No CAC CAC 19 Annex 2. List of Officials Met and Composition of the World Bank Team List of officials met: 1. H.E. Assoc. Prof. Dr Khamphay Sisavanh, Vice Minister of Education and Sport, MOES 2. Dr. Mithong Souvanhvixay, Director General, Department of Early Childhood Education, MOES 3. Ms. Khampaseuth Kitignavong, Deputy Director General, Department of Planning, MOES 4. Mr. Khamfong Sommala, Deputy Head of Division, ECU, MOES 5. Mr. Phisitxay Soulinthone, Deputy Head of Division, ECU, MOES 6. Ms. Xuenchith Selinokham, Department of Finance, Accounting Division, MOES 7. Mr. Chansamouth Keosoutha, Director General, Institute for Education Administrator Development, MOES 8. Mr. Khamphan Khamone, Deputy Director General, Department of Teacher Training, MOES 9. Ms. Viengkeo Phommachack, Deputy Director General, Department of Early Childhood Education, MOES 10. Ms. Soutsaichay Duangsavanh, Deputy Director General, Department of Early Childhood Education, MOES 11. Mr. Khambay Sorsouliphone, Deputy Director General, Department of Finance, MOES 12. Ms. Dara Khiemthamakoun, Acting Director of Inclusive Education Center. NSMP, MOES 13. Ms. Nanna Skau, Head of school meal unit, World Food Program (WFP), Lao PDR 14. Mr. Outhai National officer, School meal unit, WFP, Lao PDR 15. Ms. Jodie Fonseca. LEARN Project Director, Plan International, Lao PDR 16. Ms. Takaho Fukami, UNICEF, Lao PDR 17. Mr. Peter Hansen, Luxembourg Development, Lao PDR 18. Ms. Corine Vis, BEQUAL, DFAT, Lao PDR 19. Ms. Deborah Leaver, Save the Children International, Lao PDR 20. Ms. Holly Cantuba, CRS, Lao PDR 21. Ms. Katherine Dooley, World Vision, Lao PDR 22. Mr. Talat Mahmud, Save the Children International, Lao PDR 23. Ms. Onechanh Phetchantharath, Room to Read, Lao PDR 24. Mr. Sadsada Phongnahak, Aide et Action, Lao PDR 25. Ms. Vithanya Noonan, Aide et Action, Lao PDR 26. Mr. Alexander Loer, Aide et Action, Lao PDR 27. Mr. Phokhaisavanh Phengkhamhac, Aide et Action, Lao PDR 28. Ms. Keoamphone Souvannaphoum, Child Fund, Lao PDR Composition of the WB team: 1. Mr. Plamen Nikolov Danchev (Senior Education Specialist) 2. Dr. Sally Brinkman (Senior Advisor, Early Childhood Development) 3. Ms. Emiko Apichaya Naka (Education Consultant), 4. Ms. Achariya Kohtbantau (Education Consultant) 5. Ms. Siriphone Vanitsaveth (Senior Financial Management Specialist) 6. Ms. Saysi Sayaseng (FM consultant) 7. Mr. Latharo Lor (Procurement Specialist) 8. Ms. Waraporn Hirunwatsiri (Senior Environmental Specialist) 9. Ms. Chansouk Insouvanh (Social Safeguards Consultant) 10. Mr. Phouvanh Phommalangsy (Education Specialist) 11. Ms. Boualamphan Phouthavisouk (Team Assistant) 20