Lao PDR Early Childhood Education Project (Credit 5370-LA and Grant H910-LA) Implementation Support Mission Aide-Memoire (September 11 – 15, 2017) Contents: I. Key Project Data ........................................................................................................................... 2 II. Introduction .................................................................................................................................. 2 III. Project Overview ........................................................................................................................ 2 A. Project Development Objective (PDO) ................................................................................... 2 B. Project Components ................................................................................................................ 2 IV. Summary of Project Status ......................................................................................................... 3 A. Implementation Progress .......................................................................................................... 3 B. Likelihood of Achieving the Project Development Objectives ................................................ 4 V. Detailed Description of the Mission’s Findings and the Agreements Reached ........................... 4 Component 1: Increasing Coverage of Early Childhood Education in Target Districts ............... 4 Component 2: Improving Quality of Early Childhood Education Services.................................. 5 Component 3: Project Management, Capacity Development and Monitoring and Evaluation .... 6 VI. Summary of Agreed Actions ...................................................................................................... 8 List of Annexes (separate documents): Annex 1. CCDG and CAC Field Trip Report Annex 2. Toy Library Instructions Annex 3. VEDC Grant Guideline for CCDG, CAC and TL Annex 4. VEDC Grant Guideline for MAT, CAC and TL Annex 5. VEDC Grant Guideline for CAC and TL Annex 6. External monitoring of ECE: DESB monitoring form Annex 7. External monitoring of ECE: Head of VEDC monitoring form Annex 8. External monitoring of ECE: CCDG Caregiver monitoring form Annex 9. External monitoring of ECE: MAT Teacher monitoring form Annex 10. External monitoring of ECE: Toy Library monitoring form Annex 11. External monitoring of ECE: TORs for EMIS ECD Indicators Task Force -1- I. Key Project Data Project Name Early Childhood Education Approval Date 02 Apr 2014 Closing Date 31 Jul 2019 Project Number P145544 IDA Credit and Grant USD 28 million Percent Disbursed 45.36% Start Mission Date 11 Sept, 2017 Amount IDA Cr.#53700 USD 3.2 million Percent Disbursed 22.86% End Mission Date 15 Sept, 2017 Disbursement IDA Grant#H9100 USD 9.5 million Percent Disbursed 67.86% Next Mission Date 4 Dec, 2017 Disbursement II. Introduction 1. An implementation support mission1 for the Early Childhood Education (ECE) Project (hereinafter referred to as the project) was carried out during the period of September 11–15, 2017. The objectives of the mission were to: (i) review and agree upon the content of the field survey questionnaires to be used for the project’s midline impact evaluation survey; (ii) discuss with relevant departments of the Ministry of Education and Sports (MOES) the potential inclusion of key early childhood development indicators into MOES’ education management information system (EMIS) for routine annual data collection to report the country’s achievements against the sustainable development goals (SDGs) for early childhood development (ECD); (iii) review the project’s annual operational plan; and (iv) agree on actions to accelerate the project’s disbursements. 2. The summary of project status is presented in Section IV, the mission findings are described in Section V, while Section VI contains a summary of the agreed actions. 3. The team wishes to thank the Ministry of Education and Sports (MOES) and Development Partners (DPs) for their guidance and valuable insights. Special appreciation goes to Ms. Khampaseuth Kitignavong, Deputy Director General of MOES Department of Planning (DOP), the staff of the Education Sector Plan Coordinating Unit (ECU) of MOES and all MOES departments who contributed to the achievement of the mission’s objectives. 4. In accordance with the World Bank’s Access to Information Policy, it was agreed with the ECU that this aide-memoire will be publicly disclosed. III. Project Overview A. Project Development Objective (PDO) 5. The PDO of the project is to increase coverage and improve the quality of early childhood education services for 3 to 5-year-olds in target districts. B. Project Components 6. The Early Childhood Education Project support is structured in the following components: (i) Component 1. Increasing Coverage of Early Childhood Education in Target Districts. This component includes construction of pre-primary classrooms in remote areas with low ECE 1 The mission consisted of Mr. Plamen Danchev (Senior Education Specialist and Task Team Leader), Dr. Sally Brinkman (Senior Researcher), Mr. Phouvanh Phommalangsy (Education Specialist), Ms. Achariya Kohtbantau (Education Consultant), and Ms. Boualamphan Phouthavisouk (Team Assistant). -2- enrollment rates and piloting non-formal Community Child Development Playgroups (CCDG) for children of ages 2 to 4 years. (ii) Component 2: Improving Quality of Early Childhood Education Services. This component pilots: (i) the formal delivery of multi-age teaching (MAT) to groups of children of ages varying from 3 to 5 years; (ii) a methodology for child disability screening (DS); (iii) a community awareness campaign (CAC) to promote good childcare and health practices and emphasize the importance of early cognitive stimulation; and (iv) expansion of the provision of school meals for primary education students, piloted under the EFA-FTI Catalytic fund program in Lao PDR (closed 2014). (iii) Component 3: Project management, capacity development and monitoring and evaluation. This component supports overall project management activities, including procurement and financial management, as well as capacity building at central, provincial and district level to ensure regular monitoring and evaluation and provision of implementation support to target communities. A rigorous impact evaluation is embedded into the project design to assess the effectiveness, the individual and combined effects of the CCDGs, MAT and CAC pilots. IV. Summary of Project Status A. Implementation Progress 7. Component 1 is progressing well. The project has already fully achieved its original end targets for newly constructed pre-primary classrooms, and has exceeded the original targets for trained CCDG caregivers and for the number of constructed CCDG shelters (from 48 to 73). The recently processed project restructuring (June 2017) has increased the number of newly constructed preprimary classrooms by 300 to a total of 550. The MOES community- based construction (CBC) team has since made good progress in preparing the grounds for the construction of the additional 300 classrooms, with works expected to commence once the rainy season ends (October/November 2017). The implementation of the CCDG, which has suffered from significant delays, has gained momentum, but not all resources (and program elements dependent on these resources) have yet reached the target communities. 8. While Component 2 progress has improved, the initial implementation delays of approximately one year of the pilot CAC and MAT programs cannot be recovered. The project has made significant investments to develop the capacity at district and village levels to implement these programs, but the evidence from the very recent monitoring visits to project villages (Annex 1) suggests that not all program elements are understood and implemented well at the local level. The project impact evaluation assesses the impact of the abovementioned activities and the CCDG. The varying levels of implementation fidelity combined with the afore-mentioned implementation delays may result in less detectable impacts on child development and result in missing the project intermediate and endline targets of the PDO-level indicators measuring child development progress. It is critically important to conduct refresher trainings for CCDG, CAC and MAT communities as soon as possible to improve their understanding about the programs and ensure all required operating resources for them are provided in full and in a timely manner. 9. Component 3 activities have progressed well. The procurement selections of the external monitoring firms that will conduct the midline and endline impact evaluation surveys and the continuous supervision of the key project activities at village level have been completed. The procurement of the teaching aids for the CCDG and MAT and the Toy Libraries (TL) has been completed as well, but the teaching and learning resources are not yet distributed to -3- project communities. The mission team emphasized the importance of making the teaching aids available at project villages as soon as possible, thus enabling proper implementation of the CCDG, MAT and TL. B. Likelihood of Achieving the Project Development Objectives 10. The PDO consists of well-defined quantitative (expanding the access to ECE programs) and qualitative aspects (raising ECE service delivery quality and achieving behavioral change). With respect to quantitative expansion of access to ECE, the project has already fully achieved the original endline targets, and considering the good pace of preparations for the additional construction of ECE infrastructure, this aspect of the PDO is on track to being accomplished. However, the reduced implementation period of the quality focused activities (CCDG, MAT, CAC and TL) may be detrimental to the project’s ability to produce detectable quality and behavioral impacts. In line with the key findings described in paragraphs 7 and 8, it is likely that the intermediate PDO targets for the developmental outcomes of the target children may be missed due to the shorter implementation period of the CCDG MAT, CAC and TL activities and the inadequate fidelity of their implementation. Until the midline impact evaluation measurement results for the PDO indicators have been collected, it would be premature to make a definitive assessment about the likelihood of PDO accomplishment at this point. It is expected that the data collection will be completed and analyzed by end January 2018. If the midline measurement results confirm the above concerns, an additional implementation period (through a potential extension of the project closing date) would help recover the lost time and improve the likelihood of full accomplishment of the PDO. V. Detailed Description of the Mission’s Findings and the Agreements Reached 10. In line with the mission’s objectives, the findings presented below cover only aspects related to the impact evaluation’s midline data collection and the key project activities included in the impact evaluation design. Component 1: Increasing Coverage of Early Childhood Education in Target Districts (ii) Sub-component 1.2: Community Child Development Group (CCDG) for 3- and 4- year-olds 11. The CCDG implementation was planned to start at the beginning of 2016, but the first training of the CCDG caregivers occurred only in October 2016. This was followed by a period of six months in which the CCDGs were left to operate without any resources (the transfer of their operating grants (CCDG grant) and remuneration to CCDG caregivers were delayed by half a year). The procurement of the CCDG teaching and learning aids has taken an unusually long time and these critically important resources are yet to reach the CCDG shelters. The aids have been stored in a warehouse awaiting a decision for their distribution. It was discussed and agreed with the ECU that the CCDG teaching aid boxes should reach the CCDG villages by no later than mid-October 2017 without trying to match the distribution of the aids with the timelines for any of the planned trainings under the project. Doing so would result in further delays and minimal savings. The money needed for transporting the boxes to the villages are already planned as part of the procurement package and should come from the ECE budget for distribution of the boxes. -4- 12. The recent monitoring visit to CCDG villages (Annex 1) has revealed that not all communities have the required level of understanding on how to conduct and manage the CCDG program. Even though the Village Education Development Committees (VEDC) have received both training and CCDG operating grants (albeit transferred with a 6-month delay), VEDCs are not yet certain about the purpose of the CCDG operating grants and have not used it much, thus leaving the CCDGs without their everyday resources for paper, stationary, workbooks and materials. CCDG caregivers and VEDCs clearly need more frequent training, supervision and implementation support. It was discussed and agreed that refresher trainings for CCDG caregivers will take place in November 2017 and will be repeated in February 2018. Meanwhile, the project will finance the production of instructional videos on the key aspects of CCDG, to be used when the District Education and Sports Bureau (DESB) officers and external monitors visit the CCDG villages, and leaving a copy of the video materials in the CCDG for continuous use by the CCDG caregivers. It was agreed that the international TA of the project will assist MOES in conceptualizing the video and preparing a script by the end of December 2017, while ECU will include the selection of a video production firm in the project procurement plan. Component 2: Improving Quality of Early Childhood Education Services (i) Sub-component 2.1: Supporting Services Package (a) Community Awareness Campaign (CAC) 13. The CAC is an important element of the project design, aiming to improve the project community members’ understanding of the importance of early childhood education, early cognitive stimulation, nutrition, health and hygiene. While the monitoring visits conducted on September 7 and 8, 2017 confirmed that all seven modules of the CAC have been delivered to the communities, there seem to be knowledge gaps at the local level that negatively affect the quality and coverage of implementation. There is an urgent need to address the shortcomings in CAC implementation. To this end, the mission team and MOES agreed that a refresher training for CAC facilitators from all 270 villages will be delivered in the period October – December 2017. This training will be complemented by dedicated sessions on the VEDC Grant guidelines and the TL instruction manual. The training budget should be urgently prepared by the concerned implementing units (IUs) and ECU, and submitted for the Bank’s review and approval by the end of September 2017, so that actual delivery could take place as planned in mid-October 2017. 14. Like the CCDGs, the CAC communities will greatly benefit from both instructional video content and short videos related to the CAC modules. It was discussed and agreed with MOES that a concept and script for these video materials will be prepared with the assistance of the international TAs of the project, by no later than end February 2018. Meanwhile the project will revise the procurement plan to include a selection of a video production firm. (b) Toy libraries (TL) 15. TLs have been introduced into the project design through the June 2017 restructuring, responding to the extremely low level of interaction between parents and their children in the project communities. The TLs contain a set of age appropriate toys and books to enable parents of pre-school age children (0-5 years old) to stimulate and promote their children’s learning. -5- 16. The mission team reviewed the draft instructions for use of the TL and proposed revisions that simplify the language, reduce the length of the document, and soften the overall tone to stimulate the use of TLs by the community, rather than discouraging parents to use them. It was agreed that the revised instructions (Annex 2) will be reviewed by the concerned IUs and resubmitted to the Bank for no objection by the end of September 2017. It is important to keep that deadline as it will make it possible to use the instruction document as a training material in the CAC refresher training scheduled for mid-October 2017. 17. Regarding procurement of the TL items, the mission discussed and agreed with MOES that TLs should be distributed to the villages as soon as possible, independent of any upcoming training timelines. The money needed for transporting the toy and book boxes to the villages should come from the ECE budget for distribution of the boxes. (ii). Sub-component 2.2: Teacher and Education Officer Training (a) Multi-age teaching (MAT) 18. Like the CCDGs, the MAT teaching aid boxes have been procured but not yet delivered to the target villages. The mission agreed with MOES that the teaching aid boxes will be distributed to the MAT villages by no later than mid-October 2017. To reinforce implementation fidelity, refresher trainings for MAT teachers will be carried out in January – February 2018 during the school semester break. VEDC Grant Guidelines 19. The project restructuring of June 2017 has introduced a new disbursement category - the VEDC Grants, with a formal endorsement of VEDC Grants manuals set as a condition for using the grants. During the mission, the Bank team reviewed and proposed revisions to the VEDC Guidelines prepared by MOES for the different treatment arms of the project impact evaluation, namely: (i) villages implementing CCDG, CAC and TL; (ii) villages implementing MAT, CAC and TL; and (iii) villages implementing CAC and TL only. The revisions were discussed and agreed upon with MOES and the final drafts are presented in Annexes 3-5. It was agreed that ECU will submit them formally for the Bank’s no objection by end September, 2017. The ECU will provide instructions on these Guidelines to the concerned MOES departments before planning the VEDG Grant Guideline training to the participating villages. The village-level training will be combined with the CAC refresher training, which is scheduled for mid-October 2017. Component 3: Project Management, Capacity Development and Monitoring and Evaluation (ii) Sub-component 3.2: Monitoring and Evaluation (M&E) Midline Impact Evaluation Data Collection 20. The mission team, jointly with the impact evaluation survey firm (Indochina RL), reviewed the impact evaluation design and the instrumentation for the midline impact evaluation data collection. The overall impact evaluation design was revised considering the significant delays in implementation of the program components. 21. Baseline data were collected from November 2015 to March 2016 on 7,520 children that were aged 18 – 81 months. This was the widest age range with very few children at the top and bottom end (e.g. <1% of children were between 18-23 months at baseline, and <1% were between 72-81 months). Midline data collection is planned to start in November 2017 and will likely take 2-3 months. If the outliers are not included, then most of the sample will -6- include children aged 48 – 95 months (4 to almost 8 years) in November 2017. Due to the delays and implementation fidelity issues described in the previous sections, it is expected that the full-fledged program implementation with all required components and better understanding of the local implementers would likely commence about the same time in November 2017. Considering the timing of the implementation, the following changes to the impact evaluation research design/sample have been discussed and agreed upon: - The midline sample will exclude all children aged 7+ years. None of these children could have been exposed to the program and it will be of no value to the study, thus representing an unnecessary expense. - Noting that this is a clustered randomized control trial (RCT) where communities are randomized and children within the communities are selected at random, the midline survey will pick up new children aged 2 through to 3 years 11 months. These children thus become replacements of the children 7+ dropped from the sample, and are likely to benefit from the full impact of the program. - At endline the sample will include additional new children aged 2 – 5. This is a back-up plan if further implementation delays occur and/or if an extension to the program is granted. 22. In terms of analyses, the impact evaluation will still include traditional longitudinal/panel analyses, however, this will be with a reduced sample including only the children aged 2 to <6 at baseline. This analysis will be focused on the difference in developmental growth patterns in the different trial arms. For the children picked up at midline it will be possible to determine the difference in growth between midline and endline between the different trial arms. Even without assessing the growth, it would still be possible to compare at endline the difference in average development between the impact evaluation trial arms. Replacements at midline will also enable the project to properly report on participation rates in the different modalities, which are key measures in the project’s results framework. 23. The baseline instrumentation for the midline survey has been adjusted to reflect the new age ranges of the surveyed children. 24. As part of Indochina RL’s contract, they will monitor the key aspects of the program modalities once every six months. This is in addition to the impact evaluation midline and endline surveys. Draft monitoring forms have been prepared (Annexes 6-10) and sent for translation. These will then be programed into tablets for data collection in the field. The first monitoring will coincide with the midline data collection for greater efficiency. 25. The meetings the WB team held with Indochina RL have verbally confirmed their agreement to these revisions. It was discussed and agreed that the ECU will formally communicate these changes to Indochina RL. Inclusion of key early childhood development indicators into EMIS 26. The mission team met with representatives of the MOES Department of Planning, Department of Inspection, Department of Preprimary Education and the EMIS division to discuss the potential inclusion of key early childhood education indicators into the EMIS as a step to ensure that the Government of Lao PDR will routinely collect key data to report its achievements against the SDGs for early childhood education. 27. The project has recently revised the terms of reference for the international technical advisor supporting the improvement of the monitoring and evaluation systems of the MOES by including a set of ECD indicators, some of which have already been widely used by the project as part of the baseline survey, covering 7,520 children. -7- 28. Since all proposed indicators are new to the EMIS system and there are no methodological documents established for their routine collection, it was discussed and agreed that by October 2017 MOES will formally set up a small task force comprising a Ministry of Health representative (School Health Task Force member), and representatives of the MOES Department of Planning, Department of Inspection, Department of Preprimary Education and the EMIS division. The work of this task force will be facilitated by the international technical advisor and closely coordinated with the ongoing EMIS initiative supported by the Australia Department of Foreign Affairs and Trade (DFAT)’s BEQUAL program. The objective of the task force will be to develop the methodological guidance for data collection at village and school levels, and advance this undertaking to the stage of formally including the indicators into EMIS by the closing date of the ECE project. Detailed information on the indicators to be included in the EMIS are presented in Annex 11. VI. Summary of Agreed Actions Deadline # Action Responsible Entity Component 1. Increasing Coverage of Early Childhood Education in Target Districts 1st round: Deliver CCDG refresher training to all 146 MOES end-November 2017 1. CCDG caregivers. (ECU and IUs) 2nd round: February 2018 Deliver CCDG teaching aid boxes to all 73 MOES 2. mid-October 2017 CCDG villages (ECU and IUs) Develop a concept and a script for CCDG MOES 3. end-December 2017 instructional videos (TA, ECU and IUs) Transfer VEDC grants component for DESB MOES 4. mid-November 2017 visits to the 73 CCDG project communities (ECU, DOF, and IUs) Launch the DESB monitoring and support visits MOES 5. mid-December 2017 to CCDG communities (ECU and IUs) Component 2. Improving Quality of Early Childhood Education Services Deliver MAT refresher training to all 62 MAT MOES 6. January-February 2018 teachers (ECU and IUs) Submit draft budget for CAC refresher training MOES 7. end September 2017 for Bank’s review and NOL (ECU and IUs) Submit VEDC Grants Guidelines for Bank’s MOES 8. end September 2017 review and NOL (ECU and IUs) Submit Toy Libraries Instructions for Bank’s MOES 9. end September 2017 review and NOL (ECU and IUs) Deliver CAC refresher training, combined with MOES 10. training on Toy Libraries and the VEDC grant mid-October 2017 (ECU and IUs) guidelines. Deliver MAT teaching aids boxes to all 62 MOES 11. mid-October 2017 MAT villages (ECU and IUs) Deliver the Toy Library boxes to all 135 CAC MOES 12. mid-November 2017 villages (ECU and IUs) Transfer VEDC grants component for DESB MOES 13. visits to the CAC, MAT and TL project mid-November 2017 (ECU, DOF and IUs) communities -8- Launch the DESB monitoring and support visits MOES 14. mid-December 2017 to CAC, MAT, and TL communities (ECU and IUs) Develop a concept and a script for CAC MOES 15. instructional videos and video materials in line end-February 2018 (TA, ECU and IUs) with the CAC modules Component 3. Project management, capacity development and monitoring and evaluation Formally set up an EMIS ECD indicators task 16. force MOES October 2017 Communicate formally the revised 17. instrumentation for midline data collection to ECU end September 207 Indochina RL 18. Start the midline data collection survey Indochina RL mid-November 2017 -9-