The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) REPORT NO.: RES35635 RESTRUCTURING PAPER ON A PROPOSED PROJECT RESTRUCTURING OF STRENGTHENING EVIDENCE-BASED POLICYMAKING WITH EDUCATION STATISTICS AND ANALYSIS APPROVED ON DECEMBER 16, 2016 TO UKRAINE EDUCATION EUROPE AND CENTRAL ASIA Regional Vice President: Cyril E Muller Country Director: Satu Kristiina J. Kahkonen Senior Global Practice Director: Jaime Saavedra Chanduvi Practice Manager/Manager: Harry Anthony Patrinos Task Team Leader: Lucia Casap The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) ABBREVIATIONS AND ACRONYMS CEQA Center for Educational Quality Assessment EMIS Education Management Information System IEA Institute of Educational Analytics IT Information Technologies MoES Ministry of Education and Science of Ukraine PDO Project Development Objectives PISA Programme for International Student Assessment TF Trust Fund ToR Terms of References The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) BASIC DATA Product Information Project ID Financing Instrument P161312 Investment Project Financing Original EA Category Current EA Category Not Required (C) Not Required (C) Approval Date Current Closing Date 16-Dec-2016 24-Jan-2019 Organizations Borrower Responsible Agency Ministry of Education and Science,Institute of Ukraine Educational Analytics Financing (in USD Million) FIN_SUMM_PUB_TBL SUMMARY Total Project Cost 0.40 Total Financing 0.00 Financing Gap 0.00 DETAILS -NewFin3 Project Development Objective (PDO) Original PDO The development objective of this project is to strengthen the capacity of Ukrainian education authorities to utilize administrative, statistical, and learning assessment data and analytical methods to inform policymaking. OPS_TABLE_PDO_CURRENTPDO The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) Summary Status of Financing Net TF Approval Signing Effectiveness Closing Commitment Disbursed Undisbursed TF-A4096 21-Dec-2016 24-Jan-2017 27-Jan-2017 24-Jan-2019 .34 .17 .17 Policy Waiver(s) Does this restructuring trigger the need for any policy waiver(s)? No I. PROJECT STATUS AND RATIONALE FOR RESTRUCTURING A. Background 1. The Strengthening Evidence-Based Policymaking with Education Statistics and Analysis Project Development Objective (PDO) is to strengthen the capacity of Ukrainian education authorities to utilize administrative, statistical, and learning assessment data and analytical methods to inform policymaking. The Project is a small recipient executed grant amount of $US 0.34 million financed out of the Multidonor Trust Fund for Statistical Capacity Development III (TF-A4096-001), to be implemented over 24 months. The Project was approved on December 21, 2016 and became effective on January 27, 2017. The current closing date for the project is January 24, 2019. 2. The Project was designed to support the government’s effort to (i) development of an integrated Education Management Information System (EMIS) (Component 1); (ii) prepare for participation in the Programme for International Student Assessment (PISA) (Component 2); and (iii) institutional capacity building (Component 3). The PDO indicators include: (i) Upgraded Education Management Information System (EMIS) is fully operational; (ii) Open data portal is functional and allows stakeholders to access relevant EMIS data; and (iii) Qualified specialists are trained to prepare the national analytical report using PISA 2018 data. 3. The project recipient and implementing agency is the relatively young State Scientific Institution "Institute of Educational Analytics" (IEA) subordinated to the Ministry of Education and Science of Ukraine (MoES) responsible according to its charter for development and implementation of a system of education statistics and analysis in Ukraine through research, analytical, and innovative methods. The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) B. Project Implementation Progress to Date 1. The overall grant implementation has been slow partially due to the fact that IEA is relatively new entity without previous experience implementing Bank-financed activities, and internal management changes in 2017. Currently, planned activities to a large extend were realized or are under the implementation. Activities under Components 2 and 3 related to PISA and institutional capacity building are on track, with majority of tasks realized. Under the Component 1, related to EMIS upgrade, many tasks are conducted or initiated. Still, some activities under this component, as system certification, development of the open data portal, and users training can be done only at the final stage or after completion of the system upgrade. In terms of procurement packages, out of 24, 15 are finished, two related to EMIS upgrade are under implementation, and 7 packages to be contracted at the final stage or after the completion of the system upgrade. 2. For the Component 1: Upgraded EMIS is fully operational, all the preparatory work was completed, and system upgrade has been started. Currently, the study visits were conducted; consultants developed the Terms of References (ToR) for the EMIS upgrade and firm to perform this work was hired. However, due to lengthy process of the ToR development, the selection work was delayed and the main task of the Project, EMIS upgrade, started only in September 2018, with 4 months delay. After the system upgrade there are three more activities to be done meant to ensure system sustainability: (i) Database security certification; (ii) Development of an open data portal; and (iii) Train users on working in EMIS after its modernization. As the EMIS upgrade is expected to be finish by the end of January 2019, the current closing date, six months extension of the Project will allow to complete this works. 3. Under the Component 2: Preparation for participation in the Programme for International Student Assessment (PISA) the activities were completed. In order to avoid overlaps with other parallel PISA activities in Ukraine, and due to the unavailability of PISA-2018 data within the Project implementation timeframes, the activities were revised. Thus, instead of two consultants to support PISA activities, the IEA decided to select one international consultant who conducted in December 20118 a 5 days training on PISA data analysis and report preparation for national experts who will work in Ukraine after data release. As Ukraine didn’t participate in other PISA waves and PISA-2018 data are not yet available, for the training were used PISA data of other countries from previous waves. 4. The activities under the Component 3: Institutional capacity building were performed in accordance with the schedule. In April 2017 a Project Management Team (Procurement Specialist, Financial Management Specialist and Translator) was formed. The English language courses have been started and during August-September 2017 the planned goods (phone, furniture, computers) were procured. In December 2017 the IEA website was developed. 5. As of now, the amount of committed and disbursed funds within the Project is USD 167,493.17 thousand or 49.26% of the total amount. The following disbursement is on each Category: Cost category Planned (USD) Current (USD) (a) Goods and consultancy services 273,500 113,200 (b) Training 58,500 21,000 (c) Operational expenditures 8,000 5,100 Total 340,000 149,300 The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) 6. Procurement. The project procurement arrangements continue to be satisfactory. 7. Financial Management. The project financial management arrangements continue to be satisfactory. Per the Bank decision from December 6, 2017 the audit was waved. Thus, a single audit for the period May 14, 2017 (start of disbursements) to July 24, 2019 (requested deadline) is to be conducted. II. DESCRIPTION OF PROPOSED CHANGES 1. The proposed restructuring seeks authorization to (i) extend the project’s closing date by 6 months, (ii) revise the PDO indicator related to training of specialists on using PISA 2018 data to prepare the national analytical report, and (iii) change the distribution of financial resources among cost categories. The modifications will enable completion of the whole scope of works planned within the Project in a timely and comprehensive manner. 2. The proposed extension of closing dates by 6 months from January 24, 2019 to July 24, 2019 would allow sustainable and reliable finalization of the works related to EMIS upgrade. These include several activities related to: (i) Database security certification; (ii) training of upgraded EMIS users meant to ensure system security and sustainability could be performed only after system development; and creation of the Open Data Portal, which would be done at the final stage of the EMIS system upgrade. The additional six months would allow accomplishment of all the tasks of the Project, including those meant to ensure EMIS security and sustainability under Component 1. 3. It is proposed to rename the current PDO indicator related to PISA, namely to change the formulation of the indicator “Qualified specialists are trained to prepare the national analytical report using PISA 2018 data”, dropping the reference to the year. The new indicator would be “Qualified specialists are trained to prepare the national analytical report using PISA data”. As Ukraine didn’t participate in previous PISA waves and due to unavailability of PISA-2018 data for Ukraine within the Project timeline, the training of experts was done using PISA data of other countries from previous waves. Approving the restructure will allow to consider the indicator achieved. 4. Considering the slight changes in Project activities and significant difference between planned and actual cost for Upgraded EMIS (US$115,000 instead of US$46,000 planned, contract No. TFSCB-CQ-01) it is proposed to change distribution of financial resources within the cost categories. Namely, to compensate this contract difference reallocate the amount of unused US$ 23,800 from category “Training” to “Goods and Consulting Services”. This would allow successful completion of all planned activities, achievement of the planned results. Below is reflected the current and proposed distribution of resources per categories: Amount of the Requested Cost funding Cost category Grant Allocated Amount percentage (expressed in USD) (USD) (a) Goods and consulting services 273,500 297,300 100% (b) Training 58,500 34,700 100% (c) Operating Cost 8,000 8,000 100% TOTAL 340,000 340,000 The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) III. SUMMARY OF CHANGES Changed Not Changed Results Framework ✔ Components and Cost ✔ Loan Closing Date(s) ✔ Reallocation between Disbursement Categories ✔ Implementation Schedule ✔ Implementing Agency ✔ Project's Development Objectives ✔ Cancellations Proposed ✔ Additional Financing Proposed ✔ Disbursements Arrangements ✔ Disbursement Estimates ✔ Change in Overall Risk Rating ✔ Safeguard Policies Triggered ✔ EA category ✔ Legal Covenants ✔ Institutional Arrangements ✔ Financial Management ✔ Procurement ✔ Other Change(s) ✔ IV. DETAILED CHANGE(S) OPS_DETAILEDCHANGES_RESULTS_TABLE RESULTS FRAMEWORK Project Development Objective Indicators PDO_IND_TABLE Upgraded Education Management Information System (EMIS) is fully operational Unit of Measure: Text Indicator Type: Custom The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) Baseline Actual (Current) End Target Action Existing EMIS is Existing EMIS is EMIS is upgraded to operational with operational with cover all GSE schools incomplete coverage of incomplete coverage of Value and preschool No Change general secondary general secondary institutions in at least education (GSE) education (GSE) 3 regions (oblasts) institutions institutions Date 01-Nov-2016 15-Jun-2018 24-Jan-2019 Open data portal is functional and allows stakeholders to access relevant EMIS data Unit of Measure: Text Indicator Type: Custom Baseline Actual (Current) End Target Action Open data portal is No open data portal No open data portal functional and allows Value No Change exists exists stakeholders to access relevant EMIS data Date 01-Nov-2016 15-Jun-2018 24-Jan-2019 Qualified specialists are trained to prepare the national analytical report using PISA data Unit of Measure: Number Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 0.00 20.00 Revised Date 01-Nov-2016 15-Jun-2018 24-Jan-2019 Intermediate Indicators IO_IND_TABLE Percentage of general secondary education schools reporting in EMIS the latest year's data on students enrolled Unit of Measure: Percentage Indicator Type: Custom Baseline Actual (Current) End Target Action Value 95.11 99.99 100.00 No Change Date 01-Nov-2016 15-Jun-2018 24-Jan-2019 Percentage of general secondary education schools reporting in EMIS the latest year's data on teachers employed Unit of Measure: Percentage Indicator Type: Custom The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) Baseline Actual (Current) End Target Action Value 82.31 99.86 100.00 No Change Date 01-Nov-2016 15-Jun-2018 24-Jan-2019 Percentage of preschool institutions reporting in EMIS the latest year's data on children enrolled Unit of Measure: Percentage Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 0.00 20.00 No Change Date 01-Nov-2016 15-Jun-2018 24-Jan-2019 Number of stakeholders trained in the use of the EMIS or the EMIS open data portal Unit of Measure: Number Indicator Type: Custom Baseline Actual (Current) End Target Action Value 0.00 0.00 200.00 No Change Date 01-Nov-2016 15-Jun-2018 24-Jan-2019 Sampling of schools for the implementation of PISA 2018 is conducted using EMIS data Unit of Measure: Text Indicator Type: Custom Baseline Actual (Current) End Target Action Sampling of schools Sampling of schools for for the No sample of schools the implementation of implementation of Value exists for PISA 2018 No Change PISA 2018 is conducted PISA 2018 is implementation using EMIS data conducted using EMIS data Date 01-Nov-2016 15-Jun-2018 24-Jan-2019 PISA 2018 assessments are carried out in Ukraine Unit of Measure: Text Indicator Type: Custom Baseline Actual (Current) End Target Action No PISA assessments PISA 2018 assessments PISA 2018 Value have been carried out are carried out in assessments are No Change in Ukraine to date Ukraine carried out in Ukraine Date 01-Nov-2016 15-Jun-2018 24-Jan-2019 The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) OPS_DETAILEDCHANGES_COMPONENTS_TABLE COMPONENTS Current Current Proposed Proposed Cost Action Component Name Component Name Cost (US$M) (US$M) Development of an integrated Development of an integrated Education Management 0.17 Revised Education Management 0.19 Information System (EMIS) Information System (EMIS) Preparation for participation in Preparation for participation in the Programme for the Programme for International 0.07 Revised 0.03 International Student Student Assessment (PISA) Assessment (PISA) Institutional capacity building 0.10 Revised Institutional capacity building 0.12 TOTAL 0.34 0.34 OPS_DETAILEDCHANGES_LOANCLOSING_TABLE LOAN CLOSING DATE(S) Original Revised Proposed Proposed Deadline TF Status Closing Closing(s) Closing for Withdrawal Applications TF-A4096 Effective 24-Jan-2019 24-Jul-2019 24-Nov-2019 OPS_DETAILEDCHANGES_REALLOCATION _TABLE REALLOCATION BETWEEN DISBURSEMENT CATEGORIES Financing % Current Allocation Actuals + Committed Proposed Allocation (Type Total) Current Proposed TF-A4096-001 | Currency: USD iLap Category Sequence No: 1 Current Expenditure Category: G,CS 273,500.00 114,717.72 297,300.00 100.00 100.00 iLap Category Sequence No: 2 Current Expenditure Category: DISB - TRAINING 58,500.00 21,015.58 34,700.00 100.00 100.00 The World Bank Strengthening Evidence-Based Policymaking with Education Statistics and Analysis (P161312) iLap Category Sequence No: 3 Current Expenditure Category: DISB - OPERATING COSTS 8,000.00 5,092.28 8,000.00 100.00 100.00 Total 340,000.00 140,825.58 340,000.00 Note to Task Teams: End of system generated content, document is editable from here.