Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004916 IMPLEMENTATION COMPLETION AND RESULTS REPORT TF 15859 ON A SMALL GRANT IN THE AMOUNT OF US$ (0.7) MILLION TO EXPERT GRUP, MOLDOVA FOR EMPOWERED CITIZENS ENHANCING ACCOUNTABILITY OF EDUCATION REFORM AND QUALITY (P147607) June 10, 2019 Education Global Practice Europe And Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective June 3, 2019) Currency Unit = Moldovan Leu (MDL) MDL =18.04 US$1 FISCAL YEAR July 1 - June 30 ABBREVIATIONS AND ACRONYMS CSO Civil Society Organization CPF Country Partnership Framework CPS Country Partnership Strategy EG Expert Grup EMIS Education Management Information System GDP Gross Domestic Product GPSA Global Partnership for Social Accountability ISR Implementation Status and Results Report ICR Implementation Completion and Results Report ICT Information and Communication Technology IEBA Independent Education Budget Analysis IRI Intermediate Result Indicator LPA Local Public Authority MERP Moldova Education Reform Project ML Management Letter MoU Memorandum of Understanding MoECR Ministry of Education, Culture, and Research NGO Nongovernmental Organization ODRA Open Data Readiness Assessment OECD Organisation for Economic Co-operation and Development PCF Per Capita Financing PDO Project Development Objective PISA Programme for International Student Assessment SAcc Social Accountability TF Trust Fund TTL Task Team Leader Regional Vice President: Cyril E Muller Country Director: Satu Kristiina J. Kahkonen Senior Global Practice Director: Jaime Saavedra Chanduvi Practice Manager: Harry Anthony Patrinos Task Team Leader(s): Lucia Casap ICR Main Contributor: Alina Sava TABLE OF CONTENTS DATA SHEET ....................................................................... ERROR! BOOKMARK NOT DEFINED. I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 4 II. OUTCOME ...................................................................................................................... 8 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 13 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 13 V. LESSONS LEARNED AND RECOMMENDATIONS .............................................................. 15 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 18 ANNEX 2. PROJECT COST BY COMPONENT ........................................................................... 26 ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS ...... 28 ANNEX 4. SUPPORTING DOCUMENTS (IF ANY) ..................................................................... 29 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name Empowered citizens enhancing accountability of Edu P147607 reform and quality Country Financing Instrument Moldova Investment Project Financing Original EA Category Revised EA Category Not Required (C) Organizations Borrower Implementing Agency Adrian Lupușor Expert Grup Project Development Objective (PDO) Original PDO The objective of the Project is to empower the Member Country’s civil society to: (i) engage local, regional and national authorities in evidence-based policy and budget dialogue in the Member Country’s education sector; and (ii) to promote an enabling environment for social accountability. Page 1 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) FINANCING FINANCE_TBL Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) Donor Financing TF-15859 696,955 696,955 686,153 Total 696,955 696,955 686,153 Total Project Cost 696,955 696,955 686,153 KEY DATES Approval Effectiveness Original Closing Actual Closing 16-Dec-2013 20-Dec-2013 20-Dec-2018 20-Dec-2018 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions KEY RATINGS Outcome Bank Performance M&E Quality Satisfactory Satisfactory Substantial RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 16-Mar-2014 Satisfactory Satisfactory 0.21 02 04-Nov-2014 Satisfactory Satisfactory 0.21 03 18-Sep-2015 Satisfactory Satisfactory 0.34 04 10-Apr-2016 Satisfactory Satisfactory 0.34 05 19-Jun-2017 Satisfactory Satisfactory 0.49 06 27-Aug-2018 Satisfactory Satisfactory 0.63 Page 2 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) 07 17-May-2019 Satisfactory Satisfactory 0.69 ADM STAFF Role At Approval At ICR Regional Vice President: Laura Tuck Cyril E Muller Country Director: Qimiao Fan Satu Kristiina J. Kahkonen Senior Global Practice Director: Ana L. Revenga Jaime Saavedra Chanduvi Practice Manager: Alberto Rodriguez Harry Anthony Patrinos Task Team Leader(s): Josef S. Trommer, Anna Olefir Lucia Casap ICR Contributing Author: Alina Sava Page 3 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES Context 1. Country context. Moldova is a small lower-middle-income economy with a population of 3.5 million with a gross domestic product (GDP) per capita of US$2,724, remaining one of the poorest countries in Europe. Despite this, Moldova has made significant progress in reducing poverty and promoting inclusive growth since the early 2000s. The economy has expanded by an average of 5 percent annually, driven by consumption and fueled by remittances. However, the current economic development model is an unsustainable one, and it is challenged even more by skills mismatch in Moldova’s labor market because of large-scale out-migration combined with decreasing fertility rates. This demographic evolution led to an alarming decline in the population and increased the share of elderly people who are limiting the country’s long-term competitiveness. Also, it was causing polarization in the society fueled even more by reduced transparency and accountability. European integration has anchored successive governments’ policy reform agendas, but reforms that are good on paper are yet to materialize. Continued economic stabilization, advancement of key socioeconomic structural reforms, and establishment of a rule-based environment for businesses are the country’s key goals to address development constraints. 2. Sector context. The Government of Moldova’s primary objective in education is to improve the quality, efficiency, and relevance of the education system to meet the needs of the labor market and the broader economy. However, the country’s demographic and fiscal realities have not made it easy for the Government to fulfill this mandate. The Moldovan education sector still encounters serious challenges. Over the years, the education sector in Moldova has witnessed uneven quality and a lack of overall efficiency. 3. The country spent 6.4 percent of GDP in 20171 (compared with 9.1 percent in 2010) for education financing—this is still a high share by any regional comparison. At the same time, because of the dramatic decline in the number of students, Moldova’s existing school network (operating at half of its capacity) has emerged as a major source of inefficiency in the country’s education sector. A small number of students lead to excess schools, unused school space and facilities, and high running costs creating the premises to reform the school network. 4. In response to the demographic trends and declining school-age population, combined with unsustainably high levels of education expenditures, the Government of Moldova has been implementing a school network optimization reform with the aim of improving spending efficiency. In 2010 the per capita financing (PCF) mechanism was piloted, targeting final adjustments and testing before expanding at the national level in January 2013. Officially, the PCF was approved in 2014 and came into force in 2015, leading to an immediate reorganization of the school network. This has involved closing and merging under-enrolled schools, consolidating classes, reorganizing several lyceums into gymnasiums2 and gymnasiums into primary schools, and creating hub schools. Between 2010 and 2019, the school network comprising primary and secondary schools decreased by 16 percent from 1,489 schools to 1,246.3 1 Education in the Republic of Moldova Statistical Publication, 2017/18, Chișinău 2018. 2 Gymnasium - schools with lower secondary level. 3 Data compiled from the National Bureau of Statistics of Republic of Moldova, 2019. Page 4 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) The same percentage of decrease was recorded for students whose numbers shrunk from 396,500 in school year 2010/11 to 334,200 in school year 2018/19. 5. Moldova’s performance in the Organisation for Economic Co-operation and Development (OECD) Programme for International Student Assessment (PISA) 2009 shows that the performance of the country’s 15-year-olds in reading, math, and science is among lowest in the region. By age 15, Moldovan students lag behind their OECD peers by around two years of schooling, and around 60 percent of Moldova's 15-year-olds lack even the basic levels of proficiency in reading and math literacy. The PISA 2015 study showed that Moldova’s education system performs well compared with countries in the same income group, but education outcomes are substantially below those of (richer) neighboring and competitor countries. For example, the 2015 PISA scores in reading, math and of Moldovan 15-year-olds were above those of some countries with higher income levels, such as Georgia, Kosovo, and the former Yugoslav Republic of Macedonia. However, Moldova’s performance was lower than the performance of even the European Union’s poorest countries, Romania and Bulgaria.4 6. Rationale for World Bank support. The National Education 2020 Strategy was the key guiding document for the World Bank’s engagement. The National Education 2020 Strategy stressed the need to increase schools’ accountability for learning outcomes and participation of communities in the decision- making process that aimed for a more transparent budget process and better results in education. The Country Partnership Strategy (CPS, Report No. 79701-MD) for FY14–17 was effective at appraisal, and one of the main goals was to promote more efficient and transparent public education service as a precondition for enhancing human capital and minimizing social risks. The project’s objectives were consistent with the CPS, directly contributing to the implementation of the CPS’s second pillar of building human capital. The project was also aligned with the World Bank Education Strategy 2020 ‘Learning for All’. 7. The new Country Partnership Framework (CPF, Report No. 115716-MD) for FY18–21 continues to support improvements in the governance sector, quality of public services, and human capital development. The current program of the World Bank in Moldova provides support in various sectors including relevant lines for this intervention such as a transparent business regulatory framework, advice and financing for education, efficient and accessible public services, improved state budget allocation process, improved financial reporting in the public sector, and rehabilitated and inclusive schools. 8. The strategic goal of the Empowered Citizens Enhancing Accountability of Education Reform and Quality Project was to empower Moldovan citizens to engage local, regional, and national authorities in evidence-based policy and budget dialogue regarding educational reform; quality of services; and development priorities of primary, secondary general, and upper secondary schools and enable an environment in which social accountability (SAcc) initiatives thrive and develop. Accountability is the key lever of the project, important for all the school stakeholders, represented by parents, students, and communities who gain the ability to hold schools and teachers responsible for the quality of education services and the use of resources. This is particularly important in the context of the ongoing education reform and wider autonomy of schools in managing resources granted under the new per-student financing mechanism. 4 Moldova: Education Sector Public Expenditure Review Selected Issues, February 2018. Page 5 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) 9. This project is a Recipient-Executed Trust Fund Grant financed by the Global Partnership for Social Accountability (GPSA) in the amount of US$0.7 million. The grant was implemented over five years, from December 20, 2013 to December 20, 2018, and is known in the country as the ‘ Scoala Mea’ (My School) Project. It was implemented by Expert Grup (EG), a Moldovan civil society organization (CSO). EG is a nongovernmental organization (NGO), an independent think-thank, specialized in economic and policy research. As part of its institutional mission, EG contributes to economic and social development of the country and promotes innovative policy solutions. Since 2007, EG has been engaged in promoting greater fiscal transparency and policy accountability in many policy areas, including education. Project Development Objectives (PDOs) 10. The objective of the project is to empower the member country’s civil society to : (a) engage local, regional, and national authorities in evidence-based policy and budget dialogue in the member country’s education sector, and (b) promote an enabling environment for social accountability. Key Expected Outcomes and Outcome Indicators 11. Achievement of the PDO was measured by the following four key indicators: (a) Number of educational sector stakeholders taking part in education budget consultations at local, regional, and national level (b) Number of users of scoalamea.md (c) Quality level of the education policy debate (d) Level of satisfaction with the quality of education services 12. Measurement of the PDO indicators is supported by the following nine intermediate result indicators (IRIs): (a) Number of regional focal points engaged (b) Number of local community-level coalitions mobilized and engaged in project implementation (c) Citizens taking part or represented by civil society organizations in participatory budgetary cycle (d) Schools’ budgets discussed (e) Active users of scoalamea.md (f) Participants’ self-assessed readiness degree (g) Feedback reports from schools’ management and relevant public authorities on Social Accountability activities (h) Citizens participating with meaningful contributions in high-level policy debates, both written and oral (i) Knowledge shared with other stakeholders nationally and internationally as part of annual conferences Page 6 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) Components 13. The project consisted of four components. Component 1 focused on strengthening the capacity of the country’s civil societies to create partnerships and use SAcc tools; Component 2 focused on empowering local and regional educational stakeholders to increase accountability for implemented policies and decisions; Component 3 promoted SAcc tools used in fostering policy and budget dialogue; and Component 4 supported implementation through communication, dissemination of knowledge, and learning. 14. The summary of all four components is as follows: (a) Component 1: Building an Environment Enabling Sustainable Social Accountability (SAcc) in Moldova. The activities under this component envisaged strengthening the capacity of the CSOs to create partnerships and use SAcc tools. During the project, the following activities were carried out: (i) assessing the training needs of CSOs, (ii) creating networks of regional and local focal points for promoting SAcc tool use, (iii) developing mechanisms and tools related to school policy and budget plans, (iv) delivering relevant trainings and workshops mainly to school management and representatives of local communities and other relevant stakeholders, and (v) periodical Open Data Readiness Assessments (ODRA) by EG. A total of 100 schools from 27 rayons and 3 municipalities and 5 CSO partners were involved in these activities. (b) Component 2: Applying SAcc Tools to Empower Local and Regional Stakeholders to Increase Policy Accountability. This component aimed at empowering local and regional educational stakeholders to increase accountability for implemented policies and decisions. The activities carried out to support an increased accountability were (i) holding meetings to discuss budget in selected schools, (ii) developing school report cards and collecting and analyzing their results, and (iii) uploading the data on the website to foster budget dialogue and exchange of information at all levels. Stakeholders from 100 schools participated in the budget debates, discussed the budget, and filled out the school report cards. (c) Component 3: Integrating SAcc with Policy and Budget Dialogue. This component promoted SAcc tools used in fostering policy and budget dialogue. The activities under this component consisted of (i) independent education budget analyses (IEBAs), (ii) monitoring activities, and (iii) workshops for relevant stakeholders. The Monitoring and Evaluation Framework was implemented under this component keeping an integratory role. (d) Component 4: Knowledge and Learning. This component aimed at disseminating both the products on SAcc results and the lessons learned during implementation. The activities were based on sharing case studies, successful stories, reporting templates, and guidance notes. The media channels used were project website, Facebook page, magazines, radio, external website, and local and international events. Page 7 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) II. OUTCOME Assessment of Achievement of Each Objective/Outcome 15. The objective of the project is to empower the member country’s civil society to: (a) engage local, regional, and national authorities in evidence-based policy and budget dialogue in the member country’s education sector, and (b) promote an enabling environment for social accountability. 16. Objective 1 - to empower the Moldovan civil society to engage local, regional, and national authorities in evidence-based policy and budget dialogue in the education sector. The main activities supporting this objective were carried out under Components 1 and 2—Component 1 ‘Building an Environment Enabling Sustainable Social Accountability’ and Component 2 ‘Applying SAcc Tools to Empower Local and Regional Stakeholders to Increase Policy Accountability’. CSOs implemented activities using SAcc tools developed by the project to engage authorities at all levels in evidence-based policy and budget dialogue in education sector, mainly in schools. The results of this obective were measured by the following indicators: (a) Indicator 1: Number of educational sector stakeholders taking part in education budget consultations at local, regional, and national level. This indicator exceeded its target. A total of 14,800 people participated at public hearings organized on education budget consultation, two-and-a-half times more than what was anticipated. EG established a national coalition, engaging all the authorities from the central to local levels: Ministry of Education, Culture, and Research (MoECR); rayonal councils and local mayors; school managers; and members of civil society: parents, students, and teachers. Actors from all levels participated in 100 public hearings organized at the community level to discuss and prioritize school needs in full consultation using the SAcc tools developed. To further support measurement of Indicator one in reaching PDO 1, four intermediary result indicators (IRIs) were monitored during implementation and all of them met their targets during implementing activities. All the IRIs contributing to the PDO 1 met their targets. • IRI-1. Number of regional focal points engaged. Five CSOs were identified to create a partnership and engage them as regional focal points in fostering budget dialogue and using SAcc tools in policy making. The CSOs were trained to become change agents in the SAcc tools’ implementation and building constructive dialogue. The CSOs are specialized in different areas (that is, education, youth policies, local development, and participatory democracy) and have national or regional coverage helping at different levels of authority. This partnership proved to be one of the main avenues to ensure sustainability and replicate project activities beyond the its closing. • IRI-2. Number of local community-level coalitions mobilized and engaged in project implementation. A total of 100 coalitions at the school level with representatives of schools, local public authorities (LPAs) students, teachers, and parents were set up to use the SAcc tools developed. Page 8 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) • IRI-3. Citizens taking part or represented by CSOs in participatory budgetary cycle. A total of 54,900 citizens participated in 100 public hearings organized to engage community members in the budgetary process. Through public hearing, citizens had the opportunity to engage and learn about budget preparation and execution while prioritizing, together with school management, school needs. The methodologies and SAcc tools developed by EG served as an implementation framework to engage stakeholders in budget dialogue. • IRI-4. Schools’ budgets discussed. EG developed the social accountability guide called ‘Guide to open your school’ to explain the roles and rules of engagement for budget disscusion (http://scoalamea.md/vrei-sa-aplici-instrumentele-de-responsabilizare- sociala-in-scoala-ta-si-ai-nevoie-de-ajutor-contacteaza-ambasadorii-scoala-mea/; http://scoalamea.md/ghidul-ambasadorului-scoala-mea/). In addition, EG developed guidelines and templates for public hearing on education budget (http://scoalamea.md/mini-ghid-audieri-publice/), an implementation guide for education stakeholders report cards (http://scoalamea.md/cum-se-implementeaza- fisele-de-raportare-in-scoala/), and templates for school report cards and independent budget analysis. As many as 100 communities met, learned how to engage in the local budget dialogue, and discussed 100 school budgets to prioritize education and learning needs for their children. (b) Indicator 2: Number of users of scoalamea.md. This indicator exceeded its target. A total of 25,125 unique users, about 170,000 visits, and 15,800 active users were registered when data were counted for the scoalamea.md results. The website scoalamea.md was developed to facilitate evidence-based policy, budget dialogue, and exchange of information. The user- friendly and intuitive format of presenting school data, indicators, and budget helped in extensive use of the platform, sharing data, and allowing people access all the tools and guidelines developed. Over the last five years, more than 170,000 people have used the website and interacted on it. This indicator is also measured through a result indicator counting the active users. • IRI-5. Active users of scoalamea.md. The active users counted are those who spent more than three minutes on the website. A total of 15,800 active users were recorded, far exceeding what was initially estimated, exceeding the target again because of consistent efforts by EG. The relevant data for methdologies and tools were uploaded on the EG website, www.scoalamea.md, that has recorded a metric of 54,900 page views and 25,125 unique users. 17. Objective 2 - to empower the Moldovan CSOs to promote an enabling environment for social accountability. The lead CSO together with the regional CSO partners promoted an enabling environment for SAcc by developing and using methodologies and SAcc tools in 100 local comunities and disseminated results further through conferences, websites, and other media chanels. The main activities supporting this objective were carried out under Components 3 and 4, respectively. (c) Indicator 3: Quality level of the education policy debate. This indicator almost met the target. The quality level of the education policy debate was calculated based on the experts’ Page 9 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) responses for the 23 questions in the questionnaire “How would you rate the level of debate on educational policies in the Republic of Moldova?,� on a scale of 1 (low level) to 10 (high level). The score varied as the number of experts who participated recorded lower numbers than expected. Also, the experts surveyed were part of the central government administration who suffered frequent changes because of political volatility throughout the last five years. The indicator was recalculated in Year 5 of the project, using only the fully completed questionnaires and the answers from experts who participated in at least three public events (17 out of 25 received, while the methodology required 20), and indicated a value of +2.6, thus reaching and exceeding the target. For more accuracy and as a lesson learned related to the perception indicators, the indicator is kept at +2.36, almost meeting the estimated target. To increase the quality level of the policy debated, participants were exposed to data assessments and trained on many occasions to learn how to improve contribution to education policy. Two IRIs, presented in the following paragraphs, contributed to the measurement of Indicator three and achievement of PDO 2. Both indicators exceeded the target. • IRI-6. Participant self-assessed readiness degree. Overall, the degree of expertise in implementing the SAcc tools was at the upper-medium level (4.5) as expected. EG developed guidelines for ODRA and carried out five readiness assessments for education. The assessment was performed considering eight dimensions useful for policy dialogue: leadership, policy, legal and institutional framework, data at central level, demands for data, open data ecosystem, financing, and information and communication technology (ICT) infrastructure. Based on the findings of the assessment reports, a set of recommendations was prepared to support the MoECR in creating a strong monitoring framework and increasing performance. Some of the key recommendations for enabling the environment for SAcc are nominating a ministry responsible for open data to publicly engage in presenting the education data in a consistent manner; using data in policy making; building capacity for the ICT department; developing a data management methodology and training plan for data collection; and developing a mechanism to identify, record, and monitor data relevant for authorities and civil society. The project has strengthened the use of data provided by the Education Management Information System (EMIS), improved access, and boosted the data availability. • IRI-7. Citizens participating with meaningful contributions in high-level policy debates, both written and oral. A total of 224 citizents contributed meaningfully, exceeding the initial target of 65 after EG delivered trainings, workshops, coaching sessions, and network events to engage citizens in budget dialogue at the community level. During trainings, the school management, principals, and accountants from 100 schools learned how to link financial and learning results to budgets, through a consultative process. During the process, schools prepared the IEBAs based on performance, availability of resources, and school priorities. At the end of the school year, the IEBAs were validated with the school balance sheets and learning results. The IEBA is a user- friendly monitoring instrument, presenting the key performance indicators (KPI) linked with activities and resources in an engaging format. The IEBA result was shared during network events where participants could learn more from their peers, update their Page 10 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) skills, and put everything in general context of sector reforms. In addition, 10 workshops on the midterm education expenditure framework were delivered to 240 school management members to provide more knowledge on the budgetary decision process and expenditure monitoring (http://scoalamea.md/managementul-celor-20- de-scoli-beneficiare-ale-proiectului-scoala-mea-la-atelierul-bugetarea-participativa- in-educatie/). (d) Indicator 4: Level of satisfaction of the quality of education services. This indicator met the target. The scoring scale is 1 to 10. The final target is 10, calculated as +0.5 to the Year 5 cohort, estimated at the beginning of the year. The baseline survey was run from March to May 2018 and established, for the fourth round of schools, a level of school openness at 9.5, much higher than in the previous years. A repeat survey run in November 2018 resulted in an openness score of 9.98. In addition, five quantitative analyses of the report cards were produced, measuring students’, parents’, and teachers’ satisfaction and perceptions of education. The quantitative analyses were prepared every year and validated with regional CSO partners and the World Bank. The consolidated results report is presented at http://scoalamea.md/analiza-cantitativa-a-rezultatelor-fiselor-de-implicare-scoala-mea- 2018/. This survey of student population will generate a great deal of information useful to institutions in their improvement efforts. Two IRIs contribute to PDO 2, one helps in considering feedback received and the other in disseminating good practices and transfering knowledge. Both indicators met their targets. • IRI-8. Feedback reports from schools’ management and relevant public authorities on Social Accountability activities. Stakeholders from 100 schools participated in budget debates and discussed the budget with more knowledge on the regulations, roles, and ways to improve the situation. Moreover, in 2018, 83 schools were still organizing public hearing and open discussion for their budgets. • IRI-9. Knowledge shared with other stakeholders nationally and internationally as part of annual conferences. EG is the main contributor in sharing knowledge, results, good practices, and success stories that created a sustainable environment for 450 persons, exceeding the target. A total of 36 success stories became popular and were disseminated on the project’s website, Facebook page, in magazines, on radio, and on the GPSA website. An international outreach was covered during the GPSA Forums in November 2017 and November 2018, in Washington, D.C., and in an Open Government Partnership that took place in July 2018, in Tbilisi. At the national level, EG participated in 10 events organized by the Independent Press Association, in the Campaign ‘Fii cu ochii pe autorități’ (Be the watchdog of the authorities), to present the SAcc instruments used in the project. The ‘My School’ Ambassador Network composed of school managers willing to serve as a local focal point of SAcc was created before project closing. Thus, a network consisting of 30 ambassadors, who gained experience over the project duration and who are evenly distributed in the 15 rayons of the country, is in place, ensuring project sustainability. 18. The full description of the objectives, specific objectives, and the indicators and intermediary result indicators that contributed to reaching the objectives is presented in annex 1. Page 11 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) Overall Outcome Rating 19. The overall outcome rating of the project was assessed as Satisfactory as of the project closing date, December 20, 2018. (a) The objective to empower the Member Country’s civil society to engage local, regional and national authorities in evidence-based policy and budget dialogue in the Member Country’s education sector was achived as by its measured indicators which were fully met or surpased. Thus, PDO indicators 1 and 2 were exceded, and the intermediate results indicators were fully achieved. (b) The objective to promote an enabling environment for social accountability was almost achieved as measured by Indicator 4 which was fully achieved and Indicator 3 which almost reached the target. All the IRIs under this objective achieved or exceeded the targets. It must be noted that the last mentioned indicator did not fully reach the established target considering its ambitious level, as well as because of the calculation methodology issues. In line with the lessons learned, the teams agreed to keep the indicator while recommending the project’s closing rating as Satisfactory. Other Outcomes and Impacts 20. Successful implementation of the SAcc tools and their positive impact motivated other donors in the country to include these instruments in their projects or implement similar projects. For example: (a) The SAcc tools were adopted by the Common Initiative for Equal Opportunities Project of the Eastern European Foundation Moldova that is going to be implemented at the rayonal level in Cahul, Causeni, and Soroca to increase the representation of the vulnerable members of the society and budget inclusiveness. The project amounts to €2.9 million (http://eef.md/index.php?pag=page&id=1049&l=ro); (b) United States Agency for International Development just launched a five-year program named ‘Comunitatea Mea’ (My Community) that aims to strengthen local governments in Moldova to become more effective, transparent, and accountable to citizens. The program is facilitating Moldova’s decentralization process, building local governments’ capacity to meet citizens’ needs, and strengthening citizen confidence that democratic institutions improve their quality of life. To do so, ‘Comunitatea Mea’ provides comprehensive assistance to LPAs in communities throughout Moldova, who engage civic groups and businesses to improve public services (https://www.irex.org/project/comunitatea-mea-my- community); and (c) A project similar to ‘Scoala Mea’ is going to be replicated in Romania (http://www.scolicurate.ro). Page 12 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME 21. The project was run in a context of frequent Government changes, followed by institutional leadership changes, and significant public administration reform. Thus, a formal collaboration with the MoECR and LPA established by EG through a Memorandum of Understanding (MoU) and extensive involvement of stakeholders at all decision levels were critical for successful implementation of the project. 22. At the same time, the formulation of some monitoring and evaluation indicators and their calculation methodology represented a constraint during project implementation. The most relevant example is related to the qualitative project outcome indicator measuring the level of satisfaction with the quality of education policy debates and the level of satisfaction with the quality of education services. The ambitous target of +2.5 to the baseline, frequent changes in Government structures, and methodology issues caused difficulties in measurement and results reporting. (a) Specifically, the quality level of the education policy debate was calculated based on the experts’ responses for the 23 questions in the questionnaire “How would you rate the level of debate on educational policies in the Republic of Moldova?,� on a scale of 1 (low level) to 10 (high level). The score registered variation as the number of experts who participated recorded lower numbers than expected. Also, the experts surveyed were part of the central government administration who suffered frequent changes because of political volatility throughout the last five years, which led to skewed results. (b) It needs to be considered that measurement of satisfaction level targets permits small increases over time and the baseline value is usually low. The target required more scrutiny and different design to achieve a more realistic scoring instead of expecting high satisfaction levels for the quality of education policy debates. 23. In this context, the flexibility offered by the Grant Agreement to revise the table of indicators, with proposals of recalculation or dropping some of them, was an adequate approach. The teams agreed to keep the indicator as it was calculated as being almost fully achieved, 2.36 instead of 2.60, while recommending the project’s closing rating as Satisfactory. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME 24. Bank performance was satisfactory at entry. The Bank, GPSA and EG was closely collaborating from the beginning of the Project, prioritizing and designing it in support to the initiated reforms in educational sector. There were several moderate shortcomings, mostly related to the frequent changes in the Governmental structure. The continuous engagement of the Bank in the sector though the ongoing in parallel Moldova Education Reform Project (MERP, P127388), from one side, and EG collaboration with various stakeholders from other side, ensured the Project smooth implementation. 25. Bank performance was satisfactory at supervision. There were regular supervision missions from the Bank sides. The EG counterparts repeatedly expressed their gratitude to the Bank for the prompt reaction to the requests, as well as for continuous support in implementation. Hired by the GPSA Page 13 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) consultants, contributed the development of high quality SAcc tools, as well as to a better promotion of Project results. Close and intense collaboration between the Bank, GPSA and EG teams contributed to the successful Project completion, recognized at the national and international levels. 26. Bank performance was satisfactory at completion. Over the five years of implementation, the project had four TTLs. Despite a high turnover of task team leaders, it is worth mentioning that their collaboration, experience, and knowledge sharing even after the handover, plus Trust Fund (TF) accreditation, contributed to the smooth oversight of project implementation and its successful completion. 27. The project has been implemented over the initially expected period without any extension and spent 98 percent of the funds allocated. The PDO remained unchanged and has been rated Satisfactory. There was no risk in achieving sustainable results as the project bridged the gap between the former CPS and the current CPF, providing a good framework and tools for extending the engagement of citizens from 100 schools to 1,200 schools. 28. There were no significant Procurement or Financial Management compliance issues. The Project have had an established operational budget related to the staff costs, as well as a simplified Procurement Plan. Disbursements were done in tranches linked to the agreed results and in compliance to established rules. This flexible approach towards the disbursements and Procurement Plan contributed to an easy adjustment to the Project needs. Audits were on time and unqualified. The final project audit was submitted in March 2019 and was unqualified. 29. The supervision budget allocated for the project was estimated at US$113,500 and was exceeded by 10 percent only. Most of the funds were allocated to staff costs, followed by short-term consultants, as presented in table 1. Table 1. Supervision Expenses by Category Category Expenses Percentage of Expenses Staff 67,139 54 STC and STT 34,522 28 Travel 17,498 14 Others 5,658 5 Grand Total 124,817 100 Note: STC = Short-term consultant; STT = Short-term temporary. 30. Risk to development outcome is rated as low. The established 30 ‘My School’ Ambassador Network, servs as a local reference point of social accountability. This offers the possibility for other interested parties that want to implement social responsibility tools on their own to benefit from the experience that Ambassadors have gathered these past 5 years. Yet, the process to increase accountability and transparency in the education sector by application of SAcc tools, needs to be expanded to other levels of education and continuously reinforced to consolidate achieved results and to become fully sustainable. Page 14 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) 31. From this perspective, the World Bank’s corporate indicators, which were applied to the current Moldova Education Reform Project (P127388) (P127388) in relation to citizen engagement, can be an important component. This shall be supported and expanded to other similar activities. Other external partners activities using the SAcc tools developed under the Project are also contributing to its sustainability. V. LESSONS LEARNED AND RECOMMENDATIONS 32. During the five years of project implementation, a significant amount of experience was gained, many lessons were learned, and recommendations were collected. These can be summarized in three main categories—general, national, and project—and at the used instruments’ level. 33. At the general level (a) Choosing strong CSOs is critical for project success. EG is a sound education partner in the decision-making process especially dedicated to transparency, autonomy, and accountability, bringing to the table evidence-based and consultative processes. This was directly reflected in project activities and its further sustainability by EG’s contribution in: (i) Implementation pilot of the newly adopted Code of Education, development of new supporting tools; (ii) Partnership with the National School inspectorate on the development of annual evaluation questionnaires for teachers and schools. The ‘My School’ team promoted these activities in schools; (iii) Expertise provided as a permanent member of the working group for the following: staff remuneration in schools, self-evaluation of school management, introducing of indicators for school budget consultation, and parents’ involvement in education process; and (iv) Partnership with the Ministry of Education culture and Research to implement the regulation on the organization and functioning of the administrative board of schools using good practices from the ‘My School’ project and its well-functioning administrative boards. (b) Additional CSO partnerships at the regional level bring value added and help better coverage of geographical spaces, as well as increase collaboration with schools by reducing the distance and increasing availability (smaller number of schools per CSO). EG’s partnerships with five regional CSOs, distributed geographically to cover all the country regions, allowed them to allocate more time to schools as partners were closer and the distance smaller. Also, distributing the schools evenly (20 for each regional partner) allowed them to increase collaboration between the school and partner, because there was a smaller number of institutions to take care of. At the same time, this allowed EG to have more time to focus on the development of the tools and integration of the project results in policy documents. Page 15 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) (c) Signed MoUs were critical to consolidate the collaboration and ensure activities’ sustainability. The transfer of power from upper levels is not always easy and this process was helped by signing of the MoU at the beginning of the year where parties agreed on data sharing and participation. An MoU was signed between the MoECR and EG. This was extremely important because of high political volatility and turnover of the ministerial management staff. The MoU established the roles and responsibilities of the parties to respect the conditions that enabled the decentralization process. (d) In addition, the cooperation and advisory role was played by the Advisory Board consisting of members such as the Secretary of State from the MoECR, the Ombudsman for Child Rights, the Head of Finance of the education sector within the Ministry of Finance, and other experts in education who supported the project development during several changes in the Government. 34. At the national level (a) Low interest for the project by the schools teaching in Russian may indicate a linguistic barrier as all the project materials were initially available only in Romanian and English. In 2017, all the SAcc materials (guides, mini-guides, questionnaires, leaflets, and change stories) were translated to Russian to increase the likelihood of applying the SAcc tools. As a result, there were several applicant Russian-speaking schools, and one school was selected as a beneficiary of the project in 2017. (b) The project team learned that it is important to facilitate communication between the beneficiary schools of the project, as well as their communication with non-beneficiary schools. Beneficiary schools’ collaboration offered them the option of knowledge sharing and a learning opportunity on how to improve the application of the SAcc tools and use. Also, their collaboration with non-beneficiary schools allowed expansion of the project activities and results and contributed to increase in transparency and accountability in the educational system and beyond. 35. At the project level (a) An important lesson learned was related to the perception indicator used at the PDO level. The project had two types of indicators and there were difficulties in measuring and reporting both. As such, indicators often depend not only on project interventions but also on many other factors that are unpredictable and out of the task team leaders' (TTLs) control, their use should be carefully considered while designing the indicators matrix. Also, as this type of results are hard to obtain, the targets need to be established at a realistic level. (b) The methodology used for indicators’ calculation should be well defined and followed to avoid reporting issues later. The method used for data collection, for example, the number of interviewed persons considered, should be consistent over the type to allow accurate measurement. Page 16 of 56 The World Bank Empowered citizens enhancing accountability of education reform and quality (P147607) (c) After several rounds of implementation of participatory cards, they were adapted and simplified. As regional partner CSOs and the quantitative analysis of the participatory cards conducted by EG concluded, the participatory cards are easier to fill currently, though still long. (d) The technical issues of the SACc tools should be addressed more. Although the participation cards are considered useful as information collected is anonymous, it is difficult for schools to implement the cards independently in an online version (especially by parents in rural areas). . . Page 17 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: Increased transparency and public participation in the policymaking and budgeting processes in the educational sector at local, regional and national level in Moldova Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of educational-sector Number 0.00 5500.00 5500.00 14800.00 stakeholders taking part in education budget consultations 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 at local, regional and national level Comments (achievements against targets): Participants at public hearings counted. The target is overachieved. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of users of the Number 0.00 10000.00 10000.00 169600.00 scoalamea.md 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Page 18 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Comments (achievements against targets): Unique users counted. The target is overachieved. Objective/Outcome: Higher-quality education policy debates at local, regional and national level in Moldova Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Quality level of the education Text 5.14 7.64 +2.5 to baseline +2.36 to baseline policy debate 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): The quality level of the education policy debate is calculated based on the experts’ responses for the 23 questions in the questionnaire “How would you rate the level of debate on educational policies in the Republic of Moldova?�, on a scale from 1 (low level) to 10 (high level). The low rate of public debates was caused by frequent changes in the Ministry of Education, Culture and Research leadership (former Ministry of Education) during the last years of the project. The central government administration reform also delayed the implementation of some of the activities. Also, 2014 – Year 1 of the project – was a period of reforms and changes in the education sector; therefore, the score was higher and further expectations of experts in education were also higher. Another aspect of the lower results is the fact that many experts who participated in the survey were part of various governments throughout these years, which may skew the results of the survey. A recalculated indicator for the Y5 survey conducted from November to December 2018, using only the full completed questionnaires and the answers from experts that participated in at least three public events (17 out of 25 received, while methodology was requiring 20) indicates a +2.6, thus reaching and exceeding the target. Objective/Outcome: Higher-quality education policy debates at local, regional and national level in Moldova Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Level of satisfaction of the Text 8 10 Y5: +0.5 to the current +0.5 to the Year 5 quality of education services year cohort's baseline, cohort’s baseline, estimated at the estimated at the beginning of the year beginning of the year Page 19 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) = 9.5 31-Dec-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): The scoring scale is 1 to 10. The final target is 10 calculated as +0.5 to the year 5 cohort’s, estimated at the beginning of the year. The baseline survey was run in March-May 2018 and established for the fourth round of schools a level of school openness at 9.5, much higher than in the previous years. A repeated survey run in November 2018 resulted in an openness score of 9.98. A.2 Intermediate Results Indicators Component: Building an environment enabling sustainable social accountability in Moldova Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of regional focal- Number 0.00 5.00 5.00 5.00 points engaged 17-Feb-2014 17-Feb-2014 20-Dec-2017 20-Dec-2018 Comments (achievements against targets): 5 regional focal points engaged- partners in the project Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Number of local community- Number 0.00 100.00 100.00 100.00 level coalitions mobilized and engaged in project 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 implementation Page 20 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Participants' self-assessed Text absent upper-medium upper-medium upper-medium readiness degree 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): Overall degree of expertise in implementing the SACc tools is at the upper-medium level (4.5) Component: Applying social accountability tools to enhance accountability of education management and transparency of school budget Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Citizens taking part or Number 0.00 25000.00 25000.00 54900.00 represented by CSOs in participatory budgetary cycle 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): Only citizens taking part in participatory budgetary cycle are measured. Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Schools' budgets discussed Number 0.00 100.00 100.00 100.00 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): Page 21 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Feedback reports from the Number 0.00 100.00 100.00 100.00 school management and relevant public authorities on 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Social Accountability Activities Comments (achievements against targets): Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Active users of the Number 0.00 600.00 600.00 15800.00 www.scoalamea.md 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 Comments (achievements against targets): An active user spends more than 3 minutes on the website. Component: Integrating Social Accountability activities with national education policy Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Citizens participating with Number 0.00 65.00 65.00 224.00 meaningful contributions in high-level policy debates (both 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 written and oral) Page 22 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Comments (achievements against targets): School management (directors, deputies, accountants) that meaningfully contributed are counted. Component: Knowledge & Learning Unit of Formally Revised Actual Achieved at Indicator Name Baseline Original Target Measure Target Completion Knowledge shared with other Number 0.00 200.00 200.00 450.00 stakeholders nationally and internationally as part of 17-Feb-2014 17-Feb-2014 20-Dec-2018 20-Dec-2018 annual conferences Comments (achievements against targets): Page 23 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) A. ORGANIZATION OF THE ASSESSMENT OF THE PDO Objective/Outcome 1 - empower the Moldovan civil society to engage local, regional and national authorities in evidence-based policy and budget dialogue in the education sector 1. Number of educational sector stakeholders taking part in education budget consultations at Outcome Indicators local, regional, and national level (Indicator One in the Project Paper) 2. Number of users of scoalamea.md (Indicator Two in the Project Paper) 1. Number of regional focal points engaged 2. Number of local community-level coalitions mobilized and engaged in project implementation Intermediate Results Indicators 3. Citizens taking part or represented by CSOs in participatory budgetary cycle 4. Schools’ budgets discussed 5. Active users of scoalamea.md 1. A national coalition was set up, engaging all the authorities from the central to local levels but also individuals representing civil society: parents, students and teachers. 2. Five CSOs functioned as regional focal points and regional partners in the project. The CSO continue to use the SAcc tools developed, extending their activities in other schools. 3. 100 local coalitions at school level were set up and 80 percent of them are still functioning, Key Outputs by Component and they comprise representatives of schools, local public authorities (LPAs) students, (linked to the achievement of the teachers, and parents. They represent 8 percent of the actual school network, enabling the Objective/Outcome 2) environment for extending the use of SAcc tools. 4. 50,000 citizens participated in the public hearing of the budget or other events related to budget cycle. The results doubled the initial estimation, showing a very high engagement of both parts of the society, CSOs and community members. 5. 25,000 users of scoalamea.md compared with target of 10,000 is a great result to prove social engagement, participation, and promotion of social accountability. Objective/Outcome 2 - empower the Moldovan CSOs to promote an enabling environment for social accountability Outcome Indicators 1. Quality level of the education policy debate (Indicator Three in the Project Paper) Page 24 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) 2. Level of satisfaction with the quality of education services (Indicator Four in the Project Paper). The way the indicator related to level of satisfaction with the quality of services was estimated, needed to reach a perfect score. It was almost close which is admirable. However, usually measurement of satisfaction levels target small increases over time and perfect scores are not relevant. 1. Participants’ self-assessed readiness degree 2. Feedback reports from schools’ management and relevant public authorities on social accountability activities Intermediate Results Indicators 3. Citizens participating with meaningful contributions in high-level policy debates, both written and oral 4. Knowledge shared with other stakeholders nationally and internationally as part of annual conferences 1. Overall degree of expertise in implementing the SACc tools is at the upper-medium level (4.5) as assessed by participants. 2. Feedback reports on SAcc activities were received from the 100 schools. 3. 224 citizens contributed meaningfully in the policy debates, three times more than initially Key Outputs by Component estimated. (linked to the achievement of the 4. Knowledge sharing, results, good practice, success stories, open data reports, and other Objective/Outcome 1) analytical research open to the audience is what creates sustainability and frameworks that last. 5. Trainings, workshops, coaching sessions, and network events delivered to learn how to use SAcc tools in the budget dialogue at all levels Page 25 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) . ANNEX 2. PROJECT COST BY COMPONENT Amount at Actual at Percentage of Components Approval Project Closing Expenses (US$) (US$) Component 1. Building an Environment Enabling 129,987 124,862 18 Sustainable Social Accountability in Moldova Component 2. Applying SAcc Tools to Empower Local and 100,983 97,459 14 Regional Stakeholders to Increase Policy Accountability Component 3. Integrating Social Accountability with 139,685 138,084 20 Policy and Budget Dialogue Component 4. Knowledge and Learning 35,323 57,651 8 Component 5. Project Implementation, Supervision, and 290,977 268,097 39 Administration/Audit cost Total 696,955 686,153 100 1. PDO 1 of the project to empower the Moldovan CSOs to promote an enabling environment for social accountability was achieved with funds allocated from Components 1 and 2, 32 percent of the expenses. Around 28 percent of the funds were allocated for PDO 2 to empower the Moldovan civil society to engage authorities in evidence-based policy and budget dialogue in the education sector. The highest share of expenses is related to implementation and amounts to 39 percent of the project. The percentage of expenses after five years of implementation of the grant funds were at 98 percent of the TF amount. 2. As a category of expenses, consulting services is the one that leads the ranking with 83 percent, as shown in table 2.1. Table 2.1. Expenditure by Category of Costs Type of Costs Allocation Expenses Percentage Percentage spent of Spent of the Allocation Total Consulting services 559,820 566,617 101 83 Dissemination costs 14,177 14,044 99 2 Training/capacity building 38,088 27,476 72 4 Others 84,870 78,016 92 11 Total 696,955 686,153 98 98 3. The disbursement graph (figure 2.1) illustrates that EG disbursed periodically in almost equal tranches of funds, ensuring a permanent flow of funds to carry out the activities. Page 26 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Figure 2.1. Project Disbursement Graph Source: Operations Portal 4. The selected auditor, Grant Thornton, performed the auditing services during February–March and submitted the report on March 15, 2019. The auditors provided a clean unqualified opinion covering all grant expenses. Page 27 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) ANNEX 3. RECIPIENT, CO-FINANCIER AND OTHER PARTNER/STAKEHOLDER COMMENTS There are no comments provided; the Recipient completion report is attached in annex 4. Page 28 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) ANNEX 4. SUPPORTING DOCUMENTS (IF ANY) Recipient Implementation Completion Report MOLDOVA EDUCATION GPSA COMPLETION REPORT Moldova GPSA Grant “Empowered Citizens Enhancing Accountability of Education Reform and Quality� (P147607) I. Introduction 1. The strategic goal of “Scoala Mea� (“My School�) initiative is to empower Moldova’s citizens to engage local, regional and national authorities in evidence-based policy and budget dialogue regarding educational reform, quality of services, and development priorities of primary and general secondary schools; and to enable an environment in which Social Accountability (SAcc) initiatives thrive and develop. 2. Expert-Group chose to work in this field back in 2013 because of its dimension, importance, both social and economic, resource allocation, thus potentially generating the widest outcome. Promoting social responsibility at schools’ level is very important as it reaches many individuals (students, parents, teachers and other community members but also decision-making factors) and also reaches some of them at young ages. In this way the Project aims for long-term impact as raising a new responsible and active generation along with shorter goals as supporting the reform of the educational sector by promoting transparency in budgetary sector and higher participation of community in educational process. 3. What is the project about? Within 5 years of the project “My School�, 100 schools were guided and supported in implementing SAcc tools. Three SAcc tools were developed for this purpose and implemented in beneficiary schools: participation cards, independent school budget analysis and public hearings of school budget. Through regional partners of the project (5 regional CSOs) project supported 100 schools to implement Sacc tools independently. Moreover, the project supports the schools to share the experience with other non-beneficiary schools from the same district. 4. The Project followed these specific objectives: • Facilitate engagement of local stakeholders in approximately 100 schools using social accountability tools and promoting a dialogue on school budgets. • Facilitate the flow of information from users of education services to local and national authorities. • Promote the use of three new social accountability tools as inputs into the formal budgetary processes at the school level in education. • Inform the public about the impact of wider economic and financial conditions on the educational sector and reforms (e.g., current situation, availability of budgetary resources, forecast). Page 29 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) • Support the Ministry of Education and other policy stakeholders in improving the quality of data to better support an evidence-based policymaking process. 5. The Project's direct beneficiaries are: • An estimated 60 thousand students, parents, and staff in approximately 100 schools. • Five regional CSOs in the towns of Balti, Cahul, Ungheni, Chișinău, and Soroca. It is with these organizations that the 5 regional partnership were established. These organizations served as links between the EXPERT-GRUP and local communities. • Local public authorities in the communities targeted by the Project (including about 80 mayors and some 650 local councilmen). • About 30 regional authorities (27 district-level, two municipal, and one autonomous region) that had access to the information produced by the Project. • The Ministry of Education due to more reliable and timely feedback on the status and impact of its educational reforms. 6. The Project's indirect beneficiaries are: • Students, parents, and staff at schools not directly targeted by the Project, but who can study its results and make better informed decisions in their schools (e.g. Knowledge and Learning component). We have to mention that there are 15 schools that are independently applying the SAcc tools outside the project framework. • Populations in communities targeted by the Project that will gain from efficiencies of improved local education and better educated young people. • The Ministry of Finance which will have additional information on how effectively education public funds are spent at the local level. • International donor programs supporting the education sector in Moldova. II. Key results 7. The initiative was launched in 2014 by the Independent Think-Tank Expert-Grup and its regional partners, with the support of the World Bank Group within the Global Partnership for Social Accountability and the informational support of the Ministry of Education of Moldova. The project’s mission is to involve citizens, together with authorities in creating efficient, transparent and better schools. The project promotes changes in attitudes, mentalities and participatory behavior using three social accountability (SAcc) tools ii) public hearings of school budgets, ii) Implementation of participatory cards measuring the satisfaction on the education process, iii) independent budget analysis, etc. Page 30 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) 8. In five years, the project engaged about 60,000 people in 30 districts and municipalities. 100 beneficiary schools learned and applied social accountability tools for school budget analysis and created opportunities for students and their parents to make their schools better. 9. During the public hearings, students, parents, teachers, school managers and local public authorities discussed how to improve the educational process and make their schools safe and efficient. The proposals for budget planning vary from bike parking and lunches to modern science labs and video- security of schools. Local officials (more than 900 people, ranging from local councilors, mayors, to rayon presidents) that took part in public hearings promised immediate attention to critical school infrastructure, such as repair of heating and sanitation facilities. 10. The feed-back collected through anonymous participatory cards filled in by students, parents and teaching staff of the beneficiary schools revealed a relatively high level of satisfaction of the educational process in the schools, but also identified the most important issues to be addressed as need for increased transparency in the execution of the school budget (i.e. regular presentation of budgetary information in a user-friendly format and higher transparency of parents’ monetary contribution spending) and higher priority of students’ health and safety issues (i.e. transportation, school lunches and sanitary facilities). 11. The experience of past years’ beneficiary schools engaged in the project has led to positive outcomes as they became more transparent and efficient in budgetary execution and more successful in attracting donor projects and other extra-budgetary funds, which paves the way to better schools, better educated society and, finally, better country. An example is the Ungheni district, where the Council adopted a decision to allocate about USD 2,000 for each beneficiary school to co-finance priorities identified during the public hearings. The schools used the funds for interactive boards, books, projectors, TVs, etc. Therefore, most of them continued the implementation of SAcc tools independently in 2016.Moreover, several local public authorities from Cahul and Riscani districts took the example of public hearings in their community to discuss the local budget and priorities with a wider audience. Examples of change stories at individual school level are presented bellow • Lyceum “Alecu Russo� (Sângerenii Noi, Sângerei): after the public hearings, with the support of the regional partner of the project and the insistence of parents in the community, the school obtained financing for the pupils’ transportation from nearby villages from the Rayonal Council. • Gymnazium Falestii Noi (Falestii Noi, Falesti): the reconstruction of the heating system was possible and already in progress after the discussions at the public hearing in 2016. At the public hearings in 2016, the mayor promised to contribute financially for one of the most important problems faced by the school: the heating system. In 2017, the mayor not only kept his promise, but also participated in negotiations with the Rayonal Council in order to receive the entire amount needed. Currently the reconstruction is in progress. • Gymnasium Rosietici (Rosietici, Floresti): managed to receive a grant of US Embassy for the renovation of toilets and sanitation system (issued discussed at the public hearings) due to demonstrated transparency. The school director mentioned that they achieved this high degree of transparency after participating in “Scoala Mea� project. Page 31 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Examples of initiatives at general school level • A network of beneficiary schools was created with the support of the regional partner of the project from Ungheni. Thus, 16 beneficiary schools of 2014-2017 from Ungheni, Calarasi and Nisporeni rayons are members of the network “Local partnerships for a modern school� and share their experience in promoting social responsibility in their schools, but also inform each other about different projects, grants and events designed for schools and communities. • On its own initiative, a beneficiary school form 2016 (from Puhoi, Ialoveni) organized an event for sharing the experience in “Scoala Mea� project and promoting social responsibility among schools with the support of the regional partner of project from Chisinau and the rayonal division of education of Ialoveni. Thus, some of the participant schools at the event expressed their interest to implement SAcc tools independently. • Directors of non-beneficiary schools participated in trainings of local coalitions and public hearings organized by the beneficiary schools with the support of “Scoala Mea� team. They were motivated by the achievements of the beneficiary schools nearby in the previous years and interested to implement SAcc tools independently if not selected for the next round of the project in 2018. Examples at community level • After the participation at the public hearing at school level, some authorities decided to organize public hearings of local / rayonal budgets (Cahul city, Ungheni city and rayon, Soroca city). • After participating in public hearings, mayor of Cahul promotes social responsibility at school and community level in southern region of the Republic of Moldova in different events. • Ungheni Rayonal Council allocates financial resources for beneficiary schools in order to cover some of the needs identified during the public hearings by pupils. This motivates and inspires students. • Other success stories are available on the website scoalamea.md at http://scoalamea.md/category/blog/ or http://scoalamea.md/category/istorii-de-succes/ Component 1: Building an Environment Enabling Sustainable Social Accountability (SAcc) in Moldova. 12. The summary of activities under this component are: 13. The team established a national coalition between CSOs, minister, local public authorities, schools, parents, pupils, school managers, WB, that engaged in the process of social accountability to improve educational services in Moldova; Page 32 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) 14. We identified and trained 5 CSOs to empower them to become “actors of change� in implementing the Social Accountability Tools in schools and to engage in a constructive dialogue with the decision-makers; methodologies. 15. 100 coalitions were established at the school level between school managers, LPAs’ representatives, pupils’ representatives, teachers and parents’ representatives, other interested parties, who implemented the social accountability tools to monitor the quality of educational services. 16. A social accountability model applied to monitoring public education in Moldova consisting of a set of mechanisms and their corresponding methodologies was developed early on during the first years of implementation of the project, that was subsequently adapted into a „Guide to making your school more open� that consists and combines 3 complementary products: • 1 methodology for implementing social accountability at the school level. The guide by itself represents also a sustainability instrument for the Ambassadors in the project. Since the launch of the "My School" project in December 2013, the Independent Expert Group, together with local partners, has been interested in setting up a platform for cooperation and sharing experience in order to promote good practice in implementing social accountability tools. Expert-Grup collaborates with teachers and representatives of training institutions or educational authorities, regional partners, local and central public authorities. One of the multiplication platforms of the project's good practice is the establishment of the "My School" Ambassador Network, where the institutions, after completing the "My School" project, are willing to serve as a local reference point of social accountability. This offers the possibility for other interested parties that want to implement social responsibility tools on their own to benefit from the experience that 30 Ambassadors have gathered these past 5 years. They are located in 15 rayons in all of the three regions of the country, and can be contacted by email or by phone. http://scoalamea.md/vrei-sa-aplici-instrumentele-de-responsabilizare-sociala-in-scoala-ta- si-ai-nevoie-de-ajutor-contacteaza-ambasadorii-scoala-mea/ http://scoalamea.md/ghidul-ambasadorului-scoala-mea/ • 1 methodology for organizing and reporting on public hearings on education budget (guide and mini-guide/how-to-note) (http://scoalamea.md/mini-ghid-audieri-publice/) • 1 methodology for designing, implementing and reporting on Education Stakeholders Report Cards (guide and mini-guide/how-to-note) (http://scoalamea.md/cum-se- implementeaza-fisele-de-raportare-in-scoala/) 17. For all of the aforementioned instruments, not only the methodology was elaborated, but also templates for all the support materials needed. For example, in the case of the public hearings (applicable also for the School Cards Reports, Independent Budget Analysis): a guide and a mini guide on the organization of public hearings was produced describing the general framework of the event. The support materials with templates were produced and disseminated and consisted of templates of agenda, press Page 33 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) release, invitation models, press invitation model, list of participants, list of speakers, presentation template with the subjects outline, posters, school budget template etc. 18. Five reports on open data readiness assessment were produced during the project implementation. The assessment was conducted on the basis of the methodology developed by the World Bank for the assessment of open data at the Government level. Thus, the assessment was performed on eight dimensions: (i) Leadership, (ii) Policy and legal framework, (iii) Institutional framework, responsibilities and skills of the staff, (iv) Data of the Ministry of Education, Culture and Research, (v) Demand for Open Data, (vi) Open Data Ecosystem, (vii) Financing, (viii) National ICT infrastructure and skills. Based on the general methodology, 1 guide and miniguide on how to evaluate the Open Data Readiness Assessment (ODRA) was produced. The fifth report, was disseminated in November 2018, during the National Conference of “Scoala Mea� Project. http://scoalamea.md/studiu-de-evaluare-a- datelor-deschise-in-educatie-2018/ Narrative description of the efforts under this component. 19. The school is still perceived as a main institution of the local community, but in the process of transition of schools from community budget to financial autonomy after 2012, the dialogue between the school management and local and district authorities became deficient in some cases. Therefore, setting a formal framework between these stakeholders was important in order to bring together the community stakeholders and ensure that each of them has the necessary contribution to the process. The parts (schools, local and district authorities and the implementing team) sign Memoranda of Collaboration at the beginning of each year where they agree on sharing the necessary data and participation in the activities of the project that would ensure its proper implementation. 20. Later, in 2015 a Memorandum of Understanding was signed between the Ministry of Education and Expert-Grup. The decision was due to political instability in Moldova and changes in the leadership and technical staff of the Ministry of Education. Even though the new leadership and contact persons may be good-intended, it takes time to take over all responsibilities and the project team considered appropriate to set a formal cooperation in order to avoid delays in the project. The Memorandum of Understanding describes the activities of the project the Ministry of Education supports, the data be shared, etc. 21. While formal cooperation is very important, the team also opted for a less formal cooperation within the project through its Advisory Board. Currently, among the board members are the State Secretary by the Minister of Education, Culture and Research, the Ombudsman for Child Rights, the Head of Finance of Educational sector of the Ministry of Finance and other experts in education. The board meetings are good occasions to have broad discussions regarding the social responsibility process in the educational sector in a less formal setting that allow for brainstorming and continuous improvement in the activities of the project. Also, the Board members can influence the discussions on social responsibility that the project promotes at higher level, which is very important. 22. Being a wide project that concentrates both on national level (i.e. aiming to influence national policies) and at local level (by working directly with schools and local authorities) it is also necessary to have strong partners to promote the goals at different levels. Expert-Grup had the support of five CSOs as regional partners of the project specialized in different areas as education, youth policies, local Page 34 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) development, participatory democracy, etc. This cooperation allowed and alloes for simultaneous work at different policy levels. It also determined the establishment of different types of contacts at local and national levels and increased the efficiency of the project. All activities of the project are consulted with the regional partners and adapted if necessary to the particularities of a school or community. 23. This collaborative approach at the national and sub-national level made the beneficiary schools to be more active as they would think of their role not only in participating in a project and improving things in their own school, but also as being pioneers in a process of social accountability that initiates in Moldova. This has increased their motivation to contribute to the scope of the project by sharing the experience with other schools in the district or town (not necessarily beneficiary-schools) in different ways. Also, the formal cooperation made schools more confident in approaching local and district authorities when needed, especially when they also have the parents by their side. 24. In our effort to engage all stakeholders in the process we tried to identify and respond to the needs of each of them. Thus, while seeking the support of the Ministry of Education, the project team offered support in implementing actions already specified in the recently adopted Educational Code, but that could not been implemented immediately due to financial, technical and human limitation. As a result, the project offered some kind of pilot exercise for many tools and a valuable experience that currently may be used by national and local authorities in developing further policies and implementing better tools. The experience accumulated by the project is very valuable and the collaboration continues with higher implication of the team in the reforming the educational sector, in particular: (a) Participatory cards have served as an example for the institutional questionnaires for evaluation of educational institutions and teachers. In this respect, Expert-Grup started to collaborate with the National School Inspectorate in order to finalize the questionnaires for the annual evaluation activities of the educational institutions and teachers. Also, „My School� team will provide the necessary support for the promotion of these activities in educational institutions. (b) Expert-Grup is part of the working group working on the concept for remuneration in general education institutions and self-evaluation of school management and promotes the introduction of such activities as school budget consultation and parents' involvement in educational process among indicators for the evaluation of the management of the general education institutions. (c) Expert-Grup will support the Ministry of Education to implement the Framework „Regulation on the organization and functioning of the Administrative Board of the general education institutions�, as well as its promotion in the educational institutions based on positive practices of the beneficiary schools of "My School" project that have functional administrative boards. 25. While the project team managed to establish a collaboration with many educational stakeholders, it is still a continuous process as the state of the country changes and political, economic and social motivations also change. This might be a difficult task sometimes, especially when the stakeholders are very different. At the same time, engagement of stakeholders at different levels may foster the overall collaboration. Thus, for instance, initial engagement with the Ministry of Education fostered the Page 35 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) collaboration at district level, while later, when the leadership of the Ministry changed it worked vice- versa in a period of political changes. Partnerships 26. The Ministry of Education, Culture and Research is the main policy maker on educational, research, youth and culture domains in the Republic of Moldova. It is the s the branch of government charged with overseeing public and private education institutions, and covers culture, sport and youth. 27. Relationship over the years: Change in leadership at the Ministry of Education (currently Ministry of Education, Culture and Research) had an impact over some of the goals of the project (mainly on the policy issues promoted within the project). However, the new entity – Ministry of Education, Culture and Research expressed its openness in being a partner for the project and all activities of the project starting with August 2017, were conducted under the new partnership with the Ministry of Education, Culture and Research. 28. Also, Expert-Grup managed to involve actively during 2018 with Governmental counterparts. (a) As of July 2017, Expert-Grup was part on the working group elaborating the concept for remuneration in general education institutions and self-evaluation of school management and promoted the introduction of such activities as school budget consultation and parents' involvement in educational process among indicators for the evaluation of the management of the general education institutions. The recommendations subsequently materialized in the Law on the Single Pay Framework, which as of December 2018, is being implemented nationwide. Although the recommendations have been partially approved on the subject of remuneration in general education institutions and self-evaluation of school management, a clear mechanism has to be yet established, posing some technical problems for the beneficiaries nationwide. (b) Currently, Expert-Grup is member of the Public Internal Financial Control Board by the Ministry of Finance, Member of the Council by the Prime Minister of Moldova, Work Group 3 for education; Representative of Independent Think Tank Expert-Grup in the National Council of NGOs in Moldova; Member of Middle Term Budgetary Framework Group responsible for the remuneration and employment policy in the budgetary sector; Member of Middle Term Budgetary Framework Groups responsible for education, social protection and welfare expenditures 29. Over the years Memoranda of Cooperation were signed in 27 rayons, 2 municipalities and 1 Territorial Administrative Unit (UTA Gagauzia) (with district and local authorities). As a result regional and local authorities were involved in many project activities, supporting the values promoted by the project. Thus, over 730 local and regional public servants (including, mayors, local councilmen, head or deputy head of Rayonal Councils, representatives of educational and finance departments of rayonal councils). As a direct result of this type of partnership, Ungheni rayonal council continues to allocate financial resources to beneficiary schools for the priorities identified during public hearings organized in the project “Scoala Mea� as permanent action to support social accountability in the educational sector. Also after Page 36 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) being involved in the project “Scoala Mea�, the Mayor of Cahul promotes SAcc tools among schools and local public authorities in southern region. It also the case of the city of Călărași, Strășeni and Căușeni. 30. Regional partners – the project works with a nation-wide network of 5 regional CSOs who were trained to guide and steer the local stakeholders in usage of SAcc tools. Partners skills created to engage in SAcc exercises related to education are measured yearly. The readiness degree is estimated based on a questionnaire filled in by project regional partners. Based on their experience gathered during the first four years of implementing SACc tools, three of our five regional partner CSOs self-assessed their level of expertise in implementing the SACc tools in the educational sector of our country as being excellent, choosing the level 5 (maximum); while two partners self-assessed their level of expertise as 4 (upper- medium). These results allow concluding that the current overall degree of expertise in implementing the SACc tools of our regional partners is at the upper-medium level (4.8) an improvement of 2 points from the first year of implementation (2.8 in 2014). In conducting a follow-up self-assessment of our five regional partners, in implementing SACc tools in the education sector of our country, we asked for the lessons learned and potential suggestions on making certain changes to the approved tools and methods. 31. Access to the necessary data and interpretation. All five regional partners stated that they do not encounter any difficulties in accessing and interpreting data on the school budgets, which is an important development considering their direct involvement in providing assistance for schools in preparing the information on school budget and spending for the wider public in an easy to understand format. However, a concern indicated by one of our regional partners is limited capacity in developing visual tools as to present the data on school budgets in an attractive and easy to understand format for the wide public. Some of the partners’ suggestions refer to: (i) creation of an on-line platform where all interested parties might access the data; (ii) placements of the information on school budget on the corresponding school website. 32. Public dialogue. An essential progress after five years of implementation of the current initiative is that all five partners state that they do not encounter any difficulties in participating in the budgetary process, though as a concern stated by all five partners is the reduced openness of local authorities starting with the last year in involving the civil society in the budgetary process. Most regional partners affirmed that it is important for the beneficiary schools to interact among them-selves and share their experience with other schools from their communities and districts, as to encourage them to apply SACc tools and open their budgets to the public. An important issue mentioned is the need to speak about the advantages that this process offers to the schools and at the same time to mention the challenges that need to be addressed with joint effort of all interested parties, as local authorities, the community members, parents and students, as well as the local CSOs and media. A concern in this respect was stated the fact that the Regulation on Administration Councils approved by the Ministry of Education in 2015 was not yet published in the Official Monitor and thus the schools encounter difficulties in formalizing the activity and decisions taken by the created Councils. 33. Training needs. Our regional partners believe it is important to continue the organization and provision of trainings on public finance management and monitoring, budgetary participation, public procurement, as well as social accountability tools (e.g., public hearings, report cards, etc.) for the schools. They also highlighted the need to organize general school assemblies on discussing the importance of engaging parents and students in the participative budgeting process. At the same time it was stressed the importance of holding periodic round-tables at the initiative of Administration Councils of schools with Page 37 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) teachers and representatives of grass roots CSOs, as well as key community actors on the main school issues that they perceive and potential solutions to the identified challenges, as well as costs to be planned in the next year school budget. The participants should also be informed on the execution of the current school budget and the potential bottlenecks. 34. All five partners highlighted the importance of institutionalizing the developed social accountability tools within this initiative and emphasized the benefits, high interest in continuing on their- own and positive feedback showed this far by the majority of the beneficiary schools. They also underlined that the initiative “Scoala Mea� should become a national priority for the educational system of Moldova. 35. School Administrations and School Boards - The feed-back collected during the trainings of local coalitions, the networking session at the MTEF workshop and also the feed-back forms at the end of the project, three main issues were addressed by school administrations and school boards: (a) Adaptability of SAcc tools to the school and / or community context. Also, after the event a note on the participatory budgeting process in education was elaborated by Expert-Grup and distributed to schools in order to inform them about different forms of community consultations. Some of the schools already applied the proposed versions of the consultations (among them: „Onisifor Ghibu, Chișinău, „Ion Pelivan�, Răzeni, Ialoveni, „Constantin Stere�, Soroca). (b) Flexibility in scheduling the activities. While for the project purpose the activities follow a predefined schedule, outside the project the actives can be plan according to school or community needs. In fact, past beneficiary schools have their own calendar of activities for social accountability processes. However, some of the activities should be organized in specific periods given the national budgetary process. In order to help the schools to be flexible in their own activities, but also follow the general budgetary process in Republic of Moldova, Expert-Grup elaborated the Budgetary Calendar in Education. The calendar is available online, but was also printed and distributed to all beneficiary schools from 2014- 2018 for the facilitation of the participatory budgeting activities. (c) Need for financial management trainings for managers. While the project “Scoala Mea� does not cover this aspect, it was raised often during trainings and meetings. The team raised this issue at the Advisory Board Meeting and will also raise it in the discussions with the Ministry of Education, Culture and Research. During the MTEF and participatory budgeting workshop, Expert-Grup invited Ms. Victoria Catanoi (WB consultant for Ministry of Education) for a second year continuously to talk about “Budgetary planning and execution based on school priorities�. The presentation cannot replace the need for a training, but contributed to better understanding of the processes by directors, deputy directors and accountants. Page 38 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Component 2: Applying SAcc Tools to Empower Local and Regional Stakeholders to Increase Policy Accountability. 36. A short outline of the main results for this component are presented below: (a) The team has guided the implementation of 3 social accountability tools – School Report Card, independent school budget analysis, public hearings; organized 100 public hearings where community members discussed school’s problems and created the necessary premises to solve them; (b) Over 1400 school board members received training on the functioning and organization of the school, the social accountability tools, communication with stakeholders; (c) Over 30000 people were directly involved in appreciating the quality of educational services; (d) 16000 citizens (pupils, parents, teachers, authorities) were involved in school budget discussions; (e) Contributed to the change in perception of school in the context of the financial autonomy of school institutions through the need for partnerships for effective administration; (f) Increased the level of accountability of decision-makers at different levels (school management, public and central authorities) and beneficiaries (pupils, parents, community); (g) Developed the scoalamea.md website to facilitate evidence-based policy and budget dialogue and exchange of information. The site presented budgetary data and school performance indicators for all the schools in the country in an intuitive format, mapping out their performance and expenditures at the national and regional levels. Also the website became a knowledge – sharing platform for all the stakeholders, by hosting the guides and methodologies on the implementation of the 3 SAcc tools, 36 success stories etc. (h) Overall, only in 2018, ScoalaMea.md had 53139 pageviews, with 25.125 unique users, and more than 2871 unique users that have spent more than 3 minutes on the website. All in one, more than 170 thousand people interacted with the website and it content over the last 5 years. Component 3: Integrating SAcc with Policy and Budget Dialogue. 37. Activities and this component were concentrated around the realization of 4 distinct activities:(i) independent education budget analyses; (ii) monitoring activities; (iii) workshops for relevant stakeholders; and (iv) quantitative analysis on education budget and policy. According to Comp. 3 of Results framework, policy level-documents include: (a) Independent Education Budget Analyses (IEBA). The IEBA was an analysis that provided an explanatory view into the school finances. For the first time, 100 schools had analyzed their expenses through the lenses of the beneficiary, translating all the financial data into data about how the money spent is ensuring the functionality of the institution. It also invited the Page 39 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) beneficiaries to ask questions about the prioritization of the allocations, the link between the school budget and overall educational performances, standards of education etc. Data for IEBA was usually collected after the training of local coalition from the school administration, with a further update on the school performance during final exams in late June. The validation of data was usually happening during or after the MTEF workshops in July ((IEBA validation roundtable. During the networking session at the MTEF workshops organized on July each year the IEBA was validated with schools directors present at the event (45 participants: directors, deputy-directors and accountants)). http://scoalamea.md/managementul-celor-20-de-scoli-beneficiare-ale- proiectului-scoala-mea-la-atelierul-bugetarea-participativa-in-educatie/ IEBA were conducted during July – August each year. Final IEBA were sent to school directors in late august, before the beginning of the school year. IEBA was used by schools to make user- friendly presentation during the public hearings, but also to present the information at general schools’ assemblies in several schools. Also, Expert-Grup prepared posters for school boards and leaflets for parents with the most relevant information in IEBA. (b) Education Mid-Term Expenditures Framework workshops held bi-annually. One of the greatest problem while dealing with the principals, was the low level of acknowledgement and implication in the budgetary process. The principals weren’t involved in the planning of expenditures, analysis of priorities, but they also didn’t expect that their voice could be heard at the national level. By organizing these workshops, a two-fold effect was expected: i) for them to understand better the link between the Public National Budget, Local Budgets and the budget of the school, ii) give them the entry points into the budgetary decision making process, by explaining the budgetary cycle, the “champions� of change, how to write budgetary proposals, what groups should they join at various levels to promote their interest. 10 workshops were organized with parallel sessions with the participation of all beneficiary schools’ representatives. Over 240 participants were divided in two groups who attended the work-shop and the networking session. The theme of the workshop, while always linked to the MTEF, was annually different in ways to accommodate the discussion of one specific subject (public procurement, creation of administrative councils, policy making, with the assistance of other EG colleagues, school budget planning (with the assistance of Victoria Catanoi) etc. During the last workshop participants learned about participatory budgeting and priority-based expenditures in education. http://scoalamea.md/managementul-celor-20-de- scoli-beneficiare-ale-proiectului-scoala-mea-la-atelierul-bugetarea-participativa-in-educatie/ (c) Quantitative analyses based on econometric techniques highlighting factors influencing satisfaction of the education conducted. Five quantitative analysis were developed over the years, based on the results of the participatory cards filled in in April – May each year. The analysis was validated with regional partners and schools directors present at the MTEF workshops on July and comments received from participants after the event. (The revised document was sent for review to the project TTL (Ms. Lucia Casap) and to the Ministry of Education, Culture and Research (Mr. Valentin Crudu). Final report was published on ScoalaMea.md after its presentation at the annual conference on December 4, 2018. http://scoalamea.md/analiza-cantitativa-a- rezultatelor-fiselor-de-implicare-scoala-mea-2018/ 38. Open Data Readiness Assessment (ODRA). Five reports on open data readiness assessment were produced during the project implementation. The assessment was conducted on the basis of the Page 40 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) methodology developed by the World Bank for the assessment of open data at the Government level. Thus, the assessment was performed on eight dimensions: (i) Leadership, (ii) Policy and legal framework, (iii) Institutional framework, responsibilities and skills of the staff, (iv) Data of the Ministry of Education, Culture and Research, (v) Demand for Open Data, (vi) Open Data Ecosystem, (vii) Financing, (viii) National ICT infrastructure and skills. A set of recommendation was presented to the MoECR an followed up upon their implementation for the last 5 years. Based on the general methodology, 1 guide and miniguide on how to evaluate the Open Data Readiness Assessment (ODRA) was produced. 39. Some of the general recommendations that are still valid and would certainly improve the score in the educational sector: designating a person at the level of the ministry responsible for open data; public engagement in relation to opening data in the sector; the inclusion of open data measures in the ministry's action plans and activity reports; capacity building of the ICT Department; developing a quality control system for open data and services provided; development and approval of personal data protection policy in education; Audit of data available in the sector; developing a data management methodology, called "Open data development in the education sector", which will include aspects of collection, processing, control, approval, publication; developing a training plan for all actors involved in collecting data in the education sector; publishing reports on open data in education; developing a mechanism for identifying and recording data by authorities, including in the education sector; developing a mechanism for identifying, monitoring and evaluating data requests from society, civil society organizations, the media, the private sector. The recommendations also concern the Open Data Ecosystem, Data Opening Funding, the Center for Information Technologies and Communications in Education. 40. The methodological framework in collecting, processing and analyzing open data is essential to ensure transparency and evidence-based policy making. Empowering beneficiaries and actors in the education sector in collecting, processing and interpreting data is also essential in implementing social accountability tools. Despite the availability of data in the education sector, its beneficiaries face difficulties in using them in decision and policy making. 41. In terms of open data in the education sector, the project has strengthened the use of data provided by the Information Management System in Education (SIME), has helped to improve access and has boosted the diversification of publicly available data types. At the same time, the project intervention measures included processing, interpreting and analysing data available from the educational system. This helped the capacity needs assessment of the actors in the education sector. As both school managers and LPAs can use open data to streamline school management, their capacities to interpret such data is essential. This was also highlighted by the central authorities, thus they supported SIME and the use of data in the development of evidence-based policy interventions. The fifth report, was disseminated in November 2018, during the National Conference of “Scoala Mea� Project. http://scoalamea.md/studiu- de-evaluare-a-datelor-deschise-in-educatie-2018/ 42. Independent monitoring of the education policy environment implemented (MEGA Reports). Implementation of reforms set forth by Expert-Grup in MEGA publications (2014-2018). MEGA is the English acronym for Moldovan Economic Growth Analysis. MEGA is a bi-annual publication issued by Expert-Grup since 2009. Its main purpose is to explain the fundamentals of the recent economic trends in Moldova, to analyze the economic policies and to come up with strategic macro solutions for the development of the country. These solutions are usually presented in form of brief, but rather general Page 41 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) recommendations. An important feature of the publication is the forecast model employed by the authors to predict future dynamics of Moldova’s economy. Forecasts predicted by the model largely determine the shape of recommendations. 43. MEGA includes a few key chapters: Domestic Demand, Labor Market, Public Finance, Foreign Relationships, Monetary Policy and Financial sector. Each chapter of the publication is also a result of Expert-Grup’s other analytical and advocacy efforts in each corresponding direction. The main results of the policy and advocacy efforts are described in short in the document MEGA 2.0, Implementation of reforms set forth by Expert-Grup in MEGA publications (2014-2018), available in the shares google drive folder. Among the success to be mentioned are the reform of the State owned enterprises, the institutionalization of the Administrative Boards in schools, by the structure proposed by EG (at least 9 members, with representatives from the schools management, teachers, pupils and parents board representatives, LPAs representatives, businessmen, community leaders etc). 44. MEGA spring edition was released on April 24, 2018. https://www.expert- grup.org/ro/biblioteca/item/1597-mega-editia-a-xviii-a-perspectivele-economice-intr-un-an- electoral&category=178 45. MEGA fall edition was released on December 13, 2018 https://www.expert- grup.org/ro/biblioteca/item/1714-mega-editia-a-xix-a-economia-in-centuri-de-siguranta-urmeaza-un- an-electoral&category=178 46. All editions were consulted with Mr. Marcel Chistruga from the World Bank Moldova. Component 4: Knowledge and Learning. 47. Results aside, the project had benefited from the popularization of the instruments implemented in the project. The collaboration and partnership with the press has helped the team to share to a wider audience the results, but also create a space of knowledge sharing. This was made especially through the popularization of the case studies, the so-called „success stories� elaborated by the partners, with the help of the communicator from EG. Beside the PR person in the “Scoala Mea� Project team, EG hired a journalist to work on the case studies and success stories. A total of 36 studies and stories were disseminated on the Project website, Facebook page, in magazines, on radio, on GPSA website and during the international events as: GPSA Forum in November 2017, 2018 in Washington DC and Open Government Partnership in July 2018 in Tbilisi. At the national level, Starting October 2017 till March 2018, the project coordinator, Tatiana Savva, has participated in 10 events organized by the Independent Press Association, in the Campaign “Fii cu ochii pe autorități� (Be the watchdog of your authorities!), during which she has presented the instruments of social responsibility used in the project. http://www.moldovacurata.md/campania-fii-cu-ochii-pe-autoritati/fii-cu-ochii-pe-autoritati. Page 42 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Lessons learned Component 1: Building an environment enabling Social Accountability Key objective: Creating partnerships and skills to ensure project success Lessons learned • Establishing formal relations and better connections with all stakeholders, including the Ministry of Education (as the Memoranda of Cooperation was signed with the former Ministry of Education) are very important to ensure (i) greater connotation of the project at the national level as seen by the beneficiary-schools who are pioneers in a process of social responsibility that initiates in Moldova (ii) smooth running of the project in case the leadership changes. Despite the challenges encountered during the current year, the project managed to ensure these two aspects due to formal and informal collaboration with all stakeholders. • Not only official support of the leadership of the Ministry of Education in the past, and currently Ministry of Education, Culture and Research is important for the project, but also direct contacts with representatives of the Ministry responsible for areas related to project goals and activities. By the end of 2016 – beginning of 2017, Expert-Grup team managed to establish good relations and cooperation with the former Ministry of Education. This implied both a Memorandum of Cooperation and official support of the Ministry leadership assigned in the mid of 2015, but also direct collaboration with other staff in the Ministry (particularly the Division of E-transformation, Division for Secondary Education, Financial expert in education within the Ministry). In May 2017 the leadership of the Ministry of Education resigned and in August 2017, the Public Administration Reform was launched with the reorganization of the Ministry of Education by merging with two other ministries into the Ministry of Education, Culture and Research. Thus, it is already half a year with no formal leadership for the educational sector as only the Minster was assigned, but the state secretary for the education sector was appointed only on December 18, 2017. However, the project activities could continue as the team worked with other staff in the Ministry from the divisions mentioned above. • Maintaining a network of five regional partners with the previous years’ knowledge helped boost the project’s efforts in the new round of 20 schools and also enforce the collaboration between them and many district or local authorities. The partner Civil Society Organizations (CSOs) have already established connections with local authorities from several 27 districts and municipalities (out of 35 in the country) that ensures a better collaboration in most of them at the local level. • Strong school coalitions can mobilize communities. It is a challenge to create and maintain active School Administrations Boards (SABs) as the Regulation for its functioning has not entered into force yet (while approved in 2015). Nevertheless, in all beneficiary schools of the project, Boards are active and performed most of the intended activities. Expert-Grup will continue its efforts to promote the enforcement of the regulation. Page 43 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) • During the training program for SABs and workshops on Mid-term Expenditure Framework (MTEF) and Participatory Budgeting in Education directors mentioned the scarce of information and knowledge on finance administration issues and expressed their interest for more information regarding the implication in the budgetary process. Thus, the project extended the workshops presentations, that were not limited only to participatory budgeting, but also a training on the distribution of school budgets by needs and priorities with the participation of the financial expert form the Ministry of Education that also implied longer session of Q&A. Also, the project elaborated the Budgetary Calendar in Education, which illustrates who, when and how can involve in the budgetary process. The Calendar was distributed among the schools (not only project beneficiaries). • Lower interest for the project of the schools teaching in Russian may indicate over a linguistic barrier as all project materials were initially available only in Romanian and English. In 2017 all SAcc materials (guides, mini-guides, questionnaires, leaflets and change stories) were translated to Russian to increase their chances in applying SAcc tools. As a result there were several applicant Russian-speaking schools and one school was selected as beneficiary of the project in 2017. Component 2: Applying SAcc tools Key objective: Empower local and regional stakeholders to increase policy accountability Lessons learned • Starting earlier the activities in 2017 allowed for more flexibility and better planned events. While each year the selection of the beneficiary schools was made in the beginning of the year, for 2018, the call for application was launched in end-November 2017 that will allow for earlier start of activities. At the same time, for the independent implementation of the SAcc tools outside the project, schools may choose another schedule of events, not bound by the calendar year, as it is the case in many past beneficiary schools. • After several rounds of implementation of Participatory Cards, they were adapted and simplified. As our regional partner CSOs and the Quantitative Analysis of the Participatory Cards conducted by Expert-Grup concluded, the participatory cards are easier currently, though still long. There is evidence of a number of beneficiary schools from 2014-2016 and also intensions of the beneficiary schools of 2017 in readjusting the participatory cards to meet the school’s needs or to make them shorter. In 2018, for the purpose of the project, the participatory cards from 2017 will be used. At the same time, Expert-Grup collaborates with the National School Inspectorate on the elaboration of questionnaires to be applied to parents and pupils for the evaluation of the teachers and schools that contain questions in the areas covered by the project’s participatory cads. • We should address more the technical issues of the SAcc tools. Thus, the participation cards that are considered very useful as they collect information in an anonymous way are still difficult to be implemented independently by schools in online version (especially on parents in rural areas). "Scoala Mea" team also discusses with the National School Inspectorate on Page 44 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) the issue, with the possibility to organize trainings in the future on the use of online questionnaires in schools and at district level. • For long-term use, the sample for the participation cards may be adjusted with (i) simplified questionnaires for younger students that do not participate currently in the process and (ii) parents of students of all ages should be given the possibility to fill in the questionnaires. In 2017 and 2018, in several school the cards were applied for students of 6th grade and parents of children of all ages. While for the purpose of the project and comparison, we will use the cards for the usual sample in 2018, each school may apply them to a larger sample, which was already possible. In fact, the feed-back collected from parents of children in primary schools was reported more relevant due to their younger age and higher involvement in school activities. • Discussions in advance regarding the use of the participatory cards (during the networking session after the workshop of participatory budgeting workshop in 2018) and sharing the results in early summer helped school directors to be more prepared at the public hearings of the school budget. The administration already analyzed the priorities identified by parents / students / teachers and started the discussions based on the results collected, also indicating exact numbers / shares and being able to rank priorities for the schools. Thus, in most of the cases, the information from participatory cards was used at the public hearing (summarized or full presentation dedicated to it). • The data for the Independent Budget Analysis was collected from schools in the beginning of the second quarter (immediately after the annual reporting to the Rayonal Divisions of Finance), making also easier for schools to report. This allowed or earlier finalization of the analyses and their validation with the school directors. Overall, a better graphic presentation of the data in Independent Education Budget Analyses leaflets increased the interest and understanding by parents with lower levels of education and also students in 7-8 grades. • With sustained efforts of the project implementation team, a synergy between the SAcc activities was established. Thus, the purpose of the public hearing is not the event only, but the establishment of the process of continuous communication and collaboration in the community on the school performance and budgets. Some important achievements were registered in the fifth year of the project as the schools’ understanding of this process has increased. The collaboration of schools with local authorities started already at the beginning of the year, by their participation at the trainings of Local Coalitions (in 14 out of 20 schools local authorities participated in the training). The discussions were continued in many cases during the year and at the public hearing the authorities came more informed and more willing to involve, already knowing the problems and the ways they may help. In public hearings, local public authorities participated in 16 out of 20 beneficiary schools and rayonal authorities were present in 13 out of 20 beneficiary schools. The same is valid for the collaboration with parents, after the implementation of the participatory cards, parents were informed on their use and later the proposals were discussed at the public hearings. This collaboration should continue and the project team encourages schools to report further to all stakeholders on the implementation of the related issues discussed at the public hearing. Page 45 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Component 3: Integrating SAcc with policy and budget dialogue Key objective: Ensuring that results of SAcc tools are effectively used by the Ministry of Education, Culture and Research and local authorities to complement formal Monitoring and Evaluation (M&E) and to inform policy planning and execution Lessons learned • Lack of leadership in the Ministry of Education since May 2017 stopped or delayed several initiatives started earlier in 2017, including some of the issue the project was working on (publishing of the School Administration Board Regulation, finalization of the questionnaires for external evaluation of teachers and general schools). While all project activities went smoothly, the team will put more effort in integrating the abovementioned processes in the policy dialogue in the following period. • While formal cooperation is very important, the team also continued the less formal cooperation within the project through its Advisory Board. In 2017, among the board members were the Deputy Minister of Education (until August 2017), the Head of National School Inspectorate, the Ombudsman for Child Rights, the Head of Finance of Educational sector of the Ministry of Finance and other experts in education. The board meetings are good occasions to have broad discussions regarding the social responsibility process in the educational sector in a less formal setting that allow for brainstorming and continuous improvement in the activities of the project. Also, the Board members can influence the discussions on social responsibility that the project promotes at higher level, which is very important. Expert-Grup was waiting for the appointment of the state secretary for education within the Ministry of Education, Culture and Research (appointed only on December 18, 2017) and will send an invitation to join the project Advisory Board in early January 2018. • Expert-Grup collaborates with the School National Inspectorate to elaborate the institutional questionnaires for parents and pupils for evaluation of educational institutions and teachers. The experience accumulated by the project team was discussed with the representatives of School Inspectorate in order to be used in the process. Also, „My School� team will provide the necessary support for mediatization of these activities in educational institutions. • Expert-Grup was part of the working group working on the concept for remuneration in general education institutions and self-evaluation of school management in 2017 and promoted the introduction of such activities as school budget consultation and parents' involvement in educational process among indicators for the evaluation of the management of the general education institutions. • It is time to redirect the need to redirect efforts in promoting social responsibility from school level to local and district level. In four years of implementation many schools seem to understand the benefits of the SAcc and to appreciate the tools implemented within the project. However, a challenge remains the understanding of the process by public authorities (especially when they change and when there are political factors that influence Page 46 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) these relations). In this respect the executive director of Congress of Local Public Authorities of Moldova was invited to join the project Advisory Board. Expert-Grup is still waiting for a response. • School Administration Boards can take on the role of community mobilizers, as already proved by many examples in the project. Therefore, the project promotes the good examples through case studies intended to foster the creation and activity of School Boards, while he regulation is still pending to be enforced. • There are examples of local and rayonal public authorities which took the example of public hearings in their community to discuss the budget and priorities. Expert-Grup and regional partners are open to offer informational support for the local authorities willing to organize public hearings of the budget. Component 4: Knowledge and Learning Key objective: Ensuring that lessons learned from the implementation of the SAcc mechanisms are taken into account Lessons learned • The project team learned that the implication of the regional partners in the public dialog could be enforced. In 2017 regional partners were invited to participate at several national events organized by Expert-Grup in other projects, where they had the opportunity to meet and establish connections with relevant people for the project. Also, two partner CSOs participated in the ECA Regional Forum on Social Accountability that took place on May 23- 25 in Dushanbe, Tajikistan in May 2017, where they had the opportunity to share their experience in “My School� project and to learn more about social accountability in several sectors, including education. • The team project learnt that it is important to facilitate the communication between the beneficiary schools of the project. A networking event was organized in July after the participatory budgeting workshop, where school directors and deputy-directors had the possibility to share their experience in the project, but also discuss on other relevant issues. Also, a network of schools was created for the schools from three rayons (Ungheni, Nisporeni, Calarasi) managed by our regional partner in Ungheni, given the higher number of beneficiary schools from this rayon, where also the rayonal councils is the most active. Schools within the network organize different joint events. In Ialoveni rayon, the beneficiary schools also presented the project to other schools with the intention to create a network of schools interesting in applying SAcc in educational sector. In 2017, with the support of Expert-Grup and regional partners beneficiary and non-beneficiary school directors participated mutually in events organized by beneficiary schools (trainings of local coalitions and public hearings). • In 2017, Expert-Grup applied more intense and wide communication strategy that included boosted posts on facebook, publication of information (call for application, change stories, Page 47 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) press releases after the events) in a number of printed newspapers, permanent collaboration with the National School Inspectorate for an insertion form the project in its periodical publication. This activities increased the audience of the project. • The project team learned that it needs to disseminate more actively the change stories and make them more attractive. In this respect a professional journalist was hired to write on the most interesting change stories that will be further disseminated both in local and national media and on social networks to reach a higher number of stakeholders. Sustainability of the Project 48. Since the launch of the "My School" project in December 2013, the Independent Expert Group, together with local partners, has been interested in setting up a platform for cooperation and sharing experience in order to promote good practice in implementing social accountability tools. Expert-Grup collaborates with teachers and representatives of training institutions or educational authorities, regional partners, local and central public authorities. One of the multiplication platforms of the project's good practice is the establishment of the "My School" Ambassador Network, where the institutions, after completing the "My School" project, are willing to serve as a local reference point of social accountability. This offers the possibility for other interested parties that want to implement social responsibility tools on their own to benefit from the experience that 30 Ambassadors have gathered these past 5 years. They are located in 15 rayons in all of the three regions of the country, and can be contacted by email or by phone. http://scoalamea.md/vrei-sa-aplici-instrumentele-de-responsabilizare-sociala-in-scoala-ta-si-ai-nevoie- de-ajutor-contacteaza-ambasadorii-scoala-mea/ 49. Although, the Ministry of Education, Culture and Research is still not ready to institutionalize the SAcc tools at the national level, the subject is constantly in the public discourse of this institution, supporting the continuation of the initiative. Other than the reassurances during the National Conference, one of the points of the resolution of the National Forum for Education „A vision for our School�, adopted by the Ministry and more than 270 teachers and headmasters, was the „insurance of parents and pupils participation in the decision-making process in schools… diversification of monitoring and accountability tools at the school level.� https://mecc.gov.md/ro/content/monica-babuc-suntem-interesati-stabilirea-unui-dialog-continuu-si- eficient-dintre-minister 50. To ensure activities sustainability EG shared the SAcc tools and conducted a series of trainings for The Joint Equal Opportunity Initiative Project that seeks to improve the situation of most marginalized and vulnerable men and women in Moldova by mainstreaming equality into policy making by the Government and Parliament. 51. Over these past months, the project coordinator has presented the project “Scoala mea� to the local coalitions at the rayonal levels in Cahul and Căușeni, Soroca in the project of the Eastern European Foundation Moldova, The Common Innitiative for equal opportunities ( http://eef.md/index.php?pag=page&id=1049&l=ro). This is the first project that is going to adapt the SAcc Page 48 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) tools from ``Scoala Mea�to be implemented at the rayonal level in order to have a better representation of the vulnerable members of the society, and inclusive budgets. The project has a budget of 2.9 Mil. Euro. General Conclusions 52. Moldova’s education sector encounters serious challenges. Over the years, the education sector in Moldova has witnessed uneven quality and a lack of efficiency. The country spends large share of GDP for education financing by any regional comparisons. While reforms are in place in the educational sector and an improvement in performance of Moldovan students was registered (according to PISA 2015 results), much still remains to be done. The education is going to be further impacted by the decentralization reform that is currently unfolding in Moldova, increased autonomy of the schools and quality enhancing interventions. The strategic goal of this initiative is to empower Moldova’s citizens to engage local, regional and national authorities in evidence-based policy and budget dialogue regarding educational reform, quality of services, and development priorities of primary, secondary general and upper secondary schools; and to enable an environment in which Social Accountability (SAcc) initiatives thrive and develop. The need for social accountability in the educational sector increases even more in the context of the ongoing education reform and wider autonomy of schools in managing resources granted under the per-student financing mechanism. 53. The project was successfully implemented during the last year of its implementation and most of the Results Framework indicators have been achieved. However, due to lack of leadership in the educational sector for some periods, the Administration Reform in August 2017 with the reorganization of the Ministry of Education by merging with two other ministries into the Ministry of Education, Culture and Research and state secretary for education appointed only on December 18, 2017, the general involvement of the Ministry of Education, Culture and Research in the current year to promote the educational reforms and higher-quality educational policy debate was moderate. Another challenge was the political environment, with 2018 being a pre-electoral year, thus a series of political events coincided with the public hearings dates, and end-of-project events. Component 1: Building an environment enabling sustainable social accountability in Moldova • Established a national coalition between CSOs, minister, local public authorities, schools, parents, pupils, school managers, WB, that engaged in the process of social accountability to improve educational services in Moldova; • Identified and trained 5 CSOs to empower them to become “actors of change� in implementing the Social Accountability Tools in schools and to engage in a constructive dialogue with the decision-makers; • Established coalitions at the school level between school managers, LPAs’ representatives, pupils’ representatives, teachers and parents’ representatives, other interested parties, who implemented the social accountability tools to monitor the quality of educational services; Component 2: Applying social accountability tools to empower local and regional stakeholders to increase policy accountability • Implemented 3 social accountability tools – School Report Card, independent school budget analysis, public hearings; • Organized 100 public hearings where community members discussed school’s problems and created the necessary premises to solve them; Page 49 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) • 1416 school board members received training on the functioning and organization of the school, the social accountability tools, communication with stakeholders; • 29,930 people were directly involved in appreciating the quality of educational services; • 16000 citizens (pupils, parents, teachers, authorities) were involved in school budget discussions; • Contributed to the change in perception of school in the context of the financial autonomy of school institutions through the need for partnerships for effective administration; • Increased the level of accountability of decision-makers at different levels (school management, public and central authorities) and beneficiaries (pupils, parents, community); Component 3: Integrating social accountability with policy and budget dialogue • Independent analysis of school budgets in 100 schools; • Capacity building trainings of various stakeholders on middle-term budgetary framework; • Drafting policy documents, including in the education sector, containing information about the financial management of schools; Component 4: Disseminating knowledge about applying social accountability tools • Dissemination of the success stories with partners; • Dissemination of the lessons learned; • Dissemination of impact and success among school community members, thus increasing the prestige of participation in the project. Basic Project Data TABLE 1. Geographic and Target achieved in reporting period Total project Country’s population targeting target number total number Geographic coverage Number of provinces/regions: 2 districts+ municipalities of Chișinău, 11 31 Bălți Number of districts: 2014 – 7 districts (Rașcani, Sangerei, For 2014 – 5 31 districts Ungheni, Cahul, Floresti, Soroca, districts Hancești) For 2015 – 2 In 2015 – 5 new districts (9 total: Briceni, districts Edinet, Cahul, Leova, Criuleni, Orhei, Falești, Sangerei, Ungheni) For 2016 – 2 In 2016 – 6 new districts (11 in total: districts Donduseni, Soroca, Floresti, Cahul, Anenii For 2017 – 2 Noi, Ialoveni, Straseni, Calarasi, Ungheni, districts Falesti, Rascani) For 2018 – 2 In 2017 – 6 new districts (Causeni, districts Soldanesti, Cantemir, Glodeni, Nisporeni, Drochia) In 2018 – 5 new districts (Cimișlia, Ștefan Vodă, Taraclia, Ocnița și Rezina) Number of sub-district entities In 2014 – 1 municipality (Chisinau) n/a 2 (municipalities/counties/villages/et In 2015 – 2 municipalities (Chisinau, Balți) municipalitie c.): + 1 autonomous territorial unit (UTA s Page 50 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Gagauzia) 1 In 2016 – 1 municipality (Chisinau) autonomous In 2018 – 2 municipalities (Chisinau, Balți) region Number of Public services facilities reached (E.g. local health clinics, hospitals, elementary/ middle/high-schools) In 2014 - http://scoalamea.md/au- In 2018 – 20 schools 100 schools (20 As of 2019, fost-selectate-scolile-beneficiare- In 2014-2018 – 100 schools schools per 1,197 ale-proiectului-scoala-mea/ year) schools In 2015 - http://scoalamea.md/scoli-2015/ In 2016 - http://scoalamea.md/scoli-2016/ In 2017 - http://scoalamea.md/alte- 20-de-institutii-de-invatamant- general-din-tara-vor-beneficia-de- asistenta-in-cadrul-proiectului- multianual-scoala-mea/ In 2018 - http://diez.md/2018/01/19/20-de- licee-si-gimnazii-vor-beneficia-de- asistenta-cadrul-proiectului-scoala- mea-anul-2018-lista-scolilor- selectate/ Number of beneficiaries reached Men: n/a (the project did not plan on n/a n/a measuring beneficiaries based on gender) 2014 Report cards completed by a total of 5419 students, out of which 2401 men. Parents respondents – 962 men 2015 Report cards gender not measured Public hearings – 447 men participants (15,8% of total) Local coalition training – 31 men (11,7% of total) 2016 Report cards filled in by 2511 students, out of which 49.2% men and 1130 parents, out of which 28.5% men Local coalition training – 309 participants, out of which 10% men 2017 Report cards gender not measured Local coalition trainings – 276 participants, out of which, 12% men 2018 Report cards filled in by 2312 students, Page 51 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) 1927 parents, and 415 teachers. Report cards gender not measured Local coalition trainings – 236 participants, out of which 80% women Women: 2014 n/a n/a Report card completed by 5419 students, out of which 3018 women. Parents respondents – 3212 women 2015 Report cards gender not measured Public hearings – 2377 women (84,2% of total) Local coalition training – 233 women (88,3% of total) 2016 Report cards filled in by 2511 students, out of which 50.8% women and 1130 parents, out of which 71.5 men Local coalition training – 309 participants, out of which 90% women 2017 Report cards gender not measured Local coalition trainings – 276 participants, out of which 88% women 2018 Report cards gender not measured Report cards filled in by 2312 students, 1927 parents, and 415 teachers Local coalition trainings – 236 participants, out of which 80% women Children and youth: In 2014 – 12145 students in 20 n/a n/a beneficiary schools In 2015 – 10957 students in 20 beneficiary schools In 2016 – 7860 students in 20 beneficiary schools In 2017 – 8275 students in 20 beneficiary schools In 2018 – 6723 students in 20 beneficiary schools Total: Est. 45,960 students n/a n/a Number of non-implementing beneficiary civil society groups: (E.g. CSOs, community-based orgs., citizens’ associations, unions, PTAs, etc.): Joint Equal Opportunity Initiative, http://eef.md/index.php?pag=page&id=1 3 districts n/a East-European Foundation Moldova 051&l=en Total: 1 3 Page 52 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Table 2. Partners and Key Project Stakeholders Names of Formal Project Partner CSOs Type of partnership arrangement (and other types of formal partners as (contract or implementation applicable) agreement) 1. Centrul Regional CONTACT-Bălți Contract 2. Asociația pentru Cooperare și Comunicare Contract Democratică „DIALOG� 3. Fundația pentru Dezvoltare din Moldova Contract 4. Centrul de Resurse pentru Tineret Contract „DACIA� 5. Centrul Regional de Dezvoltare Durabilă Contract Public Sector Institutions & Counterparts the project has/is directly engaged with through formal and informal constructive/collaborative arrangements during reporting period [List government counterparts, positions, and levels of public sector institutions (e.g. Director of Budget & Planning at Ministry of Finance; District Government Councils; Director of Education Planning at Provincial Min of Ed, Schools Directors; Hospital Directors; etc.)] Corina Fusu - former Minister of Education (until May 2017) Elena Cernei – former Deputy Minister of Education until August 2017 (also member of the Project Advisory Board) Valentina Ichim – Head of E-Transformation and Computerization Department in the Ministry of Education, Culture and Research Valentin Crudu – Head of Secondary Education Department in the Ministry of Education, Culture and Research Ana Vivdici – Head of National School Inspectorate (until may 2018)(also member of the Project Advisory Board) Oxana Șoclea - Head of Department for Education Financing at the Ministry of Finance (also member of the Project Advisory Board) Vitalie Babalau, Main advisor to Monica Babuc, Minister of Education, Culture and Research Angela Cutasevici, State Secretary for Education of the Ministry of Education, Culture and Research, who was appointed on December 18, 2017. Other Key Project Beneficiaries or Stakeholders [the project has engaged with but not as main project counterparts] National and sub-national public sector Heads of General Directorates of Education (but also Heads and institutions (Executive branch): Deputy Heads of rayons) from the following districts: • Anenii Noi • Bălți • Briceni • Cahul • Cantemir • Causeni • Călărași • Cimișlia • Criuleni • Drochia • Edineț • Dondușeni • Comrat Page 53 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) • Falesti • Florești • Glodeni • Ialoveni • Nisporeni • Soroca • Soldanesti • Ungheni • Rascani • Taraclia • Cimișlia • Ștefan Vodă • Taraclia • Ocnița • Soroca • Strășeni • Sângerei • Rezina State Accountability Institutions (e.g. • Maia Banărescu - Parliamentary Advocate for children’s rights parliamentary committees, Ombudsman’s protection (also member of the Project Advisory Board) office, judiciary, anti-corruption • Ion Pruteanu, Head of the Anticorruption Education commission, national human rights Department at the National Anticorruption Centre institutions, national audit commission/auditor general): Media organizations: • Faclia • Ziarul de Garda • Radio Chisinau • Europa Libera • Invatatorul Modern • TVR Moldova Others (e.g. political and social leaders, • Eugenia Veverița – former coordinator of the Moldova policy influencers, opinion makers, etc.): Education Reform Project, currently a public finance management expert (on the Project Advisory Board) • Liliana Nicolaescu-Onofrei - Independent expert. Former Deputy Minister of Education (on the Project Advisory Board) • Victoria Catanoi – consultant in finance for Ministry of Education from World Bank • Victoria Coroban – journalist at Radio Chisinau specialized on educational topics. Also, leader of UpGrade in Education (A community promoting innovation in education in Republic of Moldova) • Vitalie Dogaru - Journalist/TV host from a popular TV station in Moldova, who is covering education (inclusively) (on the Project Advisory Board) • Viorel Furdui (executive director of Congress of Local Authorities of Moldova) Page 54 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) Key outcome tracking: Selected intermediary Indicators Expected Targets for Monitoring data for outcomes reporting period reporting period 1. Outcome 3: Higher-quality Level of satisfaction 0.5 to the baseline +7.64 education policy debates at of the quality of each year from the local, regional and national education services 5.14 established in level in Moldova; 2014, +2.5 from the baseline 2. Comp. 1. Output 5: Regional Rayons where the 13 rayons 29 rayons + Chisinau mechanism for citizens proposals for municipality + Balti oversight created and implementing a municipality + UTA replicated; citizens oversight Gagauzia mechanism are submitted 3. Comp 2. IO 1: Local Schools' budgets 100 schools 100 beneficiary school, and communities effectively use discussed as of February 2019 and 63 the Social Accountability out of 80 past beneficiary tools to participate in schools that are setting budget priorities in continuously using the SAcc their schools; tolls in their activity. 1. Comp. 1. Output 5: Regional mechanism for citizens oversight created and replicated / Rayons where the proposals for implementing a citizens oversight mechanism are submitted (a) While under the initial approach, the project was supposed to be implemented in 5 pilot rayons in the first year of its implementation and adding 2 rayons per year in the following years, due to lack of sufficient applications of schools in the selected rayons in the first year, the project was extended for more rayons. The initial setting was reasoned by the anticipated resistance from rayonal and local authorities and necessary efforts of the regional partner organizations for working with a limited number of rayons. As the SAcc exercise succeeded even with more rayons than planned, the project continued its activities in more rayons in the following years, also taking into consideration the possibility of the regional partners to work simultaneously in more rayons (this also depends on the openness of the rayonal authorities). At the same time, in two most active rayons (Ungheni and Cahul) the project continued to select more schools each year that allows the evaluation of the project impact in rayons where SAcc activities are implemented on larger scale and in consecutive years and, therefore, the pressure on the rayonal authorities might be higher. To be noted, the spillover effects in these two rayons are higher as public hearings of local budgets were organized both in Cahul and Ungheni. Therefore, the experience of these rayons may be shared to other rayons. In 2018, the project was implemented in 13 rayons, of which 5 new rayons. (b) As more rayons are involved in the project and more rayonal authorities have participated in the events organized, they are already aware of the SAcc tools in the educational system and might promote social accountability in their rayon. Thus, the processes covers currently almost entire country. 2. Comp 2. IO 1: Local communities effectively use the Social Accountability tools to participate in setting budget priorities in their schools; / Schools’ budgets discussed (a) All beneficiary schools engaged in public hearings as planned with the support of Scoala Mea team. Additionally a questionnaire was sent to past beneficiary schools (80 schools) in order to identify schools that apply SAcc tools independently after they benefited of support from the team. Thus, 63 out of 80 beneficiary schools organized public hearings in 2019, and as of February 2019 organize reporting cards completion process. This was possible due to higher engagement of Expert-Grup in communication with past beneficiary schools in 2017 - 2018. Before each stage of the project, Expert-Grup sent to all past beneficiary schools of Y1-Y5 information and materials for the activities (usually provided to the beneficiary schools) and also Page 55 of 56 The World Bank Empowered citizens enhancing accountability of Edu reform and quality (P147607) offered to provide support upon request. Some of the schools invited Expert-Grup or regional partners to participate in the events. (b) As social responsibility is a continuous process, only the implementation of separate tools more than once can generate long-term impact. Therefore, based on the questionnaires, those schools that applied several times specific tools witnessed higher impact of the process (on parents, local and rayonal authorities, etc.). Moreover, as more schools engage in the project, there are spillovers on the community level. Thus, in rayons where more beneficiary schools were selected (Cahul and Ungheni), public hearings of local budgets were organized. The spillover effects may be even higher taking into consideration that some directors of non- beneficiary schools participated at School Board trainings and public hearings of the beneficiary schools in the current year to learn from their experience. Page 56 of 56