Report No: AUS0001095 . Moldova Socioemotional Skills Assessment . September 4, 2019 . Education Global Practice . . © 2019 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved This work is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because the World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: “World Bank. 2019. Moldova Socioemotional Skills Assessment. © World Bank.� All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522- 2625; e-mail: pubrights@worldbank.org. Table of Contents ABBREVIATIONS AND ACRONYMS ......................................................................................................................... 1 ACKNOWLEDGMENTS ............................................................................................................................................ 2 EXECUTIVE SUMMARY ........................................................................................................................................... 3 1. INTRODUCTION ............................................................................................................................................ 6 2. EDUCATION AND HUMAN CAPITAL IN MOLDOVA ........................................................................................ 8 3. THE SOCIOEMOTIONAL SKILLS ASSESSMENT IN MOLDOVA ........................................................................ 10 4. HOW SOCIOEMOTIONAL SKILLS RELATE TO STUDENTS’ MATH GRADES AND WELL -BEING ......................... 11 4.1. WHAT ARE SOCIOEMOTIONAL SKILLS? .....................................................................................................................11 4.2. DO SOCIOEMOTIONAL SKILLS PREDICT MATH GRADES AND WELL-BEING? .....................................................................13 5. THE PATTERNS OF SOCIOEMOTIONAL SKILLS IN MOLDOVA ....................................................................... 15 6. THE ROLE OF SCHOOLS AND FAMILIES IN FOSTERING SOCIOEMOTIONAL SKILLS ........................................ 18 6.1 THE ROLE OF SCHOOL AND FAMILIES.........................................................................................................................18 6.2. SCHOOL CHARACTERISTICS ASSOCIATED WITH CHILDREN’S SOCIOEMOTIONAL SKILLS .......................................................20 7. IMPLICATIONS FOR POLICIES AND PRACTICES ............................................................................................ 25 REFERENCES......................................................................................................................................................... 29 ANNEX 1: SAMPLING FOR THE SOCIOEMOTIONAL SKILLS ASSESSMENT .............................................................. 31 ANNEX 2: BACKGROUND QUESTIONNAIRE OF THE SOCIOEMOTIONAL SKILLS ASSESSMENT ............................... 32 ANNEX 3: DESCRIPTIVE STATISTICS FROM THE MOLDOVA SOCIOEMOTIONAL SKILLS ASSESSMENT .................... 61 Figures FIGURE 1: SOCIOEMOTIONAL SKILLS FRAMEWORK ....................................................................................................11 FIGURE 2: THE COMPOSITE MEASURE OF SOCIOEMOTIONAL SKILLS .........................................................................13 FIGURE 3: DISTRIBUTION OF SOCIOEMOTIONAL SKILLS .............................................................................................15 FIGURE 4: DISTRIBUTION OF SOCIOEMOTIONAL SKILLS BY GENDER ..........................................................................16 FIGURE 5: LEVELS AND DISTRIBUTION OF SOCIOEMOTIONAL SKILLS BY REGIONS .....................................................17 FIGURE 6: PROPORTION OF VARIANCE IN SOCIOEMOTIONAL SKILLS EXPLAINED BY SCHOOL AND FAMILY FACTORS ...........................................................................................................................................................................19 Tables TABLE 1: THE RELATIONSHIP BETWEEN SOCIOEMOTIONAL SKILLS AND MATH GRADES ...........................................13 TABLE 2: RELATIONSHIP BETWEEN SOCIOEMOTIONAL SKILLS AND WELL-BEING ......................................................14 TABLE 3: MEASURES OF SCHOOL AND FAMILY LEARNING CONTEXTS FROM THE SOCIOEMOTIONAL SKILLS ASSESSMENT ......................................................................................................................................................20 TABLE 4: THE RELATIONSHIP BETWEEN LEARNING CONTEXTS AND SKILLS ................................................................22 TABLE 5: RELATIONSHIP BETWEEN TEACHERS’ QUALITY AND STUDENTS’ SOCIOEMOTIONAL SKILLS .......................24 TABLE 6: SUMMARY OF RESULTS AND POLICY CONCLUSIONS....................................................................................25 Abbreviations and Acronyms ASA Advisory Services and Analytics BFI Big Five Inventory CASEL Collaborative for Academic, Social, and Emotional Learning HCI Human Capital Index MSMS Moldova Skills Measurement Survey MoECR Ministry of Education, Culture and Research MoHLSP Ministry of Health, Labor and Social Protection OECD Organisation for Economic Co-operation and Development PISA Programme for International Student Assessment SEL Social and Emotional Learning STEP Skills Towards Employability and Productivity Program VET Vocational Education and Training 1 Acknowledgments This report was prepared under the Advisory Services and Analytics titled “Moldova Socioemotional Skills Assessment� (P169987). It was prepared by Koji Miyamoto (Senior Economist), Miguel Sarzosa (Consultant), and Oliver John (Consultant). The analysis was based on the World Bank’s Socioemotional Skills Assessment in Moldova administered in April 2019. This assessment was coordinated by Koji Miyamoto and Syd Dinlemez (Consultant) and administered by CIVIS led by Ruslan Sintov (Director, CIVIS) with local coordination and support by Lucia Casap (Education Specialist). The data analysis was performed by Miguel Sarzosa and Oliver John, both of whom contributed to preparing the socioemotional assessment items, background questionnaire, and sampling design with support from Syd Dinlemez. The authors acknowledge the guidance and directions from Janssen Teixeira (Task Team Leader and Senior Education Specialist), draft inputs from Lucia Casap, translations by Diana Sestacovschi (Consultant), and valuable comments from peer reviewers: Julia Liberman (Operational Analyst), Manal Quota (Education Specialist), and Alexandra Valerio (Lead Education Specialist), as well as Harry Patrinos (Practice Manager). The World Bank team is grateful to the Ministry of Education, Culture and Research (MoECR) for supporting and facilitating the Socioemotional Skills Assessment, including Prof. Virginia Rusnac (Director of Republican Center for Psychopedagogical Assistance, MoECR) who provided guidance on the survey design including measurement instruments as well as comments on the draft report. The team would like to express gratitude to the 80 schools across Moldova, and their principals, teachers, and students, for participating in the assessment and providing valuable data, which drove the substance of this report. This activity was part of the two outputs of the World Bank activity on Socioemotional Skills Assessment led by Janssen Teixeira. 2 Executive Summary Moldovan employers voice concerns that workers’ low levels of socioemotional capabilities, such as poor work ethics, lack of motivation, and limited problem-solving skills, are among the major constraints for business development and productivity. The World Bank’s 2013 Business Environment and Enterprise Performance Survey, the 2016 Labor Market Forecast of Moldova, and a qualitative interview among private sector representatives in 2017 suggest that employers face skills shortages, particularly in areas of work ethics, motivation, and problem solving. The magnitude of skills shortages in Moldova is considered to be the highest in the Europe and Central Asia region. The Moldovan labor market would benefit from an increased supply of employees, including new recruits, with a range of relevant socioemotional as well as cognitive capabilities. While the Moldovan education system has made efforts to foster and measure academic skills, limited attention has been devoted to students’ socioemotional development. Moldova has managed to improve educational attainment, raise learning outcomes, and introduce metrics of internationally recognized cognitive abilities during the last few years. However, limited progress has been made to raise and measure socioemotional skills. On the positive side, the Moldovan government has very recently launched new curricula on ‘counselling and personal development’ and ‘education for society’, which include some elements of social and emotional learning (SEL) with the aim to enhance students’ civic engagement, entrepreneurship, and cultural awareness. Moldova may also consider introducing measures of socioemotional skills that can be used to improve policies and learning practices. This report provides a first diagnostic of socioemotional skills in Moldova to profile the skills composition of grade 9 students, identify learning context measures that are strongly associated with socioemotional skills, and clarify options for policy makers and practitioners to foster socioemotional skills. To this end, the Moldovan Socioemotional Skills Assessment was launched in April 2019. This assessment focused on grade 9 students to better understand how much socioemotional skills students exiting the end of the compulsory school cycle self-reported, and how these skills were associated with learning inputs from schools and families. This assessment was the first to evaluate a range of socioemotional skills among school-age children in Moldova, using measures that had been validated internationally. The measures were designed to capture five broad domains of socioemotional skills, including students’ self-reported capacity to ‘work with others’, ‘care for others’, ‘engage with others’, ‘explore new horizons’, and ‘manage emotions’. The assessment provided not only technical validation of the measurement tools in Moldova but also initial guidance for policies and practices to foster these skills. The results suggest that such measurement tools can be mobilized in Moldova in the future. The analysis of data from the Moldovan Socioemotional Skills Assessment suggests that various measures of socioemotional skills are strongly associated with math grades and students’ well-being. The measure of socioemotional skills that demonstrate particularly high predictive validity for math grades is the composite socioemotional skills, which takes into account all five domains of socioemotional skills. Domain-specific socioemotional skills such as students’ capacity to ‘work with others’, ‘engage with others’, and ‘explore new horizons’ also show high correlations with math grades. The composite socioemotional skills also show high predictive validity for various measures of well-being which include happiness, sense of belonging, (not) being teased, (not) being bullied, and (not) being victimized. These results further signify the importance of fostering socioemotional skills in Moldova, beyond the concerns 3 raised by Moldovan employers related to socioemotional skills shortages that may affect labor productivity. The Moldovan Socioemotional Skills Assessment points to gender differences in the self-reported measures of socioemotional skills across the skills distribution. Consistent with evidence from other countries, girls tend to self-report higher levels of socioemotional skills across most of the skill dimensions including their capacity to ‘engage with others’, ‘care for others’, ‘work toward goals’, and ‘explore new horizons’. The only exception is the girl’s capacity to ‘manage emotions’, which they tend to self-report at a lower level compared to boys. The skills assessment also suggests that school and family contexts are associated with students’ levels of socioemotional skills. School and family factors explain a large share of the variations (that is, up to 20 percent) in the self-reported socioemotional skills of grade 9 students in Moldova. Among the school-level factors, being in a classroom with fewer disruptions and teachers of higher quality (that is, longer teaching experience and better pedagogical practices) as well as belonging to a school with a lower proportion of part-time teachers are particularly associated with higher levels of students’ socioemotional skills. Among the family-level factors, receiving parental support (that is, having caring and supportive parents), not being in a hostile family environment (that is, a home where there is no physical and verbal abuse against children and between parents), and being in a family with more learning assets (for example, a quieter room and desk to study, as well as a home with books and computers) are associated with students’ levels of socioemotional skills. Policy makers and educators may benefit from better understanding how fostering students’ socioemotional skills can help improve their learning outcomes, labor market success, and well-being. The evidence presented in this report, albeit based on correlational analysis using self-reported measures of socioemotional skills, suggests that empowering children’s socioemotional capabilities can be an important policy option for decision makers to help address the skills shortages in the labor market and for teachers and school administrators to improve students’ learning outcomes. There is emerging international evidence that points to successful large-scale interventions (including cost-effective ones) that schools and parents could explore introducing. There are also experiences from school systems (for example, the State of Illinois in the United States and the Unified School Districts in California) that have successfully integrated SEL in education standards, as well as introduced formative assessments that can be used for monitoring by central administrators and for adjusting pedagogical practices by teachers. Moreover, there are ways to usefully integrate the principles of socioemotional learning in existing curricula and school practices without introducing major reforms. The evidence suggests that SEL can be best designed and implemented (a) in a holistic manner, involving teachers, parents, and students; (b) in an inclusive way, making sure that those who would best benefit from SEL will receive the interventions; and (c) by starting earlier, taking into account that children’s socioemotional capabilities are particularly malleable from early childhood through adolescence. Policy makers and educators may first consider introducing regular socioemotional skills measurements at the system and school levels, which can in turn provide valuable information for them to explore integrating SEL in policies and practices. The first proposed step for policy makers is to initiate a regular socioemotional assessment at the system level using validated instruments. This can be the basis for encouraging schools to also integrate socioemotional skills measurements in a formative fashion to allow teachers to monitor the progress of children’s socioemotional development. This new information can be 4 used together with other vital data on children’s academic skills and well-being, to allow policy makers and educators to adapt policies and teaching practices in line with children’s learning needs and the skill needs of the labor market. 5 1. Introduction Skills shortages represent a major challenge among Moldovan employers. The World Bank’s 2013 Business Environment and Enterprise Performance Survey suggests that employers find inadequate workforce skills as the third most important constraint to expanding business in Moldova, while the other two reasons are corruption and political instability (Rutkowski, Levin, and Bargu 2017). Approximately 40 percent of firms in Moldova cited skills deficits as a major or severe constraint to growth, reflecting one of the highest percentages in the Europe and Central Asia region. A more recent interview among private sector representatives in Moldova suggests that employers are dissatisfied with the quantity and quality of workforce skills, especially among ‘blue-collar’ jobs (Tetyora, Lee, and Velazquez 2017). Skills shortages usually force firms to downsize and reconsider their development plans. Moldovan employers identify socioemotional skills as one of the key areas of skills shortages. The 2016 Labor Given the socioemotional skills are Market Forecast Survey of Moldova reports that poor considered among the most important work ethics and lack of motivation among the workforce areas of skills shortages in Moldova, the are two of the three major constraints for firms’ education system may consider devoting performance (Rutkowski, Levin, and Bargu 2017). They more attention to these skills. The also highlight insufficient analytical and problem-solving analysis presented in this report builds skills, as well as low willingness to learn new issues as on the Socioemotional skills Assessment driving the skills gaps for middle-skilled workers. This is in Moldova in providing insights for likely to reflect modern workplaces increasingly policy makers and practitioners to involving nonroutine analytical and interpersonal tasks, explore options to cater to the skill an evidence found in the United States (Autor, Levy, and needs. Murnane 2003; Deming 2018). These findings suggest the importance of socioemotional development of children and youth in the context of Moldova. While the Moldovan education system has made efforts to foster and measure academic skills, limited attention has been devoted to socioemotional skills. Education policies and practices across the world have traditionally focused on academic skills and monitored student’s progress toward the standards set by ministries of education or local education authorities (OECD 2015). Not surprisingly, most of the existing metrics to evaluate students’ success are based on academic or cognitive skills. Given that employers are concerned about the insufficient supply of workers with socioemotional skills, and the fact that these skills are core foundations to foster academic skills, health, and well-being (OECD 2015), the Moldovan education system may benefit from exploring ways to better prepare students with a particular attention to socioemotional skills. The purpose of this report is to identify (a) which socioemotional skills matter for grade 9 students’ learning outcome and well-being in Moldova, (b) which students tend to self-report lower levels of these skills, (c) what are the learning contexts that matter, and (d) the implications for policies. This report mobilizes the Socioemotional Skills Assessment in Moldova to address these questions. The assessment measures socioemotional skills among grade 9 students across the country and the underlying school and family contexts in Moldova (see section 3 and the annexes 1 and 2 for more details of this assessment). Early adolescence is a malleable period of children’s socioemotional development. The analysis helps understand whether children finishing the end of the compulsory education cycle have 6 sufficient socioemotional capability to continue learning and flourish in the society. The analysis presented in this report is expected to provide insights for policies and practices that would help decision makers, educators, and parents to explore options to better cater to the skills shortages. 7 2. Education and Human Capital in Moldova A child born in Moldova today is expected to achieve only 58 percent of his or her human capital potential by age 18, reflecting the low quality of education. The Human Capital Index (HCI)1 provides an aggregated measure of the expected levels of child survival, school performance, and health of the next generation in the country where that child lives. The HCI of 0.58 in Moldova represents the 75th rank among 157 countries around the world.2 Within the Europe and Central Asia region, Moldova is ranked below the average HCI level of 0.7 in the region. This low overall performance mirrors the low expected school performance of children, driven by the below-average harmonized test scores—at 436 points, compared to the Europe and Central Asia average of 495.3 While Moldova has recently improved education quality, there is still a large proportion of students below basic skills proficiency levels, especially in the rural areas. Between 2009 and 2015, Moldova has successfully improved its Programme for International Student Assessment (PISA) reading, math, and science proficiencies, at a level equivalent to almost one added year of schooling. However, Moldova’s proficiency levels are still considerably below those in the Organisation for Economic Co-operation and Development (OECD) countries, with reading, math, and science scores at 416, 420, and 428 points, respectively, compared to the OECD average scores of 493, 490, and 493 points, respectively, in 2015. The low average scores are driven by a large proportion of Moldovan students performing below basic proficiency levels in PISA reading, mathematics, and science, at 46, 50, and 42 percent, respectively, in 2015. These low performers are concentrated among children from low-income households in rural areas who are about three years behind in schooling compared to high-income households. While similar data do not exist for socioemotional skills, the employers’ perceptions of employees’ socioemotional skills shortages suggest there is likely to be a significant proportion of students lacking socioemotional skills (Tetyora, Lee, and Velazquez 2017). The Moldovan government has recently adopted a new curriculum designed to foster key competencies for While Moldova has recently launched lifelong learning. In December 2018, the Moldovan a new curriculum, which includes Parliament approved the National Development Strategy some elements of social and emotional ‘Moldova 2030’, 4 highlighting human capital as a core learning, the education system does driver for national development. Education has also not yet measure and monitor the become the main focus of the government’s Midterm progress of students’ socioemotional Sectorial Policy Priorities (2019–21). Moreover, in development. December 2018, the government launched a new curriculum based on the European Council’s 1 http://www.worldbank.org/en/publication/human-capital. 2 https://databank.worldbank.org/data/download/hci/HCI_2pager_MDA.pdf and https://www.worldbank.org/en/events/2019/04/17/moldova-skills-for-jobs-conference#3 (see presentation by Harry Patrinos titled: “Human Capital Project and Index�). 3 https://databank.worldbank.org/data/download/hci/HCI_2pager_MDA.pdf and https://www.worldbank.org/en/events/2019/04/17/moldova-skills-for-jobs-conference#3 (see presentation by Harry Patrinos titled: “Human Capital Project and Index�). 4 https://cancelaria.gov.md/en/content/national-development-strategy-moldova-2030-parliament. 8 Recommendation on Key Competences for Lifelong Learning.5 This implies a major shift in teaching and learning, which involves a transition from acquiring subject-specific knowledge to life skills with a particular focus on acquiring literacy, foreign languages, science, technology, engineering, mathematics, and digital skills.6 The new approach also highlights socioemotional skills associated with learning to learn, civic engagement, entrepreneurship, and cultural awareness. The new curriculum includes new subjects that are likely to affect students’ social and emotional development and well-being. The new curricular framework introduces new subject areas which may help improve students’ social and emotional skills, such as ‘Counseling and personal development’ and ‘Education for society’. Moreover, one of the priorities of the new curriculum and its framework is to introduce the concept of pupils’ nonviolence and well-being. This reflects the fact that violence, including bullying, has become prevalent in Moldovan schools, where about 75 percent of children have been subjected to some form of physical or psychological violence in their daily lives.7 The Moldovan education system does not measure and monitor students’ SEL. Moldova has a history of administering national examinations and skills assessments designed to measure students’ academic performance. This includes participation in PISA 2009 and 2015, which provide internationally comparable metrics of reading, math, and science. Therefore, policy makers have a wealth of information to design policies and programs to foster students’ academic and cognitive skills. Teachers and parents also receive regular information to help design and adapt curricular activities according to children’s performance. In contrast, the Moldovan education system does not systematically collect information on socioemotional skills. This implies that policy makers have limited information on the socioemotional capacity of children, which has a far-reaching impact on their academic, labor market, and social outcomes. While teachers may use their own perceptions to form a judgment about students’ socioemotional capability pertaining to academic development, they still lack conceptual understanding of the range of socioemotional skills that drive children’s lifetime success, and how to measure and foster these skills. Investments in socioemotional skills outside of the formal education system are limited. Results from the Moldova Skills Measurement Survey (MSMS), which measures cognitive and socioemotional skills as well as investments in training of the working-age population, suggest very low take-up of training outside of the formal education system (World Bank 2019). This suggests Moldova faces the twin challenges of fostering socioemotional skills for both the school-age and working-age populations. To the extent that the demand for adult education and training partly depends on the outputs of initial education, there is an urgent need to enhance the quality and efficiency of initial education to address the cognitive and socioemotional skill needs. 5 https://ec.europa.eu/education/education-in-the-eu/council-recommendation-on-key-competences-for-lifelong- learning_en. 6 https://mecc.gov.md/sites/default/files/plan-cadru_2018-19_rom_rus_final.pdf. 7 https://www.unicef.org/moldova/ce-facem/protec%C5%A3ia-copilului/combaterea-violen%C5%A3ei- %C3%AEmpotriva-copiilor. 9 3. The Socioemotional Skills Assessment in Moldova Recognizing the need to address skills shortages, the Government of Moldova and the World Bank administered the Socioemotional Skills Assessment in Moldova. The Moldovan Ministry of Education, Culture and Research (MoECR) requested the World Bank to launch an Advisory Services and Analytics (ASA), which included the Socioemotional Skills Assessment and the Moldova Skills for Jobs Conference, both of which took place in 2019. These events highlight significant interest from the Moldovan government in mobilizing education policies to improve socioemotional skills of the workforce. The Socioemotional Skills Assessment in Moldova was delivered in 2019 to over 2,000 grade 9 students across The Socioemotional Skills Assessment in 80 randomly selected schools in Moldova. The data Moldova randomly sampled 80 schools collection took place between April 1 and 26, 2019, covering over 2,000 grade 9 students. covering 2,050 grade 9 students across 80 randomly selected schools in Moldova. The average number of assessed students per class was 16.7, and the minimum number of participating students per class was 4. The assessment included both the school background questionnaire for school directors and the student questionnaire. The assessments for school directors and grade 9 students were delivered using tablets (see annex 2 for the full assessment items). The school director assessment for collecting information on school quality was conducted in the form of enumerator interviews. By contrast, the student assessment for collecting information on students’ socioemotional skills, well-being, and background characteristics was self-administered. An algorithm was used to select 20 students in classes of more than 20 students. In classes of less than 20 students, all students took the assessment.8 The school background questionnaire included diverse measures of school characteristics. The school director assessment included school background measures such as enrollment, teachers’ experience, parental school engagement, extracurricular activities, and classroom climate. Items on classroom climate were adapted from the World Bank’s classroom observation tool Teach for student reporting. At the end of the school director’s interview, the enumerator asked for the last semester’s math grades for grade 9 students (that is, the only available administrative information on students’ math capability) who took the assessment. These math grades were then matched to the student respondents. The student questionnaire included a range of individual background characteristics and self-reported Socioemotional Skills Assessment. The student questionnaire contained background measures such as age, parental education, and parental occupation. Moreover, the questionnaire included information on the learning environment at home and school, students’ perception of math and Romanian language classes and teachers, and students’ perception of their parents’ engagement in school. The items for the student assessment were adapted to the local context from OECD’s PISA 2015 and the World Bank’s classroom observation tool Teach. These items were followed by socioemotional skills items, which were based on the Big Five Inventory (BFI-2) (Soto and John 2017). 8 In schools with more than one grade 9 class, the assessment was administered in two classes. The average duration of the assessment for students was 42 minutes. The shortest assessment was 8 minutes and the longest assessment was 93 minutes. For school directors, the average duration of the assessment was 24 minutes. 10 4. How Socioemotional Skills Relate to Students’ Math Grades and Well-being 4.1. What Are Socioemotional Skills? Socioemotional skills, also known as noncognitive, soft, or character skills, are foundational human capabilities involved in achieving life goals, interacting with others, and managing emotions. As such, they manifest themselves in countless everyday life situations. While a variety of conceptual frameworks exist to characterize socioemotional skills, many of them either directly or indirectly cover the following five domains: (a) working toward goals, (b) exploring new horizons, (c) managing emotions, (d) engaging with others, and (e) caring for others (Figure 1). These five domains mirror the well-known Big Five socioemotional skills factors, which have been extensively studied in the psychological literature (John and Srivastava 1999).9 The Big Five factors are associated with a large pool of evidence across diverse countries and cultures and are known to represent distinct individual socioemotional capabilities as well as their development and outcomes (Kautz et al. 2014; OECD 2015). Figure 1: Socioemotional Skills Framework Working Exploring Managing Engaging Caring towards goals new horizons emotions with others for others Intellectual Resillience Organization Social initiative Compassion curiosity under stress Aesthetic Respect for Productiveness appreciation Self-confidence Assertiveness others Creative Frustration Responsibility Energy Trust imagination tolerance Source: Adapted based on John and De Fruyt 2015. Multiple socioemotional skills exist within each of the five broader socioemotional skills domains. Figure 1 shows that each of the five Socioemotional skills contain socioemotional skills domains hosts a family of related but distinct a broad set of human socioemotional capabilities. For example, social initiative, capabilities that drive assertiveness, and energy belong to the family of skills that facilitate children’s lifetime success. engaging with others. Yet, each does so in a different way: social initiative involves learning how to make connections with new people; assertiveness involves students 9 There are no universally agreed concepts and measures to describe socioemotional skills. The Big Five factor model is one of the most regularly used measurement approaches found in the literature. 11 learning how to ask for what they need or stand up for their own rights and the rights of others; and energy involves learning to become enthusiastic and excited about activities at school. Each of these skills can be taught, practiced, and developed in the school context. The Big Five Framework, therefore, represents a wide encompassing and complex representations of human capabilities that are believed to drive individuals’ lifetime success. A balanced set of cognitive and socioemotional skills empowers children to flourish in education, society, and the labor market. Cognitive and academic skills have traditionally been emphasized in education policies and school practices. However, evidence from intervention and longitudinal studies suggests that socioemotional skills such as perseverance, self-confidence, and compassion are as important as cognitive skills in explaining educational attainment, employability, health conditions, civic engagement, and well-being (Kautz et al. 2014; OECD 2015). Moreover, studies suggest that socioemotional skills can even help foster cognitive development (Cunha, Heckman, and Schennach 2010; OECD 2015). This means that investments in socioemotional skills can contribute to efforts to raise academic achievement. Results from the Socioemotional Skills Assessment in Moldova suggest that the Big Five Framework (Figure 1) holds among the representative sample of grade 9 students. The Socioemotional Skills Assessment employed the BFI-2, the latest version of the most widely used self-reported measures of socioemotional skills in the world (Soto and John 2017).10 The assessment introduced multiple measures at the lower-order facet levels (for example, organization, productiveness, and responsibility for the ‘Working towards goals’ domain) as shown in Figure 1.11 The data suggest that the items used within each of the five factors show high levels of correlations among each other (that is, internal validity)12 while relating less to other factors (that is, discriminant validity). Confirmatory factor analysis suggests that the Big Five Framework holds for Moldova. Figure 2 suggests that there is an overarching socioemotional factor (that is, the principal component which will henceforth be called Composite) that captures a large proportion of the common variations across all the five factors. While the psychometric performance of the self-reported BFI-2 in Moldova was reassuring, the results presented in this report should be 10 The 60 item BFI-2 has already been tested and validated across a number of countries (Soto and John 2017). Note that BFI-2 is an enhanced version of the BFI-1 which was the basis for the socioemotional skills measures used in the World Bank’s Skills Towards Employability and Productivity Program (STEP) Household Survey. Adaptation of BFI-2 to Romanian language involved two professional translators as well as a local psychologist (Prof. Virginia Rusnac, Director of Republican Center for Psychopedagogical Assistance, MoECR) who participated in the preparation and finalization of the assessment items and background questionnaires. The Socioemotional Skills Assessment conducted a pilot data collection in two schools (one rural and one urban) before the main assessment and adjusted both the items and survey administration protocol after evaluating the pilot data and the data collection process. The main assessment employed three formats of BFI-2 (that is, five-point Likert scale, forced choice, and bipolar) as well as anchoring vignettes which were separately prepared for boys and girls. These self-reported items were divided across three groups, and each group within the class received a different combination of socioemotional items. While self-reported measures of socioemotional skills tend to suffer from random and nonrandom errors (Laajaj et al. 2019), the BFI-2 has been designed specifically to minimize some of these biases that arise from acquiescence and mistranslations from the international items. Nevertheless, readers are warned to be cautious about interpreting the results. 11 Note that the measures of BFI-2 are generally considered to work better for individuals above a certain age, among those who would be able to understand and self-report on the socioemotional descriptions including adjectives. Grade 9 (that is, age 15) is considered an appropriate age to deliver the BFI-2. 12 The indicators of internal validity (Cronbach’s Alpha) for all the five factors were higher than 0.7. 12 interpreted with caution given the known limitations of self-reported measures (Duckworth and Yeager 2015). Figure 2: The Composite Measure of Socioemotional Skills 100% 80% 60% 40% 20% 0% Working Caring Engaging Exploring Managing towards goals for others with others new horizons emotions Variance explained by the compositive measure of socio-emotional skills Residual Source: Socioemotional Skills Assessment in Moldova (World Bank 2019) Note: The figures are based on standardized data on socioemotional skills (mean of 0 and standardized deviation of 1) after adjusting for acquiescence bias following Soto and John (2008). The results come from a standard analysis of principal components. 4.2. Do Socioemotional Skills Predict Math Grades and Well-being? Socioemotional skills, such as children’s capacity to ‘engage with others’, ‘work towards goals’, and ‘explore horizons’ are key Socioemotional skills are correlates of math grades in Moldova. Evidence and common sense key correlates of math suggest that children’s socioemotional skills such as social initiative, grades in Moldova. responsibility, and intellectual curiosity are likely to play a central role in raising children’s capacity to learn at school and improve school grades (Kautz et al. 2014; OECD 2015). The Socioemotional Skills Assessment in Moldova suggests that grade 9 students’ socioemotional skills are important drivers of math grades, which were extracted from the official school records (Table 1). A one standard deviation increase in the composite measure of socioemotional skills is associated with 27 percent of a standard deviation increase in math grades, even after controlling for other factors that drive grades. Moreover, among the five dimensions of socioemotional skills described in Figure 1, ‘working towards goals’, ‘exploring new horizons’, and ‘engaging with others’ are shown to have a particularly strong association with math grades. This result is consistent with the global evidence base (Soto and John 2017). Table 1: The Relationship between Socioemotional Skills and Math Grades Math Grades Composite 0.268***     Engaging     0.134*** Caring     −0.025 Working     0.119*** Managing     0.007 Exploring     0.122*** Source: Socioemotional Skills Assessment in Moldova (World Bank 2019). *** represents statistically significant coefficient estimate at the 0.01 level. All other results are not statistically significant at the 0.1 level. 13 Note: The analysis is based on standardized data on socioemotional skills after adjusting for acquiescence. The results shown are based on standard regression models net of school-fixed effects and including control of gender and age. Socioemotional skills are key correlates of well-being. The Socioemotional Skills Assessment includes a number of indicators Socioemotional skills are designed to capture well-being in schools. They include measures of also important correlates (a) feeling happy with school activities, (b) school belonging, (c) being of well-being in Moldova teased, (d) being bullied, and (e) been victimized.13 Table 2 shows that the composite measure of socioemotional skills correlates with all the five measures of school-based well- being, and in particular with happiness and school belonging.14 Moreover, students’ capacity to ‘care for others’ and ‘work towards goals’ shows the strongest and positive association with happiness. Students’ capacity to ‘engage with others’ and ‘manage emotions’ shows the strongest and positive association with a sense of belonging. Last, students’ socioemotional capability to explore new horizons is strongly associated with happiness. Table 2: Relationship between Socioemotional Skills and Well-being Happy Belonging Teased Bullied Victimized Composite 0.324*** 0.425*** −0.050** −0.042** −0.082*** Engaging 0.040   0.297*** −0.011    −0.008    0.002    Caring 0.100*** 0.093***  −0.010    0.007    −0.034**  Working 0.172*** 0.073**  −0.005    0.002    −0.021    Managing 0.007    0.130*** −0.087*** −0.060*** −0.093*** Exploring 0.100*** −0.043*  0.045*** 0.020**  0.042*** Source: Socioemotional Skills Assessment in Moldova (World Bank 2019). Note: The analysis is based on standardized data on socioemotional skills (mean of 0 and standardized deviation of 1) after adjusting for acquiescence. The results shown are based on standard regression models net of school-fixed effects and include controls for gender and age. ***, **, and * represent statistically significant coefficient estimates at the 0.01, 0.05 and 0.1 levels, respectively. All other results are not statistically significant at the 0.1 level. Socioemotional and cognitive skills can be important drivers of labor market outcomes. While the Moldovan Socioemotional Skills Assessment does not allow for the analysis of the relationship between socioemotional skills and labor market outcomes, international evidence suggests that socioemotional as well as cognitive skills are key drivers of employment and income (Kautz et al. 2014; OECD 2015; World Bank 2018). There is also evidence suggesting that life satisfaction and positive affect influence later income (De Neve and Oswald, 2012). 13 This includes been threatened, robbed, or severely beaten. 14 Note that the causal effects can go in both directions. While students’ capacity to ‘work towards goals’ and ‘explore new horizons’ can lead them to happiness, it may well be that happy children are more likely to be able to ‘work towards goals’ and ‘explore new horizons’. 14 5. The Patterns of Socioemotional Skills in Moldova A higher proportion of Moldovan students tend to self-report lower levels of capacity to ‘manage emotions’ in comparison to A relatively large proportion of other socioemotional skills. Figure 3 presents distributions of Moldovan students self-report socioemotional skills in Moldova. 15 They suggest that the difficulties in managing emotions. distributions are generally skewed toward higher response options. The exception is the distribution of students’ capacity to ‘manage emotions’, which has a relatively higher share of students that are concentrated in the lower tail and lower share of students in the upper tail. Figure 3: Distribution of Socioemotional Skills Source: Socioemotional Skills Assessment in Moldova (World Bank 2019). Note: The figures are based on standardized data on socioemotional skills after adjusting for acquiescence bias following Soto and John (2008). Girls, in comparison to boys, are more likely to self-report higher levels of socioemotional skills, particularly their Girls are generally more likely to self- capacity to ‘engage with others’ and ‘care for others’. The report higher levels of socioemotional international literature on socioemotional skills for skills than boys, with the exception of children and adolescents consistently points to the gender their capacity to ‘manage emotions’ for differences in socioemotional skills whereby girls perform which boys tend to self-report higher better than boys. Figure 4 suggests that in Moldova, there levels. is also a considerable gender difference in self-reported socioemotional skills among grade 9 students. The composite indicator of socioemotional skills shows that 15 Note that the scores have been adjusted for acquiescence bias and standardized to have mean of 0 and a standard deviation of 1. The acquiescence bias happens when respondents tend to systematically self-report positive answers. 15 girls are generally much more likely to self-report higher levels of socioemotional skills across most of the skills distribution. When looking at the specific socioemotional factors, Figure 4 suggests that for those associated with ’engaging with others’, ‘caring for others’, ‘working with others’, and ‘exploring new horizons’, girls self-report higher skills levels than boys. Note that the gender difference only starts to emerge among those who are above the first quartile of the socioemotional skills distribution. The only exception to the girls’ advantage is for the ‘managing emotions’ factor for which the boys tend to self- report higher levels. Figure 4: Distribution of Socioemotional Skills by Gender Source: Socioemotional Skills Assessment in Moldova (World Bank 2019). Note: The figures are based on standardized data on socioemotional skills after adjusting for acquiescence bias following Soto and John (2008). Students from the Central region tend to self-report higher levels of socioemotional skills than those from other regions, while students from Chisinau tend to rate higher levels of ‘exploring new horizons’ than those from other regions. Figure 5 (panels A and B) presents differences in the levels and distributions of socioemotional skills across the four regions of Moldova, including the North, the Central, the South, and the capital region of Chisinau. Panel A suggests that students from the Central region tend to self-report higher levels of socioemotional skills compared to other regions. While students in Chisinau generally self-report lower levels of socioemotional skills compared to other regions, their self-ratings on exploring horizons are higher than those from other regions. Students in the Southern region tend to self- rate average levels of socioemotional skills compared to other regions, except for their self-rating on exploring horizons which is much lower than ratings made by students from other regions. Panel B suggests that cross-regional differences in the levels of socioemotional skills generally emerge after the lowest quintile of the distribution, meaning that much of the cross-regional variations are driven by those at the higher socioemotional skills levels. While the curricula should be similar across regions, there may be cross-regional variations in the quality of teachers, facilities, and parental background that may have implications on the distributions of socioemotional skills. 16 Figure 5: Levels and Distribution of Socioemotional Skills by Regions Panel A: Average Levels of Socioemotional Skills by Region 0.1 0.05 0 Composite Engaging with Caring for others Working with Managing Exploring horizons others others Emotions -0.05 -0.1 -0.15 -0.2 North Center South Chisinau Panel B: Distributions of Socioemotional Skills by Region Source: Socioemotional Skills Assessment in Moldova (World Bank 2019). Note: The figures are based on standardized data on socioemotional skills after adjusting for acquiescence bias following Soto and John (2008). 17 6. The Role of Schools and Families in Fostering Socioemotional Skills 6.1 The Role of School and Families Schools can play a central role in fostering socioemotional skills. Evidence from both OECD and International literature suggests that developing countries suggests that schools can make a schools and families play important meaningful difference in improving students’ social and roles in cost-effectively fostering emotional skills by introducing SEL curricular activities, socioemotional skills. extracurricular programs, novel pedagogical methods, and inclusive school climate (Ikesako and Miyamoto 2015; Puerta, Valerio, and Bernal 2016).16 For instance, schools can introduce history lessons to foster students’ sense of compassion and respect for others, by introducing major events (for example, the Holocaust) in a way that encourages students to contextualize facts, debate what drove the events, and role-play the oppressed and the oppressors to take different perspectives.17 Schools can also enrich extracurricular activities (for example, music and arts), introduce service-learning (for example, volunteering), and vocational skills training (for example, apprenticeships), all of which have been shown to foster students’ social and emotional skills.18 Schools can also improve the learning climate by creating an open, inclusive, and collaborative environment using targeted campaigns (for example, anti-bullying). 19 Moreover, schools can provide teacher training to improve their pedagogical practices (for example, project-based learning) to allow students to better collaborate, solve problems, and self-reflect during regular curricular activities. 20 Last, schools can take preventive measures to minimize the incidence of students’ psychosocial distress by improving communication with parents and assigning qualified psychologists who would cater to those in need of counseling and guidance. Families can also play a fundamental role in fostering socioemotional skills. Many studies point to the benefits of parenting on children’s psychosocial development (Ikesako and Miyamoto 2015; Puerta, Valerio, and Bernal 2016). Parents can identify potential issues (for example, bullying) that may lead their children to emotional distress and subsequently provide guidance and support to help overcome the situation. Simply showing care and support to their children at home can also make a difference. To the contrary, parents who tend to engage in fights with household members, or abuse them, are more likely to generate emotional distress to children. For some of the disadvantaged children who do not have parents that can provide much needed psychosocial support, programs that offer surrogate parents from the local community have also shown a positive impact on children’s socioemotional development and learning outcomes (Kautz et al. 2014). 21 Parents’ socioeconomic circumstances and psychosocial 16 More generally, the literature suggests that socioemotional skills are malleable, although the degree in which they are malleable depends on the type of socioemotional skills and age (Kautz et al. 2014). There is some evidence suggesting that childhood and adolescence are particularly malleable periods of socioemotional development (Kautz et al. 2014). 17 A good example of this approach is Facing History and Ourselves: https://www.facinghistory.org/. 18 World Bank (2019) provides recent evidence suggesting the importance of extracurricular activities in the Russian Federation. 19 One of the well-recognized approach is the bullying prevention program run by Second Step: https://www.secondstep.org/bullying-prevention. 20 A good example of this approach is the Responsive Classroom: https://www.responsiveclassroom.org/. 21 A well-known cross-national program is the Big Brothers and Big Sisters program: http://www.bbbsi.org/. 18 conditions also affect their capacity to provide an emotionally nurturing environment. Last, the evidence also points to the importance of ensuring consistency between the school and home environment for SEL initiatives to be effective. This is why many successful SEL programs have a home outreach component. SEL can also be a cost-effective way to improve children’s learning outcomes and subsequently their lifetime success. Emerging evidence points to the role of school and family-based interventions as causal drivers of children’s socioemotional development, labor market success, and improved health conditions (Kautz et al. 2014; OECD 2015). Some of these successful SEL programs are considered to be among a class of cost-effective approaches to foster children’s outcomes. For instance, a meta-analysis of six socioemotional skills intervention programs covering kindergarten and upper secondary schools in the United States, Canada, and Sweden demonstrated a benefit-cost ratio of 11 to 1 (Belfield et al. 2015). Many early childhood and childhood intervention programs also yielded high rates of returns (Kautz et al. 2014). Schools and families explain a large share of the variations in socioemotional skills in Moldova. Do Schools and families in Moldova schools and families explain differences in children’s explain a large share of the variations socioemotional skills? Figure 6 suggests that a large in socioemotional skills. share of variance in the composite measure of socioemotional skills is explained by families (14.7 percent) and schools (5.7 percent), even after taking into account differences in gender and age across students.22 Figure 6: Proportion of Variance in Socioemotional Skills Explained by School and Family Factors Source: Socioemotional Skills Assessment in Moldova (World Bank 2019). Note: The figures are based on standardized data on socioemotional skills after adjusting for acquiescence. The results are based on a standard analysis of variance decompositions. 22 Note that the smaller proportion of variations captured by schools compared to families are likely to be partly driven by the fact that the analysis is based on variations across only 80 schools versus 2,050 families. 19 While family factors explain the largest share of variations in socioemotional skills, schools explain the largest share of variations in math grades. This may be a result of Moldovan lower secondary schools having focused more on children’s academic learning, the results of which are translated into grades. Schools and families, taken together, explain as many variances in the composite measures of socioemotional skills as in the math grades (that is, about 20 percent). 6.2. School Characteristics Associated with Children’s Socioemotional Skills The Socioemotional Skills Assessment in Moldova measures learning contexts and skills of grade 9 students to identify which school and family environment factors are more strongly associated with socioemotional skills. This assessment provides a rich set of measures not only of socioemotional skills, but also of learning contexts (Table 3). School learning contexts include extracurricular activities (engagement in arts and theater), classroom climate (class size and disruptions), school climate (violence and part-time teachers), teacher quality (teaching experience and pedagogical styles23), and preventive measures (parental engagement in school and availability of school-based psychologists). Family learning context measures include parental support (for example, affection, conversation, interest in and support of school activities, and sharing meals), hostile environment (for example, verbal or physical abuse toward spouse or child), learning assets (for example, have a desk to study, computer, books, and dictionary), and number of siblings at home. Table 3: Measures of School and Family Learning Contexts from the Socioemotional Skills Assessment Measures Scale Extracurricular: arts and A binary measure which captures if a school offers both arts and theater theater as extracurricular activities, as reported by the school principal Classroom climate: class A continuous measure that captures the average class size of a school, size as reported by the principal Classroom climate: A standardized measure based on the average score of scales designed disruptions to capture the frequency of disruptions during Romanian and math School classes within a school, as reported by students School climate: violence A standardized measure based on the average score of scales designed to capture the extent to which violence takes place or there are delinquent juveniles within a school, as reported by students School climate: part-time A continuous measure that captures the proportion of teachers who are teachers working part-time within a school, as reported by the principal Teachers’ quality: A continuous measure that captures the average years of teaching experience experience within a school, as reported by the principal 23 The measure of pedagogical styles is designed to capture some of the key elements of teachers’ classroom practices that evidence suggests play an important role in fostering students’ socioemotional skills. The measurement instrument for pedagogical styles is derived from the World Bank’s Teach launched in early 2019. See Table 5 for a more detailed description of the measure. 20 Measures Scale Teachers’ quality: A standardized measure based on the average score using scales pedagogical style designed to capture pedagogical style and attitudes (TEACH) of Romanian language and math teachers, as reported by students within a school. The scales cover the following dimensions: (a) supportive learning environment, (b) positive behavioral expectations, (c) autonomy, (d) perseverance, and (e) social and collaborative skills. Preventive measures: A standardized measure based on the principal component of scales Parental engagement and designed to capture the frequency of parental communication, parental psychologist involvement in school decisions, frequency of parent-teacher meetings, and assignment of a psychologist in school, as reported by the principal Parental support A standardized measure which captures the principal component of the categorical answers provided by students to 17 questions regarding the relationship between the child and his or her parents (The principal component loads heavily on questions regarding parental practices, such as whether parents are interested in the child’s school activities, help him or her overcome difficulties at school, treat him or her with love and affection, and have frequent conversations.) Hostile environment A standardized measure which captures the second component of the Family categorical answers provided by students to 17 questions regarding the relation between the child and the parents (The second component loads mainly on questions addressing physical and verbal abuse against the child and between the parents.) Learning assets A standardized measure based on the average score of 13 binary scales designed to capture whether a student had access to important elements to study at home including a desk, a quiet room to study, a computer, access to the Internet, and books Number of siblings Number of siblings in the household Source: Socioemotional Skills Assessment in Moldova (World Bank 2019). Classroom disruptions and parental support are strongly correlated with grade 9 students’ self-reported A grade 9 student in Moldova is more socioemotional skills in Moldova. Table 4 presents results likely to self-report higher levels of from the data analysis after verifying the robustness of socioemotional skills if she/he belongs measures and analytical approaches. A number of schools to a classroom with a lower incidence and family learning context measures highly correlate with of disruptions and has parents who are socioemotional skills. In terms of scale, classroom more caring and supportive. disruptions exhibit the largest association with socioemotional skills, with one standard deviation increase in the student-reported frequency of disruptions in Romanian and math classes associated with a 15.3 percentage points reduction in socioemotional skills. The association is particularly high for the domains of ‘caring for others’ and ‘managing emotions’. Reducing classroom disruptions may help establish a learning environment that is conducive to expressing compassion and respect for classmates as well as to raising self-confidence.24 Parental support also shows large association with socioemotional skills, with one standard deviation increase in the student-reported perception of parental support associated with 12.1 percentage points 24 This result is also consistent with the reverse causality: having more students with a lower capacity to ‘care for others’ and ‘manage emotions’ may lead to increased frequency of disruptive behaviors in classrooms. 21 increase in socioemotional skills. The association is particularly strong for the domains of ‘caring for others’ and ‘working towards goals’. Having parents who are caring and supportive to their children are likely to help improve their capacity to be compassionate and respectful to others as well as be responsible and productive.25 Table 4 also shows that classroom disruptions and parental support are also correlated with students’ math grades, although the association is much weaker than the correlation with socioemotional skills. Table 4: The Relationship between Learning Contexts and Skills Learning contexts Composite Engaging Caring Working Managing Exploring Math Extracurricular: arts −0.033 −0.037    −0.014    −0.008    −0.029    0.005    0.071*  and theater Classroom climate: −0.002 0.001    0.000    −0.003    0.000    −0.003    −0.007* class size Classroom climate: −0.153*** −0.089*** −0.152*** −0.130*** −0.152*** −0.088*** −0.040* disruptions School climate: 0.036*   0.021    0.028    0.013    0.123*** −0.047**  0.010 violence School School climate: part- −0.263*** −0.044    −0.215**  −0.313*** −0.168    −0.250** −0.350*** time teachers Teachers’ quality: 0.028** 0.014    0.039*** 0.008    0.017    0.017    −0.030** experience Teachers’ quality: 0.038*  0.024    0.030    0.051**  −0.003    0.047**  0.000 pedagogical style Preventive −0.032    −0.039*  −0.038*  −0.009    −0.036    −0.035    −0.010 measures: parents and psychologists Parental support 0.121*** 0.075*** 0.120*** 0.116*** 0.071*** 0.050*** 0.028*** Hostile environment −0.081*** −0.051*** −0.054*** −0.080*** −0.048*** −0.047*** −0.073*** Family Learning assets 0.059*** 0.088*** 0.006    0.025**  0.037*** 0.098*** 0.110*** Number of siblings 0.025    0.004    0.021    0.027    0.052*** −0.004    −0.074*** Source: Socioemotional Skills Assessment in Moldova (World Bank 2019). Note: The figures are based on standardized data on socioemotional skills after adjusting for acquiescence. The results are based on a standard linear regression model including key controls such as gender and age. ***, **, and * represent statistically significant coefficient estimates at the 0.01, 0.05 and 0.1 levels, respectively. All other results are not statistically significant at the 0.1 level. 25 This result is also consistent with the reverse causality: having a child who is caring and responsible may affect parents’ propensity to be caring and supportive. 22 Contrary to having supportive parents, students who face hostile family environment and limited learning assets A grade 9 student who experiences are less likely to report higher levels of socioemotional hostile family environment and limited skills. Children who live in a home environment involving learning assets is also less likely to frequent physical and verbal abuse are more likely to self-report higher levels of struggle to develop a spirit of engagement, caring, and socioemotional skills. working toward life goals. Evidence suggests that learning assets, such as having access to a quiet study space, a computer, and books are important determinants of learning outcomes. Learning assets can also play an important role in nurturing socioemotional skills as these skills develop hand-in-hand with academic skills. Students from classrooms with higher-quality teachers are more likely to self-report higher levels of Teachers’ experience and pedagogical socioemotional skills. Table 4 shows that teachers’ style as reported by students are also experience and pedagogical style are both correlated associated with the students’ with the composite measure of socioemotional skills. socioemotional skills. Among teachers’ When looking at specific domains of socioemotional pedagogical styles, their capacity to skills, teachers’ experience is correlated with students’ encourage students’ autonomy as well self-reported tendency to ‘care for others’, while as provide supportive, perseverant, and teachers’ pedagogical style is correlated with students’ collaborative pedagogical style are self-reported tendency to ‘work with others’ and more strongly associated with students’ ‘explore new horizons’. Although the association is not socioemotional skills. very strong, these results suggest that teachers’ quality can help improve children’s socioemotional skills. Among the measures of teachers’ pedagogical styles, their capacity to encourage students’ autonomy as well as provide supportive, perseverant, and collaborative pedagogical style are more strongly associated with students’ socioemotional skills. What are the aspects of teachers’ pedagogical styles and attitudes that are more strongly correlated with students’ socioemotional skills? Table 5 presents simple correlations between students’ self-reported socioemotional skills and measures of teachers’ pedagogical practices across the five dimensions measured using TEACH. With the exception of students’ socioemotional capacity to ‘manage emotions’, all the socioemotional skills were highly correlated with the five dimensions of teachers’ pedagogical practices. In particular, teachers’ capacity to encourage students’ autonomy (that is, giving students choice and opportunities to take roles as well as encouraging students to participate) shows a consistent association with all the dimensions of socioemotional skills. It is also worth noting that the four other elements of TEACH, namely, teachers’ capacity to provide supportive (respectful and inclusive), positive behavioral (setting clear expectations and acknowledging positive behavior), perseverant (acknowledging student efforts and encouraging goal setting), and social and collaborative pedagogical style show high and consistent correlations with all the measures of socioemotional skills except for students’ capacity to manage emotions. 23 Table 5: Relationship between Teachers’ Quality and Students’ Socioemotional Skills Teachers’ Pedagogical Styles Composite Engaging Caring Working Managing Exploring Supportive learning environment 0.068*** 0.044** 0.053** 0.077*** 0.022 0.049** Positive behavioral expectations 0.057*** 0.034 0.045** 0.067*** 0.022 0.046** Autonomy 0.088*** 0.068*** 0.084*** 0.093*** 0.036* 0.055** Perseverance 0.077*** 0.055** 0.057*** 0.089*** 0.025 0.055** Social and collaborative skills 0.067*** 0.045** 0.055** 0.073*** 0.031 0.039* Source: Socioemotional Skills Assessment in Moldova (World Bank 2019). Note: The results present pairwise correlations between each of the Romanian language teachers’ pedagogical styles (classroom average after excluding the students’ scores) and socioemotional skills. The analysis is based on standardized data on socioemotional skills after adjusting for acquiescence. ***, **, and * represent statistically significant coefficient estimates at the 0.01, 0.05 and 0.1 levels, respectively. All other results are not statistically significant at the 0.1 level. Having a larger share of part-time teachers in a school is strongly associated with lower self-reported levels of A larger share of part-time teachers in socioemotional skills. A one standard deviation increase schools is strongly associated with a in the proportion of teachers who are working part-time in lower self-reported level of a school is associated with a 26.3 percentage point socioemotional skills. decrease in the self-reported levels of composite socioemotional skills. This negative relationship is particularly strong for ‘working towards goals’ and ‘exploring new horizons’. Many schools outside of the Moldovan capital that rely on part-time teachers may not be able to provide a socioemotionally nurturing environment in a consistent manner, possibly due to the challenges in establishing a stable relationship with students. Students’ engagement in extracurricular activities (arts and theater) is not associated with socioemotional skills. This is contrary to the recent evidence from the Russian Federation which shows that engagement in extracurricular activities is associated with higher levels of social skills (World Bank 2019). International evidence suggests that extracurricular activities can be a promising way to foster socioemotional skills. It may well be that extracurricular activities in Moldova based on arts and theater have not emphasized much the SEL aspects, or that the time spent on extracurricular activities may not be sufficient to sufficiently improve students’ socioemotional skills. It would be useful to further investigate the content and intensity of extracurricular activities in Moldova and reflect on the objectives of these programs. 24 7. Implications for Policies and Practices This report presents results from the first countrywide assessment of socioemotional skills in Moldova, targeting children at grade 9 who are at an important juncture of continuous academic development and labor market transitions. 26 This assessment relied on one of the most cross-nationally validated self- reported measures of socioemotional skills, namely the BFI-2. The main conclusion is that the school and the family students belong to are important predictors of their levels of socioemotional skills. Various measures of school and family learning contexts are highly correlated with learning outcomes and well- being. Policy makers, teachers, and parents can make a meaningful difference to children’s lives by improving policy options, adapting pedagogical approaches, enriching parenting practices, and introducing regular assessments. Table 6 summarizes the main results from the analysis and provides implications for policies and practices. Table 6: Summary of Results and Policy Conclusions Main Findings Implications for Education Policies and Practices • Socioemotional skills can be • Policy makers may consider including socioemotional skills measures usefully measured in Moldova. as part of the regular national assessments. This information will • Grade 9 students’ measures of provide valuable insights for improving policies and programs socioemotional skills predict designed to raise students’ learning, well-being, and labor market math grade as well as various outcomes. A repeated assessment of socioemotional skills will help measures of student well-being. identify progress made in fostering skills that matter. Policy makers may also help build local capacity to carry out socioemotional assessments in the future. • Schools may consider regularly measuring socioemotional skills as part of formative assessments, to allow teachers to better understand and support students’ socioemotional development. Parents may also benefit from receiving regular updates on children’s socioemotional development. This would provide opportunities to adapt pedagogical and nurturing practices at home. While students can usefully self- report on their socioemotional capability, teachers can also provide valuable information on students’ socioemotional skills. 26 This report has established that socioemotional skills measured at grade 9 are important correlates of learning inputs as well as learning outcomes and well-being. The direction of causal effects cannot be identified with the microdata. Given that students’ socioemotional skills can evolve over time, the correlates can also change depending on the timing of the assessment of these skills. 25 Main Findings Implications for Education Policies and Practices • The composite socioemotional • A balanced set of socioemotional skills is likely to matter for students’ skills measure, which has been outcomes. constructed to capture all five • Teachers struggling to improve students’ math performance may dimensions of socioemotional explore not only providing them with academic support but also skills, shows particularly strong enhancing their capacity to ‘work towards goals’ (for example, correlations with measures of perseverance and responsibility), ‘engage with others’ (for example, math grades and well-being. assertiveness and energy), and ‘explore new horizons’ (for example, • Among various dimensions of curiosity and creativity). socioemotional skills, a student’s ability to ‘work towards goals’, ‘engage with others’, and ‘explore new horizons’ were shown to have a strong relationship with math grades. A student’s ability to ‘work towards goals’, ‘care for others’, and ‘explore new horizons’ showed strong associations with self-perceived happiness. • Girls are much more likely to • Teachers can benefit from being mindful about gender differences in self-rate higher levels of students’ socioemotional skills. While girls can be encouraged to build socioemotional skills than boys, on their strengths such as being engaging, caring, goal achieving, and with the exception of their explorative, some may also benefit from receiving extra support in capacity to ‘manage emotions’. managing emotions. A larger proportion of boys, in turn, may benefit • Students from schools in the from receiving extra support to be more engaging, caring, goal Central region are much more achieving, and explorative. Managing emotions is a particularly likely to self-rate higher levels of important and challenging aspect of children’s (both girls and boys) socioemotional skills than those socioemotional capability to address, and there are likely to be from the North, South, and differences in the most effective approaches to address this by Chisinau regions. gender. There are also boys who may benefit from receiving support to manage emotions. • Decision makers and teachers from Chisinau may benefit from knowing students’ generally low self-reported levels of capacity to ‘care for others' and ‘work towards goals’. Those from the South region could be mindful about the generally low levels of students’ capacity to ‘explore new horizons’. • School quality matters. This is • Schools can benefit from implementing a strategy to reduce the levels particularly the case with of classroom disruptions and at the same time improve teachers’ classroom climate (that is, pedagogical practices designed to foster children’s socioemotional classroom disruptions) and skills. There is emerging evidence on school-based practices designed quality of teachers (teachers’ to foster children’s socioemotional skills by (a) delivering SEL experience and pedagogical programs to improve teachers’ capacity to manage the classroom practice). climate and (b) improving teachers’ pedagogical skills. • Extracurricular activities (that is, • A detailed study of extracurricular activities in Moldova can be useful arts and theater) are not to provide policy makers and teachers with more information to associated with socioemotional evaluate if existing practices are conducive to children’s skills. socioemotional development. 26 Main Findings Implications for Education Policies and Practices • Good parenting, by providing • Policy makers may consider ensuring that programs related to SEL parental support (for example, take into account (a) the importance of better informing parents improving encouragement and about children’s socioemotional skills and (b) the improvement in care for children) and avoiding parental capacity to support socioemotional skills development. hostile home environment (for • Schools may consider strengthening outreach to parents (for example, minimizing physical and example, through parent-teacher meetings and regular information verbal abuse at home) are key exchanges) so that they are better informed about children’s correlates of children’s socioemotional development and ways to foster these skills at home. socioemotional skills. • Moldova does not have SEL as • Policy makers may explore introducing SEL programs by adapting part of the regular school findings from the international evidence on ‘what works’ to local curricula. settings. SEL programs include stand-alone SEL programs (for example, SEL curricula) and programs designed to integrate the core principles of SEL in teaching and learning practices across curricular activities. There are no formal SEL programs to date in Moldova. Moldova has very recently started introducing new curricula which include areas such as ‘Counseling and personal development’ and ‘Education for society’, which can have potentially important implications for SEL depending on their design and delivery mechanisms including teacher preparation. These initial efforts by the Moldovan government and the need to collect data and evidence on students’ socioemotional capabilities can be highlighted in relevant research and policy documents. • When introducing SEL programs, policy makers may first explore initiating an awareness campaign on the concept and benefits of fostering children’s social and emotional development. • Policy makers may also consider ways to foster teachers’ understanding of socioemotional skills and pedagogical development in SEL during pre- and in-service training. SEL is a new area of teaching and learning for Moldova as in many countries around the world. Emerging global evidence on effective SEL intervention programs can be mobilized to explore ways to integrate SEL into the Moldovan education system.27 Some of the large-scale SEL interventions and curricular reforms can have considerable fiscal implications, while others (for example, adaptations of existing curricula by integrating the principles of SEL) can be financially more feasible. Whatever the approach of SEL to be considered, Moldova can benefit from exploring its potential to foster academic outcomes and child well- being as well as to improve labor market outcomes, given the cost-effectiveness of successful SEL programs (Kautz et al. 2014; Belfield et al. 2015). SEL programs must be designed and implemented in a holistic manner, involving teachers, parents, and students. It is fundamental to ensure that all stakeholders fully understand, appreciate, and engage in the process. The outreach of SEL programs must be inclusive, to be sure that those who are in dire need of psychosocial support receive enough attention. In countries where SEL has already been introduced (for example, the United States and the United Kingdom), there are many programs that are focused on the vulnerable and disadvantaged students who would benefit the most (Kautz et al. 2014). Like academic 27 See, for example, inventory of successful SEL programs prepared by the Collaborative for Academic, Social, and Emotional Learning (CASEL) (https://casel.org/guide/). Boxes 4.1, 5.1, 5.2, 5.3, 5.6, and 5.7 in OECD (2015) also provide various concrete country examples. 27 skills, programs designed to foster social and emotional skills are the most effective when they are introduced earlier in children’s lifecycle. A period between early childhood and adolescence is a particularly malleable period of child development (Harvard Center on the Developing Child 2007; Kautz et al. 2014). This report has focused on the role of education policies and practices. Labor market policies and practices can also play an important role in reducing the skills gaps and fostering school to work transitions. The Ministry of Health, Labor and Social Protection (MoHLSP) is responsible for assessing the labor market trends, determining the state-funded positions in vocational education and training (VET) institutions and universities together with the MoECR, and developing the Occupational Standards for the craft professions. The MoHLSP is also responsible for addressing those who are unemployed, including their reskilling. The MoECR and the MoHLSP can usefully work together to address the shared goals of improving labor market outcomes. One of the most urgent priorities among the above recommendations is to start regular data collection on socioemotional skills and subsequently conduct detailed diagnostics. This information can provide the MoECR with indispensable inputs to designing SEL programs. The data on socioemotional skills can also provide the MoHLSP with valuable information to help design training and active labor market policies. Moreover, the diagnostics of socioemotional skills supply would be valuable to employers who could build on this information to improve strategies for recruitment and enterprise training.28 28 The Moldovan Chamber of Commerce and Industry could also contribute to communicating the nature of socioemotional skills shortages. 28 References Autor, D., F. Levy, and R. J. Murnane. 2003. “The Skill Content of Recent Technological Change: An Empirical Exploration.� Quarterly Journal of Economics 118 (4). Belfield, C., B. Bowden, A. Klapp, H. Levin, R. Shand, and S. Zander. 2015. The Economic Value of Social and Emotional Learning. New York: Center for Benefit-Cost Studies in Education. Cunha, F., J. J. Heckman, and S. Schennach. 2010. “Estimating the Technology of Cognitive and Non- Cognitive Skill Formation.� Econometrica 78 (3): 883–931. Deming, D. J. 2018. The Growing Importance of Social Skills in the Labor Market. Harvard University. Cambridge, MA. Duckworth, A. L., and D. S. Yeager. 2015. “Measurement Matters: Assessing Personal Qualities Other than Cognitive Ability for Educational Purposes.� Educational Researcher 44 (4): 237–251. Harvard Center on the Developing Child. 2007. The Science of Early Childhood Development: Closing the Gap between What We Know and What We Do. Cambridge, MA. Ikesako, H., and K. Miyamoto. 2015. "Fostering Social and Emotional Skills through Families, Schools and Communities: Summary of International Evidence and Implication for Japan's Educational Practices and Research." OECD Education Working Paper 121. OECD Publishing, Paris. John, O., and F. De Fruyt. 2015. Social and Emotional Skills Framework for the Longitudinal Study of Skills Development in Cities. Paris: OECD. John, O. P., and S. Srivastava. 1999. “The Big Five Trait Taxonomy: History, Measurement, and Theoretical Perspectives.� In Handbook of Personality: Theory and Research (2nd ed.), edited by L. A. Pervin and O. P. John, pp. 102–138. New York: Guilford. Kautz, T., J. J. Heckman, R. Diris, B. ter Weel, and L. Borghans. 2014. “Fostering and Measuring Skills: Improving Cognitive and Non-cognitive Skills to Promote Lifetime Success.� OECD Education Working Paper 110. OECD Publishing, Paris. Laajaj, R., K. Macours, D. A. P. Hernandez, O. Arias, S. Gosling, J. Potter, M. Rubio-Codina, and R. Vakis. 2019. “Challenges to Capture the Big Five Personality Traits in Non-WEIRD Populations.� Science Advances 5. The American Association for the Advancement of Science. Washington, DC. OECD (Organisation for Economic Co-operation and Development). 2015. Skills for Social Progress: The Powers of Social and Emotional Skills. Paris: OECD Publishing. De Neve, J. E., and A. J. Oswald. 2012. “Estimating the influence of life satisfaction and positive affect on later income using sibling fixed effects.� Proceedings of the National Academy of Sciences of the United States of America (PNAS) 109 (49): 19953-19958. Princeton: Princeton University. Puerta, M. L. Sanchez, A. Valerio, and M. Bernal. 2016. Taking Stock of Programs to Develop Socioemotional Skills. Washington, DC: World Bank. 29 Rutkowski, J., V. Levin, and A. Bargu. 2017. “Missing Skills: Results of the Skills Module of the Moldovan Labor Market Forecast Survey.� Washington, DC: World Bank. Soto, C. J., and O. P. John. 2008. “The Developmental Psychometrics of Big Five Self-Reports: Acquiescence, Factor Structure, Coherence, and Differentiation from Ages 10 to 20.� Journal of Personality and Social Psychology 94 (4): 718–737. American Psychological Association. ———. 2017. “The Next Big Five Inventory (BFI-2): Developing and Assessing a Hierarchical Model with 15 Facets to Enhance Bandwidth, Fidelity, and Predictive Power.� Journal of Personality and Social Psychology 113 (1): 117–143. Tetyora, V., U. J. Lee, and J. L. Garcia Velazquez. 2017. Assessment of Firm-level Skills Demand and Engagement in Skills Development: Creating a Demand-led Skills Ecosystem in Moldova (English). Washington, DC: World Bank. World Bank. 2018. The World Development Report 2018. Learning to Realize Education’s Promise. World Bank, Washington, DC. World Bank. 2019. “Russian Federation: Doing Extra-Curricular Education.� World Bank, Washington, DC. 30 Annex 1: Sampling for the Socioemotional Skills Assessment The sampling strategy proposed below is determined by three characteristics of the particular setting we will be working on: 1. The sampling unit is school (cluster), not a student. 29 That means we will interview multiple students from each sampled school. 2. Classrooms vary in size. The average class size in the final sample frame is 21 students.30 However, one quarter of the classrooms have 17 students or less and another quarter of classrooms have 26 students or more. 3. Due to capacity constraints, the maximum number of students that we will interview in a given classroom will be 20. With these constraints in mind, we propose a probability sampling proportional to size methodology to accommodate the fact that some schools will have all their students interviewed (i.e., those with 20 students or less), while others do not. Hence, the probability P of a student being surveyed depends on the class size. If the student belongs to a small class, her sampling probability depends only on the sampling probability of her school. If the student belongs to a big class, her sampling probability depends on that of her school and class size. In a nutshell, we need proportionally more of the big classrooms in our sample to account for the fact that the sampling probability of a student from a big classroom is less than that of student from a small classroom, given that their classrooms have been selected into the sample. Formally, we will draw 80 schools (clusters). Let N be the total number of students in the sample frame and Ns the number of students in cluster S. The probability of selection of a student in cluster S, PS, is 80�𝑆 𝑚𝑖𝑛{20, �𝑆 } ( )( ) � �𝑆 Thus, for classrooms smaller than 20, PS collapses to the probability of the school being selected. For classrooms larger than 20, PS is inflated by the probability of a student being selected into the set of 20 that will be interviewed in that classroom. For cases where class size exceeds 20, the surveyors should implement a process for the random selection of 20 students to take the survey. That process should be independent from the students’ characteristics like academic performance, personality, age, gender, classroom assignment or place of birth or residency. In the rare cases (3 schools in the sample) where schools have more than four eligible classes, survey children from the largest 3 classrooms. 29 There are some multi-class schools. They are relatively few in the final sample frame (28%). In order to keep the sampling strategy relatively simple, we agreed to define the sampling of multi-class schools based on their largest class. Thus, in what follows, classrooms and schools are used interchangeably, except in the final paragraph where the distinction is made explicit. 30 The final sample frame excludes Russian speaking schools, small classrooms (i.e., 10 students or less), schools with more than one shift, localities with three or less classrooms. 31 Annex 2: Background Questionnaire of the Socioemotional Skills Assessment Principal What is the total enrollment in this school? o [Insert total] o [Insert boys] o [Insert girls] What is the total enrollment in grade-9? o [Insert total] o Class 1 [Insert number] o Class 2 [Insert number] o Class 3 [Insert number] o Class 4 [Insert number] o Class 5 [Insert number] o Class 6 [Insert number] How many grade 9 teachers do you have in this school? o Teachers in total o Full-time (18 hours or more) o Part-time (Less than 18 hours) o Teachers that are staff o [Insert number] o Teachers that are outsourced o [Insert number] o Teachers without bachelor’s degree qualification o [Insert number] o Teachers with bachelor’s degree qualification o Full-time o Part-time o Teachers with master’s degree or more o Full-time o Part-time What is the average teaching experience of grade 9 teachers in this school? o 1-2 years o 2-5 years o 5-7 years o 7-10 years o 10-15 years o 15+ years How often does the school organize parental meetings/discussions/conferences in an academic year for grade 5-9 students? o Once a month 32 o Once every semester. o Twice every semester. o Once per year o Twice every year. o Other (please specify) What are the means of communication between the school and parents of grade 5-9 students? o Mail (post) o Telephone o E-mail o WhatsApp, Telegram, Viber etc.) o Meetings o Web page or page in social media o School billboard o Other (please specify) How often do you send those communications? o More than once a week o Once a week o Once every two weeks o Between one to three times per month o Between one to three times per semester o Once a year o Never Strongly Agree Neither Disagree Strongly Agree agree disagree nor disagree This school involves children’s parents o o o o o in the school’s decision-making process. Strongly Agree Neither Disagree Strongly Agree agree disagree nor disagree This school offers parents’ education o o o o o (e.g., courses on family literacy) or family support programs (e.g., to assist o o o o o with health, nutrition). Does the school offer any opportunity for extra classes/remedial classes? o Yes 33 o No (please describe why) Can teachers increase their compensation through performance bonuses? o Yes o No If “YES� o Related to not skipping work during school days o Related to administrative services for the school or the education system o Related to attending personal/professional development courses or programs o Related to students’ outcomes o Other conditions for increase (please specify) Does the school offer counselling and guidance (career, psychological etc.) for students? o Yes o No If “YES� Does the school have a psychologist/counsellor? o Yes o No If “YES� Is he/she in-house? o Yes o No How many students used the counseling/guidance services last semester? • [Insert number] What are the range of extra-curricular activities available in your school? (events organized on holidays are not included) o Number of activities available: o 0 o 1 o 2 o 3 o 4 o 5 o 6 34 o More than 6 o Do you have sports as extracurricular activities? o Yes ▪ How many students participated during the past semester? • [Insert number] o No o Do you have music as extracurricular activities? o Yes ▪ How many students participated during the past semester? • [Insert number] o No o Do you have theatre as extracurricular activities? o Yes ▪ How many students participated during the past semester? • [Insert number] o No o What other extracurricular activities do you have? o (please specify) Classroom culture Strongly Agree Neither Disagree Strongly Agree agree disagree nor disagree Teachers in this school treats all o o o o o students respectfully Teachers in this school use positive o o o o o language with students Teachers in this school respond to o o o o o students’ needs Teachers in this school do not exhibit o o o o o gender bias and challenges gender stereotypes in the classroom Teachers in this school sets clear o o o o o behavioral expectations for classroom activities Teachers in this school acknowledges o o o o o positive student behavior Teachers in this school redirects o o o o o misbehavior and focuses on the expected behavior, rather than the undesired behavior 35 Teachers in this school gives students o o o o o autonomy and provides students with choices Teachers in this school provides o o o o o students with opportunities to take on roles in the classroom Students in this school volunteer to o o o o o participate in the classroom Teachers in this school acknowledge o o o o o students’ efforts Teachers in this school have a positive o o o o o attitude towards students’ challenges Teachers in this school encourage goal o o o o o setting among students Teachers in this school promote o o o o o student collaboration through peer interaction Teachers in this school promote o o o o o students’ interpersonal skills, such as perspective taking, empathizing, emotion regulation, and social problem solving Students in this school collaborate with o o o o o one another through peer interaction Frequency of disruptions in school Every Most Some Never or lesson lessons lessons hardly ever Students don’t listen to what teacher’s say o o o o There is noise and disorder o o o o The teacher has to wait a long time for students o o o o to quiet down Students cannot work well o o o o Student’s don’t start working for a long time o o o o after the lesson begins School quality True Somewhat Neither Somewhat Untrue true true or untrue untrue There are places in my school o o o o o where crimes such as robbing, 36 stealing, violence, etc. frequently occur There are places in the o o o o o surroundings of my school where crimes such as robbing, stealing, violence, etc. frequently occur There are many delinquent o o o o o juveniles in my school 37 Students This assessment is composed of two sections. The first section is background questions. The second section is on socioemotional skills. You have one class slot to complete the assessment. There are no right or wrong answers. Please fill in the questionnaire in full in your own pace. This assessment is anonymous, and the results will be strictly confidential. The assessment will NOT affect your grades. If you do not understand any of the questions, please ask the survey administrator for support. Choose the locality. Choose the school you are studying. Write your class name (e.g., 9, 9A, 9B etc.) Name Name: Surname: Write your number in the class registry/catalogue [Insert number] Birth date Month: Year: • January • 2000 • February • 2001 • March • 2002 • April • 2003 • May • 2004 • June • 2005 • July • August • September • October • November • December Gender o Female o Male What is your height? (in cm) o [Insert height] What is your weight? (in kg) o [Insert weight] 38 How many people live in the house with you? o [Insert number] How many people over the age of 18 live in the house with you? o [Insert number] How many people under the age of 18 live in your household? (including yourself) o [Insert number] Does your mother live in your household? o Yes o No Does your father live in your household? o Yes o No How many older brothers do you have? o [Insert number] How many older sisters do you have? o [Insert number] How many younger brothers do you have? o [Insert number] How many younger sisters do you have? o [Insert number] How many bedrooms are there in your house? o [Insert number] Have you been involved in paid work outside the family during the school year? o Yes o No What’s your mother’s highest level of schooling? o Primary school; o Secondary school; o Technical college/two-year degree; o Bachelor’s/four-year degree; o Did not complete schooling 39 What’s your father’s highest level of schooling? o Primary school; o Secondary school; o Technical college/two-year degree; o Bachelor’s/four-year degree; o Did not complete schooling What’s your mother’s occupation? o What’s your father’s occupation? o Which of the following do you have at home? o A desk to study at o A room of your own o A quiet place to study o A computer you can use for school work o A link to the Internet o Classic literature o Books of poetry o Works of art (e.g., paintings) o Books to help with your school work o Technical reference books o Dictionary o Books on art, music, or design Which of the following do you have at home? o Television(s) o Car(s) o Room(s) with bath or shower o Smartphones o Computers (desktop or laptop) o Tablets o Musical instruments How many books are there at your home? (Besides school books) o 0-10 o 11-25 o 26-100 o 101-200 o 201-500 o More than 500 What language do you speak at home most of the time? o Romanian 40 o Russian o Other (please specify) Did you attend pre-school? o No o Yes, for one year or less o Yes, for more than one year Have you ever repeated a grade? o Yes o Which level? ▪ Once ▪ Twice or more o No Feeling towards school and learning: Now, we would like to ask you some questions about your feelings towards school and learning. School belonging Strongly Agree Neutral Disagree Strongly Agree (Neither disagree agree or disagree) I feel like an outsider (or left out of o o o o o things) at school I make friends easily at school o o o o o I feel like I belong at school o o o o o I feel awkward and out of place in my o o o o o school Other students seem to like me o o o o o I feel lonely at school o o o o o Joy of learning Strongly Agree Neutral Disagree Strongly Agree (Neither disagree agree or disagree) I generally have fun when I am o o o o o learning at school I like reading about my school subjects o o o o o I am happy working on school work o o o o o I enjoy acquiring new knowledge o o o o o 41 I am interested in learning about the o o o o o world through my school work Value of learning Strongly Agree Neutral Disagree Strongly Agree (Neither disagree agree or disagree) Making an effort in school is worth it o o o o o because this will help me in the work I want to do later on What I learn in my school is important o o o o o for me because I need this for what I want to do later on Studying at school is worthwhile o o o o o because what I learn will improve my career prospects Many things I learn at my school will o o o o o help me get a job School quality True Somewhat Neither Somewhat Untrue true true or untrue untrue There are places in my school o o o o o where crimes such as robbing, stealing, violence, etc. frequently occur There are places in the o o o o o surroundings of my school where crimes such as robbing, stealing, violence, etc. frequently occur There are many delinquent o o o o o juveniles in my school Have you… Yes No Been severely teased or bantered in school during the last year o o Been threatened in school during the last year o o Been collectively bullied in school during the last year o o Been severely beaten in school during the last year o o Been robbed in school during the last year o o 42 Teachers: Next couple of sections in this assessment is about your perception of the certain classes and teachers in this school. Please read the questions carefully and respond to the best of your knowledge. Below are questions about your Romanian language class and teacher. Frequency of disruptions in Romanian language class. Every Most Some Never or lesson lessons lessons hardly ever Students don’t listen to what teacher’s say o o o o There is noise and disorder o o o o The teacher has to wait a long time for students o o o o to quiet down Students cannot work well o o o o Student’s don’t start working for a long time o o o o after the lesson begins Romanian teacher’s performance Every Most Some Never or lesson lessons lessons hardly ever The teacher shows an interest in every student’s o o o o learning The teacher gives extra help when students o o o o need it The teacher helps students with their learning o o o o The teacher continues teaching until students o o o o understand The teacher gives students an opportunity to o o o o express opinions Teacher Romanian class Strongly Agree Disagree Strongly Agree disagree Romanian class teacher treats all students o o o o respectfully Romanian class teacher uses positive language o o o o with students Romanian class teacher responds to students’ o o o o needs 43 Romanian class teacher does not exhibit gender o o o o bias and challenges gender stereotypes in the classroom Romanian class teacher sets clear behavioral o o o o expectations for classroom activities Romanian class teacher acknowledges positive o o o o student behavior Romanian class teacher redirects misbehavior o o o o and focuses on the expected behavior, rather than the undesired behavior Romanian class teacher gives students o o o o autonomy and provides students with choices Romanian class teacher provides students with o o o o opportunities to take on roles in the classroom Romanian class teacher encourages volunteer o o o o to participate in the classroom Romanian class teacher acknowledges students’ o o o o efforts Romanian class teacher has a positive attitude o o o o towards students’ challenges Romanian class teacher encourages goal setting o o o o among students Romanian class teacher promotes student o o o o collaboration through peer interaction Romanian class teacher promotes students’ o o o o interpersonal skills, such as perspective taking, empathizing, emotion regulation, and social problem solving Students in this class collaborate with one o o o o another through peer interaction Below are questions about your Mathematics class and teacher. Frequency of disruptions in Mathematics class. Every Most Some Never or lesson lessons lessons hardly ever Students don’t listen to what teacher’s say o o o o There is noise and disorder o o o o The teacher has to wait a long time for students o o o o to quiet down Students cannot work well o o o o Student’s don’t start working for a long time o o o o after the lesson begins 44 Math teacher’s performance Every Most Some Never or lesson lessons lessons hardly ever The teacher shows an interest in every student’s o o o o learning The teacher gives extra help when students o o o o need it The teacher helps students with their learning o o o o The teacher continues teaching until students o o o o understand The teacher gives students an opportunity to o o o o express opinions Teacher Math class Strongly Agree Disagree Strongly Agree disagree Math class teacher treats all students o o o o respectfully Math class teacher uses positive language with o o o o students Math class teacher responds to students’ needs o o o o Math class teacher does not exhibit gender bias o o o o and challenges gender stereotypes in the classroom Math class teacher sets clear behavioral o o o o expectations for classroom activities Math class teacher acknowledges positive o o o o student behavior Math class teacher redirects misbehavior and o o o o focuses on the expected behavior, rather than the undesired behavior Math class teacher gives students autonomy o o o o and provides students with choices Math class teacher provides students with o o o o opportunities to take on roles in the classroom Math class teacher encourages volunteer to o o o o participate in the classroom Math class teacher acknowledges students’ o o o o efforts Math class teacher has a positive attitude o o o o towards students’ challenges 45 Math class teacher encourages goal setting o o o o among students Math class teacher promotes student o o o o collaboration through peer interaction Math class teacher promotes students’ o o o o interpersonal skills, such as perspective taking, empathizing, emotion regulation, and social problem solving Students in this class collaborate with one o o o o another through peer interaction Home environment, career aspirations and parents: The next sections in this assessment is related your academic and career aspirations and relations with your parents. . Please read the questions carefully and respond to the best of your knowledge. How much education would you like to receive? o No more than 9th grade o Lyceum/professional school o Collegium o University o Post university Have you decided which occupation (including starting your own business) you want to pursue in the future? o Yes If “yes� or “I have some idea�, what is the occupation you want to o I have some idea pursue in the future? o No o [Insert occupation name] If “yes� or “I have some idea�, do you think you are in the right track to get that occupation? o Yes o No If “yes� or “I have some idea�, are you learning the right skills necessary to get that occupation? o Yes o No If “no�, do you lack information to make that decision? o Yes o No If “no� do you lack your parent’s support to make that decision? o Yes o No 46 How often do your parents do the following things at home with you? Never Once Once or Once or Every or or twice a twice a day or hardly twice a month week almost ever year every day Discuss how well I am doing at school o o o o o Eat the main meal (i.e., dinner) with me o o o o o around a table Spend time talking to me o o o o o Help me with my school work o o o o o (homework) Ask how I am performing in classes o o o o o Obtain school related materials and o o o o o support (e.g., applications, study guides, private tutoring etc.) for me Discuss how things I learn at school are o o o o o used in everyday life Discuss career options with me o o o o o Thinking about the last academic year (or semester), to what extent do you agree with the following statements? Strongly Disagree Neither Agree Strongly disagree agree or agree disagree My parents are interested in my school o o o o o activities My parents are supportive of my o o o o o efforts and achievements at school My parents support me when I face o o o o o difficulties at school My parents encourage me to be o o o o o confident Relations with parents Agree Somewhat Neither Somewhat Disagree agree agree or disagree disagree My parents always treat me with o o o o o love and affection 47 My parents and I understand each o o o o o other well My parents and I candidly talk o o o o o about everything I frequently talk about my o o o o o thoughts and what I experience away from home with my parents My parents and I have frequent o o o o o conversations When I go out, my parents usually o o o o o know where I am When I go out, my parents usually o o o o o know whom I am with When I go out, my parents usually o o o o o know what I am doing When I go out, my parents usually o o o o o know when I will be back home I frequently see my parents o o o o o verbally abuse each other I frequently see one of my parents o o o o o beat the other one I am often verbally abused by my o o o o o parents I am often severely beaten by my o o o o o parents This is the end of the first part of the questionnaire. Thank you for collaboration. Please proceed to the next part of the questionnaire. 48 Which Statements Describe Me Best? (BFI-2 Simplified Likert) Some kids like to meet new people. Other kids feel uncomfortable or nervous around people they have not met before. Below are several statements like that. Some of the statements may describe really well what you are like, whereas other statements may not describe you at all—in fact, they may be the exact opposite of you! So, for example, if you agree you like to meet new people, then you probably disagree that you are uncomfortable around people you’ve not met before. Take a look at the scale below. It has 5 steps and goes from 1 to 5. On the right-hand side are two numbers to say that you agree. If you agree strongly, then click the number “5�. If you agree only somewhat, click the number “4�. On the left-hand side, the scale has two number to say that you disagree. If you disagree strongly, click the number “1�. If you disagree only somewhat, click the number “2�. Finally, click the middle number “3� only when you are right in the middle—that is when you neither agree nor disagree with the statement. Circle only one number per statement. Disagree Disagree Neither Agree Agree Do you agree this statement describes you? strongly somewhat somewhat strongly 1 2 3 4 5 1 I am outgoing, I like to meet new people 1 2 3 4 5 2 I care for others, I do nice things for others all the time 1 2 3 4 5 3 I lose things and forget where things are 1 2 3 4 5 4 I am relaxed, I deal well with pressure and difficult situations 1 2 3 4 5 5 I am not interested in creative things, like drawing, poems, singing 1 2 3 4 5 6 I say what I believe, I won’t let others push me around 1 2 3 4 5 7 I treat everybody with respect 1 2 3 4 5 8 I do as little work as possible, I tend to be lazy 1 2 3 4 5 9 I stay positive even after a setback 1 2 3 4 5 10 I am interested in many different things 1 2 3 4 5 11 I don’t often get excited about things, I won’t “jump with joy� 1 2 3 4 5 12 I find problems with others, I point out their mistakes 1 2 3 4 5 13 I will do what I have promised, people can rely on what I say 1 2 3 4 5 14 I have up and down feelings, I go very quickly from feeling happy to very unhappy. 1 2 3 4 5 15 I am creative, I find clever ways to do things 1 2 3 4 5 16 I tend to be quiet 1 2 3 4 5 49 17 I am not sympathetic to others 1 2 3 4 5 18 I like things to be well-organized, in order 1 2 3 4 5 19 I can be tense and nervous 1 2 3 4 5 20 I am interested in art or literature (books, stories) 1 2 3 4 5 21 I am willing to go first, to be the leader of the group 1 2 3 4 5 22 I start arguments, fights with others 1 2 3 4 5 23 It takes a long time before I get started with a job or task 1 2 3 4 5 24 I feel ok with who they are, has self-confidence 1 2 3 4 5 25 I stay away from intellectual discussions about could or might be 1 2 3 4 5 26 I move slowly, I don’t like to do much 1 2 3 4 5 27 I forgive others that weren’t nice or did something wrong 1 2 3 4 5 28 I don’t pay close attention to details 1 2 3 4 5 29 I am calm, not easily upset 1 2 3 4 5 30 I do not feel creative 1 2 3 4 5 31 I tend to be shy around new people 1 2 3 4 5 32 I am helpful and generous with others 1 2 3 4 5 33 I keep my things neat and tidy 1 2 3 4 5 34 I worry a lot 1 2 3 4 5 35 I like art and things that please the eye 1 2 3 4 5 36 I don’t know how to get what they want 1 2 3 4 5 37 I am sometimes rude to others 1 2 3 4 5 38 I get things done quickly and correctly 1 2 3 4 5 39 I often feel sad 1 2 3 4 5 40 I enjoy thinking about problems that are difficult, complex 1 2 3 4 5 41 I have a lot of energy to do things 1 2 3 4 5 42 I don’t trust others, I think other people are not honest 1 2 3 4 5 43 I am always there when needed, reliable 1 2 3 4 5 44 I am able to keeps my feelings under control 1 2 3 4 5 45 I do not have an active imagination, I won’t think up stories or pictures 1 2 3 4 5 46 I talk a lot 1 2 3 4 5 47 I can be unfriendly and uncaring 1 2 3 4 5 48 I leave a mess, I don’t clean up 1 2 3 4 5 49 It takes a lot before I get worried or afraid 1 2 3 4 5 50 I think stories, poems, or words in songs are really boring. 1 2 3 4 5 51 I prefer to have others take charge 1 2 3 4 5 52 I am very polite to others 1 2 3 4 5 53 I work until a task is finished, I keep going 1 2 3 4 5 54 I tend to be in low spirits 1 2 3 4 5 50 55 I am not interested in theories, like whether there’s life on other planets 1 2 3 4 5 56 I show a lot of excitement, enthusiasm 1 2 3 4 5 57 I believe the best about people, that most people want to be nice 1 2 3 4 5 58 I sometimes do things I know I shouldn’t do; I am not always responsible 1 2 3 4 5 59 I get upset easily 1 2 3 4 5 60 I come up with new thoughts and ideas 1 2 3 4 5 51 Impressions of Others (Boys) (Anchoring Vignettes) This is the last and the shortest section in the survey. Almost done! Below there are descriptions of three kids and what they typically do. Please read each description, so you get an idea about what the three kids are like. Ion is kind and considerate to everyone he meets. He really cares about what others need or want. He shares and lends his things and takes good care of his friends when they have a problem. Bogdan thinks only about himself and what he needs or wants right now. He doesn’t care about the needs and feelings of other people. He bosses his friends around and tells them to do things for him. Sandu cares about others some of the time. At other times, he doesn’t want to think about what others might need or want. He feels it’s OK to take care of his own needs first. What do you think about Ion, Bogdan and Sandu? How are they different from each other? Which of these kids is helpful and generous with others, and which kid is not? What is your impression? Ion is kind and considerate to everyone he meets. He really cares about what others need or want. He shares and lends his things and takes good care of his friends when they have a problem. In the I disagree I disagree I agree I agree Ion is helpful middle somewhat strongly strongly (5) somewhat (4) and generous with others. (3) (2) (1) 52 Bogdan thinks only about himself and what he needs or wants right now. He doesn’t care about the needs and feelings of other people. He bosses his friends around and tells them to do things for him. In the I disagree I disagree Bogdan is helpful and generous with others. I agree I agree middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) Sandu cares about others some of the time. At other times, he doesn’t want to think about what others might need or want. He feels it is OK to take care of his own needs first. Sandu is helpful In the I disagree I disagree I agree I agree and generous with others. middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) How about yourself? In the I disagree I disagree I agree I agree I am helpful and generous with others. middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) 53 Here is one final question about Ion, Bogdan and Sandu. Remember: Ion is kind and considerate to everyone she meets. He really cares about what others need or want. He shares and lends his things and takes good care of his friends when they have a problem. Ion is unfriendly and uncaring to others. In the I disagree I disagree I agree I agree middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) Bogdan thinks only about himself and what he needs or wants right now. He doesn’t care about the needs and feelings of other people. He bosses his friends around and tells them to do things for him. Bogdan is unfriendly and uncaring to others. In the I disagree I disagree I agree I agree middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) Sandu cares about others some of the time. At other times, he doesn’t want to think about what others might need or want. He feels it is OK to take care of his own needs first. 54 Sandu is unfriendly and uncaring to others. In the I disagree I disagree I agree I agree middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) How about yourself? In the I disagree I disagree I am unfriendly and uncaring to others. I agree I agree middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) 55 Impressions of Others (Girls) (Anchoring Vignettes) This is the last and the shortest section in the survey. Almost done! Below there are descriptions of three kids and what they typically do. Please read each description, so you get an idea about what the three kids are like. Maria is kind and considerate to everyone she meets. She really cares about what others need or want. She shares and lends her things and takes good care of her friends when they have a problem. Ana thinks only about herself and what she needs or wants right now. She doesn’t care about the needs and feelings of other people. She bosses her friends around and tells them to do things for her. Elenei cares about others some of the time. At other times, she doesn’t want to think about what others might need or want. She feels it’s OK to take care of her own needs first. What do you think about Maria, Ana and Elenei? How are they different from each other? Which of these kids is helpful and generous with others, and which kid is not? What is your impression? Maria is kind and considerate to everyone she meets. She really cares about what others need or want. She shares and lends his things and takes good care of her friends when they have a problem. 56 In the I disagree I disagree I agree I agree Maria is helpful middle somewhat strongly strongly (5) somewhat (4) and generous with others. (3) (2) (1) 57 Ana thinks only about herself and what she needs or wants right now. She doesn’t care about the needs and feelings of other peop le. She bosses her friends around and tells them to do things for her. In the I disagree I disagree Ana is helpful and generous with others. I agree I agree middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) Elenei cares about others some of the time. At other times, she doesn’t want to think about what others might need or want. She feels it’s OK to take care of her own needs first. Elenei is helpful In the I disagree I disagree I agree I agree and generous with others. middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) How about yourself? In the I disagree I disagree I agree I agree I am helpful and generous with others. middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) 58 Here is one final question about Maria, Ana, and Elenei. Remember: Maria is kind and considerate to everyone she meets. She really cares about what others need or want. She shares and lends her things and takes good care of her friends when they have a problem. Maria is unfriendly and uncaring to others. In the I disagree I disagree I agree I agree middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) Ana thinks only about herself and what she needs or wants right now. She doesn’t care about the needs and feelings of other peop le. She bosses her friends around and tells them to do things for her. Ana is unfriendly and uncaring to others. In the I disagree I disagree I agree I agree middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) 59 Elenei cares about others some of the time. At other times, she doesn’t want to think about what others might need or want. She feels it’s OK to take care of her own needs first. Elenei is unfriendly and uncaring to others. In the I disagree I disagree I agree I agree middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) How about yourself? In the I disagree I disagree I am unfriendly and uncaring to others. I agree I agree middle somewhat strongly strongly (5) somewhat (4) (3) (2) (1) 60 Annex 3: Descriptive Statistics from the Moldova Socioemotional Skills Assessment N Mean Std.Dev Min Max Student Reported Female 2046 0.51 0.50 0 1 Household Size 2046 4.12 1.30 2 8 Number of Siblings 2046 1.50 1.11 0 6 Father Present in Household 2046 0.65 0.48 0 1 Mother’s Education Primary 2046 0.03 0.17 0 1 Gymnasium 2046 0.24 0.42 0 1 Lyceum 2046 0.12 0.32 0 1 Technical 2046 0.22 0.42 0 1 Bachelor 2046 0.17 0.38 0 1 Postgraduate 2046 0.05 0.22 0 1 Index of Learning Assets at Home 2046 0.00 1.68 -6.83 2.25 Index of Parental Support 2046 0.00 2.47 -14.01 2.29 Index of Family Hostility 2046 0.00 1.78 -4.68 6.89 Pedagogical Practices Romanian Teacher Supportive Learning Environment 2046 0.00 1.67 -7.23 1.74 Positive Behavioral Expectations 2046 0.00 1.36 -5.87 1.86 Autonomy 2046 0.00 1.41 -5.83 1.78 Perseverance 2046 0.00 1.47 -6.29 1.58 Social and Collaborative Skills 2046 0.00 1.45 -5.56 1.95 Pedagogical Practices Math Teacher Supportive Learning Environment 2046 0.00 1.75 -6.7 1.82 Positive Behavioral Expectations 2046 0.00 1.41 -5.72 1.82 Autonomy 2046 0.00 1.48 -5.54 1.77 Perseverance 2046 0.00 1.52 -5.79 1.62 Social and Collaborative Skills 2046 0.00 1.51 -5.06 1.9 Teacher Reported Has Extracurricular Programs 80 0.34 0.48 0 1 Class Size 80 21.32 6.26 11 37 Index of Class Disruption 80 -0.08 0.48 -.93 1.29 Index of School Violence 80 1.91 0.05 1.71 2 Half Time Teachers 80 0.39 0.27 0 1 Teacher Experience 80 14.64 1.48 10.46 17 61