101561 Hungary: Skilling up the next generation An analysis of Hungary’s performance in the Program for International Student Assessment Hungary: Skilling up the next generation An analysis of Hungary’s performance in the Program for International Student Assessment Contents Figure 10. Problem-solving scores and comparison with mathema cs scores, selected PISA countries, 2012 ... 22 Acknowledgments .................................................................................................................................................. 6 Figure 11. Equity performance: Hungary, neighboring countries, and OECD average, 2012 ............................... 24 Abbrevia ons and Acronyms.................................................................................................................................. 6 Figure 12. Evolu on of socioeconomic context for Hungarian students in 2009 - 2012 ...................................... 25 Execu ve Summary ................................................................................................................................................ 7 Figure 13. Performance differences between rural and urban schools, PISA countries, 2012 ............................. 27 Why skills ma er for Hungary .............................................................................................................................. 11 Figure 14. Within-school and between-school variance in mathema cs performance, Hungary and select PISA countries, 2012 ........................................................................................................................... 28 The importance of cogni ve skills ........................................................................................................................ 13 Figure 15. School social stra fica on and PISA performance in mathema cs, Hungary and select Hungary’s educa on system................................................................................................................................. 16 PISA-par cipa ng countries worldwide, 2012 ..................................................................................................... 29 Cogni ve skills of Hungarian 15-year-olds............................................................................................................ 19 Figure 16. PISA 2012 performance and socioeconomic composi on by school tracks, Hungary......................... 30 Snapshot of Hungary’s aggregate performance in PISA ....................................................................................... 20 Figure 17. Performance differences in PISA math scores between 2009 and 2012, by student achievement Performance and equity ....................................................................................................................................... 23 group and contribu ng factors ............................................................................................................................. 32 Performance and school type............................................................................................................................... 28 Figure 18. PISA 2012 mathema cs scores by socioeconomic status and school type.......................................... 32 Policy implica ons: Promo ng quality and equity ............................................................................................... 33 Figure 19. Age of first selec on between general and voca onal tracks, ............................................................ 35 Delaying the age of selec on into general and voca onal educa on tracks........................................................ 34 Figure A1. Share of under-18-year-olds at risk of poverty and social exclusion, Hungary and neighboring Raising the quality of educa on in voca onal and voca onal secondary schools ............................................... 36 countries, 2008 – 2013 ......................................................................................................................................... 43 Tackling socioeconomic disadvantage and its impact on learning ....................................................................... 36 References ............................................................................................................................................................ 39 Tables Table 1 Hungary’s public spending on educa on, 2004 - 2011 ............................................................................ 18 Boxes Table 2. ESCS evolu on between 2009 and 2012, decomposed by factors. ........................................................ 26 Box 1. PISA’s Index of Economic, Social, and Cultural Status ............................................................................... 25 Table A1. Percent of popula on aged 15+ by highest level of schooling a ained and average years of schooling in Hungary, 1990-2010 ..................................................................................................................... 43 Box 2. Roma Communi es in Hungary ................................................................................................................ 26 TableA2. Indices of learning strategies and teaching prac ces ............................................................................ 44 Box 3. The Complex Instruc on Program (CIP) in Hungary ................................................................................. 37 Table A3. Decomposi on of PISA Scores in Math between 2009 and 2012. ........................................................ 45 Figures Figure 1. Mathema cs scores in Hungary’s schools vary significantly by school type and their students’ socioeconomic background .................................................................................................................................... 9 Figure 2. Hungary’s income convergence: GDP (PPS) per capita, Hungary and neighboring countries, 1995 – 2013 .......................................................................................................................................................... 12 Figure 3. Age distribu on of Hungary’s popula on, 2010 and 2050 (projected) ................................................. 13 Figure 4. Three dimensions of skills ..................................................................................................................... 15 Figure 5. The structure of Hungary’s educa on system ....................................................................................... 17 Figure 6. PISA scores and public expenditures per student, selected countries worldwide, 2012....................... 18 Figure 7. PISA 2012 scores: Hungary, neighboring countries, and OECD average ................................................ 20 Figure 8. Summary of Hungary’s PISA scores, 2000 to 2012, including 2009-2012 performance change by achievement percen le....................................................................................................................... 21 Figure 9. Distribu on of students by PISA proficiency level in mathema cs, 2000 - 2012 .................................. 22 Acknowledgments This report was prepared by a World Bank team consis ng of Chris an Bodewig, Lucas Gortazar, Sandor Karacsony, Jeremie Amoroso, and Mar n Moreno, under the overall guidance of Mamta Murthi, Country Director, Central Europe and the Bal cs, and Cris an Aedo, Educa on Global Prac ce Manager. The note benefited from peer review comments from Juan Manuel Moreno. Marc De Francis edited the report. Abbrevia ons and Acronyms ESCS ECA Economic, Social, and Cultural Status Europe and Central Asia Executive Summary ECE Early childhood educa on EU European Union GDP Gross domes c product OECD Organiza on for Economic Co-opera on and Development OLS Ordinary least squares PIRLS Progress in Interna onal Reading Literacy Study PISA Programme for Interna onal Student Assessment RIF Re-centered influence func ons TIMSS Trends in Interna onal Mathema cs and Science Study UN United Na ons UNESCO United Na ons Educa onal, Scien fic and Cultural Organiza on VET Voca onal educa on and training page 8 page 9 Executive Summary Figure 1. Mathema cs scores in Hungary’s schools vary significantly by school type and their students’ socioeco- Facing the prospects of rapid aging and demographic decline over the coming decades, Hungary needs a highly skilled nomic background workforce to help generate the produc vity growth that it needs to con nue fueling a convergence of its living standards with those of its West European neighbors. Skilling up Hungary’s workforce should start by equipping youth with the right cogni ve and social-emo onal founda on skills. Interna onal research has iden fied three dimensions of skills that ma er for good employment outcomes and economic growth: cogni ve skills, such as literacy, numeracy, crea ve and cri cal thinking, and problem-solving; social-emo onal skills and behavioral traits, such as conscien ousness, grit, and openness to experience; and job- or occupa on-specific technical skills, such as the ability to work as an engineer. Cogni ve and social-emo onal skills forma on starts early in a person’s life. Good cogni ve and social-emo onal skills provide a necessary founda on for the subsequent acquisi on of technical skills. Put differently, poor literacy and numeracy skills severely undermine a person’s ability to benefit from further training and lifelong learning. Hungary can do significantly be er in preparing its next genera on with the right cogni ve founda on skills. This report focuses on cogni ve skills and examines results for Hungary from the Program for Interna onal Student Source: World Bank Staff es mates using PISA 2012 data. Notes: The 2012 PISA sample for Hungary contained 204 Schools. For the purposes Assessment (PISA), which assesses the mathema cs, reading, and science competencies of 15-year-olds. Although of this chart, basic educa on schools (49 schools with very few observa ons each) and schools with less than 12 students in secondary (7 schools) were removed, leaving 149 schools. An increase of 40 points is the equivalent to what an average student learns in a single school year. Hungary’s aggregate scores in these areas are largely on par with OECD averages, its mathema cs scores declined OECD averages of ESCS index and PISA mathema cs score are 0 and 500, respec vely. The ESCS school average is calculated by compu ng the between 2009 and 2012. Moreover, Hungary’s 15-year-olds perform poorly on problem-solving. weighted average of student’s ESCS at each school. Hungary’s educa on system is one of the most inequitable in the European Union (EU). Three findings stand out: This significant variance in performance is linked with three contribu ng causes: First, performance in PISA varies significantly by students’ socioeconomic backgrounds – differing by the equivalent  A large share of children and youth at risk of poverty or social exclusion; of more than three years of schooling between students from the top and bo om quin le, which is significantly more than elsewhere in the EU. Second, performance varies by type of school – general secondary, voca onal secondary  Significant social stra fica on of schools hand-in-hand with early ability-based selec on into general second- and voca onal schools. Third, 15-year-olds from socioeconomically disadvantaged background are dispropor onately ary, voca onal secondary and voca onal tracks; and represented in voca onal schools and voca onal secondary schools, where the quality is significantly below that of  Insufficient efforts to tackle inequity in learning condi ons faced by Hungarian students from an early age. general secondary schools. While PISA data does not allow for an analysis disaggregated by ethnic background of students, Roma youth are likely to be suffering dispropor onately from the inequi es in the educa on system due to How can the quality and inclusiveness of Hungary’s educa on system be raised? The dual challenge for Hungary is their socioeconomic disadvantages. to ensure that all students acquire basic competencies on the one hand and to promote con nued excellence at the top on the other. The examples of other countries such as Poland show that improvements in equity and excellence Figure 1 summarizes the evidence, in a single picture, of how students from different socioeconomic strata are can go hand in hand, and interna onal research has documented that these two goals, if achieved simultaneously, distributed into different educa onal tracks and achieve widely differing levels of cogni ve skills as measured by PISA promote economic growth. mathema cs scores, with gaps represen ng the equivalent of mul ple years of schooling. While Hungary’s mean scores in general secondary schools are on par with the best-performing countries par cipa ng in PISA, the scores in This report lays out a policy agenda consis ng of two parallel elements: First, improving socioeconomic condi ons for Hungarian voca onal schools are worse than the aggregate scores in the weakest par cipa ng countries in PISA. children and youth in general and in school through policies targeted to the poor and disadvantaged such as welfare and employment policies for parents and educa on support for children. Second, promo ng equity and reducing More than a quarter of Hungarian 15 year-olds perform at the bo om level in PISA’s mathema cs test. These socioeconomic segrega on in basic educa on through inclusive educa on policies. Hungary has been a leader in the students risk leaving school without the minimum literacy and numeracy skills needed to succeed in obtaining a EU in expanding access to early childhood educa on, including for children from disadvantaged backgrounds. It needs produc ve job, in subsequent training, and in lifelong learning. This may be one of the explana ons for why 15 to build on that solid founda on and promote greater equity and quality for all students in the rest of its compulsory percent of Hungarian 15- to 24-year-olds are idle and neither in employment, educa on, or training, a phenomenon educa on system if it wants to have the kind of well-skilled workforce it needs for con nued inclusive economic that an economy with an aging and shrinking popula on can ill afford. growth and subsequent convergence in living standards with the economies in Western Europe. page 11 Chapter 1 Why Skills Matter for Hungary page 12 page 13 Why skills matter for Hungary But what about skills? This report places a spotlight on the next genera on and asks if Hungary’s youth are leaving the Hungary faces the challenge of achieving convergence in living standards with its Western European neighbors at a compulsory educa on system with the right set of skills needed for further educa on and training and for produc ve me when its popula on is aging and shrinking. Hungary has seen remarkable income growth over the last two de- employment. It argues that there is considerable scope for Hungary to raise the skills of the next genera on and cades but s ll has a long way to go to catch up with EU15 living standards. In 1995, Hungary’s GDP per capita stood at prepare them be er for the demands of a growing and changing economy. about 45 percent of the EU15 average, and by 2013 it had risen to 65 percent (Figure 2). Hungary will need brisk eco- Figure 3. Age distribu on of Hungary’s popula on, 2010 and 2050 (projected) nomic growth over the coming decades if it is to deliver EU15 living standards for its popula on. However, Hungary’s long-term economic growth prospects are put at risk by demographic change: the country is projected to see a decline (by more than 10 percent between 2010 and 2050) and significant aging of its popula on (Figure 3). With fewer work- ers and more old-age dependents, labor produc vity improvements are key for con nued sustained economic growth. Figure 2. Hungary’s income convergence: GDP (PPS) per capita, Hungary and neighboring countries, 1995 – 2013 Source: World Bank staff es mates using UN Popula on Prospects. Medium variant. Source: World Bank Staff es mates using Eurostat data Making be er use of its human capital is at the heart of policies to accelerate Hungary’s convergence in living The importance of cognitive skills standards. Mi ga ng the risk to economic growth from popula on aging and shrinking will require expanding the Interna onal evidence shows how much the skills of a country’s workforce ma er for economic growth and shared number of workers; that is, increasing the employment rate at all ages, especially among young and older workers, prosperity. Interna onal evidence suggests that the quality of educa on is one of the most important determinants and encouraging immigra on. It also requires enhancing their produc vity, raising the skills of the current and future of long-term economic growth.1 Recent research (Hanushek and Woessmann, 2007 and 2012), drawing on student workforce, in addi on to other measures such as reforms in product and capital markets. assessment surveys from 1960 onward, es mates that an improvement of 50 points in scores in the Organisa on for According to Eurostat labor force survey data, Hungary’s employment rate for the 20-64 age group in 2014 was 66.7 Economic Co-opera on and Development (OECD) Program for Interna onal Student Assessment (PISA) would imply percent, below the European Union average. While this marks a significant improvement over previous years, there an increase of 1 percentage point in the annual growth rate of GDP per capita.2 is room for further increase. More worryingly, a significant number of young Hungarians are struggling with the Both the share of students achieving basic literacy and the share of top-performing students ma er for growth transi on from school to work and are idle. More than 15 percent of the 15-24-year-olds neither in employment, (Hanushek and Woessmann, 2007; OECD, 2010b). A recent OECD (2015) report presents economic returns to educa on or training (NEET), and recently the shares of early school leavers has increased as well, according to 1 See Sala-i–Mar n, Doppelhofer, and Miller (2004). Eurostat data. 2 See Hanushek and Woessmann (2007) and Hanushek (2010). Using these tests as measures of cogni ve skills of the popula on, they show that countries that had be er quality of educa on in the 1960s experienced faster economic growth during the years 1960-2000, controlling for other factors. page 14 page 15 universal basic skills, defined as all students achieving level 1 skills (420 points) in PISA. While low-income countries Figure 4. Three dimensions of skills with lagging educa on systems stand to gain the most, advanced middle-income and high-income countries can expect a significant boost for long-run economic growth simply from making their educa on systems deliver be er for the weakest students: The report finds that on average, high-income countries could gain a 3.5 percent higher discounted average GDP over the next 80 years if they were to ensure that all students achieved basic skills, defined as level 1 in PISA. As will be presented in this report, a significant and growing share of Hungarian 15-year-olds currently perform below level 1 of PISA. Ensuring universal basic skills in Hungary would add 4.1 percent discounted future GDP. Ensuring basic cogni ve skills for all also helps to make growth inclusive. Beyond aggregate economic growth, educa on improves the living standards of individuals, since the more educated are able to acquire more and higher- order skills, making them more produc ve and employable and extending their labor market par cipa on over their life me, which in turn leads to higher earnings and be er quality of life.3 Educa on is an engine of social mobility: Human capital is a key asset in income genera on and hence cri cal to reducing poverty and increasing shared prosperity (Bussolo and Lopez-Calva, 2014). “Skills” can be differen ated into separate, mutually reinforcing dimensions: cogni ve, social-emo onal, and technical. Figure 4 presents the differen a on across the different dimensions. CogniƟve skills include literacy and Source: Bodewig and Badiani-Magnusson (2014) numeracy, as measured in PISA, and also include competencies like cri cal thinking and problem-solving. Social- Cogni ve skills built in childhood and youth are a necessary founda on for successful acquisi on of technical emoƟonal skills capture one’s ability to interact with others as well as determina on and focus on ge ng a job done. and job-specific skills later in life. The founda ons of cogni ve (and behavioral) skills are formed early and are the Technical skills in turn capture one’s ability to perform technical tasks in any occupa on, such as work as a plumber pla orm upon which later skills are built. Skill forma on is cumula ve, benefi ng from previous investments, and or engineer. Measuring the level of educa onal a ainment does not automa cally mean measuring actual skills. the most sensi ve periods for building a par cular skill vary across the three dimensions. Technical and job-specific While many countries in Central and Eastern Europe have seen educa onal a ainment expand since the start of skills – o en acquired last, through technical and voca onal educa on and training (TVET), higher educa on and the economic transi on, as measured by years of educa on as well as level of educa on completed, they have not on-the-job learning – benefit from strong cogni ve and behavioral skills acquired earlier in the educa on system. In necessarily seen performance improvements in interna onal student assessments that measure cogni ve skills, such other words, the cogni ve skills acquired in childhood and youth, such as those measured by PISA, will help workers as PISA (Sondergaard and Murthi, 2012). to con nuously update their technical skills during their working lives. This is of par cular importance in aging economies such as Hungary’s, where workers need to adapt to technological progress during their longer working lives. This report focuses on cogni ve skills and examines evidence from the PISA assessment of mathema cs, reading, and science competencies of Hungarian 15-year-olds. Introduced in 2000 by the Organiza on for Economic Co- opera on and Development (OECD), PISA is a worldwide study of 15-year-old school students’ performance in three different disciplines: mathema cs, science, and reading. PISA focuses on the competence of students and their ability to tackle real-life problems in those three disciplines, emphasizing skills that are cri cal for individuals’ personal and professional development. Hungary has been par cipa ng in all PISA rounds since 2000. A sample ques on from the mathema cs assessment illustrates the applied nature of the PISA tests: “Nick wants to pave the rectangular pa o of his new house. The pa o has length 5.25 meters and width 3.00 meters. He needs 81 bricks per square meter. Calculate how many bricks Nick needs for the whole pa o.”4 In assessing the performance of 15-year-olds, the test largely captures those Hungarian students who are in any one track in upper secondary educa on and a small share 4 Addi onal sample ques ons can be found at Source: h p://pisa-sq.acer.edu.au/ 3 See Hanushek (2013). page 16 page 17 of students who are s ll in basic educa on. Given that skills forma on is cumula ve, the assessment results reflect Figure 5. The structure of Hungary’s educa on system not just competencies acquired in those schools but competencies acquired even earlier in students’ educa on. PISA’s scoring system is standardized so that the mean score for each discipline among OECD countries in year 2000 is 500 points, with a standard devia on of 100 points. According to OECD, a 40 point gain in PISA is equivalent to what students learn in one year of schooling.5 This report is part of a series of World Bank reports that examine PISA data in depth to analyze educa on systems and provide policy makers with op ons for evidence-based reforms. Due to its focus on policy, the series aims to address key challenges in several countries, with a focus on improving educa on quality and equity. This report provides a snapshot of the performance of Hungarian 15-year-olds in PISA both over me and compared with their peers in other countries. In analyzing Hungary’s performance in PISA, it analyzes the roles of (i) socioeconomic and family background characteris cs; (ii) school types and school segrega on; and (iii) learning strategies and teaching prac ces. It also offers policy recommenda ons on how Hungary can make its educa on system more equitable and be er prepare the next genera on for produc ve employment. Hungary’s education system Hungary’s educa on system is aligned with the skills forma on process, by emphasizing early childhood educa on and by focusing on cogni ve skills-oriented learning content in the beginning of the educa on system. The compulsory educa on system starts with kindergarten (ISCED 0). Hungary has a wide network of kindergartens, low associated costs (there are no tui on fees, and children pay only for meals and extracurricular ac vi es; meals are free for disadvantaged children), and condi onal cash transfers for families with mul ple disadvantages who enroll their children in preschool before the age of four and maintain stable a endance during the school year.6, 7 Since 2008, local governments have been required to offer free kindergarten placements to children from families with mul ple disadvantages from the age of three. Preschool a endance has been compulsory for children by age five, Source: ONISEP and since 2014 preschool educa on is compulsory from the age of three (this rule enters into force in the 2015–16 As elsewhere in Europe, in Hungary the educa on system introduces selec on into different educa onal school year due to capacity constraints). These policies mark a strong commitment to invest in skills forma on at an tracks at the start of upper secondary educa on. Upper secondary educa on (ISCED 3, typically for pupils ages early age. The results are impressive, with 95 percent of students par cipa ng in PISA 2012 repor ng that they had 14 to 18, usually covering grades 9 through 12) is provided in three parallel tracks: general secondary schools par cipated in two or more years of preschool. Primary and lower secondary educa on (ISCED 1 and 2) is organized (Gimnázium), voca onal secondary schools (Szakközépiskola), and voca onal schools (Szakiskola). General secondary as a single-structure system in eight-grade basic schools (typically for pupils ages 6 to 14, covering grades one through schools provide general educa on and prepare for the secondary school leaving examina on, the prerequisite for eight). admission to higher educa on. Secondary voca onal schools provide general and pre-voca onal educa on, prepare for the secondary school leaving examina on and offer voca onal post-secondary non-ter ary programs. Voca onal schools provide general, pre-voca onal and voca onal educa on, and may also provide remedial lower secondary general educa on for those who have not accomplished basic school. Figure 5 depicts the structure of the system.8 Hungary’s public investment in educa on (4.7 percent of GDP) is lower than the OECD average (5.3 percent of GDP for public educa on and 6.1 percent for total public and private spending). The investment in the sector seems to 5 PISA 2009 Technical Report (OECD 2012). have declined since the beginning of the economic crisis in 2008 (Table 1). The government expenditure per student 6 Disadvantaged children are those who are eligible for the regular child protec on allowance (rendszeres gyermekvédelmi támogatás); that is, those who 8 Higher educa on programs are offered by public or private universi es and colleges and follow the three-cycle Bologna degree structure (bachelor’s degree, come from families with income below 130 percent of the lowest pension benefit; single-parent families; and families with disabled children. master’s degree, doctoral degree) except for undivided long programs (10-12 semesters) in some disciplines, including medicine and law. Adult educa on and 7 At least six hours per day spent in kindergarten and a share of absences below 25 percent. training includes part- me general educa on programs at all ISCED levels, voca onal educa on, and a wide range of non-formal courses. page 18 page 19 has increased, in part due to the declining student popula on. The country’s performance is marginally above what would be expected given its current level of public expenditure per student, although it is similar to that of neighboring economies. Nonetheless, there are countries with similar levels of investment, like Poland and Latvia, that do have much higher cogni ve outcomes (Figure 6). Table 1. Hungary’s public spending on educa on, 2004 - 2011 2004 2005 2006 2007 2008 2009 2010 2011 Government expenditure on educa on As percent of 5.4 5.5 5.4 5.3 5.1 5.1 4.9 4.7 GDP As percent of total 11.1 10.9 10.4 10.4 10.4 9.9 9.8 9.4 government expenditure Government expenditure per student (PPP$) Primary educa on Secondary educa on 3,781.8 3,808.9 4,361.5 3,931.6 4,703.6 4,269 4,800.6 4,443.4 4,482.9 4,690.5 4,545.3 4,666.3 4,759.5 4,609.5 4,395.4 4,695.2 Chapter 2 Ter ary 3,943.4 4,050.1 4,377.7 4,593.4 5,059.1 5,817 5,351.3 6,450.5 Cognitive Skills of Hungarian educa on 15-year-old Students Source: UNESCO UIS, 2015. Figure 6. PISA scores and public expenditures per student, selected countries worldwide, 2012 Source: World Bank staff es ma ons using PISA 2012 data and UNESCO 2012 data. Note: The curve represents a logarithmic approxima on of the sca er plots. page 20 page 21 Cognitive skills of Hungarian 15-year-olds Figure 8. Summary of Hungary’s PISA scores, 2000 to 2012, including 2009-2012 performance change by achieve- ment percen le Snapshot of Hungary’s aggregate performance in PISA At first glance, Hungary’s 15-year-olds performed rela vely well in mathema cs, reading, and science in 2012. Hungary’s PISA 2012 scores were roughly on par with the OECD average and the average of countries in Central Europe and the Bal cs (see Figure 7). At the same me, Hungary could be doing be er: its 15-year-olds scored worse in PISA 2012 than several of their neighbors in the Visegrad9 group of countries and in the Bal c countries. Figure 7. PISA 2012 scores: Hungary, neighboring countries, and OECD average Source: World Bank staff es mates using PISA data. Hungary’s PISA performance in mathema cs and science has weakened of late. Hungary has seen a significant decrease in aggregate mathema cs and science scores, and a slight (but not sta s cally significant) decrease in the reading score in 2012 compared to 2009 (Figure 8, panel A.). The decline in performance was par cularly pronounced Source: World Bank staff es mates using PISA data. among students in the bo om and middle of the performance distribu on (between the 10th and 60th percen le; see The weakening of Hungary’s aggregate PISA scores in mathema cs and science was also associated with an Figure 8, panel B). At the same me, Hungary’s top PISA mathema cs, reading, and science performers in 2012 have increase in the share of func onally innumerate students. PISA categorizes scores in six levels of proficiency; remained at similar levels of performance compared to 2009. students who score below level 2 in the reading and mathema cs tests are considered func onally illiterate and innumerate, respec vely. This means that they are not able to understand and solve simple problems, severely limi ng their development and subsequent cogni ve and technical skill acquisi on process. As Figure 9 shows, despite its rela vely good performance overall, a significant share of students – more than a quarter of 15-year-olds – perform below level 2 in mathema cs. The concentra on of the performance decline between 2009 and 2012 among students in the middle and bo om part of the performance distribu on (Figure 8, Panel B) has meant a shi of students in PISA level 3 and 4 toward levels 2 and 1 in mathema cs. 9 The Visegrad group is an economic, energy, and military alliance of four na ons: Czech Republic, Hungary, Poland and Slovakia. page 22 page 23 Figure 9. Distribu on of students by PISA proficiency level in mathema cs, 2000 - 2012 Source: World Bank staff es mates using PISA 2000, 2003, 2006, 2009, and 2012 data. Hungarian 15-year-olds also performed considerably below their peers in the crea ve problem solving assessment. In 2012, the OECD introduced a new element of assessment: a problem solving category that measures students’ capacity to respond to non-rou ne analy cal problems in a digital environment in order to achieve their poten al as construc ve and reflec ve ci zens by making use of the new technological tools. Although Hungary performed Source: World Bank staff es ma ons using PISA 2012 data. on par with OECD averages in reading, mathema cs, and science, the results in the crea ve problem solving test were substan ally below those in comparator countries like Poland, the Slovak Republic, and the Czech Republic. Figure 10 presents the evidence on problem-solving (Panel A). It also shows the gap with respect to the mathema cs Performance and equity assessment and the influence of computer skills on rela ve performance in problem solving: the significant gap of Hungary’s students show large wealth-based dispari es in cogni ve skills. PISA results can be reviewed to assess 34 points between Hungary’s performance on this test and its performance on the paper-based mathema cs test equity in educa on systems and its rela on to socioeconomic status, because the assessment collects informa on not can be par ally a ributed (around 45 percent) to a lack of computer skills, reflec ng a gap in both digital literacy and only on student performance but also on student background (the OECD’s ESCS Index, see Box 1). In this report, two problem-solving skills (Panel B). measures are used to examine equity in educa on: (i) the strength of the rela onship between student performance and socioeconomic status, and (ii) the PISA score gap between top and bo om ESCS quin les. Figure 10. Problem-solving scores and comparison with mathema cs scores, selected PISA countries, 2012 Figure 11 summarizes the evidence from those two measures. Almost a quarter of the variance in performance in Hungary can be explained by ESCS, significantly more than among neighbors (Panel A). Moreover, the difference in PISA reading and mathema cs scores between the top and bo om ESCS quin les is around 120 points, the equivalent of three years of schooling (Panel B). This means that a student’s household background dispropor onately determines cogni ve skill acquisi on as measured in PISA. page 24 page 25 Figure 11. Equity performance: Hungary, neighboring countries, and OECD average, 2012 Socioeconomic condi ons among poorer students have worsened in recent years. Child and youth poverty is a significant and growing challenge for Hungary. According to the EU defini ons, 42.7 percent of Hungarians below the age of 16 were at risk of poverty and social exclusion in 2013 (see Figure A1 in the Annex). This represents the third highest youth poverty rate in the EU, a er Bulgaria and Romania, and it is significantly higher than in 2008 (when the rate was 33 percent). Consistent with this, the socioeconomic condi ons (as measured by the ESCS) faced by Hungarian 15-year-olds par cipa ng in PISA dropped significantly between 2009 and 2012 due to the effects of the economic and financial crisis that has hit Europe since 2008 (see Figure 12, le panel). This decline in has been concentrated among the most disadvantaged students (the bo om 40 percent of the socioeconomic distribu on), with no significant decline occurring among students at the high end of the distribu on (Figure 12, right panel). The decline in the ESCS index stems mainly from a change in the composi on of the occupa onal status of parents and a decline in certain home possessions, such as durable goods or space at home, as well as cultural goods or home resources for educa onal purposes (Table 2). Figure 12. Evolu on of socioeconomic context for Hungarian students in 2009 - 2012 Source: World Bank staff es mates using PISA 2012 data. Note: The Index of Equality of Opportunity is the percent of the variance in reading scores explained by the main predetermined socioeconomic characteris cs in a linear regression (Ferreira and Gignoux 2011). Item Response Theory (IRT) was used to derive the Index of Economic, Social, and Cultural Status (ESCS). The es mated parameters of this index may vary between countries, resul ng in different levels of reliability. Source: World Bank staff es mates using PISA 2009 and 2012 data. The ESCS index has been rescaled using common item ques ons for consistent comparisons across years. page 26 page 27 Table 2. ESCS evolu on between 2009 and 2012, decomposed by factors Figure 13. Performance differences between rural and urban schools, PISA countries, 2012 Components of Index of Home Possessions Years of Occupa onal Home Aggregate Educa on Cultural ESCS (index) Status (highest) Wealth Educa onal Home (highest) Possessions (index) Resources Possessions (index) (index) (index) 2009 11.738 47.635 −0.390 0.333 0.005 −0.101 −0.159 2012 12.036 46.137 −0.538 0.171 −0.096 −0.276 −0.253 Source: World Bank staff es mates using PISA 2009 and 2012 data. The Aggregate Home Possessions Index and its three components (Wealth, Cultural Possessions, and Home Educa onal Resources) have been rescaled using common item ques ons, for consistent comparisons across years. Roma households on average face significantly more difficult socioeconomic condi ons. Evidence from the 2011 EC/UNDP/World Bank Regional Roma Survey shows that Roma children in Eastern Europe, including in Hungary, face an even higher risk of poverty and social exclusion (World Bank, 2015) than their non-Roma neighbors (see Source: World Bank staff es mates using PISA 2012 data. Box 2). Recent quan ta ve studies inves ga ng the school performance of eighth graders in Hungary (not PISA) Performance in PISA 2012 also varies between students in urban and rural schools across all subjects and point to parental educa on and poverty as powerful transmission mechanisms for the inequali es experienced by between girls and boys in reading. In Hungary, around 19 percent of PISA par cipants live in rural areas, defined Roma adolescents (Kertesi and Kézdi , 2011; Kertesi and Kézdi, 2013). This evidence shows that the gap in test scores as geographical units with a popula on smaller than 15,000. The difference between urban and rural students is between Roma and non-Roma decreases drama cally once other socioeconomic characteris cs are controlled for, 58 points in mathema cs and 61 points in reading (equivalent to one and a half years of schooling), which is high becoming insignificant for reading and decreasing by 85 percent for mathema cs. compared to neighboring countries like Latvia, Romania, Germany, and Poland (where the difference is between 30 and 40 points; see Figure 13). Moreover, student performance gaps disaggregated by gender show that, while performance in mathema cs is similar for boys and girls, girls outperform boys in reading by 40 points (the equivalent of one year of schooling). Although this feature is comparable to other countries like the Slovak Republic or Poland, it reflects challenges ahead that need tailored policy interven ons. In par cular, even though this gap only shows absolute differences, a further decomposi on analysis shows that part of the gap can be explained by differences in learning strategies for acquiring reading skills (see Annex). page 28 page 29 Performance and school type PISA tend to have less socially segregated school systems, sugges ng significant room for a more inclusive policy There is more varia on in student performance in Hungary between schools than within schools. The varia on in agenda. performance between schools is a measure of how big the “school effects”10 are, and the results show that these ef- There is also an ethnic dimension to school social stra fica on, with ethnic segrega on of schools on the rise due fects are closely related to how students are allocated, or selected into schools. It can be argued that the higher the to a variety of factors. School choice and selec ve commu ng are among the most important mechanisms behind between-school variance, the more inequitable is the system. Between-school differences in mathema cs perfor- ethnic segrega on of Hungarian schools between Roma and non-Roma students. An analysis of long-run geographic mance in 2012 accounted for as much as 63 percent of the varia on in student performance in Hungary. This is sig- trends shows that ethnic segrega on between Hungarian schools increased substan ally between 1980 and 2011 nificantly above the OECD average and that of top-performing neighboring countries such as Estonia and Poland (see – with a break between 2006 and 2008 that coincided with the most intensive desegrega on campaigns – and that Figure 14). the size of the educa onal markets (defined as the number of schools) and the frac on of Roma students in the area Figure 14. Within-school and between-school variance in mathema cs performance, Hungary and select PISA being the strongest predictors of between-school segrega on (Kertesi and Kézdi, 2012). Further analysis reveals that countries, 2012 local educa onal policies that were in place un l recently have tended to exacerbate between-school segrega on, in addi on to the segrega on implied by student mobility (Kertesi and Kézdi, 2013). Figure 15. School social stra fica on and PISA performance in mathema cs, Hungary and select PISA-par cipa ng countries worldwide, 2012 Source: OECD (2014b) High performance stra fica on of lower secondary schools in Hungary goes hand in hand with high stra fica on in terms of socioeconomic background. The index of school social stra fica on is defined as the correla on between the PISA student’s socioeconomic status and the school’s average socioeconomic status. In a world without social stra fica on (and thus with an index equal to zero), families from different socioeconomic backgrounds would Source: World Bank staff es mates using PISA 2012 data. Note: PISA mathema cs scores on ver cal axis. Index of School Social Segrega on on randomly se le across the country and students from different backgrounds would study together, making schools as horizontal axis. The index ranges from 0 to 1. A higher index indicates a higher correla on between students’ and schools’ socioeconomic status. diverse as society as a whole. Although there is varia on by country, students usually a end school with peers who OECD mathema cs score average is 500 points. OECD average Index of School Social Segrega on is 0.525. have certain similari es in socioeconomic status. PISA data suggests that Hungary has one of the most socially stra fied secondary school systems in the EU, similar in stra fica on to countries in La n America, characterized by a large socioeconomic segrega on between schools (see Figure 15). This is especially striking in a country A closer look shows that PISA performance in Hungary varies not just between schools in general but also between with rela vely low income inequality.11 Interes ngly, Figure 15 also shows that the best performing countries in different types of schools. The performance of students in the general secondary schools (Gimnázium), which follow a purely academic track, is equivalent to the average of top PISA-performers like Switzerland, Japan, or Korea. The 10 According to the OECD, school effects are the effects on academic performance of a ending one school rather than another, with differences in performan- ce that usually reflect schools’ differences in resources or policies and ins tu onal characteris cs. See OECD (2013b) for a detailed descrip on. performance of students in voca onal secondary schools (Szakközépiskola) is at similar levels to the country average 11 Income inequality in Hungary, as measured by the GINI index, was 28.0 in 2013, below than the EU28 average of 30.5. For more informa on, see Eurostat and EU-SILC data. and between one and a half and two years behind the general secondary schools performance. These two tracks represent around 75 percent of PISA par cipa ng students in Hungary. The remaining 25 percent are in voca onal page 30 page 31 schools (Szakiskola) or s ll in basic schools, and their performance lags significantly behind:12 Compared to their The extent of decline in PISA scores between 2009 and 2012 also varied by school type. The performance of peers in voca onal secondary and general secondary schools, this third category of students is two years and three students in voca onal schools declined significantly across reading, mathema cs, and science (Figure 16, Panel and a half years behind in performance, respec vely (Figure 16, Panel A). C). The same was true for students in voca onal secondary schools in mathema cs and science but less so in reading. This is not surprising, given that the student body of voca onal and voca onal secondary schools is Moreover, the socioeconomic composi on of the student body varies significantly by school type. A closer look at dispropor onately composed of students from the bo om two ESCS quin les, and these students faced worse the socioeconomic composi on of students in tracks shows that while only 9 percent of students from the bo om socioeconomic condi ons in 2012 than in 2009. ESCS quin le a end general secondary schools, 72 percent of students from the top ESCS quin le do so. Although voca onal secondary schools offer a more heterogeneous composi on in socioeconomic status, they are mostly Mathema cs competencies were weaker in 2012 than in 2009 for students in all types of secondary schools, for populated by students from the bo om 40 percent of the ESCS distribu on (Figure 15, Panel B). example declining by more than 10 points for students in general secondary schools. The decline in mathema cs performance is par ally explained by changes in the composi on of students in schools, especially for the lowest Figure 16. PISA 2012 performance and socioeconomic composi on by school tracks, Hungary performing students. Econometric analysis shows that changes in peer characteris cs of students as well as the student composi on of school tracks (both aggregated as school environment in Figure 17) explain around 50 percent of the differences in performance for the average student between 2009 and 2012, and even more for poorly performing students. The overall message from the analysis is that educa on in Hungary does not act as an engine of social mobility but appears to further deepen socioeconomic disadvantage. Three key messages emerge from this analysis. First, performance in PISA varies significantly by the student’s socioeconomic background. Second, performance varies by type of school. Third, 15-year-olds from socioeconomically disadvantaged background are dispropor onately represented in voca onal schools and voca onal secondary schools or, indeed, are s ll in basic schools, where aggregateperformance is significantly behind that in general secondary schools. Figure 18 summarizes the above evidence in a single picture, showing how students from different socioeconomic strata distribute into different educa onal tracks and achieve widely differing levels of cogni ve skills as measured by PISA. Source: World Bank staff es mates using PISA 2012 data. 12 Among 15-year-old Hungarian students who took PISA 2012, 38 percent of students a ended grammar schools, 36 a ended voca onal secondary schools, 14 percent a ended voca onal schools, and 12 percent were s ll in primary (mostly due to either late entrance to school or repe on). page 32 page 33 Figure 17. Performance differences in PISA math scores between 2009 and 2012, by student achievement group and contribu ng factors Chapter 3 Policy Implications: Remaining Challenges Source: World Bank staff es mates using PISA 2012 data. Note: Results decomposi on was done using an Oaxaca-Blinder method on RIF- regressions for each quan le of the distribu on of performance (Firpo, For n, and Lemieux, 2009). Low, middle, and high achievers are students in the Polish Education System in the 20th, 50th, and 80th percen les, respec vely. By decomposing differences, one o en finds that one of the explanatory factors is nega ve or higher than the actual difference, meaning that other factors outweigh their impact. Figure 18. PISA 2012 mathema cs scores by socioeconomic status and school type Source: World Bank Staff es mates using PISA 2012 data. Notes: The 2012 PISA sample for Hungary contained 204 schools. For the purposes of this chart, basic educa on schools (49 schools with very few observa ons each) and schools with less than 12 students in secondary (7 schools) were removed, leaving 149 schools. 40 points is the equivalent to what an average students learns in a school year. OECD averages of ESCS index and PISA Math Score are 0 and 500, respec vely. The ESCS school average is calculated by compu ng the weighted average of students’ ESCS at each school. page 34 page 35 Policy implications: Promoting quality and equity students into elite general secondary schools for extended six- and eight-year programs star ng at ages 10 or 12. Hungary is facing the dual challenge of a rapidly aging and shrinking popula on and significant cogni ve skill short- Most countries with successful educa on systems stream students at later stages of schooling, usually at age 16. ages among youth. The most important response to ensure growth and convergence in mes of demographic decline Interna onal evidence suggests that early selec on is bad for equity and does not improve the overall quality is to raise produc vity and, as part of that, the skills base of the popula on. Yet Hungary is not doing as well as it of educa on. Hanushek and Woessmann (2006) used previous PISA data to show how early tracking systems lead needs to in equipping all its youth for employment and for longer working lives. Moreover, it is already experiencing to a systema c increase in inequality of student performance without affec ng average performance levels. At the nega ve consequences, evident in a rela vely high share of youth ages 15-24 who are not in employment, educa on, na onal level, similar evidence has been found in Poland (see Jakubowski et al., 2010) and Germany (Piopiunik, or training (15.4 percent in 2013, according to Eurostat data). While Hungary is not alone among its neighbors in fac- 2014). The findings suggest that there are no efficiency gains from introducing early selec on of students and that, in ing this challenge, these sta s cs are an early indica on that too many youth are leaving the educa on system with- fact, delayed tracking can promote be er performance for all students. out the necessary founda on skills to either find employment or con nue in educa on and training. This is exactly the kind of outcome an emerging economy with an aging and declining popula on should avoid. Figure 19. Age of first selec on between general and voca onal tracks, Hungary and select OECD countries Hungary needs to adopt policies that promote quality and address socioeconomic disadvantage at the same me. It is important to note that most students are in their first year of secondary educa on when they take the PISA test, conducted at age 15, so not all of their varying performance can be a ributed to the type of secondary school they are a ending. However, the analysis points toward the fact that, for reasons not captured by the PISA data, students in Hungary get selected into different tracks in such a way that enrolment and performance across secondary school types is stra fied along socioeconomic background. This suggests that factors related to school segrega on based on socioeconomic condi ons, even during basic educa on, do play a role in aggregate performance and the inequity in outcomes. Given these performance differences, improvements in socioeconomic status should improve student performance in Hungary. However, the strong rela onship between socioeconomic disadvantage and student performance also means Hungary needs to enhance quality equally across the en re educa on system – and not just among schools of one type – so that all students can acquire the cogni ve founda on skills needed for good employment outcomes. The examples of other countries in the EU such as Poland, Estonia, and Finland show that access to high quality educa on does not need to be limited to only a few students. Poland, Estonia and Finland combine high aggregate Source: OECD (2010a). PISA scores with a high degree of equity. Quality and equity can go hand in hand. This sec on lays out elements of a Recent analysis reveals that tracking has a causal effect on student performance in Hungary. Hermann (2013) policy agenda toward this dual goal, focusing on interven ons at the household and school levels targeted to children uses eight and tenth grade Na onal Assessment of Basic Competencies (NABC) data as well as administra ve data from poor socioeconomic backgrounds and measures to raise quality and equity across the educa on system. on admissions to upper-secondary ins tu ons in Hungary. He finds that significantly poorer test score results (0.21- Raising the quality of educa on for all can be achieved through two policy channels: (i) delaying the selec on of 0.28 standard devia on of test scores) in voca onal schools compared to secondary voca onal schools that can students between voca onal and general educa on tracks, and (ii) raising the quality of educa on in voca onal and be explained by the school type. He also finds smaller varia ons between general secondary and the voca onal voca onal secondary schools. secondary tracks--about half of the voca onal track effect. His analysis further shows that the impact of the academic track on performance does not differ consistently from the impact of be er schools within tracks. This suggests that if Delaying the age of selection into general and vocational education tracks the academic and mixed tracks were replaced by a single general track, the average achievement level and equality of The selec on into general and voca onal educa on tracks in Hungary happens at an earlier age than elsewhere opportunity would probably not change, as school choice and the selec on of students would reproduce the present in the region and across OECD countries. Figure 19 presents OECD data on the ages of first selec on into different stra fied school system with similar outcomes. Lastly, the analysis shows that equality of opportunity could be educa on tracks. Hungary stands out, alongside Austria, Germany, the Czech Republic and the Slovak Republic, as improved mostly by shrinking the voca onal track, even though some of the effect would be expected to be diluted the OECD members with the earliest age of selec on. While Hungary selects students into different secondary tracks by greater ability sor ng within the remaining two tracks. at age 14 or a er the end of basic educa on a er grade 8, the Hungarian system allows even earlier selec on of page 36 page 37 Interna onal experience, including Poland’s, shows that delaying the age of selec on between voca onal and general tracks can promote quality and equity. It is a reform that is highly relevant for Hungary. Early tracking in Hungary is resul ng in a high degree of stra fica on of students into different types of schools along socioeconomic lines. It channels a sizeable propor on of Hungarian students into voca onal and voca onal secondary schools, where quality is significantly lower on average than in general secondary schools. In the case of the voca onal school, it also represents an irreversible choice into a schooling track that is closed in the sense of specializing students early in a certain trade and not allowing students the possibility of proceeding toward higher educa on. Interna onal evidence suggests that delaying the selec on of students into voca onal and general tracks, for example un l a er the end of compulsory educa on, may be a first step toward achieving excellence and greater equity. The example of Poland is instruc ve: In 2000, Poland delayed the selec on into voca onal and general tracks by one year, un l the age of 15, thereby extending the exposure to general curriculum content as part of a comprehensive reform of secondary educa on, and it has seen significant improvements in aggregate PISA scores and in equity indicators since then (Jakubowski et al., 2010). Raising the quality of education in vocational and vocational secondary schools The performance of students in voca onal and voca onal secondary schools is below the performance of students in the general academic secondary schools. While some difference between the three groups may be expected, the gap in performance is very large and results in significant shares of young Hungarians remaining with poor and insuf- ficient cogni ve founda on skills. It raises the ques on whether the curriculum in voca onal schools and voca onal secondary schools devotes sufficient a en on to general cogni ve content and whether teachers are equipped with the tools to impart those skills effec vely to the students that need par cular support. This suggests the need to in- crease the quality of educa on in these two types of schools, even if, as recommended in this report, the age of selec- on between voca onal and general secondary schools is raised. Tackling socioeconomic disadvantage and its impact on learning Socioeconomic disadvantage has an impact on educa on performance in Hungary, all else equal. Consequently, raising the skills of the next genera on will involve efforts to improve the socioeconomic condi ons of children and youth. This will mean, first, household-targeted interven ons with an effec ve mix of social and employment services and benefits and tax incen ves to promote the employability of parents and address mul ple drivers of poverty and depriva on affec ng families with children. Second, it will mean measures targeted to schools with large shares of students from disadvantaged socioeconomic backgrounds to reduce the effect of socioeconomic disadvantage on student performance, for example through addi onal teacher and educa onal resources. Extracurricular ac vi es can provide compensatory learning s mula on for disadvantaged children. Hungary has a network of schools that follow a model developed in Hejőkeresztúr. This model is based on the Complex Instruc on Program (CIP) instruc on method from the United States, and offers higher quality educa onal services to children from disadvantaged backgrounds (See Box 3 for more details.). Addi onally, special incen ves could be offered to a ract the most talented teachers to work in hard-to-staff schools and preschools, for example based on the Teach for America/Teach for Bulgaria programs. page 38 page 39 References Annex page 40 page 41 References PIAAC.” Working Paper 199762, Na onal Bureau of Economic Research, Cambridge, Mass. Amermueller, A. 2004. “PISA: What Makes the Difference? Explaining the Gap in Pisa Test Scores Between Finland and Hermann, Z. 2013. “Are You on the Right Track? The Effect of Educa onal Tracks on Student Achievement in Upper- Germany.” Discussion Paper No. 04-004, ZEW Center for European Economic Research, Mannheim, Germany. Secondary Educa on in Hungary.” Budapest Working Papers on the Labor Market, BWP-2013/16. Autor, D., F. Levy, and R. Murnane. 2003. The Skill Content of Recent Technological Change: An Empirical Explora on. Jakubowski, M., H. A. Patrinos, E. E. Porta and J. Wisniewski. 2010. “The Impact of the 1999 Educa on Reform in Quarterly Journal of Economics, 118(4), November 2003, 1279-1334. Poland.” Policy Research Working Paper 5263, World Bank, Washington, DC. Barrera-Osorio, F., V. Garcia-Moreno, H. A. Patrinos, and E. Porta. 2011. “Using the Oaxaca-Blinder Decomposi on Kertesi, G., and G. Kézdi. 2011. “The Roma/Non-Roma Test Score Gap in Hungary.” American Economic Review 101 Technique to Analyze Learning Outcomes Changes Over Time: An Applica on to Indonesia’s Results in PISA (3): 519–25. Mathema cs.” Working Paper 5584, World Bank, Washington, DC. ________. 2012. “Ethnic Segrega on Between Hungarian Schools: Long-Run Trends and Geographic Distribu on.” Barro, R., and J. Lee. 2013. “A New Data Set of Educa onal A ainment in the World, 1950-2010.” Journal of Budapest Working Papers on the Labour Market, BWP 2012/8. h p://www.econ.core.hu/file/download/bwp/ Development Economics 104: 184–98. bwp1208.pdf Bodewig, C., and R. Badiani Magnusson. 2014. “Skilling Up Vietnam: Preparing the Workforce for a Modern Market Kertesi, G., and G. Kézdi. 2013. “School Segrega on, School Choice, and Educa onal Policies in 100 Hungarian Economy,” DirecƟons in Development. Washington, DC: World Bank. Towns.” Roma Educa on Fund. h p://www.romaeduca onfund.hu/sites/default/files/publica ons/school_ segrega onschool_choice_and_educa onal_policies_-_final_2013.pdf Bussolo, M., and L. F. Lopez-Calva. 2014. “Shared Prosperity: Paving the Way in Europe and Central Asia.” Washington, DC: The World Bank. OECD. 2010a. PISA 2009 Results: What Makes a School Successful? Vol. 4. Paris: OECD Publishing. Ferreira, H. G., and J. Gignoux. 2011. “The Measurement of Educa onal Inequality: Achievement and Opportunity.” ________. 2010b. The High Cost of Low EducaƟonal Performance: The Long-Run Impact of Improving PISA Outcomes. Discussion Paper 6161, IZA, Bonn. Paris: OECD Publishing. Retrieved May 12, 2015 from h p://www.oecd.org/pisa/44417824.pdf Firpo, S., N. For n, and T. Lemieux. 2009. “Uncondi onal Quan le Regressions.” Econometrica 7 (3): 953–73. ________. 2011. PISA 2009 Results: Learning to Learn. Volume 3. Paris: OECD Publishing. Hanushek, E. 2013. “Economic Growth In Developing Countries: The Role of Human Capital,” Economics of EducaƟon ________. 2012. PISA 2009 Technical Report. Paris: OECD Publishing. Retrieved April 10, 2014 from h p://www.oecd. Review 37: 204–12. org/pisa/pisaproducts/50036771.pdf ________. 2009. “School Policy: Implica ons of Recent Research for Human Capital Investments in South Asia and ________. 2014 (a). PISA 2012 Results: What Students Know and Can Do. Volume 1. Paris: OECD Publishing. Retrieved Other Developing Countries.” EducaƟon Economics 17 (3): 291–313. April 10, 2014 from h p://www.oecd.org/pisa/keyfindings/pisa-2012-results-volume-I.pdf ________. 2010. The High Cost of Low EducaƟonal Performance: The Long-Run Economic Impact of Improving PISA ________. 2014 (b). PISA 2012 Results: Excellence through Equity. Volume 2. Paris: OECD Publishing. Outcomes. Paris: OECD Publica ons. ________. 2014 (c). PISA 2012 Results: CreaƟve Problem Solving. Volume 5. Paris: OECD Publishing. Hanushek, E., and L. Woessmann. 2006. “Does Educa onal Tracking Affect Performance and Inequality? Differences- ________. 2015. Universal Basic Skills: What Countries Stand to Gain. Paris: OECD Publishing. in-Differences Evidence Across Countries.” The Economic Journal 116 (510): C63-C76. Piopiunik, M. 2013. The Effects of Early Tracking on Student Performance: Evidence from a School Reform in Bavaria, ________. 2007. “The Role of Educa on Quality in Economic Growth.” Working Paper 4122, World Bank, Washington, Ifo Working Paper No. 153 DC. Sala-i-Mar n, X., G. Doppelhofer, and R. I. Miller. 2004. “Determinants of Long-Term Growth: A Bayesian Averaging of ________. 2012. “Do Be er Schools Lead to More Growth? Cogni ve Skills, Economic Outcomes, and Causa on.” Classical Es mates (BACE) Approach.” American Economic Review 94 (4): 813 – 35. Journal of Economic Growth 17: 267–321. Sondergaard, L. and M. Murthi. 2011. Skills Not Just Diplomas. World Bank, Washington DC. Hanushek, E., G. Schwerdt, S. Wiederhold, L. Woessmann. 2013. “Returns to Skills Around the World: Evidence From page 42 page 43 United Na ons Educa onal, Scien fic, and Cultural Organiza on (UNESCO). 2015. Slovakia Country Profile. Montreal: Annex UNESCO Ins tute for Sta s cs (UIS). Figure A1. Share of under-18-year-olds at risk of poverty and social exclusion, Hungary and neighboring countries, 2008 – 2013 Willms, J. D. 2006. “Learning Divides: Ten Policy Ques ons about the Performance and Equity of Schools and Schooling Systems.” UNESCO Ins tute for Sta s cs Working Paper, Montreal. World Bank. 2011b. “Benchmarking School Autonomy and Accountability in Selected European Countries.” Washington, DC. ________. 2014. World Development Report 2015: Mind, Society, and Behavior. Washington, DC. ________. 2015. Equality of Opportunity: A Fair Chance for Marginalized Roma. Washington, DC. Source: World Bank Staff es mates using Eurostat data Table A1. Percent of popula on ages 15 and older, by highest level of schooling a ained and average years of schooling, Hungary, 1990-2010 Highest Year level Average years of schooling attained Primary Secondary Tertiary Total Primary Secondary Tertiary Total Completed Total Completed Total Completed 1990 56.4 35.6 33.7 18.0 8.8 7.5 8.79 7.08 1.38 0.33 1995 26.9 22.1 61.7 33.0 10.4 8.5 10.41 7.73 2.31 0.38 2000 10.5 9.8 77.8 44.0 11.0 9.4 11.20 7.92 2.88 0.41 2005 5.9 5.2 78.2 50.3 15.3 12.6 11.66 7.92 3.18 0.56 2010 2.0 1.8 80.6 52.9 17.2 15.4 11.85 7.69 3.51 0.65 Source: Barro-Lee Dataset (2012) page 44 page 45 Table A2. Indices of learning strategies and teaching prac ces (1) Learning Strategies Control How students set clear goals for themselves and monitor their own progress in reaching them Memoriza on To what extent students try to memorize texts where is an indicator func on and is the density of the marginal distribu on of scores. A crucial Elabora on How students relate acquired knowledge to other contexts (own life, characteris c of this technique is that it provides a simple way of in outside school, and prior knowledge) Metacogni on: Compares students’ strategies for understanding and remembering understanding and with what experts rate as the most appropriate strategies Table A3. Decomposi on of PISA scores in math between 2009 and 2012. remembering Metacogni on: Compares students’ strategies for summarizing with what experts rate Percen le Percen le Percen le Average summarizing as the most appropriate strategies VARIABLES 15 50 85 Teaching Prac ces Discipline, order, and What is the disciplinary climate in the classroom (e.g., noise and me me management taken for students to quiet down)? Year 2012 479.6*** 382.6*** 476.3*** 577.9*** Discussion and Extent to which teachers engage students in discussion (5.440) (2.337) (4.921) (4.669) debate Year 2009 491.0*** 398.3*** 495.1*** 583.1*** Rela ng knowledge Whether teachers help students relate knowledge to different contexts (5.729) (13.50) (7.293) (3.245) (prior knowledge, and personal experiences) Difference −11.42 −15.68 −18.84** −5.228 Clarifying Whether teachers outline how student-teacher interac on will be from (7.900) (13.70) (8.798) (5.686) expecta ons the beginning Unexplained −5.216 −6.304 −12.49** −1.419 Managing Whether teachers mark assignments, check if students understood the (3.565) (11.82) (4.864) (4.021) assignments lesson, and mo vate students Explained −6.208 −9.373 −6.358 −3.808 Quality of Educa onal Shortage or Science laboratory equipment, instruc onal materials (including (7.142) (16.00) (8.782) (3.274) Resources inadequacy of the textbooks), computers for instruc on, internet connec vity, computer Age 0.0656 −0.0230 0.0941 0.0378 following factors (as so ware for instruc on, library materials, and audio-visual resources reported by school (0.0657) (0.144) (0.0992) (0.0477) principals) Female −0.504 −0.780 −0.601 −0.241 (0.577) (0.912) (0.692) (0.278) Note: Indices were constructed by World Bank staff based on PISA 2009. See OECD 2014, “PISA 2009 Results: Learning to Learn – Student Engagement, Strategies and Prac ces (Volume 3)” for more details on the indices. ESCS Index −0.737 −0.280 −0.754 −0.715 (0.555) (0.545) (0.599) (0.540) The analy cal approach used in the third sec on of this report is based on the Firpo, For n, and Lemieux (2009) Entrance Age Primary 0.657** 1.563** 0.771** 0.298** methodology. Typically, the literature on decomposi on of student scores in PISA through groups (Amermueller (0.258) (0.698) (0.329) (0.142) 2004) and years (Barrera et al. 2011) has focused on the mean differences, with li le a en on to what happens at ESCS Index (School) −5.292 −7.470 −5.511 -3.317 the tails of the distribu on. The Firpo, For n, and Lemieux (FFL) method allows one to decompose gaps in student (3.933) (5.941) (4.157) (2.540) performance not only for the mean but also for other sta s cs of the distribu on. Tradi onally, the problem with Rural 0.0470 0.278 0.00411 −0.00436 quan le regressions has been that the law of iterated expecta ons does not apply, thus making it impossible to (0.506) (2.991) (0.0611) (0.0526) interpret the uncondi onal marginal effect of each independent variable on a student’s performance. However, Quality of Educa onal Resources −0.141 −0.311 −0.168 −0.0610 recent econometric techniques, such as the one proposed by FFL, have solved this methodological difficulty. The (0.429) (0.964) (0.517) (0.202) FFL technique is based on the construc on of re-centered influence func ons (RIF) of a quan le of interest, , as a Voca onal School 0.0557 0.536 −0.0549 −0.0374 dependent variable in a regression: (0.494) (4.745) (0.488) (0.332) Voca onal Secondary −1.118 −5.791 −1.096 0.274 (3.424) (17.73) (3.359) (0.847) page 46 page 47 Grammar School 0.758 2.905 0.957 −0.0417 (4.762) (18.25) (6.013) (0.280) Constant −29.98 31.95 −54.33 −182.3** (85.98) (268.5) (127.5) (85.19) Observa ons 9,118 9,118 9,118 9,118 Robust standard errors in parentheses *** p<0.01, ** p<0.05, * p<0.1 Notes: Robust standard error in parentheses and clustered at the school level. *** p < 0.01, **p < 0.05, and *p < 0.1. Variable effects are grouped and include individual characteris cs (age, gender, entrance age, and socioeconomic status), school environment characteris cs (socioeconomic status of peers at school, rural/urban area, or type of stream of school), and quality of school resources.