91010 The Skills Road Skills for Employability in Uzbekistan Mohamed Ihsan Ajwad, Ilhom Abdulloev, Robin Audy, Stefan Hut, Joost de Laat, Igor Kheyfets, Jennica Larrison, Zlatko Nikoloski, and Federico Torracchi The Skills Road Skills for Employability in Uzbekistan Mohamed Ihsan Ajwad, Ilhom Abdulloev, Robin Audy, Stefan Hut, Joost de Laat, Igor Kheyfets, Jennica Larrison, Zlatko Nikoloski, and Federico Torracchi Standard Disclaimer: . This volume is a product of the staff of the International Bank for Reconstruction and Development/ The World Bank. The findings, interpretations, and conclusions expressed in this paper do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. . Copyright Statement: . The material in this publication is copyrighted. Copying and/or transmitting portions or all of this work without permission may be a violation of applicable law. The International Bank for Reconstruction and Development/ The World Bank encourages dissemination of its work and will normally grant permission to reproduce portions of the work promptly. For permission to photocopy or reprint any part of this work, please send a request with complete information to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, USA, telephone 978-750-8400, fax 978- 750-4470, http://www.copyright.com/. All other queries on rights and licenses, including subsidiary rights, should be addressed to the Office of the Publisher, The World Bank, 1818 H Street NW, Washington, DC 20433, USA, fax 202-522-2422, e-mail pubrights@worldbank.org. Suggested citation: Ajwad, Mohamed Ihsan, Ilhom Abdulloev, Robin Audy, Stefan Hut, Joost de Laat, Igor Kheyfets, Jennica Larrison, Zlatko Nikoloski, and Federico Torracchi. 2014. “The Skills Road: Skills for Employability in Uzbekistan.” World Bank. Washington, DC. i Table of Contents List of Figures..................................................................................................................................................................... iii Abbreviations and Acronyms ........................................................................................................................................... v Acknowledgements............................................................................................................................................................ vi Overview .............................................................................................................................................................................. 1 1 Country Context .......................................................................................................................................................... 4 2 Labor Market Outcomes ............................................................................................................................................ 5 2.1 Job creation has kept pace with population growth.................................................................................................. 5 2.2 Jobs are distributed unevenly and labor market discouragement is high ................................................................... 7 2.3 The labor market has undergone significant transformations................................................................................. 10 2.4 Job quality remains a significant concern for policy makers ................................................................................... 12 2.5 Labor market information systems are weak, hindering the job search and skills signaling process ........................ 15 3 The Demand for Skills.............................................................................................................................................. 17 3.1 Employers demand skilled workers, but have difficulty finding them ..................................................................... 18 3.2 Skills enhance employment outcomes, even when other factors are held constant ..................................................... 24 3.3 Higher skilled workers use their skills more often in the workplace ...................................................................... 29 4 Skill Formation over the Life Cycle ........................................................................................................................ 32 4.1 Skills are developed throughout the life cycle of an individual ................................................................................ 32 4.2 Access to general education is good, but preschool and tertiary school coverage is low .............................................. 33 4.3 The correlation between educational attainment and cognitive skills outcomes is weak, and there is significant variation within education levels........................................................................................................................... 35 4.4 There is a weak correlation between educational attainment and non-cognitive skills ............................................. 37 5 The Skills Roadmap in Uzbekistan ......................................................................................................................... 39 5.1 Get children off to the right start by expanding access to quality early childhood development programs .................. 41 5.2 Ensure that all students learn by modernizing the curricula and improving teaching quality .................................. 41 5.3 Build job-relevant skills that employers demand by implementing selective active labor market programs ................ 42 5.4 Encourage entrepreneurship and innovation by increasing tertiary education access ................................................ 43 5.5 Match the supply of skills with employer demand by improving labor market information systems ......................... 44 References .......................................................................................................................................................................... 46 Appendix A: Questionnaire Sections............................................................................................................................. 53 Appendix B: Constructing Cognitive Skills Scores Methods for Scale Development and Scoring .................... 54 Appendix C: Constructing Non-Cognitive Skills Scores Methods for Scale Development and Scoring ........... 62 Appendix D: Summary Tables........................................................................................................................................ 70 Appendix E: Cognitive and Non-cognitive Skill Mean Scores.................................................................................. 78 ii List of Figures Figure 1: Uzbekistan’s GDP growth has been stronger than most ECA and OECD countries, 1996–2012 . 5 Figure 2: Productivity has grown considerably but continues to lag behind comparator countries, 2000– 2011 .................................................................................................................................................................. 6 Figure 3: Favorable demographics present a window of opportunity for increased economic growth in Uzbekistan, 2013 ............................................................................................................................................ 7 Figure 4: Male employment rates mirror those in OECD countries, but a much smaller share of women is employed in Uzbekistan compared to the OECD countries, 2013 ....................................................... 8 Figure 5: Labor market discouragement is high among young men and women, as well as older men, 2013 8 Figure 6: International migration rates are high among young men in Uzbekistan, 2013 .................................. 9 Figure 7: The share of agriculture in GDP has decreased, while the share of value added in services and industry has grown, 1995–2012 ................................................................................................................. 10 Figure 8: The services sector employs more than half of all workers in Uzbekistan, 2013 .............................. 11 Figure 9: State-owned enterprises dominate the labor market and employ roughly twice as many persons as the private sector, 2013 ............................................................................................................................... 11 Figure 10: Self-employment is dominated by micro-businesses in the services sector, 2013............................. 12 Figure 11: Informal salaried work is common in agriculture and industry jobs, as well as among lower educated individuals, 2013 .......................................................................................................................... 13 Figure 12: High shares of physical work and repetitive labor, 2013 ....................................................................... 14 Figure 13: Use of computers at work is relatively low in Uzbekistan, but younger workers in the public sector are more likely to use a computer on the job, 2013 ................................................................... 15 Figure 14: The majority of individuals face constraints to finding a job, 2013..................................................... 16 Figure 15: A large share of firms report difficulties in hiring sufficient numbers of qualified specialists with higher education, 2013 ................................................................................................................................ 18 Figure 16: Language and non-cognitive skills are reported by employers to be most lacking in workers in Uzbekistan, 2013 .......................................................................................................................................... 19 Figure 17: Tertiary graduates and secondary special/technical graduates have better labor market outcomes than individuals with a secondary general or less than secondary education, 2013 .......................... 20 Figure 18: Employment outcomes are positively correlated with preschool attendance as a child, 2013........ 21 Figure 19: There are considerable average returns to tertiary education among salaried workers aged 25–64, circa 2009....................................................................................................................................................... 22 Figure 20: Tertiary graduates report that their education is useful for work, while less educated workers report less satisfaction, 2013 ...................................................................................................................... 22 Figure 21: There is a mismatch between jobs created and fields in which students are graduating.................. 23 Figure 22: Employed people exhibit better cognitive and non-cognitive skills than inactive people and those who are discouraged, 2013 ......................................................................................................................... 25 Figure 23: Cognitive and non-cognitive skills are generally stronger among public sector workers................. 25 Figure 24: Formal sector workers have better memory skills, but other cognitive or non-cognitive skills are statistically similar, 2013 .............................................................................................................................. 26 Figure 25: Cognitive and non-cognitive skills are generally better in young compared to older workers, especially among the inactive population 2013 ....................................................................................... 27 Figure 26: Adults with migration intentions on average have significantly higher cognitive and non-cognitive skills than adults without migration intentions, 2013............................................................................. 29 Figure 27: Returned migrants on average have significantly higher cognitive and non-cognitive skills than non-migrants, 2013 ...................................................................................................................................... 29 iii Figure 28: Workers with cognitive skills use those skills and are less likely to perform physical work ............ 30 Figure 29: Workers who are more open and sociable tend to have more contact with clients, customers, and colleagues, 2013 ............................................................................................................................................ 31 Figure 30: Skills are formed in all stages of life—very stylized ............................................................................... 33 Figure 31: While Uzbekistan’s primary and secondary enrollments are relatively high, preschool and tertiary enrollments fall short, 2011 ........................................................................................................................ 34 Figure 32: Tertiary education is accessible mostly to better off families, 2013..................................................... 34 Figure 33: Few Eurasian firms offer formal training programs to full-time employees...................................... 35 Figure 34: While cognitive skills are generally correlated with educational attainment among women, they are not as clearly correlated among men, 2013 .............................................................................................. 36 Figure 35: Cognitive skills distributions overlap despite educational attainment, 2013 ...................................... 37 Figure 36: Non-cognitive skills are not significantly different by education level, 2013 ..................................... 38 Figure 37: Actors that play a role to build skills throughout the life cycle of an individual ................................ 39 Figure 38: Policy reform priorities to boost skill outcomes of the current and future workforce in Uzbekistan..................................................................................................................................................... 40 iv Abbreviations and Acronyms ALMP Active Labor Market Programs BEEPS Business Environment and Enterprise Performance Surveys BNPP Bank Netherlands Partnership Program ECD Early Childhood Development EDS Education Development Strategy ETF European Training Foundation GDP Gross Domestic Product GNI Gross National Income GIZ German Society for International Cooperation ILO International Labor Organization LMP Labor Market Program OECD Organization for Economic Co-operation and Development OJT On-the-Job Training PISA Program for International Student Assessment SMS Short Message Service SOE State-Owned Enterprise STEP Skills toward Employment and Productivity WDR World Development Report v Acknowledgements The Skills Road: Skills for Employability in Uzbekistan has been prepared by a core team led by Mohamed Ihsan Ajwad and comprising Ilhom Abdulloev, Robin Audy, Stefan Hut, Joost de Laat (former task team leader), Igor Kheyfets, Jennica Larrison, Zlatko Nikoloski, and Federico Torracchi. Dariga Chukmaitova and Alexander Danzer contributed significantly. The World Bank team worked under the supervision and guidance of Omar Arias. The survey was implemented jointly by a team from the World Bank and the German Society for International Cooperation (GIZ). The project was conducted with generous support from the Bank Netherlands Partnership Program (BNPP) for the “Skills for Competitiveness and Growth: The Challenge of Labor Exporting Countries” Initiative, led by Joost de Laat. The cognitive and non-cognitive skills scores were constructed by Carly Tubbs (Consultant, New York University) and Louise Bahry (Consultant, University of Massachusetts Amherst). The team has collaborated closely with a World Bank team administering a similar survey in Bulgaria, comprising Silvia Guallar Artal, Victoria Levin, Alessandra Marini, and Abla Safir. The team is grateful for the insightful comments, support, and technical inputs of Jishnu Das, Tatyana Shin, Naveed Hassan Naqvi, and Janssen Teixeira. Administrative and logistical support was provided by Zinaida Korableva and in the World Bank’s Uzbekistan office by Iqboljon Ahadjonov. Shivanthi Gunasekera designed the cover page. Guadalupe Paz edited the report. The peer reviewers are: Elena Glinskaya, Edmundo Murrugarra, and María Laura Sánchez Puerta. The team is particularly grateful for assistance from the regional and country management teams including from: Laura Tuck, Saroj Kumar Jha, and Takuya Kamata. vi Overview The government of Uzbekistan has set itself an ambitious development agenda known as the Vision 2030 Strategy. Under the Strategy, the government intends to accelerate economic growth to lift its 2012 gross national income per capita from US$1,700 to more than US$4,000 by 2030, which is the income of an upper- middle-income country. The government’s strategy to achieve these goals in a sustainable fashion—ultimately aiming for an expansion of the middle class, the promotion of shared prosperity, and the elimination of poverty—has hinged on creating quality jobs for its citizens. This report contributes to Uzbekistan’s Vision 2030 Strategy by offering a policy-relevant assessment of the impact that skills gaps have on employment outcomes. The report presents findings of a multi-year project involving a large World Bank team and a group of researchers in Uzbekistan. This report builds on existing labor market studies and makes an additional contribution by measuring and analyzing various types of skills in the working-age population. 1 The study on worker skills is the first of its kind in Uzbekistan to go beyond the traditional data on educational attainment. More specifically, large-scale assessments of cognitive and non-cognitive skills of workers in both the formal and informal sectors, of job seekers, and of those who are inactive by testing and interviewing respondents is a relatively rare occurrence in middle- and low-income countries, though OECD countries tend to conduct these assessments more frequently. Data for this report draws primarily from an innovative survey on jobs, skills, and migration of citizens in Uzbekistan. The survey was developed specifically for this study and was conducted jointly by the German Society for International Cooperation (GIZ) and the World Bank in 2013 (see Box 1). The study introduces international benchmarks where relevant, a unique feature in the case of Uzbekistan, where the lack of data has historically limited the comparability of labor market indicators. The main finding of the report is that worker skills gaps are hindering employment outcomes in Uzbekistan. In fact, beyond worker characteristics and educational attainment, Uzbek employers—particularly formal sector employers—seek workers who possess both cognitive and non-cognitive skills. The higher employability and higher wage rates among higher skilled workers is mostly explained by the use of those skills in the workplaces. But, despite the higher employability and higher wage rates among higher skilled workers, skills gaps persist in Uzbekistan. Inactive and discouraged individuals have significantly lower cognitive and non-cognitive skills than employed individuals. And, a large share of employers report shortages of high-skilled workers. The above finding on skills gaps hindering employment outcomes is important as policy makers contend with the labor market challenges facing the population. While Uzbekistan has been able to maintain an overall job creation rate that is fast enough to keep pace with population growth, the achievement has disappointed critics who point to the insufficient job growth in the formal sector. The task of creating quality jobs is formidable because of certain characteristics of the Uzbek labor market. Two stand out: an uneven distribution of jobs, where women are particularly underrepresented among the employed; and the fact that youth are more discouraged—the phenomenon whereby people who are willing to work leave the labor force because they feel that there are no jobs available—than in other countries. 1 Arias et al. (2014), Sondergaard and Murthi (2012), Gill et al. (2014), World Bank (2012), and World Bank (forthcoming). 1 Among the employed, job quality—which is a multidimensional concept that includes earnings, workplace safety, job security, learning and advancement opportunities, and health and social protection benefits, mental and physical health, etc.—is of particular concern given that more than half of Uzbekistan’s workers are employed in the informal sector, and most workers do not learn new things on the job, performing predominantly repetitive tasks. Compounding these problems are the challenges stemming from labor force misallocations resulting from weak labor market information systems. These misallocations lead to poor job placement and skills signaling, but ultimately to low productivity and economic growth. As discussed earlier, this report is unique because of its analysis of skills among the working age population in Uzbekistan. The report defines worker skills as cognitive, non-cognitive (social and behavioral), and technical skills, and focuses on the first two. Cognitive skills capture the ability to use logical, intuitive, and critical thinking as well as skills such as problem solving, verbal ability, and numeracy. These skills are the basis for the formation of technical and job-specific skill acquisition later in life. The cognitive skills measured in this report include memory, literacy, and numeracy skills. Non-cognitive skills represent personality traits and socio-emotional skills that are relevant in the labor market, including extraversion, conscientiousness, openness to experience, agreeability, and emotional stability. This study measures the following non-cognitive skills: openness/sociability, workplace attitude, decision making, achievement striving, and growth mindset. The global demand for skills is shifting from routine, manual and cognitive skills toward more non-routine, higher-order skills, including socioemotional (“soft”) skills. However, the education system in Uzbekistan has a mixed track record of imparting the type of cognitive and non-cognitive skills that are increasingly demanded by employers. For women, higher educational attainment levels are generally associated with higher cognitive skills, but there is no relationship between female educational attainment and non-cognitive skills. To put it bluntly, women do not gain non-cognitive skills from additional levels of schooling. For men, there is no relationship between educational attainment and either cognitive or non-cognitive skills. While the reason for this lack of association between skills and educational attainment requires further work, the study raises questions about the admissions, curricula, and graduation process, especially at the tertiary level. Furthermore, there is considerable variation in skills scores for a given educational attainment level, which raises questions about the quality of the education system more generally and its ability to deliver on labor market-relevant skills. This report offers a framework that can be helpful to Uzbekistan in light of budget and capacity constraints to up-skill the current and future workforce. The policy goals can be informed by the Skills Toward Employability and Productivity (STEP) Framework, which brings together research-based evidence and practical experience from diverse areas—from research on the determinants of early childhood development and learning outcomes to policy experience in the reforming of vocational and technical education systems and labor markets. 2 This report recommends adopting five policy goals to improve the skills of the current and future workforce in Uzbekistan:  Getting children off to the right start by expanding access to quality early childhood development (ECD) programs, which are critical to ensuring that all children acquire the cognitive and non- cognitive skills that are conducive to high productivity and flexibility that are observed later in working life. 2 Valerio et al. (2014). 2  Ensuring that all students learn by modernizing the curricula and improving teaching quality in order to address the weak link between educational attainment and cognitive and non-cognitive skills.  Building job-relevant skills that employers demand by implementing selective active labor market programs, with a particular focus on discouraged workers and on increasing the female labor force participation, and incentivizing firms to provide on-the-job training to workers.  Encouraging entrepreneurship and innovation by increasing quality tertiary education access for motivated students, which can ensure that higher education graduates possess market-valued skills and that investments in higher education pay off.  Matching the supply of skills with employer demand by improving labor market information systems, which can help to make labor markets more efficient by improving the flow of information between job seekers and employers and by helping to secure jobs through job signaling. Box 1: World Bank/GIZ Uzbekistan Jobs, Skills, and Migration Survey (2013) The World Bank/German Society for International Cooperation (GIZ) Uzbekistan Jobs, Skills, and Migration Survey is one of three identical household surveys conducted in Central Asia in 2013—the other countries covered are the Kyrgyz Republic and Tajikistan.+ Conducted from July to September 2013, the survey collects comprehensive information not typically captured by traditional household surveys and is representative at the national, regional (Oblast), and urban/rural levels. Two distinct instruments are employed in the survey: a core questionnaire and a skills questionnaire. The sample size of the core questionnaire is 1,500 households with a total of 8,622 individuals. One individual per household was randomly selected to partake in the skills questionnaire. This second skills questionnaire sample thus consists of 1,500 individuals. Qualitative testing and pre-pilots helped fine-tune the questionnaires and organize the modules in order to administer the survey efficiently and consistently. 1. Core questionnaire* The core questionnaire contains modules focusing on the following topics: education, employment, migration, health expenditure, remittances, government transfers, financial services, subjective poverty, housing conditions, and household expenditures. The core questionnaire concludes with the random selection of a household member aged 15 to 64 who is not a current migrant (the selection is based on a random number table or Kish grid) to be the subject for the skills questionnaire. 2. Skills questionnaire* The skills questionnaire contains detailed modules on labor and work expectations, migration and preparation for migration, language skills, and technical skill training. A unique aspect of the survey is the battery of cognitive and non- cognitive questions which help to test a respondent’s ability. The cognitive skills module is based on a recent instrument developed for a similar survey in Bulgaria. The non-cognitive test modules of the skills questionnaire are based on World Bank Skills Toward Employment and Productivity (STEP) surveys. The skills modules were developed with the support of a multi-disciplinary panel of experts in psychology, skills assessment, education, and labor markets. + See Ajwad et al. (2014), “The Skills Road: Skills for Employability in the Kyrgyz Republic,” and Ajwad et al. (2014), “The Skills Road: Skills for Employability in Tajikistan.” * A more detailed overview of the questionnaire sections is available in Appendix A: Questionnaire Sections. 3 1 Country Context Uzbekistan is a doubly-landlocked lower-middle income country of 30 million people located at the crossroads of Central Asia. Having experienced impressive macroeconomic growth since the early 2000s, the country has set an ambitious goal of attaining upper-middle income status by 2030. To achieve this, the government’s strategy focuses on expanding the middle class, promoting shared prosperity, and further eliminating poverty among its citizens. At the core of these objectives lies the need to continuously expand the supply of quality jobs available to Uzbekistan’s growing population. The engine of sustained long-term growth in a robust economy, therefore, is a well-educated workforce that meets the demands of a dynamic labor market focused on high value-added industries. This report uses the most recent data to assess three areas of jobs and skills in Uzbekistan. First, the report presents the current labor market outcomes to examine job creation, job distribution, job quality, and the flows of information that characterize the country’s labor market. Second, the report evaluates the demand for skills using innovative skill measurement instruments developed by the World Bank. Third, the report presents the skill formation over the life cycle to assess whether Uzbekistan’s education and training systems adequately meet the current and future demands of the country’s economy. Combining the jobs and skills information on Uzbekistan, the report lays out a skills roadmap offering policymakers concrete solutions to address the labor market challenges facing Uzbekistan. The groundbreaking research that underpins these findings marks the first ever use of detailed skill measurement surveys in Uzbekistan aimed at informing public policies. 4 2 Labor Market Outcomes This section presents an overview of Uzbekistan’s labor market outcomes and seeks to answer three fundamental questions that shed light on the country’s ability to meet the evolving labor market demand. The questions addressed are: (i) Has job creation in Uzbekistan kept pace with population growth? (ii) What is the quality of jobs in Uzbekistan? (iii) Are workers in Uzbekistan able to find jobs that match their skills with employers’ needs? As further explained below, job creation has kept up with population growth, and this is particularly noteworthy in Uzbekistan where population growth rates are high; however, a number of challenges affecting labor market outcomes remain. For example, jobs are distributed unevenly and youth labor market discouragement—the phenomenon whereby people who are willing to work leave the labor force because they feel that there are no jobs available—is high. Moreover, job quality is of particular concern given that more than half of Uzbekistan’s workers are employed in the informal sector, and most workers do not frequently learn new things on the job, performing predominantly repetitive tasks. However, a constraint facing job seekers is that labor market information systems are weak and this, in turn, has led to poor job placement and skills signaling. In addition to the results presented in the main body of the report, Appendix D: Summary Tables contains more detailed results on labor market outcomes. 2.1 Job creation has kept pace with population growth The Uzbek economy has experienced strong economic growth in the last decade. After a period of deteriorating socio-economic indicators in the post-independence era, economic growth in the late 1990s and early 2000s consisted primarily of “catch-up growth.” However, more recent economic growth has been driven predominantly by strong exports, increasing domestic demand, expansionary government policies, and a strong inflow of remittances. Attaining a growth rate of nearly 8 percent per year, Uzbekistan’s economic performance in the past decade outpaced not only its peers in ECA, but also the OECD countries (Figure 1). Figure 1: Uzbekistan’s GDP growth has been stronger than most ECA and OECD countries, 1996– 2012 15 10 GDP growth (% per annum) 5 0 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 -5 -10 -15 Uzbekistan ECA OECD Source: Authors’ calculations using World Bank, World Development Indicators, 2013. 5 Job creation has kept pace with the country’s rapid population growth. Despite Uzbekistan’s relatively high population growth rates, job creation rates in both the formal and informal sectors have generally outpaced population growth rates. Uzbekistan’s employment rate has grown at an average rate of 2.87 percent per year since 1996, while the working-age population has grown at an equivalent rate of 2.63 percent per year. However, job creation has fallen short when compared to economic growth rates. Especially in recent years, Uzbekistan’s growth has been capital intensive. In a forthcoming World Bank report on Vision 2030 in Uzbekistan, the Bank states that continuing the current capital-intensive growth model would not deliver the income-generating opportunities envisaged under Vision 2030 nor help Uzbekistan continue along a poverty-reducing trajectory. While Uzbekistan’s productivity is low relative to other countries, productivity has grown significantly in the last decade. Uzbekistan’s output per worker has grown by over 4 percent per year between 2000 and 2011 (Figure 2). This productivity growth has been higher than those seen in Brazil, Chile, Poland, South Africa, and Turkey. The productivity increase in Uzbekistan has been pronounced since 2004 reflecting higher labor quality, more productive capital, and some resource reallocation to more efficient sectors such as communications, transport, and industry. 3 However, real wage rates have consistently outpaced labor productivity. While high real wage growth is potentially good for poverty reduction, if the trend continues the economy’s competitiveness will be undermined unless sizable total factor productivity gains materialize. 4 Figure 2: Productivity has grown considerably but continues to lag behind comparator countries, 2000–2011 % Growth in GDP per person engaged 60,000 250 2011 GDP per person engaged (constant 1990 US$ at PPP) (constant 1990 US% at PPP), 50,000 200 40,000 150 2000 to 2011 30,000 100 20,000 10,000 50 0 0 GDP Per Person Engaged (Left) Productivity Growth (Right) Source: Authors’ calculations using World Bank, World Development Indicators, 2000–2011. Note: Productivity is defined as GDP per person employed/engaged. Uzbekistan’s growing working age population (until about 2040) presents a window of opportunity for increased economic growth (Figure 3). If current labor force participation rates hold, the labor force is projected to increase by 3.9 million people by 2030, reaching the fifth largest labor force in all of Europe and Central Asia (after Russia, Turkey, Ukraine, and Poland). If tapped to its full potential, a young and growing population places the country in an ideal position to reap the full benefits of economic growth. With dependency ratios (the proportion of the population above 65 and below 15 years of age to the population 3 IMF (2013). 4 Ibid. 6 aged 15–65) projected to contract from 52 percent in 2010 to 46 percent in 2030, Uzbekistan is poised to enjoy a significant demographic dividend in the coming years. The rate at which Uzbekistan replaces its unskilled older workers with skilled youth will be important to maximizing the impact of the demographic dividend on growth and prosperity, and propel Uzbekistan’s economy toward upper-middle-income status. The two panels in Figure 3 also show that the widow of opportunity to take advantage of the demographic dividend will not last forever. Beginning in the 2040s, the window will start to close and policy makers in Uzbekistan will begin to face an aging population. Figure 3: Favorable demographics present a window of opportunity for increased economic growth in Uzbekistan, 2013 Population Composition Working Age Population in Uzbekistan 100 30000 72 Share of working age population in total 90 70 (thousands of individuals) Working-age Populations 25000 Share of Total Population (%) 68 80 20000 66 70 64 60 15000 62 population 50 60 (%) 10000 58 40 5000 56 30 54 20 0 52 10 2000 2015 2030 2045 2060 2075 2090 0 Working Age Population (left) Aged 0-14 Aged 15-24 Aged 25-65 Aged 65+ Share of Working Age Population in Total Population (right) Source: Authors’ calculations using United Nations, World Population Prospects, 2012 revision. 2.2 Jobs are distributed unevenly and labor market discouragement is high Women are underrepresented in employment and, hence, they remain an under-tapped resource. Women also make up a disproportionately large share of the country’s unpaid care work. The disparity in employment rates between women in Uzbekistan and women in OECD countries is almost 20 percentage points for 25- to 34-year-olds and, more significantly, 42 percentage points for 55- to 59-year-olds (Figure 4, panel B). If Uzbekistan’s women enjoyed the average female employment rate of OECD countries, there would be 1.03 million more women contributing to the Uzbek economy today; if Uzbekistan’s female employment rate was the same as Russia’s, there would be 1.6 million additional contributors to the Uzbek economy; and if Uzbekistan’s female employment rate was the same as South Korea’s, there would be 400,000 additional contributors to the Uzbek economy. 5 5 World Bank, World Development Indicators, 2013. 7 Figure 4: Male employment rates mirror those in OECD countries, but a much smaller share of women is employed in Uzbekistan compared to the OECD countries, 2013 A. Employment rate: Men B. Employment rate: Women 100 100 Employment rate (% of population) Employment rate (% of population) 90 80 80 70 60 60 50 40 40 30 20 20 10 0 0 Cohort (age in years) Cohort (age in years) Uzbekistan OECD Average Uzbekistan OECD Average Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. A considerable portion of the population is discouraged and is consequently not looking for work (Figure 5). Discouraged workers are defined as persons who are not in the labor force and, although they are available to work, they are no longer seeking employment because they do not believe they will find any. 6 Youth are particularly affected by labor market discouragement—approximately one in ten people aged 20– 24 are not looking for a job because they do not believe they can find one. By comparison, the average share of discouraged workers among the young labor force (aged 15–24) was just 0.5 percent in OECD countries in 2012. Older men, especially those aged 55–59, also report higher levels of labor market discouragement. For men, discouragement rates peak for youth as well as for men aged 50–60. There is no second peak in the female population likely because older women play a role in caring for grandchildren and the elderly at home, while this is not as widespread a practice for men. Figure 5: Labor market discouragement is high among young men and women, as well as older men, 2013 A. Labor discouragement: Men 12 B. Labor discouragement: Women 12 Share of working-age population (%) Share of working-age population (%) 10 10 8 8 6 6 4 4 2 2 0 0 Discouraged Discouraged Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: The figures depict the number of unemployed (individuals looking for work) and discouraged men and women as a share of the population in the age cohort. 6 International Labour Organization, Key Indicators of the Labour Market (KILM), 2013. 8 The population of Uzbekistan is internationally mobile, with an estimated 2 million citizens living abroad as of 2010, which amounts to an emigration rate of approximately 7 percent of the population. 7 The country’s emigration rate is more than double the world average (3.2 percent) and that of other middle-income countries (2.7 percent), yet lower than that of Europe and Central Asia as a whole (10.7 percent). Due in part to regional cooperation agreements permitting visa-free entry and waiving requirements for employment guarantees pre-arrival, the Russian Federation is the primary destination for international labor migrants, hosting nearly half of Uzbek emigrants as of 2011. 8 Other CIS 9 member states (particularly Ukraine and Kazakhstan, neither of which require entry visas), the Gulf Cooperation Council countries, Israel, and the Republic of Korea also attract Uzbek workers, although in substantially smaller numbers. 10 Among the working-age population in Uzbekistan, one in five males is an international migrant, and among the youth population the international migration rate is even higher. These high migration rates stem from deficiencies in the domestic labor market as well as from significant international demand for Uzbek labor. Among youth, the migration rates are particularly high—one in three males between the ages of 20 and 24 is a migrant. By comparison, female migration rates are not as significant as those of the male population. In contrast to international migration, domestic migration rates are very low, which suggests that labor allocation within the country may be less than optimal. Domestic migration, or internal migration, plays a key role in fostering local agglomeration economies. Figure 6 illustrates these trends. Figure 6: International migration rates are high among young men in Uzbekistan, 2013 Internal and international migration: Internal and international migration: Men Women 15-19 90 9 1 15-19 98 2 0 20-24 63 35 3 20-24 95 23 25-29 70 27 3 25-29 91 5 4 30-34 69 26 5 30-34 91 5 4 35-39 72 26 3 35-39 92 53 40-44 75 20 5 40-44 95 23 45-49 77 21 2 45-49 93 34 50-54 90 10 1 50-54 99 1 1 55-59 93 42 55-59 98 02 60-64 95 23 60-64 98 02 Total 76 21 3 Total 95 33 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Not a migrant International Domestic Not a migrant International Domestic Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. The vast majority of external migrants work in the Russian Federation, and to a lesser extent in Kazakhstan. As shown in Table 1, the Russian Federation currently hosts over three-quarters of external labor migrants from Uzbekistan (86 percent). Kazakhstan, the second most important host of labor migrants, accounts for 12 percent of Uzbek migrants. 7 World Bank (2011). 8 MiRPAL (2011) and World Bank (2011). 9 The Commonwealth of Independent States (CIS) is a regional organization comprised of former Soviet republics. 10 MiRPAL (2011) and World Bank (2011). 9 Table 1: The vast majority of Uzbek migrants work in the Russian Federation, 2013 Country of Destination Proportion (%) Russian Federation 86.41 Kazakhstan 11.93 Kyrgyz Republic 0.33 Tajikistan 1.06 Other 0.28 Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. 2.3 The labor market has undergone significant transformations Uzbekistan’s economy has undergone fundamental structural changes over the last 15 years, shifting employment from the agricultural sector to industry and services. Since the mid-1990s, the Uzbek government has adopted an industrialization strategy designed to move away from heavy dependence on agricultural and natural resources and transform the economy into a modern industrial one. These policies have changed the structure of the economy considerably—while services continue to be a dominant economic activity, the share of employment in industry has exceeded that of agriculture since 2006 (Figure 7). 11 By comparison, in the OECD countries the share of employment in services on average is larger (73.9 percent), while employment in industry (22.6 percent) and in agriculture in particular (3.5 percent) is less prevalent. The shift away from agriculture employment has been an important part of the productivity improvements experienced in Uzbekistan in the last decade. 12 Figure 7: The share of agriculture in GDP has decreased, while the share of value added in services and industry has grown, 1995–2012 60 50 Share of value added (%) 40 30 20 10 0 Agriculture Industry Services Source: Authors’ calculations using World Bank, World Development Indicators 2013. Services and agricultural sectors account for the majority of employment in Uzbekistan (Figure 8). Four out of five workers are engaged in the services or agricultural sectors, but some of these workers are 11 World Bank (2013a). 12 Ibid. 10 family workers or entrepreneurs. The share of employment in services is proportional to its share in GDP, but the value added in industry appears to be higher than the share of employment. Figure 8: The services sector employs more than half of all workers in Uzbekistan, 2013 Employment by sector Agriculture 27% Services 56% Industry 17% Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. The majority of the employed population works in state-owned firms. More than one in three people work in state-owned enterprises (SOEs), which is nearly double the number of people working for private firms (37 percent compared to 20 percent, respectively). Moreover, 34 percent are self-employed, representing in particular small, informal businesses (Figure 9). Figure 9: State-owned enterprises dominate the labor market and employ roughly twice as many persons as the private sector, 2013 Employment by ownership-type, urban and rural 100% 90% 36% 33% 34% 80% 70% 60% Percent (%) 16% 24% 20% 50% 40% 11% 9% 7% 30% 20% 37% 36% 37% 10% 0% Urban Rural Total State-owned enterprise Government Private Sector Self-employed and other Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: Respondents aged 25–64. Entrepreneurship in Uzbekistan is dominated by micro enterprises that operate in the services sector. Ninety percent of all the self-employed workers interviewed in the survey work at firms with fewer than six workers. About 70 percent of all self-employed individuals do not employ any additional workers and 11 another 21 percent employ fewer than five additional workers. Compared to salaried workers, the self- employed are more likely to engage in the services sector as opposed to industry and agriculture sectors. Note that the majority of individuals working in agriculture in are unpaid family workers (53 percent). Figure 10: Self-employment is dominated by micro-businesses in the services sector, 2013 Self-employed: size of businesses Sector of employment self-employed and salaried workers 80% 70% 100% 70% 90% Share of self-employed (%) 80% 60% Share of self-employed (%) 70% 53% 66% 50% 60% 50% 40% Services 40% 18% 30% 12% Industry 30% 20% 21% 29% Agriculture 10% 22% 20% 0% 8% 10% 1% 0% 1 2-5 6-50 50+ Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. 2.4 Job quality remains a significant concern for policy makers Job quality is a multidimensional concept that includes earnings, but also other concepts such as workplace safety, job security, learning and advancement opportunities, and health and social protection benefits, mental and physical health, etc. 13 At the other extreme, not having a job undermines life satisfaction and especially in countries where wage employment is the norm and where the lack of opportunities translates into open unemployment rather than underemployment. In this section, the concept of informality, which is work without a labor contract; the type of work performed at typical workplaces; and the use of technology at work is explored in more detail. The informal sector employs slightly more than half of all Uzbek workers, raising significant concerns about possible worker protection from employer exploitation. Although the definition of informality varies, for the purposes of this analysis the following definition, which is guided by other research in Europe and Central Asia, is applicable: informal sector workers are those who lack an employment contract or are unpaid family workers. 14 In Uzbekistan, the people most likely to be engaged in the informal sector are men, rural residents, and individuals with lower educational attainment levels (Figure 11). In 13World Bank (2012). 14In some studies, individuals who are self-employed in businesses with fewer than six employees are also considered part of the informal sector. In Uzbekistan, however, there is some indication that a considerable share of these small businesses pay taxes, so while they are non-corporate, they should not be considered informal. 12 agriculture, for example, almost 80 percent of all workers are engaged in the informal sector according to the above definition. Figure 11: Informal salaried work is common in agriculture and industry jobs, as well as among lower educated individuals, 2013 Informal salaried work by sector Informal salaried work by education level 90% 90% 80% 77% 80% Share of informal employment 80% 68% 70% Share of informal employment 70% 60% 54% 49% 60% 50% 60% 54% 40% 50% 30% 38% 20% 16% 40% 10% 30% 0% 20% 10% 0% Agriculture Services Industry Total Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Uzbekistan’s recent economic success should not overshadow the importance of understanding the causes of the country’s widespread economic informality. Understanding how informal work and economic transactions in unregulated and untaxed markets affect employment, well-being, and risk management are key steps toward creating more quality jobs in Uzbekistan. In all middle-income countries, even those with large manufacturing and services sectors, it is common to find a share of the labor force engaged in the informal sector, and thus beyond the reach of taxation, regulation, and protection. Consider that in the Philippines more than 40 percent of the labor force is engaged in the informal sector and in Thailand more than 50 percent; in the Republic of Korea, however, fewer than 25 percent of workers are in the informal sector and in Japan about 10 percent. There is a vigorous debate about whether firms and workers “exit” or are “excluded” from formality. However, the consequences are similar. That is, informality imposes costs to the economy at large. For example, people working informally face explicit and implicit barriers to public and privately provided insurance instruments to manage shocks. In addition, bigger firms are often over-taxed to make up for revenue lost to the government from widespread tax evasion. Finally, a large informal economy imposes heavy costs that tend to deteriorate the provision of services and public goods. 15 Physical work and repetitive tasks are key components of most jobs in Uzbekistan and only half of all workers seem to learn new things on the job. In this study, physical work is defined as regularly lifting or pulling anything weighing at least 50 pounds (25 kilograms). Physical work is unsurprisingly common in the agricultural and industrial sectors, and less so in the services sector (Figure 12). The majority of tasks performed at work are repetitive in nature (56 percent), and this holds primarily for jobs in the agriculture and 15 World Bank (2014b). 13 industry sectors. Manual, repetitive tasks limit the scope for on the job learning, which is confirmed by survey respondents in all three sectors. Only about 41 percent of all respondents working in agriculture and 53 percent in industry state that they learn new things at least once a week. This share is slightly higher, at 61 percent, in services. Figure 12: High shares of physical work and repetitive labor, 2013 100 90 90 79 76 80 80 70 71 70 61 56 Percent (%) 60 53 54 50 41 40 26 30 20 10 0 Perform physical work Learning new things at Perform repetitive tasks regularly least once a week more than half the time Agriculture Industry Services Total Source: Authors’ estimates using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. The use of computers on the job is relatively low, although computer use is slightly more common among younger employees in state-owned enterprises, working in the service sector. In Uzbekistan, only one in five workers report using a computer. This share is low compared to other developing countries. In the Yunnan province in China 55 percent of workers use computers, in Bolivia and Vietnam 35 percent of workers use computers, and in Sri Lanka 30 percent of workers use. 16 In Uzbekistan, younger and middle age workers are twice as likely to use computers (21 percent) as older workers (10 percent). The share of workers using computers is the highest in state-owned enterprises or the government (36 percent) and in the services sector (29 percent). Moreover, workers in richer households are considerably more likely to use computers at work than workers in poorer households, suggesting that higher paying jobs are more likely to require computer use. 16 World Bank (2013e). 14 Figure 13: Use of computers at work is relatively low in Uzbekistan, but younger workers in the public sector are more likely to use a computer on the job, 2013 Use of a computer at work 40% 36% 35% 29% 30% 30% 25% 23% 21% 19% 19% 20% 16% 15% 12% 10% 10% 6% 5% 4% 5% 0.80% 0% 0% Middle Older Agriculture Industry Fifth Young Services Private Sector Self-employed and Other Third Total First Second SoE/Government Fourth Cohort Employer Sector Quintile Source: Authors’ estimates using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: cohorts are defined as: young (aged 25–34), middle (aged 35–54), and older (aged 55–65). Quintiles represent household consumption quintiles. 2.5 Labor market information systems are weak, hindering the job search and skills signaling process Obstacles to learning about job vacancies or demonstrating skills to employers may make it difficult for workers to optimally match their skills in the labor market. Workers need ready access to information regarding job openings, job search strategies, and methods to effectively present their qualifications to employers. There is evidence that information gaps and signaling problems hinder efficient and equitable job matching in Uzbekistan. Among eight possible survey responses, individuals reported that learning about job vacancies is the biggest constraint to finding work in Uzbekistan. More than half of all respondents in Uzbekistan (58 percent) indicated that they do not feel they have ready access to vacancy announcements. This suggests that there is room to improve labor market information in Uzbekistan, not only to ensure that workers know where to learn about job openings but, most importantly, to ensure that a search effectively seeks to match jobs to their skillsets. In addition, workers state that they have difficulty signaling their skills to employers. Once workers have found the right job vacancy, a crucial part of the job matching process is for workers to signal their skills to employers. Given incomplete information about applicants, a resume, references, and/or interviews are typically used as a first screening mechanism. However, certifying or demonstrating qualifications as well as preparing a resume are the second and third most important constraints mentioned by respondents in 15 Uzbekistan, respectively (Figure 14). These are all significant obstacles to workers, making job matching very difficult process. Figure 14: The majority of individuals face constraints to finding a job, 2013 70 58 Sharer of respondents identifying 60 52 49 46 44 50 39 as a constraint (%) 40 29 28 30 20 10 0 Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. 16 3 The Demand for Skills This section presents information about the demand for skills in Uzbekistan, mostly drawing from the 2013 household survey data. The section addresses a fundamental question, namely what skills are demanded by Uzbek employers? The details ensue, but the study finds that employers seek workers who possess both cognitive and non-cognitive skills, even when other factors—in particular the level of educational attainment—are held constant. The analysis also indicates that these skills are generally utilized in the workplace, which partially explains their demand in the labor market. The section begins by defining skills. Box 2 defines skills for the purposes of this report. Because of data constraints, this study focuses on cognitive and non-cognitive skills. In addition to the results presented in the main body of the report below, Appendix E: Cognitive and Non-cognitive Skill Mean Scores contains more detailed information on cognitive and non-cognitive skill outcomes. Box 2: Defining skills Workers’ skills consist of cognitive, non-cognitive, and technical skills (Figure B2). Because of data constraints, this study focuses on cognitive and non-cognitive skills. Cognitive skills capture the ability to use logical, intuitive, and critical thinking as well as skills such as problem solving, verbal ability, and numeracy. Social and behavioral skills represent personality traits that are relevant in the labor market, including extraversion, conscientiousness, openness to experience, agreeability, and emotional stability. Figure B2: A worker’s skillset can be divided into three types of skills: cognitive, social/behavioral, and technical Source: Pierre et al. (forthcoming); “STEP Skills Measurement Surveys: Innovative Tools for Assessing Skills,” cited in World Bank (2013b). The three cognitive skills measured in this study are memory, literacy, and numeracy. The working memory score is based on twelve items that asked respondents to repeat a sequence of numbers of increasing length. The literacy score represents reading comprehension skills and builds on five text comprehension questions about a story card. The informational numeracy score is built using a total of 10 questions measuring comprehension of a medicine instructions card, a bus schedule card, publicity, and a graph. It should be noted that the numeracy score represents various aspects of numeracy skills, which often also require a broader set of cognitive skills such as being literate. In particular, individuals with a high score on numeracy have the ability to recognize and manipulate numbers contained in and represented by various formats. 17 The five non-cognitive skills measured in this study are openness, workplace attitude, decision making, achievement striving, and the growth mindset scale. The skills are built using the following items: (1) Openness to New Ideas and People (5 items; e.g., “Are you outgoing and sociable?”; “Are you interested in learning new things?”); (2) Workplace Attitude and Behavior (5 items; e.g., “Do you enjoy working on things that take a very long time to complete?”; “Are people mean/not nice to you?”); (3) Decision Making (5 items; e.g., “Do you think about how the things you do will affect others?”; “Do you think carefully before making an important decision?”); (4) Achievement Striving (3 items; e.g. “Do you do more than is expected of you?”; “Do you try to outdo others, to be best?”); and (5) Growth Mindset Scale (4 items; e.g. “The type of person you are is fundamental, and you cannot change much”; “You can behave in various ways, but your character cannot really be changed.”). * A detailed description of the cognitive scores and their construction is included in Appendix B: Constructing Cognitive Skills Scores Methods for Scale Development and Scoring. ** A detailed description of the non-cognitive scores and their construction is included in Appendix C: Constructing Non-Cognitive Skills Scores Methods for Scale Development and Scoring. 3.1 Employers demand skilled workers, but have difficulty finding them Employers in Uzbekistan report that inadequate skills in the workforce pose significant obstacles to firm growth. A 2008 survey of Uzbekistan’s employers revealed that 73 percent of firms identify inadequate skills of the country’s workers as an obstacle to doing business—up from 60 percent in 2005. More than one- third of the firms (35 percent) indicated that employee skills posed a “major” or “very severe” obstacle to growth. A 2013 survey found that industrial enterprises were the most likely to have difficulty finding the right skills among the available workforce, with nearly half (49 percent) reporting a lack of sufficient numbers of qualified specialists with a higher education degree (Figure 15). 17 Figure 15: A large share of firms report difficulties in hiring sufficient numbers of qualified specialists with higher education, 2013 Industry 49% Construction, transport, communications 41% Trade and catering 14% Social services 32% Other services 42% Total 35% 0% 10% 20% 30% 40% 50% 60% Source: Authors’ calculations using World Bank (2014c). 17 The findings in this paragraph are from the 2014 World Bank Uzbekistan higher education report. It relies on the 2005 and 2008 BEEPS data, as well as an employer survey commissioned in 2013 for the higher education report. 18 Among the traits that employers value in their workers, several non-cognitive as well as cognitive skills (e.g. Russian and English language knowledge) emerge as those that are most lacking. In a recent analysis of a survey that interviewed 232 enterprises 18 in Uzbekistan that employed higher education graduates, supervisors/managers were asked to rate the “importance” of categories of skills and “satisfaction” with worker skills. Therefore the difference between the importance score and the satisfaction score is a measure of the importance weighted skills gaps. The largest discrepancy between importance and satisfaction scores—representing the weighted skills gap—are found in the area of Russian language skills and a number of non-cognitive skills categories. Among the non-cognitive skills that employers report are both important and lacking are accepting responsibility for one’s actions, self-motivation, and creativity (Figure 16). Figure 16: Language and non-cognitive skills are reported by employers to be most lacking in workers in Uzbekistan, 2013 Non-cognitive skills lacking Cognitive/technical skills lacking Accepting Russian language Self-motivation English language Analytical thinking Creativity Customer relations Willingness to learn Project management Reliability Written skills Integrity Financial or budgeting Self-discipline Presentation skills Teamwork Literacy Oral communication Following Numeracy Flexibility Problem solving skills Honest, sincerity Uzbek language skills Empathy Computer skills 0 0.2 0.4 0.6 0.8 1 0 0.2 0.4 0.6 0.8 1 Index of skills gap Index of skills gap Source: Authors’ calculations using World Bank (2014c) based on the results of a survey of 232 enterprises that hired graduates of higher education institutions between 2009 and 2012. Note: The skills lacking index measures the gap between the importance and satisfaction of each skill. Skill importance and satisfaction are assessed on a five-point scale, where 1 = “not at all” and 5 = “extremely.” Employment prospects are stronger for university and secondary special/technical educated individuals, though the gap is more pronounced among women. While individuals who have completed a secondary special/technical or tertiary education enjoy high employment rates, individuals who have only completed secondary general are less likely to be employed. Overall, the employment rate among tertiary graduates is 77 percent, compared to 57 percent for secondary general graduates. This positive correlation is mostly driven by employment outcomes for women, however. Among men, employment rates vary from 81 percent for secondary general graduates to 84 percent for tertiary graduates. 18Firms were surveyed in the following sectors: industry (21 percent); construction, transport and communications (19 percent); trade and catering (16 percent); social services, excluding education (20 percent); and other services (20 percent). Firms in the private sector make up 67 percent of the sample, the remaining 33 percent of firms surveyed are public/state-owned. 19 Figure 17: Tertiary graduates and secondary special/technical graduates have better labor market outcomes than individuals with a secondary general or less than secondary education, 2013 Employment rate by education level 90 81 83 84 82 77 80 67 68 70 62 Employment rate (%) 57 60 50 50 43 40 35 30 20 10 0 All Men Women Secondary general Secondary technical/special Tertiary Total Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: Respondents aged 25–64. In Uzbekistan, preschool attendance correlates with employment outcomes. That is, preschool attendance as a child is correlated positively with the probability of both being employed and having a better job later in life (Figure 18), but mainly through higher educational attainment. While not implying causality, on average, adults who attended preschool as a child are more likely to be employed (73 percent) compared to adults who did not attend preschool (58 percent). Among the employed, a larger share of adults who went to preschool as a child have formal sector jobs. Doing agricultural work is also less common among adults who went to preschool as a child. However, when taking into account demographic characteristics such as age, gender, marital status, geographic location, and educational attainment, then preschool attendance is no longer a significant predictor of employment outcomes. Educational attainment, instead, determines labor market outcomes. Therefore, preschool attendance does not impact employment outcomes directly, but rather through higher educational attainment. 20 Figure 18: Employment outcomes are positively correlated with preschool attendance as a child, 2013 Employment outcomes among individuals who did and did not attend preschool 80% 73% 70% 58% 60% 53% 50% 46% 43% 38% 40% 27% 30% 20% 14% 10% 0% Share employed *** Share working in formal Share working in public Share doing agricultural sector *** sector* work *** Did not attend preschool Attended preschool Source: Authors’ estimates using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: Respondents aged 25–64. ***/**/* represent significant differences in outcome between individuals with and without preschool at the 1%/5%/10% significance level, respectively. A considerable wage premium is paid to tertiary graduates in Uzbekistan. Figure 19 depicts the average percentage earnings premium for workers with a tertiary education level relative to workers with secondary education (both general and technical) who otherwise possess similar observed characteristics. In Uzbekistan, workers with a tertiary education on average earn a 55 percent higher wage than similar workers with a secondary education. A high return to tertiary education is a signal of strong demand for higher educated individuals in the labor market. As such, there is a positive correlation between the degree of modernization (reforms to transition to a market economy) and the returns to tertiary schooling. 19 Average college and university premiums are highest in most EU-10 countries and are comparable to other middle- and high-income countries. Hence, the value and importance of tertiary education is likely to increase in the coming years as Uzbekistan progresses toward upper-middle-income status. Ensuring adequate access to quality tertiary and professional education is crucial for meeting the skills needs of an expanding and rapidly changing economy. In particular, a gap currently exists between the need for and the provision of highly specialized post-secondary technical education in Uzbekistan. By diversifying the options for tertiary education provision—including through the introduction of flexible, short-term technical degree and non- degree programs in specific technical fields—the education system can begin to meet the varied skills needs of the labor market. 19 See, for instance, Staneva et al. (2010) and Rutkowski (1996 and 2001). 21 Figure 19: There are considerable average returns to tertiary education among salaried workers aged 25–64, circa 2009 100% Earnings premiums over secondary 90% 80% 70% education 60% 50% 40% 30% 20% 10% 0% Source: Authors’ estimates for the Kyrgyz Republic, Tajikistan, and Uzbekistan using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Estimates for other countries in: Arias et al., 2014. Note: Salaried workers aged 25–64. Workers with a tertiary education report that their education is more useful for their job than workers with a less than secondary or secondary general education. Four out of five tertiary graduates report that their formal education is “very useful” for their work. In contrast, this share is much lower for people with less than secondary or a secondary general level of educational attainment. Hence, not only do employers complain that the education of the workforce in Uzbekistan is inadequate, but workers themselves also report that their formal education is not as useful as it should be for their job (Figure 20). Figure 20: Tertiary graduates report that their education is useful for work, while less educated workers report less satisfaction, 2013 Tertiary 3 13 6 79 Secondary technical/special 13 14 21 52 Secondary general 14 32 35 19 Less than secondary 19 24 27 30 0 20 40 60 80 100 Not useful at all Somewhat useful Moderately useful Very useful Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Educational institutions are disconnected from enterprises in Uzbekistan, and this is leading to significant mismatches. A survey of 232 enterprises in Uzbekistan reveals that employers desire to be more involved in the education process, but report a low level of interest from higher education institutions to partner with businesses in this area. 20 The lack of such cooperation may in part explain the mismatch between 20 World Bank (forthcoming). 22 the fields in which students graduate and the fields in which jobs are created. In the agricultural sector in particular, a considerable number of specialized secondary education (SSE) students graduated in the period between 2005 and 2010, while there was in fact a loss of jobs in the sector (Figure 21). In healthcare, physical culture, and sport, too, the number of SSE graduates by far exceeded the number of jobs created. Figure 21: There is a mismatch between jobs created and fields in which students are graduating 125 100 Jobs created HE graduates SSE graduates 75 50 25 0 Agriculture Education and art Industry and construction communications Healthcare, physical Other services Transport and -25 culture and sport Source: Ajwad et al. (2014) and World Bank (2014c). 23 3.2 Skills enhance employment outcomes, even when other factors are held constant Past research has shown a strong and robust relationship between cognitive skills and labor market outcomes. Much of that research has focused on developed countries, mostly because of data availability, but there is a growing body of literature on emerging economies. Box 3 summarizes the key evidence in the literature. Box 3: Evidence that skills and labor outcomes are related Past research has shown a strong and robust relationship between cognitive skills and labor market outcomes. Studies using longitudinal household surveys in the United States find that cognitive test scores during schooling years are good predictors of the level of wages attained in the labor market (Heckman, 2000; Heckman and Carneiro, 2003; Cunha et al., 2006. Moreover, the empirical evidence shows that a shortage of skills is considered to be one of the biggest barriers to employment (Sánchez Puerta, 2009). The empirical literature on cognitive skills and labor market outcomes distils two types of causal pathways: (i) direct—e.g. Murnane et al. (1995) assess the role of math skills of graduating high school seniors on their wages at age 24 and find a positive and increasing impact of cognitive skills on wages; and (ii) indirect—e.g. Cunha et al. (2005) argue that cognitive skills increase the likelihood of acquiring a higher level of education, which in turn leads to higher economic returns. There is also growing evidence that non-cognitive skills are important for labor market outcomes. Even though a more recent phenomenon, the empirical literature on the skills/labor market outcomes nexus finds a strong and robust relationship between certain non-cognitive skills, such as dependability, persistence, and docility and labor market outcomes (Heckman et al., 2006; Blom and Saeki, 2011; and Cunha and Heckman, 2010). A separate strand of the literature argues that non-cognitive skills are particularly valued in certain sectors (e.g. services). Finally, recent evidence in the context of high income countries suggests that employers value non-cognitive abilities more than cognitive ability or independent thought (e.g. Bowles et al., 2001). In Uzbekistan, employed people have better cognitive and non-cognitive skills than inactive people. Employed workers performed better on memory, literacy and numeracy tests than inactive individuals. The gap between those who are employed and those who are inactive is particularly wide for the memory test. Most of the non-cognitive scores for employed individuals are higher than scores for inactive individuals. Of particular note are decision making and achievement striving, whose scores deviate considerably between the employed and the inactive. Individuals who are discouraged from participating in the labor market have significantly lower cognitive skills and—to a certain extent—non-cognitive skills than the employed. In fact, those who are discouraged from participating in the labor market possess similar cognitive skills as inactive individuals. In terms of non-cognitive skills, interestingly, decision making in particular seems to be low among discouraged individuals, while self-reported workplace attitude and achievement striving do not seem problematic. The skill gaps among discouraged individuals results may, in part, explain why such individuals face difficulties finding a job. Of course, skills alone do not explain labor market discouragement. Individuals may exit the labor force for a variety of reasons including high reservation wages, immobility, a lack of connections needed to secure jobs, or simply unrealistic expectations. However, low skill levels among the 24 discouraged are particularly noteworthy because youth are overrepresented in the discouraged population, and given the youth bulge in Uzbekistan, the mismatch in skills raises the stakes for policymakers. Figure 22: Employed people exhibit better cognitive and non-cognitive skills than inactive people and those who are discouraged, 2013 Cognitive skill mean scores by Non-cognitive skill mean scores by employment status employment status 0.2 0.2 Standard deviations from the mean Standard deviations from the mean 0.15 0.15 0.1 0.1 0.05 0.05 0 Memory*** Literacy* Numeracy* 0 -0.05 -0.05 -0.1 -0.1 -0.15 -0.2 -0.15 Employed Discouraged Inactive Employed Discouraged Inactive Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Cognitive and non-cognitive skills are generally stronger among public sector workers, while the private sector does poorly (Figure 23). The public sector and SOEs attract highly skilled workers—both in terms of cognitive and non-cognitive skills. Self-employed workers also appear to have high cognitive skills, but do not score as well on many of the non-cognitive skills except decision-making skills, where they are particularly strong. The private sector workers, who include workers in the informal sector, do poorly with respect to cognitive and non-cognitive skills without exception. Figure 23: Cognitive and non-cognitive skills are generally stronger among public sector workers Cognitive skill mean scores Non-cognitive skill mean scores by type of employer by type of employer Standard deviations from the sample mean 0.4 0.2 Standard deviation from the sample mean 0.3 0.1 0.2 0 0.1 -0.1 0 -0.2 Memory Literacy Numeracy*** -0.1 -0.3 -0.2 -0.4 Public Sector Private Sector Self-employed and other Public Sector Private Sector Self-employed and other Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. 25 Formal sector workers have better memory skills, but other cognitive or non-cognitive skills are statistically similar between formal and informal sector workers. Figure 24 depicts cognitive and non- cognitive skill outcomes among formal and informal salaried workers. Despite visible discrepancies in average cognitive skills between formal and informal sector salaried workers, only memory skills are significantly better among employees in the formal sector. Among the other measures cognitive and non-cognitive skills, no statistically significant differences are observed between formal and informal sector workers. Figure 24: Formal sector workers have better memory skills, but other cognitive or non-cognitive skills are statistically similar, 2013 Cognitive mean scores Non-cognitive mean scores among formal and informal salaried among formal and informal salaried workers workers 0.3 0.16 0.14 Standard deviations from the mean Standard deviations from the mean 0.25 0.12 0.1 0.2 0.08 0.06 0.15 0.04 0.02 0.1 0 -0.02 0.05 -0.04 0 Memory** Literacy Numeracy -0.05 Formal salaried workers Informal salaried workers Formal salaried workers Informal salaried workers Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Holding all else constant, cognitive and non-cognitive skills are positively associated with employment in Uzbekistan. An analysis was conducted in order to assess the probability of being employed, conditional on a respondent’s set of cognitive and non-cognitive skills. 21 In addition, the model controls for the usual socio-demographic variables (age, gender, place of residence, and educational attainment). The findings suggest that certain cognitive and non-cognitive skills are significantly associated with a higher probability of being employed. In particular, memory score (cognitive skill) and decision making (non-cognitive skill) are found to be positively and significantly associated with the probability of being employed, as opposed to being out of work. More specifically, an increase in the memory score by one standard deviation is associated with a 15 percent increase in the likelihood of an individual being employed; similarly an increase in the decision-making score by one standard deviation is associated with an increase in the likelihood of being employed by 14 percent. The probability of employment in the more modern sector, i.e., industry and services, is significantly affected by an individual’s cognitive and non-cognitive skills. As economies develop and prosper, they also undergo a process of structural shift, whereby jobs are shifted from the traditional sectors (agriculture) to the modern ones (industry and services). This shift also implies a rise in importance of the cognitive and non- cognitive skills in the so-called “modern” sector. 22 In order to gauge this relationship, we restrict the analysis conducted above only on the probability of employment in the industry and services sectors. The results 21 See Nikoloski and Ajwad (forthcoming) for details. 22 OECD (2010). 26 reveal that numeracy and decision making again are significant determinants of employment in the modern sector, as opposed to working in agriculture or being out of work. However, for women, openness/sociability is also found to be positive and statistically significant; and for men, workplace attitude is found to be positive and significant. More work is needed to understand why there is a gender difference in the types of skills valued by employers, especially given that these hold true even when sectors of employment are also held constant. Workers with cognitive and non-cognitive skills are generally employed in formal sector jobs. Higher skills imply not only higher employability, but also a higher chance of obtaining a job in the formal sector, typically jobs in the state administration or SOEs, as well as in medium or large privately-owned companies. There are numerous benefits associated with employment in these sectors, including job security and various social protection benefits. In order to explore the relationship between a worker’s skills and the probability of having a formal sector job, the study analyzes employment in the state administration, SOEs, as well as private enterprises employing more than 6 employees. 23 Workers with cognitive and non-cognitive skills are generally employed in formal sector jobs. Young adults possess better cognitive skills than older adults, but the pattern is not as clear for non- cognitive skills. Figure 25 shows two interesting patterns. First, young adults generally have better cognitive skills than older adults. Second, with the exception of literacy, employed people have higher cognitive skills than inactive people. This holds both for the young and old age cohort. Similarly, employed people score higher than inactive people for most measures of non-cognitive skills as well, for both young and older cohorts. Among the employed, young people show more openness and achievement striving attitude, a finding that is also observed in other countries. Older workers and younger workers have similar workplace attitudes and decision-making skills. Note that young inactive adults scored lowest on all non-cognitive skills, with the exception of workplace attitude. Also, nearly all cognitive scores reported are negative, indicating that individuals in the middle-age group (35–54 years old) have better cognitive skills than both young and old adults. Figure 25: Cognitive and non-cognitive skills are generally better in young compared to older workers, especially among the inactive population, 2013 Cognitive skills in Uzbekistan Non-Cognitive skills in Uzbekistan 0.20 0.10 0.10 0.05 0.00 0.00 literacy memory numeracy -0.10 -0.05 -0.20 -0.10 -0.30 -0.15 -0.40 -0.20 -0.50 -0.25 Young Employed Young Inactive Young Employed Young Inactive Old Employed Old Inactive Old Employed Old Inactive Source: Authors’ estimates using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: Young adults are 25–34 years old; old adults are aged 55–64. 23 A probit analysis was carried out. For details see Nikoloski and Ajwad (forthcoming). 27 Box 4: Skills and migration in Uzbekistan and Kyrgyzstan Existing studies find that migrants and non-migrants differ with respect to education and skills. Among the reasons are the selective migration of groups who can gain disproportionally from mobility (Borjas, 1987), investments in higher education for those who aspire to migrate (Mountford, 1997), or specific pre-migration investments in human capital (Danzer and Dietz, 2014). There is a broad range of literature on the self-selection of migrants with respect to formal educational attainments (e.g., Chiquiar and Hanson, 2005; Lanzona, 1998; Orrenius and Zavodny, 2005). However, evidence on the cognitive and non- cognitive skill endowment of migrants is scarce. An analysis of adults with the intention to migration in the Kyrgyz Republic and Uzbekistan reveals that working age adults who plan to migrate typically possess above average cognitive and non- cognitive skills, compared to adults who have no migration plans (Figure 26). Note that this analysis cannot be conducted for each of the countries separately because the sample size is too small. For cognitive skills, the gap between working age adults who do and do not plan to migrate is sizeable for memory skills (greater than 20% of a standard deviation) and modest for literacy skills, but there is no difference for numeracy skills. For all measured non-cognitive skills, individuals with migration plans perform better than individuals without migration intentions; the gap is especially large with respect to achievement striving, reflecting the fact that migrating abroad implies a strongly positive contribution to family income in Central Asia. The finding that individuals who are planning to migrate, on average, have better cognitive and non- cognitive skills than others in the working-age population supports existing selection theories of migration. The results also suggest that studies focusing exclusively on education may draw very different conclusions. Similarly, migrants who have returned after working abroad have significantly higher cognitive and non-cognitive skill outcomes than non-migrants (Figure 27). The gaps in cognitive skills between migrants who have returned and non-migrants are very large in absolute terms and much larger than the gaps seen between adults who intend to migrate and adults who do not plan to migrate. While this potentially implies learning effects through migration, it could also point to the fact that not all individuals who intend to migrate follow through and actually move migrate. On the other hand, the skills gaps between migrants who return and non-migrants are lower than the gaps found between adults with the intention to migrate and working age adults with no migration intentions. The notable exception is decision making, in which migrants who return have high scores. Hence, while individuals who plan to migrate do not have much higher decision making skills, actual migrants have much higher decision making skills. This result could point either to the fact that adults with good decision making skills following through with their migration intentions, or that migrants learn such decision making skills while abroad. Disentangling these different possibilities remains for future research. 28 Figure 26: Adults with migration intentions on average have significantly higher cognitive and non- cognitive skills than adults without migration intentions, 2013 Cognitive skills gap between Non-cognitive skills gap between individuals with and without individuals with and without migration intentions migration intentions 0.5 0.3 0.4 0.3 0.2 0.2 0.1 0 0.1 0 Memory skills Literacy skills Numeracy skills Source: Authors’ estimates using World Bank/GIZ, Kyrgyz Republic and Uzbekistan Jobs, Skills, and Migration Survey, 2013. Figure 27: Returned migrants on average have significantly higher cognitive and non-cognitive skills than non-migrants, 2013 Cognitive skills gap between returned Non-cognitive skills gap between returned migrants and non-migrants migrants and non-migrants 0.3 0.3 0.2 0.2 0.1 0.1 0 0 Memory skills Literacy skills Numeracy skills Source: Authors’ estimates using World Bank/GIZ, Kyrgyz Republic and Uzbekistan Jobs, Skills, and Migration Survey, 2013. 3.3 Higher skilled workers use their skills more often in the workplace Comparing a worker’s skillset and use of skills on the job can help establish whether skills are effectively put to use in the labor market. If the labor market were to effectively make use of workers’ skills, then there would be a positive correlation between ability and the use of skills on the job. For example, a person with better numeracy ability would use math skills more frequently and intensely on the job. This section examines whether the labor market in Uzbekistan indeed uses workers’ skills efficiently. It is important to keep in mind that the results presented are correlations and they do not denote causation. It may well be that individuals who make more frequent use of math skills on the job score higher on a math ability test precisely because they use those skills on a daily basis. Such bi-directional links between skill use and skill ability is an important caveat when interpreting the results. 29 In Uzbekistan, workers with better cognitive skills are more likely to use those skills on the job, and are less likely to engage in physical work routinely. Figure 28 depicts the percentage of individuals with a low, medium, and high numeracy ability who use math skills (doing any multiplication) on the job (left panel) and engage in physical work (right panel). There is a clear positive correlation between numeracy ability and using those numeracy skills on a regular basis on the job. In addition, the share of individuals doing physical work is lower among those with a high cognitive ability, compared to those with a low cognitive ability. There is an exception in urban areas, where people with a medium cognitive ability tend to do more physical work than both the low and high ability groups. As expected, however, physical work is less common across the board in urban areas compared to rural areas. Figure 28: Workers with cognitive skills use those skills and are less likely to perform physical work 80 80 Share of respondents using math skills on the job Share of workers engaging in physical work 70 70 60 60 50 50 40 40 30 30 20 20 10 10 0 0 All Urban Rural Men Women All Urban Rural Men Women Low Numeracy Ability Medium Numeracy Ability Low Cognitive Ability Medium Cognitive Ability High Numeracy Ability High Cognitive Ability Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: Respondents aged 16–64. Low, medium, and high numeracy ability are defined as the bottom, middle, and top third of the numeracy ability distribution. Physical work is defined as regularly lifting or pulling anything weighing at least 50 pounds (25 kilograms) as part of work. For women and rural workers, higher non-cognitive skills scores meant more frequent use of those skills on the job. The share of respondents having contact with people other than colleagues (such as clients, customers, or students) is higher among those who self-reported to be relatively open and sociable (Figure 29). Questions designed to determine openness/sociability include “are you talkative?” and “are you outgoing and sociable?” More details on the non-cognitive skills measures are presented in Appendix B. 30 Figure 29: Workers who are more open and sociable tend to have more contact with clients, customers, and colleagues, 2013 80 Share of respondents having contact with clients, 70 customers, and colleagues (%) 60 50 40 30 20 10 0 All Urban Rural Men Women Low Openness/Sociability Medium Openness/Sociability High Openness/Sociability Ability Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: Respondents aged 16–64. 31 4 Skill Formation over the Life Cycle This section assesses skill formation coupled with educational attainment levels in Uzbekistan. Relying on the rich household survey data from Uzbekistan in 2013, it addresses a fundamental question: namely, does the education and training system in Uzbekistan meet the employers’ demand for skills? Or more specifically, does the education and training system impart the cognitive and non-cognitive skills needed to successfully participate in the labor market in Uzbekistan? The short answer is that although skills are formed in every stage of an individual’s life and by a variety of actors, including families who play a very important role, the government’s track record of meeting the demand for skills in the workplace is mixed. Uzbekistan has performed better than most middle-income countries to ensure access to general education, but employer demands for workers with tertiary level skills has gone unmet. With a higher demand for “new economy” skills in the labor market, as seen around the world almost without exception, Uzbekistan will need to find a way to compete globally. This study finds that a positive correlation exists between the education and training system and cognitive and non-cognitive skills for women in Uzbekistan. However, the correlation is statistically insignificant for men, which is a peculiar finding and raises questions about the admissions, curricula, and graduation process, especially at the tertiary level. 4.1 Skills are developed throughout the life cycle of an individual Skills are developed through all stages of life—from conception to preschools, in primary and secondary schools, in tertiary education, and on the job—and there are sensitive and critical development periods for each type of skill. Recent evidence suggests that the most sensitive and critical moments for skill-building differ by skill type; these “malleable” periods are depicted in green in Figure 30. Cognitive and non-cognitive skills are formed earlier on in life, while technical skills are developed later. The early childhood period is critical in the development of cognitive skills. This stage marks the first step of skill- building, and it can be particularly critical in closing the gap between children from poorer and better-off households. In fact, there are strong indications that the most critical moment for cognitive skill-building is before the age of 5. By ages 8–10, the foundations of an individual’s cognitive abilities are well set. Non- cognitive skills are then continuously developed throughout adolescence and into adulthood. 24 24 World Bank (2013g). 32 Figure 30: Skills are formed in all stages of life—very stylized Source: World Bank (2013g). To develop the skills needed for productive employment, past studies have shown that developing solid cognitive and non-cognitive skills early in life is critical. Strong cognitive and non-cognitive skills feed into the successful acquisition of technical skills, as solid cognitive and non-cognitive foundations will help workers to strengthen their technical skills throughout their working lives. 25 They also determine a person’s readiness to learn over their life cycle by shaping the capacity and motivation to absorb new knowledge, adapt, and solve new problems. This is crucial in a dynamic economic environment where specific skills can be rendered obsolete. This is not to say that generic skills, particularly non-cognitive skills, are an alternative to academic qualifications. Instead, careful attention to them is a powerful way to improve educational attainment, life-long learning, and thus employability. 26 4.2 Access to general education is good, but preschool and tertiary school coverage is low Uzbekistan has made great strides to ensure universal access to basic education, yet important challenges remain. Even though enrollment levels in primary and secondary education are now on par with the developed countries of the OECD, access to pre-primary and tertiary education falls short, given Uzbekistan’s level of development. Only 26 percent of preschool age children were enrolled in 2011, which compares unfavorably to Kazakhstan (48 percent), Malaysia (69 percent), Russia (90 percent), and the economies of the OECD (86 percent). Preschool education is important because healthy cognitive and emotional development in the early years lead to tangible economic returns. Early interventions are more cost effective when compared with remedial services later in life. Meanwhile, enrollment in tertiary education is among the lowest in the world, with only 9 percent of secondary graduates pursuing further studies. In contrast, tertiary enrollment rates in comparator countries range from 41 percent in Malaysia and Kazakhstan to 76 percent in Russia and the OECD countries (Figure 31). The consequences of underinvestment in preschool and post-secondary education are serious. Research shows that investing in children early on can be the most cost-effective way to impart skills that contribute to higher productivity later in life. Furthermore, 25 World Bank (2013g). 26 Arias et al. (2014). 33 expanding access to preschool services would allow more women to enter the labor market, increasing Uzbekistan’s low female labor force participation rates. In addition to the results presented, Appendix D: Summary Tables contains more detailed results on educational attainment in Uzbekistan among the working age population. Figure 31: While Uzbekistan’s primary and secondary enrollments are relatively high, preschool and tertiary enrollments fall short, 2011 Gross enrollment rates, by education level 111 104 103 106 102 99 100 100 95 96 100 90 89 89 % of population in age group) 86 82 80 7576 69 69 Enrollment (as 57 5556 60 48 4142 40 26 26 20 9 0 Pre-primary Primary Secondary Tertiary Uzbekistan Kazakhstan Malaysia Turkey ECA OECD Russia Source: Authors’ calculations using World Bank, EdStats database. Note: The gross enrollment rate is the ratio of the number of individuals who are actually enrolled in schools divided by the number of children who are of the corresponding school enrollment age. More than half of all students currently enrolled in higher education belong to households in the top consumption quintile, which may imply financial accessibility barriers (Figure 32). This suggests that there may be individuals with a high cognitive ability in low-income households who are unable to attend higher education. This, too, can explain why some individuals with a low level of completed education have a stronger cognitive ability than others with a high level of completed education. Figure 32: Tertiary education is accessible mostly to better off families, 2013 100% 90% 17% 17% 29% 80% 18% 70% 22% 59% 60% 20% 23% 50% 23% 40% 18% 30% 22% 17% 20% 18% 20% 16% 10% 21% 17% 14% 5% 0% 2% Primary Basic Secondary special/technical Higher education First Quintile Second Quintile Third Quintile Fourth Quintile Fifth Quintile Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. 34 Of course, learning does not end at the secondary or tertiary level; a significant portion of learning takes place on the job and in what is often termed adult (post-formal) education. This includes skills acquired while learning by doing and during on-the-job training. In the United States, it is estimated that on- the-job training contributes approximately one-quarter to one-half of all human capital (Heckman et al., 1998). Not surprisingly, there is ample literature documenting (albeit largely in OECD countries) that adult education and training increases worker productivity. 27 Despite international evidence about the importance of post-formal education, few firms in Uzbekistan offer formal training programs to full-time employees. Firms tend to underinvest in their own employees, possibly as a result of market failures that dissuade such investments. In Uzbekistan, less than one-quarter of all firms offer their full-time employees formal training programs. This is significantly less than the proportion of firms offering training in Eastern Europe and less than the training offered in neighboring Eurasian countries. To illustrate, consider that almost 70 percent of firms in the Czech Republic offer formal training to their full-time employees and 60 percent of Polish firms do the same (Figure 33). Figure 33: Few Eurasian firms offer formal training programs to full-time employees 80 70 60 50 40 30 20 10 0 Lithuania Moldova Belarus Czech Republic Poland Kazakhstan Russia Ukraine Tajikistan Uzbekistan Armenia Kyrgyzstan Georgia Azerbaijan Eurasia Comparators Source: Gill et al. (2014), based on the EBRD-World Bank Business Environment and Enterprise Performance Surveys (BEEPS), 2009. 4.3 The correlation between educational attainment and cognitive skills outcomes is weak, and there is significant variation within education levels Cognitive skills are correlated with educational attainment among women, but surprisingly, among men this correlation only holds for numeracy skills. For women, there is a significant difference in all three cognitive skills indicators (memory, literacy, numeracy) across education levels (Figure 34). Among working-age men, there are no significant differences in memory and literacy skills outcomes across educational attainment levels, not even between men with a secondary general education and men with 27 A study by the OECD (2004) shows, among other things, that employee training impacts wage growth of young or highly educated employees, and employee training allows attaining and maintaining the competences required to bring productivity in line with market wages of older and lower-educated workers. 35 tertiary education. However, there is a significant difference in numeracy skills between men with secondary general education and men with either secondary special/technical or tertiary education. While the cognitive skills outcomes are not a direct test of a person’s grasp of school-related knowledge, it is nevertheless surprising that men do not improve their literacy skills with higher educational attainment. Further work is clearly warranted, but the results raise questions about the admissions, curricula, and graduation process in Uzbekistan, especially at the tertiary level. Figure 34: While cognitive skills are generally correlated with educational attainment among women, they are not as clearly correlated among men, 2013 Cognitive skill mean scores, by Cognitive skill mean scores, by educational attainment: Men educational attainment: Women 0.4 0.4 Standard deviations from the sample mean Standard deviations from the sample mean 0.3 0.3 0.2 0.2 0.1 0.1 0 0 Memory Literacy Numeracy** Memory*** Literacy** Numeracy*** -0.1 -0.1 -0.2 -0.2 -0.3 -0.3 -0.4 -0.4 Secondary general Secondary special/technical Tertiary Secondary general Secondary special/technical Tertiary Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: Respondents aged 25–64. F-test results are depicted by *** significant at the 1% level; ** significant at the 5% level; * significant at the 10% level. The wide dispersion in cognitive skills among individuals with the same education level may suggest issues of quality of the education system, though results have to be interpreted with caution. It is important to note that there are several limitations to interpreting the cognitive skills outcome distribution (Figure 35). The cognitive skills measures are likely to suffer from measurement error. In particular, the cognitive skills questions are not able to precisely distinguish individuals with a high ability from individuals with a very high ability. This can be a reason for the heterogeneity in skills outcomes among individuals with the same education level. Given these caveats, however, there are individuals who have completed secondary education but have higher cognitive skills outcomes than individuals who have completed a tertiary education. These individuals might have had the potential to continue their education but have not had the opportunity to do so. Barriers to accessing education may play a role in these results. In addition, variation in the quality of education could be another explanation for the observed results. 36 Figure 35: Cognitive skills distributions overlap despite educational attainment, 2013 Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: Respondents aged 25–64. There is little data available about the quality of education in Uzbekistan, but the most recent OECD Program for International Student Assessment (PISA) results in the neighboring country of Kazakhstan indicate that there is cause for concern. PISA participation has led a number of countries to realize that their education systems are greatly in need of reform and has, in fact, prompted reforms. While there are several examples, Germany and Poland are good case studies of countries that were spurred by weak PISA results to reform their education systems and thereby improve their PISA performance. The OECD PISA captures the cognitive abilities—reading, math, and science—of 15-year-olds and thus reflects the new generation of labor market entrants. Both Kazakhstan and the Kyrgyz Republic participated in 2009, and both scored well below other participating countries, such as Mexico and Turkey, both of which have comparable GNI per capita levels. 4.4 There is a weak correlation between educational attainment and non-cognitive skills Good quality schooling can enhance the non-cognitive skills of students. Non-cognitive skills can be produced in schools under the right circumstances. Several studies in the psychology literature have shown the important role of non-cognitive skills on schooling performance, 28 comparable to that of cognitive skills. 28 Wolfe and Johnson (1995); Duckworth and Seligman (2005). 37 At the same time, schooling itself is also a determinant of non-cognitive skills in individuals. 29 In Uzbekistan, the majority of non-cognitive skills outcomes are not significantly correlated with educational attainment. Figure 36 shows that the majority of the non-cognitive skills measured do not differ significantly across individuals with varying educational attainment levels. Among men, only openness/sociability is significantly correlated with educational attainment. Among women, the decision- making score and the growth mindset indicator is significantly higher if a woman completed more education than a secondary general degree. In general, however, there is a lack of correlation between educational attainment and non-cognitive skills, which suggests a disconnect in the educational system. Figure 36: Non-cognitive skills are not significantly different by education level, 2013 Non-cognitive skill mean scores, by Non-cognitive skill mean scores, by educational attainment: Men educational attainment: Women 0.3 0.3 Standard deviations from the sample mean Standard deviations from the sample mean 0.2 0.2 0.1 0.1 0 0 -0.1 -0.2 -0.1 -0.3 Secondary general Secondary special/technical Tertiary Secondary general Secondary special/technical Tertiary Source: Authors’ calculations using World Bank/GIZ, Uzbekistan Jobs, Skills, and Migration Survey, 2013. Note: Respondents aged 25–64. F-test results are depicted by *** significant at the 1% level; ** significant at the 5% level; * significant at the 10% level. 29 Heckman, Stixrud and Urzua (2006). 38 5 The Skills Roadmap in Uzbekistan Uzbekistan aims to take advantage of its young and growing population and make better use of its human capital. By boosting its employment outcomes, Uzbekistan intends to attain its Vision 2030 goal of becoming an upper-middle-income country by the year 2030. Policy makers, however, recognize that attaining that goal will require skilling up Uzbekistan’s current and future workforce. While this is not an easy goal, it is an achievable one. Policy makers will have to respond to the increased demands from employers for strong cognitive and non-cognitive skills. There is a strong demand for skills in the Uzbek economy, as evidenced by significant positive labor market returns to both cognitive and non-cognitive skills, yet workers themselves complain about the inadequacy of their training for productive employment. A number of actors play a role in building skills throughout the life cycle of an individual, targeting the current and future workforce to different degrees. Policies can target the future workforce, usually by focusing on families and communities and the formal education system, and/or the current workforce, by focusing on adult training institutions and firms. Families and communities play an important role in skill development of the future workforce, especially during the early years by ensuring good nutrition and stimulation, but they continue to play a role throughout the life cycle. Formal educational institutions, beginning with pre-schools and extending through to tertiary education, are also important for skill formation of future workers. Adult training institutions, which include non-traditional training institutions and second- chance educational institutions, can help to strengthen the skillsets of the current workforce. Similarly, adults derive skills at work, either during on-the-job training programs or by learning by doing. Figure 37: Actors that play a role to build skills throughout the life cycle of an individual Families and communities Future workforce Formal Adult training Skills Formation educational institutions institutions Current workforce Firms (learning by doing; on- the-job training) Source: Authors’ illustration based on Heckman (2000). 39 The policies pursued can be informed by the Skills Toward Employability and Productivity (STEP) Framework within the government’s strategic vision. The STEP framework which brings together research-based evidence and practical experience from diverse areas—from research on the determinants of early childhood development and learning outcomes to policy experience in the reforming of vocational and technical education systems and labor markets. 30 This report recommends five policy goals to improve the skills of the current and future workforce in Uzbekistan:  Getting children off to the right start by expanding access to quality ECD programs, which are critical to ensuring that all children acquire the cognitive and non-cognitive skills that are conducive to high productivity and flexibility that are observed later in working life.  Ensuring that all students learn by modernizing the curricula and improving teaching quality in order to address the weak link between educational attainment and cognitive and non-cognitive skills.  Building job-relevant skills that employers demand by implementing selective active labor market programs, with a particular focus on discouraged workers and to increase the female labor force participation, and incentivizing firms to provide training to workers.  Encouraging entrepreneurship and innovation by increasing quality tertiary education access for motivated students, which can ensure that higher education graduates possess market-valued skills and that investments in higher education pay off.  Matching the supply of skills with employer demand by improving labor market information systems, which can help to make labor markets more efficient by alleviating lack of information about vacancies by jobseekers and help to secure jobs through job signaling. Figure 38: Policy reform priorities to boost skill outcomes of the current and future workforce in Uzbekistan 5. Matching the supply of 4. Encouraging skills with entrepreneursh employer 3. Building job- ip and demand by relevant skills innovation by improving labor that employers increasing market 2. Ensuring demand by quality tertiary information that all systems. students learn implementing education access. 1. Getting children off to by modernizing labor market the right start the curricula and programs. by expanding improving access to quality teaching quality. early childhood development programs. Source: Authors’ illustration for Uzbekistan based on Valerio et al. (2014). 30 Valerio et al. (2014). 40 5.1 Get children off to the right start by expanding access to quality early childhood development programs Uzbekistan needs to expand access to quality ECD programs to enhance cognitive and non- cognitive skills development from an early age. Getting children off to the right start, with quality ECD, can contribute to technical, cognitive, and non-cognitive skills conducive to high productivity and flexibility in the work environment. As discussed earlier, only 26 percent of preschool age children were enrolled in 2011, which is far below the enrollment rate of OECD countries (86 percent), Russia (90 percent), and neighboring countries such as Kazakhstan (48 percent). The case for investing in preschools has been made multiple times and by numerous researchers. The argument is that the intervention is usually the most cost effective way to produce the desired cognitive and non-cognitive skills in students, and hence, to produce the desired characteristics that will lead to productive workers. Well-conceived preschool education has been shown to produce students who are more successful in subsequent schooling and are well adjusted socially and emotionally. Furthermore, the benefits of well-conceived preschool programs generally dwarf the costs and hence, universal access is a desirable goal. The Nobel laureate James Heckman estimates that every dollar invested in high-quality ECD programs yields a 7–10 percent return per annum, and, in fact, policies that provide ECD to disadvantaged children have even higher returns. 31 These high rates of return are fairly consistent in the literature because ECD, it is argued, raises the returns to investment later in life as children learn how to learn. A robust literature has concluded that delays in cognitive development during the early years of a child’s life lead to reduced employability, productivity, and overall welfare. 5.2 Ensure that all students learn by modernizing the curricula and improving teaching quality Policy makers need to ensure that all students learn by modernizing the curricula and improving teaching quality. Developing modern curricula and teaching methods will strengthen the link between educational attainment and cognitive and non-cognitive skills. There is a tremendous amount of research and emerging consensus on the benefits of interventions designed to support skills development during distinct life stages. Although many of these programs have only been evaluated in the United States and other high- income countries, there is increasing evidence of their universal effectiveness. In a number of countries, social-emotional interventions have been integrated into the regular academic curriculum. The empirical evidence for Uzbekistan clearly shows that stronger cognitive and non-cognitive skills enable workers to obtain better jobs. Therefore, improving skills is a key policy objective for Uzbekistan’s development plans. While the country has achieved universal access to general education and completion rates are good, policy makers also need to improve the quality of education, which also implies improving the cognitive and non- cognitive skills gained through schooling. Higher quality education will yield better problem-solvers, more critical thinkers, better communicators, and effective team players—in short, a better workforce. Policy makers also need to focus on reforming the pre-primary and primary education system so as to strengthen non-cognitive skills formation. An increasing number of countries worldwide have integrated non-cognitive learning into the regular academic curriculum by training teachers, adopting a structured curriculum, and investing in efforts to improve the school climate. Non-cognitive skills acquired at an early age can lead to lasting habits and characteristics of social interaction. Schools are a key channel for skills development at a young age, given that children are typically in a single classroom with a single teacher 31 Heckman et al. (2009). 41 and the same group of peers for an entire school year. This “single point” of entry reduces the cost of interventions and increases the likelihood of impacting skills development. 5.3 Build job-relevant skills that employers demand by implementing selective active labor market programs Uzbekistan could build job-relevant skills that employers demand by implementing selective active labor market programs that respond to domestic and international labor market needs. Building job- relevant skills will require a multi-pronged effort that includes: (i) addressing the technical or job-specific skills gaps by implementing labor market programs; (ii) addressing market failures that prevent firms from providing on-the-job training (OJT) and incentivizing firms to provide OJT; (iii) improving migrant skills to increase their earning capacity, and therefore their ability to support their families in Uzbekistan; and (iv) strengthening the link between migrants’ skills and labor market needs abroad, the quality of workers’ skills, and the visibility of those skills. Up-skilling the workforce in Uzbekistan would boost employment rates by addressing the technical or job-specific skills gaps by implementing selective active labor market programs (ALMPs). While the country’s job creation rate has kept up with population growth rates, ALMPs can further boost the employment rates by activating youth and women. Uzbekistan spends relatively little on ALMPs in comparison to other European and Central Asian countries; therefore, allocating funds to support effective programs is likely to yield desirable outcomes. ALMPs can include job placement assistance services, counseling with employment advisors, job application and interview preparation, CV composition, informational interviews, and in-depth assessment of skills and abilities. One element in common is that all ALMPs are designed to encourage the unemployed, the discouraged, and the inactive populations to more actively seek jobs, thereby improving their prospects for employment. Countries offer a menu of ALMPs including training programs (including socio-emotional skills), public works projects, employment subsidies, and matching workers and jobs through intermediation services. As with any program, their efficacy should to be evaluated to gauge whether the design of the program is optimal given stated objectives. Uzbekistan can also benefit from addressing market failures that prevent firms from providing on- the-job training (OJT) and incentivizing them to do so. OJT is an important channel through which workers upgrade their skills during their time at work. It is also a vehicle that can help firms adopt new technologies and new business practices. In Uzbekistan, less than one-quarter of all firms offer their full-time employees formal training programs. This is a lower proportion than most countries in Europe and Central Asia. In Europe, OJT is far more common. For example, almost 70 percent of firms in the Czech Republic offer formal training to their full-time employees; 60 percent of Polish firms do so; and 54 percent of Lithuanian firms do the same. Identifying the reasons for the varying levels of OJT in each country context is a prerequisite for designing effective policy responses. Uzbekistan must design policies that strive to support firms that, despite positive expected returns, do not train their workers. To encourage the implementation of ALMPs, policy makers have several instruments at their disposal, such as credit and subsidy programs or tax grants that can be used to deal with liquidity constraints and incentivize training. These types of programs have been used successfully in a number of countries in Western Europe and Eastern Asia. Similarly, improving migrant skills increases their earning capacity, and therefore their ability to support their families in Uzbekistan. To do so, policy makers can introduce pre-departure training programs for migrants to ensure that they have the basic language skills and knowledge of social services 42 provision and migrant protection programs. The Philippines, for example, carries out pre-departure reviews and approvals of contract terms, in addition to providing a mandatory pre-departure orientation. In addition, in order to help migrants secure better jobs abroad, it is important that existing skills are appropriately recognized and valued. Enhancements to the existing migrant job placement system, including better registration and pre-selection assessment, could help avoid the “brain waste” that often impacts mid-skilled workers, devaluing not only their skills while abroad, but also the benefits of the international migration experience for both the individual workers and Uzbekistan. Education and labor market reforms—such as public-private partnerships on business-friendly curriculum development, support for on-the-job training and apprenticeship programs, and improved labor market diagnostics—can also benefit international migrants. Such programs could increase Uzbek migrants’ ability to apply their skills abroad by making skills more visible to employers. This would in turn enable them to expand their skills abroad, which could then be absorbed into the domestic market upon their return, resulting in productivity and wage improvements at home. This is particularly important given the large number of migrant workers in sectors such as construction and those with secondary or technical education, and the fact that mid-skilled workers are often at the highest risk of brain waste. 32 To improve the link between migrants’ skills and labor market needs abroad, the quality of workers’ skills, and the visibility of those skills, the government of Uzbekistan could pursue a three-pronged strategy. First, develop partnerships with Ministries of Labor and business leaders in key destination countries and sectors to identify skills needs and raise the profile of Uzbek laborers. Second, conduct labor market diagnostics to identify sectors with demand for laborers, both domestically and abroad. And third, invest in improved vocational and technical training programs. 5.4 Encourage entrepreneurship and innovation by increasing tertiary education access One way for policy makers to encourage entrepreneurship and innovation is to increase tertiary education access together with other measures to create an environment that encourages investments in knowledge and creativity. Emerging evidence shows this requires innovation-specific skills (which can be developed starting early in life) and investments to connect people with ideas (such as through collaborations between universities and private companies) as well as risk-management tools that facilitate innovation. Increasing access to quality tertiary education is essential for the development of a high-skilled workforce that is entrepreneurial and innovative. Demand for high-skilled labor can be met by increasing the number of quality tertiary graduates. This is likely to become more important as the economy evolves and demands more non-routine skills, as observed in other middle- and high-income countries. A recent World Bank report has highlighted some of the key policies that can address problems associated with access to tertiary education. 33 They include: increasing the number of spaces available to entering cohorts, especially women, and differentiating degree and non-degree programs so that short-term technical and occupational courses can offer more immediate responses to the skills demanded. To ensure quality at the tertiary level, measuring the skills produced is important. The development of an independent quality assurance agency is critical for a modern higher education system. The existing State Testing Center in Uzbekistan can be further equipped to perform this role. In addition, individual institutions 32 World Bank (2013a). 33 World Bank (2014c). 43 of higher education should perform “internal” quality assurance through so-called Quality Enhancement Cells based partly on self-assessments and peer reviews by other higher education institutions. Introducing some elements of the Bologna Process, which aims to make academic degrees and quality assurance standards more comparable and compatible across Europe, would provide a structure for quality enhancement and systems integration that could allow the quality assurance system in Uzbekistan to achieve globally recognized operating standards. Tertiary graduates should be equipped with market-relevant skills. This requires a three-pronged approach. First, regular and independent market surveys should monitor the skills requirements in the labor market. Second, partnerships with both domestic and foreign academic institutions (research partnerships, faculty exchanges, and training) as well as domestic and foreign industry (modernizing curricula, laboratories, innovation platforms, research, and joint business development) can help strengthen the links between higher education institutions and the labor market. Third, more generally, ensuring high-quality equipment in relevant and priority technical fields, together with modern curricula, trained faculty and staff, and related university-industry linkages is crucial. 5.5 Match the supply of skills with employer demand by improving labor market information systems Uzbekistan needs to match the supply of skills with employer demand improving labor market information systems. These systems can help to make labor markets more efficient enables skills to be transformed into actual employment and productivity. A key reform priority could be to improve labor market information systems to ease the transition from school to work. In Uzbekistan, more than half of all respondents (58 percent) indicated that they do not feel they are able to learn about vacancies. The problems caused by asymmetric information between job seekers and employers are more far reaching because they affect students’ educational choices, firms’ selection of workers, and/or the time that it takes to fill vacancies. 34 In other words, labor market information systems speak to the efficient allocation of resources in a country. Facilitating information flows in Uzbekistan will be important, especially for youth and first-time job-seekers, because it will help to dismantle the current rigid manpower planned system in which the number of university slots is determined based on the number of government jobs available for graduates. A number of modernizing countries have successfully labor market information systems designed to dismantle planned manpower education structures. In Poland, for example, employment observatories were introduced to provide information on job availability, wages, career prospects, and hiring expectations. 35 Employment observatories are also available in Chile and Colombia. The key concept behind employment observatories is that information about major industries, recent growth areas, occupations experiencing shortages, qualifications needed for jobs, etc., can help people make more informed choices about their courses and careers. Information of this type is routine in the United States, EU countries, and Australia. A number of emerging countries are also beginning to expand their labor market information systems. Employment observatories use a rich array of data to monitor and disseminate information about the labor market. The data managed by employment observatories include: (1) administrative data from public employment offices on unemployment, vacancies, and active labor market programs; (2) data from the national statistics office including labor force survey and household survey information, usually disaggregated 34 Jensen (2010); Kaas and Manger (2010); World Bank (2012). 35 Arias et al. (2014). 44 by region; and (3) data from special-topic surveys (usually “sociological”). Employment observatories use multi-media to disseminate information, ranging from traditional paper-based information to YouTube videos and text/SMS messaging. The information is disseminated at irregular intervals, dictated by the speed with which the information is processed. 45 References Aggarwal, Reena, Asli Demirgüç-Kunt, and Maria Soledad Martinez Peria. 2006. “Do Workers’ Remittances Promote Financial Development?” Policy Research Working Paper 3957, July, World Bank, Washington, DC. Agunias, Dovelyn Rannveig. 2008. “Managing Temporary Migration: Lessons from the Philippine Model.” MPI Insight, October 2008, Migration Policy Institute, Washington, DC. ———. 2011. “Running in Circles: Progress and Challenges in Regulating Recruitment of Filipino and Sri Lankan Labor Migrants to Jordan.” Migration Policy Institute, Washington, DC. Ajwad, Mohamed Ihsan, Pamela Dale, Susanna Hayrapetyan, Stefan Hut, Igor Kheyfets, Rebecca Lacroix, Manjula M. Luthria, Marvin Ploetz, and Mark C. Woodward. 2014. “Uzbekistan Vision 2030— Background Note: Achieving Social Sustainability in Uzbekistan.” World Bank, Washington, DC. Ajwad, Mohamed Ihsan, Joost de Laat, Stefan Hut, Jennica Larrison, Ilhom Abdulloev, Robin Audy, Zlatko Nikoloski, and Federico Torracchi. 2014. “The Skills Road: Skills for Employability in the Kyrgyz Republic.” World Bank, Washington, DC. Ajwad, Mohamed Ihsan, Stefan Hut, Ilhom Abdulloev, Robin Audy, Joost de Laat, Sachiko Kataoka, Jennica Larrison, Zlatko Nikoloski, and Federico Torracchi. 2014. “The Skills Road: Skills for Employability in Tajikistan.” World Bank, Washington, DC. Alam, Asad, Mamta Murthi, Ruslan Yemtsov, Edmundo Murrugarra, Nora Dudwick, Ellen Hamilton, and Erwin Tiongson. 2005. Growth, Poverty, and Inequality: Eastern Europe and the Former Soviet Union. Washington, DC: World Bank. Almeida, Rita, Jere Behrman, and David Robalino, eds. 2012. The Right Skills for the Job? Rethinking Training Policies for Workers. Washington, DC: World Bank. Almlund, Mathilde, Angela Lee Duckworth, James Heckman, and Tim Kautz. 2011. “Personality Psychology and Economics.” In Handbook of the Economics of Education, Vol. 4, eds. Eric A. Hanushek, Stephen J. Machin, and Ludger Woessmann, 1–181. Amsterdam: Elsevier. Arias, Omar S., Carolina Sánchez-Páramo, María E. Dávalos, Indhira Santos, Erwin R. Tiongson, Carola Gruen, Natasha de Andrade Falcão, Gady Saoivici, and Cesar A. Cancho. 2014. Back to Work: Growing with Jobs in Europe and Central Asia. Washington, DC: World Bank. Autor, David H., Frank Levy, and Richard J. Murnane. 2003. “The Skill Content of Recent Technological Change: An Empirical Exploration.” The Quarterly Journal of Economics 118 (4): 1279–333. Avato, Johanna, Johannes Koettl, and Rachel Sabates-Wheeler. 2009. “Definitions, Good Practices, and Global Estimates on the Status of Social Protection for International Migrants.” Social Protection Discussion Paper No. 0909, World Bank, Washington, DC. Barro, Robert J. 2001. “Human Capital and Growth.” American Economic Review 91 (2): 12–7. 46 Berlinski, Samuel, Sebastian Galiani, and Paul Gertler. 2009. “The Effect of Pre-Primary Education on Primary School Performance.” Journal of Public Economics 93 (1–2): 219–34. Blom, Andreas, and Hiroshi Saeki. 2011. “Employability and Skill Set of Newly Graduated Engineers in India.” Policy Research Working paper 5640, World Bank, Washington, DC. Borghans, Lex, Angela Lee Duckworth, James J. Heckman, and Bas ter Weel. 2008. “The Economics and Psychology of Personality Traits.” Journal of Human Resources 43 (4): 972–1059. Borjas, George J. 1987. “Self-Selection and the Earnings of Immigrants.” The American Economic Review 77 (4): 531–53. Bowles, Samuel, Herbert Gintis, and Melissa Osborne. 2001. “The Determinants of Earnings: A Behavioral Approach.” Journal of Economic Literature 39 (4): 1137–76. Carneiro, Pedro, James J. Heckman, and Edward Vytlacil. 2006. “Estimating Marginal and Average Returns to Education.” Unpublished manuscript. University of Chicago, Department of Economics. Chiquiar, Daniel, and Gordon H. Hanson. 2005. “International Migration, Self-Selection, and the Distribution of Wages: Evidence from Mexico and the United States.” Journal of Political Economy 113 (2): 239–81. Cunha, Flávio, and James J. Heckman. 2007. “The Technology of Skill Formation.” American Economic Review 97 (2): 31–47. ———. 2008. “Formulating, Identifying and Estimating the Technology of Cognitive and Noncognitive Skill Formation.” Journal of Human Resources 43 (4): 738–82. ———. 2010. “Investing in our Young People.” Working Paper 16201, National Bureau of Economic Research (NBER), Cambridge, MA. Cunha, Flávio, James J. Heckman, Lance Lochner, and Dimitryi V. Masterov. 2006. “Interpreting the Evidence on Life Cycle Skill Formation.” In Handbook of the Economics of Education, Vol. 1, eds. Eric A. Hanushek and Finis Welch, 697–812. Amsterdam: North-Holland. Cunha, Flávio, James J. Heckman, and Salvador Navarro. 2005. “Separating Uncertainty from Heterogeneity in Life Cycle Earnings.” Oxford Economic Papers 57 (2): 191–261. Cunha, Flávio, James J. Heckman, and Susanne M. Schennach. 2010. “Estimating the Technology of Cognitive and Noncognitive Skill Formation.” Econometrica 78 (3): 883–931. Danzer, Alexander M., and Barbara Dietz. 2014. “Labour Migration from Eastern Europe and the EU’s Quest for Talents.” Journal of Common Market Studies 52 (2): 183–99. del Rosario, Teresita. 2008. “Best Practices in Social Insurance for Migrant Workers: The Case of Sri Lanka.” International Labour Organization (ILO) Asian Regional Programme on Governance of Labour Migration, Working Paper No. 12, ILO, Bangkok. Díaz, Juan José, Omar Arias, and David Vera Tudela. 2012. “Does Perseverance Pay as Much as Being Smart? The Returns to Cognitive and Non-Cognitive Skills in Urban Peru.” Unpublished paper, World Bank, Washington, DC. 47 Duckworth, Angela L., and Martin E.P. Seligman. 2005. “Self-Discipline Outdoes IQ in Predicting Academic Performance of Adolescents.” Psychological Science 16 (12): 939–44. European Centre for the Development of Vocational Training (Cedefop). 2010. The Skill Matching Challenge: Analysing Skill Mismatch and Policy Implications. Luxembourg: Publications Office of the European Union. European Commission. 2012. Employment and Social Developments in Europe 2012. Luxembourg: Publications Office of the European Union. Gaventa, John, and Gregory Barrett. 2010. “So What Difference Does it Make? Mapping the Outcomes of Citizen Engagement.” Working Paper Vol. 2010, No. 347, Institute of Development Studies. University of Sussex. Gaventa, John, and Rosemary McGee. 2013. “The Impact of Transparency and Accountability Initiatives.” Development Policy Review 31 (s1): s3–s28. Gill, Indermit S., Ivailo Izvorski, Willem van Eeghen, and Donato De Rosa. 2014. “Diversified Development: Making the Most of Natural Resources in Eurasia.” World Bank, Washington, DC. Glewwe, Paul. 2002. “Schools and Skills in Developing Countries: Education Policies and Socioeconomic Outcomes.” Journal of Economic Literature 40 (2): 436–82. Glewwe, Paul, and Hanan Jacoby. 1994. “Student Achievement and Schooling Choice in Low-Income Countries: Evidence from Ghana.” Journal of Human Resources 29 (3): 843–64. Hanushek, Eric A., and Ludger Woessmann. 2008. “The Role of Cognitive Skills in Economic Development.” Journal of Economic Literature 46 (3): 607–68. ———. 2012. “Do Better Schools Lead to More Growth? Cognitive Skills, Economic Outcomes, and Causation.” Journal of Economic Growth 17 (4): 267–321. Heckman, James J. 2000. “Policies to Foster Human Capital.” Research in Economics 54 (1): 3–56. ———. 2006. “Skill Formation and the Economics of Investing in Disadvantaged Children.” Science 312 (5782): 1900–2. Heckman, James J., and Pedro Carneiro. 2003. “Human Capital Policy.” IZA Discussion Paper Series No. 821, Bonn, Germany. Heckman, James J., Lance Lochner, and Christopher Taber. 1998. “Explaining Rising Wage Inequality: Explorations with a Dynamic General Equilibrium Model of Labor Earnings with Heterogeneous Agents.” Review of Economic Dynamics 1: 1–58. Heckman, James J., Lance J. Lochner, and Petra E. Todd. 2006. “Earnings Functions, Rates of Return and Treatment Effects: The Mincer Equation and Beyond.” In Handbook of the Economics of Education, Vol. 1, Chapter 7, 301–458, eds. Eric Hanushek and Finis Welch. Amsterdam: Elsevier. 48 Heckman, James J., and Dimitriy V. Masterov. 2007. “The Productivity Argument for Investing in Young Children.” Applied Economic Perspectives and Policy 29 (3): 446–93. Heckman, James J., Jora Stixrud, and Sergio Urzua. 2006. “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior.” Journal of Labor Economics 24 (3): 411–82. Heckman, James J., Seong Hyeok Moon, Rodrigo Pinto, Peter A. Savelyev, and Adam Yavitz. 2010. “The Rate of Return to the HighScope Perry Preschool Program.” Journal of Public Economics 94 (1-2): 114– 28. Helmers, Christian, and Manasa Patnam. 2011. “The formation and evolution of childhood skill acquisition: Evidence from India.” Journal of Development Economics 95 (2): 252–66. Human Rights Watch. 2009. Are You Happy to Cheat Us? Exploitation of Migrant Construction Workers in Russia. New York: Human Rights Watch. International Monetary Fund (IMF). 2013. “Republic of Uzbekistan: 2012 Article IV Consultation.” IMF, Washington, DC. International Organization for Migration (IOM). 2011. “Labour Migration from Colombo Process Countries. Good Practices, Challenges, and Way Forward.” IOM, Geneva. Iskander, Natasha. 2013. “Moroccan Migrants as Unlikely Captains of Industry: Remittances, Financial Intermediation, and La Banque Centrale Populaire.” In How Immigrants Impact their Homelands, ed. Susan Eckstein and Adil Najam, 156–90. Durham, NC: Duke University Press. Jensen, Robert. 2010. “The (Perceived) Returns to Education and the Demand for Schooling.” Quarterly Journal of Economics 125 (2): 515–48. Joshi, Anuradha. 2013. “Do They Work? Assessing the Impact of Transparency and Accountability Initiatives in Service Delivery.” Development Policy Review 31 (S1): s29–s48. Kaas, Leo, and Christian Manger. 2010. “Ethnic Discrimination in Germany’s Labor Market: A Field Experiment.” IZA Discussion Paper 4741, Institute for the Study of Labor, Bonn, Germany. Kibria, Nazli. 2011. “Working Hard for the Money: Bangladesh Faces Challenges of Large-Scale Labor Migration.” Migration Policy Institute, Washington, DC. Koettl, Johannes, and Michael Weber. 2012. “Does Formal Work Pay? The Role of Labor Taxation and Social Benefit Design in the New EU Member States.” IZA Discussion Paper No. 6313, Institute for the Study of Labor, Bonn, Germany. Koettl, Johannes, Truman Packard, and Claudio E. Montenegro. 2012. In From the Shadow: Integrating Europe’s Informal Labor. Washington, DC: World Bank. Lanzona, Leonardo A. 1998. “Migration, Self-Selection and Earnings in Philippine Rural Communities.” Journal of Development Economics 56 (1): 27–50. 49 Mansoor, Ali, and Bryce Quillin, eds. 2007. Migration and Remittances. Eastern Europe and the Former Soviet Union. Washington, DC: World Bank. Migration and Remittances Peer-Assisted Learning (MiRPAL) Network and World Bank. 2011. Migration and Remittance Profiles: Armenia, Belarus, Kazakhstan, Kyrgyz Republic, Moldova, Russian Federation, Tajikistan, Ukraine, and Uzbekistan. World Bank, Washington, DC. Mincer, Jacob A. 1974. Schooling, Experience and Earnings. New York: Columbia University Press. Mountford, Andrew. 1997. “Can a brain drain be good for growth in the source economy?” Journal of Development Economics 53 (2): 287–303. Murnane, Richard J., John B. Willett, and Frank Levy. 1995. “The Growing Importance of Cognitive Skills in Wage Determination.” The Review of Economics and Statistics 77: 251–66. Nadeau, Sophie, Naoko Kataoka, Alexandria Valerio, Michelle J. Neuman, and Leslie Kennedy Elder. 2011. “Investing in Young Children: An Early Childhood Development Guide for Policy Dialogue and Project Preparation.” World Bank, Washington, DC. Nikoloski, Zlatko, and Mohamed Ihsan Ajwad. Forthcoming. “Cognitive and Non-Cognitive Skills Affect Employment Outcomes: Evidence from Central Asia.” World Bank, Washington, DC. Organisation for Economic Co-operation and Development (OECD). 2004. “Improving Skills for More and Better Jobs: Does Training Make a Difference?” In OECD Employment Outlook 2004, 183–224. Paris: OECD Publishing. ———. 2010. “Investing in Human and Social Capital: New Challenges.” OECD Publishing, Paris. ———. 2012. Free Movement of Workers and Labour Market Adjustment: Recent Experiences from OECD Countries and the European Union. Paris: OECD Publishing. Orozco, Manuel. 2005. “International Financial Flows and Worker Remittances: Best Practices.” Inter- American Dialogue, Washington, DC. ———. 2006. “International Flows of Remittances: Cost, competition and financial access in Latin America and the Caribbean—toward an industry scorecard.” Report presented at the meeting on Remittances and Transnational Families, sponsored by the Multilateral Fund of the Inter-American Development Bank and the Annie E. Casey Foundation, 12 May. Inter-American Dialogue, Washington, DC. Orrenius, Pia M., and Madeline Zavodny. 2005. “Self-selection among undocumented immigrants from Mexico.” Journal of Development Economics 78 (1): 215–40. Pierre, Gaële, María Laura Sánchez Puerta, and Alexandria Valerio. Forthcoming. “STEP Skills Measurement Surveys, Innovative Tools for Assessing Skills.” World Bank, Washington, DC. Ratha, Dilip. 2006. “Leveraging Remittances for Development.” In Migration, Trade, and Development: Proceedings of the 2006 Conference on Migration, Trade and Development. Dallas, Texas: Federal Reserve Bank of Dallas. 50 ———. 2008. “Temporary Agricultural Worker Programs between Jamaica and North America.” Working paper prepared for the World Bank, Mimeo, World Bank, Washington, DC. Ratha, Dilip, Sanket Mohapatra, and Elina Scheja. 2011. “Impact of Migration on Economic and Social Development. A Review of Evidence and Emerging Issues.” Policy Research Working Paper No. 5558, World Bank, Washington, DC. Rutkowski, Jan. 1996. “High Skills Pay Off: The Changing Wage Structure during Economic Transition in Poland.” Economics of Transition 4: 89–112. ———. 2001. “Earnings Inequality in Transition Economies of Central Europe: Trends and Patterns during the 1990s.” Social Protection Discussion Paper 0117, World Bank, Washington, DC. Sánchez Puerta, María Laura. 2009. “Towards Comprehensive Training.” SP Discussion Paper No. 0924, World Bank, Washington, DC. Segal, Carmit. 2008. “Motivation, Test Scores, and Economic Success.” Working Paper No. 1124, Department of Economics and Business, Universitat Pompeu Fabra, Barcelona. Siddiqui, Tasneem. 2012. “Impact of Migration on Poverty and Development.” Migrating Out of Poverty Research Programme Consortium Working Paper No. 2, September 2012. Available online at: http://migratingoutofpoverty.dfid.gov.uk/files/file.php?name=wp2-impact-of-migration-on- poverty-and-development.pdf&site=354 (accessed on 28 January 2014). Shonkoff, Jack P., and Deborah A. Phillips. 2000. “From Neurons to Neighborhoods: The Science of Early Childhood Development.” Washington, DC: National Academy Press. Sondergaard, Lars, Mamta Murthi, with Dina Abu-Ghaida, Christian Bodewig, and Jan Rutkowski. 2012. Skills, Not Just Diplomas: Managing Education for Results in Eastern Europe and Central Asia. Washington, DC: World Bank. Spear, Linda P. 2000. “The adolescent brain and age-related behavioral manifestations.” Neuroscience Biobehavior Review. 24 (4): 417–63. Staneva, Anita V., Reza Arabsheibani, and Philip D. Murphy. 2010. “Returns to Education in Four Transition Countries: Quantile Regression Approach.” IZA Discussion Paper 5210, Institute for the Study of Labor, Bonn, Germany. Tomas, Patricia. 2009. “Protecting Migrant Workers from the Philippines.” Asian Regional Programme on Governance of Labour Migration, ILO Regional Office for Asia and the Pacific Working Paper No. 21, ILO, Bangkok. Vietnamese Ministry of Labor. 2012. “Vietnamese TVET Background Paper. Reflections on Conference Topics.” Prepared for the Regional TVET Conference in Viet Nam, October 10–11, 2012. Valerio, Alexandria, Maria Laura Sanchez Puerta, Gäelle Pierre, Tania Rajadel, and Sebastian Monroy Taborda. 2014. “STEP Skill Measurement: Snapshot 2014.” World Bank, Washington, DC. World Bank. 2010. “BEEPS At-A-Glance: 2008 Cross Country Report.” World Bank, Washington, DC. 51 ———. 2011. Migration and Remittances Factbook 2011. Washington, DC: World Bank. ———. 2012. World Development Report 2013: Jobs. Washington, DC: World Bank. ———. 2013a. Creating Valuable Skills: A New Framework for Migration as Development. Washington, DC: World Bank. ———. 2013b. “Migrant Welfare Funds in South and East Asia: Key Features and Lessons Learned.” Unpublished draft note, World Bank, Marseille. ———. 2013c. “Remittances Prices Worldwide.” Issue No. 8, December. World Bank, Washington, DC. ———. 2013d. “Uzbekistan—Economic Development and Reforms: Achievements and Challenges.” Uzbekistan Economic Report No. 3, World Bank, Washington, DC. ———. 2013e. “Republic of Uzbekistan: Improving Early Childhood Care and Education.” Report No. 71930-UZ, World Bank, Washington, DC. ———. 2013f. “Global Stock-Take of Social Accountability Initiatives for Budget Transparency and Monitoring: Key Challenges and Lessons Learned.” Budget Transparency Initiative Report No. 81543, World Bank, Washington DC. ———. 2013g. Skilling Up Vietnam: Preparing the Workforce for a Modern Market Economy. Vietnam Development Report 2014. Washington, DC: World Bank. ———. 2014a. Global Financial Development, World DataBank. Available on the World Bank Data website: http://databank.worldbank.org/data/views/variableselection/selectvariables.aspx?source=global- financial-development#c_u. ———. 2014b. East Asia Pacific At Work: Employment, Enterprise, and Well-Being. Conference Edition. Washington, DC: World Bank. ———. 2014c. “Modernizing Tertiary Education in Uzbekistan.” World Bank, Washington, DC. World Bank and GIZ. 2013. Survey on Jobs, Skills, and Migration (Survey data). Unpublished. World Bank, Washington, DC. 52 Appendix A: Questionnaire Sections Visit 1: (All) Household Members Visit 2: Selected Household Member 1. Demographic Profile Card 1. Labor Conditions 2. Education 2. Labor Market Expectations 3. Education Expenditure 3. Russian Language Skills 4. Immigration 4. Return Migrants Pre-Departure Preparation 5. Employment 5. Future Migrants Pre-Departure Preparation 6. Labor Market 6. Pre-Departure Questions about Skills Acquisition for Future Migrants and Return Migrants 7. Work Migration Cycle 7. Most Recent Technical Skill Training 8. Most Recent Migration Event 8. Technical Skills: Reading and Writing 9. Remittances/Gifts from Non Household Member 9. Workplace Skills 10. Migration Intent 10. Non-Cognitive Skills: Part A 11. Health Expenditure 11. Non-Cognitive Skills: Part B 12. Financial Services 12. Cognitive Skills: Memory 13. Subjective Poverty 13. Cognitive Skills: Language 14. Habits And Adaptation 14. Cognitive Skills: Text Comprehension A 15. Food Consumption 15. Cognitive Skills: Text Comprehension B 16. Non-Food Consumption 16. Cognitive Skills: Table Comprehension 17. Other Non-Food Consumption 17. Cognitive Skills: Publicity Comprehension 18. Large Items of Non-Food Consumption 18. Cognitive Skills: Graph Comprehension 19. Fuel 20. Payment for Utilities and Electricity 21. Dwelling 22. Energy 23. Availability of Utility Equipment 24. Gifts 25. Government Transfers 26. Subjective Budget—Remittances 27. Selection of Member for Follow-Up Survey 53 Appendix B: Constructing Cognitive Skills Scores Methods for Scale Development and Scoring Prepared by Carly Tubbs, Ph.D. Candidate, New York University; Louise M. Bahry, Ph.D. Candidate, University of Massachusetts Amherst; Robin Audy, World Bank. Background and Measures Data for this study come from a 34-item survey module designed for use by the World Bank to assess five different “cognitive” skills. These cognitive skills can be conceptualized as falling into two domains: (1) Executive functioning skills, defined as the cognitive control capacities that enable individuals to “organize their thinking and behavior with flexibility, decrease their reactive responding to contextual cues and contingencies, and engage in self-regulated … behavior” (Welsh et al., 2010). Researchers in developmental psychology and elsewhere propose that such skills are important for school readiness and labor force attainment since they enable individuals to regulate cognitive and emotional responses that in turn allow individuals to engage more effectively in learning activities (Fuchs et al., 2005). We assessed one component of executive functioning—working memory—using a 12-item memory scale adopted from the Skills and Labor Market Survey (ENHAB) 36. These items tested the short-term recall of increasingly longer number sequences (starting with two numbers and ending with 9 numbers). Enumerators gave respondents three practice examples with two-number sequences to train the respondents on how to answer the questions, and were instructed to read out numbers at a regular pace to avoid grouping. (2) Domain-specific skills, consisting of “knowledge of ideas, facts and definitions, as well as … formulas and rules” (Boekarts, 1997, p. 164) about specific domains such as literacy and numeracy. In turn, each broader domain can be conceptualized as including other branches; mathematics, for example, includes concepts such as number recognition, arithmetic, and graph comprehension (Fuchs et al., 2005; Pinker, 1990). In this study, we assessed various concepts within the domains of literacy and numeracy using multiple-choice questions with four answer choices. Within literacy, these concepts include: (1) semantics, assessed using seven items, with five items assessing respondents’ familiarity with vocabulary, one item testing understanding of a national idiom, and one item measuring comprehension of the meaning of a complex sentence; 37 (2) reading comprehension, assessed by asking respondents to read a 257-word non-technical narrative text and then answering five questions about the text; and (3) information comprehension, assessed using four items based on instructions for taking a medicine and reading a timetable describing inter-city bus schedules. Within numeracy, concepts include: (1) arithmetic, assessed using three questions about prices in an advertisement; and (2) graph comprehension, assessed using three questions based on a graph of Bulgaria’s population growth from 1900 to 2011. The items assessing reading comprehension and semantics were taken from existing instruments fielded by the World Bank with Bulgarian students, while the items assessing mathematics and information comprehension were adapted from the Adult Literacy and Lifeskills Survey (Murray, Clermont, & Binkley, 2005). These domains are not meant to be exhaustive, but to serve as useful heuristics. Moreover, executive functioning skills and domain-specific skills are related: A number of recent studies provide evidence that 36 The ENHAB is a recent survey in Peru which gathers data on cognitive and socio-emotional test scores, individual’s characteristics, educational trajectory, and wages. 37 An issue with translation of the items comprising the semantics scale rendered the data from this set of items unusable. The semantics scale was thus not considered for analysis, leaving the total number of assessed skills at five. 54 executive functioning skills such as working memory actually contribute to the development of literacy and numeracy skills (Blair & Razza, 2010; Swanson, Jerman, & Zheng. 2008). From a policy perspective, this suggests that educators should focus on the promotion of both executive functioning and domain-specific skills, particularly in the pre-school and elementary school years when such functions are most malleable to intervention (Welsh et al., 2010). Analysis Strategy All missing values were recoded as incorrect answers, resulting in a set of 33 dichotomous or binary items. 38 In choosing how to score the items, we were motivated by a primary concern of reducing the measurement error in each score. That is, when we administer a survey measure or test, we want to ensure that the variability in scores is due to what we are trying to measure—in this study, executive functioning or domain- specific skills—as opposed to error or bias. Traditional or unrefined methods of scoring—such as summing the survey items—do not account for this measurement error, leading to bias in future regression analyses (for more information, see Box C1, “What is Factor Analysis and Why do We Use It?” in Appendix C). Refined scoring methods that account for measurement error include the production of factor scores using factor analysis or item response theory (IRT) methods. Box B1: What is Item Response Theory and When Can We Use It? Item Response Theory (IRT) is an approach, or family of statistical models, used to analyze assessment item data, such as cognitive skills assessment data. Several IRT models have been developed to estimate ability or person parameters that are scored either dichotomously (i.e. only two response categories) or polytomously (i.e. more than two response categories; Hambleton, Swaminathan, & Rogers, 1991). Traditionally, IRT has been used for educational applications for Computerized Adaptive Testing (CAT), test score equating, item analysis, and test banking. However, due to the advantages of IRT, other disciplines have recently developed an interest in using IRT for scoring, validation, and other psychometric analyses (Reise & Henson, 2003). There are two over-arching families of item response models which differ greatly in theoretical and mathematical background and analysis. The first of the two families, the logistic models, relate examinee ability (θ) and item parameters using logistic functions. The logistic family of IRT models allow for the estimation of up to three item parameters, or characteristics. The one-parameter (1PL) model is the most basic and involves, as the name states, only one item parameter: the b-parameter is included in every IRT model and is considered the difficulty parameter (Yen & Fitzpatrick, 2006). The b-parameter is at the point on the θ scale where the probability of a correct response is equal to 0.50 and typically varies from -2.00 to 2.00 (Hambleton et al., 1991; Yen & Fitzpatick, 2006), increasing as items become more difficult. The two- parameter model (2PL) includes a second item parameter, the discrimination parameter, a. a is the slope of the item characteristic curve (ICC) at the point of inflection and the higher the value of a, the more sharp the discrimination (Yen & Fitzpatrick, 2006). Finally, the three-parameter model (3PL) includes the c-parameter, called the guessing or pseudo-chance parameter. This parameter was introduced to account for the possibility 38 Ideally, we would be able to identify four, not two, sets of responses: answered correctly; answered incorrectly; not answered and didn’t know; and not answered due to time constraints or motivation but known. While such codes were initially included in the survey instrument, issues with data processing rendered such codes unusable. We were thus forced to collapse the codes into a dichotomous response: correct or incorrect. The implications of this choice are discussed further in the Implications and Future Directions section. 55 that even students with low ability have some chance of answering even difficult questions correctly. This parameter is not always necessary, and if set to zero, equates the 3PL with the 2PL (Yen & Fitzpatrick, 2006). One of the big advantages of using IRT is that the ability or person parameters (θ) are not item or test dependent, and item and test characteristics are not dependent on the ability or person parameters. This is called the property of invariance (Hambleton et al., 1991; Lord, 1980). It means that the test and item parameters remain the same regardless of the sample of respondents, and the ability or person parameters do not vary depending on the test items administered or the time of test, provided the items are relevant to and representative of the same domain of interest. Although there are clear benefits to the invariance property, there are two integral assumptions of IRT. First, there is an assumption regarding the dimensionality of the underlying ability or trait. While there are multi- dimensional IRT models (MIRT), the traditional IRT model requires that a single trait or ability accounts for an individual’s θ score. When this assumption of the data holds, the examinees can be placed along a single, meaningful scale (Hambleton et al., 1991). Second, there is the assumption of local item independence. When the items on an assessment are locally independent, a response to any item is independent of a response to any other item on the same assessment for a given individual. This assumption allows us to determine the probability of an individual response pattern occurring given the individual’s ability or trait level (Hambleton et al., 1991; Lord, 1980). If either of these assumptions is not met, item and person parameters will not be properly estimated and thus, indefensible. In addition to these assumptions, an assessment of model-data fit is also important in IRT. A poorly specified model creates problems with estimating both item parameters and θ scores. Consider the following: An analyst mistakenly specifies a model which only specifies two parameters when in fact the data fit a model consisting of three item parameters. Because the pseudo-guessing parameter has not been specified, the θ values may be over-estimated as the individual’s ability to correctly guess the answer has not been taken into consideration. Guessing is not considered to be included in ability and, as such, it should not be allowed to unduly influence scores. While IRT provides distinct advantages to classical methods of analyzing assessment data, these advantages come with several very restrictive assumptions which, if violated, calls into question the validity of the results. In order to assess whether it was appropriate to employ an IRT model with this data, we decided to first empirically determine the dimensionality of the items by conducting an exploratory factor analysis (EFA) with an oblimax rotation on a randomly selected half of participants stratified by country (N = 3,965). 39 Should a one-factor model provide a good fit to the data, we would be able to proceed with IRT analyses. Should a multi-factor model provide a good fit to the data, the dimensionality assumption required by IRT methodologies would be violated. In that case, we proceed by examining the results of the EFA and confirming the factor structure using the second half of the sample (N = 3,964). All analyses were conducted in MPlus (Muthén & Muthén, 1998–2012; Version 6.12) and adjusted for any clustering of the data due to sampling design. 40 Responses were treated as ordered categorical data to account for the skewed nature of the data. 39An oblimax rotation was chosen to account for the hypothesized correlation between factors. 40In Tajikistan—but not in Uzbekistan or Kyrgyzstan—up to two individuals per household were administered the non- cognitive skills module. To account for any non-independence of the data that may occur due to individuals being nested in households, we used the Type=Complex and Cluster=psuid commands in MPlus. 56 Once we determined a factor structure that provided a good fit to the data, we created individual scores on each of these factors using refined factor scoring techniques. As detailed above, factor scoring is preferable in this case to traditional sum scoring methods given that factor scores account for: (1) the weight of individual item loadings; and (2) shared variance between the items and the factors and measurement error (DiStephano, Zhu, & Midrila, 2009). Factor scores were created using maximum a posteriori (MAP) estimation in MPLUS, which accounts for the non-normal distribution of item response (Muthén & Muthén, 1998–2012). Results The initial EFA indicated that a one-factor model did not provide a good fit to the data (χ2 (324) = 8981.68, CFI: .888, RMSEA: .082, .081 < 95% CI < .084). 41 Thus we decided that it was not feasible to proceed with an IRT analysis due to the plausibility of violating the dimensionality assumption. In examining the factor loadings, we noted that the 12 items making up the original construct of working memory loaded cleanly onto one factor. This factor was left intact and removed from the exploratory analyses. We then chose a 2-factor solution to model associations between the remaining 15 items. This model provided a good fit to the data (χ2 (76) = 1261.15, CFI=.951, RMSEA=.063, .060 < 95% CI < .066) while modeling the observed indicators parsimoniously. A confirmatory factor analysis then confirmed the fit of a 3-factor model for all 27 items in which factors were allowed to correlate (χ2 (321) = 3128.37, CFI=.981, RMSEA=.033, .032 < 95% CI < .034). 42 The three identified factors described in Table 1, below, were: (1) Working Memory (12 items); (2) Reading Comprehension (5 items); and (3) Informational Numeracy (10 items). In addition, preliminary measurement equivalence analyses indicate that this same factor structure provides a good fit to the data in Uzbekistan, Kyrgyzstan, and Tajikistan (χ2 (97c3) = 10531.15, CFI=.953, RMSEA=.061, .060 < 95% CI < .062). 43 Finally, given the high correlation between the literacy and informational numeracy items, initial analyses were also conducted to determine whether a higher-order “cognitive” factor may account for the covariation between factors (Cattell, 1978). 44 This model was uninterpretable due to factor loadings above 1. 41 In assessing model goodness of fit, the following criteria are used: A RMSEA < .08 provides an acceptable fit to the data, while an RMSEA < .05 provides a good fit to the data; a CFI > .9 provides an acceptable fit to the data while a CFI > .95 provides a good fit to the data (Kline, 2011). 42 Factor correlations in the CFA were: Working Memory-Literacy (r=.428, p<.001), Working Memory-Informational Numeracy (r=.480, p<.001), and Literacy-Informational Numeracy (r=.69, p<.001). 43 Tests of measurement invariance seek to establish whether we are measuring the same construct in the same way across different groups. As of this writing, our preliminary analyses have established configural invariance: that the same factor structure (e.g., the same number of factors and the same pattern of loadings) exists in the samples from all three countries. Future analyses will examine other levels of invariance, establishment of which increases our certainty that observed differences between countries is attributable only to true differences in the variability of the scores. 44 For over a century, researchers have been interested in defining and measuring an overall measure of cognitive ability, or “g” factor (Jensen, 1998; Heckman, Stixrud, & Urzua, 2006). It is beyond the scope of this paper to comment extensively on such research; however, as developmental psychologists with an interest in applying research to policy, we take the position that it is useful to identify and understand the components of cognitive ability to better design programs to support the development of such skills. 57 Table B1. Unstandardized Results from Final CFA of Cognitive Skills Module Loading SE Working Memory 1. Working Memory Item 1 0.974 0.009 2. Working Memory Item 2 0.985 0.006 3. Working Memory Item 3 0.987 0.005 4. Working Memory Item 4 0.962 0.004 5. Working Memory Item 5 0.926 0.006 6. Working Memory Item 6 0.904 0.006 7. Working Memory Item 7 0.862 0.006 8. Working Memory Item 8 0.866 0.006 9. Working Memory Item 9 0.816 0.008 10. Working Memory Item 10 0.795 0.011 11. Working Memory Item 11 0.861 0.012 12. Working Memory Item 12 0.900 0.013 Reading Comprehension 13. Reading Comprehension Item 13 0.800 0.012 14. Reading Comprehension Item 14 0.748 0.011 15. Reading Comprehension Item 15 0.843 0.009 16. Reading Comprehension Item 16 0.734 0.009 17. Reading Comprehension Item 17 0.788 0.010 Informational Numeracy 18. Information Comprehension Item 18 0.522 0.014 19. Information Comprehension Item 19 0.553 0.013 20. Information Comprehension Item 20 0.588 0.013 21. Information Comprehension Item 21 0.812 0.009 22. Arithmetic Item 22 0.574 0.013 23. Arithmetic Item 23 0.741 0.010 24. Arithmetic Item 24 0.591 0.013 25. Graph Comprehension Item 25 0.726 0.012 26. Graph Comprehension Item 26 0.832 0.009 27. Graph Comprehension Item 27 0.667 0.011 Interpretation and Future Directions Our analyses indicated that the data from the cognitive skills module is best represented by three related factors that correspond to some—but not all—of the five cognitive skills described above. For example, our analyses indicated items 1–12 all indexed the hypothesized underlying executive functioning skill of Working Memory, while items 13–17 corresponded to the hypothesized underlying domain-specific skill of Reading Comprehension. Substantively, this indicates that individuals that have higher Working Memory factor scores are better able to temporarily store and manipulate information that is necessary for domain-specific cognitive tasks such as reading comprehension (Baddeley, 1992). Individuals with higher Reading Comprehension 58 scores have a better ability to read and process text and understand its meaning than individuals with lower Reading Comprehension scores (National Reading Panel, 2000). The other factor represented in the data is a combination of items meant to index facets of both Literacy (items 18–21) and Numeracy (items 22–27). This pattern of relationships can be understood in that the Information Comprehension items all involved number recognition (a component of numeracy), while the Numeracy items all tapped the ability to locate and use information contained in various formats such as advertisements and graphs (a component of information comprehension). Individuals who score highly on Informational Numeracy have the ability to recognize and manipulate numbers contained in and represented by various formats. There are three things to consider when interpreting the above analysis. First, the factor scores created through the factor analysis procedures described above are not invariant across different tests assessing cognitive ability. While such scores could have resulted from using IRT methodologies, we have evidence that using IRT with this cognitive assessment is not defensible given the likely violation of the assumption of dimensionality and as a result, item dependence. As such, we proceeded with creating refined factor scores that—although they do not inherently have the property of invariance—reduce the amount of measurement error contained in the scores. It should be noted, however, that invariance is a property that can be assessed through the use of factor analytic methods. Second, many of the items included in the cognitive skills assessment are not “clean” items. That is, they assess more than one skill at the same time: Items meant to tap the construct of Arithmetic, for example, also involve elements of reading comprehension and information comprehension. The factors—particularly Reading Comprehension and Information Numeracy—are thus highly correlated, which may be problematic for establishing predictive validity. To address this, we recommend that future analyses with this data consider a bi-factor analysis in which orthogonal or non-correlated grouping factors are created by allowing a “general” trait to correlate with the items (Reise, Moore, & Haviland, 2010). Finally, as noted in footnote 2, we were limited in our ability to discriminate between correct, incorrect, and missing answers due to issues in data processing. Given that missing answers were all recoded to be incorrect, it is likely that the scores underestimate the cognitive ability level present in the sample population. To address this, we recommend that future data collection activities carefully assess the type and extent of missing data to allow for better sensitivity tests of results to such specifications. References Baddeley, Alan. 1992. “Working Memory.” Science 255 (5044): 556–9. Blair, Clancy, and Rachel Peters Razza. 2007. “Relating Effortful Control, Executive Function, and False Belief Understanding to Emerging Math and Literacy Ability in Kindergarten.” Child Development 78: 647–63. Boekaerts, Monique. 1997. “Self-regulated Learning: A New Concept Embraced by Researchers, Policy Makers, Educators, Teachers, and Students.” Learning and Instruction 7 (2): 161–86. DiStefano, Christine, Min Zhu, and Diana Mindrila. 2009. “Understanding and Using Factor Scores: Considerations for the Applied Researcher.” Practical Assessment, Research & Evaluation 14 (20): 1–11. 59 Fuchs, Lynn S., Douglas Fuchs, Donald L. Compton, Sarah R. Powell, Pamela M. Seethaler, Andrea M. Capizzi, Christopher Schatschneider, and Jack M. Fletcher. 2006. “The Cognitive Correlates of Third-grade Skill in Arithmetic, Algorithmic Computation, and Arithmetic Word Problems.” Journal of Educational Psychology 98 (1): 29–43. Hambleton, Ronald K., H. Swaminathan, and H. Jane Rogers. 1991. Fundamentals of Item Response Theory. Newbury Park, CA: Sage Publications. Heckman, James J., Jora Stixrud, and Sergio Urzua. 2006. “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior.” Journal of Labor Economics 24 (3): 411–82. Jensen, Arthur R. 1998. The g Factor: The Science of Mental Ability. Westport, CT: Praeger. Kline, Rex B. 2011. Principles and Practice of Structural Equation Modeling, 3rd edition. New York: Guilford Press. Lord, Frederic M. 1980. Applications of Item Response Theory to Practical Testing Problems. Hillsdale, NJ: Erlbaum. Murray, T. Scott., Yvan Clermont, and Marilyn Binkley. 2005. Measuring Adult Literacy and Life Skills: New Frameworks for Assessment. Ottawa: Statistics Canada. Muthén, Linda K., and Bengt O. Muthén. 1998–2012. Mplus User’s Guide, Seventh Edition. Los Angeles, CA: Muthén & Muthén. National Reading Panel, National Institute of Child Health & Human Development (US). 2000. Report of the National Reading Panel: Teaching Children to Read: An Evidence-based Assessment of the Scientific Research Literature on Reading and its Implications for Reading Instruction: Reports of the Subgroups. Washington, DC: National Institute of Child Health and Human Development, National Institutes of Health. Pinker, Steven. 1990. “A Theory of Graph Comprehension.” In Artificial Intelligence and the Future of Testing, ed. Roy Friedle. Hillsdale, N.J.: Erlbaum. Rasch, George. 1980. Probabilistic Models for Some Intelligence and Attainment Tests (revised and expanded ed.). Chicago: The University of Chicago Press. (Originial work published 1960.) Reise, Steven P., and James M. Henson. 2003. “A Discussion of Modern versus Traditional Psychometrics as Applied to Personality Assessment Scales.” Journal of Personality Assessment 81 (2): 93–103. Reise, Steven P., Tyler M. Moore, and Mark G. Haviland. 2010. “Bifactor Models and Rotations: Exploring the Extent to Which Multidimensional Data Yield Univocal Scale Scores.” Journal of Personality Assessment 92 (6): 544–59. Swanson, H. Lee, Olga Jerman, and Xinhua Zheng. 2008. “Growth in Working Memory and Mathematical Problem Solving in Children at Risk and not at Risk for Serious Math Difficulties.” Journal of Educational Psychology 100: 343–79. 60 Welsh, Janet A., Robert L. Nix, Clancy Blair, Karen L. Bierman, and Keith E. Nelson. 2010. “The Development of Cognitive Skills and Gains in Academic School Readiness for Children from Low- income Families.” Journal of Educational Psychology 102 (1): 43–53. Yen, Wendy M., and Anne R. Fitzpatrick. 2006. “Item Response Theory.” In Educational Measurement (4th ed.), ed. Robert L. Brennan, 111–54. Westport, CT: American Council on Education. 61 Appendix C: Constructing Non-Cognitive Skills Scores Methods for Scale Development and Scoring Prepared by Carly Tubbs, Ph.D. Candidate, New York University Background and Measures Data for this study come from a 33-item survey module designed for use by the World Bank to assess 11 different “non-cognitive” skills (see Table 1, below; Duckworth & Guerra, 2012). These non-cognitive skills can be conceptualized as falling into two domains: Personality traits, defined as enduring patterns of thinking, feeling, and behaving which are relatively stable across time and situations (Borghans, Duckworth, Heckman, & ter Weel, 2008; Paunonen, 2003). The “Big Five” factors of personality—openness, conscientiousness, extraversion, agreeableness, and neuroticism (or emotional stability)—are the most widely accepted taxonomy of broad personality traits (Goldberg, 1990), having been validated for use across developmental stages (John & Srivastava, 1999) and cultures (Soto, John, Gosling, & Potter, 2008). The survey assessed each of these five factors with three items in the short Big Five Inventory (BFI-S) originally developed by John and Srivastava (1999) and later validated in large-scale panel surveys (Lang et al., 2011). Given its association with important labor market outcomes, assessed grit—a component of conscientiousness—was also assessed, with three items from the Grit Scale (Duckworth et al., 2007). Socio-emotional skills, defined as the learned knowledge, attitudes and skills necessary to understand and manage emotions, set and achieve positive goals, establish and maintain positive relationships, and make responsible decisions (CASEL, 2014). Although different cultures may differentially name, conceptualize, and prioritize such skills, socio-emotional skills and learning are of critical importance across all regions of the world (Torrente, Alimchandani, & Aber, in press). There does not currently exist an organization of socio- emotional skills similar to that developed for personality traits; as such this survey measures socio-emotional skills that are both valued by employers in countries in Europe and Central Asia (World Bank, 2009, 2013) and amenable to intervention efforts (Yeager & Dweck, 2012). These skills include: hostile bias (2 items; Dodge, 2003), decision making (4 items; Mann, Burnett, Radford, & Ford, 1997), achievement-striving, and self-control (3 items and 2 items, respectively; Goldberg et al., 2006). In addition, the fixed vs. growth mindset, or the belief that intelligence is fixed versus malleable, was measured (4 items; Yeager & Dweck, 2012). These domains are not meant to be exhaustive, but to serve as useful heuristics. Moreover, personality traits and socio-emotional skills are related: individuals with certain personality traits may tend to employ certain socio-emotional skills (McAdams, 1995). For program and policy purposes, however, there is a key distinction between personality traits and socio-emotional skills: while personality traits are predictive of labor market outcomes, they are less amenable to direct change via intervention. Socio-emotional skills, however, have been shown to be malleable to various intervention efforts across cultures (e.g., Jones, Brown, Aber, 2011; Torrente et al., 2014). In turn, building socio-emotional skills can result in changes to enduring patterns of thinking and behaving (Dweck, 2008). 62 Table C1. Original 33 Items Included in the Non-Cognitive Skills Module 45 Extraversion Are you talkative? Do you like to keep your opinions to yourself? Do you prefer to keep quiet when you have an opinion? (R) Are you outgoing and sociable, do you make friends easily? Conscientiousness When you perform a task, are you very careful? Do you prefer relaxation more than hard work? (R) Do you work very well and quickly? Openness Do you come up with ideas others haven't thought of before? Are you interested in learning new things? Do you enjoy beautiful things, like nature, art, and music? Emotional Stability Are you relaxed during stressful situations? Do you tend to worry? (R) Do you get nervous easily? (R) Agreeableness Do you forgive other people easily? Are you very polite to other people? Are you generous to other people with your time or money? Personality Traits Grit Do you finish whatever you begin? Do you work very hard? For example, do you keep working when others stop to take a break? Do you enjoy working on things that take a very long time to complete? Hostile Bias Do people take advantage of you? Are people mean/not nice to you? Decision Making Do you think about how the things you do will affect your future? Do you think carefully before you make an important decision? Do you ask for help when you don't understand something? Socioemotional Skills Do you think about how the things you do will affect others? Achievement Striving Do you do more than is expected of you? Do you strive to do everything in the best way? Do you try to outdo others, to be best? 45All items except the Fixed Versus Growth Mindset items were scaled using a 4-point Likert scale (1 = Almost always – 4 = Almost never). The Fixed Versus Growth Mindset items employed a 6-point Likert scale (1 = Totally agree – 6 = Strongly disagree). Items that are marked with an (R) were reverse coded so that a low value indicates the same valence of response on every item. 63 Self Control Do you spend more than you can afford? Do you do crazy things and act wildly? Fixed Versus Growth Mindset The type of person you are is fundamental, and you cannot change much. You can behave in various ways, but your character cannot really be changed. As much as I hate to admit it, you cannot teach an old dog new tricks. You cannot change their most basic properties. You have a certain personality and not much can be done to change that. Note: Items and scales in blue are personality trait measures, items and scales in orange are socio-emotional skill measures. Analysis Strategy Our initial analyses revealed three main issues with the data. First, correlations between items in the same groupings (e.g., openness, grit) were low—generally ranging from .2 - .4—suggesting that each item is measuring a different facet of the grouping. Second, sum-scoring items according to the 11 hypothesized constructs and computing reliability coefficients indicated the scores were composed of a significant degree of measurement error. Third, the distribution of item responses across the Likert scales deviated substantially from normality, invalidating the assumptions inherent in traditional statistical measurement techniques. To address these issues, factor analyses were conducted in a multi-step process. Box C1: What is Factor Analysis and Why Do We Use it? Factor analysis is a statistical technique that can be used to examine the relationship between observed items or indicators (see Table 1, above) and unobserved latent constructs or factors that are hypothesized to underlie the associations between indicators (in this study, openness, conscientiousness, etc.). There are three primary goals of or reasons to use factor analysis: (1) data reduction; (2) scale structure; and (3) to reduce measurement error. First, survey instruments provide a lot of data—some surveys to assess adult personality factors include over 500 items. Not only is it not practical to analyze that much data, but testing effects on multiple discrete indicators increases the likelihood of having a “false positive,” or Type I error. Factor analysis assists with data reduction by establishing a lesser number of factors that account for the variation between indicators. Second, surveys are frequently designed to capture multiple constructs (in our study, various personality traits and socio-emotional skills) using items that may relate more strongly to some constructs than others. For example, in our study, the item “Do you think about how the things you do will affect your future?” may be a better indicator of Decision Making than, “Do you ask for help when you don't understand something?” Factor analysis allows us to understand the internal scale structure by quantifying the number of factors in the data and the extent to which items are related to each factor. Finally, when we administer a survey measure or test, we want to ensure that the variability in scores is due to what we are trying to measure—in this study, personality traits or socio-emotional skills—as opposed to error or bias. Traditional or unrefined methods of scoring—such as summing the survey items—do not account for this measurement error, leading to biases in regression analyses. Factor analysis allows us to adjust for measurement error by fitting an underlying model accounting for both variation among observed items and random error variance. 64 There are two primary types of factor analysis: exploratory factor analysis (EFA) and confirmatory factor analysis (CFA). While both EFA and CFA attempt to model the relationship between observed items using a smaller set of latent constructs, they differ in the a priori restrictions that are placed on the model. EFA is a data-driven technique that is primarily used when the factor structure (e.g, the appropriate number of underlying factors and the relationships of the items to the factors) is unknown, whether because the survey has never been administered before or is being administered in new contexts. In CFA, a researcher uses a strong theoretical foundation to specify at the outset the number of hypothesized factors and the patterns of how the items relate to the factors. This solution is then evaluated with respect to how well it fits the observed data. EFA is used most frequently early in the process of scale development, while CFA is used once the researcher has established the factor structure based on prior empirical and theoretical grounds. Given that the non-cognitive skills module has never before been administered in the countries of interest in this study, we decided to proceed by first conducting exploratory factor analyses (EFAs) with an oblimax rotation on a randomly selected half of participants stratified by country (N = 3,885). 46 In doing so, we are not making a priori assumptions about the factor structure of the module in these new contexts. Then, to support the EFA results, the factor structure was confirmed (in a confirmatory factor analysis, or CFA) using the second half of the sample (N = 3,887). All analyses were conducted in MPlus (Muthén and Muthén, 1998–2012; Version 6.12) and adjusted for any clustering of the data due to sampling design.47 Responses were treated as ordered categorical data to account for the skewed nature of the data, and full information maximum likelihood (FIML) estimation was employed to handle missing data. 48 Once we determined a factor structure that provided a good fit to the data, we created individual scores on each of these factors using refined factor scoring techniques. As detailed above, factor scoring is preferable in this case to traditional sum scoring methods given that factor scores account for: (1) the weight of individual item loadings; and (2) shared variance between the items and the factors and measurement error (DiStephano, Zhu, and Midrila, 2009). Factor scores were created based on the exploratory factor analysis solution using maximum a posteriori (MAP) estimation in MPLUS, which accounts for the non-normal distribution of item response (Muthén and Muthén, 1998–2012). Results The initial EFA revealed two groupings of items: those that loaded well onto one factor, and those that did not. The 4 items making up the original construct of “Fixed Versus Growth Mindset” loaded cleanly onto one factor. This factor was left intact and removed from the exploratory analyses; it was subsequently confirmed to provide a good fit to the data (χ2 (2) = 27.52, CFI: .996, RMSEA: .057, .039 < 95% CI 46 An oblimax rotation was chosen to account for the hypothesized correlation between factors. 47 In Tajikistan—but not in Uzbekistan or Kyrgyz Republic—up to two individuals per household were administered the non-cognitive skills module. To account for any non-independence of the data that may occur due to individuals being nested in households, we used the Type=Complex and Cluster=psuid commands in MPlus. 48 FIML utilizes all available data points, even for cases with missing item responses, by assessing during parameter estimation missing data patterns as well as by using information from all available data points. While FIML does not impute missing data, its use of information from all observed data is conceptually similar to missing data imputation, where a missing value is computed conditioned on several other included variables (Muthén, Kaplan & Hollis, 1987). In this sample, 120 cases did not have data on any of the items and were removed from the analysis. 65 < .077). 49 Also removed from analyses at this juncture were items that loaded below .2 on any construct and items that were reverse coded due to factor-item correlations in unexpected directions. We then chose a 4- factor solution to model associations between the remaining 18 items; in this solution, items were allowed to cross-load on multiple factors and factors were allowed to correlate. 50 This model provided an excellent fit to the data (χ2 (87) = 530.89, CFI=.985, RMSEA=.036, .033 < 95% CI < .039) while modeling the observed indicators parsimoniously. The four identified factors described in Table 2, below, were: (1) Openness to New Ideas and People (5 items; e.g., “Are you outgoing and sociable?”; “Are you interested in learning new things?”); (2) Workplace Attitude and Behavior (5 items; e.g., “Do you enjoy working on things that take a very long time to complete?”; “Are people mean/not nice to you?”); (3) Decision Making (5 items; e.g., “Do you think about how the things you do will affect others?”; “Do you think carefully before making an important decision?”); and (4) Achievement Striving (3 items; “Do you do more than is expected of you?”). As detailed above, confirmatory factor analysis confirmed the fit of this model (χ2 (129) = 2336.52, CFI=.922, RMSEA=.066, .064 < 95% CI < .069). In addition, preliminary measurement equivalence analyses indicate that this same factor structure provides a good fit to the data in Uzbekistan, Kyrgyz Republic, and Tajikistan (χ2 (459) = 69484.24, CFI=.932, RMSEA=.068, .066 < 95% CI < .070). 51 Table C2. Unstandardized Results from Final CFA of Non-Cognitive Skills Module Loading SE Extraversion 1. Are you talkative? 0.502 0.015 2. Are you outgoing and sociable, do you make friends easily? 0.672 0.012 3. Are you interested in learning new things? 0.635 0.013 4. Do you enjoy beautiful things, like nature, art, and music? 0.528 0.015 5. Are you very polite to other people? 0.648 0.013 Workplace Attitudes and Behaviors 6. Do you come up with ideas others haven't thought of before? 0.575 0.019 Do you work very hard? For example, do you keep working when others stop to 0.693 0.018 7. take a break? 8. Do you enjoy working on things that take a very long time to complete? 0.506 0.019 9. Do people take advantage of you? 0.360 0.020 10. Are people mean/not nice to you? 0.207 0.024 Decision Making 11. Do you finish whatever you begin? 0.622 0.013 49 In assessing model goodness of fit, the following criteria are used: A RMSEA < .08 provides an acceptable fit to the data, while an RMSEA < .05 provides a good fit to the data; a CFI > .9 provides an acceptable fit to the data while a CFI > .95 provides a good fit to the data (Kline, 2011). 50 Factor correlations in the final EFA ranged from .1 to .65. The highest correlations were: Openness-Decision Making (.535), Openness-Achievement Striving (.556), and Decision Making-Achievement Striving (.65). 51 Tests of measurement invariance seek to establish whether we are measuring the same construct in the same way across different groups. As of this writing, our preliminary analyses have established configural invariance: that the same factor structure (e.g., the same number of factors and the same pattern of loadings) exists in the samples from all three countries. Future analyses will examine other levels of invariance, establishment of which increases our certainty that observed differences between countries is attributable only to true differences in the variability of the scores. 66 12. Do you think about how the things you do will affect your future? 0.673 0.011 13. Do you think carefully before you make an important decision? 0.683 0.011 14. Do you ask for help when you don't understand something? 0.592 0.013 15. Do you think about how the things you do will affect others? 0.669 0.011 Achievement Striving 16. Do you do more than is expected of you? 0.587 0.014 17. Do you strive to do everything in the best way? 0.723 0.013 18. Do you try to outdo others, to be best? 0.463 0.016 Fixed Versus Growth Mindset 19. The type of person you are is fundamental, and you cannot change much. 0.678 0.009 20. You can behave in various ways, but your character cannot really be changed. 0.711 0.009 As much as I hate to admit it, you cannot teach an old dog new tricks. You 0.697 0.008 21. cannot change their most basic properties. 22. You have a certain personality and not much can be done to change that. 0.704 0.008 Interpretation and Future Directions Our analyses indicated that the data from the non-cognitive skills module is best represented by five factors that correspond to some—but not all—of the 11 personality traits and socio-emotional skills described in Table 1. For example, our analyses indicated that items 19-22 and 16-18 index the hypothesized underlying socio-emotional skills Fixed Versus Growth Mindset and Achievement Striving, respectively. Substantively, this indicates that individuals that have higher Achievement Striving factor scores tend to strive to go “above and beyond” and to do more than is expected of them, while individuals who have higher Fixed Versus Growth Scores tend to believe new skills can be learned. The other three factors represented in the data are combinations of items meant to index both personality traits and socio-emotional skills; this pattern of relationships can be understood in that certain personality traits tend to be related to certain learned attitudes and skills. For example, our factor of Decision Making consists of items originally thought to index both decision-making skills and the trait of grit. In this case, individuals who think carefully and thoroughly about the repercussions of their decisions and behaviors (see items 12–15) tend to follow through with their actions (see item 11)—perhaps anticipating the repercussions of not following through. Our factor of Workplace Attitudes and Behaviors consists of items meant to index both Grit and Hostile Bias. Individuals who work very hard when others take a break (see items 6–8) may tend to feel that others take advantage of them or are mean (see items 9–10). Thus individuals who score higher on this construct may be workers who work hard and are innovative but perceive interactions with others as hostile; individuals who score lower on this construct tend to work less hard and on discrete projects, without perceiving workplace interactions as negative. Finally, our construct of Openness to New Ideas and People reflects items thought to index the personality traits of extraversion, agreeableness, and openness. Individuals who score high on this construct are social and open to new ideas, people, and things (see items 1–5). There are two plausible reasons why the data did not reflect the expected 11 traits and skills. First, only 2–4 items were used to originally index each trait/skill; this may not have been enough to validly and reliably fully “capture” the constructs of interest. Instead, these items appear to reflect weak to moderately related aspects 67 of a trait/skill that co-vary with aspects of other traits/skills. This is unsurprising given demonstrated correlations between: (a) Big Five personality traits (Digman, 1997); and (b) personality traits and socio- emotional skills (McAdams, 1995). To address this issue, future surveys should consider including a broader range of items to represent each trait/skill. A second explanation that we cautiously proffer is that the items do not relate to each other in the same way in Tajikistan, Uzbekistan, and Kyrgyz Republic as in the samples from which the items were developed. For example, in the Grit scale in this sample, “finishing what was begun” is not related to “enjoying working on things that take a long time to complete.” In ECA contexts, grit might not be well indexed by such behaviors. To investigate this, future research should: (1) conduct qualitative research to better understand how these traits and skills are understood in ECA contexts; and (2) test for measurement invariance between the non-cognitive items administered in this study and in other studies. References Borghans, Lex, Angela Lee Duckworth, James J. Heckman, and Bas ter Weel. 2008. “The Economics and Psychology of Personality Traits.” Journal of Human Resources 43 (4): 972–1059. CASEL. 2014. “What is Social and Emotional Learning?” Retrieved from: http://www.casel.org/social-and- emotional-learning. Dodge, Kenneth A. 2003. “Do Social Information Processing Patterns Mediate Aggressive Behavior?” In Benjamin B. Lahey, Terrie E. Moffitt, and Avshalom Caspi, eds., Causes of Conduct Disorder and Juvenile Delinquency, 254–74. New York: Guilford Press. DiStefano, Christine, Min Zhu, and Diana Mindrila. 2009. “Understanding and Using Factor Scores: Considerations for the Applied Researcher.” Practical Assessment, Research & Evaluation 14 (20): 1–11. Duckworth, Angela L., Christopher Peterson, Michael D. Matthews, and Dennis R. Kelly. 2007. “Grit: Perseverance and Passion for Long-Term Goals.” Journal of Personality and Social Psychology 92: 1087– 101. Dweck, Carol S. 2008. “Can Personality Be Changed? The Role of Beliefs in Personality and Change.” Current Directions in Psychological Science 17 (6): 391–4. Goldberg, Lewis R. 1990. “An Alternative ‘Description of Personality’: The Big-Five Factor Structure.” Journal of Personality and Social Psychology 59 (6): 1216–29. Goldberg, Lewis R., John A. Johnson, Herbert W. Eber, Robert Hogan, Michael C. Ashton, C. Robert Cloninger, and Harrison G. Gough. 2006. “The international personality item pool and the future of public-domain personality measures.” Journal of Research in Personality 40 (1): 84–96. John, Oliver P., and Sanjay Srivastava. 1999. “The Big Five Trait Taxonomy: History, Measurement, and Theoretical Perspectives.” In Lawrence A. Pervin and Oliver P. John, eds., Handbook of Personality: Theory and Research, 2nd ed., 102–38. New York, NY: Guilford Press. 68 Jones, Stephanie M., Joshua L. Brown, and J. Lawrence Aber. 2011. “Two-Year Impacts of a Universal School-Based Social-Emotional and Literacy Intervention: An Experiment in Translational Developmental Research.” Child Development 82 (2): 533–54. Kline, Rex B. 2011. Principles and Practice of Structural Equation Modeling, 3rd edition. New York: Guilford Press. Lang, Frieder R., Dennis John, Oliver Lüdtke, Jürgen Schupp, and Gert G. Wagner. 2011. “Short assessment of the Big Five: Robust across survey methods except telephone interviewing.” Behavior Research Methods 43 (2): 548–67. Mann, Leon, Paul Burnett, Mark Radford, and Steve Ford. 1997. “The Melbourne decision making questionnaire: An instrument for measuring patterns for coping with decisional conflict.” Journal of Behavioral Decision Making 10 (1): 1–19. McAdams, Dan P. 1995. “What Do We Know When We Know a Person?” Journal of Personality 63 (3): 365– 96. Muthén, Bengt O., David Kaplan, and Michael Hollis. 1987. “On structural equation modeling with data that are not missing completely at random.” Psychometrika 42 (3): 431–62. Muthén, Linda K., and Bengt O. Muthén. 1998–2012. Mplus User’s Guide, Seventh Edition. Los Angeles, CA: Muthén & Muthén. Paunonen, Sampo V. 2003. “Big Five factors of personality and replicated predictions of behavior.” Journal of Personality and Social Psychology 84 (2): 411–22. Soto, Christopher J., Oliver P. John, Samuel D. Gosling, and Jeff Potter. 2008. “The Developmental Psychometrics of Big Five Self-Reports: Acquiescence, Factor Structure, Coherence, and Differentiation From Ages 10 to 20.” Journal of Personality and Social Psychology 94 (4): 718–37. Torrente, Catalina, Anjali Alimchandani, and John Lawrence Aber. Forthcoming. “International Perspectives on Social-Emotional Learning.” In Handbook on Social and Emotional Learning: Research and Practice. Torrente, Catalina, Brian M. Johnston, Edward Seidman, and Alana Gross. 2014. “Improving learning environments and children's social-emotional wellbeing in the Democratic Republic of the Congo: Preliminary results from a cluster randomized trial.” Paper presented at Society for Research on Educational Effectiveness (SREE), Washington, DC. Yeager, David Scott, and Carol S. Dweck. 2012. “Mindsets That Promote Resilience: When Students Believe That Personal Characteristics Can Be Developed.” Educational Psychologist 47 (4): 302–14. 69 Appendix D: Summary Tables Employment Rate Table D1. Employment Rate by Age Cohort All Male Female Age Cohort (%) (%) (%) 16-19 20.2 26.1 15.8 20-24 50.6 67.4 39.4 25-29 63.9 88.7 43.5 30-34 65.6 89.5 44.7 35-39 67.4 88.5 52.3 40-44 71.1 89.5 55.1 45-49 65.1 83.4 52.8 50-54 58.2 80.6 40.6 55-59 36.5 64.5 14.5 60-64 9.4 14.0 4.8 Total 48.6 66.1 35.0 Excluding current migrants. Table D2. Employment Rate by Consumption Quintile Consumption All Male Female quintile (%) (%) (%) 1 42.6 62.3 28.9 2 46.7 66.1 31.9 3 49.5 68.3 34.7 4 49.6 67.7 35.9 5 53.2 65.5 42.6 Total 48.6 66.1 35.0 Excluding current migrants. Working-age population (16-64). Table D3. Employment Rate by Rural/Urban Location All Male Female (%) (%) (%) Urban 45.1 66.3 28.7 Rural 52.0 65.9 41.3 Total 48.6 66.1 35.0 Excluding current migrants. Working-age population (16-64) Table A4. Employment Rate by Education Level All Male Female Education level (%) (%) (%) Less than secondary 47.3% 78.6 24.4 Secondary general 56.5% 81.0 34.8 Secondary technical/special 67.1% 83.0 50.3 Tertiary 76.4% 82.4 67.9 Total 62.0% 81.8 42.5 Including current migrants. Population aged 25-64 y.o. 70 Labor Force Participation Rate Table A5. Labor Force Participation Rate by Age Cohort All Male Female Age cohort (%) (%) (%) 16-19 21.7 27.1 17.6 20-24 53.1 70.0 42.0 25-29 65.8 90.1 45.7 30-34 66.9 91.2 45.9 35-39 69.3 89.5 54.8 40-44 71.9 90.2 56.0 45-49 66.2 84.7 53.8 50-54 59.4 81.9 41.6 55-59 37.3 66.0 14.8 60-64 9.9 14.0 5.8 Total 49.5 66.6 36.2 Excluding current migrants. Table A6. Labor Force Participation Rate by Consumption Quintile Consumption All Male Female quintile (%) (%) (%) 1 43.9 64.1 30.0 2 47.9 66.7 33.6 3 49.8 67.9 35.6 4 50.9 68.7 37.6 5 53.5 65.5 43.1 Total 49.5 66.6 36.2 Excluding current migrants. Working-age population (16-64). Table A7. Labor Force Participation Rate by Rural/Urban Location All Male Female (%) (%) (%) Urban 46.1 66.9 29.9 Rural 52.8 66.3 42.4 Total 49.5 66.6 36.2 Excluding current migrants. Working-age population (16-64) Table A8. Labor Force Participation Rate by Education Level All Male Female Education level (%) (%) (%) Less than secondary 48.3 79.9 25.1 Secondary general 57.6 82.0 36.0 Secondary technical/special 68.4 84.0 52.1 Tertiary 77.5 83.2 69.4 Total 63.1 82.8 43.9 Including current migrants. Population aged 25-64 y.o. 71 Employment Status Table A9. Employment Status by Age Cohort: All Out of labor force Employed Unemployed Age cohort Discouraged Inactive (%) (%) (%) (%) 16-19 20.2 1.5 8.9 69.4 20-24 50.6 2.6 10.7 36.2 25-29 63.9 1.8 5.5 28.7 30-34 65.6 1.4 6.1 27.0 35-39 67.4 1.9 2.1 28.6 40-44 71.1 0.8 2.9 25.2 45-49 65.1 1.1 5.0 28.8 50-54 58.2 1.2 3.7 37.0 55-59 36.5 0.9 4.3 58.4 60-64 9.4 0.5 0.4 89.7 Total 48.6 1.4 5.3 44.7 Excluding current migrants. Table A10. Employment Status by Age Cohort: Male Out of labor force Employed Unemployed Age cohort Discouraged Inactive (%) (%) (%) (%) 16-19 26.1 1.0 5.9 66.9 20-24 67.4 2.6 10.0 20.0 25-29 88.7 1.4 6.8 3.0 30-34 89.5 1.6 4.3 4.5 35-39 88.5 1.1 3.1 7.4 40-44 89.5 0.7 4.0 5.7 45-49 83.4 1.3 5.5 9.9 50-54 80.6 1.3 7.3 10.8 55-59 64.5 1.5 9.3 24.7 60-64 14.0 0.0 0.0 86.0 Total 66.1 1.3 5.6 27.0 Excluding current migrants. Table A11. Employment Status by Age Cohort: Female Out of labor force Employed Unemployed Age cohort Discouraged Inactive (%) (%) (%) (%) 16-19 15.8 1.8 11.1 71.3 20-24 39.4 2.6 11.1 46.9 25-29 43.5 2.2 4.5 49.8 30-34 44.7 1.1 7.6 46.5 35-39 52.3 2.6 1.3 43.9 40-44 55.1 0.9 1.9 42.1 45-49 52.8 1.0 4.7 41.6 50-54 40.6 1.1 0.8 57.5 55-59 14.5 0.3 0.4 84.7 60-64 4.8 1.0 0.8 93.4 Total 35.0 1.5 5.1 58.3 Excluding current migrants. 72 Table A12. Employment Status by Consumption Quintile: All Out of labor force Consumption Employed Unemployed quintile (%) (%) Discouraged Inactive (%) (%) 1 44.9 1.9 9.4 43.8 2 49.7 1.7 8.1 40.6 3 53.0 1.2 3.8 42.0 4 53.9 1.5 4.6 39.9 5 56.2 1.4 3.4 39.0 Total 51.8 1.5 5.7 40.9 Excluding current migrants. Working-age population (16-64). Table A13. Employment Status by Consumption Quintile: Male Out of labor force Consumption Employed Unemployed quintile (%) (%) Discouraged Inactive (%) (%) 1 66.5 3.0 11.3 19.3 2 69.6 1.3 9.7 19.4 3 73.7 0.7 2.9 22.7 4 72.9 1.2 4.2 21.7 5 68.4 1.0 3.4 27.2 Total 70.2 1.4 6.0 22.4 Excluding current migrants. Working-age population (16-64). Table A14. Employment Status by Consumption Quintile: Female Out of labor force Consumption Employed Unemployed quintile (%) (%) Discouraged Inactive (%) (%) 1 30.5 1.1 8.2 60.2 2 34.2 2.0 6.8 56.9 3 36.9 1.5 4.5 57.1 4 39.4 1.8 5.0 53.9 5 45.3 1.7 3.5 49.6 Total 37.5 1.6 5.5 55.3 Excluding current migrants. Working-age population (16-64). Table A15. Employment Status by Education Level: All Out of labor force Employed Unemployed Education level (%) (%) Discouraged Inactive (%) (%) Less than secondary 43.1 1.1 2.4 53.5 Secondary general 51.5 1.2 5.3 42.0 Secondary technical/special 63.1 1.5 4.1 31.3 Tertiary 74.8 1.3 1.5 22.5 Total 57.8 1.3 4.1 36.8 Including current migrants. Population aged 25-64 y.o. 73 Table A16. Employment Status by Education Level: Male Out of labor force Employed Unemployed Education level (%) (%) Discouraged Inactive (%) (%) Less than secondary 76.7 1.6 1.2 20.5 Secondary general 77.2 1.2 7.8 13.9 Secondary technical/special 79.4 1.3 4.7 14.7 Tertiary 80.7 1.0 2.3 16.0 Total 78.5 1.2 5.2 15.0 Including current migrants. Population aged 25-64 y.o. Table A17. Employment Status by Education Level: Female Out of labor force Employed Unemployed Education level (%) (%) Discouraged Inactive (%) (%) Less than secondary 21.2 0.7 3.1 75.0 Secondary general 33.7 1.2 3.5 61.5 Secondary technical/special 49.0 1.7 3.6 45.6 Tertiary 67.0 1.7 0.5 30.9 Total 41.1 1.4 3.2 54.3 Including current migrants. Population aged 25-64 y.o. Table A18. Employment Status by Rural/Urban: All Out of labor force Employed Unemployed Discouraged Inactive (%) (%) (%) (%) Urban 48.1 1.5 5.1 45.3 Rural 55.5 1.6 6.3 36.6 Total 51.8 1.5 5.7 40.9 Excluding current migrants. Working-age population (16-64). Table A19. Employment Status by Rural/Urban: Male Out of labor force Employed Unemployed Discouraged Inactive (%) (%) (%) (%) Urban 69.8 1.2 5.1 23.9 Rural 70.7 1.5 6.9 20.9 Total 70.2 1.4 6.0 22.4 Excluding current migrants. Working-age population (16-64). Table A20. Employment Status by Rural/Urban: Female Out of labor force Employed Unemployed Discouraged Inactive (%) (%) (%) (%) Urban 30.9 1.6 5.2 62.3 Rural 44.0 1.6 5.9 48.5 Total 37.5 1.6 5.5 55.3 Excluding current migrants. Working-age population (16-64). 74 Educational Attainment Table A21. Educational Attainment by Age Cohort: All Less than Secondary Secondary Age cohort secondary general technical/special Tertiary (%) (%) (%) (%) 25-29 11.4 37.9 37.7 13.1 30-34 12.3 43.7 31.9 12.1 35-39 9.5 48.6 34.7 7.3 40-44 5.1 48.1 31.2 15.5 45-49 4.0 46.0 36.3 13.8 50-54 4.9 48.7 30.7 15.7 55-59 7.7 44.7 28.6 19.0 60-64 12.8 40.1 26.2 20.9 Total 8.7 44.3 32.9 14.1 Excluding current migrants. Table A22. Educational Attainment by Age Cohort: Male Less than Secondary Secondary Age cohort secondary general technical/special Tertiary (%) (%) (%) (%) 25-29 9.4 34.4 37.3 18.9 30-34 10.4 42.4 32.4 14.8 35-39 11.2 44.2 34.4 10.2 40-44 7.2 41.4 33.1 18.2 45-49 2.0 45.5 38.5 14.0 50-54 3.7 46.2 30.8 19.2 55-59 3.7 38.4 33.9 24.0 60-64 11.3 33.7 29.1 25.9 Total 7.7 40.5 34.0 17.9 Excluding current migrants. Table A23. Educational Attainment by Age Cohort: Female Less than Secondary Secondary Age cohort secondary general technical/special Tertiary (%) (%) (%) (%) 25-29 13.0 40.7 38.0 8.3 30-34 13.9 44.9 31.5 9.7 35-39 8.3 51.7 34.9 5.2 40-44 3.3 53.9 29.6 13.2 45-49 5.3 46.3 34.7 13.7 50-54 5.8 50.7 30.7 12.9 55-59 10.9 49.6 24.5 15.1 60-64 14.4 46.7 23.2 15.8 Total 9.6 47.4 32.0 11.1 Excluding current migrants. 75 Table A24. Educational Attainment by Consumption Quintile: All Less than Secondary Secondary Consumption secondary general technical/special Tertiary quintile (%) (%) (%) (%) 1 14.3 55.9 24.8 5.0 2 8.4 53.6 28.6 9.5 3 8.2 42.2 33.4 16.2 4 8.4 38.6 36.3 16.7 5 5.1 33.3 39.8 21.7 Total 8.7 44.3 32.9 14.1 Excluding current migrants. Population aged 25-64 y.o. Table A25. Educational Attainment by Consumption Quintile: Male Less than Secondary Secondary Consumption secondary general technical/special Tertiary quintile (%) (%) (%) (%) 1 13.2 53.9 27.5 5.3 2 6.1 50.3 30.7 13.0 3 7.0 35.7 35.4 21.8 4 7.7 36.3 34.6 21.4 5 5.4 29.3 40.0 25.3 Total 7.7 40.5 34.0 17.9 Excluding current migrants. Population aged 25-64 y.o. Table A26. Educational Attainment by Consumption Quintile: Female Less than Secondary Secondary Consumption secondary general technical/special Tertiary quintile (%) (%) (%) (%) 1 15.1 57.3 22.9 4.8 2 10.3 56.4 26.8 6.5 3 9.2 47.5 31.8 11.5 4 8.9 40.4 37.7 13.0 5 4.9 36.9 39.6 18.6 Total 9.6 47.4 32.0 11.1 Excluding current migrants. Population aged 25-64 y.o. Table A27. Educational Attainment by Urban/Rural: All Less than Secondary Secondary secondary general technical/special Tertiary (%) (%) (%) (%) Urban 11.5 36.8 34.5 17.3 Rural 5.9 52.1 31.2 10.8 Total 8.7 44.3 32.9 14.1 Excluding current migrants. Population aged 25-64 y.o. 76 Table A28. Educational Attainment by Urban/Rural: Male Less than Secondary Secondary secondary general technical/special Tertiary (%) (%) (%) (%) Urban 11.3 34.0 33.9 20.8 Rural 3.9 47.4 34.0 14.7 Total 7.7 40.5 34.0 17.9 Excluding current migrants. Population aged 25-64 y.o. Table A29. Educational Attainment by Urban/Rural: Female Less than Secondary Secondary secondary general technical/special Tertiary (%) (%) (%) (%) Urban 11.7 39.1 34.9 14.3 Rural 7.5 55.8 29.0 7.8 Total 9.6 47.4 32.0 11.1 Excluding current migrants. Population aged 25-64 y.o. 77 Appendix E: Cognitive and Non-cognitive Skill Mean Scores Cognitive Skills Non-Cognitive Skills Openness/ Workplace Decision Achievement Growth Memory Literacy Numeracy sociability attitude Making Striving Mindset Total -0.059 -0.039 -0.049 0 -0.015 0 -0.016 -0.015 Region Urban -0.1 -0.096 -0.075 -0.108 -0.233 -0.003 -0.152 -0.116 Region Rural -0.021 0.015 -0.025 0.102 0.191 0.003 0.113 0.079 Gender Male 0.107 0.044 0.006 -0.039 -0.058 0.017 -0.016 0.051 Gender Female -0.143 -0.081 -0.077 0.02 0.007 -0.009 -0.015 -0.049 Consumption quintile Quintile 1 -0.204 0.058 0.018 -0.106 0.012 -0.262 -0.061 -0.021 Consumption quintile Quintile 2 -0.088 0.075 0.002 -0.063 0.027 -0.072 -0.048 -0.056 Consumption quintile Quintile 3 -0.045 -0.052 -0.119 0.063 -0.056 0.002 -0.113 -0.009 Consumption quintile Quintile 4 -0.156 -0.243 -0.179 0.02 -0.089 0.106 0.026 0.01 Consumption quintile Quintile 5 0.182 -0.026 0.037 0.077 0.035 0.2 0.112 -0.001 Age cohort: 16-35 years old Young 0.051 0.003 -0.017 -0.077 -0.016 -0.053 0.013 -0.001 Age cohort: 50-65 years old Old -0.254 -0.055 -0.136 0.06 0.005 -0.065 -0.066 -0.033 Employment status Employed 0.165 0.061 0.074 0.043 0.041 0.088 0.045 -0.003 Employment status Out of work -0.313 -0.132 -0.188 -0.039 -0.073 -0.098 -0.097 -0.054 Sector of employment Agriculture 0.008 -0.013 -0.068 -0.034 0.043 0 -0.031 0.072 Sector of employment Industry -0.062 0.035 -0.028 -0.165 -0.161 0.046 -0.178 -0.245 Sector of employment Services 0.296 0.095 0.149 0.097 0.06 0.143 0.114 0.041 Type of employer SOE/Gov’t 0.332 0.068 0.173 0.109 0.112 0.129 0.118 0.069 Type of employer Private Sector -0.002 -0.035 -0.172 -0.218 -0.34 -0.106 -0.243 -0.076 Type of employer Self-employed + other 0.145 0.12 0.079 0.027 -0.034 0.156 0.024 -0.109 Educational attainment level Secondary general -0.233 -0.131 -0.196 -0.063 -0.066 -0.053 -0.1 -0.072 Educational attainment level Secondary technical/special 0.041 -0.02 0.024 0.057 0.021 0.013 0.061 0.014 Educational attainment level Tertiary 0.238 0.191 0.225 0.062 0.06 0.129 0.069 0.089 78