SOCIAL PROTECTION AND JOBS SEPTEMBER 2019 THE MALAYSIA DEVELOPMENT EXPERIENCE SERIES Monitoring Occupational Shortages Lessons from Malaysia’s Critical Occupations List CONNECT WITH US wbg.org/Malaysia @WorldBankMalaysia @WB_AsiaPacific http://bit.ly/WB_blogsMY SEPTEMBER 2019 THE MALAYSIA DEVELOPMENT EXPERIENCE SERIES Monitoring Occupational Shortages Lessons from Malaysia’s Critical Occupations List SOCIAL PROTECTION AND JOBS About KNOWLEDGE & RESEARCH The World Bank Group’s current partnership with Malaysia is focused on knowledge- sharing. It is centered on support for Malaysia’s vision to join the ranks of high-income economies by 2020 through inclusive and sustainable growth, and to share its lessons with developing countries. In March 2016, the World Bank Group officially launched its Global Knowledge and Research Hub in Malaysia. The new Hub is the first of its kind, serving both as a field presence in Malaysia and as a global knowledge and research hub. It focuses on sharing Malaysia’s people-centered development expertise and creating new innovative policy research on local, regional and global issues. Knowledge & Research reports are flagship work emanating from the teams based in the Malaysia Hub. The Malaysia Development Experience Series captures key lessons from Malaysia relevant for emerging economies in Asia, Africa and elsewhere that are transitioning out of poverty and into shared prosperity. Cover Photo attribution: © bigstockphoto.com The findings, interpretations, and conclusions expressed in this report do not necessarily reflect the views of the Executive Directors of the World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of the World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Please contact Harry Moroz, the lead author of this report, at hmoroz@worldbank.org if you have questions or comments with respect to content. Table of Contents Acknowledgements 4 List of Figures 5 List of Tables 5 List of Boxes 5 Introduction 6 Part I: The Problem 8 Part II: The Solution 10 What is the Critical Occupations List? 11 Who creates the Critical Occupations List? 13 How is the Critical Occupations List created? 13 What is the result of creating the Critical Occupations List? 16 How are the Critical Occupations List and other shortage lists used? 16 Part III: Lessons Learned 18 Learning from the strengths of the Critical Occupations List 19 Shortage lists should be updated regularly and improved continually 19 Shortage lists should be based on rigorous evidence 19 Shortage lists should be transparent 22 Shortage lists should be produced by a specialized agency 22 Shortage lists should involve private and public sector stakeholders in their production 22 Learning from challenges faced by the Critical Occupations List 23 Shortage lists should weigh the pros and cons of regional disaggregation 23 Shortage lists should invest in standardized occupational data 23 Part IV: Conclusion 24 References 26 Appendix 1: The 2018/2019 Critical Occupations List and top associated skills 28 Appendix 2: The methodology for the Critical Occupations List 31 Top-down approach 31 Bottom-up approach 34 Dovetailing 35 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 3 Acknowledgements The report was prepared by a team in the Social Protection and Jobs Global Practice of the World Bank consisting of Harry Moroz (task team leader), Achim Schmillen, and Amanina Abdur Rahman. The team received guidance and support from Firas Raad and Philip O’Keefe. Mauro Testaverde led the initial work on the Critical Occupations List, and is owed tremendous thanks. Joshua Foong, Kane Chong, and Elizabeth Schwinn provided excellent editorial assistance. We are indebted to TalentCorp, the Institute of Labour Market Information and Analysis, and the Critical Skills Monitoring Committee for their leadership in establishing the Critical Occupations List and valuable collaboration with the World Bank. In particular, the team is grateful to Mahuran Saro Sariki, Sivakumeren A. Narayanan (both Deputy CEOs of TalentCorp), the late Dr. Lee Chee Sung (Advisor at ILMIA), and all past and present members of the Critical Skills Monitoring Committee. We would like to dedicate this publication to the late Dr. Lee Chee Sung whose wisdom shaped the Critical Occupations List. 4 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List List of Figures Figure 1. Defining critical occupations 13 Figure 2. The process of compiling the Critical Occupations List 15 Figure 3. Summarizing the top-down approach 33 List of Tables Table 1. Sample shortage indicators 14 Table 2. Potential future applications of the Critical Occupations List 17 Table 3. Final indicators included in the top-down approach in the 2018/2019 COL 32 List of Boxes Box 1. Shortage lists in the United Kingdom and Australia 12 Box 2. Using real-time labor market information to improve policymaking 20 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 5 Introduction 6 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Introduction Many emerging economies have skills shortages Plan to monitor skills imbalances in Malaysia. To do but fail to effectively deploy students and so, the CSC created an annual Critical Occupations job seekers towards filling those shortages. List (COL) of occupations that are middle- or high- In emerging economies, new technologies, skilled, sought-after, and strategic. The COL has digitization, automation, and other trends like evolved during the last several years to become a Industry 4.0 result in a constantly changing demand best-practice tool for monitoring skills. The COL for sophisticated skills. In this environment, there is updated regularly and improved continually, are often students and job seekers who could is based on rigorous evidence, and is widely potentially fill skills gaps but face significant circulated. The production of the list is undertaken challenges in identifying job opportunities and by the CSC and incorporates a wide range of input the skills needed to obtain them. The schools, from both the public and private sector. training centers, and public service providers that are responsible for preparing the workforce to fill This is a case study of the origins, development, skills gaps often face similar challenges in terms of and lessons of the Critical Occupations List. deploying attention and resources towards filling The case study has four parts. Part I describes those gaps. the problem that the COL aims to solve, namely policymakers’ lack of a tool to monitor labor market In Malaysia, the Critical Skills Monitoring shortages. Part II discusses the COL’s origins, Committee (CSC) is charged with producing a methodology, and use. Part III compiles lessons Critical Occupations List to serve as a platform learned from the COL, and provides key takeaways for coordinating human capital development for policymakers in other countries considering the policies. The CSC is a specialized interagency body development of a skills monitoring tool. Part IV that was established as part of the Eleventh Malaysia concludes. Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 7 PART I The Problem 8 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Part I: The Problem M a l a y s i a ’s l a b or m a r ket h a s evo lved service providers responsible for preparing them significantly in recent decades. An economy to fill those gaps—face a significant challenge in that was once driven by agriculture and trying to identify where job opportunities are and manufacturing is increasingly shifting to growth what skills are needed to obtain them. based on productivity, innovation, and knowledge creation. The transformation has had significant Filling labor market shortages is strategically consequences for the labor market. As Malaysia important for an economy whose growth urbanized and the economy grew, jobs shifted strategy is based on knowledge production and first to the manufacturing sector and then to the innovation. Labor market shortages can disrupt services sector, which now accounts for more than economic growth. Such shortages reduce output half of Malaysia’s GDP and around 60 percent of and productivity in the short run when high-potential employment. firms cannot hire the talent needed to grow. If such shortages persist, there is a risk of long-run Malaysia’s shift to an economy based on losses in competitiveness and businesses’ ability high productivity and knowledge creation has to innovate. Identifying labor market shortages as resulted in shortages in the labor market. Labor they arise and developing strategies to fill them is market or skills shortages arise when the supply of therefore important to maintain productivity and workers with particular skills is insufficient to meet competitiveness. demand (Veneri 1999). Put simply, labor market shortages are jobs that employers want to fill but The government of Malaysia created the Critical have difficulty filling. The condition occurs when Occupations List as a tool to identify labor wages or labor supply do not adjust to demand, or market shortages. Despite investing substantial when information problems prevent job searchers public resources in human capital development, the and employers from finding each other. government of Malaysia did not have an effective mechanism to monitor labor market shortages. These shortages are not unique to Malaysia but Three problems hindered the government’s ability they are particularly acute there, for several to monitor those shortages effectively. First, the reasons. First, Malaysia has a small working-age workforce development system was supply-side population compared to regional competitors like driven with few channels for information to flow China, Indonesia, Thailand, and Vietnam, which from employers to decision makers, education and limits its labor supply. Second, Malaysia’s strong training providers, or individuals eager to invest in economic growth in recent years has resulted in a human capital. Second, formal responsibility for generally tight labor market: The unemployment identifying the skills needed to propel the economy rate has been below 4 percent for the last two was dispersed among a number of government decades, further limiting the number of workers ministries, government agencies, and state-owned available for employers to hire. Finally, labor enterprises that did not coordinate well with each market shortages can be particularly problematic other. Third, efforts to identify skill needs followed in emerging economies like Malaysia’s, where new an outdated “manpower planning” approach based technologies, digitization, automation, and other on predicting quantities of workers that was not trends like Industry 4.0 result in a constantly changing well-suited to keep pace with a dynamic, globally demand for particular skills. In that environment, integrated, market-led economy. the students and jobseekers who could fill skills gaps—and the schools, training centers, and public Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 9 PART II The Solution 10 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Part II: The Solution In 2014, the government of Malaysia established a Critical Skills Monitoring Committee (CSC) with the mandate to monitor skills imbalances. In its five-year national planning strategy, the Eleventh Malaysia Plan, the government called for the creation of a committee to identify critical skills gaps in strategic sectors to enhance human capital planning. The result was the CSC, which is jointly chaired by Talent Corporation (TalentCorp) and the Institute of Labour Market Information and Analysis (ILMIA), both agencies in the Ministry of Human Resources (MOHR). The CSC’s primary objective is to manage and update the COL to identify labor market shortages. The COL is a platform for the coordination of human capital development policies that is adapted from international best practices in skills monitoring. More practically, the COL is a list of occupations for which there is strong evidence of a labor shortage that may be alleviated through government action. Occupations on the COL are middle- or high-skilled, sought-after, and strategic. The COL seeks to identify and draw stakeholder attention to occupations that are critical to the continued growth and development of the Malaysian economy but that are currently difficult to fill. What is the Critical Occupations List? Occupational shortage lists identify and monitor critical skills and needed occupations. Occupational shortage lists are an approach to skills monitoring that is designed to help align workforce development policies with employers’ needs. They do this by monitoring the industries, occupations, and skills that are in demand; by monitoring the skills of the current labor force and current students; and by assessing how well the supply of workers meets the demands of employers. The result is a list of occupations, and sometimes the qualifications and skills associated with them, for policymakers who are involved in making decisions about human capital development policy. Shortage lists are common in the OECD. Sixteen OECD countries employ shortage lists. The application of the lists varies, but policymakers generally use the lists to identify short- and medium-term occupational and skills gaps, frequently seeking to fill those gaps through immigration but also by directing resources to training programs and job seekers. The United Kingdom and Australia are prominent examples of countries that use shortage lists, and experts from both countries advised the CSC regarding the creation of the COL (Box 1 ). In both countries, the methodologies draw on quantitative and qualitative data sources to make determinations about labor market shortages. The COL is Malaysia’s labor market shortage list. The COL is primarily concerned with identifying labor market shortages that are associated with Malaysia’s growing knowledge-based economy. To accomplish this, the COL focuses on identifying occupations that are (mid- or high-) skilled, according to the Malaysian Standard Classification of Occupations (MASCO). The COL also determines whether there are mismatches between employers’ demand for certain occupations and the supply of skills associated with those occupations. As a result, the COL seeks to identify shortages in occupations that are sought after by employers. Finally, the COL is designed to be a tool to help policymakers make decisions. Thus, after shortages in skilled occupations are identified, an occupation is considered critical only if filling that occupational shortage is consistent with Malaysia’s strategic economic development objectives. Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 11 Part II: The Solution BOX 1 Shortage lists in the United Kingdom and Australia An independent body of economists is responsible for developing the UK’s Shortage Occupation List. The UK’s Migration Advisory Committee (UK MAC) is an independent body established in 2007 and staffed by economists that provides evidence-based advice on migration to the government. Its work includes advising the government on skills shortages by developing and revising a Shortage Occupation List (SOL). The primary objective of the SOL is to help the government fill labor market shortages with workers from outside the European Economic Area, though the list also informs policymakers’ decisions about training and education. Australia has a research program devoted to identifying labor market shortages, as well as skilled occupation lists that are designed to fill labor market gaps through migration. The Department of Jobs and Small Business, formerly the Department of Employment, has a research program to identify shortages in skilled occupations. The research program informs education, training, employment, and migration policies. The department also produces the Short Term Skilled Occupation List (STSOL) and the Medium and Long Term Strategic Skills List (MLTSSL), which guide decisions about temporary and permanent migration. Source: Authors 12 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Part II: The Solution Who creates the Critical Occupations List? The CSC is an interagency body that creates the COL. The CSC is composed of representatives from TalentCorp and ILMIA, which are both under the Ministry of Human Resources. Each agency brings a unique strength to the CSC. TalentCorp has a mandate to nurture talent in Malaysia through a suite of initiatives for graduates and professionals that build skills, create connections with employers, and seek to make living and working in Malaysia more attractive. Thus, TalentCorp brings to the CSC its partnerships with the private sector and its knowledge of the skills ecosystem in Malaysia. ILMIA analyzes Malaysia’s labor market and human capital, and has built a data warehouse of labor market information drawn from a variety of sources. Thus, ILMIA brings to the CSC expertise in labor market analytics and access to and understanding of Malaysia’s different sources of labor market information. The CSC has been supported by the World Bank, which provided technical advice and hands-on support during the creation of the COL. How is the Critical Occupations List created? The methodology for the COL combines rigorous quantitative methods with stakeholder engagement. The methodology for the COL builds on international best practices and uses a mixed methods approach to define occupations that appear on the list. The criteria for inclusion is three-fold: occupations must be skilled, sought-after, and strategic (Figure 1 ). The process of defining skilled occupations relies on Malaysia’s 10-tiered MASCO. The first eight tiers of MASCO, which range from Managers (tier 1) to Plant and Machine Operators (tier 8), are considered to be mid- or high-skilled occupations. The process of defining occupations that employers are in need of (where demand exceeds supply) and that are strategic (central to economic development) requires rigorous analysis of quantitative and qualitative evidence in the “top- down,” “bottom-up,” and “dovetailing” approaches summarized below (which are described in more detail in Appendix 2 ). FIGURE 1. Defining critical occupations Critical occupations Criteria Skilled Sought-after Strategic Demand Central to Mid- or Definition high-skilled exceeds economic supply development Top-down, bottom-up, Methodology MASCO 1-8 and dovetailing Note: MASCO stands for Malaysian Standard Classification of Occupations. Source: Authors Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 13 Part II: The Solution • The top-down approach provides the basis for determining whether an occupation is sought-after by identifying shortages. The top-down approach generates objective evidence of the current labor market that is comparable over time and across occupations. That evidence draws on multiple quantitative data sources such as labor force surveys and administrative data. It relies on multiple indicators of whether an occupation has a shortage and provides initial evidence of whether employers are seeking workers in that occupation. Those indicators can also offer guidance about which occupations are strategic. Table 1 provides examples of shortage indicators used in the top-down process. TABLE 1. Sample shortage indicators Data source Indicator Shortage rationale • An increase in the number of employees in an occupation is Labour Force 3-year employment a sign of a previous increase in the number of vacancies (or a Survey growth reduction in firing/quits). Rising employment in an occupation suggests that the relative demand for that occupation is rising. Survey of • Rising median wages in an occupation relative to other 3-year median wage Wages and occupations may indicate increased demand for workers in that growth Salaries occupation. Online job • A large number of vacancies in an occupation is positively Vacancy rate posting data associated with rising labor demand. Source: Authors • The bottom-up approach complements the top-down approach to generate additional evidence of sought-after occupations. The bottom-up approach serves two functions. The first is to build an evidence base directly from stakeholders that, in conjunction with top-down information, allows for a systematic assessment of which occupations merit inclusion on the COL. The second is to build contextual knowledge about occupations and sectors to allow the CSC to continually update its efforts to interpret indicators, communicate decisions, and plan monitoring efforts between COL rounds. The bottom- up approach involves a Call for Evidence (CfE), which is a survey of employers, and consultations with employers and industry associations. The CfE asks a wide range of employers about occupations they believe are in shortage to generate as complete a picture as possible of employers’ hiring challenges. Consultations are an opportunity to collect information similar to that gathered by the CfE, but also allow for the collection of additional contextual information that can aid in interpretation of the CfE and of the top-down evidence. 14 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Part II: The Solution • A process of “dovetailing” the top-down and bottom-up approaches ensures that the final COL reflects both quantitative evidence and the input of knowledgeable stakeholders. Dovetailing helps synthesize the evidence from the top-down and bottom-up approaches to develop the final list. When used together, the top-down and bottom-up components combine objective and contextualized indicators of the degree to which a skilled occupation is sought-after. The strategic importance of occupations that meet the skilled and sought-after criteria for inclusion on the COL is also assessed during dovetailing. This assessment is based on a variety of factors, including which industries report shortages in certain occupations, the degree to which employers are actively seeking to fill shortages, the potential impact of shortages on businesses and industries, and the ability to automate particular occupations. The use of both quantitative and qualitative sources of information in creating the COL allows for robust justifications for an occupation’s inclusion on the list. Figure 2 summarizes the process of compiling the COL. FIGURE 2. The process of compiling the Critical Occupations List All occupations Top-down Bottom-up Skilled Skilled screen Call for Indicators of Stakeholder Other data Sought-after shortage Evidence consultations sources Survey Strategic Dovetailing Final COL Source: Authors Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 15 Part II: The Solution What is the result of creating the Critical Occupations List? The COL has been created annually since 2015. The CSC first launched a pilot list for 2015/2016. Lists were also created in 2016/2017, 2017/2018, and 2018/2019. The 2018/2019 COL included 59 occupations, ranging from managing directors and chief executives to mechanical engineers, aircraft pilots, software developers, and heavy truck and lorry drivers. Most of the occupations on the COL are high- skilled occupations at the managerial, professional, and associate professional level but some are mid-skilled occupations. The latest COL represented 12 percent of the 480 non-military 4-digit occupations in MASCO and about 23 percent of total employment in Malaysia in 2017. Appendix 1 shows the full list of occupations on the COL and the top three skills required for each occupation. Each iteration of the COL seeks to improve on the previous version. The CSC has learned from the United Kingdom and Australia’s experiences with shortage lists as well as its own experience. The methodology for producing the COL has evolved over time. The list has been expanded to include mid- as well as high-skilled occupations. Improvements have been introduced to allow for more holistic consideration of labor market conditions and their potential impact on Malaysian employers. An important improvement that has increased the quality of the COL over time is the growing availability of data sources, in part because of the creation of ILMIA’s Labour Market Information Data Warehouse, which has centralized labor market information in a single location. To further enhance the capacity of the COL to be a useful tool for decision making, the most recent COL includes detailed information about each occupation’s characteristics and skill requirements. How are the Critical Occupations List and other shortage lists used? Governments around the world have implemented shortage lists to improve policymaking for human capital development. Most countries have used shortage lists in two areas. First, countries like Australia and Ireland use them to inform decisions about education and training programs. Because they identify areas where skills are lacking, shortage lists are a useful tool to help guide resources to training programs that can generate those missing skills. Second, countries like New Zealand and the United Kingdom use the lists to inform immigration decisions. Shortage lists are often used to attract high-skilled labor that can fill gaps in the labor market immediately, without the time lag involved in developing and delivering training programs. In some cases, shortage lists have also been used to identify highly skilled individuals who can meet longer- term skills needs. The uses of shortage lists are not mutually exclusive. Occupations with a shortage of workers that can be filled through immigration in the short term can also be targeted by investments in education and training in the medium term. Many stakeholders use Malaysia’s COL as a source of labor market intelligence and to inform immigration policy. Various stakeholders, including policy makers, training institutions, institutions of higher education, job seekers and students use the COL as a source of information. The recent Mid-Term Review of the Eleventh Malaysia Plan accords the COL a prominent place and mentions that “[t]he COL report will 16 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Part II: The Solution be continuously updated to identify shortages of workers in critical occupations that will have an impact on the economy. The Government will intervene based on the report by reprioritising the field of studies and leveraging the Malaysian diaspora.” Institutions of higher education are using the COL in developing courses of study that meet industry demand. As part of the government’s efforts to leverage the Malaysian diaspora, TalentCorp is using the COL in very concrete terms in its Returning Expert Programme (REP). TalentCorp’s REP seeks to attract qualified Malaysians who are working abroad back to Malaysia. The REP uses a points-based system to assess candidates’ qualifications. Points are awarded for education, salary, and work experience. Points are also awarded to candidates in occupations that appear on the COL. Thus, the REP is a strategic tool to fill labor market shortages identified by the COL. The COL has the potential to help adapt workforce development policies in Malaysia to economic needs in the context of Industry 4.0. The Malaysian economy and the global economy are changing rapidly as new advances disrupt labor markets. Those changes will require Malaysia’s workforce development policies to be responsive and flexible as new occupations emerge and employers demand new skills. The COL has the potential to enhance that flexibility in three areas: First, the COL can further improve the responsiveness of upskilling and reskilling programs to industry needs. Second, the COL can further improve the efficiency of the job-matching process for displaced and retrenched workers. Third, the COL can further improve the responsiveness of immigration admissions decisions to economic needs (See Table 2 ). TABLE 2. Potential future applications of the Critical Occupations List Policy area Application • Inform development of TVET and higher education programming • Inform development of occupational standards and accreditation Upskilling and reskilling • Target funding for TVET and higher education programming • Target incentives to businesses for apprentices • Inform career counsellors about occupations in shortage Employment services • Target upskilling and reskilling for jobseekers Immigration • Target admissions to workers in occupations on the COL Source: Authors Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 17 PART III Lessons Learned 18 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Part III: Lessons Learned Learning from the strengths of the Critical Occupations List Lessons from the CSC’s experience with the COL suggest several key principles for implementing a shortage list that is accurate, accepted, and relevant. Those key principles can help guide the creation and use of shortage lists in other countries. Shortage lists should be updated regularly and improved continually Shortage lists should be updated regularly and improved continually. The COL is updated each year to ensure that it remains relevant to current labor market needs. Improvements to successive iterations of the COL have expanded the number of data sources reviewed, expanded and improved the calculation of shortage indicators, and expanded the number of stakeholders consulted in the bottom-up process. The first COL was a pilot that included just two shortage indicators from two datasets and involved limited consultations with stakeholders and no survey. The depth and breadth of evidence considered by the COL was expanded over time as the CSC became more experienced. By the 2018/2019 iteration of the COL, 14 shortage indicators were included based on review of 14 different datasets. They were supplemented with evidence from 23 consultations and nearly 4,000 surveys of stakeholders about critical occupations. The process of expansion involved regular consideration of what was working and what was not, and regular communication with government and private sector stakeholders to ensure that those in a position to provide relevant data were motivated to do so. Shortage lists should be based on rigorous evidence The methodology for creating the shortage lists should be evidence-based. The process of developing the COL begins with a consideration of all available data sources that potentially have information relevant to the labor market. Rather than rely on a single measure of labor market shortages, multiple measures that are both quantitative and qualitative in nature are preferred (Veneri 1999; Shah and Burke 2005; Richardson 2007; Infometrics 2006). That helps ensure that diverse sources of labor market information are considered. Information about employment and unemployment rates, vacancy and hard-to-fill vacancy rates, and changes in wages; employer surveys; and the opinions of employers, regulators, educational institutions, and other labor market stakeholders are all important to understand the future skills of the workforce, future demand by employers, and future shortages. A complete set of information is almost never available, so careful consideration of which available indicators are best suited to the labor market is key. Those indicators should be evaluated rigorously for how well they capture shortages. The COL methodology considers each indicator on its own against shortage benchmarks; then it considers indicators in combination; finally, it considers different combinations of indicators. Qualitative information from surveys and consultations is incorporated to supplement and provide context for quantitative data. Real-time labor market information can enhance shortage lists by providing a proxy for vacancies and detailed occupational information. Real-time labor market information takes advantage of the proliferation of online job search by “scraping” or “spidering” the Web for job postings, resumes, and other employment-related information (Box 2 ). This scraping results in raw data that can be assembled and Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 19 Part III: Lessons Learned structured into information on occupations, vacancies, salaries, skills, and career pathways. Real-time labor market information can be collected constantly, resulting in monthly or even daily data on labor market trends. Real-time data is particularly useful when vacancy data is limited or unavailable. Additionally, real- time labor market information techniques can generate detailed data about job openings and worker skills. The COL incorporates real-time labor market information in two ways. First, online job posting data is used as a proxy for job openings to create indicators of labor market shortages because information on vacancies is not available from other sources. Second, online job posting data provide detailed information on the skills and experience requirements of occupations in high demand. Shortage lists should be transparent The methodology for creating shortage lists should be transparent. Making shortage lists and their methodologies publicly available increases trust that they accurately reflect labor market needs. Each iteration of the COL is accompanied by a report. That report is both a methodological manual that makes the process of producing the COL transparent (including any shortcomings and recommendations for improving it) and a guide to how the COL can be useful to policymakers. Helping stakeholders understand the tool, its purpose, and its benefits is essential to motivate policymakers, training institutions, employers, and others to use the shortage list and to contribute to its improvement by providing their own evidence of shortages. Shortage lists should be produced by a specialized agency The body responsible for creating shortage lists should be specialized and make use of relevant expertise. The staff of the CSC devotes a significant portion of time to creating the COL. That allows them to develop occupational and sectoral expertise and deep knowledge of the COL’s methodology. A significant advantage of the CSC’s structure is the partnership between TalentCorp and ILMIA. The institutional structure allows the CSC to draw on ILMIA’s labor market information sources, including its Labour Market Information Data Warehouse, and on TalentCorp’s ties with stakeholders in the private sector. Shortage lists should involve private and public sector stakeholders in their production The production of shortage lists should involve many different labor market stakeholders. Soliciting input from stakeholders such as employers, industry associations, unions, and others not only contributes valuable evidence to the shortage lists, but also raises the profile of the lists and generates buy-in from potential users. The CSC has devoted significant effort to developing relationships with private sector stakeholders. The number of responses to the annual survey about critical occupations grew from fewer than 200 to nearly 4,000 in just four years. During the same period, the number of consultations grew from 9 to 23. Those sources provide vital intelligence from the employers that know the most about current demands in the labor market. Relationships with data providers are also essential because data is the lifeblood of the quantitative evidence included in shortage lists. Relationships should be cultivated with data providers and they should be kept informed about how their data are used. 20 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Part III: Lessons Learned BOX 2 Using real-time labor market information to improve policymaking The benefits of real-time labor market information are its timeliness and granularity. Real-time information is generated constantly and can be useful almost immediately. Different from traditional surveys of the labor market that take weeks or months to be completed and published, real-time data can provide users with an up-to-date portrait of the labor market, assuaging concerns that information is outdated. Real-time information on the labor market provides a very granular picture of occupations, skills, salaries, industries, education levels, and other characteristics, allowing users of the information to enhance their knowledge of the labor market and improve policy interventions. Labor market information systems and workforce development policies around the world incorporate real-time labor market information. Job search websites are increasingly being used by academics to study labor market dynamics, job search, the relationship between skills and wages, and labor market concentration (Marinescu 2017; Deming and Kahn 2018; Azar et al. 2018). Policy applications are also proliferating. The European Centre for the Development of Vocational Training is using online job ads to establish a pan-EU tool for collecting data on skills demand. Local governments in the United States, including economic development agencies, educational institutions, and workforce development agencies, are using real-time labor market information to inform career counsellors about occupations in demand and the qualifications that can help lead to careers in those occupations; to inform sector strategies; and to target education and training investments (Alstadt 2011; Maher & Maher 2015). The City University of New York uses real-time labor market information to create “career maps” for medical assistants, home health aides, and cooks (Maher and Maher 2015). The Consortium for Workforce and Economic Development of the New Jersey Council of County Colleges uses data from online job posts to help community colleges direct training opportunities to occupations that are in demand. Source: Authors Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 21 Part III: Lessons Learned 22 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Part III: Lessons Learned Learning from challenges faced by the Critical Occupations List The Critical Occupations List, like all shortage lists, faces challenges that the CSC continues to work on. The CSC continues to face challenges as it works to increase the effectiveness of the COL. However, those challenges are not unique to the COL. They, and the CSC’s responses to them, have important lessons for other countries considering shortage lists. Shortage lists should weigh the pros and cons of regional disaggregation As in many countries, labor markets in Malaysia vary between regions. In Malaysia, geographic labor market differences are most clearly reflected in different employment rates, labor force participation rates, and sectors of employment. For instance, the unemployment rate in Peninsular Malaysia was 3 percent in 2017, compared to 5 percent in East Malaysia on the island of Borneo. Seven percent of employment was in agriculture in Peninsular Malaysia, compared to 28 percent in East Malaysia. The variation suggests that labor market shortages likely vary across geographies as well. While the current version of the COL does not directly consider such variations, its most recent version included a pilot of a subnational COL that divided Malaysia into Peninsular Malaysia and East Malaysia. Although the two geographic areas are quite large and labor markets may vary within them, the approach demonstrated the potential benefits and challenges of creating a subnational COL. Still, developing full subnational shortage lists requires significant additional resources because surveys are not always representative at subnational levels and consultations must be undertaken for different regions, which multiplies the number of meetings required. Shortage lists should invest in standardized occupational data Obtaining data with standardized occupational classifications is not always possible. Obtaining datasets with standardized occupations is essential since occupations are the backbone of the COL and similar shortage lists. However, occupational information is not always classified in a standardized format. While survey data in Malaysia is frequently provided in the standardized MASCO scheme, administrative and qualitative data—and some survey data—is not. The CSC has worked with data providers to classify occupations in MASCO. The CSC has also employed machine-learning techniques that can classify unstructured text, such as job titles or job descriptions, into MASCO. Nonetheless, standardized occupational data continues to be a challenge to obtain and will require significant investment by practitioners developing their own shortage lists. Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 23 PART IV Conclusion 24 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Part IV: Conclusion The Government of Malaysia established the CSC to monitors skills imbalances. One of the CSC’s primary objectives is to develop a COL annually to serve as a platform for the coordination of human capital development policies. The CSC is jointly chaired by TalentCorp and the Institute of Labour Market Information and Analysis, which are agencies in the Ministry of Human Resources. This institutional structure allows the CSC to draw on ILMIA’s labor market information sources, including its Labour Market Information Data Warehouse, and on TalentCorp’s ties with stakeholders in the private sector. The CSC has published an annual Critical Occupations List (COL) since 2015. The COL is a list of occupations for which there is strong evidence that there is significant labor market shortage that may be alleviated through government action. Occupations on the COL meet the criteria of being mid- or high- skilled, sought-after, and strategic. The COL seeks to identify and draw stakeholder attention to this set of occupations that are critical to the continued growth and development of the Malaysian economy but that are currently difficult to fill. The experience of Malaysia with the COL has several lessons for other countries exploring tools to monitor skills shortages. 1. Shortage lists are a useful tool for monitoring labor market shortages, and they are particularly critical now as new technologies disrupt labor markets. 2. Shortage lists should be updated regularly and improved continually, should be based on rigorous evidence, and should be transparent. Governments can take advantage of new data sources, such as online job postings, to create lists that are up-to-date and include detailed information about skills requirements. 3. Shortage lists should be developed by a specialized agency but should incorporate evidence from public and private sector stakeholders. Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 25 References Altstadt, David. 2011. “Aligning Community Colleges to Their Local Labor Markets: The Emerging Role of Online Job Ads for Providing Real-Time Intelligence about Occupations and Skills in Demand.” Jobs for the Future and Credentials That Work, Boston, MA. Azar, José A., Ioana Marinescu, Marshall I. Steinbaum, and Bledi Taska. 2018. “Concentration in US Labor Markets: Evidence from Online Vacancy Data,” Working Paper 24395, National Bureau of Economic Research, Cambridge, MA. Critical Skills Monitoring Committee (CSC). 2019. Critical Occupations List 2018/2019: Technical Report. CSC, Kuala Lumpur. Deming, David and Lisa B. Kahn. 2018. “Skill Requirements across Firms and Labor Markets: Evidence from Job Postings for Professionals,” Journal of Labor Economics 36(S1):S337-69. Infometrics. 2006. “Indicators of Skill Shortage,” Prepared by Infometrics Ltd. for Department of Labour, New Zealand. MAC (United Kingdom Migration Advisory Committee). 2017. Assessing labour market shortages: A methodology update. London: United Kingdom Migration Advisory Committee. Maher & Maher. 2015. Promising Practices in Leveraging Traditional and Real-Time Labor Market Information,. U.S. Department of Labor Employment and Training Administration Office of Workforce Investment. Marinescu, Ioana. 2017. “The General Equilibrium Impacts of Unemployment Insurance: Evidence from a Large Online Job Board,” Journal of Public Economics 150:14-29. Richardson, Sue. 2007. What is a skill shortage? Adelaide: National Centre for Vocational Education Research. Shah, Chandra and Gerald Burke. 2005. “Skills Shortages: Concepts, Measurement and Policy Responses,” Australian Bulletin of Labour 31(1):44-71. Veneri, Carolyn M. 1999. “Can occupational labor shortages be identified using available data?” Monthly Labor Review 122:15-21. 26 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Appendix Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 27 Appendix Appendix 1: The 2018/2019 Critical Occupations List and top associated skills MASCO title Top skill Second skill Third skill Managing Directors and Chief Executives Communication Skills English Planning Finance Managers Accounting Finance Budgeting Human Resource Managers Communication Skills English Planning Policy and Planning Managers Planning Communication Skills English Business Services Managers - - - Business Services and Administration Communication Skills Project Management English Managers Not Elsewhere Classified Sales and Marketing Managers Sales Communication Skills English Advertising and Public Relations Managers Communication Skills English Social Media Research and Development Managers Communication Skills Research English Manufacturing Managers Communication Skills Planning English Construction Construction Managers Chinese English Management Information and Communications Communication Skills English Project Management Technology Managers Quality Assurance and Chemists Chemistry English Control Geotechnical Geologists and Geophysicists Civil Engineering Communication Skills Engineering Mathematicians, Actuaries and Microsoft Excel Communication Skills English Statisticians Quality Assurance and Industrial and Production Engineers Communication Skills English Control Civil Engineers Civil Engineering Communication Skills English Mechanical Engineers Mechanical Engineering Communication Skills English Chemical Engineers Process Engineering Communication Skills Troubleshooting Mining Engineers, Metallurgists and Teamwork / English Communication Skills Related Professionals Collaboration Engineering Professionals (Excluding Electrotechnology) Not Elsewhere Communication Skills English Chinese Classified 28 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Appendix MASCO title Top skill Second skill Third skill Electrical Engineers Electrical Engineering Communication Skills Troubleshooting Electronic Engineers Troubleshooting Communication Skills English Graphic and Multimedia Designers Graphic Design Creativity Adobe Photoshop Aircraft Pilots and Related Professionals English Communication Skills Chinese Manufacturing Professionals Welding English Chinese Environmental and Occupational Health Occupational Health and English Communication Skills and Hygiene Professionals Safety University and Higher Education Lecturer English Research Professional Teachers Early Childhood Educators Teaching Chinese English Accountant and Auditor Accounting Communication Skills English Financial and Investment Advisers Sales English Planning Financial Analysts Accounting Communication Skills Finance Advertising and Marketing Professionals Marketing English Social Media Systems Analysts Communication Skills English SAP Software Developers Software Engineering Software Development SQL Applications Programmers SQL English .NET Software and Applications Developers Quality Assurance and English Communication Skills and Analysts Not Elsewhere Classified Control Systems Administrators Troubleshooting Communication Skills English Teamwork / Computer Network Professionals Troubleshooting Communication Skills Collaboration Database and Network Professionals Not Communication Skills Information Security English Elsewhere Classified Civil Engineering Technicians Troubleshooting English Communication Skills Electrical Engineering Technicians Troubleshooting Preventive Maintenance English Mechanical Engineering Technicians Troubleshooting Preventive Maintenance English Physical and Engineering Science Quality Assurance and Troubleshooting English Technicians Not Elsewhere Classified Control Manufacturing Supervisors - - - Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 29 Appendix MASCO title Top skill Second skill Third skill Other Supervisor Not Elsewhere Classified English Communication Skills Chinese Commercial Sales Agents Sales Chinese English Chefs Cooking English Food Preparation Receptionists Administrative Support English Communication Skills Welders and Flame Cutters - - - Agricultural and Industrial Machinery - - - Mechanics and Repairers Electrical Mechanics and Fitters - - - Bakers, Pastry, Pasta and Confectionery - - - Makers Tailors, Dressmakers, Furriers and Hatters - - - Rubber Products Machine Operators - - - Steam Engine and Boiler Operators - - - Stationary Plant and Machine Operators - - - Not Elsewhere Classified Heavy Truck and Lorry Drivers - - - Mobile Farm and Forestry Plant Operators - - - Note: “-“ indicates not available. Source: CSC 2019. 30 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Appendix Appendix 2: The methodology for the Critical Occupations List The methodology for creating the COL combines rigorous quantitative methods with stakeholder engagement. The criteria for inclusion on the list is threefold: occupations must be skilled, sought-after, and strategic. Occupations are determined to be skilled based on the MASCO occupational classification scheme. This process is relatively mechanical based on classifications of occupations established by MASCO. The first eight tiers of MASCO are considered mid- or high- skilled occupations. Sought-after occupations are identified using quantitative indicators of shortage and qualitative evidence from employers and other stakeholders. Strategic means that an occupation is central to Malaysia’s economic development objectives. The process of defining occupations that are sought-after and that are strategic requires rigorous analysis of quantitative evidence through a top-down approach, rigorous analysis of qualitative evidence through a bottom-up approach, and a thorough process of dovetailing evidence resulting from both approaches. This appendix describes the top-down, bottom-up, and dovetailing processes that identify sought-after and strategic occupations for inclusion on the COL. Top-down approach The “top-down” approach provides the basis for determining whether an occupation is sought-after by identifying shortages. Developing shortage indicators is challenging because labor market indicators can mean different things in different contexts. For example, salaries might fluctuate before reaching equilibrium, or a large number of vacancies can be caused by high turnover, undermining the effectiveness of those measures as shortage indicators (MAC 2017). The top-down methodology takes such issues into consideration by looking at a range of data sources to define both quantity indicators (for example, employment levels and vacancies) and price indicators (for example, earnings and wage premiums). The methodology also uses indicator-specific thresholds to define when each indicator is likely to provide evidence of shortage, and employs a “traffic light” approach so that evidence of a shortage in a certain percentage of indicators is the final gauge of a shortage, rather than evidence of a shortage in a single indicator. The top-down approach is a process of identifying data sources, shortage indicators, and shortage occupations. The first step in the top-down approach is to identify data sources with information about shortages. Next, a set of shortage indicators is calculated with those data sources and a number of different indicators are tested singly and in combination. A final preferred combination of indicators for the top-down methodology emerges from this process. • Identify data sources. The first step in compiling the top-down list of shortage occupations is to compile all of the available datasets that can be used to construct quantitative indicators of labor market shortage. This step seeks to be over-inclusive: at this stage the relevance of datasets is evaluated rather than the practicality of constructing indicators from them. Datasets to consider include both surveys and administrative data. The 2018/2019 COL considered the Labour Force Survey, the Survey of Wages and Salaries, job vacancies registered with the public employment agency JobsMalaysia, databases of vocational and university student graduates, immigration databases, and data collected from online job-search websites. Those datasets are then evaluated on two criteria: 1) the availability of data in the most recent year and in previous years; and 2) the feasibility of calculating indicators using occupation as the unit of analysis. Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 31 Appendix • Calculate shortage indicators. The identification of data sources with potential shortage indicators that have occupational information permits the compilation of an initial list of shortage indicators. The initial list includes all indicators that have an economic rationale for using them as evidence of an occupational shortage and that have sufficient data based on occupation. In the 2018/2019 COL, there were 19 initial indicators from five datasets. This initial list is refined through a process that ensures that each indicator adds unique shortage information to the top-down methodology. This involves looking at the correlation between indicators. This also involves evaluating which occupations are found to be in shortage using each indicator. This second step requires defining thresholds for each indicator using a benchmark period defined as a period of high economic and employment growth. At this stage, multiple thresholds are used to test more and less restrictive scenarios. The result is a set of intermediate shortage indicators that provide unique shortage information. Two indicators were pared from the initial list for the 2018/2019 COL, leaving 17 intermediate indicators from five datasets. • Select final specification. The final step in the top-down approach is to combine the intermediate shortage indicators to create lists of occupational shortages. Thus far, each indicator has been considered individually. In order to make final decisions about which shortage indicators to include, however, different combinations of indicators must also be considered to ensure that when they are combined they all add pertinent information about labor market shortages. Creating those lists of occupational shortages requires setting a rule for how to combine the evidence from the different shortage indicators. Following the UK Migration Advisory Committee, the COL considers occupations TABLE 3. Final indicators included in the top-down approach in the 2018/2019 COL Data source Indicator 1) 1-year employment growth 2) 3-year employment growth 3) 1-year working hours growth Labour force survey 4) 3-year working hours growth 5) 1-year education level decrease 6) 3-year education level decrease 7) 1-year wage premium growth Survey of Wages and Salaries 8) 3-year wage premium growth 9) 1-year RP-T applications growth Resident Pass-Talent database 10) 3-year RP-T applications growth 11) # of vacancies 12) Vacancy rate (% of employment) Online job posting data 13) 1- year education level decrease 14) 1-year experience level decrease Note: The Resident Pass-Talent (RP-T) is a program run by TalentCorp to encourage talented expatriates who are currently working in Malaysia to remain in Malaysia for work. Source: Authors 32 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Appendix in shortage to be those with values exceeding the shortage threshold for more than half of the available indicators. Potential lists of shortage occupations are created by testing different combinations of indicators. Those tests are evaluated on a number of criteria. One of the most important is the number of occupations found to have a shortage. Longer lists are preferred because they permit further scrutiny during dovetailing (see below). In the 2018/2019 COL, the selected specification included 14 indicators from 4 datasets ( Table 3). The top-down approach results in a top-down shortage list. The process of identifying data sources, calculating shortage indicators, and selecting a final combination of indicators results in an initial top-down list of shortage occupations (Figure 3). In the 2018/2019 COL, 53 such occupations were identified. The list provides evidence that the occupations are sought-after, but additional evidence is also collected through surveys of and consultations with labor market stakeholders in the bottom-up approach. FIGURE 3. Summarizing the top-down approach What data What data sources have sources What initial Identify data have usable indicators information sources about information can be shortages? about constructed? occupations? Initial indicators Calculate shortage Which indicators provide unique shortage information? indicators Intermediate indicators Select final Which combinations of indicators provide specification unique shortage information? Final indicators Final top-down shortage list Source: Authors Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 33 Appendix Bottom-up approach The bottom-up approach builds an evidence base that occupations are sought-after by directly asking stakeholders for information. Like the top-down approach, the bottom-up approach draws on several sources of data. They include a broad survey of employers, in-depth consultations with employers and industry associations, and sector-based “environmental” scans. • Call for Evidence Survey. The primary purpose of the Call for Evidence (CfE) survey is to allow employers to nominate occupations for inclusion on the COL. Employers are provided with a brief description of the COL’s skilled, sought-after, and strategic criteria and are asked to nominate occupations meeting those criteria. The CfE then asks the same series of questions for each occupation. The questions solicit information about each nominated occupation’s key responsibilities, number of employees, experience level, strategies to fill vacancies, average time taken to fill vacancies, and reasons why the occupation is hard to fill. Employers to fill out the survey are drawn from several government databases with company contact information that is available publicly or compiled by the CSC. A survey research firm assists the CSC in identifying respondents, building the questionnaire, monitoring responses, and cleaning the final data. The CfE is not meant to be representative, but to gather information for the CSC. • Stakeholder consultations. Like the CfE, the primary goal of the stakeholder consultations is to generate a list of occupations that participants nominate for inclusion on the COL. During the consultations, the CSC team first seeks to compile a complete list of occupations that could potentially be nominated. The team subsequently gathers information from participants to support the contention that the occupation meets the skilled, sought-after, and strategic criteria. This discussion is only semi-structured; it aims to allow for discussion of different types of information in greater breadth than is possible with the CfE. That discussion includes the business impact of shortages. It also covers whether a shortage is limited to one job type or is present in an entire occupation. The consultations should make participants feel that their input is valued, inform them of key milestones in producing the COL, and motivate them to fill out the CfE if they have not yet done so. The consultations also increase knowledge within the CSC about economic sectors, their needs, and their trends. • Sector-based environmental scans. Environmental scans have been undertaken by different Malaysian agencies in several sectors: machinery equipment and advanced engineering; food and beverage; medical devices; pharmaceutical manufacturing; chemical and petrochemical; and electrical and electronics. The scans profile current industry trends, including the current workforce and occupations in the sector; project how workforce supply and demand will evolve; and identify potential skill gaps. The scans also seek to identify in-demand occupations. The CSC considers the information when assessing occupations on the list of nominated occupations. The CfE and the stakeholder consultations result in a list of occupations that employers and industry associations nominate for the COL. Evidence from the environmental scans is considered when compiling the list of nominated occupations and during the next stage of the process, dovetailing (see below). Additionally, both the CfE and the consultations generate information about recent hiring experiences, companies’ reported reasons for hiring difficulties, desired educational profiles, strategies used to address hiring difficulties, and the impact those hiring difficulties have on operations. Consultations also provide employers with an opportunity to contribute other information to support their nominations. To make the bottom-up approach compatible with the top-down approach, occupations are classified using the MASCO occupational classification scheme. 34 Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List Appendix Dovetailing Dovetailing is the process of consolidating evidence from the top-down and bottom-up approaches to determine whether the evidence shows that an occupation is sought-after and should be included on the COL. While the dovetailing process involves a set of rules to guide decisions, the process does not involve mechanically applying a strict set of weights for different indicators or fixed thresholds for determining if an occupation should be included. Rather, the coherence of top-down and bottom-up evidence and the plausibility of the case that emerges from this evidence are the primary drivers of the decision for inclusion. In the dovetailing process, the CSC is seeking credible evidence for an occupation’s inclusion on the COL. The absence of such evidence results in an occupation not being included on the list. Potential uses of the COL include targeting government resources or providing preferential treatment for an occupation on the list. The usefulness of the list depends on its highlighting only occupations where a strong case for such treatment exists. Accordingly, the COL, like similar lists in other countries, seeks to be relatively selective. The default status of an occupation is that it is not included on the COL. The CSC seeks sufficient evidence to override that default, meaning that the absence of sufficient evidence has the same result as credible evidence that an occupation should not be included. Thus, the process is set up in a way that limits the occurrence of false positives. There are two rounds of dovetailing analysis and a process of validation. In the first round of dovetailing, occupations are prioritized for inclusion on the COL by the strength of the evidence that employers are seeking them. Most of the time is spent evaluating occupations with weak evidence for inclusion on the COL. The decision about each occupation is documented in reports that capture both the top-down and bottom-up evidence, as well as the team’s rationale for recommending an occupation’s inclusion or exclusion. Marginal cases are marked for additional data gathering during a validation process. The preliminary COL resulting from the first round of dovetailing is validated with employers, government agencies, industry associations, and other stakeholders. A second round of dovetailing is conducted for those occupations for which additional information is received during the validation process. The process is the same as in the first round of dovetailing, but also makes use of the information gathered in the validation process. The strategic importance of occupations that meet the criteria for being skilled and sought-after is assessed during the second round of dovetailing. This assessment is based on a variety of factors, including which industries report shortages in certain occupations, the degree to which employers are actively seeking to fill shortages, the potential impact of shortages on the health of businesses and industries, and whether occupations can be automated. The second dovetailing stage results in the final COL. The second dovetailing stage is the final stage of identifying critical occupations. At this stage, a final list of critical occupations has been identified. The use of both quantitative and qualitative sources of information in creating the COL allows for the creation of occupation reports, which are robust justifications for an occupation’s inclusion on the list. Monitoring Occupational Shortages: Lessons from Malaysia’s Critical Occupations List 35 CONNECT WITH US wbg.org/Malaysia @WorldBankMalaysia @WB_AsiaPacific http://bit.ly/WB_blogsMY