Document of The World Bank FOR OFFICIAL USE ONLY Report No: 50768-KZ PROJECT APPRAISAL DOCUMENT ON A PROPOSED LOAN IN THE AMOUNT OF US$29.23 MILLION TO THE REPUBLIC OF KAZAKHSTAN FOR A TECHNICAL & VOCATIONAL EDUCATION MODERNIZATION PROJECT (TVEMP) June 1, 2010 Human Development Sector Unit Central Asia Country Unit Europe and Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective April 9, 2010) Currency Unit = Kazakhstan Tenge (KZT) KZT146.78 = US$1 US$1 = SDR 0.66 FISCAL YEAR January 1 ­ December 31 ABBREVIATIONS AND ACRONYMS ATAMEKEN Union of Employers Association CBT Competency-based Training CFC Committee of Financial Control EBT Enterprise-based Training EMF Environmental Management Framework EMP Environmental Management Plan ETF European Training Fund EU European Union FD Finance Department FM Financial management GDP Gross domestic product GOK Government of Kazakhstan GPN General Procurement Notice GTZ German Technical Cooperation IBRD International Bank for Reconstruction and Development ICB International Competitive Bidding IDP Institutional Development Plan IFAC International Federation of Accountants IFRs Interim un-audited financial reports IFRS International Financial Reporting Standards IPSAS International Public Sector Accounting Standards ISA International Standards on Auditing KZT Kazakhstan Tenge (currency) LCMS Learning Content Management System M&E Monitoring and evaluation MES Ministry of Education and Science MLSP Ministry of Labor and Social Protection of the Population MOF Ministry of Finance MTR Midterm Review NOS National Occupational Standards NPV Net present value NQF National Qualifications Framework NQS National Qualifications System NSIID National Strategy for Industrial Innovation Development PAP Project Advisory Panel PFM Public Financial Management PFSs Project Financial Statements PMU Project Management Unit POM Project Operational Manual PP Procurement Plan PSC Project Steering Committee 2 ROSC Reports on the Observance of Standards and Codes SAI Internal Control Department of the Ministry of Finance and the Accounts Committee SIL Specific Investment Loan SOE Statement of Expenditure SPN Specific Procurement Notices TA Technical assistance TORs Terms of reference TVE Technical and vocational education The Center National Scientific Methodological Center for Vocational Education Development and Qualifications Award TVEM Technical and Vocational Education Modernization TVEMP Technical & Vocational Education Modernization Project TVET Technical and Vocational Education and Training Vice President: Philippe H. Le Houerou Country Director: Motoo Konishi Country Manager: Sebnem Akkaya Sector Manager: Alberto Rodriguez (Acting) Task Team Leader: Dingyong Hou 3 KAZAKHSTAN TECHNICAL & VOCATIONAL EDUCATION MODERNIZATION PROJECT CONTENTS Page I. STRATEGIC CONTEXT AND RATIONALE ................................................................. 9 A. Country and sector issues.................................................................................................... 9 B. Rationale for Bank involvement ....................................................................................... 14 II. PROJECT DESCRIPTION ............................................................................................... 15 A. Lending instrument ........................................................................................................... 15 B. Project Development Objective and key indicators .......................................................... 15 C. Project components ........................................................................................................... 15 D. Lessons learned and reflected in the project design .......................................................... 18 E. Alternatives considered and reasons for rejection ............................................................ 19 III. IMPLEMENTATION .................................................................................................... 20 A. Partnership arrangements .................................................................................................. 20 B. Institutional and implementation arrangements ................................................................ 20 C. Monitoring and evaluation (M&E) of outcomes and results ............................................ 21 D. Sustainability..................................................................................................................... 22 E. Critical risks and possible controversial aspects ............................................................... 22 F. Loan conditions and covenants ......................................................................................... 24 IV. APPRAISAL SUMMARY ............................................................................................. 25 A. Economic and financial analyses ...................................................................................... 25 B. Technical ........................................................................................................................... 26 C. Fiduciary ........................................................................................................................... 27 D. Social................................................................................................................................. 29 E. Environment ...................................................................................................................... 29 F. Safeguard policies ............................................................................................................. 30 G. Policy exceptions and readiness........................................................................................ 30 Annex 1: Sector Background ..................................................................................................... 31 Annex 2: Major Related Projects Financed by the Bank and/or other Agencies ................. 36 Annex 3: Results Framework and Monitoring ........................................................................ 37 4 Annex 4: Detailed Project Description ...................................................................................... 49 Annex 5: Project Costs ............................................................................................................... 61 Annex 6: Implementation Arrangements ................................................................................. 63 Annex 7: Financial Management and Disbursement Arrangements ..................................... 65 Annex 8: Procurement Arrangements ...................................................................................... 74 Annex 9: Economic and Financial Analysis ............................................................................. 87 Annex 10: Safeguard Policy Issues ............................................................................................ 94 Annex 11: Project Preparation and Supervision ..................................................................... 96 Annex 12: Documents in the Project File ................................................................................. 97 Annex 13: Criteria and Procedures for Approval of IDP Subprojects ................................. 98 Annex 14: Statement of Loans and Credits ............................................................................ 103 Annex 15: Country at a Glance ............................................................................................... 104 Annex 16: Map: IBRD 33425 ................................................................................................... 106 5 KAZAKHSTAN TECHNICAL & VOCATIONAL EDUCATION MODERNIZATION (TVEM) PROJECT APPRAISAL DOCUMENT EUROPE AND CENTRAL ASIA ECSHD Date: June 1, 2010 Team Leader: Dingyong Hou Country Director: Motoo Konishi Sectors: Vocational training (100%) Sector Manager/Director: Alberto Rodriguez Themes: Education for the knowledge economy (Acting)/ Mamta Murthi (Acting) (67%); Small and medium enterprise support (33%) Project ID: P102177 Environmental category: B Lending Instrument: Specific Investment Loan Joint IFC: Joint Level: Project Financing Data [X] Loan [ ] Credit [ ] Grant [ ] Guarantee [ ] Other: For Loans/Credits/Others: Total Bank financing (US$m): US$29.23m Proposed terms: IBRD Flexible loan in US dollars with an interest rate equal to 6 months LIBOR, plus variable spread with a 15 year maturity, including 4 year grace period, commitment linked and with level repayment pattern. Financing Plan (US$m) Source Local Foreign Total International Bank for Reconstruction and 10.23 19 29.23 Development Republic of Kazakhstan 4 4 Total: 14.23 19 33.23 Borrower: Republic of Kazakhstan Responsible Agency: Ministry of Education and Science of the Republic of Kazakhstan Dom Ministerstv, entrance 11 Kazakhstan Tel: (7-7172) 741-573 Fax: (7-7172) 742-416 Estimated Disbursements (Bank FY/US$m) FY 2010 2011 2012 2013 Annual 4.3 10.6 14.3 Cumulative 0 4.3 14.9 29.2 Project implementation period: Start: August 15, 2010; End: December 31, 2013 Expected effectiveness date: October 15, 2010 Expected closing date: December 31, 2013 Does the project depart from the CAS in content or other significant respects? Ref. [ ]Yes [X] No PAD I.C. Does the project require any exceptions from Bank policies? Ref. PAD IV.G. [ ]Yes [X] No Have these been approved by Bank management? [ ]Yes [ ] No 6 Is approval for any policy exception sought from the Board? [ ]Yes [ X ] No Does the project include any critical risks rated substantial or high? [ X]Yes [ ] No Ref. PAD III.E. Does the project meet the Regional criteria for readiness for implementation? Ref. [X]Yes [ ] No PAD IV.G. Project Development Objective Ref. PAD II.C., Technical Annex 3 The Project Development Objective is to raise the relevance, quality, and efficiency of technical and vocational education (TVE) through an improved policy framework and institutional capacity. Project description [one-sentence summary of each component] Ref. PAD II.D., Technical Annex 4 The Project consists of three components: Component 1: Develop Standards and Quality Assurance for TVE. This component will support: (a) the development of a national qualifications system and a national qualifications framework; (b) the development of occupational standards, and realignment of qualifications assessment; and (c) the introduction of institutional accreditation in establishing the foundation for a national qualifications system. Component 2: Strengthen Governance, Management, and Financing for TVE. This component will support activities to increase employer involvement in advising TVE development through: (a) the modernization of the management and governance policies, procedures and institutional structures of TVE; (b) the introduction of formula-based financing to TVE institutions in selected regions; (c) design and implementation of an effective mechanism for monitoring and evaluation of the project; and (d) support for Project coordination, implementation and management. Component 3: Strengthen Skills Delivery Capacity of TVE Institutions. This component will support: (a) development of educational standards and core curriculum as the basis for the institutions to design new programs, (b) preparation of teacher-training programs for design and delivery of a competency-based and modularized curriculum (in select regional teacher-training centers); and (c) the development and implementation of Institutional Development Plans (IDPs) to enhance skills delivery at TVE institutions (lyceums and colleges), including development and delivery of competency-based training (CBT) programs in line with new occupational and educational standards and core curriculum. Which safeguard policies are triggered, if any? Ref. PAD IV.F., Technical Annex 10 Environmental safeguard policies are triggered B (low). Significant, nonstandard conditions, if any, for: Ref. PAD III.F. Loan/credit effectiveness: (a) The PMU has been established by the Borrower, through the Ministry of Education and Science (MES), with a composition, resources and terms of reference satisfactory to the Bank. (b) The Advisory Panel has been established by the Borrower with a composition, resources and terms of reference satisfactory to the Bank. (c) The Operational Manual1, satisfactory to the Bank, has been adopted by the Borrower. Covenants applicable to project implementation: The B (a) The Borrower, through the PMU, shall carry out the Project in accordance with the requirements, criteria, organizational arrangements and operational procedures set forth in the Operational Manual and the EMF, and shall not assign, amend, abrogate or waive any provisions of the Operational Manual or the EMF without prior approval of the Bank. 1 The POM was substantially complete by negotiation except for some minor changes relating to financial management. As a result, it was decided at negotiation that adoption of the final POM (including a complete section on financial management controls) would be a condition of effectiveness. 7 (b) At all times during the implementation of the Project, the Borrower shall maintain the Project Steering Committee, the PMU and the Project Advisory Panel with a composition, resources, and terms of reference satisfactory to the Bank. (c) Not later than thirty (30) days after the Effectiveness Deadline, the Borrower, through the MES, shall upgrade the accounting software of the PMU in a manner satisfactory to the Bank. 8 I. STRATEGIC CONTEXT AND RATIONALE A. Country and sector issues Context 1. From the late 1990s until mid-2008, Kazakhstan experienced strong economic growth fueled by rising oil production and prices. Real gross domestic product (GDP) grew by an average of nearly 10 percent per year between 2000 and 2008. Real income increased at the even faster rate of 16 percent per year over the same period, and poverty declined from 32 percent in 2000 to 13 percent in 2008. However, Kazakhstan has been severely impacted by the global economic crisis, with growth projected by the International Monetary Fund at minus 2.0 percent in 2009, with a slow recovery to a positive 1.5 percent projected in 2010. Registered unemployment, still relatively low at 7.1 percent in February 2009, is expected to increase throughout the downturn. 2. Recovery from the fallout of the global economic crisis dominates the near term, but medium-to-long-term prospects for economic growth are promising. Growth over the medium term is projected to be in the range of 3 to 8 percent of GDP yearly. These projections are driven largely by the anticipated recovery of commodity prices and higher oil production. However, reliance on commodity exports, mainly oil and metals, makes the country vulnerable to fluctuations in world prices. Kazakhstan has therefore adopted economic diversification and increased competitiveness as priority goals to ensure rapid and sustainable growth. In 2003, Kazakhstan adopted its National Strategy for Industrial Innovation Development (NSIID) for 2003­15, which lays the legislative and institutional foundations for economic diversification. 3. Kazakhstan is committed to becoming one the world's 50 most competitive economies as set forth in its national competitiveness strategy. However, the business environment has problems resulting in a lack of competitiveness. Kazakhstan's ranking in the Global Competitiveness Index has worsened from 50th in 2006 to 61st in 2007 and to 66th in 2008. Doing Business 2009 ranks Kazakhstan 70th of 181 economies, but this ranking does not capture important dimensions of the business climate, namely the practice of business regulation (compared to legislated principles) and severe constraints on competition. 4. Slow productivity growth relative to labor costs outside the resource sector poses a fundamental obstacle to achieving diversified growth in Kazakhstan, and sluggishness in labor market adjustments hinders productivity growth. The Government recognizes that efforts must be made to increase worker productivity. This requires, among other things, better knowledge and skills in the workforce as produced through quality technical and vocational education (TVE). 5. The present TVE system, engineered to support a command economy, suffers from a lack of linkages to the market economy. It does not produce sufficient graduates with the competencies required by the market. A shortage of education and skills of available labor impedes business development. According to a 2008 World Bank labor market survey of 500 firms, 64 percent of employers stated that an insufficient level of education and lack of skills are among the principal hindrances for business development (see Figure 1). The shortage of skilled labor is reported to affect the economy widely (see Figure 2). Forty percent of firms in construction and food production, and 46 percent of firms in oil and gas extraction, reported skills shortages to be a very significant problem, resulting in chronic vacancies, loss of productivity, and the need to import expensive foreign technicians. 9 6. Requirements for skills and competences are continuously increasing: 40 percent of surveyed companies reported increased requirements for skills of workers for the last two years as a result of a demand for higher-quality goods and services, more complicated technological processes, and increased competitiveness. A third of firms expect recruited technicians to have secondary TVE or college education. Fifty-seven percent of firms expect recruited skilled workers to have secondary (lyceum) TVE education and 36 percent expect postsecondary education. Figure 1: Impediments to Business in Kazakhstan % of firms reporting a specific factor to be a problem Macroeconomic instability 73.8% Excessively high tax rates 70.2% Lack of skills and education of available workers 63.6% High cost of financing 58.6% Inadequate tax administration 55.8% Poor roads' quality 55.2% Contract violations by customers and suppliers 54.0% Difficulties in gaining business licenses/permits 48.6% Difficulty in accessing financing 48.4% Inadequate labor laws/regulations 43.6% Government intervention 41.8% Corruption 41.6% Poor telecommunications' quality 38.2% Organized crime/mafia 21.2% 0% 10% 20% 30% 40% 50% 60% 70% 80% Source: Labor Market in the Republic of Kazakhstan--Project Results, BISAM Central Asia, Almaty, March 2008. Figure 2: Percentage of Firms Reporting Severe Shortages of Skilled Workers % of firms reporting a skill shortage to be a very significant problem (by economic sector) oil/gas/extraction 46.2% food production 40.0% other industry 39.4% construction 39.3% pulp/paper/w oodw ork 35.7% pow er/electricity 33.3% finance/banking 29.8% trade/services 27.6% real estate 27.1% transport/com m unications 25.5% hotels/restaurants 8.3% 0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50% Source: Labor Market in the Republic of Kazakhstan- Project Results, BISAM Central Asia, Almaty, March 2008. 10 7. After almost a decade of sustained economic growth, the country requires a new generation of skilled technical and professional citizens to ensure continued success in an increasingly global environment. At present, there is a growing need for a more skilled workforce in primary sectors such as the mineral and oil and gas industries; the building and manufacturing industries such as engineering, petrochemical production, and construction materials; and in service industries such as tourism, social work, and consumer services. The dominant problem in Kazakhstan at this time is structural unemployment. That is, there are many vacancies,2 but these vacancies cannot be filled by the currently unemployed because they do not have the required knowledge and skills. Two-thirds of the 500 firms surveyed had vacancies. In greatest demand were skilled workers (44 percent of firms) and technicians (40 percent) compared with relatively low demand for accountants (9 percent) and engineers (5 percent).3 8. In addition, as firms invest in new technologies to enhance their productivity and capacity to innovate, they are increasingly looking to hire employees who have sound transferable foundational skills such as a solid grounding in language, mathematics, and science; a capacity for using technology to communicate; the ability to acquire, manage, and use information; skill in managing resources; a real aptitude for working in teams; and simple workplace discipline. They are looking for employees who are equipped with a broader set of occupational workplace skills and competencies and who can bring to the workplace high productivity, professional flexibility, and the capacity for on-the-job learning and for the acquisition of new skills. Investment in human capital 9. The Government of Kazakhstan recognizes the importance of investment in human capital. Human resource development is one of five pillars in the draft National Strategic Plan 2020: Human capital is a basic driver of a country's innovation and increased productivity. These investments have longer term payoffs, but they are critical determinants of the country's stable and sustainable development.4 The National Strategy for Industrial Innovation Development (NSIID) includes as one of its objectives a tripling of labor productivity between 2000 and 2015. The Government recognizes that modernizing the education system, including the TVE system will be essential to reaching that objective. 10. Public spending on human capital, accordingly, has been growing, and social indicators improving, but more needs to be done. Kazakhstan's Human Development Index, compiled by the United Nations Development Programme (UNDP), improved from 80th in 2005 to 73rd in 2008, but is still inferior to many countries at comparable levels of income per capita. Kazakhstan's public expenditure on education as a percentage of GDP increased from 3 percent in 2002 to 4 percent in 2009, still well below that of economies at the same income level. The TVE System 11. In Kazakhstan, basic education through grade 9 is universal. Repetition rates are low and virtually all youth complete basic education. Kazakhstan participates in the Trends in International Mathematics and Science Study (TIMSS) and has achieved impressive results at grade 4. Adult literacy rates are almost universal for both men and women. Following 2 The Ministry of Labor estimates the number of vacancies for technicians and skilled workers nationwide at 40,000. 3 Labor Market in the Republic of Kazakhstan--Project Results, BISAM Central Asia, Almaty, March 2008. 4 Strategic Plan of Development of the Republic of Kazakhstan up to 2020, draft August 28, 2009, p. 8. 11 nine years of basic education, students have the choice of entering either general education or technical and vocational education. About 20 percent of students in grades 9­11 study in TVE institutions. 12. Technical and vocational education is provided by two types of institutions--vocational schools (professional lyceums) and colleges. Vocational schools provide initial vocational education aimed at preparing students to enter the labor market as skilled workers in technical and service fields. Vocational school programs last two to three years. Colleges provide more theoretical work in science and mathematics and prepare students as technicians, masters, and supervisors. College programs last from three to four years. Upon graduation from secondary education, after grade 11, students may enter higher education or TVE at a more advanced level or may enter the labor market. 13. A significant feature of TVE is the presence of private institutions and private financing of student fees in public institutions. At present, there are 324 vocational schools of which 26 are private; and 542 technical colleges, of which the majority--334--are private. About 111,200 students are enrolled in vocational schools taught by 10,500 teachers (a ratio of 10.6 students per teacher). Vocational schools produce about 39,500 graduates yearly among a wide range of specializations. Colleges enroll almost 500,000 students taught by 28,600 teachers (an average of 19.5 students per teacher) and produce 119,000 graduates per year. Just over half the students at the college level are in private institutions. Given limitations of public financing and strong student demand, public colleges now admit fee-paying students, who have grown to about 60 percent of total enrollment. Fee-paying students pay about half the actual cost per student at the institution. 14. As indicated, businesses claim the TVE system does not produce sufficient graduates with the required competencies. Why do shortages and vacancies for technicians and skilled workers persist in the economy? The simple answer is that the TVE system produces neither the quantity nor quality of skills needed in the market economy. Rigid institutional structures and inefficient financing mechanisms make for an inflexible system that does not provide incentives for effective and efficient use of resources. The TVE system in Kazakhstan faces daunting challenges in terms of its relevance, management, quality, and internal efficiency. These issues are well understood by the Government and have been built into the formulation of the State Program. The proposed project will support policy studies on a selected set of issues to deepen the analysis to inform future policy, and on specific interventions targeting the issues that have been fully studied. 15. Relevance. Issues of economic relevance include: (a) lack of or insufficient linkages with employers and the labor market; (b) a predominantly supply-driven orientation where the principal incentive is to fill existing programs and use existing teachers and facilities rather than adapt to changes in the market; (c) obsolete program content and incongruence with market requirements; (d) educational standards and content not based on occupational analysis; and (e) overspecialization of the curriculum leading to narrow skills and lack of sufficient foundation skills for learning on the job and becoming lifelong learners as needed in a market economy. 16. Governance and management effectiveness. TVE is government dominated to the exclusion of important stakeholders, especially employers. Employers have inadequate room for participation, even in the new National TVE Council. The division of responsibilities and functions for TVE needs to be clearly divided between the national and regional levels. At the institutional level, TVE management generally lacks the authority to operate the institutions flexibly in response to changes in markets. In particular, budgetary controls limit 12 the flexibility of redistributing available resources according to institutional needs. In addition, the TVE system lacks a culture of monitoring and evaluation to feed policy development and guide implementation. Data on financing per student and per program are weak or missing as a basis for analysis of cost-effectiveness. 17. Quality. Within the framework of the Youth Voices program for Kazakhstan, a group of 25 youth surveyed a random sample of 500 young people in five oblasts on their views on the TVE system in the country. TVE trainees focused on the weak quality of training received, including the low quality of teaching staff, resulting in superficial study in many subjects, absence and/or low quality of textbooks, and the lack of up-to-date equipment. They concluded that their studies prepared them for entering higher-educational institutions but were not good enough for entering employment (see Annex 1 for details). 18. Lack of investment in TVE for more than 15 years has resulted in obsolete and worn-out facilities and equipment. Regions and cities spend only half the recurrent expenditures per TVE student compared with national TVE institutions of the same type. Additional challenges to quality include the high turnover among instructors of practical subjects, in part because of aging of the teacher force. Inadequate inflow of qualified young technical and vocational instructors contributes to teaching vacancies owing to perceived low salaries and benefits for teachers in the TVE system. Moreover, teachers and instructors lack opportunities for in-service training or the chance to learn about good practices elsewhere, including more effective teaching methods. Institutional accreditation is input based and serves only to reissue licenses. The process provides little evidence on institutional performance and quality of outputs. 19. Finance and Internal Efficiency. Resources allocated to TVE in Kazakhstan are considerably lower as a proportion of GDP than in other middle-income, emerging-market economies: 0.2 percent of GDP goes to TVE in Kazakhstan compared with 0.6 percent in Poland and Slovakia in 2004 and 1.0 percent in the Czech Republic. Compounding the low funding level, few incentives exist for efficient use of resources and a better mix of public and private partnerships. In addition, the allocation of resources to public TVE institutions is driven by historical budgets rather than institutional needs and system priorities. As a result, per capita spending by program varies widely. In general, student-teacher ratios are low in TVE institutions, reflecting overspecialization in the curriculum, with resulting low average teaching loads in some practical subjects. 20. In most market economies the business sector is a major source of skills development, both within firms and through assistance to formal TVE. That is not the case in Kazakhstan. Business support for TVE is constrained by the tax structure. Financial support by businesses for TVE is regarded by the Government as social expenditure (education, health, community assistance) and limited to 3 percent of taxable income. Any donation by businesses of used equipment and materials to training institutions is subject to the value- added tax. No tax benefits accrue from accepting interns from TVE institutions for practical training. 21. TVE Reform Programs. The Government is aware of the need to make extensive reforms in the TVE system. The Education Law of 2007 provides the framework for restructuring TVE, including provision for better integration of secondary vocational and general education. The Government plans to adopt a system of 10 years of basic education and two years of specialized education. Vocational and general secondary education institutions will share the same core courses of general subjects, thereby providing vocational students with more transferable foundation skills. The law also provides for the 13 development of a separate postsecondary vocational education qualification with more advanced levels of technical skills. 22. The State Program for Technical and Vocational Education and Training (TVET) Development, developed with World Bank assistance, was approved by Presidential Decree 626 of July 1, 2008, with an initial phase to be carried out from 2008 to 2012. The goal of the State Program is to modernize the TVE system so it will provide the economy and employers with qualified specialists who possess competitive skills and sufficient capacity for lifelong learning. This requires a restructuring of the TVE network and a modernization of its infrastructure and technical, material, and human resource base. The objectives of the State Program are to: (a) ensure accessibility and attractiveness of TVE; (b) increase the quality training and competitiveness of the workforce produced by the TVE system; and (c) improve efficiency and accountability of the TVE management and funding systems. The State Program calls for reforms in the following areas: (a) in standards, a national qualification framework will be established along with linkage of educational with occupational standards, independent assessment, and accreditation; (b) in governance of TVE, national, regional, and institutional councils will be established with employer and stakeholder representation; (c) in finance, formula-based financing will be introduced; and (d) incentives for increased public-private partnerships in TVE. 23. Accelerated Industrial Development and Innovation Program. In 2010, the Government introduced changes to the ongoing development programs, including the State Program, in an effort to streamline these programs and refocus them on the top priorities established by the Government for 2010­14. One of key objectives of this development is to establish a competitive and productive workforce for 10 priority sectors. It calls for alignment of the supply of skills to the demand of industries and an education and training system that supports lifelong learning. It revised the key elements of the State Program to: (a) upgrade the content and structure of the education program to labor market demand and international standards; (b) improve the TVE structure and governance; (c) upgrade TVE infrastructure and facilities; and (d) upgrade the skills certification and qualification system and align them to international standards. The Government intends to achieve these declared goals by its own resources and external financing with technical assistance from international development partners. 24. The TVEM project supports pilot testing the initial steps in the implementation of this program by developing standards and quality assurance of TVE; strengthening governance, management, and financing for TVE; and strengthening the skills delivery capacity of TVE institutions. The Government has indicated its intention to scale-up the project with a follow-on loan for systemwide implementation through additional financing upon successful completion of the first phase. B. Rationale for Bank involvement 25. In 2007, at the request of the Government of Kazakhstan, the World Bank helped develop a State Program for improving the TVET system to be financed with the Government's national budget. In support of the implementation of the State Program, the Government requested a project to be prepared with an International Bank for Reconstruction and Development (IBRD) loan that provides financing and technical assistance in selected areas that best complement the State Program. In 2010, the Government reaffirmed its request for an IBRD loan to finance the first phase of implementation of the program against the renewed priority of establishing a competitive and 14 productive workforce under the accelerated industrial development and innovation plan. It is within the context of the Government's planned investment in modernizing the TVET system that the involvement of the Bank adds value. 26. The World Bank has a long record of investing in TVET reforms, and Bank involvement would fill an important gap by introducing international experience and innovation. The Bank's engagement under the Project would support innovations in helping develop skill standards for key industries, realign qualifications and assessment to new standards, improve governance and financing, upgrade curriculum and training methodology, and foster closer partnership with businesses and employers in quality assurance and service delivery. The Bank's comparative advantage includes its international experience, which would also contribute to the design of an impact evaluation for extracting lessons to be shared and replicated nationally, strengthening reforms and policy development for TVET. II. PROJECT DESCRIPTION A. Lending instrument 27. A Specific Investment Loan (SIL) has been selected as the appropriate instrument for this project. The use of a SIL provides a flexible instrument that can ensure the technical, financial, economic, and institutional viability of these specific investments. B. Project Development Objective and key indicators 28. The Project Development Objective is to raise the relevance, quality, and efficiency of technical and vocational education in Kazakhstan through an improved policy framework and institutional capacity. The attainment of the objective will be measured by the following key outcome indicators: Employer satisfaction with industry-led competency standards as a basis for development of curriculum and programs; and Share of the existing programs in project-supported schools revised in accordance with the newly introduced competency standards. C. Project components The Project consists of three components: 29. Component 1. Develop Standards and Quality Assurance for TVE (US$2.4 million). This component would finance goods, consultants' services and training to introduce a national qualifications system and national qualifications framework, occupational standards, qualification assessment, and institutional accreditation for establishing a foundation for a national qualification system that enables flexible pathways for learning and employment. This includes: (a) Development of a National Qualifications System (NQS) and a National Qualifications Framework (NQF). This sub-component supports the initial steps toward development of an NQS. (a) It would finance a study tour and technical assistance to develop a concept for an NQS based on international best practice for discussion and agreement among stakeholders. The NQS would aim to enhance opportunities for lifelong learning by enabling learners to move from one level to another and to receive credit for all types of learning. (b) Given that it takes 15 significant time and effort to develop and introduce all elements of an NQS, the Project would take an initial step by developing a National Qualifications Framework (NQF) setting out levels of occupational and educational qualifications that are compatible with the European Qualifications Framework. The NQF would allow transfer of credits and facilitate movement of labor among Kazakhstan, Europe, and other regions. The NQF would be developed in close collaboration with the private sector. (b) Development of Occupational Standards and Assessment of Qualifications. This sub-component would contribute to the NQS by developing occupational standards in the priority sectors and upgrading the assessment of qualifications. The occupational standards refer to broad basic skills and concrete occupational skills. (i) The sub-component would finance procurement and adaptation of occupational standards that are already available on the international market and development of standards that do not exist or do not correspond to occupations in Kazakhstan. Development and procurement of the standards will give priority to the oil and gas, metallurgy, machinery, agro-processing, construction, communications and information technology, geology, mining, power engineering and transportation sectors.5 This sub-component will also provide technical assistance and guidance to the developers of occupational standards. New occupational standards will form a basis for the development of new educational standards and core curriculums (to be developed under Component 3). (ii) The sub-component will also finance technical assistance to upgrade assessment tools for qualifications. The tools would align to the employers' requirements reflected in the occupational standards. The tools would be revised in a sequence that corresponds to the order and pace of occupational standards preparation. (c) Introduction of Institutional Accreditation. This sub-component would finance technical assistance to develop and implement methods and procedures for independent, voluntary institutional accreditation. Accreditation would focus on outputs and outcomes of education and training delivered by the TVE institutions. 30. Component 2. Strengthen Governance, Management, and Financing for TVE (US$1.6 million). This component seeks to establish a basis for increasing the efficiency and effectiveness of the TVE sector by studying and providing policy options on governance, management, and financing of Technical and Vocational Education and Training (TVET). (a) Modernization of TVE Structure, Management, and Governance, and Management. This sub-component would (i) support technical assistance for the new national, regional, sectoral, and institutional councils in terms of advice to the members; and (ii) finance an in-depth review of the current TVE structure, its governance and management including teacher compensation policy and practice, and elaborate policy options based on international experience. Consultation with stakeholders would follow on the advantages and disadvantages of the alternatives leading to policy decisions and planning for implementation. Finally, this sub- component would finance international and local expertise to carry out the analyses and a study tour to countries that have reformed TVE system governance and management. 5 The Government of Kazakhstan's state industrialization program defined these as priorities. 16 (b) Introduction of Formula-based Financing. This sub-component seeks to improve the efficiency of the use of TVE funds by designing, testing, evaluating, and revising per capita approaches for financing TVE institutions. This will move the TVE system away from one that merely allocates resources based on past costs to one that allocates resources toward students pursuing specialties required by the labor market. It would finance a study tour, international and domestic expertise, data collection, staff training on new procedures, the costs of a trial simulation in one or two regions (oblasts), evaluation of results, and adaptation of the model for nationwide implementation. (c) Design and implementation of a Monitoring and Evaluation (M&E) System. This sub-component would finance consultant services to design and implement an effective mechanism for M&E for the Project. The Project would support regular monitoring of project implementation activities and impact studies. The M&E will focus primarily on project-supported interventions, and M&E activities would inform and enhance the system capacity to monitor and evaluate investments in TVE. (d) Support for Project coordination, implementation and management. This sub-component would finance goods, services, and operating costs to operate and maintain a Project Management Unit (PMU) to administer the project, and a Project Advisory Panel to provide technical support to the PMU in implementation (see section III.B). 31. Component 3. Strengthen Skills Delivery Capacity of TVE Institutions (US$29.2 million). This component is designed to begin a process of gradual consolidation and integration of the programs currently offered in the professional lyceums and colleges toward an integrated, broad, competency-based curriculum. The component would finance: (a) revision and development of educational standards and core curriculum (on the national level) as the basis for the institutions to design new programs, (b) preparation of teacher- training programs for developing and delivering a competency-based and modularized curriculum (in six regional teacher-training centers), and (c) the development and implementation of Institutional Development Plans (IDPs) to enhance skills delivery at TVE institutions (lyceums and colleges), including development and delivery of competency- based training (CBT) programs in line with new occupational and educational standards and core curriculum. IDPs would be selected for financing through competitive bidding. (a) Support for development of education standards, the core curriculum, and the program. This component would finance technical assistance in developing educational standards, the core curriculum, and the program to guide the institutions in designing and delivering new programs for priority industries and occupations using the occupational standards developed under Component 1. (b) Preparation of Teacher-training programs in developing and delivery of a competency-based and modularized curriculum. This component would finance Sub-grants (for goods and consultants' services) in support of development of in- service teacher-training programs for developing and introducing a competency-based and modularized curriculum in select regional teacher-training centers. (c) Implementation of Institutional Development Plans. This component would finance Sub-grants (for goods, works, consultants' services and training) 17 through subprojects to be awarded on a competitive basis to public and private provider institutions for delivering the programs specified in the IDPs by select institutions. See Annex 4 for a detailed description of the Components. D. Lessons learned and reflected in the project design 32. Project design has benefited from a variety of lessons summarized below. 33. Critical involvement of the private sector. TVE systems need to respond to the demands of the labor market, and global experience demonstrates that this requires the active engagement of the private sector in the development of the curriculum, definition of professional standards, governance, and financing. Under the project, the private sector would be engaged directly in all these activities. Through ATAMEKEN (the Union of Employers Association), the private sector would have an active oversight role in the State Program of TVE Development of Technical and Vocational Education Modernization (TVEM) and the Project. Component 3 ensures that only TVE programs with a clear arrangement with the private sector to participate would be eligible for support. 34. Focus on reform (modernization). The experience of other World Bank projects in TVE demonstrates the need to focus on system reform, dealing with strategic policy issues, rather than financing equipment and buildings, which may lead to very poor tangible results. The TVEM is supporting important reforms in curriculum content, a financing mechanism for the system as a whole, and ensuring that program support (Component 3) goes only to TVE programs that have a proven labor market demand. 35. Improving TVE system governance. Kazakhstan faces many challenging governance issues, including in the TVE system. Kazakhstan's experience demonstrates that these issues stem from structural issues from the unfinished transition to a market-based economy and misalignment of accountabilities with the incentives in a centralized environment. Lessons learned from other World Bank-financed projects point to the importance of strengthening accountability, incentives, and institutional capacity in support of the transition from a centralized system to a decentralized structure. 36. Competitive funding mechanism for program implementation. The most effective TVE projects supported by the World Bank have introduced competitive funding mechanisms for the institutions receiving direct support. This removes a culture of entitlement among schools and provides real incentives for them to improve their performance. Component 3 introduces such a competitive funding mechanism. If proven effective, the Ministry of Education and Science (MES) may adopt this mechanism more generally for its own budget allocations. 37. Strong local ownership. World Bank global experience demonstrates the vital importance of strong local ownership for any project, particularly if the project supports reforms. The project has been developed over two years with close collaboration with the MES, and with ownership of and support from other stakeholders, especially the Ministry of Labor and Social Protection of the Population (MLSP) and ATAMEKEN. In addition, the process of developing IDPs will involve close consultation with stakeholders at the institutional level. 18 E. Alternatives considered and reasons for rejection 38. Business training. In many countries, such as Germany, Japan, and the Czech Republic, business-based training exceeds training provided through public institutions. However, this presupposes sufficient incentives and capacity of businesses to provide skills training for workers. These conditions do not exist in Kazakhstan. The proposed Project takes a necessary step toward stimulating business support for training by (a) identifying current constraints and disincentives for businesses to support training, both within firms and in the TVE system; and (b) proposing alternative ways to stimulate business training based on international experience. 39. Employer leadership. In some higher-middle-income countries such as Brazil and Singapore, employers are put in charge of policy development and allocation of funds for skills training. This is highly desirable but is seen as a longer-term goal in Kazakhstan. The Project would make a valuable start in this direction by ensuring employer participation in national, regional, sectoral, and institutional TVE councils. 40. Labor market reforms. Part of the problem with low labor productivity has to do with rigidities in labor market mechanisms, such as difficulties laying off workers, barriers to part-time work, and lack of current labor market information. Reforms in these areas were considered beyond the scope of the Project, or actions were already being taken by other agencies, such as the MLSP on market information. 41. Private provision. The proposed project does not extend the government policy framework that has opened the market to private provision; for example, it does not privatize the vocational schools. Such a move could have beneficial effects, particularly if combined with employer governance. However, this alternative is considered too radical a reform to be feasible at the moment given the embryonic nature of the training market. The Project does support private provision by ensuring equal access to project funds under the competitive process, a major change from current practices. 42. Demand for training. The team considered the alternative of focusing on promoting policies to increase the demand for TVE among parents and students. TVE in Kazakhstan, as in other countries, suffers from low prestige compared with academic secondary education. Demand could be boosted by reducing costs, such as more fee waivers among college students, and increases in student stipends in vocational schools. To some extent, these are covered in the State Program, along with increased media exposure to good employment prospects for TVE graduates. The Project, instead, focuses on the supply side of training, because evidence from the labor market indicates that the main problem at the moment is not one of demand, but of an inadequate supply of relevant, quality skills. 43. Reforms in structure, governance, and management. The Project includes modest steps toward better TVET governance in the establishment of employer participation in various advisory councils. It would also take an initial step toward consolidation by providing incentives for schools and colleges to cooperate under awards for institutional development. More extensive reforms are likely needed in the structure, governance, and management of the TVE system. However, these require careful prior study and consideration. This is provided under the policy studies in the Project. 44. Construction of new facilities. The team rejected the option of financing the construction of new facilities in favor of concentrating on standards, new programs, and making better use of existing resources. 19 45. The proposed Project by no means intends to cover the full range of reforms that eventually will be needed to make provision of skills fully demand driven and cost-effective. It does, however, take the first steps to reorient the system to employer requirements. Through the several studies it finances, it would also lay a foundation for broader future reforms while concentrating in the near term on institution-based reforms to improve training provision. III. IMPLEMENTATION A. Partnership arrangements 46. In addition to the proposed involvement of the World Bank in this Project, the Ministry of Education and Science (MES) is benefiting from a long-lasting and close relationship with the European Union (EU), the European Training Fund (ETF), German Technical Cooperation (Deutsche Gesellschaft für Technische Zusammenarbeit, GTZ), and the British Council in technical and vocational education (TVE). The Project will be implemented in close collaboration with these partners through the ongoing donor TVE coordination meetings, which are chaired and organized by the MES although there is no co-financing for this Project. The main activities of these development partners are summarized in Annex 4.1. B. Institutional and implementation arrangements 47. The Project would be implemented through a Project Management Unit located in the MES. A detailed description of implementation arrangements, including an organizational chart, is presented in Annex 6. 48. Project Management Unit (PMU). The PMU would be responsible for the day-to-day management of the Project. The PMU would include a Project Director, and staff (consultants) organized in five functional areas: (a) procurement, (b) financial management, (c) coordination for policy and systems development (Components 1 and 2), (d) coordination for Institutional Development Plan (IDP) subprojects (Component 3), and (e) administrative support. Six full-time staff would be augmented by consultants with expertise in specific fields to be employed on a short-term basis during implementation. Terms of reference (TORs) for the six key positions have been agreed upon during project appraisal and are included in the Project Operational Manual. Key members of the PMU will be appointed by effectiveness. 49. The PMU would be responsible for the following functions: (a) coordination of all aspects of project implementation; (b) liaison with government agencies, industries and businesses, and the World Bank; (c) financial management, including budgeting, accounting, reporting, internal control, audit, funds flow, including government counterpart funds and disbursement of Loan proceeds; (d) procurement of goods and services, including purchasing and contracting arrangements for all aspects of the project; ; and (e) monitoring and reporting implementation progress to government authorities and the World Bank. 50. The PMU would establish a Project Advisory Panel (PAP) as a source for technical support. The PAP would consist of local consultants in relevant fields selected for their unique expertise. Members of the PAP would help the PMU prepare TORs; review occupational standards, training programs developed under the Project, and subproject applications; and advise on specific issues as requested by the PMU. The TORs of the PAP 20 have been agreed upon and are included in the Project Operational Manual. The PAP will be established as a condition of effectiveness. 51. The PMU would prepare quarterly progress reports and a detailed midterm report within 15 months of project effectiveness as a basis for midterm review. Based on the outcomes of the midterm review, measures would be taken to ensure efficient completion of the Project. 52. A Project Steering Committee (PSC) would provide oversight on implementation and make strategic decisions in addressing key issues in implementation. This is based on a existing structure in place but with refinement of its membership and functions. The PSC, interagency in nature, would include members from concerned government agencies, such as the Ministry of Finance, the Ministry of Labor and Social Protection (MLSP), ATAMEKEN (Union of Industry Associations), and industry and employers' associations. The PSC would meet on a monthly basis. The PSC TORs have been agreed upon and are included in the Project Operational Manual. 53. Subproject Implementation at TVE institutions. TVE institutions competitively selected under Component 3 would each establish a Project Group to coordinate the work with the PMU, implement the subprojects, and report on the progress in implementation. C. Monitoring and evaluation (M&E) of outcomes and results 54. The Project would support the development and implementation of a comprehensive M&E framework to monitor project progress and evaluate impact of project interventions. The M&E activities will be limited to Project interventions and would not provide direct assistance in regular M&E of the State Program, since this function is mandated to the MES staff. However, M&E activities under the Project would enhance the system capacity to monitor and evaluate investments in TVE. 55. The progress and achievement of the Project Development Objectives will be monitored and evaluated through three types of activities: (a) regular routine monitoring, (b) midterm review, and (c) an impact evaluation at project conclusion. A set of results-monitoring indicators has been developed to measure project outputs, intermediate outcomes, and final development outcomes (Annex 3). To the extent possible, the results M&E arrangements for the project will be integrated into the existing data collection and utilization mechanism. 56. Regular routine monitoring will look at the extent to which the proposed project activities are being implemented as planned and at their direct outputs. The results monitoring will provide data for the output indicators. Regular monitoring of the Project would be the responsibility of the PMU. Results of the monitoring would be reported to the MES and the Bank in semiannual progress reports in a format agreed with the Bank. 57. A Mid-term Review (MTR) will be conducted no later than within 15 months of project effectiveness and will focus on (a) assessing early results of project implementation, (b) identifying any need for adjustment, and (c) agreeing on the next steps for the remainder of the implementation. Individual institutional data will be aggregated at the project level for analysis at the MTR in order to help the project refine the next-stage activities as necessary. A midterm report in the format agreed between the Bank and the MES would be submitted to the Bank prior to the MTR. 58. An independent impact evaluation will be conducted at the completion of the project to assess the final development impact (that is, the achievement of the Project Development 21 Objectives through the outcome and output indicators). The PMU would initiate the impact evaluation study. Detailed TORs for the impact evaluation study would be developed by the PMU/MES in accordance with the Impact Evaluation Plan in the Project Appraisal Document. D. Sustainability 59. Sustainability of the key project investments rests with (a) the TVE policy framework and regulations, (b) appropriate budget allocations, and (c) institutional sustainability. In terms of policy, the Government of Kazakhstan has clearly enunciated its commitment to modernizing the TVE system through both the State Program on TVE Development, with its budget of KZT55 billion, and the project to supplement core elements of the State Program. The Project and the Program will produce lessons learned to inform TVE policy. Financially, the Government confirmed its commitment to increase financial support to the TVE sector to sustain the State Program and project achievements. Regular allocations to infrastructure maintenance would be secured in the budget allocations for TVE. The Project would also provide incentives for greater participation of the private sector through raising overall relevance of TVE to employer demands and through increased transparency in resource allocation and monitoring of its use. No new institutions would be established under the Project. The Project would strengthen the capacity of existing institutions for skills delivery. Moreover, institutional mechanisms would be established with Project support to ensure sustainability of the reforms, that is, mechanisms to update occupational standards, incentives for institutional accreditation, and recognition of the qualifications. E. Critical risks and possible controversial aspects 60. Two critical risks are identified. One is that project-financed investments in modernizing the TVE system could result in reduced impact if the upgrading of institutional infrastructure is not closely linked with the key reform objectives and steps. Second, the Project would be implemented in an environment that is low in capacity and high in fiduciary risk and perceived corruption. As such, appropriate mitigation measures have been incorporated into the design and implementation arrangements to lower these risks. They are presented in the following matrix. Rating of Risk Factors Description of Risk Rating of Mitigation Measures Residual Risk Risk Technical/design Overemphasis on upgrading Substantial a. Tie the provision of Moderate technology and infrastructure equipment to CBT-based without linkage to curriculum reforms. Competency-based Training b. Front-load targeted (CBT) curriculum reforms. technical assistance (TA) and training for developing capacity in MES and institutions. c. Ensure proper sequencing for implementation. Implementation Technical capacity in MES Substantial a. Front-load TA and Moderate capacity and and institutions to implement training for CBT-based sustainability CBT-based curriculum and reforms and institutional management reforms. development. 22 Lack of incentive and Substantial a. Front-load advocacy, TA, Moderate resistance to change from and training. within bureaucracy and b. Conduct stakeholder among the institutions. consultation prior to implementation. Develop standards without Substantial a. Ensure employer and Moderate close involvement of industry representation and industries and employers. consultation in the development of standards. Standards developed are too Substantial a. Criteria specified in the Moderate narrow and overspecialized. TORs for ensuring broad- based standards. Willingness and readiness of Substantial a. Front-load targeted training Moderate teachers and students to and TA for CBT. adopt the new CBT approach. b. Consider incentives to encourage behavior change. Financial Lack of direct experience of Substantial a. Experienced financial Moderate management Bank Financial Management management consultants to and disbursement be hired within PMU will requirements on the part of help the MES implement MES. the Project in accordance Accounting system that does with Bank procedures. not fully meet the project's b. Appropriate Chart of reporting requirement: risk of Accounts for Project untimely or unreliable accounting will be reporting, slow disbursement, developed and the poor accounting, and weak accounting system recordkeeping. modified, with capacity to generate IFRs that will be submitted to the Bank regularly. c. Appropriate funds-flow arrangements will be agreed between MES and other participating agencies and these will be documented in the POM. Project will be subject to annual audit by independent auditors satisfactory to the Bank. Procurement Limited knowledge of Bank High Substantial procurement procedures and a. The Project will provide lack of procurement capacity. support to MES to: (a) Equipment procurement for build procurement capacity TVE is difficult due to the in accordance with World precise specifications needed Bank procurement and wide range of equipment guidelines; and (b) obtain procured; and the MES lacks external TA in developing capacity to develop adequate appropriate specifications. technical specifications for b. MES procurement staff various TVE equipment. and consultants will be invited to attend procurement training workshops when organized by the World Bank in the country. As part of the Project Launch Workshop, 23 procurement training will be provided to the PMU and MES. Social and The Project will finance Moderate An Environmental Low environmental minor rehabilitation of Management Framework safeguards technical schools and (EMF) using the Environmental colleges. It will also finance Management Plan (EMP) the purchase of new template was prepared to equipment for these determine possible educational centers. As a environmental impacts of result, there may be handling project activities. The EMF and/or disposal of hazardous will determine which materials in the process. corresponding mitigation measures are to be implemented by contractors, schools, and colleges affected by them. These measures will be included in the Project Operational Manual and in contracts. The Bank team will supervise compliance with the EMF during implementation. Overall Risk (including Reputational Risks) Moderate F. Loan conditions and covenants Loan conditions for effectiveness (a) The PMU has been established by the Borrower, through the MES, with a composition, resources and terms of reference satisfactory to the Bank. (b) The Advisory Panel has been established by the Borrower with a composition, resources and terms of reference satisfactory to the Bank. (c) The Operational Manual6, satisfactory to the Bank, has been adopted by the Borrower. Covenants (a) The Borrower, through the PMU, shall carry out the Project in accordance with the requirements, criteria, organizational arrangements and operational procedures set forth in the Operational Manual and the EMF, and shall not assign, amend, abrogate or waive any provisions of the Operational Manual or the EMF without prior approval of the Bank. (b) At all times during the implementation of the Project, the Borrower shall maintain the Project Steering Committee, the PMU and the Project Advisory Panel with a composition, resources, and terms of reference satisfactory to the Bank. 6 The POM was substantially complete by negotiation except for some minor changes relating to financial management. As a result, it was decided at negotiation that adoption of the final POM (including a complete section on financial management controls) would be a condition of effectiveness. 24 (c) Not later than thirty (30) days after the Effectiveness Deadline, the Borrower, through the MES, shall upgrade the accounting software of the PMU in a manner satisfactory to the Bank. IV. APPRAISAL SUMMARY A. Economic and financial analyses 61. The project would support the modernization and reform of the TVE system through developing standards and quality assurance for TVE; strengthening governance, management, and financing for TVE; and strengthening skills delivery capacity of TVE institutions. 62. The analysis of the labor market and demand for skills in Kazakhstan provides the basis for a strong economic rationale for government intervention and reforms in the TVE education system. If properly implemented, these reforms are expected to bring about economic benefits through the following: (a) Improved employability of TVE graduates as the quality of TVE rises and new quality TVE graduates become more demanded in the labor market. Kazakhstan, like other transition countries in the Europe and Central Asia Region, inherited the Soviet system of TVE, the relevance of which to the labor market can be described as artificial, because it was established within a tradition of central economic planning and centralized job placement. New economic realities of the fast- changing and increasingly globalized economy call for profound revisions of the TVE system to enable it to meet the demands of the dynamic labor market. (b) Increased rates of returns to TVE as the skills and competencies of the graduates of the new TVE system are significantly enhanced and match the requirements of the labor market.7 The restructuring of the education system so that the quality of the provided secondary general education (core curriculum) is the same across all institutions will also eliminate the negative selection that is evident in the TVE system; those who choose to attend technical school are often those who are perceived as having low capabilities for education at a higher level. 63. Estimated economic costs and benefits of the Project. According to the proposed TVE Project, the investment costs over July 2010­December 2013 will amount to US$29.23 million (KZT4.38 billion). These costs will be covered through the World Bank loan of the same amount.8 The cost of borrowing on IBRD loan terms over 20 years will amount to US$16.6 million (the net present value [NPV] of those payments [at a 5 percent discount rate] is estimated to be US$9.1 million in 2010 prices). In the cost-benefit analysis it is assumed that on top of these costs, the sustainability of the Project will also require additional support costs equal to about 5 percent of the Project costs per year for 10 years (2015­25) following the initial investment phase. These costs will thus amount to US$14.6 million in nominal terms (the NPV of those payments [at a 5 percent discount rate] is estimated to be US$9.3 million in 2010 prices). To summarize, the cost-benefit analysis 7 Analysis of the 2008 Labor Market Survey reveals that currently the workers with TVE education are paid on average 28 percent more than those with secondary general education. 8 This calculation is based on the exchange rate of 150 KZT / US$. 25 assumes that the Project's overall costs (IBRD loan, loan servicing, plus additional costs to sustain the reform of the TVE system) will reach US$58.4 million in nominal terms. The NPV of the Project's overall costs is estimated at US$47.6 million in 2010 prices. 64. The economic benefits of the TVE Project to the GOK are estimated by the World Bank at US$103.5 million (in 2010 prices) during 2015­30. Those benefits take the form of additional tax revenues (personal income tax collections) that result from the implementation of the Project. Similar to the calculation of costs, the calculation of benefits uses the annual rate of 5 percent to estimate the NPV. The calculated benefits are based on the premise that the new quality TVE graduates will be available for the labor market starting in 2015. 65. The net benefits of the TVE Program are estimated at US$ 55.9 million over the period of 2015-2030. However, according to the Project's expected payoff schedule, the net benefits will only start to accumulate in 2015. 66. To sum up, the cost-benefit analysis indicates that the economic benefits of the TVE Project to the GOK will outweigh the costs. The NPV of the net benefits is estimated at US$ 55.9 million. Moreover, the estimated net benefits are expected to continue after the year 2030. The estimated net benefits also do not take into account the private benefits that accrue to individuals as a result of better TVE education, as well as potential spill-over effects due to improved labor mobility, adoption of knowledge, etc. B. Technical 67. Technical Analysis. The proposed Project would contribute directly to achieving its three broad objectives, as follows: (a) Increased Relevance of TVE: Employers would benefit from the proposed Project through more appropriate skills and higher productivity of workers. First, employers would become involved in guiding development of the TVE system through national, regional, and sectoral councils. They would have an avenue through their associations to articulate the scope of skill requirements that only employers know best. Second, the knowledge and skills acquired by trainees would correspond more closely to actual job requirements. This would be assured through the development of occupational standards as a starting point for training curriculums. Students and trainees would also benefit greatly from the proposed Project. By streamlining the curriculum at the professional lyceum school level and supporting the integration of general education and TVE at this level through the development of broad TVE programs, students would be able to build broad vocational orientation skills that would allow for flexibility in employment and/or continuation of education or training. These new programs would offer career development opportunities based on broad skills and competencies that are relevant to a range of occupations in the labor market. Programs will be developed around broad occupational families in well- defined groups of occupational areas and will introduce students to career paths. These paths could lead to entry-level employment opportunities in specific careers and foundation skills for learning new occupations on the job and through periodic upgrading. Modular programs, in particular, would provide the building blocks for more flexible retraining and upgrading. The introduction of career guidance and counseling would also facilitate the entry of trainees directly into jobs in demand and later into opportunities for skill upgrading. 26 Private training providers would also benefit under the Project to the extent that they win bids for the development and implementation of new TVE programs. Private training providers are characterized by shorter response times to changes in occupational demand than are public providers. Thus, support for private providers would help make the overall system more responsive. (b) Better-quality TVE: The proposed Project would support multiple interventions to raise the quality of TVE. Quality is defined here as effective learning achievement and skills acquisition. New TVE standards would be developed on the basis of occupational standards. Extensive support would be provided to the development of new teaching curriculums and programs in priority fields. International experience shows that curriculums are implemented best when teachers are directly involved in their development. The proposed Project follows this approach, with the new programs to be developed by staff of TVE institutions selected on a competitive basis. International experience also shows that competency-based education and training is far more effective in the learning of skills than traditional time-based teaching methods. The Project also supports this approach. Another essential requirement for high-quality TVE is teacher competence. The Project therefore supports the upgrading of instructional staff to deliver the new curriculums and to develop needed teaching materials. Proper skills training cannot be acquired without access to appropriate up-to-date equipment provided under project financing. New procedures would be developed and implemented for assessment of student performance to ensure higher levels of achievement. Accreditation of TVE institutions focused on outputs would be designed and implemented as a means of quality assurance of the system as a whole. Similarly, the monitoring and evaluation system to be designed under the Project would help ensure achievement of overall reform goals by monitoring performance and highlighting problems for special attention. Finally, competition for funds as provided under the Project could stimulate innovation that raises quality. (c) Improved Financing and Internal Efficiency: The proposed Project includes several elements to ensure more efficient deployment of available resources. Formula-based financing would be designed, pilot-tested, revised, and prepared for national implementation. This system would equalize spending across the system by program and would give TVE institution managers the authority and incentives to spend the funds economically. Broadening of programs into clusters within families of occupations would reduce overspecialization and enable more efficient use of instructor time within institutions. Consolidation, part of the overall State Program supported by the Project, would also help to achieve greater economies of scale within institutions, such as the interregional TVE centers. C. Fiduciary 68. Financial management (FM) capacity assessments were carried out during the months of August 2009 and March 2010 to determine if the financial management arrangements established by the MES were satisfactory to the Bank. These financial management arrangements include budgeting, accounting, reporting, internal control, audit and disbursement procedures. The assessments established that the FM arrangements in the MES 27 system are satisfactory, but could be further strengthened during implementation to fully support project accounting and reporting requirements. 69. Fiduciary Risk at the Project Level: Based on the FM assessment fiduciary risk at project level was assessed to be substantial before mitigation and moderate after risk mitigation measures. A number of actions were agreed to further strengthen the financial management arrangements, including development of FM manual, modification of the automated accounting system with capacity to generate interim financial reports and hiring of FM consultant to take charge of project FM and disbursement functions, under the direction of the MES Chief Accountant. Terms of reference for the modification of the accounting software and for the financial management consultant were developed and approved by the MES prior to negotiations. 70. Fiduciary Risk at the Country Level: Based on the PEFA assessment, issued in May 2009, the overall fiduciary risk in Kazakhstan is assessed as substantial. The key reasons are: inadequate capacity of core control and supervisory agencies performing audits within public sector; (ii) although most of the basic laws are in place within with respect to private sector and public enterprises, including state non-commercial organizations), financial reporting, compliance remains a problem and authorities need to improve quality of auditing, monitoring and supervision. Annex 7 presents details of assessment and mitigation measures. 71. Procurement. The latest Country Procurement Assessment Report is dated June 2000. A Law on Public Procurement of Goods, Works and Services was passed in May 2002. The GOK and the Bank conducted a Joint Public Expenditure and Institutional Review in 2004, which included a review of the efficiency of public procurement. This assessment led to Kazakhstan being ranked as a High Risk country from the public procurement point of view. A new Law on Public Procurement was enacted in July 2007 and took effect on January 1, 2008. While the new law includes many provisions that reflect international practices and Bank recommendations, it remains in need of further improvement and, therefore, the country's procurement risk category remains unchanged. 72. The overall responsibility for implementation of the Project rests with the MES through its Department of Technical and Vocational Education. The responsibility for day-to-day management, coordination, and monitoring rests with the PMU, established within the MES. The PMU will ensure an ongoing dialogue with and feedback from MES management and project stakeholders to ensure ownership and promptly resolve problems, and will be liaising with the World Bank. Direct responsibility for implementation of the Project, including procurement activities, will fall on the PMU. 73. Based on the assessment of MES capacity to implement procurement, the overall project procurement risk is rated as High. It is recognized, however, that this rating is determined by the overall procurement environment in the country, while the MES has very solid technical experience in dealing with public procurement. The MES Department of State Procurement and Capital Construction is staffed by experts with the relevant technical experience in conducting procurement according to national legislation. However, this department does not have sufficient experience dealing with management of the international procurement cycle. 74. Procurement for the Project will be carried out in accordance with the Bank's "Guidelines: Procurement under IBRD Loans and IDA Credits" published in May 2004 and revised in October 2006 and May 2010 (Procurement Guidelines); and "Guidelines: 28 Selection and Employment of Consultants by World Bank Borrowers" published in May 2004 and revised in October 2006 and May 2010 (Consultant Guidelines), and the provisions stipulated in the Loan Agreement. More details on procurement arrangements are in Annex 8. D. Social 75. There are no ethnic issues affecting the Project, nor are there any policies or interventions in the system that negatively impact gender parity in TVE. However, TVE institutions, particularly vocational lyceums, suffer from low prestige compared with academic secondary schools. Lyceums are the destination for academically unsuccessful students, which accounts for part of the stigma. The student population in TVE tends to come from lower-income groups because of prior educational disadvantage. In addition, TVE has suffered from neglect in terms of new investment, which reduced the quality of instruction provided. The proposed Project addresses the issue of stigma in three ways. First, the Project will help introduce a national qualifications framework, which provides more flexible pathways for further study, that is, easier entry into and exit from formal education. Second, the Project supports the integration of curriculums with general secondary education, thereby enhancing the possibility for graduates to pursue higher education. Third, project design directly addresses youth concerns, as articulated in the survey, Youth Voices. The Project seeks to raise the quality and relevance of teaching, which will make graduates more employable and able to command higher wages. E. Environment 76. The Project is classified as Category B due to its moderate environmental impacts per World Bank Operational Policy 4.01. The Project finances renovation and refurbishing of existing facilities and adaptation of interior space for the accommodation of existing laboratories and training facilities through the provision of sub-grants under Component C of the Project. Project activities are not expected to cause significant adverse environmental impacts. Nevertheless, to ensure adequate safeguards to minimize the impacts, the Project identified the following issues that should be addressed: (a) dust and noise due to the rehabilitation activities; (b) dumping of construction wastes, accidental spillage of machine oil, lubricants, and paints and solvents; and (c) asbestos. It is not known whether asbestos was used in premises to be proposed for rehabilitation, but considering its common use in the former USSR, it is possible to find such material in insulation and roofing material. Inappropriate handling of asbestos could cause a real health concern for construction workers and for the general public in the vicinity of the rehabilitated premises, particularly if it is inhaled. 77. Since the subprojects to be financed are yet to be identified, the MES prepared an Environmental Management Framework (EMF), which outlines environmental assessment procedures and mitigation requirements for the selected subprojects. The EMF provides details on procedures, criteria, and responsibilities for subproject screening, conducting Environmental Impact Assessments and preparing Environmental Management Plans, as well as supervision and monitoring procedures. The EMF also includes Environmental Guidelines containing an assessment of potential impacts and generic mitigation measures to be undertaken for identified subprojects in all stages, from selection and implementation to the monitoring and evaluation of the results, and provides details of MES EMF institutional responsibilities. 29 78. In accordance with the Bank's policy, the EMF was disclosed and consulted in the country prior to appraisal. F. Safeguard policies Safeguard Policies Triggered by the Project Yes No Environmental Assessment (OP/BP 4.01) [X ] [] Natural Habitats (OP/BP 4.04) [] [X] Pest Management (OP 4.09) [] [X] Physical Cultural Resources (OP/BP 4.11) [] [X] Involuntary Resettlement (OP/BP 4.12) [] [X] Indigenous Peoples (OP/BP 4.10) [] [X] Forests (OP/BP 4.36) [] [X] Safety of Dams (OP/BP 4.37) [] [X] Projects in Disputed Areas (OP/BP 7.60)* [] [X] Projects on International Waterways (OP/BP 7.50) [] [X] * By supporting the proposed project, the Bank does not intend to prejudice the final determination of the parties' claims on the disputed areas. G. Policy exceptions and readiness 79. There are no policy exceptions. 80. The project meets the regional implementation readiness criteria. 30 Annex 1: Sector Background KAZAKHSTAN: TVEMP Formal Technical and Vocational Education in Kazakhstan and Reform Plans Structure 1. Following nine years of basic education, students in Kazakhstan have the choice of entering either general education or technical and vocational education (TVE). General education is provided through general secondary schools, lyceums, and gymnasiums. Technical and vocational education is provided through vocational schools (professional lyceums) and colleges. Vocational schools provide initial vocational education aimed at preparing students for entry into the labor market as skilled workers in technical and service fields. Vocational school programs last two to three years. Colleges provide more theoretical work in science and mathematics and prepare students as technicians, masters, and supervisors. College programs last from three to four years. Colleges are also able to offer initial vocational education, as in vocational schools, but reportedly few do so. Upon graduation from secondary school, after grade 11, students may enter higher education or TVE at a more advanced level, or enter the labor market. Quantitative overview 2. The transition to a market economy in Kazakhstan dramatically impacted the TVE system. The number of state-funded TVE schools declined by one-third between 1991 and the present. However, the number of non-state-funded institutions increased substantially, especially among colleges. Table A1.1 shows the basic dimensions of the formal TVE system. Table A1.1: TVE Elements in 2008 by Ownership Type Institutions Teachers Student Total Number of Intake in Student Graduates 2008 Enrollment Schools (lyceums) ­ Total 324 10,480 49,520 111,180 39,490 Public 298 n.a. 48,290 108,180 38,320 Private 26 n.a. 1,230 3,000 1,160 Colleges ­ Total 542 28,580 166,680 499,280 118,760 Public 208 n.a. 75,350 237,470 57,080 Private 334 n.a. 91,330 261,810 61,680 n.a. = Not applicable. Source: MES. 3. A significant feature of TVE is the presence of private institutions and private financing of student fees in public institutions. As Table A1.1 shows, in the aggregate about 42 percent of the total number of schools and colleges are private. Similarly, about 43 percent of student enrollment is in private schools and colleges, as are VET teachers. The student- teacher ratio in VET schools is lower at 10.6 students per teacher than in VET colleges (19.5 students per teacher). Between 1998 and 2008, the number of private colleges increased from 246 to 334 and enrollment in them increased from 55,300 students to 261,810 students. The bulk of the student population was concentrated in business-related studies. Given limitations of public financing and strong student demand, public colleges now admit fee- 31 paying students, who have grown to about 60 percent of total enrollment. Fee-paying students pay about half the actual cost per student at the institution. Governance and management 4. The Ministry of Education and Science (MES) is responsible for all levels of education, including TVE. It develops regulations and procedures regarding policy implementation. TVE is managed by the Department of Technical-Vocational Education within the MES. 5. All public vocational schools (vocational lyceums) and all other public colleges are administered by the departments of education of 14 regions (oblasts) and two cities (Astana and Alma Ata). The regions/municipal education administrations have a TVE unit. Responsibilities include inspecting TVE delivery in accordance with state mandatory standards, preparation of the TVE section of the regional/city budget, certification of teachers, licensing of private educational institutions, approving charters of educational institutions, and retraining and upgrading. Financing 6. Spending on TVE amounts to about 5.9 percent of total public expenditures on education. State run colleges are financed by the national government budget. Locally operated institutions including vocational schools (professional lyceums) and colleges are funded by local governments. Schools and colleges defined as state enterprises are allowed to raise and use revenue outside government sources. The main source of outside income is from fee-paying students. In 2007, 60 percent of total enrollments in public colleges were fee paying. The fees charged generally cover about 50 to 60 percent of the full costs. Strengths and weaknesses of TVE in Kazakhstan Table A1.2 shows an initial assessment of the strengths and weaknesses of current TVE provision according to four criteria: external efficiency, organizational and management effectiveness, quality of training, and finance and internal efficiency. 32 Table A1.2: Strengths and Weaknesses of Current TVE Provision Criterion Strengths Weaknesses 1. Plans for integration of general with 1. Content of programs not linked to labor market vocational education through a common core requirements. program of studies (Education Law of 2007). 2. Decreased output and insufficient graduates in 2. Private colleges and lyceums are allowed, fields in demand. which tend to respond more quickly than 3. Supply driven--incentives are oriented to filling public institutions to changes in labor market existing capacity and using existing teachers and 1. External Efficiency demand. facilities. 4. Educational standards and the content of teaching programs are not based on occupational analysis. 5. Overspecialization of the curriculum leading to narrow skills. Programs overly theoretical and inflexible. 6. Lack of sufficient foundation skills that would enable learners to learn new skills on the job and become lifelong learners, as needed in a market economy. 7. Limits on institutions (e.g., compulsion to follow rigid state standards; budget controls) limit flexibility of vocational and technical institutes to respond to changes in the labor market. 1. Most TVE institutions are managed at the 1. Narrow governance structure that excludes 2. Organizational & regional or municipal levels. stakeholders, especially employers. Inadequate role Management 2. Adoption of State TVE Modernization for employer participation. Efficiency Program in 2008. 2. Central-government dominated. 3. Government, not institutions, decides on the number of students admitted by specialization. 4. Lack of an M&E culture and structures to feed policy development and guide implementation. 1. National assessment system exists, which 1. High turnover among instructors of practical provides data on graduate performance by subjects and teacher shortages. school, region, and program. 2. Tendency toward aging of the teacher force. Low 2. 50 Methodological Centers exist--mainly inflow of qualified young technical-vocational schools with experience in program instructors owing to low salaries and benefits for development. teachers in the TVE system. 3. Quality of Training 3. Plan to adopt modular, competency-based 3. Teachers and instructors lack opportunities for in- curriculums and assessments. service training. No systematic approach to 4. Establishment of six centers in higher collection and dissemination of good practices to education to train technical and practical pedagogical staff. instructors. 4. Regions/cities spend on TVE only about half what the central government spends on national institutions. 5. Lack of significant investment in TVE renewal since independence. 6. Little investment in updating equipment and facilities--aging infrastructure and educational technology. 7. Insufficient materials and teaching aids. 33 1. Relatively low dropout rates and 1. Constraints on and disincentives for business correspondingly high completion rates for investment in Enterprise-based Training (EBT) and vocational schools and colleges. support for TVE. 2. High degree of fee-paying students in public 2. Low student-teacher ratios (either from colleges expands revenue for TVE. overspecialization or low teaching loads). 4. Finance and Internal Efficiency 3. Expansion in the number of private colleges 3. The allocation of resources to public TVE and some schools potentially good for market institutions is driven by historical budgets rather responsiveness. than institutional needs and system priorities. 4. A wide variance exists on expenditure by student and program across institutions. 5. Budgetary controls limit the flexibility of allocating available resources according to institutional needs. 6. Lack of good data on actual unit costs by program category. 7. Fee-paying students in public colleges do not pay full costs. 8. Overlap between initial vocational education and initial college level. 9. Resources allocated to TVE are considerably lower than in other middle-income, emerging-market economies. Source: Bank compilation. 7. Within the framework of the Youth Voices program for Kazakhstan, a group of 25 youth undertook a survey of a random sample of 500 young people in five oblasts on their views on the TVE system in Kazakhstan. The main conclusions were as follows: The most critical challenge of the TVE system is the low quality of teaching staff. The absence and/or low quality of textbooks is an acute problem. TVE schools and colleges need significant improvement in their education materials and equipment. The key factor in students selecting TVE schools and colleges was having no-fee education or low-cost education. The key factor in selecting a specific school was its proximity to the student's home. Students of TVE schools and colleges indicated that the knowledge acquired at school was very useful when entering higher educational institutions or other educational facilities. However, it was not good enough for entering employment. Reform plans 8. The Education Law of 2007 provides the framework for restructuring TVE, including provision for better integration of secondary vocational and general education and the development of separate postsecondary vocational education qualifications with more advanced levels of technical skills. The Government plans to change to a system of 10 years of basic education and 2 years of specialized education beginning in 2011. Vocational and general education secondary institutions will share the same core courses of general subjects comprising about 40 percent of instructional time. 9. The State Program for TVE Development, developed with World Bank assistance, was approved by Presidential Decree 626 of July 1, 2008, with an initial phase to be carried out from 2008 to 2012. The goal of the State Program is to modernize the TVE system so that it will provide the economy and employers with qualified specialists who possess competitive skills and sufficient capacity for lifelong learning. These goals will be achieved by: (a) modernizing content, support materials, infrastructure, and equipment to ensure TVE 34 accessibility and attractiveness; (b) improving the quality of training and competitiveness of the workforce produced by the TVE system; and (c) improving efficiency and accountability of TVE management and funding systems. The State Program will be financed by KZT55 billion (approximately US$366.7 million) from the National Budget. The Program calls for reforms in standards, TVE governance and finance, and incentives for increased public- private partnerships. The TVEM Project supports pilot testing the initial steps in the implementation of this program by developing standards and quality assurance of TVE; strengthening governance, management, and financing for TVE; and strengthening skills- delivery capacity of TVE institutions. The Government has indicated its intention to scale-up the project with a follow-on loan for system wide implementation through additional financing upon successful completion of the first phase. 10. The TVE reform program is focused on restructuring the TVE system in Kazakhstan with two levels. The first level, upper-secondary TVE, would aim to be fully integrated with general secondary education, offering limited skill-based vocational content in grades 11 and 12 instead of a large amount of occupationally based vocational content. The second level, postsecondary professional TVE, would operate with flexibility and modern governance that includes private/public partnerships and would be focused on quickly responding to regional or national job-training demands. 11. The Program is in line with recent social and economic policy reforms, including the National Competitiveness Strategy, the National Industrialization Strategy, and the National Development Strategy 2020. 35 Annex 2: Major Related Projects Financed by the Bank and/or other Agencies 1. GTZ (German Technical Cooperation) Project: Assistance for Development of Vocational Training in Kazakhstan (1995­1999). 2. GTZ Project: Interregional Cooperation of Central Asia and Transcaucasia Countries in Vocational Training (2000­2003). 3. GTZ Project: Advanced Training with a View to Further Organization of Multiplication Work (Kazakhstan, Kyrgyz Republic, 1995.4804.1). 4. GTZ Project: Vocational Training of Girls and Women and Support of Educational System Reforming in Kazakhstan. 5. GTZ/Government of Germany Project: Support of National Minorities in Central Asia Countries. 6. Invent NGO: Support of Economic Reforms in Central Asia Partner Countries by Means of Vocational Education (2006­2010). 7. European Training Fund: Public Private Partnership in Vocational Education and Training in the Republic of Kazakhstan. 8. European Training Fund: Development of National Qualifications Frameworks in Central Asia. 9. European Training Fund: Development of Skills for Poverty Reduction. 10. Technical Aid for the Commonwealth of Independent States (TACIS): Relation between Vocational Training and Education and Development of Small and Medium- sized Business in Kazakhstan. 11. World Bank: Kazakhstan Technology Commercialization Project. 36 Annex 3: Results Framework and Monitoring KAZAKHSTAN: TVEMP 1. The Project Development Objective of the Technical & Vocational Education Modernization Project (TVEMP) is to raise the relevance, quality, and efficiency of the technical and vocational education in Kazakhstan through an improved policy framework and institutional capacity. The attainment of the objective will be measured by the following key outcome indicators: Employer satisfaction with the industry-led competency standards as a basis for development of curriculum and programs; and Share of the existing programs in the project-supported schools revised in accordance with the newly introduced competency standards. 2. The development impact of the project will be measured by a set of indicators as summarized in the Results Framework in Table A3.1. (Arrangements for results monitoring are presented in Table A3.2.) 3. The objective of the monitoring and evaluation (M&E) is to determine the extent to which the project will have an impact on the technical vocational education and training system and enabling conditions for further demand-driven development of the system. The short implementation period does not allow sufficient time to evaluate the impact on students and employers. 4. The project M&E will: (a) monitor the progress of project implementation and provide timely feedback to the Project Management Unit (PMU) and project implementers (schools); (b) monitor, assess, and summarize the achievement (output and outcomes); (c) analyze factors affecting the project's implementation and achievement; and (d) summarize experiences and lessons learned for policymaking and dissemination. Arrangements for results monitoring and evaluation 5. To the extent possible, arrangements for the results M&E will be integrated into the existing data collection and utilization system. Three types of M&E activities will be carried out during and after the intervention, as outlined below. Regular routine monitoring 6. Regular routine monitoring assesses whether the proposed project activities have been implemented as planned and their direct deliverables (outputs) achieved, and monitors in a systematic fashion changes in project outcome indicators over time. This will provide the project with information on the progress of project implementation and help the project make adjustments as necessary. 7. Monitoring the direct deliverables and outputs of proposed project activities measures the extent to which an individual activity has been implemented in a timely fashion and expected outputs have been achieved according to a set of predetermined targets. 37 8. Given the variation among project schools in terms of detailed project activities, individual project schools will develop a set of output indicators and targets as part of the Institutional Development Plans (IDPs). The schools will monitor these indicators regularly during implementation using the existing school administrative system and project management mechanism. A set of output indicators developed by the project schools (and reflected in the IDPs) will be tracked regularly by the project schools, through school administrative sources, and reported to the Ministry of Education and Science (MES) and the World Bank via the PMU on a semiannual basis. Mid-Term Review (MTR) 9. The Mid-Term Review (MTR) will be conducted no later than within 15 months of project implementation. It will assess the progress in implementing the three components, the output produced, whether project benchmarks in the Project Results Framework have been achieved, and identify any need for adjustment. 10. The MTR will analyze aggregated school data and analyze and summarize experiences accumulated and lessons learned during project implementation. The results of the analysis will help the Project refine the next stage of project implementation. Most original data measuring the Project's intermediate outcomes will be collected by project schools and reported to the PMU on a semiannual basis. The PMU (with support of the Project Advisory Panel [PAP]) will lead the MTR, conduct data aggregation and analysis at the project level, and provide a review report to the MES and the World Bank. 11. A process evaluation will be conducted at the MTR the objective of which is to assess the effectiveness and efficiency of the project implementation, addressing implementation questions such as which activities have been implemented and who has been reached through the activities. Process evaluation is a necessary step in understanding why the project has the effects it has or why it does not have the effects that were expected. Without information on project implementation, it will be difficult to link any changes/outcomes observed at the school level to the intervention. The process evaluation will also assess whether inputs or resources have been allocated or mobilized and whether activities are being implemented as planned. It will identify project strengths, weaknesses, and areas that need improvement. A combination of qualitative and quantitative approaches is proposed for the collection of data for the evaluation process, including document review, interviews with various stakeholders (school administration and management, project team, and other stakeholders), and satisfaction surveys with a sample of employers and teachers involved in the project. Impact evaluation 12. An independent impact evaluation will be conducted at the completion of the project to assess project achievements. It will also identify which types and modalities of interventions are effective and efficient in contributing to the project achievements, and conditions or factors that can enhance or hinder the effectiveness and efficiency of the various intervention strategies at both the institutional and policy levels. 13. The evaluation will test the main hypothesis of the intervention: that the reforms of the project have made a significant contribution in supporting Technical and Vocational Education and Training (TVET) institutions to improve the capacity, quality, and relevance 38 of technical and vocational training. This hypothesis will be further detailed and divided into sub-hypotheses for testing in the terms of reference (TORs) for impact evaluation. 14. The impact evaluation, the details of which are presented in the next section of this document, will be carried out by an independent research team or a consulting firm. The provincial PMU will coordinate and facilitate data collection for the baseline study and impact evaluation. During implementation, the project schools are also required to collect outcome indicator data through the existing administrative systems and report to the MES and the World Bank via the PMU on a semiannual basis for the purpose of monitoring the trend of improvement over time. Table A3.1: Results Framework Objectives Project Outcome Indicators Use of Project Outcome Information The Project Development 1. Employer satisfaction with the industry-led Indicates the extent to Objective is to raise the competency standards as a basis for development which improved policy relevance, quality, and of curriculum and programs. framework and efficiency of technical and institutional capacity meet vocational education (TVE) 2. Share of the existing programs in the project- market demand and through an improved policy supported schools revised in accordance with the quality standards. framework and institutional newly introduced competency standards. capacity. Intermediate Outcomes Output Indicators Use of Output (Outputs) Monitoring Component 1. Develop Standards and Quality Assurance for TVE 1.1. National Qualification 1. Concept of National Qualifications System Evaluation of Project Framework (NQF) approved (NQS) and National Qualifications Framework progress and improvement by GOK and employers. (NQF) approved by GOK and employers. of implementation 2. Legal framework, including regulations, revised arrangements. and approved by GOK. 3. NQF and key elements of NQS formally adopted by the GOK. 1.2. Quality and Relevance of 4. Share of occupations covered by new standards Evaluation of Project TVE improved through: out of total number of occupations in the priority progress and improvement - Occupational standards sectors supported by the project. of implementation - Skills assessment tools. 5. Share of newly developed occupational arrangements. standards for which qualification assessment tools prepared. 1.3. Independent institutional 6. Accreditation standards and mechanism Evaluation of Project accreditation mechanism developed. progress and improvement introduced. 7. Number of project schools that passed of implementation independent institutional accreditation. arrangements. Component 2. Strengthen Governance, Management, and Financing for TVE 2.1. National, regional, 1. National and regional councils have at least one- Evaluation of Project sectoral, and institutional third of members from the industry. progress and improvement councils operating 2. Share of sectors that established councils. of implementation effectively. 3. Share of project schools with established boards arrangements. (with participation of employers). 2.2. Options for reform of 4. Study completed on governance and 39 TVE governance and management of TVE at all levels with policy management system options generated. generated and adopted. 5. Implementation plan for governance and management improvement prepared, budgeted, and adopted by the MES. 2.3. Effectiveness of 6. Formula-based financing policy options studied Evaluation of Project financing in TVE system and proposal developed. progress and improvement improved through 7. Formula-based financing mechanism piloted in of implementation introduction of formula-based one or two regions. arrangements. financing mechanism. 8. Plan developed for nationwide implementation of formula-based financing. 2.4. Monitoring and 9. Project M&E plan developed and implemented. Evaluation of Project evaluation (M&E) system 10. Impact evaluation study completed and report progress and improvement developed and feeding generated. of implementation information into project arrangements. implementation. Component 3. Strengthen Skills Delivery Capacity of TVE Institutions (institution-based innovations/reforms) 3.1. National educational 1. Number of educational standards updated in line Evaluation of Project standards and core with occupational standards. progress and improvement curriculum updated in line 2. Number of updated educational standards of implementation with occupational standards. supported with revised core curriculum. arrangements. 3.2. Capacity of the teacher- 3. Number of in-service teacher-training programs Evaluation of Project training institutions in the developed in the area of CBT. progress and improvement area of Competency-based of implementation Training (CBT) for TVE arrangements. sector improved. 3.3. Skills delivery system 4. Share of TVE schools that received grants (out Evaluation of Project improved. of total number). progress and improvement 5. Share of private TVE schools that received of implementation grants (out of schools that received grants). arrangements. 6. Share of TVE schools that implement the grants in accordance with the targets set in their Institutional Development Plana (IDPs) (out of schools that received grants), as verified by regular operational audit. 7. Share of teachers in each project school trained in developing and delivering new CBT programs. 8. Share of project schools that planned and introduced programs that use competency-based curriculum. 40 Table A3.2. Arrangements for Results Monitoring Target Values Data Collection and Reporting Project Outcome Indicators Baseline YR1 YR2 YR3 Frequency and Data Collection Responsibility for 2011 2012 2013 Reports Instruments Data Collection 1. Employer satisfaction TBD at TBD n/a TBD at Baseline study and Baseline study and MES / PMU with industry-led YR1 YR1 impact study impact study competency standards as a basis for development of curriculum and programs. 2. Share of the existing TBD at TBD n/a TBD at Baseline study and Baseline study and MES / PMU programs in the project- YR1 YR1 impact study impact study supported schools revised in accordance with the newly introduced competency standards. Output Indicators Component 1. Develop Standards and Quality Assurance for TVE Sub-component 1.2. NQF and NQS 1. Concept of NQS and NQF n.a. Con- NQF Semiannual Implementation MES / PMU approved by GOK and cept Progress Report employers. of NQS 2. Legal framework, including n.a. Done Semiannual Implementation MES / PMU regulations, revised and Progress Report approved by GOK. 3. NQF and key elements of n.a. Done Semiannual Implementation MES / PMU NQS formally adopted by the Progress Report GOK. Sub-component 1.2. Occupational Standards and Assessment 4. Share of occupations 0 5% 25% 100% Semiannual Implementation MLSP / MES/ covered by new standards out Progress Report PMU of total number of occupations in the priority sectors 41 supported by the Project (about 147). 5. Share of newly developed 0 40% 80% 100% Semiannual Implementation MES / PMU occupational standards for Progress Report which qualification assessment tools prepared. (out of number of new standards developed to date) Sub-component 1.3. Institutional Accreditation 6. Accreditation standards and n.a. 0 0 Done Semiannual Implementation MES / PMU mechanisms developed. Progress Report 7. Number of project schools n.a. 0 0 10 Semiannual Implementation MES / PMU that passed independent Progress Report institutional accreditation. (at least one school from each of 10 priority sectors) Component 2. Strengthens Governance, Management, and Financing for TVE Sub-component 2.1. Governance and Management 1. National and sector councils 35% (4 50% 75% 100% Semiannual Implementation MES / PMU have at least one-third of existing Progress Report members from the industry. out of priority 10) 2. Number of priority (10) 4 5 7 10 Semiannual Implementation MES / PMU sectors which established Progress Report councils. 3. Share of project schools n.a. n.a. n.a. 100% Semiannual Implementation MES / PMU with established boards (with Progress Report participation of employers). 4. Study completed on n.a. 0 Done Semiannual Implementation MES / PMU governance and management Progress Report of TVE at all levels with policy options generated. 5. Implementation plan for the n.a. 0 0 Done Semiannual Implementation MES / PMU 42 governance and management Progress Report improvement prepared, budgeted, and adopted by MES. 6. Formula-based financing n.a. Done Semiannual Implementation MES / PMU policy options studied and Progress Report proposal developed. 7. Formula-based financing n.a. Done Semiannual Implementation MES / PMU mechanism piloted in one or Progress Report two regions. 8. Plan developed for n.a. Done Semiannual Implementation MES / PMU nationwide implementation of Progress Report formula-based financing. Sub-component 2.3. Monitoring and Evaluation 9. Project M&E plan n.a. 1 1 1 Semiannual Implementation MES / PMU developed and implemented. Progress Report 10. Impact evaluation study n.a. Baseli n.a. Done FY1 and FY3 Implementation MES / PMU completed and report ne Progress Report generated. collec ted Component 3. Strengthen Skills Delivery Capacity of TVE Institutions Sub-component 3.1. National Educational Standards and Core Curriculum 1. Number of educational n.a. 40 60 47 Semiannual Implementation MES / PMU standards updated in line with Progress Report occupational standards. (progressive total, out of total 220 educational standards in Kazakhstan) 2. Number of updated n.a. 25 50 100 Semiannual Implementation MES / PMU educational standards Progress Report supported with revised core curriculum. (progressive total, out of 147 educational standards supported under the project) 43 Sub-component 3.2. TVE Teacher-training Program 3. Number of in-service 0 4 5 6 Semiannual Implementation MES / PMU teacher-training programs Progress Report developed in the area of CBT. (progressive total) Sub-component 3.3. Institutional Development Plan 4. Share of lyceums/colleges n.a. Not Not less Semiannual Implementation MES / PMU that received grants (out of less than Progress Report total number). than 10% 5% 5. Share of private TVE n.a. Not Not less Semiannual Implementation MES / PMU institutions that received less than Progress Report grants (out of schools that than 35% received grants). 35% 6. Share of schools that n.a. 50% 90% Semiannual Monitoring Studies MES / PMU implement the grants in (e.g., regular accordance with the targets set operational audit); in their IDPs (out of schools Implementation that received grants). Progress Report 7. Share of teachers in each n.a. 0 2­5% 2­5% Annual Implementation MES / PMU project school trained in Progress Report developing and delivering new CBT programs. 8. Share of project schools that n.a. 0 25% Up to Semiannual Implementation MES / PMU planned and introduced 100% Progress Report programs that use competency-based curriculum. n.a. = Not applicable. TBD = To be determined. 44 IMPACT EVALUATION PLAN Evaluation Objectives and Components The Impact Evaluation Plan for the TVEM Project will assess the results of the intervention and draw lessons in project design and implementation that can guide future replication efforts under a follow-up operation or state programs. The proposed evaluation comprises three components: (a) a baseline study, (b) a review of project design and implementation, and (c) an impact assessment. Baseline study The baseline study will describe the initial condition of the project before the interventions in the reform areas of the Project (for example, school management capacity, teaching and training capacity, and teaching and training facility). The study will be carried out during the first year of project implementation. Review of project design and implementation The review will examine the design and implementation process of the project. In conjunction with impact assessment, the review will identify the ways that variation in the current design and implementation process might affect project impacts. Such information is essential for replication and scaling up of similar future interventions. Impact assessment The impact assessment will be conducted after the intervention is completed. The main questions the evaluation will answer are: 1. Have expected final outcomes/Project Development Objectives been achieved? 2. What are the reasons for achieving or not achieving the outcomes? Are changes attributable to the intervention? 3. What lessons and experiences can be drawn from the Project? In addressing evaluation questions 1 and 2, the evaluation will test the main hypothesis of the intervention, that is, whether the project has made significant contributions in supporting TVET institutions to raise the relevance, quality, and efficiency of the technical and vocational education. This hypothesis will be further detailed in the TORs for impact evaluation and divided into sub-hypotheses for testing. This hypothesis and sub-hypotheses will be tested through analyzing changes in school operation and performance before and after for the schools supported under the project and between the project schools (treatment group) and non-project schools (control group/comparison group). Evaluation Methodology Evaluation design 45 The impact evaluation seeks answers to questions regarding the effectiveness of the intervention and establishment of a causal link between the intervention and results. Ideally, answering such a question requires an experimental design in which all eligible schools will be randomly allocated into project (treatment) or non-project (control) schools and will be followed for the entire intervention period. Although experimental design is considered the most robust evaluation methodology, it is not completely feasible in the current project since all project schools will be determined as a result of competitive selection. Given this limitation, the evaluation proposes two designs: (i) Before-and-after design A before-and-after design will be applied to estimate project effectiveness, especially on school operation, performance, and quality of education and training. Most school reform data will be collected routinely during the study period, such as the number of newly introduced competency-based training (CBT) programs; instead of looking at a given outcome at two points in time, the evaluator can look at the outcome indicator at several points in time. This strategy may improve the usefulness of classical before-and- after design and may also help draw causal inferences. (ii) Quasi-experimental design (matching) Since the current intervention has partial coverage, it is possible to construct a comparison group using the sub-population of non-project schools that is most similar to the treatment group (project schools). Because the time span of the intervention is rather lengthy (up to three years), it may be possible that some school characteristics may not hold constant during the study period; for example, some non-project schools may also, through different mechanisms, initiate similar school reform. In realizing these possibilities and in the absence of randomization, it is proposed that a group of non- project schools be identified according to a set of school characteristics retrospectively to match with the project schools. Meanwhile, students from the project schools will be matched with students from non-project schools (using propensity score matching based on some student characteristics). These two matching processes serve as a correction strategy and will help the evaluation eliminate external influences and correct for the selection biases. Data collection and analysis The evaluation will collect information from two levels of sources: various stakeholders including students, employers, school teachers, and the project staff as the primary data source; and school administrative systems as the secondary data source. It will employ a mix of qualitative and quantitative methodological approaches to collecting data needed for analysis. Some possible data sources and collection methods may include, but not be limited to: (i) Baseline survey A survey will be conducted with project and non-project schools to establish a baseline in the reform areas of school management capacity, teaching and training capacity, and teaching and training facility. (ii) Analysis of school administrative data 46 Time series for school operation data (availability of CBT programs, qualification of teachers, management and governance arrangements, and so forth), and early student learning outcome data (for example, results of the graduation tests and qualification certification tests) will be analyzed and compared for both project and non-project schools at the end of the project. The analysis will control for school-level variables and external variables that appear to influence the outcome indicators (for example, local labor market fluctuation, social and economic status). Several levels of analysis will be carried out including: (a) trends over the project period for the project schools; and (b) differences observed in the impact study between the project and non-project schools, after controlling baseline data and school-level variables. Analysis of school operation data and student learning outcome data in relation to different interventions supported under IDPs will be conducted. (iii) Survey of employers Employers that have been recruiting earlier graduates from project and non-project schools will be surveyed to obtain their assessment and information on, among other things: (a) their perceptions on the relevance and quality of the school teaching and training provided; (b) their perceptions on the relevance of the school management and governance policies; and (c) their perceptions and assessment of the competency of students graduated from the CBT programs, non-CBT programs in the project schools, and from other non-project schools. (iv) Survey of graduates A sample of graduates from project schools will be surveyed. The sample will be stratified by several of the categories (to be determined later) and a sample size will be calculated to ensure the representation of the study population. The purpose of the survey is to obtain information on, among other things: (a) their perceptions on the relevance and quality of the school teaching and training provided; (b) their experience with CBT training; and (c) early evidence of economic benefits gained (for example, employment opportunity, salary, and promotion) as a result of receiving training and education. (v) Focus group discussion (FGD) with project schools' teachers Teachers involved in the school reform activity supported by the IDPs and CBT will be interviewed via FGDs at the end of the project to assess the benefits of the school reform, especially the benefit of adapting the CBT approach to students, to the teachers themselves, and to the school system, and their satisfaction with PMU staff capacity in project planning, management, and monitoring and evaluation. (vi) Interview with stakeholders Face-to-face or telephone interviews will be conducted at the end of the Project with various stakeholders including: School principals/key personnel for both project and non-project schools to obtain information on the changes that occurred during the study period in the areas of school management, administration, system and teacher training, and the contribution of the project or other interventions to the changes observed and lessons learned from the reform. 47 Project leaders and PMU staff to identify lessons and experience gained relating to project design and implementation. (vii) Desk review A desk review will be conducted of research publications, policy notes, and other products produced during the Project to identify and summarize evidence of enhanced capacity at the local and national levels in terms of policy development and utilization of policy and research findings in supporting the decision-making process. Project documentation/records on project implementation will be reviewed at both the school and PMU levels to identify good practices and summarize useful lessons. (viii) Case studies Case studies may be employed to highlight lessons or good practices as learned from interviews with school principals/key personnel, teachers, and project implementation and management teams. Evaluation Approach A participatory approach to the impact evaluation is highly recommended. It would be useful to consult with project stakeholders such as school principals, students, teachers, school boards, industrial and regional councils, and policymakers about the evaluation design, data collection instruments, and survey methodology in order to achieve maximum and balanced contributions. 48 Annex 4: Detailed Project Description KAZAKHSTAN: TVEMP 1. Component 1: Develop Standards and Quality Assurance for Technical and Vocational Education (TVE). This component would establish the foundation for the national TVE system acknowledging qualifications and different types of learning, including a qualification framework, occupational standards, qualification assessment and recognition mechanisms, and introduction of a mechanism for independent voluntary institutional accreditation. The component would be implemented through intensive consultations with all types of educational and research institutions, employers, and other stakeholders. Employers would take the leading role in developing occupational standards. Sub-Component 1.1: Development of the Concept for a National Qualifications System (NQS) and a National Qualifications Framework (NQF). This sub-component would support development of a Concept for NQS and NQF as outlined in the table below. Objectives Activities 1. To develop the Concept 1.1 Review Models: Complete a desk review of the European Qualifications Note of an NQS and Framework and qualifications systems of other countries. develop NQF for 1.2 Implement Study Tour: Initiate a two-week study tour of 2 countries for Kazakhstan. up to 15 representatives of Ministry of Education and Science (MES), Ministry of Labor and Social Protection of the Population (MLSP) academic institutions, employers, occupational associations, and national/regional TVE representatives. 1.3 Develop Concept Note of the NQS: Develop a draft Concept Note for NQS for Kazakhstan, and discuss with stakeholders (that is, universities, colleges, lyceums, schools, employers, occupational associations, and political leaders). 1.4. Develop the NQF: Develop a draft NQF for Kazakhstan and discuss with stakeholders. 2. To implement the NQS 2.1 Policies: Develop necessary changes to laws and administrative regulations and NQF. for application of the NQS and NQF. 2.2 Dissemination: Develop public information materials to explain the rationale and content of the NQS and NQF, and disseminate to stakeholders and implementing agencies via media and training programs at the national, regional, and institutional levels. Sub-component 1.2: Occupational Standards and Qualifications Assessment. This sub- component would contribute to the NQS by developing occupational standards and upgrading qualifications assessment and recognition mechanisms. i. Occupational standards. National Occupational Standards (NOS) set out measurable performance outcomes to which an individual is expected to work in a given occupation. In particular, NOS set out the skills, knowledge, and understanding required to perform competently in the workplace. NOS should comprise: Performance criteria, defining in detail what is expected of the individual. 49 Knowledge and understanding, that is, what the individual needs to know and/or understand to enable them to meet the performance criteria. Scope, specifying the range of circumstances or situations that have a critical impact on the activity when carrying out the performance criteria. Elements. An NOS can be divided into two or more discrete elements that describe the activities the person has to carry out. Values and behaviors are the personal attributes an individual is expected to demonstrate within the NOS. Develop occupational standards will be led by employers (through employers' associations and or sector skill councils) and will be used to: o Develop educational standards and curriculum materials for vocational education; o Evaluate and certify qualifications of employees and graduates of TVE schools; and o Personnel management, for example, development of motivation and remuneration policies, job descriptions, personnel selection, and probation. Detailed procedures for the occupational standards development are outlined in Figure A4.1 at the end of the annex. ii. Qualifications assessment. The sub-component would finance continuous technical assistance to upgrade the qualification assessment tools available in the Republican Scientific and Methodological Center for TVE Development and Qualifications Award in order to match these tools with employer requirements reflected in the occupational standards. The tools would be revised in a sequence that corresponds to the order and pace of occupational standards preparation, as outlined in the following table. Objectives Activities 1. To define 1.1 Review Methodology: Examine administrative and technical procedures, and administrative and actual standards developed in other countries and complete short study tours (within technical procedures Component 1.1. with the group on NQS) to the selected countries to review procedures. and develop occupational 1.2 Develop National Methodology: Develop administrative structure, technical standards in the support requirements, and procedures for development of occupational standards in priority sectors. Kazakhstan. Development of the methodology will include several stages: Creation of a National Register for Occupational Standards; Development of procedures for occupational standards development; Development of procedures for occupational standards updating; Development of procedures for occupational standards dissemination; Development of approaches to define: (i) Occupational functions; (ii) Levels of significance for each particular function; (iii) Knowledge and skills needed to perform each function; (iv) Levels of qualifications appropriate for performing the functions; Development of a generic occupational standard (model standard). 1.3 Stakeholder Review: Present methodology to technical stakeholder representatives, particularly employers, for review. 1.4 Prioritizing: Define priorities in developing occupational standards and identify a group of standards that can be purchased internationally and a group that will be 50 Objectives Activities developed locally. 1.5 Deliver Training and Technical Assistance: The experts involved in the methodology review and development will deliver training and ongoing technical advice (where appropriate) to the developers of the occupational standards. 1.6 Purchase of International Occupational Standards: According to the agreed list, purchase about 20 occupational standards and further adaptation/localization. Purchase of the occupational standards will include several stages, which are discussed in detail at the end of this Annex. 1.7 Develop Standards: According to the agreed list, develop about 100 standards. Development of the occupational standards will include several stages, which are discussed in detail at the end of this Annex. 2. To review and 2.1 Review Methodology: Examine existing assessment methodology and tools update qualification currently in use at the National Center for Qualification Assessment and Research in assessment tools with TVE (The Center), in relation to international benchmarks, and make recommendations direct links to on how methodology can be strengthened, including basing assessments on occupational occupational standards, not curriculums. standards. 2.2 Revise Procedures and Assessment Tools: Provide training, technical advice, and assistance to the staff of the Center is in charge of updating assessments procedures and tools. (Staff of the Center will revise the procedures and tools without additional financing from the Project.) When the standards for particular occupations are not (fully) developed by the time of the assessment tools revisions, requirements of employers for qualifications will be studied and used as a basis for the tools revision. 2.3 Training: Train additional evaluators from the Center in applying revised tools. The training will be delivered by the experts that developed new procedures and updated the tools. 2.4 Assessment: Conduct assessment and certification of qualifications using new procedures and tools under existing mandate and recurrent financing of the Center (no loan funds will be required). Analyze results of assessment and certification process and provide recommendations on the TVE policy implications. Sub-component 1.3: Institutional Accreditation. This sub-component would contribute to the NQS by introducing institutional accreditation procedures. The Project would support establishing a mechanism for independent voluntary institutional accreditation external to the Ministry of Education and Science (MES) with a focus on outputs and outcomes of education and training delivered by TVE institutions. The MES would use results of such accreditation to manage the TVE network and allocate resources targeted to support institutional development rather than daily operation of TVE schools. Requirements for such accreditation procedures would be developed based on the best international practice and benchmarks. For accreditation an institution would be selected through competition to develop and new institutional accreditation mechanisms and stakeholders would be informed about the new approach to institutional accreditation. See table below for details. Objectives Activities 1. To develop 1.1 Review Methodology: Examine the existing TVE accreditation and licensing institutional process (used by the MES and regional education departments) for private and public accreditation lyceums and colleges. Compare with international benchmarks (that is, the procedures to improve International Standards Organization [ISO], the International Labour Organization 51 Objectives Activities internal and external [ILO], and United Nations Educational, Scientific and Cultural Organization efficiency of TVE [UNESCO]) for accreditation and licensing. Develop requirements for such investments. accreditation procedures. 1.2 Develop Accreditation Procedures: Develop and introduce procedures for independent voluntary institutional accreditation. Adjust, as necessary, laws, regulations, and procedures to update TVE accreditation and licensing procedures. 1.3 Implement: Disseminate updated procedures to all stakeholders and conduct accreditation of volunteering colleges (not less than one from each priority sector) according to the new procedures. The accreditation will be conducted under existing recurrent financing or using grants for the Institutional Development Plan (IDP) implementation of the volunteering colleges (no additional loan funds will be required). Component 2: Strengthen Governance, Management, and Financing for TVE Sub-Component 2.1: Establishing a Basis for Reforming TVE Structure, Governance, and Management. This sub-component seeks to support Councils to ensure their effective operation, modernize the governance and management policies within TVE by identifying priority problems and constraints in the existing system, studying and discussing alternatives, and planning for implementation of new structures, as outlined in the following table. Objectives Activities 1. To promote the 1.1 Develop Structure and Functions: Expert services to help design the structure effective operation of and functions of the councils at the various levels, including criteria for membership. national, regional, 1.2 Advice: Provision of ongoing expert advice to the Councils. sectoral, and institutional councils. 2. To analyze the 2.1 Review Models: Complete an in-depth review of the present system of TVE present TVE governance and management including teacher compensation policy and practice; governance and complete a desk review of alternative models, and define pros and cons of management system alternative structures and policies with respect to governance, and identify alternate administration/management, teacher compensation, financing, delivery, and evaluation policies for of TVE at the national, regional/municipal, and institutional levels. Kazakhstan. 2.2 Implement Study Tour: Undertake a study tour for TVE national/regional/institutional managers and employer representatives. 2.3 Policies: Generate alternative structures, systems, and procedures, with arguments for and against and implications in terms of legislation, regulations, and resources. 2.4 Dissemination: Develop and distribute public information materials on the options and hold consultative workshops with stakeholders. 3. To plan for 3.1 Policy Decisions. Based on wide consultation with stakeholders, government implementation of takes decisions on reforms. reforms in governance 3.2 Implementation Planning. Prepare detailed action plans for implementation of and management of policy decisions, including activities, steps, responsibilities, timing, and costs. TVE. Sub-component 2.2: Introducing Formula-based Financing: Improve the efficiency of use of TVE funds by designing and pilot-testing formula-based approaches to TVE financing, as outlined in the table below. Objectives Activities 1. Design: To identify 1.1 Review Models: Complete a desk review of alternative ways to channel 52 alternate policies for formula-based financing for TVE institutions, including arguments for and against formula-based each. funding of TVE 1.2 Implement Study Tour: Initiate an in-depth study tour for MES to examine the operational costs and process and procedures of formula-based financing. develop revised 1.3. Policy Options: Present policy options pros and cons and complete requirements polices for for each for discussion and policy decisions. Kazakhstan. 1.3. Policy Decision: Adopt the specific model for implementation. 1.4 Identify Implementation Requirements: Prepare implementation plan for pilot- testing in one region. 2. Test: Complete a 2.1. Select Participating Region: Develop criteria for selecting one region for pilot test of formula- implementation of the pilot scheme. based financing in one 2.2. Data Collection: Collect data at the institution level as a basis for formula-based region, make allocations. Aggregate and analyze the data to determine average rates. adjustments, and then 2.3 Training: Train MES personnel and representatives of regional departments in prepare a plan for pilot oblasts. nationwide 2.3 Pilot Test: Conduct experimental calculation of school budgets in accordance implementation. with the new per capita formula. 2.4 Evaluation and Adjustments: Evaluate the implementation of formula-based financing in the pilot region and make necessary adjustments in the technical implementation procedures. 2.4 Implementation Planning: Prepare detailed plan for nationwide implementation of revised budgeting of TVE sector using formula-based approach, including necessary changes in legislation and regulations. 2.5. Decision to Proceed. Sub-Component 2.3: Establishing TVE Monitoring & Evaluation (M&E). Develop and implement effective M&E of the TVE system that will provide an empirical basis for policy development and better implementation of TVE reforms, as follows. Objectives Activities 1. To design and 1.1 Develop Monitoring Methodology: Develop methodology for regular monitoring implement a regular and Mid-Term Review. monitoring system for 1.2 Conduct Regular Data Collection, including operational monitoring of the the project. progress in implementing IDPs at the school level. 1.3 Prepare Regular Monitoring Reports, including MTR report. 2. To deliver Impact 2.1 Design: Design an impact evaluation. Evaluation: Complete 2.2 Baseline Data Collection: Implement baseline data collection. an analysis of the 2.3 Impact Data Collection: Implement collection and analysis of impact data. impact of the project 2.4 Dissemination of Findings: Preparation of evaluation report noting achievements on relevance, quality, and shortfalls, with analysis of reasons and recommendations for future actions. and efficiency of TVE in Kazakhstan. Sub-Component 2.4: Support for Project coordination, implementation and management. Establish a PMU to manage the Project and act as the main liaison with the World Bank, as follows. Objectives Activities 1. PMU established, 1.1 Coordination: Help coordinate project implementation through: (a) convening staffed, and trained to regular meetings of the Technical Coordinators to discuss common issues; (b) coordinate project ensuring that the Project is being implemented in accordance with legal agreements execution and manage with the World Bank; and (c) assisting with Project implementation as requested by project resources. the technical agencies involved including establishing and convening a tripartite project steering committee composed of representatives of the Government, 53 employers, and other stakeholders that can provide guidance for overall project implementation. 1.2 Training: Train PMU members on Bank procedures. 1.3 Procurement: Assist the MES in managing all procurement under the Project, in cooperation with the technical agencies. 1.4 Financial Management: Assist the MES in setting up and operating a financial management system based on technical agreements reached with the World Bank. 1.5 World Bank Liaison: (a) Manage disbursement of all World Bank Loan proceeds; (b) monitor availability of Government's counterpart contribution to the Project; (c) monitor implementation progress of Project components; and (d) prepare quarterly progress reports in accordance with a project progress reporting format acceptable to the Bank. 1.6 Audit Arrangements: Organize annual audits to be made according to Bank requirements and agreements with the Government. 1.7 Accounts Maintenance: Maintain Project accounts acceptable to the Bank. 2. Establish Project 2.1 To select core members of the Panel--6 experts in specific fields Advisory Panel representing industry, employers and government agencies. The PMU proposes candidates; the Project Steering Committee approves the core Panel members. 2.2 To create a pool of experts in specific fields as extension of the Panel. The PMU proposes candidates; the Panel approves the pool. 2.3 Regularly update the pool. 2.4 Panel provides independent technical advice to the PMU, Steering Committee, and MES. Component 3: Strengthen Skills Delivery Capacity of TVE Institutions. This component is designed to begin a process of gradual consolidation and integration of the programs currently offered in the professional lyceums and colleges toward an integrated, broad, competency-based curriculum. The component would finance: (a) support for development of educational standards and core curriculum (on the national level) as the basis for the institutions to design new programs, (b) preparation and implementation of teacher-training programs for developing and delivering a competency-based and modularized curriculum (in six select regional teacher- training centers), and (c) the development and implementation of Institutional Development Plans (IDPs) to enhance skills delivery at TVE institutions (lyceums and colleges). IDPs would be selected for financing through competitive bidding. Sub-component 3.1: Support for development of education standards, core curriculum, and program. This component would finance revising and developing educational standards and a core-competency-based curriculum. These educational standards and the core curriculum will be developed on the national level using revised procedures and approaches. The core curriculum will guide the TVE institutions to design new programs as part of IDP implementation. Occupational standards for selected industries developed under Component 1 will be used as a basis for standards and core curriculum development. Sub-component 3.2: Teacher-training programs in developing and delivery of competency- based and modularized curriculum. This sub-component would finance Sub-Grants (goods and consultants' services) in support of development of in-service teacher-training programs for developing and introducing the competency-based and modularized curriculum. The programs will be established and delivered in select regional teacher-training centers. Each of the centers would prepare a proposal for developing a program to be reviewed by the Projects' Advisory Panel (PAP) and approved by MES. The program should include teacher training in: (a) 54 development of CBT-based curriculum, (b) development of key competences in the technical specialization, and (c) pedagogical strategies in implementing the CBT-based curriculum. The criteria for evaluation of the proposals are specified in the Project Operational Manual. Proposals would range in size between US$100,000 and US$200,000. PAP will provide support to the centers in the program development. This sub-component would cover the costs of developing the programs. Delivery of the training programs to teachers will be financed from the funds allocated to the schools through Institutional Development Plan (IDP) grants. Sub-component 3.3: Institutional Development Plans. This sub-component would finance Sub-Grants (goods, works, consultants' services and training, (staff development)) to be awarded on a competitive basis to public and private provider institutions for implementing their IDPs. Each applicant institution will undertake a strategic planning exercise for developing an IDP with close involvement of and in consultation with the relevant industries, employers, and other stakeholders, and supported by technical assistance from the project's working groups. The IDP would be the basis for the institutions to compete for financing. Depending on the strategies to be followed, each IDP could contain an integrated package of CBT modular program development, professional development for teachers and administrators, alignment of assessment instruments and processes, upgrading of training equipment and technology, development of learning and teaching materials, e-learning modalities, strengthening of institutional governance, and provision of services for career guidance and counseling. Implementation of the IDP would be closely sequenced with the development of the standards and National Qualifications Framework (NQF) described in Component 1. Selected criteria and procedures are detailed in the Operational Manual. The level of financing of each application would depend on the evaluated IDP, but winning applications are expected to receive between US$200,000 and US$500,000. Each applicant institution would co-finance up to 30 percent of the total cost proposed. Each winning bid would constitute a subproject. Each application for subproject financing would specify selected priority CBT modular programs to be developed that cater to the provision of the competences at both the secondary and postsecondary levels leading to a relevant qualification. Technical assistance would be provided to institutions competing for subproject financing. The assistance would cover strategic planning, new program and curriculum development, and related technical areas. The project's Advisory Panel would review the IDPs in accordance with the criteria and procedures established in the Operational Manual. Approval would be made by the MES and signed by the Director of the PMU. See details in the table below. Objectives Activities 1. Develop educational 1.1 Review methodology for educational standards and core curriculum standards and core development. curriculum in line with 1.2 Revise existing and develop new education standards in line with occupational standards. occupational standards. 1.3 Revise existing and develop new core curriculum in line with new education standards. 2. Develop the teacher- 2.1 Review the proposals for new teacher-training programs from the 55 Objectives Activities training program. teacher-training centers. 2.2 Assist in improving the proposals. 2.3 Training and support for program developers to be provided by working groups and consultants. 2.4 Training Programs to be developed include upgrading of knowledge of technical subjects and methodology. 2.5 New training programs to be pilot-tested before being implemented. 3. Develop and implement 3.1 Selection of IDPs for support under the project. the IDP. 3.2 Technical Assistance to assist in undertaking strategic planning and enhancing the IDP. 3.3 Training to be provided for the teachers developing new CBT modular training programs. 3.4 CBT modular programs to be developed based on the new occupational and educational standards. 3.5 Training of additional teachers (in addition to developers of the programs that are already equipped with skills to train students using new programs) from project schools to be provided for implementation of the CBT curriculum. 3.6 Equipment/Technology/Materials to be selected based on the new program and in line with recommendations available in core curriculums. 3.7 Equipment/Technology/Materials to be procured centrally and provided to project schools. 56 Figure A4.1: Occupational Standards Development Options Option 1 ­ Purchase Program Industry-specific Working Group Industry Associations Identify Occupational Finalizes Draft and ATAMEKEN Standards to be Inputs from MOL and Reviews and other sources Occupational Purchased Standard(s) Recommends Changes to Make Standards More contractor may be Acceptable to used to identify Employers Identify Sources for sources Purchase of Returned for Standards further revision Revised Draft Occupational Standards are Standards are Reviewed by Project Review Sample of Reviewed and Steering Committee Standards to Changes are Made to (PSC) Determine Quality Reflect Local Needs and Appropriateness and Conditions Gaps in the Required Contact and Standards for Each No Negotiate Terms for Industrial Grouping PSC Provides Purchase and (Cluster) are Endorsement Delivery Schedule Identified for Development using Options 2 or 3 It may be necessary to set up more than one Yes Industry-specific working group per Working Groups industry Enter into No Established, Some representatives No Purchase Comprised of from the Project Advisory MOL and Line Agreement Curriculum Design Panel may be asked to Ministries Review Specialists and serve on these ad hoc & Endorse Knowledgeable committees Yes Persons from Business/Industry Occupational Accept Delivery of Standard is Sent to Standards Curriculum Documents in All Occupational Development Team Available Format and Standards are for Conversion into Process Payment Organized into Course of Study Industry (cluster) Groupings All Occupational Standards are Draft Curriculum is No Translated into Piloted in One or Appropriate More TVE Institutions Language & Format Occupational Standard and Curriculum Accepted 57 Occupational Standards Development Options Option 2 ­ Local Development Approach Often these Identified Inputs may be from Identify MOL and other Occupational Occupational Yes Standards will be those sources; possible use PAP Provides Standards to be of a contractor to not found through Developed Endorsement Standardized Option 1 and Option 3 identify sources format provided by PMU Identify Task Lists and No Information Task Lists and Performance Resources to Performance Requirements Support Requirements are Converted to Development are Reviewed Standardized and Endorsed by Format for PAP Occupational Standards PMU Obtains Returned for Resources and Further Translates into Revision Appropriate Language Task Lists and Performance Draft Standards Requirements are Reviewed by are Reviewed by Project Steering Development Business/industry Committee (PSC) PMU Organizes Team representatives must be Development intimately knowledgeable Team, Including about the occupation Curriculum being studied Development Specialists and A contractor may be Collected Data is Representatives used in place of the Organized and No from Industry development team Analyzed into PSC Provides Simple to Endorsement Complex Task Lists w/ Some representatives Development Associated from the PAP Team Leader Performance Serve on this Ad Hoc Organizes Short Requirements Committee Workshop to Required by Develop Business/ Yes Occupational Industry Analysis Data Occupational Collection Standards are Instrument Sent to Curriculum Team Leader Development Organizes Data Team for Collection Conversion into Occupational Activities in a Course of Study Analysis Data Reasonable Collection Sample of Instrument is Businesses/ Reviewed by Industries PAP Draft Curriculum is Piloted in One No No or More TVE Institutions A contractor may be used in place of the MOL & No development team PAP Provides Yes Line Ministries Endorsement Occupational Review & Standards and Endorse Curriculum Accepted 58 Occupational Standards Development Options Option 3 ­ Obtained From Local Sources Often these Identified Identify Occupational Occupational Inputs may be from Revised Draft Standards will be those Standards to be MOL and other Standards are not found through Developed sources; Reviewed by Option 1 and Option 2 Project Steering Committee (PSC) Contractor may be used Identify Local to identify and obtain Firms that local occupational Employ Persons standards in the Desired Occupations No PSC Provides Yes Endorsement Standards may be Obtain all obtained directly from Occupational PAP Provides firms or from industry Standards Endorsement Yes associations that represent Currently in Use No those firms within the Firms ­ Translate if Necessary Standard format provided MOL & Line by PMU Ministries Business/Industry Review & PMU Organizes representatives must be Revised Draft, in Endorse Development intimately knowledgeable Team, Including Standard Format about the occupation Curriculum is Reviewed and being studied Development Endorsed by Specialists and PAP A contractor may be used Representatives instead of the from Industry development team Occupational Standards are Collected Data Sent to are Organized, Curriculum Some representatives Team Leader Analyzed, and Development from the PAP Organizes Short Combined into Team for Serve on this Ad Hoc Workshop to Existing Returned for Conversion into Committee Review Existing Occupational Further Course of Study Occupational Standard Revision Standards Documents Draft Curriculum is Piloted in One Team Leader or More TVE Organizes Data Returned for Institutions All Available Collection Further Occupational Activities in a Revision Standards for a Reasonable Single Sample of Occupation Businesses/ are Rationalized Industries Occupational into a Single Standards and Occupational Curriculum Standard and Accepted Analyzed to Determine Need for Additional Business/industry Inputs from associations and Business/ ATAMEKEN may provide Industry access to sample of firms required for data collection 59 Curriculum Development Model Inputs from MOL and Option 1 ­ Competency Based other sources; For each task/ Obtain Occupational subtasks listed, Standards to support prepare one or more curriculum learning/teaching Identify the development effort activities that will sequencing required enable the student to for all teaching learn and retain the activities cognitive, Check to make sure psychomotor, and that the Occupational affective skills for that Standards are Yes activity approved by MOL, Contractor conducts MOE, and workshops w/ appropriate line curriculum Contractor development ministry conducts workshop - review specialists and task/subtask lists and teachers to write performance-based teaching plans for Identify the objectives (often called performance each task and No curriculum guide) to ensure requirements for all subtask to be taught relevance to Occupational ­ include activities in the Standard Occupational performance Standards objectives, teaching activities, equipment/ tools to be used, materials to be used, expected products/ For each identified outputs from Prepare RFP and job/task and students, and method issue for contractors subtasks, prepare of individual to prepare curriculum one or more competency performance-based assessment objectives that reflect agreed upon Occupational Standards Develop Pre- and Post-assessment Instruments Single or Review proposals Logical Groups of and select contractor For each task, list Performance subtasks and lower- Objectives level teaching elements for each occupation Local Employers and Skilled Employees to Prepare summary Provide Input ­ document. Often absolute requirement called a `course of study' Contractor-based on Negotiate contract Curriculum Plan, list and award required tasks/Jobs contract No in occupation Yes MOE, MOL & Stakeholder Yes review and endorsement No Contractor develops curriculum Review and development plan for Concurrence by Yes the occupation based PAP School initiates VTE on the Occupational Program Standards No 60 Annex 5: Project Costs KAZAKHSTAN: TVEMP Kazakhstan International Bank Technical and Vocational Education Modernization for Reconstruction The Government of Components by Financiers and Development Kazakhstan Total Local (Excl. Duties & Amount % Amount % Amount % For. Exch. Taxes) Taxes A. Develop Standards and Quality Assurance for TVE Development of the Concept of NQS and NQF 300,379.1 88.0 40,960.8 12.0 341,339.9 1.0 130,284.0 170,095.1 40,960.8 Occupational Standards and Qual Assessment 1,719,810.4 88.0 234,519.6 12.0 1,954,330.0 5.9 207,379.4 1,512,431.0 234,519.6 Institutional Accreditation 134,088.2 88.0 18,284.8 12.0 152,372.9 0.5 97,602.4 36,485.7 18,284.8 Subtotal Develop Standards and Quality Assurance for TVE 2,154,277.7 88.0 293,765.1 12.0 2,448,042.8 7.4 435,265.8 1,719,011.9 293,765.1 B. Strengthen Governance, Management & Financing for TVE Modernization of TVE Structure, Management and Governance 259,666.9 88.0 35,409.1 12.0 295,076.0 0.9 124,186.6 135,480.3 35,409.1 Formula-based Financing 236,940.0 88.0 32,310.0 12.0 269,250.0 0.8 114,749.0 122,191.0 32,310.0 Monitoring and Evaluation 346,570.5 88.0 47,259.6 12.0 393,830.1 1.2 319,343.7 27,226.8 47,259.6 Implementation Support 540,595.0 88.0 73,717.5 12.0 614,312.6 1.8 29,164.1 511,431.0 73,717.5 Subtotal Strengthen Governance, Management & Financing for TVE 1,383,772.5 88.0 188,696.2 12.0 1,572,468.7 4.7 587,443.4 796,329.1 188,696.2 C. Strengthen Skills Delivery Capacity for TVE Systems Development of Core Curricula and Educational Standards 1,234,099.2 88.0 168,286.2 12.0 1,402,385.4 4.2 - 1,234,099.2 168,286.2 Institutional Development Plans 23,577,307.8 88.0 3,215,087.4 12.0 26,792,395.2 80.7 17,382,814.7 6,194,493.1 3,215,087.4 TVE Teacher Training Program 877,537.6 88.0 119,664.2 12.0 997,201.8 3.0 653,917.7 223,619.9 119,664.2 Subtotal Strengthen Skills Delivery Capacity for TVE Systems 25,688,944.6 88.0 3,503,037.9 12.0 29,191,982.5 87.9 18,036,732.4 7,652,212.2 3,503,037.9 29,226,994.7 88.0 3,985,499.3 12.0 33,212,494.0 100.0 19,059,441.6 10,167,553.2 3,985,499.3 61 Kazakhstan (Tenge) (US$) Technical and Vocational Education Modernization % % Total Components Project Cost Summary Foreign Base Local Foreign Total Local Foreign Total Exchange Costs A. Develop Standards and Quality Assurance for TVE Development of the Concept of NQS and NQF 26,055,750 18,632,250 44,688,000 177,250 126,750 304,000 42 1 Occupational Standards and Qual Assessment 216,384,000 29,767,500 246,151,500 1,472,000 202,500 1,674,500 12 6 Institutional Accreditation 6,914,880 13,665,120 20,580,000 47,040 92,960 140,000 66 - Subtotal Develop Standards and Quality Assurance for TVE 249,354,630 62,064,870 311,419,500 1,696,290 422,210 2,118,500 20 7 B. Strengthen Governance, Management & Financing for TVE Modernization of TVE Structure, Management and Governance 21,234,150 17,794,350 39,028,500 144,450 121,050 265,500 46 1 Formula-based Financing 19,477,500 16,537,500 36,015,000 132,500 112,500 245,000 46 1 Monitoring and Evaluation 10,085,670 45,774,330 55,860,000 68,610 311,390 380,000 82 1 Implementation Support 70,236,600 3,998,400 74,235,000 477,800 27,200 505,000 5 2 Subtotal Strengthen Governance, Management & Financing for TVE 121,033,920 84,104,580 205,138,500 823,360 572,140 1,395,500 41 5 C. Strengthen Skills Delivery Capacity for TVE Systems Development of Core Curricula and Educational Standards 171,960,600 - 171,960,600 1,169,800 - 1,169,800 - 4 Institutional Development Plans 1,125,349,680 2,379,048,000 3,504,397,680 7,655,440 16,184,000 23,839,440 68 81 TVE Teacher Training Program 42,336,000 89,964,000 132,300,000 288,000 612,000 900,000 68 3 Subtotal Strengthen Skills Delivery Capacity for TVE Systems 1,339,646,280 2,469,012,000 3,808,658,280 9,113,240 16,796,000 25,909,240 65 88 1,710,034,830 2,615,181,450 4,325,216,280 11,632,890 17,790,350 29,423,240 60 100 Physical Contingencies 52,339,056 109,346,244 161,685,300 356,048 743,852 1,099,900 68 4 Price Contingencies 219,053,457 -56,205,875 162,847,582 2,164,114 525,240 2,689,354 20 9 1,981,427,343 2,668,321,819 4,649,749,162 14,153,052 19,059,442 33,212,494 57 113 62 Annex 6: Implementation Arrangements KAZAKHSTAN: TVEMP Overview The Technical & Vocational Education Modernization Project (TVEMP) will be implemented by the Project Management Unit (PMU) established under the Ministry of Education and Science (MES) (Figure A6.1). The PMU will ensure the participation of stakeholders at multiple levels, but will ultimately be responsible for the day-to-day management of the project. The PMU will be staffed by consultants with the expertise in the specific fields, selected according to the agreed term of reference (TORs). The PMU will undertake the following functions: Coordination: Coordinate project implementation through: (a) ensuring that the Project is implemented in accordance with legal agreements with the World Bank; and (b) assisting with Project implementation as requested by the technical agencies involved, including establishing and convening a Project Advisory Panel composed of local experts who would advise the PMU on specific issues and support for convening the meetings of the Project Steering Committee). Procurement: Manage all procurement under the Project, in cooperation with the technical agencies. Financial Management: Set up and operate a financial management system based on technical agreements reached with the World Bank. Monitoring and reporting: Monitor Project progress by component and report to the Project Steering Committee and the World Bank. World Bank Liaison: (a) Manage disbursement of all World Bank Loan proceeds, (b) monitor availability of Government's counterpart contribution to the Project, (c) monitor implementation progress of Project components, and (d) prepare quarterly progress reports in accordance with a project progress reporting format acceptable to the Bank. Industry Liaison: Work with sector councils and employer associations at the national and regional levels that bring together government representatives, employers, and large businesses. Audit Arrangements: Organize annual audits to be executed according to Bank requirements and agreements with the Government. Accounts Maintenance: Maintain Project accounts acceptable to the Bank. 1. The PMU would comprise the following six consultants hired on a competitive basis by effectiveness: (a) PMU Director (b) Coordinator for Components 1 and 2 (c) Coordinator for Component 3 (d) Financial Management specialist (e) Procurement specialist 63 (f) Administrative Assistant/Translator 2. The Project Steering Committee comprising representatives of government agencies, industries, and employers, will be chaired by the MES Executive Secretary. The Steering Committee will meet not less than monthly to oversee Project progress and resolve outstanding policy-related issues. The Steering Committee will monitor progress against agreed performance indicators. The detailed TORs for the Committee are shown in the Project Operational Manual. 3. A Project Advisory Panel comprising local consultants would be established to help the PMU develop TORs, review the professional standards and educational programs development process, review the applications from the institutions under Component 3, and advise on specific, field-related issues. The project Advisory Panel will be established as a condition of effectiveness. 4. Professional Lyceums and Colleges: TVE institutions meeting eligibility requirements and competitively selected under Component 3 will be obliged to establish a Project group to coordinate the work with the PMU and report on the progress made. Figure A6.1: Organization for TVEMP Implementation under the MES Project Steering Committee (comprising MES, MLSP, other govt. agencies, ATAMEKEN, employers' associations) Chaired by the Executive Secretary of MES Deputy Director, TVE Department in MES Director, Project Management Unit Project Advisory Panel Finance & Policy & Subproject Administrative Procurement Coordination Accounting Systems Support Development (IDPs) Subproject Teams at Lyceums and Colleges that Win Bids 64 Annex 7: Financial Management and Disbursement Arrangements KAZAKHSTAN: TVEMP Executive Summary 1. The Ministry of Education and Science (MES), supported by the Project Management Unit (PMU), will be responsible for implementing the financial management (FM) function of the Technical & Vocational Education Modernization Project (TVEMP), including flow of funds, budgeting, accounting, reporting, and auditing. The MES has no prior experience implementing Bank-financed projects. However, the FM arrangements of the MES were reviewed during the missions in August 2009 and March 2010 and have been found generally satisfactory. In particular: (i) the MES has a reliable accounting system; (ii) the filing system allows keeping all supporting documentation in a well systemized manner; (iii) internal controls system is adequate and (iv) the MES is being audited by the Internal Control Department of the MOF and the Accounts Committee (SAI) on bi-annual basis. In addition the MES has recently established the Internal Audit Unit which is auditing activities of agencies, subordinate to the MES. Fiduciary Risk at the Project Level. Based on the FM assessment, fiduciary risk at the project level is assessed as substantial before mitigation and moderate after risk-mitigation measures. A number of actions have been agreed to further strengthen the FM arrangements, including development of an FM manual that will form part of the Project Operational Manual, the hiring of an FM consultant to take charge of project FM and disbursement functions and modification of the automated accounting system with the capacity to generate IFRs.. The first two measures are conditions of effectiveness of the Loan; the third measure is required no later than thirty (30) days after effectiveness. 2. Fiduciary Risk at the Country Level. Based on the Public Expenditure and Financial Accountability assessment, issued in May 2009, the overall fiduciary risk in Kazakhstan is assessed as substantial. The key reasons are: (a) inadequate capacity of core control and supervisory agencies performing audits within the public sector; and (b) although most of the basic laws are in place with respect to private and public sector enterprises, including state non- commercial organizations, financial reporting compliance remains a problem and authorities need to improve quality of auditing, monitoring, and supervision. In addition, several critical Public Financial Management elements, including internal controls, internal and external audit, and financial reporting, are still weak. These risks will be mitigated by establishing a PMU within the MES to be responsible for fiduciary aspects of the project. Conclusion 3. The overall financial management risk for the project is substantial before and moderate after mitigation measures. The project will also be implemented in an environment of high perceived corruption, but adequate mitigation measures are included in the design of the project to ensure that the residual risk is acceptable, including: (a) regular review of compliance with the internal control framework as described in the Project Operational Manual, (b) regular and timely reconciliation of bank accounts, (c) timely submission of IFRs that will be used to monitor overall financial activity of the project, (d) the project financial statements will be 65 audited by independent auditors and on terms acceptable to the World Bank, and (e) regular FM supervision and procurement prior and post-reviews will be conducted to ensure continued adequacy of the financial management and procurement arrangements. The table below summarizes the financial management assessment and risk ratings of this project. Risk Analysis Risk Analysis FM Risk Risk Mitigation Measures Residual Risk INHERENT RISKS Country Level Weak Public Financial Management S The MES, supported by the PMU, will maintain M institutions. an independent FM system and use independent auditors acceptable to the Bank to audit project accounts and commercial banks for designated accounts. Entity Level FM arrangements established by the S Action plan developed to strengthen MES FM M MES are not adequate or capable of systems, including modification of the providing effective FM support to the accounting system, and development of a Project. Project Operational Manual. Project Level Project is medium sized, with a S Implementation arrangements that allow close M commercial bank used for designated monitoring of activities under the project account. No sub-accounts will be used (including flow of funds, documents, and in the Project. However, the Project reports). Flow of funds, documents, and reports includes other beneficiaries such as between the MES and other beneficiaries will be grant recipients (e.g., schools). It is described in the Project Operational Manual and expected that the flow of documents compliance monitored closely. and reports will be complicated. OVERALL INHERENT RISK S M CONTROL RISKS Budgeting. The MES has adequate L The existing budgeting system will be used for L budgeting capacity. Budget is prepared the project. No additional measures required. according to the Budget Law in detail, which is necessary for project monitoring. Budget is also supported by the financing plan approved by the Ministry of Finance (MOF); this financing plan is split by sources of finance and months. Accounting. The accounting system S Accounting system (1C software) to be M used by the MES does not have modified for the Project needs, with inbuilt capacity to fully support project FM controls to ensure reliability and enhance requirements. . timeliness in processing transactions and financial reporting. Chart of accounts will be developed to facilitate tracking of project sources and uses of funds and capture project- specific transactions. Internal Controls, including Internal M Project FM/accounting policies and procedures M Audit. The MES has an adequate including internal control procedures applicable internal control system in place, with to the project to be documented in a manual and 66 internal control procedures described in compliance reviewed periodically by the Bank. the regulations of the MOF and internal orders of the MES. An Internal Audit Unit was recently established within the MES, which audits activities of agencies and enterprises subordinate to the MES. Funds Flow. The Treasury systems S Loan funds will flow via a single designated M cannot be relied on for timely account held in a commercial bank by the MES, processing of payments, especially to while government funds will flow through the foreign suppliers and consultants, and Treasury system. Designated account will be the MES does not have experience with reconciled regularly, with every replenishment Letters of Credit. application. Financial Reporting. Although the S The automated accounting system will be M MES currently prepares quarterly modified, with capacity to generate IFRs. reports on budget execution, such Financial reports to be generated by the reports are prepared manually in Excel accounting software to ensure timeliness and spreadsheets, and reliability is not reliability. assured. The MES also does not have experience with World Bank reporting requirements. Auditing. Past audits under the S The audit will be carried out by independent M Kazakhstan portfolio have been auditors acceptable to the Bank, and under submitted to the Bank with TORs satisfactory to the Bank. Arrangements considerable delay. will be made for timely contracting of the auditor to ensure timely submission of audit reports to the Bank. OVERALL CONTROL RISK S M OVERALL FM RISK S Risk-based FM supervision M H = High. S = Substantial. M = Moderate. L = Low. Country issues 4. The Public Expenditure and Financial Accountability for Kazakhstan issued in 2009 shows a mixed scenario as far as Public Financial Management is concerned. Kazakhstan has a well- developed budgeting process that is coordinated by the Republican Budget Commission. The country scores highly on predictability of the budget, with significant low levels of deviations of spending from budgeted expenditures in the last three years. However, the country does not score as well on overall budget comprehensiveness and transparency, particularly with regard to comprehensiveness of information included in budget documentation and public access to key fiscal information. Although the budget is prepared on functional, program, and administrative classification, with economic classification being used in budget execution reports, the budget classification is not embedded yet in the chart of accounts, and the national accounting and reporting standards for the public sector are only in the process of modernization. 5. The Public Expenditure and Financial Accountability also shows that Kazakhstan has strong systems of accounting, recording, and reporting, with well-established procedures for timely and regular accounts reconciliation, making information readily available on resources received by service delivery units. Furthermore, comprehensive in-year budget reports are available both for internal use by the Government and external use by the public. Quality of annual financial statements, however, remains one of the weakest points of Public Financial Management in 67 Kazakhstan. This is primarily because essential information is missing from the annual financial statements, namely balance and financial results report. The annual financial statement consists of only one type of report, the Annual Report on the Republican Budget Execution, which is, however, timely submitted to the Accounts Committee for external audit. National accounting and reporting standards for the public sector are not fully developed, and the financial reports differ considerably from the International Public Sector Accounting Standards. 6. Public sector auditing is performed by two agencies: the Accounts Committee (AC), the equivalent of a Supreme Audit Institution, which conducts external audits; and the Committee of Financial Control (CFC), which performs a control function (the concept of internal audit is not elaborated in the budget legislation, although the CFC is supposed to be responsible for internal audit). However, the scope of external audits does not include the year-end financial statements with respect to the Republican budget and of the Republican government bodies. The government external audit system is not organized fully in line with international standards, and the Accounts Committee does not issue any audit opinion. In fact, there is still no clear distinction between internal audit (performed by the Committee of Financial Control), and external audit (performed by the Accounts Committee). 7. The 2007 Accounting and Auditing Reports on the Observance of Standards and Codes (ROSC) for Kazakhstan show that capacity of the accounting profession is still low, and there is no critical mass of professionally qualified accountants, particularly in the public sector. Knowledge of internationally recognized accounting and auditing standards, such as International Financial Reporting Standards (IFRS), International Public Sector Accounting Standards (IPSAS), and International Standards on Auditing (ISA), is limited due to the lack of capacity, in both the public and private sectors. Most project-implementing entities use the cash basis of accounting, which is not in accordance with IFRS, but is allowed under IPSAS, and in many cases can be sufficient for proper project accounting. External audit is practiced by individuals and a small number of audit firms, both local and international. Most audits required by international development partners have traditionally been performed by one of the four large international firms. The Bank conducted a review of local audit firms to determine their acceptability to audit Bank-financed projects and found only two Kazakhstan-based firms that would be eligible. The Bank does not place any reliance on audits conducted by the Accounts Committee, the equivalent of the Supreme Audit Institution in the country. 8. Corruption is widely acknowledged as a major issue in Kazakhstan, as indicated in the TI CPI for 2007 (# 150 with CPI of 2.1) and 2008 (# 147 with CPI 0f 2.2). However, the fiduciary risk of the stand-alone financial management arrangements for the Bank-financed investment projects in Kazakhstan is considered moderate. The Government counterpart funding is assessed to be adequate for a number of years. 9. Because of these weaknesses, the Bank does not place reliance on the country's Public Financial Management system, except the budgeting system, which is assessed to be satisfactory, and Treasury system for counterpart contributions. Implementation arrangements 68 10. The MES will be the implementing agency. Financial management support for the Project will be provided by the Finance Department of the MES. The Finance Department consists of four divisions, including the Accounting and Financial Reporting Division, the Division of Budget Planning for Higher Education and Youth Policy, the Division for Planning of Investment Programs and State Assets, and the Division for Consolidated Planning and Budget Execution. The Finance Department will be supported by an external FM Consultant in managing Project funds, and will maintain accounts and have the accounts audited. 11. The MES uses computerized accounting software designed for budget organizations. However, the software is old and has not been updated. There are plans to replace it with the newer version, although the timeline for such upgrade is not clear. Thus, it has been agreed that the new version of 1C accounting software needs to be purchased for the Project and to be used by the FM Consultant, and this software will be adapted to suit project accounting and financial reporting requirements, including ability to generate interim un-audited financial reports (IFRs) in a format acceptable to the Bank. The software should also be compatible with the existing MES accounting software. Control procedures established for budget organizations are also quite strong and compliance is monitored by the Financial Control Committee and the Accounts Committee. Strengths 12. The Finance Department of the MES has a strong team led by the Department Head. Department strengths include: (a) a reliable budget formulation process and accounting system, (b) a well-systemized filing system of all supporting documentation, (c) an adequate internal control system, and (d) the MES is audited by the Internal Control Department of the Ministry of Finance (MOF) and the Accounts Committee on a biannual basis and by the Internal Audit Unit of the MES. Weaknesses and action plan 13. The following weaknesses were revealed during assessments of the MES: (a) a lack of prior experience in Bank-financed projects, which can be resolved through training in the World Bank FM and Disbursement Procedures and capacity-building; and (b) an automated accounting system that does not support generation of IFRs, and that needs to be upgraded with capacity to meet the project accounting and reporting requirements. The following table lists the actions that have been agreed to be implemented to further strengthen the project financial management arrangements. Agreed Action Responsibility Deadline 1. The automated project accounting system is MES Not later than upgraded, with adequate in-built controls and capable thirty (30) days of tracking project resources and expenditures and after the generating financial reports, including interim Effectiveness financial reports (IFRs). Deadline 69 2. Approval of Project Operational Manual (POM) that MES Effectiveness9 describes the project FM system and procedures, including budgeting, accounting, reporting, internal control, flow of funds, and audit arrangements. 14. The risk associated with the implementing agency is substantial before and moderate after mitigation measures. Budgeting and planning 15. The Finance Department of the MES has a special planning division that prepares budgets, including monthly financing plans. Although the head of the planning division does not have experience in preparing budgets and financing plans for World Bank-financed projects, the existing governmental procedure for such budgeting, including external loans, is adequate and acceptable to the Bank. 16. The Project budgets will form the basis for allocating funds to Project activities, both from the loan and the Republican budget. Project budgets will be prepared in accordance with the IFR format (disbursement categories, components and activities, and account codes, broken down by quarter). Annual budgets should be agreed with the Bank before being submitted to the Ministry of Finance and the Ministry of Economy for final approval. Approved annual budgets will then be entered into the accounting system and used for periodic comparison with actual results as part of the interim financial reporting. 17. The risk associated with planning and budgeting is assessed as low. Staffing of the Accounting/FM Function. 18. There are four divisions within the Finance Department, as described above, and the Head of Finance Department will be responsible for overall FM supervision of the Project. The Head of the Accounting and Financial Reporting Division is performing functions of the Chief Accountant. Staff turnover in the Financial Service is not high. Since this is the first World Bank-financed project for the MES, staff is not familiar with World Bank requirements and procedures for FM and disbursements. It has been agreed that an experienced FM consultant will be hired within the Finance Department by Effectiveness to handle daily activities related to project FM, including accounting and reporting. The risk associated with staffing is substantial before and moderate after mitigation measures. Information Systems 19. The accounting system currently used by the MES was installed several years ago and has not been upgraded since then. The software does not have capacity to support project accounting and financial reporting requirements, including generating IFRs. The accounting software (1C) will be upgraded to fully support Project accounting, reporting, and disbursement functions. Any fixed assets bought with project funds will be recorded in the main system of the MES. A new 9 Draft POM is ready, but will need to be approved by the Borrower as an effectiveness condition. 70 Chart of Accounts will be developed to track project activities, including sources and uses of project funds. The accounting system will have capacity to support accounting and reporting requirements of the Project, including capacity to generate IFRs. The risk associated with the information system is substantial before mitigation and moderate after risk-mitigation measures. Interim Un-audited Financial Reports (IFRs) 20. Currently, the MES prepares financial reports on a regular basis, including quarterly reports on budget execution. These reports include information for budgeted and actual expenditures, both in cash and commitments, and variance analysis. Thus, these reports are similar to the IFRs required by the Bank. However, the reports are being prepared manually in Excel spreadsheets and their reliability is not assured. For the Project, the 1C accounting system will be modified to be able to generate IFRs automatically. 21. The Finance Department of the MES (with support of the FM consultant) will be responsible for submission of IFRs that will be generated by the Project accounting system based on formats agreed with the World Bank. The reports, to include Statement of Sources and Uses of funds, Uses of Funds by Project Activities (Components & Expenditure Categories), Statement of Designated Account, SOE Withdrawal Schedule and Disbursement Summary, will be submitted to the World Bank within 45 days of the end of each quarter, with the first reports under the proposed Project being submitted after the end of the first full quarter following initial disbursement. The risk associated with financial reporting is substantial before mitigation measures and moderate after mitigation measures. Accounting Policies and Procedures 22. The MES, like other governmental/budget organizations in Kazakhstan, uses various governmental instructions and orders describing accounting policies and procedures for budget organizations. However, a Project Operational Manual, incorporating a Financial Management section, will describe policies and procedures under the project, reflecting project-specific activities and transactions, internal control procedures, and accounting policies and procedures applicable to the Project, including cash basis accounting to be adopted for the project. There will be a separate Chart of Accounts for the Project. The risk associated with accounting policies and procedures is moderate before and after mitigation measures. Internal Controls and Internal Audit 23. Internal control procedures applied by the MES are described in detail in the Instruction # 30 of the Ministry of Finance, which is updatable on a regular basis, and internal orders of the MES. The internal audit function is carried out by the recently established Internal Audit Unit of the MES, which is responsible for audits of activities of institutions and enterprises under the MES. This is relevant to the Project because such institutions will be participating. The MOS itself is audited by the Accounts Committee and Internal Control Department of the MOF on a biannual basis. The recent audit by the Accounts Committee was finalized just before the FM Assessment mission. The MOS does not work with commercial banks and uses the Treasury system for all 71 settlements. The risk associated with internal controls is substantial before and moderate after mitigation. External Audits 24. The audit of the Project will be conducted by independent private auditors acceptable to the World Bank, on terms of reference acceptable to the Bank, and procured by the Ministry of Finance out of Republican budget funds, and according to the International Standards on Auditing (ISA) issued by the International Auditing and Assurance Standards Board of the International Federation of Accountants (IFAC). The annual audited Project Financial Statements will be provided to the World Bank within six months of the end of each fiscal year and also at the project closing. 25. The following table identifies the audit reports that will be required to be submitted by the project together with the due date for submission. Audit Report Due Date Continuing Entity Financial Statements N/A Project Financial Statements (PFS), Within six months of the end of each fiscal including Statement of sources and Uses year and also at the project closing of Funds, Uses of Funds by Project Activities, Statement of Designated Account, SOE Withdrawal Schedule and Disbursement Summary. 26. Previous audit reports under the Kazakhstan portfolio have been submitted with considerable delay, in violation of the financing agreements. The risk associated with external audit is considered substantial before and moderate after mitigation measures. Flow of Funds and Disbursement Arrangements 27. The MES staff does not have prior experience in Bank disbursement procedures and will be supported by the FM consultant. The project Designated Account will be maintained by the MES in a commercial bank acceptable to the Bank. Loan funds will flow from the Bank as follows: (a) via a single Designated Account held by the MES for a ceiling amount of US$2,500,000, that will be replenished on the basis of full documentation and Statements of Expenditure (SOEs) on a quarterly basis, (b) reimbursement with full documentation and SOEs, (c) Direct Payments from the Loan Account, and (d) Special Commitments on the basis of withdrawal applications. The Project will follow transaction-based disbursement procedures. Withdrawals from the Loan Account will be requested in accordance with the guidance to be given in the Disbursement Letter. 28. The MES will be responsible for keeping the supporting documentation for all Project expenses, especially those reported through SOEs, and for making them available to World Bank supervision missions and to the auditors. 72 29. The Minimum Application Size for direct payments and for issuance of Special Commitments will be communicated to the Borrower in the Disbursement Letter. All withdrawal applications for direct payment or for issuance of special commitments will be supported by full documentation. 30. The risk associated with funds flow and disbursement is substantial before and moderate after mitigation measures. Supervision Plan 31. As part of its Project supervision missions, the World Bank will conduct risk-based financial management supervisions, initially after every six months and thereafter at appropriate intervals, depending on the level of assessed risk. These will pay particular attention to: (a) Project accounting and internal control systems; (b) budgeting and financial planning arrangements; (c) review of the IFRs; (d) review of audit reports, including financial statements, and remedial actions recommended in the auditor's Management Letters; and (e) disbursement management and financial flows, and counterpart funds, as applicable. 73 Annex 8: Procurement Arrangements KAZAKHSTAN: TVEMP A. General Procurement for the Technical & Vocational Education Modernization (TVEM) Project will be carried out in accordance with the World Bank's "Guidelines: Procurement under IBRD Loans and IDA Credits" published in May 2004 and revised in October 2006 and May 2010 (Procurement Guidelines); and "Guidelines: Selection and Employment of Consultants by World Bank Borrowers" published in May 2004 and revised in October 2006 and May 2010 (Consultant Guidelines), and the provisions stipulated in the Loan Agreement (LA). The various procurement actions under different expenditure categories are described in general below. For each contract to be financed under the Loan Agreement, the various procurement or consultant selection methods, the need for pre-qualification, estimated costs, prior review requirements, and time frame have been agreed between the Borrower and the Bank in the Procurement Plan (PP). The PP will be updated at least annually or as required to reflect the actual project implementation needs and improvements in institutional capacity. A General Procurement Notice (GPN) will be published in United Nations Development Business (UNDB) online and in its printed version, and in dgMarket online. Specific Procurement Notices (SPN) will be published for all International Competitive Bidding procurement and consulting contracts as per guidelines as the corresponding bidding documents and Request for Proposals (RFPs) become ready and available. B. Assessment of the agency's capacity to implement procurement The latest Country Procurement Assessment Report is dated June 2000. A new Public Procurement Law took effect in Kazakhstan on January 1, 2008. While the new law includes many provisions that reflect international practices and the Bank's recommendations, it needs further improvement, such as: (i) all bid evaluation criteria shall be quantifiable in monetary term; (ii) qualification criteria for bid evaluation shall be applied on a pass or fail basis; (iii) contracts shall be awarded to qualified bidder having submitted the lowest evaluated substantially responsive bid and no negotiation shall take place; (iv) there should be an effective and independent protest mechanist allowing bidders and interested parties to protest and to have their protest handled in a timely manner. Therefore and the country's high procurement risk category remains unchanged. The Bank conducted an assessment in August 2009 (updated March 2010) of the implementing agencies' capacity to implement project procurement, and the report is included in the project file. Procurement activities will be carried out by the Ministry of Education and Science (MES) through its Project Management Unit (PMU). The PMU will be set up before project effectiveness and will engage in an ongoing dialogue with MES management and project stakeholders. The PMU will be staffed by local consultants, assisted by full time procurement consultant with relevant procurement qualifications and international experience. The PMU will be responsible for managing the timely implementation of project activities, including: managing the day-to-day aspects of project implementation, procurement and financial management, and coordination with stakeholders and monitoring key performance indicators for both quality and 74 physical outcomes. Management responsibility for each sub-component will be clearly assigned to the first-level managers heading the organizational units within the MES. C. Procurement risk assessment The overall procurement risk is rated High. The risks associated with procurement and the mitigation measures were identified in the assessment of agencies procurement capacity and are summarized in the table below. Procurement Risk Assessment Residual Risk Description of Risk Mitigation Measures Risk Rating Rating Potential procurement delays: H Careful procurement planning and S Experience with the past and ongoing realistic scheduling; advanced projects in country show frequent preparation of technical specifications procurement delays due to poor and/or TORs; further procurement planning, lack of qualified training would be provided during project procurement staff, complex internal implementation; close Bank supervision approval process and etc. and monitoring, particularly from the country office. Low level of competition: Past S Careful procurement packaging to foster M experience indicates the procurement competition; wide and advanced in-country has not attracted adequate advertising; proactive search of and competition; often only one bid was contact to potential suppliers and received. consultants. The perceived level of corruption in S The Bank's good governance and M the country is high. Transparency anticorruption safeguards, particularly the International assigned a Corruption transparency and disclosure provisions of Preconception Index of 2.2 to the Bank Guidelines, will be enforced. Kazakhstan in 2008 (No. 145 in the The Bank will also pay close attention to world). implementation supervision including physical inspections. Inadequate contract management and M More emphasis and training on L lower-than-required quality of appropriate contract management; regular equipment. physical inspections by PMU and Bank supervision mission. Average H M H = High. S = Substantial. M = Moderate. L = Low. D. Procurement implementation and arrangements Procurement activities will be carried out by a PMU to be established within the MES. The PMU will include a full time procurement specialist. Training of the PMU will be done through the project launch workshop as well as during the project implementation. Procurement of Goods: Goods contracts to be procured under this project will include procurement of learning equipment on priority profiles and procurement of educational equipment (including software, digital resources). 75 Procurement of Works: Works contracts to be procured under this project will include the minor rehabilitation works (contract value will not exceed US$100,000) contracts under the grant program. Selection of Consultants: Consultants' services contracts to be procured under this project will include analyzing shortages in qualifications and existing secondary TVE programs, development of 175 module programs, development of concept, requirements for programs, procedures, and templates, and development of E-Learning and development/configuration of a Learning Content Management System (LCMS). Shortlists for consultants' services for contracts estimated to be less than US$200,000 or equivalent may be composed entirely of national consultants. It is also expected that consulting offices associated with local universities may be included in the shortlists. University-based consultants will not be given preference over other private consultants. Training: Training and workshops, including study tours, would be carried out according to annual training plans to be prepared by the PMU and agreed by the Bank. The institutions for training/study tours would be selected by evaluating which institutional program would be most useful, availability of services, duration of training, and reasonableness of cost. Grants Program: The grants will be awarded according to the provisions specified in the Project Operational Manual. The specific goods and services to be financed under each grant will be identified by the PMU during the grant appraisal stage. Operating Cost: These would include office rent, utility and communications, translations, bank charges, office supplies, advertisements, photocopying, mail, and travel expenses. Such costs will be financed by the project per the annual budget in the Institutional Development Plans (IDPs) approved by the Bank and according to the implementing agency's administrative procedures, which were reviewed and found acceptable to the Bank. Operating costs will not include salaries of civil servants. Technical Issues as Part of Procurement Decisions: The PMU will be responsible for developing technical specifications or TORs in collaboration with the technical departments of the implementing agency, and national or international consultants will be hired to provide needed assistance. Bidding Documents and Evaluation Arrangements: The bidding documents, including technical specifications, will be prepared by the PMU with assistance from the procurement and project management consultants. Bid or proposal evaluation will be carried out by an evaluation team (comprising 4 to 5 technical specialists). The procurement and project management consultants will provide assistance in the prequalification process and bid evaluation. Filing and Record Keeping: The PMU office will set up adequate filing and recordkeeping systems, including hard and electronic copies of related procurement documents. Agreed reporting formats are included in the Project Operational Manual. 76 Procurement Plan MES has developed a Procurement Plan for the entire project which provides information on procurement packages, methods, and Bank review requirements. The procurement plan is tentative since it covers the entire project completion period. A detailed procurement plan for the first 18 months of the project has also been prepared and will be agreed upon between the Borrower and the Bank project team at negotiations. This plan will be available in the implementing agency's project database and the version without cost estimate will be available on the Bank's external website. The procurement plan will be updated annually or as required to reflect the actual project implementation needs and improvements in the implementing agency institutional capacity, in agreement with the Bank project team. Frequency of Procurement Supervision In addition to the Bank team prior review, the Project capacity assessment recommends that ex- post reviews be done on at least 20 percent of contracts subject to post-review (see table below). It is expected that a field supervision mission will take place every six months during which post-reviews will be conducted. At a minimum, one post-review report--which will include physical inspection of sample contracts, including those subject to prior review--will be prepared each year. Not less than 10 percent of the contracts will be physically inspected. Not less than 10 percent of the contracts will be physically inspected. Physical inspection will be conducted by technical specialist in consultation with the procurement specialist and in coordination with the PMU. Anticorruption Measures The Bank Anticorruption Guidelines (October 15, 2006) and the transparency and disclosure provisions in the Bank Procurement or Consultants Guidelines (May 2004, revised in October 2006 and May 2010) will be followed. Among others, the following specific actions would be taken: Individuals involved in project management, including procurement, and tender or evaluation committees, must confirm in writing that they have no conflicts of interest, that is, relationships with suppliers, consultants, or government officials. Mechanisms would be established to ensure payments to suppliers and consultants are made according to their contract terms without delays. The Bank would be notified of every complaint received from suppliers or consultants relating to the procurement process, and these complaints would be recorded and dealt with promptly and diligently. Up-to-date procurement records would be maintained and made available to the Bank staff and auditors. 77 REPUBLIC OF KAZAKHSTAN TECHNICAL VOCATIONAL EDUCATION MODERNIZATION PROJECT PROCUREMENT PLAN - GOODS AND WORKS for Jan 2011 -Dec 2013 Compo Estimate Estimat WB Date WB No- nent Cost Contract d Cost ed Cost Procu. Revie of WB No- Date of Date of Bid objection Date of Date of Refere Contrac Plan vs Item Table Descriptio (KZT (US$ Metho w Draft objection Invitation Bid Evaluation to Contract Contract Remarks nce as t Ref. Actual Codes n equivale equivale d (Prior/ BD to to BD to Bids Opening Report Contract Signing Completion per nt) nt) Post) WB Award PAD GOODS Office Equipment and 4-Dec- 22-Dec- Plan 40,000 SH Prior 6-Dec-10 8-Dec-10 24-Dec-10 2-Jan-11 3-Jan-11 31-Jan-11 KZTVE Accounting 10 10 1 2.4 M-SH- Software 01 for PMU Actual Materials for Traning and Upgrading Plan 10,000 SH Post n/a n/a 17-Dec-11 3-Jan-12 10-Jan-12 n/a 1-Mar-11 31-Dec-13 KZTVE Qualificatio 4 2.1 M-SH- n 02 Workshops Actual Procuremen t and localization Plan 60,000 SH Post n/a n/a 12-Feb-11 26-Feb-11 10-Mar-11 n/a 15-Mar-11 31-Dec-13 KZTVE of 20 3 1.2 M-SH- Standards 03 Actual Materials for Information Disseminati Plan 20,000 SH Post n/a n/a 17-Dec-11 3-Jan-12 10-Jan-12 n/a 15-Jan-12 31-Dec-13 on, NQF KZTVE Implementa 2 1.1 M-SH- tion 04 Actual 130,000 TOTAL PROJECT COST, 100% WB funds: USD29,23 mln. Project implementation duration: 2011-2013 Expected Project Effectiveness - Dec 2010. 78 Republic of Kazakhstan Technical and Vocational Education Modernization Project, Loan # PROCUREMENT PLAN - CONSULTANTS - JAN 2011-DEC 2013 (Date of PP: ________; Update No. ______; Date of WB NOL: ________) DRAFT of April 14, 2010 Esti mate Requ WB No- WB Date Comp WB Draft Date Cost d est objectio Date of Tech. Combin Draf No- of onent Revi RFP WB No- of Ite Tabl Cont Cost Estimated for n to RFP Proposa Evaluat ed Tech t objecti Cont Refere Contract Plan vs Select. ew (incl. objectio Contr Remark m e ract (KZ Cost (US$ Exp. RFP Issue l ion & Fin Final on to ract nce as Description Actual Method (Prio TOR, n to act s Code Ref. T equivalent) Of (full d Submiss Report Eval. Cont Draft Com per r/ Short TER Signin s equi Inter package ion (TER) Report ract Contr pleti PAD Post) List) g valen est ) act on t) Development of KZT Educational VEM Planned Standards 23- 23- 20- 31- 1 3.1 - and Programs Aug- 9-Sep- 21-Sep- Sep- 28-Oct- 16-Nov- 1-Dec- 13-Dec- Dec- 3-Jan- 10- Dec- QCB (national) 351,850 QCBS Prior 10 10 10 10 10 10 10 10 10 11 Jan-11 13 S-01 Actual Development KZT of Core 23- 23- 20- 31- VEM Planned Curricula Aug- 9-Sep- 21-Sep- Sep- 28-Oct- 16-Nov- 1-Dec- 13-Dec- Dec- 3-Jan- 10- Dec- 2 3.1 - (national) 226,195 QCBS Prior 10 10 10 10 10 10 10 10 10 11 Jan-11 13 QCB S-02 Actual KZT Teacher Training Planned 23- 23- 20- 31- VEM Aug- 9-Sep- 21-Sep- Sep- 28-Oct- 16-Nov- 1-Dec- 13-Dec- Dec- 3-Jan- 10- Dec- 3 3.2 - Programs 1,000,229 QCBS Prior 10 10 10 10 10 10 10 10 10 11 Jan-11 12 QCB S-03 Actual Development of KZT Methodology VEM Planned for Per- 4- 1- 18- 10- 31- 4 2.2 - Capita Oct- 19- 29-Oct- Nov- 6-Dec- 28-Dec- 11-Jan- 8-Feb- Feb- 1-Mar- Mar- Dec- QCB Financing 195,000 QCBS Prior 10 Oct-10 10 10 10 10 11 11 11 11 11 12 S-04 Actual Expert Evaluation of KZT Educational Planned 12- 26- 10- 31- VEM Standards Jan- 15- 24-Feb- Feb- 1-Apr- 26-Apr- 4-May- Jun- 20- 1-Jun- Dec- 5 3.1 - and Programs 211,110 QCBS Prior 11 Feb-11 11 11 11 11 11 1-Jun-11 11 Jun-11 11 13 QCB S-05 Actual KZT Expert VEM Evaluation of 12- 26- 10- 31- 6 Planned - Core Jan- 15- 24-Feb- Feb- 1-Apr- 26-Apr- 4-May- Jun- 20- 1-Jun- Dec- QCB Curricula 226,195 QCBS Prior 11 Feb-11 11 11 11 11 11 1-Jun-11 11 Jun-11 11 13 79 S-06 Actual Development KZT of up to 120 Planned 3- 29- 10- 20- 31- VEM standards Jun- 17- 27-Jun- Jun- 2-Aug- 1-Sep- 13-Sep- 25-Sep- Nov- Nov- 1-Dec- Dec- 7 1.2 - 889,000 QCBS Prior 11 Jun-11 11 11 11 11 11 11 11 11 11 12 QCB S-07 Actual 3,099,579 80 Republic of Kazakhstan Technical and Vocational Education Modernization Project, Loan # PROCUREMENT PLAN - CONSULTANTS (CQS or Individual Consultants) - JAN 2011-DEC 2013 Evaluat Comp WB WB No- WB No- Date Date Estimat Estimate WB Draf ion onent No- Request objectio RFP Propo Draft objectio of of Cost ed Cost d Cost Revie t report Item Refer Contrac Contract Plan vs Firm Select. objec for Exp. n to Issued sal Final n to Cont Contr Rem Table (KZT (US$ w TOR for ence t Ref. Description Actual or Ind. Method tion Of RFP/ (for Submi Contr Draft ract act arks Codes equivale equivalen (Prior/ to Short as per to Interest Short CQS) ssion act Contrac Signi Compl nt) t) Post) WB list & PAD TOR list t ng etion RFP Project Management Unit (Director, FM, 15- 18- 5- Procurement, 20-Aug- 4-Sep- 8-Sep- 28- 1-Oct- 31- 1.1 KZTVE Planned 324,000 IC IC Prior Aug- Aug- n/a n/a Oct- Admin.Asst, 10 10 10 Sep-11 10 Dec-13 1 M/PMU 10 10 10 Coordinator -IC-01 for Comp. 1 and 2, and Coord for Comp 3) Actual Development of Methodology 20- 1- 18- for Project 8-Dec- 20-Dec- 13- 31- 2.3 Planned 3,000 IC IC Post Nov- Dec- n/a n/a n/a n/a Jan- KZTVE Monitoring 10 10 Jan-11 Dec-11 2 10 10 11 M/IC-02 and Evaluation (national) Actual Project Monitoring 30- 7- 18- and 8-Nov- 27-Nov- 7-Dec- 9-Dec- 20- 5-Jan- 15-Jan- 31- 2.3 KZTVE Planned 75,000 Firm CQ Prior Oct- Nov- Jan- Evaluation 10 10 10 10 Dec-10 11 11 Dec-13 3 M/CQ- 10 10 11 and 01 Workshops Actual Development of KZTVE Methodology 8- 18- 12-Nov- 7-Dec- 9-Dec- 20- 10- 31- 4 2.3 M/CQ- for Project Planned 34,000 Firm CQ Post Nov- n/a n/a n/a Jan- 10 10 10 Dec-10 Jan-11 Dec-11 02 Impact 10 11 Assessment Analysis 81 (intl) Actual Implementati 30- 7- 18- on of 8-Nov- 27-Nov- 7-Dec- 9-Dec- 20- 5-Jan- 15-Jan- 31- 2.3 KZTVE Planned 118,000 Firm CQ Prior Oct- Nov- Jan- Baseline 10 10 10 10 Dec-10 11 11 Dec-11 5 M/CQ- 10 10 11 Study 03 Actual Review Methodology 23- 1- 1- for 20-Dec- 28-Jan- 5-Feb- 18- 25-Feb- 31- 1.2 Planned 29,000 IC IC Prior Nov- Dec- n/a n/a Mar- KZTVE Occupational 11 11 11 Feb-11 11 Dec-11 6 11 11 11 M/IC-03 Standards Dvlpmt Actual International Models 15- 25- 5- 2-Feb- 25-Feb- 31- 1.1 Review, Planned 7,500 IC IC Post Jan- Jan- n/a n/a n/a n/a n/a Mar- KZTVE 11 11 Dec-11 7 National 11 11 11 M/IC-04 Consultants Actual Advising to 1- 10- 15- 15-Mar- 25-Mar- 10- 30- Councils Planned 25,000 IC IC Post Mar- Mar- n/a n/a n/a n/a Apr- KZTVE 11 11 Apr-11 Dec-11 8 2.1 (intl) 11 11 11 M/IC-05 Actual Advising to 1- 10- 15- 15-Mar- 25-Mar- 10- 30- Councils Planned 8,000 IC IC Post Mar- Mar- n/a n/a n/a n/a Apr- KZTVE 11 11 Apr-11 Dec-12 9 2.1 (national) 11 11 11 M/IC-06 Actual Analytical Review of TVE Governance and Mngm. 1- 7- 20- 15- 9-Feb- 25-Feb- 6-Mar- 9-Mar- 3-Apr- 10-Apr- 31- KZTVE and Planned 67,500 Firm CQ Prior Feb- Feb- Mar- Apr- 11 11 11 11 11 11 Dec-13 10 2.1 M/CQ- Workshops 11 11 11 11 04 for Councils, TVE Regional Mngrs. Actual Dvlpmnt of National Qualification 1- 9- 10- 24- 20- 10-Feb- 1-Mar- 9-Mar- 7-Apr- 15-Apr- 1.1 KZTVE s System Planned 77,000 Firm CQ Prior Feb- Feb- Mar- Mar- Apr- Dec-12 11 11 11 11 11 11 M/CQ- Concept 11 11 11 11 11 05 (NQS) for KZ Actual 82 Development of 1- 9- 10- 24- 20- 30- Methodology 10-Feb- 1-Mar- 9-Mar- 7-Apr- 15-Apr- 1.2 KZTVE Planned 59,000 Firm CQ Prior Feb- Feb- Mar- Mar- Apr- Aug- for 11 11 11 11 11 12 M/CQ- 11 11 11 11 11 11 Occupational 06 Standards Actual Revision of National Legislation and 1- 9- 10- 24- 20- 10-Feb- 1-Mar- 9-Mar- 7-Apr- 15-Apr- 31- 2.1 KZTVE Workshop on Planned 84,000 Firm CQ Prior Feb- Feb- Mar- Mar- Apr- 11 11 11 11 11 Dec-12 13 M/CQ- Mngm and 11 11 11 11 11 07 Governance Policy Options Actual Development of Accreditation Procedures 1- 9- 10- 24- 20- for 10-Feb- 1-Mar- 9-Mar- 7-Apr- 15-Apr- 31- KZTVE Planned 100,000 Firm CQ Prior Feb- Feb- Mar- Mar- Apr- Institutions 11 11 11 11 11 Dec-12 14 M/CQ- 11 11 11 11 11 (international 08 ) and Training/Wor kshop??? Actual Development of Accreditation 12- 20- 20- 24-Mar- 10-Apr- 13- 29- 15- 31- KZTVE Procedures Planned 13,500 Firm CQ Post Mar- n/a n/a Apr- Mar 11 11 Apr-11 Apr-11 Apr-11 Dec-12 15 1.3 M/CQ- for 11 11 09 Institutions (national) Actual Review Methodology 1- 13- 5- 10- for 16-Mar- 7-Apr- 9-Apr- 26- 31- KZTVE Planned 41,500 Firm CQ Post Mar- Mar- n/a May- n/a May- Qualification 11 11 11 Apr-11 Dec-12 16 1.2 M/CQ- 11 11 11 11 s Assessment 10 (QA) Actual Review Methodology 11- 19- 10- 15- for 21-Apr- 4-May- 31- Planned 4,500 IC IC Post Apr- Apr- n/a n/a n/a May- n/a May- KZTVE Institutional 11 11 Dec-11 17 1.3 11 11 11 11 M/IC-07 Accreditation (national) Actual Training for KZTVE 22- 1- 5- 15- 20- Personnel 2-Apr- 17-Apr- 20- 31- 18 2.2 M/CQ- Planned 25,000 Firm CQ Post Mar- Apr- n/a May- May- n/a May- Involved in 11 11 Apr-11 Dec-12 11 11 11 11 11 11 Pilots 83 Actual Dvlpmnt of National 22- 3- 8- 20- Qualification 5-Mar- 28-Mar- 8-Apr- 10- 29- 16-May- 31- 1.1 KZTVE Planned 122,000 Firm CQ Prior Feb- Mar- May- May- s Framework 11 11 11 Apr-11 Apr-11 11 Dec-12 19 M/CQ- 11 11 11 11 (NQF) for 12 KZ Actual Workshop 29- 8- 2- 14- 22- 30- 13-Apr- 27-Apr- 31- 1.2 KZTVE for Planned 10,000 Firm CQ Post Mar- Apr- n/a May- May- May- n/a May- 11 11 Dec-11 20 M/CQ- Stakeholders 11 11 11 11 11 11 13 Actual Training and 29- 8- 2- 14- 22- 30- Upgrading 13-Apr- 27-Apr- 31- 2.1 Planned 6,000 IC IC Post Mar- Apr- n/a May- May- May- n/a May- KZTVE Qualification 11 11 Dec-13 21 11 11 11 11 11 11 M/IC-08 (national) Actual Workshops on Training 29- 8- 2- 14- 22- 30- 13-Apr- 27-Apr- 31- KZTVE and Planned 20,000 Firm CQ Post Mar- Apr- n/a May- May- May- n/a May- 11 11 Dec-13 22 M/CQ- Upgrading 11 11 11 11 11 11 14 Qualification Actual Copy and Disseminatio n of 18- 27- 3- 10- 30- 29-Apr- 7-Apr- 16-Apr- 19- 20-May- 31- KZTVE Requirement Planned 72,000 Firm CQ Prior Apr- Apr- May- May- May- 11 11 11 Apr-11 11 Dec-12 23 M/CQ- s for 11 11 11 11 11 15 Institutional Accreditation Actual Experts 18- 27- 3- 10- 30- Evaluation of 29-Apr- 7-Apr- 16-Apr- 19- 20-May- 31- KZTVE Planned 147,000 Firm CQ Prior Apr- Apr- May- May- May- Occupational 11 11 11 Apr-11 11 Dec-12 24 M/CQ- 11 11 11 11 11 Standards 16 Actual Expert 18- 27- 3- 10- 30- Support for 29-Apr- 7-Apr- 16-Apr- 19- 20-May- 31- KZTVE Planned 165,000 Firm CQ Prior Apr- Apr- May- May- May- Standard 11 11 11 Apr-11 11 Dec-13 25 M/CQ- 11 11 11 11 11 Developers 17 Actual Expert 18- 27- 3- 10- 30- 29-Apr- 7-Apr- 16-Apr- 19- 20-May- 31- KZTVE Evaluation of Planned 49,288 Firm CQ Post Apr- Apr- May- May- May- 11 11 11 Apr-11 11 Dec-13 26 3.3 M/CQ- IDPs 11 11 11 11 11 18 Actual Development KZTVE 17- 26- 28- 5- of Procedures 28-Oct- 18-Nov- 26-Nov- 10- 19- 27-Dec- 31- 27 1.2 M/CQ- Planned 170,000 Firm CQ Prior Oct- Oct- Nov- Jan- and Tools for 11 11 11 Dec-11 Dec-11 11 Dec-13 19 11 11 11 12 QA 84 Actual Implementati on of NQF- Revision of 10- 20- 30- 22-Dec- 18-Jan- 26- 31- National Planned 19,500 IC IC Post Dec- Dec- n/a n/a n/a n/a Jan- KZTVE 10 11 Jan-11 Dec-13 28 1.1 Legislation, 10 10 12 M/IC-09 National Consultants Actual Technical Advice for 15- 23- 1- 25-May- 15-Jun- 24- 31- Conducting Planned 12,000 IC IC Post May- May- n/a n/a n/a n/a Jul- KZTVE 12 12 Jun-12 Dec-13 29 1.2 QA 12 12 12 M/IC-10 (National) Actual Analysis of 22- 1- 10- the QA 4-Oct- 25-Oct- 1-Nov- 31- Planned 12,000 IC IC Post Sep- Oct- n/a n/a n/a n/a Nov- KZTVE Results 12 12 12 Dec-13 30 1.2 12 12 12 M/IC-11 (national) Actual Four Workshops 26- 5- 10- 8-Nov- 2-Dec- 6-Dec- 19- 4-Jan- 31- KZTVE for Members Planned 20,000 Firm CQ Post Oct- Nov- n/a n/a Jan- 12 12 12 Dec-12 13 Dec-13 31 2.1 M/CQ- of the 12 12 13 20 Councils Actual Conducting 10- 20- 5- Project 23-Nov- 8-Dec- 18-Dec- 20- 13- 21- 1-Feb- 31- KZTVE Planned 150,000 Firm CQ Prior Nov- Nov- Feb- Impact 12 12 12 Dec-12 Jan-13 Jan-13 13 Dec-13 32 2.3 M/CQ- 12 12 13 Assessment 21 Actual International Models 9- 18- 5- 20-Dec- 11- 25- 1-Feb- 31- KZTVE Review, Planned 25,000 Firm CQ Post Dec- Dec- 9-Jan-13 n/a n/a Feb- 12 Jan-13 Jan-13 13 Dec-13 33 1.1 M/CQ- Consultant 12 12 13 22 with Int.exp Actual 2,095,288 85 REPUBLIC OF KAZAKHSTAN TECHNICAL AND VOCATIONAL EDUCATION MODERNIZATION PROJECT THRESHOLDS FOR PROCUREMENT METHODS AND BANK PRIOR REVIEW Contract Value Expenditure Category Procurement Method Contracts Subjects to Prior Review (US$) Threshold (US$) >500,000 ICB All ICB contracts <500,000 NCB First 2 NCB contracts Goods <100,000 Shopping First Shopping contract NA DC All DC contracts >3 mln. ICB All ICB contracts <3 mln. NCB First 2 NCB contracts Works <100,000 Shopping First Shopping contract NA DC All DC contracts >=200,000 QCBS/QBS/LCS/FBS a/ b/ >=100,000 for firms; all SSS contracts; all <200,000 CQS Consultant Services TORs (including training) NA SSS >=50,000 for individuals; all SSS contracts; all NA IC TORs Notes: a/ Shortlist may compose entirely of national consultants for assignments of less than US$100,000 equivalent per contract. b/ As appropriate, these methods may be adopted for assignments costing less than $200,000. ICB ­ International Competitive Bidding NCB ­ National Competitive Bidding DC ­ Direct Contracting QCBS ­ Quality and Cost Based Selection QBS ­ Quality Based Selection LCS ­ Least Cost Selection FBS ­ Fixed Budget Selection CQS ­ Selection Based on Consultants' Qualifications SSS ­ Single Source Selection IC ­ Individual Consultants Operating Costs: 2.4 Office Rental 90,000 10-Jan-11 31-Dec-13 2.4 Utilities 30,000 10-Jan-11 31-Dec-13 2.4 PMU Training related costs 15,000 10-Jan-11 31-Dec-13 2.4 Travel 6,000 10-Jan-11 31-Dec-13 TOTAL 141,000 Cost Tab Codes Title of the Assignment Budget in USD 1.1 Study Tour, 3 persons 15,000 2.1 Study Tour, 1 week for 5 persons 25,000 2.2 Study Tour, 1 week for 5 persons 25,000 TOTAL 65,000 86 Annex 9: Economic and Financial Analysis KAZAKHSTAN: TVEMP 1. Economic rationale for investments in upgrading the quality of technical and vocational education (TVE) in Kazakhstan The proposed Technical & Vocational Education Modernization (TVEM) Project, to be financed by a World Bank loan of US$29.2 million, will provide significant support to the modernization and reform of the TVE system. With these inputs the Government expects to modernize its TVE system, resulting in a relatively high economic rate of return on its investment. The TVEM Project provides resources for the central drivers of the modernization and reform of TVE in Kazakhstan--namely the development of the National Qualification Framework (NQF), international professional standards, guidance to students to help them make effective choices, improved governance in the TVE system, more effective financing mechanisms through per capita financing and performance-based grants, and the material support required to actually implement modernized TVE systems in the delivery institutions. An important element in this scheme is the implementation of a competitive funding mechanism for Component 3 (80 percent of the TVEM Project), which will result in a recognition (and reward) of effective institutions under the Project, thus providing a strong incentive for weaker institutions to improve their performance (Components 1 and 2 will support this effort). A 2008 survey of firms in Kazakhstan10 indicates that the lack of labor force skills is ranked third in a list of the most important obstacles to business development in Kazakhstan. Lack of education/skills of available workers was ranked by 31.2 percent of companies as a very significant obstacle to business growth, putting this factor only third after macroeconomic instability (32.8 percent) and excessively high tax rates (32.4 percent). To put this in perspective, contract violations are cited as a major problem by 22.8 percent of firms, inadequate tax administration is cited by 18.4 percent of firms, corruption is cited by 18.2 percent of firms, and organized crime is cited by 7.2 percent of firms. While 31.2 percent of the firms report lack of skills as being a very significant problem, another 32.4 percent also report it to be a problem (a lower rating). Hence, a total 63.6 percent of the firms in Kazakhstan have stated that they have significant concerns about the level of labor skills available in the country. In Kazakhstan the demand for skilled labor is very strong.11 The 2008 Labor Market Survey (LMS) indicates that 40 percent of firms hired extra workers during the past year, while only 9 percent downsized. Over the past year, 36.6 percent of firms were hiring specialists and skilled workers compared to only 18.1 percent of firms hiring unskilled workers, 8.8 percent hiring salesmen, and 8.6 percent hiring managers. The overwhelming majority of firms in Kazakhstan require high levels of education in the employees they seek. In recruiting skilled workers, 57.1 percent of the firms expect employees to have TVE education, 35.7 percent expect college education, and 4.3 percent expect 10 The Labor Market Survey of 500 business entities with a sample stratified by region, firm size, type of ownership, and industry was commissioned by the World Bank, and implemented by the survey firm BISAM in January 2008. 11 GDP increased at an average rate of 8.1 percent during 1996­2006. 87 higher education. Only 3 percent of firms are willing to consider applicants with secondary general education. Even when recruiting for positions requiring a lower level of skills, only a quarter of the firms expect to recruit graduates with secondary general or lower education. Therefore, the following considerations argue for investing in improving TVE in Kazakhstan: The analysis of the labor market and demand for skills in Kazakhstan indicates a strong economic rationale for government intervention in reforming the TVE education system. If properly implemented, such reforms are expected to bring about economic benefits through the following main channels: Increased demand for TVE graduates with a new set of skills, which will also trigger the desirability of the TVE program, and hence will increase the supply of TVE program graduates. Improved employability of the TVE graduates as the quality of TVE education improves and new quality TVE graduates become more demanded in the labor market. Kazakhstan, like other transition countries in the Europe and Central Asia Region, inherited the Soviet system of TVE education whose relevance to the labor markets can be described as artificial, because it was established within a tradition of central economic planning and centralized job placement. New economic realities of the fast- changing and increasingly globalized economy call for profound revisions of the TVE system to enable it to meet the demands of the dynamic labor market. There will be increased returns to TVE because the skills and competencies of the graduates of the new TVE system will be significantly enhanced and will better match labor market requirements.12 The restructuring of the education system so that the quality of secondary general education (core curriculum) will be the same across all institutions will also eliminate the negative selection now evident in the TVE system; those who choose to attend technical schools are often those who are perceived as having low capabilities for education at a higher level. 2. Project Objectives, Components, and Economic Relevance The main objective of the Project is to help restructure and rebuild the TVE system in Kazakhstan. In doing this, it will finance critically important, currently unfunded activities and goods within the broader State TVE Development Program). This restructuring is required as the Government seeks to: (a) increase youth employability and capabilities for continuous skills upgrading, and (b) offers local businesses a trained workforce with the skills and competencies to enable them to compete in the global economy.13 The Project consists of three strongly integrated components: A National Qualifications Framework, which develops the reforms; TVEM Governance and Financing systems to improve the efficiency and accountability of the TVE system; and TVE upgrades to improve the quality and relevance of the professional lyceum schools, colleges, and lifelong learning programs. 12 The analysis of the 2008 LMS indicates that currently workers with TVE education are paid on average 28 percent more than those with secondary general education. 13 Analysis of the 2008 LMS indicates that only 4 percent of firms recruit some workforce from abroad. The rest rely solely on the skills available in the local labor market. 88 The provision of vocational skills by the upgraded TVE system is expected to help youth find work upon leaving school, and become more productive (that is, achieve higher returns) and trainable once in the labor force. There is international evidence that this happens,14 but not for all youth or all specializations. Choices are important to meet different learning needs and circumstances of youth. Skills may be acquired in schools and beyond in non-formal settings, including the workplace. Using TVE to improve the transition of youth to work requires careful review of specializations and their purposes. It also requires awareness of the characteristics of those served and the close matching of training with employers' needs. There are definitely some risks involved in achieving economic benefits from the upgraded TVE system. Much of the unfavorable evidence for professional lyceum school programs found in developing countries, apart from the poor quality of the TVE offered, appears to rest with slow jobs growth and weak demand for employment. TVE has been introduced in developing countries to promote skills for industrialization, and where this has not happened, it has been slow to respond to the different skill needs of the informal economy that has grown in its place. Small modern sectors in these countries with high degrees of informality in employment translate to low demand for TVE, but where job growth and labor shortages in the modern sector are found, as is the case in Kazakhstan, TVE graduates are more easily absorbed into employment.15 3. Methodology of Estimating Economic Costs and Benefits In quantifying the costs and benefits of the Project we carefully considered the following key factors: (a) the time frame for Project implementation (that is, investments made); (b) the timing of when the benefits start to accrue, and over which period of time the benefits (and related costs) are analyzed; (c) quantifying direct compared to indirect benefits (and costs); and (d) methodological approaches to estimating costs and benefits, and assumptions made in the process of analysis. Each of these factors is discussed in a greater detail below. The Project, according to the currently proposed time frame, is expected to be implemented during 2010 through 2013. If the Project becomes effective as planned, it can be expected that the new quality TVE graduates will be available for the labor market starting about 2015. Hence, it would take about five years from the start of the Project to produce the first benefits, when new TVE graduates enter the labor market as potentially more productive workers. For the purposes of the analysis, we limit the benefit period to 15 years, that is, from 2015 to 2030. It does not mean though that the benefits from the new TVE will stop by then. Because the benefits become increasingly less predictable in the future, we want to keep the cost- benefit analysis within a reasonable time frame. It is possible to quantify most, but not all, of the Project's expected benefits. As discussed above, in this analysis we will focus on quantifying the economic benefits of the 14 Ryan, Paul. 2001. The School-to-Work Transition: A Cross-National Perspective. Journal of Economic Literature Vol. XXXIX, March: 34­92. 15 Note that efforts to enhance the relevance of secondary education by introducing a small amount of vocational content in the curriculum, usually less than 20 percent, show little evidence of improving employment. By providing more choices, however, these efforts may enhance educational attainment (Lauglo, Jon, and Rupert Maclean [eds.], 2005, Vocationalization of Secondary Education Revisited, Netherlands: Springer). 89 Project that will find their pass-through to the labor market in the form of improved employability and higher returns. It is not possible to quantify the direct or indirect impact of additional infrastructure, improved training for teachers, enhanced managerial capacity of the education system, a potentially lower crime rate (due to lower unemployment), and other factors. These will have to be understood as Project outcomes that make the other (quantifiable) outcomes possible--without an additive impact of their own. The analysis will therefore be restricted to the quantifiable economic impacts and benefits. This comprises: (a) an increase in employment among TVE graduates; (b) higher returns to TVE in the labor market; and (c) increased tax revenues. Certain assumptions necessarily need to be made in the process. This is driven mostly by uncertainty about the future rates of return to TVE. While the current rates of return to TVE (compared to other levels of education) can be estimated empirically from the existing data, such as the Kazakhstan Firm Survey, there is no literature that could provide solid guidance on how the returns to TVE education are expected to change as the result of reforming the TVE system and the evolving structure of the economy. Some reasonable assumptions need to be made in this regard, and will be discussed below. The cost-benefit analysis of the Project is based on a model comprising three modules. The first module contains estimated numbers of new TVE graduates, and allows for various scenarios on their employment level. The second module estimates the size of the various benefits of the Project based on the estimated returns and employment levels. The third module quantifies the costs arising from the Project. The key inputs and assumptions made in estimating the costs and benefits of the World Bank Project are: The cost of borrowing will be around 2.5 percent per year. This is based on the current standard IBRD loan terms, which are currently equal to LIBOR + 1.05 percent.16 The current one-year LIBOR is 1.09 percent. In addition, a front-end fee of 0.25 percent is charged. It would take five years from the beginning of the Project to produce the first benefits, when new TVE graduates enter the labor market as potentially more productive workers. For the purposes of the analysis we limit the benefit period to 15 years, that is, from 2015 to 2030. Because the benefits become increasingly less predictable in the future, we want to keep the cost-benefit analysis within a reasonable time frame. The sustainability of the Project will also require additional current costs equal to about 5 percent of the Project costs per year for 10 years (2015­2025) following the initial investment phase. 16 The rate of 1.05 percent is for loans of the duration greater than 14 years; the rate decreases to 0.8 percent for loans of 10 to 14 years. 90 The projected number of TVE graduates increases from about 158,000 (in the 2007/08 school year) to 190,000 (in 2015), as suggested by the Draft Law on the TVE reform, and then increases steadily by about 0.4 percent per year (the rate of population growth). The increase in the probability of finding employment for TVE graduates (due to an improved TVE program) is equal to 10 percent. The improved quality of TVE education results in the productivity improvement of workers with TVE education by 10 percent; the productivity improvement manifests in higher wages. Real wages increase in line with real GDP growth; during 2015­30, real wages increase at an average rate of 5.5 percent.17 Return to an extra year of work experience is equal to 2 percent. The economic benefits to the Government from modernizing the TVE system (that is, from implementing the Project) come in the form of taxation of personal incomes; the personal income tax rate used in the calculations is equal to 10 percent.18 4. Estimated Economic Costs and Benefits of the Project It is estimated, in the proposed TVE Project, that the investment costs for 2010 through 2013 will amount to about US$29.2 million (KZT4.38 billion). These costs will be covered through the World Bank loan of the same amount.19 The cost of borrowing on IBRD loan terms over 20 years will amount to US$16.6 million (the net present value [NPV] of those payments [at a 5 percent discount rate] is estimated to be US$9.1 million in 2010 prices). In the cost-benefit analysis it is also assumed that on top of these costs the sustainability of the Project will also require additional support costs equal to about 5 percent of the Project costs per year for 10 years (2015­2025) following the initial investment phase. These costs will amount to US$14.6 million in nominal terms (the NPV of those payments [at a 5 percent discount rate] is estimated to be US$9.3 million in 2010 prices). To summarize, the cost-benefit analysis assumes that the Project's overall costs (IBRD loan, loan servicing, plus additional costs to sustain the reform of the TVE system) will reach US$58.4 million in nominal terms. The NPV of the Project's overall costs is estimated at US$47.6 million in 2010 prices. The economic benefits of the TVE Project to the GOK are estimated to be about US$103.5 million (in 2010 prices) during 2015­30. Those benefits take the form of additional tax revenues (personal income tax collections) that result from the implementation of the Project. Similar to the calculation of costs, the calculation of benefits uses the annual rate of 5 percent to estimate the NPV. The calculated benefits are based on the premise that the new quality TVE graduates will be available for the labor market starting in 2015. The Project's total costs and benefits are presented in Figure A9.1. 17 Note that current macroeconomic scenarios for 2015­20 use the range for annual economic growth of 5.5 percent to 8 percent. In our analysis we decided to use a more conservative scenario. 18 Note that the personal income tax flat rate of 10 percent has been in effect in Kazakhstan since July 7, 2006. 19 This calculation is based on the exchange rate of 150 KZT / US$. 91 Figure A9.1: The Project's Total Costs, Total Benefits, and Net Benefits (2010­30), US$ million (in 2010 prices) Source: World Bank estimates. The net benefits of the TVE Project are estimated to be about US$55.9 million over 2015­30 (Figure A9.1). However, according to the Project's expected payoff schedule, the net benefits will only start to accumulate in 2015. The annual streams of the Project's net benefits are presented in Figure A9.2. The Project is expected to break even (that is, total net benefits become positive) by 2021, as shown in Table A9.1. Figure A9.2: Estimated Annual Stream of Net Benefits of the TVE Project (2010­2030), US$ million (in 2010 prices) Source: World Bank estimates. 92 Table A9.1: Estimated Financial Performance of the Project (NPV, discount rate of 5%) Net Total Total benefits Net benefits costs, US$ benefits, (annual), (cumulative), Year mln US$ mln US$ mln US$ mln 2010 5.8 0.0 -5.8 -5.8 2011 6.5 0.0 -6.5 -12.4 2012 6.5 0.0 -6.5 -18.9 2013 6.5 0.0 -6.5 -25.3 2014 7.6 0.0 -7.6 -32.9 2015 1.7 7.7 6.1 -26.8 2016 1.6 7.6 6.0 -20.9 2017 1.5 7.4 5.9 -15.0 2018 1.4 7.2 5.8 -9.2 2019 1.4 7.0 5.6 -3.6 2020 1.3 6.8 5.5 1.9 2021 1.2 6.7 5.4 7.4 2022 1.2 6.5 5.3 12.7 2023 1.1 6.3 5.2 17.9 2024 0.4 6.2 5.8 23.7 2025 0.4 6.0 5.7 29.4 2026 0.3 5.9 5.6 35.0 2027 0.3 5.7 5.4 40.4 2028 0.3 5.6 5.3 45.7 2029 0.3 5.5 5.2 50.9 2030 0.3 5.3 5.1 55.9 Total 47.6 103.5 55.9 Source: World Bank estimates. In summary, the cost-benefit analysis indicates that the economic benefits of the TVE Project to the GOK will outweigh the costs. The NPV of the net benefits is estimated to be about US$55.9 million. Moreover, the estimated net benefits are expected to continue after 2030. The estimated net benefits also do not take into account the private benefits that accrue to individuals as a result of better TVE education, and potential spillover effects due to improved labor mobility and adoption of knowledge. 93 Annex 10: Safeguard Policy Issues KAZAKHSTAN: TVEMP Project objective and description. The Project Development Objective is to raise the relevance, quality, and efficiency of technical and vocational education in Kazakhstan through an improved policy framework and institutional capacity. The Project consists of three components: Component 1 develops national standards and quality-assurance mechanisms; Component 2 strengthens the system of governance, management and finance; and Component 3 strengthens the skills delivery capacity of TVE institutions. Component 1: Develop Standards and Quality Assurance for TVE. This component will support the establishment of a national qualifications framework, development of occupational standards, an upgrade of assessment tools for qualifications, and introduction of institutional accreditation in establishing the foundation for a national qualifications system. Component 2: Strengthen Governance, Management, and Financing for TVE. This component will improve system efficiency and effectiveness by supporting activities to increase employer involvement in advising TVE development through national, regional, and sectoral councils; generate options for better structures, governance, and management of the TVE system; introduce formula-based financing to enable the allocation of resources toward students pursuing specialties required by the labor market; and establish a TVE monitoring and evaluation system. Component 3: Strengthen Skills Delivery Capacity of TVE Institutions. This component would finance the revision and development of educational standards and core curriculum, help develop and deliver a competency-based curriculum, and finance development of Institutional Development Plans (IDPs) of TVE institutions selected through competitive bidding, to enhance skills delivery at TVE institutions. IDPs would include establishing enterprise partnerships, designing new curriculums and programs based on occupational analysis, upgrading of management and teaching staff, development of teaching materials, renovation of facilities, procurement of teaching equipment, and establishment of career guidance and counseling services. Location. All the subprojects to be supported by the Project will be identified and screened during the implementation stage and will be implemented countrywide. Project category. In accordance with the Bank's safeguard policies and procedures, including OP/BP/GP 4.01 Environmental Assessment, the Project is placed into Bank Category B, which is applied to all proposed projects that have a potential environmental impact. Since the subprojects to be financed are not yet identified, the Bank requires the client to screen all proposed subprojects and ensure that subproject beneficiaries carry out an appropriate Environmental Assessment (EA) for each of them. For this purpose the client must prepare an Environmental Management Framework (EMF). 94 There will be no expansion of buildings beyond existing site boundaries, and no land acquisition (even minor) will be required. Thus, OP4.12 is not triggered and no resettlement documents need be prepared. Potential environmental impacts. The Project activities are not expected to cause significant adverse environmental or social impacts and may be summarized as follows: (a) dust and noise due to the rehabilitation activities; (b) dumping of construction wastes, accidental spillage of machine oil, lubricants, paints, and solvents; and (c) asbestos. It is not yet known whether asbestos was used in premises to be proposed for rehabilitation; however, considering its common use in the former USSR, it is possible to find such material in insulation and roofing material. These potential environmental impacts are minor and could be easily managed during project implementation. The Project will also bring positive social impacts since the proposed activities would increase youth employability and capabilities for continuous skills upgrading. Environmental Management Framework (EMF). The EMF outlines environmental assessment procedures and mitigation requirements for the subprojects that will be supported by the Project. It provides details on procedures, criteria, and responsibilities for subproject screening, and on preparing, implementing, and monitoring of subproject-specific Environmental Impact Assessments (EIAs). The document also includes Environmental Guidelines for proposed subprojects containing the following: (a) generic mitigation measures for potential environmental impacts of the rehabilitation activities of the selected school premises, described above. In this regard the guidelines are focused, in particular, on handling of construction- generated wastes, and storage of hazardous materials under project-supported measures to reduce noise and dust; (b) screening procedures for identifying subprojects that do not require any special Environmental Assessment (EA) and Category B subprojects for which it is necessary to apply an Environment Management Plan (EMP); (c) description of the EMP checklist; and (d) requirements for conducting monitoring activities for Category B subprojects. The EMF also contains institutional responsibilities for its implementation. Integration of the EMF into project design and implementation. The EMF will be integrated into the Project Operational Manual. An EMP checklist will be used as part of all contracts involving equipment and works. The Bank is expected to provide special EA training to the Project Management Unit (PMU) team, so they can promote compliance with the EMF and EMP. The Bank will also supervise this aspect of implementation. The subprojects' EMP will also be integrated into the construction contracts for individual sites--into both specifications and bills of quantities--and the contractors will be required to include the cost in their financial bids. TVE beneficiary institutions will also receive a copy of the EMF and EMP to enable them, when appropriate, to oversee compliance on the part of contractors. EA report disclosure and consultation. In accordance with the Bank policy, the EMF was consulted in the country with key stakeholders and has been disclosed to the public. 95 Annex 11: Project Preparation and Supervision KAZAKHSTAN: TVEMP Planned Actual PCN review 02/12/2008 02/12/2008 Initial PID to PIC 02/26/2008 02/19/2008 Initial ISDS to PIC 02/27/2008 04/03/2008 Appraisal 03/15/2010 03/25/2010 Negotiations 05/07/2010 Board/RVP approval 06/29/2010 Planned date of effectiveness 09/30/2010 Planned date of Mid-term Review 12/01/2011 Planned closing date 12/31/2013 Key institutions responsible for preparation of the project: MES, MLSP, ATAMEKEN. Bank staff and consultants who worked on the project included: Name Title Unit Dingyong Hou Team Leader; Senior Education Specialist ECSH2 John Innes Lead Social Sector Specialist ECSH3 Ernesto P. Cuadra Lead Education Specialist MNSHE Alberto Rodriguez Country Sector Coordinator; Lead Education ECSH2 Specialist Aliya Bigarinova E T Consultant ECSH2 Tatyana Li E T Consultant ECSH2 Darkhan Bilyalov Consultant ECSH2 Kirill Vasiliev Education Specialist ECSH2 Baktybek Zhumadil Operations Analyst ECSH1 Dina Zhurkabayeva Education Specialist ECSHD Nurbek Kurmanaliev Procurement Specialist ECSC2 John Otieno Ogallo Senior Financial Management Specialist ECSC3 Aliya Kim Finance Assistant ECCKA Arcadie Capcelea Safeguard Specialist ECSSD Danielle Malek Counsel LEGEM Hannah Koilpillai Senior Finance Officer CTRFC Richard Johanson Consultant ECSHD David Fretwell Consultant ECSHD Joshua Hawley Consultant ECSHD Rajendra Dhoj Joshi Senior Education Specialist SASHD Bertil Oskarsson Consultant ECSHD Robert McGough Consultant ECSHD Kari Jouko Juhani Pitkanen Consultant ECSHD Wendi Ralaingita Consultant ECSHD Karthika Radhakrishnan-Nair Program Assistant ECSH1 Anara Tokusheva Program Assistant ECCKA Aigerim Dzhamalova Team Assistant ECCKZ Estimated Approval and Supervision costs: 1. Remaining costs to approval: US$50,000 2. Estimated annual supervision cost: US$100,000 96 Annex 12: Documents in the Project File KAZAKHSTAN: TVEMP Section A ­ General Law on Education, 2007. Astana, Republic of Kazakhstan. State Program on Technical and Vocational Education Development, MES, 2008, Astana. National Pact on TVE between MES, MLSP, and ATAMEKEN, Astana 2008. Review of Vocational Education and Training and Employment in Kazakhstan, European Training Foundation, 2003. Situational Analysis on Employment, 2007, International Labour Organization (ILO), Almaty. Section B ­ World Bank Views of Kazakhstan Youth on TVE, 2006. Section C ­ Project Analysis of TVE system, 2007. Labor Market in the Republic of Kazakhstan, BISAM Central Asia, 2008. Report and proposal for Action Plan to develop per-student and performance funding for technical and vocational education in the Republic of Kazakhstan. Feasibility study for the TVEM Project, 2009. Institutional mapping of the TVE System, 2009. 97 Annex 13: Criteria and Procedures for Approval of IDP Subprojects KAZAKHSTAN: TVEMP I. Criteria for Approval of Institutional Development Plan (IDP) Subprojects 1. Eligibility criteria--who can apply and for what Reform of lyceum programs. Reform of college programs. Retraining and upgrading programs--short courses. Public institutions20. Private providers. Legal status: Institutions are legally incorporated entities that have vocational training as their main field of activity and have satisfied their tax and social security payment obligations. Social partners: Institutions have a tripartite management board or have established an employer advisory council to assist with the implementation of the IDPs. Co-financing: Technical and vocational education (TVE) institutions provide guarantee letters from the local Akimats (authorities) approving the IDPs and required local funds. 2. Program priorities Linkages with enterprises. Sectors: Oil and gas, for example. Types of interventions, for example, new program development and career counseling and guidance. Program integration between initial and secondary vocational. Adult retraining and upgrading programs--short courses. 3. Eligible expenditures--what can be financed Local training: Staff development (including travel, subsistence, and tuition). Consultant services: Domestic, international. Goods: Training materials, training equipment, office equipment. Renovation of existing facilities. Incremental operating cost. The following expenditures would be excluded: Commercial production equipment. Vehicles. New construction. Salaries or salary supplements of civil servants. 20 Participation of private providers is essential to ensure competition for the sub grants and will be monitored for during implementation. 98 4. Content of applications--the IDPs Background: Analysis of key problems, their dimensions, and causes and consequences. Justification: Solution of priority problems--identification of labor market requirements from enterprises related to the program. Objectives and strategy. Means and activities: Components with targets--programs, enrollments by gender, projected outputs. Partnerships arranged with enterprises. Implementation: Responsibilities (who will do what), staffing (and qualifications), schedule. Costs and financing: Costs per student, co-financing amounts and agreements. Evaluation. Deliverables. Dissemination plan: To spread reforms to other institutions. Attachments to the application: Organizational structure of the institution. Programs offered. Intake, enrollments, graduates by program and gender--last six years. Teaching/instructional staff by subject and qualification. Age profile of teaching staff. Data on teacher turnover by subject or specialization. Financing: By category and per student. Linkages with employers. Past placement rates for graduates by program. Detailed costing. Proposed equipment lists. Detailed implementation schedule. 5. Evaluation criteria In general, applications will be evaluated according to three broad criteria: Relevance: Relationship to overall project objectives, such as relevance, quality improvement, and priorities. Effectiveness/feasibility: o Technical o Financial, (for example, co-financing) o Implementation capacity. Efficiency (costs per student). More specifically, the applications would be reviewed for the following kinds of topics: Critical fields: Does the proposal aim at improving service delivery in critical sectors and occupations? 99 Necessary inputs: Does the proposal cover all the inputs necessary to achieve its objectives in terms of quantity and quality? Is the proposed equipment directly related to the objectives and content of the proposal? Does the design contain carefully articulated activities and sub-activities that are based on best practices in training, curriculum development, and adult learning? Is there evidence that employers have been and will continue to be involved in project design, operation, and evaluation? Realism: Is evidence provided that planned activities are capable of achieving objectives? Implementation: To what extent does the bidder have the ability to carry out the project as intended and produce quality results? To what extent has the bidder adequately identified the staffing implications of project execution and assigned the necessary technical and managerial capacity and expertise to carry out the subproject effectively? Are unit costs of the proposed activities reasonable and in line with Project Management Unit norms? Are appropriate accounting systems and procedures in place to track subproject expenditures? 6. Approval criteria and allocation guidelines Preference will be given to IDPs that plan to share new equipment and facilities with other TVE institutions. Preference will be given to proposals that integrate initial and secondary vocational programs in the same institution. Close linkages with employers. Minimum size of subprojects. Maximum size of subprojects. Regional distribution: While keeping in line with the regional development priorities defined in the State Program of Accelerated Industrial Development and Innovation, equal opportunities for competing in subgrants should be ensured for TVE institutions from all regions. Private: Floor on funds for private training providers (at least a third of subproject approvals or total funds). Composition of funds: No more than 80 percent of total costs for equipment and facilities; at least 20 percent for staff and systems development. II. Procedures for Approval of Grants for IDP Subprojects Stage 1: Program Announcement (for IDP subprojects) Generating Proposals First, the PMU will advertise the availability of funds for financing IDPs at TVE institutions. This will be done through the media, through regional and city administrations, and by direct contact with training providers. Second, the PMU will conduct workshops for interested applicants to train them on the formulation and design of IDPs, covering both methods and content. Third, the PMU will contract a group of consultants in IDP proposal design that 100 interested applicants can call on to assist them in preparing the IDP. (These consultants would not be eligible to review applications for the PMU due to conflict of interest.) Stage 2: Issue Request for Subproject Proposals The PMU program officer responsible for IDPs issues an official announcement and request for proposals with deadlines. The officer responds to requests for information by potential applicants. The PMU offers assistance and training in proposal preparation to potential bidders if necessary. Stage 3: Initial Screening of Proposals The PMU program officer receives IDP proposals and records receipt. The officer checks proposals received for compliance with the specified format and other conditions. The officer examines the applications for completeness and returns to the bidder those not in compliance, with an explanation of the reasons. Stage 4: Evaluation of Proposals (see above criteria for assessment) The PMU will have appointed an Advisory Council of experts comprised of specialists in different fields to form subgroups to review applications. The program officer will ensure that the reviewers have no conflict of interest and receive clear guidelines (and training, initially) about ethical issues, the content and format of the review, and the criteria to be applied. The program officer organizes proposals by program field and sends a batch of applications to the (sub) Panel for evaluation. The (sub) Panel evaluates the proposals according to the agreed criteria. It undertakes an economic review (relationship of the proposal to enterprises), a technical review (good training design and learning practices), and a financial review on the efficient use of funds and proper control over expenditures. It follows an agreed format and scoring sheet for the evaluation. The (sub) Panel sends a written report and recommendation to the PMU for each evaluated proposal. In case of disagreement, a third-party expert may be called in. In some cases a field review of proposals may be necessary. Proposals that receive a favorable rating in all three areas (economic, technical, and financial) will be forwarded by the program officer to the PMU Director. The PMU Director checks whether the approved procedures and criteria have been applied by the Advisory Panel. The Director decides which proposals to put forward and which to return to the applicant. If returned to the applicant, the PMU will attach the reasons for rejection. If the application has merit, the PMU may also recommend revisions before resubmission. Stage 5: Financing Decisions The PMU Director forward these recommendations to the Project Director for consideration. Special considerations or approval criteria include whether the proposal meets the agreed criteria and whether the feasibility of cost-sharing has been demonstrated convincingly. The Project Director or the PMU may ask for further staff work on the proposal or reject it with explanation. If approved, the application is sent to the PMU Director for contractual arrangements. If not approved, a communication will be sent to the proposal originator stating the reasons why the project was not approved. The information will be as clear and detailed as possible to encourage rejected bids to be improved and resubmitted. 101 The figure below graphically displays the process. 102 Annex 14: Statement of Loans and Credits KAZAKHSTAN: TVEMP Difference between expected and actual Original Amount in US$ Millions disbursements Project ID FY Purpose IBRD IDA SF GEF Cancel. Undisb. Orig. Frm. Rev'd P099270 2009 SOUTH WEST ROADS 2,125.00 0.00 0.00 0.00 0.00 2,125.00 0.00 0.00 P101928 2008 HLTH SEC TECH (JERP) 117.70 0.00 0.00 0.00 0.00 117.70 0.00 0.00 P096998 2008 CUSTOMS DEVT (JERP) 18.50 0.00 0.00 0.00 0.00 18.50 0.82 0.00 P090695 2008 TECHNOLOGY 13.40 0.00 0.00 0.00 0.00 13.40 0.00 0.00 COMMERCIALIZATION PROJECT P078342 2007 UST-KAMENOGORSK ENV REMED 24.29 0.00 0.00 0.00 0.00 24.29 0.00 0.00 P078301 2006 FORESTRY 30.00 0.00 0.00 0.00 0.00 27.92 2.59 0.00 P095155 2006 N-S ELEC TRANSM 100.00 0.00 0.00 0.00 0.00 33.80 31.13 0.00 P058015 2005 AG POST PRIV ASSIST (APL #2) 35.00 0.00 0.00 0.00 0.00 34.83 28.83 0.00 P049721 2005 AGRIC COMPETITIVENESS 24.00 0.00 0.00 0.00 0.00 19.31 12.91 0.00 P059803 2003 NURA RIVER CLEAN-UP 40.39 0.00 0.00 0.00 0.00 13.83 10.34 0.00 P046045 2001 SYR DARYA CONTROL N. ARAL SEA 64.50 0.00 0.00 0.00 0.00 4.56 4.56 0.02 Total: 2,592.78 0.00 0.00 0.00 0.00 2,433.14 91.18 0.02 KAZAKHSTAN STATEMENT OF IFC's Held and Disbursed Portfolio In Millions of U.S. Dollars Committed Disbursed IFC IFC FY Approval Company Loan Equity Quasi Partic. Loan Equity Quasi Partic. Total portfolio: 0.00 0.00 0.00 0.00 0.00 0.00 0.00 0.00 Approvals Pending Commitment FY Approval Company Loan Equity Quasi Partic. 2001 Kazkommertsbk 2 0.02 0.00 0.00 0.00 Total pending commitment: 0.02 0.00 0.00 0.00 103 Annex 15: Country at a Glance KAZAKHSTAN: TVEMP E uro pe & Uppe r- P O V E R T Y a nd S O C IA L C e nt ra l m iddle - D e v e lo pm e nt dia m o nd* Ka za k hs t a n A s ia inc o m e 2007 P o pulatio n, mid-year (millio ns) 15.5 445 823 Life expectancy GNI per capita (A tlas metho d, US$ ) 5,060 6,052 6,987 GNI (A tlas metho d, US$ billio ns) 78.3 2,694 5,750 A v e ra ge a nnua l gro wt h, 2 0 0 1- 0 7 P o pulatio n (%) 0.7 0.0 0.7 GNI Gro ss Labo r fo rce (%) 1.3 0.5 1.3 per primary M o s t re c e nt e s t im a t e ( la t e s t ye a r a v a ila ble , 2 0 0 1- 0 7 ) capita enro llment P o verty (% o f po pulatio n belo w natio nal po verty line) 15 .. .. Urban po pulatio n (% o f to tal po pulatio n) 58 64 75 Life expectancy at birth (years) 66 69 71 Infant mo rtality (per 1,000 live births) 26 23 22 Child malnutritio n (% o f children under 5) .. .. .. A ccess to impro ved water so urce A ccess to an impro ved water so urce (% o f po pulatio n) 96 95 95 Literacy (% o f po pulatio n age 1 5+) .. 97 93 Gro ss primary enro llment (% o f scho o l-age po pulatio n) 105 97 11 1 Kazakhstan M ale 105 98 1 12 Upper-middle-inco me gro up Female 105 96 109 KE Y E C O N O M IC R A T IO S a nd LO N G - T E R M T R E N D S 19 8 7 19 9 7 2006 2007 E c o no m ic ra t io s * GDP (US$ billio ns) .. 22.2 81.0 103.8 Gro ss capital fo rmatio n/GDP .. 15.6 32.8 31.0 Trade Expo rts o f go o ds and services/GDP .. 34.9 51.1 47.6 Gro ss do mestic savings/GDP .. 13.1 43.5 40.3 Gro ss natio nal savings/GDP .. 12.0 30.5 28.4 Current acco unt balance/GDP .. -3.6 -2.2 -3.0 Do mestic Capital Interest payments/GDP .. 0.8 2.4 .. savings fo rmatio n To tal debt/GDP .. 18.4 91.5 .. To tal debt service/expo rts .. 6.2 33.7 .. P resent value o f debt/GDP .. .. 89.1 .. P resent value o f debt/expo rts .. .. 167.4 .. Indebtedness 19 8 7 - 9 7 19 9 7 - 0 7 2006 2007 2 0 0 7 - 11 (average annual gro wth) GDP -7.1 9.0 10.7 8.5 5.9 Kazakhstan GDP per capita -6.1 8.8 9.5 7.3 5.6 Upper-middle-inco me gro up Expo rts o f go o ds and services -4.6 7.7 6.9 9.3 8.1 S T R UC T UR E o f t he E C O N O M Y G ro wt h o f c a pit a l a nd G D P ( %) 60 19 8 7 19 9 7 2006 2007 G ro wt h o f c a pit a l a nd G D P ( %) (% o f GDP ) 40 60 A griculture .. 12.0 5.9 6.6 20 Industry .. 27.3 42.1 44.3 40 M anufacturing .. 14.0 12.4 .. 0 20 Services .. 60.7 52.0 49.1 02 03 04 05 06 07 0 Ho useho ld final co nsumptio n expenditure .. 74.5 46.3 48.5 02 03 04 05 06 07 General go v't final co nsumptio n expenditure .. 12.4 10.2 1 1 .2 GCF GDP Impo rts o f go o ds and services .. 37.4 40.4 38.3 19 8 7 - 9 7 19 9 7 - 0 7 2006 2007 G ro wt h o f e xpo rt s a nd im po rt s ( %) (average annual gro wth) 20 A griculture -9.6 4.6 6.0 5.0 Industry .. 10.8 13.4 10.0 10 M anufacturing .. 8.4 7.9 7.0 Services .. 9.0 10.9 10.0 0 Ho useho ld final co nsumptio n expenditure 1 -1 .7 7.7 13.8 10.0 02 03 04 05 06 07 General go v't final co nsumptio n expenditure -8.0 7.3 6.2 10.0 -10 Gro ss capital fo rmatio n -26.7 17.4 28.8 10.0 Exports Imports Impo rts o f go o ds and services -17.2 6.1 12.1 9.6 No te: 2007 data are preliminary estimates. This table was pro duced fro m the Develo pment Eco no mics LDB database. * The diamo nds sho w fo ur key indicato rs in the co untry (in bo ld) co mpared with its inco me-gro up average. If data are missing, the diamo nd will be inco mplete. 104 Kazakhstan P R IC E S a nd G O V E R N M E N T F IN A N C E 19 8 7 19 9 7 2006 2007 Inf la t io n ( %) D o m e s t ic pric e s 30 (% change) Co nsumer prices .. 17.4 8.6 8.8 20 Implicit GDP deflato r .. 16.1 21.6 14.8 10 G o v e rnm e nt f ina nc e (% o f GDP , includes current grants) 0 Current revenue .. 20.7 27.1 27.0 02 03 04 05 06 07 Current budget balance .. -3.8 12.9 10.8 GDP deflator CPI Overall surplus/deficit .. -7.1 7.6 4.7 TRADE 19 8 7 19 9 7 2006 2007 E xpo rt a nd im po rt le v e ls ( US $ m ill.) (US$ millio ns) To tal expo rts (fo b) .. 6,899 38,762 46,329 60,000 Fuel and o il pro ducts .. 2,21 6 26,279 30,303 Ferro us metals .. 951 2,411 2,555 40,000 M anufactures .. 1,491 3,978 6,797 To tal impo rts (cif) .. 7,1 76 24,1 20 30,289 Fo o d .. 474 1 74 ,1 1,374 20,000 Fuel and energy .. 628 3,051 4,051 Capital go o ds .. 1,462 10,722 14,222 0 01 02 03 04 05 06 07 Expo rt price index (2000=100) .. 85 283 309 Impo rt price index (2000=100) .. 120 249 286 Exports Imports Terms o f trade (2000=1 00) .. 71 1 13 108 B A LA N C E o f P A Y M E N T S 19 8 7 19 9 7 2006 2007 C urre nt a c c o unt ba la nc e t o G D P ( %) (US$ millio ns) Expo rts o f go o ds and services .. 7,741 41,570 49,437 2 Impo rts o f go o ds and services .. 8,300 32,840 39,731 0 Reso urce balance .. -559 8,730 9,705 01 02 03 04 05 06 07 Net inco me .. -315 -9,317 1 -1 ,321 -2 Net current transfers .. 75 -1,207 -1,500 -4 Current acco unt balance .. -799 -1,795 1 -3,1 7 -6 Financing items (net) .. 1,279 12,869 1 3,1 7 Changes in net reserves .. -480 1 -1 ,075 0 -8 M emo : Reserves including go ld (US$ millio ns) .. 2,291 9,1 1 27 9,1 1 27 Co nversio n rate (DEC, lo cal/US$ ) 2.40E-3 75.4 126.1 122.6 E X T E R N A L D E B T a nd R E S O UR C E F LO WS 19 8 7 19 9 7 2006 2007 C o m po s it io n o f 2 0 0 6 de bt ( US $ m ill.) (US$ millio ns) To tal debt o utstanding and disbursed .. 4,078 74,148 .. IB RD .. 648 502 427 A: 502 D: 321 IDA .. 0 0 0 E: 939 G: 12,579 To tal debt service .. 483 14,532 .. IB RD .. 34 161 172 IDA .. 0 0 0 Co mpo sitio n o f net reso urce flo ws Official grants .. 54 51 .. Official credito rs .. 444 -23 .. P rivate credito rs .. 777 25,768 .. Fo reign direct investment (net inflo ws) .. 1,321 6,143 .. P o rtfo lio equity (net inflo ws) .. 0 2,797 .. F: 59,807 Wo rld B ank pro gram Co mmitments .. 247 30 0 A - IBRD E - Bilateral Disbursements .. 202 29 67 B - IDA D - Other multilateral F - Private P rincipal repayments .. 0 130 143 C - IM F G - Short-term Net flo ws .. 202 -101 -76 Interest payments .. 34 31 29 Net transfers .. 167 -132 -105 No te: This table was pro duced fro m the Develo pment Eco no mics LDB database. 9/24/08 105 Annex 16: Maps KAZAKHSTAN: TVEM 106