BAHRAINǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Bahrain SABER Country Report STUDENT ASSESSMENT 2013 Status Key Policy Areas for Student Assessment 1. Classroom Assessment In Bahrain, a formal, system-level document provides guidelines for classroom assessment. Some resources, such as scoring criteria and rubrics for grading students' work, are available system-wide to support teachers in their classroom assessment activities. Opportunities to build skills in classroom assessment are available to teachers through pre- and in-service training. At the same time, classroom assessment practices are perceived as being of moderate quality with a tendency to overly focus on information recall rather than higher-order skills. 2. Examinations The Examination for the Certificate of the Completion of General Secondary Education is administered to students in grades 10, 11, and 12. It covers the subjects of social studies, mathematics, science, language arts, Islamic studies, sports, and art. Examination results are mainly used for decisions about student promotion to the next grade level in secondary school, and student selection to higher education institutions. Regular funding is allocated by the government, and the Examination Directorate, which has primary responsibility for running the examination, is adequately staffed. However, there are no formal mechanisms in place to monitor the overall impact of the examination on the education system, particularly in terms of its effects on students, teachers, and schools. 3. National Large-Scale Assessment (NLSA) The National Examinations, which monitors education quality at the system level and informs policy design, evaluation, and decision-making, assesses all students in grades 3, 6, 9, and 12. First administered in 2009, it has have since been carried out in 2010 and 2013. Regular funding is allocated by the government. The National Authority for Qualifications and Quality Assurance of Education and Training, which implements the assessment, has an adequate number of staff to carry out activities effectively. However, only some mechanisms are in place to monitor the actual impact of the NLSA on policy dialog, design, and decision-making. While a permanent oversight committee is in place, there are no expert review groups. 4. International Large-Scale Assessment (ILSA) Bahrain has participated in three ILSAs in the past 10 years – TIMSS 2003, 2007, and 2011. Funding for ILSAs is allocated at discretion. A national coordinator and team are responsible for carrying out ILSA activities. In-country opportunities to learn about ILSAs are not very varied, but available to a broad audience. Country-specific ILSA results are widely disseminated in a timely manner. The results are used in a variety of ways to inform decision making in Bahrain, including tracking the impact of reforms on student achievement levels and informing resource allocation. THE WORLD BANK BAHRAINǀ STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Introduction student assessment systems. The framework is structured around two main dimensions of assessment Bahrain has focused on increasing student learning systems: the types/purposes of assessment activities and outcomes by improving the quality of education in the the quality of those activities. country. An effective student assessment system is an important part of improving education quality and Assessment types and purposes learning outcomes as it provides the necessary information to meet stakeholders’ decision-making Assessment systems tend to be comprised of three main needs. In order to gain a better understanding of the types of assessment activities, each of which serves a strengths and weaknesses of its existing assessment different purpose and addresses different information system, Bahrain decided to benchmark this system using needs. These three main types are: classroom standardized tools developed under The World Bank’s assessment, examinations, and large-scale, system level Systems Approach for Better Education Results (SABER) assessments. program. SABER is an evidence-based program to help countries systematically examine and strengthen the Classroom assessment provides real-time information to performance of different aspects of their education support ongoing teaching and learning in individual systems. classrooms. Classroom assessments use a variety of formats, including observation, questioning, and paper- What is SABER-Student Assessment? and-pencil tests, to evaluate student learning, generally on a daily basis. SABER-Student Assessment is a component of the SABER Examinations provide a basis for selecting or certifying program that focuses specifically on benchmarking students as they move from one level of the education student assessment policies and systems. The goal of system to the next (or into the workforce). All eligible SABER-Student Assessment is to promote stronger students are tested on an annual basis (or more often if assessment systems that contribute to improved the system allows for repeat testing). Examinations cover education quality and learning for all. the main subject areas in the curriculum and usually involve essays and multiple-choice questions. National governments and international agencies are increasingly recognizing the key role that assessment of Large-scale, system-level assessments provide feedback student learning plays in an effective education system. on the overall performance of the education system at The importance of assessment is linked to its role in: particular grades or age levels. These assessments (i) providing information on levels of student typically cover a few subjects on a regular basis (such as learning and achievement in the system; every 3 to 5 years), are often sample based, and use (ii) monitoring trends in education quality over multiple-choice and short-answer formats. They may be time; national or international in scope. (iii) supporting educators and students with real- time information to improve teaching and Appendix 1 summarizes the key features of these main learning; and types of assessment activities. (iv) holding stakeholders accountable for results. SABER-Student Assessment methodology The SABER-Student Assessment framework is built on the available evidence base for what an effective assessment system looks like. The framework provides guidance on how countries can build more effective SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 2 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Quality drivers of an assessment system Table 1: Framework for building an effective assessment system, with indicator areas The key considerations when evaluating a student assessment system are the individual and combined quality of assessment activities in terms of the adequacy of the information generated to support decision- making. There are three main drivers of information quality in an assessment system: enabling context, system alignment, and assessment quality. Enabling context refers to the broader context in which the assessment activity takes place and the extent to which that context is conducive to, or supportive of, the assessment. It covers such issues as the legislative or policy framework for assessment activities; institutional and organizational structures for designing, carrying out, The indicators are identified based on a combination of or using results from the assessment; the availability of criteria, including: sufficient and stable sources of funding; and the presence of trained assessment staff. x professional standards for assessment; x empirical research on the characteristics of effective System alignment refers to the extent to which the assessment systems, including analysis of the assessment is aligned with the rest of the education characteristics that differentiate between the system. This includes the degree of congruence between assessment systems of low- versus high-performing assessment activities and system learning goals, nations; and standards, curriculum, and pre- and in-service teacher x theory — that is, general consensus among experts training. that it contributes to effective assessment. Assessment quality refers to the psychometric quality of Levels of development the instruments, processes, and procedures for the assessment activity. It covers such issues as design and implementation of assessment activities, analysis and The World Bank has developed a set of standardized interpretation of student responses to those activities, questionnaires and rubrics for collecting and and the appropriateness of how assessment results are evaluating data on the three assessment types and reported and used. related quality drivers. Crossing the quality drivers with the different The questionnaires are used to collect data on the assessment types/purposes provides the framework and characteristics of the assessment system in a particular broad indicator areas shown in Table 1. This framework country. The information from the questionnaires is then is a starting point for identifying indicators that can be applied to the rubrics in order to judge the development used to review assessment systems and plan for their level of the country’s assessment system in different improvement. areas. The basic structure of the rubrics for evaluating data collected using the standardized questionnaires is summarized in Appendix 2. The goal of the rubrics is to provide a country with some sense of the development SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 3 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 level of its assessment activities compared to best or Education in Bahrain recommended practice in each area. For each indicator, the rubric displays four development levels—Latent, Emerging, Established, and Advanced. These levels are Bahrain is a high-income country in the Middle East and artificially constructed categories chosen to represent North Africa region. In 2010, GDP per capita (current key stages on the underlying continuum for each US$, 2010) was $18,334, and annual growth was 4.5 indicator. Each level is accompanied by a description of percent. what performance on the indicator looks like at that In Bahrain, basic education lasts for 9 years and is level. compulsory for children between the ages of six and 14. Secondary education lasts three years. In 2002 (the year x Latent is the lowest level of performance; it for which the latest data are available), net enrolment represents absence of, or deviation from, the desired rates at the primary and secondary levels were 99 attribute. percent and 90 percent respectively. In 2008, 11.7 x Emerging is the next level; it represents partial percent of government spending in Bahrain was presence of the attribute. allocated to education, equivalent to 2.9 percent of GDP. x Established represents the acceptable minimum standard. Bahrain recently implemented King Hamad’s Schools of x Advanced represents the ideal or current best the Future project, with the goal to integrate Information practice. and Communications Technology (ICT) into various aspects of the education system. For example, the A summary of the development levels for each project seeks to help teachers individually assess their assessment type is presented in Appendix 3. students’ progress through the use of various e-learning activities. In addition, in 2008, the country instituted the National Authority for Qualifications and Quality In reality, assessment systems are likely to be at Assurance of Education and Training (NAQQAET) to different levels of development in different areas. For monitor the quality of education at all levels. Bahrain is example, a system may be Established in the area of also implementing other reforms to improve its examinations, but Emerging in the area of large-scale, education system, including introducing greater system-level assessment, and vice versa. While intuition flexibility for students at the secondary stage to pursue suggests that it is probably better to be further along in vocational training and enter the labor force, as well as as many areas as possible, the evidence is unclear as improving recruitment, training, and employment to whether it is necessary to be functioning at conditions for teachers. Advanced levels in all areas. Therefore, one might view the Established level as a desirable minimum outcome to achieve in all areas, but only aspire beyond that in those Detailed information on Bahrain’s student assessment areas that most contribute to the national vision or system was collected using the SABER-Student priorities for education. In line with these considerations, Assessment questionnaires and rubrics. It is important to the ratings generated by the rubrics are not meant to be remember that these tools primarily focus on additive across assessment types (that is, they are not benchmarking a country’s policies and arrangements for meant to be added to create an overall rating for an assessment activities at the system or macro level. assessment system; they are only meant to produce an Additional data would need to be collected to determine overall rating for each assessment type). The actual, on-the-ground practices in Bahrain, particularly methodology for assigning development levels is by teachers and students in schools. The following summarized in Appendix 4. sections discuss the findings for each assessment type, accompanied by suggested policy options. The suggested policy options were determined in collaboration with key local stakeholders based on Bahrain’s immediate interests and needs. Detailed, completed rubrics for each assessment type are provided in Appendix 5. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 4 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Classroom Assessment classroom assessment is used to diagnose student learning issues, provide feedback to students on their learning, inform parents about their child's learning, and Level of development plan next steps in instruction. Limited mechanisms are in place to systematically monitor the quality of classroom assessment practices. In Bahrain, Educational Assessment System is a formal, For example, classroom assessment is a required system-level document that provides guidelines for component of a teacher's performance evaluation and classroom assessment. It identifies methods of school inspection. System-wide reviews of the quality of classroom assessment, as well as tools that can be used education also include a focus on classroom assessment. to measure students’ performance. Some resources are available system-wide to support Suggested policy options teachers in their classroom assessment activities. For example, curriculum documents for different subject areas outline what students are expected to learn at 1. Introduce additional systematic mechanisms to different grade levels. The Educational Assessment monitor the quality of classroom assessment practices. System and the Teacher's Guide for Assessing the Daily For example, expand the NAQQAET school quality Tasks documents outline the level of performance that assurance reports to include more in-depth reviews of students are expected to reach in different subject areas classroom assessment practices. (Responsibility: at different grade levels. Textbooks and workbooks, as NAQAAET) well as scoring criteria and rubrics for grading students' work, also provide support for teachers’ classroom 2. Expand the availability of system-wide resources for assessment activities. teachers to conduct classroom assessment activities. For example, establish online assessment resources or At the same time, mechanisms are in place to computer-based testing with instant reports on systematically ensure that teachers develop skills and students’ performance. (Responsibility: MoE) expertise in classroom assessment. For example, there is 3. Introduce additional system-level mechanisms to in-service teacher training on how to design ensure that teachers develop skills and expertise in achievement tests and use formative classroom classroom assessment. For example, mandate that all assessment methods; pre-service teacher training; and teacher training programs include a required course on opportunities to participate in developing and scoring classroom assessment. (Responsibility: MoE) questions for large-scale assessments and examinations. In addition, school inspection and teacher supervision include a component focused on classroom assessment. Classroom assessment practices are perceived to be of moderate quality. Classroom activities tend to be focused on information recall and grade inflation is a serious problem. Classroom assessment information is required to be disseminated to all key stakeholders, including students, parents, and school district and Ministry of Education officials. There are adequate required uses of classroom assessment to support student learning. For example, SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 5 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Examinations However, there are very limited systematic mechanisms in place to ensure the quality of the examination. Only Level of development internal reviews and observers are used in the quality assurance process. There are no external reviews or observers. In addition, there is no piloting or field testing The Examination for the Certificate of the Completion of of instruments. Finally, there are no mechanisms, such as General Secondary Education has been administered a permanent oversight committee or expert review since 1958. A formal, system-level document, General groups, available to monitor the consequences of the Examination Regulations (2010), authorizes the examination in terms of its impact on students, teachers, examination, which is administered to students in grades and schools. 10, 11, and 12. The examination covers the subjects of Social Studies, Mathematics, Science, Language Arts, Suggested policy options Islamic Studies, Sports, and Art. The main purposes for which examination results are used are decisions about student promotion to the next grade level in secondary 1. Introduce additional systematic mechanisms to ensure school, and student selection to higher education the quality of the examination. For example, conduct a institutions. pilot before the examination is administered or commission external certifications or audits of the Regular funding is allocated by the government for the examination. (Responsibility: MoE) examination. Funding covers all core examination activities, including examination design and 2. Introduce a variety of system-level mechanisms to administration, data analysis and reporting, long- or monitor the consequences of the examination. For medium-term planning of program milestones, research example, establish a permanent oversight committee or and development, and staff training. expert review groups to review the impact of the examination on students, teachers, and schools. The Examination Directorate within the MoE is (Responsibility: MoE) responsible for running the examination. The Examination Directorate has all of the required facilities to carry out the examination and is adequately staffed. Bahrain offers a wide range of opportunities that prepare individuals to work on the examination. For example, there are university graduate programs in Bahrain University and other private universities in the country, as well as individual university courses (graduate and non-graduate) on educational measurement and evaluation relevant to examinations. There are also opportunities to attend non university training courses or workshops, and funding is available for attending international programs or workshops on educational measurement and evaluation. Up-to-date, compulsory courses and workshops on the examination are available to teachers. Teachers are also involved in most examination-related tasks, such as creating and selecting examination questions and administering and scoring the examination. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 6 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 National Large-Scale Assessment (NLSA) television. The main reports on the NLSA results contain information on achievement levels overall and by subgroups, as well as trends over time, overall and for Level of development subgroups. Some mechanisms are in place to monitor the The National Examinations, which monitor education consequences of the NLSA, including a permanent quality at the system level and inform policy design, oversight committee, which is the Education and evaluation, and decision making, assess all students in Training Reform Committee. In addition, funding is made grades 3, 6, 9, and 12. For grade 3 students, the available for internal research on the impact of the NLSA. assessment covers Arabic language and Mathematics. For grade 6 and grade 9 students, the examination covers Suggested policy options Arabic and English languages, Mathematics, and the Sciences. For grade 12 students, the assessment covers 1. Ensure that results from the National Examinations are Arabic language, English language, and problem solving. used to inform curricula, teacher training, and relevant Formal, system-level documents, Royal decrees 32 policies. In addition, ensure that the NLSA is aligned with (2008), 6 (2009), and 83 (2012), authorize the National other assessments carried out in the system, including Examinations, and are available to the public online. In classroom assessments and high-stakes examinations. addition, there is a written NLSA plan for the coming Additionally, establish regular focus groups or surveys of years, which specifies which grade levels will be tested key stakeholders to monitor the consequences of the and in which subject areas. NLSA for policy dialog, design, and decision making. (Responsibility: NAQAAET and MoE) Regular funding for the National Examinations is allocated by the government. This funding covers all core 2. Ensure that the comprehensive technical report on the activities as well as research and development. National Examinations is readily available to the general public. (Responsibility: NAQAAET and MoE) The NAQQAET, formerly known as the Quality Assurance Authority for Education and Training (QAAET), is a permanent, independent national authority operating under the guidance of a Board of Directors that is responsible for carrying out the National Examinations. The NAQQAET reports to the Cabinet of Ministers. The NAQQAET has an adequate number of full-time staff to carry out the National Examinations effectively, with no issues. A variety of mechanisms are in place to ensure the quality of the NLSA. For example, all proctors and administrators are trained according to a protocol and a pilot is conducted before the main data collection takes place. In addition, all booklets are numbered and discrepancies are required to be recorded on a standard sheet. Although a comprehensive technical report is available, its circulation is restricted. NLSA results are disseminated within 12 months after the assessment is administered. The results are featured in newspapers and magazines, and on radio and SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 7 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 International Large-Scale Assessment (ILSA) Country-specific ILSA results are disseminated throughout Bahrain in a timely manner. For example, the national report on the TIMSS 2007 results was published Level of development and made available on the MoE website. Copies of the national report were distributed to key stakeholders, and country results were communicated through a press release. Products providing feedback to schools and Bahrain has participated in three ILSAs in the past 10 educators are also systematically made available in years—Trends in International Mathematics and Science Bahrain. However, there has been limited media Study (TIMSS) 2003, 2007, and 2011—and has taken coverage of ILSA results, including commentary in concrete steps to participate in two more in the near editorials or columns. future—Progress in International Reading Literacy Study (PIRLS) 2016 and TIMSS 2015. Although there are formal Results from the ILSAs have been used to inform decision policy documents addressing Bahrain's participation in making in a variety of ways, including to track the impact ILSAs, they are not available to the public. of reforms on student achievement levels, as well as to inform curriculum improvement, teacher training Regular funding for ILSAs is allocated at the programs, and resource allocation. Decisions based on government’s discretion, and covers all core activities, ILSA results have had a positive impact on students' such as international participation fees, implementation achievement levels as evidenced by the advancement in of the assessment exercise in Bahrain, and processing achievement that occurred between TIMSS 2003 and and analysis of the data collected. In addition, funding TIMSS 2007. covers the implementation of a remedial action plan to improve students' performance, which is developed Suggested policy options after the ILSA results are made available and weaknesses in various competencies have been identified. 1. Improve dissemination and reporting of ILSA results A national coordinator and team carry out the ILSA through the media. For example, collaborate with activities in Bahrain. There are a sufficient number of various media outlets to ensure extensive coverage of team members with prior experience working on ILSAs results. (Responsibility: NAQAAET and MoE) who have the training and experience necessary to carry out the assessment activities effectively. 2. Expand local opportunities to learn about ILSAs. For example, develop local university or online courses on Bahrain offers some opportunities to learn about ILSAs, international assessments. (Responsibility: NAQAAET including workshops and meetings on using international and MoE) assessment databases, which are available to various audiences, including teachers, principals, and university students studying assessment or a related area. However, there is no funding available to attend international workshops or training on this topic. Further, there are no university or online courses on international assessments. Bahrain has not only met all technical standards required to have its data presented in the main displays of the international report, but has also contributed to new knowledge on ILSAs through being featured in the TIMSS 2007 Encyclopedia and the and TIMSS 2011 Encyclopedia, which are publicly available on the TIMSS website. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 8 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 1: Assessment Types and Their Key Differences Classroom Large-scale assessment Examinations Surveys National International Exit Entrance Purpose To provide To provide To provide To certify To select immediate feedback on feedback on the students as they students for feedback to overall health of comparative move from one further inform the system at performance of level of the educational classroom particular the education education opportunities instruction grade/age system at system to the level(s), and to particular next (or into the monitor trends grade/age workforce) in learning level(s) Frequency Daily For individual For individual Annually and Annually and subjects offered subjects offered more often more often on a regular on a regular where the where the basis (such as basis (such as system allows system allows every 3-5 years) every 3-5 years) for repeats for repeats Who is All students Sample or A sample of All eligible All eligible tested? census of students at a students students students at a particular grade particular grade or age level(s) or age level(s) Format Varies from Usually multiple Usually multiple Usually essay Usually essay observation to choice and short choice and short and multiple and multiple questioning to answer answer choice choice paper-and-pencil tests to student performances Coverage of All subject areas Generally Generally Covers main Covers main curriculum confined to a confined to one subject areas subject areas few subjects or two subjects Additional Yes, as part of Frequently Yes Seldom Seldom information the teaching collected from process students? Scoring Usually informal Varies from Usually involves Varies from Varies from and simple simple to more statistically simple to more simple to more statistically sophisticated statistically statistically sophisticated techniques sophisticated sophisticated techniques techniques techniques SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 9 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 2: Basic Structure of Rubrics for Evaluating Data Collected on a Student Assessment System Development Level LATENT ESTABLISHED (Absence of, or EMERGING (Acceptable deviation from, (On way to meeting minimum ADVANCED Dimension attribute) minimum standard) standard) (Best practice) Justification EC—ENABLING CONTEXT EC1—Policies EC2—Leadership, public engagement EC3—Funding EC4—Institutional arrangements EC5—Human resources SA—SYSTEM ALIGNMENT SA1—Learning/quality goals SA2—Curriculum SA3—Pre-, in-service teacher training AQ—ASSESSMENT QUALITY AQ1—Ensuring quality (design, administration, analysis) AQ2—Ensuring effective uses SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 10 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 3: Summary of the Development Levels for Each Assessment Type Assessment Type LATENT EMERGING ESTABLISHED ADVANCED Absence of, or deviation On way to meeting Acceptable minimum Best practice from, the attribute minimum standard standard There is no system-wide There is weak system- There is sufficient There is strong system- institutional capacity to wide institutional system-wide wide institutional support and ensure the capacity to support and institutional capacity to capacity to support and quality of classroom ensure the quality of support and ensure the ensure the quality of assessment practices. classroom assessment quality of classroom classroom assessment CLASSROOM practices. assessment practices. practices. ASSESSMENT There is no standardized There is a partially There is a stable There is a stable examination in place for stable standardized standardized standardized key decisions. examination in place, examination in place. examination in place and a need to develop There is institutional and institutional institutional capacity to capacity and some capacity and strong run the examination. limited mechanisms to mechanisms to monitor EXAMINATIONS The examination monitor it. The it. The examination is of typically is of poor examination is of high quality and is quality and is perceived acceptable quality and is perceived as fair and as unfair or corrupt. perceived as fair for free from corruption. most students and free from corruption. There is no NLSA in There is an unstable There is a stable NLSA in There is a stable NLSA in place. NLSA in place and a place. There is place and institutional need to develop institutional capacity capacity and strong institutional capacity to and some limited mechanisms to monitor NATIONAL (OR run the NLSA. mechanisms to monitor it. The NLSA is of high SYSTEM-LEVEL) Assessment quality and it. The NLSA is of quality and its LARGE-SCALE impact are weak. moderate quality and its information is ASSESSMENT information is effectively used to disseminated, but not improve education. always used in effective ways. There is neither a Participation in an ILSA There is more or less There is stable history of participation has been initiated, but stable participation in an participation in an ILSA in an ILSA nor plans to there still is need to ILSA. There is and institutional participate in one. develop institutional institutional capacity to capacity to run the ILSA. INTERNATIONAL capacity to carry out the carry out the ILSA. The The information from LARGE-SCALE ILSA. information from the the ILSA is effectively ILSA is disseminated, but used to improve ASSESSMENT not always used in education. effective ways. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 11 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 4: Methodology for Assigning 6. Scores for certain key dimensions under ‘Enabling Development Levels Context’ (in the case of EXAM, NLSA, and ILSA) and under ‘System Alignment’ (in the case of CLASS) were set as ceiling scores, i.e., the overall mean score for the 1. The country team or consultant collects information particular assessment type cannot be greater than the about the assessment system in the country. score for these key dimensions. These key variables include formal policy, regular funding, having a 2. Based on the collected information, a level of permanent assessment unit, and the quality of development and score is assigned to each dimension in assessment practices. the rubrics: x Latent = 1 score point x Emerging = 2 score points x Established = 3 score points x Advanced = 4 score points 3. The score for each quality driver is computed by aggregating the scores for each of its constituent dimensions. For example: The quality driver, ‘Enabling Context,’ in the case of ILSA, has 3 dimensions on which a hypothetical country receives the following scores: Dimension A = 2 points; Dimension B = 2 points; Dimension C = 3 points. The hypothetical country’s overall score for this quality driver would be: (2+2+3)/3 = 2.33 4. A preliminary level of development is assigned to each quality driver. 5. The preliminary development level is validated using expert judgment in cooperation with the country team and The World Bank Task Team Leader. For scores that allow a margin of discretion (i.e., to choose between two levels of development), a final decision has to be made based on expert judgment. For example, the aforementioned hypothetical country has an ‘Enabling Context’ score of 2.33, corresponding to a preliminary level of development of ‘Emerging or Established.’ Based on qualitative information not captured in the rubric, along with expert judgment, the country team chooses ‘Emerging’ as the most appropriate level. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 12 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Appendix 5: SABER-Student Assessment Rubrics for Bahrain This appendix provides the completed SABER-Student Assessment rubrics for each type of assessment activity in Bahrain. In each row of the rubric, the relevant selection is indicated by a thick border and an asterisk. The selection may include a superscript number that refers to the justification or explanation for the selection. The explanation or justification text can be located in the “Development-level rating justifications” section at the end of each rubric. If a row includes a superscript, but not a thick border and an asterisk, this means that insufficient information was available to determine the relevant selection in the row. 13 SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 BAHRAIN Classroom Assessment SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 14 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT AND SYSTEM ALIGNMENT Overall policy and resource framework within which classroom assessment activity takes place in a country or system, and the degree to which classroom assessment activity is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT AND SYSTEM ALIGNMENT 1: Setting clear guidelines for classroom assessment There is no system-level document that There is an informal system-level There is a formal system-level document This option does not apply to this provides guidelines for classroom document that provides guidelines for that provides guidelines for classroom dimension. assessment. classroom assessment. assessment.1 * This option does not apply to this This option does not apply to this The availability of the document is The document is widely available. 2 dimension. dimension. restricted. * ENABLING CONTEXT AND SYSTEM ALIGNMENT 2: Aligning classroom assessment with system learning goals There are no system-wide resources for There are scarce system-wide resources There are some system-wide resources There are a variety of system-wide teachers for classroom assessment. for teachers for classroom assessment. for teachers for classroom assessment. 3 resources available for teachers for classroom assessment. * There is no official curriculum or There is an official curriculum or There is an official curriculum or There is an official curriculum or standards document. standards document, but it is not clear standards document that specifies what standards document that specifies what what students are expected to learn or to students are expected to learn, but the students are expected to learn and to what level of performance. level of performance required is not clear. what level of performance. 4 ENABLING CONTEXT AND SYSTEM ALIGNMENT 3: * Having effective human resources to carry out classroom assessment activities There are no system-level mechanisms to This option does not apply to this There are some system-level mechanisms There are a variety of system-level ensure that teachers develop skills and dimension. to ensure that teachers develop skills and mechanisms to ensure that teachers expertise in classroom assessment. expertise in classroom assessment.5 develop skills and expertise in classroom assessment. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 15 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Quality of classroom assessment design, administration, analysis, and use. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of classroom assessment Classroom assessment practices suffer Classroom assessment practices are Classroom assessment practices are Classroom assessment practices are from widespread weaknesses or there is known to be weak. known to be of moderate quality. 6 known to be generally of high quality. no information available on classroom assessment practices. * There are no mechanisms to monitor the There are ad hoc mechanisms to monitor There are limited systematic mechanisms There are varied and systematic quality of classroom assessment the quality of classroom assessment to monitor the quality of classroom mechanisms in place to monitor the practices. practices. assessment practices. 7 quality of classroom assessment * practices. ASSESSMENT QUALITY 2: Ensuring effective uses of classroom assessment Classroom assessment information is not This option does not apply to this Classroom assessment information is Classroom assessment information is required to be disseminated to key dimension. required to be disseminated to some key required to be disseminated to all key stakeholders. stakeholders. stakeholders. 8 * There are no required uses of classroom There are limited required uses of There are adequate required uses of There are adequate required uses of assessment to support student learning. classroom assessment to support student classroom assessment to support student classroom assessment to support student learning. learning, excluding its use as an input for learning, including its use as an input for external examination results. 9 * external examination results. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 16 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Classroom Assessment: Development-level rating justifications 1. The Educational Assessment System document, authorized by the Center of Measurement and Assessment in Bahrain in 2010, provides guidelines for classroom assessment. It identifies the methods of classroom assessment, as well as the tools that may be used to measure student's performance in applying each method. In addition, guidelines for classroom assessment are found in the Teacher's Guide for Assessing the Daily Tasks of Students in Basic Education. 2. The document has been made available to all schools and teachers. It is available online, as in-service courses for teachers, and in all schools and educational directorates at the Ministry of Education. 3. There are some system-wide resources available for teachers to engage in classroom assessment. For example, curriculum documents for different subjects outline what students are expected to learn in different subject areas at different grade or age levels, and the Educational Assessment System and the Teacher's Guide for Assessing the Daily Tasks documents outline the level of performance that students are expected to reach in different subject areas at different grade or age levels. In addition, there are textbooks or workbooks that provide support for classroom assessment and scoring criteria or rubrics for students' work. However, there are no online assessment resources or computer-based testing with instant reports on students' performance. 4. An official curriculum or standards document specifies what students are expected to learn and to what level of performance. 5. There are some system-level mechanisms in place to ensure that teachers develop skills and expertise in classroom assessment. For example, there is in- service teacher training on elaborating achievement tests and using classroom assessment methods, pre-service teacher training, opportunities to participate in item development for or scoring of large-scale assessments or exams, and school inspection or teacher supervision which includes a component focused on classroom assessment. However, not all teacher training programs include a required course on classroom assessment, there are no online resources on classroom assessment, and there are no opportunities to participate in conferences or workshops. 6. Classroom assessment practices are known to be of moderate quality. It is uncommon to observe errors in the scoring or grading of work, and teachers commonly use explicit or a priori criteria for scoring or grading work. Classroom assessments do not rely only on multiple-choice, selection-type questions, and the uneven application of standards for grading students' work is not a serious problem. Classroom assessment activities provide useful feedback to students, and parents are informed about students' grades. However, classroom assessment activities are commonly about recalling information, and grade inflation is a serious problem. 7. There are limited systematic mechanisms in place to monitor the quality of classroom assessment practices. Although classroom assessment is a required component of a teacher's performance evaluation and school inspection or teacher supervision, and there are national or other system-wide reviews of the quality of education which include a focus on classroom assessment, there is no external moderation system which reviews the difficulty of classroom assessment activities and the appropriateness of scoring criteria, and no government funding is available for research on the quality of classroom assessment activities and how to improve classroom assessment. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 17 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 8. Classroom assessment information is required to be disseminated to all key stakeholders, including students, parents, and school district and Ministry of Education officials. 9. There are adequate required uses of classroom assessment to support student learning, excluding its use as an input for external examination results. Classroom assessment is used to support student learning by diagnosing student learning issues, providing feedback to students on their learning, informing parents about their child's learning, and planning the next steps in instruction. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 18 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 BAHRAIN Examinations SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 19 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which assessment activity takes place in a country or system and the extent to which that framework is conducive to, or supportive of, the assessment activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies No standardized examination has taken The standardized examination has been The examination is a stable program that This option does not apply to this place. operating on an irregular basis. has been operating regularly. 1 dimension There is no policy document that There is an informal or draft policy * There is a formal policy document that This option does not apply to this authorizes the examination. document that authorizes the authorizes the examination.2 dimension. examination. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public.3 public. dimension. This option does not apply to this This option does not apply to this * The policy document addresses some key The policy document addresses all key dimension. dimension. aspects of the examination. aspects of the examination. 4 * ENABLING CONTEXT 2: Having strong leadership All stakeholder groups strongly oppose Most stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the the examination or are indifferent to it. examination. examination. 5 examination. There are no attempts to improve the This option does not apply to this There are independent attempts to * There are coordinated attempts to examination by stakeholder groups. dimension. improve the examination by stakeholder improve the examination by stakeholder groups. groups.6 * Efforts to improve the examination are This option does not apply to this Efforts to improve the examination are This option does not apply to this not welcomed by the leadership in charge dimension. generally welcomed by the leadership in dimension. of the examination charge of the examination.7 * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 20 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding There is no funding allocated for the There is irregular funding allocated for There is regular funding allocated for the This option does not apply to this examination. the examination. examination.8 dimension. * This option does not apply to this Funding covers some core examination Funding covers all core examination This option does not apply to this dimension. activities: design, administration, data activities: design, administration, data dimension. processing or reporting. processing and reporting. 9 * This option does not apply to this Funding does not cover research and This option does not apply to this Funding covers research and dimension. development.10 dimension. development. * ENABLING CONTEXT 4: Having strong organizational structures The examination office does not exist or is The examination office is newly The examination office is a stable This option does not apply to this newly established. established. organization. 11 dimension. * The examination office is not accountable This option does not apply to this The examination office is accountable to This option does not apply to this to an external board or agency. 12 dimension. an external board or agency. dimension. * Examination results are not recognized by Examination results are recognized by Examination results are recognized by Examination results are recognized by any certification or selection system. certification or selection system in the one certification or selection system in two or more certification or selection country. another country. systems in another country.13 * The examination office does not have the The examination office has some of the The examination office has all of the The examination office has state-of-the- required facilities to carry out the required facilities to carry out the required facilities to carry out the art facilities to carry out the examination. examination. examination. examination. 14 * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 21 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having effective human resources There is no staff to carry out the The examination office is inadequately The examination office is adequately The examination office is adequately examination. staffed to effectively carry out the staffed to carry out the examination staffed to carry out the assessment examination, issues are pervasive. effectively, with minimal issues. effectively, with no issues. 15 * The country does not offer opportunities This option does not apply to this The country offers some opportunities The country offers a wide range of that prepare for work on the dimension. that prepare for work on the opportunities that prepare for work on examination. examination. the examination. 16 * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 22 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 SYSTEM ALIGNMENT Degree to which the assessment is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning examinations with learning goals and opportunities to learn It is not clear what the examination This option does not apply to this There is a clear understanding of what the This option does not apply to this measures. dimension. examination measures.17 dimension. What the examination measures is This option does not apply to this What is measured by the examination is * This option does not apply to this questioned by some stakeholder groups. dimension. largely accepted by stakeholder groups.18 dimension. * Material to prepare for the examination is There is some material to prepare for the There is comprehensive material to There is comprehensive material to minimal and it is only accessible to very examination that is accessible to some prepare for the examination that is prepare for the examination that is few students. students. accessible to most students. 19 accessible to all students. * SYSTEM ALIGNMENT 2: Providing teachers with opportunities to learn about the examination There are no courses or workshops on There are no up-to-date courses or There are up-to-date voluntary courses or There are up-to-date compulsory courses examinations available to teachers. workshops on examinations available to workshops on examinations available to or workshops on examinations for teachers. teachers. teachers. 20 Teachers are excluded from all Teachers are involved in very few Teachers are involved in some Teachers are involved in * most examination-related tasks. examination-related tasks. examination-related tasks. examination-related tasks. 21 * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 23 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Degree to which the assessment meets quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring quality There is no technical report or other There is some documentation on the There is a comprehensive technical report There is a comprehensive, high quality documentation. examination, but it is not in a formal but with restricted circulation. 22 technical report available to the general report format. public. * There are no mechanisms in place to This option does not apply to this There are limited systematic mechanisms There are varied and systematic ensure the quality of the examination. dimension. in place to ensure the quality of the mechanisms in place to ensure the quality examination.23 of the examination. * ASSESSMENT QUALITY 2: Ensuring fairness Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the Inappropriate behavior surrounding the examination process is high. examination process is moderate. examination process is low. examination process is marginal. 24 The examination results lack credibility The examination results are credible for The examination results are credible for This option does not apply to this * for all stakeholder groups. some stakeholder groups. all stakeholder groups.25 dimension. * The majority of the students (over 50 A significant proportion of students (10 A small proportion of students (less than All students can take the examination; percent) may not take the examination percent to 50 percent) may not take the 10 percent) may not take the examination there are no language, gender or other because of language, gender, or other examination because of language, because of language, gender, or other equivalent barriers.26 equivalent barriers. gender, or other equivalent barriers. equivalent barriers. * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 24 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 3: Using examination information in a fair way Examination results are not used in a Examination results are used by some Examination results are used by most Examination results are used by all proper way by all stakeholder groups. stakeholder groups in a proper way. stakeholder groups in a proper way. stakeholder groups in a proper way. 27 * Student names and results are public. This option does not apply to this Students’ results are confidential. 28 This option does not apply to this dimension. dimension. * ASSESSMENT QUALITY 4: Ensuring positive consequences of the examination There are no options for students who do There are very limited options for There are some options for students who There is a variety of options for students not perform well on the examination, or students who do not perform well on the do not perform well on the examination. who do not perform well on the 29 students must leave the education examination. examination. system. * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There is a variety of mechanisms in place monitor the consequences of the dimension. monitor the consequences of the to monitor the consequences of the examination.30 examination. examination. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 25 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Examinations: Development-level rating justifications 1. The Examination for the Certificate of the Completion of General Secondary Education was first administered in 1958. The examination covers the subjects of Social Studies, Mathematics, Science, Language Arts, Islamic Studies, Sports, and Art in grades 10, 11, and 12. The examination's main purposes are for student selection or promotion to grades, courses, or tracks in secondary school, and student selection to university or other higher education institutions. 2. The Ministry of Education’s General Examination Regulations from 2010 authorizes the examination. 3. The policy document is available only to parties in charge of the preparation and implementation of the examination. 4. The policy document addresses all key aspects of the examination, including governance, distribution of power, responsibilities among key entities, the purpose of the examination, authorized uses of results, procedures for special or disadvantaged students, and procedures to investigate and address security breaches, cheating, or other forms of inappropriate behavior. Additionally, it states funding sources, specifies who can sit for the examination, identifies rules about preparation, and explains alignment with curricula and standards and the format of the examination questions. 5. Most stakeholder groups support the examination. Policy makers, educators, students, parents, the media, and universities all support the examination. However, it is unclear what level of support teacher unions, think tanks, NGOs, and employers have for the examination. 6. Stakeholder groups have made coordinated attempts to improve the examination. 7. Leadership in charge of the examination generally welcomes efforts to improve the examination. 8. There is regular funding allocated by the government for the examination. 9. Funding covers all core examination activities, including examination design and administration, data analysis and reporting, long- or medium-term planning of program milestones, research and development, and staff training. 10. Funding for the examination does not cover research and development activities. 11. The Examination Directorate, which has had primary responsibility for running the examination since 1958, is an office or branch within the Ministry of Education. 12. The examination office is not accountable to an external board or agency, as it is a branch within the Ministry of Education. 13. Examination results are recognized by certification or selection systems in other Gulf countries. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 26 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 14. The examination office has all of the required facilities to carry out the examination, including computers for all technical staff, secure storage facilities, a secure building, access to adequate computer servers, the ability to backup data, and adequate communication tools. 15. The examination office is adequately staffed with full-time and permanent staff to carry out the assessment effectively, with no issues. 16. Bahrain offers a wide range of opportunities that prepare for work on the examination. For example, on the topic of educational measurement and evaluation, there are university graduate programs housed in the Bahrain University and other private universities in the kingdom, as well as university courses (graduate and non-graduate). There are also opportunities to attend non-university training courses or workshops and funding for attendance at international programs, courses, or workshops on educational measurement and evaluation. 17. There is a clear understanding that the examination measures national school curriculum guidelines or standards. 18. Stakeholder groups largely accept what is measured by the examination. 19. There is comprehensive material to prepare for the examinations that is accessible to most students. All previous exam questions and their corresponding answers are available on the Ministry of Education's website and that of the HM King Hamad’s Schools of the Future Project, which is accessible at www.moe.gov.bh/khsfp/. However, information on how to prepare for the examination, the framework document explaining what is measured on the examination, and the report on the strengths and weaknesses in student performance are unavailable to students preparing for the examination. 20. Up-to-date compulsory courses or workshops on examinations are available for teachers. 21. Teachers are involved in most examination-related tasks. For example, teachers are involved in selecting or creating examination questions and examination scoring guides, administering and scoring the examination, acting as a judge, and supervising examination procedures. 22. A comprehensive technical report is available but with restricted circulation. 23. There are very limited systematic mechanisms in place to ensure the quality of the examination. Internal review or observers are in place; however, there is no external review or observers, external certification or audit, pilot or field testing, or translation verification. 24. Inappropriate behavior surrounding the examination process is marginal. Issues such as copying from other candidates, using unauthorized materials such as prepared answers and notes, and intimidation of examination supervisors do not typically occur during the examination process. 25. All stakeholder groups perceive the examination results as credible. 26. All students can take the examination regardless of location, background, or ability to pay. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 27 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 27. All stakeholder groups use examination results in a proper way. 28. Students' results are confidential as only the student and persons with a legitimate, professional interest in the test taker can know the results. 29. There are some options for students who do not perform well on the examination. For example, students may retake the examination or attend remedial or preparatory courses in order to prepare to retake the examination. However, there are no options for students to repeat the grade, or opt for less selective schools, universities, or tracks. 30. There are no mechanisms, such as a permanent oversight committee or expert review groups, in place to monitor the consequences of the examination. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 28 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 BAHRAIN National (or System-Level) Large-Scale Assessment (NLSA) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 29 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which NLSA activity takes place in a country or system and the extent to which that framework is conducive to, or supportive of, the NLSA activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for NLSA No NLSA exercise has taken place. The NLSA has been operating on an The NLSA is a stable program that has This option does not apply to this irregular basis. been operating regularly. 1 dimension. * There is no policy document pertaining to There is an informal or draft policy There is a formal policy document that This option does not apply to this NLSA. document that authorizes the NLSA. authorizes the NLSA.2 dimension. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public. public.3 dimension. * There is no plan for NLSA activity. This option does not apply to this There is a general understanding that the There is a written NLSA plan for the dimension. NLSA will take place. coming years. 4 * ENABLING CONTEXT 2: Having strong public engagement for NLSA All stakeholder groups strongly oppose Some stakeholder groups oppose the Most stakeholders groups support the All stakeholder groups support the NLSA. the NLSA or are indifferent to it. NLSA. NLSA. 5 * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 30 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having regular funding for NLSA There is no funding allocated to the NLSA. There is irregular funding allocated to the There is regular funding allocated to the This option does not apply to this NLSA. NLSA. 6 dimension. This option does not apply to this Funding covers some core NLSA activities: Funding covers all core NLSA activities: * This option does not apply to this dimension. design, administration, analysis and design, administration, analysis and dimension. reporting. reporting.7 This option does not apply to this Funding does not cover research and This option does not apply to this * Funding covers research and dimension. development activities. dimension. development activities. 8 * ENABLING CONTEXT 4: Having strong organizational structures for NLSA There is no NLSA office, ad hoc unit or The NLSA office is a temporary agency or The NLSA office is a permanent agency, This option does not apply to this team. group of people. institution or unit.9 dimension. This option does not apply to this Political considerations regularly hamper Political considerations sometimes * Political considerations never hamper dimension. technical considerations. hamper technical considerations. technical considerations. 10 * This option does not apply to this The NLSA office is not accountable to a The NLSA office is accountable to a clearly This option does not apply to this dimension. clearly recognized body. recognized body. 11 dimension. * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 31 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 5: Having effective human resources for NLSA There is no staff allocated for running an The NLSA office is inadequately staffed to The NLSA office is adequately staffed to The NLSA office is adequately staffed to NLSA. effectively carry out the assessment. carry out the NLSA effectively, with carry out the NLSA effectively, with no minimal issues. issues. 12 * The country does not offer opportunities This option does not apply to this The country offers some opportunities to The country offers a wide range of that prepare individuals for work on dimension. prepare individuals for work on the NLSA. opportunities to prepare individuals for 13 NLSA. work on the NLSA. 13 * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 32 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 SYSTEM ALIGNMENT Degree to which the NLSA is coherent with other components of the education system. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Aligning the NLSA with learning goals It is not clear if the NLSA is based on This option does not apply to this The NLSA measures performance against This option does not apply to this curriculum or learning standards. dimension. curriculum or learning standards. 14 dimension. * What the NLSA measures is generally This option does not apply to this What the NLSA measures is questioned by What the NLSA measures is largely questioned by stakeholder groups. dimension. some stakeholder groups. accepted by stakeholder groups. 15 * There are no mechanisms in place to There are ad hoc reviews of the NLSA to There are regular internal reviews of the This option does not apply to this ensure that the NLSA accurately ensure that it measures what it is NLSA to ensure that it measures what it is dimension. measures what it is supposed to measure. intended to measure. intended to measure.16 SYSTEM ALIGNMENT 2: * Providing teachers with opportunities to learn about the NLSA There are no courses or workshops on the There are occasional courses or There are some courses or workshops on There are widely available high quality NLSA. workshops on the NLSA. the NLSA offered on a regular basis. courses or workshops on the NLSA offered on a regular basis. 17 * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 33 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Degree to which the NLSA meets technical standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of the NLSA No options are offered to include all This option does not apply to this At least one option is offered to include all Different options are offered to include all groups of students in the NLSA. dimension. groups of students in the NLSA. groups of students in the NLSA. 18 * There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to There are a variety of mechanisms in ensure the quality of the NLSA. dimension. ensure the quality of the NLSA. place to ensure the quality of the NLSA. 19 * There is no technical report or other There is some documentation about the There is a comprehensive technical report There is a comprehensive, high quality documentation about the NLSA. technical aspects of the NLSA, but it is not but with restricted circulation. 20 technical report available to the general in a formal report format. public. * ASSESSMENT QUALITY 2: Ensuring effective uses of the NLSA NLSA results are not disseminated. NLSA results are poorly disseminated. NLSA results are disseminated in an This option does not apply to this effective way. 21 dimension. * NLSA information is not used or is used in This option does not apply to this NLSA results are used by some NLSA information is used by all ways inconsistent with the purposes or dimension. stakeholder groups in a way that is stakeholder groups in a way that is the technical characteristics of the consistent with the purposes and consistent with the purposes and assessment. technical characteristics of the technical characteristics of the assessment. 22 assessment. There are no mechanisms in place to This option does not apply to this There are some mechanisms in place to * There are a variety of mechanisms in monitor the consequences of the NLSA. dimension. monitor the consequences of the NLSA. 23 place to monitor the consequences of the NLSA. * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 34 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 National (of System-Level) Large Scale Assessment (NLSA): Development-level rating justifications 1. The National Examinations, which monitor education quality at the system level and assist with policy design, evaluation, and decision making, assess all students in grades 3, 6, 9, and 12. These assessments have been in place for less than five years. Initially, the assessments were administered to students in grades 3 and 6 in 2009, with grades 9 and 12 added in 2010 and 2013 respectively. For grade 3 students, the assessment covers Arabic language and Mathematics, while for students in grade 6 and grade 9, the examination covers Arabic and English languages, Mathematics, and Science. For grade 12 students, the assessment covers Arabic language, English language, and problem solving. 2. HM the King of the Kingdom of Bahrain authorized Royal decree number 32 in 2008 to establish and organize the National Authority for Qualifications and Quality Assurance of Education and Training (NAQQAET). This decree was subsequently amended by decree number 6 in 2009 and decree number 83 in 2012. 3. Royal decrees 32 (from 2008), 6 (from 2009), and 83 (from 2012), which authorize the NLSA, are available online to the public at www.legalaffairs.gov.bh. 4. There is a written NLSA plan for the coming years. As one of the initiatives mentioned in the National Economic Strategy to help achieve Economic Vision 2030, the NAQQAET has been adopted to reform the National Education System. It was given the responsibility "of standard setting and quality control by using a series of national examinations and periodic assessments in public schools and of submitting general reports thereto."(Source: National Economic Strategy 2009-2014, p. 49). The plan specifies who will be tested and in which subject areas. The plan is easily accessible by the public. Information related to the national examinations may be retrieved through the NAQQAET website: http://www.qaa.bh. 5. Most stakeholder groups support the NLSA. While policy makers, educators, the media, universities, and employers support the NLSA, students and parents are neutral to it. It is unclear what level of support teacher unions, think tanks, NGOs, and similar organizations maintain. 6. The government allocates regular funding to the NLSA. 7. Funding covers all core NLSA activities, including design, administration, analysis, reporting, long- or medium- term planning of program milestones, research and development, and staff training. 8. Funding for the NLSA covers research and development activities. 9. The NAQQAET, formerly known as the Quality Assurance Authority for Education and Training (QAAET), is a permanent, independent national authority that operates under the guidance of a Board of Directors established by decree number 7 of 2009, and reports to the Cabinet of Ministers. 10. Political considerations never hamper technical considerations, and assessment results have never been withheld due to political reasons. 11. The NAQQAET is accountable to a clearly recognized body, the Cabinet of Ministers, and is subject to its supervision. NAQQAET's performance is monitored by the Education and Training Reform Committee, as it is one of the national initiatives established to develop education and training in the Kingdom of Bahrain. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 35 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 12. The NAQQAET has an adequate number of permanent and full-time staff to carry out the NLSA effectively, with no issues. 13. Bahrain offers university courses on educational measurement and evaluation, which prepare individuals for work on NLSAs. 14. The NLSA measures performance against national curriculum guidelines or learning standards for grades 3, 6, and 9 and internationally recognized curriculum guidelines or learning standards for grade 12. Grade 12 National Examinations in Arabic and Problem Solving are set at the Cambridge AS Level (UK), and the grade 12 National Examination in English is set at the Vantage, Level B2 of the Common European Framework (CEF). 15. Stakeholder groups largely accept what is measured by the NLSA. 16. There are regular internal reviews of the alignment between the assessment instrument and what it is supposed to measure. 17. There are widely available high quality courses or workshops on the NLSA offered on a regular basis. Additionally, most courses provide teachers with relevant resources that they can use in their classrooms. 18. Accommodations or alternative assessments are provided for students with disabilities, and the NLSA is offered in the language of instruction for almost all student groups. 19. There are a variety of mechanisms in place to ensure the quality of the NLSA. For example, all proctors or administrators are trained according to a protocol, there is a standardized manual for large-scale assessment administrators, and a pilot is conducted before the main data collection takes place. In addition, all booklets are numbered and discrepancies must be recorded on a standard sheet. Also, there are external and internal reviewers or observers, double scoring and processing of data, and training for scorers to ensure high inter-rater reliability. 20. Although a comprehensive technical report is available, its circulation is restricted. 21. NLSA results are disseminated in an effective way. Results are disseminated within 12 months of the large-scale assessment being administered and are featured by media such as newspapers, magazines, radio, or television. In addition, reports with results are made available for all stakeholder groups, there are workshops or presentations for key stakeholders on the results, and a media briefing is organized to discuss the assessment results. The main reports on the results contain information on overall achievement levels and subgroups, and trends over time overall and for subgroups. 22. Assessment information is used by some stakeholder groups in a way that is consistent with the purposes and technical characteristics of the assessment. 23. There are some mechanisms in place to monitor the consequences of the NLSA, including a permanent oversight committee, which is the Education and Training Reform Committee, and funding for internal research on the impact of large-scale assessment. However, there are no expert review groups, themed conferences that provide a forum to discuss research and other data on the consequences of the large-scale assessment, or regular focus groups or surveys of key stakeholders. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 36 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 BAHRAIN International Large-Scale Assessment (ILSA) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 37 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ENABLING CONTEXT Overall framework of policies, leadership, organizational structures, fiscal and human resources in which ILSA takes place in a country or system and the extent to which that framework is conducive to, or supportive of, ILSA activity. LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 1: Setting clear policies for ILSA The country/system has not participated This option does not apply to this The country/system has participated in at The country/system has participated in in an ILSA in the last 10 years. dimension. least one ILSA in the last 10 years. two or more ILSA in the last 10 years.1 * The country/system has not taken This option does not apply to this The country/system has taken concrete This option does not apply to this concrete steps to participate in an ILSA in dimension. steps to participate in at least one ILSA in dimension. the next 5 years. the next 5 years. 2 * There is no policy document that There is an informal or draft policy There is a formal policy document that This option does not apply to this addresses participation in ILSA. document that addresses participation in addresses participation in ILSA. 3 dimension. ILSA. * This option does not apply to this The policy document is not available to The policy document is available to the This option does not apply to this dimension. the public. 4 public. dimension. * ENABLING CONTEXT 2: Having regular funding for ILSA There is no funding for participation in There is funding from loans or external There is regular funding allocated at There is regular funding approved by law, ILSA. donors. discretion. 5 decree or norm. * This option does not apply to this Funding covers some core activities of the Funding covers all core activities of the This option does not apply to this dimension. ILSA. ILSA. 6 dimension. * Funding does not cover research and This option does not apply to this This option does not apply to this Funding covers research and development activities. dimension. dimension. development activities.7 * (CONTINUED) SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 38 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 LATENT EMERGING ESTABLISHED ADVANCED ENABLING CONTEXT 3: Having effective human resources for ILSA There is no team or national/system There is a team or national/system There is a team and national/system This option does not apply to this coordinator to carry out the ILSA coordinator to carry out the ILSA coordinator to carry out the ILSA dimension. activities. activities. activities. 8 * This option does not apply to this The national/system coordinator or other The national/system coordinator is fluent This option does not apply to this dimension. designated team member may not be in the language of the assessment. 9 dimension. fluent in the language of the assessment. This option does not apply to this The ILSA office is inadequately staffed or The ILSA office is adequately staffed or* The ILSA office is adequately staffed and dimension. trained to carry out the assessment trained to carry out the ILSA effectively, trained to carry out the ILSA effectively, effectively. with minimal issues. with no issues. 10 * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 39 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 SYSTEM ALIGNMENT Degree to which the ILSA meets technical quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED SYSTEM ALIGNMENT 1: Providing opportunities to learn about ILSA The ILSA team has not attended The ILSA team attended some The ILSA team attended all international This option does not apply to this international workshops or meetings. international workshops or meetings. workshops or meetings. 11 dimension. The country/system offers no This option does not apply to this The country/system offers some * The country/system offers a wide range opportunities to learn about ILSA. dimension. opportunities to learn about ILSA. 12 of opportunities to learn about ILSA. * This option does not apply to this This option does not apply to this Opportunities to learn about ILSA are Opportunities to learn about ILSA are dimension. dimension. available to the country's/system's ILSA available to a wide audience, in addition team members only. to the country's/system's ILSA team members. 13 * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 40 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 ASSESSMENT QUALITY Degree to which the ILSA meets technical quality standards, is fair, and is used in an effective way. LATENT EMERGING ESTABLISHED ADVANCED ASSESSMENT QUALITY 1: Ensuring the quality of ILSA Data from the ILSA has not been The country/system met sufficient The country/system met all technical This option does not apply to this published. standards to have its data presented standards required to have its data dimension. beneath the main display of the presented in the main displays of the international report or in an annex. international report. 14 The country/system has not contributed This option does not apply to this This option does not apply to this * The country/system has contributed new new knowledge on ILSA. dimension. dimension. knowledge on ILSA. 15 * ASSESSMENT QUALITY 2: Ensuring effective uses of ILSA If any, country/system-specific results Country/system-specific results and Country/system-specific results and Country/system-specific results and and information are not disseminated in information are disseminated irregularly information are regularly disseminated in information are regularly and widely the country/system. in the country/system. the country/system. disseminated in the country/system. 16 Products to provide feedback to schools This option does not apply to this Products to provide feedback to schools Products to provide feedback to schools * and educators about the ILSA results are dimension. and educators about the ILSA results are and educators about ILSA results are not made available. sometimes made available. systematically made available. 17 * There is no media coverage of the ILSA There is limited media coverage of the There is some media coverage of the ILSA There is wide media coverage of the ILSA results. ILSA results. 18 results. results. * If any, country/system-specific results Results from the ILSA are used in a limited Results from the ILSA are used in some Results from the ILSA are used in a variety and information from the ILSA are not way to inform decision making in the ways to inform decision making in the of ways to inform decision making in the used to inform decision making in the country/system. country/system. country/system. 19 country/system. * It is not clear that decisions based on ILSA This option does not apply to this This option does not apply to this Decisions based on the ILSA results have results have had a positive impact on dimension. dimension. had a positive impact on students' students' achievement levels. achievement levels.20 * SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 41 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 International Large Scale Assessment (ILSA): Development-level rating justifications 1. Bahrain has participated in three ILSAs in the past 10 years, TIMSS 2003, 2007, and 2011. 2. Bahrain has taken concrete steps to participate in two ILSAs in the next five years, including PIRLS 2016 and TIMSS 2015. 3. There are formal policy documents addressing Bahrain's participation in ILSAs. 4. The policy documents addressing ILSA participation are not available to the public. 5. There is regular funding allocated at the government’s discretion. 6. Funding covers all core activities of the ILSA, including international participation fees, implementation of the assessment exercise in Bahrain, processing and analyzing data collected from the implementation of the assessment exercise, reporting and disseminating the assessment results in Bahrain, attendance at international expert meetings for the assessment exercise, and research and development. In addition, funding covers the implementation of a remedial action plan to improve students' performance, which is developed after results of the assessment are available and the weaknesses in various competencies are identified. 7. Funding covers research and development activities. 8. There is a team and coordinator to carry out the ILSA activities in Bahrain. 9. The ILSA coordinator in Bahrain is fluent in the language of the assessment. 10. The ILSA office is adequately staffed and trained to carry out the ILSA effectively, with no issues. In addition to the national coordinator being fluent in the language of the assessment, the team is sufficiently staffed, has previous experience working on international assessments, has the necessary training or experience to carry out the required assessment activities effectively, and has attended all international meetings related to the assessment. 11. The ILSA team has attended all international workshops or meetings. 12. Bahrain offers some opportunities to learn about ILSA, including workshops or meetings on using international assessment databases. However, there are no university or online courses on international assessments, or funding to attend international workshops, or training on international assessments. 13. Opportunities to learn about ILSA are available to a wide audience, in addition to Bahrain's ILSA team members. For instance, opportunities are available to teachers, principals, and university students studying assessment or a related area. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 42 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 14. Bahrain met all technical standards required to have its data presented in the main displays of the international report. 15. Bahrain has contributed new knowledge on ILSA. For example, it is featured in the TIMSS 2007 Encyclopedia and TIMSS 2011 Encyclopedia, which are publicly available at the TIMSS website. 16. Country-specific results and information are regularly and widely disseminated in Bahrain. For example, the Kingdom of Bahrain's report on the results of TIMSS 2007 was published and previously available on the Ministry of Education's website. In addition, copies of the national report were distributed to key stakeholders, and country results were communicated through a press release. Results received coverage on the television, radio, and newspapers, and brochures and PowerPoint presentations with Bahrain's results were made available online or distributed to key stakeholders. In addition, products providing feedback to the schools or educators about the results were made available. 17. Products providing feedback to schools and educators about ILSA results are systematically made available in Bahrain. 18. There is limited media coverage, including commentary in editorials or columns, of the ILSA results. 19. Results from the ILSA are used in a variety of ways to inform decision making in Bahrain, including tracking the impact of reforms on student achievement levels, as well as informing curriculum improvement, teacher training programs, other assessment activities in the system, and resource allocation. 20. Decisions based on the ILSA results have had a positive impact on students' achievement levels, evidenced by the advancement in achievement between TIMSS 2003 and TIMSS 2007. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 43 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 Acknowledgements References This report, part of a 16-country benchmarking exercise Clarke, M. 2012. “What Matters Most for Student in the Middle East & North Africa and Africa regions, was Assessment Systems: A Framework Paper.” READ/SABER prepared by the World Bank SABER-Student Assessment Working Paper Series. Washington, DC: World Bank. team, in partnership with the Arab League Educational, Cultural, and Scientific Organization (ALECSO) and the The Education Project. 2009. “Bahrain: Education and United Nations Educational, Scientific, and Cultural Training.” Data retrieved from Organization (UNESCO), which led data-collection http://www.educationprojectbahrain.org/download/ED efforts. The benchmarking exercise is part of the Arab B%20- Regional Agenda for Improving Education Quality %20Education%20and%20Training%20September%200 (ARAIEQ), led by ALECSO in partnership with the World 9.pdf on October 3, 2013. Bank. It benefited from feedback and review from Lianqin Wang, Senior Education Specialist, and Alyaa Embassy of the Kingdom of Bahrain. 2013. “Bahrain’s Alaali, Education Specialist, in the World Bank’s Education System.” Data retrieved from Education Global Practice, as well as comments received http://www.mofa.gov.bh/Default.aspx?tabid=7741 on during a national validation workshop held in Bahrain. October 3, 2013. Kingdom of Bahrain Ministry of Education.2003. “King Hamad’s Schools of the Future Project.” Data retrieved from http://www.moe.gov.bh/khsfp/khsfpdoc/future- eng.pdf on October 3, 2013. United Nations Educational, Scientific, and Cultural Organization-Institute for Statistics. 2011. UIS Statistics in Brief. “Education Profile: Bahrain.” Montreal: UNESCO-UIS. World Bank. 2012. Bahrain Country Indicator Data. Washington, DC: World Bank. Data retrieved from http://data.worldbank.org/ on October 3, 2013. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 44 BAHRAIN ǀ SABER-STUDENT ASSESSMENT SABER COUNTRY REPORT |2013 www.worldbank.org/education/saber The Systems Approach for Better Education Results (SABER) initiative produces comparative data and knowledge on education policies and institutions, with the aim of helping countries systematically strengthen their education systems. SABER evaluates the quality of education policies against evidence-based global standards, using new diagnostic tools and detailed policy data. The SABER country reports give all parties with a stake in educational results—from administrators, teachers, and parents to policymakers and business people—an accessible, objective snapshot showing how well the policies of their country's education system are oriented toward ensuring that all children and youth learn. This report focuses specifically on policies in the area of student assessment. This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. SYSTEMS APPROACH FOR BETTER EDUCATION RESULTS 2