OBSERVER MANUAL Teach Teach OBSERVER MANUAL Teach TABLE OF CONTENTS FOREWORD........................................................................................................................................................ ii TEACH TEAM.....................................................................................................................................................iv INTRODUCTION................................................................................................................................................ 1 PROCEDURES FOR CODING....................................................................................................................5 OBSERVER MANUAL.................................................................................................................................13 TIME ON TASK TIME ON LEARNING.......................................................................................................................................... 17 QUALITY OF TEACHING PRACTICES CLASSROOM CULTURE 19 SUPPORTIVE LEARNING ENVIRONMENT................................................................................................... 20 POSITIVE BEHAVIORAL EXPECTATIONS...................................................................................................... 21 INSTRUCTION 22 LESSON FACILITATION.................................................................................................................................... 23 CHECKS FOR UNDERSTANDING....................................................................................................................24 FEEDBACK.......................................................................................................................................................... 25 CRITICAL THINKING......................................................................................................................................... 26 SOCIOEMOTIONAL SKILLS 28 AUTONOMY........................................................................................................................................................ 29 PERSEVERANCE.............................................................................................................................................. 30 SOCIAL & COLLABORATIVE SKILLS.............................................................................................................. 31 FREQUENTLY ASKED QUESTIONS................................................................................................... 32 i FOREWORD School enrollment increased substantially over the last 25 years in low-and middle-income countries. Schooling, however, does not guarantee learning. A large share of children complete primary school lacking even basic reading, writing, and arithmetic skills1 — a state of affairs UNESCO2 dubbed the “global learning crisis.” The learning crisis is, at its core, a teaching crisis.3 A growing body of research indicates teaching is the most important school-based determinant of student learning.4 The difference between the impact of a weak and great teacher on student test scores is equivalent to one to two years of schooling.5 Moreover, evidence suggests several consecutive years of effective teaching can offset the learning shortfalls of marginalized students6 and significantly improve students’ long- term outcomes.7 Identifying effective teaching is not easy, however. Research shows that teacher characteristics such as formal education, years of experience (beyond the first two), cognitive skills, and entry exam performance only explain a small fraction of the variation in student learning.8 Recent evidence highlights the crucial role teaching practices play in explaining this variation. For example, a seminal study in Ecuador found better teaching practices are associated with improved learning outcomes.9 Moreover, a study of over 60 coaching programs found those designed to advance teaching practices also resulted in increased student learning.10 Although better teaching practices are needed to tackle the learning crisis, most education systems in low-and middle-income countries do not regularly monitor them, either because they do not recognize the importance of such practices, or do not know how to adequately monitor them. The situation is exacerbated by a prevalence of ineffective professional development programs, which tend to be theory-based and rarely monitor or provide actionable guidance for teachers to improve their practice.11 Without a reliable instrument, even experienced education professionals struggle to distinguish between effective and ineffective teaching. ii Given this reality, what can be done? Teach, a new classroom observation tool, was developed to address these challenges. First, Teach holistically measures what happens in the classroom. It does so by considering not just time spent on learning but, more importantly, the quality of teaching practices. Second, Teach captures practices that nurture children’s cognitive and socioemotional skills. Third, Teach was developed with low- and middle-income countries in mind, and includes the use of local video footage to contextualize the training. Finally, Teach is free and includes a toolkit that facilitates data collection, analysis, and validation of Teach scores. Before the tool was launched, Teach underwent a rigorous development and validation process over a two-year timeframe. A Technical Advisory Panel12 provided extensive feedback and inputs on the tool’s design. Teach was also piloted in over 1,000 classrooms across Mozambique, Pakistan, the Philippines, and Uruguay, and tested with global video footage from 11 low- and middle-income countries. Preliminary analyses of pilot data indicate Teach scores have high levels of interrater reliability and are internally consistent. Moreover, teachers who display effective practices, as measured by Teach, are associated with students who achieve higher learning outcomes.13 Teach is intended to be used as a system diagnostic and for professional development. Coach, a forthcoming protocol, will help principals, inspectors, and coaches use Teach to provide feedback on how teachers can improve their classroom practices. Coach aims to reduce the gap between practice and evidence by utilizing evidence-based strategies to improve professional development.14 We hope these new resources will bring us one step closer to ensuring every child has a skilled, supported, and motivated teacher, conditio sine qua non to achieve Learning for All. Omar Arias Practice Manager, Global Knowledge and Innovation Team iii TEACH TEAM Acknowledgments Teach was prepared by a team led by Ezequiel Molina. The core team was composed of Carolina Melo Hurtado, Adelle Pushparatnam, and Tracy Wilichowski. Jenny Beth Aloys, Alice Madeleine Danon, Syeda Farwa Fatima, Carolina Moreira Vásquez, and Iva Trako were members of the extended team. The team received guidance from a technical advisory panel composed of Lindsay Brown, Pam Grossman, Heather Hill, Andrew Ho, Sara Rimm-Kaufman, Andrew Ragatz, Erica Woolway, and Nick Yoder. The manual and observation instrument were designed by Danielle Willis. Amy Gautam was the chief copy editor. Restituto Jr. Mijares Cardenas and Cassia Miranda provided administrative support. The team is grateful for the contributions of the Teach Classroom Observation Working Group, composed of Salman Asim, Tara Beteille, Marguerite Clarke, Michael Crawford, David Evans, Deon Filmer, Francisco Haimovich, Samira Halabi, Amer Hassan, Peter Holland, Dingyong Hou, Nathalie Lahire, Toby Linden, Javier Luque, Juan Manuel Moreno, Shawn Powers, Halsey Rogers, Shwetlena Sabarwal, Shabnam Sinha, Lars Sondergaard, Simon Thacker, Waly Wane, and Noah Yarrow. The team also benefited from the support of Hafsa Alvi, Tamara Arnold, Jennifer Bulley, Yanina Gallo, Julia Hahn, Julia Ladics, Anahita Matin, Abdal Mufti, Octavio Medina Pedreira, Mahjabeen Raza, Hina Saleem, Marie Evane Tamagnan, and Sergio Venegas Marin. A number of colleagues provided insightful comments, feedback, and inputs on the tool, including Gonzalo Dibot, Guadalupe Goyeneche, Michael Handel, Amer Hassan, Ines Kudo, Victoria Levin, Alonso Sanchez, Virginia Tort Gómez, Paula Prendeville, Elina Rostan, Kirill Vasiliev, and Noah Yarrow. In addition, the team is grateful to the teams that applied a preliminary version of Teach in their various contexts. This included Franco Russo, Binh Thanh Vu, and Javier Luque in the Philippines; Koen Martijn Geven, Tazeen Fasih, and Ali Ansari in Pakistan; Francisco Haimovich Paz and Helena Rovner in Uruguay; Marina Bassi in Mozambique; and Sara Rimm-Kaufman at the University of Virginia. Overall guidance for the development and preparation of Teach was provided by Omar Arias, Practice Manager for the Global Knowledge and Innovation Team. The team gratefully acknowledges support from the Global Lead of the Curriculum, Instruction and Assessment Thematic Group, Michael Crawford, and the Global Leads of the Teachers Career and Professional Development Thematic Group, Tara Beteille and David Evans, for their guidance and advice throughout the process. The team is especially grateful to the Senior Management Team of the Education Global Practice — Jaime Saavedra, Senior Director, Luis Benveniste, Director, and Keiko Miwa, Director — for their leadership, direction, and relentless support. The team is grateful for generous support provided by the Systems Approach for Better Education (SABER) Trust Fund, funded in large part by the United Kingdom’s Department for International Development (DFID) and Australia’s Department of Foreign Affairs and Trade (DFAT). The team apologizes to anyone inadvertently omitted from this list and expresses its gratitude to all who contributed to Teach, including those whose names may not appear here. Lastly and most importantly, the team members would like to thank all the teachers who welcomed us into their classroom as part of this project. HAVE QUESTIONS? Contact Us at teach@worldbank.org. iv INTRODUCTION 1 What does Teach measure? Teach differs from other classroom observation tools in that it captures (i) the time teachers spend on learning and the extent to which students are on task, and (ii) the quality of teaching practices that help develop students’ socioemotional and cognitive skills. As part of the Time on Task component, 3 “snapshots” of 1–10 seconds are used to record both the teacher’s actions and the number of students who are on task throughout the observation. The Quality of Teaching Practices component, on the other hand, is organized into 3 primary areas: Classroom Culture, Instruction, and Socioemotional Skills15 (see graphic on page 3).These areas have 9 corresponding elements that point to 28 behaviors. The behaviors are characterized as low, medium, or high, based on the evidence collected during the observation. These behavior scores are translated into a 5-point scale that quantifies teaching practices as captured in a series of two, 15-minute lesson observations. 1 CLASSROOM CULTURE: The teacher creates a culture that is conducive to learning. The focus here is not on the teacher correcting students’ negative behaviors but rather the extent to which the teacher creates: (i) a supportive learning environment by treating all students respectfully, consistently using positive language, responding to students’ needs, and both challenging gender stereotypes and not exhibiting gender bias in the classroom; and (ii) positive behavioral expectations by setting clear behavioral expectations, acknowledging positive student behavior, and effectively redirecting misbehavior. 2 INSTRUCTION: The teacher instructs in a way that deepens student understanding and encourages critical thinking and analysis. The focus here is not on content-specific methods of instruction, but rather the extent to which the teacher: (i) facilitates the lesson by explicitly articulating lesson objectives that are aligned to the learning activity, clearly explaining content, and connecting the learning activity to other content knowledge or students’ daily lives, and by modeling the learning activity through enacting or thinking aloud; (ii) does not simply move from one topic to the next but checks for understanding by using questions, prompts, or other strategies to determine students’ level of understanding, by monitoring students during group and independent work, and by adjusting his/her teaching to the level of students; (iii) gives feedback by providing specific comments or prompts to help clarify students’ misunderstandings or identify their successes; and (iv) encourages students to think critically by asking open-ended questions and providing students with thinking tasks that require them to actively analyze content. Students exhibit critical thinking ability by asking open-ended questions or performing thinking tasks. 3 SOCIOEMOTIONAL SKILLS: The teacher fosters socioemotional skills that encourage students to succeed both inside and outside the classroom. To develop students’ social and emotional skills, the teacher: (i) instills autonomy by providing students with opportunities to make choices and take on meaningful roles in the classroom. Students exhibit their autonomy by volunteering to participate in classroom activities; (ii) promotes perseverance by acknowledging students’ efforts, rather than focusing solely on their intelligence or natural abilities, by having a positive attitude toward students’ challenges by framing failure and frustrations as part of the learning process, and by encouraging students to set short- and long-term goals; and (iii) fosters social and collaborative skills by encouraging collaboration through peer interaction and by promoting interpersonal skills, such as perspective taking, empathizing, emotion regulation, and social problem solving. Students exhibit social and collaborative skills by collaborating with one another through peer interaction. 2 TEACH FRAMEWORK 3 How was Teach developed? To finalize a working version of the tool, the Teach development team rigorously researched, revised, and piloted different iterations of the tool over a 2-year timeframe: 1 The development team — which comprised 1 education measurement expert, 1 instructional expert, 1 psychologist, and 1 teacher — assessed 5 classroom observation tools widely used in the United States to create an inventory of teaching practices that are commonly evaluated.16 The team then built upon this list to include behaviors from international classroom observation tools used in low- and middle-income countries.17 Based on this preliminary analysis, the team created an inventory of 3 areas and 43 elements.18 2 The development team hosted a working group of 22 education experts and practitioners to help further reduce and prioritize elements for the Teach framework. Participants were asked to indicate whether any elements were missing from the inventory, to rank the elements and areas by relevance, and to identify elements they characterized as unobservable. This process resulted in a downsized framework of 25 elements. 3 The development team reviewed the theoretical and empirical evidence from low- and middle-income countries to further eliminate elements from the framework. This process resulted in a downsized framework of 14 elements. 4 These 14 elements comprised the first working version of the tool, which aimed to capture both quality and frequency of teaching practices as measured by each element.19 This preliminary tool was piloted in person in Pakistan and Uruguay and through classroom video footage in Afghanistan, China, Pakistan, the Philippines, Tanzania, Uruguay, and Vietnam. From these pilots, it became apparent that observers struggled to code reliably when they had to simultaneously capture the frequency and quality of teaching practices for each element. In response, the development team revised the structure of the tool to address this challenge. This process resulted in a tool comprised of 10 elements. 5 The development team convened a technical advisory panel, including Lindsay Brown, Pam Grossman, Heather Hill, Andrew Ragatz, Sara Rimm-Kaufman, Erica Woolway, and Nick Yoder, to provide written feedback on the tool. These comments were compiled and addressed as part of a 1-day technical workshop. During the workshop, the experts advised the team on which issues to prioritize and how to incorporate the comments to further improve the tool. 6 This updated version of the tool was applied in 4 settings, where observers were given a certification exam that ensured they could reliably code using Teach. In Mozambique, 74% of the observers passed the certification exam; in Pakistan and the Philippines, 96% passed; and in Uruguay, 100% passed. The observers also provided comments on the tool and training that was considered during the revision process. 7 The development team worked closely with Andrew Ho20 to analyze psychometric properties of the tool. Using the data from Punjab, Pakistan the team found those teachers who exhibit better teaching practices, as measured by Teach, are associated with an additional .08-.12 SD increase in student test scores. This is after controlling for a host of variables, including class size, teacher content knowledge, and other student and teacher characteristics. Based on this analysis and feedback from the trainers and observers, the development team revised each element’s structure and complementary examples to improve the tool’s consistency and clarity. As part of this process, the Time on Learning element was modified to capture teachers’ time on instruction and students’ time on task through a series of snapshots. This process resulted in a tool that comprised 1 low inference and 9 high-inference elements. The final stage involved testing these revisions using video footage of 11 low- and middle-income countries from the Teach video library. 4 PROCEDURES FOR CODING 5 Protocol Before, during, and after an observation, observers should be cognizant and respectful of the school environment by following this protocol: BEFORE DURING AFTER SUPPLIES: SETUP: CONCLUSION: Ensure you have the manual, Sit toward the back of the Thank the teacher for being able to observation packet, a pencil/pen, classroom to view the entire conduct the observation. consent forms,21 and a classroom; ensure your presence watch/phone. does not block students’ view of When the lesson concludes, remove the lesson. yourself from the classroom and ARRIVAL: finish coding in a different location Introduce yourself to the principal If visiting a classroom with another to mitigate distractions. and arrive at the designated observer, seat yourself separately classroom at least 10 minutes and refrain from talking with him/ DISCRETION: before class begins. her at any point during the lesson. Avoid discussing any of the scores with the teacher. If a teacher asks Introduce yourself to the teacher, Ensure your cellphone is muted and how s/he performed, politely explain the purpose of the abstain from texting, phone calls, remind him/her this is not a visit, and remind the teacher of Facebook/Twitter, taking pictures, performance evaluation. For the confidential nature of the or any other distracting activities. example: observation: OBSERVATION: “The goal of the observation was “Good morning, Mr./Ms. [teacher’s Begin the observation when class to learn about teaching practices; surname], I work with [affiliate is scheduled to begin; if the teacher the notes from this observation organization]. Your school has been is delayed, wait until s/he arrives will be used as part of a larger randomly selected to partake in and make note of the time on the study on teaching practices in a survey that includes classroom observation sheet. [district/city name]. I very much observations. The purpose of the enjoyed watching your lesson and survey is to learn about teaching In the case of multigrade appreciate you allowing me into practices in [district/city name]. As classrooms, treat the observation your classroom.” such, I’m here to simply learn from as one grade and document it on you — these observations will not the observation sheet. Refrain from discussing the be used for evaluative purposes, classroom scores with anyone. and your identity will remain entirely NO INTERACTION: You can provide your supervisor’s confidential. Please proceed with Avoid engaging with or distracting number if the teacher insists. the lesson as you normally would.” the students or teacher and do not participate in classroom activities, Refrain from discussing what DISSENT: even if explicitly asked. occurred during the lesson in a If a teacher does not want to be joking or disrespectful way. observed, kindly remind him/ Do not check students’ textbooks, This may affect your credibility her that the observation is not worksheets, notebooks, or other as an observer. an evaluation, his/her identity classwork. will be kept anonymous, and no information about the observation Avoid positive or negative nonverbal will be shared with school expressions and convey a neutral authorities. Kindly note, a teacher attitude to avoid inadvertently cannot be forced to be observed; distracting the teacher. if the teacher continues to decline consent, exit the classroom and Redirect the teacher and students document what happened on the to the lesson if they ask questions observation sheet. or focus their attention on your presence. 6 Length of the observation Observations should be divided into two, 15-minute segments.22 The first observation segment begins at the scheduled class time; however, if the teacher or students are not present during the scheduled class time or the lesson is delayed, the observation begins when the teacher enters the classroom. After each 15-minute observation, observers should spend 10–15 minutes scoring the observation, depending on the length of the class. For example, in a 45-minute class, the first observation segment begins at the scheduled class time and is 15 minutes in length. The observer then stops (even though class is still going) and spends the next 15 minutes scoring segment 1. The observer then spends the remaining 15 minutes of class observing segment 2. After the class has concluded, the observer spends another 15 minutes scoring segment 2. Observers should always record the length of each observation segment on the scoring sheet. If the lesson ends before the predetermined length of the observation, observers should still code the segment. It is important to accurately record the information on segment length, delayed starts, and early finishes, as this will be used in data analysis. Note-taking Once the observation begins, the observer uses the note-taking form to document what the teacher says by noting specific behaviors, questions, instructions, and actions. These notes are essential to code objectively and reliably, since they provide evidence for the chosen scores. When note-taking, it is important to be as descriptive as possible. Observers will use their notes and compare them with the descriptions in the manual to determine the behavior quality ranges and assign an overall combined score for each element. As soon as observers finish an observation, every score should be justified with evidence from the observation. When note-taking, it is important to look for specific student and teacher behaviors that are clearly included in the tool. All observers should create a note-taking system that works for them; below are some helpful note- taking techniques.23 TECHNIQUE WHAT IS OBSERVED WHAT IS WRITTEN SCRIPTING: After a lesson on forming past tense sentences, the T: Who can take an action quotes by teacher asks students to relate the current lesson to verb from yesterday and teachers (T) or a previous one on action verbs by forming a sentence create a past tense sentence? students (S) using both strategies. She asks, “Who can take an action verb from yesterday and create a past tense S: Amna jumped over the sentence?” A student raises her hand and responds, puddle. “Amna jumped over the puddle.” TALLIES: Throughout the lesson, the teacher says “very good” 8 “Very good” shortcuts for times in response to student participation and answers. frequently used words or phrases SHORTHAND: The teacher reviews a student’s paragraph and provides FB- T: opening paragraph is specific symbols or feedback by saying, “Great job on the first paragraph. compelling b/c of personal letters to represent The way you open with a personal story is very story behaviors compelling.” ANECDOTES: At the start of an activity, the teacher asks if everyone 6 Ss no book, T cont. teaching summaries of what has a textbook. Six students raise their hands to at board, 3 Ss playing was seen or heard indicate they do not. The teacher continues teaching at (disruptive). the board. Meanwhile, 3 students are playing with a ball of paper and distracting others. 7 The tool does not exist without the manual; the tool is constituted by the observer manual and observation sheet; observers should actively use and read the manual to determine scores. Measuring Time on Task For the Time on Learning element, observers will take 3 “snapshots,” or 1–10 second scans of the classroom, and use only information gathered during the snapshot to code the behaviors. For the first behavior, observers will record whether the teacher is providing a learning activity for most students by indicating “no” if s/he is not providing a learning activity and “yes” if s/he is. If the teacher is providing a learning activity, scan the classroom from left to right to determine whether students are on task. If 0 or 1 student is off task, score the second behavior high(H). If 2 to 5 students are off task, score it medium(M). If 6 or more students are off task, score it a low(L). If the teacher does not provide a learning activity for most students, record OBSERVATION a “not applicable” SHEET (N/A) for the second behavior and continue coding the other elements of the tool. See Page 17 for more details on the SCHOOL ID: TEACHER ID: CODER ID: GRADE: SUBJECT: SEGMENT 1 snapshot method CLASS SIZE: girls and _____ boys _____how to code this SCHEDULED TIME:element. __ __ : __ __ to __ __ : __ __ ACTUAL TIME: __ __ : __ __ to __ __ : __ __ SEGMENT LENGTH: _____ min TIME ON TASK 0. TIME ON LEARNING 1st Snapshot (4m) 2nd Snapshot (9m) 3rd Snapshot (14m) 0.1 Teacher provides learning activity to most students Y N Y N Y N 0.2 Students are on task N/A L M H N/A L M H N/A L M H QUALITY OF TEACHING PRACTICES Measuring Quality of Teaching Practices Areas / Elements / Behaviors Scoring Final Scores A. CLASSROOM CULTURE 1. Assigning (i) quality SUPPORTIVE LEARNING ranges to each behavior ENVIRONMENT 1 2 3 4 5 1.1 The teacher treats all students respectfully L M H To assign the most objective score, the manual describes each behavior in 3 qualityLranges: 1.2 The teacher uses positive language with students M low, medium, H and high. These are detailed descriptions 1.3 The teacher responds to students’ needs and include examples that help observers N/A decide L which M quality H score best applies 1.4 The to each teacher does element. not exhibit gender bias After the first and challenges observation gender stereotypes in thesegment classroom concludes, the observer N/A L assigns M Ha “low, medium, or high” rating to each behavior. For this, it is necessary to read the notes and compare them with the 2. POSITIVE BEHAVIORIAL EXPECTATIONS 1 2 3 4 5 descriptions provided 2.1 The teacher sets in the clear behavioral manual. expectations It is very for classroom important for observers to adhere to the activities L manual M closely as as H possible, whether 2.2 The teacher or not acknowledges they positive agree student with it. This symbol behavior signifies that the given behavior L has a corresponding M H FAQ; observers should thoroughly familiarize themselves with the FAQs prior to carrying out observations, and OBSERVATION L M H SHEET 2.3 The teacher redirects misbehavior and focuses on the expected behavior, rather than the undesired behavior should refer to the FAQs while coding to help clarify any confusion. B. INSTRUCTION SCHOOL ID: TEACHER ID: CODER ID: GRADE: SUBJECT: SEGMENT 1 3. SIZE: girlsFACILITATION LESSON CLASS _____ boys _____ SCHEDULED TIME: __ __ : __ __ to __ __ : __ __ 1 : __ __ 2 ACTUAL TIME: __ __ : __ __ to __ __ 3 SEGMENT 4 LENGTH: 5 _____ min It is very important that observers give 1 score for every behavior. If observers want to change an answer, 3.1 The teacher explicitly articulates the objectives of the lesson and relates classroom activities to the objectives L M H they must clearly remove the invalid score by fully erasing or striking through it. Some behaviors may not be TIME ON TASK 3.2 The teacher’s explanation of content is clear L M H 0. TIME ON LEARNING 1st Snapshot (4m) 2 nd Snapshot (9m) 3 rd Snapshot (14m) observed. For those behaviors, the manual provides the option to write “N/A.” Observers canM only score “N/A” if 3.3 The 0.1 teacher Teacher makes provides connections learning the in to activity lesson most that relate to other content knowledge or students’ daily students Y lives N YL N HY N presented 3.4 The the teacher option models 0.2 Students are on task by on enacting the or scoring thinking aloud sheet (0.2, 1.3, 1.4, 4.2). If a behavior N/A L M is scored H N/A L “N/A,” L M this M H behavior H N/A L should M H not influence the overall score for the corresponding element. The following example shows what this would 4. CHECKS FOR UNDERSTANDING QUALITY OF TEACHING PRACTICES 1 2 3 4 5 look 4.1 like Areas The in/ Behaviors practice: teacher / Elements uses questions, prompts or other strategies to determine students’ level of understanding Scoring L M H Final Scores 4.2 The teacher monitors most students during independent/group work N/A L M H A. CLASSROOM CULTURE 4.3 The teacher adjusts teaching to the level of students L M H 1. SUPPORTIVE LEARNING ENVIRONMENT 1 2 3 4 5 1.1 FEEDBACK 5. The teacher treats all students respectfully 1L 2 3 M 4 H5 5.1 1.2 The teacher The provides teacher uses specific positive comments language or prompts that help clarify students’ misunderstandings with students L L M M H H 5.2 1.3 The teacher The teacher responds provides specific to students’ comments needs or prompts that help identify students’ successes N/A L L M M H H 1.4 The teacher does not exhibit gender bias and challenges gender stereotypes in the classroom N/A L M H 6. CRITICAL THINKING 1 2 3 4 5 6.1 POSITIVE 2. The teacher BEHAVIORIAL asks open-ended questions EXPECTATIONS 1L 2 M 3 4 H5 6.2 2.1 The teacher teacher provides The thinking sets clear tasks behavioral expectations for classroom activities L L M M H H 6.3 2.2 The teacher The ask open-ended studentsacknowledges questions positive or perform student behaviorthinking tasks L L M M H H 2.3 The teacher redirects misbehavior and focuses on the expected behavior, rather than the undesired behavior L M H C. SOCIOEMOTIONAL SKILLS 7. B. AUTONOMY INSTRUCTION 1 2 3 4 5 7.1 LESSON 3. The teacher provides students with choices FACILITATION 1L 2 M 3 4 H5 8 7.2 The teacher explicitly providesarticulates students with opportunities take on to lesson roles in the classroom L M H 3.1 The teacher the objectives of the and relates classroom activities to the objectives L M H 7.3 3.2 The students The teacher’svolunteer to participate explanation in clear of content is the classroom L L M M H H OBSERVATION SHEET SCHOOL ID: TEACHER ID: CODER ID: GRADE: SUBJECT: SEGMENT 1 CLASS SIZE: girls _____ boys _____ SCHEDULED TIME: __ __ : __ __ to __ __ : __ __ ACTUAL TIME: __ __ : __ __ to __ __ : __ __ SEGMENT LENGTH: _____ min (ii) Assigning scores to each element TIME ON TASK After 0. assigning TIME ON LEARNING quality ranges to the behaviors, the element scores should 1st Snapshot (4m) be decided according 2nd Snapshot (9m) to 3rd the (14m) Snapshot overall 0.1 Teacher quality providesof each learning element. activity Scores range from 1 to 5, with 1 being to most students Y theNlowest score Y and N 5 the highest. Y N It 0.2 necessary is Students to carefully read the descriptions for the different are on task N/Abehavior L M levels H and L N/A assignM H element an N/A L M H score that best describes the observed scenario in the classroom. While the final score should follow the QUALITY OF TEACHING PRACTICES calculated scores from the behaviors, observers should always go back and Areas / Elements / Behaviors reread the element description Scoring and Final Scores its corresponding A. CLASSROOM CULTURE behaviors to determine if the score fits the overall description of the element. For example, observers 1. SUPPORTIVE may score an element a 4 even if it contains high, medium, LEARNING ENVIRONMENT 1 and 2 low 3 behavior 4 5 scores if what was observed exceeds the overall medium description, but does not constitute a high description. 1.1 The teacher treats all students respectfully L M H The final 1.2 The score teacher uses need not bewith positive language a mathematical students calculation and the score should reflectLthe evidence M presented H in 1.3 The teacher responds to students’ needs N/A L M H the entire segment. 1.4 The teacher does not exhibit gender bias and challenges gender stereotypes in the classroom N/A L M H 2. POSITIVE BEHAVIORIAL EXPECTATIONS 1 2 3 4 5 4 2.1 The teacher sets clear behavioral expectations for classroom activities L M H H 2.2 The teacher acknowledges positive student behavior L M H L 2.3 The teacher redirects misbehavior and focuses on the expected behavior, rather than the undesired behavior L M H M B. INSTRUCTION Common 3. challenges in classroom observations LESSON FACILITATION 1 2 3 4 5 3.1 The teacher explicitly articulates the objectives of the lesson and relates classroom activities to the objectives L M H Before coding with a classroom observation tool, it is crucial to understand the importance 3.2 The teacher’s explanation of content is clear L M of inter-rater H reliability, which describes the degree to which observers agree on the scores 3.3 The teacher makes connections in the lesson that relate to other content knowledge or students’ daily lives associated L with M a specific H observation. For by 3.4 The teacher models example, an observation enacting or thinking aloud is reliable if 2 observers use the tool to observe L the M same H teacher and arrive at the same (or nearly the same) scores. 4. CHECKS FOR UNDERSTANDING 1 2 3 4 5 4.1 The teacher uses questions, prompts or other strategies to determine students’ level of understanding L M H Observers should be aware of several challenges when conducting classroom 4.2 The teacher monitors most students during independent/group work N/A observations L M that have H the potential to adversely affect objectivity 4.3 The teacher adjusts teaching to the level of students and reliability when using the tool: L M H 5. Personal FEEDBACK Experiences 1 2 3 4 5 5.1 The teacher provides specific comments or prompts that help clarify students’ misunderstandings L M H In some cases, past experiences and personal opinions influence how observers score the rubric. This 5.2 The teacher provides specific comments or prompts that help identify students’ successes L M H is particularly problematic for people who have preexisting notions of what constitutes “good teaching.” 6. CRITICAL Moreover, 1 2 their THINKING their exposure to different teaching styles has the potential to influence 3 reliability. 4 5 For 6.1 The teacher asks open-ended questions L M H example, some observers may think, “When I went to school, this is how we learned” or “My daughter’s 6.2 The teacher provides thinking tasks L M H 6.3 teacher does this.” Despite this prior knowledge, it is important to remember L The students ask open-ended questions or perform thinking tasks codes must M based solely be H on the manual, regardless of opinion or experience. C. SOCIOEMOTIONAL SKILLS 7. Additional AUTONOMY Information 1 2 3 4 5 7.1 In The teacher some provides cases, students withobservers choices adjust their scores based on additional or preexisting L information M they have H 7.2 on the teacher, school, or students. Sometimes, they also assume certain behaviors The teacher provides students with opportunities to take on roles in the classroom L M H by incorrectly 7.3 The students volunteer to participate in the classroom L M H inferring the teacher’s intentions. For example, “I am going to give the teacher a 5 for positive environment, 8. because even though she was impatient with the student, I know it is because PERSEVERANCE 1 2 worked she 3 4 a double-shift 5 8.1 The teacher acknowledges students’ efforts L M H today.” This additional information should not influence the scoring of the observation as codes should 8.2 The teacher has a positive attitude towards students’ challenges L M H solely reflect what happens in the classroom during the allocated observation time. 8.3 The teacher encourages goal setting L M H 9. Comparison SOCIAL & COLLABORATIVE SKILLS 1 2 3 4 5 9.1 Often, The teacher observers promotes conduct students’ collaboration several through observations peer interaction during a short timeframe and L compareM teaching H styles and 9.2 ability The teacher promotes across interpersonal skills — this ultimately hinders the reliability of the tool. L students’ observations For example, M an H observer 9.3 Students collaborate with one another through peer interaction L M H may rate a teacher lower on a behavior because in an earlier observation, s/he saw the same teacher, or a different teacher, use a better strategy to communicate the same information. It is necessary to observe 1 each segment independently and avoid comparison to other situations or teachers to maintain reliability. 9 Separation of Elements In some cases, separating the content of the elements can feel forced since everything that happens in the classroom is interconnected; that is, observers may strongly feel an action falls under more than 1 element. One observed action can serve as evidence for more than 1 Teach behavior or element, but the scoring of each must be done independently. For example, a teacher may provide feedback during the lesson so students reflect on their mistakes. This feedback may encourage students to think critically; however, this does not mean the teacher automatically scores high on the critical thinking element, since other behaviors in the critical thinking element may be absent. In this case, observers should keep both elements separate and score them independently. Weighing of Specific Events or First Impressions In some cases, observers may witness a situation that surprises them or triggers a negative or positive impression. This incident may influence how they assess the entire observation. To maintain reliability, it is important to consider the event in the broader context of the observation and not let first impressions or salient events disproportionately influence the overall score. Therefore, observers should write detailed notes of the observation to determine how much weight to give a specific event. Additionally, each segment should be considered in-and-of itself, and observers should focus on what occurs in the current segment. For instance, even if the teacher intends to do an activity later in the class, it is important for observers to only score what actually happens in that segment, rather than boosting the score of 1 of the behaviors based on an intention that never occurred. This is particularly applicable for distinguishing what occurs in segment 1 versus segment 2 (i.e., what is observed in segment 1 may not be considered for scoring in segment 2, and vice versa). Central Tendency In some cases, observers assign medium-level scores more often than they should. This reluctance to assign high or low scores occurs (i) when observers are not confident in their ability to identify the appropriate level, or believe that high or low scores are very rare and are largely unattainable; or (ii) due to the fear (for themselves or the teacher) of assigning more extreme scores. It is important observers score the behaviors exactly as defined in the manual without being influenced by how the scores may be used or how they reflect on the observer or the teacher. 10 Observer certification and Reliability Exam A training participant must pass the Teach Reliability Exam before becoming a certified reliable Teach observer. Observer certification provides quality control and increases reliability of the Teach tool across observers. It ensures all certified observers can use the tool to accurately and consistently score classroom observations in accordance with the Teach scale. The Teach Reliability Exam consists of watching and scoring three, 15-minute video segments and scoring them according to the Teach rubric. Participants have 15 minutes to code each segment and cannot stop, rewind, or rewatch the videos during the exam. To pass the exam, participants must be reliable on 8 of the 10 elements for each segment. For example, if an observer scores 100% on the first segment, 100% on the second segment, and 70% on the third segment, s/he would not pass the exam. For the Time on Learning element, participants are considered reliable if they are in exact agreement with the master score for 2 out of the 3 snapshots. For all the other elements, participants are considered reliable if they score within 1 point of the master score. Participants who do not pass on the first attempt will be given feedback and allowed 1 additional opportunity to pass the exam. The second exam will consist of 3 different videos. Participants who do not pass the second attempt will not be certified as Teach observers. Teach certification is valid for 1 year. 11 1 World Bank (2018). 2 UNESCO (2013). 3   Bold et al. (2017). 4   Hanushek and Rivkin (2010); Snilstveit et al. (2016). 5   Araujo et al. (2016); Bau and Das (2017); Hanushek and Rivkin (2010); Evans and Yuan (2018). 6   Hanushek and Rivkin (2006); Hanushek and Rivkin (2010); Nye, Konstantopoulos, and Hedges (2004). 7   Chetty, Friedman, and Rockoff (2014). 8   Staiger and Rockoff (2010); Araujo et al. (2016); Bau and Das (2017); Cruz-Aguayo et al. (2017). 9   Araujo et al. (2016). 10  Kraft, Blazar, and Hogan (2018). 11  Popova et al. (2018). 12 The team received guidance from a technical advisory panel composed of Lindsay Brown, Pam Grossman, Heather Hill, Andrew Ho, Sara Rimm-Kaufman, Andrew  Ragatz, Erica Woolway, and Nick Yoder. 13 Molina et al. (2018). 14  Popova et al. (2018). 15 It should be noted that it is impossible to draw a clear line between teaching practices linked to academic versus socioemotional learning. Many teaching practices   included in common professional teaching frameworks do impact students’ socioemotional development, but are usually thought of in terms of academic rather than socioemotional learning. Explicitly linking teaching practices with socioemotional outcomes in measures used for assessment will serve to increase the salience of students’ socioemotional skills to teachers, as well as to other stakeholders and policy makers, thus ensuring a focus on both academic and socioemotional learning in the classroom. 16 The Teach framework built upon the inventory created by Gill and others (2016), who conducted a content analysis of the differences in dimensions of instructional   practice of 5 commonly used classroom observation tools comparing the behaviors they measure with the extent to which they predict student learning. The tools included CLASS, FFT, PLATO, Mathematical Quality of Instruction, and UTeach Observational Protocol. The content, predictive power, and potential bias of these instruments were also analyzed as part of this preliminary framework (Gill, Brian, Megan Shoji, Thomas Coen, and Kate Place. 2016. “The Content, Predictive Power, and Potential Bias in Five Widely Used Teacher Observation Instruments.” National Center for Education Evaluation and Regional Assistance, Washington, DC. 17  These included OPERA, SCOPE, SDI, Stallings, and TIPPS. 18  Elements refer to groups of multiple, similar behaviors that aim to capture teaching practices related to positive learning outcomes. 19 For example, the tool aimed to capture not just the quality with which a teacher checked for understanding (adjusting the lesson, prompting students to determine their   level of understanding, etc.), but the frequency with which the teacher checked for understanding in each lesson. 20 Andrew Ho is Professor of Education at the Harvard Graduate School of Education. He is a psychometrician whose research aims to improve the design, use, and   interpretation of test scores in education policy and practice. 21  Protocol to enter the classroom may vary from context to context; however, it is important to have the necessary approvals in place before arriving at the school. 22 These times may differ slightly from context to context. 23 Adapted from Archer, Jeff, et al. 2016. “Better Feedback for Better Teaching: A Practical Guide to Improving Classroom Observations.” San Francisco, CA: Jossey-Bass. © 2019 The International Bank for Reconstruction and Development / International Development Association (the World Bank) 1818 H Street NW, Washington, DC 20433 Telephone: 202-473-1000; Website: www.worldbank.org This work is a product of the staff of the World Bank, and is subject to revision. The Teach Manual is intended for use in classroom observations and is not for sale. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. The World Bank accepts no responsibility or liability for any omissions or errors (including, without limitation, typographical errors and technical errors) in any content and the use of the manual or for reliance thereon. 12 OBSERVER MANUAL 13 OBSERVATION SHEET SCHOOL ID: TEACHER ID: CODER ID: GRADE: SUBJECT: SEGMENT 1 CLASS SIZE: girls _____ boys _____ SCHEDULED TIME: __ __ : __ __ to __ __ : __ __ ACTUAL TIME: __ __ : __ __ to __ __ : __ __ SEGMENT LENGTH: _____ min TIME ON TASK 0. TIME ON LEARNING 1st Snapshot (4m) 2nd Snapshot (9m) 3rd Snapshot (14m) 0.1 Teacher provides learning activity to most students Y N Y N Y N 0.2 Students are on task N/A L M H N/A L M H N/A L M H QUALITY OF TEACHING PRACTICES Areas / Elements / Behaviors Scoring Final Scores A. CLASSROOM CULTURE 1. SUPPORTIVE LEARNING ENVIRONMENT 1 2 3 4 5 1.1 The teacher treats all students respectfully L M H 1.2 The teacher uses positive language with students L M H 1.3 The teacher responds to students’ needs N/A L M H 1.4 The teacher does not exhibit gender bias and challenges gender stereotypes in the classroom N/A L M H 2. POSITIVE BEHAVIORIAL EXPECTATIONS 1 2 3 4 5 2.1 The teacher sets clear behavioral expectations for classroom activities L M H 2.2 The teacher acknowledges positive student behavior L M H 2.3 The teacher redirects misbehavior and focuses on the expected behavior, rather than the undesired behavior L M H B. INSTRUCTION 3. LESSON FACILITATION 1 2 3 4 5 3.1 The teacher explicitly articulates the objectives of the lesson and relates classroom activities to the objectives L M H 3.2 The teacher’s explanation of content is clear L M H 3.3 The teacher makes connections in the lesson that relate to other content knowledge or students’ daily lives L M H 3.4 The teacher models by enacting or thinking aloud L M H 4. CHECKS FOR UNDERSTANDING 1 2 3 4 5 4.1 The teacher uses questions, prompts or other strategies to determine students’ level of understanding L M H 4.2 The teacher monitors most students during independent/group work N/A L M H 4.3 The teacher adjusts teaching to the level of students L M H 5. FEEDBACK 1 2 3 4 5 5.1 The teacher provides specific comments or prompts that help clarify students’ misunderstandings L M H 5.2 The teacher provides specific comments or prompts that help identify students’ successes L M H 6. CRITICAL THINKING 1 2 3 4 5 6.1 The teacher asks open-ended questions L M H 6.2 The teacher provides thinking tasks L M H 6.3 The students ask open-ended questions or perform thinking tasks L M H C. SOCIOEMOTIONAL SKILLS 7. AUTONOMY 1 2 3 4 5 7.1 The teacher provides students with choices L M H 7.2 The teacher provides students with opportunities to take on roles in the classroom L M H 7.3 The students volunteer to participate in the classroom L M H 8. PERSEVERANCE 1 2 3 4 5 8.1 The teacher acknowledges students’ efforts L M H 8.2 The teacher has a positive attitude towards students’ challenges L M H 8.3 The teacher encourages goal setting L M H 9. SOCIAL & COLLABORATIVE SKILLS 1 2 3 4 5 9.1 The teacher promotes students’ collaboration through peer interaction L M H 9.2 The teacher promotes students’ interpersonal skills L M H 9.3 Students collaborate with one another through peer interaction L M H 1 14 OBSERVATION NOTES SEGMENT 1 0.1 0.2 _______ 1.1 1.2 1.3 1.4 _______ 2.1 2.2 2.3 _______ 3.1 3.2 3.3 3.4 _______ 4.1 4.2 4.3 _______ 5.1 5.2 _______ 6.1 6.2 6.3 _______ 7.1 7.2 7.3 _______ 8.1 8.2 8.3 _______ 9.1 9.2 9.3 2 15 OBSERVER MANUAL TIME ON TASK 16 0 The teacher maximizes time on learning. TIME ON The teacher maximizes time on learning by ensuring most students are on task and provided with a learning activity most of the time. This can be observed in the classroom through the LEARNING following behaviors: YES NO 0.1 LEARNING ACTIVITIES: NONLEARNING ACTIVITIES: This includes any activity that is related to class content, This includes any activity that is not related to class content, The teacher is teaching independent of its quality. including activities related to classroom management such as or provides a learning For example, learning activities can include a teacher lecturing, small taking attendance or disciplining students or any other activity that leaves students waiting. activity for most group/team work, or students working on a worksheet or reading independently. Note that if the teacher leaves the classroom, but has For example, when the teacher is silently writing on the board without students provided students with a learning activity, this would still count as a asking students to copy. Other examples of nonlearning activities include: learning activity. when a teacher takes attendance, s/he may read the children’s names individually; when there are misbehaviors, s/he may stop the lesson to redirect student misbehavior; when there are outside disruptions, s/he may stop teaching to see what is going on; when checking homework, s/he may check each student’s homework individually, while the other students wait with nothing to do. In addition, basic classroom processes may be prolonged, such as transitioning to a new activity, getting materials ready for a lesson, or completing administrative tasks. LOW MEDIUM HIGH 0.2 6 or more students are off task 2–5 students are off task All students are on task (one student may be off task) Students are on task1 Students off task: This includes students who are not participating in the learning activity provided by the teacher either because they are quiet but distracted, or because they are disrupting the class. For example, in the first category, students may be staring out the window, resting their head on the desk, looking down to the floor or at the observer, or sleeping. In the second category, they may be passing notes, whispering, talking to another student during an activity that does not require talking, moving around the class, shouting, or in any other way disrupting the class. 1 This behavior is scored as N/A if the teacher is not teaching or providing a learning activity (i.e., 0.1 is scored N/A). 17 OBSERVER MANUAL QUALITY OF TEACHING PRACTICES 18 CLASSROOM CULTURE SUPPORTIVE LEARNING ENVIRONMENT POSITIVE BEHAVIORAL EXPECTATIONS 19 A.1 CLASSROOM CULTURE The teacher creates a supportive learning environment. SUPPORTIVE The teacher creates a classroom environment where students can feel emotionally safe and LEARNING supported. Moreover, all students feel welcome, as the teacher treats all students respectfully. ENVIRONMENT This can be observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality Range In this classroom, the In this classroom, the teacher is In this classroom, the teacher is teacher is ineffective at somewhat effective at creating a effective at creating a supportive creating a supportive supportive learning environment. learning environment. learning environment. 1.1 The teacher does not treat all The teacher treats all students The teacher treats all students respectfully. students respectfully. somewhat respectfully. For example: The teacher uses students’ names, The teacher treats all For example: The teacher may For example, the teacher does not treat students says “please” and “thank you,” or shows some students respectfully yell at some students, scold them, disrespectfully (e.g., s/he does not yell at or other culturally relevant sign of respect. shame/ridicule them, or use physical ridicule students), but the teacher does not show punishment to discipline them. outward signs of respect toward students either (e.g., call students by their names, say “please” or “thank you,” or other culturally relevant signs of respect). 1.2 The teacher does not use The teacher uses some positive language The teacher consistently uses positive positive language in his/her in his/her communication with students. language in his/her communication The teacher uses communication with students. with students. For example: The teacher may say “well done” or positive language “good”, although this happens infrequently. For example: The teacher consistently uses encouraging phrases such as “Great job!” when with students2 students show their work to him/her, or “You can do this!”, or “You are such a talented group of children.” 1.3 The teacher is not aware The teacher responds to students’ needs The teacher promptly responds to students’ of students’ needs OR does not but may not address the problem at hand. needs in a way that specifically addresses The teacher responds address the problem at hand. the problem at hand. For example: A student may be upset because s/ to students’ needs3 For example: A student may not have he does not have a pencil, and the teacher asks For example: If a student does not have a pencil, the required supplies for the lesson, another child to share his/her pencil, but s/he the teacher allows the child to borrow one from and the teacher does not notice or refuses. The teacher carries on with the lesson his/her spare pencil box. sees it and ignores it. Alternatively, without solving the problem. a student may be upset because of a bad grade or a personal problem, and the teacher ignores the student or is dismissive of the issue (e.g., the teacher tells the student to “get over it” or “pull yourself together”). 1.4 The teacher exhibits gender The teacher does not exhibit gender The teacher does not exhibit gender bias bias or reinforces gender bias, but does not challenge gender AND challenges gender stereotypes in the The teacher does not stereotypes in the classroom. stereotypes either. The teacher classroom. The teacher could show this by provides children of all genders with exhibit gender bias providing students with unequal equal opportunities to participate in the For example: The teacher assigns cleaning tasks to children of all genders, and calls equally on all and challenges gender opportunities to participate classroom and has similar expectations genders to answer difficult questions. In addition, in classroom activities, or by for all students. stereotypes in the expressing unequal expectations the teacher uses examples and explanations For example: The teacher assigns cleaning tasks that portray female rather than male scientists, classroom4 for students’ behaviors or to children of all genders, and calls equally on all doctors, and astronauts. capabilities. genders to answer difficult questions. For example: A teacher seats girls exclusively at the back of the classroom or only calls on boys to answer difficult questions. Alternatively, the teacher calls equally on students of all genders to answer difficult questions, but only assigns girls to classroom cleaning tasks. 2 Only verbal communication is counted as positive language; nonverbal displays of positive language would not count toward this behavior 3 This behavior is scored as N/A if there are no observable emotional, material, or physical needs. 4 The chances to participate should be considered proportionally to the ratio of different genders in the classroom; this behavior is only applicable in mixed-gender classrooms. 20 A.2 CLASSROOM CULTURE The teacher promotes positive behavior in the classroom. POSITIVE The teacher promotes positive behavior by acknowledging students’ behavior that meets or exceeds BEHAVIORAL expectations. Moreover, the teacher sets clear behavioral expectations for different parts of the EXPECTATIONS lesson. This can be observed in the classroom through following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality Range In this classroom, the In this classroom, the teacher is In this classroom, the teacher teacher is ineffective somewhat effective at promoting is effective at promoting positive at promoting positive positive behavior. behavior. behavior. 2.1 The teacher does not set The teacher sets unclear or superficial The teacher sets clear behavioral behavioral expectations for behavioral expectations for classroom expectations throughout the lesson for The teacher sets clear classroom tasks and/or activities. tasks and/or activities. classroom tasks and/or activities. behavioral expectations For example: The teacher says, “Work For example: When introducing a group For example: Upon introducing a group activity to on your reading comprehension skills,” activity, the teacher says, “Please sit in your the class, the teacher explicitly states the expected for classroom activities without providing instruction on what preassigned groups and behave,” without behavior for students in the group. This may include, the expected behavior is for the activity. clarifying what such behavior would entail. “Use a quiet indoor voice” or “Take turns speaking.” If students are working independently, the teacher gives directions on what to do when they complete the activity. The teacher says, “Please quietly get up, bring your worksheet to me, and read while you wait for your classmates to finish.” Alternatively, the teacher is not observed setting clear behavioral expectations, but students are well-behaved5 throughout the lesson. 2.2 The teacher does not The teacher acknowledges some The teacher acknowledges students’ acknowledge student behavior students’ behavior, but is not specific positive behavior that meets or exceeds The teacher that meets or exceeds about their expected behavior. expectations. expectations. acknowledges positive For example: If a group is following behavioral For example: A teacher says to the class, “I just expectations, the teacher says, “This group is noticed that members of Group A are taking turns student behavior working well together” or “This group is doing a to speak and are proactively working on the next good job,” without clarifying why or how. assignment.” 2.3 Redirection of misbehavior Redirection of misbehavior is effective When a problem arises, redirection of is ineffective and focuses on but focuses on misbehaviors rather than misbehavior effectively addresses the The teacher redirects misbehaviors, rather than the the expected behavior. Alternatively, problem at hand and focuses on the expected behavior. redirection of misbehavior is somewhat expected behavior. misbehavior and effective and focuses on the expected For example: If s/he notices a For example: If students are talking loudly and focuses on the expected distracted student, the teacher stops behavior. being disruptive during a lesson, the teacher says, behavior, rather than the lecturing and calls out the name of the For example: Upon noticing that 3 students “Remember to use quiet voices,” and the students student, asking her, “Why are you not are not working on the assigned problems, quiet down. undesired behavior5 paying attention in class?” Alternatively, the teacher says, “You 3 need to stop talking the teacher continues to ignore the now, you are making too much noise.” This Alternatively, the teacher is not observed student who is distracted, but the statement focuses on the disruptive students’ redirecting students’ behavior, but the distracted student begins to tease and negative behavior, rather than on what is students are well-behaved throughout argue with the peer sitting next to her. expected of them. Consequently, the disruptive the lesson. This shifts the focus of the entire class students quiet down. In another scenario, the away from the lesson and onto those teacher redirects the students by asking them 2 students. to “Focus on the task at hand.” Even though the teacher focuses on the positive behavior expected from the students, for the most part, they continue to talk. 5 A misbehavior occurs when a student causes a disruption in the classroom that either interferes with the flow of the lesson, distracts other students, or upsets the teacher. 21 INSTRUCTION LESSON FACILITATION CHECKS FOR UNDERSTANDING FEEDBACK CRITICAL THINKING 22 B.3 INSTRUCTION The teacher facilitates the lesson to promote comprehension. The teacher facilitates the lesson to promote comprehension by explicitly articulating the objectives, LESSON providing clear explanations of concepts, and connecting the lesson with other content knowledge or FACILITATION students’ experiences. This can be observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality Range In this classroom, the In this classroom, the teacher In this classroom, the teacher is teacher is ineffective in is somewhat effective in effective in facilitating the lesson to facilitating the lesson to facilitating the lesson to promote promote comprehension. promote comprehension. comprehension. 3.1 The teacher does not state the lesson objective(s), nor can one be The teacher either explicitly states a broad The teacher explicitly states a specific lesson lesson objective OR the objective is not objective (i.e. a learning goal) and the lesson The teacher explicitly inferred from the lesson activities. explicitly stated, but can be inferred from activities align to the stated objective. the lesson activities. articulates the objectives For example: The teacher asks students For example: Near the beginning of class the teacher to take turns reading a text about For example: The teacher says, “Today we’re states, “Today we’re going to learn to multiply of the lesson and relates planting and harvesting crops. S/he then going to learn about multiplication,” without fractions.” Each lesson activity is clearly related to the classroom activities to spends the rest of the lesson discussing further specification. Alternatively, the lesson objective of multiplying fractions. farming and the specific processes activities may clearly work toward how to the objectives involved. The teacher does not state a divide whole numbers, but this is not explicitly lesson objective, and it is difficult to infer articulated by the teacher. a lesson objective from the activities as the objective could be developing oral reading fluency, developing vocabulary, or learning about agriculture. 3.2 The teacher’s explanations of the The teacher’s explanations of the content, The teacher’s explanations of the content content are confusing OR content when they occur, are somewhat clear. are clear and easy to understand. The teacher’s is simply not explained. Although parts of these explanations may The teacher’s explanations are logical, be clear, others are confusing or superficial. and may be accompanied by graphic explanation of content For example: The teacher uses too representations or examples. many technical terms without explaining For example: While reading a story, the teacher is clear what s/he means and/or may explain identifies difficult words and defines them, but For example: In a lesson on fractions, the teacher ideas without a logical order or does not relate them to what is happening in provides a clear and thorough definition of a fraction, connection. Moreover, the teacher may the story. including defining “numerator” and “denominator.” S/he say, “A fraction is a combination of a has drawn several fractions on the board as examples. numerator and denominator,” without saying what either of those words mean. Alternatively, the teacher may not provide any explanation of content. 3.3 The teacher does not connect The teacher may attempt to connect the The teacher meaningfully connects the lesson to what is being taught to other lesson to other content knowledge or other content knowledge or students’ daily lives. The teacher makes content knowledge or students’ students’ daily lives, but the connections daily lives. The teacher may use are superficial, confusing, or unclear. For example: When teaching a class on fractions, the connections in the examples that may be related to teacher relates the content to students’ experiences For example: When introducing a lesson on by asking, “Who has had to slice a birthday cake? lesson that relate to other content or students’ lives, but fractions, the teacher says, “When we cut a cake, How did you make sure there were enough slices for the teacher does not attempt to other content knowledge connect it to the learning activity. we use fractions” and go on to explain fractions. everyone? Learning about fractions can help us divide The connection to students’ lives is superficial a cake between people.” The teacher also connects the or students’ daily lives For example: During a lesson on and nonspecific. Alternatively, the teacher says, lesson to a prior lesson on halves by saying, “Remember fractions the teacher uses a picture “Remember yesterday we learned the rules for yesterday when we learned about halves? We learned of a cake and divides it into fourths, adding whole numbers? Now we are going to use that when we cut a cake in half, we can share it equally but does not make a connection to those rules and apply them to adding fractions.” between 2 people. Today we will learn how to divide the students’ experience with slicing However, when explaining how to add fractions, cake into fourths, so 4 people can share the cake. When cake. Alternatively, the teacher says, the teacher does not link the rules back to the we were forming halves we made sure we had 2 halves “Remember, yesterday we learned rules for adding whole numbers. of identical size. The same thing is true when we are about whole numbers? Today, we are forming fourths: we have to make sure to keep slices going to learn how to add fractions.” of the same size.” The connection between the current lesson and other content knowledge and/or students’ daily lives is clear. 3.4 The teacher does not model. The teacher partially models the The teacher completely models the learning learning activity. activity by enacting all parts of the procedure The teacher models by OR by enacting the procedure AND thinking For example: In an English class where the aloud. enacting or thinking objective of the activity is to write a paragraph, the teacher only demonstrates how to write a topic For example: The teacher demonstrates different ways aloud6 sentence. In a math class, the teacher shows to solve a math problem (enactment of a procedure) (enacts) how to draw a bar graph, but does not and while doing so, s/he says what s/he is thinking at clarify how s/he extracted the data from the text to each step of the equation (think aloud). Or if students create the bar graph. are calculating the area of their desk, the teacher demonstrates each step in the process (full enactment of a procedure). 6  odeling can take place at any time in the lesson (including at the end). If the learning activity is procedural in nature, modeling will include an enactment of the procedure for children to observe; M however, if the activity focuses on developing a thinking skill, a complete model will include a think aloud. An action is considered modeling so long as the teacher demonstrates/enacts procedures or thinking processes related to the learning activity. 23 B.4 INSTRUCTION The teacher checks for understanding for most students. CHECKS FOR The teacher checks for understanding to ensure most students comprehend the lesson content. Moreover, the teacher adjusts the pace of the lesson to provide students with additional learning UNDERSTANDING opportunities. This can be observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality Range In this classroom, In this classroom, the teacher is In this classroom, the teacher is the teacher does not effective at checking only a few effective at checking for most check for any student’s students’ understanding. students’ understanding. understanding. 4.1 The teacher either does not ask The teacher uses questions, prompts, The teacher uses questions, prompts, questions/prompt students at or other strategies that are effective at or other strategies that are effective The teacher uses all OR when s/he does, the class determining only a few students’ level of at determining most students’ level of responds in synchrony, which understanding. understanding. questions, prompts, is accepted without further For example: The teacher asks, “What is 7+8?” For example: The teacher says, “Please put your or other strategies to clarifying for understanding. Only a few students respond by raising their thumb up if you agree or down if you disagree determine students’ For example: When explaining a hand, a group from which the teacher calls with this statement: Equilateral triangles have concept, the teacher asks, “Have you upon 1 or 2 students to provide an answer. equal angles.” The teacher also asks students to level of understanding all understood?” The students in the Alternatively, the teacher asks the question demonstrate their knowledge by having all students class respond in unison, “Yes, we have.” but does not ask students to raise their hands share their answers, e.g., by asking each student Another example is that the teacher in response and simply allows students to to read out the sentence s/he wrote using past inquires, “This is correct, right?” after willingly volunteer their answers. tense verbs. completing a problem set. The class or an individual student replies, “Yes, this is correct.” 4.2 The teacher does not monitor The teacher monitors some students The teacher systematically monitors most students when they are working when they are working independently or students by circulating the classroom and The teacher monitors independently or in groups. in groups to check their understanding. approaching individual students or groups to check their understanding. most students during For example: The teacher sits at his/ For example: The teacher observes some her desk or remains standing in front of student work for accuracy, clarifies concepts, For example: When students are working, the independent/group work7 the class when students are working. or asks questions. teacher walks around the classroom, making sure to approach students or groups in a systematic way. The teacher observes most students’ work, clarifies concepts, and asks questions. 4.3 The teacher does not adjust The teacher slightly adjusts teaching, The teacher substantially adjusts teaching for students.8 but this adjustment is brief and teaching for students. When students have The teacher adjusts superficial. misconceptions, the teacher may initiate back and forth exchanges to help them teaching to the level For example: As students complete an alphabet understand all points of misunderstanding, worksheet, the teacher notices they are not of students dotting their ‘i’s. In response, s/he briefly which provides students with more opportunities to learn. The teacher may also reminds the class to dot their ‘i’s. provide more challenging tasks for those who already have an advanced understanding. For example: As students complete an alphabet worksheet, the teacher notices they are not dotting their ‘i’s. In response, s/he briefly stops the activity and reviews the differences between capital and lower case ‘i’s before continuing with the alphabet activity. Alternatively, if the teacher notices that a student has already completed the worksheet, s/he may give that student another activity to complete while waiting for the rest of the class. 7 This behavior is scored as N/A if there is no observable group or independent work. 8 Even if there is no perceived need for adjusting, if the teacher does not adjust teaching, this behavior is scored as low. 24 B.5 INSTRUCTION The teacher provides feedback to deepen student understanding. FEEDBACK The teacher provides specific comments or prompts9 to help identify misunderstandings, understand successes, and guide thought processes to promote learning. This can be observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality Range In this classroom, the teacher is ineffective In this classroom, the teacher is somewhat effective at providing In this classroom, the teacher is highly effective at providing at providing feedback feedback to deepen students’ feedback to deepen students’ to deepen students’ understanding. understanding. understanding. 5.1 The teacher either does The teacher provides students with The teacher provides students with specific not provide students with general or superficial comments/ comments/prompts that contain substantive The teacher provides comments/prompts about prompts about their misunderstandings. information that helps clarify students’ their misunderstandings OR the misunderstandings. specific comments comments provided are simple, For example: In a math class, the teacher says, “You forgot to include the negative sign,” For example, the teacher says, “Do you remember or prompts that help evaluative statements (e.g., without providing further information what happens when we multiply a positive and a “That is incorrect”). clarify students’ or prompts. negative number? Let’s look at your notes. Now, let’s For example: When a student answers look at your answer. What do you need to change to misunderstandings a teacher’s question incorrectly, the find the correct answer?” teacher responds by saying, “That is not the correct answer,” and moves on. 5.2 The teacher either does not The teacher provides students with The teacher provides students with provide students with comments/ general or superficial comments/ specific comments/prompts that contain The teacher provides prompts about their successes prompts about their successes. substantive information that helps identify OR the comments provided are students’ successes. specific comments simple, evaluative statements For example: If students are writing stories as part of an assignment, the teacher says, For example: If students are writing stories, the or prompts that help (e.g., “That is correct”). “Good job on the third paragraph,” without teacher says, “You do a good job getting the reader identify students’ For example: When a student answers specifying what that particular student did interested in this paragraph when you write ‘no one a teacher’s question correctly, the that made it good. knew what would happen.’ This sentence makes successes teacher responds by saying, “That is me want to read more.” Alternatively, the teacher correct,” and moves on. highlights one student’s work and say to the class, “Look at the work of this classmate, see how s/ he used the number line to solve this subtraction problem?,” and then proceed to explain how the student solved it. 9 Prompts are pieces of information, such as guiding hints or questions, which are given by the teacher and encourage students to think through misunderstandings or identify successes. 25 B.6 INSTRUCTION The teacher builds students’ critical thinking skills. CRITICAL The teacher builds students’ critical thinking skills by encouraging them to actively analyze content. THINKING This can be observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality Range In this classroom, the In this classroom, the teacher In this classroom, the teacher is teacher is ineffective at is somewhat effective at effective at developing critical developing critical thinking developing critical thinking skills. thinking skills. skills. 6.1 The teacher does not ask open- The teacher asks students at least The teacher asks students 3 or more open- ended questions OR asks only 1 2 open-ended questions but does not ended questions AND at least 1 of them The teacher asks open-ended question. The teacher build on student responses, OR the builds upon student responses by asking may ask closed-ended questions teacher asks 2 open-ended questions students to justify their reasoning, further open-ended questions that have a predetermined answer. and 1 of them is a follow-up to a explain, or clarify their ideas. student response. that require reasoning, For example: The teacher asks, “Who For example: The teacher asks, “How do you think is the main character in this story?” or For example: The teacher asks, “Why was the the main characters in the story would prepare for explanation, or generalization “Which is greater, -2 or -6?” character unhappy? What makes you think the competition?” After a student responds, the or have more than 1 correct that?” OR “Why is -2 greater than -6?” And teacher then follows up by asking, “What facts or answer then asks, “How do you use the number line to ideas make you think that?” Then s/he asks another determine if -8 or -4 is greater?” student, “What do you think happens next?” In a math class, the teacher asks, “How do you know -2 is greater than -6?” After the student responds, the teacher follows up by asking, “What would happen if the numbers were positive?” Later in the lesson, the teacher asks, “How do you use the number line to determine if –8 or –4 is greater?” 6.2 The teacher does not provide The teacher provides superficial thinking The teacher provides substantial thinking thinking tasks. Classrooms with tasks. Superficial thinking tasks are tasks. Substantial thinking tasks are tasks The teacher provides no thinking tasks include those tasks such as matching sets of items, such as making predictions, identifying where students simply listen to the identifying concepts or key pieces patterns, explaining thinking, making thinking tasks teacher or perform rote tasks. of information, and comparing and connections, and interpreting information. contrasting characteristics. They also They also include applying learned information that require students to actively For examples, refer to the thinking task include applying learned information or or techniques to new tasks the teacher has table on the next page. analyze content, as opposed techniques to tasks similar to those the not demonstrated. to simply receiving information teacher has already demonstrated. For examples, refer to the thinking task table on the or building fluency (i.e., rote For examples, refer to the thinking task table on next page. learning) the next page. 6.3 Students do not ask open-ended Students do not ask open-ended Students ask open-ended questions. questions nor do they perform questions; however, they do perform thinking tasks. superficial thinking tasks. For example, after working on subtraction problems, The students ask open- a student asks, “Why does 6 – 9 equal a negative ended questions or For examples, refer to the thinking task For examples, refer to the thinking task table on number?” table on the next page. the next page. perform thinking tasks Alternatively, they perform substantial thinking tasks. For examples, refer to the thinking task table on the next page. 26 Thinking Task Table These examples are intended to help observers decipher what constitutes a thinking task and to differentiate between the quality levels. It is important to note that these examples are not comprehensive. In addition, context and students’ learning levels should be weighed considerably when scoring 6.2 and 6.3. Language LOW MEDIUM HIGH Classes 1. Learning Students repetitively read Students match photos to a letter. For example, The teacher has several short words written on the the alphabet. different letters are written on the board. The teacher calls board. S/he reads “cat” while pointing at the letters to read students up one at a time and gives them an image of a and asks students what would happen if they piece of fruit. S/he says, "What piece of fruit do you have? changed the first letter to “p” or “s.” Then s/he asks Think, what is the first letter of the name of your fruit them to choose a word and see what happens if and put your picture on the board under the appropriate they change the first letter. letter." 2. Reading Students take turns After reading a story, the teacher writes a series of After reading a story, the teacher says, "Now I want reading a story or simply questions on the board that students need to answer you to predict what might happen next in the story. comprehension listen to the teacher read independently. These questions ask students to identify Write down what you think would happen next and a story. key aspects of the story, such as the protagonist, the then share with your neighbor when you're done." setting, and the sequence of events. 3. Learning writing Students repetitively write The teacher asks students to write sentences where the Students are asked to analyze 3 different sentences example sentences. focus is on a specific sentence structure using a list of by listing the similarities and differences between specific verbs or nouns. the sentence structures and to explain why using one sentence structure is better than another. Math LOW MEDIUM HIGH Classes Learning about 1.  The teacher has students Students compare numbers based on size and The teacher puts sequences of numbers on the memorize numbers 1–100. organize them by descending or ascending order. For board and has students find the patterns. For numbers example, the teacher writes 8, 29, 72, 63, and 7 on example, the teacher writes the following 3 number the board. S/he tells students to write the numbers sequences on the board: 3, 13, 17, 23; 6, 15, 24, 30, in ascending order. Alternatively, the teacher tells 36; and 4, 12, 28, 32, 40. S/he then tells students to students, "Look at this set of numbers: 2, 5, 10, 19, 24. identify what each group has in common. Write down in 2 columns which are even numbers and which are odd numbers." Learning about 2.  Students listen to the The teacher explains the process of subtraction. She The teacher explains the process of subtraction. teacher explain the then asks students to complete several subtraction The teacher then writes a “menu” on the board subtraction concept and then copy the problems (e.g., “What's 10-5?") and to write their including prices. The teacher has students imagine examples from the board. answers in their notebooks. they have $20 and asks them to figure out how much change they would receive from buying different items. Learning about 3.  Students listen to the In a lesson on bar graphs, the teacher uses a chart of In a lesson on bar graphs, the teacher draws a bar teacher explain the numbers and draws a bar graph showing the class's graph showing the class's favorite foods. S/he bar graphs concept and then copy the favorite foods. S/he then asks students, "Which bar is then asks students to work in pairs to interpret the examples from the board. tallest? Which bar is the shortest?" information to identify and rank foods from the most preferred to the least preferred. S/he then tells them to calculate how many more students prefer the most preferred compared to the least preferred food. Learning about 4.  Students are told to repeat In a lesson on fractions, students are given pieces of The teacher tells students to fold a piece of paper the definition of a fraction paper cut into various shapes and are instructed to fold into sixths. Then s/he says, "Color in 3/6ths of your fractions to their neighbor. the paper into various shapes that represent fractions. paper. Write down the fraction of the colored part The teacher shows them how to fold into various of the paper and see how many other fractions you fractions and then, having them work in pairs, says, can write that represent this area. What patterns do "One of you will fold your paper into 1/5, the other will you identify among the fractions?” fold your paper into 1/3. Then, whoever has the larger fraction should stand up." Finding the area 5.  The teacher calculates After explaining how to find the area of a rectangle, After learning how to find the area of a rectangle, the area of 3 different the teacher draws a rectangle on the board, gives students are asked to compute the area of the of a rectangle rectangles on the board measurements, and has students use formulas they classroom, which is in the shape of a rectangle. and has students copy know to determine the area. the information in their notebooks. Solving word 6.  The teacher writes a The teacher writes a word problem on the board and The teacher writes a word problem on the board word problem on the board shows students how to solve it. The teacher then gives and shows students how to solve it. The teacher problems and shows students how students a set of word problems to solve. then gives students a set of word problems to to solve it. solve. The teacher calls on students to explain how they solved the different problems. 27 SOCIOEMOTIONAL SKILLS AUTONOMY PERSEVERANCE SOCIAL AND COLLABORATIVE SKILLS 28 C.7 SOCIOEMOTIONAL SKILLS The teacher allows students to make choices and AUTONOMY encourages students to participate in the classroom. The teacher provides students with opportunities to make choices and take on meaningful roles in the classroom. Students make use of these opportunities by volunteering to take on roles and expressing their ideas and opinions throughout the lesson. This can be observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality Range In this classroom, the teacher is ineffective In this classroom, the teacher is somewhat effective at In this classroom, the teacher is effective at developing students’ at developing students’ developing students’ autonomy. autonomy. autonomy. 7.1 The teacher does not explicitly The teacher explicitly provides students The teacher explicitly provides students with provide students with choices. with at least 1 superficial choice that is at least 1 substantive choice that is related to The teacher provides The teacher decides how learning not related to the learning objective. the learning objective. activities should be completed, students with choices without providing different options For example: The teacher allows students to For example: The teacher allows students to for how students can approach choose between different colored pencils to choose between writing an essay or doing a the task. complete an assignment, decide where to sit in presentation about their favorite sport. In a science the classroom when completing a task, choose class, the teacher lets students choose an animal For example: Students are asked to the order in which to complete the activities, to investigate. complete a set of math problems or vote on which student presentation was following a prescribed set of steps. the best.  Alternatively, the teacher tells students to write sentences without providing intentional choices. 7.2 The teacher does not provide The teacher provides students with The teacher provides students with students with opportunities to opportunities to take on limited roles in opportunities to take on meaningful roles in The teacher take on roles in the classroom. the classroom. the classroom, in which they are responsible for parts of a learning activity. provides students For example: The lesson is primarily For example: Students take attendance, assign lecture-based and highly structured; tasks, pass out materials, or write on the board. For example: The teacher gives a student the with opportunities subsequently, students’ participation Limited roles also include housekeeping tasks opportunity to solve an equation on the board and to take on roles in is limited to copying down information. such as fetching water, wiping the board, or explain to the class how s/he tackled the main In this lesson, students never get cleaning the classroom. challenges of the problem. the classroom the chance to come to the board or read a text. 7.3 Students do not volunteer to Only a few students volunteer to Most students volunteer to participate by participate in the classroom. participate by expressing their ideas expressing their ideas and taking on roles. The students volunteer and taking on roles. For example: When the teacher asks a question, to participate in the For example: When the teacher asks a many students put their hand up to share their question, only a few students put their hand answers. The students could also volunteer without classroom up to answer; later when the teacher asks the teacher asking (e.g., a student offers to share another question, the same few students put a related experience when the teacher is explaining their hand up. a concept). 29 C.8 SOCIOEMOTIONAL SKILLS The teacher promotes students’ efforts, has a positive PERSEVERANCE attitude toward challenges, and encourages goal setting. The teacher promotes students’ efforts toward the goal of mastering new skills or concepts, instead of focusing solely on results, intelligence, or natural abilities. In addition, the teacher has a positive attitude toward challenges, framing failure and frustrations as useful parts of the learning process. The teacher also encourages students to set short- and/or long-term goals. This can be observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality Range In this classroom, the In this classroom, the In this classroom, the teacher teacher is ineffective at teacher is somewhat effective is effective at developing developing students’ at developing students’ students’ perseverance. perseverance. perseverance. 8.1 The teacher does not In this classroom, the teacher sometimes In this classroom, the teacher frequently acknowledge student efforts. acknowledges student efforts, but acknowledges students’ efforts toward The teacher Although the teacher may praise most praise is focused on outcomes or mastering new skills or concepts, and students for “being smart” or student intelligence. identifies these efforts explicitly. acknowledges “intelligent,” the teacher does not For example: When a student does well on For example: When students solve a difficult problem students’ efforts focus on students’ efforts or work. a test, the teacher says, “I know you put so they had been struggling with, the teacher praises For example: The teacher says, much hard work into this!,” but most times, the and highlights the efforts they made to solve the rather than focusing only “Very good! You’re the smartest teacher praises students by saying they are problem. The teacher says, “You have progressed on results, intelligence, or student in the class” or “Well done! “smart” or “intelligent.” so much on our multiplication problem sets! I’m natural abilities You’re so smart!” glad you asked me for help. If you keep practicing and using the strategies we learned in class, you’ll master them all very soon!” 8.2 The teacher has a negative The teacher has a neutral attitude The teacher has a positive attitude toward attitude toward students’ toward students’ challenges. Although students’ challenges, and helps students The teacher has a challenges. the teacher does not penalize a student understand that failure and frustration are for making mistakes or struggling with normal parts of the learning process. positive attitude towards For example: The teacher explicitly a new concept, the teacher does not scolds students for making mistakes For example: When a student is struggling with a students’ challenges10 or becomes impatient with a student make it clear that failure and frustration problem set, the teacher says, “Remember, it’s okay are normal parts of the learning process for taking time to understand a new either. to feel frustrated when we’re trying to do something concept. new! Let’s think about how we can go about this.” For example: When a student is struggling The teacher also encourages students to think to solve a math problem on the board, the through different resources they could turn to for teacher simply gives the student the answer help (e.g., asking a friend for advice, looking for in a neutral manner (i.e., not in an angry or answers in the textbook). impatient manner). 8.3 The teacher does not encourage The teacher encourages students to set The teacher encourages students to set students to set short- or long- either short- OR long-term goals.11 short- AND long-term goals.11 The teacher The teacher encourages term goals.11 may reference both long- and short-term For example: For short-term goal setting, the goals at the same time, particularly when goal setting teacher says, “How many pages of the book will encouraging students to set a short-term goal you read each day this week?” For long-term that would help them achieve a long-term goal. goal setting, the teacher says, “I want you to write down how much progress you’ve made For example: The teacher says, “Let’s think about the on the goals we set at the beginning of the goals we set for ourselves at the beginning of the school year.” school year. What is one thing you will do this week that will get you closer to that goal?” Alternatively, Alternatively, the teacher may talk about the teacher talks about the short- and long-term the importance of setting goals in a goals separately (as in the examples for “Medium”). general way. For example: The teacher says, “It’s important to think about what you want to be when you grow up.” In addition, the teacher highlights how characters in a story set a short- or long- term goal for themselves and how they worked toward it. 10 These challenges may include making mistakes, scoring low on a test, or feeling frustrated when trying to understand a concept. 11 Short-term goals are goals that students aim to achieve within a month or less, and long-term goals are goals that span a longer timeframe (e.g., over the school year, when they grow up). 30 C.9 SOCIOEMOTIONAL SKILLS The teacher fosters a collaborative classroom environment. The teacher encourages students’ collaboration with one another and promotes students’ SOCIAL & interpersonal skills. Students respond to the teacher’s efforts by collaborating with one another COLLABORATIVE in the classroom, creating an environment free from physical or emotional hostility. This can be SKILLS observed in the classroom through the following behaviors: Score 1 2 3 4 5 Behavior LOW MEDIUM HIGH Quality Range In this classroom, the In this classroom, the In this classroom, the teacher is teacher is ineffective teacher is somewhat effective effective at developing students’ at developing students’ at developing students’ collaborative skills. collaborative skills. collaborative skills. 9.1 The teacher does not promote The teacher promotes superficial The teacher promotes substantial student collaboration among students. student collaboration through sharing collaboration by asking them to work together The teacher promotes opinions, materials, or ideas. to produce a product, solve a problem, For example: The teacher does not complete a worksheet, or present a new idea. students’ collaboration provide any opportunities to work in For example: The teacher asks students to groups or pairs. read their neighbor’s work or share crayons For example: The teacher asks students to form through peer interaction with each other. pairs or groups to complete a task that requires collaboration, such as creating a diagram of the water cycle or coming up with skits to illustrate a set of vocabulary words. 9.2 The teacher does not promote The teacher promotes students’ The teacher promotes students’ interpersonal students’ interpersonal skills. interpersonal skills in a brief or skills by encouraging perspective taking, The teacher promotes superficial manner. empathizing, emotion regulation, or social problem solving. students’ interpersonal For example: The teacher tells students to “Help each other” during a group exercise, asks For example: The teacher asks a student, “How do skills, a child to “Say you’re sorry” to a classmate, or you think that made him/her (classmate or character encourages children to take turns during an in a book) feel?” such as perspective taking, activity. However, the teacher does not explain empathizing, emotion regulation, why these behaviors are important. See FAQ 9.2 for more examples. and social problem solving12 9.3 Students do not collaborate OR Students collaborate superficially; Students collaborate with one another when students interact with one there may also be minimal instances by working together to produce a product, Students collaborate another, they display negative where students display negative solve a problem, complete a worksheet, behaviors. behaviors (e.g., teasing, pushing, or present a new idea. There are no displays with one another through bullying); however, these behaviors of negative behavior. For example: When asked to pick peer interaction partners for an activity, students are isolated and minor or playful (i.e., For example: Students work in groups to complete a no student is upset) and are not a core purposefully exclude one or more characteristic of the classroom. task that requires collaboration, such as creating a of their peers. diagram of the water cycle or coming up with skits For example: Students share materials among to illustrate a set of vocabulary words. themselves in a group, but they complete the activity independently and do not collaborate with one another on problem sets. 12 Perspective taking: The ability to consider a situation from a different point of view. Empathizing: The ability to recognize and share another’s emotions. Emotion regulation: The ability to effectively manage and respond to an emotional experience. Social problem solving: The process that an individual goes through to solve an interpersonal problem. This may involve applying aspects of perspective taking, empathizing, or emotion regulation to a social situation. 31 FREQUENTLY ASKED QUESTIONS 32 Time on Task (0.1a) When the class is in transition, how do I know when the transition has ended? Transitions occur in most classes. As indicated in the manual, consider what most of the students are doing and if the teacher is providing opportunities to learn. A transition officially ends when most students are provided with the next learning activity. For example, if the teacher says, “Take out your workbooks and complete the exercise on page 3,” but students have not yet taken out their workbooks at the time of the snapshot, this is still considered a learning activity as the teacher has provided a learning activity for most students. However, the students may be off task. (0.1b) How do I code the snapshot if a learning activity happens concurrently with administrative activities? Even though the teacher is doing administrative tasks (which are considered nonlearning activities), it counts as a learning activity if most students are provided with a learning activity. For example, while taking attendance, a teacher may ask children to identify phonemes and put their names on the wall under the first letter of their name. (0.2) Are students off task if they leave the room during the snapshot? They are counted as off task. If they leave the room before the snapshot, observers should not count them as off task. Quality of Teaching Practices (1.1) Must a teacher use students’ names to treat students respectfully? In some cultures, the use of names may not be a common sign of respect. If the teacher does not use names but exhibits other signs of respectful behavior (e.g., the teacher uses terms of endearment to refer to students, uses a respectful form of a word, or speaks to students in a warm tone of voice), this may still be scored a high. (1.2a) Is nonverbal communication counted as positive language? Although praise for students may come in many forms, behavior 1.2 only seeks evidence of “positive language.” As such, nonverbal communication, such as clapping or smiling, does not impact the overall score. However, if the teacher makes a statement such as “Let’s give a round of applause,” this is counted toward positive language — not because of the applause, but because the teacher verbally communicates positive language. (1.2b) What is considered “consistent” positive language? Specifically, where should I draw the line between a medium and a high score? Both the consistency and the quality of the comments should be taken into consideration. For example, if a teacher simply says, “You are such a talented group of students” and “Awesome!” in a 15-minute segment, it is weighted more heavily than the teacher saying “Good” 4 times. However, if the teacher says “Very good” 7 times, this would constitute a high rating. The following basic thresholds may be used as a loose guide to determine scoring: 0 instances of positive language constitutes a low score, 1–4 instances is a medium score, and at least 5 instances is a high score. (1.3a) If a student needs to go to the bathroom, is that considered a need? Yes, although the examples in the manual have to do with providing materials or emotional support, please remember that these are simply examples and are not comprehensive. Any observable emotional, material, or physical needs are captured here. If a student needs to go the bathroom, that could affect how s/he pays attention during the class, and it is important for the teacher to address. It is important to note, what is not captured here is a student’s need to understand academic content as this is captured when the teacher adjusts the lesson (behavior 4.3). (1.3b) During a partner activity, the teacher rearranges partners to include a student without a partner. Does this count as responding to a student need? Yes, although rearranging students in the classroom is not automatically considered responding to student needs, if a student does not have a partner or group for an activity and the teacher rearranges students to include the student, then this is considered to be addressing a student need. For this to count, there must be an identifiable student need — e.g., the student either has to visibly not have a partner, or the teacher might ask, “Who doesn’t have a partner?,” and the student responds that s/he does not have a partner. 33 (1.3c) Does asking a student if s/he has a specific need automatically count as responding to a student need? No, a teacher simply asking if a student has a need does not necessarily count as responding to a student need. For example, if the teacher asks students if they are hungry or tired in an attempt to engage them, this does not automatically count as responding to a student need. However, this is scored a medium if a student indicates the perceived need does indeed exist by indicating s/he is tired or hungry, or if it is clear that a student is tired or hungry. If the teacher addresses the problem by giving that student something to eat, this is scored a high. (2.1) How are behavioral expectations different from directions or instructions for an activity? Behavioral expectations focus on the expected behavior during an activity, whereas instructions for an activity focus on the steps required to complete an activity. For instance, the teacher may provide instructions for an activity by saying, “Read the first paragraph and then answer the questions on page 12” — this tells students what they need to do to carry out the activity. On the other hand, the teacher may state behavioral expectations by saying, “If you have any questions, quietly raise your hand” — this sets clear behavioral expectations for students to follow during the activity. (2.3) A student was sleeping in class, but I know he was up all night working. The teacher seems sympathetic toward him and is letting him sleep. Does this affect the score? There are 2 issues here. First, observers need to be very careful to not let any outside information influence their coding. No matter what the reason, only code and score what is observed during the coding segment. The second issue is the definition of misbehavior. Two factors may be considered when deciding if the student is misbehaving: if the student is causing a disruption in the classroom (distracting students who are trying to pay attention to the lesson), AND if the teacher is bothered by this disruption. If neither the teacher nor the other students are bothered by the student sleeping and it is not disruptive to the flow of the lesson, the behavior 2.3 score could still be a high, depending on the other evidence in the classroom. (3.1) Students are just reading and discussing a story for class. The teacher says, “Today we’re going to talk about [title of story].” Does this count as stating the lesson objective? A lesson objective should say why the class is doing the activity, rather than what activity students will be doing. For example, an activity may be to read a passage about plants and to answer questions based on the text, while the objective of the activity is to learn about photosynthesis. In this case, although the teacher clearly defines the activity for the class, there should be some objective to explain why students are reading the story (to learn new vocabulary, different parts of speech, etc.). Thus, this statement alone does not count as an explicit definition of the lesson objective. (3.2a) How do I code this behavior if the focus of the class is an activity? Content includes directions for activities. If the teacher’s directions for activities are confusing, the score is marked a low. If the teacher’s directions are somewhat clear, this is marked a medium. If the directions are clear, this is marked a high. (3.2b) The teacher explains the content clearly; however, the explanation s/he gives is incorrect. Do I still score this a high? Yes, if the teacher’s explanation of content is clear and easy to understand, this is still scored a high. This element does not assess the accuracy of content, but rather, how the content is delivered. Thus, if the teacher’s explanations are wrong but consistently clear, it is still scored a high. This element does not require observers to discern correct from incorrect material. (3.3a) What exactly counts as students’ daily lives and how is it determined to be “meaningful?” The teacher needs to explicitly state how the content is related to students’ lives, rather than observers inferring what they think is related to students’ lives. If the teacher only mentions objects students may encounter in their daily lives, such as “Let’s count the flowers,” this is not considered a meaningful connection. However, if the teacher makes an explicit statement that connects to students’ lives, such as “Here is a flower like the one that we have in the garden,” that would be an attempt to make a connection. In the above example, barring other evidence, the behavior is scored a medium because it is not explicitly connected to the lesson objective. However, if after making the explicit connection to their own garden, s/he connects the example to the lesson objective by saying “So if we have 2 gardens with 6 flowers each, how many flowers are there total?,” this constitutes a high score because the teacher explicitly relates the example to both students’ daily lives and the lesson objective. 34 (3.3b) What counts as making connections to other content knowledge? Does recalling what was learned in a previous lesson count as a connection? It may — particularly if the teacher attempts to explicitly connect the lesson to the past content knowledge. For example, if the teacher says, “Remember when we learned the alphabet? Today we will use the alphabet to form syllables,” this is scored a medium because although the teacher explicitly connects new content to past content, s/he only does so superficially. However, if the teacher further explains how to use the alphabet to form syllables, this is scored a high because the teacher not only recalls what was learned in a past lesson and references how it connects to new content, but builds upon past content to contextualize new material. If the teacher simply recalls what was learned in a previous lesson without making an explicit connection to the current lesson, this is scored a low. For example, the teacher may say, “Remember how we learned about fractions yesterday? Today we’re going to learn about decimal places.” (3.4a) I’m having trouble with modeling. How do I know when I see it? What should I specifically look for in modeling? Modeling a procedure or skill will mirror an activity that students are asked to do in that lesson or in the near future. Teachers can model by enacting the procedure (showing how to perform a task) or thinking aloud. Cognitive modeling, or a “think aloud,” refers to when a teacher explicitly discusses a thought process or strategy to students by thinking through the challenge aloud (e.g., how to extract important information from a word problem, how to determine theme in a text). When the teacher enacts a procedure, s/he shows all, or some, of the steps in a process for a complete or partial model. Showing the end product could look different across disciplines; however, it essentially gives students an example for which to strive. (3.4b) Does modeling always have to happen before the activity? Although the traditional idea of modeling is when the teacher enacts or thinks aloud a task and then students complete the same activity, modeling does not always have to take place before the activity. Modeling can occur whenever the teacher enacts a procedure or thinks aloud regardless of whether it is at the beginning or end of the activity. For this to occur, it is important that the enacted task or presented think aloud is the same as the task students are expected to perform or have performed. Modeling can occur at the end of class if the teacher walks students through the thinking process as s/he solves a problem. However, simply revealing the answer to a learning activity or a math problem is not considered modeling. (3.4c) What is the difference between an instructional explanation and modeling? To model for students, the teacher needs to perform the task or parts of the task s/he is asking students to do. This is different than giving them directions or explaining an activity as it involves teacher demonstration. The teacher may also demonstrate his/her thinking process as part of the modeling. If the task is to learn the meaning of new words in a text and the teacher simply provides students with a definition of a word, this may contribute to a clear explanation (3.2), but it does not necessarily constitute modeling. An example of modeling is if the teacher were to demonstrate how s/he uses context clues to find the meaning of a word. For example, the teacher may say, “When I don’t know the meaning of a word (in this case, “abrupt”), I reread the sentence, and think about the context, here I read……., therefore I know this means something like sudden or unexpected.” In a math classroom, the teacher may be working with students to estimate lengths in standard units. S/he may explain the length of a centimeter and provide examples of common objects that are a centimeter long — this is part of his/her instructional explanation (3.2). To model, the teacher may show students how to estimate. For example, s/he may show the width of his/her finger is approximately 1 cm and that s/he can use this knowledge to try to estimate the length of a pencil by thinking about (or measuring) how many of his/her finger widths fit along the length of the pencil. (3.4d) I’m still having troubling identifying modeling. Any other tips? To determine whether the teacher has modeled, ask yourself: 1. What is the learning activity? What are students being asked to do or learn? Did the teacher show students what this process or skill looks like? 2. Is the thing students are being asked to do a process or a thinking skill? a. If students are asked to do a thinking skill, the teacher has to do a think aloud to be scored a high. If the task is procedural, the teacher should show students all steps in the process. b. Students then complete a similar activity in that lesson or in the near future. 35 (3.4e) If the teacher models a procedure — for division, for example — but then students are requested to do a different division activity, is it considered modeling? If students do some of the procedure, it could be partial modeling. However, if what students do is unrelated to the procedure shown by the teacher, it does not count as modeling. So, while the activity does not need to be identical, some or all of the procedures modeled need to be included in the activity to be counted as evidence toward modeling. (3.4f) Can students and teachers co-construct a model or should it be entirely teacher-led? Although we often think of teachers presenting a model for the benefit of the student, some cases arise where modeling is not completely led by the teacher and students may be a part of the process. For example, students and the teacher co-construct knowledge by enacting a procedure together to get to the final product. (4.1a) Can an activity be a way to check for understanding? It is important to stick to the manual by remembering that the teacher needs to ask questions to check for understanding. However, the questions asked by the teacher can be written or verbal, which would be inclusive of an activity. For instance, the teacher may pass out a written quiz to students and check their answers to determine their level of understanding. It is important to note that just giving a quiz is not a check for understanding; the teacher must check students’ answers during the segment for it to count as a check for understanding. Additionally, checking homework (or work that was assigned prior to the observed segment) is not counted toward checking for understanding unless it is clear that the content of the work is related to the current lesson. (4.1b) How do I know what constitutes an “effective” check for understanding? Specifically, what is the difference between a medium and a high score? This behavior is designed to capture the extent to which the teacher makes an effort to check if students understand the content. In an effective check for understanding, the teacher gives individual students the opportunity to show what they know. For example, a highly effective way to check for understanding is by having students come to the board to complete a math problem. This is classified as such because the teacher is able to see the extent to which each individual student understands and is able to complete the task; however, this system does not allow for the teacher to gain information about MOST students’ understanding. What differentiates between a medium and a high score is whether the teacher gains information on MOST students’ understanding over the course of the lesson. For example, a highly effective way a teacher could determine most students’ understanding is by asking them to agree or disagree with statements by showing a thumbs up or down. This behavior does not capture if the teacher does something with that information (this is captured in behavior 4.3). (4.2a) During independent/group work the teacher walks around but does not approach or talk to students at all. Does this count as monitoring? Yes. The teacher can verify students’ understanding without providing comments; at times it is difficult to tell whether the teacher is looking at student work as s/he walks around the classroom. Thus, if the teacher simply walks around the classroom during independent or group work, this is scored a medium. Visual cues should also be taken into account: e.g., the teacher points to students’ work, leans in, or says something observers may not be able to hear. If the teacher is observed monitoring most students in this way, it may be scored a high. (4.2b) The teacher asks students to write the school name and date in their notebooks. They spend a significant amount of time doing this. Does this count as independent work? Yes, writing in their notebooks is a learning task for students who do it independently. Other examples of independent work are: copying down examples from the board when the teacher asks them to and independently completing tasks given by the teacher (e.g., write down a proper noun, draw a picture, complete math equations, etc.). If students are reading something in unison (e.g., the alphabet) and the teacher circulates the classroom and approaches individual students and corrects them, this would count as a whole-group activity. Thus, it does not count as independent/group work. The teacher’s comments are captured under feedback (5.1) and/or adjusting (4.3). (4.3) Most of the adjustment examples are about explanation of content. Are there other ways a teacher could adjust? Although the teacher may effectively adjust by further explaining content, adjusting teaching means giving more opportunities to learn, so the teacher may also do this in other ways. For example, the teacher may give more time to finish a task, provide students who finish early with additional or more advanced tasks, or provide feedback. Sometimes an overlap between feedback and adjusting teaching may occur, since the teacher can comment on students’ work and adjust the lesson; however, not all feedback should be counted as adjusting. 36 (5.1/5.2) There is only 1 instance where the teacher provides specific comments. Is this enough for scoring a high? Yes, but it depends on the quality of the teacher feedback. If the teacher gives 1 comment and provides substantive information about what a student did well on or helps clarify misunderstandings, this could be scored as a high. For example, while giving feedback to a student, the teacher may say, “With what number should ascending order start? Biggest or smallest? The smallest. But you started with the biggest. Ascending order starts with smallest, so it has to be like this.” However, if the comment is somewhat vague or in the form of a hint, this would likely be considered a medium. For example, while students are completing independent work the teacher may circulate and tell a student, “Don’t write it there, start writing it from here” or “Leave room for your words to breathe.” These comments are not specific. (6.1) The teacher asks many open-ended questions, but does not give students a chance to respond or answers on behalf of students. How should I score this? This is a good example of what may distinguish a high from a medium. If a teacher asks many open-ended questions, but does not give students a chance to respond or answers on behalf of students, then the teacher cannot build upon student responses. Thus, this is scored a medium. To score a high, the teacher must ask 3 or more open-ended questions AND at least 1 of these questions should build upon student responses. (6.2/6.3) How do I score this behavior if students are completing a worksheet? How do I know if the worksheet includes a thinking task or not? If it is impossible to determine what is on the worksheet, this would not count toward a thinking task. Remember, you can only score what you see or hear. If you receive some indication of what is on the worksheet (e.g., through the teacher’s instructions or students’ questions), score the task according to the quality ranges outlined in the manual. (6.3) Does answering thinking questions count as performing a thinking task? Answering an open-ended question counts as performing a thinking task if students perform a thinking task with their answer. For example, after reading a story, the teacher could ask, “How do you think the main character felt after losing the competition?” If a student responds, “I think he felt sad because he practiced very hard, and really wanted to win the competition,” this would count as performing a substantial thinking task as the student is explaining his/her thinking. (refer to the Thinking Task Table for more examples of thinking tasks). However, if the teacher asks an open-ended question and students answer by simply repeating knowledge they have learned, it is not considered a thinking task. For example, the teacher may ask, “What happened after the main character lost the competition?” If a student says, “He cried,” this does not count as a thinking task because the student is simply recalling information. (7.1a) Can an open-ended question/task count as providing students with choices? If the teacher asks an open-ended question, this would likely not count as a choice. An open-ended task could be counted toward the teacher providing students with choices if the teacher’s instructions explicitly imply s/he intends for students to make a choice. For example, the teacher could say, “Select one of these topics for your essay” or “You can decide which method to use to solve the problem.” (7.1b) How do I code this behavior if there is no clear learning objective? If there is no stated learning objective or if the objective cannot be inferred from the learning activities, this behavior cannot be scored a high. It is scored a medium if a choice is presented and a low if no choice is presented. (7.3a) What contributes as evidence toward volunteering? What is captured under this behavior is whether students are volunteering information or simply doing as required in a certain situation. Reciting information in call-and-response fashion or responding in unison to the teacher’s questions in a rehearsed or expected fashion — e.g., all students answering “Yes” when the teacher asks, “Do you understand?” — does not count as volunteering to participate in the classroom. Although the example in the manual is “students raise their hand,” students are also volunteering information when they answer questions without being called upon. Therefore, even if they do not raise their hand, if most students volunteer answers in response to the teacher’s questions, this is still scored a high. For example, the teacher may ask, “Who knows the answer?” If most students call out their responses (with or without raising their hand) (e.g., “Me!,” “The answer is 5!,” etc.), then this is scored a high; if only a few students answer, then it is scored a medium. 37 (8.1a) What if students do not seem to be making any effort in the class? How do I score this behavior? If the teacher does not acknowledge any effort, even if s/he does not provide any tasks or questions that seem to challenge students or they do not seem to be making any effort, this should still be scored a low. Teachers can always find things students are doing or have done (recent homework, for example) that can earn them acknowledgment for their efforts, even if it seemed to be easy for them. (8.1b) What is the difference between acknowledging students’ effort (8.1) and using positive language (1.2)? Acknowledging students’ effort includes comments that focus specifically on the work and effort of the student. While acknowledging students’ effort may also count as positive language, a comment that constitutes positive language does not necessarily constitute acknowledging students’ effort. For example, “You have made so much progress on your writing! I can tell you have been practicing!” is a comment that counts toward positive language AND acknowledging students’ effort. “Good job!! You are such a fast writer!” is an example of positive language, but does NOT count toward acknowledging students’ effort. (8.2a) If no mistake is observed, how can I tell the teacher’s attitude toward challenges? As the 3 choices are low, medium, and high, the teacher’s attitude will always fit into 1 of those 3 categories. Any question could be a challenge to students, so watching the teacher throughout the segment should provide enough information to code this behavior. If the teacher has a neutral attitude, does not get angry/impatient, or does not scold or penalize students for making mistakes, then it is scored a medium. (8.2b) The teacher did not scold a student, but did seem annoyed. How should I score this? The example of a negative attitude toward challenges includes “scolding,” but it is important to consider other forms of negativity, such as annoyance and impatience. It is important to take cultural differences into consideration (like for 1.1). (8.2c) In scoring positive attitude toward students’ challenges, should I consider the “best” incident or the average over the course of the segment? For this behavior, observers should consider the average attitude of the teacher over the course of the segment. For example, the teacher might show a positive attitude toward students’ challenges when a student makes a mistake and the teacher says, “It’s ok, we’re learning.” However, if besides that isolated incident the teacher consistently and explicitly scolds or becomes impatient with students, this is scored a low or a medium (depending on the balance of incidents over the segment). However, if no clear indications of a negative attitude arise, then 1 instance of a positive attitude is enough to make the score for this behavior a high. (9.2) How could a teacher promote perspective taking, empathizing, emotion regulation, and social problem solving? An example of perspective taking is: A boy gets upset because his classmates excluded him from a game. The teacher encourages perspective taking by explaining to the boy that his classmates might not have known that he wanted to join in the game, and then encouraging him to ask them if he could participate. An example of empathizing is: When a group of students are teasing a classmate, the teacher promotes empathy by asking the group members to consider how they would feel if they were the ones being teased. An example of emotion regulation is: When a student is upset, the teacher promotes emotion regulation by providing strategies for the student to deal with his or her emotions, such as taking a deep breath or counting to 10. An example of social problem solving is: There is a problem between 2 students. The teacher encourages social problem solving by acknowledging the issue, recognizing students’ emotions, and suggesting they brainstorm a solution together. The teacher may also intentionally model interpersonal skills; for example, the teacher may demonstrate how to stand up to a bully. What if I still have a question? Read, read, read the manual and these FAQs. If your question remains unanswered, ask your trainer or email teach@worldbank.org. It is much better to address your question than to make an assumption and incorrectly code an observation segment. 38 “Teach is a magnificent example of research taken to the practical level with the possibility of providing enormous social value. The imaginative use of an observational platform as a device for monitoring what actually happens in the classroom could be absolutely revolutionary. Instead of just bemoaning the need to improve classroom teaching, this initiative turns research and evaluation into a clear improvement mechanism.” . Eric Hanushek Paul and Jean Hanna Senior Fellow, Hoover Institution, Stanford University “Up to this point, the lack of open source, flexible, easy-to-learn observational measures that can be used systematically in classrooms has stood as a major stumbling block in international efforts to improve education. Observing not just what teachers teach, but how teachers teach — it is a critical step for improvement. Teach not only fills this gap but stands apart from typical observational measures in that it can be used systematically but has some flexibility built into to adjust and adapt to cultural variation. The Teach development process has been meticulous, building on a strong theoretical base and decades of empirical research. Further, as part of the validation work, Teach has been tested in more than 15 countries in four continents which represents a unique strength. Teachers in classrooms around the world hold great power in shaping our future. The Teach measure offers a unique window into this space.” Sara Rimm-Kaufman Professor of Education, Center for Advanced Study of Teaching and Learning, Curry School of Education, University of Virginia “Teach represents a major innovation in our efforts to improve education for all. It helps us address crucial questions such as: How can teachers create an engaging and supportive learning environment? How should teachers teach so they can help students develop strong content foundations and critical thinking skills? How can teachers nurture independent, resilient, and socially competent learners? Teach will be catalytic for enhancing learning all around the world!” . Oon Seng Tan Director, Centre for Research in Child Development, National Institute of Education, Singapore “Observing teachers in the classroom is a powerful strategy for understanding what drives learning and giving individual teachers the feedback they need to become more effective. Teach — the first-ever classroom observation instrument designed for developing countries — is an essential resource for countries seeking to improve education results. Teach combines the best features of prior instruments into an all-in-one package that analyzes teachers’ time on task, socioemotional support for students, and use of high-quality teaching practices. Its observer training materials and software are all open source and user-friendly, and Teach results can be benchmarked against a growing number of developing countries. Teach makes a huge contribution to education quality in the developing world.” . Barbara Bruns Center for Global Development and Walsh School of Foreign Service, Georgetown University “Teach provides excellent guidance for observing and rating global classroom instruction. The instrument is impressive not only for its comprehensiveness but also its specificity — naming key classroom practices and describing concrete examples of how those practices occur at different levels of quality. A useful and accessible tool for heads of school, administrators, and even teachers themselves.” Heather Hill Jerome T. Murphy Professor in Education, Harvard Graduate School of Education; Creator of the Mathematical Quality of Instruction (MQI) instrument “Teach provides a practical tool for educators around the world who are serious about improving the quality of classroom practice. Designed specifically for a global audience, Teach builds on a strong research base and has been tested in multiple countries. Although observation protocols have been used primarily to evaluate teaching, their greatest promise lies in the possibility of creating a common instructional vision and providing specific feedback to teachers on how to improve their instruction. Teach will no doubt provide such learning opportunities for teachers and leaders worldwide.” . Pam Grossman Dean and George and Diane Weiss Professor, Graduate School of Education, University of Pennsylvania; Creator of the Protocol for English Language Arts Teaching Observation (PLATO) instrument “Teach is a classroom observation tool that has clearly been designed with the realities of the Global South in mind. The clear explanations, well-crafted examples, and FAQs ease interpretation and ensure commonality of understanding between observers. The simplicity of the tool makes it particularly suited for the purpose of monitoring classrooms and also for capturing insights for further improvement in teacher practices. Teach is also the first-ever classroom observation tool that capture teachers’ efforts to foster soft skills.” . Sara Ruto Director, People’s Action for Learning (PAL) Network Contact us at teach@worldbank.org and visit us at www.worldbank.org/education/teach