Enhancing School Quality in Vietnam through Participative and Collaborative Learning Vietnam Escuela Nueva Impact Evaluation Study S. D. Parandekar, F. Yamauchi, A. B. Ragatz E. K. Sedmik, A. Sawamoto Enhancing School Quality in Vietnam through Participative and Collaborative Learning S. D. Parandekar, F. Yamauchi, A. B. Ragatz E. K. Sedmik, A. Sawamoto © 2017 International Bank for Reconstruction and Development / The World Bank 1818 H Street NW, Washington, DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved This work is a product of the staff of The World Bank with external contributions. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of The World Bank, its Board of Executive Directors, or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Nothing herein shall constitute or be considered to be a limitation upon or waiver of the privileges and immunities of The World Bank, all of which are specifically reserved. Rights and Permissions This work is available under the Creative Commons Attribution 3.0 IGO license (CC BY 3.0 IGO) http:// creativecommons.org/licenses/by/3.0/igo. Under the Creative Commons Attribution license, you are free to copy, distribute, transmit, and adapt this work, including for commercial purposes, under the following conditions: Attribution—Please cite the work as follows: Parandekar, Suhas D., Futoshi Yamauchi, Andrew B. Ragatz, Elisabeth K. Sedmik, Akiko Sawamoto. 2017. “Enhancing School Quality in Vietnam through Participative and Collaborative Learning—Vietnam Escuela Nueva Impact Evaluation Study.” World Bank, Washington, DC. License: Creative Commons Attribution CC BY 3.0 IGO Translations—If you create a translation of this work, please add the following disclaimer along with the attribution: This translation was not created by The World Bank and should not be considered an official World Bank translation. The World Bank shall not be liable for any content or error in this translation. Adaptations—If you create an adaptation of this work, please add the following disclaimer along with the attribution: This is an adaptation of an original work by The World Bank. Views and opinions expressed in the adaptation are the sole responsibility of the author or authors of the adaptation and are not endorsed by The World Bank. Third-party content—The World Bank does not necessarily own each component of the content contained within the work. The World Bank therefore does not warrant that the use of any third-party-owned individual component or part contained in the work will not infringe on the rights of those third parties. The risk of claims resulting from such infringement rests solely with you. If you wish to re-use a component of the work, it is your responsibility to determine whether permission is needed for that re-use and to obtain permission from the copyright owner. Examples of components can include, but are not limited to, tables, figures, or images. All queries on rights and licenses should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. Cover page: Used with the permission of the Vietnam Ministry of Education and Training, Hanoi/Vietnam (2017). Cover design by Shepherd Incorporated Contents Acknowledgments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . ix Abbreviations and Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xi Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xiii 1. Why Vietnam Escuela Nueva (VNEN)? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1 The Escuela Nueva Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.1 Origins in Colombia . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.1.2 Adaptation in Vietnam . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3 1.2 VNEN Theory of Change . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 1.2.1 Participative and Collaborative Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 1.2.2 Community Involvement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.2.3 Movement and Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2.4 Play and Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .12 1.2.5 Complex Adaptive Systems . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13 1.3 Overview of the VNEN Impact Evaluation (IE) Study Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.3.1 Randomized Selection of Intervention and Control Groups . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.3.2 Survey Rounds . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.3.3 Implementation Heterogeneity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 1.3.4 Use of Item Response Theory (Rasch Indices) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 1.3.5 Qualitative Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.4 Preview of Findings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 Endnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2. School Leadership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.1 Strength of Implementation of the Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.1.1 School Autonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 2.1.2 VNEN Implementation from Principal’s Perspective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.1.3 School Grants and Student Scholarships . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 2.1.4 Community Engagement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.1.5 Readiness for Self-Paced Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 2.2 Contextual Factors Influencing Implementation and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.2.1 Principal’s Values and Beliefs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 2.2.2 Principal’s Political Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 2.2.3 Principal’s Innovative Attitude . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.2.4 Principal’s Active Engagement on the Job . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 2.2.5 School Size, Infrastructure and Facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 2.3 Estimation of Program Impact . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.3.1 Traditional Mindset about Schooling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 2.3.2 Principal’s Self-Belief about Influence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Endnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 Enhancing School Quality in Vietnam through Participative and collaborative Learning iii Contents 3. Teacher Beliefs, Attitudes, and Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.1 Who Are the Teachers? A Profile . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.1.1 Demographics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.1.2 Job Satisfaction and Salary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 3.1.3 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 3.1.4 Time Use . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.2 Teachers’ Beliefs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.2.1 Beliefs Regarding Educational Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 3.2.2 Beliefs Regarding Pedagogy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 3.2.3 Innovativeness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.3 Impact on Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.3.1 Self-Efficacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 3.3.2 Influence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Endnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4. Pedagogy and Teaching Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.1 The Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.1.1 Classroom Space and Furniture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.1.2 Artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 4.1.3 Learning Guides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.2 Pedagogical Elements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.2.1 Teaching Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.2.2 Group Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 4.2.3 Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 4.2.4 Student Government . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 4.2.5 Parental Engagement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65 4.3 VNEN Implementation Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66 4.3.1 VNEN Implementation Index: Principal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67 4.3.2 VNEN Implementation Index: Parents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.3.3 VNEN Implementation Index: Students . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 4.3.4 VNEN Implementation Index: Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Endnote . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80 5. Parents, Home, and Community Situations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.1 Parent Demographics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.1.1 Family and Living Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.1.2 Education and Labor Market . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 5.2 Parent Beliefs, Values, and Personality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 5.2.1 Parental Beliefs Regarding Education . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 5.2.2 Values Children Learn at Home . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 5.2.3 Parental Personality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 5.3 Knowledge, Practices, and Preferences Regarding VNEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 5.3.1 Parents Understanding of VNEN Activities at School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 5.3.2 Parents Participation in VNEN Activities at School . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 5.3.3 Home Academic Environment for Student . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 5.3.4 Parent Opinion and Preference Regarding VNEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 iv www.worldbank.org 6. Cognitive and Non-Cognitive Student Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.1 Visual Representation of Test Score Trends . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.1.1 Overall Trends for Cognitive Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 6.1.2 Trends in Cognitive Scores by Priority I Provinces . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 6.2 Impact Evaluation on Test Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 6.3 Non-Cognitive Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.3.1 Estimation of Non-Cognitive Outcome . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 6.3.2 Impact Evaluation of Non-Cognitive Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Endnotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 7. Findings from the Qualitative Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .111 7.1 Overview of Qualitative Component . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111 7.2 Teaching and Learning Practices in the Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 7.2.1 Lesson Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 7.2.2 Instructional Practices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 7.2.3 Type and Extent of Classroom Interaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 7.3 VNEN Practices and 21st Century Skills in the Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113 7.4 The Deep-Dive: Exploration on What Takes Place for 21st Century Skills and Why It Takes Place . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 7.4.1 Leadership . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115 7.4.2 Teamwork and Cooperative Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117 7.4.3 Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 118 7.4.4 Self-Managed Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119 7.5 Extent and Nature of Use of the Learning Guide . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120 7.6 Stakeholders’ Views on Benefits and Challenges of VNEN Implementation . . . . . . . . . . . . . . . . . . . . 122 7.6.1 Development of Socio-Emotional Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122 7.6.2 Increased Parental Involvement in School and Student Learning . . . . . . . . . . . . . . . . . . . . . . . 123 7.6.3 Improved Relationships among Stakeholders . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123 7.7 Challenges of VNEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 7.7.1 Challenges Attributed to the Mindset of Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 7.7.2 Challenges Resulting from Ethnic Minority Students’ Proficiency in the Vietnamese Language . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124 7.7.3 Challenges Associated with Parents . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125 7.8 Further Exploration of Patterns in Classroom Practices and Interactions . . . . . . . . . . . . . . . . . . . . . . 125 8. Policy Conclusions and Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 8.1 Positive Impact of VNEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 8.2 Theory of Change of VNEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129 8.3 Leadership of VNEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130 8.4 Training of Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 131 8.5 Deepening Participation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 8.6 Communications Program . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 132 8.7 Monitoring Socio-Emotional Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 133 8.8 Directions for Further Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 134 Enhancing School Quality in Vietnam through Participative and collaborative Learning v Contents Bibliography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 135 Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141 Picture Credits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 List of Figures Figure ES.1 Vietnam Escuela Nueva Impact Evaluation: 2013–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . xx Figure 1.1 Replicated from the OECD Publication—Causality and Correlation of Cognitive and Non-Cognitive Skills in Korea . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 Figure 1.2 Theoretical Model of Peer Assisted Learning from Topping, 2005 . . . . . . . . . . . . . . . . . . . . . . . . . 9 Figure 2.1 Decentralization of Decision Making, 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Figure 2.2 School Autonomy Index, 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Figure 2.3 Principal Familiarity with VNEN Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Figure 2.4 Principal View of VNEN Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 Figure 2.5 Principal Report of Parent Participation, 2015 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 Figure 2.6 Community Engagement Measure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Figure 2.7 Readiness for VNEN (per grade level) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 Figure 2.8 Principal Values and Beliefs (average 2013–2015) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 Figure 2.9 Principal Political Skills . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 Figure 2.10 Innovation and Correlations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 Figure 2.11 Principal as Busy as a Bee—Every Semester, 2015 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Figure 2.12 School Wealth Index, 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 Figure 2.13 Principal Attitude about Traditional Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 Figure 2.14 Principal Self-Perception of Influence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Figure 3.1 VNEN Teachers with Higher Satisfaction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Figure 3.2 Attitude toward Traditional Model and Familiarity with VNEN . . . . . . . . . . . . . . . . . . . . . . . . . . 45 Figure 3.3 Educational Objectives and Implementation Compared . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Figure 3.4 Beliefs Regarding Importance of Pedagogical Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 Figure 3.5 Hypothetical Block-Diagonal Pattern Expected for Heat Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . .49 Figure 3.6 Challenges Regarding Pedagogical Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 Figure 3.7 Innovativeness of Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51 Figure 3.8 Self-Efficacy of Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Figure 3.9 Teachers Self-Perception of Influence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Figure 4.1 Classroom Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 Figure 4.2 Artifacts in VNEN Classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Figure 4.3 Measure of Existence of Artifacts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 Figure 4.4 Gender and Ethnic Minority Equality in the VNEN Classroom, 2014 . . . . . . . . . . . . . . . . . . . . . . . 65 Figure 4.5 VNEN Implementation Index: Principal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Figure 4.6 VNEN Implementation Index: Parents (Part I) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70 Figure 4.7 VNEN Implementation Index: Parents (Part II) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .71 Figure 4.8 VNEN Implementation Index: Students (Part I) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Figure 4.9 VNEN Implementation Index: Students (Part II) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Figure 4.10 VNEN Implementation Index: Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Figure 4.11 VNEN Implementation Index: Formative Assessment Intervention . . . . . . . . . . . . . . . . . . . . . . . . 76 Figure 4.12 VNEN Implementation Combined Index: By Year and Overall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 77 Figure 4.13 VNEN Implementation Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78 Figure 5.1 Family Demographic Situation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82 Figure 5.2 Parents Education and Occupation Situation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 83 Figure 5.3 Parents Opinion about Maintaining Traditional Model, 2016 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84 Figure 5.4 Parent Beliefs Regarding Education and Values . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86 Figure 5.5 Qualities Learned at Home . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87 Figure 5.6 Parental Big Five Personality Traits . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88 vi www.worldbank.org Figure 5.7 Parent Participation and Opinion about Usefulness . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90 Figure 5.8 Opinion about VNEN among Parents Who Know about VNEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Figure 5.9 Reason for Positive Opinion of Parents about VNEN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92 Figure 6.1 Vietnamese Test Scores for 15 Randomly Selected Schools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95 Figure 6.2 Same Schools with VNEN Information Added . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96 Figure 6.3 District FSQL (Fundamental School Quality Level) Audit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 97 Figure 6.4 National Trends in Cognitive Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98 Figure 6.5 Trends in Provinces (at least 20 schools in sample) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99 Figure 6.6 Matched Scores . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Figure 6.7 Construction of Non-Cognitive Outcome Score . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 104 Figure 6.8 Non-Cognitive Score Overall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 105 Figure 7.1 Map of Analytical Components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112 Figure 7.2 Time Spent per Lesson and Level of Advancement for VNEN 21st Century Skills . . . . . . . . . 114 Figure 7.3 Percent of Time Not Following the Learning Guide and Its Effectiveness and Type . . . . . . . 120 Figure 7.4 Visual Example of Multiple Layers Coded over the Course of the Lesson . . . . . . . . . . . . . . . . 126 List of Tables Table 1.1 Students Report of Group Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 Table 2.1 Sufficiency of VNEN School Grants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 Table 2.2 Scholarships Compared to Enrollment, 2015 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 Table 2.3 Distribution by Gender and Ethnic Minority, 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 Table 2.4 School Facilities Compared . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 Table 2.5 Impact of VNEN on Principal Self-Perception of Influence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 Table 3.1 Profile of Surveyed Teachers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 Table 3.2 Satisfaction of Teachers, 2015 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40 Table 3.3 Excellent Teacher Awards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41 Table 3.4 Teacher Training Received, 2012 to 2015 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42 Table 3.5 Teacher Interaction and Cluster Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 Table 3.6 Teacher Time Use, 2012 to 2015 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Table 3.7 Elements of Teacher Self-Efficacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52 Table 3.8 Teachers as Primary Influencers in Decision Making, 2013 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 Table 4.1 Opinions and Use of Learning Guides . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 Table 4.2 VNEN Methods among Top 3 Teaching Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 Table 4.3 Opinions and Use of Group Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62 Table 4.4 Teacher Use of Assessment Methods in Mathematics Lessons . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63 Table 4.5 Teacher Engagements with Parents (mean number of occasions per year) . . . . . . . . . . . . . . . . 66 Table 4.6 Principal Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68 Table 4.7 Parents (Part I) Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69 Table 4.8 Parents (Part II) Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71 Table 4.9 Student (Part I) Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72 Table 4.10 Student (Part II) Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73 Table 4.11 Teachers Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74 Table 5.1 Parent Reports of Classroom Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89 Table 5.2 Home Study Environment and Activities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91 Table 6.1 Variables Used for Propensity Score Matching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 100 Table 6.2 Impact on Learning Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 102 Table 6.3 Estimating Impact on Non-Cognitive Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 106 Table 6.4 Additional Controls for Test Score Regressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 107 Enhancing School Quality in Vietnam through Participative and collaborative Learning vii  Acknowledgments This report is based on the work done by a World The team thanks the Global Partnership of Bank (WB) team with the support, encourage- Education and Dubai Cares for their financial ment and participation of Vietnam’s Ministry of contributions to the project and Impact Evalu- Education and Training, the Dubai Cares Founda- ation Study and associated training and capac- tion, the Global Partnership for Education (GPE), ity building activities. The team would also like and the Mekong Development Research Insti- to thank the United Nations Educational, Scien- tute. The WB team functioned under the leader- tific, and Cultural Organization (UNESCO), as the ship and guidance of Ousmane Dione, Country GPE coordinating agency in Vietnam. Finally, the Director; Amit Dar, (Acting) Senior Director of the team would like to thank all students, teachers, Education Global Practice; Harry Patrinos, Prac- principals, parents, school management teams, tice Manager; and Michel Welmond, Program community members, and field investigators Leader. The team benefited from leadership of and videographers, that, through their participa- the following individuals during preparation tion, made the Impact Evaluation Study possible. and implementation, based on the positions they held at the time: Victoria Kwakwa, Country The WB task team was led by Suhas D. Parandekar, Director; Luis Benveniste, Sector Manager; and Senior Economist, and included Adam Gann, Christian Bodewig, Country Sector Coordinator. Consultant; Akiko Sawamoto, Consultant; Andrew B. Ragatz, Senior Education Specialist; The Ministry of Education and Training (MOET) Anh Lan Vu, Education Specialist; Elisabeth  K. team was led by Dr. Nguyen Vinh Hien, Vice Sedmik, Consultant; Evelyn Wang, Consultant; Minister at the time. The MOET team included Futoshi Yamauchi, Senior Economist; Hai-Anh Dr. Pham Ngoc Dinh, then Director of the Pri- Dang, Economist; Shang Gao, Consultant; and mary Education Department; Dr. Dang Tu An, Takiko Igarashi, Education Specialist. Senior Specialist and Advisor of the GPE-VNEN project; and Mr. Le Tien Thanh, Senior Special- The findings, interpretations, and conclusions ist and Advisor of the GPE-VNEN project. The expressed in this and all other volumes of this implementation of the impact evaluation survey report are entirely those of the authors. They do and test preparation was carried out under the not necessarily represent the views of the Interna- responsibility of Dr. Hai Pham Thi Thanh, M&E tional Bank for Reconstruction and D ­ evelopment/ Specialist of the GPE-VNEN project. World Bank and its affiliated organizations, or those of the Executive Directors of the World Bank The Mekong Development Research Institute or the governments they represent. (MDRI) functioned under the leadership of Phung Duc Tung, Director of MDRI. The team Peer reviewers for the World Bank were Erik included Nghiem Hoang Son, Researcher; Ho Bloom, Senior Economist, and Gabriel Demom- Van Bao, Researcher; Le Hai Chau, Researcher; bynes, Program Leader. The team obtained very Pham Linh Chi, Researcher; Nguyen Thanh Long, useful suggestions and feedback from Michel Researcher; and Nguyen Hong Thuy, Researcher. Welmond, Program Leader; Michael Crawford, The team would also like to thank Fernando Lead Education Specialist; as well from exter- Cartwright, CEO Polymetrika Canada for his con- nal reviewers: Eduardo Velez Bustillo, Pham Vu tributions to the study. Thanh Giang, and Pablo Jaramillo. Enhancing School Quality in Vietnam through Participative and collaborative Learning ix Abbreviations and Acronyms AI artificial intelligence ATT average treatment effect on the treated (model) BFI Big Five Inventory CAS complex adaptive systems CHC Confucian Heritage Culture EFA Education for All EFA-FTI Education for All-Fast Track Initiative EN Escuela Nueva EPDF Education Program Development Fund GPE Global Partnership for Education IE Impact Evaluation IPA Importance-performance analysis IRT Item Response Theory LG Learning Guide LTP Long-term Potentiation MDRI MeKong Development Research Institute MIT Massachusetts Institute of Technology MOET Ministry of Education and Training OECD Organisation for Economic Co-operation and Development PEDC Primary Education for Disadvantaged Children PISA Programme for International Student Assessment PPP Purchasing power parity PSI Political skills inventory PSM Propensity score matching RISE Research on Improving Systems of Education SL Self-managed learning T&CL Teamwork and cooperative learning t.e.t time-engaged-with-task t.o.t time-on-task UNESCO United Nations Education, Scientific, and Cultural Organization VND Vietnam dong VNEN Vietnam Escuela Nueva WB World Bank WMLE weighted mean likelihood estimate WVS World Values Survey ZPD zone of proximate development Enhancing School Quality in Vietnam through Participative and collaborative Learning xi Executive Summary Findings Highlights The Vietnam Escuela Nueva (VNEN) program has had a positive impact on cognitive and non- cognitive achievement of children in Vietnam, which should be recognized and made known to all stakeholders. This study has reported the findings of positive impact using a rigorous impact evaluation procedure that followed a cohort of students from Grade 3 through to Grade 5. Almost any educational program tracked over a period of time will show growth in the cognitive and non- cognitive abilities of children. This study compares the growth of the children in the VNEN program with a counterfactual group to see the difference in growth. A rough calculation of the cost of the GPE-VNEN program is also instructive. The program cost approximately US$85 million, and benefited about a half million children over four years, not counting the children to be benefited in the future. This works out roughly to about US$40 per student per year. From a recent public expenditure review, we know that the average per student expenditure for primary education in Vietnam is about US$1,000 per year in purchasing power parity (PPP) terms (approximately Vietnam dong (VND) 8 million), which indicates a reasonable benefit-cost ratio. Effective participative and collaborative learning is a complex undertaking with heavy cultural influences. A sustained awareness building and motivational campaign is required that targets both communities of the program schools and wider audiences. This campaign needs to clarify the origin and purpose of the reform, explain the expected benefits, and lay out the required behavior from all stakeholders for successful implementation. Further high quality research at all levels needs to be encouraged to learn about the approaches that are best suited to Vietnamese conditions. Vietnam’s education system is widely regarded internationally as a ‘success story’. Continued engagement with the international education research community will bring mutual benefits for Vietnam as well as other countries who wish to learn from Vietnam. Introduction strong socio-emotional skills are also more likely to persist with effort in the face of hardship and Together with structural transformations in achieve high test scores. Vietnam’s Fundamental the economy, the single biggest determinant and Comprehensive Education Reform adopted of future living standards in Vietnam will be an by the government in 2013 provides the policy education system that enables Vietnamese cit- basis for a major transformation of the education izens to be competitive in the globalized econ- system so that it can provide the elements neces- omy. To this end, learners must not only master sary for learners to obtain these 21st century skills traditional knowledge and facts, but also develop and competencies. Other innovations include a competencies for independent and innovative competency based curriculum, rigorous learn- thinking which belong to the set of non-cognitive ing assessment, school-based teacher support, or socio-emotional skills. Research has also shown and universal full day schooling at the general that non-cognitive and cognitive skills are caus- education level. ally linked and related—children who develop Enhancing School Quality in Vietnam through Participative and collaborative Learning xiii Executive Summary As part of its overall education reform strat- focus on developing learner competencies and egy, Vietnam introduced a “whole school” character. The ideals and goals enshrined in these reform to teaching and learning as part of the directive policy documents have to be translated Vietnam Escuela Nueva (VNEN) program. The to daily classroom activities. VNEN incorporates and integrates several innova- tive and globally recognized practices including: Vietnam Escuela Nueva is a Vietnamese pro- (a) participative and collaborative learning; (b) gram that melds Vietnamese capabilities with self-paced learning guides; (c) student govern- rich and varied international knowledge and ment; (d) formative assessment combined with experience of educational reform under the summative assessment; (e) application or real-life Escuela Nueva or New School model. Escuela oriented learning, with community integration; Nueva or New School is an award winning interna- and (f ) teacher professional networks. The com- tionally recognized program with elements that bination of these elements is intended to spur a have origins in a number of programs around the transformative and powerful learning experience. world such as Waldorf and Montessori schools. This approach to learning is expected to lead to The ideas underlying the model have origins the new skills and competencies expected by in most important pedagogical-cultural move- the reform effort. The GPE-VNEN project and the ments in Europe and the Americas in the early larger VNEN program sought the implementation decades of the twentieth century. VNEN includes of this “whole school” approach in 1,447 schools the application of a set of elements developed for from 2012–2016. the Vietnamese context that has an intellectual inspiration from the Colombian program. Escuela The motivation for establishment of the pro- Nueva provides some philosophical bases—for gram comes from decisions at the highest instance the idea that human beings learn best level in Vietnam to undertake a program of as part of a community; and Escuela Nueva pro- Fundamental and Comprehensive Reform of vides the practical experience of decades with the education sector. Resolution 29 of Novem- self-learning taught using learning guides. In Viet- ber 4, 2013, of the 8th plenum of the 11th Party nam the philosophy has to be adapted to meld Congress and the related Resolution 44 of June 9, with the thinking of Vietnamese philosophers 2014, from the Prime Minister’s office, provide the such as Nguyen Trai (1380–1442) and Chu Van policy setting for the Fundamental and Compre- An (1292–1370), who stressed the importance of hensive Reform that motivated the establishment living a virtuous life. On a practical plane, learn- of the VNEN program. Resolution 29 notes the ing guides for Vietnamese children needed to be achievements of the education system and points developed for Vietnam by Vietnamese textbook out the challenge of developing well-rounded experts. citizens for the modern world. Educational litera- ture uses the term ‘competency’ to mean not only The Vietnam Escuela Nueva Impact Evaluation the acquisition of a skill, but the real-life appli- aims to produce evidence-based input for pol- cation of the skill, typically in a social context. icy decision making by using state-of-the-art Resolution 29 exhorts Vietnamese educators to research methodologies to examine the results develop creativity in children and the ability for of this pilot. The basic research questions that self-learning. General education should provide motivated the study were: (a) how useful was children with ethical as well as practical learning the new approach for students, and (b) how well to apply knowledge. Teachers are encouraged were teachers able to incorporate the new peda- to avoid the imposition of knowledge, passive gogy into their practice. The impact evaluation learning, and rigid memorization. Resolution 44 measured the extent to which students acquired describes the commitment of the government to cognitive and non-cognitive skills by comparing xiv www.worldbank.org results from a randomly chosen set of VNEN and cognizant about the problems with the status traditional schools from a nationally representa- quo. This is also a common sense view—you will tive sample. Considering that the pedagogical only make an effort to change if you think there is changes introduced by VNEN constitute a para­ need for change. In this study we asked principals digmatic shift for students, parents, teachers, about their opinion regarding the statement that and education managers—by no means a sim- “the traditional model is working fine, there is no ple task—the study also aimed to better under- need for change.” About 75% of VNEN principals stand the challenges to implement this approach disagreed with the statement throughout the successfully. three years of the survey. The interesting finding is that in 2013, 73% of control group principals were The study introduces important methodologi- of the same opinion, but this group has declined cal innovations that set an important prece- to 63% in the next two years. dent for future education research in Vietnam. The study used impact evaluation methods to Principal’s self-belief about their influence was compare the results and outcomes in VNEN and also positively impacted by participation in traditional schools. Because the VNEN project had the program, indicating a pathway for success not selected participating schools through ran- in reform. When dealing with a paradigmatic domization, the study used a propensity score change in the educational model, even if you matching exercise to determine a matched panel believe in the need for reform and have a favor- of 650 control and treatment schools. Question- able opinion about the new school model, as a naires were administered to principals, teachers, school principal you may doubt your ability as parents, and students. Standardized tests were a school leader to bring about positive change. administered to a sample of randomly selected We tracked the self-perception of the principals students from within each selected school in about their influence regarding a set of decisions Grade 3 in 2013. The cohort of students were that affect education delivery at the school. These tested again in Grade 4 in 2014 and Grade 5 in decisions included issues such as ‘adapting cur- 2015. This method allowed the researchers to riculum to local conditions’ and ‘evaluating teach- compare growth across two years. Finally, using ers’ performance’. When we plotted the principal digital video to film teaching and learning prac- influence index for 2015 as compared to the prin- tices from a sample of 15 schools, educational cipal influence index for 2014, we found an inter- researchers gained important insights of class- esting pattern. The value of the index grew over room activity in both VNEN and traditional time, and it grew twice as much for VNEN school schools. These methods hold great promise for principals as compared to traditional school understanding the effects of the Fundamental principals. VNEN requires principals to exercise and Comprehensive Reform that other research- some measure of autonomy and have a leader- ers plan to extend in the future. ship role in integrating the school activities with the local community. The finding shows again the importance of practical experience of principals Key Findings in implementing the model. Principals, the frontline leaders of the reform, Turning from belief to understanding, teachers have a strong belief about the need to reform appear to have understood the model concep- the traditional model, and this belief is posi- tually quite well, but may be facing challenges tively related with their actual program expe- in practical implementation. We consider four rience. The literature on reform and change statements about teacher activities to character- management points out that in order for change ize the traditional model: give effective lectures, to be possible, the stakeholders have to be Enhancing School Quality in Vietnam through Participative and collaborative Learning xv Executive Summary maintain discipline and order, strive that stu- using the learning guide with a certain level of dents do not make mistakes, and repeat expla- flexibility as they see fit by making modifications nations until the student understands. Similarly to and/or deviations from activities, yet still meet- four statements characterize the VNEN model: ing quality standards and teaching all required stress student enquiry, students helping other content. Some of the challenges faced by VNEN students, students engaged in self-learning, and include the persistence of a traditional mindset organizing fun activities for students. We asked among some teachers, the limited Vietnamese teachers to rank the eight activities according to language proficiency of ethnic minority students, importance—the four activities that form the tra- and the inability of some parents to provide aca- ditional model received low ranks—for example, demic support for their children. lowest rank, by 42% of the teachers was for giv- ing effective lectures, while 57% gave first rank to Program classrooms counted on the presence student self-learning. In contrast to the ranking of physical artifacts from real life to bring edu- on importance, teacher responses regarding how cation closer to real life, but the use of this ele- challenging they found it to implement the activ- ment may have been below the potential. The ity indicated that encouraging student enquiry data indicate that there was widespread adoption is a highly ranked challenge—27% of teachers of the practice of bringing in some real-life arti- mentioned it as first rank. Students not making facts such as models of farming implements or mistakes was mentioned by 25% of teachers as local handicrafts in the classroom to help children a first ranked challenge. Hopefully, teachers are relate education to real life. However, it appears not trying too hard in that area, because mistakes that full utilization was not made of this element are very useful when students learn from them. of VNEN in all schools. While there is a great deal of energy and excitement when materials are The video analysis shows how VNEN students first brought into the classroom, after a while are exposed to multiple pathways of learning. they tend to lose their novelty value as well as They have a more prominent pedagogical use of their didactic utility. Not all teachers have utilized exploration and discussion as well as opportu- a rotation policy regarding artifacts, and there nities to practice and exercise problem solving is not always a new set of artifacts to help stu- through both individual and group work. While dents engage in conversation or study related they tend to spend a similar amount of time on to exploration around the artifacts. For instance, individual work as non-VNEN students, a key dif- the teacher could direct students to be obser- ference appears to be that VNEN classrooms spend vant about the kind of hoes used on the farm and a lot of time in group activity rather than whole to compare them with the prototype they have class work. To the extent that group work allows in the classroom. In addition to having different for greater cognitive engagement as compared to objects, different activities can be organized by whole class work, the VNEN classroom may reflect the teacher that relate to the object she has in the greater efficiency in teaching. Despite some varia- classroom, like the community map. tion across schools and within classrooms, VNEN schools provide much more space for students Student government is another program inno- to develop and practice 21st century skills, such vation that likely is instrumental in providing as leadership, teamwork and cooperative learn- students with socio-emotional skills as well as ing, communication, and self-managed learning. enriching values for 21st century skills. One Some of the interactions involving 21st century idea under VNEN is that socio-emotional skills skills are still at a basic level, but the majority tend including collaboration and responsibility as to be at intermediate and advanced levels. With well as character strength building can best be training support from MOET, VNEN teachers are done through practical experience. Every VNEN xvi www.worldbank.org class has a student government and teachers researchers regarding the measurement of socio- are encouraged to rotate roles so that many stu- ­ emotional skills. Indeed, the opportunity to dents can get a chance to serve. By organizing develop a program in Vietnam in view of the committees for different tasks, students develop potential related to implementation of VNEN is organizational skills, which is another way to one of the recommendations of this study. We make education relevant to real life. It appears have utilized a method that appears to be a rea- that some of the instances of student govern- sonably sound way for the purpose at hand, based ment were implemented quite well, while others on parent feedback regarding the behavioral utilized this method in a limited way. In order to development of their children over the three-year be effective, there needs to be clear definition development of the VNEN program. The results of the roles and responsibilities of all members, show that students in the VNEN program did bet- and feedback from the teacher and the student ter than the students in traditional schools. VNEN regarding performance and improvement. Even students did particularly better at the lower end though only a few teachers may have imple- of the distribution. This is an important finding mented the full extent of student government, it because these students tend to come from dis- is likely that this VNEN element has been effective advantaged groups and are often a special focus in helping children acquire socio-emotional skills. of educational policy makers. Close parent participation in school and Test scores for Vietnamese and mathemat- classroom activities is a program activity ics show that program students perform as that appears to have been only sporadically well or better compared to traditional school adopted though parents may be quite in favor students. The scores of students from VNEN of it. One of the aspects of VNEN which may as well as control group schools moved up as have proved difficult to implement is the close they moved through the grades. However, the involvement of parents in school activities. Given students from VNEN schools already showed a the semi-urban and rural location of most VNEN higher mean score at the baseline of the study, schools, the overwhelming majority of parents are which was 18 months after the initiation of the farmers and few have education beyond second- project. This difference is retained throughout ary school. However, the data show that parents the years, though it narrows down over the last were quite well informed about the details of the two years. Regression analysis shows a statistically VNEN model and what goes on in VNEN schools. significant effect of the VNEN program, with some Parents who knew about VNEN were in support of specifications showing effect sizes of about one- the program. When we asked parents about the fifth of a standard deviation, about 15 points in usefulness of parent participation in classroom Vietnamese and 18 points in mathematics. These activities, we found both groups of parents, VNEN findings involved a number of technical issues parents and the control group parents, expressed related to measurement and statistical inference a favorable opinion. However, actual participation and will be presented for discussion and review of parents was sporadic—participation for VNEN in the academic literature. parents was quite similar as participation from control group parents. Variation in the level of performance of stu- dents within the program group of schools is The data indicate that the program has had a related to variation in the intensity of the pro- positive impact on the socio-emotional skills gram. The study documents a considerable level of children enrolled in program supported of variation or heterogeneity in the implementa- schools. However, there is not yet an agreed tion of VNEN practices. Merely seating children in and established method among educational groups and providing them with learning guides Enhancing School Quality in Vietnam through Participative and collaborative Learning xvii Executive Summary does not ensure that adequate participative and century. The ‘new school’ was conceived as an collaborative learning will take place. While some alternative relevant to the modern world, where principals and teachers were able to take the education follows a natural process more attuned opportunities provided to them by the model to the psychological development of children and very far, there were others who appear to have cognizant of the social nature of human beings. lacked the knowledge or the inspiration. It would “Modern schools” or “progressive schools” were not be advisable to attempt to mandate or direct also terms associated with the movement that all schools to rigidly adhere to specific methods. included some of the foremost educational think- Rather, there is scope for feedback loops to be ers of the time—Jean-Olvide Decroly (1871–1932) generated, where those who are implementing from Belgium; John Dewey (1859–1952) in the the program well can influence others. United States; Adolphe Ferrière (1879–1960) and Jean Piaget (1896–1980) from Switzerland; María Montessori (1870–1952) from Italy; and the bril- Policy Implications liant but short lived Lev Vygotsky(1896–1934) from Russia. Vygotsky’s idea of the ‘zone of prox- The VNEN program has had a positive impact imal development’ created during social inter- on cognitive and non-cognitive achievement actions is key to the VNEN practice of self-paced of children in Vietnam, which should be rec- learning. Participative and collaborative learning ognized and made known to all stakeholders. is expected to work because in a fundamental This study has reported the findings of positive way, all human learning, some would say all learn- impact using a rigorous impact evaluation pro- ing, can be understood as consisting of a com- cedure that followed a cohort of students from bined process of exploration and engagement Grade 3 through to Grade 5. Almost any educa- (Pentland, 2014). tional program tracked over a period of time will show growth in the cognitive and non-­ cognitive The championship of leaders at every level, abilities of children. This study compares the from the national, through the provincial, dis- growth of the children in the VNEN program trict and school levels, appears to be a neces- with a counterfactual group to see the differ- sary condition for success. We have seen that ence in growth. A rough calculation of the cost in certain provinces, there appears to be a higher of the GPE-VNEN program is also instructive. The incidence of complete implementation of VNEN. program cost approximately US$85 million, and We found that schools with an advanced level of benefited about a half million children over four implementation tended to have principals with years, not counting the children to be benefited in a deep understanding of the VNEN model, who the future. This works out roughly to about US$40 encouraged teachers in the school to follow it. per student per year. The average per student Activities such as periodic meetings among teach- expenditure for primary education in Vietnam is ers in the school to discuss and practice VNEN about US$1,000 per year in PPP terms (approxi- approaches seem to contribute to its adoption. mately VND 8 million), which indicates a reason- Ensuring a high level of implementation by influ- able benefit-cost ratio. encing leaders at all levels is a challenging prob- lem. The best chance for success would come from All stakeholders should be encouraged to feedback loops generated through sharing of develop an accurate idea of the educational successful implementation. Events such as video thinking that underlines the reform. The competitions, which had been organized at some ideas underlying the Escuela Nueva or New point during implementation, can be conducted School model have origins in most important with dedicated attention and would be instru- ­ pedagogical-cultural movements in Europe and mental in reducing the width of the distribution the Americas in the early decades of the twentieth xviii www.worldbank.org of the implementation index towards the higher The close involvement of parents represents end through a process of imitation and experi- a cultural shift for many stakeholders, and ence of rewards. greater effort from national and provincial leaders is needed to make this happen at scale. The complexity and difficulty of a paradig- Some principals and teachers may believe that matic change in the teaching approach calls the predominantly large community of farmers for a constant process of experimentation and that represents the parent body may not be ame- rigorous accompanying research to learn the nable to close involvement in the school. Others best approaches. It is hoped that this study has may have tried and failed to engage the parents provided insights to the education research com- and may have persuaded themselves of the futil- munity and policy makers of the usefulness of ity of trying again. School leaders who were more two features of research—the importance of hav- persistent likely met with success. As is the case ing comparable intervention and control groups with getting teachers to implement all aspects of for valid inference to be possible, and the use of the model, the generation of multiple feedback analysis of digital video through the development loops is probably the best way to bring about and application of coding trees as has been done wider adoption. in this study. There are a number of issues with regard to participative and collaborative learn- From a policy perspective, it is useful to ing which need to be investigated closely—for understand how the feedback loops may be instance, how often should group formation be ­ engendered—a case in point is the use of changed; how much independence should the artifacts in the classroom. Typically, these have teacher utilize in having students apply the learn- been one-off activities done with varying levels of ing guide; how can parents be assured that stu- enthusiasm. In our study we tried to capture this dents are learning if the teacher does not provide diversity by asking not only about the presence any marks in their notebooks and only provides of artifacts but also about their use. We found that descriptive and advisory commentary; and many the use of artifacts was made very seldom, and other questions like these. the reason may be that teachers may not have been encouraged or provided ideas regarding The new method of training where commit- how to use artifacts. For instance, once a com- ted, high performing teachers were co-opted munity map is made, it is a static document that early to become trainers themselves should may not be referred to again. But fun activities be made part of regular practice. This practice using the map could be organized that would should be considered for retention going ahead, serve a didactic purpose for both cognitive and and teachers should be encouraged to take on non-cognitive skills. an investigative mindset with regard to their own practices. In this way, the training was not A video competition to share practices could something developed by trainers removed from provide a mechanism to generate awareness practical realities and provided to teachers in a about the true nature of the program, given mimic of the traditional frontal model of teach- the success of the small-scale video compe- ing. Rather, training was conducted in the form tition. A key phenomenon that was observed of a VNEN classroom, complete with groups and throughout the program was the clustering of group leaders and warm-up games. In the case of good practices because of the exchange and VNEN teachers, the practical acts of implementing replication of ideas. Video forums are a form of what they learned in the training reinforced and multipartite communication. Other forms of feed- strengthened their beliefs. back from implementers on the ground to the policy makers is quite important. Such feedback Enhancing School Quality in Vietnam through Participative and collaborative Learning xix Executive Summary happened to a large extent during the program, insights if the data are made available to as senior Ministry of Education and Training lead- researchers. While the impact evaluation results ers crisscrossed the country visiting schools and presented here can be further deepened and incorporating adjustments in the program based developed from this data, the team from Research on feedback. Greater awareness about this phe- on Improving Systems of Education (RISE) will col- nomenon, which also took place at the provincial lect further data from the same cohort of children level, would go a long way toward generating as they go through secondary school. The primary ownership among all stakeholders. data collected under the current study itself pro- vide a veritable data mine of research on various The qualitative component also has many topics including teacher efficacy, principal lead- potential avenues for further research with ership, and other issues related to educational more time to fully analyze the video data psychology and student achievement. It is hoped than has been possible so far. With 810 videos that the data will be made available by the Viet- from 270 lessons (with each lesson containing a namese government to researchers in the future video of the teacher and of two student groups), to carry out further investigation. there are many opportunities to delve deeper into specific aspects of what takes place in the Vietnam’s education system is widely classroom. The videos also provide ample oppor- regarded internationally as a ‘success story’. tunity to create vignettes for use as examples in Continued engagement with the interna- training and professional development. Many tional education research community will examples of advanced VNEN practices have bring mutual benefit for Vietnam as well as already been identified in this round of analy- other countries who wish to learn from Viet- sis, but this could be extended with the creation nam. This research study has provided such of a database of examples. The vignettes could benefits, with the promise of more to follow. Two also include a set of more specific tags of what key points of this research study are indicated the videos represent. For example, clips identi- in Figure ES.1. First, in order to evaluate a pro- fied for high levels of leadership could be further gram, it is important to compare performance catalogued as showing where the leader effec- growth for the program schools as well as a con- tively leads a discussion involving higher order trol group of schools; merely evaluating ‘before’ thinking, where the leader ensures all members and ‘after’ performance is not enough for attri- are actively participating, or many other ways in bution of changes to the program. Second, with which leaders demonstrate unique skills. methodological considerations to be detailed in forthcoming academic papers, the study found The quantitative data collection over a period positive impact, especially for non-cognitive of nearly three years has generated a huge skills. amount of data that can generate many further xx www.worldbank.org Figure ES.1  Vietnam Escuela Nueva Impact Evaluation: 2013–2017 Vietnam Escuela Nueva Impact Evaluation Average effect size First large-scale impact evaluation study of Vietnamese 0.16 Escuela Nueva shows a strong relationship between implementation Mathematics 0.18 and cognitive and non- cognitive achievement Non-cognitive skills 0.41 An effect size of 0.50 is equivalent to 1 year of instruction Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning xxi Why Vietnam Escuela Nueva (VNEN)? 1 1.1  The Escuela Nueva Model sauce’ of Escuela Nueva? Though the purpose of this book is to investigate the impact of imple- 1.1.1  Origins in Colombia mentation of Escuela Nueva in Vietnam, the fol- Escuela Nueva began to be implemented in lowing quotation provides an insightful context Colombia in the 1970s as a program that trans- setting about the original Colombian program: formed rural education in that South American country. From being behind their urban counter- “Imagine a primary school where children work parts, rural children were provided an education together to learn things that are relevant to that catered to their particular context. Escuela their lives, where teachers are facilitators who Nueva included a number of elements that were foster comprehension and democratic behav- highly innovative at that time, and can be consid- ior, and where parents and communities are ered to be modern and forward-looking to this actively involved in their children’s education. day. These included self-paced learning guides Now imagine that this is not an exclusive pri- instead of textbooks and a flexible school calen- vate school open to a wealthy few, but a state dar adapted to imperatives of rural life such as school in a developing country attended by the intense labor of the harvesting season. children who are among the poorest in the world. This is not a utopian dream but reality Rather than retrofitting an education model for over 20,000 schools in Colombia attended designed for cities in industrial societies, Escuela by 5 million children today. They are ‘Escuela Nueva sought to build on strengths of the rural Nueva’ schools.”1 contexts. This was inclusive education before inclusion became fashionable, by doing simple things such as bringing in farmer parents to 1.1.2  Adaptation in Vietnam classrooms to help relate education to liveli- A conference was organized by the World Bank hoods. It started as a grassroots initiative that in Cebu City, Philippines, from February 9–12, became a national and then an international 2009, with financial support from the Education program (Kline, 2002). While the initial focus of Program Development Fund (EPDF) of the Educa- Escuela Nueva was on multigrade schools, as the tion for All-Fast Track Initiative (EFA-FTI), the pre- program began to be implemented in diverse cursor organization of the Global Partnership for contexts in other countries, other elements of the Education (GPE). The title of this conference was program, such as its basis on participative and “Multiple Approaches of Education for Rural Areas: collaborative learning, became more important. Lessons from Good Practices” and it brought Juárez, Associates, and Inc., 2003 provide a use- together experts and practitioners from around ful review of the application of Escuela Nueva in the world including Vicky Colbert and Pablo Jara- Guatemala, Nicaragua, and the Philippines. millo from Colombia, who made presentations regarding Escuela Nueva.2 Also attending the con- Why did Escuela Nueva become such a popu- ference were policy makers from the Vietnamese lar and long living program? What is the ‘secret Ministry of Education and Training (MOET). 3 Why Vietnam Escuela Nueva (VNEN)? The Vietnamese delegation included the then strategic terms in Resolution 29 of the Central Deputy Minister in charge of Primary Education, Steering Committee of the Eighth Congress of Mr. Nguyen Vinh Hien and Mr. Dang Tu An, then the Communist Party of Vietnam. Resolution 29 Project Director of the Primary Education for emphasizes the quality of education with a Disadvantaged Children (PEDC), a multi-donor focus on developing competencies and charac- financed MOET project. The Vietnamese delega- ter in children as a means toward helping Viet- tion was quite impressed by the presentations nam become a prosperous postindustrial nation and discussions with the Colombian experts and within one generation. The reform encompasses a follow-up visit was organized for a MOET team a profound revision of the curriculum, an adapta- to visit rural Colombia on a study tour to learn tion of pedagogical methodology, and attendant more details about Escuela Nueva. Even more teacher professional development to provide impressed by the reality they saw in Colombia, Vietnamese children with so-called ‘Twenty-First the Vietnamese found some resources gener- Century Skills’. ated from efficiency savings in the PEDC project to design a pilot of Escuela Nueva in Vietnam that The final element of the storyline regarding the began implementation in 2010. adoption of Escuela Nueva in Vietnam is the role of the Global Partnership for Education (GPE). The The pilot program was developed for a geo- Education for All (EFA) goal had been defined graphical spread of six provinces from three dif- as “Ensuring that by 2015 all children, particu- ferent regions of Vietnam with disadvantaged larly girls, children in difficult circumstances, and ethnic minority populations. In each prov- and those belonging to ethnic minorities, have ince, two districts were chosen, in each district access to, and complete, free and compulsory two schools, and in each school two Grade 2 education of good quality.” Vietnamese officials classrooms—for a total of 24 schools and 48 class- put together a detailed package of interventions rooms. According to the MOET officials at the that would help scale up the Escuela Nueva time, the pilot generated an unexpectedly high model on a much broader scale than the small level of enthusiasm and energy among the par- pilot. The argument presented to the GPE author- ticipants and stakeholders. For example, though ities was the opportunity for the GPE to support materials and teacher training were provided the establishment of quality education for the only for two class sections in each pilot school, disadvantaged population in a country that had other teachers observed the Escuela Nueva class- largely met the quantitative goal of EFA, but still rooms and began to adopt elements of Escuela needed help. The enlightened conceptualiza- Nueva, such as learning corners and group seat- tion of school quality is presented eloquently by ing. In fact, some of the neighboring schools also Epstein and Yuthas, 2012: sought information and knowledge to adopt Escuela Nueva (EN) practices on their own. And “The traditional definition of school qual- this was a nearly universal phenomenon across ity in the developing world is based on con- the diversity of contexts, not just isolated in one tent mastery. But using traditional schooling or two of the pilot schools. approaches during the few precious years most children will spend in school leads to A main reason to initiate a pilot regarding the wasted resources and forgone opportunities pedagogical approach was the plan in Vietnam for individuals and communities. Governmen- to undertake a major overhaul of the educa- tal agencies and organizations that support tion system called the “Fundamental and Com- and promote quality education for all children prehensive Reform of Education.” The reform, must move beyond traditional models to help currently in implementation, is formalized in children develop the knowledge, skills, and 4 www.worldbank.org attitudes that are relevant to their lives and long-term growth prospects and the risk of fall- that can lift them out of poverty.” ing into the so-called middle-income trap.3 Vietnamese leaders seek through the educational 1.2  VNEN Theory of Change reform to both enhance the acquisition of socio- emotional skills such as creativity, independence, An impact evaluation study is by design a kind and problem solving, and also enhance perfor- of scientific experiment. It is a very difficult kind mance on cognitive skills. Non-cognitive skills are of experiment because for ethical and practical sought to be enhanced as an end in themselves. reasons, it is usually impossible to exercise a high One body of research evidence points to the level of control over the conditions. Experimenta- direct impact of non-cognitive skills on life out- tion is usually meant to follow a certain theory comes. For instance, Kautz et al., 2014 provides a or set of theories. In this complicated case of a literature review regarding the direct impact of whole school model implemented at a sizable non-cognitive skills: scale of 1,447 schools and 450,000 children, there is not a simple theory that could be cited. “The literature establishes that achievement With regard to VNEN, there are at least five sets of tests do not adequately capture character theories or bodies of knowledge that were instru- skills/personality traits, goals, motivations, and mental either explicitly or implicitly in designing preferences that are valued in the labor mar- and then implementing the VNEN model. Each of ket, in school, and in many other domains. . . . them is dealt with in turn in this section. Character is a skill, not a trait. At any age, char- acter skills are stable across different tasks, but By way of introduction, it is useful to trace the skills can change over the life cycle. Charac- chain of reasoning from the objective of the ter is shaped by families, schools, and social program to specific actual or targeted actions. environments. Skill development is a dynamic The ultimate objective of VNEN is for children to process, in which the early years lay the foun- develop their human potential and to grow up dation for successful investment in later years. as adults to a peaceful and prosperous society. High-quality early childhood and elementary While some Vietnamese youth excel in Math school programs improve character skills in a Olympiads and Vietnamese children perform lasting and cost-effective way. Many of them better than the Organisation for Economic Co- beneficially affect later-life outcomes without operation and Development (OECD) average on improving cognition.” the Programme for International Student Assess- ment (PISA) in Mathematics and Science, the Another strand of the literature examines the Vietnamese education system still faces a num- connection between non-cognitive skills and ber of unmet challenges. The problem definition student academic achievement (Farrington in Vietnam has been that the education system et al., 2012). In this formulation there are cer- is geared to cramming children’s heads with tain ‘academic behaviors’ conducive to good information and knowledge. When they grow performance as measured through teacher pro- up and the children from middle- and upper- vided class grades and standardized test scores. income families go to college, they continue to These behaviors are related to responsible or be fed facts and more information. For most of disciplined class attendance, homework assign- the poor children, who would be lucky to get to ments, and preparation for and participation and finish high school, they often grow up with in class activities. There is a related concept of deficient livelihood skills. There is an overall lack ‘academic perseverance’ that deals with general of entrepreneurship and innovation in society, character traits or skills such as grit or tenacity and Vietnam faces the prospect of a slowdown in Enhancing School Quality in Vietnam through Participative and collaborative Learning 5 Why Vietnam Escuela Nueva (VNEN)? and self-discipline. Academic behaviors and per- small description of the application of complex severance in turn arise from an ‘academic mind- systems thinking to the design and implementa- set’ said to be manifested through motivation, tion of VNEN. Elaborate and far reaching reform effort, and self-belief. These three individual ‘aca- in the very basics of the teaching and learn- demic’ characteristics are ideally combined with ing process is highly dynamic and somewhat the development of social skills such as commu- unpredictable. A final subsection deals with the nication and cooperation. Students then learn dynamics of reform. to develop and deploy appropriate ‘learning strategies’ including goal-setting, self-regulated learning, and study skills. The ultimate result is 1.2.1  Participative and improved academic performance (Farrington Collaborative Learning et al., 2012). The central and most important aspect of the VNEN program is the fundamental transforma- Finally, there exists a more applications oriented tion in the teaching and learning method in literature. An excellent example is OECD, 2015 the classroom. The new teaching and learn- that provides a detailed overview of evidence ing method is meant to impart, directly and from longitudinal studies in nine OECD coun- indirectly, improvement in both cognitive and tries. The findings from this study are very useful non-cognitive skills and competencies of the stu- to examine, and the interested reader is directed dents. Various terms are used interchangeably to to the actual OECD study which is freely available describe this method or aspects of the method for download from the link in the bibliography that constitutes practice in the VNEN classroom. section at the end of this book. Two of the salient These terms include active learning, participative conclusions are worthwhile to replicate here. The and collaborative learning, peer learning, coop- finding report that “Teachers and parents can erative learning, and so on. Much of the initial lit- help improve children’s social and emotional erature regarding EN in Colombia focused on the skills by promoting strong relationships with multigrade setting.4 The multigrade classroom children and mobilizing practical learning expe- is usually not a choice as much as it is imposed rience.” As will be seen presently, VNEN interven- on some education systems, typically in remote tions follow this logic, and although this study areas because it is economically or financially not represents only a beginning as far as Vietnam is feasible to have mono-grade classrooms. concerned, at some point in the recent future, researchers should be able to obtain empirical Multigrade is actually an extreme manifestation data to find out the extent to which Figure 1.1 of something that is universal in classrooms— regarding Korea from the OECD book would also differential levels of ability along various cogni- be similar to Vietnam. tive domains, coupled with diversity of preferred learning strategies of individual students. In a With the above introduction regarding what multigrade classroom, it is almost impossible or VNEN hopes to achieve—the joint improve- definitely lacking in efficiency if the teacher tries a ment over time in cognitive and non-cognitive command-and-control lecture mode of teaching skills—we can turn to the five pathways through to impart knowledge. For example, a teacher may which the program hopes to achieve this out- be lecturing to Grade 4 children, while the Grade 3 come: (i) participative and collaborative learn- children listen with vacant stares, or vice-versa. ing; (ii) community involvement in education; There are all kinds of downstream negative effects (iii) incorporation of bodily or physical movement of loss of interest and motivation. But a moment’s in learning; (iv) the serious inclusion of play and reflection would show that this also happens in the connection to real life in learning; and (v) a a mono-grade classroom even though children 6 www.worldbank.org Figure 1.1  Replicated from the OECD Publication—Causality and Correlation of Cognitive and Non-Cognitive Skills in Korea Source: OECD (2015), Skills for Social Progress: The Power of Social and Emotional Skills, Figure 4.3, OECD Publishing, Paris. tend to be of a very similar age (abstracting away bored and apathetic. As enunciated by the famed from the issue of repeaters that would widen the Russian psychologist, Lev Vygotsky (1896–1934) age distribution). In a traditional classroom in who can be said to be the intellectual grandfa- Vietnam, the teacher typically would aim for the ther of VNEN, for children to learn well, they need middle of the ability spectrum—children who to be in the “zone of proximal development” or need more help to understand may be simply left ZPD. The best reference to the reader unfamil- behind, and children above the median may get iar with the concept of ZPD is available from the Enhancing School Quality in Vietnam through Participative and collaborative Learning 7 Why Vietnam Escuela Nueva (VNEN)? excellent video “ Vygotsky in the classroom” pro- changes over the last 25 years has been a greater duced by some highly creative education stu- focus upon implementation integrity.” dents and available at https://www.youtube.com/ watch?v=I9CnZoFUlT0. Peer learning is meant to Seating children in small groups of three to five enhance the probability for a student to be in children in each group while the teacher contin- his or her ZPD, with more ‘time-on-task’ or ‘t.o.t.’ ues to direct the entire classroom’s activity from and more ‘time-engaged-with-task’ or ‘t.e.t.’ See the front of the classroom is clearly not active Hogan and Tudge, 1999 for more details on the learning. What is the meaning in context of VNEN connection between Vygotsky’s thinking and par- of participative and collaborative learning, also ticipative and collaborative learning. termed as cooperative learning? Johnson and Johnson, 1999 describe five key elements of Figure 1.2, reproduced from Topping, 2005 cooperative learning that are often mentioned describes the elaborate process whereby par- in the literature: (i) positive interdependence: our ticipative and active learning may be expected own individual growth is linked to the goals of to lead to non-cognitive and cognitive learning ­ others—group members have assigned roles; gains. The figure purports to show how the five mutual learning goals exist to make sure every- basic subprocesses at the top lead successively to one in the group helps to define and to attain deeper interactions between the group members the objectives; (ii) individual accountability: to so that at the end there is improved metacogni- make sure that each child is contributing and tion (thinking about thinking) and enhanced learning and not free-riding to his or her own self-esteem and self-attribution (I know how to detriment and that of the group; (iii) face-to-face learn and I am good at it). In turn they feed back promotive interaction: this may be at the core of to reinforce the initial five subprocesses outlined the interaction as it involves working with others at the top. The author seeks to summarize the in pairs and as a whole group to explain, ques- vast recent literature on peer learning that is tion, encourage and support or agree when the used across levels of education for children and teacher needs to be asked for help; (iv) social adults alike. In elementary school settings such skills: students learn by doing—they learn appro- as VNEN, perhaps the related term ‘participative priate behavior to question and consult, to praise and collaborative learning’ is more appropriate as when deserved, to build empathy and trust; the teacher plays a key facilitating role. The fol- (v)  group processing refers to norms and proce- lowing quote from Topping, 2005 is particularly dures to work together—for example if you need insightful: to borrow someone’s pencil, you may not just grab it, but request and be granted the request “However, many schools might think they in a polite, overt communication exchange. The are implementing peer tutoring or coopera- group mostly needs to learn to police itself; there tive learning, when all they are really doing is is hardly any fun or innovation in the teacher putting children together and hoping for the shouting across the room to maintain discipline best . . . found that while children were often and order. In the VNEN context, group processes placed in groups, mostly they worked as indi- were termed evocatively by the Colombian con- viduals. Only one-sixth of the time was spent sultants working in Vietnam to help MOET as ‘el interacting with other pupils, and most of this ritmo de la aprendizaje’ meaning ‘the rhythm of was not related to the task. learning’—evocative because it uses a musical or dance metaphor to describe the organic pro- Sometimes hoping for the best works for some cess at the heart of a VNEN group, the technically children, but typically not for those most in accurate term would be intersubjectivity as in need. . . . Accordingly, one of the most important Figure 1.2. Group processes in a VNEN classroom 8 www.worldbank.org Figure 1.2  Theoretical Model of Peer Assisted Learning from Topping, 2005 Organization & Cognitive Scaffolding & Communication Affect engagement con ict error management language—thought, motivation, t.o.t., t.e.t.; to liquify ZPD management; listen, explain, question, accountability, goals, plans, primitive information modulation; clarify, simplify, prompt modeling, individualisation, cognitions modeling, monitoring; rehearse, revise, ownership, interactivity, and error detection, summarise, speculate, self-disclosure immediacy, beliefs diagnosis, correction hypothesise variety Situated accretion, returning, restructuring Inter-subjective cognitive co-construction Practice, uency Generalisation automaticity, retention supported—independent mostly implicit implicit—explicit Feedback & reinforcement Self-monitoring self-regulation implicit and explicit Metacognition Self-attribution explicit, intentional, self-esteem strategic I want to know I know I know I want to know how, when, if I know I know how I believe I can know how, when, if I know I know when and if In iterative cycles: surface—strategic—deep declarative—procedural—conditional Source: Adopted based on Topping, Keith J. (2005). “Trends in peer learning.” In: Educational psychology 25:6, 631–645, Figure 1. Enhancing School Quality in Vietnam through Participative and collaborative Learning 9 Why Vietnam Escuela Nueva (VNEN)? are mediated by the use of textbook supplanting consultation with teachers, parents, and other learning guides that provide self-directed and members of the school community. self-guided sets of activities designed to meet the curriculum requirements.5 One interesting aspect with regard to com- munity involvement is the element of ‘student government’. VNEN seeks to inculcate values of 1.2.2  Community Involvement service and respect for others as well as helping Very closely related to the participative and col- to develop skills of communication, teamwork, laborative learning at the heart of VNEN is the and leadership. Rather than merely teach about practice of community involvement in education. the relevant virtues and behaviors through the This comes in the form of three kinds of actions: teacher’s lectures, the students are meant to act together responsibly. This aspect of community (i) involvement of the community in the day-to- involvement has been particularly important in day administration of the school and decision countries with recent histories of conflict (Baessa, making about the deployment of resources that Chesterfield, and Ramos, 2002; Forero-Pineda, is meant to result in greater ownership of the Escobar-Rodríguez, et al., 2006). Well imple- school and its success by the community; (ii) mented student government would involve incorporation of local knowledge and culture in election of the officeholders by the students, the the curricular content which improves the rel- clear assignment of roles and responsibilities to evance of education as well as enhancing social different positions, and oversight and feedback cohesion; and (iii) the direct incorporation of par- from the teacher. Students establish various com- ents, grandparents, and other community mem- mittees such as a committee for cultural activities bers in the teaching and learning activities. and community outreach, a learning committee, a library or reading committee, and so on. Under The notion that greater community control over the tutelage of adults, the idea here is for children school administration may lead to improved to learn at an early age to take responsibility for school effectiveness has been studied exhaus- things they can influence. tively in the literature on decentralization of pub- lic service delivery. It is highly cited and discussed As part of the effort to make education more rele- by Mundial, 2004 and Ahmad et al., 2005. In the vant, VNEN schools encourage parents and other particular case of VNEN, the context for all schools community members to take part in the school’s has generally included the involvement of the activity. Local traditions and cultural practices, village or commune people’s committee (local especially from ethnic minority communities, government). The special aspect of VNEN was can be preserved and enhanced at school. Par- the provision of an annual school grant, of about ents are encouraged to visit VNEN schools and US$2,000 per school for the school to spend on teach skills such as local dances and music as well its priorities associated with the implementation as artisan skills including weaving and prepara- of VNEN. The school community could decide to tion of food products. Farmers provide ‘show and spend the resources for rehabilitation and repairs tell’ demonstrations regarding their actual work of the physical plant, furniture suitable for seating with crops and animals. In certain instances, bor- children in groups, didactic materials in addition rowing from Colombian precedents, there is also to the materials provided by MOET, professional an attempt to bring in linkages with the domi- development meetings for teachers, hiring of nant local ­industry—such as coffee and tourism. temporary teaching assistants, and so on. Deci- Drawing clear links between the curriculum and sions are usually made by the school principal in livelihood, for example measuring distance and 10 www.worldbank.org visiting the farm at the time of rice transplanting, useful to better understand the VNEN model. are fun for children at the same time as they serve Movement at the beginning of the class helps a didactic and motivational purpose. grab students’ attention and prepares them for learning—this has deep physiological reasons to do with the relationship between attention 1.2.3  Movement and Learning and ‘state changes’ in the body and mind. Atten- There was a time when it was thought that the tion and motivation are highly correlated, which best classroom is one where the students are all makes intuitive sense—it would appear to be quiet, demurely sitting in rows, without fidget- difficult to study hard if you are not even paying ing or indeed making any movement—except attention! A state change refers to a change in perhaps raising their hand when asked by the the student’s mental, emotional, or physiologi- teacher. In the traditional primary school in Viet- cal state. In a VNEN class, every learning activity nam, there is even a norm of the correct way to typically begins with a physical warm-up. This raise your hand, with one arm angled horizontally is especially effective when done as a game— in front and holding the elbow with the other what is termed as ‘ludic activity’ in the academic hand, which should be perpendicular, with open literature. As eloquently written by Summerford, palm facing inward and the arm not sloping this 2009, “Our brain’s attentional system is set up way and that. Talking to your neighbor, even in to prefer high contrast, originality, and exciting whispers, may be regarded as a negative sign connotations.” about your being a ‘good girl’. Perhaps the tradi- tion that associates good learning with quiet and Hannaford, 1995 describes a very interesting lack of movement is one of the toughest ones to relationship between the strategies she used to break. In a Confucian Heritage Culture (CHC) like help the learning of disabled children compared Vietnam, the inertia probably goes back a few to the learning of children without any disabili- thousand years (Littrell, 2005). ties. She describes the very meaning of the term ‘learning’ as it is a process that begins at birth or Summerford, 2009 has compiled a long and even before. According to Hannaford: detailed timeline of research regarding the con- nection between the body and the brain that “Learning is a highly natural process, invigo- leads her to prescribe a series of actions for the rated by our interactions with other people elementary classroom teacher. The key point she through our sensory-motor experiences and makes is that cognition is facilitated by move- sense of connectedness and appreciation.” ment. This is because movement anchors learn- ing through the body, it energizes and integrates This is a description in perhaps an academic the body and brain for optimal learning, and it language of something that is obvious to any- makes learning fun. The scientific basis of the one who has observed closely the behavior of linkages between movement and learning is also human babies. It is immediately clear by look- explained carefully by Hannaford, 1995 who cites ing at this famous video (with more than 4 mil- book length treatments such as Damasio, 1999 as lion views as of December, 2016) guided by well as scientific papers regarding the structure Dr. Edward Tronick, Director of the Infant Par- of the human brain and the parts of the brain ent Mental Health Program at the University that are engaged through movement (Middleton of Massachusetts: https://www.youtube.com/ and Pl, 1994). watch?v=apzXGEbZht0. The video shows how a one-year-old infant interacts with her mother The literature is wide ranging and covers dif- and gets emotionally distressed (only temporar- ferent topics, but a short summary would be ily, for the benefit of science) when her mother Enhancing School Quality in Vietnam through Participative and collaborative Learning 11 Why Vietnam Escuela Nueva (VNEN)? shows an expressionless face and is unrespon- altruism and understanding. . . . Our primary sive to the baby’s movements and antics to get emotions (happiness, surprise, fear, disgust, the mother’s attention. What begins at infancy anger and sadness) are not often activated in carries through for all of our life. We develop an our consciously controlled society. But when understanding of the world initially in a spatial we truly play, these emotions are allowed to way, by exploring space through movement, surface in a safe way. . . . When we are able including touch. And we also develop emotional to take in our fill of sensory stimuli, process intelligence and social skills such as empathy, and integrate it with richly developed base through mirroring or mimicking actions of oth- patterns, and express new insights in a cre- ers. It is well known that emotion is linked to the ative way, both physically and verbally, we are formation of lasting memories (long-term poten- then truly at play. . . . Play on words, punning tiation or LTP in the scientific literature). Emotion and wit have contributed to creative thought and the physical body are inextricably linked— in literature and drama throughout the ages. the very definition of emotion concerns involun- Composers, choreographers, and artists of all tary physiological changes (Ekman, 2007). The kinds express their integration through play. ideal VNEN classroom, therefore, is humming and The exhilarating play of modern science spills vibrant with movement—children get to express over into mathematics, paradoxical puzzles their natural selves, they are transported or alle- and language.” viated to a positive affective state (also known as ‘having fun’), and consequently absorb mate- Johnson, 2016 takes the story much further, rial faster and retain it longer, all the while devel- claiming that all of human history can be oping deeper socio-emotional skills. This ‘fun’ is explained as originating in humanity’s love of actually deadly serious, as explained further in playfulness. Singer, Golinkoff, and Hirsh-Pasek, the next subsection. 2006 and Brown, 2009 claim a bit less for playful- ness, but just as with movement and the need to anchor learning in real life, they provide a con- 1.2.4  Play and Learning vincing case for the critical importance of ludic Serious interest in play by the general public was activities in a well-rounded education. The estab- stimulated recently by the widely reported news lishment of boundaries and rules, and the explo- that the software company Google (now part of ration of those boundaries are key to explaining Alphabet, Inc.) provides 20 percent of free time the impact of play on developments of skills. to employees to engage in any pursuit of their Singer, Golinkoff, and Hirsh-Pasek, 2006 quote own choice, without any obligation with regard the selfsame Lev Vygotsky with regard to the to their regular work. A number of commercially importance of play in stretching a child’s compe- successful products have been reported to have tencies. The author provides a very detailed and come from free-thinking employees engaged in well referenced exposition of how and why play play (Copeland and Savoia, 2011). A branch of the is equated with learning and how play is impor- same neuroscience literature mentioned with tant for the development of self-regulation and regard to movement also examines the question creativity. of play. Indeed, Hannaford, 1995 describes the role of play: One issue that bears greater attention is the diffi- culty in convincing parents and even some teach- “Play provides the emotional spark which ers that play is really useful. As Johnson, 2016 activates our attention, problem solving and says, “The pleasure of play is understandable. The behavior response systems so we gain the productivity of play is harder to explain.” Singer, skills necessary for co-operation, co-creativity, Golinkoff, and Hirsh-Pasek, 2006 document how 12 www.worldbank.org many important stakeholders are not yet con- the government was interested in generating a vinced about the usefulness of play inside the widespread national awareness and feedback classroom. The authors cite a study carried out generation campaign about the new pedagogi- by the Lego Corporation in five OECD countries cal model. A seeding model of demonstration which showed that 94% of the parents agreed VNEN schools was adopted for provinces with a that time spent playing is time spent learning. lower disadvantaged population; Priority I prov- Yet the same parents are apparently confused, inces would have a high density of VNEN schools, because they indicate that time in ‘cognitive roughly every other school would be a VNEN tasks’ should take precedence over free time in school. In 19 Priority II provinces with a moder- play. Another study quoted by the authors put ate number of disadvantaged children, each ‘learning’ as the 12th ranked benefit out of 14 district within a province would have one or two benefits of play. We will present some findings demonstration VNEN schools, with an average of from Vietnam regarding the attitudes of parents. about 10 schools per province. Finally, in 24 rela- tively wealthy provinces with a very small popu- lation of disadvantaged children, there would 1.2.5  Complex Adaptive Systems be one demonstration school per province. This Complexity sciences have now become very method was instrumental in helping VNEN be popular, with numerous dedicated journals and a truly national program whereby widespread scientific conferences. The field itself is very vast, awareness about VNEN schools was generated with Mitchell, 2009 providing a useful introduc- and about 2,000 additional schools adopted the tory overview of this amalgamation of many VNEN model even though they did not receive disciplines. A central concept in complexity sci- any financial resources from the project. ence is Complex Adaptive Systems or CAS. This term was introduced and explained at length by The second important application of complexity John Holland—see Holland, 1992 and Holland, science important in understanding the VNEN 1995. Holland characterizes CAS as having cer- model of change is the use of the biweekly clus- tain properties and mechanisms—these include ter meetings for continued teacher professional aggregation, tagging, nonlinearity, flows, diver- development. Indeed, the entire training model sity, internal models, and building blocks. Mitch- for VNEN was based on an application of com- ell, 2009 provides further explanations regarding plexity science principles. In the usual ‘cascade’ self-similar replicating algorithms and the very model, an initial ‘training of trainers’ is followed important concept of scaling associated with by successive rounds of training so that by the what she terms as the science of networks. The time the training reaches the school level, the classic CAS originally studied were living systems understanding of the original model may be such as ant colonies or forest ecosystems, but this significantly distorted. In the case of VNEN train- soon was extended to human organization. The ing, a few school teachers were involved at the importance of complexity science insights for national level training, and subsequently there human organization is laid out very well in the were multiple feedback loops for the training to evocatively titled book ‘Emergence: The Con- be grounded in reality and to reduce the loss of nected Lives of Ants, Brains, Cities, and Software’ signals through successive rounds. (Johnson, 2002). Development of the learning guides also The first important application of complexity sci- included a participative approach, so it was not ence in the VNEN program was the design of the just a group of writers preparing material to be program. In preparation for the upcoming funda- subsequently deployed without much recourse mental and comprehensive reform of education, for modification. Enhancing School Quality in Vietnam through Participative and collaborative Learning 13 Why Vietnam Escuela Nueva (VNEN)? 1.3  Overview of the VNEN treatment and control group schools that were to be followed for two years. The data used for Impact Evaluation (IE) Study the PSM was the same school census data that Design had been used to prioritize provinces. It was determined that for adequate power to detect an This impact evaluation study is based on a design effect size of one-tenth of standard deviation in that follows a cohort of students for two years, test scores, a sample size of 300 treatment group starting from a sample in Grade 3 in 2013–14 schools and 300 control group schools would be and tracking the cohort as it moved through to adequate. Given the chances of logistical issues Grade 4 in 2014–15 and Grade 5 in 2015–16.6 This or possible school mergers or other unforeseen section begins with an overview of the propen- events, a buffer of an additional 10% of sampled sity score matching (PSM) method used to ran- schools was defined, so that a total of 660 schools domize the selection of treatment and control would be sought to be sampled. As the province group schools. The data collection methodology prioritization was a defining element of the pro- was comprised of a set of field survey question- gram, the PSM was conducted separately for two naires that were administered each year to school groups—Priority I provinces with a total sample principals, teachers, parents, and students, and of 440 schools, and Priority II and III schools with a standardized assessment test for students in a total sample of 220 schools. mathematics and Vietnamese. As in any experi- ment involving human subjects, it was not fea- It was fairly straightforward to find nearly sible to control all aspects of the intervention, matched neighbors in both samples with trim- and we found out early that there is considerable ming to enhance the common support, and a implementation heterogeneity. Implementation random number generator was used to select was varied among the treatment group schools, the actual sample. Post selection balance tests and there was spillover or leakage in terms of showed very high levels of similarity between the control group schools adopting elements of treatment and control group schools. Within the VNEN practice. We discuss this aspect in some selected schools, the number of which eventu- detail and conclude the section with a descrip- ally came down to 648 because of logistics issues, tion of the qualitative component of the study a random number generator was used to select using video. The methodology for video analysis 20 students at random from Grade 3 during the was developed in the initial period and deployed baseline, and the same cohort was followed over in the final year when the cohort was in Grade 5. the two years. The benchmark round of data col- lection was held in December, 2013. 1.3.1  Randomized Selection of Intervention and Control Groups 1.3.2  Survey Rounds There was an initial delay in the availability of Field survey instruments were designed to cap- funding before Dubai Cares stepped in with the ture three sets of variables—variables that would funding. The program implementation began help determine the extent to which VNEN was in the summer of 2012, with children then in implemented at a school; an extensive set of Grade 2. As the VNEN program had already been contextual variables that were deemed to be rel- implemented the year before the initiation of evant either as determinants of implementation the impact evaluation study, it was not possible or as causal variables leading to outcomes; and a to undertake a random assignment of the pro- set of outcome variables. The standardized tests gram. A propensity score matching method was were administered under strict vigilance by proc- used to determine the randomized sample of tors other than the teachers of the same school, 14 www.worldbank.org with established time limits and the shuffling In order to have a reasonably robust index, of questions across booklet variants to prevent we include a number of measures from three copying of answers. Wherever possible, ques- respondent groups: students, parents, and teach- tions were used that had been established in the ers. As the behaviors in question are only ascer- literature to be useful; for example, the items to tained through asking questions based on recall, determine teacher self-efficacy, an important we adopt an aggregating strategy to avoid the construct related to teacher performance, was problem that the noise in the measurement of based on an extensive literature around this the index may overwhelm the signal. An exam- concept (Gibson and Dembo, 1984; Woolfolk and ple with one of the elements that comprises the Hoy, 1990; Senemoğlu et al., 2009). index will illustrate the idea. The students were randomly sampled from within the sampled Data collection for the quantitative survey was school; hence, students within a school belonged done by a well-trained team of field investiga- to different classroom sections. In the sample, tors who used handheld electronic devices each school had an average of just over three for recording answers. The data was uploaded sections, and each section would have different at the end of the day to a cloud-based reposi- class teachers with different practices. Student tory for checking and quality control so that recollections and interpretations may also vary, any errors could be rectified while the team which means that it will be useful to aggregate was in the field. Tests were prepared by a team over the students to estimate the level of imple- of trained MOET test preparers who received mentation of group activities at the level of the guidance and support from a Canadian interna- school. tional testing expert with extensive experience regarding standardized tests. The usual protocol The construction of the implementation index of a pre-test followed by the selection of items is outlined in Chapter 4. An example of one of for the actual testing was used, depending on the elements used in the index is included in the Item Response Theory (IRT) parameters of this introductory section by way of illustration. the items. A question was asked of the students: “Do you do group activities in Vietnamese subject?” with the choices being ‘Yes, often’, ‘Yes, sometimes’, 1.3.3 Implementation and ‘No’. We construct a variable as an element Heterogeneity of the implementation index by counting the A key issue in this impact evaluation is that the number of students who said ‘Yes, often’, sub- VNEN program has multiple elements. Even tracting the number of students who said ‘No’, though there existed randomly selected treat- and expressing the difference as a percentage of ment and control group schools, in fact, quite the number of students who answered the ques- a few of the control group schools adopted ele- tion. Now, if there is an overwhelming majority ments of the VNEN model. Conversely, only a few of students who answered ‘Yes, often’, and did of the treatment group schools applied all the so in each of the three rounds of data collection, elements of the model. There exists a consider- the chances are high that the teaching of Viet- able heterogeneity in the implementation of the namese language was being done with students model. In order to deal with this issue, we con- working in groups. Conversely, with a majority struct a ‘VNEN implementation index’. Section 4.3 of students saying that group work was not the of this report provides a detailed account of the norm, chances would appear high that the teach- construction of the implementation index, and ing method was traditional rather than VNEN. an introductory summary with examples is pre- Table 1.1 presents the data regarding the high sented here. incidence of group activities for Vietnamese and Enhancing School Quality in Vietnam through Participative and collaborative Learning 15 Why Vietnam Escuela Nueva (VNEN)? Table 1.1  Students Report of Group Activities Variable Group Mean (Standard Deviation) 2013 Vietnamese group activity VNEN 72.51 (25.34) Control 21.14 (41.41) Mathematics group activity VNEN 67.47 (29.32) Control 6.04 (45.76) 2014 Vietnamese group activity VNEN 74.38 (18.17) Control 27.89 (28.46) Mathematics group activity VNEN 69.09 (21.17) Control 15.34 (32.90) 2015 Vietnamese group activity VNEN 72.05 (17.09) Control 34.80 (26.85) Mathematics group activity VNEN 63.78 (18.15) Control 23.31 (32.20) Source: VNEN Impact Evaluation study primary data, Authors calculations. mathematics for each of the three survey years that for a robust measure of implementation, a from 2013 to 2015. number of measures need to be combined. The mean for the VNEN or treatment group is about two to three times the control group. The 1.3.4  Use of Item Response Theory distribution is tighter for the treatment group, as (Rasch Indices) shown by the lower relative standard deviation. Throughout the study, as mentioned in the rel- The VNEN group shows a slight increase from the evant sections, we have summarized the col- first to second year, and then a decline in the third lected data through the construction of Rasch year in the level of the variable. The control group indices (Masters, 1982; Green, Frantom, et al., shows a progressive increase in the mean value 2002; Bond and Fox, 2015). This method has great over the two years, and a decline in the disper- intuitive appeal. The raw score on the categori- sion, though the standard deviation for the VNEN cal survey items is only the starting point for the group always remains smaller and in fact declines construction of a weighted mean likelihood esti- monotonically. The aggregated measures of mate (WMLE) measure that captures the pattern Table 1.1 do not show the value of the variable of variation in scoring of the elements (Linacre for individual schools, which tends to stay stable and Wright, 2000). By standardizing the resulting across the three years for some schools, and fluc- value to a mean of 0 and standard deviation of 1, tuate for other schools. This fluctuation indicates 16 www.worldbank.org the findings using the index can be easily inter- main criteria for school selection being to ensure a preted and understood. range of contexts and learning outcomes among the schools. The focus was on VNEN schools, as the aim is to clearly describe and illustrate what 1.3.5  Qualitative Research are advanced VNEN outcomes. However, in an The qualitative component of the Impact Evalua- impact evaluation setting, it was important to tion, presented in Chapter 7, is intended to com- include a few control group schools to provide plement the quantitative component in order to a reference or benchmark of practices in a tradi- provide a more complete picture of VNEN. The tional setting. In primary schools in Vietnam, as qualitative component provides a deeper under- in most other countries, the same teacher is typi- standing of how the VNEN program has been cally responsible to teach both Vietnamese and implemented in schools and the extent and ways mathematics. In each of the fifteen schools sam- in which it has been adopted. pled for the qualitative study, three mathemat- ics and three Vietnamese lessons were filmed for There are multiple pathways that could lead to three different teachers, for a total of 270 lessons superior outcomes, and the outcomes them- (15 schools × 3 teachers per school × 6 lessons selves are multidimensional. Such complexities per teacher). necessitate an approach of in-depth qualitative analysis. The analysis examines specific aspects The videos were analyzed by developing mul- and classifies practices of teachers and stu- tiple tracks of content analysis. The develop- dents into Beginner, Intermediate, or Advanced ment of tracks by human coders to examine depending on both (i) the extent to which the the incidence and duration of specific actions teacher and/or students follow the practices and in the classroom was popularized by Stigler and (ii) the level of effectiveness. Hiebert, 1999 in their insightful TIMSS Video Study. The standardized use of classroom obser- While students’ practices are determined in large vation, including video to provide data that can part by the teacher and how the teacher oper- be used to make comparisons over time and ates the lesson, the students also develop their across units of observation in a valid and reliable own way of operating and interacting. Therefore way, is the subject of a vast amount of literature.7 it is critical to study students’ practices both in This VNEN IE study seeks to contribute to the the context of student-teacher interaction and development of a robust and scalable research separately from student-to-student interaction. method for meaningful measurement regarding The design of the classroom analysis therefore the acquisition of non-cognitive skills. Further involved the use of three separate cameras: one development, including technological advances focused on the teacher and two focused on two such as wearable computer devices and machine separate student learning groups of four to six readable video are possible sequential steps. students. The analysis of these three perspectives Data generated in this way may be a revolution- gives a rounded picture of what is taking place in ary means to develop an understanding of non- the classroom. cognitive skills, and this study forms part of baby steps along a long but promising journey. A sample of fifteen schools across four provinces was used for the qualitative component, with the Enhancing School Quality in Vietnam through Participative and collaborative Learning 17 Why Vietnam Escuela Nueva (VNEN)? 1.4  Preview of Findings VNEN has a positive impact on Vietnam This study reports the findings of positive impact using an impact evaluation procedure that fol- lowed a cohort of students from Grade 3 through to Grade 5. The positive impact holds for both mathematics and Vietnamese standardized tests scores as well as for measures of non-cognitive achievement. These findings need to be examined through the scrutiny of academic publication as the underlying models used are complex and require specialized knowledge of statistical meth- odology regarding inference on attribution. The study finds evidence of learning gains for both cognitive and non-cognitive aspects in context of a cohort of intervention and control groups; it finds that learning progress is better for VNEN students. Full scale impact requires implementation feedback loops What drives the positive results of the program is not so much the label of VNEN or participation in the program as much as the full implementation of VNEN elements. While all VNEN schools met the letter of the program, we found that implementation intensity varied. In a complex intervention with many elements, and which requires teachers and stakeholders to transform their mindset, the heterogeneity is not unexpected. A complex reform cannot be enforced uniformly by mandate. Going forward with the reform, it is very important for strong and multiple feedback loops for all stakeholders. These feedback loops will help generate ownership of the model. The feedback loops will also enable learning regarding what works and what does not work. Implementation was strongest when school leaders and teachers showed initiative and creativity regarding participative and collaborative learning. Video analysis indicates how and why the results are obtained Source: © World Bank The qualitative portion of this study seeks to understand the how and why of impact. We know that VNEN had a positive impact, we know that impact was related to intensity of implementa- tion, and the video analysis helps us to understand what that means. Future development of the methodology will provide rich insights and policy inroads. Vietnam has potential to lead the developing world regarding the achievement of the goal of learning for all, providing 21st century skills in an inclusive way. 18 www.worldbank.org Endnotes 4. Psacharopoulos, Rojas, and Velez, 1993; McEwan, 1998; Benveniste and McEwan, 2000; McEwan, 2008. 5. The learning guide activities follow a prescribed 1. Retrieved from http://isites.harvard.edu/fs/docs/icb sequence which starts with prior knowledge, moves .topic566774.files/Escuela%20Nueva.doc on Decem- through the introduction of a new concept, then the ber 15, 2016. As the focus of this book is on Escuela application of the concept in known situations and Nueva in Vietnam, we skip details regarding the finally applications in unknown or future situations. program in Colombia. An excellent written overview Sometimes there is also follow-up work to be done at can be found in Psacharopoulos, Rojas, and Velez, home or in the community. For a detailed description 1993. Introductory accounts are also available in of the way in which the theories of participative or Colbert, Chiappe, and Arboleda, 1993 and Colbert, collaborative learning were prescribed for imple­ 1999. mentation as part of the VNEN program, please refer 2. Details of the conference and related materials can be to the thirteen VNEN dimensions—organizational found at Multiple Approaches of Education for Rural factors for peer learning, in the language of Topping, Areas: Lesson from Good Practices. Cebu, Philippines, 2005. February 9–12, 2009. 6. The school term in Vietnam starts at the end of 3. Vietnam faced a declining trend of growth in product August or early September and ends in May of the diversification at the beginning of this decade. A following calendar year. network analysis of world trade flows showed that 7. See Erickson et al., 2006; Rich and Hannafin, 2009; Vietnam’s revealed comparative advantage was Bell et al., 2012; Danielson, 2012; and Goldman et al., restricted to a few raw-material product categories 2014. (World Bank, 2011). Enhancing School Quality in Vietnam through Participative and collaborative Learning 19 School Leadership 2 This chapter presents findings in three sections. 2.1.1  School Autonomy In the first section, we discuss issues and fac- School autonomy is a part of the VNEN model by tors related to the strength of implementation implication. Autonomy is needed to include local of the program. This includes issues such as elements in the curriculum, involve the commu- school autonomy and the sufficiency of school nity, and transform the teaching method. The grants. The next section deals with contextual extent of autonomy of schools as determined in factors influencing implementation and results. 2013 is depicted in Figure 2.1. It covers issues such as the values and beliefs of principals and their political skills. Finally, a third Figure 2.1 is based on responses from the school section provides evidence of estimation of pro- principal about the locus of decision making gram impact. for the different decisions shown in the figure. This figure is useful as a scene-setting to under- stand the setting of the educational reform in 2.1  Strength of Vietnam. Decisions with greater local influence Implementation get higher points and vice-versa. The decisions of the Program with a high level of autonomy toward the left side of Figure 2.1 relate to financial matters such School leadership is often considered to be one as financial contribution levels and exemptions of the most important dimensions of school as well as carrying out of school infrastructure effectiveness or the performance of programs.1 improvements. Decisions marked in gray are the Related research has examined the aspect of five decisions with the lowest level of local auton- recruitment or election of school principals, the omy. They include the establishment of academic professional development of principals, and standards as well as decisions regarding school the autonomy provided to school principals as closure or key features of teacher administration. important in developing effective school lead- ers. Researchers have examined closely issues An autonomy index was constructed using a of performance, political skills, leadership styles, Rasch index method outlined in the previous and personality of school leaders.2 In the case of chapter for overall VNEN implementation. Fig- VNEN, there are many reasons why the role of ure 2.2 presents the index values for Priority 1 and the school principal may be vitally important. Priority 2 and Priority 3 provinces together. The The principal provides leadership in terms of VNEN group of schools had a higher autonomy belief and commitment about the reform. It is score as compared to the control group schools. the principal who leads the school community By construction the autonomy index has a mean in the application of the reform, by securing and of 0 and a standard deviation of 1. The difference deploying resources, or by solving problems in mean values for both priority groupings was along the way. about 0.25 points or one-fourth of a standard 21 School Leadership Figure 2.1  Decentralization of Decision Making, 2013 Source: VNEN Impact Evaluation study primary data, Authors calculations. Figure 2.2  School Autonomy Index, 2013   (a) Priority 1 provinces (b) Priority 2 & Priority 3 provinces Source: VNEN Impact Evaluation study primary data, Authors calculations. deviation, with a mean of –0.13 for the control the model and be favorably disposed toward group and 0.12 in 2013, the only year when this it. Figure 2.3 shows the evolution of principal question was asked. awareness over the three years of the study. A surprising 12% of VNEN Principals in 2013—well into the second year of the project—expressed 2.1.2  VNEN Implementation disagreement with the sentence “I am com- from Principal’s Perspective pletely familiar with the VNEN model.” However, In order for VNEN to be implemented adequately, this number came down to 5% by 2015. Within the principals need to be well informed about the control group, we find that 67% in 2013 22 www.worldbank.org Figure 2.3  Principal Familiarity with VNEN Model   (a) VNEN principals (b) Control group principals Source: VNEN Impact Evaluation study primary data, Authors calculations. expressed lack of familiarity with the model, but activities covering traditional teaching methods this number came down to 46% in 2014. From and ten items for VNEN pedagogy were used (see 2014 to 2015 the trend of awareness generation Figure 2.4a). Principals were asked how often among control group principals slowed down. In teachers in the school used that activity in class, this study, we often find that a change from the with the options being ‘always’, ‘often’, ‘sometimes’, first year to the second was followed by a slow- and ‘never’. An index was computed counting down in the third year. the frequency of mentions of ‘always’ and ‘often’ for the VNEN pedagogy items, less the count for The principals were asked more detailed ques- ‘always’ and ‘often’ for the traditional items, and tions about the implementation of VNEN in expressing the number as a percentage, so that the years 2014 and 2015. A set of five items or the highest possible value would be 100%. Figure 2.4  Principal View of VNEN Implementation Traditional Teaching a. Teacher reads while whole class listens and takes notes b. Teacher works with 1–2 students while whole class observes c. Teacher asks several students to read while whole class listens and takes notes d. Teacher writes instruction on board and students copy down instruction in notebook e. Teacher arranges for students to sit in rows facing the teacher VNEN Pedagogy f. Teacher arranges for students to sit together in groups i. Class has discussion about real life objects (natural like fruit or plant) or manmade objects j. Teacher finds a way to relate the lesson to real life k. Teacher encourages children to find out new things by themselves l. Teacher gives feedback to class about performance m. Teacher gives feedback to individual student p. Teacher uses a game as part of the lesson q. Teacher uses the 'role-play' method when teaching Vietnamese r. Teacher makes children solve many problems when teaching mathematics s. Teacher is able to help students individually with a particular problem   (a) List of pedagogical activities (b) Principal implementation measure of VNEN for 2014 and 2016 Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 23 School Leadership Figure 2.4b indicates that the VNEN pedagogy 2.1.3  School Grants and Student score was 67 for control group schools in 2014 Scholarships and 82 for VNEN group schools. The variable One key implementation measure of the VNEN vnen_ori takes on the value of 1 for VNEN group program was the provision of school grants. Every schools and 0 for control group schools. One school received a grant of US$3,000 per year, and interesting observation from Figure 2.4b is the schools with satellite campuses received an addi- decline from 2014 to 2015 in the index value tional grant of US$1,300 per year. This amount is for both VNEN and control group schools, with estimated to be about 10–15% of the average the VNEN group measure declining from 82% to school’s regular operational budget, without 77%. In the case of the VNEN pedagogy measure considering payment for teachers, who are paid from principals, we did not have a measure for directly by the provincial government. The rea- 2013, but for many of the variables, for instance soning behind the grants was that the disadvan- in the case of the familiarity with VNEN, we see taged communities targeted by the VNEN project a big movement from 2013 to 2014 and then a needed financial support to be able to imple- smaller or even negative movement from 2014 ment VNEN. The grant was expected to be spent to 2015. on teaching materials, professional development meetings at the cluster level, minor rehabilitation, There could be many possible reasons for lack of and repair and renovation of furniture if needed adoption of full VNEN practices. As an example, for children to be able to be seated in groups. The while 73–75% of VNEN group principals reported choices of expenditures were delegated to the that their teachers used games ‘always’ or ‘often’, school community—with the principal making the remaining 25% of VNEN principals may have decisions in consultation with teachers, parents, beliefs that preclude faith in the importance of and other stakeholders in the community. play or ludic activities in the classroom. Or it is possible that teachers had difficulty coming In the principal survey, a question was asked up with ideas of appropriate activities, though to the principal in VNEN schools about the suf- the VNEN learning guide is supposed to solve ficiency of the grant received for various items that particular problem by outlining suggested of expenditure and the results are presented in activities. Table 2.1. In this table a measure of 2 indicates Table 2.1  Sufficiency of VNEN School Grants Priority 1 Priority 2 & 3 Dedicated Use of School Grant 2013 2014 2015 2013 2014 2015 Professional development meetings 1.68 1.68 1.78 1.59 1.75 1.7 Learning space & teaching materials 1.09 1.63 1.67 0.97 1.64 1.72 Teaching tools & communication 1.13 1.53 1.62 1.05 1.61 1.57 material Organizing events and awards 1.12 1.49 1.53 1.24 1.56 1.48 Renovating furniture 0.86 1.38 1.38 0.83 1.40 1.34 Minor classroom renovations 0.78 1.10 1.20 0.67 1.23 1.16 Teaching assistant & extra-curricular 1.00 0.66 0.50 1.12 0.54 0.34 Source: VNEN Impact Evaluation study primary data, Authors calculations. 24 www.worldbank.org that the principal thought the funds were suffi- 2.1.4  Community Engagement cient for the purpose, and lower numbers suggest Community Engagement is an important facet of an opinion about less sufficiency. The interesting the VNEN program, with at least three interven- finding from Table 2.1 is that professional devel- tions: the involvement of parents in providing opment meetings received a score quite close to training for livelihood skills and local customs 2—lack of resources would not have held back and traditions, at school and through ‘applica- the school’s ability to organize biweekly cluster tions’ homework; the involvement of parents in meetings. Schools made use of the grants for classroom instruction; and support for decision teaching materials and supplies associated with making in schools. Figure 2.5 provides a compar- VNEN and organizing events with parental par- ison between VNEN and control group schools ticipation. Infrastructure related investments and from the point of view of principal invitations to school furniture are long-term spending items parents. The items in the lower panel serve as con- for which the grant was probably not considered trols which show parity between VNEN and con- to be sufficient. trol group schools. The upper panel of Figure 2.5 shows the variation in VNEN implementation on One interesting aspect of school finance in Viet- the dimension of community engagement, with nam is the prevalence of scholarships for children 94 VNEN schools not inviting parents even once from poor families and other disadvantaged situ- for livelihood or culture related training, and 149 ations. In primary education, there are no official doing so only once in the school year. fees, but parents and the school community typi- cally get together and determine the contribu- Figure 2.6 shows the comparison between VNEN tions they will collect. This direct local financing and control group schools on a composite mea- is in addition to expenses for salaries and basic sure of community engagement including nine education material including textbooks and more items including parent’s labor contribu- school supplies, which is covered by the provin- tions and discussion of financial matters. cial government. Table 2.2 indicates the variation in the pattern of scholarships. Overall, less than 10% of students receive scholarships, but the 2.1.5  Readiness for Self-Paced percentage is higher in Priority 2 & 3 provinces. Learning VNEN and control group are matched very well With the VNEN method centered around self- in Priority 1 provinces, but scholarship awardees paced ‘learning guides’, Grade 1 is dedicated appear to be slightly higher in VNEN schools in to get children up to speed in being able to Priority 2 & 3 provinces. Table 2.2  Scholarships Compared to Enrollment, 2015 Priority 1 Priority 2 & 3 Control Group VNEN Group Control Group VNEN Group Quantile Quantile Quantile Quantile 0.25 0.50 0.75 0.25 0.50 0.75 0.25 0.50 0.75 0.25 0.50 0.75 Students 137 243 372 124 237 372 198 281 398 139 329 505 Scholarships 1 7 22 1 6 19 1 8 30 2 10 30 % Scholarships 0.69 2.75 8.96 0.62 2.78 7.61 0.25 2.63 9.52 0.56 3.56 11.55 Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 25 School Leadership Figure 2.5  Principal Report of Parent Participation, 2015   (a) Livelihood skill teaching (b) Invite to classroom   (c) Inform about rules (d) Inform about achievements Source: VNEN Impact Evaluation study primary data, Authors calculations. read and follow the instructions. Figure 2.7 The skill ‘Read a sentence’ on average goes shows for all schools the principal’s opinion beyond Grade 1 (blue line), with VNEN and about the grade by which at least 75% of the control groups not shown separately as there children at the school acquire a certain skill. were no differences. 26 www.worldbank.org Figure 2.6  Community Engagement Measure   (a) Priority 1 provinces (b) Priority 2 & Priority 3 provinces Source: VNEN Impact Evaluation study primary data, Authors calculations. Figure 2.7  Readiness for VNEN (per grade level) Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 27 School Leadership 2.2  Contextual Factors ‘Positive approach’ comes from WVS items about a positive approach to life, good health, trust Influencing Implementation and a belief that people have choices. ‘Informa- and Results tion junkie’ comes from a question about the number of news sources a person accesses fre- 2.2.1  Principal’s Values and Beliefs quently, including print media, television, and The first contextual factor is based on a sub- the Internet. Finally. ‘Confucian heritage values’ set of questions from the World Values Survey comes from aspirations regarding being rich (WVS)3. The WVS includes a detailed set of ques- and successful, doing good for society, behaving tions asking people about their personal beliefs. properly, and following tradition. Along all four We generated four constructs from a subset measures, one does not find any appreciable dif- of WVS questions. ‘Political operative’ is com- ference between the VNEN and control group prised of questions about the importance to schools (Figure 2.8). the person of work, leisure, politics, and pride. Figure 2.8  Principal Values and Beliefs (average 2013–2015)   (a) Political operative (b) Positive approach   (c) Information junkie (d) Confucian heritage values Source: VNEN Impact Evaluation study primary data, Authors calculations. 28 www.worldbank.org 2.2.2  Principal’s Political Skills Figure 2.9 indicates an even distribution of Values and beliefs of a principal may affect their political skills across VNEN and control group motivation and interest in implementing VNEN principals. In the case of the VNEN program, it is successfully. A key aspect of leadership is the feasible that principals with higher levels of polit- ability to get things done with and through peo- ical skills would be better able to implement the ple, which depends on political skills.4 The four program and obtain superior performance from dimensions of political skills shown in Figure 2.9, teachers, parents, and students. It is also possible together are defined by Ferris, Treadway, et al., that a principal’s political skills may be used to 2005 as ‘the ability to understand others at work not implement VNEN very well, if the principal and to use that knowledge to influence others to has beliefs against the program—political skills act in ways that enhance one’s personal or orga- could be a double-edged sword. nizational objectives’. Figure 2.9  Principal Political Skills   (a) Social astuteness (b) Interpersonal influence   (c) Networking ability (d) Apparent sincerity Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 29 School Leadership The four dimensions of political skills are under- 2.2.3  Principal’s Innovative stood as follows: Social astuteness refers to the Attitude powers of observation, awareness, and under- Since VNEN represents a fundamental innova- standing in social contexts. According to Ferris, tion in teaching and learning, it is important to Davidson, and Perrewe, 2005, individuals with be able to calibrate, beyond the principal’s beliefs high levels of social astuteness ‘comprehend and political skills, about innovativeness as a per- social interactions and in social settings they sonality trait or behavioral habit.5 accurately interpret their own behavior as well as that of others’. The ability to have a high self- As with the earlier graphs on values and politi- awareness and empathy for others likely has cal skills, the left-hand side density estimate biological roots, with some individuals having depicted in Figure 2.10 indicates a parity higher capabilities through a combination of between VNEN and control group principals. The nature and nurture (Iacoboni, 2009). right-hand side of Figure 2.10 shows color coded correlation coefficients between the nine con- Interpersonal influence refers to the ability to structs measured for principals. There are only a understand others and oneself to get other few correlations that are statistically significant people to willingly and pleasantly follow sug- between the political skill measures. Further gestions and guidance. Networking ability refers research would be needed into these constructs to the development and use of networks of dif- to understand fully their application to principal ferent stakeholders, in this case of teachers, par- behavior and school performance. ents, district, provincial, and national officers and external experts. Finally, apparent sincerity refers to the impression that other people have about 2.2.4  Principal’s Active the individual’s sincerity—for instance, when Engagement on the Job the principal greets the invited parent with a We sought to measure the activity level of the smile and inquires about the harvest that year, principal as a contextual variable related to imple- the parent should perceive that the principal is mentation and performance (Figure 2.11). Three genuinely interested in the answer. groups of activities were considered—related to Figure 2.10  Innovation and Correlations   (a) Principal innovativeness (b) Correlations Source: VNEN Impact Evaluation study primary data, Authors calculations. 30 www.worldbank.org Figure 2.11  Principal as Busy as a Bee—Every Semester, 2015 Meeting Observing Meeting Meeting Observing Meeting teachers classes officials teachers classes officials 10 times 15 times 9 times 9 times 15 times 8 times Meeting Visiting Meeting Visiting Travelling parents teachers at Travelling parents teachers at 4 times 4 times home 12 times 4 times 3 times home 10 times Visiting Training Other Visiting Training Other students at event events students at event events home 7 times 5 times 3 times home 6 times 5 times 3 times (a) VNEN group (b) Control group Source: VNEN Impact Evaluation study primary data, Authors calculations. teachers (shown in dark blue); activities with par- 2.2.5  School Size, Infrastructure ents (in light blue); and administrative activities and Facilities (in green). There were no statistically significant The average school sizes were about 240 stu- differences between VNEN and control groups. dents in Priority 1 provinces and 300 students By the third year of the program, it might have in Priority 2 & 3 provinces. This works out to been expected that VNEN principals would an average of about 50 to 60 students in each report higher levels of activities with regard to Grade. Table 2.3 shows the enrollment by grade, observing classes or meeting with parents, but together with female enrollment and ethnic no such evidence was seen in the data. minority enrollment. This table indicates the Table 2.3  Distribution by Gender and Ethnic Minority, 2013 Priority Control Group VNEN Group Total Female Ethnic E. Fem. Total Female Ethnic E. Fem. Priority 1 Grade 1 57 27 16 7 56 27 21 10 Grade 2 55 26 15 7 52 25 19 9 Grade 3 58 28 17 8 54 26 21 10 Grade 4 61 29 19 9 54 26 22 10 Grade 5 62 30 18 9 55 27 21 10 Priority 2 & 3 Grade 1 72 35 11 6 68 33 10  5 Grade 2 66 32 11 5 64 31 10  5 Grade 3 64 30 10 5 65 31  9  4 Grade 4 63 31 10 5 65 31  9  5 Grade 5 66 32 10 5 66 33  9  4 Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 31 School Leadership Figure 2.12  School Wealth Index, 2013   (a) Item response theory map (b) Kernel density school wealth index Source: VNEN Impact Evaluation study primary data, Authors calculations. gender parity that exists in Vietnamese primary wealth index. Items like a digital video camera schools in poor as well as less poor provinces. and computer for teachers were rare, with pho- There do appear to be marginally less than 50% tocopiers and sports equipment a bit less rare. female students, but it is not possible to deduce Almost all schools had libraries and at least one a statistically significant effect without accurate television set. information about gender and age population figures. The table clearly shows the absence of a Finally, to complete the examination of school major problem of repetition and dropout during facilities, Table 2.4 presents some comparative primary school. The smaller number of students numbers. With the primarily rural or semi-urban in Grade 1 as compared to Grade 5 in Priority 1 schools that constituted the sample, one can schools likely reflects a demographic transition see that the schools have a land area averaging with declining fertility. The lower panel does not between 6,000 to 8,000 square meters, with many reflect a similar trend likely due to in-migration trees—the 25th percentile mark had 20 trees. to the wealthier, relatively more urban provinces. Students had about 2 square meters of space in the classroom, which is very important for VNEN We also generated an index of school wealth schools because of the emphasis on movement based on the facilities that the school possessed. and flexibility of seating opportunities. Vietnam’s schools are well equipped in a com- parative international context (Parandekar and Sedmik, 2016). Apart from textbooks and learn- 2.3  Estimation of Program ing aids that are available for all children, quite a Impact number of schools also possess electronic equip- ment such as projectors. A school wealth index 2.3.1  Traditional Mindset about was constructed that shows parity between Schooling VNEN and control group schools (Figure 2.12). In order for the VNEN reform or the teaching Figure 2.12 (a) shows the Item Response Theory philosophy that VNEN represents to be success- map that is useful to understanding the school ful, principals need to be convinced about the 32 www.worldbank.org Table 2.4  School Facilities Compared Priority 1 Priority 2 & 3 Control Group VNEN Group Control Group VNEN Group Quantile Quantile Quantile Quantile 0.25 0.50 0.75 0.25 0.50 0.75 0.25 0.50 0.75 0.25 0.50 0.75 School wealth –0.88 0.06 0.66 –0.88 0.06 0.36 –0.56 0.06 0.98 –0.56 0.36 0.98 Distance 6.00 10.00 17.00 5.00 10.00 20.00 4.00 9.00 15.00 3.75 7.50 13.50 Land area 4,000 5,800 9,080 3,906 6,219 8,399 4,200 6,336 10,005 4,937 7,587 10,049 Number of trees 20 35 60 21 40 90 20 38 65 24 41 84 Number of 137 243 372 124 237 372 198 281 398 239 329 505 students Area per student 1.69 2.38 4.35 1.72 2.56 4.35 1.52 2.04 3.23 1.67 2.13 2.96 Class student ratio 0.04 0.06 0.10 0.04 0.06 0.10 0.04 0.05 0.06 0.03 0.04 0.06 Number of teachers 11 16 23 12 17 23 14 19 24 15 18 26 Teacher student 0.05 0.07 0.09 0.05 0.07 0.10 0.05 0.06 0.07 0.05 0.06 0.07 ratio Source: VNEN Impact Evaluation study primary data, Authors calculations. need for change. Rogers, 2010 begins his book statement that ‘The traditional teaching model with a timeless quotation from the year 1513 is working fine. There is no need for change.’ attributed to Machiavelli ‘Whenever his enemies The answers are reproduced in the panel of Fig- have the ability to attack the innovator, they do ure 2.13. The percentage figures along the left- so with the passion of partisans, while the oth- hand side of each element of the panel indicate ers defend him sluggishly, so that the innova- the total percentage who disagreed with the tor and his party alike are vulnerable’. Ellsworth, statement; the figures on the right-hand side 2000 and Hargreaves, 2007 provide book length indicate the percentage who agreed with the treatments reviewing the copious literature on statement. The number in the middle shows the change in education or perhaps on why there is percentage of people who were neutral. so little change in education. It is clear from this literature that reform will not take place unless Figure 2.13 shows that the majority of principals a majority of the key stakeholders are in favor disagree with the statement, which implies that of change to begin with, or unless a majority they are in favor of change. The majority is higher become persuaded about the benefit of change. (75%) for VNEN schools, which is healthy consid- ering that VNEN school principals have actual Changing the mindset about traditional educa- experience with the model so their continued tion may seem to be a key intermediate goal of rejection of the traditional model augurs well the VNEN program. As a way to determine the for the future of educational reform in Vietnam. success in achieving this goal, in each year of However, the right-hand panel of Figure 2.13 the survey, principals were asked to agree or a may also indicate a cause for concern. The per- disagree on a seven-point Likert scale to the centage of control group principals who disagree Enhancing School Quality in Vietnam through Participative and collaborative Learning 33 School Leadership Figure 2.13  Principal Attitude about Traditional Model (a) VNEN principals (b) Control group principals Source: VNEN Impact Evaluation study primary data, Authors calculations. with the statement can be seen to be declining how influential they are in affecting change. each year. Conversely, 25% of control group Prin- Each year of the survey, principals were asked cipals who agreed in 2013 that the traditional to rate how influential they were on each set model was working fine and there was no need of decisions, reproduced in Figure 2.14 (a). The for change increased to 33% by 2015. A majority responses were used to generate a Rasch index still want change, but the trend may be a cause for influence, following the same practice as for for concern. the other composite indices used in this study. An interesting phenomenon can be observed from Figure 2.14 (b), which shows the influence self- 2.3.2  Principal’s Self-Belief rating for 2015 as compared to 2014. The index about Influence is defined so as to have a zero mean and a stan- A second important impact that we measured for dard deviation of 1. Figure 2.14 (b) shows that the principals concerns their own self-belief about slope is higher for VNEN school principals. 34 www.worldbank.org Figure 2.14  Principal Self-Perception of Influence a. Initial assignment of teachers to a particular school b. Regular rotation of teachers to a particular school c. Dismissing teachers d. Evaluating teacher performance e. Setting teacher salaries f. Selecting teachers for training g. Choosing teaching methods to use h. Developing teaching materials i. Adapting curriculum to local conditions j. Determining working hours of teachers k. Determining class size l. Deciding admission for students m. Setting standards for student promotion n. Evaluating students' periodical performance o. Shutting down a school p. Adding new grades to or removing grades from existing school q. Setting the amount of financial contribution from parents/students r. Deciding which students are exempted from financial contribution s. Deciding on the construction of school facilities t. Maintaining and rehabilitating facilities u. Deciding on how to spend school funds v. Scheduling meetings with community   (a) List of decisions (b) Movement from 2014 to 2015 Source: VNEN Impact Evaluation study primary data, Authors calculations. Table 2.5  Impact of VNEN on Principal Self-Perception of Influence Dependent Variable msr_p202_15 (Control Group) (VNEN Group) msr_p202_14 0.248*** 0.522*** (0.047) (0.056) School wealth 0.138*** −0.007 (0.051) (0.059) Percentage scholarships 0.117 −0.126 (0.238) (0.297) Distance 0.0001 0.006 (0.005) (0.005) Total students 0.0001 0.0001 (0.0002) (0.0003) Constant −0.082 −0.068 (0.110) (0.134) Observations 322 325 R 2 0.115 0.217 Adjusted R2 0.101 0.205 Residual std. error 0.859 (df = 316) 0.972 (df = 319) F statistic 8.230*** (df = 5; 316) 17.728*** (df = 5; 319) Note: *p < 0.1; **p < 0.05; ***p < 0.01 Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 35 School Leadership Table 2.5 shows that the coefficient of the schools might indicate an important phenom- principal’s measure of influence in 2014 when enon related to the reform. In diverse social con- regressed on the principal’s measure of influ- texts, change is held back because agents are not ence for 2015 as the dependent variable was aware themselves of the power they wield, given twice as high for VNEN schools as compared to existing norms and regulations, to bring about control group schools. The regressions included change. When change is implemented without four other control variables—the school wealth first changing the regulations, people sometimes index, the percentage of students on scholar- find to their own surprise that transformation has ships, the distance of the school from the district always been feasible, and the magnitude of the center, and the total number of students at the transformation may be even greater when for- school. None of the control variables were sta- mal regulations are introduced. It is possible that tistically significant for VNEN schools. The school Figure 2.14 and Table 2.5 provide a glimpse into wealth index was statistically significant for con- the unfolding of this process. When the system trol group schools, with higher levels of school wide fundamental and comprehensive educa- wealth indicating higher levels of influence rat- tion reform is rolled out, at least some principals ings for principals. The higher slope for VNEN may be more than ready. 36 www.worldbank.org TakeAways: School Leadership This chapter examined the VNEN program from the regarding being rich and successful, doing good point of view of the school principal. The chapter pro- for society, behaving properly, and following vided evidence in three parts—the implementation of tradition. This study has provided constructs the VNEN program; the contextual factors regarding regarding principal political skills and inno- the school principal and the school itself; and finally vativeness. All these measures show an equal the impact of the program in terms of the change in distribution across VNEN and control group the principals’ mindsets and in their self-belief. schools, supporting the assertion that the samples were balanced well and can be safely SCHOOL AUTONOMY used for drawing causal inference. • There is a link between program implementation TRADITIONAL MINDSET and school autonomy. There was a higher level of autonomy in VNEN schools as evidenced by • Principals, the frontline leaders of the reform, more frequent instances of local involvement in have a strong belief about the need to reform decision making. Key areas where local school the traditional model, and this belief is posi- communities exercised autonomy included con- tively related with their actual program expe- vocation of community meetings, student eval- rience. When we asked principals about their uation, and rehabilitation of school facilities. opinion regarding the statement that “the tra- ditional model is working fine, there is no need IMPLEMENTATION for change,” about 75% of VNEN principals dis- agreed with the statement throughout the three • There was a variation in the implementation years of the survey. 73% of control group prin- of the VNEN program across schools from the cipals were of the same opinion, but this group perspective of the school principal. With regard has declined to 63% in two years. So those who to pedagogical activities, there was a signifi- experienced VNEN for real continue to believe in cant jump in implementation between 2013 and the need for reform; those who may only have 2014 and a smaller increase from 2014 to 2015. hearsay knowledge about VNEN may develop One area that appears to be disappointing is the a stronger belief in the status quo of the tradi- low level of community engagement in VNEN tional model. schools—in quite a few VNEN schools, there was hardly any engagement with parents. Principals who are successful in implementing the reform are among the most important change CONTEXTUAL EQUALITY agents for the future success of the reform—the • Principals show variation with regard to their change in their mindset and their ability to influ- value orientation, including the feature defined ence others has been a key factor of the reform as Confucian heritage values, based on beliefs effort. Enhancing School Quality in Vietnam through Participative and collaborative Learning 37 School Leadership Endnotes 4. Pfeffer, 1981; Ferris, Treadway, et al., 2005; Ferris, Davidson, and Perrewe, 2005. We use the standard 18 item ‘political skills inventory’ (PSI). For applications 1. Cheng, 1994; Hallinger and Heck, 1998; Teddlie and in education and to principal leadership, see Doug- Reynolds, 2000; Creemers et al., 2003. las and Ammeter, 2004; Brosky, 2011; and Fidan and 2. Marzano, Waters, and McNulty, 2005; Oyetunji, 2006; Balcı, 2016. Beatriz, Deborah, and Hunter, 2008; Parandekar, 5. Hurt, Joseph, and Cook, 1977. Oliveira, and Amorim, 2008; Hallinger and Huber, 2012; Parandekar, 2014. 3. Inglehart, Basanez, and Moreno, 1998; Dakhli and De Clercq, 2004; Dalton and Ong, 2005; Minkov and Hof- stede, 2010. 38 www.worldbank.org Teacher Beliefs, Attitudes, and Training 3 This chapter presents findings in three sections. of teachers—it seeks for teachers to behave in a In this first section, we provide a profile of teach- fundamentally different way. With decades and ers to answer the question ‘who are the teach- more of cultural inertia at their back, VNEN seeks ers?’. This includes demographics, salary and to transform teachers from being all powerful satisfaction levels of the teachers and their train- providers of knowledge to become facilitators of ing. The next section deals with teachers’ beliefs. inquiry. For such a transformation to be possible, It covers beliefs regarding educational objectives teachers need to be able to transform their very and pedagogy as well as measures of teacher beliefs about education and learn to teach in a innovativeness and values. Finally a third section new way. Our investigation begins by exploring provides evidence of the program’s impact on the demographic profile of teachers. teachers, examining their self-efficacy and per- ceptions about influence in decision making. Table 3.1 provides a summary demographic profile. A word is in order about the sample of teachers. The sampled schools had a median of 3.1  Who Are the Teachers? 18 teachers, and in the first year of the survey, a A Profile random sample of teachers were surveyed. With 20 students also being selected randomly for 3.1.1 Demographics testing in the first year and only half of those sur- Teachers and students make up the two sides veyed through a questionnaire, only one-third of the pedagogical equation, and the effective- of the students could be matched to the teacher ness of any reform depends critically on the who taught them in that year. The teacher sam- enthusiasm and uptake from teachers. A com- pling strategy was revised the following year to prehensive reform like VNEN is very demanding be able to match teachers to students. Though Table 3.1  Profile of Surveyed Teachers Priority 1 Priority 2 & 3 2013 2014 2015 2013 2014 2015 Female gender 82% 78% 77% 85% 79% 80% Age in years 39.8 39.2 40.2 40.3 39.0 41.12 Percentage ethnic minority 24% 22% 22% 6% 5% 4% Teacher home possessions index 64.75 65.88 67.76 68.78 70.10 71.55 Percentage with college/university degree 47% 57% 64% 47% 57% 62% Years of teaching experience 17.6 17.0 18.0 18.3 16.8 18.9 Tenure at school 11.3 10.5 11.7 12.3 10.7 12.2 Source: VNEN Impact Evaluation study primary data, Authors calculations. 39 Teacher Beliefs, Attitudes, and Training some schools in the sample do have the same 0.75 million. There are no statistically significant teacher who move up the grades as the students differences across control and VNEN groups. Pri- are promoted, typically the teacher is different. ority 1 province salaries are lower by about VND As a result, though there are some teachers who 100,000 per month, likely reflecting differences were surveyed in each of the three years, the in cost of living. Using PPP exchange rate of VND sample composition is different in each year. 7,600 per USD (World Development Indicators, This is the reason that Table 3.1 presents demo- 2015), the average monthly salary was slightly graphic data separately for the three years. less than US$500 per month, about the same as the per capita income of Vietnam. The salary fig- Roughly 80% of the teachers are female, with an ures are rough generalizations, but they convey average age of 40 years and teaching experience the fact that the surveyed teachers are not poor of about 17 years. Slightly less than 60% of the nor are they rich. teachers have undergraduate degrees and the average tenure at the current school is about Teachers were asked about their level of satis- 11 years. Of the teachers in Priority 1 provinces, faction on a five-point Likert scale, ranging from 20% are from ethnic minorities, and the propor- ‘very satisfied’ to ‘very dissatisfied’, on five work tion of ethnic minority teachers is much smaller related dimensions: salary; appreciation from the at about 5% in Priority 2 & 3 provinces. Nearly parents of the students they serve; social status in 33% of the students in Priority 1 provinces and the community; work environment; and the stu- about 13% of the students in Priority 2 & 3 prov- dent study results. A Rasch index of satisfaction inces are from ethnic minorities. Considering the was generated from the responses and Table 3.2 lag in educational attainment of ethnic minori- presents the item measures compared for the ties at higher education levels, the proportion control and VNEN groups for the year 2015. The of ethnic minority teachers appears to be fairly results are reproduced in Figure 3.1 for compar- healthy. The teacher home possession index ing satisfaction levels across VNEN and control comes from a count of 22 home possessions. groups. There was no statistically significant difference 3.1.2  Job Satisfaction and Salary between the two groups in 2013, but the 2015 Teachers surveyed in the study received a sample shows a small statistically significant monthly salary averaging Vietname dong (VND) advantage for VNEN teachers. By construction 3.46 million in 2013, going up to VND 3.64 mil- the index has a mean of zero and standard devi- lion in 2015. The standard deviation is about VND ation of 1; the mean values for 2013 and 2015 Table 3.2  Satisfaction of Teachers, 2015 Control Group VNEN Group Index Index Item Measure Item Measure a. Salary 2.05 a. Salary 1.91 e. Student study result –0.26 e. Student study result 0.16 Social or community status c.  –0.58 c. Social or community status –0.42 d. Work environment –0.64 b. Parents appreciation –0.78 b. Parent appreciation –1.09 d. Work environment –0.88 Source: VNEN Impact Evaluation study primary data, Authors calculations. 40 www.worldbank.org Figure 3.1  VNEN Teachers with Higher Satisfaction (a) 2013 comparison (b) 2015 comparison Source: VNEN Impact Evaluation study primary data, Authors calculations. for the two groups are –0.07 and +0.07. Table 3.2 recognition of ‘excellent teacher’ awards at shows that the ‘most difficult’ satisfaction was for school, provincial, and national levels. Table 3.3 salary and the ‘least difficult’ satisfaction was for shows a progression with passing years toward parent appreciation and work environment. An awards at higher levels—between 2013 and interesting result is that the satisfaction measure 2015 there is a decline in the first column from is easier or higher for the work environment for 332 to 128 who did not receive any award and the VNEN group while parent appreciation is an increase from 121 to 281 for provincial level higher for the control group. awards. At the provincial and national levels, more VNEN teachers got awards. Even in 2013, Another aspect related to compensation and sat- the first survey year at which time VNEN had isfaction is the practice in Vietnam of providing been in implementation for 1 year, 9% of VNEN Table 3.3  Excellent Teacher Awards No School District Province National Year Group Total Award Level Level Level Level 2013 All 1770 332 567 748 121 2 Control 100% 20.7% 32.3% 41.9% 5.0% — VNEN 100% 16.9% 31.8% 42.6% 8.6% — 2014 All 1847 255 515 873 194 10 Control 100% 14.2% 28.3% 48.6% 8.3% — VNEN 100% 13.4% 27.5% 45.9% 12.7% — 2015 All 1768 128 410 937 281 12 Control 100% 7.5% 25.5% 53.4% 13.1% — VNEN 100% 7.0% 20.9% 52.6% 18.7% 1% Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 41 Teacher Beliefs, Attitudes, and Training teachers received provincial level awards as com- Three aspects of the training have an important pared to 5% for control group teachers. By 2015, bearing on the program impact: (i) the training 19% of VNEN teachers were provincial award win- followed the philosophy that for an effective ners as compared to 13% of control group teach- training program the training event itself is only a ers. Eight out of twelve national award winners small proportion of time.1 Accordingly, the teach- in 2015 were from the VNEN group. Awards can ers received all training materials at least 2 weeks be very useful as rewards for good performance before the training and came to the training pre- to motivate further effort from all teachers, as pared; (ii) the training event consisted of practi- well as to provide recognition of efforts already cal implementation of VNEN classrooms rather made. Further research needs to capture more than hearing lectures about VNEN—and train- information about the role of awards in schools ees discussed in a facilitated way the behavior in Vietnam. they were meant to replicate in classrooms; and (iii) the focus on teachers themselves as trainers meant that some of the teachers got multiple 3.1.3 Training opportunities to take part in the training within Training of teachers to be able to teach using the the school year. VNEN pedagogical method was a crucial input of the program. Given the fact that the VNEN While it was not mandatory for the schools that pedagogical model represented a dramatic were not implementing the VNEN model to change for most teachers, providing adequate attend VNEN training—presumably the provin- training in the limited time period available was cial education department would organize their a big challenge. All primary school teachers in regular annual training—it appears that many of Vietnam receive two weeks of summer training, the control group teachers actually attended the which constituted the initial VNEN training for VNEN training. We asked all surveyed teachers most teachers. In order to prevent a loss of fidel- each year whether or not they had received any ity that comes from the usual ‘training of train- VNEN training. All teachers from the VNEN group ers’ model, teachers were involved at all levels answered in the affirmative. The percentage of of training. The prior small-scale VNEN pilot in control group teachers who answered ‘yes’ went six provinces (see section 1.1.2) had generated up from 37% in 2013 to 71% in 2015. the benefit of having a few teachers who already had experience with the EN methodology. The The findings shown in Table 3.4 have an impor- most skilled and motivated among these teach- tant implication for this impact evaluation study. ers were included as part of the group of trainers. It shows a near parity in the provision of training Training was held in overlapping cascades, and between VNEN and control groups, which by itself was required to reach tens of thousands of teach- is not surprising as two weeks or roughly 10 days ers within a few weeks in the summer. of summer training is mandated for all teachers. Table 3.4  Teacher Training Received, 2012 to 2015 Control Group VNEN Group Year 2012 2013 2014 2015 2012 2013 2014 2015 Number of days 8.93 9.80 8.10 9.42 9.44 10.9 9.35 8.69 Number of instances* 1.18 1.26 1.35 1.36 1.11  1.39 1.32 1.31 * Instances could be national, provincial, and school levels. Source: VNEN Impact Evaluation study primary data, Authors calculations. 42 www.worldbank.org The variation in the table from 10 days likely average of three to four school cluster meetings reflects the practices at different training events per year. This number should have been closer to with regard to the use of the fifth day in each 15 or 16 as they were supposed to be held once week, which may have been used for socializing, every two weeks. Formally scheduled classroom sometimes explicitly recognized as part of the observations and discussions appear to also have training. However, the high proportion of teach- taken place in the control group schools. The last ers from control group schools who took VNEN row of Table 3.5 shows that VNEN school teachers training indicates that even control group teach- did consult more often with teachers from other ers are likely to have developed a good familiar- schools. ity with VNEN teaching and learning methods. This would have helped those schools who were not program schools to become ‘voluntary’ VNEN 3.1.4  Time Use schools. But it would also help teachers adopt Teachers from VNEN schools appear to have part of the VNEN practices that they wanted to worked about one hour more per week than adopt even though their school may not have teachers from control group schools, with the institutionally adopted the VNEN model. extra time mainly coming from time in the class- room. There is a noticeable jump in average As part of the training philosophy of VNEN, in classroom hours from 2014 to 2015 for both con- addition to the off-site two week training at the trol group and VNEN group schools. It is likely beginning of the school year, the program also that this difference reflects the parallel reform of included provisions for sustained professional increasing classroom hours in schools toward full development through the year. day schooling. Typically schools would increase the number of classes for one or two days of the Table 3.5 provides information regarding a week, including additional afternoon classes. The teacher’s exposure to this form of professional number of days with afternoon classes would development. The table indicates that teach- subsequently be increased to approach full day ers in VNEN schools reported taking part in an schooling. Table 3.5  Teacher Interaction and Cluster Training Control Group VNEN Group Year 2012 2013 2014 2012 2013 2014 Principal classroom observation of teacher 5.96 4.99 4.97 6.22 6.17 5.35 Same grade teacher observation of teacher 13.99 12.35 — 13.43 13.81 — Same school teacher observation of teacher 8.82 8.62 — 9.08 9.90 — Biweekly cluster training 0.11 0.30 0.44 2.53 3.60 2.97 Option chosen when faced with difficulty in teaching Research teaching guide book 68% 61% 79% 74% 68% 81% Exchange views with principal 40% 34% 45% 61% 49% 59% Exchange views with teachers at same school 89% 85% 94% 92% 93% 96% Search for information on the internet 57% 54% 74% 62% 59% 75% Exchange views with teacher in other school 40% 37% 48% 56% 47% 55% Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 43 Teacher Beliefs, Attitudes, and Training Table 3.6  Teacher Time Use, 2012 to 2015 Control Group VNEN Group Class Other Total Class Other Total Year Prep Classroom Work Working Prep Classroom Work Working 2013 12.65 24.25 3.11 40.01 11.42 27.37 2.60 41.38 2014 13.15 23.96 3.41 40.51 12.78 26.31 3.31 42.40 2015 13.63 30.03 4.55 48.21 12.79 31.96 4.23 48.98 Source: VNEN Impact Evaluation study primary data, Authors calculations. 3.2  Teachers’ Beliefs and then slightly more to 16% in 2015. Interest- ingly, the percentage of teachers who agreed Teachers’ behaviors regarding educational reform with the traditional model remained about the are guided by their beliefs and their prepared- same. For some reason there was a lowering of ness for reform. We explored three aspects—the the perceived need for change among control beliefs of teachers regarding educational objec- group teachers. Among VNEN group teachers tives; their beliefs regarding the importance of (right-hand side panel of Figure 3.2), a converse certain pedagogical activities; and a general pattern is observed—teachers who had agreed measure of innovativeness. that the traditional model was good and there was no need for change declined from 25% in 2013 to 19% in 2014 and 15% in 2015. It is pos- 3.2.1  Beliefs Regarding Educational sible that the control group teachers were influ- Objectives enced by a strong move on the part of traditional We examined the change across the three years minded opinion leaders in Vietnam to criticize of the questions asked of school principals about the reform. The VNEN group teachers may have (i) the belief in the need for change and (ii) the had a closer knowledge from practical experi- level of familiarity with the VNEN model. Teachers ence of the VNEN model and hence understood were now asked their opinion on these questions. and appreciated the need for change better. To the extent that teachers believe that the tradi- Further research would be required to have a tional model is working fine, without any need for definitive conclusion about the reasons for the changes, the less will there be ownership of the possible trend identified in this study. reform. Figure 3.2 shows the responses to these two Likert scaled questions. In 2013, about 70% Figure 3.2 also shows the responses from the of both control group and VNEN group teachers question of familiarity with the VNEN model. The disagreed with the statement that the traditional specific statement was “I am totally familiar with model was working fine. However, there is a pos- the VNEN model.” The question indicates that sibly problematic trend with regard to the atti- the first year’s implementation of VNEN deliv- tude from control group teachers. ered the biggest impact on the familiarity, with 83% of VNEN group teachers agreeing with the Figure 3.2 indicates that the percentage of teach- statement. The percentage strongly agreeing is ers from control group schools who disagreed shown in dark blue, and it increased for familiar- (red areas of graph) declined from 71% in 2013 ity for VNEN teachers from 11% in 2013 to 17% in to 62% in 2014 and further to 56% in 2015. The 2014, and was 16% in 2015. The phenomenon of percentage of control group teachers who were a sizeable impact in the first year of the study (the neutral jumped from 2% in 2013 to 14% in 2014 second year of the program) and a stabilization 44 www.worldbank.org Figure 3.2  Attitude toward Traditional Model and Familiarity with VNEN (a) 2013: Control group (b) 2013: VNEN group (c) 2014: Control group (d) 2014: VNEN group (continued) 45 Teacher Beliefs, Attitudes, and Training Figure 3.2  Attitude toward Traditional Model and Familiarity with VNEN­ —Continued (e) 2015: Control group (f) 2015: VNEN group Source: VNEN Impact Evaluation study primary data, Authors calculations. in the following year is a pattern observed often survey. The interesting fact to be observed from in this study. Figure 3.3 is the consistency of results across both VNEN and control groups and across the two As teacher beliefs are such a vital part of an edu- years. The elements in Quadrant I (text in red font) cational reform, we also sought to understand indicate that the most important educational teacher beliefs about educational objectives objectives for both VNEN and control group more deeply by asking them to rate or rank items teachers were the so-called non-cognitive skills in a set of educational objectives. We asked the of ethical strength, cooperative spirit, problem teachers also to rate or rank the same items in solving, and communication. When the objec- terms of their implementation by the teachers. tive of self-study quality was added in 2015,it This so called ‘importance-performance analysis’ also figures prominently in Quadrant I. There are originates from literature in marketing.2 hardly any entries in Quadrant IV (high impor- tance and low implementation). Interestingly, Figure 3.3 provides the results of the importance- the item ‘academic excellence’ figures in Quad- performance analysis for the years 2014 and 2015 rant II, surprisingly behind the non-cognitive skills when the section was included in the teacher in the horizontal importance dimension. The 46 www.worldbank.org Figure 3.3  Educational Objectives and Implementation Compared (a) 2014: Control group (b) 2014: VNEN group (c) 2015: Control group (d) 2015: VNEN group Source: VNEN Impact Evaluation study primary data, Authors calculations. elements in Quadrant III are somewhat sobering or teaching methods in the classroom. In a vari- with regard to understanding teachers’ beliefs ation of the importance-performance analysis, regarding educational objectives. “Competency we also asked teachers about how challeng- for livelihood” is rated the lowest on impor- ing they found each pedagogical activity. In tance and implementation. Similarly, real-world this instance we analyze the two dimensions knowledge, and civic preparation with regard to separately. Figure 3.4 shows the results from democratic citizenship and leadership qualities teachers’ rankings of the importance of eight do not appear high in teacher ranking. It should pedagogical tasks. Each panel in Figure 3.4 is a be noted that these are relative rankings; in abso- ‘heat map’ that shows the percentage of teach- lute terms almost everything is rated by teach- ers who provided the particular rank for each ers as being important and being implemented. of the pedagogical activities arrayed vertically Further research is required to investigate if the (Warnes et al., 2013). The cells color coded with findings are valid and accurate representations of red indicate the lowest percentages and green teacher’s beliefs and actions. indicates the highest percentages, while yellow indicates percentages in the middle, for the par- ticular distribution of each heat map. Each row 3.2.2  Beliefs Regarding Pedagogy and each column of this particular set of heat We look closely at beliefs of teachers regarding maps adds up to 100. the importance of various pedagogical activities Enhancing School Quality in Vietnam through Participative and collaborative Learning 47 Teacher Beliefs, Attitudes, and Training Figure 3.4  Beliefs Regarding Importance of Pedagogical Activities (a) 2014: Control group (b) 2014: VNEN group (a) 2015: Control group (b) 2015: VNEN group Source: VNEN Impact Evaluation study primary data, Authors calculations. From the top left panel we can see that 25% of 2014 ranked student self-learning as the top control group teachers in 2014 ranked ‘Effective ranking pedagogical objective. lectures’ as the first ranked pedagogical objec- tive. This is not the cell with the highest of the The eight pedagogical objectives are listed in a first ranked items—which is perhaps surprising particular way. The first four are traditional peda- for control group teachers—30% of teachers gogical objectives, with titles shortened in the ranked ‘Student self-learning’ as first. Compared heat map for presentation clarity: give effective lec- with the VNEN group teachers for 2014 (top right- tures; ensure discipline in the classroom; strive hard hand panel)—only 7% of VNEN group teachers to make sure that students do not make mistakes; in 2014 ranked effective lectures as first in 2014. and repeat explanations to make sure that all stu- A much higher 46% of VNEN group teachers in dents understand. The next four are pedagogical 48 www.worldbank.org objectives attuned to the VNEN model: encour- In fact, the right-hand panels of Figure 3.4 match age student enquiry; provide chances for students the hypothetical pattern. Interestingly, even the to help other students; use more time for student’s left-hand panels for the control group show a self-learning and less time for instructions; and pattern similar to the one expected for the VNEN organize fun activities for student learning. group—low percentages in the northwest and southeast sections. With some exceptions that With this configuration of objectives and the are quite interesting in their own right, the con- color coding used, the null hypothesis would trol group teachers in aspirational terms would be of a block-diagonal structure of a 64 celled appear to share the importance of VNEN model matrix. The set of northwest cells would be elements. The change from 2014 to 2015 in Fig- expected to be green or green-yellow for the ure 3.4, especially for the VNEN group, is note- control group—representing the high ranks worthy. One can see that ‘effective lectures’ expected in this zone; conversely for the VNEN ranked last by 29% of VNEN group teachers is group teachers, we would expect the northwest marked last by 42% of VNEN teachers in 2015. A section to be red—for example, few VNEN group similar pattern is seen for high ranking of ‘student teachers should be saying that students not mak- helps others’ from 2014 to 2015. It would appear ing mistakes is a big priority. With the same logic, that in terms of beliefs about the importance of we would expect to see green and green-yellow pedagogical activities, the program was making for the control group in the southeast section good headway. of cells and the area being red for VNEN group teachers. In brief, the configuration expected We now turn to examine the results in Figure 3.6 would be as shown in Figure 3.5 for the control with regard to teacher ranking of how challeng- and VNEN groups—with green and yellow block ing they found it was to apply the pedagogical diagonal structure, and some yellow cells here practice. The colors have been changed for Fig- and there to indicate percentages not too high ure 3.6, with maroon or brown colors represent- and not too low. ing low percentage cells, blue color representing Figure 3.5  Hypothetical Block-Diagonal Pattern Expected for Heat Map (a) Control group (b) VNEN group Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 49 Teacher Beliefs, Attitudes, and Training Figure 3.6  Challenges Regarding Pedagogical Activities (a) 2014: Control group (b) 2014: VNEN group (c) 2015: Control group (d) 2015: VNEN group Source: VNEN Impact Evaluation study primary data, Authors calculations. high percentages and cream colors representing quadrant to increase, but these results show that percentage numbers that are not too high and within one year the improvement may only be not too low. marginal. The reader can explore any particu- lar thread through the panels; paucity of space One does not observe even an imperfect block- prevents a comprehensive exposition here. For diagonal structure. Rather, Figure 3.6 indicates instance, ‘encouraging student enquiry’ appears a similar pattern in all four panels. If we add up to be consistently among the top two ranked the 16 southeast cells for the VNEN groups, we challenges, with the cells in dark blue or light get a total of 181 in 2014, which goes up to 187 blue. Providing ‘repeated explanations’ and giv- in 2015. As VNEN teachers progressively gain ing ‘effective lectures’ likewise are rated the low- mastery, one would expect the numbers in this est in terms of challenges. 50 www.worldbank.org Figure 3.7  Innovativeness of Teachers (a) 2013 (b) 2014 (c) 2015 Source: VNEN Impact Evaluation study primary data, Authors calculations. 3.2.3 Innovativeness only do as well as the teacher expects him or her Using the same standard inventory due to Hurt, to do—if the teacher sets high expectations of Joseph, and Cook, 1977 for principals, we esti- the student, it is likely that the student will strive mated the overall innovativeness for teachers. As hard and make an effort to justify or live up to for principals, we did not find any discernible dif- the teacher’s expectations. The student’s effort ference between control group and VNEN group reaps its own rewards, completing, in a sense, teachers on this measure (Figure 3.7). the teacher’s prophecy about the student. An important issue is the formation of those teacher expectations regarding students. According to 3.3  Impact on Teachers Bandura, 1993, “Teachers’ beliefs in their personal efficacy to motivate and promote learning affect The section on beliefs of teachers did show some the types of learning environments they create movement or change across the program years. and the level of academic progress their students However, changes in beliefs are more often the achieve.” result of a re-evaluation of previous beliefs in light of tangible results. But what brings about Social learning is a cornerstone of the VNEN the results in the first place? In this section, we model, hence teacher self-efficacy becomes trace any evidence regarding impact on mea- even more important for VNEN. Essentially, self- sures of self-efficacy and perceived influence of efficacy works through feedback mechanisms, teachers. The reasoning is that as teachers’ self- and these mechanisms are modified in VNEN efficacy and perceived influence grow, they will classrooms because of network effects. In the be more confident and better able to implement traditional model, the relationship was between the VNEN model. the teacher and each individual student, to the extent that the teacher was able to carve out time for interacting with individual students. In 3.3.1 Self-Efficacy the VNEN model, there is a dynamic where the A substantive literature exists regarding self- group of four to five students sometimes ampli- efficacy and work-related performance, particu- fies and at other times dampens the feedback. larly in education and for teachers.3 Self-efficacy is defined as an individual’s judgment about how The self-efficacy measure, constructed from well or effectively they can perform certain tasks teacher responses regarding eight key tasks, or deal with situations. It is said that a student can does not vary greatly between control and VNEN Enhancing School Quality in Vietnam through Participative and collaborative Learning 51 Teacher Beliefs, Attitudes, and Training Figure 3.8  Self-Efficacy of Teachers (a) 2013 (b) 2014 (c) 2015 Source: VNEN Impact Evaluation study primary data, Authors calculations. group teachers (Figure 3.8). The measure is con- Table 3.7, included ones such as ’How much can structed so as to have a mean of zero and stan- you do to manage the class time effectively?’ and dard deviation of 1. In 2013, the mean for the ‘How much can you do to organize teaching in a VNEN group was +0.05 as compared to a mean way that your students can use the learned mate- of –0.05 for the control group; and in 2015 the rial in daily life effectively?’ values were –0.04 and +0.04 respectively; in 2014 there was no statistically significant difference. The items in Table 3.7 are arranged according to Insight into self-efficacy is better obtained by declining values of the first column which shows considering the composition of the self-efficacy the WMLE (Weighted Mean Likelihood Estimate) measure. Teachers responded to eight tasks on a Rasch item measure. The next two columns show nine-point scale with 1 meaning ‘I cannot at all’ the number (percentage) of surveyed teachers and 9 meaning ‘I am fully capable of doing it.’ The from the control and VNEN groups in 2015 who eight task statements, abbreviated for space in gave themselves very high ratings of 7, 8, or 9 on Table 3.7  Elements of Teacher Self-Efficacy Measure Choice 7, 8, 9 Choice 7, 8, 9 Item 2015 Control (n = 878) VNEN (n = 890) Getting students to produce original products 0.89 651 (74%) 732 (82%) Help develop creativity in students 0.57 735 (84%) 776 (87%) Use taught material in daily life 0.06 800 (91%) 811 (91%) Use teaching aids and materials effectively 0.05 775 (88%) 831 (93%) Provide reinforcements for positive behavior –0.13 799 (91%) 839 (94%) Communicate with problematic students –0.31 790 (90%) 837 (94%) Monitor performance improvement reliably –0.49 825 (94%) 827 (93%) Manage class time effectively –0.63 832 (95%) 834 (94%) Source: VNEN Impact Evaluation study primary data, Authors calculations. 52 www.worldbank.org each item. While the Bandura self-­ rating scales interesting that ‘managing class time effectively’ for teacher efficacy have been applied in multiple is rated as the easiest task by the teachers, with contexts, we are not aware of a previous applica- about 95% of the teachers rating themselves 7 tion in Vietnam. It is possible that in the Vietnam- or higher on the nine-point rating scale used in ese cultural context, teachers may have very high the study, with no difference for this item across internally and externally imposed expectations the control and VNEN groups. Overall, one does of performance. This expectation may render it observe an edge for VNEN teachers for five out of inconceivable or unacceptable that any teacher the eight items, with the biggest difference being may admit that she is less than perfectly capable for the first item—‘getting students to produce of performing any task expected of her. Admit- original products.’ ting a less than perfect ability as a normal part of professional development may not be the same as in other cultures where the self-efficacy scale 3.3.2 Influence has been applied. The methodological aspect With the VNEN model stressing aspects of col- requires further research; for this study we note laborative and participative learning related to in Table 3.7 the very high percentages of teach- the livelihood particular to each community, it is ers who gave themselves ratings of 7, 8, or 9 on important that teachers are empowered and feel the items. empowered. We measured the influence level that teachers had on decision making regarding Table 3.7 shows that relatively the most difficult key factors affecting teachers. Table 3.8 shows task encountered by Vietnamese primary school the eight kinds of decisions and the percentage teachers is getting students to produce original of teachers who answered that teachers were the products. The other two most difficult tasks (from primary influencers for the decision. Figure 3.9 the WMLE estimates), include developing creativ- shows the Rasch index measure of influence from ity in students and applying learning to activi- a four-point scale—‘not influential’; ‘somewhat ties in daily life; these are deficits that underpin influential’; ‘influential’; and ‘very influential’. the need for VNEN education reform. It is quite Table 3.8  Teachers as Primary Influencers in Decision Making, 2013 Control Group VNEN Group Item N = 864 N = 906 Number Percentage Number Percentage Choosing teaching methods to use 729 84.4% 766 84.5% Developing teaching materials 203 23.5% 203 22.4% Adapting curriculum to local conditions 221 25.6% 339 37.4% Selecting teachers for training 27 3.1% 29 3.2% Evaluating teacher performance 16 1.9% 27 3.0% Dismissing teachers 7 0.8% 9 1.0% Rotation of teachers to schools 2 0.2% 2 0.2% Initial school assignment of teachers 1 0.1% 2 0.2% Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 53 Teacher Beliefs, Attitudes, and Training Figure 3.9  Teachers Self-Perception of Influence (a) 2013 (b) 2014 (c) 2015 Source: VNEN Impact Evaluation study primary data, Authors calculations. The Rasch index measure of teacher perception of teachers reported being the primary influenc- of influence shows no difference between control ers on the choice of teaching methods. How- and VNEN groups in 2013. In 2014, there is a sta- ever, for the item ‘adapting curriculum to local tistically significant difference, with mean value conditions’, a higher percentage of VNEN teach- being +0.10 for the VNEN group, compared to ers (37% compared to 26%) reported having the –0.10 for the control group of teachers. By 2015 main influence, among groups including various the difference had narrowed down to +0.05 and levels of government authorities and the school –0.05. Table 3.8 shows that in 2013 both VNEN principal. and control groups, a high percentage (84–85%) 54 www.worldbank.org TakeAways: Teacher Beliefs, Attitudes, and Training This is the first of two chapters that deals with provide knowledge. All teachers appear to have teachers. In this chapter, we presented some been exposed to some VNEN training, even if details about the profile of teachers in terms of they were not teaching in VNEN schools. The their demographic background and training. We biweekly cluster training was not fully imple- next presented details about teacher beliefs mented in some schools. regarding the reform. Finally, we examined the evidence regarding the impact of the program on BELIEFS REGARDING PEDAGOGICAL teachers self-efficacy and perceptions of influence. OBJECTIVES DEMOGRAPHIC PROFILE • Teachers show a sharp understanding regard- ing the pedagogical objectives of the reform, • On an average, the teacher age of the sample and the VNEN teachers appear to have devel- covered in this study was about 40 years, the oped ownership of some of the objectives—for same for VNEN or control group schools. More instance student self-learning as being more than half of the teachers had a college or uni- important than giving effective lectures. How- versity degree and they had about 18 years of ever, some teachers find it difficult to imple- teaching experience, with 11 to 12 tenure at the ment the objectives. sampled school. About one in five teachers in Priority 1 provinces were ethnic minorities. IMPACT ON TEACHER SELF-EFFICACY SALARY AND SATISFACTION • Further research is required to test the validity of the Bandura self-efficacy scale in Vietnam, • The salary of teachers is close to the per capita but the results presented here show very high income of Vietnam, suggesting that teachers self-efficacy of the teachers. The two items are neither rich nor poor. Job satisfaction was which are relatively more difficult for teach- equally matched across VNEN and control group ers were “Getting students to produce original schools, with more VNEN teachers receiving products” and “Helping develop creativity in excellent teacher awards at the provincial level. students.” TRAINING • The most important issue to note about teacher Teachers appear to have sound conceptual under- training was that teachers received material standing of VNEN, but some teachers have faced prior to the training, and the training event itself difficulties in implementation—a reason for this used an actual emulation of the VNEN classroom difficulty may have been the sporadic implementa- with active discussion, rather than a lecture, to tion of the biweekly cluster meetings. Enhancing School Quality in Vietnam through Participative and collaborative Learning 55 Teacher Beliefs, Attitudes, and Training Endnotes useful survey of the literature using IPA in different fields. Anderson, Hsu, and Kinney, 2016 is a recent application in the field of education. 1. See Brinkerhoff, 2006 and Leimbach and Emde, 2011. 3. Albert Bandura is the great grandfather of this lit- Training is only useful when trainees are motivated erature, also termed as the social-cognitive theory and well prepared to absorb the training before the (Bandura, 1977). Bandura, 1993 describes the appli- event, and feedback loops exist for the application of cation of self-efficacy to teachers and learning. Ban- learning after the training. dura, 1997 is a book length treatment. Stajkovic and 2. Martilla and James, 1977 is the seminal piece that Luthans, 1998 and Judge et al., 2007 provide useful spawned a large amount of literature regarding literature surveys. The measurements scale used in importance-performance analysis or IPA. Oh, 2001 this study is based on Senemog ˘ lu et al., 2009; Wool- provides a critical methodological investigation of its folk and Hoy, 1990; and Gibson and Dembo, 1984. application and Azzopardi and Nash, 2013 provide a 56 www.worldbank.org Pedagogy and Teaching Methods 4 In this chapter we explore further the imple- province level box plots. The mean classroom mentation of the VNEN program. In the first sec- area available to each student was about 0.5 tion, we examine some program related details squared meters, without any statistically sig- regarding the classroom. In the next section we nificant difference between control and VNEN examine various aspects of VNEN pedagogy groups. When teachers were asked in 2013 if such as teaching methods, assessment and stu- there was a problem of space being restrictive dent government. We conclude this chapter by to prevent moving around or to establish learn- an exposition of the construction of the VNEN ing corners, 11% of VNEN teachers indicated this implementation index. was the case. Interestingly, 31% of control group teachers indicated a problem of space. Teachers were also asked about any problem with furni- 4.1  The Classroom ture, for instance being too heavy to rearrange in optimal group seating. While 79% of VNEN teach- We investigate three aspects regarding the VNEN ers indicated no problem with the furniture, the classroom in this section. In order for the VNEN proportion of control group teachers was lower model to be applied correctly, there needs to be at 72%. Over the next two years, there was an adequate space for children to be able to move improvement in the furniture situation for VNEN around as well as for the teacher to be able to schools, with 82% of teachers indicating no prob- move from group to group when needed. We lem with furniture in 2015. first investigate the aspect of adequate space and the furniture in the classroom. Second, the VNEN model calls for the incorporation of certain 4.1.2 Artifacts artifacts in the classroom—these include fea- VNEN pedagogy includes features that are directly tures such as a ‘student mailbox’ for students to observable in the classroom. We have termed these write messages to one another. Third, the VNEN as ‘artifacts’, which include the following: (i) Com- model of teaching is centered around the use of munity Map: as part of the effort to integrate learn- learning guides, and we investigate the teacher’s ing with the community and enhance the sense of use of learning guides. belonging, at the same time as providing useful conceptual skills to students, each VNEN classroom is supposed to make a community map, which lays 4.1.1  Classroom Space out the homes of every student in the classroom and Furniture together with major landmarks; (ii) Classroom Overall, the data indicates that there was ade- Library: this library includes the learning guides for quate space available for movement of children children as well as other reference material, story- and teachers as well as parents and other visitors books, magazines, etc.—­ usually it is arranged by in the classroom. Figure 4.1 indicates that the a student group, as shown in the accompanying median classroom area was 42 squared meters, picture (Figure 4.2), and groups are meant to be with a fairly even distribution as shown by the responsible for its neatness. Some classrooms also 57 Pedagogy and Teaching Methods Figure 4.1  Classroom Space Source: VNEN Impact Evaluation study primary data, Authors calculations. have a ‘mobile library’ which includes titles children Installing the artifacts is relatively easy to do and may take home; (iii) Learning Corner: this is a part can be quite fun for everyone, especially if par- of the classroom where other didactic material and ents are also involved. For instance, making the student’s work is stored and displayed; (iv) Com- community map requires discussion, consensus munity Corner: is part of or an extension of the building, and creativity. The task itself would help learning corner; it includes objects related to the to create a sense of community and lead to very community life of the school—ethnic handicrafts useful curriculum related discussions. Similarly, or dresses, crops, or other products; parents are setting up and maintaining a community corner invited to the classroom to help set up the commu- and library are very useful tasks related to the nity corner and the community map; (v) Student development of socio-emotional skills. Figure 4.3 Mailbox: is a poster with envelopes for each stu- indicates the measure of existence in the class- dent to be able to write notes to each other—the room of seven artifacts. Most of the VNEN schools use of the mailbox is meant to be a tool to apply had 75% or more of the artifacts. However, in a language learning and build communication skills; follow-up question about the use of artifacts, we (vi) Student Government Organization Chart: stu- found that some of them, such as the commu- dents form a student government with elected or nity map or the student mailbox were in disuse nominated members in different committees such in some schools. Of VNEN teachers 13% in 2014 as academic committee, health and hygiene com- and 11% of VNEN teachers in 2015 reported not mittee, external affairs committee, and so on. using the artifacts at all. 58 www.worldbank.org Figure 4.2  Artifacts in VNEN Classroom (a) Community map (b) Learning corner (c) Student government chart (d) Classroom library Source: © World Bank. Figure 4.3  Measure of Existence of Artifacts (a) 2013 (b) 2014 (c) 2015 Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 59 Pedagogy and Teaching Methods Table 4.1  Opinions and Use of Learning Guides Priority 1 Priority 2 & 3 2013 2014 2015 2013 2014 2015 Preference mathematics LG vs. textbook 90% 87% — 96% 91% — Preference Vietnamese LG vs. textbook 89% 87% — 96% 91% — Use of LG—different tasks at same time—very easy 82% 74% 75% 83% 80% 81% or easy Training for LG—very effective or effective 94% 98% 97% 96% 98% 97% For knowing each student’s progress LG—very 79% 94% 94% 83% 97% 98% effective or effective For learning communication skills LG—very effective 82% 97% 98% 86% 99% 99% or effective For students to be more creative LG—very effective 77% 94% 93% 82% 97% 97% or effective For students to know their own progress LG—very 72% 93% 92% 78% 97% 97% effective or effective Supporting ‘vote’ for LG to replace textbooks for — 85% 83% — 90% 89% entire system Source: VNEN Impact Evaluation study primary data, Authors calculations. 4.1.3  Learning Guides drastic change in the teaching method using LGs, it Learning Guides (LGs) replace textbooks under is surprising to see a strong positive response from the VNEN model. A learning guide has lessons or teachers regarding the LGs. Teachers prefer the LGs chapters like a regular textbook. In addition, it to textbooks and find them useful in a number of also has a workbook with exercises and activities ways. The training for the use of LGs was effective. for students to undertake, together with a series Teachers would also support the adoption of LGs of instructions about activities to be carried out in lieu of textbooks for the whole system. in a sequence, similar to a teacher’s lesson guide, except it is in the hands of students. Advantages of the LGs include: (i) turning individual differ- 4.2  Pedagogical Elements ences along different dimensions of intelligence From the three physical or tangible features of into being advantages in learning rather than VNEN teaching in the previous section, we turn disadvantages; (ii) developing and deepen- in this section to study five further elements in ing multiple socio-emotional skills; (iii) helping the process of VNEN teaching. We examine closer teachers better identify and direct attention the teaching methods used in the classroom toward students who need specific help; and and group work, and the modified assessment (iv) helping students themselves better identify framework that focuses on formative feedback their own progress and take corrective action as to students. Findings are presented regarding and when required. the implementation of student government and parental or community engagement in class- Table 4.1 presents key findings regarding percent room activities. of teacher receptiveness about LGs. Given the 60 www.worldbank.org 4.2.1  Teaching Method regarding learning guides (LGs) reported in The three main teaching methods reported by Table 4.1, with some important differences. As teachers across both VNEN and control groups with LGs, teachers express a clear preference for were as follows: (i) divide students into groups; the use of group organization over the traditional (ii) assign tasks to students who work individually classroom. However, while a majority of teach- before reporting back on results; and (iii) guide ers found the use of LGs to be easy or very easy, students in doing exercises in exercise notebooks there were no teachers who found group work or in study notes. Even though the task assignment to be very easy. The distribution between teach- for students or exercises follows the sequence laid ers who considered group teaching to be easy out in the learning guide, VNEN teachers may be and those who considered it to be difficult was considering the application of the methods to be about even. Group work and LGs go together; it quite similar. The key differences would be that in would be rather difficult to teach using groups VNEN classrooms checking of answers is often a without using LGs. The teacher’s view of group pairs or group exercise, where the teacher is only work as difficult likely stems from its inherent called in cases of doubt or dispute. complexity—the teacher has to look for interac- tions within and across groups, in addition to the A follow-up question determines from teachers overall interaction with the class as a whole and the three most frequently used activities includ- with individual students. Interestingly, the train- ing three actions that are central to VNEN—stu- ing for group work does get positive reviews dents encouraged to raise questions, students from the teachers. studying in groups, and students studying on their own. Table 4.2 shows the comparison of The teachers were also asked in 2014 how often VNEN and control groups through the three sur- they found themselves teaching or explaining vey years. The reported use of group studying something to the whole class even though the by control group teachers is puzzling and needs students are seated in groups. Just over half of the to be investigated further. Note in Table 4.2 the teachers, 53% to be exact, said that they had to do increase in the percentages for items ‘b’ and ‘c’ this less than one-fourth of the time or only when which may be one of the spillover effects of the the learning guide required. The other half would VNEN program. apparently be teaching to the whole class more often. Accurate measurement of the exact extent to which whole class teaching in VNEN classrooms 4.2.2  Group Work happens requires more sophisticated methods VNEN teachers were asked about their experi- that we investigate in the qualitative section of ence with group work. The responses, shown this report. For now, it would be useful to note in Table 4.3, match to some extent the findings that when teachers are not letting work proceed Table 4.2  VNEN Methods among Top 3 Teaching Methods Control Group VNEN Group Year 2013 2014 2015 2013 2014 2015 a. Students studying in groups 94% 95% 95% 98% 99% 98% b. Students encouraged to raise questions 38% 55% 61% 53% 64% 71% c. Students study on their own 11% 17% 25% 36% 42% 46% Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 61 Pedagogy and Teaching Methods Table 4.3  Opinions and Use of Group Work Priority 1 Priority 2 & 3 2014 2015 2014 2015 Prefer group work for mathematics 86% 82% 90% 86% Prefer group work for language 90% 87% 93% 88% Easy to teach with group organization 50% 53% 49% 52% Training for group work—very effective or effective 96% — 99% — For knowing each student’s progress—very effective or effective 96% — 96% — For learning communication skills—very effective or effective 99% — 100% — For students to be more creative—very effective or effective 94% — 99% — For students to know their own progress group work—very 76% — 77% — effective or effective Supporting ‘vote’ for group work for entire system 87% — 88% — Source: VNEN Impact Evaluation study primary data, Authors calculations. in groups, the method being used is the traditional semester. The percentage of control group teach- teaching method rather than the VNEN method. ers in 2013 who reported formal testing with fre- quency greater than once a semester was 95%. The percentage of VNEN teachers reporting for- 4.2.3 Assessment mal assessment more often than once a semes- The VNEN model stresses formative rather than ter went up to 62% in 2014 compared to 65% of summative assessment—meaning the accent control group teachers. In 2015, there were more is on oral and written feedback rather than on VNEN teachers (69%) reporting formal testing repetitive testing. The data indicate that teachers more often than once a semester, compared to in VNEN classrooms may have drastically reduced 60% of control group teachers. the amount of informal testing or marking of stu- dent performance at the beginning of program There are a number of alternative methods implementation. However, it also appears that under VNEN in lieu of testing with marks for VNEN teachers subsequently migrated toward students to obtain feedback about their learn- using more testing while traditional teachers ing: (i) students are meant to check each other’s reduced their testing. Using a frequency table work in pairs; (ii) the group leader checks the to approximate the actual number of times that work of each group member; (iii) the group informal or routine tests were used, the data indi- arrives at a consensus about the work of each cate in the year 2013 that an average of 39 tests group member; and (iv) groups work on proj- were applied by VNEN teachers as compared to ects and each group obtains feedback from the 100 tests throughout the year for control group teacher. These assessment techniques seek to teachers. The question on informal tests was provide feedback beyond basic mechanics of not repeated in subsequent years, but a similar computation and getting the correct answer. question on formal tests was monitored. In 2013, By way of contrast, the traditional preferred 79% of VNEN teachers reported having formal methods are (i) teachers call on one of the stu- assessment once a year and 21% reported hav- dents to demonstrate the correct answers and ing formal assessment more often than once a (ii) teacher orchestrates a sequence where each 62 www.worldbank.org Table 4.4  Teacher Use of Assessment Methods in Mathematics Lessons 2014 2015 VNEN Preferred Methods Control VNEN Control VNEN (i) Student pair checking—all the lessons 47% 64% 54% 76% —about half the lessons 30% 26% 29% 21% —some of the lessons 22%  9% 16%  3% —never use this method  1%  0%  1%  0% (ii) Group leader checking—all the lessons 38% 74% 47% 74% —about half the lessons 27% 16% 27% 19% —some of the lessons 33% 10% 24%  7% —never use this method  1%  0%  2%  0% (iii) Group consensus check—all the lessons 26% 32% 27% 36% —about half the lessons 21% 25% 28% 31% —some of the lessons 45% 38% 39% 30% —never use this method  8% 5%  7%  3% (iv) Project work checking—all the lessons 35% 49% 37% 42% —about half the lessons 31% 23% 37% 28% —some of the lessons 33% 25% 25% 28% —never use this method  1%  1%  2%  2% Traditional Preferred Methods (i) Teacher sequenced steps—all the lessons 28%  8% 24%  6% —about half the lessons 17%  6% 19% 10% —some of the lessons 38% 40% 36% 38% —never use this method 17% 47% 21% 47% (ii) Student show and tell—all the lessons 59% 19% 60% 16% —about half the lessons 15% 14% 17% 21% —some of the lessons 22% 54% 19% 51% —never use this method  4% 14%  4% 12% Source: VNEN Impact Evaluation study primary data, Authors calculations. step is explained on the blackboard and stu- The first thing to be observed from Table 4.4 is dents check their own answers from their work- that the VNEN preferred methods are also popu- books or slates. Table 4.4 shows the teacher lar with traditional teachers. While (i) student responses regarding how often they use the pair checking and (ii) group leader checking specific assessment method. show a higher incidence with VNEN teachers, Enhancing School Quality in Vietnam through Participative and collaborative Learning 63 Pedagogy and Teaching Methods the traditional teachers also use these methods. community and to help develop socio-emotional Control group teachers may even have increased skills of responsibility and leadership.1 Student their use of VNEN preferred methods as aware- government is learning by doing. VNEN utilizes ness about the new assessment methods spread. the philosophy that the best way to learn respon- As a matter of fact, for the specific issue of assess- sibility is to be actually assigned responsibility, ment, MOET encouraged all schools to adopt for- together with help to execute it. Group lead- mative assessment methods. ers learn to make sure that the group members obtain the resources they need and follow the The bottom two items of Table 4.4 show that old tasks indicated in the learning guide or the direc- methods have a tendency to persist. One clas- tions of the teacher. They learn or are meant to sic approach that is commonly used in primary learn about communicating clearly, persuading classrooms in Vietnam, especially for mathemat- others, and acting collectively in a responsible ics, is for the teacher to orchestrate the computa- manner. tion of answers with the use of a long wooden ruler, which she would tap once to indicate that Inclusiveness is a core value regarding student students are to work on a specific step, and tap government. Teachers are encouraged to give twice to indicate that they should move to the everyone a chance, which they appear to be next step. This is similar to another approach doing quite faithfully. As mentioned by 95% where the teacher calls a student, typically a of the teachers, each student gets a chance to student whom she considers to be bright and be group leader. The group leader position is able, to demonstrate the answer on the black- changed every week by 15% of VNEN teachers. A board and the children check their own work further 40% to 50% of teachers change the group against the blackboard. Sometimes this is com- leader every month and about 20% change the bined with the students subsequently lining up group leader every two months. to have their notebooks initialed by the teacher. These methods are very inefficient forms of cog- In addition to the group leaders, the classroom nitive engagement and have negative equity also establishes various student committees, and implications. While nearly half the VNEN teach- each student has to be a member of one commit- ers report they never use the orchestration tee, with rotation encouraged. The committees method, only one in ten of the VNEN teachers typically established in VNEN schools are ones report never using the class demonstration by for learning, community relations, arts and craft, the star student. health and hygiene, sports, culture, and so on. Student government is one of the areas where Table 4.4 does not show the answers for the the school leadership and class teachers can same question that was asked regarding assess- exercise their energy and creativity with regard ment in Vietnamese language and other sub- to the depth of implementation. Respect for dif- jects. When the answers are combined they give ferences and appreciation of diversity are deep a fairly strong idea about the strength of VNEN social goals that VNEN seeks to further through implementation. student government. Figure 4.4 shows the selection of girls and eth- 4.2.4  Student Government nic minorities in student government positions. A very important part of the VNEN model is the The black dotted line shows the percentage of institution of classroom or student government. female students (nearly 50%) and the percent- The primary purpose of the student govern- age of ethnic minority students (about 38%) in ment is to help inculcate values of service and the overall sample. Rather than nomination by 64 www.worldbank.org Figure 4.4  Gender and Ethnic Minority Equality in the VNEN Classroom, 2014 (a) Office holder gender (b) Office holder ethnicity Source: VNEN Impact Evaluation study primary data, Authors calculations. the teacher, students elect the representatives, 4.2.5  Parental Engagement which itself is an exercise in fomenting democ- We now turn to the eighth and last element of racy and civic values. The gender distribution VNEN in the classroom, parental engagement. A shown in Figure 4.4 (a) provides quite a fascinat- deeper examination of parental backgrounds and ing insight both into how stereotypes are formed the home environment of the child is covered in and sustained and how they can be eliminated. the next chapter. In this section we focus on the Quite unlike the situation for adult women in teacher’s perspective regarding parents and par- national leadership (Munro and Of, 2012), with ent roles as part of the VNEN teaching method. two notable exceptions, the girls appear to domi- nate student government. With a core element of VNEN philosophy con- cerning the relationship between learning and Among the two posts with the highest female real life, parents are meant to play a bigger role representation is the position of Secretary. Fur- in a child’s education as compared to the tra- ther research is required to investigate more ditional model. Parent engagement as part of deeply into the functioning of student govern- VNEN has various facets—both cognitive and ment, but it is likely that the job of Secretary non-cognitive learning as well as the building of involves the highest amount of real work. Girls community and cohesion. Teachers were asked are underrepresented by far as leaders of the how often they engaged with parents for a speci- sports committee and also for the committee fied list of activities or purposes. The findings are on student’s rights. At the same time, girls are reported in Table 4.5. The overall level of interac- grossly overrepresented on the Arts and Crafts tion with parents for VNEN teachers is lower than committee leadership. From panel (b) of Fig- would be expected if a substantial proportion of ure 4.4, the percentage of ethnic minority leaders teachers had closely adhered to the VNEN phi- appear to be slightly lower than the percentage losophy of involving parents. The data do indi- of ethnic minority children in the student popu- cate a higher level of parent interaction for VNEN lation. However, there is no pattern by position teachers, but with considerable heterogeneity that would suggest any kind of discrimination or within both VNEN and control group schools. On bias. The percentage information is derived from average, parents were invited only once a year to two different sources—school enrollment from make classroom artifacts, and less than once to the principal and ethnic identity of student gov- demonstrate or teach a skill. ernment office holder from the class teacher. Enhancing School Quality in Vietnam through Participative and collaborative Learning 65 Pedagogy and Teaching Methods Table 4.5  Teacher Engagements with Parents (mean number of occasions per year) 2014 2015 Traditional interactions Control VNEN Control VNEN a. Inform parents about child’s learning progress 2.39 2.47 2.54 2.55 b. Discuss well-being and behavior of child 2.09 2.20 2.16 2.21 c. Discuss parents’ concerns or wishes 2.17 2.28 2.20 2.28 d. Discuss homework or home based support 1.97 2.09 2.05 2.14 f. Voluntary parent labor for school improvement 1.07 1.29 1.04 1.26 j. Discuss financial matters regarding fees charged 1.04 1.03 1.29 1.33 k. Discuss financial matters—general fund use 0.77 1.16 0.76 1.09 VNEN encouraged interactions e. Invite parents to demonstrate or teach a skill 0.59 0.86 0.45 0.79 g. Invite parents to help make classroom artifacts 0.65 1.16 0.52 1.11 h. Invite parents to help in teaching-learning 0.55 0.97 0.38 0.88 i. Take parent’s language help (ethnic minority) 0.42 0.57 0.38 0.45 l. Discuss use of VNEN grant 0.33 1.12 0.33 0.97 Overall or total number of interactions 14.47 17.23 14.53 17.19 Source: VNEN Impact Evaluation study primary data, Authors calculations. 4.3  VNEN Implementation VNEN schools where students performed well on the standardized tests. While this model has been Index implemented and the ensuing results reported in Chapters 6 and 7, the study team also came to The initial conceptualization of this study was to realize from baseline data and from school vis- implement a standard average treatment effect its and discussions with teachers and education on the treated (ATT) model of impact evaluation. administrators that the ATT model may not be We had a randomized control and treatment completely appropriate to evaluate the impact of group cohort who would monitor from Grade 3 VNEN. The reason is the presence of considerable to Grade 4 and 5, and compare the student heterogeneity in VNEN implementation. performance to determine the impact of the model through standardized tests in Vietnamese Awareness and knowledge about VNEN spread and mathematics. The accompanying surveys fairly quickly across both VNEN and control were meant to collect background information group schools. Quite a few VNEN practices do including attitudes, preferences, and values not require an additional resource outlay and of the stakeholders as well as endowments to are easy to copy—for example, to invite parents understand or explain the variation in test score to teach how to make a traditional ethnic handi- performance. The qualitative part of the study craft, or to institute a student government. Some (reported in Chapter 7) was meant to provide practices do require resources, such as the acqui- further insight on the relationship between class- sition of learning guides, which were provided room actions and the results for a small sample of 66 www.worldbank.org free of charge to project schools due to the GPE on various dimensions, a once a year retrospec- financed project. However, MOET also followed tive survey cannot possibly measure dimension a practice of allowing non-project schools to level implementation with reasonable accu- adopt the VNEN model, and some schools were racy. However, by aggregating or averaging able to arrange for funds, for instance from the across dimensions and over respondents and local government’s allocation for textbooks to years, the signal to noise ratio can be consider- be redirected to purchase the learning guides ably improved. While the surveys were initially at marginal cost. Other schools attempted to designed to capture background information emulate the VNEN pedagogy without the detail to understand causality better, some of the regarding learning guides. From the viewpoint questions do provide information that can be of the impact evaluation, this was a problem repurposed to determine the level of implemen- of sample contamination, the fact that schools tation. More implementation related questions assigned to the control group began to adopt were also added after the baseline year, so the substantive elements of the VNEN project. When understanding of implementation improves over such a phenomenon is proportionately small, time, much as implementation itself may have one can simply eliminate the contaminated part improved cumulatively with time. of the sample with some checks for systemic bias. However, in this study, as seen from some of the In this subsection we present information about previously reported findings, quite large propor- the construction of the VNEN implementa- tions of control group schools adopted VNEN tion index. We consider variables related to the elements. intensity of VNEN that are available for both the control and VNEN groups so that an index can At the same time as control group schools were be developed common to the two groups. We adopting elements of the VNEN model, there examine the respective empirical distributions of were VNEN schools that were skipping elements the variable superimposed on the same axis to or adopting them with lower intensity. For exam- examine for lateral separation. If there is a clearly ple, a VNEN classroom may have had the mailbox visible separation, it is taken as an indication that artifact installed in the classroom for children to the variable tracks VNEN well and is included as share messages with one another to improve an element for the VNEN index. If there is not their communication and writing skills; but after a very clear separation, we do not include the installation of the mailbox, there was little actual variable. At the end the variables are combined use made of it. Or parents may have been invited into a single implementation index defined to for one meeting to help make the community have a mean of zero and standard deviation map, but never after that. Meanwhile, some of 1. The elements are considered by respondent VNEN schools may have taken parent participa- groups—principals, teachers, parents, and stu- tion to heart, and set aside one morning every dents. We construct a year-wise implementation week for parents to come in to teach a traditional index and an overall implementation index. skill. And teachers at the schools may have taken steps to make sure that children used the mail- boxes to communicate messages to one another 4.3.1  VNEN Implementation Index: and to the teacher, and the same teacher may Principal have also provided formative assessment by pro- The questionnaire on principals includes two possi- viding feedback on the notes the children wrote. ble elements for the VNEN index—an item on prin- cipal community outreach, expected to be higher Given the degrees of freedom that existed with for VNEN and a Rasch index of VNEN actions, shown regard to implementation on a day-to-day basis in Table 4.6. The Rasch index includes items such Enhancing School Quality in Vietnam through Participative and collaborative Learning 67 Pedagogy and Teaching Methods as ‘Teachers in my school are always looking for measure does not track VNEN, while the Rasch opportunities to integrate students’ learning with index does. Interestingly, in a recurrent theme real life’ and ‘Community members are involved in throughout this study, we find that the Rasch mea- curricular and extra-curricular activities’. The results sure shows a big step towards separation in the are shown in Figure 4.5. The community outreach second year, seen from the Kolmogorov-Smirnov Table 4.6  Principal Variables Variable Name Concept Included 2013 hcv_all13 Principal community outreach (home visits) X pri_msr_13_05 VNEN actions Rasch measure ✓ 2014 hcv_all14 Principal community outreach (home visits) X pri_msr_14_05 VNEN actions Rasch measure ✓ 2015 HCV_all15 Principal community outreach (home visits) X pri_msr_15_05 VNEN actions Rasch measure ✓ Source: VNEN Impact Evaluation study primary data, Authors calculations. Figure 4.5  VNEN Implementation Index: Principal (a) K-S distance 0.1063 (b) K-S distance 0.1120 (c) K-S distance 0.1116 (d) K-S distance 0.3506 (e) K-S distance 0.5233 (f) K-S distance 0.4454 Source: VNEN Impact Evaluation study primary data, Authors calculations. 68 www.worldbank.org distance measure, but the gap scales back from the extremely easy question about implementation, second year to the third year. or one that was very difficult for a school to get wrong. Indeed, Figure 4.6(a) shows a wide gap between control group and VNEN parents on 4.3.2  VNEN Implementation Index: this measure. We can also see from Figure 4.6(b) Parents that the gap narrows from 2014 to 2015, mostly Parents were interviewed only in 2014 and 2015 due to more control group parents becoming but the parents are able to provide more useful aware of the VNEN program. Another variable information regarding VNEN implementation which appears to discriminate strongly between for the purpose of generating an implementa- VNEN and control group schools is classroom tion index, see Table 4.7. By combining the data participation by parents—Figure 4.7 (a) and over the sample of students from each school, Table 4.8. Midway through the program, MOET a more robust measurement is possible—this had issued Circular 30 that sought to modernize is also true of information from students and the assessment method, with a more detailed teachers. Parents were asked about their own formative assessment by teachers. This circular involvement in school activities, an important was applicable to all schools, and is probably the VNEN consideration, and about the learning reason behind the lack of separation in measures related activities of children at home. Parents observed in Figure 4.7 (c). Other variables show a were also asked about their knowledge of moderate level of VNEN tracking. VNEN implementation in the school. While a set of variables were measured in each of the two survey years, some measures are available only 4.3.3  VNEN Implementation Index: for a single year. This is because the question- Students naires were being revised after every round to Student measures regarding classroom interac- uncover more details, while seeking to maintain tion are among the most useful variables explain- repeated measurement when asking the same ing VNEN implementation. It had been shown question was more useful. Figures 4.6 and 4.7 earlier that teachers in all schools, including provide details. control group schools, reported using a group- based teaching method. This may have occurred All parents were asked about their knowledge because of a problem in communicating the fact of VNEN. Using Rasch terminology, this was an that we were interested in knowing about near Table 4.7  Parents (Part I) Variables Variable Name Concept Included 2014 par_kvnen14 Parent knowledge about VNEN ✓ par_ask14b Child interaction at home X par_pvnen14 Parent participation in school activities X 2015 par_kvnen15 Parent knowledge about VNEN ✓ par_chsh15 Child sharing about school happenings X par_pvnen15 Parent participation in school activities X Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 69 Pedagogy and Teaching Methods Figure 4.6  VNEN Implementation Index: Parents (Part I) (a) K-S distance 0.8121 (b) K-S distance 0.1365 (c) K-S distance 0.2041 (d) K-S distance 0.5218 (e) K-S distance 0.1107 (f) K-S distance 0.1814 Source: VNEN Impact Evaluation study primary data, Authors calculations. full-time use of the group method, not just occa- 2015, shows a clear demarcation and an increase sional use of group teaching which may happen from the first year to the second year, followed by at any school. a decline in the gap. The fact that every sampled school had approxi- We also used a Rasch measure constructed out mately 20 randomly chosen students from the of 23 items that were asked of students regard- grade cohort provides a robust check on actual ing the VNEN method used in classroom instruc- group work (Table 4.9 and Figure 4.8). A simple tion. These items included questions related to question was asked about how often the group the use of games and role-play in the classroom method is used in teaching—with choices being and the use of help from peers. The items also 1: ‘Yes, often’; 2: ‘Yes, sometimes’ and 3: ‘No’. We included practices that are typically seen only defined the measure, hi_grpv13 and hi_grpm13 in traditional classrooms such as the teacher respectively for Vietnamese and mathemat- reading out loud and the students taking notes. ics lessons, based on the number of students As with the measurement of group work, the who answered l less the number of students combined happening of VNEN practices and who answered 3, expressed as a percentage of the absence of happenings expected in con- the total number of students at the school. The trol group classrooms makes for a fairly robust resulting measure shown also for years 2014 and measure of VNEN practice. Conversations with 70 www.worldbank.org Table 4.8  Parents (Part II) Variables Variable Name Concept Included 2015 par_clrmp15 Parent report of child’s classroom setup ✓ par_rsa15 Parent report of child’s classroom activities ✓ par_cir3015 Parent receives written feedback from teacher X par_real15 Child does applied real life assignments ✓ par_stg15 Parent reports about student government election ✓ Source: VNEN Impact Evaluation study primary data, Authors calculations. Figure 4.7  VNEN Implementation Index: Parents (Part II) (a) K-S distance 0.7989 (b) K-S distance 0.3261 (c) K-S distance 0.0439 (d) K-S distance 0.2049 (e) K-S distance 0.2998 Source: VNEN Impact Evaluation study primary data, Authors calculations. parents had suggested that one difference that cognitively engaged and experienced greater the VNEN program made was the willingness and novelty in their day-to-day activities. The formal enthusiasm of children to tell their parents about survey question regarding this only covers a rel- what had happened at school that day after atively low magnitude discrimination power, as they got home. This would make sense as chil- seen in Figures 4.9 (b) and (c). dren at VNEN schools were expected to be more Enhancing School Quality in Vietnam through Participative and collaborative Learning 71 Pedagogy and Teaching Methods Table 4.9  Student (Part I) Variables Variable Name Concept Included 2013 hi_grpv13 High incidence of working in groups for Vietnamese ✓ hi_grpm13 High incidence of working in groups for mathematics ✓ 2014 hi_grpv14 High incidence of working in groups for Vietnamese ✓ hi_grpm14 High incidence of working in groups for mathematics ✓ 2015 hi_grpv15 High incidence of working in groups for Vietnamese ✓ hi_grpm15 High incidence of working in groups for mathematics ✓ Source: VNEN Impact Evaluation study primary data, Authors calculations. Figure 4.8  VNEN Implementation Index: Students (Part I) (a) K-S distance 0.5764 (b) K-S distance 0.7437 (c) K-S distance 0.6361 (d) K-S distance 0.6228 (e) K-S distance 0.6910 (f) K-S distance 0.6069 Source: VNEN Impact Evaluation study primary data, Authors calculations. 72 www.worldbank.org Table 4.10  Student (Part II) Variables Variable Name Concept Included 2014 stu_msr_14_02 Student report of VNEN class activities ✓ stu_tell_mom_14 Student volunteers to tell parents about class activities X stu_gv_peer14 Student enthusiastic participation in peer activities ✓ 2015 stu_msr_15_02 Student report of VNEN class activities ✓ stu_tell_mom_15 Student volunteers to tell parents about class activities X stu_gv_peer15 Student enthusiastic participation in peer activities ✓ Source: VNEN Impact Evaluation study primary data, Authors calculations. Figure 4.9  VNEN Implementation Index: Students (Part II) (a) K-S distance 0.7779 (b) K-S distance 0.1744 (c) K-S distance 0.2543 (d) K-S distance 0.6914 (e) K-S distance 0.0941 (f) K-S distance 0.2462 Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 73 Pedagogy and Teaching Methods 4.3.4  VNEN Implementation Index: single variable would not be an accurate measure Teachers of implementation for this reason, but repeated Figure 4.10 indicates that nearly all the variables averaging and triangulation across groups would derived from the survey of teachers prove to be lead closer toward a reasonably accurate mea- useful in tracking or discriminating between sure of implementation. VNEN and control groups schools. In some cases, there is a movement toward a greater level of Figure 4.11 (d) and (e) show the result of a policy implementation across years. For instance, the level intervention to encourage all schools to measurement of variables tc_gsq_13, tc_gsq_14, adopt the use of formative assessment methods. and tc_gsq_15 comes from a question that asked Control group distribution shows a bigger right- teachers to mention the top three teaching activ- ward shift leading to a decline in the discriminat- ities they use from a list of nine activities. This ing power of this particular variable. included items such as ‘encourage students to raise questions in class’ together with items such Figure 4.12 shows the year-wise Rasch aggre- as ‘write on blackboard for students to copy’. To gated measure of the constituent elements some extent, teachers may have become more shown so far, and the summary implementa- aware over the study period that the latter was tion measure combining all three years, named less and less of a socially acceptable answer. The vindex2a. Table 4.11  Teachers Variables Variable Name Concept Included 2013 tc_gsq_13 Frequency of VNEN and traditional classroom activities ✓ tc_arti13 Existence of VNEN artifacts in the classroom ✓ tc_artu1_13 Utilization of VNEN artifacts in the classroom ✓ tc_pvnen13 Parent participation at school ✓ tc_s14_13 Composite index of VNEN actions in the classroom ✓ tc_pclm13 Incidence of bi-weekly teacher cluster meetings ✓ 2014 tc_gsq_14 Frequency of VNEN and traditional classroom activities ✓ tc_arti14 Existence of VNEN artifacts in the classroom ✓ tc_artu1_14 Utilization of VNEN artifacts in the classroom ✓ tc_pp_14 Parent participation (stronger specification) at school ✓ tc_ped_14 Composite index of VNEN pedagogy in the classroom ✓ tc_chwk_14 Teacher utilizing VNEN supported assessment methods ✓ 2015 tc_gsq_15 Frequency of VNEN and traditional classroom activities ✓ tc_arti15 Existence of VNEN artifacts in the classroom ✓ tc_artu1_15 Utilization of VNEN artifacts in the classroom ✓ tc_pp_15 Parent participation (stronger specification) at school ✓ tc_ped_15 Composite index of VNEN pedagogy in the classroom ✓ tc_chwk_15 Teacher utilizing VNEN supported assessment methods ✓ | tc_sg_15 Teacher report on functioning student government ✓ tr_wct_15 Teacher teaching whole class even if group seating ✓ Source: VNEN Impact Evaluation study primary data, Authors calculations. 74 www.worldbank.org Figure 4.10  VNEN Implementation Index: Teachers (a) K-S distance 0.2929 (b) K-S distance 0.3087 (c) K-S distance 0.2777 (d) K-S distance 0.8047 (e) K-S distance 0.8515 (f) K-S distance 0.7958 (g) K-S distance 0.8247 (h) K-S distance 0.7044 (i) K-S distance 0.6569 (j) K-S distance 0.1869 (k) K-S distance 0.2130 (l) K-S distance 0.3309 (continued) Enhancing School Quality in Vietnam through Participative and collaborative Learning 75 Pedagogy and Teaching Methods Figure 4.10  VNEN Implementation Index: Teachers—Continued (m) K-S distance 0.3603 (n) K-S distance 0.5861 (o) K-S distance 0.6347 Source: VNEN Impact Evaluation study primary data, Authors calculations. Figure 4.11  VNEN Implementation Index: Formative Assessment Intervention (a) K-S distance 0.3098 (b) K-S distance 0.5918 (c) K-S distance 0.1883 (d) K-S distance 0.3483 (e) K-S distance 0.2777 Source: VNEN Impact Evaluation study primary data, Authors calculations. 76 www.worldbank.org Figure 4.12  VNEN Implementation Combined Index: By Year and Overall (a) K-S distance 0.8512 (b) K-S distance 0.8912 (c) K-S distance 0.8083 (d) K-S distance 0.9131 Source: VNEN Impact Evaluation study primary data, Authors calculations. The different elements were combined using a standard deviations between control group and Rasch technique. In addition to the kernel den- treatment group. But we have to be very careful sities shown earlier, Figure 4.13 provides a clear in comparing items because the experimental depiction of the issue of implementation hetero- design and tags of VNEN and control have many geneity. The smoothed dashed line shows how layers in between. on average implementation difference is two Enhancing School Quality in Vietnam through Participative and collaborative Learning 77 Pedagogy and Teaching Methods Figure 4.13  VNEN Implementation Index Source: VNEN Impact Evaluation study primary data, Authors calculations. 78 www.worldbank.org TakeAways: Pedagogy and Teaching Methods This is the second of two chapters that deals with community and to help develop socio-emotional teachers. In this chapter, we presented details skills of responsibility and leadership. A strong of implementation of the teaching method in the positive outcome concerns diversity and gen- classroom and examined each element of the VNEN der equity, with many girls exercising their model. We described the construction of an index skills as student leaders. that can be used to understand the implementation of the VNEN model. PARENTAL ENGAGEMENT CLASSROOM • Parental involvement in the day-to-day activi- ties of the school forms an important part of the • Mean classroom area was 42 squared meters, VNEN concept, but implementation of this activ- which appears to have been adequate in most ity appears to have been limited to some schools. cases for a group configuration of seating. VNEN The tendency was for parents to be invited only classrooms had 75% or more of seven artifacts once a year to help make classroom artifacts such as community maps and learning corners. and less than once a year to demonstrate or However, regular use of the artifacts as part teach a skill. of classroom instruction was not widespread. IMPLEMENTATION INDEX LEARNING GUIDES • The painstaking construction of an implementa- • Learning guides are a central element of the tion index lies at the heart of this study. Based VNEN approach to combine socio-emotional on the elements of the VNEN model, an index skill development together with cognitive skills. is constructed from information gathered Teachers expressed positive opinions about the together from principals, teachers, students, effectiveness of learning guides and a large and parents. The resulting index shows the majority expressed support for using learning continuum of VNEN practices—while overlap is guides in lieu of textbooks. minimal, within the group of VNEN schools, there was a wide dispersion of the index, as shown in GROUP WORK AND ASSESSMENT Figure 4.13. • Teachers appreciated the effectiveness of group The VNEN model included many elements—some work in enhancing creativity and communica- of them such as seating arrangements and class- tion skills. However, about half the teachers room artifacts were easy to implement; other ele- found it difficult to teach using group work. ments such as advanced assessment methods and Formative assessment methods such as check- student government were more difficult. The final ing in pairs, group checking, and project work result is that VNEN implementation took place is more often found in VNEN schools, but tra- along a distribution, with some schools applying ditional teachers also used the recently intro- nearly all elements and other schools implementing duced assessment methods. only a few of the elements. STUDENT GOVERNMENT • The primary purpose of student govern- ment is to help inculcate values of service and Enhancing School Quality in Vietnam through Participative and collaborative Learning 79 Pedagogy and Teaching Methods Endnote Chesterfield, 1994, Baessa, Chesterfield, and Ramos, 2002 and Forero-Pineda, Escobar-Rodríguez, et al., 2006. The study in Guatemala found a positive rela- 1. The relationship between Escuela Nueva and the tionship between democratic behaviors and reading development of civic values has been investigated achievements of children in Escuela Nueva schools. in detail in the case of Colombia and Guatemala; see 80 www.worldbank.org Parents, Home, and Community Situations 5 The purpose of this chapter is to provide an for which there is an appreciable difference that overview of the home context of the students is statistically significant. There are slightly more as uncovered from a survey of the parents. This ethnic minority parents in the VNEN sample. Both chapter presents findings in three sections. In VNEN and control group families were of similar the first section, we present a brief overview of size with between two to three children, based the parent demographics. The second section on the reported number of siblings. There is no presents the beliefs, values, and personality of difference with regard to parents’ age and their the parents. The third section presents findings wealth as measured by an index of household regarding the knowledge, practices, and prefer- possessions. ences regarding VNEN, including the study envi- ronment at home and parental interaction with the VNEN model. 5.1.2  Education and Labor Market Unlike for the sample of students, teachers, and principals who were interviewed every year, we 5.1  Parent Demographics did not expect to uncover major differences from year to year in the parent interviews. Two While the selection of the random sample for the rounds of parental interviews were carried out study was based on the school characteristics, it (for the second and third year of the study), but is important to determine whether there was most information was collected only one time. parity or equality between parents of children As seen in Figure 5.2, the parent population is in the VNEN and control group samples. We con- roughly equally split between those who have a sider two main sets of variables—family demo- primary level of education or below and those graphics in terms of dwelling conditions and with higher than a primary level. Lower second- the educational attainment of parents, as well as ary education or nine years of schooling is the their occupation groups. most commonly reported parental educational level. It is remarkable that there is a parity of edu- cation between mother and father, a reflection 5.1.1  Family and Living Conditions of the past investments for education made in As expected, mothers are the primary caregivers Vietnam. for the children, and they take care of the main aspects with regard to the child’s education. The The fact that the average level of education is less program was directed toward high ethnic minor- than the high school level has an important impli- ity areas, so it is not surprising that more than cation regarding the implementation of a reform a third of the parents report being from ethnic of the nature of VNEN. VNEN seeks to support minorities. As seen in Figure 5.1(b), the ethnic the transformation toward a competency based background of the parents is the only variable curriculum, and some of the methods such as 81 Parents, Home, and Community Situations Figure 5.1  Family Demographic Situation     (a) Child’s primary caregiver (b) Ethnic background (c) Number of siblings   (d) Age distribution of parents (e) Household possessions index Source: VNEN Impact Evaluation study primary data, Authors calculations. encouraging questioning from children are quite level of parents from both groups was between contrary to the upbringing of most of the par- VND 2 to 3 million, equivalent to about USD 300 ents. Later in this chapter we look more closely at in PPP terms. Only half of the parents reported parent views according to their education levels. having only one job. A further one-fourth of par- Figure 5.2 also shows that nearly two-thirds of ents reported two jobs or more than two jobs. the parents were farmers, with both mother and Regardless of income or education levels, 97% of father reporting that occupation. Not reported parents reported that they wanted their child to in Figure 5.2 is that the average monthly income proceed to lower secondary school. 82 www.worldbank.org Figure 5.2  Parents Education and Occupation Situation (a) Mother’s education (b) Father’s education (c) Mother’s occupation (d) Father’s occupation Source: VNEN Impact Evaluation study primary data, Authors calculations. 5.2  Parent Beliefs, Values, they were neutral about the pedagogical model, children would not receive a consistent message and Personality at home and in the school, and the dissonance would likely not be healthy for the children. Parental beliefs and values are particularly Hence what parents think about the education important for VNEN because in a sense the VNEN being imparted to their children, and the values program seeks to bring about a positive cultural that parents seek to instill in children are cru- transformation. If parents do not believe in the cial for the success of the educational reform. need for a cultural transformation, it is likely that Personality of the parent also affects learning they would not support the VNEN model. Even if and it would be important to ascertain equality Enhancing School Quality in Vietnam through Participative and collaborative Learning 83 Parents, Home, and Community Situations between control groups and VNEN groups for the five-point Likert scale. The results for control and purpose of impact evaluation. VNEN groups are presented in Figure 5.3. The results show that 80% of the control group par- ents disagreed with the statement, and a smaller 5.2.1  Parental Beliefs Regarding 60% of VNEN group parents disagreed. Education We asked the same question to parents as we Even though a majority of VNEN parents dis- had to principals and teachers (reported in Sec- agreed with the statement that change was not tions 2.3 and 3.2 respectively) about the satis- needed, the proportions are lower than the prin- faction with the traditional model. Specifically, cipals and the teachers. Principals are the biggest they were asked how much they agreed or dis- believers in the VNEN model, with a consistent agreed with the statement: “Vietnam’s traditional 75% believing in the need for change. We did teaching model where children sit in rows, fac- not measure evolution in parents’ attitude, but ing the teacher/blackboard is good, there is no clearly for the success of the reform, the findings need for reform.” Choices for parents ranged indicate the need for deeper engagement of the from “Strongly Agree” to “Strongly Disagree” on a parents. Figure 5.3  Parents Opinion about Maintaining Traditional Model, 2016 (a) Control group parents (b) VNEN group parents Source: VNEN Impact Evaluation study primary data, Authors calculations. 84 www.worldbank.org It is usually difficult for parents to acknowledge a children’s performance on homework depended problem with the schools or school system that on the level of education of parents. Turning to they send their children to. The finding has been the right-hand side, it is useful to observe that consistently reported in the United States, where even though Vietnam is classified as belonging to nationally representative parent surveys were a Confucius heritage culture with high emphasis carried out for many decades. Negative opinions on authority values (Dalton and Ong, 2005), 94% about the education system decline consistently of parents disagreed that children need to always as parents are asked about public schools nation- be obedient. At the same time, only 27% of the ally, state-wide, locally, and so on, as reported in parents disagreed that it is more important for Rose, Gallup, and Elam, 1990 and Tompson, Benz, their child to make many friends than to do well and Agiesta, 2013. There has been a large body of academically. research in advanced economies that indicated that parenting and parental resources were more The relationship between parental effort, school important than the school attended by the child characteristics, and student effort and achieve- and the resources provided by the school. For ment has acquired renewed interest, and some a long time, it has been thought that the situa- of the recent findings need to be explored fur- tion was different in poor countries—with public ther for developing country contexts like Viet- schools being typically deprived of resources, it nam. Houtenville and Conway, 2008 explore was often found that the relationship was differ- the relationship between parental effort, school ent (Fuller, 1987). Perhaps in the case of Vietnam resources, and student achievement. They find sooner than other countries, the challenge is now sizeable impacts of parental effort that are shifting toward what parents do. We probed fur- not captured by information regarding paren- ther into parental attitudes regarding education. tal background such as income and educa- tion. Parental effort is defined through reports We asked parents to respond to a set of Likert of parent actions: discussion with children of scaled items regarding the role of parents in edu- school activities, academic matters, and course cation and about child characteristics that were selections as well as participation in meetings important. Figure 5.4 indicates the findings, with and volunteering for school activities. De Fraja, the left-hand panel showing the direct education T. Oliveira, and Zanchi, 2010 present an interest- related items and the right-hand panel showing ing game theoretic model which combines the the items regarding importance of child out- effects of efforts from parents, children, and the comes. There was nearly a complete coincidence school. In their empirical estimates, they find that between parents in the VNEN and control groups, efforts tend to build on one another—parental but the responses are still interesting to explore. effort spurs further effort from the child and also Starting from the lower bar of panel (a) 85% of from teachers and other actors at school. The parents disagreed that ‘the teacher bears the paper concludes with the interesting observa- primary responsibility for the child’s educational tion that “It may be easier and more effective to development.’ Parents also disagreed (70%) with stimulate effort in households with low socio- the statement that ‘the child’s performance on economic background rather than hope for their homework assignment depends on the parents’ economic conditions to change.” Another prom- efforts’. Only 24% of the parents agreed with ising approach is from Dufur, Parcel, and Trout- the statement that ‘parents should participate man, 2013, who studied the issue through the regularly at in-class activities’. Likewise, only lens of social capital, with family social capital 26% of the parents agreed that parents need defined from communication, trust, and ‘active to sit together with children while they study engagement in the child’s academic life.’ They and about half of the parents disagreed that contrast the impact of family social capital with Enhancing School Quality in Vietnam through Participative and collaborative Learning 85 Parents, Home, and Community Situations Figure 5.4  Parent Beliefs Regarding Education and Values (a) Education by item (b) Values by item (c) Education composite index (d) Values composite index Source: VNEN Impact Evaluation study primary data, Authors calculations. school social capital, which includes teacher presented in Figure 5.4, parents were asked to morale and the ability of teachers to address the select up to five most important values from a needs of individual students. These aspects are list of 12 values that children can learn at home. closely related to the VNEN methodology which In this case as well, there was not any variation puts emphasis on both these kinds of social capi- between VNEN and control group parents, which tal. Further research can build on the intriguing bodes well for unbiased inference regarding initial findings of this impact evaluation study. impact evaluation. The findings, presented in Fig- ure 5.5, are important with regard to the nature and communication with parents that may be 5.2.2  Values Children Learn at Home required for the education reform in Vietnam to In addition to the Likert scaled question regard- succeed. ing educational and child development values 86 www.worldbank.org Figure 5.5  Qualities Learned at Home Source: VNEN Impact Evaluation study primary data, Authors calculations. Findings reported earlier regarding the sometimes but parents may need to be made more aware ambivalent attitude of parents about the tradi- that school is a place where such values can be tional model suggest the need to understand bet- strengthened or deepened. Responsibility, for ter the content and nature of the message required instance, can be developed by learning to take by reform proponents. By far the highest mention care of small tasks through roles in the student of values from parents (83%) was for ‘Indepen- government committees at school. Parents may dence’, closely followed by ‘Respect’, ‘Hard Work’, not be aware of the knowledge developed by ‘Obedience’ and ‘Responsibility’. The values of educational experts regarding the close linkages ‘imagination’ and ‘self-expression’ are at the lower between non-cognitive and cognitive skills. But end of the responses, together with ‘religious faith’, the fact that they care about these qualities may which is widely known not to be critical in Vietnam provide an opening to mold public opinion in among large segments of the population. favor of reform that seeks to enhance competen- cies of children along these dimensions. To some extent the views of the parents may appear to be contradictory—how will it be possible for a child to become independent 5.2.3  Parental Personality without also being given rein to develop their We conclude this section with an overview expression? Perhaps the parents want the self-­ regarding parental personality. As expected that child to develop independence within a circum- does not vary across VNEN and control group scribed set of boundaries, which may or may not parents in any discernible way. The idea that the allow freedom of self-expression. However, the personality of parents, particularly mothers, has low importance given to imagination cannot be an impact on child outcomes has been studied explained in this way. Clearly, values such as thrift for a long time in the literature on personality and determination are indeed learned at home, (Kochanska, Clark, and Goldman, 1997). In the Enhancing School Quality in Vietnam through Participative and collaborative Learning 87 Parents, Home, and Community Situations Figure 5.6  Parental Big Five Personality Traits Source: VNEN Impact Evaluation study primary data, Authors calculations. past decade or so there has been a veritable The measures are based on a shortened 15 item explosion of studies relating personality to vari- list of items—parents responded to a five-point ous human outcomes. The empirical validation of agree-disagree scale of items such as ‘I am a per- the so-called ‘Big Five Inventory’ or BFI plays a big son who is talkative’, a constituent item for the role in this (John, Naumann, and Soto, 2008). BFI extraversion score, and ‘I am a person who val- refers to the finding that a substantive portion ues artistic, aesthetic experiences’, an item on the of personality differences across human beings openness dimension. Scores on each dimension of diverse ages and cultures can be captured are arithmetic averages, with reverse coding for through five personality dimensions—openness, some items. conscientiousness, extraversion, neuroticism, and agreeableness. Metsäpelto and Pulkkinen, 2003, and Urman, 2012 provide instances of 5.3  Knowledge, Practices, using BFI to explore the relationship between and Preferences Regarding personality and parenting techniques that lead VNEN to child outcomes. In the case of this study, we utilized a short form of the BFI inventory from 5.3.1  Parents Understanding Lang et al., 2011. of VNEN Activities at School Awareness about VNEN was not universal in Figure 5.6 shows the measures of BFI dimensions VNEN schools—only 71% of VNEN school parents for the surveyed parents. It shows the equality mentioned knowing about VNEN. At the same between the control group and VNEN group. time, 37% of control group schools mentioned 88 www.worldbank.org Table 5.1  Parent Reports of Classroom Activities Classroom Feature Variant Control Group VNEN Group Seating arrangement Group seating 22% 85% Traditional 69% 8% Do not know 10% 7% Didactic materials Learning guide 5% 42% Textbook 81% 38% Do not know 14% 20% Teaching method Facilitate groups 26% 65% Podium lecture 51% 13% Do not know 23% 22% Student government SG does exist 43% 80% No SG 23% 3% Do not know 34% 17% Warm-up games Game played 51% 63% Game not played 17% 9% Do not know 33% 28% Student assessment Useful written comments 92% 92% No useful written comments 2% 2% Do not know 6% 6% Source: VNEN Impact Evaluation study primary data, Authors calculations. knowing about VNEN. The issue of implementa- salience among VNEN parents. 85% of VNEN tion heterogeneity across the VNEN and control group parents were aware about the group seat- groups has been discussed in detail in the pre- ing at school. 80% of parents reported about vious chapter. The roughly 6,000 parents who student government, which confirms anecdotal knew about VNEN were asked further about their evidence about some parents becoming closely opinion regarding VNEN. It is possible that some involved in ‘lobbying’ the teacher for inclusion of VNEN group parents may not have been aware their children in student government. As men- of the VNEN name, but they were familiar with tioned earlier, the assessment method adopted the elements implemented in their child’s school. for schools in the VNEN program was mandated Table 5.1 shows the responses from all parents for all primary schools around the second year of regarding a set of VNEN elements. the program, which is reflected in 92% of parents reporting receiving written comments on the The seating arrangement and student govern- performance of their children at school. ment appear to be the elements with the highest Enhancing School Quality in Vietnam through Participative and collaborative Learning 89 Parents, Home, and Community Situations Figure 5.7  Parent Participation and Opinion about Usefulness   (a) Discuss with teacher about student (b) Discuss with teacher about student   (c) Take part in classroom activities (d) Take part in classroom activities   (e) Take part in school activities (f) Take part in school activities Source: VNEN Impact Evaluation study primary data, Authors calculations. 5.3.2  Parents Participation For instance, they are expected to collaborate in in VNEN Activities at School the presentation of artifacts related to real life in Figure 5.7 shows variation in the extent of paren- the learning corner in the classroom. A number tal participation in school for three activities. The of schools have initiated particular days of the first item was discussion with parents about week for parents to come to school to share or the child’s academic performance at school teach about customs and practices of the local and behavioral aspects. This is an activity that is community. For instance, there could be activities expected to take place for all schools and panel related to cooking of local delicacies or the mak- (a) corroborates this expectation, with panel ing of handicrafts or other livelihood related skills. (b) indicating that nearly all parents find this Table 5.1 shows that this particular aspect of VNEN activity to be very useful or useful. may have been implemented only in a few schools, with a very high 73% of VNEN parents reporting Parents in the VNEN program are expected to take never having taken part in classroom activities and part in activities in the classroom and the school. 54% never having taken part in school activities. 90 www.worldbank.org 5.3.3  Home Academic Environment 5.3.4  Parent Opinion and for Student Preference Regarding VNEN Table 5.2 indicates parity between control and Figures 5.8 and 5.9 present the opinion of parents VNEN groups with regard to efforts made by regarding VNEN among the 6,000 parents (54% students and only a slightly higher mention of of the overall sample) who reported being aware homework assignments related to real life for about VNEN. The 85% of parents who expressed a children attending VNEN schools. The phenom- positive opinion about VNEN were asked to pick enon of private classes in Vietnam has been out the main reasons for their opinion. Figure 5.9 well researched and is an important policy issue indicates that the top three reasons were better (Dang, 2007). Nearly one out of five children academic performance, improved communica- are taking additional classes. Future research is tion skills, and improved leadership skills. In the needed to determine the impact of additional next chapter we examine the evidence from classes on the educational reform. standardized tests and whether indeed the per- formance of children in VNEN schools was better. Table 5.2  Home Study Environment and Activities Home Environment Aspect Control Group VNEN Group Permanent study corner exists 87% 87% Assignment for real life application 53% 63% Help from parent when doing real life application 54% 58% Student takes extra private classes 20% 17% Average study time per day (hours) 1.6 1.5 Average time per day carrying out chores (hours) 1.5 1.5 Parent help studying hours per week 2.05 2.00 Source: VNEN Impact Evaluation study primary data, Authors calculations. Figure 5.8  Opinion about VNEN among Parents Who Know about VNEN Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 91 Parents, Home, and Community Situations Figure 5.9  Reason for Positive Opinion of Parents about VNEN Source: VNEN Impact Evaluation study primary data, Authors calculations. 92 www.worldbank.org TakeAways: Parents Home and Community This chapter described the background of parents PARENTAL PARTICIPATION in terms of demographics and occupation, followed by a description of parents’ values and beliefs • Parents appear to know a lot about the ele- ments of VNEN reform being implemented regarding education. It was rounded off with find- at their child’s school including the seating ings about their opinions regarding VNEN. arrangement, the use of warm-up games before each lesson, and the functioning of student gov- DEMOGRAPHICS ernment. However, a high 73% of VNEN parents • Nearly two-thirds of the parents are farmers, report never having taken part in classroom and the same proportion have less than a high activities and 54% reported never having taken school level education. It is interesting to note part in school activities. Parent discussions with that there is parity of education level between teachers about the student’s performance was mothers and fathers, something not common evenly matched across VNEN and control group for a country with Vietnam’s level of per capita schools. income. One-third of the parents are from eth- nic minorities. These figures provide a useful OPINION ABOUT VNEN overview of the context of the VNEN program. • All parents, whether of children going to VNEN or control group schools, were asked if they PARENTAL BELIEFS were aware of the VNEN model. Just over half • It is quite interesting to note that only 60% of of the 6,000 surveyed parents mentioned being VNEN school parents as compared to 70% of aware of VNEN. Those parents were asked about VNEN teachers and 75% of VNEN principals dis- their opinion about VNEN. Of the parents, 85% agree with the statement that the traditional reported having a highly favorable or favorable model is doing fine. A much higher 80% of control opinion about VNEN. The top three reasons for group parents felt the same way. This finding their favorable rating of VNEN was superior aca- indicates the importance of involving parents demic performance and better communication more closely in the reform effort. and leadership skills. VALUES • Even though the parents are mostly farmers with less than a high school education, some of their educational views are quite progressive. Of Parents are a key figure in VNEN, which is perhaps the parents, 85% disagreed with the statement different from other kinds of educational reform that the teacher bore primary responsibility for that focus only on teachers and school authori- the child’s educational development, and 94% of ties. VNEN seeks to engage parents at school and parents disagreed with the notion that children with education at home in values, livelihood skills, should always be obedient. However, pointing and application or practice exercises at home to to the low outreach toward parents, only about increase relevance of education. In practice, paren- a quarter of the parents agreed that parents tal engagement under VNEN appears to be sporadic should regularly participate in class activities. at best. Enhancing School Quality in Vietnam through Participative and collaborative Learning 93 Cognitive and Non-Cognitive Student Outcomes 6 6.1  Visual Representation comparable score across years; the mathematics score (not shown in Figure 6.1) was also com- of Test Score Trends puted in the same way. The dots represent the achievement of an individual student and each 6.1.1  Overall Trends box of the box plot represents a school with the for Cognitive Scores central line being the mean and the edges of the At the outset of this chapter, it is very important box representing 25th and 75th percentile scores to have clarity about the measurement of cog- for the school. The dots are ‘jittered’ so that indi- nitive outcomes. The study used standardized vidual dots can be visible. The purple and black tests designed to follow the cohort from Grade 3 lines in Figure 6.1 represent the year wise means in 2013 to Grade 5 in 2015. About 15 to 20 ran- for this particular random sample, with the mean domly selected students were chosen from each score being 500 for Grade 3 in 2013 and 600 in school at the beginning of the study and followed Grade 5 in 2015. The very important takeaway throughout the three years. Figure 6.1 shows a from Figure 6.1 is the imprecision with which window into what the test score data looks like, cognitive achievement is measured. The only for a random sample of 15 schools chosen for this useful conclusions that can be made from analy- demonstration. The scores used in the analysis sis of data like this are through techniques of sta- are based on the results for Vietnamese using tistical inference. the Item Response Theory to obtain a scaled and Figure 6.1  Vietnamese Test Scores for 15 Randomly Selected Schools Mean score Source: VNEN Impact Evaluation study primary data, Authors calculations. 95 Cognitive and Non-Cognitive Student Outcomes Figure 6.2  Same Schools with VNEN Information Added Mean score Source: VNEN Impact Evaluation study primary data, Authors calculations. This point becomes clearer looking at Figure 6.2, performing in Vietnamese was one of the target- which is the same as the earlier figure, with infor- ing criteria for the project and was included in the mation added through shading of whether the propensity score matching equation. The school school was originally classified as a VNEN school records have data about the number of Grade 1 or a control group school. It so happens that in children who were in one of four categories— these particular randomly selected 15 schools excellent, good, average, and poor. As schools for demonstration purposes, eight (in blue) were are of different sizes, we first convert this num- control group schools and seven (in red) were ber into a percentage. We then compute a raw VNEN schools. The means are represented by the overall score for each school by weighting excel- horizontal lines in respective colors, with dotted lent, good, average, and poor students by 4, 3, 2 lines being the score in 2013 and filled lines the and 1 respectively. Finally, in order to increase the score in 2015. viability of comparison with the score from test- ing, we standardize the resulting raw score to a The fact that in 2013 the red line is already above mean of 500 and a standard deviation of 100. The the blue line reflects the fact that the baseline resulting score is not purely normal because it is survey was conducted in December 2012 after truncated at the higher end, as seen in Figure 6.3. the VNEN program had already been in existence for about 18 months. There was no standardized The advantage of this study is the relatively large test available to compare the schools at the start sample size. The random sample for demonstra- date of the VNEN program in June 2012. tion purposes used for Figures 6.1 and 6.2 hap- pened to show the VNEN schools on an average We do not have standardized test data before doing better than the control group schools, but the baseline. However, we do have data from the for the small sample of 15 schools it could have school records when the cohort was in Grade 1 been the other way as well. As we increase the about the performance of students in math- numbers considered, in spite of the considerable ematics and Vietnamese. Indeed, the school’s noise in the data that can be seen clearly with the overall percentage of children who were weak window provided by Figures 6.1 and 6.2, we are 96 www.worldbank.org Figure 6.3  District FSQL (Fundamental School Quality Level) Audit (a) Vietnamese (b) Mathematics Source: VNEN Impact Evaluation study primary data, Authors calculations. able to progressively reduce the noise and get a larger for mathematics than for Vietnamese and sharper signal. We present the key summarized for Priority II & III schools as compared to Priority I values in the panel of four graphs in Figure 6.4. schools. It is not immediately obvious from the scale of these graphs, but as shown subsequently Figure 6.4 shows the summarized distributions in the quantitative parts of this chapter, the big- of test scores for the control group and VNEN gest impact of the program appears to have group schools matched according to propensity taken place in the first year of the survey and the scores following the elimination of control group second year of the program, that is between 2013 schools that adopted VNEN and VNEN group and 2014, with the subsequent increment from schools with low implementation indexes. The 2014 to 2015 being lower than the initial incre- tests were constructed so as to have a mean of ment. Test results of May 2016 are also reported 500 at the baseline in December 2013. Succes- in the figures but not used in the quantitative sive pairs of years had matching items which analysis because of lack of compatibility. allowed the test difficulty to be calibrated for a comparable score. Each panel shows how the test scores went up as the students went up the 6.1.2  Trends in Cognitive Scores grades. The tests were also designed so that the by Priority I Provinces mean score in 2015, two years from the baseline, Figure 6.5 looks at the trends by a group of nine would be 600. The score is scaled so that the stan- provinces. Though the sample is representative dard deviation is 100, and it gives a good rubric only up to the level of priority order of prov- to understand the evolution of test scores. Mov- inces, these are the provinces which had at least ing one standard deviation requires about two 20 schools in the sample and provide some sug- years of development. In this setting, the focus gestive insights. The black dotted line in each of investigation is the comparison between the panel shows the score of 500, which was the ini- green boxes showing control group schools and tial national mean for Grade 3. One can observe the red boxes showing VNEN schools. Across all interesting contrasts. While VNEN schools initially the groups, the red box is seen to be on par or underperformed in some provinces, the perfor- higher than the green box. The differences are mance gap caught up by the end of the program; Enhancing School Quality in Vietnam through Participative and collaborative Learning 97 Cognitive and Non-Cognitive Student Outcomes Figure 6.4  National Trends in Cognitive Scores (a) Priority I mathematics (b) Priority II & III mathematics (c) Priority I Vietnamese (d) Priority II & III Vietnamese Source: VNEN Impact Evaluation study primary data, Authors calculations. in some cases there has been a parity from the adoption of VNEN elements across all primary beginning to end. In some cases there was a posi- schools. Local decision making would take place tive gap for VNEN that has widened over time. only in the context of leadership from the pro- vincial level. A key reason for this is the control of The province level variations reinforce the impor- financial resources. Control group schools which tance of the implementation index because decided to adopt the model partially or fully provincial authorities would have been the would likely have needed financial support from key decision makers regarding promoting the the provincial government. 98 www.worldbank.org Figure 6.5  Trends in Provinces (at least 20 schools in sample) (a) Mathematics (b) Vietnamese Source: VNEN Impact Evaluation study primary data, Authors calculations. 6.2  Impact Evaluation and nonintervention (control) groups. Second, propensity score-based school weights are used on Test Scores in the first-differenced form, which differences out unobserved fixed components of errors that To identify the impact of VNEN on learning can otherwise potentially bias impact estimates.1 outcomes, we take two approaches. First, the average treatment effect is estimated with pro- The conditioning variables used to estimate pro- pensity scores that essentially compare similar pensity scores are depicted in Table 6.1 and Fig- schools between the intervention (treatment) ure 6.6, with a graph of the resulting propensity Enhancing School Quality in Vietnam through Participative and collaborative Learning 99 Cognitive and Non-Cognitive Student Outcomes Table 6.1  Variables Used for Propensity Score Matching Variable Description gg1_5t School size g15poorsh Percentage students from poor families eg15sh Percentage students from ethnic minorities mg15sh Percentage of weak performing students in mathematics vg15sh Percentage of weak performing students in Vietnamese tthkm Distance from city center cmsupport Level of community support and interest (0, 1, or 2) tthsatellites Number of satellite campuses fii2007 Fundamental School Quality Input Index 2007 computer Number of computers bboksh Percentage of blackboards in good working condition seatoksh2 Percentage of student seats in good working condition schstd1m School meets national standard 1 or higher tups Percentage of teachers with upper secondary education ttrain Percentage of teachers with college training i.Priority Dummy for province priority grouping i.matinh Province dummy Source: VNEN Impact Evaluation study primary data, Authors calculations. Figure 6.6  Matched Scores Propensity Score before matching Sampled Schools Propensity Score after matching 0.4500000 0.6000000 0.3000000 0.4000000 Propensity Score 0.1500000 0.2000000 0.0000000 0.0000000 6 4 2 0 2 4 6 6 4 2 0 2 4 6 control treatment   control treatment (a) Mathematics (b) Vietnamese Source: VNEN Impact Evaluation study primary data, Authors calculations. 100 www.worldbank.org score distribution. The procedure used in estima- the point estimate decreased from the period tion trimmed off observations if the estimated of 2013–2014 to that of 2013–2015, which propensity was smaller than 0.10 or greater than means that the control groups have caught up 0.90. (See Crump et al., 2009). over time with the treatment group. Since the impact on mathematics scores in 2013–2015 As discussed in previous sections, the distinction is insignificant, the two groups were statisti- between the intervention and nonintervention cally indistinguishable in 2015. However, it is schools was quite subtle in reality. Many of the impossible to deduce any clear conclusion on nonintervention schools learned VNEN pedagog- Vietnamese since the two impact estimates are ical methods from the intervention schools over both insignificant. time, which makes the initially defined treatment less sensible in the above estimation. In Panel B, we estimate the impact of VNEN implementation without priori defining treat- To overcome, an implementation index was con- ment and control schools. Interestingly, the structed to measure the degree to which VNEN impacts on mathematics and Vietnamese are was introduced (see Section 4.3), and the index significantly positive in 2013–2014. The mar- will be directly used in the second approach. The ginal effects are almost the same in both sub- implementation index is a continuous variable jects. In 2013–2015, though the impact on that measures the VNEN practice. Note that the Vietnamese scores is insignificant, we found index itself, by construct, captures a change of that the point estimates are smaller than that of pedagogical methods as VNEN was not imple- 2013–2014, which is consistent with the aver- mented at any school prior to the intervention, age treatment effect estimates in the first panel. which fits in the first-differenced form too. Here Since the standard deviation is normalized as we do not apply the propensity score-based unity in the implementation index, the short- weights. term impact is substantial in the two subjects (a Standard Deviation (SD) increase of the index Outcome variables of interest are individual- changes the mathematic score by 0.18 SD and level mathematics and Vietnamese test scores the Vietnamese score by 0.16 SD). described visually in the prior section. Two dif- ferent periods were used to investigate the The results in Table 6.2 clearly show that (i) the dynamic nature of VNEN impacts: 2013–2014 impact is more significant in mathematics than and 2013–2015. In all estimations, we control the Vietnamese, but in the short run, VNEN had a sig- initial-stage or predetermined school, students, nificant impact on both subjects, and (ii) the non- and parents’ characteristics. (See Table 6.4 at the intervention schools seemed to catch up over end of this chapter for the list of control variables time with the intervention schools. Again, the used.) second point is indirectly supported by insignifi- cant estimates in the first panel that relied on the Table 6.2 displays the estimated impacts on initially defined treatment versus control schools. mathematics and Vietnamese test scores. Panel A shows the average treatment effect Far reaching education reform usually takes and Panel B, the implementation effect. First, time to develop, and the higher level of impact the average treatment effect is significantly between the second and third year of the pro- positive for the change in mathematics score in gram (2013–2014) as compared to the third and 2013–2014. The impact on Vietnamese scores fourth year (2014–2015) requires further research was insignificant. Second, we observe that to be carried out. Enhancing School Quality in Vietnam through Participative and collaborative Learning 101 Cognitive and Non-Cognitive Student Outcomes Table 6.2  Impact on Learning Outcomes Panel (A) Dependent Variable: Change in Math 2013–14 Math 2013–15 Viet. 2013–14 Viet. 2013–15 (1) (2) (3) (4) VNEN 13.06082* 6.508364 −0.1917586 −4.926378 (1.78) (1.03) (0.03) (0.95) Number of students −0.0036567 −0.0018838 0.0034362 0.0115182 (0.41) (0.22) (0.38) (1.61) Female 9.048301*** 3.056733 15.66617*** −7.882858*** (3.49) (1.26) (5.95) (3.69) Ethnic Kinh 4.393798 0.1755553 9.437553* −9.080281* (0.92) (0.04) (1.71) (2.05) Other controls included yes yes yes yes Number of obs 5,206 5,205 5,206 5,204 F (114, 5091) 3.96 3.92 7.89 10.67 Prob > F 0.0000 0.0000 0.0000 0.0000 R-squared 0.1174 0.0861 0.1686 0.2042 Root MSE (Mean Squared 90.366 84.104 90.066 72.541 Error) Panel (B) Dependent Variable: Change in Math 2013–14 Math 2013–15 Viet. 2013–14 Viet. 2013–15 (1) (2) (3) (4) Index 18.12793*** 8.186649*** 15.91217*** 2.509689 (5.60) (2.66) (4.46) (0.91) Number of students −0.003316 0.0010311 0.0017258 0.0088508 (0.38) (0.13) (0.20) (1.27) Female 9.591044*** 3.908525* 16.30728*** −7.247989*** (3.81) (1.67) (6.55) (3.61) Ethnic Kinh 6.027562 1.738576 9.724907* −7.934034* (1.22) (0.37) (1.79) (1.95) Other controls included yes yes yes yes Number of obs 5,299 5,298 5,299 5,297 F (123, 5175) 3.75 4.14 6.90 11.36 Prob > F 0.0000 0.0000 0.0000 0.0000 R-squared 0.1051 0.0914 0.1538 0.2022 Root MSE 90.621 84.257 89.564 71.991 Note: *p < 0.1; **p < 0.05; ***p < 0.01 Source: VNEN Impact Evaluation study primary data, Authors calculations. 102 www.worldbank.org Measurement issues could be one explanation 6.3.1  Estimation of Non-Cognitive of the pattern across the two years—with the Outcome cohort based measurement being used here, the In order to be able to capture meaningful infor- top performing students may still be improving mation about a child’s non-cognitive develop- but the measurement may not be able to cap- ment we asked parents about the following ture it. There is a further possible explanation. It statements, using a scrambled list of items: is possible that the VNEN program has a great “Every child develops his/her competencies as novelty value which fundamentally alters the he/she gets older. Parents/guardians usually teaching-learning interaction. Topping, 2005 have expectations about their child’s develop- explains in some detail the strong cognitive and ment progress. Please compare your child’s com- affective impacts of peer learning. The author petency progress in the last three years with your mentions “the sheer excitement and variety of a expectation.” And for each of 21 items, we asked novel kind of learning interaction.” After a period the parents to rate on a four-point scale: ‘Much of time, it is possible that the novelty wears off higher than my expectation’; ‘Somewhat higher for both teachers and students and the instruc- than my expectation’; ‘Same as my expectation’; tion slips back to the previously ingrained pat- and ‘A bit lower than my expectation’. terns of habit, such as frontal instruction from the teacher. It is quite possible that for the VNEN pro- The items themselves were chosen so as to be gram to be truly effective over time, a sustained easily understood by parents and classified into innovative effort may be required, together with four categories as shown in Figure 6.7. The par- accompanying research.2 ticular framing of these statements allowed vari- ation to be generated in the responses, rather than every parent saying their child is obviously 6.3 Non-Cognitive the best on every dimension. The information Outcomes was converted into a 500 mean Rasch score for the individual constructs as well as the overall While the technology for standardized test non-cognitive assessment score. The person- scores is well established, the measurement item map shown in Figure 6.7 makes intuitive of non- cognitive outcomes is not as straight- sense. Helping parents with housework chores forward. To measure non-cognitive outcomes, and taking care of siblings was an ‘easy’ item, we depend on parent inputs regarding their checked off by most parents. Expressing ideas children’s behavior, the idea being that parents clearly and making plans and following through observe and interact with their children in mul- with them were among the most difficult items tiple situations and contexts. Parents, perhaps to rank. more than teachers, would likely be biased to see their children always in a good light, which While much further research would be required we have to account for in some way if we are to to validate and confirm the non-cognitive use a valid and accurate measurement. In this scores measured in this way, for our purpose section we present an overview of our measure it is sufficiently plausible that the technique is of non-cognitive assessment followed by an not biased in any systematic way toward the estimation of VNEN impact. VNEN or control groups. Figure 6.8 shows the Enhancing School Quality in Vietnam through Participative and collaborative Learning 103 Cognitive and Non-Cognitive Student Outcomes Figure 6.7  Construction of Non-Cognitive Outcome Score (a) Item response theory map Dimension Variable Description intra-personal md_intr_r Making plans and following through with them md_intr_t Neatness and tidiness with their personal belongings md_intr_d Curiosity about how things work md_intr_i Managing his/her own schedule md_intr_u Taking good care of personal hygiene without having to be reminded md_intr_n Responsibility (e.g., keep promise, admit mistakes, complete assigned tasks) social md_soci_c Confidence (e.g., in greeting parents' guest, child's guest at home) md_soci_h Persuasiveness md_soci_i Defending his/her point of views md_soci_l Proactively obtain help when they need md_soci_f Sharing objects with siblings/friends md_soci_p Helping parents with housework chores, taking care of siblings md_soci_g Getting along well with other children ethical md_ethi_m Being sensitive to emotional needs of others md_ethi_e Love of nature (plants, animals, river, mountain, etc.) md_ethi_q Being respectful to elders md_ethi_k Helping friends with their classwork comm md_ethi_e Love of nature (plants, animals, river, mountain, etc.) md_acad_s Making artworks/craftworks as a gift/use at home md_acad_v Express their ideas clearly (b) Item descriptions Source: VNEN Impact Evaluation study primary data, Authors calculations. 104 www.worldbank.org kernel density estimates of the resulting com- of the extreme groups (approximately 2,300 parison, centered on the mean of 500. We use children in each group), which are ‘pure’ VNEN the information regarding implementation of and control groups, with half of the children VNEN to look at three groups—children who (about 6,000) in the middle group. Figure 6.8 went to schools with an implementation index shows that the full VNEN group dominates at higher than 1 (see Figure 4.13) as ‘Full VNEN’, the lower end of the distribution, with scores children who went to school with an imple- being even toward the higher end, this phe- mentation index lower than −1 as ‘No VNEN’, nomenon being substantively an artifact of the and the intermediate group with indexes measurement technique. Table 6.3 presents between –1 and + 1. The sample children are regression results that show the significant distributed approximately one-fourth in each impact of VNEN on non-cognitive outcomes. Figure 6.8  Non-Cognitive Score Overall Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 105 Cognitive and Non-Cognitive Student Outcomes 6.3.2  Impact Evaluation of Non-Cognitive Outcomes Table 6.3  Estimating Impact on Non-Cognitive Outcomes Dependent Variable Non-Cognitive (all) Intrapersonal Social (1) (2) (3) newdef1 30.699*** 24.385*** 26.694*** (dummy int. VNEN) (4.502) (4.592) (4.678) newdef2 41.218*** 31.497*** 36.946*** (dummy full VNEN) (4.792) (4.888) (4.980) priority2 −73.908*** −79.979*** −58.087*** (Priority II & III) (10.059) (10.255) (10.451) totstum −0.025*** −0.016** −0.035*** (school size) (0.007) (0.007) (0.007) sch_wealth 7.778*** 7.185*** 5.743*** (school wealth) (1.504) (1.534) (1.563) st_tch −0.795*** −0.918*** −0.581*** (student to teacher ratio) (0.202) (0.206) (0.210) distance −0.351*** −0.103 −0.468*** from city center (0.110) (0.113) (0.115) stu_gen 13.916*** 14.268*** 11.118*** (girl is 1) (1.744) (1.778) (1.812) stu_ethnic −0.283 6.753** −0.013 (ethnic minority is 1) (3.250) (3.315) (3.376) Constant 555.307*** 549.982*** 552.708*** (10.144) (10.343) (10.539) Observations 10,768 10,761 10,765 R2 0.134 0.126 0.101 Adjusted R2 0.129 0.122 0.096 Residual Std. Error 252.430 257.334 262.254 (df = 10709) (df = 10702) (df = 10706) F Statistic 28.515*** 26.669*** 20.699*** (df = 58; 10709) (df = 58; 10702) (df = 58; 10706) Note: *p < 0.1; **p < 0.05; ***p < 0.01 Source: VNEN Impact Evaluation study primary data, Authors calculations. 106 www.worldbank.org Table 6.4  Additional Controls for Test Score Regressions i.st_14_q102a Gender of student ethnic_kinh Ethnic minority of student i.pa_14_q115a Mother’s education level i.pa_14_q115b Father’s education level tc_gen Teacher gender tc_age Teacher age tc_ethnic Teacher ethnicity tc_wlth Teacher wealth index tc_degr Teacher years of educational experience tc_trgds13 Teacher days of training in current school year tc_tenall Teacher years of teaching tc_tensch Teacher years of teaching at latest school tc_excel Level of excellent teacher award tc_postvol Teacher was voluntarily posted to his/her school tc_numlang Number of ethnic minority languages spoken by teacher tc_pweek Number of periods teaching per week tc_stuhelp Teacher beliefs that best students should support other students tc_stuqn Teacher encourages students to raise questions tc_stugps Teacher puts students in learning groups tc_stuself Teacher lets students self-study tc_sprestr Classroom has enough space for learning corners tc_furnr Teacher satisfied with furniture in classroom tc_iasst Frequency of informal cognitive assessments tc_fasst Frequency of formal cognitive assessments tc_pobs12 Frequency of classroom observations by head teacher/principal tc_tgobs12 Frequency of classroom observations by teacher of same grade tc_clumg13 Frequency of classroom observations by teacher of same school tc_pincls12 Frequency of parents participating in teaching activities tc_pmuseful Usefulness of parents participating in teaching activities tc_hrs_class Time spent teaching in class tc_hrs_prep Time spent prepping for teaching tc_hrs_oth Time spent on other things (excl. teaching and preparations for teaching) tc_arti13 Index on classroom furniture and infrastructure p_wv_polop Index on principal world values (view on politics, work, leisure, pride) p_wv_posap Index on principal world values (view on positive approach to life) p_wv_injun Index on principal world values (information junkie) p_wv_cohecu Index on Confucian heritage values p_310_hcv Whether principal is proactive p_psi_sa Index on principal management style—social astuteness p_psi_ii Index on principal management style—interpersonal influence p_psi_na Index on principal management style—networking ability p_psi_as Index on principal management style—apparent sincerity pri_msr_13_05 IRT score on principal’s pedagogical attitudes schlships Measure of need based student scholarships msr_p701 IRT score on principal’s innovativeness Source: VNEN Impact Evaluation study primary data, Authors calculations. Enhancing School Quality in Vietnam through Participative and collaborative Learning 107 Cognitive and Non-Cognitive Student Outcomes TakeAways: Cognitive and Non-Cognitive Student Outcomes This chapter discussed the findings regarding NON-COGNITIVE ACHIEVEMENT student outcomes for both cognitive and non- cognitive abilities. As the baseline cognitive test- • Weconstructed measures of non-cognitive abili- ties on four dimensions: intrapersonal, social, ing was conducted in December 2013, approxi- ethical, and communication. Scores for each mately eighteen months after the GPE-VNEN student were based on parental reports of the project started, the parity between VNEN and growth of children’s abilities over the three years control groups prior to the start of the project was of the program. For each of the individual con- established using administrative census data. structs, as well as a combined measure of non- The chapter provided evidence of superior VNEN cognitive skills, VNEN students were performing outcomes for both cognitive and non-cognitive better than the control group students. The abilities. Variation in implementation intensity of effect sizes were quite large, above one-third of VNEN implied that the impact evaluation based a standard deviation in some cases. An interest- on a binary definition may not be complete. We ing finding for both non-cognitive and cognitive introduced a method using the implementation abilities, was that girls performed better than index to account for the variation in implemen- boys on average. tation. The methodological details, including a discussion of why attribution can be made in the quasi-experimental setting rather than mere cor- IMPLEMENTATION INDEX relation, is planned to be presented in forthcoming • An elaborately constructed VNEN implemen- academic research reports. tation index was able to discriminate quite effectively between VNEN and non-VNEN VIETNAMESE AND MATHEMATICS TEST schools. We used the implementation index SCORES instead of the program participation in some of the regression specifications. The effect • Standardized tests were taken by the same considering only the binary variable for pro- cohort of students from Grade 3 to Grade 5, gram participation was statistically insig- over approximately two years. The tests were nificant in some cases, and the effect size designed so that item response theory (IRT) was only as high as one-tenth of a standard based measures could be used, with a national deviation. However, when we incorporated the mean of 500 and a standard deviation of 100. implementation index, the effect size increased As the students progressed through the grades, almost to one-fifth of a standard deviation in their scores would be expected to grow, and the mathematics for 2013–14, and more than one- research question of interest was to compare third of a standard deviation for non-cognitive the performance of students from VNEN and outcomes. control group schools. The evidence indicated that students from VNEN schools on an aver- The VNEN program is seen to lead to superior out- age, kept up to the same level or exceeded the comes regarding both non-cognitive and cognitive growth of control group students. abilities. The methodologies used to arrive at this result need to be investigated further through aca- demic research. 108 www.worldbank.org Endnotes 2. Thanks to the financial support from Research on Improving Systems of Education a sample of the cohort of students followed in the present study will 1. We utilize a variant of the methodology proposed continue to be tracked over the next few years. See by Hirano, Imbens, and Ridder, 2003. Their method further discussion in the conclusions and recommen- calls for weighting by the inverse of the nonpara- dations section. metric estimates of the propensity score. Their paper explains how the method is more efficient than sim- ply comparing differenced treatment and control means. See also Wooldridge, 2007. Enhancing School Quality in Vietnam through Participative and collaborative Learning 109 Findings from the Qualitative Component 7 This chapter summarizes the qualitative com- cognitive and non-cognitive outcomes. Building ponent results in six sections. The first section on this premise, the objective for the qualitative provides a brief description of the methodology study is: “To clearly describe and illustrate what used for conducting the qualitative research. The are advanced VNEN outcomes and to determine second section gives an overview of what occurs how they might have been achieved.” in the classroom, as captured through coded les- son videos, including the approaches and activi- To address the objective, two overarching ties used. The third section presents the extent research questions are asked: and level of the adoption of key 21st century skills across VNEN schools and classrooms, and 1. What do we mean by desired outcomes of the fourth section details those skills regarding VNEN in the classroom and in the school? how they were enacted in VNEN classrooms. The 2. What are the factors that can explain how fifth section sheds light on both the extent and these desired outcomes have been made pos- nature of the use of learning guides by VNEN sible in practical experience? teachers. The final section provides stakeholder insights and perceptions of VNEN regarding its For the first question, the study developed meth- benefits and challenges. While the study findings ods centered on analysis of observations by video have been condensed and summarized for this and interviews to define, for future reference, what chapter, full results and a more detailed meth- it means in practical terms for children to develop odology of the qualitative study are to be the competencies in the focal areas of: (i) leadership, subject of a future research paper. (ii) teamwork and cooperative learning, (iii) com- munication, and (iv) self-­managed learning. VNEN 7.1  Overview of Qualitative encompasses an ambitious array of objectives with regard to sought after outcomes. These are Component also new and modern outcomes, going beyond the traditional learning of facts explained by the The qualitative research was designed to provide a teacher. Competencies like leadership are very deeper understanding of how the VNEN program important for the future in the implementation of has been implemented in schools and to what the fundamental and comprehensive education extent and the ways in which it has been adopted. reform, and the study hopes to provide a useful reference from the experience of the VNEN model One of the critical findings from the quantitative by investigating: analysis, through creation of the VNEN imple- mentation index, is that there is a great deal of a. How are teachers implementing the standard variation in the extent of VNEN adoption and that elements of the VNEN practices? the level of adoption has a strong relationship with student outcomes. The schools that more b. Why do teachers use certain practices and fully adopted the program tended to have higher teach in the ways they do? 111 Findings from the Qualitative Component Figure 7.1  Map of Analytical Components Source: VNEN Impact Evaluation study primary data, Authors schematic representation. c. What are the factors for students in the adop- advanced). An example snapshot of a partial cod- tion of VNEN? ing tree is shown in Figure 7.4 at the end of this chapter. The first step in analyzing teaching and d. What role do the parents play in the VNEN learning classroom practices was to gather sum- model? maries of measures of the coded lessons of both The qualitative analysis involved 45 teachers, VNEN and traditional classrooms. The purpose of 15 principals, 72 parents, and 393 students of these summaries was to ascertain what occurred students of grades 3–5 across 15 schools. Three in the classroom as well as how VNEN may have mathematics and three Vietnamese lessons were changed the way the lessons functioned. To do filmed for each teacher, for a total of 270 lessons. so, three different dimensions were examined— For each lesson there were three cameras, with lesson structure, instructional methods, and type one focused on the teacher and two focused on and extent of classroom interactions. two different student groups. In total there are 810 videos. Multiple tools were utilized to cap- ture the necessary information in the case stud- 7.2.1  Lesson Structure ies (see Figure 7.1). Lesson structures were analyzed by comparing the lesson times, which were classified by activity type and segment purpose, between VNEN and 7.2  Teaching and Learning traditional classrooms. It was found that VNEN Practices in the Classroom classrooms spent much less time on whole class activities and more time on group work than For analysis of the videos, a coding tree was traditional classrooms. In addition to spending developed that is made up of approaches approximately a third of the lesson for group and activities used in the classroom. Often the work, VNEN classrooms presented a more bal- activities and approaches have key attributes anced picture of students working individually, associated with them which provide further in groups, and as a class. In contrast, traditional detailed breakdowns and specify quality or lev- classrooms spent nearly two-thirds of the total els of advancement (i.e., basic, intermediate, lesson time on whole class activities and slightly 112 www.worldbank.org over a quarter on individual work but little time 7.3  VNEN Practices on group and pairs work. Regarding segment purposes, VNEN classrooms spent approxi- and 21st Century Skills mately the same amount of time as traditional in the Classroom classrooms on practicing. However, VNEN classes devoted more time to exploration, whereas tra- An important element of the VNEN model is that ditional classes allocated more time for review. student learning is not only cognitive but also involves the development of what have been termed as 21st century skills. In VNEN, some of 7.2.2  Instructional Practices these skills include leadership, teamwork and VNEN and traditional classrooms used both similar cooperative learning (T&CL), communication, and distinct instructional methods during lessons. and self-managed learning (SL). The lessons were For example, both types of classroom used prob- examined to capture instances in the classroom lem solving and exercises for three-fourths of all where such skills might have been developed lessons. However, the difference was that VNEN and utilized. In doing so, the extent to and level classrooms spent more time on discussion but at which practices occurred to facilitate these less time on lecturing than traditional classrooms. skills in VNEN classrooms were reviewed relative to traditional ones. An analysis also determined whether the time spent and the level of advance- 7.2.3  Type and Extent of Classroom ment in these 21st century skills’ activities varied across VNEN schools and classrooms. To account Interaction for variations in the adoption of VNEN practices, The lessons were examined to determine the teacher characteristics were examined to discern ways in which teachers and students were inter- what might influence their receptivity to and acting. This was done by classifying the types of effectiveness in implementing VNEN. questions asked by teachers, their level of com- plexity, and the extent of opportunities for stu- Extent and level of adoption of VNEN practices dents to address questions to teachers and other and their effects on 21st century skills: In this students. In both VNEN and traditional class- study, VNEN classrooms provided significantly rooms, test questions were the most common more time for students to practice and develop type asked by teachers. Nonetheless, compared key 21st century skills (leadership, teamwork to traditional classrooms, VNEN classrooms had and cooperative learning, communication, and much more variation in types of questions, with self-managed learning) at a higher level than a greater use of guiding and directing questions, what was offered in traditional classrooms (see genuine questions, and progress questions. In Figure 7.2). This result is neither surprising nor terms of complexity, most questions from both should it be considered a critique of the tradi- VNEN and traditional lessons fell into the low and tional model. VNEN was designed to give stu- medium levels; VNEN lessons had a small propor- dents more opportunities in these areas, and the tion of high complexity questions, whereas tra- results indicate that the design is being success- ditional lessons did not. VNEN classrooms also fully adopted. tended to facilitate significantly more opportu- nities for students to interact with other students Regarding quality, levels of advancement in as well as with teachers. On average, VNEN stu- VNEN classrooms were generally high, with dents posed approximately eight times as many most at either the intermediate or advanced questions to teachers and nearly nineteen times level. This indicates that teachers and students as many questions to other students as did their can grasp the model and move beyond either traditional counterparts. Enhancing School Quality in Vietnam through Participative and collaborative Learning 113 Findings from the Qualitative Component Figure 7.2  Time Spent per Lesson and Level of Advancement for VNEN 21st Century Skills Source: VNEN Impact Evaluation study primary data, teacher camera videos of lessons. a rudimentary or ineffective adoption of the may still perform VNEN activities frequently and practices. at an advanced level. Finally, some teachers spent a significant amount of time on activities in one Variation across VNEN schools and classrooms: area of the 21st century skills, yet little on oth- The time spent on key 21st century skills var- ers. In summary, the adoption of the VNEN model ied across VNEN schools and classrooms. When varied both by school and by teacher. examining VNEN lessons on a school-by-school basis, some schools spent a significant amount Teacher characteristics that help account for dif- of lesson time on activities that were designed ferences: Both VNEN adoption and advancement to practice all four categories of the key 21st cen- levels were influenced by the characteristics tury skills, while others did not. When examining of the teachers who implemented VNEN. One lessons within VNEN schools, similarities across important teacher factor that emerged was age. grades emerged, with certain exceptions of high Lessons were grouped into levels of advance- variation, which suggests a school-level effect ment categories, and teacher characteristics where teachers in the same school followed were then examined based on those categories. similar practices. Nevertheless, there were a few Younger teachers tended to have a higher pro- contrasting examples, such as one teacher within portion of lesson time that utilized VNEN prac- a school implementing VNEN in a very different tices, and they tended to be at a more advanced way than two other teachers. These examples level. Teachers at the basic and basic-to-middle indicate an important teacher-level effect in the levels had an average age of over 40, while the implementation of VNEN: A teacher in either a middle, middle-to-advanced, and advanced lev- generally non-supportive environment or one els had an average age below 38. Although it is where VNEN is not integrated into school culture important to not overanalyze these differences, 114 www.worldbank.org age distribution may indicate that more expe- students’ own and others’ opinions and feelings. rienced teachers may be more set in their ways Asking group members questions was the most and find it more difficult or be less willing to used method of leadership. However, most of the modify their practices, whereas younger teachers questions were not creative and simply followed might be more flexible. Interestingly, the middle the learning guide. In many cases, the leader took category had the lowest average age, with slight on a facilitative role and thus encouraged and increases seen when moving toward advanced. allowed members to solve problems. In many This may indicate that some experience is ben- lessons, the leaders were high-performing stu- eficial for optimal implementation. dents; they quickly completed individual work and then guided others. This raises the issue of a need for leader assignment tradeoffs vs. rotation 7.4  The Deep-Dive: to ensure that all students have opportunities to Exploration on What Takes lead. The types of questions and answers used Place for 21st Century Skills when leadership was deemed to be advanced tended to go beyond the learning guide ques- and Why It Takes Place tions. They also tended to be open and encour- aged opinion sharing and discussion beyond It is important to understand what occurs when simple answers. 21st century skills emerge in lessons and what factors lead to either more or less advanced lev- How leadership was either enabled or hin- els. Using five (four VNEN and one traditional) dered: VNEN enabled leadership by separating selected schools, lessons were explored in the class into small groups, choosing student greater detail to identify markers that indicated group leaders and committee leaders, and oblig- the level of advancement for each of the key ing those student leaders to run group and/or VNEN skills: leadership, teamwork and coopera- class activities, including creating more suitable tive learning, communication, and self-managed activities, such as a warm-up activity, a sharing- learning. Lessons were examined for the contexts the-lesson’s-objective activity, and group activi- in which each of these skills occurred, their ben- ties, to improve and nurture leadership. efits and downsides, the ways in which they were either enabled or hindered, and ways to improve Benefits and downsides of leadership: One their effectiveness. The markers and identified of the benefits of more advanced/high student video instances from both the teacher and stu- leadership, based on the lesson observations, dent perspectives can be leveraged when train- was that, when the leader aptly involved every- ing teachers on how to encourage and support one, there were more chances to share ideas the development of non-cognitive skills. and discuss differences. Additionally, when the leader was able to relate the questions to real-life events, other members appeared more enthu- 7.4.1 Leadership siastic in the discussion. During group work, Contexts of leadership: Leadership appeared students seemed to gain more valuable experi- more often in whole class and group activities ences regarding good leadership and how to that included a specifically assigned “leader” handle difficult, unexpected situations. Some compared to pairs activities. Leadership in both students were both more active and more pro- whole class and group activities exhibited simi- active in contributing ideas to group discussions, lar characteristics, such as asking group mem- and all students shared more readily when the bers questions, assigning tasks to each member, teacher suggested what to discuss and how to reaching consensus, and reflecting on the Enhancing School Quality in Vietnam through Participative and collaborative Learning 115 Findings from the Qualitative Component Video examples of leadership demonstrated by students Example (a) Clear lead role in question and answer: The leader had a clear role in leading the question and answering task on the sports in the pictures. All group members participated. After being interrupted by the teacher, the leader actively continued with exercise 3 by expressing her own opinion and receiving feedback from every member. Example (b) Discussing opinions: The leader asked the rest of the group questions in the exercise (which was supposed to be a pair activity) and everyone raised their hands. She chose one student to answer, then asked if there were any other opinions. She asked another student just to make sure the answer was right. When there was a conflict in the answers, she calculated herself and concluded the correct answer. One student actively shared ideas on another method of solving the question. The leader agreed that there were various ways of solving the questions. Video examples of teachers supporting leadership Example (c) Sharing work: One student shared her work in front of the class. She spoke loudly and clearly, but when asked to comment on the exercise, she could not answer. The teacher then gave her hints, and only when she still could not answer, the teacher asked another student to help her answer the questions. The teacher thanked the students and concluded the answer. Example (d) Questions to finalize answers: After asking questions in pairs, the leader asked her group the same questions to finalize the answers. Group members volunteered to answer and also expressed their opinions on how the answers should be. The teacher only observed the group and did not interfere as the group was doing well on its own. do so. Despite these benefits, the downside was • Teachers should encourage student leaders to that in some instances both the leader and the take a more active leadership role to encour- group members became somewhat mechani- age others to engage more in discussions. cal in group activities, especially when asking and answering questions. Only one case had a • Teachers should encourage academically stronger students to assist weaker students. flexible leader who facilitated the group discus- sion differently. Possible other downsides for stu- • Teachers should encourage students to com- dent leaders include that other members might municate when disagreements occur, and become dependent on the leader (e.g., only they should not immediately offer the cor- speaking when being asked to; not contributing rect answer. ideas). If leadership is only basic/low, students may feel that learning is an obligation and lose • Leaders should develop methods for gaining every member’s attention and elicit participa- interest. tion in group activities. How could leadership be made more effec- • Leaders should be educated on tips for han- tive? Ways to improve leadership and make it dling group conflicts and distracted members. more effective include, but are not limited to, the following: • Leaders should be given ample time to con- sider the optimal ways to run group work 116 www.worldbank.org well and to provide effective demonstration How teamwork and cooperative learning were instead of directly giving commands. either enabled or hindered: VNEN enabled T&CL by organizing the class into groups, which encouraged (and obligated) students to work 7.4.2  Teamwork and Cooperative together. T&CL appeared to occur either when Learning activities involved sharing opinions and feelings Contexts of teamwork and cooperative learn- or when there were differences among answers, ing: Most of the evidence on Teamwork and Coop- which forced students to cooperate. Teachers erative Learning (T&CL) came through group provided students opportunities for T&CL by activities, which was mainly because there were observing group work and interacting with the more group activities than pairs activities. There groups only when they either needed guidance tended to be more discussion due to more com- on activities or required answer clarification. The plex questions in the group activities, whereas teachers occasionally reminded the group to the pairs activities were often simple (e.g., cross- work cooperatively by asking if they had held checking of each other’s results). There were few a group discussion. Some teachers also created observations regarding T&CL during whole class chances for T&CL by having academically stron- activities because most of those were questions ger students assist weaker students with exer- and answers between either one student leader cises. Each group usually had a few students who or the teacher and the rest of the class. T&CL truly enjoyed teamwork and actively raised their usually appeared in exercises that required stu- hands, while other students were passive and dents to ask and answer questions, which were appeared to answer questions out of obligation. generally already available in the learning guide. Hence, the questions and answers were often Benefits and downsides of teamwork and taken from the materials in the learning guide cooperative learning: Students often under- and rarely truly creative. Characteristics of T&CL stood lessons on a deeper level and tended to that were identified to be more advanced/high find them more fun if they truly enjoyed team- included actively and enthusiastically participat- work and were actively sharing and receiving ing in group discussions, all members working in ideas. However, teamwork might not be as effec- a cooperative manner and trying hard to achieve tive for timid students who do not enjoy work- results, interest in the discussion content, and ing with others; they might work alone more going beyond the learning guide. Characteristics effectively. of T&CL identified as basic/low included requir- ing a significant amount of teacher assistance, How could teamwork and cooperative learn- no discussion, and students working individually ing be made more effective? T&CL could be instead of together on group activities. more effective if the student leaders, instead of Video examples of teamwork and cooperative learning Example (a) Active discussion: Students were having a discussion where all participants were actively contributing. They politely took turns and provided an opportunity for every member of the group to ask questions as well as to listen while others were speaking. Example (b) Cooperating to finalize an answer: Students were demonstrating good coopera- tive skills to come up with an answer. Some students had a better grasp of an issue and the others listened, while contributing on their own at another stage of the discussion. Enhancing School Quality in Vietnam through Participative and collaborative Learning 117 Findings from the Qualitative Component asking questions all the time, would allow other sharing personal ideas using original sentences. members to ask questions as well; that way, they Characteristics of basic/low communication skills would feel more equal, less dependent on the were identified as only answering reluctantly leader, and more eager to share their opinions. and/or mechanically, repeating what others had Regarding the teachers’ perspectives, T&CL could said without having one’s own opinion, arguing be more effective if teachers interacted with the about unimportant issues (e.g., why one student group only when necessary, such as when it was had more cards than another), and not commu- not active enough. The group results showed nicating when it was required. that the teachers should focus more on low-­ performing students, who might not understand How communication was either enabled or the lesson yet feel too timid to ask others for help. hindered: VNEN enabled communication by Teachers should also allow students to fix each creating activities where students had to work other’s mistakes before explaining the solutions together and communicate with each other. themselves. Communication skills were often improved through discussing feelings, stating opinions, and sharing ideas and other issues that are not 7.4.3 Communication simply found in books. Contexts of communication: Most high-level communication tended to appear in group Benefits and downsides of communication: activities while doing exercises. Communica- Through communication, the students learned tion between pairs was usually short because from different perspectives (different ideas, opin- the difficulty for pair activities was rather low. ions of other students, different ways to solve a Communication skills in whole class activities problem). They learned to be more comfortable were generally less creative because the answers and confident when speaking in front of an audi- tended to be predetermined. Conversely, in ence over time. The apparent downsides were group communication, the activities tended that some students did not express an opinion to function more in the form of back-and-forth but rather only repeated answers from others. discussion and there were more chances for students to actively express their own opinions. How could communication be made more Most communication was in the form of asking effective? From the student perspective, com- and answering questions that were taken from munication skills could be more effective when the learning guide. The discussions were livelier the leader is able to elicit everyone’s opinions when disagreements emerged between stu- and come to a consensus on a problem. From dents, and they needed to determine who was the teacher perspective, if the teacher was able wrong and why. Both the whole class and group to devise creative questions that were thought activities had students who were enthusiastic provoking, then the students would tend to be and who actively raised their hands to state their engaged in a more interesting discussion. Teach- opinion and communicate with others. Notably, ers should avoid interrupting students’ discus- there were a few who were timid and thus com- sion flow and should suggest topics related to municated in a quiet voice, and they only did the lesson that would be interesting for students so when asked. Students other than the leader to discuss. mostly answered the teacher’s and student lead- er’s questions but rarely had questions of their own. Characteristics of more advanced/higher 7.4.4  Self-Managed Learning communication skills were identified as speak- Contexts of self-managed learning: As ing clearly, fluently, and confidently, and actively would be expected, self-managed learning (SL) 118 www.worldbank.org Video examples of communication Example (a) Actively sharing ideas with own thoughts: Students were going beyond just dis- cussing the given question and elaborating on the topic. Example (b) Teacher facilitating group discussions: Students were going through the motions to answer questions. The teacher enabled students’ opportunities for exercising their communi- cation skills by doing the following: reminding the group to discuss between themselves, asking one student to help out another in figuring out how to do the exercise, and asking if the group had discussed about a particular topic. activities mostly appeared during group and dividing classes into groups so that students pairs activities. While there was almost no SL could work at their own pace. SL very much during whole class activities in terms of learn- depended on the group leader actively leading ing material, the warm-up activities presented a group activities, according to each member’s subtle form of SL from the standpoint of students pace, and adjusting activities to achieve the great- choosing games and other activities. SL mostly est group benefits. Teachers enabled opportuni- appeared when the group leader would move on ties for SL by only interfering with group work if to a new activity after finishing the previous one necessary; they sometimes hindered SL by telling without the teacher’s reminder. It was identified students which activity to do next. as being more advanced/high when a student or a designated leader actively changed the form of Benefits and downsides of self-managed the activity so that it suited the group work (e.g., learning: The benefits of SL are that a group can actively read a poem prior to any teacher instruc- save time by working at its own pace and not tion, discussed content before it was taught, dis- having to wait for other groups to finish. A group cussed how to do the take-home exercise). In could also devise either extra activities or other many lessons, student leaders actively moved methods for operating activities that would suit on to the next activity and operated the group them best. Learning styles differ among students without guidance from the teacher. The pat- and while most appear to operate effectively on tern of SL was that, after finishing one activity, self-managed tasks, a few tend to struggle with if the teacher did not give a reminder, the leader the approach and the added responsibility. This would actively let the group move on to the next could result in them becoming disengaged and activity. learning less. How self-managed learning was either How could self-managed learning be made enabled or hindered: VNEN enabled SL by more effective? SL would be more effective if Video examples of self-managed learning Example (a) Moving to the next topic without teacher guidance: The group did not wait for instructions from the teacher, and moved on to the next activity on their own. Example (b) Use of learning corner: Students finished their task and automatically got materials from the learning corner for the next activity. Enhancing School Quality in Vietnam through Participative and collaborative Learning 119 Findings from the Qualitative Component teachers would only interfere when necessary. spent on a new activity (e.g., the teacher intro- They should otherwise let students operate the duced an activity not specified in the guide), and activities themselves and actively encourage stu- 4% represented a change in content. Although it dents to do activities in a SL form. Teachers could was anticipated that activities might be skipped, let groups decide how they would like to operate this rarely happened. Regarding quality of activi- activities and only participate to either confirm ties that deviated from the learning guide, 40% the results or offer further guidance. of the time was considered high, 44% medium, and only 16% was considered low quality. This indicates that deviations often benefited and/ 7.5  Extent and Nature or enhanced the lessons. While generally simi- of Use of the Learning Guide lar, there were some interesting differences in deviation by grade. Grade 3 had a much larger The learning guide is an important element of proportion of deviation from the learning guide the VNEN model regarding what occurs in the regarding additional activities (45%) than other classroom. It provides the content to be learned grades (26% in Grade 4 and 19% in Grade 5). as well as the manner in which it could be con- However, Grade 5’s deviations, at 78%, typically ducted. However, teachers are not only given the involved operations. Quality of activities also opportunity to modify the learning guide’s activi- tended to rate higher in Grade 3, with 49% per- ties as they see fit but are even trained on how to cent considered as high quality compared to effectively do so. This policy recognizes that each 34% in Grade 4 and 38% in Grade 5. class is different, and what might work in one environment may not work as well in another. Findings from interviews: For many VNEN This section examines how often teachers devi- teachers, the learning guide has been a welcome ate from the learning guide and what takes place change from the traditional textbooks they used when they do. previously. Teachers at the Case Study 6 (CS6) school commented that the learning guide is Findings from video analysis: Figure 7.3 sum- more flexible and structured more logically than marizes the results of the video analysis, in which the previous textbook. The fourth grade teacher the lessons of VNEN classrooms were examined at CS6 found the learning guide particularly help- to determine the types and extent of modifica- ful in teaching Vietnamese because of the logi- tions to and/or deviations from activities speci- cal presentation of its content, which includes a fied in the learning guide and their levels of sequence of reading, vocabulary practice, and effectiveness. Overall, 23% of lesson time was dif- sentence construction. Teachers in CS7 and CS9 ferent from the learning guide. Of this time, 67% remarked that the learning guide, with its color- differed in terms of operations (e.g., assigning an ful pictures and clear fonts, has been received activity in pairs instead of individually), 29% was well both by the teachers and students. Teachers Figure 7.3  Percent of Time Not Following the Learning Guide and Its Effectiveness and Type % off Low Med High Operation Content Addition Skip Overall 23% 16% 44% 40% 67% 4% 29% 0% Grade 3 20% 22% 29% 49% 55% 0% 45% 0% Grade 4 23% 10% 56% 34% 66% 9% 26% 0% Grade 5 26% 17% 45% 38% 78% 3% 19% 0% Source: VNEN Impact Evaluation study primary data, Authors calculations. 120 www.worldbank.org in CS7 and CS9 also commented that they found types of activities conducted, with few changes that the “three-in-one” functionality of the learn- to the content itself. Similar to CS9 teachers, CS12 ing guide very useful; it is designed to be used teachers sometimes changed various types of as a textbook, exercise book (for students), and activities. In a math lesson that was filmed and teaching guide (for teachers). In addition to the reviewed in the stimulated recall interview with sequential order of the learning guide and its the third grade teacher, she stated that she presentation, the content itself matters as well. switched individual work (as instructed in the The fifth grade teacher at CS9 commented that learning guide originally) with group and whole she is content with the learning guide because class activities, because she felt that some stu- it provides accurate and current information. In dents would not understand the content fully if contrast, the third grade CS3 (traditional school) they worked individually first. The fourth grade teacher stated that because some questions CS12 teacher explained that in choosing the listed in the textbook are out of date, teachers types of activities appropriate for her students, have to make them fit with the current context. she bases her strategies on the level of difficulty of the exercise; easy ones can be worked on indi- Teacher comments on the patterns of adjust- vidually, but difficult ones are more suitable for ments corroborated the findings from the video group or whole class activities. analysis. Teachers rarely change the content of the learning guide but mostly adjust the opera- Another way to adjust the learning guide is to tions or carry out additional activities. One of add activities or questions that are not specified the few examples of teachers adjusting con- in the guide. For example, the stimulated recall tent included the third grade teacher at CS6, with CS12’s fifth grade students revealed that who stated that she makes minor modifications their teacher had not only modified the opera- to the content occasionally when it is not suit- tion by switching paired work with group work able or relevant in the local context. The fourth but also added an individual activity before the grade teacher at CS4 said that, although he does students shared their answers in groups in their not feel the need to adjust the content much, math lesson. The observation of the lesson videos he occasionally adds conclusions at the end of also confirmed that CS12 teachers often make the lesson so that his students can have a better deviations by adding more activities before or grasp of the main theme of the lesson. However, after students work individually; these deviations most of the teachers interviewed stated that they generally improve student-teacher interactions. rarely make major changes to the learning guide In another example, in her stimulated recall inter- content. view about a math lesson, the fourth grade CS12 teacher stated that she developed some addi- A common way to adjust the learning guide is tional questions for students to discuss, with the to change the types and/or sequence of learning goal of facilitating group work. The third grade activities. For example, the third grade teacher CS12 teacher indicated that she sometimes poses at CS9 commented, “I do not adjust questions, several sub-questions, in addition to the ques- but only the types of learning activities. For tions set out in the learning guide; this enables example, if an exercise is designed for students students to answer each question more easily to work individually but my students do not and understand the content systematically. The have the required level of academic capacity to fifth grade CS6 teacher takes a similar approach tackle it on their own, I will change it to a pairs by breaking complex questions into smaller ones or group activity so they can support each other so that students can answer them more easily in working on the exercise.” Other CS9 teachers and understand the content more fully. echoed that most of their changes are in the Enhancing School Quality in Vietnam through Participative and collaborative Learning 121 Findings from the Qualitative Component Some teachers, however, expressed reservations said, “The advantage is that the students have about making spontaneous adjustments to the more opportunities to study in groups. This learning guide. At CS2 (one of the basic level allows them to discuss and solve the problems VNEN schools), the fourth grade CS2 teacher together, and also to become more confident.” A remarked that she is hesitant to make modifi- fifth grade CS6 teacher indicated that the VNEN cations because any change she might make model helps students to learn more proactively, needs to be discussed with other teachers. The be more creative, and acquire knowledge by fifth grade CS2 teacher said bluntly that making themselves. As a result, students learn more adjustments is too much of a task for teachers deeply and the knowledge they gain stays with and that they are not “researchers.” It was difficult them for longer periods of time. Similarly, a fifth to determine to what extent VNEN teachers are grade CS15 teacher stated that the VNEN model obliged to discuss any possible changes, how- assists students to remember what they learned ever minor, with other teachers and/or princi- better because this new model emphasizes self- pals. Nonetheless, given the fact that flexibility discovery and exploration in learning. In the tra- in making adjustments appears to have positive ditional model, students would try to memorize effects on student learning, more examination everything but would quickly forget what they will be necessary to find out the way in which studied. the rigidness of the policy for internal discus- sions on adjustments might affect the teacher’s Parents also cited various positive behavioral motivation to modify/devise the learning guide changes in their children. Some CS12 parents to better accommodate the needs of students indicated that their children have become more and their level of understanding. industrious and happier about going to school; others said that the children demonstrate more self-confidence and are more comfortable with 7.6  Stakeholders’ Views public speaking. Similarly, CS6 parents com- on Benefits and Challenges mented that their children have not only become of VNEN Implementation confident, but also improved self-managed learning skills. One CS6 parent said, “Since the Summarized below are stakeholder insights and school adopted the VNEN model, my child’s per- perceptions of VNEN regarding its benefits and formance has improved and he can study by him- challenges. self without me reminding him.” Another parent said, “My child’s attitude has changed very much. She has become more self-disciplined. She gets 7.6.1 Development up early and cleans the house herself. She goes of Socio-Emotional Skills to school very early without me telling her. She Teachers and parents observed that VNEN’s has become polite and actively greets everyone.” implementation has had positive effects on CS2 parents and teachers also made similar com- socio-emotional skills, both in school and at ments, citing the way in which their children/ home. Many of the teachers and parents inter- students now communicate with them more viewed stated that students have become more confidently and have become more motivated to active and confident; they attributed the change study at home and help around the house. One to VNEN, which encourages active learning and CS9 parent commented, “I see the benefit that provides increased opportunities for them to both students and my child have become more improve their communication and leadership confident and proactive in reading. They prefer skills. Commenting on the benefits of applying reading by themselves rather than waiting for the VNEN model, a third grade teacher at CS9 guidance and support from others.” 122 www.worldbank.org 7.6.2  Increased Parental “In my opinion, the roles of parents have Involvement in School changed. The school now invites the parents and Student Learning to discuss students’ performance and other VNEN has also led to increased parental involve- related issues and to participate in school ment. In the VNEN model, parents play vital and activities. Overall, this model is more innova- active roles, both in student learning and school tive than the last one.” (CS9, Parent) activities. At home, parents are involved in chil- dren’s learning, not just by checking on their schoolwork but also by helping them work on 7.6.3  Improved Relationships the application exercises in the learning guide. among Stakeholders In addition to attending the parent meetings, VNEN has improved the relationships among the parents provide support to the school by mak- stakeholders. The teachers at CS2, CS9, and CS12 ing monetary and/or physical contributions to indicated that VNEN has brought teachers and decorate and repair classrooms. They provide students closer. Students share their ideas with real-life learning by decorating the “community teachers more freely and even talk about their corner” and by giving talks and presentations for family issues. The third grade CS15 teacher also students about local traditions and culture, such stated that students are not afraid to talk to teach- as embroidery, cooking, dancing, and farming. In ers any more like they used to be. She attributed this way, VNEN integrates parents into learning the change to VNEN, which she believes has cre- and makes it more relevant and enjoyable for stu- ated a more open and friendly learning environ- dents. These activities are designed to increase ment for both students and teachers. understanding and appreciation for the culture and ethnic traditions of the students. In addi- The improvement in parent-child relationships tion, parents participate in the student govern- also is a welcome change for many parents. CS2 ment election process and student performance parents attribute their children’s transformation assessment. to their exposure to VNEN, through which they have greater opportunities to share ideas and Selected quotes from the interviews of teach- opinions with their classmates and teachers. ers and parents are cited below to illustrate the Some parents indicated that they communicate way in which VNEN has facilitated more active with their children and their teachers more than parental involvement in different dimensions of ever before. CS2 parents described the way in student learning at the study schools. which the VNEN model has fostered open com- munication between parents and children: they “Under the traditional model, parents did not not only spend more time talking with their chil- participate in student assessment, but under dren at home, but their mutual communication this [VNEN] model, parents are involved in vari- also has become more open and friendly. CS9 ous activities, including student assessment. I parents also indicated that increased parental think that the parents have an essential role to involvement in student learning has allowed play here.” (CS6, Grade 5 Teacher) children and parents to discuss what they are studying in lessons more frequently. One CS9 “The parents actively participate in class activi- parent said, “The relations between parents, ties and they also remind the students to study school staffs, and students have become closer, at home. A parent even called me once to ask while the students’ respect for their parents and why there was no homework that day. They grandparents remains unchanged.” VNEN also care a lot about their children’s study.” (CS9, has changed relationships between parents and Grade 3 Teacher) teachers dramatically. CS12 parents indicated Enhancing School Quality in Vietnam through Participative and collaborative Learning 123 Findings from the Qualitative Component that their lines of communication with teachers limited proficiency of ethnic minority students in have improved since the school adopted VNEN, Vietnamese. This is particularly true for schools and they are in frequent contact with teachers that are heavily populated with ethnic minority who tell them how their children are doing at students. The CS6 principal and teachers reported school. that applying the VNEN model has been particu- larly challenging for lower grade students with poor Vietnamese vocabulary and limited reading 7.7  Challenges of VNEN skills. Similarly, at CS12, where almost all students are non-Kinh and have limited proficiency in Viet- 7.7.1  Challenges Attributed namese, teachers stated that their students have to the Mindset of Teachers difficulty studying on their own using the learn- Despite increased levels of effectiveness of teach- ing guide. As one fourth grade teacher stated, ers in applying the VNEN model, it takes time to “They often use the same words repeatedly and change their mindsets toward fully adopting make incorrect sentences in both writing and it, as illustrated in a comment made by an CS2 speaking.” Hence, the teachers spend a significant teacher: amount of time adjusting the learning guide to better accommodate the learning needs of their “I think the biggest challenge [for teachers] is to students, which is not always easy; thus, they feel change the teaching methods and approaches the need for more professional support. At CS9, with which we have been familiar. For example, a large proportion of the students are Khmer, I have been teaching for twenty years with the and they have difficulty communicating and fol- old model; so, firstly, I have to change my way lowing the lessons in Vietnamese. A third grade of thinking and learn how to implement the teacher there stated: “The disadvantage is that new one properly. The old teaching method most students are from ethnic minority groups has been inculcated in me, so the change is a coming from remote areas. These students make long process of self-learning.” up about 70 percent of the class, so their ability to speak Vietnamese and to present in front of Although it is not an easy task to alter teacher the class is still limited.” mindsets, a sustained effort to expose teachers to the VNEN model through professional develop- To cope with the challenges presented by ethnic ment is crucial. Biweekly cluster meetings would minority students’ lack of language proficiency, serve as an important platform for teachers not teachers take different approaches. Teachers only to discuss and practice VNEN approaches in CS9 said that they work closely with ethnic but also to learn from fellow VNEN teachers minority students and encourage them to prac- (especially those at a more advanced level) about tice speaking Vietnamese at home with their their experiences with the model and the posi- parents. Both CS12 and CS9 teachers addressed tive effects they have realized through facilitating the importance of combining the VNEN model more participative and collaborative learning. with the traditional (teacher-centered) model to some extent. The fourth and fifth grade CS12 teachers commented that using both VNEN and 7.7.2  Challenges Resulting traditional models would be more effective for from Ethnic Minority Students’ teaching ethnic minority students rather than Proficiency in the Vietnamese abandoning the traditional model altogether. Language Similarly, the fourth grade CS9 teacher remarked One of the major challenges faced by teach- that combining the two models would help eth- ers and principals in implementing VNEN is the nic minority students—who tend to be passive 124 www.worldbank.org learners and still have difficulty following the grandparents, should be encouraged to forge learning guide on their own—absorb informa- stronger family-school-community ties and sup- tion more effectively. The teacher believes that port students in need of substantial academic continuing to provide lectures when needed will help at home. help students improve their performance. While challenges regarding proficiency of ethnic 7.8  Further Exploration minority students in the Vietnamese language are of Patterns in Classroom not unique to VNEN schools, a more concerted Practices and Interactions effort could be made to help those students improve their readiness for school by combin- The lesson coding methodology utilized a mul- ing the program with early childhood education tidimensional structure to capture instances of interventions and/or by developing innovative specified events and activities in the classroom. approaches to address those challenges. As displayed in Figure 7.4, the coding captured the events with timestamps, including durations, frequencies and cross-dimensional combinations 7.7.3  Challenges Associated of what takes place at the same time. Many of the with Parents codes included attributes that capture important Teachers often associate inability of parents to qualifying categorizations and high-inference help their children academically with the insuf- features such as level of student engagement or ficient knowledge of parents as well as their level of advancement. The data allow for over- temporary migration status. The CS6 teachers all snapshot summaries across lessons, but also stated that many parents are not knowledgeable allow for intricate pattern analysis of individual enough to help their children with their applica- lessons and search for dynamic patterns across tion exercises at home. To cope with this chal- lessons. The sequence of events can be analyzed lenge, the fifth grade teacher gave her contact to understand how events unfold and what trig- numbers to parents in the hope that they would gers certain instances. The coding is heavily ori- ask questions directly, if necessary. In addition, ented to how interactions between teachers and migration of parents was a common issue among students, and students with other students, can the studied schools. Teachers alleged that par- be analyzed in terms of types of questions asked ent absence leads to less parental involvement and responses given or how teachers interact in student learning, both at home and in school. with their students for feedback and support. The third grade teacher at CS6 commented that Interactions can be contextualized based on the some students could not work on application activities taking place and the teaching-learning exercises because their parents worked away methods being utilized in order to gain deeper from home. Students normally stay with their insights on why things take place in the way grandparents during those times, and many that they do. These data can even be extended grandparents do not actively participate in stu- through more advanced data capture with soci- dent learning or school activities. These parental ometers that use variables to explore interper- challenges are common, especially in areas with sonal and intrapersonal relationships. While this disadvantaged student populations. Nonethe- phase of the study has touched upon aspects in less, given the emphasis of the VNEN model on relation to the research questions posed, there community involvement, a more active engage- are many opportunities for further analysis. ment of the broader local community, including Enhancing School Quality in Vietnam through Participative and collaborative Learning 125 Figure 7.4  Visual Example of Multiple Layers Coded over the Course of the Lesson Findings from the Qualitative Component Source: VNEN Impact Evaluation study primary data, Authors calculations. 126 www.worldbank.org TakeAways: Qualitative Research Using Videos This chapter presented a summarized version of TEACHER TRAINING analytical findings from a method based on videos of VNEN classrooms. The analysis sought to exam- • Important advancement markers for 21st cen- tury skills from both the teacher and student ine closely the practical manifestations of VNEN perspectives can be leveraged when training practices. The chapter presented the factors that teachers on how to both encourage and sup- appear to be associated with VNEN implementa- port the development of non-cognitive skills. tion and the challenges faced by teachers and other stakeholders. LEARNING GUIDE FLEXIBILITY MULTIPLE PATHWAYS • With training support from MOET, VNEN teach- • The video analysis suggests that VNEN students ers are using the learning guide with a certain level of flexibility as they see fit by making mod- are not only given multiple ways to learn, with ifications to and/or deviations from activities, a more prominent pedagogical use of explora- yet still meeting quality standards and teaching tion and discussion, but also opportunities to all required content. practice and exercise problem solving through both individual and group work. CHALLENGES 21st CENTURY SKILLS • Some of the challenges faced by VNEN include • Despite some variation across schools and the persistence of traditional mindset among some teachers, the limited Vietnamese lan- within classrooms, VNEN schools provide more guage proficiency of ethnic minority students, space for students to develop and practice 21st and the inability of some parents to provide century skills, such as leadership, teamwork academic support for their children. and cooperative learning, communication, and self-managed learning, at both intermediate and advanced levels. Video analysis shows that VNEN implementation has multiple pathways to effectiveness. The reform in the future will be well advised to continue to use analytical coding of classroom videos for continu- ous learning and feedback. Enhancing School Quality in Vietnam through Participative and collaborative Learning 127 Policy Conclusions and Recommendations 8 8.1  Positive Impact of VNEN resources—the student–teacher ratio does not change, and learning guides replace textbooks. The VNEN program has had a positive impact A rough calculation of the cost of the GPE-VNEN on cognitive and non-cognitive achievement of program is also instructive. The program cost children in Vietnam. This study reported the find- approximately US$85 million, and benefited ings of positive impact using a rigorous impact about a half million children over four years, evaluation procedure that followed a cohort of not counting the children to be benefited in the students from Grade 3 through to Grade 5. The future. This works out roughly to about US$40 per program placement was not randomized, but a student per year. The average per student expen- propensity score matching exercise was carried diture for primary education is about US$1,000 out to determine a matched panel of both a con- per year in PPP terms (approximately VND 8 mil- trol and a treatment group of schools. Balance lion). The main element of fixed costs was for tests of student achievement two years prior to developing the teaching method and materials; the baseline show a parity in performance prior the marginal cost of reaching additional students to the program initiation. Just in case there may was the same as any other public school in Viet- be unobservables that may still introduce bias, nam. Considering the impact on test scores and we were able to adopt a differencing approach non-cognitive development, and the fact that where the dependent variable was the differ- the program itself was not more or less expensive ence in Vietnamese and mathematics test scores than traditional teaching, the overall conclusion as children progressed through Grade 3, Grade 4, was one of positive impact. and Grade 5. We also estimated the non-cognitive development of the children over the three years of the program and found a positive impact. 8.2  Theory of Change of VNEN Almost any educational program tracked over a period of time will show growth in the cogni- The VNEN program is a comprehensive program tive and non-cognitive abilities of children. This of pedagogical reform. It seeks to consolidate on study compared the growth of the children in the past gains in Vietnam to make a direct impact the VNEN program with a counterfactual group on cognitive achievement as well as impact non- to see the difference in growth. The preference cognitive achievement. Non-cognitive achieve- in this study was to collect detailed data on the ment, aspects of which are termed as ‘21st teaching and learning processes; we did not col- Century Skills’ because of their increased rele- lect data on costs. However, the lack of precise vance in the hyper-connected and dynamically cost data is not a problem because the operating accelerating modern world, are useful for their costs of the program are substantively the same own sake, and also because they are associated over the treatment and control group. VNEN is with improved cognitive outcomes. Vietnam is merely a different way of organizing the same committed to the so-called ‘Fundamental and 129 Policy Conclusions and Recommendations Comprehensive reform’ of the education system 8.3  Leadership of VNEN of which the VNEN program was a vanguard. The impact evaluation of VNEN has a very important This study has uncovered a critical nexus role to fill because it helps to provide early feed- between the implementation of VNEN and the back on a reform that will roll out over more years outcomes or benefits derived from the program. as the cohort of primary school children move For example, while the mere nomenclature of through secondary school and new cohorts of a ‘VNEN school’ appears to have a statistically primary school children enter the system. insignificant impact on Vietnamese learning, the results depicted in Table 6.1 (Panel B) show With such an ambitious scope and long-term time an impact of 0.16 standard deviations when we frame, it will be very useful for all the stakehold- consider moving halfway along the distribution ers to have an accurate idea of the theoretical of the implementation index we devised and underpinnings of the VNEN reform. As described presented in Chapter 4. Yet the same table also in detail in Colbert, Chiappe, and Arboleda, 1993, shows a slowdown in the growth from imple- the Escuela Nueva program in Colombia has prac- menting the VNEN program, a convergence over tical and empirical roots, with solutions devised to time in results so that the early results of VNEN, tackle the issue of multigrade instruction and the from Grade 2 to Grade 3 and Grade 3 to Grade 4 exigencies of rural life. However, as the program are not as strong as from Grade 4 to Grade 5. grew over time in Colombia and other countries, Some part of this would be an artifact of mea- and most definitely in Vietnam, the conceptual surement, and future research will be required basis has benefited from the accretion of various to sift out this issue more carefully than we have extremely useful and time-tested educational been able to do. Yet, to the extent that the phe- and philosophical principles. As sketched out in nomenon does indeed take place, it points to the Figure 1.2 about peer-assisted learning, due to need for emphasis on leadership. Topping, 2005, very deep philosophical consid- erations underlie the VNEN model. Participative With a reform as wide ranging as VNEN, the and collaborative learning is expected to work championship of leaders at every level, from the because in a fundamental way, all human learn- national, through the provincial, district, and ing, some would say all learning, can be under- school levels, appear to be a necessary condition stood as consisting of a combined process of for success. We have seen that in certain prov- exploration and engagement (Pentland, 2014). inces, there appears to be a higher incidence of complete implementation of VNEN. We found And the philosophical wellspring for VNEN does that schools with advanced levels of implemen- not go dry even after drawing from the substan- tation tended to have principals with a deep tively deep literature regarding why learning understanding of the VNEN model, who encour- together should work. This study provided useful aged teachers in the school to follow it. Activities references to the body of knowledge regarding such as periodic meetings among teachers in the movement and learning, and play and learning. school to discuss and practice VNEN approaches From an administrative point of view regarding seem to contribute to its adoption. The ability implementation, we have referred to the litera- to provide leadership regarding VNEN appears ture on complex adaptive systems. While VNEN to be inversely proportional to the individual’s will remain a practical program which will thrive direct experience and knowledge about the pro- only so far as it receives empirical validation, it gram. Those who have been part of or observed would be very useful for critics to continue learn- advanced implementation of VNEN are best ing from the various theoretical bases of VNEN, placed to inform others about the program. and to nurture its further development. 130 www.worldbank.org Implementation heterogeneity explains to some pedagogical beliefs show a close alignment to extent the different levels of acceptance and the ideas sought to be imparted by training. In interest in VNEN among provincial leaders. It is the case of VNEN teachers, the practical acts of difficult to separate cause and effect as provincial implementing what they learned in the training leaders are instrumental in driving implementa- reinforced and strengthened their beliefs. Both tion, but it is clear that the impression of leaders qualitative and quantitative analysis indicates will be driven by what they have observed. If they that teachers are using new teaching practices have observed a low level of implementation, related with the VNEN approach. However, we they might have a less than favorable impres- have also seen the high level of heterogeneity in sion of VNEN; if they have seen animated chil- implementation. It is quite likely that one of the dren with high levels of cognitive and affective factors that led to this distribution is the low inci- engagement, they might appreciate and under- dence in the implementation of cluster training. stand the program better. Biweekly interaction between teachers appears to have taken place only sporadically in many Ensuring a high level of implementation by schools. influencing leaders at all levels is a challenging ­ problem—clearly. Trying to mandate or regi- Going forward, VNEN teachers should be helped ment the implementation would not work at with the on-site cluster training to better acquire all, because it would be logically inconsistent to a set of skills that would help facilitate class- mandate a feature like community participation. room activities more effectively. To reinforce and The best chance for success would come from improve their role as facilitators, teachers should feedback loops generated through sharing of learn to become more observant by carefully successful implementation. At any level, provin- monitoring each group and minimizing their cial leaders or teachers would be able to respond direct involvement as long as students are tack- positively to demonstrations of success. Events ling their task together. Second, teachers could such as video competitions which had been receive more substantive training to sharpen organized at some point during implementation their questioning skills, which would allow them can be conducted with dedicated attention and to guide students to develop a deeper under- would be instrumental in reducing the width of standing through an enquiry mode of learning. the distribution of the implementation index Third, more rigorous training should be offered toward the higher end through a process of imi- to diversify the techniques they use to motivate tation and experience of rewards. students to study better and exercise 21st cen- tury skills. 8.4  Training of Teachers Some VNEN teachers addressed the difficulty of training student group leaders, while others The VNEN program pioneered a new paradigm indicated the lack of students with sufficient of training where committed, high performing ability to assume the leadership role. One way to teachers were co-opted early to become train- address this challenge would be to expand their ers themselves. In this way, the training was not perception of leadership and make them aware something developed by trainers removed from that it is not something certain children have nat- practical realities and provided to teachers in a urally, but that it can be strengthened through mimic of the traditional frontal model of teach- practice and training. Another way to respond ing. Rather, training was conducted in the form to this challenge would be to restructure group of a VNEN classroom, complete with groups and activities in such a way that team members have group leaders and warm-up games. As shown in more shared responsibilities. the heat maps presented in Figure 3.4, teachers’ Enhancing School Quality in Vietnam through Participative and collaborative Learning 131 Policy Conclusions and Recommendations 8.5  Deepening Participation the map the homes which had a person with a birthday that month, or houses where there were As with the limited implementation of biweekly four generations of people living there and so on. cluster training, we also found limited engage- Successful examples can be replicated through ment with parents and community members as demonstration and replication, done on a com- compared to the intention of the VNEN model to pletely voluntary basis. closely engage with parents. The close involve- ment of parents in instructional processes rep- resents a rather large cultural shift for many 8.6 Communications stakeholders. Some principals and teachers may Program believe that the predominantly large commu- nity of farmers that represents the parent body This study was focused on the content of the may not be amenable to close involvement in VNEN program—how the program was imple- the school. Others may have tried and failed to mented, the context in which it was implemented engage the parents and may have persuaded and the impact that the program had. The pro- themselves of the futility of trying again. Others gram did include a communication element, may have been more persistent and met with with the view to generate awareness and inter- success. Still others may have been very lucky est in the model among all stakeholders. To some from the outset, or even have found particular extent, the communication program must have parents as champions of the effort, who lob- been successful, otherwise the phenomenon bied and pressured for implementation. As is of control group schools voluntarily adopting the case with teachers implementing aspects of the program and many VNEN elements spilling the model, such as facilitating group work rather over into VNEN schools may not have happened. than directing children who happen to be sitting However, it is also true that some of the program in groups, the generation of multiple feedback schools themselves had a low level of implemen- loops is probably the best way to bring about tation and part of the reason might have been an wider adoption. inadequate understanding of the VNEN program to the same extent that the high implementing From a policy perspective, it is useful to under- VNEN schools had managed to attain. stand how the feedback loops may be engen- dered. A case in point is the use of artifacts in the Going ahead, there are four key elements of a classroom, including objects from real life and the communications program that would be very community map. Typically, these have been one- useful to keep in mind: (i) Factual basis: As far as off activities done with varying levels of enthusi- possible all stakeholders need to be informed asm. In our study we tried to capture this diversity about the results and outcomes conducted in by asking not only about the presence of artifacts as objective a manner as possible. Thanks to the but also about their use. We found that the use financial support from Dubai Cares, it was pos- of artifacts was made very seldom, and the rea- sible to carry out this impact evaluation study, son likely was that teachers may not have been but effort needs to be made to spread the results encouraged or provided ideas regarding how to of the study; (ii) Champion stakeholders: A key bring it about. For instance, once a community phenomenon that was observed throughout the map is made, it may be a static document that program was the clustering of good practices no one refers to again. But fun activities could be because of the exchange and replication of ideas. organized that would serve a didactic purpose When one teacher found a way to provide facilita- for both cognitive and non-cognitive skills. An tion to student government that was particularly example would be for children to identify on effective, other teachers in her school tended to 132 www.worldbank.org copy her, but the idea may not have been trans- the adoption of a particular method—the use of mitted to a neighboring school. If the successful sociometers combined with artificial intelligence idea were to be recognized as such, the chances (AI) algorithms, to provide a highly innovative are that the original teacher or group of teach- and potentially game changing method to mea- ers may wish to share the idea with others—but sure certain socio-emotional skills in real time they would only be able to do so if provided with and actually enhance the acquisition of these the mechanism and the incentives to do so. The skills through real-time feedback to teachers and video competition was a case in point; (iii) Coun- students. These devices capture real-time physi- tering misconceptions: It is quite interesting that ological data from individuals and data regarding when we sought to track attitudes of stakehold- the behavioral dynamics of groups. The resulting ers regarding the traditional model or status quo, stream of ‘Big Data’ will be analyzed using sophis- we found a possible deepening of belief about ticated computer algorithms. The method has the appropriateness of the traditional model—as been used extensively in a multitude of contexts measured by people’s agreement with the idea around the world and reported in scientific jour- that ‘there is no need to change’. It is quite likely nals and books. that attitudes may have resulted from misap- prehensions about trade-offs between cognitive At present, the teacher in a VNEN classroom has a skills and non-cognitive skills, or a simple lack limited understanding of the dynamic of interac- of awareness about the latest research showing tion within a group and the classroom as a whole. links between play and learning achievement, However, the adoption of a sociometer would but these can be corrected with appropriately enable the teacher to have a real-time monitor- designed communications; and (iv) Bi-partite ing of the health level of interaction within a or multi-partite communication: Finally, in the group and direct her energies and attention to case of a far reaching reform program it is always help improve the dynamics. The students can important to have feedback loops from imple- obtain the same feedback themselves and be menters on the ground to the policy makers. This able to self-correct their behavior for improving happened to a large extent during the program, the health of their interaction. Pentland, 2012 as senior MOET leaders crisscrossed the country provides evidence regarding the utilization of visiting schools and incorporating adjustments this method in corporate and academic settings, in the program based on the feedback. Greater where the use substantially enhanced the quality awareness about this phenomenon, which also of outcomes. The underlying theoretical model took place at the provincial level, would go a is based on the influence that humans have on long way toward generating ownership among one another’s behavior through direct and indi- all stakeholders. rect interaction (Pan et al., 2012). The extension of this model is quite feasible from the corpo- rate context to the context of the VNEN school. 8.7 Monitoring Recent advances in technology make these Socio-Emotional Skills devices quite inexpensive, perhaps US$5 apiece and the use can be scaled up quite easily. How- Currently, available methods for measur- ever, the development of the calibration of the ing acquisition of social and emotional (non-­ instruments and generation of valid and usable cognitive) skills rely on qualitative questionnaires skill measures will require a major research effort of self-reported behaviors or reports by parents for which the study team, in collaboration with and teachers, collected at discrete points in time the technology’s founders at the Massachusetts but referring to larger periods of time like the Institute of Technology (MIT), are engaged in the previous day, week, or semester. We propose search for research sponsorship. Enhancing School Quality in Vietnam through Participative and collaborative Learning 133 Policy Conclusions and Recommendations 8.8  Directions for used in the classroom. For example, teacher use of questioning can be further explored to analyze Further Research questions that elicit responses involving higher order thinking from students. The quantitative data collection over a period of nearly three years has generated a huge amount The videos also provide ample opportunity to of data that can generate many further insights. create vignettes of instances for use as examples While the impact evaluation results presented in training and professional development. Many here can be further deepened and developed examples of advanced VNEN practices have from this data, the team from Research on Improv- already been identified in this round of analy- ing Systems of Education (RISE) will collect further sis, but this could be extended with the creation data from a part of the same cohort of children as of a database of instances. The vignettes could they go through secondary school. The primary also include a set of more specific tags of what data collected under the current study itself pro- the videos represent. For example, clips identi- vide a veritable data mine of research on various fied for high levels of leadership could be further topics including teacher efficacy, principal lead- catalogued as being where the leader effectively ership, and other issues related to educational leads a discussion involving higher order think- psychology and student achievement. It is hoped ing, where the leader ensures all members are that the data will be made available by the Viet- actively participating, or many other ways in namese government to researchers in the future which leaders demonstrate unique skills. to carry out further investigation. From the teacher perspective, many instances The qualitative component also has many poten- can be created to demonstrate effective facili- tial avenues for further research. With 810 videos tation of activities. This would be particularly from 270 lessons (with each lesson containing a helpful in training new teachers who are accus- video of the teacher and of two student groups), tomed to running their lessons in a teacher-led there are many opportunities to delve deeper manner and are not familiar with facilitation into specific aspects of what takes place in the techniques. All transcripts have been cata- classroom. Since 540 of these videos are coded, logued using Nvivo and have already been it is easy to quickly identify instances of specific tagged with specific categories and subcate- types of instruction, methods, and activities. gories. While most of the interview transcripts For example, the exploratory method (which have been utilized to answer a specific set of is found in VNEN but is rare in traditional class- questions posed in this round of analysis, there rooms) could be analyzed in more depth to better are countless ways in which the transcripts understand what makes for effective exploratory could be further explored. activities. It is possible to analyze the dialogue 134 www.worldbank.org Bibliography Ahmad, J. K., et al. (2005). “Decentralization and In: International Review of Education 46.1–2: Service Delivery.” In: World Bank Policy Research 31–48. Working Paper No. 3603. http://papers.ssrn Bond, Trevor, and Christine M. Fox (2015). Apply- .com/sol3/papers.cfm?abstract_id=753505 ing the Rasch Model: Fundamental Measure- Anderson, Sheri, Yu-Chang Hsu, and Judy Kinney ment in the Human Sciences. Routledge. (2016). “Using Importance-Performance Anal- Brinkerhoff, Robert O. (2006). “Increasing Impact ysis to Guide Instructional Design of Experi- of Training Investments: An Evaluation Strat- ential Learning Activities.” In: Online Learning egy for Building Organizational Learning 20.4. Capability.” In: Industrial and Commercial Train- Azzopardi, Ernest, and Robert Nash (2013). ing 38.6: 302–307. “A Critical Evaluation of Importance–­ Brosky, Donald (2011). “Micropolitics in the Performance Analysis.” In: Tourism Manage- School: Teacher Leaders’ Use of Political Skill ment 35, 222–233. and Influence Tactics.” In: International Journal Baessa, Yetilú de, Ray Chesterfield, and Tanya of Educational Leadership Preparation 6.1: n1. Ramos (2002). “Active Learning and Demo- Brown, Stuart L. (2009). Play: How It Shapes the cratic Behavior in Guatemalan Rural Primary Brain, Opens the Imagination, and Invigorates Schools.” In: Compare 32.2, 205–218. the Soul. Penguin. Bandura, Albert (1977). “Self-efficacy: Toward Cheng, Yin Cheong (1994). “Principal’s Leader- a Unifying Theory of Behavioral Change.” In: ship as a Critical Factor for School Perfor- Psychological Review 84.2, 191. mance: Evidence from Multi-Levels of Primary ———. (1993). “Perceived Self-efficacy in Cogni- Schools.” In: School Effectiveness and School tive Development and Functioning.” In: Educa- Improvement 5.3: 299–317. tional Psychologist 28.2, 117–148. Chesterfield, Ray (1994).“Indicators of Democratic ———. (1997). Self-efficacy: The exercise of con- Behaviour in Nueva Escuela Unitaria (NEU) trol. Macmillan. Schools.” In: Basic Education Strengthening– Beatriz, Pont, Nusche Deborah, and Moorman BEST–Project). Guatemala: United States Agency Hunter (2008). Improving School Leadership, for International Development [USAID]. http:// Volume 1 Policy and Practice. OECD publishing. pdf.usaid.gov/pdf_docs/PNABX940.pdf Bell, Courtney A., et al. (2012). “An Argument Colbert, Vicky (1999). “Titulo: Mejorando el Approach to Observation Protocol Validity.” acceso y la calidad de la educacion para el In: Educational Assessment 17.2–3: 62–87. sector rural pobre. El caso de la Escuela Nueva DOI: 10.1080/10627197.2012.715014. eprint: en Colombia.” In: Revista Iberoamericana de http://dx.doi.org/10.1080/10627197.2012 Educacion 20: 107–135. .715014. http://dx.doi.org/10.1080/10627197 Colbert, Vicky, Clemencia Chiappe, and Jairo .2012.715014. Arboleda (1993). “The New School Program: Benveniste, Luis A., and Patrick J. McEwan (2000). More and Better Primary Education for Chil- “Constraints to Implementing Educational dren in Rural Areas in Colombia.” In: Effective Innovations: The Case of Multigrade Schools.” 135 Bibliography schools in Developing Countries, H. M. Levin Matter More than Capital at School? Social and M. E. Lockheed (Eds.). Capital Effects on Academic Achievement.” In: Copeland, Patrick, and Alberto Savoia (2011). Research in Social Stratification and Mobility “Entrepreneurial Innovation at Google.” In: 31: 1–21. Institute of Electrical and Electronics Engineers: Ekman, Paul (2007). Emotions Revealed: Recogniz- Computer 0018-9162/11: 7. http://www.alberto ing Faces and Feelings to Improve Communica- savoia.com/uploads/1/4/0/9/14099067/ent tion and Emotional Life. Macmillan. repreneurialinnovationatgoogle_published Ellsworth, James B. (2000). Surviving Change: A version-1.pdf Survey of Educational Change Models. ERIC. Creemers, Bert, et al. (2003). World Class Schools: Epstein, Marc J., and Kristi Jane Yuthas (2012). International Perspectives on School Effective- “Redefining Education in the Developing ness. Routledge. World.” In: Stanford Social Science Innovation Crump, Richard K., et al. (2009). “Dealing with Review. Limited Overlap in Estimation of Aver- Erickson, Frederick, et al. (2006). “Definition age Treatment Effects.” In: Biometrika 96.1: and Analysis of Data from Videotape: Some 187–199. Research Procedures and Their Rationales.” In: Dakhli, Mourad, and Dirk De Clercq (2004). Handbook of Complementary Methods in Edu- “Human Capital, Social Capital, and Innova- cation Research 3: 177–192. tion: A Multi-country Study.” In: Entrepreneur- Farrington, Camille A., et al. (2012). Teaching ship & Regional Development 16.2: 107–128. Adolescents to Become Learners: The Role Dalton, Russell J., and Nhu-Ngoc T. Ong (2005). of Noncognitive Factors in Shaping School “Authority Orientations and Democratic Atti- Performance—A Critical Literature Review. tudes: A Test of the ‘Asian values’ Hypothesis.” ERIC. In: Japanese Journal of Political Science 6.02: Ferris, Gerald R., Sherry L. Davidson, and 211–231. Pamela L. Perrewe (2005). Political Skill at Damasio, Antonio R. (1999). The Feeling of What Work: Impact on Work Effectiveness. Davies- Happens: Body and Emotion in the Making of Black Publishing. Consciousness. Houghton Mifflin Harcourt. Ferris, Gerald R., Darren C. Treadway, et al. (2005). Dang, Hai-Anh (2007). “The Determinants and “Development and Validation of the Political Impact of Private Tutoring Classes in Viet- Skill Inventory.” In: Journal of Management nam.” In: Economics of Education Review 26.6: 31.1: 126–152. 683–698. Fidan, Tuncer, and Ali Balcı (2016). “Principal Danielson, Charlotte (2012). “Observing Class- Proactivity: School Principals’ Proactive.” In: room Practice.” In: Educational Leadership 70.3: Applied Chaos and Complexity Theory in Edu- 32–37. cation, p. 29. De Fraja, Gianni, Tania Oliveira, and Luisa Zan- Forero-Pineda, Clemente, Daniel Escobar- chi (2010). “Must Try Harder: Evaluating the Rodríguez, et al. (2006). “Escuela Nueva’s Role of Effort in Educational Attainment.” In: Impact on the Peaceful Social Interaction of The Review of Economics and Statistics 92.3: Children in Colombia.” In: Education for All and 577–597. Multigrade Teaching. Springer, pp. 265–300. Douglas, Ceasar, and Anthony P. Ammeter (2004). Fuller, Bruce (1987). “What School Factors Raise “An Examination of Leader Political Skill and Achievement in the Third World?” In: Review of Its Effect on Ratings of Leader Effectiveness.” Educational Research 57.3: 255–292. In: The Leadership Quarterly 15.4: 537–550. Gibson, Sherri, and Myron H. Dembo (1984). Dufur, Mikaela J., Toby L. Parcel, and Kelly P. “Teacher Efficacy: A Construct Validation.” In: Troutman (2013). “Does Capital at Home Journal of Educational Psychology 76.4: 569. 136 www.worldbank.org Goldman, Ricki, et al. (2014). Video Research in Iacoboni, Marco (2009). Mirroring People: The the Learning Sciences. Routledge. New Science of How We Connect with Others. Green, Kathy E., Catherine G. Frantom, et al. Macmillan. (2002). “Survey Development and Validation Inglehart, Ronald F., Miguel Basanez, and Alejan- with the Rasch Model.” In: International Con- dro Moreno (1998). Human Values and Beliefs. ference on Questionnaire Development, Evalu- University of Michigan Press. ation, and Testing, Charleston, SC. John, Oliver P., Laura P. Naumann, and Christo- Hallinger, Philip, and Ronald H. Heck (1998). pher J. Soto (2008). “Paradigm Shift to the Inte- “Exploring the Principal’s Contribution to grative Big Five Trait Taxonomy.” In: Handbook School Effectiveness: 1980–1995.” In: School of Personality: Theory and Research 3: 114–158. Effectiveness and School Improvement 9.2: https://www.ocf.berkeley.edu/~johnlab/ 157–191. 2008chapter.pdf Hallinger, Philip, and Stephan Huber (2012). Johnson, David W., and Roger T. Johnson (1999). “School Leadership That Makes a Difference: “Making Cooperative Learning Work.” In: The- International Perspectives.” In: School Effective- ory into Practice 38.2: 67–73. ness and School Improvement 23.4: 359–367. Johnson, Steven (2002). Emergence: The Con- Hannaford, Carla (1995). Smart Moves: Why nected Lives of Ants, Brains, Cities, and Soft- Learning Is Not All in Your Head. ERIC. ware. Simon and Schuster. Hargreaves, Andy (2007). Extending Educational ———. (2016). Wonderland: How Play Made the Change: International Handbook of Educa- Modern World. New York: Riverhead Books. tional Change. Volume 2. Springer Science & Juárez, Associates, and Inc. (2003). “The Effects Business Media. of Active Learning Programs in Multigrade Hirano, Keisuke, Guido W Imbens, and Geert Schools on Girls’ Persistence in and Com- Ridder (2003). “Efficient Estimation of Aver- pletion of Primary School in Developing age Treatment Effects Using the Estimated Countries.” In: Girls’ Education Monitoring Propensity Score.” In: Econometrica 71.4: System, United States Agency for Interna- 1161–1189. tional Development. http://pdf.usaid.gov/ Hogan, Diane M., and Jonathan R. H. Tudge pdf_docs/pnacs291.pdf (1999). “Implications of Vygotsky’s Theory for Judge, Timothy A., et al. (2007). “Self-efficacy Peer Learning.” In: Cognitive Perspectives on and Work-Related Performance: The Integral Peer Learning. Edited by Angela M. O’Donnell Role of Individual Differences.” In: Journal of and Alison King. Lawrence Erlbaum Associ- Applied Psychology 92.1: 107. ates Publishers. Kautz, Tim et al. (2014). Fostering and Measuring Holland, John H. (1992). “Complex adaptive sys- Skills: Improving Cognitive and Non-Cognitive tems.” In: Daedalus, pp. 17–30. Skills to Promote Lifetime Success. Working Holland, John Henry (1995). Hidden Order: How Paper 20749. National Bureau of Economic Adaptation Builds Complexity. Basic Books. Research. DOI: 10.3386/w20749. http://www Houtenville, Andrew J., and Karen Smith Conway .nber.org/papers/w20749 (2008). “Parental Effort, School Resources, and Kline, Rachel (2002). “A Model for Improving Student Achievement.” In: Journal of Human Rural Schools: Escuela Nueva in Colombia and Resources 43.2: 437–453. Guatemala.” In: Current Issues in Comparative Hurt, H. Thomas, Katherine Joseph, and Education 2.2: 171–181. Chester D. Cook (1977). “Scales for the Mea- Kochanska, Grazyna, Anna Clark, and Marguerita surement of Innovativeness.” In: Human Com- Sitrin Goldman (1997). “Implications of Moth- munication Research 4.1: 58–65. ers’ Personality for Their Parenting and Their Enhancing School Quality in Vietnam through Participative and collaborative Learning 137 Bibliography Young Children’s Developmental Outcomes.” from the World Values Survey.” In: Journal of In: Journal of Personality 65.2: 387–420. Cross-Cultural Psychology, pp. 3–14. Lang, Frieder R., et al. (2011). “Short Assessment Mitchell, Melanie (2009). Complexity: A Guided of the Big Five: Robust Across Survey Methods Tour. Oxford University Press. Except Telephone Interviewing.” In: Behavior Mundial, Banco (2004). “World Development Research Methods 43.2: 548–567. Report 2004: Making Services Work for Poor Leimbach, Michael P., and Ed Emde (2011). People.” In: Banque mondiale, Washington, DC. “The 80/20 Rule for Learning Transfer.” In: Munro, Jean (2012). Women’s Representation in Chief Learning Officer 10.12: 64–67. http://m Leadership in Viet Nam. .cedma-europe.org/newsletter%20articles/ OECD (2015). Skills for Social Progress: The Power Clomedia/The%2080-20%20Rule%20for%20 of Social and Emotional Skills, OECD Skills Stud- Learning%20Transfer%20(Dec%2011).pdf ies, OECD Publishing, Paris. Linacre, John Micheal, and Benjamin D. Wright Oh, Haemoon (2001). “Revisiting Importance– (2000). “WINSTEPS: Multiple-choice, Rating Performance Analysis.” In: Tourism Manage- Scale, and Partial Credit Rasch Analysis [Com- ment 22.6: 617–627. puter software].” In: Chicago: MESA. Oyetunji, Christianah Oluwatoyin (2006). “The Littrell, Romie F. (2005). “Learning Styles of Stu- Relationship between Leadership Style dents in and from Confucian Cultures.” In: and School Climate in Botswana Second- Working Paper. ary Schools.” PhD thesis. University of South Martilla, John A., and John C. James (1977). Africa. “Importance-performance Analysis.” In: The Pan, Wei, et al. (2012). “Modeling Dynamical Journal of Marketing, pp. 77–79. Influence in Human Interaction: Using Data Marzano, Robert J., Timothy Waters, and Brian A. to Make Better Inferences about Influence McNulty (2005). School Leadership That Works: within Social Systems.” In: IEEE Signal Process- From Research to Results. ASCD. ing Magazine 29.2: 77–86. Masters, Geoff N. (1982). “A Rasch Model for Par- Parandekar, Suhas (2014). “Benchmarking Public tial Credit Scoring.” In: Psychometrika 47.2: Policy: Methodological Insights from Mea- 149–174. surement of School Based Management.” McEwan, Patrick J. (1998). “The Effectiveness of In: World Bank Policy Research Working Paper Multigrade Schools in Colombia.” In: Inter- 6938. national Journal of Educational Development Parandekar, Suhas, D., Isabel de Assis Ribeiro 18.6: 435–452. Oliveira, and Erica P. Amorim (2008). “Desem- ———. (2008). “Evaluating Multigrade School penho dos alunos na Prova Brasil: diversos Reform in Latin America.” In: Comparative Edu- caminhos para o sucesso educacional nas cation 44.4: 465–483. redes municipais de ensino.” In: Brasilia: Inep. Metsäpelto, Riitta-Leena, and Lea Pulkkinen Parandekar, Suhas, and Elisabeth Sedmik (2016). (2003). “Personality Traits and Parenting: Neu- “Unraveling a Secret: Vietnam’s Outstanding roticism, Extraversion, and Openness to Expe- Performance on the PISA Test.” In: World Bank rience as Discriminative Factors.” In: European Policy Research Working Paper 7630. Journal of Personality 17.1: 59–78. Pentland, Alex (2012). “The New Science of Build- Middleton, F. A., and Strick P. (1994). “Anatomi- ing Great Teams.” In: Harvard Business Review cal Evidence for Cerebellar and Basal Ganglia 90.4: 60–69. Involvement in Higher Cognitive Function.” In: ———. (2014). Social Physics: How Good Ideas Science 266: 458–461. Spread—The Lessons from a New Science. Minkov, Michael, and Geert Hofstede (2010). Penguin. “Hofstede’s Fifth Dimension: New Evidence 138 www.worldbank.org Pfeffer, Jeffrey (1981). Power in Organizations. Summerford, Cathie (2009). Action-packed Class- Volume 33. Pitman Marshfield, MA. rooms, K–5: Using Movement to Educate and Psacharopoulos, George, Carlos Rojas, and Edu- Invigorate Learners. Corwin Press. ardo Velez (1993). “Achievement Evaluation of Teddlie, Charles, and David Reynolds (2000). The Colombia’s ‘Escuela Nueva’: Is Multigrade the International Handbook of School Effectiveness Answer?” In: Comparative Education Review Research. Psychology Press. 37.3: 263–276. ISSN: 00104086, 1545701X. Tompson, Trevor, Jennifer Benz, and Jennifer URL: http://www.jstor.org/stable/1188512 Agiesta (2013). Parents’ Attitudes on the Qual- Rich, Peter J., and Michael Hannafin (2009). ity of Education in the United States. NORC Cen- “Video Annotation Tools: Technologies to ter for Public Affairs Research. Scaffold, Structure, and Transform Teacher Topping, Keith J. (2005). “Trends in Peer Learn- Reflection.” In: Journal of Teacher Education ing.” In: Educational Psychology 25.6: 631–645. 60.1: 52–67. Urman, Andrea (2012). “Relationship between Rogers, Everett M. (2010). Diffusion of Innova- Parenting Perceptions and Personality.” tions. Simon and Schuster. In: ETD Collection for Pace University Paper Rose, Lowell C., Alec M. Gallup, and Stanley M. AAI3496709. URL: http://digitalcommons Elam (1990). “The Twenty-second Annual Gal- .pace.edu/dissertations/AAI3496709 lup Poll of the Public’s Attitudes Toward the Warnes, Gregory R., et al. (2013). gplots: Vari- Public Schools.” In: Phi Delta Kappan, vol. 72, ous R Programming Tools for Plotting Data. no. 1, 41–55. R Package Version 2.12. 1. http://CRAN.%20 Senemoğlu, N., et al. (2009). “Elementary School R-project.%20org/package=%20gplots Teachers’ Self-efficacy Beliefs: A Turkish Case.” Wooldridge, Jeffrey M. (2007). “Inverse Probabil- In: Humanity & Social Sciences Journal 4.2: ity Weighted Estimation for General Missing 164–171. Data Problems.” In: Journal of Econometrics Singer, Dorothy G., Roberta Michnick Golinkoff, 141.2: 1281–1301. and Kathy Hirsh-Pasek (2006). Play = Learn- Woolfolk, Anita E., and Wayne K. Hoy (1990). ing: How Play Motivates and Enhances Chil- “Prospective Teachers’ Sense of Efficacy and dren’s Cognitive and Social-emotional Growth. Beliefs about Control.” In: Journal of Educa- Oxford University Press. tional Psychology 82.1: 81. Stajkovic, Alexander D., and Fred Luthans (1998). World Bank (2011). “Project Concept Note on a “Self-efficacy and Work-Related Performance: Proposed Grant of 80.00 (US$M) to the Social- A Meta-analysis.” In: Psychological Bulletin ist Republic of Vietnam.” In: Unpublished World 124.2: 240. Bank Project Documentation. Stigler, James W., and James Hiebert (1999). World Development Indicators, 2015, http://data The Teaching Gap: Best Ideas from the World’s .worldbank.org/data-catalog/world- Teachers for Improving Education in the Class- development-indicators room. Simon and Schuster. Enhancing School Quality in Vietnam through Participative and collaborative Learning 139 Index Artifacts, 57 Objective, 44 Assessment, 62 Awareness, 66, 88 Parental Engagement, 65 Parents, 69, 81, 88–91, 125 Beliefs, 28, 39, 44, 47, 83, 84 Participative and Collaborative Learning, 6 Pedagogical Elements, 60 Classroom, 57, 112, 113, 125 Principal, 22, 28–30, 34, 67 Colombia, 3 Community Engagement, 25 Readiness, 25 Community Involvement, 10 School Autonomy, 21 Education Program Development Fund, 3 School Grants, 24 Educational Objectives, 44 Self-Efficacy, 51 Student Government, 64 Furniture, 57 Students, 69, 124 Study Design, 14 Group Work, 61 Teacher Demographics, 39 Influence, 34, 53 Teacher Profile, 39 Innovativeness, 51 Teacher’s Beliefs, 39, 44 Teachers, 51, 74, 124, 131 Job Satisfaction, 40 Teaching Method, 57, 61 Learning Guides, 60 VNEN Implementation Index, 66–78 Ministry of Education and Training, 3 Multigrade, 6 141 Picture Credits Title Page: Used with the permission of the Chapter 2: Used with the permission of the Viet- Vietnam Ministry of Education and Training, nam Ministry of Education and Training, Hanoi/ Hanoi/Vietnam (2017). Vietnam (2017). Acknowledgments: Used with the permission of Figure 4.2: © The World Bank (2016). the Vietnam Ministry of Education and Training, Hanoi/Vietnam (2017). Chapter 6: Used with the permission of the Viet- nam Ministry of Education and Training, Hanoi/ Abbreviations and Acronyms: Used with the Vietnam (2017). permission of the Vietnam Ministry of Education and Training, Hanoi/Vietnam (2017). Chapter 7: © Chau Doan/World Bank (2008). Executive Summary: Used with the permission of Chapter 8: Used with the permission of the Viet- the Vietnam Ministry of Education and Training, nam Ministry of Education and Training, Hanoi/ Hanoi/Vietnam (2017). Vietnam (2017). Chapter 1: Sunset on the bay of Danang, Central Index: © Charles Van den Broek (2014). Vietnam, licensed under Creative Commons 0 Public Domain (1.0 Universal). Section 1.4: Preview of Findings: © The World Bank (2016). 142 The Main TakeAways: VNEN Has a Positive Impact on Vietnam • The reform needs champions throughout the • Substantial school-based training and support system, from top to bottom. The champions through the school year is needed—information need to be informed accurately about both the from classroom observation and actual prac- underlying educational theories and the empiri- tice, as well as the tacit knowledge of teachers, cal facts regarding implementation in practice. should be accessed for a cycle of continuous • A sustained information campaign is required learning. that targets both communities of the program • Effective participative and collaborative learn- schools and wider audiences. This information ing is a complex undertaking with heavy cultural campaign needs to clarify the origin and pur- influences. Further high quality research at all pose of the reform, explain the expected ben- levels needs to be encouraged to learn about the efits, and lay out the required behavior from all approaches that are best suited to Vietnamese stakeholders for successful implementation. conditions. • In addition to the adequate understanding of • Vietnam’s education system is widely regarded both concept and practice, conditions need internationally as a ‘success story’. Continued to be put in place to motivate stakeholders to engagement with the international education fully adopt the reforms in teaching and learn- research community will bring mutual benefit ing practices. for Vietnam as well as other countries who wish • Well-functioning monitoring and evaluation to learn from Vietnam. systems that provide feedback throughout all levels of the system are a key element of such Vietnam has the potential to lead the developing favorable conditions—with feedback going both world regarding the achievement of the goal of ways, and including the crucial feedback from learning for all, providing 21st century skills in an students, parents, and teachers. inclusive way. The World Bank 1818 H. Street N.W. Washington, DC 20433  USA www.worldbank.org