Document of The World Bank Report No.: 55407 PROJECT PERFORMANCE ASSESSMENT REPORT NEPAL SCHOOL COMMUNITY PROJECT SUPPORT (CR. 3808) 28,2010 JUNE Sector Evaluation Division Independent Evaluation Group Currency Equivalents (annual averages) Currency Unit = (rupee) Community School Support Project As of March 2003 (Months before Appraisal) As of September 30,2008 US$1 =NPR78.3 US$1 = NPR73.22 NPR 1 = US$0.0133 NPRl = US$0.013657 Abbreviations and Acronyms BPEP Basic and Primary Education Project NGO Nongovernmental organization CAS Country Assistance Strategy NLSS Nepal Living Standards Survey CERID Center of Educational Innovation and NORAD Norwegian Agency for Development Cooperation Development CSSP Community Schools Support Project OECD Organ`ization for Economic Cooperation and Development DANIDA Danish International Development PAD Project Appraisal Document Agency DCA Development Credit Agreement PCU Project Coordination Unit DDC District Development Committees PISA Programme for International Student Assessment DFID United Kingdom Department for PIU Project Implementation Unit International Development EFA Education for All PHRD Policy and Human Resources Development EMIS Educational Management information PPAR Project Performance Assessment Report system EU European Union PRSP Poverty Reduction Strategy Paper FTI Fast-Track Initiative to achieve PRSC Poverty Reduction Strategy Credit Education for All GDP Gross domestic product QAG Quality Assurance Group ICR Implementation Completion Report SAR Staff Appraisal Report IDA International Development Association SMC School management committee IEG Independent Evaluation Group SWAP Sector-Wide Approach ISR Implementation Status and Results TIMSS Trends in International Mathematics and Science Report Study JICA Japan International Cooperation VDC Village Development Committees Agency LIL Learning and Innovation Loan UNESCO United Nations Educational, Scientific, and Cultural Organization MoES Ministry of Education and Science UNICEF United Nations Children's Fund Fiscal Year Government: January 1 -December 3 1 Director-General, Evaluation : Mr. Vinod Thomas Director, Independent Evaluation Group, World Bank : Ms. CherylGray Manager, Sector Evaluation Division : Ms. Monika Huppi Task Manager : Ms. Helen Abadzi 1 IEGWB Mission: Enhancing development effectiveness through excellence and independence in evaluation. About this Report The Independent Evaluation Group assesses the programs and activities of the World Bank for two purposes: first, to ensure the integrity of the Bank's self-evaluation process and to verify that the Banks work is producing the expected results, and second, to help develop improved directions, policies, and procedures through the dissemination of lessons drawn from experience. As part of this work, IEGWB annually assesses about 25 percent of the Banks lending operations. In selecting operations for assessment, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming studies or country evaluations; those for which Executive Directors or Bank management have requested assessments; and those that are likely to generate important lessons. The operations, topics, and analytical approaches selected for assessment support larger evaluation studies. A Project Performance Assessment Report (PPAR) is based on a review of the Implementation Completion Report (a self-evaluation by the responsible Bank department) and fieldwork conducted by IEGWB. To prepare PPARs, IEGWB staff examine project files and other documents, interview operational staff, and in most cases visit the borrowing country to discuss the operation with staff of the Bank and the government, other stakeholders, and beneficiaries. The PPAR thereby seeks to validate and augment the information provided in the ICR, as well as examine issues of special interest to broader IEGWB studies. Each PPAR is subject to peer review and IEGWB management approval. Once cleared internally, the PPAR is reviewed by the responsible Bank department and amended as necessary. The completed PPAR is then sent to the borrower for review; the borrowers' comments are attached to the document that is sent to the Bank's Board of Executive Directors. After an assessment report has been sent to the Board, it is disclosed to the public. About the IEGWB Rating System The time-tested evaluation methods used by IEGWB are suited to the broad range of the World Bank's work. The methods offer both rigor and a necessary level of flexibility to adapt to lending instrument, project design, or sectoral approach. IEGWB evaluators all apply the same basic method to arrive at their project ratings. Following is the definition and rating scale used for each evaluation criterion (additional information is available on the IEGWB website: http://worldbank.org/ieg). Outcome: The extent to which the operation's major relevant objectives were achieved, or are expected to be achieved, efficiently. The rating has three dimensions: relevance of objectives, efficacy, and efficiency. Relevance of objectives is the extent to which the project's objectives are consistent with the country's current development priorities and with current Bank country and sectoral assistance strategies and corporate goals (expressed in Poverty Reduction Strategy Papers, Country Assistance Strategies, Sector Strategy Papers, Operational Policies). Efficacy is the extent to which the project's objectives were achieved, or expected to be achieved, taking into account their relative importance. Efficiency is the extent to which the project achieved, or is expected to achieve, a return higher than the opportunity cost of capital and benefits at least cost compared to alternatives. The efficiency dimension generally is not applied to adjustment operations. Possible ratings: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Risk to Development Outcome: The risk, at the time of evaluation, that development outcomes (or expected outcomes) will not be maintained (or realized). Possible ratings: High, Significant, Moderate, Negligible to Low, Not Evaluable. Bank Performance: The extent to which services provided by the Bank ensured quality at entry of the operation and supported effective implementation through appropriate supervision (including ensuring adequate transition arrangements for regular operation of supported activities after loanlcredit closing, toward the achievement of development outcomes. The rating has two dimensions: quality at entry and quality of supervision Possible ratings: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. Borrower Performance: The extent to which the borrower assumed ownership and responsibility to ensure quality of preparation and implementation, and complied with covenants and .agreements, towards the achievement of development objectives and sustainability. The rating has two dimensions: government performance and implementing agency performance. Possible ratings: Highly Satisfactory, Satisfactory, Moderately Satisfactory, Moderately Unsatisfactory, Unsatisfactory, Highly Unsatisfactory. .. 11 Table of Contents Table of Contents .............................................................................................................. .. 11 Principal Ratings .............................................................................................................. ... 1 11 Key Staff Responsible ...................................................................................................... ... 1 11 Preface ................................................................................................................................. v Summary .......................................................................................................................... .. v11 1. Background .................................................................................................................. 1 2. Project Objectives, Relevance, and Implementation ................................................ 3 3. Achievement of objectives ........................................................................................... 9 4. Ratings ........................................................................................................................ 17 5. Issues: Sectoral Strategy and Community Management ....................................... 21 6 . Lessons ........................................................................................................................ 23 References ......................................................................................................................... 25 Annex A.Implementation of project components ........................................................ 31 Annex B . Supplementary Tables .................................................................................... 33 . Annex C An Ongoing Impact Evaluation of the Formal Transfer of School Management in Nepal ................................................................................................ 39 Annex D. Basic Data Sheet .............................................................................................. 41 Annex E. Borrower's Comments .................................................................................... 43 Figures Figure 3-1 : Resource Generation from Different Sources for Physical Facilities Development ... 15 Table 2-1: Objectives of the Community School Support Project (Cr. 3808) ................................. 6 Table 3-1: Funds raised by source in 30 selected schools ............................................................. 15 This report was prepared by Helen Abadzi. who assessed the project in September 2008 . Marie-Jeanne Ndiaye provided administrative support . ... 111 Principal Ratings ICR * ICR Review * PPAR Community ~- School Support Project (Cr. 3808) ~ Outcome Satisfactory Unsatisfactory Unsatisfactory Risk to Development Negligible to low Significant Significant Outcome Bank Performance Satisfactory Unsatisfactory Unsatisfactory Borrower Satisfactory Moderately Moderately Performance Satisfactory Satisfactory * The Implementation Completion Report (ICR) is a self-evaluation by the responsible operational division of the Bank. The ICR Review is an intermediate IEG product that seeks to independently verify the findings of the ICR. ** According to the 2006 harmonization guidelines, sustainability has been replaced with a "risk to development outcome" rating. Key Staff Responsible Task Manager/ Division Chief/ Country Director ~- Leader Sector Director Community School Support Project (Cr. 3808) Appraisal Rajendra Dhoj Joshi Michelle Riboud Kenichi Ohashi Completion Rajendra Dhoj Joshi Amit Dar Susan Goldmark V Preface This is the Project Performance Assessment Report (PPAR) on an education project in Nepal. The Community School Support Project (Cr. 3808) in the amount of US$5 million equivalent was approved on June 30,2003. The credit closed on September 30, 2008 after two extensions totaling 24 months, and fully disbursed. The PPAR was conducted to assess the outcomes of Bank policy and investments in a low-income country that underwent a period of civil conflict and political upheaval. The document is based on the following sources: Implementation Completion Report (ICR), Project Appraisal Document (PAD), Development Credit Agreement (DCA), PHRD Grant Agreement, and project files, particularly the supervision reports. Also, IEG consulted the research literature, reports on Nepal, and data on schooling trends. An IEG mission visited Nepal in September 2008 to interview officials, donors, and beneficiaries, observe instruction in schools, and collect other pertinent information. Field visits took place in Kathmandu, Dulikhel, Okhreni, and Charikot districts. The author thanks the government officials who received the mission for their extensive cooperation. Following standard IEG procedures, copies of the draft PPAR were sent to government officials and agencies for their review and comments. The Borrower's response has been taken into account in the report and included as Annex E. vii Summary This document reviews the performance of the Community School Support Project (Cr. 3808) approved in FY03. The Community School Support Project was supported by a Learning and Innovation Loan (LIL) that aimed to develop a knowledge base for the successful transfer of school management responsibility to local communities following legislation that devolved decision-making powers to that level. The project financed social mobilization and a menu of grants as incentives for school committees to accept formal management of their schools. According to the project design, the committees that agreed to do so were expected to raise funds, enroll out-of-school children, and be accountable to other school committees, such as parent-teacher associations and monitoring committees. The project surpassed its target of formally transferring 1,500 schools to community management. However, implementation suffered from fiduciary challenges and resistance by teachers' unions and insurgents. Deficiencies in the evaluation design impeded formation of the knowledge base needed to learn what could help communities manage schools effectively. The development objectives of this project were substantially relevant to the country's human resource development, but the relevance of project design was negligible. Project objectives were not clearly stated and design documents did not specify which responsibilities would be transferred to communities. There was no evaluation design on the basis of which the formal school transfer could be assessed. The overall rating for relevance was modest. The outcome of the Community School Support Project is rated unsatisfactory. the Despite the use of a LILY project did not generate the knowledge base needed to determine what makes for successful transfer of school management responsibility to local communities and to determine what the effects of transfers are on quality of learning and efficiency of resource use. Risk to development outcome is rated signijkant, because community involvement in poorer areas may be limited or financially unsustainable without extra funding. Bank performance is rated unsatisfactory. The Bank used a LIL to test certain incentives in promoting school management, but failed to design an evaluation system that would establish cause-effect relationships; instead it financed in subsequent projects a scale up of the experiment before adequately evaluating the results. Also, it did not sufficiently consult with all key stakeholders on project design. Finally, borrower performance is rated moderately satisfactory; the government agreed to the project strategy but many implementation arrangements did not materialize, and district education staff often had limited knowledge about the grants and terms offered under the project. This assessment provides a number of lessons for the education sector: Community management of schools offers many potential benefits, but it is important to understand what communities of different educational and income levels are likely to implement. The education of the poorest children should not depend heavily on the means available to the very poor communities. ... Vlll Before scaling up pilot activities, it is important to evaluate those pilots using robust evaluation designs. In particular, LILs should strive to create best practice in evaluation in order to maximize learning. Major educational reforms should be based on solid analytical work and take into account relevant research and evaluations of similar experiences elsewhere. There is little evidence that the design of the CSSP drew on research and evaluation findings of experiences with community management of schools, particularly as it relates to learning quality and outcomes, further compounding the lack of robust evaluation carried out under the LIL itself. Vinod Thomas Director-General Evaluation 1 1. Background 1.1 Nepal is a largely mountainous, predominantly agricultural country with a US$340 per capita income. The terrain and the country's cultural norms have impeded access to education for women, minority ethnic groups Oanjatis) and lower-caste Hindus (dalits). I Schooling opportunities for women have improved over time, but social disparities in access to education have remained high; in 2001, gross primary enrollment in the poorest fifth of households was only 22 percent compared to 90 percent enrollment in the wealthiest fifth.' 1.2 Alleviating the poverty of socially excluded groups, particularly women, became an important goal of Bank and donor strategy. Lending started in 1978 with two technical education projects, but the Bank soon focused on access to primary education through the Education I11 project; it then continued with the first and second Basic and Primary Education Projects (BPEP 1-11), Following the completion of BPEP I1 in 2004, the Bank co-financed, along with eight other donors, an Education for All (EFA) project (to be completed in January 20 lo), and it approved a School Sector Reform Program project in September 2009 (Annex ByTable 1). 1.3 Community management and user groups have been Nepalese traditions in multiple sectors. Until 1971, the country's educational system consisted of about 4,000 schools, mainly for the upper castes, which were managed by community committee^.^ In 1971 the National Education System Plan made all schools public, and the lower castes were eventually permitted to enroll. Generally school management committees that had traditionally run schools continued to exist, though they were often politically appointed (Carney and Shrestha 2009). 1.4 Social inequities gave rise to a civil war that started in 1996, intensified in 1999, and ended effectively in 2007. The projects implemented in Nepal in the early 2000s were affected at various times by the civil war. About 2,000 schools (out of about 2S,OOO) closed at some point. The need to flee conflict areas affected the education of about 250,000 children that often crowded classrooms in the Tarai region. Many supervisors could not visit their posts or visited only those close to roads. Some regional education offices were damaged, and records were destroyed. 1.5 Institutional weaknesses, geographical isolation, and the civil war revived government interest in community management of schools. In 1999, a Self Governance Act was passed, decentralizing the management of services to sub-national level of World Bank Country at a Glance, 2008; Janjatis account for about 37 percent of the population, and dalits for about 13 percent (Gurung 2002). About 42 percent ofNepalis are illiterate, 28 percent have only primary education, 32 gercent secondary and 1.6 percent higher education (Yadav 2008). World Bank (2001), p. 7 and Basic and Primary Education Project 11 (BPEP 11) Project Appraisal Document (PAD), p.58. Nepalese schools offer primary education (grades 1-5), lower secondary education (grades 6-8), secondary education (grades 9-10) and higher secondary education (grades 11-12). The Department of Education regulates grades 1 to 10, and the Higher Secondary Education Board grades 11-12. Schools tend to have primary and secondary grades in one facility. G. Shrestha et al. 2002. 2 government (municipalities/villages and district levels) and strengthening community participation in their management for sectors such as education, health, drinking water, irrigation and forestry. Support for decentralized decisions in education has become an integral part of the government and donors' education strategy. The Bank has also supported community management in almost all sectors through investment credits and the First Poverty Reduction Support Credit (PRSC; FY06).4 Furthermore, the Basic and Primary Education Project (BPEP II), approved in 1999, supported community management in all schools of the country and built capacity at the central, district and local levels to support all committees (IEG 2009). Under BPEP 11, the Department of Education undertook extensive training to promote decentralization policies and strengthen field staff competencies at district and local education offices (e.g. inspectors and supervisors were trained on financial management). A number of evaluative studies were also undertaken to study the effects of community management vis-a-vis other factors influencing student learning. 1.6 In 2001 the Education Act was amended to provide a legal framework for partnerships between elected local governments (municipalities for urban areas, Village Development Committees (VDC) for rural areas and District Development Committees (DDC)) and the central government at the national and sub-national levels (mostly district level administration of line ministries). All public schools were designated as "community schools" and were to be managed by a School Management Committee (SMC). The regulations provided for a committee consisting of one chair and three parental members, including one female, to be selected by the parents from among themselves. Members include the Ward president of the VDC or Municipality where the school is situated, one person nominated from among the founders of the school or donors to the school, one teacher selected by the teachers, and the headmaster, who is also the secretary.' The Act did not specify the procedures for committee member selection. 1.7 Before the BPEP I1 appraisal there was a pilot of organizing committees consisting mainly of parents, including mothers. Community responsibilities included hiring and paying temporary teachers, reviewing annual school performance, controlling school finance, managing school property and funds, mobilizing additional resources, distributing textbooks and scholarships, nominating teachers for training, and drawing up school improvement plans. These tasks would be performed through the head master (BPEP I1 PAD, p. 60). 1.8 A 2006 study of schools in three districts (regular and transferred) found that community-based monitoring was functional in many schools (CERID 2006; also see Annex B Table 3), but overall had many limitations. There were no conceptual frameworks or minimum standards, training for committees had been insufficient, and the task was perceived differently by different people. In particular, the study found that committees did little monitoring or supervision of classroom instruction. The better educated committees tried to ensure that students did homework, but less educated communities had little interest in the school. School development initiatives were taken by PRSC ICR, p. I . The PRSC priorities included strengthening community user groups in irrigation and rural roads and also a handover of sub-health posts to management committees. Nepal Education Act 2028 (l971), as amended by the Seventh Amendment 2058 (2001) and in effect since 2002. individual committee chairs rather than member teams, but many chairmen were not guardians of students, and their interests in education were unclear. For example, only 34 percent of preschool chairmen were actual guardians of children in the school. The study also found that leadership of the head teacher was cruciaL6 The findings raised questions regarding the effectiveness of school management committees as the central strategy for improving educational quality, particularly in low-income areas. 1.9 A longitudinal study (2002-2006) found that in general, school-level planning had a positive and significant effect on student promotion rates, even controlling for socioeconomic status (CERID 2007). The effect may to some extent indicate teacher quality; teachers were involved heavily in formulating school improvement plans, and it is possible that the schools with more task-oriented teachers were able to plan better and also help students learn more. Also, school-level expenditures had a highly positive effect on examination score and promotion, although controlling for students' background reduced the magnitude of the effect. This finding suggests that community fundraising may significantly contribute to learning outcomes. Thus better-off communities may give their students an advantage separate from management effects. 1-10 The projects after BPEP I1 (see para. 1.2) continued to fund various initiatives in all schools, but emphasis was placed on empowering communities by offering them a formal transfer of responsibility along with various incentives. This methodology was piloted through the Community Schools Support Project (CSSP) and adopted by subsequent projects. Various sections of this report discuss relevance, implementation experience, mission visits, ' and evidence regarding the extent to which the activities and inputs of the Community Schools Support Project may have contributed to outputs, outcomes, and impacts. 2, Project Objectives, Relevance, and Implementation 2.1 The Community School Support Project was a US$5 million LIL that became effective in 2003. As mentioned earlier, BPEP I1 financed training and inputs to help management committees in all schools of the country operate schools more effectively (IEG 2009). By contrast, this LIL tested a model that was applicable only to committees in the entire country that agreed, after an advocacy process, to take over formally the responsibility of managing their schools (objectives in Table 2-1). As an incentive, committees would receive a one-time grant of about US$1,200 (NRS 100,000) to use as they saw fit and become eligible to apply for a menu of other grants. The committees would have to be democratically elected, prepare action plans for the school, raise funds, and be accountable for expenditures to other school committees that would be formed for CERID 2006, p, 6. ' The IEG mission visited nine schools in Kathmandu, Dulikhel, Okhreni, and Charikot districts: Viswaniketan primary and secondary school, Kathmandu; Samajik Kalyan primary school, Dulikhel; Kalika Praveen school, Manthali; Sri Janakalyan, Okhreni district; Gaurishankar higher secondary school, Ramechhap; Banghari primary and lower secondary school, Ramechhap; Bhimeshwar primary school, Ramechhap; Sri Janavijaya primary, rural Charikot, Bhairawi secondary school, Sunarpani, Charikot. This was a convenience sample and was used for illustration rather than rating projects. Nearly all schools were in poor rural areas of hilly terrain. Classes were extensively observed, and interviews were held with teachers, principals, and some management committee members. Pertinent mission findings are discussed in various sections. 4 this purpose (a process called a "social audit"). However, some important details were not clarified.8 2.2 In 2001 , the Education Act had just passed, and means were being sought to implement it effectively. According to project documents, the Bank and the Government started discussing the concept of "formally" transferring schools to community management in late 200 1. In the 2003 Country Assistance Strategy, the transfer of schools to community management became the Bank's main strategy for quality improvement in primary education in Nepal, along with provision of textbook^.^ According to some officials interviewed, there was limited government interest, partly because the insurrection made it difficult to reach out to some communities. In 2002 the Ministry of Education and Sports (MoES) adopted the formal transfer of schools to community management as a policy and committed to targets on the number of formal transfers (PRSC ICR 2006, p. 12). 2.3 A comparison of the DCA and the PAD reveals a lack of clarity about project objectives (Table 2- l), the definition of a "community-managed school", the additional responsibilities to be transferred to communities beyond those given to other community- managed schools, and the relationship between the objectives and components. Specifically, the following issues have arisen: (a) The definition of community-managed schools in the PAD is unclear. l o Technically speaking, all Nepalese schools are managed by communities in the sense that all government-funded schools are supervised by a School Management Committee (SMC) which included community representatives. It is likely that committees have varying levels of activity, but the PAD did not make the distinction between the existing government-funded schools supervised by a SMC and those that were supported by the project. Readers may thus get the impression that only certain schools are community-managed. The PAD and subsequent project documents also failed to clarify that the regular community schools and the formally transferred schools essentially differed in just two items: a letter of transfer (Figure 3-6) and a one-time grant of about US$1,200. Further grants required an application. The IEG mission School catchment boundaries are often unclear, and it is uncertain who can vote in an election, how many votes a family has, or whether the ballot will be secret. Parents and guardians would participate, but voting participation rates were not reported. In principle, anyone can sponsor a child and run for committee membership. The MoES Flash report showed that only 5 percent of schools had actual elections for committees in 2006, and the remaining schools selected the committee members. The 2008 Flash report showed that 65 percent of management committees and 63 percent of garent-teacher associations were selected rather than elected. CAS 2003. For primary and secondary education, the Bank's strategy for improving access and quality was to (a) ensure efficient and timely textbook distribution to all students in the beginning of the year; and (b) convince management committees of schools throughout the country to take up formally the responsibility of managing schools. By the end of the 2003 CAS period, the expected outcome was that 25% of schools would be under community management and that the management would provide (a) inclusive and (b) higher quality education (World Bank 2003, Annex B IO). l o The Nepal Country Assistance Evaluation states on p. 58:"In principle, since 2002, the management of all government-funded schools is supervised by an SMC. The "formally transferred" schools refer to government-funded schools for which the SMC has entered in a formal agreement with the DEOs that stipulates respective parties' responsibilities. In exchange for more responsibilities and autonomy (hire and pay temporary teachers, review annual school performance, control school finance, mobilize additional resources, distribute textbooks and scholarships, nominate teachers for training, and prepare school improvement plans), these communities would receive a grant of approximately US$1,200 and eligibility for a menu of additional supporting grants." 5 verified in discussions with the task team and Nepalese educators that the schools participating in the formal transfer process received no additional authority or resources over other schools, beyond the one-time transfer. (b) The responsibilities to be transferred to communities are not described in the DCA or in the PAD. Since the government has stated that it has not withdrawn from any responsibilities, it is unclear what responsibilities were supposed to be transferred to communities, if any. It is also unclear how one would define a successful transfer of responsibility. l 2 (c) The objectives in the PAD and the DCA differ in several key points. The DCA refers to the formation of a knowledge base on transferring management to communities, whereas the PAD has two formulations that are different: one, which refers to mere acquisition of critical experience prior to scaling up management transfer, and a second that defines the development objective as testing "whether community management of schools can better contribute to enhance (i) participation rates, (ii) quality and efficiency, and (iii) accountability." If the development objective was simply to acquire critical experience without formal documentation, then the project did not conform to the LIL guidelines. Also the PAD did not specify what topics a knowledge base would contain or what format it would have. (d) The financed activities and numerous statements in the project files suggest that the project de facto had an implementation rather than an experimentation and learning goal. Every implementation status and results (ISR) report mentions as targets the numbers of schools transferred to community management rather than the development of a knowledge base. Rather than use the LIL for experimentation and evaluation, the credit proceeds were used to scale up the formal transfer of schools to community management before the concept could be evaluated. l3 During appraisal, the government had reportedly planned to give transferred schools extra resources for temporary teachers, but finally other schools received temporary teachers as well. Also, the Eighth Amendment of the Education Act (2005) was passed to suspend central recruitment of teachers and enable transferred schools to hire all teachers locally, but this has not been implemented. The ICR of the project (finalized six months after the PPAR mission, p. 1) states that transferred schools may appoint head teachers and release non-performing permanent teachers, but the exclusivity of these functions was unclear. No regulations show the government withdrawing from oversight responsibilities, as also discussed in project documents. During interviews, the IEG mission verified this. l 3 The EFA project included US$7 million for incentive grants, and the Additional Financing for the EFA project included another US$25 million (World Bank 2007b, p. 24). 6 Table 2-1: Objectives of the Community School Support Project (Cr. 3808) Objectives Components Development Credit Agreement: a School grants (US$3.66 m at appraisal, US$4.9 m The objective of the Project is to develop a final) for (i) incentive grants for committees taking knowledge base needed for successful over management of government-funded schools; (ii) transfer of responsibility for management of block grants tied to performance for post-primary government-funded community schools to schools transferred to community management; (iii) local communities. performance grants for improving access to primary education and promotion rates; (iv) supplementary Project Appraisal Document: grants to community schools for expanding The objective of the proposed Community School enrollment to cover additional costs; and (v) other Support Project (CSSP), a Learning Innovation Loan grants for support to marginalized communities, and (LIL) was to assist the government to gain critical piloting of innovative approaches. experience for successfully up-scaling the transfer of 3 Scholarships (US$0.44 m at appraisal, US$0.13 m management of government-funded schools to final) for scholarships to poor out-of-school children communities (p. 2). (Rs. 500-1000 for the first child of disadvantaged families in primary and secondary schools and Rs. Note: PAD Annex 1, p. 16 lists the DCA objective 250 for subsequent children. as a "follow-on development objective" and includes a new "project development objective:" 3 Capacity building (US0.42 m at appraisal, US$0.3 To test whether community management of schools m final) for (i) assistance to communities for can better contribute to enhance (a) participation managing schools; (ii) need-based training for rates, (b) quality and efficiency, and (c) teachers and committee members; (iii) orientation for accountability. The DCA, Annex 2, under Part D of civil servants, local government officials and the project also specifies the following activity to be resource persons; and (iv) policy communications. undertaken: "Developing a system of Project zMonitoring and Evaluation (US$0.48 m at monitoring and developmental impact evaluation" appraisal, US$O. 10 m final) for a well-established (P. 14). and smoothly operating M&E system that could support periodic improvements in the experimental design of the LIL. 2.4 In 2005, a recipient-executed PHRD grant (TF055 146), was approved for the project. It was a Policy and Human Resources Development (PHRD) implementation grant for US$699,600, which disbursed US$633,206.18 for: (a) community awareness and social mobilization, (b) monitoring system for school enrollments, retention and completion, (c) quality monitoring system for decentralized schools, (d) study on the role of community characteristics on project take-up and recommendations for strengthening of capacities, (e) best-practice handbook and network peer-to-peer support activities, and (f) project implementation plan. Project documents also show that the grant paid for other activities, such as NRS 50,000 to each of 30 schools to buy computers. The grant application uses a different statement of objectives from others found in the PAD: "The project's short-term objective is to put in place the set of conditions needed to ensure an effective management of schools by communities and thus, to permit a successful . implementation and scaling-up of the decentralization policy." 2.5 This PPAR presents results on achievement of two objectives: (a) the DCA objective that has a relatively specific output, a knowledge base; and (b) the objective mentioned in the Annex 1 of the PAD, which is more specific and was used for the Implementation Completion Report (ICR) and in the project's supervision reports. 7 RELEVANCE 2.6 The relevance of objective(s) for this project is rated substantial. Community management is a Nepalese tradition and reflects a government strategy for improved services in many sectors. The 2003 Country Assistance Strategy (CAS) and the Poverty Reduction Strategy Paper (PRSP) focused on community management of schools as the primary means of improving quality of education. Therefore a knowledge base would be important for successful implementation as would be an assessment on whether community-managed schools enhance participation, quality, efficiency, and accountability to citizens. 2.7 The relevance of project design is rated negligible. According to Bank guidelines, a LIL should last about 3.5 years and focus "on experimentation, learning and piloting in search of possible developmental solutions, prior to potential larger-scale operations." l 4 However, the formal transfer of schools to community management was not tested against a control group. No clear rationale was given for the grant amounts chosen. l5 The eligible schools were self-selected on willingness to undertake this process, but the project did not monitor the characteristics of schools that rejected the offer. 2.8 It would have been wise to assess the incidence and level of activity of school committees in the country during or shortly after project appraisal and to adjust project design accordingly. A sample of 80 schools surveyed in 2004 showed committees in all schools (New Era 2005, p. 39). But another survey available the time of project effectiveness showed very poor preparation of the committees for the tasks envisaged for them.16However, the study received no attention in modifying the project design. Thus, the population of schools formally transferred to community management was the result of a selection process whose determinants were unknown. Financing, socioeconomic status, and the $1200 financial incentive became confounding variables. Given these design faults, the LIL could not evaluate the effectiveness of the CAS strategy to use community management as a means to improve quality of education in Nepal (paras. 2.2,2.6). 2.9 Bank staff interviewed by the IEG mission expressed a strong belief that democratically elected management committees would be accountable to citizens and perform better than formally or informally appointed committees. " However, the Bank 14 According to Operations Policy and Country Services (OPCS) guidelines, LILs are loans of $5 million or less financing small, experimental, risky and/or time-sensitive projects in order to pilot promising initiatives and build consensus around them, or experiment with an approach in order to develop locally based interventions prior to widespread implementation. A LIL focuses on experimentation, learning and piloting in search of possible developmental solutions, prior to potential larger-scale operations. Clarity of the learning objectives and the system to monitor and evaluate them are vital to the purpose of the project (Operations Manual, dated since 1998). l 5 "...the proposed incentive grants together with performance grants described below, will be kept within the range of the supports schools are receiving from the development budget (PAD p. 23)." However this range was not provided, and there was no piloting to determine which amounts would be sufficient to motivate communities. l 6 CSNN 2005 (p. 11) cites a 2003 CERID of Ilan and Morang districts ("Management Transfer of Public Schools under Formative Research Project") which presaged the problems that later became salient. I 7 The PAD states that "Community-managed schools are expected to perform better than government-managed schools, as such schools will be accountable to the community, whereas the latter is accountable only to the Government" (PAD p. 22). Annex 2, p. 27 states that "The main hypothesis of the project is that community management of government-funded schools can better enhance access to schools for all children, including children 8 did not offer behavioral or other research to support the hypothesis that a nominal transfer to community management in Nepal would increase motivation among community members. Some behavioral research findings might predict positive outcomes, but other research suggests that nine-member committees lacking individual responsibility may fall victims to `social loafing' (e.g., Latane et al. 1979). The project had no plan to test the sustainability of committee activities. 2.10 The LIL design did not consider the likely effects of the insurgency on the project or the lessons of earlier Bank projects. Some operations completed in Nepal before,the LIL appraisal raised questions regarding elite ca ture and the ability of the very poor to build schools or use communal forest resources.` Yet the PAD did not explore the possibility that community management might have an adverse impact on equity. Also the risk assessment made no mention of the possible impact of the insurgency or of the political risk that the teachers' unions would prevent the local hiring and management of teachers. The only risk seen with respect to communities was the inability to mobilize additional resources (PAD p. 14). 2.1 1 The design was also weak in fiduciary aspects. Various administrative arrangements (e.g. how the budget was to flow to schools) were outlined in the PAD in great detail, but fiduciary checks and balances relied on a set of four committees per school that did not exist during appraisal and were not formed later. The feasibility of their establishment was not explored. (See quality at entry rating in para. 4.4). Implementation Experience 2.12 The project experienced considerable problems in all aspects of its operation. The most important are discussed below. 2.13 Several administrative arrangements foreseen in the PAD did not materialize. The implementing agency was the Department of Education, as with BPEP 11, and a project council should have been constituted in the MoES, but a council was not active. District Education Offices did not act with the expected degree of attention to the program and gave limited counterpart funds. Project documents show that some district staff were confused by the multiple grant options. l9 2.14 Internal controls remained weak during most of the project life. Most of the schools did not maintain the required information to the level of detail agreed (New Era 2008). Project documents and audit reports show repeated warnings for delayed audits, including a warning by the Auditor General in 2008. In 2007, about 70 percent of the from disadvantaged communities, and better improve efficiency and quality of school education compared to schools not transferred to community management". l8 E.g. the Earthquake Schools Rehabilitation project (Cr. 2048) ICR and the Project Performance Audit Reports of the Hill Community Forestry Project (Credit 2028-NEP) and the Second Forestry Project (Credit 1400-NEP) 2001, OED Report No 22438. A 2007 supervision document for example stated that district education office staff, including resource persons, did not have clear a understanding about the objectives, justification and implementation detaiEs of the policy of transfer of schools to community management and that they were thus not able to confront arguments against community management of schools put forward by the opponents of the policy. 9 transferred schools were completing social audits. 2o Action plans for improving financial management and procurement practices were to be developed, but it is unclear what remedies were applied. Some teachers filed a corruption case with the government against some district office staff in August 2005. 2.15 Some Bank procurement practices in the project also raise management questions. The Bank financed consultant contracts for school surveys to the Community Schools National Network (CSNN), an organization purportedly consisting of school committee members. CSNN was also hired by the government with PHRD grant funds to conduct surveys. However, the PHRD grant agreement has a condition that consultants paid by the grant would not also be paid by the Bank during the grant period.21 Informants raised questions regarding a potential conflict of interest in the use of CSNN as an entity for designing and conducting surveys; a 2006 supervision report mentions concerns that those collecting the data were over-reporting enrollments of poor children (para. 3.7). 2.16 According to project documents and IEG mission interviews, opposition by teachers, educational researchers, academics, and the Maoist insurgents to the concept of handing schools over to communities delayed implementation. The project was extended twice by a total of 27 months, and the Bank funded a public information campaign2'. 3. Achievement of objectives 3.1 The project financed the transfer of 2,035 schools to community management, exceeding the planned 1,500 referred to in the PAD, and an,additional5,216 schools were transferred under the subsequent Education for All As there were roughly 38,000 schools in Nepal in 2006-7, the CSSP covered about 7 percent of the total (Annex ByTable 6). This chapter reviews the evidence on the extent to which the objectives of the project were achieved in these formally transferred schools. 2o The MoES management information system recorded in 2007/08 that nationwide 62.3 percent of schools completed social audits, whereas among the formally transferred schools 70 percent completed social audits (ISR 9/21/07). The Ministry formed a committee chaired by the Director of Administration and Physical Service Division to oversee the social and financial audits of schools. *' The Community Schools National Network (CSNN) was set up in 2004 with technical assistance paid by the Bank's budget, and has received money from other donors as well. It only represents the `transferred' schools rather than management committees in general. According to informants, an undetermined number of management committee members are not parents per se but have `adopted' children to become eligible for membership. " A team of consultants specializing on political economy of reforms was engaged to work with schools and to help strengthen the capacity of Community School National Network - an umbrella organization of community-managed schools" (Excerpt from project documents, 9/27/07). In 2004 -2007, Bank funds were used to hire a consultant group referred to as the Educational Journalist Group to document and disseminate project outcomes through television, radio, and print media. 23 World Bank (2009), p. iii and Annex 2. Schools in Nepal often have grades 1-10 together; primary, lower secondary, and middle secondary count as one unit each. At the end of the project, 6,590 school units had agreed to the formal transfer process. 10 Develop a knowledge base needed for successful transfer of responsibility for management of government-funded community schools to local communities (modest) 3.2 The project and the PHRD grant financed several thoughtful and innovative activities that in principle could help make school committees throughout the country more effective. Supporting organizations, such as local NGOs, assisted in implementation. The activities notably included (a) advocacy to children's guardians about their roles in school management; (b) peer-to-peer networks and internal study that (according to reports) resulted in useful information exchange among committee members; (c) a report card to 100 schools that described their conditions and challenges; and (d) a best-practice handbook. Other outputs included a case study of successful and not so successful community schools and a study on the role of community characteristics on project take-up and recommendations for strengthening of capacities.'' 3.3 The activities produced some important lessons regarding service delivery to the poor. The cooperation of teachers and headmasters in the handover process was crucial (CRPeC 2008). The evaluation of an operations manual showed that many committees depended on the headmaster for reporting and handling committee financial accounts (a potential conflict of interest). Financial reporting requirements were complex, and committee members needed accountancy training. The educational level needed to read the operations manual and follow its directives suggest that many of the transferred schools belong to communities with educated residents. Poorer communities might be unable to handle the accounting and reporting demands of formal management. Some NGO staff lacked sufficient training for effective social mobilization, and many district education officers had conflictive relations with committees. Other studies (CERID 2006) also resonated on the difficulties of managing poor schools effectively (para. 1.7). 3.4 The studies conducted by the project could have given considerable insight on community decisions, but most were poorly designed and could offer little specific information. (See monitoring and evaluation section.) Some appeared to be advocacy pieces, with unsubstantiated statements about the schools being `transformed'.26The studies focused on the transferred schools, thus missing the opportunity to learn how the regular community-managed schools operated across time and how the two types of schools differed in inputs, outputs, and outcomes. 3.5 An important argument made in the PAD in favor of formal transfer to community management was the expectation that committees would lower teacher absenteeism and delays because of increased accountability of the schools to the communities. Thus, parents would normally be expected to monitor teacher absenteeism 24 World Bank (2006). Study visits were arranged to effectively managed schools so that others could learn from them Upadhyay et al. 2008). 15 CPReC 2008, Chapter 1. 26 For example, a follow-up of 10 schools contains comments such as `Once the agreement between the SMC [School Management Committee] ... and the D E 0 [District Education Officer] is signed which transfers the management of school to the community, the school gains a new status and new sense of belonging. The school finds itself in a new context of responsibility and a compelling reason to take a new direction.' (G. Shrestha 2005). 11 The ratesa2? project would have also been expected to monitor the plans made by communities for vigilance.of teacher performance as well as the number of teachers fired or transferred due to community vigilance. However, MoES confirmed that these data were not collected. Some self-reported teacher attendance data are available from a set of 30 selected schools, but their reliability is unknown.28Overall, the project collected little generalizeable information that would serve as a knowledge base. Lack of pertinent knowledge handicaps the government?sintent to scale up citizen involvement in schools in an effective and efficient manner. Test whether community management of schools can better contribute to enhance: (a) participation rates, (b) quality and efficiency, and (c) accountability (negligible). 3.6 The project limited this test to schools self-selected for transfer to community management, so this objective was not met for community management in general. Also, the project?s de facto evaluation design - which involved baseline and endline surveys -- did not take the self-selection of communities for formal transfers into account, nor did these surveys track the same communities over time. Nevertheless, this section presents the evidence collected with respect to this subset of schools. Part (a): Enhancing participation rates 3.7 The term ?participation? refers to enrollment rates. In communities that agreed to the formal transfer of school management, students? guardians were eligible for various types of grants if the schools (i) mainstreamed out-of-school children; (ii) mobilized in order to include indigenous, disadvantaged and dalit communities; and (iii) improved student promotion rates.29A follow-up study of 30 selected schools in 2004-06 found that out-of-school children in the service areas of formally-transferred schools dropped from 41 to 15 percent, out-of-school girls from 44 to 15 percent, while out-of-school janjatis and dalits dropped from 44 to 15 percent and from 50 to 18 percent, respectively. This met or exceeded the project?s targets set in the PAD (p. 19). However, the ?service area? was in fact a selected sample of intensely supported schools where the highest levels of interventions were provided. Furthermore, concerns were raised about the reliability of the underlying data. 30 Another report stated that enrollments had fallen short of PAD targets. The latter study also reported discrepancies between household data and school survey data as well as between school and MoES figures, so the magnitude of the 27 Management committees are not authorized to fire permanent teachers or principals, but they may request their transfer. According to informants interviewed during the mission, head teachers often agreed to the ?transfer? of only one school unit (e.g. the primary or lower secondary section of a K-10 school); this would ensure that they would keep their position. 28 A study of 220 ?regular? community schools showed that only 40 and 54 percent of schools had all teachers present on each of two surprise visits. Teacher absenteeism rate was 17.8 percent on the first visit and 12.8 percent on the second. About 50 percent of principals had taken no action on absenteeism, and only 5 percent had requested a teacher?s dismissal (New Era 2008). 29 DCA p. 3. According to the ICR, 335 supplementary grants were given, some to multiple schools. At least 160 schools received ?intensive support?, that is, multiple grants, based on a first-come-first served basis. 30 World Bank (2006b), Table 4.1 and Figure 4.3. Aide-memoires also express concerns about exaggerated self- reporting ofjanjati and dalit enrollments by school management committees. If the figures are correct, then the 41 percent out-of-school children in the catchment area seemed inordinately high given the approximately 20 percent recorded by New Era (2005, p. 19) and an average net enrollment ratio of 88 percent for the rest of the country in 2005. The ICR offered no evidence that community management rather than other factors were responsible for this change. 12 reduction attributable to the transfer to community management, if any, is unclear (World Bank 2006b). In any event, the effect of formal transfer on results could not be established without comparing the results to those of an adequate control group. Further studies were to be undertaken in 2009- 10. 3.8 To encourage committees to enroll out-of-school children in their schools, the government pays to all schools block grants on the basis of the number enrolled. Grants are small and barely cover teacher salaries. There is no money to buy any educational materials or repair premises (Vaux 2006). Management committees may find it necessary to charge school fees (called exam or entry rather than tuition fees), but in Nepal education is free for the primary grades, so this practice is strictly speaking illegal.31 Funding limitations have created disappointment, and may have also encouraged overstatement of enrollments and charges of fees. Important equity issues arise when the poor must raise money to send other poor children to school (Section 4). 3.9 During IEG mission visits, significant absenteeism rates were noted in all poor schools, transferred or otherwise. (Three schools, two of them transferred, where a complete count could be made had average attendance of 57-67 percent; the 2008 MoES statistical report ("Flash report") showed grade-wise average attendance of 58-67 percent.) Two committee chairmen of formally transferred schools who were asked about the absenteeism rates replied that it was not possible to bring certain children to school every day. Some lived too far, or parents were not yet conscious of schooling need. Overall, the extent to which participation rates were raised as a result of community management is unknown, and the difficulties in raising them were not investigated. Part (b):Enhancing quality 3.10 Project reports suggest that a number of schools taken over by committees function as envisaged. Many of the communities that chose to take over schools had the financial or intellectual means to manage them.32 They could provide a "separate room for each grade/section, library, computer class and office room. In some schools, there was provision of separate office room for the committee. Similarly, because of plastered walls in all rooms, display of different records in the office and display of student and teacher- made materials in the classrooms were possible. In well-partitioned and spacious classrooms, teachers felt comfortable to conduct class activities in an interactive way." 31 See reports in Annex C; according to Carney et ai. (2006) there were cases of parents forced to withdraw children from certain schools because of fees. 32 About 88 percent of committee members were literate, about 55 had more than primary education (New Era 2005 p. 128). Also, according to a survey of a'stratified random sample of 220 regular community-managed schools that had not yet been transferred, in 16 districts (New Era 2008), a third of management committee members had some primary education, 11 percent had completed primary, 28 percent had completed secondary, and 11 percent had higher education. They were 83 percent male and held about 10 meetings annually. In meetings the most frequent discussion topics were improving school infrastructure (91%), finances (85.2%) and enrollment of out of school children in schools (85.2%). Less frequently discussed matters were: improving quality of teaching and learning (83.3%), school administrative issues (75.7%), mobilizing local resources for school (69%) and community participation in school affairs (61.9%) (New ERA 2008, pp. 35-37). About 23 percent of household heads reported participating in a meeting, nearly 68 percent of respondents reported that they had donated in one way or the other to the school in their community, and in 86 percent of the cases it was physical labor. About 48 percent of the respondents thought the school was good or very good, 36 percent thought it was average, and 13.8 percent thought it was bad; 79.4 percent were satisfied with teachers' performance and 77 percent thought teachers were dedicated (New ERA 2008, p. 47-50). 13 (World Bank 2007d). Some schools in the Ilam district provided computers and email service. Some other districts introduced English-medium instruction in grades 1-3.33 After such improvements, some students reportedly moved from private to community schools. Project documents report such cases as evidence of improving quality, but generalizeable evidence was not obtained. 3.1 1 A lot of student performance data were collected, though the datasets have weaknesses. The differences by school type may reflect the different socioeconomic levels of the communities electing to participate in the formal transfer, plus effects of additional financing. Also, self-selection bias may exist independently of advocacy efforts. Background variables did not include consistent measurements of variables necessary for improved information processing, such as instructional time use, textbook availability, and teacher knowledge. Thus, individual studies could not provide conclusive evidence that changes in test scores are due to formal transfer to community management. Below is a summary of the main studies and findings (also see Annex B, Table 5). 3.12 An initial purposive sample was taken of 80 transferred and not transferred schools (data collected in May-October 2004; New Era 2005). Test scores of over 1300 students showed baseline differences among schools receiving an incentive vs. additional types of grants. The latter, as well as their controls, often scored higher than the incentive grants schools, but there was much variation in Nepali, math, and social studies among grades. The test was not comparable to the many achievement tests given earlier (Annex ByTable 2), so it was not possible to compare the performance level of this sample with that of other schools of Nepal. This dataset showed that all schools had management committees and that in most characteristics the `transferred' and `regular' schools did not significantly differ (New Era 2005). This dataset was apparently not used further. 3.13 A subsequent pilot impact evaluation study was carried out by a research team using a random sample of 80 communities/schools from six districts in the country, none of which had yet undergone the formal transfer; 40 were randomly "treated" with NGO advocacy and 40 were controls (did not receive advocacy; see Chaudhury and Parajuli (2009). The objective of the pilot was to ascertain roughly what share of the schools that received advocacy were likely to transfer, as the basis for designing the subsequent main impact evaluation in 220 communities/schools nationwide. Between the baseline of the pilot impact evaluation study (December 2005-February 2006) and the follow-up survey (August-November 2007), 15 of the 40 schools in the treatment group and 5 of the 40 schools in the control group agreed to be transferred to formal community management. The pilot baseline and follow-up surveys included an achievement test for 1300 5th graders in 65 schools. Initial evidence suggested that 18 months of formal community management in this sample made no difference in the learning outcomes except in science. However, the sample size of the pilot study was too small (only 80 schools) to draw any robust conclusions about the impact of the formal transfer of school management. The results of the much larger impact evaluation research project (not financed under this project), launched in 2007, will be able to address this question, 33 Sumansinh Tamang. Kantipur, September 23, 2008. Samudayak vidyalaya angrezi madiam (Community schools go English medium.) 14 controlling for the self-selection bias inherent in the voluntary nature of the formal transfer (see Annex C).34 3.14 A different test was administered once to students of grades 1, 3, 5 in 300 schools during 2008; it showed small and inconsistent differences between regular and transferred In addition, a cross-sectional achievement test given to a sample of Nepalese schools in 2008 showed small differences between schools that were formally transferred to community management and regular schools (1.9-3 percentage points; EDSC 2008; Annex B, Table 7). However, the simple comparison of outcomes between formally transferred and non-transferred schools cannot be interpreted as the impact of the formal transfer because they do not take into account the selection bias of communities electing to formally transfer. Also it is not known how each type of school performed prior to the transfer. 3.15 According to the PAD, teachers of transferred schools were to receive customized training, but reports do not mention whether the training took place, what it consisted of, and whether it resulted in improved classroom behaviors. Committees often gave first priority to financing buildings and furniture,36so it is unclear how much attention teaching quality received (Annex B, Table 8). Furthermore, tests given to the teachers of a selected sample of schools showed that teachers in transferred schools did not score better than those of regular schools. In Nepali, teachers in the two sets of schools had essentially the same scores (55.6 vs. 55 percent respectively), whereas in English and math, teachers of regular schools scored better (41.2 vs. 44.8 percent and 54.1 vs. 5 1.3 per~ent).~' Perhaps teachers of transferred schools had lower baseline knowledge. But baseline data did not exist, so the benefits of formal community management transfer on teacher knowledge could not be documented. 3.16 The IEG mission visited four schools formally transferred to community management, all in poor rural areas3* Two committee chairmen who were present showed little surprise at the teacher absenteeism and delays, student absenteeism, lack of teaching aids or noise from other classrooms coming through open transoms. In IEG interviews, chairmen were found to consider the way schools operate in poor areas as their norm. It was unclear, from these interviews, how management in these communities would result in improved quality. . Part (c): Enhancing efficiency 3.17 The PAD did not specify whether the efficiency objective of transferring school management to communities was to enhance internal efficiency (dropout and graduation 34 The impact evaluation research was funded by the Education Program Development Fund (EPDF) of the Fast Track Initiative, and was independent of the CSSP project. The results measuring the impact of formal transfers are expected in 2010. 35 Full Bright (2008), tables 5.7-5.10, p. 67-68, 82. In English, Nepali, and math, the two types of schools had differences of 1-2 percentage points, with regular schools often scoring higher. The magnitude and direction of selection effects are unclear. 36 World Bank 2007 (draft p. 36). Also CERID 2006. 37 Full Bright (2008). 38 See the school observations in IEG (2009), Chapter 2. 15 rates) or to improve the efficient use of public funds. The key performance indicators in the PAD include transition, repetition, and dropout rates. The ICR did not present data on these and had no indicators of management efficiency or cost-effectiveness. The various evaluation reports had limited evidence regarding either type of efficiency. 3.18 An important project expectation was that committees would raise funds and supplement the government's funding. The ICR (p. 11) argues that one rupee of investment generated 1.5 additional rupees, which might increase the efficiency of government funds use. The survey of 30 selected schools showed that committees did raise funds (CSNN 2007), though most resources were obtained from international NGOs. Only about 15 percent of the funds raised were generated by community members (Table 3-1). Furthermore, all schools visited by the IEG mission, both transferred and otherwise, reported that they were raising funds and paying contract teachers and other small expenses. It is unknown whether transferred schools raised more funds than regular schools, if the socioeconomic status of the community were held constant. Table 3-1: Funds raised by source in 30 selected schools Figure 3-1: Resource Generation from Different Sources for Physical Facilities Development Sources International 2004 2005 2006 Total I DDC. 7 Local NGQ, NGO 429 90,710 140,439 231,577 National NGO 13,787 9,344 1,852 24,983 Local Agencies 18,495 19,621 17,979 56,095 Community 56,279 92,495 82,006 230,779 71.2 Total 88,989 212,170 242,276 543,434 Local NGOm VDC INGO Corn DDC Conversion in 2008 US$ dollars Note: DDC: District Development Committee; VDC: Village Development Committee; INGO: international NGO; Com: Community 3.19 Earlier efforts to encourage poor communities to complete or repair schools had limited outcomes because of limited capacity and resources (e.g. the Earthquake Schools Rehabilitation Project, Cr. 2047). Some of the schools visited by the IEG mission clearly did not have the means or organization to carry out civil works satisfactorily. A 2006 survey of selected transferred schools showed that many did not spend the incentive grant on any activities and just deposited in a bank account (World Bank 2006b). This may have been due to the fact that US$1,200 is a small amount given the schools' needs. (A block of classrooms costs about four times that amount.) However, well-to-do schools may have had little need for it, while poor schools may have been unable to spend it. About 7 percent of the communities lent the project grant money at high 39 Annex Table B-11, rates and conditions were unknown (World Bank 2006b, Community Managed Schools in Nepal, final version 2007). By contrast, in 2004, 55% of schools surveyed by CSNN (2005) used the money for construction and only 11% deposited it in the Bank. But also New Era (2005; annex 5.1, p. 121) shows that communities contributed only about 1.5% of expenses, and it is described as negligible (p. 43). 16 Part (d): Enhancing accountability 3.20 The wording of the PAD and the DCA does not precisely specify toward whom accountability would be enhanced and what evidence would be pr~vided.~' Project documents suggest that management committees would be accountable to other school committees through social audits (para. 2.1). MoES Flash reports showed that social audits were conducted in about 70 percent of the transferred schools, but they often did not function as expected (para. 2.13). One reason may be the poor-quality questionnaire disseminated to schools by the government, which also required high-level literacy41. Furthermore, the four-committee structure envisaged in the PAD for schools was too complex to be set up. As a result, evidence of enhanced accountability was very limited. Furthermore, the fact that this complex structure could not be generalized does not bode well for accountability arrangements in the long term. 3.21 Overall the tests intended by the alternative objective are inconclusive on participation rates, quality, efficiency, and accountability. The evaluation framework necessary to discover whether the formally transferred schools (with and without intensive support) performed better than other community-managed schools was not proposed at the design stage or subsequently incorporated into the project. The studies raise the concern that a considerable amount of money was spent on monitoring and evaluation activities without creating interpretable experimental designs and obtaining interpretable findings. An impact evaluation financed by the Education for All-Fast Track Initiative (EFA-FTI), Education Program Development Fund (EPDF), and the Bank- Netherlands Partnership Program (BNPP) is underway and may be able to answer some of these questions in 20 10 (see Annex C for details.) Stakeholder Concerns on the Transfer of Schools to Community Management 3.22 The IEG mission interviewed 26 persons (teachers, parents, academics, MoES staff and consultants, journalists, NGO and donor staff) regarding the effects of the policy of transferring school management to communities. Everyone agreed that the general concept of community-managed schools has potential benefits, but there was much skepticism about the specific transfer concept. Interviewees made references to articles, cartoons, and even a television program that questioned this concept.42The main issues raised were: unclear portrayal of the formal transfer concept as community management of schools, given that all schools in Nepal are technically community-managed. 40 See Reinikka and Smith 2004. 41 CERID (2007b) found a disconnect between authority and accountability at the school levels. Accountability remained at a moral rather than at a legal level, and the report advocated that the law should be amended to strengthen legal means for accountability. The report also found that community mobilization efforts had limited significance. There are similar reports that NGOs do not provide assurance of quality. Staff and management are often, in effect, contractors, rather than nonprofit organizations per se, and social mobilizers may themselves be disempowered (Hobley and Paudyal2008 42 Sagarmatha TV programs September 26 and October 4, 2008, Shah 2008; K. Shrestha 2003. 17 lack of clarity regarding the duties of the transfer; a lack of provisions for handing schools back to the government if future committees refuse the responsibility or stop functioning; lack of a venue for appealing committee decisions. the political role of the committee chairmanship and the importance of this post as a stepping stone into politics; risks of arbitrary teacher dismissals in favor of teachers preferred by committee members, or demand of rent^."^ 0 anxiety by poorer communities about managing the schools and concerns about funding once the US$1200 are spent. Resource persons and head teachers reported to the BPEP I1 evaluation mission that communities with limited education found it hard to understand what must be done and hesitated to take up school management. Many viewed the transfer as an additional financial burden and risk to the community (Also see Parajuli 2007). Limited communications from district offices contributed to m i ~ ~ ~ n ~ e p t i o n ~ . ~ ~ Allegations of imposition of illegal school fees. 4. Ratings Outcome 4.1 Overall, project outcome is rated unsatisfactory. Relevance is rated modest (average of substantial relevance of objectives and negligible relevance of design; para. 2.1). Efficacy is rated modest. The project was not designed to develop a knowledge base for community-managed schools in general or transferred schools in particular. Although the project exceeded its targets of transferring schools to community management, it did not test whether and under what circumstances the formal transfer of schools improved or detracted from enrollment, efficiency, quality, or accountability, as stated in the objectives. Neither the project documents nor the ICR provided sufficient information to assess whether the management transfer was more efficient than regular community management or how well the project funds were used. The project spent $5 million to learn whether the formal transfer of schools to community management raised schooling outcomes and accountability relative to schools that were not formally transferred, but it did not finance an evaluation that could answer the question. Instead, many other studies were financed, most not comparable, that could not address the project objective, and resources were used to transfer more schools than was planned, even though the efficacy of the intervention had not been demonstrated. Therefore, overall, efficiency is rated negligible. 4.2 Formal transfers were scaled up as a policy during the CSSP and in the follow-on Education for All (EFA) project without further evidence of their effectiveness or equity 43 There have been newspaper reports suggesting demands for payments or removals for political reasons. For example, in the Kaski district the chairman of the Maoist party was named head teacher (Kantipur September 30, 2008). However, according to several sources, no teachers are known to have been fired or transferred for incompetence. 44 DANIDA 2004, vol2, p. 13; footnote of para. 2.12. 18 impact. Incentive grants accounted for US$1 million during CSSP, US$7 million during the EFA project, and US$25 million during the EFA additional financing. The follow-on School.Sector Reform Project aims to transfer all schools to formal community management by 2014 (World Bank 2009b). The project PAD refers to the formal community transfer as a radical reform that was independently Risk to Development Outcome 4.3 Risk to development outcome is rated significant. Popular resistance and the governance problems found during implementation do not bode well for a long-term strategy of transferring school management to communities. The transferred schools supported by the project were to be managed by committees whose members' tenure would be limited to three years. Inevitably other community members may succeed them, whose views and dedication are not ensured.46 It is unclear how closely and who will monitor committee actions and ensure good governance and fair treatment towards teachers and students. The risk to development outcome goes beyond the purpose of the LIL, since formal transfer of school management to communities is supported by the government and is the cornerstone of Bank strategy for improving school quality and access. The failure to evaluate the impact of formal transfers to community management on accountability and educational outcomes before scaling up the intervention could adversely affect country strategy. Bank Performance 4.4 Overall, Bank performance is rated unsatisfactory. The quality at entry of the project was highly unsatisfactory because there was no evaluation design for the project or risk assessment with respect to the insurgency or the teachers' unions, objectives were poorly articulated, and systemic obstacles had not been identified. The envisaged four- committee implementation structure did not exist during appraisal. The Bank also disregarded multiple worrisome lessons regarding the ability of poor communities to implement complex schemes (e.g. World Bank 2002, CSNN 2005). While the project proposed a new strategy - the formal transfer of schools to community management - it was not based on prior analytic work that documented the problems and outcomes of the preceding strategy. This led critics to suggest that the Bank had a privatization agenda and was downplaying the primary education achievements of the last two decades (para. 4.6). 4.5 Supervision is rated moderately unsatisfactory. Teams of Bank staff and consultants regularly visited schools during supervision missions. A parents' organization (the Community Schools National Network) was contracted to visit schools and collect data to be used for supervision missions, but some project documents express concerns regarding the reliability of these data. Survey findings of poor attendance in the 45 World Bank 2009b, p. 52 states "Recently completed EFA evaluation report has independently verified that the transfer of schools to community management has been a successful initiative and has already led to positive outcomes." The evaluation team of NORAD 2009, however, reported to the IEG mission that they had relied on CSSP documents and consultants. 46 One implication is a constant need for training of new committees. Expenditures can be large, as Bank projects from Tanzania and Honduras have shown (World Bank 2008 b, c ) , 19 2005 CSNN report were disregarded. Aides-memoire focused on interactions with specific schools that agreed to be transferred and paid little attention to evaluation deficiencies. When it became obvious that all schools in Nepal would operate essentially under the same rules, the project design was not revised to assess the specific effects of the transfer incentives. Also, the Bank seemed unable to deal with the auditing and fiduciary aspects of the project. It did not deal with the conflicts of interest involving contracts for the Community Schools National Network that also received grant funds (para. 2.14-2.15). Some informants expressed concerns to the IEG mission that there was no dialogue with the education community on these perception^,^' that Bank documents . made unsupported allusions regarding the effectiveness of the government's strategy preceding CSSP.48 4.6 Donor and government staff interviewed by the IEG mission stated satisfaction and appreciation with Bank performance in terms of donor coordination, prompt action, and support for government programs. Some donors, however, expressed a concern that the Bank brings to Nepal limited technical knowledge on education issues as well as a private-sector bias and that it did not sufficiently consult with stakeholders. Information on the project was not adequately shared with donors, and they were not consulted when the funding guidelines were developed. Furthermore, unclear portrayal of the concept during the CSSP has confused staff from other donor agencies, creating in many the erroneous impression that there are two types of schools -- regular and community- the managed. 49 Some respondents raised concerns that by financing this LILY Bank failed to study alternatives, such as strengthening management in all communities, particularly the poorest. Borrower Performance 4.7 Borrower performance is rated moderately satisfactory. The government was supportive through policies and legislation, and government performance is rated satisfactory. The implementing agency performance is rated moderately satisfactory. For much of the implementation period the project was administered by the Department of Education without specially assigned staff, and administrative arrangements fell short of the scheme envisaged in the PAD. Audits and financial monitoring reports were often late. Given the project design faults, it is unclear to what extent the government intended to learn more about the capacities of communities to manage schools from the operation, as opposed to the ability of schools to absorb cash grants. 47 This concern is also expressed in a a peer-reviewed publication which stated: "It appears that the World Bank is engaged in a concerted effort to re-write Nepal's recent educational history, eulogizing the earlier period of locally owned (but small-scale) `community' schools and portraying the State as a major impediment to change" (Carney et al. 2006). 48 Examples cited were articles and reports such as G. Shrestha (2005). According to the latter report, the transfer strategy aimed to mitigate a deteriorating performance of public primary schools in spite of heavy government investment. Also, the 2007 Public Sector Expenditure Review stated that education has deteriorated over time (p. 27- 28). However, no data are provided on the performance of the 4000 schools that existed in 1971, and it is difficult to assess the validity of comment in the absence of evidence. Overall achievement trends have been positive in comparable tests (EDSC and CERID, Annex Table B-2). 49 Teachers' unions were not consulted in the development of the Operation Manual for Community-Managed schools (Vaux et al. 2006, p. 25). Adhikari (2007) expresses the erroneous belief that the Community Schools National Network serves all schools and is a volunteer network. The Joint Evaluation of the EFA project also incorrectly presented the role of public sector and community management (NORAD 2009, p. xvi, 50). 20 Monitoring and Evaluation Design, Implementation, and Utilization 4.8 The monitoring and evaluation design is rated negligible. The project required a carefully tailored research design to establish cause-effect relationships. The PAD outlined the need for some treatment and control groups, 50 but the design did not take into account the systematic differences between communities choosing to transfer and receive additional grants. There was no effort to use designs that would statistically control selection decisions in the transfer process (e.g. pair-wise matching or propensity score matching). Any differences in outcomes between these groups could be due to the factors leading them to participate, rather than to the interventions. 4.9 Monitoring and evaluation implementation is rated modest. As prescribed in the PAD, pilot baseline data were collected but two different datasets were abandoned. The first (New Era 2005) used purposive sampling, but the next selected schools at random. 5 1 Neither the mid-term review documents (May-June 2005) nor supervision documents in early 2006 (6 months before the original closing date), mention any concerns about the inadequacy of the pre-post survey evaluation strategy for measuring the impact of the intervention^.^^ Aside from the baseline and end line surveys, multiple other studies were conducted through the PHRD grant and the project itself, but most were qualitative narratives, with information mainly obtained through self-reports and one-time surveys.53 There was little longitudinal follow-up other than self-reports, statistical analyses were few, and the studies were not linked or coordinated into an evaluation design. They used purposive samples of 10-100 schools chosen on the basis of unclear criteria. 4.10 Finally, the introduction of new achievement tests made it impossible to compare achievement with that of earlier cohorts and other sample^.'^ Survey validity and reliability were also issues of concern; several survey questions focused on accounting issues, but few inquired about the educational level of committee members, involvement level in decisions-making, amount of time available, and distance from school. Some questionnaires (notably the 2005 CSNN-managed study) asked questions regarding community activities that only permitted positive answers. Crucial details were missing, "For the purpose of the outcome-level testing, a baseline will be established in a sample of government-funded - community-managed schools treatment group, government-funded schools transferred to community but not receiving project support except for the incentive grant - partial treatment group, and government-funded schools managed by the Government-control group. The achievement of these groups of schools at the end of the project will be compared with the baseline to find out whether community-management leads to better outcomes." (PAD p. 23). 5 1 The New Era 2005 survey surveyed 80 purposefully schools classified as `incentive', `performance', and `control." It took place shortly after a CSNN survey of 95 schools. 52 In March-April 2008, the CSSP endline survey was conducted for 300 schools and their teachers, children in the catchment areas, and SMCs, for which new achievement tests were developed for Nepali language and math in grades 1,3, and 5. This survey had a completely different design than the baseline, with four types of schools in each of 15 districts - community aided, community-managed, community-managed unaided, and institutional (Full Bright 2008). As mentioned earlier, the learning assessments were not comparable with any previous learning assessments in Nepal. 53 For example, the Community Schools National Network (CSNN) did a survey of 95 purposively sampled transferred schools (CSNN 2005, April 17), while the Center for Policy research and Consultancy (CPReC 2008, p. 2, volume 2) studied another 100 schools that chose community management voluntarily, after advocacy, after rigorous advocacy, or rejected even after advocacy. However, it is mainly an advocacy piece itself and offers limited data. 54 New Era 2008 and Full Bright 2008. Both documents refer to a `TIMSS type test' without clarifying in what way that test was different from earlier tests conducted in Nepal. New Era (2008) analyzed data of the 220 baseline schools in terms of percentiles rather than by using the TIMSS scale. The report also stated that some of the students taking this test did not live in the sampled school catchments areas. 21 such as the means whereby likely children's guardians were identified, or what the social mobilization included. No survey questions were found on school fees and on whether the poor could afford them. Some surveys had questions with unclear definitions, such as "lower socioeconomic condition of school". 4.1 1 The 2005 PHRD grant financed a number of studies. The grant application stated that the government's monitoring systems were weak and requested funds to install a new monitoring and evaluation system for 1,500 schools, although the MoES flash reports were working reasonably well. It also stated incorrectly that Nepal had no achievement tests; it proposed to develop them and administer three rounds of tests, but only two were conducted. Overall, about US$633,206 from the trust fund was spent for monitoring and evaluation functions, but only a fraction of the planned activities took place. 4.12 The use of data to guide policy is rated negligible. Project documents and reports to external bodies show a focus on the number of schools that were transferred, and the For many reports generated did not seem to influence ~trategy.'~ example, one study found that the vast majority of regular schools also had functional committees, but the finding did not instigate a change in strategies or research design. The Bank did not issue a synopsis of the information obtained through all the studies by the end of the project, and dissemination promised in the ICR in the first half of 2009 was not carried out. There is little evidence that test results were taken into account for quality improvement purposes. Aside from a May 2008 internal World Bank workshop, data on students' and teachers' test scores were not discussed or presented extensively. The outcomes and evaluation design issues were also not discussed in the PAD of the follow-on operation, the Schools Sector Reform Project (World Bank 2009b). 4.13 Because of major deficiencies in design implementation and use of findings, the monitoring and evaluation of the project is rated negligible. 5. Issues: Sectoral Strategy and Community Management 5.1 The aim of the 2003 Country Assistance Strategy (CAS) was to improve education quality and learning outcomes. The 2002 CAS progress report included a "pulse-taking" exercise which showed considerable concerns regarding the ability of poor communities to deal with complex implementation issues.56 Nevertheless, the CSSP was appraised at about the same time without any special concerns for the poor. The 2003 CAS did not attempt to strengthen all community organizations through some means known from the vast community-driven development literature. Instead it focused only " See for example Shrestha and Joshi 2007, report to a Quality of Education Conference in New Delhi. "User groups-which have proliferated-tend to be more participatory and successful in the delivery of services. They are by and large still dominated by local elites and, hence not inclusive (of the poorest)." "NGOs.. . use much of the money themselves and do not reach the poorest of the poor (except for some dalit NGOs). " `The poorest of the poor are excluded. As a local official said "they (the poor) are not coming to us so we cannot help them and we have our hands full.' "(Appendix 11, p. 3). 22 The on the willing comm~nities.~? official transfer to community management became a development objective in its own right5*and the cornerstone of the Bank?s education strategy. Without obtaining further evidence, the ICR of the first PRSC (World Bank 2006a, p. 12) stated that the handover pace had exceeded all previous expectations and was interpreted as ?symptomatic of the demand that existed for this reform.? 5.2 Some informants interviewed by IEG raised a concern that the selective transfer of schools to communities may create equity issues. Communities have social stratifications and informal relations that determine who gets benefits (Hobley and Paudyal2008). When committee members determine how to use resources, local elites may benefit disproportionately. The poorest may live too far or lack the needed time to attend meetings, and may not even know what their rights are (Carney at al. 2006).? Certainly unleashing the power of communities who can safeguard their own interests is useful, but efforts to do so should build on the knowledge base established through research and evaluation from similar experiences elsewhere. Research from Latin America to Indonesia on the effectiveness of community management for promoting education quality suggests that community participation does not automatically lead to improvements in education quality and learning outcomes.60 These and other issues should have been explored during the preparation and implementation of the CSSP. 5.3 Ultimately, it may be difficult to generalize the model of community transfer. If all schools received the grant amounts given to transferred schools, public expenditures for education might become unsustainable. As mentioned in Chapter 1, merely increasing the expenditures per student may improve learning outcomes irrespective of school management. In a budget constrained environment, targeting low performing schools in the poorest area might be called for (also see conclusions of CERID 2007a in para 1.9). Currently resources are spent to encourage communities to take over, without explicitly targeting lower performing schools in poorer areas. 5.4 Research shows that people?s prior experiences determine their expectations, so the poorest may be satisfied with low-quality education61. To help poor communities 57 In some respects, the ?transfer?program worked like a school-level voucher system. Students? vouchers have produced social stratification (e.g. Chile) but not much better learning outcomes. (E.g. Perry and Leipziger 1999; Mizala and Romaguera 1998, Carnoy and McEwan 1997 are included in Perry and Leipziger 1999). 5 8 ?This process at the primary level-a fundamental shift from the currently centrally-managed system-has begun over the past year with over 250 schools transferred by end-September 2003 and an additional 500 applications pending a proval.? (World Bank 2003, p. 13). 59 Sowton, circa 2004. The author raises concerns about decision-makers being from influential families and deciding on educational matters according to their socioeconomic experiences. Studies such as Sibieta et al. 2008 also suggest that a successful funding system for schools needs to encourage equity, targeting resources where they are most required, and be reasonably comprehensible to all stakeholders. 6o Banerjee et al. 2008, found that interventions to encourage beneficiaries? participation, such as providing information, training community members in a new testing tool, had no impact on community involvement in public schools, and no impact on teacher effort or learning outcomes in those schools. Gunnarsson et al. 2004, in LAC study found no impact of local control on learning. An Indonesian study found limited community management capacity (SGP 2002). A study showed that Honduran rural schools were open for 154 days a year, but PROHECO schools operated for 180 days out of the 200 official days. However, the study also found little difference in learning outcomes. (Programa Hondurefio de Educacidn Comunitaria, Honduran Program of Community Education; PROHECO ICR, World Bank 2008). See Abadzi 2009 for a review 23 effectively manage schools, such communities might benefit from training on what to expect and monitor. Means to raise citizens' interest and expectations in schools could include radio broadcasts on school activities and committee decisions, and instructions to parents on how well their children should be reading or calculating. Nepal can benefit from the research that already exists on the opportunities and challenges related to community based school management, including its equity implications. 6 . Lessons 6.1 This assessment provides a number of lessons for the education sector: Community management of schools offers many potential benefits, but it is important to understand what communities of different educational and income levels are likely to implement. The education of the poorest children should not depend heavily on the means available to the very poor communities. Before scaling up pilot activities, it is important to evaluate those pilots using robust evaluation designs. In particular, LILs should strive to create best practice in evaluation in order to maximize learning. 0 Major educational reforms should be based on solid analytical work and take into account relevant research and evaluation from similar experiences elsewhere. There is little evidence that the design of the CSSP drew on research and evaluation findings of experiences with community management of schools, particularly as it relates to learning quality and outcomes, further compounding the lack of robust evaluation carried out under the LIL itself. 25 References Abadzi, H. 2009. "Community management as a development tool in the social sectors: Why more should be known before its use is scaled up." Paper submitted for publication. Abadzi, H. 2006. Eficient Learning for the Poor. Washington DC: World Bank. Abadzi, H. 2003. Adult Literacy Outcomes. Washington, DC. World Bank Acharya, Sushan and Bal Chandra Luitel. 2006. "Functioning and Effectiveness of Scholarship and Incentive Schemes". UNESCO, Kathmandu. Series of Monographs and Working Papers no. 9. Adhikari, Prakash Singh. 2007. "Analysis: Danish Aid for Education and Civil Society Involvement in Nepal." Unpublished report submitted to DANIDA. Ammenwerth, E., C. Iller, and U. Mansmann. 2003. "Can evaluation studies benefit from triangulation? A case study." International Journal of Medical Informatics 70(2- 3):237-48. Aryal, Mohan Prasad. 2005. "Higher and higher secondary education in Nepal - Institutions, Enrollment, Financing and Performances." Final report, June. Ministry of Education, Kathmandu. Asian Development Bank. 2004. "Nepal Education Sector Policy and Strategy." ADB TA 4326. September. Asian Development Bank. 2008. "Community Driven Development. Nepal: Country Profile." Kathmandu. Bayne, Sarah. 2005. "Background paper and analytical framework for a conflict assessment of the Education for All Programme in Nepal". International Alert. Banerjee, Abhijit V., Rukmini Banerji, Esther Duflo, Rachel Glennerster, and Stuti Khemani. 2008. "Pitfalls of participatory programs: Evidence from a randomized evaluation in education in India." World Bank Policy Research Working Paper, no. 4584. World Bank, Washington, D.C. Bista, Min Bahadur. 2006. "Status of Female Teachers in Nepal." UNESCO. Bista, Min Bahadur, and Stephen Carney. 2004, 2006. "Reforming the Ministry to Improve Education: An Institutional Analysis of the Ministry of Education and Sports (MoES) of Nepal" ED499633, International Institute for Educational Planning (IIEP), UNESCO. Carney, S., M. B. Bista, and J. Agergaard. 2006. `"Empowering' the `Local' through Education? Exploring Community-Managed Schooling in Nepal". Oxford Review of Education, 33(5): 61 1-628. Carney, S., and M. B. Bista. 2009. "Community Schooling in Nepal: A Genealogy of Education Reform since 1990." Comparative Education Review 53(2): 189-211. CERID. 2006. "Education for All Formative Research Report. Synthesis Report of the case studies conducted in 2006." Tribhuvan University. Research Centre for Educational Innovation and Development (CERID). 26 CERID. 2007a. "Education for All Formative Research Report. Longitudinal Study on System Indicators: Cohort analysis." Tribhuvan University. Research Centre for Educational Innovation and development (CERID). CERID 2007b. "Understanding School Autonomy: A Study on Enabling Conditions for School Effectiveness." Education for All 2009-20 14 Formative Research Project, Report no. 14. Kathmandu, Tribhuvan University. Research Centre for Educational Innovation and development (CERID). Community Learning Center (CLC). 2005. "Educational Needs of Internally Displaced Persons living in Western Kathmandu." Survey Report prepared for UNICEF. CPReC. 2008. "Social Mobilization for participation in the Community Managed School Project." (PHRD-Package No. 2) GRANT # TF 055 146. Implementation Completion Report, Centre for Policy Research and Consultancy, Vols I and I1 (CPReC) Kathmandu. CSNN. 2005. "Community Managed Schools (CMSs) Nepal Survey Report - 2005." Community Schools National Network. Kathmandu CSNN. 2007. "Community Managed Schools in Nepal", Kathmandu. Chaudhury, N. and D. Parajuli. 2009. "Giving it Back: Devolution of School Management to Communities". Initial findings from an impact evaluation in Nepal. Unpublished report. DANIDA. 2004/3. "Nepal: Evaluation - Joint Government-Donor Evaluation of Basic and Primary Education Programme 11." Ministry of Foreign Affairs. EDSC. 2003. "National Assessment of Grade 5 Students." Educational and Developmental Service Centre for the Ministry of Education and Sports. EDSC. 2008. "National Assessment of Grade 8 Students." Educational and Developmental Service Center of the Ministry of Education and Sports. FHD and RIDA. 2009. "Improving local service delivery for MDGs in Asia: Country case study." Foundation for Human Development (FHD) and Research Inputs and Development Action (RIDA). Gunnarsson, Louise Victoria, Peter Orazem, Mario Sanchez, and Aimee Verdisco. 2004. "Does Local School Control Raise Student Outcomes?: Theory and Evidence on the Roles of School Autonomy and Community Participation." World Bunk. Working Paper no. 04005. World Bank, Washington, D.C. Gurung, H. 2002. "Janagadana 2001Anusar Jatiya Tyathanka, Prarambhik Lekhajokha" (Based on 200 1 Census, Preliminary Investigation on the Number of Caste/Ethnic Groups). Kathmandu: Dharmodaya Sabha. Hobley, Mary, and Bimala Ray Paudyal. 2008. "Study of Income Generating Activities in Kapilavastu and Ramechhap Districts, Nepal" (draft report). Independent Evaluation Group (IEG). 2009. "Project Performance Assessment Report: Nepal: Basic and Primary Education Project 11." World Bank, Washington, D.C. 27 IEG. 200 1."Nepal: Hill Community Forestry Project (Credit 2028-NEP), Second Forestry Project (Credit 1400-NEP)." Project Performance Audit Reports. Washington, DC: World Bank. Report No 22438 (IEG was formerly known as OED). IEG. 2008. Decentralization in Client Countries. An Evaluation o World Bank Support, f 1990-2007. IEG. 2004. Books, buildings, and learning outcomes: An impact evaluation o World f Bank support to basic education in Ghana. Washington, D.C.: World Bank. Johnson, K. and S.E.K. Bradley. 2008. "Trends in Economic Differentials in Population and Health Outcomes: Further Analysis of the 2006 Nepal Demographic and Health Survey." Calverton, Maryland, USA: Macro International Inc. Kantipur 14 Asoj, 2065 (September 30,2008). Anerasvaviyu Krantikari Jhadap (Maoist Nepal students' association fight). Khaniya, T. R. 2004. "Transfer of School Management to Community." Ministry of Education. Khaniya, Tirth Raj, Mana Prasad Wagley, Pramila Rajbhandari, Dr Bijaya Kumar Thapa, Shekhar Dhungana. 1999. "National Assessment of Grade 5 Students." Educational and Developmental Service Centre (EDSC). Khaniya, Tirth Raj. 2007. "New Horizons in Education in Nepal." Office Support Service Center, Kathmandu. Khaniya, Tirth Raj. 2008, October 23. "Reform in Higher Education. Key stakeholders should be kept informed about the World Bank-funded Higher Education Project." Kathmandu Post. King, Elizabeth M., Peter F. Orazem, and Elizabeth M. Paterno. 2008. "Promotion with and without Learning: Effects on Student Enrollment and Dropout Behavior." World Bank Policy Research Working Paper Series, no. 4722. World Bank, Washington, D.C. Latane, B., K. Williams, and S. Harkins.1979. "Many Hands Make Light The Work: The Causes and Consequences of Social Loafing". Journal o Personality and Social f psycho lo^ 37: 822-832. Lockheed, M., and A. Harris. 2005. "Beneath Education Production Functions: The Case of Primary Education in Jamaica." Peabody Journal o Education 80(1): 6-28. f Mansuri, G. and V. Rao. 2004. "Community Based (and Driven) Development: A Critical Review." World Bank Policy Research Working Paper, no. 3209. World Bank, Washington, D.C. Mathieu, Alexandra. 2006. "Reaching Girls in South Asia: Differentiated Needs and Responses in Emergencies." UNGEI, March. Mikhail Samih. 2008. "The Alternative Tertiary Education Sector: More than non- University Education." Education Working Paper Series, no. 10. World Bank, Washington, D.C. MoES. 2007a. "Education for All: Mid-Decade Assessment." Kathmandu. MoES. 2007b. "Annual Strategic Implementation Plan (ASIP) 2007-2008" 28 MoES. 2004. "Nepal Education Sector Development Policy and Strategy. Interim Report 11." September-December. MoES. 2005. "School Level Educational Statistics of Nepal." Consolidated Report 2005 (2062) and 2006 (Flash Reports) MoES. 2006. "Household Survey of Service Areas of Community Managed Schools." Department of Education. Nepal, Kishore. 2005. "Under the Shadow of Violence." Kathmandu: CJPS. New Era. 2008. "Baseline and Follow-up Surveys for Nepal Community Managed Schools Impact Evaluation." Kathmandu. New Era. 2005. "Baseline Survey Of Selected Community Managed And Government Managed Public Schools In Selected Districts." P.O. Box 722 Rudramati Marga, Kalopool, Kathmandu (June). Nielson, K.A., and T. Bryant. 2000. "Memory Modulation by Intrinsic and Extrinsic Reward." Neurobiology of Learning and Memory 84: 42-48. NORAD 2009. "Joint Evaluation of Nepal's Education for All 2004-2009 Sector Programme." Evaluation Report., January. Parajuli, Mahesh Nath. 2007. "People's participation in school governance? Realities of educational decentralization in Nepal." In Holger Daun (Ed.) School Decentralization in the Context of Globalizing Governance. New York: Springer (pp. 195-211). Perry, G., and D. Leipziger (Eds). 1999. "Chile: Recent Policy Lessons and Emerging Challenges." World Bank Institute, Washington, DC. Reinikka, Ritva, and Nathanael Smith. 2004. "Public expenditure tracking surveys in education." UNESCO-IIEP. Rivarola, M., and B. Fuller. 1999. "Nicaragua's experiment to decentralise schools: Contrasting views of parents, teachers and directors", Comparative Education Review 43(4): 489-521. Sagarmatha TV. 2008. "Samudayak Vidyalaya" (Community School). Aired in Kathmandu on September 26 and October 4. Shrestha, Gajendra Man. 2005. "Community schools in the Process of Transformation." Community School Support Project, Kathmandu, May. Shrestha, Gajendra Man, Ganga Ram Gautam, and Saloni Singh. 2002. "SIP - Based Development Programme: A status statement." Paper submitted to the EC Education Coordination Office and World Bank/ Nepal. Kathmandu. Shrestha, Gajendra Man, and Rajendra Dhoj Joshi. 2007. "Community Managed Schools in Nepal: Glimpses of Transformation." Paper presented at a World Bank Quality of Education conference, New Delhi October 24-26 (siteresources.worldbank.org/INTINDIA/Resources/Shrestha.ppt ) Shrestha, Kedar N. 2003. "Transfer of public school to community." Kathmandu Post, November 14. 29 Shah, Prakash. 2008. ?Adha talab matra paunchan rahat shikshaharu (Relief teachers receive only half the salary).? Shikshak, October. SIGP. 2002. ?Community participation in the scholarship and grants program (sgp) and school improvement grants program.? Jakarta: Independent Monitoring and Evaluation of the Scholarships and Grants Program and the School Improvement Grants Program. Sibieta, Luke, Haroon Chowdry, and Alastair Muriel. 2008. ?Level playing field? The implications of school funding.? CfBT Trust, UK (www.uquebec.ca/observno/fichiers/743 17 2.pdf ) Siprek Kendra. 2007 (2064). Samudayak Vidyalaya Vyavasthapan Sachetna. Hate Pustaka 101 (Community School management Advocacy: Handbook 101). Kanthmandu, Nepal. Sowton, C. R. (undated, circa 2004) ?Educational Elitism to ?Education For All?: An Analysis and Evaluation of the Nepalese Education System.? Sultana, Ronald. 2006. ?Facing the hidden dropout challenge in Albania.? Tirana: UNICEF. TRSE. 2003-2006. ?Technical Review of School Education?. Asian Development Bank. UNESCO. 2006. ?A 10-year year literacy/NFE Policy and Programme Framework.? UNESCO. Kathmandu Office. Jawalkhel, Lalitpur Kathmandu 977-1-5554396 / 769. UNESCO. 2006. ?Activities in Nepal. Biennium Report. 2004-2005 .?? Education Culture Communication and Information Sciences. Upadhyay, Uttam Prasad. 2008. ?Social mobilization for participation in the community managed project? (grant TF055 146). Implementation completion report. Centre for Policy Research and Consultancy, Kathmandu, June. Vaux, Tony, Alan Smith, and Sirjana Subba. 2006. ?Nepal Education for All: Review from a Conflict Perspective.? International Alert, Embassy of Finland. World Bank. 1992. ?Nepal: Primary Education Project 111.? Project Completion Report. World Bank. 2001. ?Nepal: Priorities and Strategies for Education Reform.? Report no. 22065-NEP. Washington, D.C, World Bank. 2002. ?Country assistance strategy progress report.? Report No. 24 170- NEP World Bank. 2003. ?A Country Assistance Strategy for the Kingdom of Nepal.? Report No. 26509-NEP. Washington, D.C. World Bank. 2006a. ?First Poverty Reduction Support Credit.? Implementation Completion Report. Report No. 35462 World Bank. 2006b. ?A Follow-up Study of ISP Schools?. (draft report). World Bank, 2007a. ?Managing public finances for a new Nepal.? Report No. 43384-NP 30 World Bank, 2007b. "Nepal: Project Paper on a Proposed Additional Financing (Grant) in the Amount of US$60 million equivalent for the Education for All Project." Washington DC, no. 41 346-NP. World Bank. 2007c "Attaining the Health and Education Millennium Development Goals in Nepal South Asia: Human Development Sector", Report No 12. World Bank. 2007d. "Community Managed Schools in Nepal: A trend in school reform." Final report. December. World Bank. 2008a. "South Asia Region: Ending Poverty in a Region of Growing Contrasts." Regional Briefing, South Asia Vice-presidency, SecM2008-0232 May 29. World Bank. 2008b. "Honduras: Community-Based Education Project." Implementation Completion and Results Report. Washington, D.C. World Bank. 2008c. "Tanzania: Secondary Education Development Program." Implementation Completion and Results Report, Report No: ICR0000744. Washington, D.C. World Bank. 2009a. "Nepal: Community School Support Project", Implementation Completion and Results Report. No: ICROOOO 1025, March 23. World Bank. 2009b. "Nepal. School Sector Reform Program: Project Appraisal Document", August 26, Report No: 50157-NP. World Food Programme. 2006. "Small Area Estimation of Poverty, Caloric Intake, and Malnutrition in Nepal." [reports on CBS household survey], National Planning Commission Secretariat, Bureau of Statistics. Yadav, Ram Prakash. 2008. "Relation between Literacy and Poverty." Poverty Outlook. PAF , 31 Annex A Annex A. Implementation of project components Table A-1: Community School Support Project (Cr. 3808) Outcomes Components/ Activities Targets to be achieved outputs Info obtained during subcomponents mission Grants for transferring 1SO0 schools About 2035 of about 23.000 8000 schools were expected schools to community schools (according to ICR). to be transferred to management (about The project funded transfer of communities by the end of US$1200). 1,746, rest funded by 2007 Education for All Project Supplementary grants 250 335 Matching grants for libraries, (fully supported schools) computers, science labs introduced at midterm review (average NRS 82,000). Salary grants introduced, schools were required to monitor attendance of students and teachers and to recruit teachers competitively through a public ad Out-of-school children's 7619 17250 Students need to catch up, but scholarships coaching classes were not found Maintenance scholarships 1972 1 42640 The scholarship component was apparently successful in the primary schools, but less so in lower secondary and secondary schools. Training NGOs, support 300 About 146 NGO staff trained Only a minority of the NGOs organizations in 6-day orientation programs proved useful; a number were in fact private providers of services responding to demand Customized teacher 1000 Teacher support grants to It is uncertain what the training for instruction, 1047 schools, 45% customization options community relations achievement of the target included and how effective (Household community the training was survey) Training civil servants 1000 Unsure if target was achieved, Impact unknown, district and politicians several workshops done education officers not very knowledgeable about the program according to project files Communication of About 146 NGOs hired, NGOs may be best for policies to communities maybe 50-60 doing mobilization of enrollments, acceptable work not really experienced in community school management Orienting school 1300 Total number unknown In many cases only chairman community members Final US$200,000 of grant and head teacher were trained spent on dialog with Three-year terms mean that communities, that according there must be a constant to the task manager consisted training program for new of training committees Peer to peer networks and 81% of respondents found study visits to efficiently visits effective managed schools in Nepal 78% subsequently prepared The networking activities action plans were conducted in 17 districts involving 207 schools and 458 participants Staff supported for 10 -Instead a study tour was Assignments did not happen, developmental carried out in Latin America a the Ministry w s not assignments interested in sending staff to work in schools 32 Annex A Outcomes Components/ Activities Info obtained during subcomponents Targets to be achieved outputs mission Monitoring and Monitoring tasks were US$699,600 PHRD About US$633,206.18 ofthe Impact evaluation not evaluation financed by a PHRD grant TF055146 grant disbursed for such tasks completed by project end due grant to poor quality of baseline data Provision of goods and 9 computers for the There was to be participatory consultants' services to Department of Education monitoring of changes in the facilitate transfer of faxes, computers in schools school, but most of the management of schools did not maintain the government-funded required information as community schools to demanded by survey communities instruments (New Era 2008). Develop, by June 30, Pension scheme became According to the task team, No effect 2004, a policy Redundant after communities had no interest framework, acceptable tot communities were in providing pensions to their the Bank to establish a allowed to hire teachers. pension scheme for permanent teachers teachers of community- managed schools funded, in part, by teachers' contributions Adopt, by September 30, If teachers did not Legislation passed after the No effect 2003, procedures, perform, the community expected deadline, but has not acceptable in substance to could request their been implemented due to Removal on transfers of the Association, for transfer. teacher unions' concerns teachers for incompetence is redeploying MOES- about local biases virtually unknown, despite appointed teachers from IEG mission found that few if committee powers to do so community-managed any teachers have been schools based on requests removed for incompetence or Incompetent teachers may from relevant absence still teach or be absent management committees extensively within six months from the date of such request Adopt, by December 31, Formula shall be revised Block grants adopted, but To address inequitable 2004, a regulatory on a bi-annual bases and formula not clearly developed distribution of resources the framework, acceptable in include, inter alia, taking or updated government has introduced substance to the into account the student per capita financing for Association, for formula- enrollment. salaries in addition to the based government- existing per capita financing provided block grants for non-salary costs. funding to eligible schools. Adopt, by October 3 1, Not done The effects of delayed and 2003, procedures, Budgets are sent three times a limited payments on teacher acceptable to the year, always delayed, so performance are unknown Association, for teachers are paid every four releasing grants to months. community-managed Community hired teachers schools on an advance ought to be paid monthly if basis to ensure that communities have sufficient teacher salaries are paid resources in a timely manner Source: Project documents and information obtained during the PPAR mission f4 3 E 4 ?? s 0 4 .- Y cd rn (u 8 00 vi m m 2 m m m 2 1 0 m -. 2 (u m z P v, m (u vi m m 13 z f4 Y m 3 e E 4 F 0 0 cu z 6. z .I p3 Q) .?; -z 0 a a 1 m L Q) -0 8 d Q 3 z El .I z c u , 1 d a M E c, v) v) Q) c, c, 1 Q) 5 + Q c, 8 a 1 c, m m 1 0 .I L * Q 43 0 z L 8 .. m cu Q) zi b E9 pe 2 4 X a2 E 4 0 r, 9 N - 39 9 N N m 0 0 N 2 3 x 9 N N I N 0 0 N 39 z : : N N R R 10 d N m n e N ? N 3 3 I 36 Annex B Annex B Table 5: Datasets with Test Scores Pertinent to Management Transfer Monitoring and Pertinent Design Features Outcomes Evaluation Study New Era 2005 Selected sample of 80 transferred schools Transferred and control schools were Data collected May-October and matched controls fairly similar; schools given 2004 performance grants (and their matched controls) outperformed others in some tests. Full Bright 2008 Cross-sectional survey of 300 primary In English, Nepali, and math, the two Student tests given in March- schools types of schools had differences of 1- April 2008 Contractor won bid for an `impact 2 percentage points, with regular evaluation study' schools often scoring higher. Full Bright 2008 Cross-sectional survey of 300 primary In Nepali, teachers in the two sets of Teacher tests given in March- schools schools had essentially the same April 2008 scores (55.6 vs. 55 percent respectively), whereas in English and math, teachers of regular schools scored better (41.2 vs. 44.8 oercent A and 5 1.3 vs. 54. lpercent) EDSC 2008 Longitudinal monitoring achievement test Schools formally transferred to Tests given in 2008 analyzed data cross-sectionally based on community management scored 1.9-3 community management type. No baseline percentage points over regularly data were compared managed schools Pilot impact evaluation Random sample of 80 non-transferred There is no formal report on the baseline and additional data schools from six districts; 40 were results of the pilot - only a collection randomly "treated" with NGO advocacy preliminary powerpoint presentation Baseline December 2005- and 40 were not. The purpose of the pilot (that does not control for self- February 2006, follow-up tests was to ascertain how many schools were selection of communities) and a in August -November 2007 likely to transfer in communities with single table in a research proposal for Data collection by New Era; advocacy, as input into the design of a the larger study that shows results for results disseminated at a much larger research project. Both this only four variables, but controlling for World Bank workshop in May pilot and the larger project were designed to self-selection into the transferred 2008. use the random assignment of the advocacy group (Chaudhury and Parajuli 2009). as an instrumental variable to obtain The latter shows higher TIMSS unbiased estimates of the impact of the Science Test scores in the 5`h grade, a transfer (eliminating the self-selection bias). higher grade 5 promotion rate, a Over 18 months, 15 of the 40 schools in the higher net enrollment rate, and higher treatment group receiving advocacy elected community participation in daily to formally transfer, compared with 5 of the school affairs in transferred schools. 40 control group schools. Dataset collected However, only the test scores are with Education for All Fast track Initiative statistically significant; since the funds standard errors have not been adjusted to take into account the clustering of the sample, the significance is likely overstated. Results for most of the variables collected (including math, Nepali, and English tests in grades 1, 3, and 5 ) were not shown. Because of the small sample size of the pilot (only 80 schools), most differences would not be expected to achieve statistical significance. That is why the larger study was launched (below). As none of the schools had yet elected to transfer, all of the test scores are for non-transferred schools. Test . scores could not be compared with earlier datasets 37 Annex B Annex B Table 6: Number of reported schools by typ e and level 2006-07 Lower Higher Types of schools Primary secondary Secondary secondary Community aided 20,084 4,3 14 2,627 129 Community managed 1,896 345 143 42 Community unaided 2,239 2,073 893 510 Institutional 3,106 2,147 1,555 194 Total 27,325 8,879 5,2 18 875 Source: MoES Flash report 11; Formally transferred schools are designated as `community-managed' in comparison to others that are `community-aided' School type Nepali English Math Social Science Health Studies Community School 43.8 35.6 30.4 52.3 47.1 46.5 Formally Transferred School 45.1 37.5 33.4 55.0 49.2 49.4 Difference in points 1.3 1.9 3.0 2.8 2.1 2.9 Communities used finds for the following proposes Resources Physical Instructional Loan in School Teacher Land Maintenance improvement improvement high construction salary purchase interest CSSP 25 (17.2%) 76 (52.4%) 10 (6.9%) 18 (12.4%) 11 (7.6%) 2 (1.4%) 3 (2.1%) Grants Local 66 (50.4%) 15 (1 1.5%) 3 (2.3%) 36. (27.5%) 11 (8.4%) Resources 39 Annex C Annex C. An Ongoing Impact Evaluation of the Formal Transfer of School Management in Nepal In late 2005, researchers launched an impact evaluation to compare the effectiveness of primary schools whose management was formally transferred to the community with primary schools whose management had not been formally transferred. The research design took into account the voluntary nature of the formal transfer by randomly assigning a sample of schools that had not yet been transferred to an "intervention" that consisted of an advocacy campaign, providing information to communities and assistance in submitting the application for formal transfer. The schools in the "control" group did not receive the advocacy intervention, while those in the "treatment" group did. However, the design recognized that in both the treatment and control schools/villages, some will choose to participate in the transfer and some will not. This created four groups - control schooldvillages that did not transfer, those that did, treatment schools/villages that did not transfer, and those that did. The random assignment of the advocacy program served as the instrumental variable allowing identification of the impact of the transfer, separate from the factors leading to the self- selection of communities into the program.62 The evaluation will measure the impact of the formal transfer of school management to communities on teacher absenteeism, teacher quality, parental and community participation, enrollment rates and learning outcomes, and the equity impacts of the intervention, among others. While the design should allow an unbiased estimate of the impact of the formal transfer to community management, the sample of schools and communities was selected from among the schools that had not yet transferred as of 2005, which may be systematically different from those that transferred earlier. The evaluation will measure short-term impacts of the formal transfer, but not long-term impacts or sustainability, nor will it assess the impact of the "intensive support" offered to some of the schools in the CSSP. 62 Chaudhury and Parajuli 2009 41 Annex D Annex D. Basic Data Sheet COMMUNITY SCHOOL SUPPORT PROJECT (CR. 3808) Key Project Data (amounts in US$ million) Appraisal Actual or Actual as % of estimate current estimate appraisal estimate (PAD) (W) Total project costs 5.18 5.69 109% Loan amount 5 5.3663 107% Cancellation 0 Cumulative Estimated and Actual Disbursements (CR.3808) FY04 FY05 FY06 FY07 FY08 .~ ~-~~ Appraisal estimate (US$M) 1.5 3.8 5 .O (PAD) Actual (IJS$M) (SAP) 0.8 1.9 3.1 4.3 5.3 Actual as % of appraisal 54.7% 49.2% 62.4 Date of final disbursement: 10/16/200864 Project Dates Original (Forecast) Actual (Actual) Concept Note 0312712003 03/27/2003 Negotiations 0611212003 06/09/2003 Board approval 0612712003 06/30/2003 Signing 0712 112003 07/25/2003 Effectiveness NA 08/28/2003 Closing date 09/30/2006 09/30/2008 Staff InDuts (Labor Costs: amounts in US$ million) FY03 FY04 FY05 FY06 FY07 FY08 FY09 Total Preparation 0.2 0.02 0.22 AppraisallNegotiations 12.2 12.2 Supervision 9.4 29.1 12.0 10.6 5.3 2.5 68.9 ICR 4.0 4.0 Total 12.4 9.42 29.1 12.0 10.6 5.3 6.5 85.32 63 Difference in appraisal amount and actual could be attributed to the fluctuation of the exchange rate between the US$ and the NPR. There is no explanation in the ICR regarding that increase in the actual amount. 64 Special Account documented. 42 Annex D Names Title Unit I Responsibility/Specialty Rajendra Dhoj Joshi Sr. Education Specialist SASHD Task Team Leader Susan E. Hirshberg Sr. Education Specialist SASHD Education Specialist I SushilaRai I Program Assistant I SASHD I Team Support 43 Annex E Annex E. Borrower's Comments Fax 1 4271 $64 4211165 F MfNiSTRY O FLNANCE Singhadurbar ~ R u K ~ T ~ ~ A NEPAL , Mr. Viriod Thomas Director General 115 June 2010 Evalriatian Independent Evaluation Group 'rhe World Hank Washington DC Lkar Mr. 'I liomas, This is with reference to our letter of 20 Jutre 2010. Further rcference is made LOqcliir Draft Project Pcrfomiance Assessment Report (C'ommunity School Support Prqcct. CR. 3808) of 23 June 20 I O In this connection, 1 now wish to request you 'to kindly consider oiily pnrn 1-1, arid para 6-8 as Bonowcr's Cmimcnts. They are; I Ihe Got eminent of Nepal believes h a t 5oine ol the contents cif the report n c d tiiore rrseasch and credible analysis without wlirch the credibrlity 01. the icpon Iias k e n tiixdcnnincd. The 1L.G tcam . that carried out the evaluation ririd wrote the report mixed up t b w d i m i c t elcmontc i n onc. i.c thc issue of Bank's program rnanagcnicnt and the principles of cnmiiiutritk tiiaiiagemciit oi basic scr~icessuch as education, The documerlt nlmost rcstk :I$ i f thc IlG I~ad eaiicludcd that Corninunity Management does not work and i s trjiiig to prove the mne. 2 The Government does not hace a problem with zhc I nning tho !Link toam's pcrfoiriiance in ternis of cvaltiating this program and develop~rlga knuwlcclgc base ~ 1 Itit: brsis of this L.IL. 1 Goverriinetit rtrttnglq diwgrees with sane of the liiies in tlic docuinent that questic~iii ce of "cornmunit> manngenlei~t"as a srraleg> ~hiottgha huriictllq curricd aut %.tiid). le\ cl dcvelopmeiit activities ecoininctided rti:tt the school qystom bo handed over to thc re\pccti\ c commuiiitroa b o 115 ti) indhc p h l i c ' expenditure on education effectikc and producttve School edticatlort cariiiot bc madc cffkcii) e ~ , i t h i \ t makiiig ~t teachcrs rtcwiiiitabie t o the purents and students 3 .. Cominriizity Management was, thwcrore, a government strategy to which thc Bank later suhscribed to and the PPAR's mandate cannot extend to questionill$ thc Government's strategy. I here 8R 1 44 Annex E Tel No 1 42 11993 42 11867 4271801 -12"1837 4211748 42'1826 42?1775 .!97:e13 Fax j 4% 1Ill65 42 7 63 Singhodurbar AT^^^^^^, NEPAL numerous in%mxs of this in the documcnt and these mtist be purged before this document IS released. 4. Communirj management was found highly strcccssful in foresiiy cnnservutioii and developtneat and in other focal I c d development works durtog the pcriod of intense conflict in Nepal In fact, cominimity management o f hasic services is the most resilicnt forin of dcvetopincnt delivery ti1 confl rct-a Keeled societ). ~ 5 Thc (;ON decidcrd to hand over tlie school iiiniiaganent to coitmiinities B'J its policy stance This \\as rnttiated wtthout anticipating any exlcrtial assistance. The Rdnk came later, tlnd then came host of other donors supporting the riwpleiiient,iliciii of the idea. Wh) did not ItG team iiiquire with otliar donors iii the Education SWAp? Wcrc thcy illso "convinced t?j thc Rank"? This policy would hace been iniplerncnted even H ithotit any cxtarnai a m x e . Fui-thcrmore, the GON has not wavcretf rronl this strategy over the past decadc and do not intend to move mas) from this in the corning >ears a$ well. For the Mi iepoit to quemon this Etrategy. and not t l i i 12rznk'~perfonname in measurtng, monitoring and learning from the LlL, 1s rotally u n i ~ ~ ~ e p t t ~ b k helreie t l i d it We dcriionrtratcs the author's cvinplere lack of undcrsfaiiding of the p - 0 ~ ~ 5 5 and inadequate consultations with the concerned stahLholders i h d Icd us to where are at prescnt. 6 I'he report is fraught with unconiiected statcmeiiis that wen1 to portraj a w i o i i s problem uith regorth to community rnanagernetrt t lowe\er, the report itself fails 10 offci %ill> serious evidence of the saint problem The authur i s wily suggesting that this could be a problcm, hut offers little i n the %a) of evidence. 7 , Cornmutiity management is a Iong-tGrai 51rate o f tlie Go\ enimciit - ne ackiiowledpc that tlicre IC. alwa!s inmn for impro~enientshut the sclcc e, often unsubst.uitiated and biased prcscntalion of cvidcncc in the report seems to suggcbt that h e PPAK team has an ageiida agciinst cornmwity ITIdlkigC~lC~l~ With bcqt q a r d l ; . p, Yaurs siticerely, cc Ms. S . Goldrnsrk Country IJirector The World B a l k NEPAL 2