Document of The World Bank Report No: ICR2481 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-45900 IDA-48940) ON A CREDIT IN THE AMOUNT OF SDR 232.7 MILLION (US$ 350 MILLION EQUIVALENT) TO THE ISLAMIC REPUBLIC OF PAKISTAN FOR A SINDH EDUCATION SECTOR PROJECT (SEP) December 24, 2012 Human Development Department South Asia Region i CURRENCY EQUIVALENTS (Exchange Rate Effective June 29, 2012) Currency Unit = Pakistani Rupee (PKR) US$ 1.00 = PKR 94.627 FISCAL YEAR July 1 – June 30 ABBREVIATIONS AND ACRONYMS ADP Annual Development Plan AF Additional Financing ASC Annual Schools Census BoC Bureau of Curriculum CAS Country Assistance Strategy CPD Continuous Professional Development CPS Country Partnership Strategy DAC Departmental Accounts Committee DDO Drawing and Disbursing Officer DLI Disbursement Linked Indicator DPs Development Partners DSP Differential Stipends Program ED Education and Literacy Department EEP Eligible Expenditure Program EMIS Education Management Information System EMR Education Management Reforms EPDF Education Program Development Fund ESMF Environmental and Social Management Framework ERU Economic Reform Unit EU European Union FD Finance Department FM Financial Management FMIS Financial Management Information Specialist GoS Government of Sindh IAS Internal Audit Specialist IDA International Development Association ITE Initial Teacher Education LAC Latin America and Caribbean Region KPI Key Performance Indicator MFSS Minimum Funding Standards for Schools MDG Millennium Development Goal M&E Monitoring and Evaluation ii MTBF Medium Term Budget Framework MTFF Medium Term Fiscal Framework NER Net Enrollment Rate PDV Present Discounted Value PEACe Provincial Education Assessment Center PIFRA Project for Improvement of Financial Reporting and Auditing PITE Provincial Institute for Teacher Education PPRS Promoting Private Schooling in Rural Sindh PRSP Poverty Reduction Strategy Paper PSLM Pakistan Social and Living Standards Measurement Survey RSC Reform Steering Committee RSU Reform Support Unit SBD Standard Bidding Document SEF Sindh Education Foundation SEDPC Sindh Education Sector Development Policy Credit SEMIS Sindh Education Management Information System SEP Sindh Education Sector Project SERP Government of Sindh’s Medium Term Education Sector Reform Program SESM Sindh Education Sector Managers Course SIP School Improvement Plan SMC School Management Committee SPPRA Sindh Public Procurement Regulatory Agency STBB Sindh Textbook Board STEDA Sindh Teacher Education Development Authority SWAp Sector Wide Approach TA Technical Assistance TPV Third Party Validation TED Teacher Education Development TOP (District) Terms of Partnership UNICEF United Nations Children Fund Vice President: Isabel M. Guerrero Country Director: Rachid Benmessaoud Sector Manager: Amit Dar Project Team Leader: Umbreen Arif and Dhushyanth Raju ICR Team Leader: Surendra Agarwal and Salman Asim iii PAKISTAN Sindh Education Sector Project TABLE OF CONTENTS Page DATA SHEET .................................................................................................................... v A. Basic Information....................................................................................................... v B. Key Dates ................................................................................................................... v C. Ratings Summary ....................................................................................................... v D. Sector and Theme Codes .......................................................................................... vi E. Bank Staff .................................................................................................................. vi F. Results Framework Analysis .................................................................................... vii G. Ratings of Project Performance in ISRs ................................................................. xiv H. Restructuring (if any) .............................................................................................. xiv I. Disbursement Profile ................................................................................................ xv 1. Project Context, Development Objectives and Design ................................................... 1 2. Key Factors Affecting Implementation and Outcomes .................................................. 5 3. Assessment of Outcomes .............................................................................................. 13 4. Assessment of Risk to Development Outcome ............................................................. 22 5. Assessment of Bank and Borrower Performance ......................................................... 22 6. Lessons Learned............................................................................................................ 25 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners............... 26 Annex 1. Project Costs and Financing .............................................................................. 28 Annex 2. Outputs by Component...................................................................................... 29 Annex 3. Economic and Financial Analysis ..................................................................... 38 Annex 4. Bank Lending and Implementation Support/Supervision Processes................. 41 Annex 5. Beneficiary Survey Results ............................................................................... 43 Annex 6. Stakeholder Workshop Report and Results....................................................... 44 Annex 7. Summary of Borrower's ICR ............................................................................. 45 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ........................... 58 Annex 9. List of Supporting Documents .......................................................................... 59 MAP .................................................................................................................................. 61 iv DATA SHEET A. Basic Information Pakistan: Sindh Country: Pakistan Project Name: Education Sector Project (SEP) Project ID: P107300 L/C/TF Number(s): IDA-45900,IDA-48940 ICR Date: 12/27/2012 ICR Type: Core ICR ISLAMIC REPUBLIC Lending Instrument: SIL Borrower: OF PAKISTAN Original Total XDR 200.60M Disbursed Amount: XDR 230.02M Commitment: Revised Amount: XDR 230.03M Environmental Category: B Implementing Agencies: Reform Support Unit, Education and Literacy Department, Government of Sindh Cofinanciers and Other External Partners: European Union (EU) B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 12/23/2008 Effectiveness: 06/11/2009 06/11/2009 Appraisal: 04/13/2009 Restructuring(s): Approval: 06/04/2009 Mid-term Review: 03/01/2011 06/30/2012 Closing: 06/30/2012 06/30/2012 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Substantial Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Satisfactory Overall Borrower Satisfactory v Performance: Performance: C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Moderately Closing/Inactive status: Satisfactory D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) General education sector 25 25 Public administration- Education 50 50 Secondary education 25 25 Theme Code (as % of total Bank financing) Education for all 69 69 Gender 15 15 Public expenditure, financial management and 11 11 procurement Rural services and infrastructure 5 5 E. Bank Staff Positions At ICR At Approval Vice President: Isabel M. Guerrero Isabel M. Guerrero Country Director: Rachid Benmessaoud Yusupha B. Crookes Sector Manager: Amit Dar Amit Dar Project Team Leader: Dhushyanth Raju/Umbreen Arif Reema Nayar/Tahseen Sayed ICR Team Leader: Surendra K.Agarwal/Salman Asim ICR Primary Author: Surendra K. Agarwal Salman Asim Ayesha Khan vi F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The development objective of the proposed project was to support the Government of Sindh's Medium Term Education Sector Reform Program (SERP). The objectives of SERP were to increase school participation, reduce gender and rural-urban disparities, increase progression and improve the measurement of student learning. Revised Project Development Objectives (as approved by original approving authority) The PDO was not revised. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : Net Enrollment Rate (NER), primary level (age group 5-9 year olds, grades 1-5). Value 50.3% [PSLM survey 53% [PSLM survey quantitative or 55.6% 2006/07] 2010/11] Qualitative) Date achieved 12/31/2007 06/30/2012 09/30/2011 51% of target achieved by 2010/11 (one year before the target year). PSLM Comments survey data 2011/12 is not yet available. Large scale floods in 2010 & 2011 are (incl. % likely to have resulted in a shortfall in the achievement against the target (para achievement) 3.2.2). Female-male ratio NER, rural Sindh, primary level (age group 5-9 year olds, Indicator 2 : grades 1-5). Value 61.2% [PSLM survey 71.9% [PSLM quantitative or 64.6% 2006/07] survey 2010/11] Qualitative) Date achieved 12/31/2007 06/30/2012 09/30/2011 Comments 315% of target achieved by 2010/11(one year before the expected assessment (incl. % date). PSLM survey data for 2011/12 is not yet available. Absent the floods, achievement) actual achievement could have been even higher. Indicator 3 : Transition rate from Class 5 to Class 6, girls, rural, public schools. 63.4% 55% [ASC 51.2% [ASC 2007/08] 2011/12] Value 61.9 (new quantitative or Bank estimate, new definition, see para Bank estimate, new Qualitative) definition = 49.7%, 3.2.11) definition = 55%, 2007/08 (para. 3.2.11) 2011/12 (para. 3.2.11) Date achieved 12/31/2008 06/30/2012 05/31/2012 Comments GoS dropped rural/urban classification in 2011/12. Actual value calculated (incl. % excluding main urban districts (Karachi & Hyderabad) is not comparable with achievement) indicator (para 3.2.11-3.2.13). Floods also likely impacted the transition rate. vii Learning levels monitored through diagnostic learning assessments (in district Indicator 4 : representative samples of public schools). Baselines for math, language, science & social studies, class 4 students established in All programmed No disaggregated 08/09, 09/10, Value student diagnostic information on learning 10/11 & 11/12, quantitative or assessments in levels within the respectively. Qualitative) target subjects were province. Follow up tests, completed. math & language, class 4 in 10/11 & 11/12 respectively. Class 8 math & language tests in 10/11 & 11/12. Date achieved 12/31/2008 06/30/2012 05/31/2012 All targets achieved. Analysis & reports of class-4 math & language tests (08/09 Comments & 09/10) completed. Analysis of follow-up math test, class 4 (10/11) not yet (incl. % done. System for learning assessments is in place & operational at provincial achievement) level. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Fiscal & budget management: Education budget, including SERP expenditures, Indicator 1 : consistent with Medium Term Fiscal Framework, MTFF [this indicator represents DLI 1]. FY10/11 education expenditures FY10/11 education aligned with Budgetary processes not budget (including Value FY10/11-12/13 linked to medium term SERP) executed in (quantitative MTBF; target was strategic education sector alignment with or Qualitative) achieved after planning FY10/11-12/13 delayed release of MTBF budget for the PPRS program. Date achieved 12/31/2008 06/30/2011 05/31/2012 Comments DLI targets for all years were achieved. Financing for SERP was protected (incl. % during project implementation despite the tight fiscal situation. achievement) District Terms of Partnership (TOP): Share of prioritized schools completed to Indicator 2 : meet stipulated design, construction quality and functionality standards [this indicator represents DLI 2]. viii Inadequate prioritization 68% by 6/30/2011, of schools for At least 60% as Value and 80% by rehabilitation. Insufficient validated by third (quantitative 5/31/2012, monitoring to ensure party construction or Qualitative) validated by the adequate infrastructure supervision firm. third party. quality. Date achieved 12/31/2008 06/30/2011 05/31/2012 113% of FY10/11 target achieved by 6/11 (133% by 5/12). 80% of schools (by Comments 5/12) constructed & met minimum standards. Dissolution of local government & (incl. % withdrawal of committed funds from accounts for program & floods affected achievement) further progress. Monitoring & Evaluation (1) No. of months to complete administration of Annual Schools Census (ASC); (2) Share of schools with less than 10% Indicator 3 : discrepancy in ASC data from validation data based on sample validation [this indicator represents DLI 3]. (1) Achieved. (1) Timeliness - 2007/08 (1) At most 4 (2) Less than 75% Value administration of ASC months. (2) At but confidence (quantitative took 8 months. (2) least 75% based on building measures or Qualitative) Quality - No systematic third party implemented to sample based verification. validation. improve data quality. Date achieved 12/31/2008 06/30/2011 10/31/2011 FY09 & FY10 targets achieved. FY11 ASC completed on time but data accuracy Comments below agreed threshold (actual 55%, target 75%). FY10/11 DLI was considered (incl. % met after RSU implemented agreed remedial actions to improve quality for achievement) FY11/12 ASC & onwards. District and Sub-district Education Management: Implementation of Education Management Reform (EMR) to strengthen capacity & accountability of Indicator 4 : education managers [indicator represents DLI 4 and extension of DLI in SEP AF]. Continue Target was implementation of expanded in Overlap in roles and EMR pilot SEP AF to responsibilities in including implementatio EMR program Value education management; inspection & n of EMR piloted in a total of (quantitative lack of career path and advice/ support initiative from 14 talukas in 14 or Qualitative) professional development system. Rollout previous 2 districts. and quality assurance of plan in place based pilot districts schools. on results of to additional 3 evaluation. districts. Date achieved 12/31/2008 06/30/2011 06/30/2011 10/31/2011 Targets for all 3 years achieved (EMR piloted in 9 more districts than target Comments requirement). The focus was on the introduction of cluster-based school (incl. % management & school-based support which was only a part of a wider agenda achievement) envisaged in EMR. School Management Committees (SMCs): (1) Share of public schools with Indicator 5 : active SMCs; (2) No. of schools in SMC pilot mobilization for capacity building in school improvement planning [this indicator represents DLI 5]. ix (1) At least 70% of SMCs activated; (1) 81.4% of SMCs Large numbers of inactive expenditures activated and SMCs. Financing and validated by third received SMC Value capacity building party. (2) At least grants (target 70%). (quantitative programs for SMCs 5,000 schools A survey-based or Qualitative) discontinued. Limited supported under grant expenditure involvement of SMCs in pilot following review completed. school improvement. impact evaluation (2) 5,020 schools. design. Date achieved 12/31/2008 06/30/2011 06/30/2012 Comments (1) 112% of FY08/09, 119% of FY09/10 & 116% of FY10/11 targets achieved. (incl. % Progress sustained in 2011/12. (2) 5,020 schools were supported for capacity achievement) building. Stipends for Girls in Public Schools: (1) Share of beneficiary girls that received Indicator 6 : stipends; (2) No. of low transition-rate talukas (sub-districts) with higher stipends (Differential Stipends Program-DSP) [this indicator represents DLI 6]. (1) Over 90% of intended beneficiaries (1) At least 90% of received stipends intended via money orders in beneficiaries in FY10/11, validated (1) 80% delivery of Value each district as by a third party 2007/08 stipends by (quantitative validated by third sample survey. (2) 2008/09. (2) No targeted or Qualitative) party. (2) At least Higher stipends program. 45 talukas offered to about following impact 85,000 beneficiaries evaluation design. in 45 talukas in FY10/11, consistent with agreed impact evaluation. Date achieved 12/31/2008 06/30/2011 10/31/2011 Over 90% of intended beneficiaries received stipends each year (in 2010/11, Comments money orders (MOs) issued to about 409,000 beneficiaries). DSP implemented (incl. % as planned. Results sustained in FY12 (98.5% of intended beneficiaries received achievement) stipends). Public Private Partnership: No. of private schools supported under fully-funded pilot public subsidy program in qualifying underserved rural localities Indicator 7 : (Promoting Low-Cost Private Schooling in Rural Sindh - PPRS) [this indicator represents DLI 7]. No pilot program. PPRS under design by Value At least 450 PPRS government and SEF, the Over 450 PPRS (quantitative schools implementing schools operational. or Qualitative) operational. organization for the program. Date achieved 12/31/2008 06/30/2011 05/31/2012 Comments FY09, FY10 targets achieved. FY11 target met in FY12 after delay. Number of x (incl. % operational schools dropped to 400 in FY12/13 as contracts for schools entering achievement) in the latest round of entry & that not meeting unserved community criteria were cancelled. Merit Based School Specific Recruitment of Teachers: Implementation of merit Indicator 8 : and needs based teacher recruitment [this indicator represents DLI 8 and was also a Global DLI]. New teachers Implementation of recruited on merit teacher in Round I (5,271 Merit-based teacher Value recruitment teachers). Round II recruitment initiative (quantitative following merit recruitment of approved in April 2007 or Qualitative) based criteria as approximately and revised in June 2008. validated by third 8,000 teachers party. finalized by May 2012. Date achieved 12/31/2008 06/30/2011 06/30/2011 Comments DLI targets met satisfactorily each year & progress continued per agreed policy. (incl. % The first round of recruitment was additionally validated by a third party. Agreed achievement) protocol followed for recruitment & placement of teachers. Teacher Education Development (TED): (1) Standards and competency based continuous professional development (CPD); (2) quality assurance of CPD Indicator 9 : providers; (3) new diploma for certification of teachers [this indicator represents DLI 9]. (1) New CPD program piloted in 3 districts, (2) 6 CPD providers (1) No standards and (1) New CPD accredited. competencies guiding program piloted in ED/RSU teacher in-service training at least 2 districts. implemented (CPD); (2) no quality (2) At least 3 CPD Value accreditation assurance of training providers (quantitative process with programs; (3) no plan for accredited. (3) or Qualitative) guidance and transition of Initial New diploma in support of STEDA Teacher Education (ITE) use in training of Board of from present certification all new teachers in Governors. (3) (CT PTC) courses. at least 5 colleges. New diploma piloted in 2 colleges and launched in 10 colleges. Date achieved 12/31/2008 06/30/2011 05/31/2012 Comments Targets achieved with delay in CPD providers' accreditation in FY10/11 due to (incl. % pending legal status of STEDA. Bank accepted reduction from 5 to 2 colleges, achievement) for piloting of new diploma course, from a feasibility point of view. Learning Assessment of Students: (1) Subject and grade for fielding of diagnostic assessment in district representative sample of public schools; (2) Indicator 10 : Subject & grade for analysis report of diagnostic assessment [indicator represents DLI 10]. Value No district representative (1) Class 4 math, (1) Achieved. xi (quantitative assessment information science & class 8 (2) Report produced or Qualitative) on learning outcomes math tests available in province. implemented (representative at district level). (2) Class 4 language report produced. Date achieved 12/31/2008 06/30/2011 05/31/2011 All DLI targets achieved in 3 years incl. class 4 math & language tests in Comments FY08/09 & FY09/10, respectively. In addition, reports of class-4 math (incl. % assessment (08/09) and language assessment (09/10) were produced & achievement) disseminated. Preparation & management of school specific nonsalary budgets [this indicator Indicator 11 : represents a DLI in SEP AF]. Preparation of school specific School budgets No school-level budget nonsalary budgets prepared by 3 Value planning and using preliminary districts, (quantitative management for primary, MFSS and incorporated in the or Qualitative) middle and elementary incorporation into budget books for government schools. the FY2011/12 FY2011/12. budget books in at least 3 districts. Date achieved 01/31/2011 06/30/2011 06/30/2011 Comments 100% of the target achieved. Subsequently, all districts have prepared school- (incl. % specific budgets (non-salary and salary) for FY2012/13, which have been achievement) included as Annex in the FY2012/13 Finance Department budget book. Teacher rationalization across schools and the formal allocation of teaching posts Indicator 12 : to schools [this indicator represents a DLI in SEP AF]. Allocation of 3 districts prepared sanctioned salary budgets teaching posts to Teaching posts are rationalizing the schools and Value not at the school level teaching force in incorporation of (quantitative for primary, middle, each school (and school-specific or Qualitative) and elementary post teachers teaching posts in government schools. according to the FY2011/12 budget rationalized books in at least 3 number). districts. Date achieved 01/31/2011 06/30/2011 06/30/2011 Comments Target was achieved. Subsequently for FY2012/13, all districts have prepared (incl. % school-specific budgets in which the numbers of teaching and non-teaching staff achievement) are identified. Net Enrollment Rate (NER), secondary level (age group 10-14 year olds, grades Indicator 13 : 6-10). Value 26.4% [PSLM survey 29.8% [PSLM (quantitative 27.9% 2006/07] survey 2010/11] or Qualitative) Date achieved 12/31/2007 06/30/2012 09/30/2011 xii By 2010/11(one year before the target assessment date), 227% of the target Comments (expected increase in the NER) achieved. PSLM survey data for 2011/12 is not (incl. % yet available. Absent floods, actual result could have potentially been even achievement) higher. Female-male ratio NER, rural Sindh, secondary level (age group 10-14 year olds, Indicator 14 : grades 6-10). Value 32.3% [PSLM survey 40.3% [PSLM (quantitative 39.2% 2006/07] survey 2010/11] or Qualitative) Date achieved 12/31/2007 06/30/2012 09/30/2011 Comments By 2010/11(one year before the target assessment date), 116% of the target (incl. % achieved. PSLM survey data for 2011/12 is not yet available. Absent floods, achievement) actual result could have potentially been even higher. Primary school completion rate (share of 12 years olds completing class 5 Indicator 15 : conditional on ever attending school). Value 53.4% [PSLM survey 58.3% [PSLM (quantitative 55.2% 2006/07] survey 2010/11] or Qualitative) Date achieved 12/31/2007 06/30/2012 09/30/2011 By 2010/11(one year before the target assessment date), 272% of target (or of the Comments expected increase in the completion rate) achieved. PSLM survey data for (incl. % 2011/12 not yet available. In absence of the floods, result could have potentially achievement) been higher. Indicator 16 : Teacher absenteeism. Not tracked. But the first round of school survey as part of TPV of teachers recruited Baseline and target to be Value under Round 1 established after school Decline in teacher (quantitative found absenteeism survey by a third party absenteeism or Qualitative) rate of 16% for firm in 2009/10 these contract teachers compared to 21% among regular teachers in same schools. Date achieved 06/30/2010 06/30/2012 06/30/2012 Comments Rounds of school surveys by a third party firm could not be carried out because (incl. % the survey was linked to the TPV of the teacher recruitment program & Round 2 achievement) recruitment was not completed in time. Indicator 17 : Student absenteeism. Baseline and target to be Value established after school Decline in student (quantitative Not tracked survey by a third party absenteeism or Qualitative) firm in 2009/10 Date achieved 06/30/2010 06/30/2012 06/30/2012 Comments Rounds of school surveys by a third party firm were not carried out, and the (incl. % indicator was not tracked. xiii achievement) Improving Transparency and Accountability in Education Expenditures: Indicator 18 : Number of residual audit paragraphs resolved in education sector [this indicator represents a covenant]. Reduction, by 04/09, of 35% of cumulative outstanding advance audit paras since 7,679 cumulative FY05/06; by Value outstanding advance audit 04/10, reduction of (quantitative Achieved paragraphs since 60% of residual or Qualitative) FY2005/06 advance audit paras for FY07/08; and by 04/11, a reduction of 60% of residual advance audit paras for FY08/09 Date achieved 11/01/2007 04/30/2011 06/30/2011 Comments Target for each year achieved. First year target (35%) was exceeded by 5 (incl. % percentage points, second year target (60%) was exceeded by 3 percentage points achievement) and the third year target (60%) was exceeded by 4 percentage points. G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 07/29/2009 Satisfactory Satisfactory 102.03 2 02/13/2010 Satisfactory Moderately Satisfactory 119.27 3 11/10/2010 Satisfactory Moderately Satisfactory 187.89 4 06/27/2011 Satisfactory Moderately Satisfactory 274.77 5 01/12/2012 Satisfactory Satisfactory 325.54 6 06/25/2012 Moderately Satisfactory Satisfactory 350.08 H. Restructuring (if any) Not Applicable xiv I. Disbursement Profile xv 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal 1.1.1 At the time of preparation of the Sindh Education Sector Project (SEP), Pakistan’s economic growth had slowed down to below 4% owing to major external and internal shocks, after having experienced a remarkable turnaround since 2000 in part as an outcome of Government initiated reforms, as articulated in its Poverty Reduction Strategy Paper (PRSP-I, 2003). Despite the economic progress, social indicators were low and compared poorly with those of other countries with similar levels of per capita income. The country was off track in meeting the education Millennium Development Goals (MDGs) for 2015, which are universal primary completion and gender parity in enrollment. 1.1.2 Sindh, Pakistan’s second largest province in terms of both population and economic output, performed poorly across a range of socioeconomic indicators despite its strong relative economic position and growth performance. The incidence of household poverty, especially in rural Sindh, was high (31% in 2005/06) and the rural population was among the most vulnerable to falling into poverty. About half the children of the primary school population were enrolled in school. The net enrollment rate (NER) for rural girls in primary school (ages 5-9 years, grades 1- 5) was 31% in 2006/07, and only about one half of rural girls stayed enrolled for some middle or secondary education after completing primary schooling. Lastly, the level of student achievement in the province, as in the rest of the country, was low. 1.1.3 Major sector issues affecting the performance of public education system and causing poor outcomes in Sindh were: (i) insufficient and inefficient budgetary processes; (ii) minimal monitoring, supervision and accountability of service delivery agents; (iii) limited parent and community oversight of and participation in school level management and decision-making; (iv) ineffective implementation of demand side programs such as secondary school stipends for girls and the provision of free textbooks; (v) patronage-based public school teacher hiring, potentially compromising the quality of in-coming teachers and their accountability; (vi) lack of teacher standards and quality assurance in teacher education and professional development; and (vii) limited extent and quality of evidence-based decision-making due to lack of timeliness and quality of data from existing monitoring and evaluation (M&E) systems. 1.1.4 To address these challenges, the Government of Sindh (GoS) initiated a multi-faceted, medium-term Education Sector Reform Program (SERP) in 2006/07. Reflecting an important shift from previous development efforts which focused on the levels of expenditures, inputs and benefits, SERP sought to improve access, equity and quality in education by (i) improving governance and accountability in education service delivery, and (ii) combining education reforms with fiscal and public finance management interventions to improve the effectiveness and efficiency of public expenditures. 1.1.5 The Bank supported the design and implementation of the first year of SERP through the Sindh Education Sector Development Policy Credit (SEDPC) in June 2007. The Bank continued to support the SERP through the Sindh Education Sector Project (SEP) and Additional Financing (AF) for SEP, approved by the Board in June 2009 and March 2011, respectively (see Box 1). 1 Box 1: SEP Instrument Design: The SEP, along with a similar education project in Punjab, was the first results-based education sector operation in IDA countries, with four key innovative design features: • The design was performance based with disbursements tied to pre-specified annual implementation progress and performance targets, referred to as Disbursement Linked Indicators (DLIs). The DLIs captured intermediate results, central to the medium-term achievement of the project development objectives. • The design was underpinned by education sector governance initiatives that attempted to set the structures and incentives right for service delivery performance. • The design also supported improvements in budget and fiscal management, financial management, procurement management, and environmental safeguard principles and practices. While the specific focus was on the education sector, benefits of many of these improvements extended beyond the education sector. • The design aimed at further strengthening of the monitoring and evaluation (M&E) systems. The design likely helped in orienting and focusing the GoS's efforts on agreed program implementation progress and performance targets (see Section 2), and enabled the GoS to take forward important but politically-difficult, governance-oriented reforms by providing the desired backing in the face of opposition from status-quo proponents. Annual sets of DLIs also likely promoted steady, incremental progress, scale up, and improvements in program implementation over the project's life. 1.1.6 Rationale for Bank Assistance: The SERP provided a sound and promising strategy for improving educational outcomes in Sindh, helping the country make progress towards meeting the education MDGs. The Bank was in a unique position to add considerable value, beyond financing the expenditures, through the associated technical and global knowledge that underpins the Bank’s guidance, and through the political support it leverages domestically for introducing significant institutional changes. 1.1.7 Higher-level Objectives: At the time of approval, the Bank’s Country Assistance Strategy (CAS, 2006a) and the national priorities emphasized increased academic achievement of primary-school students, reduction in gender gaps in school participation, and improvement in access to and quality of secondary education. One key pillar of the Government’s PRSP II (2009) was Human Development which emphasized (a) improving access to education for all, (b) removing urban-rural and gender imbalances in school participation, (c) enhancing the quality of education and student learning at all levels, and (d) improving governance and management of the education sector. PRSP II also emphasized the need to increase financing for the sector, strengthen planning and implementation capacity, improve resource utilization, enhance governance for greater accountability of education providers to the community, build capacity of district and local institutions, and strengthen the role of school committees, all of which are central in SERP and the SEP. 1.2 Original Project Development Objectives (PDO) and Key Indicators 1.2.1 The development objective of the project was to support the SERP. The objectives of SERP were to increase school participation, reduce gender and rural-urban disparities in school participation, increase progression and improve the measurement of student learning. 1.2.2 The success in meeting the PDO was to be measured using the following four PDO level outcome indicators and targets (page 5, PAD): (i) NER in primary schools was expected to increase from 50.3% in 2006/07 to 55.6% in 2011/12; (ii) female-male ratio NER in primary schools in rural Sindh was expected to increase from 61.2% in 2006/07 to 64.6% in 2011/12; 2 (iii) transition rate from class 5 to 6 for girls in rural public schools was expected to increase from 51.2% in 2007/08 to 63.4% in 2011/12, and (iv) learning levels were expected to be monitored through diagnostic learning assessments. The project was to also regularly monitor the achievement of the DLIs to measure success of the program (see PAD, pages 36-38 and SEP AF PP, page 9). Although the PAD (page 35) included other intermediate and outcome indicators, the project’s primary focus was on the four PDO level indicators and the DLIs to measure the project’s success. 1.3 Revised PDO and Key Indicators, and Reasons/Justifications 1.3.1 The PDO and the PDO level key indicators were not revised. However, in March 2011, under the SEP AF, two additional DLIs were added and one of the original DLIs in the SEP was expanded in scope (see paragraph 1.7.1). One key justification for the SEP AF was to support the GoS’s commitment and effort in the design and early implementation of the selected key education reform activities to improve school system management and service delivery performance, in order to enhance the SEP’s effectiveness. 1.4 Main Beneficiaries 1.4.1 All students in GoS primary and secondary schools and low-cost private schools that were covered by the SEP initiatives were the immediate beneficiaries and targets of the project (about 4.29 million children in 49,496 schools at appraisal). The other main beneficiaries were the Education Department (ED), its Reform Support Unit (RSU), district and school level management for service delivery, and parents and teachers. The initiatives taken together covered the entire province, and also covered all levels (grades 1-10). Nine of the ten areas with DLIs covered government schools, and two of the ten areas with DLIs covered the Sindh Education Foundation’s (SEF’s) Promoting Private Schooling in Rural Sindh (PPRS) program, a public-private partnership to extend service delivery to disadvantaged communities. Several interventions in the SERP and SEP were to disproportionately generate larger performance gains in rural areas and/or benefit girls (see paragraphs 2.4.6, 2.4.7, 3.2.3 and 3.2.8). 1.5 Original Components 1.5.1 The project consisted of two components: (1) Financing of SERP (cost at appraisal US$2,166 million, IDA US$294 million); and (2) Technical Assistance (IDA US$6 million). 1.5.2 Component 1 (Financing of SERP) was to finance key eligible expenditure programs (EEPs) in SERP, with disbursements based on achievements of agreed DLIs that reflect implementation progress and performance targets, critical to achieving the PDO (see Annex 2). SERP had four broad thrust areas: I. Improving fiscal sustainability and effectiveness of public education expenditures, through: (i) improved budgetary processes to introduce medium-term, results based budgeting through the preparation of Medium-Term Fiscal and Budget Frameworks (MTFF, MTBF); (ii) enhanced internal controls and financial accountability through the resolution of audit paragraphs; and (iii) improved procurement management. It included one DLI: DLI 1- Execution of education budgets (including SERP expenditures) in alignment with the MTFF which integrates SERP strategy and results. II. Improving education sector management, through reforms to (i) strengthen provincial monitoring of education sector and evaluation of priority programs including the implementation of the Annual School Census (ASC); (ii) strengthen district education management cadre; (iii) improve performance of districts in school rehabilitation and education service delivery through performance based funding partnerships; and (iv) strengthen School Management Committees’ (SMCs) capacity and effectiveness through the implementation of a pilot capacity building 3 program. It included four DLIs: (a) DLI 2 - Share of prioritized schools to meet stipulated design, construction quality and functionality standards; (b) DLI 3 - Time taken to complete administration of the ASC and share of schools with ASC data within 10% deviation from validation data; (c) DLI 4 - Steps in the implementation of the new education management structure and advice/support system in at least one taluka of two pilot districts; and (d) DLI 5 - Share of public schools with active SMCs, and the number of schools supported by the SMC capacity building pilot. III. Improving access to quality schooling particularly in rural areas and for girls, through reforms to (i) improve the performance and effectiveness of GoS’s existing programs (free textbooks program, and the secondary school girls’ stipend programs for attending public schools and a phased-in introduction of the DSP (higher annual stipend amount) in talukas with the lowest transition rates for girls from class 5 to class 6) to provide incentives to poor families to send their children to school; and (ii) leverage the provision of schooling by private entrepreneurs in underserved rural communities. Under the SEF’s PPRS program, the GoS was to offer a per student subsidy to private entrepreneurs subject to their meeting specified quality standards for school infrastructure, providing free schooling. It included two DLIs: (a) DLI 6 - Share of beneficiary girls receiving stipends and number of talukas covered under the DSP; and (b) DLI 7 - Number of private schools supported by the pilot PPRS program. IV. Improving the quality of teaching and student learning, through instituting merit and needs based recruitment of teachers, improving programs and procedures for their continuous professional development (CPD) and merit-based career advancement, improving effectiveness of teacher training programs, and strengthening the provincial diagnostic student learning assessment and improved classroom assessment. It included three DLIs: (a) DLI 8 - Implementation of merit based teacher recruitment; (b) DLI 9 - Steps in the implementation of standards and competency based teacher education and development (number of districts with pilot CPD program, number of CPD providers accredited, number of colleges offering new teaching diploma); and (c) DLI 10 - Subjects and grades for annual district representative diagnostic assessments and production and dissemination of analysis reports. 1.5.3 Component 2 (Technical Assistance, TA) was to finance essential technical, advisory and capacity building support initiatives to strengthen fiduciary, environmental management, administrative and M&E activities to assist GoS in SERP implementation towards the achievement of DLIs and verification of their achievement. Activities were to entail the design and implementation of reviews, validations, and evaluations by third-party firms. Significant TA was also financed by other Development Partners (DPs), particularly the EU, and by the GoS itself (paragraphs 1.7.2 and 2.2.1(c)). 1.6 Revised Components Project components were not revised. 1.7 Other significant changes 1.7.1 Scope of reform activities. Two new education sector reforms (rationalization of teachers across schools and formal allocation of teaching posts to schools, and preparation and management of school-specific nonsalary budgets) were added to SERP and one existing reform activity (implementation of the district and sub-district Education Management Reform (EMR) initiative) was extended to additional districts, at the time of the SEP AF approved by the Board in March 2011. The SEP AF included three DLIs (see Annex 2). The two new DLIs associated with the above first two changes were: (i) Allocation of sanctioned teaching posts to schools and incorporation of school-specific teaching posts in FY2011/12 budget books in at least three districts; and (ii) preparation of school-specific nonsalary budgets using preliminary minimum funding standards for schools (MFSS) and incorporation into the FY2011/12 budget books in at 4 least three districts. The third DLI was an extension of the DLI discussed in Section 1.5.2.II above, i.e. DLI 4 to include three additional districts. 1.7.2 Funding allocations. The original credit amount was US$300 million equivalent and the AF was US$50 million equivalent. At the request of the Borrower, SDR 2.67 million (or US$4.01 million equivalent) was cancelled in June 2012 from the amount allocated to Component 2 (Technical Assistance) because several TA activities envisaged at the start of the SEP to be financed by IDA were either financed by other DPs (EU, USAID via its ED-LINKS initiative, and Asian Development Bank) or by the GoS itself (see paragraph 2.2.1(c)). The possibility of first using grant financing for TA before using the IDA credit was foreseen at appraisal and thus was planned for (PAD, page 51). 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 2.1.1 Background Analysis: The SEP was a three year investment lending operation after the close of the first SEDPC in December 2007 and the decision that the next two parts of the DPC series would not proceed owing to the macro environment and uncertainties regarding the IMF program. SEDPC supported the design and the first year of implementation of SERP. The new GoS that came to Office in February 2008 made education a top priority, unambiguously stated its ownership of and commitment to SERP, showed determination in accelerating implementation, and requested continued Bank support for SERP. The GoS also continued to build on the results of the SEDPC (PAD, pages 100-101) by further developing and approving the new SERP policies (e.g., district education management) and the specific design of key sub-programs (e.g., public private partnership program). The design of other sub-programs was also further refined or improved based on experience (e.g., school rehabilitation under the district TOP program). 2.1.2 The preparation of SEP benefitted from the lessons of the GoS’s own implementation experience, from the experience of Punjab which has been implementing its education sector reform program since 2003, from a range of analytical and advisory work for Sindh as well as Pakistan (PAD, pages 98-99), from broader international experience (importance of addressing teacher quality and the importance of student learning assessments), and from the experience of similar performance-based Sector Wide Approaches (SWAps) used in other countries, particularly in the Latin America and Caribbean (LAC) region, mostly in IBRD settings. The key lessons taken into account were the following: (a) criticality of strong political commitment and ownership in particular to successfully address governance constraints; (b) combining sector interventions with cross-cutting fiscal and fiduciary interventions to enhance the likelihood of results and sustainability; (c) provision of complementary non-lending analytical and advisory support and just-in-time policy and technical assistance notes to support design and implementation of the reform program; (d) sustained high level policy dialogue to ensure implementation of key reform actions to improve sector governance; (e) simultaneously addressing the issues of school access and quality of education to improve the socio-economic prospects of individuals; and (f) the importance of a good M&E system to provide timely and reliable information to inform the design and implementation. 5 2.1.3 Project Design: The GoS’s program (SERP), which the project supports, was prepared in a participatory approach through consultations with stakeholders including parents, students, teachers, communities, SMCs, civil society, GoS officials at the provincial, district and sub- district levels and the private sector. The Bank team that had already engaged with GoS on the design of SEDPC continued to be proactively engaged on the SEP program design including refinements of its subprograms and design of new reform areas, reinforcing ownership and active participation of the various stakeholders through consultations and technical and advisory support. The SEP was a Sector Investment Credit with a SWAp to support implementation of the GoS’s SERP for primary and secondary educations (see Box 1, page 2). Credit disbursements were to be made against selected key education budget line items or EEPs, based on the achievement of pre-specified results or DLIs, determined in partnership with GoS (PAD, pages 36-38). The project also included a small TA component. 2.1.4 DLIs reflected priority elements in the SERP, including implementation performance, institutional change or intermediate outcome indicators that built incrementally over the project life. They also represented the conditions needed to overcome the budgetary, governance and accountability constraints and in turn contribute towards the attainment of the key PDO outcome indicators. Some of the DLIs were expected to improve the efficiency and effectiveness in education sector interventions during the credit life, and others constituted incremental institutional steps that would have sustained impacts beyond the project life. The DLI relating to implementation of the merit-based recruitment of public school teachers was made a necessary condition for all disbursements, constituting a “global DLI�, because the GoS and the Bank considered this DLI as (i) central to the achievement of education access and quality targets, (ii) essential to improve sector governance but entailing high implementation risks, and (iii) an indicator whose achievement could benefit considerably from the leverage provided by the DP support. 2.1.5 The European Union (EU) had prepared its support, the Sindh Education Sector Plan Support Program, in tandem with the Bank’s previous SEDPC and prior to the SEP. The EU’s support was Euros 33 million in four tranches (budget support) and Euros 6 million in EU- managed complementary services. The EU supported the SEP and financed a similar set of DLIs, and held joint review missions with the Bank during the SEP’s implementation. 2.1.6 Risks and Mitigation Measures: All risks were properly identified and well mitigated through project design and preparation. Despite substantial risks associated with some of the sector governance improvements (e.g., merit-based teacher recruitment, new district education management structure and quality assurance of schools, accreditation of teacher training providers, prioritized school rehabilitation with construction supervision and procurement ex-post reviews by independent third parties, and internal and external controls for stipends and textbooks), the high degree of GoS commitment and the use of results based disbursements (DLIs) mitigated most risks and contributed to the success in implementing even difficult sector governance improvements. 2.1.7 Quality at Entry: No quality-at-entry assessment of the project was carried out by the former Quality Assurance Group. A Quality Enhancement Review (QER) was conducted in February 2009 which endorsed the project design, including the PDO and DLIs, fiduciary arrangements, and risk assessment and mitigation measures (see Annex 9 for reference). In view of the sound background analysis, incorporation of the lessons learned, innovative project design with disbursements linked to achievements of results, high GoS commitment, and comprehensive assessment of risks, the quality at entry is rated satisfactory. 2.2 Implementation 2.2.1 Implementation was satisfactory or moderately satisfactory throughout the course of the 6 project as rated in the Implementation Status and Results (ISRs) reports. The Bank’s support to the design of SERP and the first year of its implementation via SEDPC laid a good foundation for the SEP DLI matrix. Annex 2 shows how the DLIs and their targets under the project SEP were achieved. The following main factors positively affected implementation: (a) The results-based design of the SEP with disbursements tied explicitly to the satisfactory achievement of the DLIs likely helped in orienting and focusing the GoS’s efforts on agreed program implementation progress and performance targets (see Box 1, page2). (b) Close donor partnership and coordination throughout project design and implementation including joint review missions helped ensure that a single program was being supported by the DPs, also minimizing transaction costs for the GoS. (c) The TA component played an important complementary role to the results-based component. Together with the EU and other DPs, the Bank financed selected implementation support and capacity building activities, efforts to strengthen fiduciary, safeguard and M&E systems and activities, third party validations, and operational and impact evaluations of selected reforms and activities. TA resources were the main source of funding for contracting third parties for program implementation reviews, validations and evaluations. The EU provided TA support for the SMC subprogram, student diagnostic testing, district and sub-district education management reforms, the ASC, textbook procurement and overall provincial procurement reforms, logistical support to the RSU, monitoring visits to the field, and training/information dissemination activities. The USAID, via its ED-LINKS initiative, provided TA support for the ASC subprogram in selected districts and the Teacher Education Development (TED) subprogram. The Asian Development Bank provided TA support for the MTBF activity. The GoS used its own resources for the third- party supervision of school rehabilitation under the district TOP program. (d) Strong and continued GoS commitment was central to successful implementation of the program and for addressing governance constraints to effective service delivery. Many reform actions to improve the governance and accountability environment required significant breaks from past practices and challenging deep seated vested interests. Examples include: (i) replacing the patronage based recruitment of teachers by merit and needs-based, objective and transparent recruitment and placement of teachers in GoS schools; (ii) replacing past school infrastructure efforts characterized by poor screening and shoddy or incomplete construction, by a school rehabilitation program that offered annual grants to districts tied in part to the district’s performance in meeting construction quality and reporting indicators; (iii) starting a pilot PPRS program to use GoS funding to leverage the private sector to cost-effectively extend quality and free schooling to un/under-served communities in order to raise school participation and student achievement; and (iv) introducing, during the SEP implementation, new bold reforms including teacher rationalization across schools and fixing teaching posts to schools (SEP AF), and school system consolidation starting with the merging of adjoining schools. (e) Regular and sustained dialogue between the Bank and senior level GoS officials and stakeholders at the district and school level facilitated continuity in SERP and SEP implementation, particularly in difficult sector governance reforms despite frequent changes in key senior GoS officials (e.g., the Secretary of Education, GoS). Technical inputs from the Bank, including analytical reports and just-in-time policy and technical assistance notes, supported design and implementation of key reform actions. (f) The Bank team pursued and supported the GoS in the implementation of important governance actions to reduce fiduciary risks in the sector broadly even when such reform actions were not explicitly a part of the SEP design and DLIs. Two examples are: (i) substantial reduction in the risk of leakage through the success of concerted efforts of the RSU, in 7 collaboration with the Finance Department (FD), District Treasury Offices, Accountant General Offices and the PIFRA Directorate (Project for Improvement of Financial Reporting and Auditing), in reducing the payment of salaries to education sector employees in cash through Drawing and Disbursing Officers (DDOs) in the districts rather than via direct deposits into their personal bank accounts from about 23,000 to 1,200 employees (or from about 10% to 0.5% of the workforce) as of June 30, 2012; and (ii) agreement with the ED/RSU and the Sindh Textbook Board (STBB), after extensive dialogue during the SEP implementation, to procure textbooks starting for FY2012/13 under a single publishing contract rather than through separate contracts for paper, printing and publishing, to reduce potential inefficiencies and address contract management concerns. 2.2.2 Main factors that slowed down implementation and hindered progress: (a) The large scale floods and heavy rains of 2010 and 2011 caused extensive and major damage to school infrastructure and loss of school and student resources, disruption to school operations, and economic losses for and the physical displacement of households. They also caused the diversion of attention, efforts, and scarce funds of the GoS to disaster relief, recovery, and reconstruction activities. (b) Frequent turnover of Secretaries of Education during SEP (eight Secretaries of Education from FY08 to FY11) disrupted continuity and slowed down implementation. (c) Inadequate staffing in the RSU in particular for procurement (consulting services and contract management) and environment management affected implementation. Frequent turnover of GoS officers working at the RSU affected implementation performance and oversight of the SEP. (d) Inadequate mobility allowances for travel of RSU staff to the field negatively affected management and monitoring of program implementation progress and performance on the ground. (e) Weak implementation capacity of the districts to simultaneously implement specific sub- programs of SERP is likely to have significantly contributed to variable implementation progress and in some cases may have reduced the substance of the reform actions. Preference of the districts for maintaining the status quo instead of embracing important reforms and changes is likely to have also significantly contributed to variable implementation speed. (f) Implementation performance of the TA (component 2) was affected because of inadequate planning with sufficient lead time to carry out needed procurement, limited capacity for contract management, and limited use of the TA resources for meeting the important capacity needs at the district level and lower levels as well as for key entities, other than the RSU, associated with SERP (e.g., the Economic Reform Unit (ERU), PEACe, Sindh Education Foundation (SEF), and those concerned with the teacher education development (TED) and EMR programs). 2.2.3 Quality of Supervision: No quality of supervision assessment of the project was carried out by the former Quality Assurance Group. The Bank’s Operations Committee reviewed the project implementation progress after the first year of implementation, and endorsed the Task Team’s supervision approach and efforts. Very close supervision, timely and intensive technical support across the reform program, timely preparation of the SEP AF, and a dialogue on the GoS’s SERP II and follow-on SEP II project helped in successful implementation of the project. The quality of implementation is rated satisfactory. 8 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 2.3.1 M&E Design: Four key PDO level outcome indicators and a total of 33 DLI targets (over three years) were identified to monitor progress towards achieving the PDO. Household sample surveys (Pakistan Social and Living Standards Measurement (PSLM) Survey) were to provide data for two of the four PDO indicators (NER primary and female-male ratio NER primary) and the ASC was to be used for a third indicator (grade 5-6 transition rate, rural girls). SEP was to also conduct and make use of annual student diagnostic assessments by The Provincial Education Assessment Center (PEACe) in grades 4 and 8 in district representative samples of government schools. 2.3.2 Data reliability for monitoring had been an issue in Sindh for a long time. Continued strengthening of the M&E was integrated into the SERP design and the SEP supported the M&E strengthening through DLIs, EEPs and TA. Selected M&E activities, including rigorous impact evaluations, were to be also supported through Bank staff inputs and grants from the Education Program Development Fund (EPDF). The M&E in the GoS’s program includes the annual census of public schools (ASC), monitoring and evaluation of SERP interventions by the RSU and third parties, monitoring of student learning outcomes over time via PEACe, expenditure tracking and benefit incidence analysis, and rigorous impact evaluations using prospective designs of selected sub-programs. In addition, results of PSLM surveys were to be used to monitor and triangulate findings on outcomes. 2.3.3 M&E Implementation: The M&E system worked generally as planned except for the pace of improvements in the ASC. First, performance of key project activities was maintained and compliance with the agreed parameters under the DLIs was assessed through independent, transparent third party validations (TPVs). For example, the TPVs were used to monitor and evaluate compliance with the agreed DLIs for the rehabilitation of schools, ASC data (quality), activation of the SMCs, delivery of stipends to girls, and merit and needs based recruitment of teachers. Second, several impact evaluations were carried out, such as of the capacity building pilot of SMCs in school improvement planning, stipends for girls in low transition-rate talukas under the DSP, and the SEF PPRS program, to help inform policy and design discussions and decisions on scale-up of the sub-programs. Third, the GoS through PEACe successfully completed a programmed cycle of diagnostic assessments of student learning in selected subjects in grades 4 and 8; data from the most recent assessments (2011 and 2012) is still under analysis. 2.3.4 Fourth, the ASC remained the GoS’s primary data activity for its public schools and provided information on school characteristics, enrollment by grade and gender, and teacher level information. Under SERP, the Bank, EU and USAID (through its ED-LINKS initiative) supported agreed efforts to strengthen the ASC at both the province and district levels, in particular the ASC administrative procedures and strengthening human and technological resources at RSU which managed the ASC activity. Despite significant improvements in timeliness and quality of data of the ASC under SERP (though the ASC had been conducted for nearly 18 years, there was no verification of the quality of the data in the ASC until the start of the SERP), several weaknesses remain in the quality and reliability of the ASC. Weaknesses in the ASC data arise in part due to the capacity issues at the district and taluka level, poor record keeping in school registers, and poor field visit practices by supervisors to schools to collect information. These weaknesses have hampered the ability to track certain indicators and monitor performance of SEP in areas that rely on the ASC data (e.g., transition rates). Continued strengthening of the ASC to yield regular and reliable information on public schools will remain a key initiative under SERP II. 2.3.5 Baseline data on teacher and student absenteeism were to be established in 2009/10 and decline in the absenteeism rates were to be measured in 2011/12 through school surveys by third party firms. Rounds of school surveys by a third party firm on teacher absenteeism could not be 9 carried out because the survey was linked to the TPV of the teacher recruitment program and recruitment under Round 2 was not completed on time. However, the first round of school survey as part of the TPV of the teachers recruited under Round 1 found an absenteeism rate of 16% for these contract teachers compared to 21% among regular teachers in the same schools. On student absenteeism, rounds of school surveys by a third party firm were also not carried out, and thus the indicator could not be tracked. These surveys are now planned to be carried out in SERP II. 2.3.6 M&E Utilization: The M&E data was used by the RSU and the Bank and EU to inform assessment of the SERP and SEP and decision making. For example: • The M&E data on the results-based specific targets (DLIs) to be achieved for disbursements to take place was very helpful in keeping implementation of the reform actions on track, and/or in the case of delays or problems to discuss and agree on corrective actions to get back on track. For example, when the target for accurate data in the ASC (FY2010/11) was not met, the Bank, EU and the ED/RSU agreed on a remedial action plan for the ED to send a strong message to district education and school administrators of zero tolerance for misreporting of data starting with the ASC FY2011/12, and including a number of confidence building measures (Annex 2). • Data on the timeliness of delivery of textbooks and stipends for girls in public schools proved helpful in keeping such service delivery on track. • When due diligence by the Bank team during analysis and review of the payroll data (education staff) revealed that a significant share of staff was receiving salaries in cash through DDOs rather than as direct deposits into their personal bank accounts posing a potential risk of leakage, the Bank, ED, ERU/FD, and others in GoS worked to correct the practice (see paragraph 2.2.1 (f)). 2.3.7 Quality and reliability of data used in the ICR is generally good except for the questionable quality and reliability of the ASC data, particularly the transition rate (see paragraphs 2.3.4, 3.2.12 and 3.2.13). 2.4 Safeguard and Fiduciary Compliance Safeguard compliance is rated satisfactory on account of the following achievements/ issues: 2.4.1 Environmental aspect: The SERP overall was expected to result in environmental benefits. However, some project activities such as rehabilitation and/or upgrading of school buildings and addition of missing facilities under the district TOP program could lead to non- significant and minor negative environmental impacts. In early 2010, some of the recommendations from the reports on water quality, construction quality and natural hazards prepared by consultants as well a draft UNICEF report on water, sanitation and hygiene were incorporated into the designs and guidelines prepared by the third party construction supervision firm and provided to the district governments, and used in PC-1s and Standard Bidding Documents (SBDs) for the school rehabilitation initiative under the TOP program. 2.4.2 The environmental management benefits under the district TOP program were major. The area of significant accomplishment was in the school design and construction. The use of unframed structures was abandoned and replaced by the use of framed reinforced concrete structures (RCC), which are stronger and thus less vulnerable to earthquakes. To reduce the vulnerability to floods, measures such as raising school floors above known flood levels were also adopted. Pakistan has a long history of earthquakes and severe floods with thousands of fatalities and extensive damages (e.g., according to the Pakistan government, the death toll of the 2005 earthquake in Kashmir alone was more than 75,000, including thousands of school children). Improved designs and well-built structures might therefore save thousands of lives. Framed 10 structures allow the use of larger windows that resulted in well-lighted and better ventilated classrooms (without the use of electricity) that are more appealing to children, particularly in the hot summer months when the temperatures can be quite high. Boundary walls were built with proper heights, making it safer for children to attend school and to prevent vandalisms in off- school hours. Some variations in the construction quality and degree of completeness indicate the need of more strict construction supervision by district engineers, and by the third-party construction supervision firm. 2.4.3 The project also reached out to the private sector, distributed the Environmental and Social Management Framework (ESMF) to the contractors, and to entrepreneurs under the SEF’s PPRS program as well as held orientation workshops for the PPRS program entrepreneurs. The RSU did not recruit an Environmental Coordinator but ensured that the third party firm hired for construction supervision and validation of school rehabilitation works under the TOP program had an environmental specialist on the team. Overall, the above project activities remained in compliance with the ESMF, and the use of the firm to screen and supervise construction in the field was a significant positive step towards increased attention to environmental aspects. 2.4.4 Despite compliance, there were some challenges during implementation, including weak ownership of the ESMF particularly in the Public Works Department, and limited capacity building in districts for ESMF implementation. Overall, however, teachers and headmasters of the rehabilitated schools commented favorably on the quality of construction, concrete reinforced structures to protect buildings against earthquakes and floods, and boundary walls for safety. Planned environment enhancement pilots (i.e., to provide low-cost power from renewable sources to schools without electricity, and cost-effective interventions to remove contaminants such as arsenic and fluoride, and disinfect water to ensure safe drinking water facilities at schools) did not take off due to problems in hiring a firm, and will now be carried out under SERP II. 2.4.5 Social aspects: Land for school rehabilitation activities under the district TOP program was obtained via donations. School buildings under the PPRS program were either rented or owned by the entrepreneurs. 2.4.6 Successful implementation of several interventions in the SERP supported by the SEP is likely to have contributed to the significant increase of girls in schools in rural areas. The female- male ratio NER, rural Sindh, primary level, increased from 68% in 2007/08 to 72% in 2010/11 and in secondary schools in rural areas from 32% in 2007/08 to 40% in 2010/11 exceeding the targets for 2011/12. The PPRS program extended quality schooling to un/underserved communities in rural areas, targeting the worst-off districts. Rigorous evaluation results show that the program had a major impact on improving participation and achievement outcomes for rural children and for both girls and boys. The program drew nearly all young children into school (the average share of children going to school in communities with PPRS program schools is 90% versus 59% in control communities), and produced substantially higher achievement gains than other (principally GoS) schools that children attend in the evaluation communities. Children enrolled in school as a result of the PPRS program answered additional 14 and 9 questions correctly in the math and local-language tests, respectively, while children enrolled in school (principally GoS schools) answered additional 7 and 5 questions correctly in the math and local- language tests, respectively. 2.4.7 In addition to the PPRS program, key interventions that are likely to have contributed to the positive achievements were: (i) higher stipends to girls in low-transition rate (rural) talukas through DSP, (ii) gender sensitive teacher recruitment (additional marks to female candidates given in the objective scoring of the candidates), (iii) school rehabilitation through the district TOP program to bring infrastructure improvements including provision of toilets and boundary walls that were critical for girls, and (iv) incentives such as free and timely delivery textbooks 11 and stipends for girls enrolled in classes 6 to 10 in public schools that are likely to have influenced particularly poor households to send their children to school. The Bank’s impact evaluation of the DSP indicates that the initiative increased the retention of girls in secondary grade, thus contributing to higher student attainment. Fiduciary compliance is rated moderately satisfactory on account of the following achievements and issues: 2.4.8 Financial management (FM) rating remained moderately satisfactory through the project. With (i) the GoS’s demonstrated strong commitment to improving public financial management (PFM), (ii) the provision of PIFRA computer terminals to the ED and RSU for budget execution reporting and monitoring across the province, and (iii) the fact that education expenditures constitute a significant outlay in the GoS budget, timely, comprehensive and accurate expenditure reporting arrangements for the entire sector were already a key area of attention at both the RSU and the FD. The required staff for the FM unit in the RSU was already in place at the SEP start up. However, there were issues in retaining the full services of the FM staff. The internal audit specialist (IAS) had to dedicate his attention towards meeting the targets for resolution of the outstanding audit paragraphs, and could not make tangible recommendations for strengthening the internal controls in the education sector. After the IAS left the RSU in April 2011, it was decided to revise the Terms of Reference but the position could not be filled by the SEP closing date. All required financial reports were submitted; however there were delays in the submission of the interim unaudited financial reports. 2.4.9 A key achievement was the strengthening of the system of reviews by the Departmental Accounts Committee (DAC) and resolution of the cumulative backlog of advance audits paragraphs for the education sector per the agreed schedule, facilitated by the strong support from the Secretary of Education. This activity is likely to have fostered the accountability regime within the education sector. In April 2011, concerns arose concerning the payment of salaries of about 23,000 education sector employees in cash through the DDOs in the districts. Through the concerted efforts of the RSU and others in the GoS, the number of employees receiving salary in cash was reduced to 1,200 or from 10% to 0.5% of the education sector workforce. Steps are however yet to be taken for systematic process for reviewing and correcting the historical payroll data for education sector staff and ensuring its consistency with personnel records held by the line department, and for formally adopting the necessary measures to strengthen internal controls in this important area of expenditure. 2.4.10 Procurement: At the time of SEP preparation, a revised procurement law was already in place. The SERP and SEP supported through legal covenants and TA the implementation of an improved procurement regulatory and institutional framework for the province, and good practices in implementation of these procurement rules in the education sector. SERP provided a platform for continued engagement in the procurement reform dialogue with the Sindh Public Procurement Regulatory Agency (SPPRA) which led towards SPPRA’s notification of procurement rules and standard bidding documents (SBD) for goods and works; and standard Request for a Proposal (RFP) for consultancy services. SPPRA is in the process of developing some implementing regulations. The next steps identified by SPPRA are the development of sector specific SBDs, establishment of an M&E system for procurement performance, development of procurement competencies and e-procurement strategy. The GoS had already set up a system of quarterly updates on the Composite Schedule of Rates (CSR) but its robust implementation is yet to be seen. 2.4.11 Procurement of all EEPs and TA under the SEP and SEP AF was carried out in accordance with the Bank guidelines. The civil works contracts under the district TOP programs signed between districts and RSU for civil works were not an EEP, and hence they were required 12 to follow SPPRA rules. TOPs emphasized the application of SPPRA rules, use of bidding documents agreed by the Bank (till SPPRA SBDs are notified), and an ex-post procurement review to be commissioned by the RSU. The past practice had been that the contracts were awarded on school basis which resulted in thousands of scattered contracts, diluting the focus on quality of contractors, construction as well as contract management. For school rehabilitation works under the district TOP program, contracts were planned at the taluka level rather than at school level which consolidated contracts to a marketable size attracting good firms, making contract management more efficient, and the RSU hired a third party construction supervision firm to ensure compliance with the contractual stipulations. The ex-post reviews were also conducted by this construction supervision firm. Findings of ex-post procurement reviews provided a useful basis for discussion with the districts as well as private sector to improve upon the procurement practices. The project also provided a platform to discuss the textbook procurement (though not an EEP in SEP) and agree on actions to improve the procurement procedures (see paragraph 2.2.1 (f)). 2.4.12 Issues in procurement were mainly linked to continued capacity constraints. The RSU did not appoint staff conversant with consultancy selection and contract management, citing a dearth of such procurement specialists in the local market. The lack of procurement staff with relevant experience led to (i) delays in updating procurement plans, in getting the website operational, and in contract awards due to the several iterations necessary before the Bank could clear documents; (ii) decreased efficiency of consultancy assignments; and (iii) lags in deliverables. The procurement rating therefore remained moderately satisfactory throughout the project. 2.5 Post-completion Operation/Next Phase 2.5.1 Even before the SEP closed, the GoS had started design and early implementation of the second Sindh Education Sector Reform Program (SERP II), incorporating the lessons learned on challenges, constraints and opportunities from the SEP implementation and elsewhere, and building on the reforms initiated and carried out under the SEP. The GoS has asked for the Bank’s support for its SERP II through financing a proposed Second Sindh Education Sector Project (SEP II), scheduled for delivery in early 2013. The SERP II is being designed to carry forward some of the successful governance and accountability initiatives under SERP and to add in complementary governance and accountability initiatives that can maximize the reform dose felt and quickly yield positive results at the school level. To ensure the success of SERP II, the GoS also plans to strengthen implementation arrangements and coordination. SEP II design will be similar to that of the SEP, and include a results based component disbursing against achievement of DLIs and a well-defined TA component. Currently, the GoS is itself financing key consultancies to support SERP II implementation whose costs will be retroactively reimbursed from SEP II upon approval and effectiveness. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation 3.1.1 Objectives: The SEP objectives remain highly relevant to the Government of Pakistan’s development priorities articulated in PRSP II (see paragraph 1.1.7). The project objectives (to increase school participation, reduce gender and rural-urban disparities, increase progression, and improve the measurement of learning) were well connected to the education sector priorities of the PRSP II. 3.1.2 The SEP objectives also remained highly relevant and consistent with the Bank’s Country Partnership Strategy (CPS) for FY2010-13 which was later extended to FY2014 at the time of the CPS Progress Report (November 2011). Two of the four Pillars of the CPS emphasized (i) 13 improving access to education with a focus on equity to address regional and gender imbalances, school outcomes of female children, and quality of education including through improved systems to measure student learning, and strengthened governance and accountability of education service providers at all levels (Pillar 2); and (ii) strengthened public expenditures, financial and procurement management, and enhanced capacity and accountability in public sector management (Pillar 1). The SEP objectives and its reform activities as reflected in the DLI matrix were consistent with the above priorities of the CPS. 3.1.3 Although the Bank’s Education Sector Strategy 2020 came into effect after the SEP, yet the project development objectives were also relevant to the new sector strategy which emphasizes (i) “learning for all� (not just schooling but learning); (ii) quality, with learning gains as a key metric of quality, needs to be a focus of education investments (access alone is not enough); (iii) lowering the barriers that keep girls (and women) from attaining as much education as other population groups; (iv) strengthening education systems, namely, aligning governance, management of schools and teachers, financing rules, accountability mechanisms, and incentive mechanisms with the goal of learning for all; and (v) evidence-based policy making and innovations. 3.1.4 Design: The SEP design (Box 1 on page 2, Section 1.5 and Annex 2) reflected a proper diagnosis of the sector development priorities that have remained relevant throughout the life of the project. It also focused explicitly on the achievement of results (project components and implementation arrangements) which is consistent with both the Bank’s current CPS and the Government’s PRSP II priorities. The SEP’s results framework and the DLI matrix reflect the connections between the SEP objectives and the Bank’s CPS and the Government’s PRSP II. 3.1.5 Implementation: The Bank’s implementation support together with the DPs was and remained responsive to the government’s needs to achieving its education objectives and priorities (enunciated in the SERP and PRSP II), and to the Bank’s CPS for the country and the Education Sector Strategy 2020. The joint review missions throughout the project implementation engaged in high-level policy dialogue and conducted the necessary due diligence and provided continued technical advice as needed. The SEP objectives remained unchanged even at the time of SEP AF. 3.2 Achievement of Project Development Objectives 3.2.1 Achievement of the PDO was to be measured using four PDO-level results indicators (paragraph 1.2.2). Achievement of the DLIs for the key reform subprograms in SERP was also a key to the success of the program and achievement of the SEP’s project development objectives (Section 1.5 and Annex 2). In addition, several other indicators including the NER secondary level and female-male ratio NER, rural Sindh, secondary level were also a part of the results framework (PAD, page 35). 3.2.2 Effect of floods: Sindh was affected by the large scale floods and heavy rains of 2010 and 2011 (a natural event) that are likely to have significantly impacted the functioning and performance of the education sector. Each flooding event by itself caused extensive, major damage to school infrastructure and loss of school and student resources, disruption to school operations, economic losses for and the physical displacement of households, and the diversion of attention, efforts, and scarce funds of the GoS to disaster relief, recovery, and reconstruction activities. Two events in a row, with significant overlap in flood-affected districts, were crippling, likely resulting in a prolonged recovery period. The floods are likely to have affected the achievement of all PDO and other indicators including participation, transition, and completion rates, regardless of whether an indicator had a shortfall against the target or exceeded the target. Adverse effects of these floods on the results indicators are likely to be persistent for some time 14 to come and reflected in the future rounds of the PSLM survey and ASC data when they are released. PDO 1: Increase school participation. Indicators: (1) NER, primary education (5-9 years old, 1-5 grades) -- PDO outcome indicator, (2) NER, secondary education (10-14 years old, 6-10 grades) -- other key performance indicator (KPI). 3.2.3 SEP supported the design and implementation of several key demand and supply side reform initiatives under SERP that directly promoted school participation (see Annex 2): (i) PPRS program extended quality schooling to un/underserved communities. About 450 new schools were established with an enrollment of about 65,000 students; (ii) merit-based, objective, and transparent recruitment and placement of teachers in GoS schools with over 13,000 teachers locally recruited and placed in schools; (iii) school rehabilitation under the district TOP program to provide missing facilities (e.g., toilets and boundary walls) and buildings for shelterless schools, primarily in rural areas where the needs were the greatest. About 1,800 schools were rehabilitated and 250 schools were close to being completed by end FY2011/12; (iv) improvements in the delivery of stipends for girls in public secondary schools; cumulatively, a total of about 1.5 million girls in class 6 to 10 between 2008/09 and 2011/12 received stipends; (v) under a refined targeting (Differential Stipends Program or DSP), SEP provided higher stipends to secondary school girls in low transition rate talukas; cumulatively, a total of 346,000 girls received stipends under the DSP during 2007/08 and 2011/12; and (vi) improved delivery of textbooks at no cost to the students with over 90% of the books being delivered within the first month of school opening. 3.2.4 Improved infrastructure inputs increased accessibility to quality school facilities, particularly in disadvantaged areas (e.g., un/undeserved areas) and are likely to have brought school-age children into the schools that had remained out of school until now. Presence of qualified and locally recruited teachers led to improved teacher presence and better quality teaching. Reduced cost of schooling through the incentive programs (e.g., stipends and free textbooks, and their timely delivery) is likely to also have influenced households to send children to school, when they might not have done so otherwise, especially for poor families who often decide to keep some of the children out of school to assist in household work. Implementation performance of the incentive programs was substantially enhanced through strengthened delivery mechanisms and close monitoring, including by the TPVs. The above interventions were implemented satisfactorily. 3.2.5 The NER for primary education (5-9 years old, 1-5 grades) increased from 50.3% in 2006/07 to 51% in 2007/08, 54% in 2008/09 and 53% in 2010/11; value for 2009/10 is not available because the GoS did not gather comparable survey data that year (PSLM survey results for 2011/12 are also not yet released). The project achieved 51% of the target in 2010/11 (the target was 55.6% by 2011/12). While the above initiatives in SERP are likely to have improved participation, large scale floods and rains in 2010 & 2011 (see paragraph 3.2.2) – natural events -- are also likely to have resulted in a shortfall in the achievement of the NER-primary education against the target as the progress on the NER-primary that was on a positive incline in the first year of the SEP (2008/09) was likely to have been abruptly slowed (the primary-NER in 2010/11 was marginally poorer than in 2008/09). It is likely that the primary-NER value for 2011/12, when the 2011/12 PSLM survey statistics are released, would show no improvement or even a drop, as the province is still recovering from the devastating floods. 3.2.6 The NER for secondary education (10-14 years old, 6-10 grades) increased from 26.4% in 2006/07 to 27.5% in 2007/08, 30.2% in 2008/09 and 29.8% in 2010/11 (PSLM survey results). Though 227% of the target (27.9% in 2011/12) was achieved (or of the expected increase in the NER for secondary education) one year ahead of the target year, the NER-secondary was 15 marginally poorer than in 2008/09, in part due to the likely effects of the floods. Without the floods, the achievement could have been more. The targeted interventions supported by the SEP (stipends for secondary school girls, school rehabilitation including missing school facilities, focus on proactive recruitment of female teachers under the merit and needs based recruitment of teachers, and free textbooks) are likely to have contributed significantly to the improved NER for secondary education, offsetting the likely negative impact of the floods. 3.2.7 The achievement of the objective to increase school participation, primary education, is rated moderately satisfactory. While the full achievement is short of the targets, two factors warrant a moderately satisfactory rating: (i) secondary education NER target has already been significantly exceeded and is likely to exceed the target when the 2011/12 PSLM survey statistics are released; and (ii) the progress on the NER-primary that was on the positive incline until 2008/09 was likely to have been abruptly slowed by the two consecutive floods and heavy rains, a factor outside the GoS’s control. PDO 2: Reduce gender and rural-urban disparities in school participation. Indicators: (1) Female-male ratio NER, primary, rural, 5-9 years, 1-5 grades -- PDO outcome indicator, (2) Female-male ratio NER, secondary, rural, 10-14 years old, 6-10 grade -- other KPI. 3.2.8 The successful design and implementation of the SEP programs particularly the new innovative programs targeted to reach the most disadvantaged groups discussed in paragraphs 3.2.3 and 3.2.4 and Annex 2 resulted in a satisfactory achievement of the objective (PDO 2). The female-male ratio NER in primary education in rural Sindh where the gender equality has been the worst (5-9 years old, 1-5 grades) increased remarkably from 61.2% in 2006/07 to 71.9% in 2010/11 (target was 64.6% in 2011/12, the PSLM survey statistics for 2011/12 are not yet available); thus 315% of the target was achieved, one year ahead of the target year. In addition, the female-male ratio NER in secondary education in rural Sindh (10-14 years old, 6-10 grades) increased from 32.3% in 2006/07 to 40.3% in 2010/11 (target was 39.2% in 2011/12); thus 116% of the target was achieved, one year ahead of the target year. Achievement of the PDO by improving access and equity to the most disadvantaged, and through simultaneous focus on the quality of learning, is a key step towards reducing the gender and regional disparities. The targeted interventions supported by the SEP are likely to have contributed significantly to the improved female-male ratio NERs for both primary and secondary education, offsetting the likely negative impact of the floods. PDO 3: Increase progression. Indicators: (1) Primary school completion rate; and (2) Transition rate from class 5 to class 6, public schools, rural girls. 3.2.9 Initiatives under the SERP and SEP (e.g., placement of quality teachers in schools on merit and needs basis including additional marks to female candidates, teacher education development (TED), strengthening of district and sub-district education management for service delivery (EMR reforms), school rehabilitation with a particular focus on missing facilities (i.e. toilets and boundary walls), stipends for secondary school girls and higher stipends for secondary school girls in low transition rate talukas, the PPRS program, and strengthening of SMCs) were designed to promote greater school quality and better school management. The idea was that these improvements would in turn promote retention of students in school with a particular attention to girls, completion of primary schooling and progression to secondary education. The GoS satisfactorily met the DLIs for these initiatives including laying foundation for continuing the longer term TED and EMR reforms and the strengthening of SMCs (Annex 2). 3.2.10 The primary completion rate for all Sindh increased from 53.4% in 2006/07 to 58.3% in 2010/11 (PSLM survey data), exceeding the target of 55.2% for 2011/12 (a 272% achievement of 16 the target (or of the expected increase in the completion rate)). The PSLM survey statistics are of satisfactory quality and have been produced over long periods without any significant issues. 3.2.11 The target for transition rate (from class 5 to class 6, rural girls, public schools) is unmet. PAD had projected the transition rate to increase from 51.2% in 2007/08 to 63.4% in 2011/12 (a projected increase of 12.2 percentage points). In 2011/12, the GoS removed the rural/urban classification of schools due to changing rural/urban status on the ground, and hence the Bank’s Task Team could not track the original indicator. The new statistic is class 5-6 transition rate, public schools, girls in all districts excluding Karachi and Hyderabad which are predominantly urban districts. Using the same data but the new definition, the baseline value was 49.7% and the value of the statistic estimated at completion (2011/12) was 55% (an increase of 5.3 percentage points); the end-year target using the new definition would have been 61.9%. The original end- year target of 63.4% in the PAD is not applicable due to the definition change. 3.2.12 The ICR team feels that the poor achievement on the transition rate is very likely the result of estimating this statistic from administrative data whose reliability and quality at baseline is highly questionable. As discussed in paragraph 2.3.4, quality of ASC data is still a concern but relative to baseline data quality has shown improvements. Prior to 2009/10 when the first TPV was conducted, there was no independent assessment of the quality of enrollment data collected under Annual School Census, leading to highly incorrect data. The table below shows the Grade 5 and Grade 6 enrollment data for public schools in Sindh to illustrate the problem with the quality of the ASC data. Table 1: Enrollment in Sindh, Grades 5- 6, Public Schools, 2008/09-2010/11 2008-09 2009-10 2010-11 Grade 5 524,194 389,457 386,666 Grade 6 447,061 240,083 237,064 3.2.13 In 2009/10, with improvements in administration of school census (introduction of third party validation or TPV), more accurate estimates for enrollments of Grade 5 and Grade 6 were obtained. Also, data for secondary school enrollments had apparently improved by a greater margin than for primary school enrollments. Given the smaller proportion of secondary schools in the province (less than 20%) the probability of sampling a secondary school for TPV is higher, resulting in greater improvement in the quality of ASC data for secondary schools. Any comparison of ratios based on ASC data prior to 2009/10 and at project completion is therefore misleading. Hence the ICR team feels that owing to poor data quality the transition rate in hindsight should not have been included as a PDO level indicator. 3.2.14 The progress on the two indicators was mixed: (i) the achievement of the target for the primary completion rate for all Sindh was 272%; and (ii) the baseline value for transition rate was not applicable and hence the achievement of transition rate cannot be ascertained. Conservatively the achievement on PDO 3 is rated moderately unsatisfactory. PDO 4: Improve the measurement of learning Indicator: Learning levels monitored through diagnostic student assessments. 3.2.15 To improve the measurement of learning, the SEP supported the strengthening of PEACe as the agency with primary responsibility to carry out a programmatic cycle of diagnostic assessments in selected subjects in grades 4 and 8 in district representative samples of GoS schools. The PEACe completed the following assessments over the SEP period: (i) math test for grade 4 students (3,500 schools) in 2008/09; (ii) language test for grade 4 students (3,500 schools) in 2009/10; and (iii) follow-up math test for grade 4, science test for grade 4, and math 17 test for grade 8 in 2010/11 for a total sample of 65,920 students in 6,531 schools. In addition, PEACe conducted the social studies test for grade 4, follow-up language test for grade 4, and the language test for grade 8 in 2011/12. Analysis and reports of the grade 4 math and language assessments, completed in 2008/09 and 2009/10, respectively, were completed and findings discussed with key stakeholders (e.g., ED, STBB, Provincial Institute of Teacher Education (PITE) and RSU). Although not a part of the DLI, completion of the analysis of the 2010/11 follow-up math test in particular would have been helpful in monitoring the learning level in math; PEACe was constrained in analyzing the data by its limited capacity. PEACe needs further support to strengthen its capacity for test design, test administration, and analysis and for wide dissemination of the findings to key stakeholders and the public. Achievement of PDO 4 is rated moderately satisfactory. 3.2.16 The SERP also aimed at improving resource allocation and management for the education sector that cut across the entire sector reform program and thereby contribute to the achievement of the program’s development objectives. Though not strictly a PDO, the SEP performance of the activities in support of this pillar is rated satisfactory. Four key achievements were: (a) The GoS integrated the results based SERP into the MTFF, and the education budget was executed in alignment with the MTFF/MTBF although releases of some budget line items were delayed at times during the SEP. Fulfillment of DLI 1 (see Annex 2) related to the first thrust area -- education budget, including SERP expenditures, consistent with MTFF -- helped protect the financing of SERP and ensure effective use of public education expenditures during project implementation. (b) Financial accountability and transparency in the education sector was enhanced through the resolution of outstanding advance audit paragraphs. Actual achievements exceeded the targets: by April 2009, a reduction of 40% (target 35%) of 7,639 cumulative outstanding advance audit paragraphs since FY2005/06; by April 2010, a reduction of 63% (target 60%) of 3,854 residual advance audit paragraphs for FY2007/08; and by April 2011, a reduction of 64% (target 60%) of 3,917 residual advance audit paragraphs for FY2008/09. (c) Procurement regulatory and institutional framework and procurement management in the education sector were improved (see paragraph 2.4.10). (d) Improved procurement management and supervision of schools’ rehabilitation under the district TOP program was achieved (see paragraph 2.4.11). 3.3 Efficiency 3.3.1 Efficiency in achieving the project objectives in terms of net present value (NPV) is satisfactory. The details of the approach and assumptions are given in Annex 3. 3.3.2 A cost-benefit analysis using the present discounted value (PDV) method was carried out at appraisal. The current and future economic benefits and costs incurred for a representative child age 5 (the official age for entering primary school in grade 1 at appraisal) given a projected average school attainment profile when the child reaches age 18 were estimated for two cases: (1) without SERP, and (2) with SERP (see Annex 3). The difference between the two estimates provides the expected incremental net benefits per representative child age 5, and a positive expected incremental net benefit implies that the investment was worth pursuing at appraisal. 3.3.3 The PDV of expected economic benefits of primary and secondary schooling were derived by estimating real lifetime labor earnings based on age-earning profiles for 15-65 year olds. The PDV of expected economic costs comprised both direct (public expenditures on public and primary schooling) and indirect (forgone labor earnings of children age 10 and above) costs of attaining each school completion level. 18 3.3.4 At appraisal, the NPV of the SERP was estimated at PKR 33,150 per child using a discount rate of 5% (base case), and PKR 2,400 per child using a discount rate of 10% (sensitivity analysis). As noted in Annex 3, at completion, the ICR team estimates the NPV for SERP of PKR 130,500 per child using a discount rate of 5%, and PKR 22,800 per child using a discount rate of 10%. In both cases, estimated NPVs are higher. This shows the desirability of investment in SERP even at a high discount rate of 10%. The ICR team prefers the estimate with the 10% discount rate as, over the SEP period, interest rates stayed at 10% or more and the ICR team expects these rates to remain high due to poor current and future macroeconomic conditions, resulting in significant price increases. 3.4 Justification of Overall Outcome Rating Rating: Moderately Satisfactory 3.4.1 The overall outcome is rated moderately satisfactory. The PDOs and design remained highly relevant throughout project implementation and continue to remain relevant today. Efficiency of the project also remained satisfactory. The moderately satisfactory outcome rating reflects (i) substantial satisfactory achievement of the project’s objective to reduce gender and rural disparities (PDO 2), (ii) moderately satisfactory achievement of the objectives to increase school participation (PDO 1), and to improve the measurement of learning (PDO 4), and (iii) moderately unsatisfactory achievement of the objective to increase progression (PDO 3). Several other key indicators exceeded the targets (e.g., primary school completion, female-male ratio for secondary education, and NER for secondary education). Table 2 provides a summary of scores of different aspects of project outcomes, using a six point scale to serve as a check on the overall assessment of the outcome rating. 3.4.2 The achievement of the PDO and other indicators was however affected by the external shock of large scale floods and these effects are likely to persist for some time to come (see paragraph 3.2.2). Table 2: Project Outcome Evaluation Scores Weight Score Weighted score Relevance 0.33 Alignment with development strategy 0.07 6 0.42 Objectives 0.08 5 0.40 Design 0.08 5 0.40 Implementation 0.10 5 0.50 Efficacy 0.34 PDO 1 0.09 4 0.36 PDO 2 0.09 5 0.45 PDO 3 0.08 3 0.24 PDO 4 0.08 4 0.32 Efficiency 0.33 Benefits to costs 0.17 5 0.85 Internal efficiency 0.16 5 0.80 Total 1.00 4.74 1 = Highly Unsatisfactory, 2 = Unsatisfactory, 3 = Moderately Unsatisfactory, 4 = Moderately Satisfactory, 5 = Satisfactory, 6 = Highly Satisfactory 3.4.3 The GoS met, during the SEP implementation, all 33 DLI targets including the 3 DLI targets that were added during implementation arising from the GoS initiated extension of the initiatives under SERP to new reform areas. While in some cases, the project outputs also exceeded the DLI targets, only one DLI needed a significant adjustment (ASC in the M&E 19 subprogram). The project succeeded in supporting the adoption, refinements and implementation of some difficult policy reform actions under the SERP and the SEP. Three initiatives that yielded significant results at the service delivery level were (i) the merit and needs based recruitment of teachers (see Box 2), (ii) performance based grants to districts for school rehabilitation under the district TOP program (paragraphs 3.2.3 and 3.2.4; Annex 2, sub-program 2), and (iii) use of GoS funding to leverage the private sector to cost-effectively extend quality schooling to un/underserved communities in order to raise school participation and student achievement (paragraphs 3.2.3 and 3.2.4; Annex 2, sub-program 7). Foundation has also been laid for improved medium term education sector budgeting, and improved financial accountability and procurement policies and procedures. As a result of the project, the sector governance and accountability mechanisms have also been strengthened. Box 2: Merit-based, objective, and transparent recruitment & placement of teachers in GoS schools: Recruitment of teachers in Sindh has historically been patronage based, often without due consideration of whether the new hire was appropriately qualified and/or whether there was a genuine need at the school where the new hire was placed. In addition, academic and professional qualifications used to screen and select candidates are generally viewed in the country as a poor signal of candidate knowledge and ability in part because of concerns regarding the quality and integrity of diploma and certificate conferring institutions. This state of affairs potentially compromises teacher quality at entry and the alignment of teacher interest and efforts with desired duties and responsibilities over the teacher’s service period. The new recruitment policy under SERP delineated merit-based, objective, transparent, and standardized criteria for recruiting new teachers under school-specific, fixed-term (three-year) contracts. Under the new policy, (1) the candidate was required to pass a standardized, written knowledge test designed and administered by an independent testing agency; (2) female candidates were given additional marks in the objective scoring of the candidates; (3) the candidate was not subject to a selection interview (reducing the possibility for bias by eliminating room for discretion); (4) candidates were ranked using a composite score that summed up the test score and points for other relevant attributes and job offers were made starting from the top-ranked candidate and moving down on the list; (5) the qualifying candidate was to be placed at a school under a nontransferable, school-specific contract which implied that the teacher could not be transferred between schools; and (6) to the extent possible, the qualifying candidate for primary-school teaching service was placed locally (i.e., within the union council where the candidate had residence). Two rounds of recruitment and placement of about 13,000 teachers have been completed under SERP. In the second round, needs-based placement was also introduced where a qualifying candidate for primary- school teaching service would only be hired if an understaffed school was identified (a student-teacher ratio (STR) higher than a STR floor) in the qualifying candidate’s union council of residence. Descriptive evidence from a school sample survey conducted by a third-party firm indicates that the new teachers recruited in Round 1 had a higher presence rate than older co-teachers in the same schools. 3.4.4 Overall, key development outcomes have contributed to the improvements in the access, equity and quality of the education sector. These achievements are expected to have long-term benefits of improving education outcomes as the reform program and investments are sustained, with continued Bank support of the GoS’s SERP II program. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 3.5.1 Various elements of SERP aimed at improving equity by reducing relative female and rural disadvantages. The incentive programs -- the on-time and regular delivery of free textbooks to public schools (this incentive is likely to have had greater impact for the rural and poor), regular conditional cash transfers offered to girls as well as higher conditional cash transfers to girls attending public secondary schools in low transition-rate talukas (which are predominantly in rural areas) are likely to have helped increase enrollment by reducing the cost of schooling and providing incentives for households to send their children to school. The rehabilitation of public schools with deficient/poor quality infrastructure (again such schools are disproportionately 20 located in rural areas) under the district TOP program, the establishment of free private schools in un/ underserved rural communities, and recruitment of new teachers (with a bias at the margin towards female candidates) under merit and needs-based affirmative criteria and their local assignment under school-specific contracts are likely to have contributed to increased enrollment of children from poor and rural households. It is often assumed, at times documented, that girls’ school participation is more sensitive to distance to school, deficiencies in school infrastructure such as the lack of toilets or boundary walls (for a sense of security), and the lack of female teachers; the initiatives under SERP addressing these issues would likely have a larger impact on girls’ school participation than boys’. As a result, PDO 2 aimed at reducing gender and rural- urban disparities was achieved (see paragraph 3.2.8). (b) Institutional Change/Strengthening 3.5.2 The SEP implementation has resulted in significant institutional changes, and made progress towards strengthening capacity for the longer-term development of the education sector institutions, improving governance and institutionalizing accountability in education service delivery. Examples of the main areas of improvements as well as of the challenges are listed below. • Fiscal sustainability and effectiveness and accountability of public expenditures have been enhanced with the approval of the medium-term fiscal framework, financial management reforms including resolution of outstanding unresolved audit observations, and procurement reforms (see paragraph 3.2.16). • The project helped strengthen capacity of the ERU in the FD that assisted the education sector in budgets and releases, and effectively partnered with the RSU to improve the efficiency and effectiveness of public education expenditures. • Progress was made towards building capacity of the RSU which has become the primary agency in the ED responsible for education policy formulation, coordination, monitoring and evaluation of the efforts under the SERP, donor coordination, and implementation of selected activities. However, frequent changes in key staff of the RSU affected its performance (paragraph 5.2.4). • The Reform Steering Committee (RSC), composed of the highest ranking officials in the province and chaired by the Chief Secretary, to provide overall strategic guidance, interdepartmental coordination, and oversight support for SERP implementation could not meet regularly, mainly because of difficulties in convening meetings. • The SEP laid good foundation to strengthen the weak capacity at the district and school level to strengthen education management, resource management, service delivery and monitoring and evaluation. This is an ongoing process and more needs to be done. (c) Other Unintended Outcomes and Impacts (positive or negative) 3.5.3 Two positive outcomes are noted. First, the results based project design changed the nature of dialogue from focusing on inputs to focusing on outcomes because the timing and amount of disbursements were tied to the achievement of specified annual implementation progress and performance targets or DLIs. Second, triggered by the Bank’s due diligence, the concerted efforts of the RSU, FD and others in the GoS resulted in a sharp reduction in the payment of salaries in cash to education sector employees through DDOs at the district level to direct deposits into employees’ bank accounts, thereby reducing the significant risk of leakages. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable. 21 4. Assessment of Risk to Development Outcome Rating: Substantial 4.1 Key risks to development outcomes arise from the following: (i) continued political instability in the province that could reduce GoS commitment to the education reform program and lead to shifts in priorities; (ii) elections in 2013 and possibly a subsequent change in priorities and interests by the new GoS; (iii) weak economic management, volatile security situation, political uncertainty, and poor prospects can depress demand-side, and supply-side government investments in education; (iv) continuation of reforms in particular concerning improvements in the education system governance and strengthen accountability at all levels in order to improve school performance and service delivery are likely to remain challenging in a weak governance environment and resisted by status-quo proponents; and (v) despite progress in SEP, planning and execution capacity remains weak across the provincial departments, sub-departments and other entities of the ED and various administrative levels from the province to the school. However, the GoS’s request for continued Bank support for its SERP II through a second SEP makes continued delivery of improved education services likely. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 5.1.1 The Bank performance for ensuring quality at entry is rated satisfactory. As discussed earlier (Section 3.1), the project development objectives were strongly relevant and remained relevant throughout the project, and were aligned with the Bank’s CPS and the GoS’s priorities in the education sector. Building on the engagement with the GoS, established during the previous SEDPC in support of the SERP, the Bank team remained fully engaged with the GoS in the design of the SEP including active policy dialogue; continuity in the Bank’s efforts was facilitated by the same Task Team Leader for the SEDPC and for the SEP during its preparation and appraisal. Throughout preparation, the Bank team consulted various stakeholders at all levels in the province and districts, and took steps to reinforce ownership and active participation of the GoS in project design. In consultation with the GoS, the SEP used a results-based design with disbursements tied to the achievements of the agreed targets in the DLI matrix. 5.1.2 As noted in the PAD (pages 8-9) and in paragraph 2.1.2, the project design reflected the lessons learned not only from the GoS’s own implementation experience including from the SERP and SEDPC, and from Punjab which has been implementing its own education sector reforms since 2003, but also from the experience of other countries and regions that were implementing similar projects with similar instrument design, particularly in the LAC region. In addition, the Bank team took into account the findings and recommendations from a range of analytical and advisory work for Sindh as well as Pakistan, and broader international experience in selecting interventions in the SEP design to simultaneously address the issues of school access and quality of learning. 5.1.3 The SEP design also reinforced improvements in education system governance and accountability in all interventions, and paid attention to reducing gender and rural-urban disparities, and addressing social development aspects. The M&E arrangements were appropriate and well integrated in the project design (Section 2.3); however, in hindsight, in view of the concerns on the quality of the ASC data, it was premature to rely on data from the ASC for tracking the progress on transition rate. The SEP design paid adequate attention to the fiduciary assessment and arrangements to ensure compliance with the Bank’s relevant policies and procedures. All significant risks, including weak implementation capacity, were recognized and 22 mitigation measures formulated. In addition, the SEP AF further aimed to intensify important reforms (teacher rationalization across schools and preparation and management of school specific budgets). (b) Quality of Supervision Rating: Satisfactory 5.1.4 The Bank implementation support team was multi-disciplinary with a good skill mix with specialists in education, poverty reduction and economic management, financial management, procurement, social and environmental management, communications, and legal, complemented by international and national consultants. The continuity of this cross-sectoral team throughout the SEP starting from preparation to supervision was unusually remarkable and this contributed during project implementation to (i) the focus on development objectives, (ii) the effectiveness of the policy dialogue across the diverse reform programs with the higher level GoS officials and the implementing agencies, and (iii) a speedy resolution of the implementation issues as they arose. Close collaboration with GoS during preparation of the SEP and the SEP AF continued through implementation and the Bank team continued to provide the necessary technical advice on important policy and implementation issues. 5.1.5 The Bank team jointly with the EU and GoS reviewed project progress at least two times per year, thereby also reducing the transaction cost to the GoS. The Bank and EU teams collaborated actively throughout implementation aligning their technical inputs including the support from their respective TA components. Continuous contact over emails and phone and short missions with specific mission objectives mainly by Islamabad-based staff in between the main missions helped maintain continuous program support to the implementing agencies. During security incidents and resulting security restrictions on mission travel, the Bank team relied on virtual discussions and reviews, contracted consultancy firms and contracted individual consultants based in Karachi for project supervision and implementation support. However, visits to districts and schools were limited to those which could be visited within the day long trips due to the Bank’s security concerns; this gap was however compensated as needed by holding discussions with, for example, PITE/BoC including PEACe/STBB officials, district education officials, school teachers in Karachi as arranged by the RSU. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory 5.1.6 The Bank’s overall performance is rated satisfactory, considering the above satisfactory assessments in assuring quality at entry and quality of supervision. 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory 5.2.1 The GoS’s ownership of and commitment to the project and to its successful implementation to achieve development objectives was strong. The GoS’s attention to education sector financing for the SERP programs supported by the SEP and SEP AF, despite the floods and general efforts to address the effects of the floods, remained unabated. The GoS demonstrated its strong commitment to the project’s development objectives by instituting several politically difficult reforms such as the merit based, objective, and transparent recruitment and needs-based placement of teachers in GoS schools, and the pilot program using public funds to leverage the private sector to extend quality schooling to un/underserved communities particularly in rural areas. The GoS also supported a structured program of diagnostic assessments of student learning by PEACe. In addition to meeting the ten DLIs with three annual targets for each DLI (30 DLI targets in total), the GoS thought of new reforms and introduced them, leading to three new DLIs (with one annual target for each DLI) supported by the SEP AF. 23 5.2.2 There were however a few areas where performance was wanting: (i) the RSC did not meet as regularly as expected (paragraph 3.5.2), which at times delayed action on reforms such as in steering the submission and approval of the Sindh Teacher Education Development Authority (STEDA) Act (which was approved by the Assembly in May 2012 and is now waiting for the Governor’s assent), (ii) frequent changes in the Secretary of Education affected continuity and delayed implementation, (iii) coordination, collaboration and information sharing between the various departments of the ED and across the provincial GoS involved in the reform program outside of the Bank/EU joint review missions, and the internal impetus for collaboration were missing, (iv) provincial government (Planning and Development, FD and AG) pressing the districts to act would have helped ED/RSU to smoothen and accelerate implementation, and that support was sometimes wanting. These other entities carry more clout than ED/RSU, and (v) at times, funds were not released in full and on time. Some delays were due to bureaucratic delays, some due to ED/RSU issues in providing the needed information, and some because the message from government leadership to FD on release was missing. On the other hand, the GoS has remained committed to the education sector reforms and is preparing its SERP II and has requested Bank financing for the next project with design features similar to those of SEP. (b) Implementing Agency or Agencies Performance Rating: Satisfactory 5.2.3 The ED and RSU demonstrated strong commitment and ownership to achieving the SEP development objectives, and coordinated with the ED staff at the district and school level, and other relevant entities (SEF, STBB, PITE, Bureau of Curriculum (BOC), and PEACe). The RSU played a key role in the preparation and implementation of the reform programs supported by SEP. As noted in paragraph 5.2.1, the ED and RSU instituted politically difficult reform programs supported by the SEP. The RSU championed two new key reforms during implementation with the SEP AF support, and introduced another new reform program for school system consolidation; these reforms were not originally in SERP. The ED/RSU met all DLI targets and only one DLI target needed a significant adjustment (the ASC in the M&E subprogram in 2010/11). Satisfactory implementation of the reform program and meeting of the DLIs were significant achievements. The ED/RSU also kept program implementation from derailing despite shocks from floods. 5.2.4 The RSU had primary responsibility for conducting the ASC. Though the timeliness and reliability of data improved during the SEP, the agenda remained largely incomplete (paragraph 2.3.4). Other parts of the M&E system including the TPVs and impact evaluations worked satisfactorily to provide the required information for tracking progress of the DLIs and inform decision making and performance monitoring. RSU generally complied with covenants 1 and agreements and with the Bank’s fiduciary policies. The RSU also played a key role in coordinating with donors including the EU. Staffing however in the RSU remained inadequate in particular for procurement (consultancy services and contract management) and environment management, and frequent turnover of GoS officers working at the RSU affected implementation performance and oversight of the SEP. At times, payments in the ASC and student assessment program under PEACe were delayed. Available project TA resources should have been used 1 Covenants concerning the communication plan and its implementation, detailed inventory and investigation report of new schools without the Sindh Education Management Information System (SEMIS) codes, issuance of permanent, unique and fixed length SEMIS codes to existing government schools, and the display by all government schools of their school names and SEMIS codes following issued guidelines were only partially complied with; the reason being that the universe of government schools is yet to be fully determined by GoS. 24 more by ED/RSU to support implementation capacity of districts. The most successful example of support to the districts was the use of a third party construction supervision firm to support districts, contractors, and schools with the effective implementation of the school rehabilitation initiative. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory 5.2.5 The overall performance of the Borrower is rated satisfactory, considering the above assessments of the GoS (satisfactory) and the implementing agency performance (satisfactory) in carrying out most reforms including politically difficult reform programs, giving unabated attention to providing education sector financing, and keeping program implementation from derailing despite the external shocks from two large scale floods in 2010 and 2011. 6. Lessons Learned 6.1 Significant lessons learned from the SEP’s experience are listed below. • A results-based investment operation that disburses against DLIs with predefined implementation progress and performance targets can be a good vehicle to influence medium-term reforms and performance outcomes such as improved sector governance and increased accountability, and help change the nature of the dialogue to focus on development outcomes. There was a large amount of “learning by doing� from the design and implementation experience of this first results-based education sector operation (along with a similar project in Punjab in Pakistan) in IDA countries on the Bank side and by the GoS, and the ICR team expects to see positive long-term benefits of this approach as more Bank staff and developing countries become conversant with the approach. • Strong political commitment and ownership are critical for successfully steering a sector- wide reform program and for addressing governance and accountability constraints to effective service delivery (e.g., approval of the policy and successful implementation of the merit based recruitment of teachers and their needs-based placement following objective and transparent criteria required the GoS to challenge vested interests and achieve a significant break from the patronage based teacher recruitment in the past). • Continuous and intensive Bank engagement across the reform program and technical and advisory support (to the line department and other related core departments at all levels including program and deputy program managers) can help keep momentum and provide the needed support at critical points. Joint preparation and implementation review missions by the Bank, the EU and the GoS contributed to the effective project implementation and minimized the transaction costs to the GoS. • Simultaneous focus in SEP on inputs and reforms to improve the quality of learning is likely to be more beneficial in meeting the school participation objectives of attracting and retaining both boys and girls in school. • A realistic assessment of the capacity is important to inform the design and sequencing of the reform programs for optimal impact (e.g., the design of the subprogram for strengthening district education management for service delivery proved optimistic during implementation and an incremental approach may have resulted in a more effective implementation). • Accountability based public subsidies to private entrepreneurs to set up and operate primary schools can serve as a major instrument for the GoS to provide quality schooling to un/underserved rural communities. The success of the PPRS pilot in the SEP is an example of how proper incentives and the power of accountability can help in transforming system performance. • Regular diagnostic assessments for tracking student achievement are good initial steps towards improving student learning outcomes. However, to leverage the assessments to 25 achieve intended improvements requires capacity for test design, test administration and analysis, and wide dissemination of the findings to key stakeholders and the public. • Availability of timely and reliable data is essential for choosing the results indicators to measure the achievement of the development objectives. In SEP, the ASC could not provide reliable data to meaningfully measure progress on the transition rate. • Maintenance of the RSC was an essential step to provide high-level oversight and inter- departmental coordination but such an effort by itself could not aid oversight of the reform implementation progress with sufficient regularity and closer to the ground. The high level RSC however needed to be active and should have been followed by working groups at the provincial level to guide and help resolve critical issues and by formal coordination and oversight arrangements at lower levels. • Inadequate staffing of the implementing agency and frequent changes in critical areas such as procurement and contract management, environment management, monitoring officers, and deputy program managers can affect project implementation. • Successful implementation relies on district performance which remained wanting during the SERP. The RSU made significant efforts to support the districts and take a considerable load off the districts. The efforts to improve district performance will need to be continued in SERP II. In addition, apart from the RSC, strong coordination between departments for pushing forward the reforms is essential for successful implementation. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies 7.1.1 The Borrower’s assessment of the project is included in Annex 7. The ICR team’s comments on the observations of the Borrower are given below. • The GoS has observed that there were complaints from communities and SMCs that the amount of the grant to the SMCs (PKR 22,000) for a primary school was not enough for larger primary schools with five rooms, and that slabs in the flat amounts can be introduced. The ICR team shares the view that needs of the schools vary with size. The amount of the grants to the SMCs in the future under SERP II should however also take into account the design of the school specific budgets reform program, and arrangements and controls on the use of the funds. • On the GoS’s comment that the impact of the pilot capacity building program on the SMCs was less because selected schools were scattered in 8 districts and the impact would have been greater if all schools within the randomly selected communities in 8 districts or 5,000 schools have been selected within 2 districts only. The original design of the pilot was chosen jointly, and the ICR team agrees that in hindsight the pilot capacity building program would have likely had a greater impact if all schools (but limited to the desired number, i.e., 5,000) within a given village were chosen for the pilot. • The ICR team agrees that the assignment, by the construction supervision firm for the district TOP program, of two employees (one Assistant Resident Engineer and one Site Engineer) per district regardless of the size of the district is likely to have affected the effectiveness of the field supervision for the program in the largest districts. This and similar issues should be addressed in the proposed SEP II. • The GoS has observed that direct and closer interaction with the districts and schools was required to further tune-up the technical advice from the Bank. The ICR team feels that visits to districts and schools were limited to those which could be visited within the day long trips due to the Bank’s security concerns; this gap was however compensated as needed by holding discussions with, for example, PITE/BoC including PEACe/STBB officials, district education officials, school teachers in Karachi as arranged by the RSU. 26 • On the GoS’s comment that the Bank should have had a procurement specialist dedicated to SERP only for better facilitation of the reform areas, the ICR team feels that the arrangements for the Bank’s procurement support in place for the SEP (the Senior Procurement Specialist, based in the Country Office in Islamabad, participating twice a year in joint review missions to discuss procurement aspects, visiting Karachi as needed between the formal supervision missions to help resolve procurement issues, providing advice on phone and through emails, and conducting procurement training) were generally adequate. Adequate number of experienced procurement staff in the RSU would have also helped. • On the GoS’s comment that strengthening of the Bank’s Country Office in support of the SERP could have been more helpful, the ICR team feels that the staffing was adequate for providing implementation support to the SEP. As shown in Annex 4, a Senior Education Specialist, a Public Sector Management/Governance Specialist, and Senior FM and Procurement Specialists were based in the Bank’s Country Office in Islamabad, and a Consultant was based in Karachi. These staff together with members of the Task Team based in Washington DC supported the SEP implementation. 7.1.2 The RSU reviewed the ICR and concurred with the assessment. No substantive issues were raised (see Annex 7, page 57). The ICR team has taken into account the RSU’s comments in the ICR. No comments were received from the ED, FD and P&D. (b) Cofinanciers 7.2.1 The EU reviewed and concurred with the assessment (see Annex 8). No substantive issues were raised. (c) Other partners and stakeholders Not applicable. 27 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD million equivalent) Appraisal Actual/Latest Estimate Estimate Percentage (USD (USD of Components millions) millions) Appraisal Monitoring/SEMIS 5.19 0.28 5.43 Terms of Partnership Grants to district governments 147.66 108.42 73.43 School Management Committee Grants 68.07 33.41 49.08 Manager Training 4.60 0.15 3.21 Secondary School Girls' Stipends Program 27.85 42.17 151.44 Public-Private Partnership Grants 41.99 17.79 42.36 Textbooks 53.47 40.10 75.00 Employee related expenses of district governments - primary & secondary education (includes teachers & district education management) 1,814.48 1,666.30 91.83 Learning Assessment/PEACe 2.27 0.79 35.04 Technical Assistance 6.00 2.04 34.00 Total Project Cost 2,171.57 1,911.45 88.02 Total Financing Required 2,171.57 1,911.45 88.02 (b) Financing Appraisal Actual/Latest Type of Co- Estimate (USD Estimate (USD Percentage of Source of Funds financing millions) millions) Appraisal Borrower 1,823.12 1,514.31 83.06 IDA 300.00 352.97 117.66 EU Parallel 48.88 49.86 102.00 Total Financing 2,172.00 1,911.45 88.00 Notes: (1) USD million equivalent calculated using PKR or EUR exchange rate as at June 30 for each fiscal year. Discrepancies in figures may occur due to difference in exchange rate on actual day of disbursement. (2) EU financing estimate at appraisal calculated using EUR exchange rate as at April 30, 2009 (date of effective exchange rate in SEP PAD). TA amount includes TA disbursed before start of SEP duration (i.e. from March 2007 onwards). (3) Appraisal estimate of Borrower financing differs from the amount in the SEP PAD as the EU financing at appraisal had been included under Borrower financing, and now in this table for the ICR, the EU financing is shown separately in both the appraisal and actual/latest estimates. (4) The RSU considers the actual costs for SERP to be likely underestimated because the accuracy of data submitted by the districts through PIFRA in FY12 may have been affected due to the changes caused by the repeal of the 2001 Local Government Ordinance in FY12. 28 Annex 2. Outputs by Component Sindh Education Sector Project (SEP) and Sindh Education Sector Project Additional Financing (SEP AF) Disbursement Linked Indicators (DLIs) for SEP and SEP AF Overall Achievement: • All ten of the FY2008/09 DLIs in SEP were satisfactorily met by the GoS, including the Global DLI on merit and needs based teacher recruitment. • All ten of the FY2009/10 DLIs in SEP were satisfactorily met by the GoS, including the Global DLI on merit and needs based teacher recruitment; two DLIs -- fiscal and budget management and school management committees -- had lagged but were subsequently satisfactorily met in FY2010/11. • All ten of the FY2010/11 DLIs in SEP were satisfactorily met by the GoS, including the Global DLI on merit and needs based teacher recruitment; 2.5 DLIs -- fiscal and budget management, public- private partnership program under the SEF and Annual School Census (0.5 DLI) – had lagged beyond FY2010/11 but were satisfactorily met in FY2011/12. In the case of the Annual School Census DLI, GoS took remedial actions in FY2011/12 to improve the Annual School Census data quality for the FY2011/12 round. • All three of the DLIs for SEP AF were satisfactorily met by the GoS. Thrust Area 1: Improving Fiscal Sustainability and Effectiveness of Public Expenditures Subprogram 1: Fiscal and Budget Management – improving medium term education sector budgeting Indicator: Education budget, including SERP expenditures, consistent with MTFF. Status of targets for disbursement (DLI Targets) Comments • FY2008/09: SERP fully financed; education • DLI targets for disbursements met in each year sector strategy and results-based financing during the three program years albeit with some framework integrated in Sindh’s MTFF for delays. FY09/10 through FY11/12. • Financing for SERP was protected during project • FY2009/10: FY09/10 education expenditures implementation. were fully aligned with FY09/10 – 11/12 MTFF, and most expenditures for FY09/10 were made. • FY2010/11: FY10/11 education expenditures were fully aligned with FY10/11 – 12/13 MTBF; the FY2010/11 DLI (end target) was satisfactorily met after delayed release of the budget for the PPRS schools. Thrust Area 2: Improving Education Sector Management Subprogram 2: District Terms of Partnership (TOP) – improving efficiency and quality of school rehabilitation Indicator: Share of prioritized schools completed to meet stipulated design, construction quality and functionality standards. Status of targets for disbursement (DLI Targets) Comments • In FY2008/09, the list of prioritized schools for • The DLIs for the District TOP program were met rehabilitation was completed for TOP II. A third during the three program years. party, supervision firm was contracted to • Under this sub-program, GoS offered districts annual validate prioritized schools, supervise school grants for constructing (i) buildings for shelterless rehabilitation and ensure they meet minimum schools, (ii) additional classrooms in overcrowded standards for school design, construction quality schools, and (iii) basic missing facilities such as and functionality. toilets and boundary walls. The next year’s grant for • In FY2009/10, of the 1,008 schools screened in the program to a district was tied in part to the 29 by the third party for FY08/09 TOP (or TOP II), district’s performance in meeting construction and 437 schools (43%; target was 40%) were fully reporting indicators (as well as schooling indicators), completed and met the minimum construction with the share of the grant linked to district and school safety standards as certified by the performance increasing each year incrementally each third party. year. • In FY2010/11, 68% (target was 60%) of • Marked improvements in design, construction screened-in schools under TOP II (FY08/09) and quality and functionality standards as verified by the TOP III (FY09/10) were fully completed and third party construction supervision firm. Over 75% met minimum construction standards and school of TOP II schools were completed within 6 months safety standards as certified by the third party. for buildings of shelterless schools, and 4 months for • As of May 31, 2012, 80% of the schools under additional classrooms, toilets and boundary walls. TOP II and TOP III had been fully completed • Descriptive evidence indicates that the schools and certified by the third party. rehabilitated under this program experienced less damage than other schools in flood-affected areas, suggesting improved construction quality. Rehabilitated schools also often served as emergency relief shelters during the 2010 and 2011 floods, again presumably because the structures fared better in withstanding the floods and offering protection. • Further progress of works in FY10/11 and FY11/12 was affected principally by the dissolution of the local government system in 2011 and the withdrawal of committed funds from district accounts for the initiative, and by the floods in 2010 and 2011. Thrust Area 2: Improving Education Sector Management Subprogram 3: Monitoring and Evaluation – improving the timeliness and quality of Annual School Census (ASC). Indicators: (1) Number of months to complete administration of ASC. (2) Share of schools with less than 10% discrepancy in ASC data from validation data, based on sample validation. Status of targets for disbursement (DLI Targets) Comments • In FY2008/09, (1) ASC was completed within • 2.5 DLIs were met during three program years. For the agreed time of 5 months. (2) Data accuracy the unmet 0.5 DLI, following confidence building on key variables was 54.2% (target was 50%) measures were included in the remedial action plan based on third party validation. for the ED/RSU to send a strong message to district • In FY2009/10, (1) ASC was completed within education and school administrators of zero tolerance the agreed time of 4 months (November 1, 2009 for misreporting data starting with the ASC 2011-12 – February 28, 2010). (2) Extent of ASC: (i) disciplinary action against the district discrepancies in key variables was within education officers in the ten worst-performing tolerable bounds (50%) as validated by the third districts in terms of ASC 2010-11 data quality, (ii) party validation firm. detailed inquiries against officials in the districts • In FY2010/11, DLI (end target) was partially who misreported data in ASC FY2010-11, (iii) met by June 2011. (1) ASC was completed issuance of a letter from ED/RSU to all concerned within the agreed time of 4 months (between officials that strong action will be taken against non- December 28-30, 2010 and end-April 2011). (2) performing officials in ASC 2011-12, (iv) assurance Data accuracy on key variables was significantly from ED/RSU that the SEMIS officers will be less than the agreed threshold (55% against the inducted on merit and competency, and good officers target of 75%). This unmet 0.5 DLI was will be retained for a minimum of 3 years, and (v) considered met after ED/RSU implemented an commitment that financial payments for ASC 2011- agreed remedial action plan to improve data 12 will be cleared on time. Upon satisfactory 30 quality for the FY2011/12 ASC and onwards. implementation of these confidence building measures, the Bank agreed to declare the unmet 0.5 DLI for FY2010/11 as met, and make disbursement against the withheld 0.5 DLI in FY2011/12. • EU deployed a provincial coordinator and two district coordinators to provide day-to-day support for ASC/SEMIS activities, and helped develop an operational plan with detailed set of programmed activities. USAID also supported strengthening of the ASC activity through its ED-LINKS initiative. • Despite these efforts and progress (for example, prior to SEP, ASC FY2007-08 took 8 months, and there was no systematic sample based verification of data quality), the agenda of improving the ASC is not yet complete. Part of this could well be attributed to the capacity issues at the district and taluka level, poor record keeping in school registers, inadequate rigor in visits by supervisors to all schools to collect information, and failure of the RSU to provide the necessary hardware and infrastructure to district offices. • At times, payments in the ASC were also delayed. Thrust Area 2: Improving Education Sector Management Subprogram 4: District and Sub-district Education Management – strengthening district education management for service delivery Indicator: Implementation of Education Management Reform (EMR) to strengthen capacity and accountability of education managers. Status of targets for disbursement (DLI Targets) Comments • In FY2008/09, a baseline survey to assess • In total 1,653 clusters were formed and notified with education management was completed in 11 designated Guide/Head Teachers. The Sindh districts. Also, EMR implementation roadmap Education Sector Manager’s (SESM) course was was prepared. launched successfully at the Provincial Institute of • In FY2009/10, GoS selected and approved 3 Teacher Education (PITE), a mandatory course for districts for the pilot of the EMR program (target education managers up to Grade 17, with five was 2 districts). Draft job descriptions for and cohorts of SESM completed. The EU played an re-designation of officers to new EMR positions important role in supporting this initiative. were approved. Quality Assurance framework • The ED/RSU’s focus was on the introduction of and cluster based school management model cluster-based school management and school based were prepared and approved. The pilot EMR support, which were parts of a wider agenda districts also submitted draft district education envisaged by GoS for district EMR. development plans. • The FY2010/11 DLIs for SEP and for the SEP AF were satisfactorily met (the SEP AF required the extension of the Sub-program to at least 3 additional districts). The EMR program that was initiated in three pilot talukas in three districts was expanded to 11 additional talukas in 11additional districts. Thrust Area 2: Improving Education Sector Management Subprogram 5: School Management Committees (SMCs) – Strengthening SMC participation and capacity 31 for supporting school management. Indicators: (1) Share of public schools with active SMCs. (2) Number of schools in SMC pilot mobilization for capacity building in school improvement planning. Status of targets for disbursement (DLI Targets) Comments • In FY2008/09: (1) Data was received for 18,677 • The DLIs for the SMCs were fully met during the out of 33,100 schools. SMCs in these schools three program years. were declared ready for activation. (2) Pilot • SMC bank accounts have been reactivated in 93.8% SMC mobilization program was designed. of all main and functional public schools over • The FY2009/10 target was satisfactorily met program implementation. All these accounts were after a delay. (1) 71.1 % of the main and dormant in FY2007/08. functional government schools had activated • Revised SMC guidelines, posters and School SMCs as determined by school level information Improvement Plans (SIPs) were shared with all on credited accounts and by the receipt of local- 48,932schools in the Sindh province. language SMC guidelines (target was 60%). (2) Pilot SMC mobilization program was implemented in 2,500 schools for capacity building in school improvement planning. • The FY2010/11 target was satisfactorily met. (1) 81.4% of the main and functional government schools had activated SMCs as determined by school level information on credited accounts and by the receipt of local- language SMC guidelines, as validated using administrative data (target was 70%). A survey- based SMC grant expenditure review (917 schools) by a third party was also completed and a report submitted to the GoS. (2) SMC mobilization program was expanded to additional 2,520 schools for capacity building in school improvement planning in 8 districts following the agreed impact evaluation design. • Subsequent progress: In FY2011/12, 81.7% of main functional schools were provided SMC grants as validated using administrative data. Better designed guidelines, posters, and charts were disseminated to SMCs in all schools. Thrust Area 3: Improving Access to Quality Schooling Subprogram 6: Stipends for Girls in Public Schools – Strengthening the efficiency and effectiveness of incentives Indicators: (1) Share of beneficiary girls that received stipends. (2) Number of low transition-rate talukas (sub- districts) from Grade 5 to 6 with higher stipends (Differential Stipends Program – DSP). Status of targets for disbursement (DLI Targets) Comments • In FY 2008/09: (1) 90.3% of intended • The DLIs for the stipends program for girls in public beneficiaries received stipends before the close schools were fully met during the three program of FY2008/09 and (2) 30 low transition-rate years. talukas were in the DSP program. • According to GoS administrative data, in FY2010/11 • In FY2009/10: (1) over 90% of intended stipend MOs were issued to 409,376 beneficiaries. beneficiaries (provided in the Student Under the DSP program, 84,538 beneficiaries in 45 Attendance Pro Forma, SAP) in each district talukas received higher stipend amount (PKR 3,600), received stipends as validated by checking a consistent with the agreed impact evaluation design. 32 random sample of Money Order (MO) receipts. • 98.5% of students in district representative sample of (2) 30 low transition- rate talukas (selected intended beneficiaries in FY2011/12 have indicated consistent with an impact evaluation design) receipt of stipends under a third party survey. received the higher stipend amount (PKR • During FY2010/11, several key actions were taken to 3600/year) as validated by checking a random improve stipends administration including sample of MO receipts. identification and elimination of duplicate records in • In FY2010/11: (1) More than 90% of intended the SAP database (over 16,000 duplicate records), beneficiaries in each district received stipends as more systematic complaints management, and indicated by third-party survey of sample improved communication through newspapers and beneficiary schools and students. (2) The DSP SMS on stipend delivery schedule in advance of was introduced to 45 talukas (30 low transition delivery. rate talukas in FY2008/09, and 15 mid-transition • Stipends amounts were revised in 2011-12 from rate talukas in FY2010/11, consistent with the PKR 2,400 and PKR 3,600 per beneficiary student, agreed impact evaluation design). for regular and higher stipends to PKR 2,500 and PKR 3,500 to eliminate potential leakage from small denominations of cash. • The ED/RSU entered into an MoU with the Government Postal Office (GPO) for service standards for delivery of stipends including a maximum number of days for the delivery of stipends after their receipt by the GPO, and required the GPO to return the MO receipts to the RSU for their systematic storage. • Impact of the program on transition rates is expected to be small for secondary girls in urban areas, given already high secondary school participation rates. • Cumbersome and inefficient stipend delivery arrangements through the GPO. Thrust Area 3: Improving Access to Quality Schooling Subprogram 7: Public Private Partnership in Education – Leveraging the private sector to extend service delivery to underserved areas. Indicator: Number of private schools supported under fully-funded pilot public subsidy program in qualifying underserved rural locations (promoting Low-Cost Private Schooling in Rural Sindh – PPRS) Status of targets for disbursement (DLI Targets) Comments • In FY2008/09 funds released and 200 schools • The DLIs for support to promoting the PPRS selected following program and impact program were met during the SEP. evaluation design. • The number of operational PPRS schools however • In FY2009/10 the SEF offered subsidies to 199 has dropped in FY2012/13 to approximately 400 round I schools, with about 194 schools fully schools as contracts for schools that entered in the operational and 21 thousand children in latest round of entry and did not meet the unserved attendance. In round II, an additional 100 community criteria were cancelled. schools were selected (consistent with an impact • Experimental evidence of 200 (Round I) program evaluation design), contracted, and offered schools shows that in un/underserved communities initial entrepreneur and teacher orientation and which have received the PPRS program schools, training. student participation has risen by 30 percentage • In FY2010/11, the Sindh Education Foundation points to 90%. (SEF) added about 185 new schools to the • Even in the absence of subsidies being tied to test program. The overall target of at least 450 scores in program schools, gains in child operational schools in FY2010/11 was met with achievement (mathematics and local language tests) delay in FY2011-12. in program communities have been around two 33 • Monitoring, inspections, teacher training standard deviations. activities have been conducted. A pilot • Cost-effectiveness of the program in increasing assessment in a sample of schools has also been school participation is among the highest among all conducted. rigorously-evaluated programs in the developing world; the current cost of the program is roughly PKR 8,000 per student per year (all inclusive), and the cost per PPRS program student to induce an one percent increase in the share of children that go to school is a very low at PKR 265 (less than US$3) per year. • A shortcoming in the PPRS program was that regular testing of students and holding schools accountable for student achievement did not happen. Only a pilot test was conducted over the SEP period. Thrust Area 4: Improving the Quality of Teaching and Student Learning Subprogram 8: Merit Based School Specific Recruitment of Teachers – Improving the quality of new teacher hires Indicator: Implementation of merit and needs based teacher recruitment – Global DLI. Status of targets for disbursement (DLI Targets) Comments • In FY2008/09, 5271 new teachers (target was • The Global DLI for implementation of merit and 5,000) were recruited (Round 1) on merit and needs based teacher recruitment for improving the implementation was validated by third party. quality of new teacher hires was met during the three • The FY2009/10 target was satisfactorily met program years. with recruitment of Primary School Teachers • Teachers that enter are tested and have to score 60% (PST), Junior School Teachers (JST), and High or higher on a standardized, written knowledge test School Teachers (HST) under Round II in designed and administered by an independent testing compliance with key criteria of merit and needs agency, for further consideration. Female candidates based recruitment and placement following were given additional marks in the objective scoring transparent, explicit and objective criteria under of the candidates. The candidate was subsequently school specific and fixed term contracts. not subject to a selection interview in order to • In FY2010/11, appointment of approximately eliminate discretion, and job offers were made 8,000 teachers under Round II per the agreed starting from the top ranked candidate and moving policy was completed. down the list. The qualifying candidate was to be placed at a school under a nontransferable, school- specific contract (i.e., the teacher could not be transferred between schools), and to the extent possible, the qualifying candidate for primary-school teaching service was placed locally (i.e., within the union council where the candidate had residence). • First round of the school survey, as part of the third party validation of the teachers recruited under Round 1, found that the contract teachers had lower absenteeism rate of 16% compared to 21% among regular teachers in the same schools. • Round 1 teachers have been regularized under the Sindh Regularization of Teachers Appointment on Contract Basis Ordinance 2012 and the subsequent act passed by the Sindh Assembly. • The third round of merit and needs based teacher recruitment is current getting underway based on a 34 strengthened recruitment and placement system. Thrust Area 4: Improving the Quality of Teaching and Student Learning Subprogram 9: Teacher Education Development (TED) – Improving teacher training effectiveness and teacher quality Indicators: (1) Standards and competency based continuous development (CPD). (2) Quality assurance of CPD providers. (3) New diploma for certification of teachers. Status of targets for disbursement (DLI Targets) Comments • In FY2008/09, surveys of BoC, PITE and • The DLIs for TED were met. government teacher education institutions were • The trial of the CPD certification standards was completed, and an implementation roadmap of completed in three districts. teacher education development (TED) was • 4 CPD providers were provisionally accredited in prepared. 2010. The CPD framework has been piloted in three • The FY2009/10 target was satisfactorily met talukas of three districts with 90 schools covered but with delay. Teacher standard and under the pilot program. competencies were developed and approved by • Notification to cease all PTC and CT courses was the GoS and applied in the development of an issued in September 2011. Associate Diploma to be piloted in two • Reduction from 5 to 2 colleges for piloting the new districts. An Advisory Board for the Teacher diploma was judged by the Bank team as needed Education and Development Apex Body was from a feasibility point of view, and the change was established and became active; a budget was captured in the Bank’s Disbursement Letter for that drafted and under review. round of the DLIs. • The FY2010/11 target was considered to be • The ADE program was expanded to 12 teaching met as the CPD part of the three-part target education institutes of the province in 2011. was fully piloted but the certification of the • Rationalization of roles of BoC and PITE as CPD providers could not be formalized due to envisaged in the project document has not yet been the pending legal status of the Sindh Teacher realized. A committee, constituted in April 2012, to Education Development Authority (STEDA). rationalize the roles and responsibilities of all ED/RSU however implemented the stakeholders related to teacher education is to submit accreditation process with the guidance and its recommendations to the Board of Governors of support of the STEDA Advisory Board (now STEDA in the near future. referred to as the Board of Governors). The • Evaluation of standards under CPD framework has trial of the CPD certification standards was not yet been completed. completed in three districts. The Associate • STEDA Act was approved by the Assembly in May Degree in Education (ADE) was launched in 2012, and is now waiting for the Governor’s two GECE colleges with technical support approval. from a USAID supported project. • Fragmentation of teacher management tasks and weak coordination among the various education departments responsible for TED, including PITE, BoC, STEDA, and districts, continued to adversely impact this subprogram’s implementation. • ED’s capacity to implement the TED road map was limited and many of the administrative and management hurdles could not be overcome despite the ED’s efforts. Thrust Area 4: Improving the Quality of Teaching and Student Learning Subprogram 10: Learning Assessments of Students – Improving the regular measurement of learning. Indicators: (1) Subject and grade for fielding of diagnostic assessment in district representative sample of public schools. (2) Subject and grade for analysis report of diagnostic assessment. Status of targets for disbursement (DLI Targets) Comments 35 • In FY2008/09, district representative • The DLIs for learning assessments of students were mathematics test for grade 4 students was met. implemented in roughly 3,500 schools. • Technical capacity of PEACe remained limited with • The FY2009/10 targets were satisfactorily met. data analysis and test development outsourced to Class 4 language diagnostic test was consultants under the TA financed by the EU. Data administered in March 2010 in a district and from the assessments in 2011 is yet to be analyzed. rural/urban representative sample of GoS Since the expiry of the contracts of EU supported schools (about 3,500 schools). Data from the technical consultants, PEACe does not have the class 4 math diagnostic test conducted in April capacity for data analysis and for preparing reports. 2009 was analyzed and report produced. • Despite the successful completion of the • The FY2010/11 DLI (end target) was programmed cycle of diagnostic assessments over satisfactorily met by June 2011. Three large the SERP period, for tracking student achievement to scale student diagnostic assessments (grade 4 leverage the assessments to achieve intended science and follow-up math, and grade 8 math) improvements in learning, PEACe needs to further were administered in district representative develop its capacity for test design, test samples of schools over March 19–22, 2011. A administration and analysis and for wide total sample of 65,920 students in 6,351 schools dissemination of the findings to key stakeholders and was tested (14,460 students in the grade 8 math the public. Timely analysis of the assessments and test; 25,730 students in the grade 4 science and reports are important for monitoring learning follow-up math tests). Grade 4 language outcomes. assessment report from March 2010 test was • At times, payments in the student assessment produced program under PEACe were also delayed. • In February- March 2012, three large-scale student diagnostic assessments (grade 4 social studies and follow-up language, and grade 8 language) were successfully administered in district-representative samples of schools. Disbursement Linked Indicators added under the SEP Additional Financing (SEP AF) Additional Financing Sub-program 1: Teacher rationalization across schools and the formal allocation of teaching posts to schools Status of targets for disbursement (DLI Targets) Comments • The DLI target was met by June 2011. ED/RSU • The DLI was met. asked three districts to prepare salary budgets rationalizing the teaching force in each school (and to post teachers according to the rationalized number). To facilitate this exercise, ED/RSU developed a computer algorithm to assign appropriate numbers of teachers to schools in order to minimize the variation in the number of teachers across schools relative to the fixed median school student-teacher ratio in a given taluka, and provided the results to the districts permitting them to alter the number of teachers for up to 20% of schools. In practice, districts generally complied with the prescribed numbers. • Subsequent progress: All districts have prepared FY2012/13 school specific budgets (where the number of teachers and non-teaching staff in the 36 school are identified). Additional Financing Sub-program 2: Preparation and management of school-specific non-salary budgets Status of targets for disbursement (DLI Targets) Comments • The DLI (end target) was satisfactorily met by • The DLI was met. June 2011. School budgets were prepared by three districts (Naushero Feroz, Umerkot and Mirpurkhas) using stipulated MFSS guidelines (Minimum Funding Standards for Schools). These budgets were incorporated in the budget books for FY 2011/12 and FD released their matching grants for non-salary budgets for these districts in October 2011. • Subsequent Progress: All districts have prepared school-specific budgets (salary and non-salary) for FY 2012/13. The school-specific budgets have been included as an Annex in the FY 2012/13 budget book prepared by the FD. Additional Financing Sub-program 3: Extension of the district and sub-district education management reform initiative to additional districts Status of targets for disbursement (DLI Targets) Comments • Met. For the status of the target compliance and comments, see Thrust Area2, Subprogram 4. 37 Annex 3. Economic and Financial Analysis Approach at Appraisal The main objectives of SERP were to increase school enrollment, retention, transition between levels, and student learning. By having an impact on these indicators, it was hoped that the program would impact the probability of a child completing different school levels, and this in turn, would yield gains in labor earnings measured over the course of standard working life. The net effects of the program were estimated as the incremental benefits accruing to a representative child (private returns to schooling) as a result of the effects induced by the program. A cost-benefit analysis of SERP using the present discounted value (PDV) method was carried out at appraisal. The current and future economic benefits and costs incurred for a representative child age 5 (the official age for entering primary school in grade 1 at appraisal) given a projected average school attainment profile when the child reaches age 18 were estimated for two cases: (1) without SERP, where the projected average school attainment profile is identical to the average school attainment profile over a recent period, and (2) with SERP, where essentially the projected average school attainment profile was adjusted by raising the annualized growth rate in the probability of completing secondary school by 0.5%, 1% and 2% points for male-urban, female-urban, and female-rural population groups, while the negative forecasted annualized change without SERP for the male-rural population group was reduced to zero. The PDV of expected economic benefits of primary and secondary schooling were derived by estimating real lifetime labor earnings based on age-earning profiles for 15-65 year olds. The PDV of expected economic costs comprised both direct (public expenditures on public and primary schooling) and indirect (forgone labor earnings of children age 10 and above) costs of attaining each school completion level. Assumptions for Economic Analysis at Project Appraisal: • Urbanization rate: The forecasted annualized urbanization rate was assumed to be equal to the annualized population urbanization rate over the period 1998/99-2005/06, which stood at 1.8%. • Discount rate: The rate at which expected economic costs and benefits were discounted to the present period was assumed to be 5%. The Bank team used the trend in discount rates (State Bank of Pakistan) at the time of the appraisal to assess the appropriateness of the discount rate -- the rate had remained below 5% from 2001-2007. • Survival probabilities: Estimates of age group-specific adult mortality rates for women and men aged 15-49 years using Pakistan Demographic and Household Survey (2006-07) data were linearly extrapolated to impute survival rates for ages 50-64. • Unpaid family workers: The age-education level-earnings profile of unpaid workers was artificially constructed by applying a reduction factor of 75% to the corresponding profiles for self-employed workers. For children ages 10+ a reduction factor of 50% was applied to unpaid child workers. • Real annual earnings: The forecasted annual growth rates in real annual earnings in wage and self-employment were assumed to be equal to the annualized growth in real annual earnings in the respective employment type over the period 1998/99-2005/06 for workers ages 15-64. The growth rate for self-employment earnings was applied to the constructed earnings in unpaid employment. • Schooling length: For each school completion level, the length was assumed to be equal to the modal number of grades completed—for children ages 16+ the modal grades completed were 0, 5 38 and 10, corresponding to less than primary school completion, at most primary school completion, and at least secondary school completion, respectively. • Secondary school completion rates: See paragraph on the previous page under “Approach at appraisal.� Results of Economic Analysis at Appraisal A cost benefit analysis of the SERP, undertaken by using the PDV method, suggests that the program yields positive expected incremental net benefits under the base case as well as under most of the alternative cases considered as part of a sensitivity analysis. Specifically, under the base case the net present value of costs and benefits, discounted at 5%, was estimated at PKR 33,150 (2000-2001 PKR) per child at project appraisal. (In PKR at 2000-2001 prices) Cases Benefits Costs Net Benefits 5 % discount rate 39,842 6,695 33,147 10% discount rate 7,962 5,560 2,402 Reduced SERP effect on the forecasted growth rate in the probability of secondary school completion and 20,378 5,801 14,577 5% discount rate Reduced SERP effect on the forecasted growth rate in the probability of secondary school completion 4,299 4,852 -553 and10% discount rate Reduced forecasted real adult labor earnings growth 23,282 6,695 16,587 rates and 5% discount rate Reduced forecasted real adult labor earnings growth 5,845 5,560 285 rates and 10% discount rate Economic Re-evaluation and Scope of Data Inputs: The same model was re-estimated using the most recent databases available for post-completion evaluation by comparing them with and without project scenarios. Inputs: Data for the analysis, as was the case at appraisal, is largely obtained from province- representative household sample surveys. The present values of key variables used in the cost benefit analysis are estimated based on 2010-11 PSLM survey data which covered 19,622 households in Sindh. For estimating changes in key variables over the recent past, estimates from the above survey data were compared to estimates based on 2007/08 PSLM survey data which covered 3,765 households in Sindh. Time-series data on expenditures and students in the Sindh public education system are obtained from GoS sources. Assumptions: To keep the base parameters similar to those used in the PAD, the same basic assumptions for economic evaluation have been used, to the extent possible, as those used at appraisal. Key assumptions are summarized below. • Urbanization rate: The forecasted annualized urbanization rate is assumed to be equal to the annualized population urbanization rate over the period 2007/08-2010/11, which stands at 2.8%. • Discount rate: The rate at which expected economic benefits and costs are discounted to the present period is assumed to be 10%--Pakistan deposit interest rates adjusted for inflation have 39 remained above 10% over the period 2007-08 to 2010-11, and are expected to continue at this level over the long term. • Real annual earnings: The annual growth rate in real annual earnings in wage and self- employment is assumed to be equal to the annualized growth in real annual earnings in the respective employment type over the period 2005/06-2010/11 for workers ages 15-64. Growth rate for self-employment earnings was applied to the constructed earnings in unpaid employment. • Secondary school completion rates: With SERP annualized change in the probabilities of completing at least secondary school are 9.2%, 0.5%, 3%, and 2.4% points for the male-urban, female-urban, female-rural and male-rural population groups, respectively. • Survival probabilities, age-earning profiles for unpaid family workers, and length of schooling for each completion level follow identical assumptions as at appraisal. Re-evaluation Findings: As was the case at appraisal, the economic benefits of schooling are estimated by using adult labor earnings, and the economic costs of schooling are estimated by using schooling expenditures and, given the context, forgone child labor earnings. The present discounted values of benefits and costs are provided below. (In PKR at 2000-2001 prices) Cases Benefits (PDV) Costs (PDV) Net Benefits 10% discount rate (rate at re-evaluation) 25,491 2,689 22,802 5% discount rate (rate at appraisal) 132,897 2,383 130,514 Comparison: At appraisal, the incremental net present benefit of the SERP was estimated at PKR 33,150 per child using a discount rate of 5% (base case), and PKR 2,400 per child using a discount rate of 10% (sensitivity analysis). As noted in the table, the ICR team estimates an incremental net present benefit for SERP of PKR 130,500 per child using a discount rate of 5% and PKR 22,800 per child using a discount rate of 10%. In both cases, the estimates are higher than at appraisal. The ICR team prefers the estimate with the 10% discount rate as, over the SEP period, interest rates stayed at 10% or more and these rates are expected to remain high due to poor current and future macroeconomic conditions, resulting in significant price increases. 40 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Names Title Unit Lending Amna W. Mir Program Assistant SASHD Anwar Ali Bhatti Financial Analyst SACPK Benjamin Safran Junior Professional Associate SASED Bertha M. Mburugu Program Assistant LCSPS Chaohua Zhang Lead Social Development Specialist SASDS Chris Parel Consultant LCSPS Dhushyanth Raju Senior Economist SASED Ernesto Sanchez-Triana Lead Environmental Specialist SASDI Felipe Barrera Sr. Economist HDNED Furqan Ahmad Saleem Sr Financial Management Specialist AFTME Hanid Mukhtar Senior Economist SASEP Ismaila B. Ceesay Lead Financial Management Spec AFTME Luis C.P Miglino Consultant SASED Margaret Png Lead Counsel LEGLE Martin Serrano Counsel LEGES Michelle Riboud Consultant IEGPS Mohammad Omar Khalid Consultant SASDI Reema Nayar Sector Manager LCSHE Shaheen Malik Research Analyst SASEP Shahzad Sharjeel Senior Communications Officer SAREX Sofia Shakil Senior Education Specialist SASED Tahseen Sayed Khan Country Manager SACNP Uzma Sadaf Senior Procurement Specialist SARPS Zahid Hasnain Sr. Public Sector Specialist EASPW Supervision/ICR Alan Brian Norley Consultant SASED Amar Fateen Temporary SASED Amna W. Mir Program Assistant SASHD Anwar Ali Bhatti Financial Analyst SACPK Aarij Bashir Temporary SASED Ayesha Khan Consultant SASED Benjamin Safran Junior Professional Associate SASED Bertha M. Mburugu Program Assistant LCSPS Bilal Qureshi Temporary SASED Chaohua Zhang Lead Social Development Specialist SASDS Dhushyanth Raju Senior Economist SASED Elfreda Vincent Program Assistant SASHD Eliana Carranza Young Professional YPP Ernesto Sanchez-Triana Lead Environmental Specialist SASDI Hanid Mukhtar Senior Economist SASEP 41 Hayya Ahmed Consultant SASED Indhira Vanessa Santos Economist ECSH4 Ismaila B. Ceesay Lead Financial Management Spec AFTME Lorenza De Icaza Consultant SASED Luis C.P Miglino Consultant SASED Mariam Nusrat Adil Consultant SASED Michelle Riboud Consultant IEGPS Mohammad Khalid Khan Program Assistant SASHD Mosuf Ali Consultant SASED Quynh T. Nguyen Consultant SASED Saeeda Sabah Rashid Sr Financial Management Specialist SARFM Salman Asim Consultant SASED Shahzad Sharjeel Sr. Communications Officer SAREX Surendra Agarwal Consultant SASED Umbreen Arif Senior Education Specialist SASED Uzma Sadaf Senior Procurement Specialist SARPS Senior Public Sector Management Zubair K. Bhatti SASGP Specialist (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY08 39 180.01 FY09 99 408.40 Total: 138 588.41 Supervision/ICR FY 10 107 410.60 FY11 45 268.73 FY12 42 186.26 Total: 194 865.59 42 Annex 5. Beneficiary Survey Results Not Applicable 43 Annex 6. Stakeholder Workshop Report and Results Not Applicable 44 Annex 7. Summary of Borrower's ICR Program Objectives, Design, Implementation & Performance Parameters: Sindh Education Reform Program is a government initiative to bring-in qualitative changes in the Education sector. This Program is supported by the World Bank and European Union during the period June 2009 to June 2012. a. KEY PROGRAM OBJECTIVES OF SERP-1: The basic objectives of the program SERP are: • To increase school participation • Reduce gender and rural-urban disparities in school participation • Increase progression • Improve the measurement of student learning b. RESULTS: (Source: PSLM - 2007-08 & 2010-11 (i) 59% Increase in total budget allocation for education. (ii) Increase in Net Enrolment Rate (NER) in Primary, Elementary and Secondary from baseline 2007-08 is 51%, 18% and 11% to 52%, 19% and 11% respectively by 2010-2011. (iii) 3% increase in primary completion rate in government schools from 49% to 52%. (iv) A visible improvement in examination and assessment results. c. PROGRAM DESIGN: The program was designed in four pillars: 1. Improving Fiscal Sustainability and Effectiveness of Public Education Expenditures. 2. Improving Education Sector Management 3. Improving Access to Quality School in Rural Areas for Girls 4. Improving Quality of Teaching and Students Learning In order to monitor and assess the progress towards achieving these objectives the World Bank and the government agreed on a set of Disbursement Linked Indicators or DLI’s making the funding conditional to the achievement of these DLIs. The progress on quantifiable indicators was measured using independent evaluations by third parties besides government reports. At the end of the project, E&L Department successfully achieved all 33 disbursement linked indicators which was consequently appreciated & verified by the World Bank. However, progress towards achieving the objectives remained moderate. It should also be noted that during the period province of Sindh experienced natural calamity i.e. massive flooding twice, in consecutive years having caused serious damages to every field in Sindh. d. PERFORMANCE UNDER REFORM PROGRAM PILLARS The brief overview of performance of the reform program pillars is discussed separately along with the associated components. PILLAR-1: IMPROVING FISCAL SUSTAINABILITY AND EFFECTIVENESS OF PUBLIC EDUCATION EXPENDITURES. 1. FINANCIAL MANAGEMENT INFORMATION SYSTEM Objectives: Under the financial management objective the program was to improve fiscal sustainability and the effectiveness of public education expenditure. The following were the objectives of this component related to public finance: 45 i. Improved budgetary processes to introduce medium term result based budgeting, and preparation of medium term fiscal and budget framework. ii. Enhance internal control and financial accountability in the education sector through resolution of Audit Para. Achievements: Before SERP –I, there were 7,679 audit paras outstanding in the Education Department. During the FY 2009-10 RSU/ED was able to settle 2,413 audit paras against the target of 2,312. Thus it was able to settle overall 63% audit paras against the target of 60%. Likewise FY 2010-11 2,490 outstanding audit paras against the target of 2,350, were settled, an overall 64% settlement against the target of 60%. Previously around 23,000 employee of Education Department were drawing their salaries manually and there certain data error in payroll of education Department .During the SERP , RSU/Education Department was able to rectify errors in payroll data of Education Department in PIFRA system .Through greater collaboration with the Finance Department (FD),District Treasury Offices ,Accountant General Offices and PIFRA Directorate, RSU/Education Department was able to online salaries of around 21,000 education employees .Thus number of employees drawing their salaries through DDOs (manually ) was reduced from 23,000 to just 1,200 as on June 2012. This effort of education department was highly appreciated by the PIFRA Directorate and AG Offices at various forums. To further improve the internal controls and create awareness amongst the Drawing and Disbursement Officers (DDOs) of district education department an extensive capacity building workshops were conducted in 13 districts with the support of European Union (EU) and around 450 DDOs were trained. The School Specific Budgets under MTBF were prepared in 14 Districts during FY 2011-12 .The MTBF budget were prepared in 14 Districts as against target of 6 districts. Implementation & Operational Experience: Under SERP I, to improve budgetary processes, the Education Department has been preparing budgets under Medium Term Budgetary Framework (MTBF) since FY 2010-11. The budgets of all the provincial level organizations/units related to the education departments like Bureau of Curriculum (BOC), Sindh Text Book Board (STBB), Provincial Institute of Teacher Education (PITE) and other such sub- departments, including Reform Support Unit (RSU) and Sindh Education Reforms Program (SERP) are being prepared under Medium Term Budgetary Framework. SERP was to complement the overall Public Financial Management (PFM) to improve accountability for education sector expenditures. For that purpose, the World Bank desired better internal controls and financial accountability through settlement of advance audit paras. Government of Sindh wanted continued resolution of audit Paras for the sector, through continued improvements in internal controls and other preventive and systematic actions. PILLAR-2: IMPROVING EDUCATION SECTOR MANAGEMENT 2. MONITORING AND EVALUATION – ANNUAL SCHOOL CENSUS Objectives: • Conduct Annual Institutional Census & Producing Educational Statistics of Sindh Province. • Reliable & Timely Information to Educational Researchers, Planners, Administrators & Donor agencies by providing readymade data for effective Education Planning and Management. • Develop Coordination with donors & other organizations intervening with Education Sector for reform purpose. Achievements & Implementation: The ASC 2009-10 was conducted in due time and the DLI was met. The ASC 2010-11 was conducted in due time (4 months) as targeted. The following confidence building measures (CBM) were agreed to make a case for the release of unmet DLI. 1. A meeting by Senior Minister for Education over the TPV results & that was explained to all Concerned Officers: 46 2. A summary letter to get S. Minister's permission for the written test and time-bound appointment of DO SEMIS officers (3 years). A letter sent to AG from Secretary Education requesting timely payments for ASC this year and acknowledgment from AG office to that effect. 3. A standard 9 digit SEMIS code was awarded to all schools reported in ASC 2010-11, earlier the SEMIS code was not standardized. 4. All the districts were asked to display new fixed length and old SEMIS code with Name at the door. Almost all the districts have displayed the SEMIS codes and they have also provided evidence of certificate of all schools and snapshots of nearly 10% school. 5. A list of all new schools have been prepared and verified by the districts, all these schools will be awarded NEW SEMIS ID, as approved the Senior Minister for Education. Operational Experiences: ASC is the key intervention and is a complex process which requires lot of efforts. During the SERP following main challenges were faced 1. Shortage of Technical team at SEMIS/ RSU 2. Capacity of existing Government officers posted at SEMIS wing / RSU 3. Technical capacity of DO(SEMIS) at district level 4. Overall capacity at field level (Supervisors for data collection, ADO for data usage and data accuracy) 5. Data entry, Data Merging & Data Cleaning 6. Unavailability of EMIS/ reporting module 3. EDUCATION MANAGEMENT REFORM - EMR Objectives: The Government of Sindh (GoS) recognizes the strategic challenge of improving the quality of education service delivery both at district and sub-district levels. Towards this end, the GoS has taken a strategic initiative of introducing Education Management Reform program (EMR) in a phased approach, under the on-going Sindh Education Reforms Program (SERP), mainly supported by the European Union (EU) and the World Bank. The Design of Cluster Based Management Program under Education Management Reform Program. Sindh has more than 18,000 (approx. 40%) are single/two teacher schools at primary level. Achievements: Before the inception of program reforms, a baseline survey of 327 lead/guide schools was conducted to identify the current situation of those schools. On the basis of baseline survey findings a quality assurance model was developed to improve the overall teaching and learning environment of target school. Under EMR-QA model school-based support was provided to 327 guide/lead schools and some empirical evidences can be find which has helped in the overall improvement of teaching and learning environments of those target school. Moreover more than 118 education managers were provided institutionalized management course with the technical input of IBA-Sukkur. Operational and Implementation Experiences: The Cluster Based Management Model was piloted initially in Dadu, Mirpurkhas and Naushahro Feroze tehsil of respective districts. The baseline survey of all guide schools was conducted to identify issues and challenges of each guide schools. Before the end of pilot phase an evaluation study was conducted to gauge the improvement of each guide school. Expansion of EMR Model in Fourteen Tehsils (Dadu, Badin, Thatta, Mirpurkhas, Umerkot, Sanghar, Daur (Shaheed Benazirabad), Naushahro Feroze, FaizGanj (Khairpur), Larkana, Shikarpur, Kashmore, Jacobabad, Kamber Shahdadkot) was implemented. 4. TERMS OF PARTNERSHIP - ToP Objectives: The objectives of the Terms of the Partnership program were to rehabilitate existing schools of the Sindh Province at the primary, elementary/middle and secondary levels by providing them complete buildings (shelter less schools) and missing facilities such as Additional class room, toilet, boundary wall, water and electricity. 47 Achievements: An equal amount of 100(M) Rupees was given to each district for rehabilitating schools. Unlike that, the funds for TOP-II, TOP-III and TOP-IV were allocated on the basis of performance cum need criteria. In TOP-I phase, around 3000 schools were rehabilitated whereas around 2030 schools have been completed in TOP-II and TOP-III phase of SERP whereas civil work on TOP-IV phase has just started in a few districts of the Province. Numbers of Total Terms Of Total Schools Schools Schools Remaining Remarks Partnership completed Screened PC-Is No third party 3000 3000 TOP-I - at that time (roughly) (roughly) TOP - II 1927 1007 1003 4 TOP - III 1567 1025 780 245 (Karachi PC-Is are not TOP - IV 1335 603 0 603 available) Total: 4829 2635 1783 852 Assessment of Design & Implementation: For each district, two employees’ i.e. one ARE and one Site Engineer were placed by the Third Party Engineering Firm. The result of such equal based appointment was that site inspection was not done according to the size of the district. Hence quality was compromised by not having human resource in accordance with the geographical size of the district. Actually the screening of the schools did not happen keeping in view the future needs and the quality of the schools. In other words, the officials of the Third Party certified only surface-level things and those which were inside the boundary wall. The districts at the initial level did not accept the idea of involvement of third party in the engineering affairs. Consequently a visible friction was witnessed between AREs and XENs. There was a lacking on RSU part as well as it failed to send strong signals to districts whenever they were found lacking in the resolution of such and other their internal issues. Those contractors who opt for ADP related schemes did not learn much from the SERP concept of execution of schemes. If these contractors would have been able to get something from the SERP model of Schools Rehabilitation within the districts, in order to get even more better results in respect of quality of the schools. 5. SCHOOL MANAGEMENT COMMITTEE – SMC Objectives: • To improve enrollment and reduce dropout. • To increase and protect school assets. • To develop school improvement plan. • Effective and efficient utilization of funds • (i.e. government grant/community donation as per SMC guidelines) • To check the attendance of teachers and students. • To liaison with NGOs and Government offices for development of schools Achievements: Sindh has approximately 43,095 main & functional schools. Majority of bank’s accounts were dormant and closed due to long non disbursement of SMC funds. It was the major achievement of SMC team that 40,462 bank accounts were made operative by mutual cooperation of State Bank, Regional Banks, Districts and RSU. More than 5000 SMCs were trained by NGOs. Some NGOs also arranged the meetings of communities. All 49,000 schools got guidelines in easy version in Urdu and Sindhi languages. Two posters which were self-explanatory along with one SIP chart were also sent to every school. This was the intensive awareness campaign. Data was validated in the form of 121 Taluka Master 48 Directories (TMDs) having 0 to 5 versions in three years. A Third Party Validation - TPV was conducted in 2010-11 for expenditure tracking of SMC funds. RELEASED CREDITED FINANCIAL YEAR AMOUNT REL AMOUNT CREDITED SMC SMC IN RS. IN RS. 2009-10 32086 855,454,000 28869 765,444,000 20010-11 40249 1,078,188,000 34732 930,070,000 2011-12 40462 1,086,982,000 35211 947,340,000 Operational & Implementation Experience: Before SERP I, the SMC funds were stopped due to non-transparency of utilization of the funds and lack of capacity building of SMCs. Communities took it as positive step of Government, however various complaints have been lodged from communities and SMCs that the meager amount of Rs 22,000 for primary school was not enough for 5 roomed schools. Various slabs in flat rates can be introduced in this regard. The capacity building component of SMC also reflect less impact in the field due to the scattered schools in 8 districts. For intense impact, it may have been better to pick all schools within the randomly selected communities within 8 districts or 5000 schools have been selected within 2 districts only. Moreover, no secondary school was included in the sample. Female schools that need more capacity building were out of sample. 6. GIRL’S STIPEND PROGRAM Objectives: • Relieve parents from financial burden of educating their children • Provide an incentive for girls to remain in school • To increase and retain female participation in schools Achievements: Year No. of Schools No. of Beneficiaries Amount 2007-08 2229 259539 269,805,200 2008-09 2491 330941 363,626,800 2009-10 2752 380423 951,240,000 2010-11 2835 409376 1,083,948,000 2011-12 2865 415868 1,132,202,000 Assessment of Design & Implementation: It has been analyzed on the basis of last five years data that total girl’s enrollment has been increased from 3% to 6% and varies from year to year. Total enrolment was 259,539 in 2007 and 415,868 in 2011- 12. Despite increase in total enrolment transition rate is not sufficiently increased as per expectations, only 4 %to 5 %. There was significant year wise increase in beneficiaries and expenditure. The increased Enrollment in 45 DSP Talukas is impressive and doubled in last five years. It was 24,236 in 2007 and 51,448 in 2011-12. 49 PILLAR-4: IMPROVING QUALITY OF TEACHING AND STUDENTS LEARNING 7. MERIT BASED TEACHERS RECRUITMENT Objectives: i) To improve the quality of education by merit based recruitment. ii) To appreciate appointment of local teachers introducing methodology of school - UC specific appointment of Primary School Teachers (PSTs), and Taluka specific appointment of JSTs. iii) To ensure placement of local highly qualified / professional teachers. iv) To reduce gender disparity. Achievements: It was first time in the province of Sindh, the teachers were recruited on merit basis. For Induction of highly qualified and professional teachers in the stream, the method of gracing marks was introduced covering various aspects such as high education, professional qualification, locality and gender etc. Before start of the recruitment process, number of strategies was evolved at RSU level to facilitate other stakeholder’s, including development of tools / templates and other relevant documents to carry out. Implementation and operational experience: The whole program was implemented in very efficient manner to some extent despite some bitter experiences in some districts (Kamber Shahdadkot and Naushahro Feroze) where EDOEs violate the guidelines of the department to meet their illicit motives and earned bad name for the department. The officers were placed under suspension and enquiry was ordered against them by the department. The fate of the officers will be decided after final recommendations of the enquiry committee and decision of cases filed by the affected candidates against both officers. This paved way for meritorious candidates to lodge protest and push department to face litigation in the courts of law. This also caused delay in recruitment process in some districts. The entry of extra candidates in the stream as successful candidates also caused great problems to carry out smooth operation of the program. 8. TEACHERS EDUCATION AND DEVELOPMENT – CPD Objectives: Keeping in view the pre-SERP scenario, Government of Sindh decided to constitute a working group involving the experts and intellectuals to frame rationalized recommendations for addressing the core issues and challenges in enhancing the quality of teacher education in Sindh. The objectives of their task were to; i. Enhance quality of pre-service teacher training courses, ii. Make in-service teacher trainings effective for development of skills and competencies of the teachers, iii. Liaise in-service teacher trainings with promotions and career path of the teachers, iv. Strengthen the teacher training institutes in human, capital and physical characteristics , v. Rationalize overlapping roles of apex organizations i.e. PITE and BoC, vi. Design a mechanism for provision of equal opportunities of training for each teacher in Sindh. Achievements: • Rationalization of roles and responsibilities of the PITE and BoC: The roles of BoC and PITE have been rationalized and former performs the administrative role through its extension wing and the later, the role of Quality Assurance Agency. After 18th Amendment, the responsibility of administration of TEIs in Sindh needs to be rationalized • Introduction of a new Initial Teacher Education (ITE) program by phasing out PTC and CT courses: A two year Associate Degree in Education (ADE) program leading to 4 year B.Ed. Honors Degree, was introduced as Initial Teacher Education (ITE). 12 (twelve) TEIs offers this ITE program from session 2012-13. 50 • Introduction of Continuous Professional Development (CPD) framework: CPD framework is designed on accreditation and certification standards. It ensures that no sub-standard teacher development program would be offered for in-service trainings. • Establishment of an institutional home/ focal point: Sindh Teacher Education Development Authority (STEDA) has been established for strategically managing the whole domain of teacher education and development in the province. Operational and Implementation Experiment: The outcome of the working group was a Teacher Education Development (TED) policy which was approved in April, 2009 by the Government of Sindh. The policy mentions the proposals and its associated institutional arrangements by focusing on following four domains in teacher education in the province; i. Rationalization of roles and responsibilities of the PITE and BoC: The consultative meetings were done with both the organizations along with former heads of both the institutions to identify if they were performing their roles according to their essence of creation. It is hoped that all the interventions mentioned above would reflect both the organizations towards their original roles in near future. After clarifying roles of both the organizations, the responsibility of administration of TEIs in Sindh needs to be rationalized. ii. Introduction of a new Initial Teacher Education (ITE) program by phasing out PTC and CT courses: Higher Education Commission (HEC), in line with National Education Policy 2009, came out with an intervention in Initial Teacher Education (ITE) with an Associate Degree in Education (ADE) program leading to 4 year B.Ed. Honor’s degree. Government of Sindh, in consultation with World Bank, decided to adopt the intervention in the province. iii. Introduction of Continuous Professional Development (CPD) framework: CPD Framework was introduced for in-service training of the teachers and its relation to their career path. The framework provides equal and competitive opportunities to both public and private service providers to come up with standardized training programs. iv. Establishment of an institutional home/ focal point: The need of a regulatory body was emerged for strategically managing the whole domain of teacher education and development in the province. 9. PROVINCIAL EDUCATION ASSESSMENTS – PEACE Objectives: The main objectives of this portfolio were, • To assess student learning. • To obtain more detailed information regarding students’ learning in all districts of Sindh province. • To identify students’ strengths and weaknesses. • To assess the effectiveness of a particular instructional strategy. • To assess and recommend the effectiveness of curriculum programs. • To assess and improve teaching effectiveness. • To provide data that assist in decision making. • To communicate with and involve all stakeholders. Implementation and Achievements: • PEACE Sindh completed the 4 cycles of diagnostic assessment according to approved action plan SERP. 51 Implementation and Achievements: PEACE Sindh completed the 4 cycles of diagnostic assessment according to approved action plan SERP 2009 2010 2011 2012 Grade 4 Grade 4 Grade 4 Science Grade 4 Social Studies Mathematics Languages Grade 4 Mathematics Grade 4 Languages Grade 8 Mathematics Grade 8 Languages Details of Large Scale Testing conducted by PEACE Year Subject(s) Grade April, 2009 Mathematics IV March, 2010 Language IV March, 2011 Mathematics and Science IV (repeat) & VIII Feb- March, 2012 Language IV (repeat) & VIII Social Studies IV Assessment of Design & Implementation: PEACE, Sindh completed the SERP-I and produced two reports and broachers. The results of these tests show that there is a need for improvements in teacher training and textbook development. Analysis and reports of PEACE assessments of Mathematics and Languages in Grade IV for the financial year 2010 & 2011 could not be produced due to non-availability of local consultant and Data Analysis Expert. Following are the strategies through which targets can be achieved; • Working Groups to be functionalized to develop a mechanism for the concerned stakeholders to implement the finding of students learning achievement for improving the quality of education at school, district and provincial level. • Additional meetings with district officers, teachers and parents. • Conducting seminars, meetings and workshops to share the findings. 10. FREE TEXT BOOKS – FTB Objectives: • To increase the enrollment and overcome drop out ratio of school going children. • To attract the poor parents to send their children to school. (especially of rural areas) • To reduce the education cost. • To enhance literacy rate. Achievements: DETAILS OF TEXTBOOKS SETS DISTRIBUTED AMONG SCHOOLS BY APRIL 30TH Grades 2006-7 2007-8 2008-9 2009-10 2010-11 2011-12 Primary 3,060,825 3,703,380 3,703,380 3,949,194 3,935,508 3,485,967 Middle 446,006 679,339 679,339 674,415 645,974 677,456 Secondary 187,476 251,626 351,626 365,112 341,946 326,221 Total 3,694,307 4,634,345 4,734,315 4,988,721 4,923,428 4,489,444 52 Assessment of Design & Implementation: • Every year a cell comprising of three monitoring officers is established at RSU to examine the complaints received from the parents/teachers, in connection with text books delivery and to take remedial steps for redressal of their grievances, through different methods and monitoring tools. • As soon as the process of text books delivery ended, the monitoring teams as well as the DPM (Text) himself proceeded to different districts of the province to conduct random visit in connection to ensure the text books are delivered to real beneficiaries within due course of time. • The monitoring tools were developed at RSU before start of activity. • First time a Proforma was developed to assess number of undistributed books lying in different Taluka warehouses and STBB warehouses. • Due to effective and timely monitoring very small number of complaints were received at RSU from different corners. • Demand of books / Enrollment is obtained from the districts on prescribed Proforma to place order for printing of books with the Sindh Text Book Board. • The schedule regarding free text books distribution is issued to all districts of the province after due consultation with the Sindh Text Book Board. • The School Directory, School Dispatch Forms and Taluka Dispatch Forms are being printed at RSU about a month earlier before start of the activity. • At the time of delivery of Free Text Books the RSU assigns tasks of monitoring of the books distribution to its officers, who remain at the Warehouse until the last book is dispatched to the district concerned. Additional Financing: SCHOOL SPECIFIC BUDGET Objectives: Objective is to introduce School Specific Budget with school specific non-transferable posts across all districts to improve governance and accountability and ensure availability of adequate resources to each school. Achievements: 1. Policy of school specific budget approved and notified. 2. Software application prepared to support school specific budget 2011-12 preparation and training provided to districts government officials. 3. To satisfy School Specific Budget Disbursement Link Indicator 6 district governments prepared School specific budget. In addition to that 8 more districts prepared budget for the FY 2011-12 4. All 23 districts prepared school specific budget, approved and included in the budget book Volume-III for the financial year 2012-13. 5. School Specific Budget Utilization framework for non-salary approved and circulated to concerned quarters. Assessment of Design & Implementation: In the school specific budget management Finance Department, AG Sindh and District Account Offices have very crucial rules and they are likely to affect and be affected by the proposed increased of school level DDOs in budgets. Their better understanding of outcomes of the initiative was necessary to move in the right directions. System limitations and capacity issues in ERU-FD prevented creation of primary, middle and elementary schools DDOs in the system and making it extremely difficult to reach the program objectives particularly aligning the teaching resources with the school size and non-salary expenditure tracking. Teacher’s Rationalization: Objectives: Objective is to rationalize teachers by of fixing teaching post in schools budgets according to teacher rationalize policy of Education & Literacy Department. 53 Achievements: 1. Teacher’s rationalization incorporated in the school Specific Budget Policy. 2. To clear School specific budget Disbursement link indicator 6 district governments prepared School specific budget along with teaching post in primary schools. In addition to that 8 more districts prepared budget for the FY 2011-12. 3. All 23 districts prepared school specific budget with sanctioned post for primary schools, approved and included in the budget book Volume-III for the financial year 2012-13. Assessment of Design & Implementation: School specific budgets along with teaching post for high and higher secondary schools were incorporated in the Budget Books Volume-III as per prevailing practices. Since methods of teacher’s requirement assessment particularly with respect to subject wise teaching and data on subject wise teachers was not available, allocation of teaching post was not possible in post primary schools. Hence rationalized allocation of teaching posts restricted to primary schools only. PROMOTING PRIVATE SCHOOLING IN RURAL SINDH (inputs provided by the SEF) 1. Key Program Objectives: a. Promote long-term public-private partnerships through providing per child fee subsidy to private schools sector in order to create greater access to education in marginalized areas of Sindh. b. Enhance the quality of education and educational practices within classrooms thereby increasing student learning and achievement. c. Reduce gender disparity in education. 2. Achievements: 485 new private schools have been established in 10 districts of Sindh catering to the educational needs of approximately 72,000 students. The schools were established through local entrepreneurs selected through a comprehensive screening and selection process; these school operators were trained in school management and school leadership and have been provided regular financial, technical and administrative support. The entrepreneurs are provided funds to cover the school expenses; the amount of funds or student subsidy provided to each school is determined based on critical indicators like student headcount, overall performance and compliance to the key terms and conditions laid out in the partnership contract signed with the entrepreneurs. In addition to financial support, technical support geared towards educational quality enhancement at the school level included teachers’ professional development including district based training programs, refreshers and learning support, provision of learning material, free Oxford University Press Textbooks for each student, stationery and student bags, school monitoring and student assessment. Around 1300 teachers, mostly females, were hired from within the community to teach at these schools promoting employment and development opportunities for local women. Operational and Implementation Experiences: The challenges faced during project implementation included opening of new schools near government schools owing to non- compliance of third party research firms, which were primarily responsible for the survey, towards contractual obligations resulting in flawed site based verification. As a result around 70 schools had to be closed. Additionally, natural disasters i.e. flash floods and rains that hit the Province during the course of project implementation resulted in mass migrations of the communities, damages to the school infrastructure resulting in many schools becoming dysfunctional. Also, the project experiences in terms of partnership with entrepreneurs point towards the need for further strengthening the selection criteria of school entrepreneurs and introduction of academic qualification and relevant professional experience may be considered as some of the key conditions for selection. Some entrepreneurs associated with the program lack basic reading, writing and comprehension skills which cause huge problems vis-à- vis understanding the terms and conditions of the contract as well as school based implementation of project interventions. 54 Borrower’s Performance (a) Performance of Government & Implementation Agency: The role of government has been instrumental in implementation and getting the desired outcomes of the project. The E& L Department was successful in completing all the Disbursement Linked Indicators (DLIs) of the project including the Global DLI. It was not only successful in achieving the DLIs but was also successful in getting almost 100% disbursements against the allocated amount of the project. Moreover, it was also successful in getting the timely disbursement of all the funds under the project agreement. Under the project distribution of funds to the districts under Terms of Partnership (TOP) were made which helped the education department to allocate guaranteed funds to the District Education Department to provide missing facilities and improve the school infrastructure. This was verified through the third party validations conducted under TOP. The spending of the allocated funds was made under the set plan. However, the link between the Annual Development Plan (ADP) and TOP was missing and there is a larger need to build relationship between TOP and ADP to avoid duplications and get better results. The government was successful in the payment of Stipends to the Girls and differential stipends to some district in accordance with the project target .The veracity of the achievement has been checked through third party validations. Thus, it was able to achieve the broader objective of the project related to stipends. However, more needs to be focused on outcomes and improvement in payment methods is required to further improve and facilitate the distribution of funds . The School Management Committees funds were successfully distributed getting the set targeted results under the project. This has also been verified by the third party reports conducted in this regard. But the basic purpose of involving the community under SMC needs to be improved and more focused approach towards community participation and creating awareness amongst the community is needed. Another major achievement of the government has been allocation of funds for primary, middle and elementary schools along with high schools under non-salary component in every district of Sindh. Previously Primary, Middle and Elementary schools didn’t have any kind of non-salary budgets .The government has been quite successful in designing the policies towards allocation of funds under School Specific Budgets .There is a dire need for designing, implementation and monitoring framework for School Specific Budget which would get desired results and set outcomes. Under the Medium Term Budgetary Framework, Performance based budget for education were prepared in all districts of Sindh. Teacher Education mechanism has been significantly improved and a regulation and monitoring body has been established. Sindh Teachers Development Authority (STEDA) has been established through an Act of Parliament. The essential political will and commitment successfully carried out Merit based Teacher Recruitment. This policy precedent will be helpful to implement much needed hardcore reforms in phase of SERP. To improve the access, schools were established under Public Private Partnership the government was not only successful in establishment of schools but was also able to sustain the system. Under Provincial Education Assessment Center scheme quality assessment of students was conducted and the same was verified through third party validations. The governance issues were resolved under the Education Management Reforms (EMR) under the program. To improve the quality of management district education managers were provided training under Sindh Education School Management program. (b) Justification of Rating for Overall Borrower Performance: Rating: Satisfactory 55 The overall performance was satisfactory as all the DLI’s and Legal Covenants identified by World Bank were met in the timely manner. The Technical Assistance is a major source through which the Government of Sindh could achieve its objectives. It could improve the skills of the education department to obtain the required level of skills for the implementation of all the reforms effectively. It is needed that Technical Assistance must be utilized to its optimum level to achieve the desired outcome Bank’s Performance 1. Bank Performance in Ensuring Quality at Entry Rating: Satisfactory 2. Quality of Supervision Rating: Satisfactory 3. Justification of Rating for Overall Bank Performance Reform Support Unit kept close and continuous contact with the World Bank. The close coordination helped the implementation process significantly. However a strengthening resident mission of World Bank could have been more helpful. It is also felt that procurement section of World Bank should have been dedicated to SERP only for better facilitation of the key reform areas. The direct and closer interaction with the districts and schools is also required to further tune-up the technical advice that we receive from World Bank. As per the spirit of the Paris Declaration, European Union and World Bank joined hands. This step greatly reduced transaction cost for the Government. Institutional Arrangement for Sustainability of Reform Program: Next Phase of Sindh Education Reform: Successful implementation of first phase of Sindh Education Reform Program has provided insights and lessons that are critical for the next phase. E&L Department is in an iterative process to identify the reform areas. Key Legal and Policy Developments: After designing the SERP, some key legal and policy developments have taken place that has helped to identify the reform direction. 18th Amendment in the Constitution of Pakistan: The eighteenth amendment has changed the structure and requirements of education sector. The Sector stands fully devolved to province. The significant change in post-devolution scenario has made E&LD responsible to design and implement the curriculum. This change will have profound implications for the designing process of next phase of reform Compulsory Primary and Secondary Education: Also, 18th amendment has made education compulsory for children between the ages of 5 to 16 years. Provincial government is in process of drafting an Act to meet the constitutional requirement. It is expected that next phase of reform will revolve around this constitutional requirement. Identification of Reform Areas: The fast pace change in social, political and legal environment will require quick and constant adjustment of the reform design. Reform program of the government will remain an evolving enterprise. It is expected that thematic areas for next phase of reform will be carefully identified to meet the requirements. Government has initially identified reform areas on which threadbare discussion with the development partners, various stakeholders, district administration and provincial departments pertaining to different but relevant departments is on-going. 56 Borrower’s Comments on Draft ICR (received on December 17, 2012 from Saba Mushtaq, Deputy Program Manager, RSU) Dear Dhushyanth, Many thanks for this draft report - it is extremely comprehensive and detailed, with useful lessons learned for the Government of Sindh. We agree with the comments/assessment of the Project Development Objectives and assessment of PDO Indicator(s). We also agree with assessment of Intermediate Outcome Indicator(s) except that the following at page iv & ix need to be explained: Indicator 1- Fiscal and Budget management at page iv needs to be corrected as the payment in respect of ASC and assessment was not delayed in 2010-11& 2011-12, however, PPRS got release in 2011-12 with certain delays. Indicator 13 - Achievement by 227% at page ix and also referred at page-15, para 3.2.6 need to be substantiated with numbers for the common reader. Indicator 15 - Achievement by 272% at page ix and also referred at page-16, para 3.2.10 need to be substantiated with numbers for the common reader. We have reservations about the strong wording used about RSU under 2.2(f) at page-8 which could be revised to reflect the partial responsibility of the Bank also. This however, is compensated in para 5.2.3 at page-24 The last component promoting private school in rural Sindh needs to be kept at page 54 after teachers’ rationalization instead of page 56. We appreciate very positive responses/comments reflected in Article-7, page-26 of the ICR by the Bank on the issues raised by the Government of Sindh/Implementing Agencies and Partners. We do not have any comments on the annexes of ICR. These are the draft comments from RSU while the comments from ED, FD & P& D will be submitted on Dec. 20. Saba 57 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Comments received from the EU (Email from Ms. Wendy Fisher, Development Adviser, Education and HRD, Delegation of the European Union to Pakistan, Islamabad, December 13, 2012). Dear Dhushyanth Many thanks for this draft report - it is extremely comprehensive and detailed, with useful lessons learned for the GoS and the DPs to take forward for SERP II as well as for the wider education sector. We have no substantive comments. We look forward to continuing to work closely with you and the WB team in supporting the Government of Sindh in improving the education sector in Sindh. Wishing you happy holidays. Wendy 58 Annex 9. List of Supporting Documents Asim, Salman, Raju, Dhushyanth, Adil, Mariam, Pandey, Priyanka and De Icaza, Lorenza. (2012), [Impact of information awareness and community engagement campaign on school outcomes in Sindh, Pakistan], Baseline Survey: Raw Unpublished Data Asim Salman (2012), Financial and Economic Analysis Sindh Education Sector Project ICR Project File (the methodology was borrowed from Raju Dhushyanth (2008/09) Financial and Economic Analysis Sindh Education Sector Project) Barrera-Osorio, F. & Raju, D. (2012). The impacts of differential changes in benefit levels on female enrollment: Evidence from a gender-targeted CCT program in Pakistan. Draft. Barrera-Osorio, F., Blakeslee, D. S., Hoover, M., Linden, L. L., & Raju, D. (2011). Expanding Educational Opportunities in Remote Parts of the World: Evidence from a RCT of a Public-Private Partnership in Pakistan. European Union, Final Evaluation of Sindh Education Plan Support Program (SEPSP), May 21, 2012. European Union, Sindh Education Reform Technical Assistance (SERTA) Project Completion Report, 20 April 2012 Government of Pakistan, Finance Division (2009). Poverty Reduction Strategy Paper PRSP-II. Government of Pakistan, Finance Division (2012). PRSP-II Period Progress Report, FY09-FY11. Government of Pakistan, Pakistan Bureau of Statistics. Pakistan Social and Living Standards Measurement Surveys, 2006/07 – 2010/11. Horn, Robin S. (2009). Quality Enhancement Review Panel Report, Pakistan: Punjab Education Development Program and Sindh Education Sector Reform Project. Pakistan Floods 2010 – Preliminary Damage and Needs Assessment Report, November 2010. Pakistan Country Assistance Strategy, FY06-09, April 4, 2006 (Report No: 35718-PAK) Pakistan Country Partnership Strategy, FY10-13, July 30, 2010 (Report No: 53553-PK) Pakistan Country Partnership Strategy Progress Report, FY10-14, November 16, 2011 (Report No: 65286-PK) Procurement Post Review Reports: SERP (2011, 2012), Ernst & Young. Sindh Education Sector Project (SEP) Project Appraisal Document, May 6, 2009 (Report No: 47642-PK). SEP Additional Financing Project Paper, February 28, 2011 (Report No: 59138-PK). SEP Financing and Project Agreements, June 9, 2009 and May 19, 2011 (Additional Financing) SEP Implementation Status and Results Reports (Seq. 01 – 06) 59 SEP Aide-Memoires and letters to the Government (2008-2011) School Management Committees Expenditure Review Report 2011. Third Party Validation Report of Stipends for Girls in Public Schools (2010/2011) 60 60° 65° 70° 75° TAJIKISTAN CHINA PAKISTAN SINDH EDUCATION SECTOR PROJECT R. SINDH PROVINCE r na Ku Gilgit Chitral Indu s P R O J E C T P RO VI N CE R. S E LE C TE D CI T I E S HYB K HY ER BE R- N A TI O N A L CAPI T AL KHW P A K HT UN K A HWA M A I N R O ADS 35° R A I LR O A D S Mardan Approximate 35° P R O V I N C E BO U N DARI E S Line of Control Peshawar I N TE R N A T I O N AL BO U N DARI E S Kahat ISLAMABAD JAMMU Rawalpindi and KASHMIR Bannu Jhelum 0 100 200 300 KILOMETERS Mianwali Khushab R. Gujrat 0 50 100 150 200 MILES Sialkot Gujranwala Sargodha J h e lu m D.I.Khan Bhakkar Lahore Jhana Faisalabad R. Kasur AFGHANISTAN i Zhob P UN J AB Ra v R. Sahiwal 30° Quetta D.G. 30° Khan Multan R. j tle Bahawalnagar Su Sibi Bahawalpur Kalat BA L O C H I S T AN I N D I A Rahimyar Khan ISLAMIC REP. R. Khuzdar Sukkur 75° Rohri s OF IRAN du Canal In Larkana . UZBEK. TAJIKISTAN lR 65° 70° Panjgur TAJIK. CHINA Na TURKMENISTAN Dokri N a ra 35° 25° 35° Bela Nawabshah • • A •• •• • • • HW • • • • • •••• • Approx. Line • •• NK 25° •• TU AFGHANISTAN of Control • • KH •• • JAMMU -PA ISLAMABAD •• • AND ER • KASHMIR • • • YB KH Hyderabad Gwadar SIN D H 30° PUNJAB 30° Karachi Thatta PAKISTAN Badin ARABIAN SEA ISLAMIC BALOCHISTAN REP. OF IRAN INDIA This map was produced by the Map Design Unit of The World Bank. The boundaries, colors, denominations and any other information shown DECEMBER 2012 SINDH IBRD 36829R on this map do not imply, on the part of The World Bank Group, any 25° 25° judgment on the legal status of any territory, or any endorsement or acceptance of such boundaries. Arabian Sea 60° 65° 70° 65° 70° 75°