The Third Punjab Education Sector Project (PESP III) Mid-Term Review Mission April 23-30, 2019 I. INTRODUCTION 1. A mid-term review (MTR) mission was conducted on April 23rd-30th, 2019. During the review a World Bank mission1 met with officials from the School Education Department (SED) in Lahore to review implementation progress for the Third Punjab Education Sector Project (PESP III). The objec- tives of the mission were the following: (i) Review the overall progress over the life of the project with a view to assess the achievements in relation to overall strategic and higher-level objectives of the project; (ii) To review progress towards the achievement of project Disbursement Linked Indicators (DLIs); and (iii) to review and update the technical assistance requirements for the Project. 2. The task team met with the Secretary School Education Department, Punjab Examination Com- mission (PEC), the Quaid-e-Azam Academy for Educational Development (QAED) (formerly known as the Directorate for Staff Development), the Punjab Education Foundation (PEF), the Program Mon- itoring and Implementation Unit (PMIU), Cambridge Education (TA Firm), and Punjab Information and Technology Board (PITB). The task team would like to thank the SED and PMIU for arranging the meetings and to the officials for giving the mission valuable time for the meetings. This Aide Memoire summarizes the team’s main findings, recommendations and agreements reached with the Government of Punjab (GoPunjab). II. PROJECT DATA AND RATINGS Table 1: Key Project Data and Ratings Project Data USD (millions) Original Project Amount $300 Total Disbursement $115.62 Disbursement in FY 19 $.72 Closing Date December 2021 Project Ratings: Previous Current Project Development Objective MS MS Implementation Progress MS MS 1 Tazeen Fasih (Task Team Leader); Koen Geven (Co-Task Team Leader); Shahram Paksima (Senior Education Specialist), Akram El-Shorbagi (Senior Financial Management Specialist), Babar Naseem (Social Safeguards Specialist); Khalid Bin Anjum (Senior Procurement Specialist), Ella Humphrey, Ali Ansari, Abdal Mufti, Huma Mirza (Consultants). Component 1: MS MS Component 2: MS Project Management S Procurement S Financial Management S Ratings: HS=Highly Satisfactory; S=Satisfactory; MS= Moderately Satisfactory; MU= Moderately Unsatisfactory; U=Unsatisfactory; HU=Highly Unsatisfactory; NA=Not Applicable; NR=Not Rated. III. IMPLEMENTATION PROGRESS AND KEY FINDINGS 3. PDO and indicators. As outlined in the PAD and table 2 below, the Pakistan Social and Living Standards Measurement Surveys (PSLM) are to be used to measure the levels of school participation for the project. However, a decision has been taken at the Federal level that the PSLM will no longer be conducted, and instead provinces are expected to conduct regular household surveys in the form of the Multiple Index Cluster Surveys (MICS). Therefore, the Project will report school participation rates using the MICS be- cause this is the only regular, government administered, large-scale household survey that is available, and because the Government of Punjab itself utilizes this data to track progress against key indicators in the education sector. The MICS 2018 report was recently released and shows significant increases in school participation since 2014 when the last round was conducted. Once the raw data for the 2018 MICS is available, the task team will work with the PMIU to calculate the relevant participation rates using this data. The baseline measure of teaching-learning practices has been proposed using data from the recently con- ducted Service Delivery Indicator (SDI) survey as well as the proposed target for the end of the project which is a 0.25 standard deviation improvement over the baseline score. A baseline measure for the level of school readiness is yet to be established, and it proposed that this indicator be measured as the share of children ages 3 to 5, that are enrolled in Early Childhood Education (ECE) classes that meet quality stand- ards as prescribed in the ECE policy. The rationale for using this indicator as a measure of school readiness is that it is closely aligned to the ECE intervention supported through the Project, it serves as a measure of quality beyond simple enrollment in ECE and, it relies on data that is already being collected as part of the government’s own monitoring of ECE. Table 2. PDO level Indicators Indicator Name Data Source Baseline Current Value End Target (2020-2021) School Participa- Pakistan Social 75.8 -- 79 tion Rate (ages 6- and Living Stand- 15) ards Measurement Surveys (PSLM) Completion rate, Annual School 67 -- 70 Grade 5 Census Data Quality score of Independent Third Proposed: -- Proposed: primary-teaching Party (Baseline to 2.53 2.65 learning practices be established through SABER exercise) Level of school Independent Third Proposed: -- Improvement over readiness Party (Baseline to Share of students baseline be established by ages 3-5 enrolled in firm undertaking ECE classes meet- ing quality stand- research the ELP ards systems research grant) 4. DLI 1 Private school vouchers- Strengthening program design to expand coverage of tuition-replace- ment vouchers to children from disadvantaged households. 130,000 Eligible children received vouchers to attend private schools. Main Findings: This DLI is likely to be met. Currently, there is a total of 428,000 children enrolled in the education voucher scheme (EVS) which is 32,000 less than the target for this year.2 PEF plans on enrolling an additional 50,000 students in the EVS to meet the DLI target for this year and it is expected that a total of 470,000 voucher students will be attending school by the end of this fiscal year. Regular monitoring of PEF partner schools is ongoing, and Quality Assurance Tests (QATs) are also being administered. The mission discussed the voucher amount and whether it remains competitive for school operators. At the moment, PEF is of the view that the voucher amount is competitive; however, it is very likely that in a year or two, an increase in the amount may be necessary to ensure it remains competitive. PEF has now also been tasked with conducting QATs for Public School Support Program (PSSP) schools as well as Daanish Schools and this will place an additional strain on PEF’s human resources (HR). Given the large scale at which PEF is already operating the mission notes that there is a need for additional HR particularly to support with the development and administration of the QAT as well as to further strengthen PEF’s M&E systems.3 Under the TA component for the Project, PEF will receive support from a psycho- metrician to build PEF capacity to work on strengthening the quality of student assessments which will support the quality assurance system for the EVS, NSP and FAS programs. This expert is expected to arrive in Lahore by April 30th, and will work with the ADU team over a period of 6 weeks as part of this work. The Academic Development Unit (ADU) team will work with the expert to create a detailed schedule to optimize the use of time. PEF is still in the process of procuring the ERP software which will link PEF’s MIS system with its Financial Management systems to increase administrative efficiency. The financial bids for this software need to be reviewed and scored by the PMIU, and this activity should be completed at the soonest as it is significantly delayed. 5. DLI 2. Public-private partnerships in education. Leveraging the private sector to support schools to increase enrollment. 540,000 children are enrolled through Public-Private Partnerships under PEF ap- proved programs that have a Strengthened Quality Assurance System. Main Findings. This DLI is likely to be met. Last year PEF had around 2.1 million students enrolled in FAS and NSP schools across Punjab. Although there is no additional enrollment taking place this year due to financial constraints at the provin- cial level, the enrollment targets for this year’s DLI have been met, with overall enrolment in FAS and NSP programs just over 2.22 million. Enrollment in FAS is 1.93m and enrolment in NSP schools is 0.29m. Enrolment under the Public School Support Program (PSSP) stands at 625,829. PSSP has been restructured and absorbed by a new body termed PEIMA which is independent of PEF with its own administrative body. 2 The targets for the DLI are above and beyond the baseline enrollment in the EVS 3 At present there is a total of 52 vacant positions Adequate financing for the expansion of PEF programs remains a concern and during the implementation support mission in November, it was expected that PEF’s budget might be reduced to PKR 12.4 Billion; however PEF has received PKR 17 billion for this year. PEF has requested PKR 23 billion for next year and has received confirmation from SED that this amount will be allocated. The reduced budget for this year coupled with the relocation of the PEF head office, has meant that the results for the QAT have been delayed this year. In order to achieve the targets for subsequent years, it is essential that PEF receives adequate financing over the coming years. Even though PEF is not currently enrolling additional children in the FAS, and NSP programs this year, in expectation of expansion of PEF programs next year, school owners have enrolled additional children at their own expense. Another concern is that despite the com- mendable achievements of PEF and the SED with respect to enrolling children at such a large scale through PPPs with strong quality assurance measures, the education sector plan and the New Deal do not provide sufficient clarity on the role of PEF as well as its targets moving forward. The SED is in the process of drafting a new plan identifying targets for the Second Generation of School Education Reforms which should provide more clarity on the way forward. 6. DLI 3. Stipends for secondary school girls. Stipends for secondary school girls to increase secondary school participation/retention. SED continues to implement the strengthened program in 16 Districts. Main Findings. This DLI is likely to be met. The program is continuing in 16 districts with the card-based payment mechanism. By December 2018, the total number of card holders was 377,739 and the total num- ber of students was 448,616 with 330,023 eligible students, however only 268,482 of these students have received payments due to a lack of funds to cover pending payments from previous years. This backlog of payments has resulted from delays in registering beneficiaries for payments. These inefficiencies pose a serious risk to the program, as uncertainties in receiving payments in a timely manner undermine the in- tended impact of the program. The last disbursement to beneficiaries took place in March 2019 for the quarter spanning from July to September. The payment for October to December has not yet been made and it is expected that the students who have not yet received payments for previous quarters will also receive payments in the coming month. Using the SIS, an online attendance system has been adopted to streamline the compliance monitoring processes. A complaint redressal process has also been established and so far thousands of complaints have been addressed. A third-party validation for the program is on- going and the report is expected over the summer vacations. There is a need to further improve the pro- cesses linked to the registration of beneficiaries for payments. For example, instead of relying on parents to come to registration centers multiple times- first to register their child for the program, and then later on to be registered for payments these two processes should be done on the spot in one visit. The mission agreed that the PMIU would draft a note documenting the main implementation challenges along with proposed solutions to address these challenges. 7. DLI 4. Early Childhood Education (ECE). Improving quality and expanding access to early childhood education. 2,500 schools have classrooms that meet the Quality Standards prescribed by the ECE Policy Framework; and 50 schools have started implementation of District health/ nutrition officers’ pilot. Main Findings. This DLI is likely to be met. Data collected by both MEAs and AEOs that indicate at least 2,609 ECE rooms meet the basic quality standards outlined in the policy. The two-year ECE curriculum has been drafted and is close to approval. The PMIU had been developing the health pilot together with Policy and Strategic Planning Unit (PSPU) and have identified the 50 schools with ECE rooms to receive the health pilot, the training to be delivered to district health/nutrition supervisors and indicators to be measured. As part of the midterm review, the mission recognized achievements under DLI 4 as well as steps the govern- ment has taken to ensure the delivery of quality ECE at scale, including the approval of the ECE policy and the beginnings of a quality assurance system. At the same time, the importance of achieving quality con- sistently across all ECE rooms to support positive child development outcomes was emphasized. The in- consistency in quality was reflected in the field visits, with some schools meeting the quality standards well and others facing significant challenges, including one school that had an older student supervising the ECE class as the trained ECE teacher was teaching another class. It was also observed in all schools visited that ECE rooms were established in addition to Katchis, rather than replacing them. These new ECE rooms sometimes displaced other grades, pushing them into multi-graded classrooms. Additionally, neither district level officials nor teachers and head teachers were aware of the 2017 ECE policy and quality standards. To address these challenges, it was suggested to use Assistant Education Officers (AEOs), who appear to be a dynamic and capable cadre of professionals, to work with schools to find solutions or seek further support from the district level. Additionally, it was agreed that while the data AEOs collected was helpful, it was important that they take on more of a mentoring role for the ECE teachers and that PMIU/QAED provide guidance to AEOs on mentoring. It was noted that while the ECE policy had been distributed to the districts, it would be helpful to sensitize district level officials further by presenting on the policy and quality stand- ards at the upcoming CEO conference and include an ECE indicator in the District Ranking System to incentivize action. A soon to be hired communications specialist could also support more effective outreach to schools and teachers and ensure the communication of the quality standards. Additionally, given multiple ECE level classes are present in schools, the importance of formally approving the ECE curriculum was stressed. In the meantime, it was suggested that activity sheets that have been developed as part of the ECE teacher guide are printed and distributed to Katchis and that Katchi classes are encouraged to spend time in the ECE room. It was suggested that the government consider the establishment of an ECE cell within SED to more effectively coordinate the implementation of the ECE scale up. Finally, in order to ensure that quality standards are consistently met, scale up of ECE should be contingent on the availability of adequate classrooms in a school, and teachers to ensure that ECE does not place an additional strain on schools operating at limited capacity. 8. DLI 5. Human Resources Management. Strengthening HR processes through continuous merit-based recruitment and rationalization of teaching posts. SED continues to implement its recruitment policy for Merit-Based Recruitment of new teachers and reallocation of posts. Main Findings: This DLI is likely to be met. Due to the financial situation in the province, there is no teacher recruitment expected to take place this year. The ban on teacher transfers was temporarily lifted this year, and teachers who submitted appli- cations and met the transfer criteria, were transferred. The TPV of the teacher recruitment test is on-going as well as a review of the teacher allocation, teacher transfer, teacher recruitment and teacher promotion policies. The draft of this review is expected by the end of April, and the final report will be shared with the Bank by May 30th. Given that there is no recruitment taking place, the task team will review this DLI in greater detail to determine whether or not this DLI has been achieved. The SED has moved a summary to the Chief Minister requesting a waiver for teacher recruitment and a decision on this is yet to be taken. During the mission, the Secretary School Education, outlined his vision to develop a holistic human re- source policy for teachers that sets clear objectives and expectations for teachers and consolidates other teacher related policies. In addition, the Secretary expressed his desire to restructure the teacher cadre to move away from a typical civil service structure to a more aligned system that would link incentives and performance assessments to teacher related activities. 9. DLI 6. Quality in the primary classroom. Strengthening field based Continuous Professional De- velopment (CPD) for improved teacher performance. At least 85% of primary school teachers in 8 Districts covered by New Mainstream Phase of integrated CPD program; and DSD continue to implement the vali- dated innovative teachers support package in at least 100 schools. Main Findings. This DLI is off track since the start of the project. The broad contours of the Innovative Support Package were presented and are currently under preparation. QAED is working with the TA firm to develop a concept note and an implementation plan for the roll out of the package. Final approval of the restructured DLI is still pending Bank approval and it is expected that this DLI will be formally restructured in the next month. QAED presented four options for CPD of teachers that have been prepared with different organizations: British Council, Ali Institute, the Innovative Support Package, and LUMS School of Education itself. Since Janu- ary 2019, AEOs are conducting class observations and recording these observations through an android based application. The version of the classroom observation protocol currently in use is the same that was used by the DTEs under QAED’s previous CPD model. The revised TOR of the AEOs has been finalized and is awaiting notification from the SED. The PDO level indicator to measure the quality of teaching practices will be measured using the TEACH tool, in line with the established baseline of 2.53 measuring classroom culture, instructional practices and socio-emotional measures. QAED will finalize a classroom observation tool to be used with support from the TA firm. 10. DLI 7. Student Assessment. PEC and its instruments strengthened within a revised policy framework. 80% of ISP targets for 2018-19 are met; and strengthened Grade 5+8 PEC instruments are administered throughout the entire Punjab school system. Main Findings. This DLI is unlikely to be met. The new Commission has been notified and the first meeting is expected to take place at the end of April. The Assessment Policy Framework (APF) has been drafted and this will be presented to the commission for approval in the coming month. The Commission has constituted a technical committee to review the APF in detail and gather feedback from relevant stakeholders before presenting it to the Commission for ap- proval. The major change in the APF is that the grade 5 and 8 examinations will be replaced with three types of assessments. These include a system level assessment, a school based assessment, and Formative assessments. Once the APF is finalized, the TA team will work with PEC to finalize the Institutional Strengthening Plan for PEC which will include a detailed and costed HR staffing plan, a capacity building plan and a procurement plan which will also be submitted to the Commission for approval. The Secretary School Education, informed that mission that this will be a four year plan with a clear roadmap to implement the APF. With this ambitious new APF, it is clear that without quality HR the APF will not be effectively implemented. The mission also expressed its concern that despite the potential savings related to eliminat- ing the grade 5 and 8 exams, the activities outlined in the APF will require significant budgetary allocations particularly for building capacity for the school based assessments. A detailed costing exercise must be undertaken as part of the ISP to calculate the budgetary implication for PEC. Lastly, without the grade 5 and 8 exams, it is possible that the DLI will have to be restructured to be aligned with the new assessment framework for the province. Restructuring would have to be requested in a formal letter from the External Affairs Department, and a process will then be engaged to revise the DLI language in the Financial Agree- ment. This determination will be made once the final APF is approved by the Commission. 11. DLI 8. School Specific non-salary budget (NSB). Setting and executing school budgets in line with school resourcing needs. SED has prepared School-Specific Non- Salary Budgets for FY2018/19 in accord- ance with agreed funding formula, and submitted for inclusion in FY2019/20 District budgets in 36 Districts for all schools, including high/high secondary; and Punjab’s Finance Department has dispersed School- Specific Non-Salary Budgets bi-annually to schools with establishment of Necessary Support Structure for execution; and TPV of Year 1 and Year 2 of this DLI completed. Main Findings: This DLI is likely to be met. Out of Total NSB allocation of PKR 14 billion for FY 2018-19, PKR 5.33 billion has been disbursed to schools in all 36 districts of Punjab in September 2018. The remaining PKR 8.67 billion has recently been released by the Finance Department and will shortly be ‘online’ and transferred to districts. The TPV for this DLI is underway, and it is expected that the TA firm will submit the report by 3rd June. There is also work ongoing to make the disbursement system more efficient through the automation of certain pro- cesses, and the Punjab Information Technology Board (PITB) is working closely with the PMIU and the Bank of Punjab to streamline disbursement processes to eventually move to a model in which funds are available in the beginning of the school year. In order to mobilize and build the capacity of school councils to provide a necessary support structure for the execution of NSB funds, a School Council Mobilization Strategy was developed earlier this fiscal year, and is currently under implementation. As part of this strategy a series of 11 animated infotainment videos are under development along with 7 2d instructional videos and info graphics for school councils. The subjects of these videos range from effective utilization of NSB to child protection and nutrition. Pilot screening of these videos will take place to ensure that they are in tune with the target audience i.e. school council members. The PMIU has also developed an android based application which will be installed in Tablets available with Head Teachers in each school, who will download these videos and info graphics to be consumed by the Council members. The App will also have a feedback loop in order to gauge audience interaction with these resources and their comments on the utility of these resources. The mission also discussed the need agree on a mechanism to report on NSB utilization using existing MEA and SIS data. Meanwhile, additional evidence has been submitted regarding the 2nd year results of the DLI. The assessment of the mission is that further evidence may be required to meet the DLI, while it is likely to have been met. 12. DLI 9. Data strengthening and performance management. Strengthening management capacity and evidence-based decision making. PMIU has integrated Education sector data (from PEC, PMIU, DSD, and PEF); and PMIU has provided access and orientation to the integrated database to education departments and District education officials in 12 Districts; PMIU has prepared and delivered on a semi- annual basis District performance report cards for Year 3 following agreed format and contents, to District Education Administrations. Main Findings: This DLI is likely to be met. The data of PEC, PMIU (Annual School Census & Monthly Monitoring), QAED and PEF has been integrated. Training of technical persons of SED, PMIU, PEC, PEF and QAED were conducted on Aug 2, 2018 and Oct 8, 2018. Access was also granted to the source departments. Trainings of the SDP from DMO office and the IT in-charge from CEO office of all 36 districts were conducted. Access was also granted to the DMO and CEO offices. Moving forward, there is a need to fine tune the filter options for reports to be generated, in addition there is a need to sync the dashboard with the actions taken by district officials which are currently being monitored by the Chief Minister’s Monitoring Task force. In addition, while certain data sources are being synced in real time, other data are pushed less regularly and there is work ongoing to move towards near real time data synchronization. PITB is also working on a system to automate teacher transfers and link this with the integrated dashboard to increase the efficacy and transparency of the transfer process. While PITB’s current engagement with the SED on select projects is coming to an end, the SED has committed that budget for PITB supported activities has been allocated for the next fiscal year. District Performance Report Cards for the period Apr-Sep 2018 were delivered in November 2018. District Performance Report Cards for the period Oct 2018 to Mar 2019 will be delivered in April/May 2019. The mission discussed transitioning from district report cards in their current form, to utilizing the district rankings that are calculated by the PMIU and reviewed at District Review Committee (DRC) meetings. These rankings are reviewed by dis- trict officials on a monthly basis and actions are taken to address lagging indicators. It is proposed that in subsequent years, these district rankings serve as the basis for the district report cards and these could be in the form of either district data packs, or customized reports flashed on LCD screens in district offices. These options will be discussed in the coming months. 13. Financial Management The mission held discussions with the Project Director (PD), Internal Auditor, and Project Finance and Accounts Officers. The mission also performed review of payments from the Des- ignated Account against Component-2 expenditures. The following was noted during the visit: Staffing: A qualified and experienced FMS has joined PMIU on November 1, 2018. He is sup- ported by two qualified persons who are responsible for financial reporting and monthly reconcili- ation with the Accountant General, Pakistan using PIFRA terminal. Internal Controls: The mission was informed that internal audit of TA-Component-2 of the pro- ject for the six months ended December 31, 2018 has been conducted and audit observations have been shared with the financial management and procurement sections of PMIU. Proper segregation is now applied for the recording and reporting of transactions. Non-Salary Budget Expenditure-Funds Flow: Finance Department has released the remaining amount of PKR 8.76 billion, which is the remaining part of PKR 14 billion allocated in 2018 for DLI 8. Efforts are being made by PMIU to rectify errors and anomalies in reporting NSB using dash-board school information system (SIS) developed by the Punjab Information Technology Board (PITB). As of end of February 2019, the system shows there are: Schools with negative current cash and bank balance: 435 Schools with same cash and bank balance: 38 Schools with no entry: 964 Schools with total opening balance above PKR 2 Million : 53 Reporting: The IFRs for the project are timely received and found acceptable. Audit: Acceptable audited financial statements of PESP-III for the year ended June 30, 2018 along with the auditor’s management have been received on December 28, 2018. Review of project payments: Review of certain transactions was carried out, which is about 24% of the total payments of PKR 25,738,175 incurred during October-December 2018. These payments are adequately supported with proper documentation, approved by the PD manually recorded in the accounting ledger, as per the system envisaged. The Drawing Account bank book is properly rec- onciled. 14. Procurement. The mission held discussions with the PD PMIU and with the procurement specialist and reached the following conclusions: Procurement Capacity. Procurement capacity has been somewhat reduced, as relevant human re- sources are currently not in place. The PMIU has recently hired a contract management specialist, who joined the project in Nov 2018. Unfortunately, the PMIU had to terminate the contract of consultant for lack of presence. The Procurement specialist of the project (who has been with the PMIU for a long time) is currently providing support for contract management. The PMIU is re- quested to hire a contract management specialist on priority to manage the contracts specially the large contract like the technical assistance contract. As per the current procurement profile of the project, the Procurement specialist has adequate capacity to provide support to the procurement function of the project. It is requested that the procurement specialist attend the online training for contract management.[1] Procurement Planning. Procurement planning has improved somewhat with fewer revisions to the procurement plan. During the mission of November 2018, it was noted there have been a total of 10 revisions to procurement plan since the start of the project, despite having only 15 procure- ment activities under the project. It is appreciated that since the last mission of Nov 2018, the pro- curement plan was revised only once, avoiding the frequent revisions. PMIU should continue with this holistic approach for preparing procurement plan and avoid frequent updates. Procurement Process. Since November 2018, there are 3 contracts for individual consultants which are signed. The total amount of contracts signed since the initiation of the project is USD 7.478m. The first payment of 10% advance for TA firm has been initiated. The PKR component has been paid to M/s Mott McDonald on April 15, 2019. The foreign currency component of pay- ment is in process. There are 11 activities in STEP with status of pending implementation. It is informed by the Procurement consultant that these activities are already completed, however these are not updated in STEP. It is requested that STEP should be updated for all prior and post review activities prior to acting in actual. The STEP system automatically prevents contract award to de- barred or sanctioned firms The main update is to be made to the activity for TA firm, which is currently showing ‘under implementation’, while the contract with the firm has been signed and first payment has been made. Disclosure of Information. PMIU is posting procurement related information on its website. It is requested during last mission that contract amount is also to be published with the summary for award on PMIU website. It is again requested that this information to be disclosed on priority. The Bank team appreciates that information at PMIU website is regularly updated. Contract Management. The contract for TA firm was signed in Oct 2018 and the first advance payment was made in Apr 2019. This gap of more than 5 months is caused by confusion around Bank guarantees. This gap could have been avoided by having an experienced contract manage- ment specialist, who knows how to deal with large contracts with international firms, and who can advise the firm in getting the documents as per the laws and regulations of Pakistan. Record Keeping. During the most recent mission of November 2018, it was observed that PMIU was keeping all procurement record in separate files. Now it is reported that PMIU has made all procurement record available in individual case files for each activity. It is also requested in last mission that the electronic backup of all the Procurement related records to be maintained on reg- ular basis to avoid any data loss. It is now observed that PMIU has started to have an electronic backup of the procurement record. 15. Social Safeguards. The performance on social safeguards remains satisfactory. A TPV is underway on Girls Stipend (DLI 3) to check its effectiveness, equitable distribution and complaint resolution. As reported, there have been many complaints on the intervention, and the Project has addressed most of them. A Grievance Redress Mechanism has remained active for the DLI, whereby Punjab Social Protection Agency (PSPA) and PESP received complaints through emails, phone calls and other mediums. SED’s telephonic hotline has also received complaints on the Stipend Program, and so did the Provincial Ombuds- man Office. In order to address these issues more efficiently, PESP has plans to launch its own Grievance Redressal Mechanism (GRM) dedicated to the overall project and the Stipend Program. The design of the interventions and the GRM will benefit from the results of the TPV. IV. KEY CONCERNS, NEXT STEPS AND AGREED ACTIONS 16. Key concerns. One of the major concerns of the review is that the PMIU is severely understaffed, with all deputy director positions, the head of research position and contract management specialist position currently vacant. These positions are critical because they serve as focal persons for DLIs ensuring that implementation issues are identified and addressed. With the TA firm now on board, there is a need to ensure that positions are filled urgently. Another concern observed during the mission was the elimination of project allowances for PMIU staff (and for other project staff in the province). In the past, these allow- ances have served as useful incentives to attract quality human resources and without these allowances there is a concern that the PMIU may not be able to attract individuals who can deliver on the SED’s ambitious reform agenda. ● For DLIs 1 (Vouchers) and 2 (PPPs), the main concern is the staffing at PEF. Beyond director level positions, there is a need to fill key positions in ADU and M&E/IT Depart- ments as PEF is expanding its quality assurance system to other programs (i.e. PEIMA, Daanish schools). The Secretary School Education also expressed his desire to scale up investments in the low-fee private schooling sector by working with the private sector for the continuous professional development of teachers, and by creating a set of incentives and standards for different categories of private schools. ● For DLI 3 (Girl Stipends), the main concern is the delay in the implementation and ultimate disbursements of girls’ stipends. The TPV will clarify what actions are to be taken to speed up disbursements. ● For DLI 4 (ECE), the main concern remains the quality of implementation at scale. Human resource and classroom availability in ECE schools, while the mission has also noted that ECE schools maintain a Katchi class. A curriculum has recently been drafted, but needs to be approved. The coordination of ECE activities is still dispersed, as there is no single department taking the lead on this topic. ● For DLI 5 (Teacher Recruitment and Allocation), the main concern is that there has been no recruitment this year because of the hiring freeze. Moreover, the mission has been in- formed that there is no alignment between various teacher-related policies on allocation, transfer, recruitment and promotion. ● For DLI 6 (Teacher CPD), a new framework has been tentatively agreed during the mis- sion (Annex 3) that will support AEOs as mentors, develop an innovative support package and integrate this into QAED’s regular CPD program. A first step for this year is the fina- lization of the revision of the TORs of the AEOs. The key question going forward is the effective management of TA support to QAED to ensure quality of implementation. ● For DLI 7 (Student Assessment), the mission has been informed that a new Assessment Policy Framework has been developed. The key question going forward is the adoption of this new framework, as well as its translation into an Institutional Strengthening Plan that sets out the key implementation steps, human resource needs and budgetary allocations. This DLI may need to be restructured after approval of the framework. ● For DLI 8 (Non-Salary Budgets), the key next step will be the roll-out of the new school council mobilization strategy. A key concern is the position of school councils as well as the future of NSB within the new decentralization strategy of the government. The SED expressed that at the moment no changes to the NSB are expected under this new structure. ● For DLI 9 (Data Integration), a lot of work has gone in designing the data integration and building the Executive Dashboard. The focus should be on the utilization of the platform, and tracking utilization at both district and departmental level. A key concern is the funding of PITB’s involvement for the next years. 17. A broader concern is the overall fiscal situation of the province and the implication this has for edu- cation reforms. This year’s reduction of budget for PEF is one example, as is the freeze on teacher recruit- ment. One of the key findings from the field visits conducted also reflects that classroom space is limited in many schools in the province, and there is still a need to invest in additional classrooms and facilities across the province. Lastly, with the change in government, and delays in onboarding of the TA firm, there is a pressing need to ensure proactive contract management on the behalf of the SED and PMIU to ensure that TA deliverables are up to the standard of the SED and are produced in a timely manner. As part of this, the mission suggested monthly reviews of TA progress chaired by the PD PMIU, with active partici- pation of QAED, PEC and PEF to review progress and identify issues in TA delivery. 18. The Secretary School Education Department identified additional areas of support moving forward. First, given the new decentralized structure being proposed, powers are expected to be devolved to the Tehsil level. This will require (among other things) technical support to develop HR plans as well as ca- pacity building support to ensure tehsil level authorities have the capacity to undertake their new roles and responsibilities. In addition, the SED has plans to scale up investments in the private sector to improve the quality of education service providers. This will require technical support for design and implementation. Once the SED has developed its Second Generation of School Education Reforms roadmap, the task team will review the recommendations to determine if there is scope within the current project to support new activities. Summary of Agreed Actions DLI Actions Responsible Due Date Fill Vacant Positions at PMIU SED 1 EVS Complete procurement process for ERP Soft- PMIU 05/30/2019 ware for PEF 1 Staffing of key PEF position SED / 2 2 Capacity building sessions with Pyschometrician PMIU/TA + 07/31/2019 ADU PEF 3 TPV PMIU/TA 07/30/2019 Note on implementation challenges PMIU 05/16/2019 4 Presentation on ECE policy at CEO conference PMIU Subject to holding of CEO con- ference Establishment of ECE indicator as part of the Dis- PMIU/SED 08/30/2019 trict Monitoring System ECE class mentoring tool developed for AEOs PMIU/QAE 09/30/2019 D Distribution of activity sheets to Katchis QAED Academic Ses- sion 2020/21 Approval of 2-year ECE curriculum PCTB 06/30/2019 Establishment of ECE Cell SED/PMIU 09/30/2019 5 TPV Final Report PMIU/TA 05/30/2019 6 Revised TOR of AEO QAED/TA 05/30/2019 Design of the implementation of the Innovative Sup- QAED/TA 05/30/2019 port Package, including a work plan Finalized Classroom Observation Tool QAED/TA 06/30/2019 7 Notification of APF by Commission PEC 06/30/2019 Draft ISP PEC/TA 06/30/2019 8 TPV Report PMIU/TA 06/03/2019 Animated Videos and Info graphics developed PMIU/Firm 06/15/2019 Conduct monthly TA review meeting with Attached PMIU 05/30/2019 Departments Annex 1 – Status of outcome-level indicators Status of main results of the project PDO: To support Punjab to improve school participation, completion, and teaching-learning practices with a particular focus on low-performing districts. Baseline MTR Tar- MTR Notes get Status 1. School participation rate (6–15 77 77 -- MICS 2017-18 report as data source. Needs a check on data compatibil- years) (Percentage), ity. Need micro data. disaggregated by: Girls (6–10 years) 79 79 85 Boys (6–10 years) 86 86 89 Girls (11–15 years) 66 66 77 Boys (11–15 years) 77 77 83 LPDs 70 70 -- 2. Completion rate, Grade 5 67 68 -- Need micro data. (Percentage), disaggregated by gender: Boys 69 70 -- Girls 66 67 -- 3. Quality score of primary Proposed: - - No targets for 2018/19. End target is 2.65. teaching-learning practices 2.53 4. Level of school readiness Proposed: - 2% No targets for 2018/19. Endline target is 6%. 0 (N/A) Intermediate Results Indicators 1. Number of additional children 0 130,000 138,737 of age 6–16 years to whom a voucher is provided to pay low-cost private school tuition fees (Number), disaggregated by gender: Girls 0 65,000 63,236 Boys 0 65,000 75,501 2. Number of additional children 0 540,000 931,731 enrolled in PPP initiatives supported by the PEF under a quality assurance system (Number), disaggregated by gender: Girls 0 270,000 425,348 Boys 0 270,000 496,383 3. Number of additional schools 0 675 424 Counting 2016/17 as the baseline and including only EVS/FAS/NSP covered by PPP initiatives schools supported by the PEF under a quality assurance system (Number) 4. Number of children of age 3–5 0 75,000 39,231 years enrolled in an ECE classroom that meets quality standards (Number), disaggregated by gender: Girls 37,500 19,785 Boys 37,500 19,446 5. Percentage of new teachers 0 100 100 recruited based on merit, for all vacant posts under the initial recruitment quota reallocated to schools according to the recruitment policy (Percentage) 6. Number of districts in which at 0 8 0 least 85% of primary public schools are covered by strengthened package of field based CPD (Number) 7. Grades 5 and 8 PEC (revised) 0 Grades 5 0 results are analyzed and widely and 8 disseminated in actionable form PEC (re- (Text) vised) administered to all public school children. 8. NSB execution rate 64% Increase of 2 80% percentage points over base- line Number of Executive District 0 12 36 Offices (EDOs)-Education that analyze and act on revised district education report card generated from integrated database (Number) 9. Number of SC members 80,000 128,000 -- Number not available at present. reached through citizen engagement initiatives Annex 2 – DLI Progress and evidence matrix DLI definition 2019 targets Status as of MTR Pending Actions Disbursement Evidence required amount if DLI is met 1 Private School 130,000 Eligible Children Current enrolment at $ 8.59m 1. Summary of DLI status vouchers Received vouches to at- 428,000, which is 32,000 2. Database of registered tend private schools. under target voucher recipients (with unique (Cumulative number is Around 470,000 enrolments identifiers) 462,679) expected 3. QAT and monitoring Da- tabase of partner schools (with unique identifiers) On track to reach 2 Public Private 540,000 children (cumu- Current enrolment is $ 8.59m 1. Summary of DLI status Partnerships in lative 2.38m) are enrolled 2.089m in FAS and NSP 2. QAT and monitoring Da- education through Public-Private programs. tabase of partner schools (with Partnerships under PEF QAT has been delayed this unique identifiers approved programs that year because of office move 3. Monitoring reports have a Strengthened Qual- and budget cuts. 4. TPV reports ity Assurance System On track to reach 3 Stipends for SED continues to imple- On track to reach Action plan to ad- $ 8.59m 1. Summary of DLI status secondary ment the strengthened April 2018-September 2018 dress delays 2. Database of beneficiaries school girls program in 16 Districts Total registered students in agreed format 448,616 3. Biannual program imple- Total eligible students mentation status report with bene- 330,023 ficiary identification, confirma- Payments made to 268,482 tion, and disbursement data, in agreed format Pending evidence review. 4. Implementation chal- lenges note 4 Early Child- (a) 2,500 schools have (a) 2,609 ECE Classrooms $ 8.59m 1. Summary of DLI status hood Education classrooms that meet the are meeting the standards 2. Database of ECE classes Quality Standards pre- (b) ECE health pilot is be- with status of Quality Standards scribed by the ECE Policy ing initiated in 50 schools in 3. ECE health pilot concept Framework; 3 districts note (b) And 50 schools have On track to reach 4. ECE health pilot training started implementation of materials District health/nutrition 5. Final ECE curriculum officers pilot 5 Teacher Re- SED continues to imple- No new hiring in 2018/19. $ 8.59m 1. Summary of DLI status cruitment ment its recruitment pol- Review of teacher alloca- including teacher transfer infor- icy for Merit Based Re- tion process in progress. mation cruitment of new teachers 2. TPV on teacher recruit- and reallocation of posts ment test On track to reach 3. Review of teacher alloca- tion policy 6 Teacher Profes- At least 85% of primary Initial version of ISP is be- Agreement pend- $ 8.59m 1. Summary of DLI status sional Develop- school teachers in 8 dis- ing prepared. ing to restructuring 2. Concept note and drafted ment tricts covered by New Draft TORs have been pre- proposal. innovative support package, in- Mainstream Phase of inte- pared. cluding instruments to be used to grated CPD. Then implementa- track success of ISP implementa- tion of new plans. tion (e.g. classroom observation DSD continues to imple- Off track since start of pro- instrument) ment the validated inno- ject. 3. Approved revision of vative teachers support TORs for AEOs package in at least 100 schools 7 Student assess- 80% of ISP targets for Draft APF is being prepared Appointment of $ 8.59m Evidence for Year 1 Target: ment 2018-19 are met; and and in final stage. commission mem- 1. Summary of DLI status strengthened Grade 5+8 bers and attracting 2. Approved Assessment PEC instruments are ad- qualified staff Policy Framework (with relevant ministered Off track since start of pro- notification) throughout the entire ject. Approval of as- 3. Approved Institutional Punjab school system sessment policy Strengthening Plan (approved by framework, Commission with relevant notifi- cation) Implementation of ISP Revamping of as- sessments 8 School specific SED has prepared School- Out of Total NSB allocation This is on track $ 8.59m 1. Summary of DLI status Non-Salary Specific Non-Salary of PKR 14 billion in FY 2. TPV report on NSB for Budget Budgets for FY2018/19 in 2018-19, PKR. 5.33 billion years 1 and 2 have been dispersed to accordance with agreed schools in all 36 districts of TPV to be com- 3. Certificates submitted by funding formula, and sub- Punjab in September 2018 pleted, is entering CEO office (or equivalent for re- mitted for inclusion in fieldwork stage. lease of cheques to all primary FY2019/20 District budg- Remaining NSB funds and elementary schools ets in 36 Districts for all amounting to PKR 8.67 bil- 4. SC video 1 schools, including lion has recently been re- high/high secondary; leased by Finance Depart- ment. PMIU is in process of Punjab’s Finance Depart- formula based calculation of ment has dispersed share of each school and re- School-Specific Non-Sal- maining NSB funds are ex- ary Budgets bi-annually pected to be dispersed to all to schools with establish- schools shortly ment of Necessary Sup- port Structure for execu- Final TPV report to be sub- tion; mitted by 3rd of June TPV of Year 1 and Year 2 On track to reach of this DLI completed. 9 Data Strength- PMIU has integrated Edu- The data of PEC, PMIU PEF and DSD $ 8.59m 1. Summary of DLI status ening and Per- cation sector data (from (Annual School Census & (QAED) data to be including capacity building over- formance Man- PEC, PMIU, DSD, and Monthly Monitoring), integrated view for departments and district agement PEF); and QAED and PEF has been officials integrated. 2. Integrated education da- PMIU has provided ac- tabase in agreed format cess and orientation to Training of technical per- 3. District level lists with the integrated database to sons of SED, PMIU, PEC, signatures of relevant officials in- education departments PEF and QAED were con- dicating receipt of cards by dis- and District education of- ducted on Aug 2, 2018 and trict education administrations, in ficials in 12 Districts; Oct 8, 2018. Access was agreed format PMIU has prepared and also granted to the depart- delivered on a semiannual ments. basis District performance report cards for Year 3 Trainings of the SDP from following agreed format DMO office and the IT in- and contents, to District charge from CEO office of Education Administra- all 36 districts were con- tions ducted. Access was also granted to the DMO and CEO offices. District Performance Report Cards for the period Apr- Sep 2018 were delivered in November 2018. District Performance Report Cards for the period Oct 2018 to Mar 2019 will be delivered in April/May 2019. On track to reach Annex 3 – Tentative restructuring of DLI 6 DLI 6 Type of DLI Scalability Unit of Measure Total Allocated As % of Total Amount (USD) Financing Amount Outcome No Text 32,210,000.00 0.00 Period Value Allocated Formula Amount (USD) Baseline Field-based system of CPD has mixed operational capacity and performance, and -- poor design match with multigrade and large class teaching. 2016-17 -- -- 0.00 2017-18 -- -- 0.00 2018-19 Innovative support package is designed; AEOs task description is revised and AEOs -- 8,590,000.00 start conducting classroom observations through android app. 2019-20 Training of Primary School Teachers (as per revamped CPD design) conducted in -- 06 Districts Innovative support package is developed, validated and pilot initiated in 11,810,000.00 100 schools; QAED has trained AEOs (or equivalent) on classroom observation and feedback provision. 2020-21 Training of Primary School Teachers (as per revamped CPD design) conducted in -- 12 Districts. QAED has incorporated successful elements of innovative support 11,810,000.00 package into the regular CPD program; Description • The new CPD approach taken by QAED strengthens the existing CPD system for primary teachers, and focuses on improving teaching practices, while also improving literacy, numeracy, and large class (incl. if relevant, multi-grade) teaching. o This includes a teacher support package on basic literacy and numeracy covering Grades 1-3, curriculum sequencing of learning steps for Grades 1-3 literacy and numeracy, with learning, assessment and supplementary learning activities for each step. o This also includes a focus on strengthened pedagogic practice for teachers, use of formative assessments, and in- service training for teachers and head-teachers or principals, aligned to the broad agenda of government and stakeholders. This also includes adequate instructional resources including teacher guides, learning and assessment activity materials for each step, individual child progress monitoring cards, student workbooks, notebooks, supplementary reading books, and literacy and numeracy materials; • The main focus of the DLI is the development and piloting of an innovative support package focusing on teacher pedagogic practice, which includes the use of videos in order to show good practice and evaluate practice, and is aimed to improve teaching quality in the classroom in areas such as supportive learning environment, setting positive behavioral expectations, providing feedback, checking for understanding (formative assessment), lesson facilitation, and critical thinking. • The approach also includes modules (including an app) to support AEOs (or equivalent), headteachers, CPD experts and teachers on teacher observation and feedback provision with the aim of providing data-driven, actionable and constructive feedback to teachers, linked to standards for teaching. • A revised task description for AEOs (or equivalent) will be developed in order to refocus the system on professional development and support, rather than control, while allowing more time for classroom observation, and the development of a professional mentoring relationship between the AEO and the schools. AEOs and headteachers will also be trained and certified on how to do classroom observation and how to provide feedback. • The innovative package will be validated, which means that the innovative package has been reviewed by relevant stakeholders, has been piloted and evaluated, ideally in the setting of a randomized controlled trial that measures its effect on classroom practices and learning outcomes. • The successful elements of the innovative support package pilot will be identified by QAED and stakeholders, and will be selected for scale up in the new mainstream CPD system. Evidence: • Innovative support package published on QAED’s website; AEO training materials and certification results published on QAED website; Pilot report; evidence provided by QAED for training of primary school teachers; PMIU (AEO data on classroom observations, and MEA verification of pilot exercise). Data source/ Evidence to be provided by the PMIU with QAED inputs Agency Verification PMIU MEA monitoring system and SED's AEO system. Entity Procedure School inspectors (MEAs) visit schools regularly to check whether actions are being implemented. School mentors (AEOs) will provide classroom observations and feedback to teachers. PDO level Indicator: To improve teaching-learning practices Quality score of primary teaching-learning 5-7 2.53 -- 2.65 practices (Number) 3. Quality score of teaching and learning practices Definition/Description of Indicator The indicator is an index of 9 constructs from the 'Teach' Classroom Observation Instrument. These constructs in- cluding (1) Supportive Learning Environment, (2) Positive Behavioral Expectations, (3) Lesson Facilitation, (4) Checks for Understanding, (5) Feedback, (6) Critical Thinking, (7), Autonomy, (8) Perseverance, and (9) Social and Collaborative Skills. Each of these constructs is scored by enumerators on a five point scale (1 being very low and 5 being very high). The final result is a simple average of the score on these five constructs (public schools only). Frequency At baseline and endline. Data Source Sample-based survey of public schools and monitoring data from AEOs, applying the 'Teach' classroom observa- tion instrument. Methodology for Data Collection The baseline is set using an Independent third party, and is sample-based. The instrument is validated, and aligned with the teaching-learning practices targeted for improvement by the Quaid-e-Azam Academy for Educational De- velopment (QAED). Once the instrument has been designed and validated, and a team of observers has been trained to reliably collect data, the baseline data collection will be undertaken to determine the baseline value. The endline will be collected by school mentors (AEOs) that will be trained on a new classroom observation tool. Responsibility for Data Collection PMIU and QAED Intermediate Indicators: Indicator Name DL Baseline Intermediate Targets End Target I Year 1 Year 2 Year 3 Year 4 Year 5 Number of districts 6 0.00 0.00 0 0 6 12 implementing the strengthened field-based CPD (Number) 6. Number of Districts Implementing the strengthened field-based CPD Definition/Description of Indicator Measures the number of new teachers recruited based on merit for all vacant posts (which are to be filled by initial recruitment of teachers and not by promotions) reallocated to schools based on the recruitment policy, as a percent- age of all new teachers recruited for those vacant posts Frequency Year 4 and Endline Data Source Monitoring data Methodology for Data Collection AEOs (or equivalent) will collect data from the schools during regular school visits. Responsibility for Data Collection PMIU and QAED with coordination from SED.