The World Bank Implementation Status & Results Report Kyrgyz Health Results Based Financing (P120435) Kyrgyz Health Results Based Financing (P120435) EUROPE AND CENTRAL ASIA | Kyrgyz Republic | Health, Nutrition & Population Global Practice | Recipient Executed Activities | Investment Project Financing | FY 2013 | Seq No: 12 | ARCHIVED on 22-Nov-2018 | ISR34841 | Implementing Agencies: Mandatory Health Insurance Fund, Ministry of Health, KYRGYZ REPUBLIC Key Dates Key Project Dates Bank Approval Date: 18-Apr-2013 Effectiveness Date: 29-Jul-2014 Planned Mid Term Review Date: 26-Sep-2016 Actual Mid-Term Review Date: 26-Sep-2016 Original Closing Date: 30-Jun-2017 Revised Closing Date: 31-Dec-2018 pdoTable Project Development Objectives Project Development Objective (from Project Appraisal Document) The Project Development Objective is to (1) pilot performance-based payments and/or enhanced supervision for quality of maternal and neonatal care in randomly selected rayon hospitals; and (2) strengthen the Government’s and providers’ capacity in performance-based contracting and monitoring and evaluating for results. Has the Project Development Objective been changed since Board Approval of the Project Objective? No Components Table Name Component 1: Pilot Performance-Based Payments and Enhanced Supervision for Quality of Care:(Cost $9.59 M) Component 2: Strengthen the Government's and Providers' Capacity in Performance-Based Payment Reform and Monitoring and Evaluation f or Results:(Cost $1.41 M) Overall Ratings Name Previous Rating Current Rating Progress towards achievement of PDO Satisfactory Satisfactory Overall Implementation Progress (IP) Satisfactory Satisfactory Overall Risk Rating High High Implementation Status and Key Decisions The project is fully on track and will achieve its Project Development Objectives (PDOs) by the closing date. The rayon hospitals Balance Scorecard (BSC) results started to converge after incentive payment was set at 5% across all 64 hospitals, including in control group since the 3rd quarter of 2017 onward. Preliminary results of the Impact Evaluation showed significant and positive effects of intervention n the newborn's Apgar score and mother's blood loss, as well as blood supply, drug availability and quality, and provider's motivation. Final results will be disseminated in January 2019. 11/22/2018 Page 1 of 6 The World Bank Implementation Status & Results Report Kyrgyz Health Results Based Financing (P120435) PHC RBF approach piloted in 5 rayons for a period of one year (two under the Health Results Innovation Trust Fund (HRITF) and 3 supported by the Swiss funded Provider Autonomy project) have shown encouraging results too. Final results from a rigorous Impact Evaluation (quantitative and qualitative studies) have confirmed improvements along three dimensions of quality of care (structural quality, process quality and outcomes). These results will be disseminated at the Public Event in January 2019. The “positive incentives” scheme based on a RBF quality score is being institutionalized across all hospital and PHC levels (except national facilities) using public funds since July 2018 (184 facilities) with the MHIF taking a leadership in conducting quarterly verifications with support of the project using a quality index (Balanced Scorecards) for various types of health facilities at the oblast, city, rayon hospital and PHC levels. Risks Systematic Operations Risk-rating Tool Risk Category Rating at Approval Previous Rating Current Rating Political and Governance -- High High Macroeconomic -- Substantial Substantial Sector Strategies and Policies -- Substantial Substantial Technical Design of Project or Program -- Moderate Moderate Institutional Capacity for Implementation and Sustainability -- High High Fiduciary -- Substantial Substantial Environment and Social -- Moderate Low Stakeholders -- Substantial Substantial Other -- Low Low Overall -- High High Results PDO Indicators by Objectives / Outcomes pilot performance-based payments and/or enhanced supervision for quality of maternal/neonatal care IN00589887 ►Proportion of eligible rayon hospitals that received the authorized performance-based payments within two weeks after the MHIF’s receipt of RBF payment to their bank account (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 22.00 100.00 100.00 Date 25-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 Achieved. Actual (Previous) figure of 22 showed Number of hospitals not Percentage. This was corrected Comments: IN00589885 ►Proportion of quarterly peer verification records for Groups 1 and 2 rayon hospitals submitted to MHIF within 30 working days after the end of each quarter (Percentage, Custom) 11/22/2018 Page 2 of 6 The World Bank Implementation Status & Results Report Kyrgyz Health Results Based Financing (P120435) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 42.00 90.00 100.00 Date 25-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 Partially achieved as verification records for 2 hospitals (10%) included in Gr1 & 2 were submitted after the threshold of 30 days (on Aug 13, 2018 vs the deadline of Aug 1 2018). This is due to the extension of verification period caused by the scale up and institutionalization of RBF scheme across 184 facilities (incl Comments: hospitals and PHC) since July 1, 2018. Previous figure of 42 was a number of hospitals in Gr1 & 2 not a percentage. It has been corrected. IN00589889 ►Number of counter-verification carried out for a subset of Group 1 rayon hospitals and Group 2 rayon hospitals every six months (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 10.00 10.00 8.00 Date 25-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 Overachieved. The 7th round of counter verifications was completed on March 4-8, 2018 in 10 hospitals Comments: strengthen the Government’s and providers’ capacity in performance-based contracting and M&E results IN00589891 ►Impact Evaluation Surveys completed on schedule to generate evidence for policy (Text, Custom) Baseline Actual (Previous) Actual (Current) End Target Two rounds of Impact Impact Evaluation Evaluation Surveys baseline survey At least 2 round of (baseline and end line) completed before start of surveys implemented Value No surveys were completed by the pilot by the end of the end of the project project Date 25-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 Target is met as two rounds of IE surveys (quantitative and qualitative studies) were completed. Comments: IN00589894 ►Average quarterly quality score of Group 1 hospitals (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 9.30 77.00 79.80 75.00 Date 29-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 Overachieved. QII 2018 average score of Group 1 hospitals is 79,8% which exceeds the target of 75% Comments: Intermediate Results Indicators by Components Component 1: Pilot Performance-Based Payments and Enhanced Supervision for Quality of Care IN00589886 11/22/2018 Page 3 of 6 The World Bank Implementation Status & Results Report Kyrgyz Health Results Based Financing (P120435) ►Performance Contracts signed with the Group 1 rayon hospitals (20) and updated annually based on any modifications introduced to the Balanced Scorecard (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 22.00 22.00 20.00 Date 25-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 Overachieved. Contracts w participating providers are signed on annual basis. Performance contracts with Comments: all 22 hospitals (Group 1) were signed in Dec 2017 for the year 2018 IN00589888 ►Proportion of Group 1 rayon hospitals (20) that received quarterly performance-based payments in accordance to their level of performance as per agreed Performance Contracts (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 100.00 100.00 100.00 Date 25-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 Achieved. All 22 hospitals (Group I) received performance based payments in accordance w their BSC Comments: scores IN00589895 ►Average % score Balance Score Card (BSC) indicator 1.7 Red Blood Cell Availability (Group 1 hospitals) (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 19.00 73.00 73.00 75.00 Date 20-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 On track. There has been a significant increase from the baseline of 19% in Aug 2014 to 73% in 2018 Comments: IN00589896 ►Average % score for Category 6 Maternity Department (Group 1 hospitals) (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 5.00 72.50 80.00 85.00 Date 20-Aug-2014 03-Nov-2017 25-Oct-2018 31-Dec-2018 It is unlikely to be met as the end target of 85% has been set too ambitious. This is despite a significant increase from July 2014 to ~80% in Aug 2018 following original calculation (74.2,5% according to a new calculation). Comments: % score for Maternity Department (Gr1 hospitals) IN00589897 ►Average % score for Category 7 Pediatric/Neonatal Department (Group 1 hospitals) (Percentage, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 21.30 80.00 79.00 80.00 Date 20-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 11/22/2018 Page 4 of 6 The World Bank Implementation Status & Results Report Kyrgyz Health Results Based Financing (P120435) On track. There has been a slight decrease from 80% to 79% in QII 2018. As this is a small difference the likelihood high Comments: Component 2: Strengthen the Government's and Providers' Capacity in Performance-Based Payment Reform and Monitoring and Evaluation f or Results IN00589892 ►National RBF Steering Committee meets at least twice a year (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 2.00 2.00 2.00 Date 25-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 Achieved. National RBF Steering Committee has met three times since Nov 2017 (Nov 9, Dec 1, 2018 and June 22, Comments: scheduled for Dec 2018 IN00589893 ►Extended RBF team meets at least every 4 months (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 4.00 4.00 3.00 Date 25-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 Achieved. The RBF Extended team met 4 times since Nov 2017 (Nov 1, Dec 8, 2017 and March 23 and Comments: May 17, 2018. Next meeting is scheduled for Nov 28, 2018 IN00589890 ►MOH/MHIF produce quarterly monitoring reports within 3 weeks after the end of each quarter (Number, Custom) Baseline Actual (Previous) Actual (Current) End Target Value 0.00 -- 4.00 4.00 Date 25-Aug-2014 15-May-2018 25-Oct-2018 31-Dec-2018 MOH/MHIF have been producing quarterly monitoring reports within 3 weeks after the end of each quarter thus resulting in 4 reports annually. Previous value was 4. It was deleted as the unit of measurement was Comments: corrected to align with the restructured RF Data on Financial Performance Disbursements (by loan) Project Loan/Credit/TF Status Currency Original Revised Cancelled Disbursed Undisbursed % Disbursed P120435 TF-13310 Effective USD 11.00 11.00 0.00 9.86 1.14 90% 11/22/2018 Page 5 of 6 The World Bank Implementation Status & Results Report Kyrgyz Health Results Based Financing (P120435) Key Dates (by loan) Project Loan/Credit/TF Status Approval Date Signing Date Effectiveness Date Orig. Closing Date Rev. Closing Date P120435 TF-13310 Effective 30-Dec-2013 30-Dec-2013 29-Jul-2014 30-Jun-2017 31-Dec-2018 Cumulative Disbursements Restructuring History Level 2 Approved on 21-Jun-2017 Related Project(s) There are no related projects. 11/22/2018 Page 6 of 6