The World Bank Report No: ICR00003897 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IDA-44810) ON A CREDIT IN THE AMOUNT OF 21.9 XDR MILLION (US$ 35.00 MILLION EQUIVALENT) REVISED TO THE AMOUNT OF 12.6 XDR MILLION (US$17.2 MILLION EQUIVALENT) TO THE PEOPLE’S REPUBLIC OF BANGLADESH FOR A DISABILITY AND CHILDREN-AT-RISK PROJECT December 12, 2016 Social Protection and Labor Global Practice Bangladesh Country Department South Asia Region i ii CURRENCY EQUIVALENTS (Exchange Rate Effective October 25th, 2016) Currency Unit = Bangladeshi taka (BDT) BDT 1.00 = US$ 0.013 US$ 1.00 = BDT 78.26 FISCAL YEAR July 1 to June 30 ABBREVIATIONS AND ACRONYMS CAR Children at Risk MOSW Ministry of Social Welfare CAS Country Assistance Strategy MTRV Mobile Therapy Rehabilitation Van CPF Country Partnership Framework NFOWD National Forum for Organizations Working with Disability DCAR Disability and Children at Risk Project NGO Non-Governmental Organization DPO Disabled People’s Organization NSSA National Social Security Academy DPP Development Project Proposal NSSS National Social Security Strategy DSC Disability Service Center PAD Project Appraisal Document DSS Department of Social Services PCAR Protection of Children at Risk ERD Economic Relations Division PDO Project Development Objective FM Financial Management PIU Project Implementation Unite FY Financial Year PKSF Palli Karma Sahayak Foundation FYP Five Year Plan PRMF Procurement Risk Mitigation Framework GOB Government of Bangladesh PWD Person with Disabilities HI Handicap International SDF Social Development Fund ICPSC Integrated Child Protection Service TA Technical Assistance Center IO Intermediate Indicator UN United Nations JPUF Jatiyo Protibondhi Unnayan UNCRPD United Nations Convention on the Foundation Rights of Persons with Disabilities M&E Monitoring and Evaluation UNCRC United Nations Convention on the Rights of the Child MIS Management Information System UNICEF United Nations International Education Fund Senior Global Practice Director: Michal J. Rutkowski Practice Manager: Pablo Gottret Country Director: Qimiao Fan Project Team Leader: Yoonyoung Cho, Ashiq Aziz ICR Team Leader: Srinivas Varadan iii COUNTRY Project Name Data Sheet A. Basic Information……………………………………….………….…………….v B. Key Dates……………………………………………………...…….……………v C. Ratings Summary……………………………………………….….……………..v D. Sector and Theme Codes…………………………………………..……..………vi E. Bank Staff…………………………..…………………………….………………vi F. Results Framework Analysis……………………………………………………..vi G. Ratings of Project Performance in ISRs………………………….……………..xvi H. Restructuring…………………………………………………………………...xvii I. Disbursement Graph………………….………………………………………..xviii 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 9 3. Assessment of Outcomes .......................................................................................... 15 4. Assessment of Risk to Development Outcome......................................................... 24 5. Assessment of Bank and Borrower Performance ..................................................... 25 6. Lessons Learned ....................................................................................................... 29 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 30 Annex 1. Project Costs and Financing .......................................................................... 31 Annex 2. Outputs by Component ................................................................................. 32 Annex 3. Economic and Financial Analysis ................................................................. 34 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 38 Annex 5. Beneficiary Survey Results ........................................................................... 40 Annex 6. Stakeholder Workshop Report and Results................................................... 44 Annex 7. Summary of Borrower's ICR and/or Comments on Draft ICR ..................... 45 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 48 Annex 9. List of Supporting Documents ...................................................................... 49 MAP…………………………………………..……………………………………….50 iv A. Basic Information Bangladesh Disability Country: Bangladesh Project Name: and Children at Risk Project ID: P106332 L/C/TF Number(s): IDA-44810 ICR Date: 12/20/2016 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: SIL Borrower: BANGLADESH Original Total XDR 21.90M Disbursed Amount: XDR 12.46M Commitment: Revised Amount: XDR 12.56M Environmental Category: C Implementing Agencies: Department of Social Services Jatiyo Protibhondi Unnayan Foundation Cofinanciers and Other External Partners: B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 10/09/2007 Effectiveness: 01/22/2009 01/22/2009 06/24/2011 Appraisal: 02/27/2008 Restructuring(s): 12/31/2013 03/25/2015 Approval: 07/01/2008 Mid-term Review: 12/13/2012 12/26/2012 Closing: 12/31/2014 06/30/2016 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Moderately Satisfactory Risk to Development Outcome: Moderate Bank Performance: Moderately Satisfactory Borrower Performance: Moderately Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Moderately Quality at Entry: Government: Moderately Satisfactory Unsatisfactory Implementing Quality of Supervision: Satisfactory Moderately Satisfactory Agency/Agencies: Overall Bank Overall Borrower Moderately Satisfactory Moderately Satisfactory Performance: Performance: v C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No None at any time (Yes/No): (QEA): Problem Project at any Quality of Yes None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Public administration - Other social services 22 22 Other social services 78 78 Theme Code (as % of total Bank financing) Participation and civic engagement 10 10 Social Safety Nets/Social Assistance & Social Care 90 90 Services E. Bank Staff Positions At ICR At Approval Vice President: Annette Dixon Praful C. Patel Country Director: Qimiao Fan Xian Zhu Practice Pablo Gottret Mansoora Rashid Manager/Manager: Project Team Leader: Yoonyoung Cho Qaiser M. Khan ICR Team Leader: Srinivas Varadan ICR Primary Author: Anastasiya Denisova F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objective of the proposed Project is to expand the coverage, use, and quality of social care services for persons with disabilities and children at risk as a means of promoting equity and social inclusion. vi Revised Project Development Objectives (as approved by original approving authority) To expand coverage and improve quality of services for persons with disabilities and children at risk by strengthening and scaling up government service delivery systems. (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years 10% increase in the coverage of disabled population by JPUF- original Indicator 1 : 370,464 (160,525 Value 24,165 5-10% female and quantitative or Qualitative) 209,939 male) Date achieved 06/03/2008 12/31/2014 06/30/2016 SURPASSED: dropped in June 2011, but project data show that by the Comments closing date JPUF covered at least 370,464 beneficiaries, which represents (incl. % achievement) more than 140% increase in the coverage of disabled population by JPUF. 30% increase in early detection of disabilities and impairments-original Indicator 2 : Value 2% 40-50% quantitative or Qualitative) Date achieved 06/03/2008 12/31/2014 Comments Dropped during June 2011 restructuring. Data on current status of early (incl. % detection of disabilities is not available. achievement) 10% increase of PWD in the workforce - original Indicator 3 : Value 769,000 6-10% quantitative or Qualitative) Date achieved 06/03/2008 12/31/2014 Comments Dropped during June 2011 restructuring. Data on number of PWDs in the (incl. % workforce at any later dates is not available. achievement) 30% of participating street children attend formal or non formal education, Indicator 4 : or training - original Value 8080 30% 24% quantitative or Qualitative) Date achieved 06/03/2008 12/31/2014 06/30/2016 vii PARTIALLY ACHIEVED: Dropped during June 2011 restructuring. Comments However, according to DSS data by the end of the project more than 500 (incl. % out of not reintegrated 2100 children (24%) received technical/vocational achievement) training. Thus 80% of the target was achieved. Increased number of people with disabilities receiving services - new Indicator 5 : 370,464 (160,525 Value 15,000 100,000 female and quantitative or Qualitative) 209,939 male) Date achieved 12/31/2009 12/31/2014 06/30/2016 SURPASSED: Added during the first restructuring and rephrased to: Comments "Number of people with disabilities registered at DSCs" in Dec 2013. (incl. % During the third restructuring the baseline was revised to 8,168 based on achievement) the availability of digitized records Increased percentage of beneficiaries report satisfaction with services Indicator 6 : received at disability service centers - new 70% of 80% of beneficiaries 93.53% of beneficiaries "fully beneficiaries "fully satisfied" satisfied" "fully satisfied" Value with DSC with DSC with DSC N/A quantitative or services, services, services, Qualitative) equipment, staff equipment, equipment, staff behavior and staff behavior behavior and competency. and competency. competency. Date achieved 05/26/2011 12/31/2014 06/30/2016 06/30/2016 SURPASSED: Introduced during the first restructuring in June 2011. Comments Revised to "Share of beneficiaries that report satisfaction with services (incl. % received at disability service centers" during the second restructuring in achievement) December 2013. Increased number of children at risk receiving child protection services.- Indicator 7 : new 6,193 (3,176 Value Not available Double baseline 3,000 female and 3,017 quantitative or Qualitative) male) Date achieved 05/26/2011 12/31/2014 06/30/2016 06/30/2016 Comments SURPASSED: Introduced during the first restructuring. Rephrased to viii (incl. % "Number of children at risk registered with ICDSCs" during the second achievement) restructuring in December 2013. The baseline value was set in May 2013, and the target revised in December 2013. Increased awareness and positive change in attitude towards persons with disability and children at risk in communities where services provided. - Indicator 8 : new 29.4% of 20% aware of interviewed key individuals fully government Value aware of Not available 20% increase programs for quantitative or government Qualitative) PWDs and program including children at services at risk Disability Serv Date achieved 05/26/2011 12/31/2014 06/30/2016 06/30/2016 SURPASSED: Introduced during the first restructuring. Rephrased to Comments "Share of persons reporting greater awareness on related issues in (incl. % communities where services have been provided" in Dec 2013. Baselines achievement) were set as 12% for PWD & 9% for children at risk Performance of disability and children at risk programs regularly Indicator 9 : monitored and reviewed.-new Annual Reports prepared conference held in 2014 & 2016. where Value Workshops held Not available monitoring quantitative or in 2014 & 2016. Qualitative) reports and 2016 report to be program results finalized. are discussed Date achieved 05/26/2011 06/30/2016 06/30/2016 Comments ACHIEVED (incl. % achievement) Direct project beneficiaries - new Indicator 10 : Value 8,168.00 103,000 376,657.00 quantitative or Qualitative) Date achieved 06/24/2011 06/30/2016 06/30/2016 Comments SURPASSED: Core indicator added during December 2013 restructuring. (incl. % achievement) Baseline revised in March 2015 given new digitized records for PWD. ix Children at risk not counted for in the baseline due to lack of data at the time. Percent female beneficiaries-new Indicator 11 : Value 0.00 40.00% 43.40 quantitative or Qualitative) Date achieved 12/31/2013 06/30/2016 06/30/2016 Comments SURPASSED: Core indicator added during December 2013 restructuring. (incl. % achievement) Share of DSC staff that attain a minimum score on a technical skills Indicator 12 : assessment - new Value 66% 80% 60% quantitative or Qualitative) Date achieved 06/11/2013 06/30/2016 06/30/2016 PARTIALLY ACHIEVED: Baseline based on data collected by JPUF for Comments a training needs assessment in May 2013. Results are based on (incl. % aggregation and harmonization of various certificates and accreditation, achievement) which is not directly comparable to baseline. Children at risk reintegrated - new Indicator 13 : Value N/A 25% 70% quantitative or Qualitative) Date achieved 06/30/2011 06/30/2016 05/31/2016 Comments SURPASSED: New PDO indicator added during restructuring in (incl. % December 2013. achievement) (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years 10% increase in PWDs using the services provided by NGOs contracted Indicator 1 : by JPUF-original 8-10% increase + Value 35,000 assessment to (quantitative or Qualitative) verify performance x Date achieved 06/03/2008 12/31/2014 Comments Dropped during the first restructuring. Became irrelevant as JPUF no (incl. % longer needed to contract NGOs for service provision. achievement) Percentage increase in the number of NGOs, DPOs and community groups capable of developing and implementing rehabilitation and other Indicator 2 : disability services -original Value 521 500-600 (quantitative or Qualitative) Date achieved 06/03/2008 12/31/2014 Comments Dropped during the first restructuring. Became irrelevant due to focus on (incl. % government provision of disability services. achievement) Percentage increased in the number of NGOs, PDOs capable of providing specialized services on specific issues (such as early detection of Indicator 3 : disabilities and impairments-original Value 89 75-100 (quantitative or Qualitative) Date achieved 06/03/2008 12/31/2014 Comments Dropped during the first restructuring. Became irrelevant due to focus on (incl. % government provision of disability services. achievement) At least 10 spot surveys over a period of 5 years conducted to assess Indicator 4 : awareness on specific disability issues and need of relevant care-original Value N/A 5-8 2 (quantitative or Qualitative) Date achieved 06/03/2008 12/31/2014 06/30/2016 Comments NOT ACHIEVED: Dropped during the first restructuring, but in the (incl. % course of the project 2 spot surveys have been conducted. achievement) JPUF capable of delivering disability services nationwide-original Indicator 5 : Value N/A YES YES (quantitative or Qualitative) Date achieved 06/03/2008 12/31/2014 06/30/2016 Comments ACHIEVED: Dropped during the first restructuring. However, by close of (incl. % the project JPUF had services centers in all 64 districts of the country. achievement) Indicator 6 : Minimum standards of care for working with street children are developed xi and applied across implementing NGOs-original At least 50% Standards of care participating for working with Value NGOs apply street children are N/A (quantitative common developed and or Qualitative) standards shared with properly ICPSC staff Date achieved 06/03/2008 12/31/2014 06/30/2016 PARTIALLY ACHIEVED: The indicator was dropped during the first Comments restructuring but a similar indicator was introduced in 2013 (see Indicator (incl. % 15). As a result the minimum standards for working with street children achievement) were developed and shared with ICPSC staff Alternative care model for orphans developed and piloted - original Indicator 7 : Decision taken by MOSW/DSS Value whether to N/A (quantitative continue or Qualitative) alternative care model Date achieved 06/03/2008 12/31/2014 Comments Dropped during the first restructuring. Became irrelevant due to the (incl. % change in scope of the project. achievement) MIS established and used to monitor service delivery to street children- Indicator 8 : original Value N/A Fully operational Fully operation (quantitative or Qualitative) Date achieved 06/03/2008 12/31/2014 06/30/2016 ACHIEVED: Dropped during the first restructuring, but a similar Comments indicator was introduced during restructuring in 2013 and the JPUF owned (incl. % MIS to monitor service delivery to street children was build and made achievement) fully operational. Evidence of MOSW-DSS taking lead in planning and formulating policies Indicator 9 : and programs consistent with its institutional mandate-original Value Feasibility and MoSW-DSS N/A (quantitative performance leading agencies or Qualitative) assessment on social security xii and disability in NSSS 2015 Date achieved 06/03/2008 12/31/2014 06/30/2016 PARTIALLY ACHIEVED: Dropped during the first restructuring. Comments However, National Social Security Strategy (NSSS) 2015 presents (incl. % MOSW leading and formulating policies and programs consistent with its achievement) institutional mandate. MIS developed and able to produce relevant reports to assess the Indicator 10 : performance of programs-original Value 100% ready and 100% ready and None (quantitative operational operational or Qualitative) Date achieved 06/03/2008 12/31/2014 06/30/2016 ACHIEVED: The indicator was dropped during the first restructuring but Comments a similar one was reintroduced during the restructuring in 2013. Hence by (incl. % achievement) the end of the project DSS had a fully operational MIS. 20 additional Disability Service Centers established and functional-new Indicator 11 : Value 0 20 50 50 (quantitative or Qualitative) Date achieved 05/26/2011 12/31/2014 06/30/2016 06/30/2016 Comments ACHIEVED: Indicator revised during December 2013 restructuring to (incl. % "Disability Service Centers established and functional." achievement) Guidelines and tools for disability certification developed-new Indicator 12 : Value Partially Partially N/A Complete (quantitative Complete Complete or Qualitative) Date achieved 05/26/2011 12/31/2014 12/31/2014 06/30/2016 PARTIALLY ACHIEVED: Introduced during June 2011 restructuring Comments and dropped during December 2013 restructuring as the Government was (incl. % able to finance certification work with its own funding. The work is achievement) expected to be complete be end of calendar year 2016. Concerned government officials and service providers at union, upazilla and district levels trained on rights of persons with disabilities and Indicator 13 : available services-new Value N/A 50% of 2000 3,045 (quantitative government xiii or Qualitative) officials and service providers trained Date achieved 05/26/2011 12/31/2014 06/30/2016 06/30/2016 SURPASSED: During December 2013 restructuring the indicator is Comments revised to "Concerned government officials, DSC staff and other service (incl. % providers trained on disability issues." The target was revised to be an achievement) absolute value. ICPS centers established and functional in 7 divisional cities-new Indicator 14 : Value 0 7 11 11 (quantitative or Qualitative) Date achieved 05/26/2011 12/31/2014 06/30/2016 06/30/2016 Comments ACHIEVED: Indicator is revised during December 2013 restructuring to (incl. % "ICPSCs established and functional achievement) Standards for care for children at risk developed-new Indicator 15 : Value N/A Complete Complete (quantitative or Qualitative) Date achieved 05/26/2011 12/31/2014 06/30/2016 Comments ACHIEVED (incl. % achievement) Concerned government officials and service providers at union, upazilla and district levels trained on child protection issues and available services- Indicator 16 : new 50% of concerned Value government N/A 2500 546 (quantitative officials and or Qualitative) service providers trained Date achieved 05/26/2011 12/31/2014 06/30/2016 06/30/2016 NOT ACHIEVED: Added in 2011. Revised in Dec 2013 to "Concerned Comments government officials, NGOs, ICPSCs staff and service providers trained (incl. % on disability issues." The target was overestimated due inclusion of staff achievement) not directly involved in service provision Indicator 17 : NSSA staff trained to provide training on disability and child protection xiv issues and available services- Value 50% of NSSA N/A (quantitative staff or Qualitative) Date achieved 05/26/2011 12/31/2014 Dropped during December 2013 restructuring, but the envisioned results Comments in terms of training of relevant staff engaged in disability and children at (incl. % achievement) risk service delivery are accounted for under indicators 13 and 16. Concerned policymakers and government officials at MOSW central level trained on disability and child protection related issues and available Indicator 18 : services-new 50% of Value concerned N/A (quantitative government or Qualitative) officials Date achieved 05/26/2011 12/31/2014 Dropped during December 2013 restructuring as project decided to focus Comments on training of staff directly involved in service provision to PWDs and (incl. % achievement) children-at-risk. Assistive devices distributed-new Indicator 19 : Value 0 10% 3.48% (quantitative or Qualitative) Date achieved 06/30/2011 06/30/2016 06/30/2016 PARTIALLY ACHIEVED: Added in Dec 2013. Measured as % of the Comments total number of registered clients. Due to data constraints the latest (incl. % estimate based on spotchecks. The latest data based on device achievement) procurement & distribution from FY14 corresponds to 11.4% Persons with disabilities receiving services via mobile outreach-new Indicator 20 : Value 0 10% 23.1% (quantitative or Qualitative) Date achieved 12/30/2013 06/30/2016 06/30/2016 Comments SURPASSED: Indicator added during December 2013 restructuring. (incl. % achievement) MIS development and implementation-new Indicator 21 : xv MIS operational MIS operational Value MIS not developed for DSCs and for DSCs and (quantitative or Qualitative) ICPSCs ICPSCs Date achieved 12/31/2013 06/30/2016 06/30/2016 Comments ACHIEVED: Indicator added during December 2013 restructuring. (incl. % achievement) Service transactions at the DSCs-new Indicator 22 : Value 15,000 500,000 2,409,920 (quantitative or Qualitative) Date achieved 06/30/2011 06/30/2016 06/30/2016 Comments SURPASSED: The final achievement exceeded the end-Project target by (incl. % over 380%. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 09/14/2008 Satisfactory Satisfactory 0.00 2 03/14/2009 Satisfactory Satisfactory 0.43 3 08/14/2009 Moderately Satisfactory Moderately Satisfactory 1.37 4 03/18/2010 Unsatisfactory Unsatisfactory 1.87 5 02/27/2011 Unsatisfactory Unsatisfactory 1.89 6 09/11/2011 Unsatisfactory Unsatisfactory 1.96 Moderately 7 04/07/2012 Unsatisfactory 2.04 Unsatisfactory Moderately 8 10/08/2012 Moderately Satisfactory 2.04 Unsatisfactory 9 03/30/2013 Moderately Satisfactory Moderately Satisfactory 3.05 10 09/01/2013 Moderately Satisfactory Moderately Satisfactory 3.76 11 03/03/2014 Moderately Satisfactory Moderately Satisfactory 4.91 Moderately 12 09/06/2014 Unsatisfactory 6.35 Unsatisfactory Moderately 13 03/19/2015 Moderately Satisfactory 7.65 Unsatisfactory 14 10/07/2015 Moderately Satisfactory Moderately Satisfactory 9.92 15 06/29/2016 Satisfactory Moderately Satisfactory 17.56 H. Restructuring (if any) xvi ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions Project activities were revised to reflect a shift from an NGO based arrangement to provision of services by government. The PDO was 06/24/2011 Y U U 1.89 changed to provide a more precise and measurable reflection of the project activities and outcomes in line with the revised design. 1) Partial cancellation of US$8.8 million. Cancellation requested by the Borrower given cost savings and streamlining of activities not critical to the achievement of the PDO. 2) Modifications in project components and costs to allow focus on key critical activities. 3) Extension of 12/31/2013 N MS MS 4.29 Project Closing Date by 18 months from December 31, 2014 to June 30,2016 to allow for new and revised activities to be implemented. 4) Adjustment of the Results Framework including PDO indicators to reflect changes above. Partial cancellation of US$9.0 million without changes to indicators. Cancellation of US$5.0 million is based on the 03/25/2015 MS MU 7.79 revision of project costs to eliminate amounts not required for completion of Project activities. While a fluctuation in exchange rate of US$ to SDR explains an xvii ISR Ratings at Amount Board Restructuring Disbursed at Restructuring Reason for Restructuring & Approved Restructuring Date(s) Key Changes Made PDO Change DO IP in USD millions additional cost reduction in the amount of US$4.0 million. If PDO and/or Key Outcome Targets were formally revised (approved by the original approving body) enter ratings below: Outcome Ratings Against Original PDO/Targets Moderately Unsatisfactory Against Formally Revised PDO/Targets Satisfactory Overall (weighted) rating Moderately Satisfactory I. Disbursement Profile xviii 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal (brief summary of country and sector background, rationale for Bank assistance) 1. Designed in early 2008 as a response to a request from Government of Bangladesh (GOB), the Disability and Children at Risk (DCAR) project was the first lending operation in the Social Protection sector in the country. It also was the first project addressing provision of services to persons with disabilities (PWD) supported by the World Bank Group. 2. In August 2008, when the Project was appraised, Bangladesh was experiencing substantial economic growth that was twice the average of other low income countries and made significant progress towards reducing poverty (with national poverty line declining roughly one percent per annum for the preceding 15 years 1). Part of the poverty reduction can be attributed to variety of programs established by the GOB that were designed to support the rural poor, elderly, women, and children through provision of safety nets, microfinance and other services2. Nonetheless, GOB recognized that despite these programs some groups remained particularly marginalized with few targeted interventions. Prominent among these marginalized groups were: (i) people with disabilities; and (ii) particularly vulnerable children (street children, children working in hazardous occupations, orphans, children/adolescents in conflict with the law, etc.), especially in rapidly growing urban slums. 3. The costs of not addressing challenges faced by PWD and children-at-risk were estimated to be between US$1.2-$1.8 billion and US$9 million per annum respectively. According to the data from National Forum for Organizations Working with Disability (NFOWD) and Handicap International (HI) in 2005 PWD represented about 5.6% of population in Bangladesh. The Economic and Financial analysis performed at the appraisal estimated that due to (i) some people with disabilities not participating in the workforce 3 ; (ii) foregone schooling for some children with disability; (iii) foregone income for adult caretakers; and (iv) foregone schooling for child care takers not addressing needs of people with disability costed Bangladesh as much as US$l.2 billion, or 1.74 percent of GDP lost per annum. Moreover, an independent study that used similar data estimated the number to be even higher at US$1.8 billion4. The exact number of children-at-risk was not available at the time of appraisal. Close to 700,000 children were also estimated to be homeless5. Additionally, according to National Child Labor Survey 1 Calculations of the project team at appraisal based on various household surveys. 2 “World Bank. 2008. Bangladesh - Poverty Assessment for Bangladesh: Creating Opportunities and Bridging the East- West Divide. Washington, DC. © World Bank. https://openknowledge.worldbank.org/handle/10986/7886 License: CC BY 3.0 Unported.� 3 Accounting for PWDs being of productive age and not having disability that would prevent them from working. 4 Zulfikar, Ali. “The Economic Costs of Disability in Bangladesh.� Bangladesh Development Studies. Vol. XXXVII, December 2014, No. 4 5 According to the data from Bangladesh Development Institute conducted in 2005. 1 2002 -2003 7.4 million of children ages 5-17 were engaged in some economic activity and 4 million were neither at school nor engaged in work. The team estimated the annual loss in lifetime earnings due to street children forgoing schooling to be US$9 million. 4. The GOB was committed to improving the welfare of these groups. By August 2008, the GOB was a signatory to most of the international conventions and declarations on disability including the United Nations (UN) Convention on the Rights of Persons with Disabilities (CRPD). It also signed the UN Convention on the Rights of the Child (UNCRC). Furthermore, commitment of GOB to reaching these groups could be evidenced by creation of National Plan of Action for Children (2005-2010) and National Policy for Disability in 2005. Two departments within Ministry of Social Welfare (MoSW) were in charge of implementing these policies: (i) the National Foundation for the Development of Disabled Persons (NFDDP) which specifically addressed PWDs; and (ii) the Department of Social Services (DSS) which addressed the needs of children at risk, in addition to implementing a number of other social protection interventions. Nonetheless, the implementation of the National Plans for Disability and Children lagged. Cognizant of these challenges, the GOB experimented with various ways to improve the functioning of these departments. For example, it made NFDDP more autonomous (concurrently changing its name to the Jatiyo Protibondhi Unnayan Foundation (JPUF)) in hope that this would allow to improve its capacity by raising its profile and ability to attract top talent. 5. This vision of a need for an intervention targeting particularly marginalized groups such as PWDs and children-at- risk was aligned with Pillar 2 of the Country Assistance Strategy (CAS) 2006-2009. Pillar 2 on Empowerment of the Poor focused on enhancing the quality and efficiency of social service provision to the poor and vulnerable. The only minor difference between the approach outlined in the CAS and one used by the project is that the CAS focused on improving service delivery through strengthening of the local government and community driven interventions, while the project aimed to strengthen non-governmental organizations (NGOs) in parallel with building capacity of the central government. 6. However, the strength of the NGO sector in Bangladesh and its role as a partner of the Government in terms of service delivery is something that is acknowledged not only by the CAS 6 of the World Bank but also by the CAS prepared by the Asian Development Bank (ADB)7. Furthermore, while there was no evidence that NGOs were effective at delivering services to persons with disability in Bangladesh, in terms of addressing children at risk by the time of appraisal, UNICEF had already had a successful collaboration with the Government of Bangladesh through PCAR (Protecting Children at Risk) Project, which utilized collaboration between GOB and NGOs for service delivery. The World Bank’s success in supporting Palli Karma Sahayak Foundation8 (PKSF) also 6 Bangladesh Country Assistance Strategy: 2006-2009. World Bank. Dhaka, Bangladesh. 2006 7 Country Strategy and Program: 2006-2010. Asian Development Bank. Dhaka, Bangladesh. 2005. 8 Palli Karma-Sahayak Foundation (PKSF), an apex development organization registered under the Companies Act of 1913/1994. It was established by the Government of Bangladesh (GoB) in May 1990, 2 provided a good model for partnership between the government, NGOs and civil society. PKSF has implemented Bank projects on poverty alleviation and microfinance, livelihood restoration during disaster recovery etc. during 1990s and 2000s. 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) 7. The objective of the Project was to expand the coverage, use, and quality of social care services for persons with disabilities and vulnerable children as a means of promoting equity and social inclusion9. Outcome Indicators:  10% increase in the coverage of disabled population by JPUF  30% increase in early detection of disabilities and impairments  10% increase of PWD in the workforce  30% of participating street children attend formal or non-formal education or training 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification 8. The change of government in January 2009 (following election in December 2008) resulted in a major shift in government’s stance on social welfare policy. In particular, in early 2009, the MoSW took a decision to significantly strengthen public social welfare institutions and service delivery systems and scale up publicly provided services10. As a result, the project went through a Level-1 restructuring with the change in the PDO and all the PDO indicators in June 2011. The revised PDO read as: “To expand coverage and improve quality of services for persons with disabilities and children at risk by strengthening and scaling up government service delivery systems.� Outcome Indicators:  Increased number of persons with disability receiving services  Increased percentage of beneficiaries report satisfaction with services received at disability service centers  Increased number of children at risk receiving child protection services  Performance of disability and children at risk (CAR) programs regularly monitored and reviewed for sustainable poverty reduction through employment generation. When initially established it focused on creating self-employment opportunities in the rural off-farm sector and adopted the strategy of promoting a credit program (http://pksf-bd.org). 9 The PDO is the same as in the Financial Agreement. 10 Nevertheless, the government did express commitment to partnering with NGOs where necessary to complement publicly provided services in order to improve equity in access, quality, and coverage. 3  Increased awareness and positive change in attitude towards PWD and CAR in communities where services provided 9. The subsequent restructurings in 2013 and 2015 did not change the PDO. Nonetheless, the restructuring of 2013 introduced revisions of key performance indicators which remained as such until project closing in June 2016:  Number of persons with disabilities receiving services  Number of children at risk receiving child protection services  Share of beneficiaries that report satisfaction with services received at disability service centers (DSC)  Share of persons reporting greater awareness on related issues in communities where services have been provided  Performance of disability and children at risk programs regularly monitored and reviewed  Share of DSC staff that attain a minimum score on a technical skills assessment  Children at risk reintegrated.  Number of beneficiaries (core indicator)  Number of female beneficiaries (core indicator) 10. Detailed key performance indicators are presented in the ICR Datasheet, above, and are marked whether they were “original�, “new� and/or “revised�. Section 2.3 on Monitoring and Evaluation below provides a detailed description and analysis of the Results Framework, including the changes introduced during the project. 1.4 Main Beneficiaries 11. The primary beneficiaries of DCAR were to be persons with disabilities and children at risk11. Initially the project did not have a specific geographic focus. After the restructuring in 2011 children at risk component was to be implemented in seven divisional cities. The restructuring of 2013 added four more centers expanding the reach to eleven locations12. Similarly after restructuring in June 2011 the disability component was envisioned to complement government’s own efforts in establishment of DSCs and financed additional 20 locations. During restructuring of 2013 thirty more locations were added. As a result the disability component was implemented in one-hundred and three locations (fifty of which were entirely financed by the Project, and the remainder 11 Children at risk were defined as children below the age of 18 years who were living, working, playing and sleeping on the street and were deprived of basic rights. Minimum age of the street children to be included in the project was six years old. The restructuring papers provided a more precise geographical scope of the children at risk component of project. 12 Dhaka, Chittagong, Rajshani, Khulna, Barisal, Sylhet, Narayangong, Mymemsingh, Comilla, Rangpur, and Jessore. 4 benefitted from financing of specific costs such as equipment, equipment upgrading and training)13. 12. JPUF and DSS, were meant to be secondary beneficiaries of the project. As initially designed the project was to support institutional capacity strengthening through development of relevant guidelines, training of relevant staff at all levels, and development of MIS for implementing agencies. This focus was not affected by the restructuring. In fact as restructured the focus on institutional capacity strengthening was even stronger, which is reflected in the revised PDO. 13. Additionally, as the project was originally designed the NGOs through which the services were supposed to be delivered could also have been considered as secondary beneficiaries. Even though there was no direct capacity building intervention for NGOs/DPOs as part of the project, it was envisioned to happen due to the NGOs/DPOs having to meet a rigorous set of criteria to receive funding from JPUF. 1.5 Original Components (as approved) 14. Component 1: Expanding Disability Services, Improving Awareness, and Building Capacity (total baseline estimated costs US$21.60 million): The objective of this component was to expand disability services, improve awareness and build capacity of NGOs and government agencies serving the disabled. Sub-component 1(a) supported the restructuring, scaling up, and capacity building of the JPUF to make it a center of excellence for contracting disability services, promote awareness building, research, monitoring and evaluation, and establishing good practices. Sub-component 1(b) provided financing for disability services JPUF administered grants to NGOs and Disabled People’s Organizations (DPOs). 15. Component 2: Strengthening and Expanding the Network of Services for Children-at-Risk (total baseline estimated costs US$9.14 million): The objective of this component was to strengthen and expand the network of services for children-at-risk. Activities under sub-component 2(a) focused on institutional and capacity building of DSS and included but were not limited to development of standards for care service provision, development of “case-management� tool and training of personnel. Sub- component 2(b) provided rehabilitation and integration services for street children by supporting NGO-run centers for street children in eleven locations, thus effectively supporting expansion of PCAR from six to eleven locations and broadening the scope of services provided. Sub-component 2(c) supported the design, development and piloting of alternative approaches to institutional care for orphans by contracting NGOs as service providers. Lastly, sub-component 2(d) supported the management and operational costs of the projects. 13 Disability Service Centers were located in all 64 district headquarters, and 49 sub-districts (upazilas). 5 16. Component 3: Institutional Strengthening of the Ministry of Social Welfare and Department of Social Services (total baseline estimated costs US$2.75 million): The objective of this component was the institutional strengthening, training and capacity building of the MoSW and DSS for policy formulation, program planning, implementation, monitoring and evaluation by supporting: (i) the revision of the associated legal framework for persons with disabilities, child protection and other socially disadvantaged populations to bring local laws and regulations in conformity with the UN conventions on Disability and Child Rights; (ii) streamlining the organizational structure and procedures of DSS; (iii) training DSS personnel and other service providers on proactive social services; (iv) technical assistance and provisions of required hardware and software for monitoring and evaluation of MOSW and DSS services and programs. 1.6 Revised Components 17. As mentioned in Section 1.3 above, the change of government in January 2009 resulted in a major shift in government’s stance on social welfare policy. Furthermore, the Government reassessed the institutional status of JPUF (implementing agency for Component 1) and began the process of reverting it to a Government agency with the appointment of a civil servant as the Managing Director. 18. Consequently to ensure alignment with new government priorities the project went through a Level-1 restructuring in June 2011, which also contained a significant change in implementation arrangements, wherein it was decided the project would deliver services to the intended beneficiaries through the government’s own staffing rather than an NGO led model as originally envisioned. During this restructuring the number of components remained the same but their focus and activities were changed as follows: 19. Component 1: Institutional Strengthening, Expanding Disability Services, Improving Awareness, and Building Capacity (total estimated cost US$22 million): The objective of this component was to strengthen the capacity of JPUF to formulate and implement disability policies and programs, increase the coverage and quality of disability services and programs, and build public awareness on the rights of persons with disability in Bangladesh. Sub-Component 1(a) focused on expansion of integrated services and support for inclusion of persons with disabilities. Activities under this sub- component included: (i) strengthening and scaling up of government programs and interventions through the addition of 20 new Disability Service Centers (DSCs) to the 15 existing centers and the provision of 20 fully-equipped mobile vans for outreach services; (ii) contracting services from non-governmental service providers to reach persons with disability specifically in under-served geographical/service areas in order to complement government service delivery; (iii) design of a national disability certification and determination process through the development of relevant guidelines and tools; (iv) provision of schooling materials such as Braille textbooks, assistive devices such as spectacles and hearing aids, and mobile ramps for cyclone shelters; and (v) design and implementation of public education and information campaigns on disability issues, including the development of awareness materials. Sub-component 1(b) supported institutional strengthening of the JPUF. Activities under this sub-component included: (i) a needs assessment to inform program development and action plan formulation for 6 disability services and programs; (ii) development of detailed operational guidelines and standards of service for persons with disabilities; (iii) introduction of a system to monitor and assess JPUF progress and performance with an integrated MIS; (iv) training of disability service providers such as therapists, health workers, and school teachers; (v) training and capacity building of JPUF and MOSW staff at union, upazilla, district and national levels on disability issues; (vi) performance assessment of performance of JPUF programs and services; and (vii) provision of fiduciary and technical staff to support project implementation and associated operating costs. 20. Component 2: Strengthening and Expanding the Network of Services for Children at Risk and Strengthening Department of Social Services (DSS) (total estimated cost US$10.5 million): The objective of this component was to improve and expand the network of services for children at risk and strengthen the institutional capacity of DSS. Sub-component 2(a) focused on strengthening of services for children at risk. Activities under this component included: (i) support of Integrated Child Protection Service Centers (ICPSCs) in 7 divisional cities; (ii) contract of service provision from non-government service providers to expand services specifically in under-served geographical or service areas for children at risk to complement government service delivery; (iii) a needs assessment to inform program development and action plans for children-at-risk services and programs; (iv) development of detailed operational guidelines and standards of services for children-at-risk; and (v) design and implementation of public education and information campaigns on child protection issues including the development of awareness materials. Sub-component 2(b) was targeted towards project implementation support and included activities such as: (i) design and implementation of a system to monitor and assess DSS progress and performance, including an MIS; (ii) performance evaluation of programs and services for children at risk; and (iii) provision of fiduciary and technical staff to support project implementation and associated operating costs. 21. Component 3: Institutional Strengthening of Ministry of Social Welfare (MOSW) to Mainstream Disability and Children at Risk Issues (total estimated cost US$2.5 million): The objective of this component was to strengthen the institutional capacity of MOSW and enhance project impacts by promoting and mainstreaming disability and children at risk issues within the institutions that deal with social welfare concerns more broadly. This component supported the following activities: (i) strengthening of the National Social Service Academy (NSSA) on disability and child protection issues; (ii) building capacity of MOSW personnel through training and exposure through group study tours; (iii) training and building capacity of staff at the local levels (i.e. union, upazilla and district social workers); (iv) development and implementation of an MIS for MoSW integrated with the DSS and JPUF MIS systems (including hardware, software, training, communication technology); and (v) an evaluation to assess overall project performance and impact. 22. Subsequently in December 31, 2013 the project was restructured even further to ensure undivided focus on the key activities as agreed with GOB. This restructuring also entailed a partial cancellation of US$8.8 million. Additionally, activities from 7 Component 3 were merged into Component 2 as part of simplification and consolidation of project activities. The resulting components were as follows: 23. Component 1: Institutional Strengthening, Expanding Disability Services, Improving Awareness, and Building Capacity, renamed as Strengthening Disability Services (US$16.2 million). This component supported establishing an estimated 30 additional disability service centers (DSCs) at the sub-district level and providing additional mobile therapy vans, as well as upgrading equipment in all (Government and World Bank funded) DSCs. 24. Component 2: Strengthening Children-At-Risk Services and Social Welfare Program Administration (US$10.0 million). This component would support three main activities: (i) establishment of 4 additional ICPSCs to expand coverage and provision of additional equipment to and strengthening of emergency and outreach services; and (ii) consolidation of all MIS activities into an integrated system that covers project activities as well as key programs administered by the Ministry of Social Welfare including social pensions and cash transfer programs such as the Old Age Allowance, Widow’s Allowance, and Disability Allowance; (iii) capacity building and institutional strengthening activities previously under Component 3 of the project. 1.7 Other significant changes 25. Partial cancellations: The project went through two cancellations (US$8.8 million in December 2013 and US$5 million in March 2015). The cancellation undertaken in 2013 was driven by: (i) delays in initiating services due to the lengthy government approval processes and thus lower than planned expenditures on operating costs; (ii) cost savings in the estimated operating costs of the centers, as the package of services was a new arrangement and no comparable services were available on which to base initial cost estimates; (iii) lower than anticipated costs of some consulting contracts; and (iv) elimination of cost allocations to originally planned activities that were no longer prioritized by the government. The cancellation in March 2015 was on the other hand driven by: (i) a change in estimated cost of components based on the updated schedule of recurring payments for procurement of goods, non-consulting services and consultant services for each component, which accounted for US$5 million, and (ii) an additional reduction of this revised cost by US$4 million on account of US$ to SDR exchange rate fluctuation, - resulting in a total partial cancellation of US$9 million. 26. Changes in expenditure allocations: Restructuring in 2013 reflected the need for the project to focus on implementation of a few key activities to ensure that it had a chance to deliver what it had promised given the pace of implementation and government’s priorities. This streamlining resulted in a reallocation of funds from component 3 into components 1 and 2 and dropping of activities that no longer 8 corresponded to government’s priorities or could be completed without financing from the World Bank14. 27. Changes in timeline: Given the pace of implementation, under restructuring in December 2013, the project Closing Date was extended for an additional eighteen months from December 31, 2014 to June 30, 2016. The extension was granted to allow the additional DSCs and ICPSCs to be established under the project to become fully operational (e.g. by allowing them the time to build client roster and for the staff to undergo training). Additionally, the extension would provide time for the integrated MIS to be developed and rolled out across all sites at all levels of program administration. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry 28. The Bank team responded to the Government’s request for a project addressing needs of PWDs and children-at-risk and prepared a project in less than 10 months securing the first lending project for Social Protection sector in Bangladesh. The preparation included a stakeholder consultation and a Quality Enhancement Review (QER). The resulting design mimicked the PDO entirely and thus made sense in terms of focus and scope despite other issues (discussed below). 29. The children-at-risk component primarily built on the already existing model of service provision to this segment of the population through expansion and strengthening of UNICEF PCAR type of intervention. The UNICEF PCAR project subcontracted NGOs for service delivery. 30. However, disability component that addressed provision of services to PWDs was a first project of this type and had no best practices or documented lessons that project preparation could use. The design of this component (similarly to the design of the children-at-risk component) relied on successes of joint government/NGO implementation. This delivery model had a proven track record of being effective in other sectors in and outside of Bangladesh. The PAD in particular mentions SDF and PKSF, as examples of success, even though neither one of these could serve as an example of service delivery in the context of this project. The model of joint government/NGO implementation was also highlighted as one of the innovative and effective approaches in the CAS 2006-2010 given generally high capacity of NGO sector in Bangladesh. 31. However, given low supervision/management capacity of the implementing agency (recognized in the PAD) and lack of evidence of capacity of NGOs/DPOs in the disability sector the resulting design introduced unnecessary complexity in the project 14 For example, the GOB no longer required outside financing for the design of guidelines for national disability certification and determination process as it had sufficient funds to complete the process. 9 design. The capacity of the NGOs in the sector of intervention is crucial for the model of joint government/NGO implementation to be successful in particular in cases where government might have low capacity itself. The PAD presents no evidence in regards to the capacity of the NGOs/DPOs in disability sector. Furthermore, the PAD acknowledged that NGOs in the disability sector had not been successful at providing access to employment, aids and services for PWDs. It mentioned lack of funding as one of the potential explanations, but did not provide evidence of otherwise strong capacity and did not undertake any additional assessments. 32. Furthermore, concerns in regards to the complexity of the design and lack of evidence justifying NGO led implementation were raised during the QER discussions15. The risk of disability NGOs/DPOs having limited capacity to deliver services was acknowledged in PAD and rated as “Substantial.� The proposed mitigation measures, however, were not sufficient. Specifically, the mitigating measures consisted of building capacity for JPUF to assess the capacity of NGOs/DPOs that apply for grants, and did not directly address building the capacity of the disability NGOs/DPOs. 33. Furthermore, while risk assessment at the appraisal was quite thorough it lacked the political economy analysis that could have been useful given the upcoming elections in December 2008. On one hand it would be unrealistic to expect that the team could foresee a drastic change in the Government’s stance on service delivery to PWDs and children-at-risk especially in light of the Government agreeing to the proposed design. On the other hand, performing a thorough institutional and political analysis could have allowed the team to assess if there was a likelihood or any signs of this change happening. Additionally, the team could have at least accounted for potential delays in implementation due to the change in government. 2.2 Implementation 34. The project became effective in January 2009 exactly at the same time as the government changed following the elections of December 2008. The change in government resulted in the shift of government’s stance on service delivery (as mentioned in Section 1.3). The MoSW now prioritized the ability of the government to deliver social services through its own systems and therefore focused on strengthening and scaling up of government service delivery systems and institutions. The government remained strongly committed to agenda of improving the lives of PWD and children-at- risk, and even introduced Amendments to the Disability Welfare Act of 2001 and Children’s Act of 1974 to reflect new implementation priorities. It was open to utilizing partnerships with NGOs but only where it was needed to complement government’s own efforts. This meant that NGO-led service delivery model (as proposed in the original project design) no longer corresponded to the government priorities in the social welfare sector. 15 Based on interviews with the team and QER peer reviewers. 10 35. Furthermore, change in the institutional status of JPUF (see Section 1.3) meant that the design of the disability component that relied on JPUF as a semi-autonomous body had to undergo even more adjustments. It also implied that JPUF as an institution had to undergo another restructuring and also work with the Bank team to redesign the project in line with the revised thinking of the then government. Both of these processes were time taking. 36. The project team on the Bank side has also been altered with only two people remaining from the team at Appraisal. Therefore with significant changes on both sides almost a new dialogue had to be established, which required additional time and introduced additional political economy challenges on top of the already usually complicated discussions that surround project restructuring. 37. Nonetheless, after yet another change in the Bank team leadership the project was successfully restructured in June 2011. The restructured design of the project in 2011 reflected the government’s new priorities and its continued commitment to the cause of PWD and children-at-risk. Furthermore, the revised PDO not only improved the measurability of the initial goal to expand and improve service delivery to PWDs and children-at-risk, but also highlighted the project’s commitment to strengthening government owned systems for service delivery. 38. The restructured project design was also informed by a thorough institutional assessment, which could serve as the evidence of learning from the initial phase of project implementation. Furthermore, the team has also significantly revised the results framework to improve its ability to measure progress towards the achievement of the PDO and to inform necessary adjustments in the course of implementation. 39. However, due to unusually complex restructuring that required not just an “Amendment to the Financing Agreement,� but a reshaping of the Financing Agreement as “Amended and Restated Financing Agreement� it took about 18 months for restructuring related discussions to conclude in the form of a level 1 restructuring cleared by the Board in June 2011. And it took up to April 2012 when the first restructuring was declared effective. 40. Restructuring undertaken in December 2013 highlighted further learning from implementation experience. The team took into account: (i) the pace of implementation: proposing an extension to ensure the project achieves its objectives and partial cancellation to realistically reflect the possibilities given the speed of implementation; (ii) further learning about the capacity of implementing agencies: proposing to focus solely on activities essential for the achievement of the PDO and simplification of the design; (iii) learning about the costs of implementation: proposing cancellations due to cost savings. 11 41. The project implementation still suffered from significant delays even as the government took more ownership of the project and as the design was simplified. More specifically, slow pace of implementation noted in December 2013 restructuring and again later during March 2015 restructuring was a result of a combination of: (i) low capacity of implementing agencies; and (ii) cumbersome government approval procedures required after each of the restructurings. The simplification of the project design that is observed in all of the restructurings paired with intensification of capacity building interventions was necessary to address the first constraint. 42. The second constraint comes from the need to submit the new Development Project Proposal (DPP) for approval of the ERD after each one of the major restructurings. For instance, the Revised DPP necessary after the first restructuring (for both JPUF and DSS) was approved in December 2011, while the Board had cleared the restructuring in June 2011. 43. Additionally, each restructuring was used to reflect learning about implementing agencies’ capacity and aspirations. For example, mobile therapy vans were added during the restructuring of 2015 as a way to increase the outreach. While it could be a potentially risky step given the additional procurement capacity it required, it was the step that the implementing agency was willing to take on board and has successfully made happen by the close of the project 44. Overall, despite a rather rough start and chronic delays the project implementation followed a generally upward trajectory starting with the first restructuring. Each subsequent restructuring built on the learnings from the previous phase of implementation in terms of gradually simplifying the project design and identifying the critical activities that had to be prioritized to reach the PDO. 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization 45. Design: The PAD Annex 3 contained 4 PDO indicators; 6 intermediate outcome (IO) indicators for Component 1; 3 IO indicators for Component 2; and 2 IO indicators for Component 3. It can be argued that the initial M&E framework introduced at the design stage could have been strengthened to reflect the theory of change behind the project interventions in a more measurable and conservative way. More specifically, the original PDO aims to address the improvements in terms of coverage, use and quality of services provided to PWDs and children-at-risk. However, only one PDO indicator out of four measured the increase in coverage. Another one “30% increase in early detection of disabilities and impairments� potentially could have measured use and quality. But the remaining two PDO indicators “10% of increase of PWD in the workforce� and “30% of the participating street children attend formal or non-formal education, or training� seemed to be addressing outcomes beyond the PDO and also could not have been clearly attributed to the project. Thus, even absent the change in the project design the original M&E framework would have required revisions to improve its ability to track progress towards the PDO and to inform project implementation. 12 46. Implementation: The entire M&E framework was revised during restructuring in June 2011. As mentioned above, this mainly reflected a rather drastic change in project structure and priorities. However, revision of the framework in 2011 also reflected a more practical approach to M&E by introducing a cautiously conservative output based framework with significantly improved attribution. The M&E framework was further revised during the second restructuring in 2013 to properly reflect streamlining of the critical activities of the project and dropping of the non-essential ones. 47. The resulting M&E framework remained conservative and output oriented which matched the revised PDO better, but also allowed for a more effective measure of progress towards the PDO. More specifically, it allowed to assess progress towards expansion of coverage and improvement of quality of services provided to PWDs and children-at-risk through PDO indicators such as “number of people with disabilities receiving services,� “number of direct beneficiaries�, “number of children-at-risk re- integrated�, and “share of beneficiaries expressing satisfaction with services received at DSCs� as a proxy indicator for quality. It also further allowed to assess if the expanded coverage and improved quality could be attributed to the strengthening of and scaling up of government service delivery systems through a host of intermediate results indicators (including such indicators as number of IDSCs and IPSCs) and PDO indicators such as “share of DSC staff that attain a minimum score on technical skills assessment.� The strengthening itself was also assessed through the indicator tracking the development of the MIS. 48. Utilization: During implementation, all indicators were carefully monitored and both administrative data and third party spot checks were used. There were two third party spot checks covering DSCs, ICPSCs, beneficiaries, and communities. The findings informed the implementing agencies and helped them to identify changes necessary to improve quality of services. Furthermore, the survey instruments used during these spot checks were customized for both JPUF and DSS to use during their own regular monitoring activities in the future. Lastly, project M&E supported two stakeholder workshops in 2014 and 2016 that allowed to share the results of the spot checks and process studies and collaboratively identify approaches to improve project performance. 2.4 Safeguard and Fiduciary Compliance Procurement 49. All planned procurement of goods and services was completed within the implementation period. Four cases of misprocurement of contracts were identified during the period FY 2011 to FY 2013 (two contracts from DSS and two contracts from JPUF). All of the packages together amounted to a total of approximately US$ 50,000. They were primarily towards procurement of goods. The ineligible expenditures were in the process of being refunded (as on August 2016). 50. The DSS and JPUF with the support of procurement consultants were responsible for all procurement under the project. At the time of appraisal, the team rated the procurement risk as “Substantial� before mitigation and as “Moderate� after mitigation. The mitigation strategy included setting up of a separate unit for procurement under 13 JPUF and provision of procurement related technical assistance from the project to DSS. Following the first restructuring fiduciary arrangements and measures associated with fiduciary risks did not change significantly. Additionally, by the time of the first restructuring both JPUF and DSS had hired procurement specialists, which could serve as an indication of an improved procurement capacity even though further measures had to be undertaken. Nonetheless, both DSS and JPUF were not able to retain qualified staff due to below market salaries and both the MTR Mission in December 2012 and restructuring paper from December 2013 highlighted significant concerns in this regard. The restructuring paper of December 2013 identifies the weak administrative capacity pertaining to both financial management and procurement as a risk. This paper suggested additional mitigating measures that included provision of training on procurement to the implementing agency staff and post review for most common procurement procedures. Further measures were suggested during subsequent supervision missions including prior review for all direct procurement and single source selection contracts regardless of the amount. Financial Management 51. By the closing of the project there were no outstanding interim financial reports or audit reports. At the time of appraisal the project rated the FM risk as “Substantial� before mitigation and as “Moderate� after mitigation for both DSS and JPUF. Both agencies had staffing issues over time, however, compliance and reporting have been appropriate. There were minor concerns on incompleteness of documentation, accuracy and timeliness of withdrawal applications that have been raised by the task team on several occasions. This performance issue in particular affected DSS, wherein, the centers being operated by it in 11 locations had poor book keeping due to absence of trained human resources. Although, expenses were being incurred by these centers, the project and more importantly the government’s Development Project Proposal (DPP)16 did not have provisions to appoint an accountant. As a result, book keeping was assigned to other staff. This lack of capacity also affected the financial statements and received ‘qualified opinion’ from the auditors for FY 2012 and FY 2013. The DSS in pursuance of the audit and the task team’s follow up has taken internal measures in order to streamline their book keeping. 52. Additionally, the inability of both agencies to engage in adequate financial planning resulted in reduced budget allocations from MOSW in FY 2013-2014 budget. Lastly, just a month before the project Closing Date, the JPUF suffered from fund shortages that were a result of an increase in National Payscale 2015, under-forecasting of expenditures such as customs duties and taxes for goods procured, and depreciation of XDR since the last project restructuring (which was not budgeted for appropriately). Safeguards 16 Development Project Proposal (DPP) is the Government’s implementation document required to be prepared by the implementing agency and approved for undertaking any activity related to the project. 14 53. The project did not trigger any safeguard policies and was rated as a C category project. The project by its very nature was targeting a vulnerable population group. It intended to have a positive impact on both children at risk and persons with disabilities. Despite the project documents referring to DSCs and ICPSCs as built or established, there was no actual construction involved in the project and all the services were provided through rented premises. 15 2.5 Post-completion Operation/Next Phase 54. The project supported a total of 103 DSCs17 , 32 mobile vans and 11 ICPSCs in their provision of services to over a million beneficiaries, along with MIS systems installed and made operation at both JPUF and DSS. The MoSW has the responsibility of maintaining this large scale operation, and is working closely with the Ministry of Finance (MoF) to ensure the sustainability of the activities. More specifically, by the end of the project there were: (i) allocation of 503 million taka (US4 6.29 million) by MoSW to “Protibondhi Sheba O Shanhajji Kendro� Program under JPUF; and (ii) allocation of 205 million taka (US$2.56 million) by the Finance Division to the “Sheikh Russell Training and Rehabilitation Service Centers� under DSS. Furthermore, the project completion report prepared by THE JPUF also evidences that the GOB has taken initiative to absorb all existing project staff, 50 project financed ICDSCs and 32 project supported MTRVs with all financial liabilities into Government Revenue Budget program. These services have been well received and appreciated by beneficiary groups. The local media has also covered stories around project beneficiaries.18 3. Assessment of Outcomes 55. The assessment of the project outcomes and performance is subject to the split- assessment methodology according to the Bank guidelines. The lifetime of a project is split into three Phases that are marked by changes in the PDO or PDO indicators: (i) Phase 1 which lasts from appraisal to the first restructuring in June 2011; (ii) Phase 2 which starts with Level-1 restructuring in 2011 that resulted in the change of the PDO and continues until the second restructuring in December 2013; and (iii) Phase 3 that starts with restructuring of December 2013 which involved change in the PDO indicators and lasts until the closing day of the project as the restructuring in March 2015 did not introduce changes to either the PDO or the PDO indicators. 3.1 Relevance of Objectives, Design and Implementation 56. The original and revised PDOs were highly relevant at the time of appraisal and revisions (as discussed Section 1) and remain so now. The goals and the achievements of the project in regards to provision of services to both PWDs and people with disabilities are recognized in both Country Partnership Framework (CPF) for FY16-20 and 7th Five Year Plan (FYP): FY 2016-FY2020 of the Government of Bangladesh. In particular the 7th FYP acknowledges the achievements in terms of provision of services to PWD through IDSCs and mobile therapy vans in rural areas as an important building block towards future inclusion. Similarly it acknowledges the importance of re-integration of children-at-risk. Incorporation of marginalized groups is outlined as one of the key 17 The project supported fully 50 out of the 103 DSCs and provided equipment upgrades and staff training for the remaining 53. 18 http://www.thedailystar.net/city/free-cost-therapy-brings-smile-hundreds-1324402 16 foundational priorities in the CPF FY16-20. Thus relevance of the objective is rated as Substantial across all three Phases. 57. The initial project design was in line with priorities of the CAS and Government at the time (see Section 1). However, as given political changes that happened during Phase 1 the NGO led model was not reflective of the Government’s priorities in the social welfare sector. It was not completely irrelevant given that the Government was open to partnership with NGOs where needed, but since the main priority was strengthening and expansion of the government’s own ability to deliver services its relevance was at best Modest. 58. During the first restructuring the project showed its commitment to government developmental priorities when the PDO and project design were changed in 2011 to reflect service provision through strengthening and scaling up of government systems. The restructuring in December 2013 did streamline the focus on the main activities needed for the achievement of the PDO the design and implementation of the project, but despite leaving some room for improvement the design and implementation of Phase 2 were still relevant as they corresponded to the government priorities then and now (in particular through focus on disability certification) and dropping of some of the activities such as the certification was connected more to the ability of the government to finance those activities themselves. Thus relevance of the design and implementation for this Phase is rated as Substantial. 59. Lastly the design as it emerged in Phase 3 with focus on the key activities such as establishment and upgrading of DSCs and ICPSCs, and strengthening of DSS and JPUF as implementing agencies is in line with their increasing importance in implementing policies in regards to PWD and children-at-risk as outlined in NSSS 2015 and 7th FYP: FY2016-2020. Furthermore, while being just a subpart of Phase 3 the revisions introduced during the restructuring in March 2015 helped to achieve even more effective design in regards to the capacity building of JPUF and DSS. Thus relevance of the project objectives, design and implementation is assessed as Substantial. 3.2 Achievement of Project Development Objectives 60. The table below presents a summary of achievement of PDO and Intermediate Outcome (IO) indicators across the three project phases. 61. The achievement towards the PDO is broken down into subparts of the PDO. Initially the PDO consisted of three parts: (i) “expand coverage�; (ii) “expand use�; and (iii) “expand quality�. The revised PDO included two subparts: (i) “expand coverage�; and (ii) “improve quality�; which indicate strong continuity with the original PDO. 62. In order to make the achievement story complete we present two sets of progress indicators. Table 3.1 below presents progress towards the achievement of these PDOs within the time bounds of each of the phases. Table 3.2, on the other hand, presents progress towards the achievement of the PDOs at the time of the evaluation. 17 Table 3.1 Achievement of the PDOs within the time bounds of each Phase Phase 1 Phase 2 Phase 3 (July 2008- June (July 2011- (Jan 2014-June 2016) 2011) Dec 2013) PDO indicators Original Original and Original, 2011, 2013 indicators May 2011 Surpassed 0 2 (new) 4 revised, 3 new (including 1 new core, 1 new core supplement) Achieved 0 3 (new) 1 (May 2011) Partially achieved 0 0 1 (new) Not achieved 4 0 0 Total 4 5 (new) 9 ( 4 revised; 1 May 2011; 4 new) Total surpassed 0 2 (new) 7 (5 revised, 2 new) PDO 1: expand 0 1 surpassed 5 surpassed coverage 2 achieved PDO 2: expand N/A 1 surpassed 3 out of 3 surpassed use 1 achieved PDO 3: 0 out 2 1 2 surpassed, 1 achieved, 1 expand/improve surpassed, partially achieved quality 1 achieved Total surpassed, 0 (HU) 5 (100%) 9 (100%) (S) achieved and (S) partially achieved IO indicators Surpassed 0 0 3 (1 new, 2 revised) Achieved 0 2 (new) 4 (1 new, 2 revised, 1 May 2011) Partially achieved 0 1 (new) 2 (new) Not achieved 10 5 (new) 0 Total 10 8 (new) 9 (4 new; 4 revised; 1 May 2011) Total surpassed 0 0 3 Total surpassed, 0% 3 (new) 9 achieved and 37.5% (100%) partially achieved Weighted 0% 71% 124% Average HU MS HS Rating scale: HU/U (0-40%); MU (41-64%); MS (65-79%); S (80-94%); HS (95-100%) Note: A weighted average was calculated using the following scale: surpassed=1.5x; achieved=1x; partially achieved=0.75x. 18 Table 3.2 Achievement of the PDOs by the time of evaluation Phase 1 Phase 2 Phase 3 (July 2008- June (July 2011- (Jan 2014-June 2016) 2011) Dec 2013) PDO indicators Original Original and Original, 2011, 2013 indicators May 2011 Surpassed 1 4 (new) 4 revised, 3 new (including 1 new core, 1 new core supplement) Achieved 0 1 (new) 1 (May 2011) Partially achieved 1 0 1 (new) Not achieved 2 0 0 Total 4 5 (new) 9 ( 4 revised; 1 May 2011; 4 new) Total surpassed 1 2 (new) 7 (5 revised, 2 new) PDO 1: expand 1 3 out of 3 5 surpassed coverage surpassed PDO 2: expand N/A 2 out 2 3 out of 3 surpassed use surpassed PDO 3: 0 out of 2 1 achieved 2 surpassed, 1 achieved, 1 expand/improve achieved 1 surpassed partially achieved quality Total surpassed, 2 (50%) 5 (100%) 9 (100%) achieved and partially achieved IO indicators Surpassed 0 1(new) 3 (1 new, 2 revised) Achieved 3 3 (new) 4 (1 new, 2 revised, 1 May 2011) Partially achieved 2 0 2 (new) Not achieved 5 5 (new) 0 Total 10 8 (new) 9 (4 new; 4 revised; 1 May 2011) Total surpassed 0 0 3 Total surpassed, 5 (50%) 4 (50%) 9 (100%) achieved and partially achieved Weighted 57% 88% 124% Average MU S HS Rating scale: HU/U (0-40%); MU (41-64%); MS (65-79%); S (80-94%); HS (95-100%) Note: A weighted average was calculated using the following scale: surpassed=1.5x; achieved=1x; partially achieved=0.75x. 19 63. PDO 1 “Expand coverage�: The first subpart of both original and revised PDOs is the objective to “expand coverage.� In Phase 1 only one (“10% increase in the coverage of disabled population by JPUF�) out of four PDO indicators measured the achievement towards this PDO. Furthermore, it touched only on coverage of services for PWDs and not for children-at-risk. Given implementation delays no progress was made towards this indicator during Phase 1 itself (see Table 3.1), but by the close of the project the indicator was surpassed quite significantly as even using the original baseline the coverage increased by 140% as opposed to 10%. Nonetheless, given implementation hurdles and poor design of the initial results framework the efficacy of the project with regards to this PDO in Phase 1 can only be rated as Negligible. 64. As the results framework improved after the first restructuring three out of four indicators tracked achievement towards this PDO and included both services for PWDs and children-at-risk: (i) increase in number of PWD receiving services; (ii) increase in number of children-at-risk receiving child protection services; and (iii) increased awareness and positive change in attitudes towards PWD and children-at-risk where services are provided. By the time of the second restructuring December 31st, 2013 the project had already reached close to 1,700 children at risk and close to 53,000 clients at DSCs 19 (surpassing the goal of 25,000 for that period in implementation timeline). Associated IOs, such as number of IDSCs and ICPSCs, were also achieved both by the time of the evaluation and within the time bounds of Phase 2. Thus efficacy of the project with regards to the achievement of this PDO in Phase 2 is ranked as Substantial. 65. Lastly in Phase 3 measurement of the achievement towards this PDO was strengthened by refining indicators above and adding two core indicators: (i) Number of beneficiaries; and (ii) number of female beneficiaries. By the close of the project in June 2016, the project surpassed key PDO indicators reaching more than 6,000 children at risk through ICPSCs with 70% of them reintegrated; and reaching more than 370,000 PWDs through DSCs. Communities in the project locations were made aware of services available both for children at risk and for persons with disabilities resulting in almost two-threefold increase from the baseline. Lastly, the second restructuring added the provision of mobile therapy vans to project activities, which definitely contributed to the increased coverage of services for PWDs (with 23% of beneficiaries being reached in this way). Thus the efficacy of the project with regards to this PDO in Phase 3 is thus rated as Substantial. 66. PDO 2 “expand use�: Even though one of the objectives of the initial PDO was to expand use of the services for PWD and children-at-risk there were no PDO or IO indicators that tracked this, which is a weakness mentioned in Section 2.3. Naturally the rating for efficacy of Phase 1 in regards to this PDO is Negligible. 19 Restructuring paper from December 2013 states two different numbers: 37,200 registered clients at DSCs based on data from May 2013 and 53,000 based on data from October 2013. 20 67. The objective of expanded use was dropped from the revised PDO, while the objectives to expand coverage and improve quality remained. Nevertheless, some of the PDO indicators that were introduced to assess the success of the project with respect to the expansion of coverage can be used to assess the expansion of use of the services for PWDs, and the argument can be made that the project did achieve significant results in the expansion of the use of the services for its target groups. In particular the number of people receiving services at DSCs and ICPSCs does represent the use of services and the project achieved and surpassed its targets with respect to those in Phases 2 and 3. Thus the efficacy with respect to this PDO in Phases 2 and 3 is rated as Substantial. 68. PDO 3 “expand/improve quality�: From the very beginning the project set out to expand quality of services provides to PWDs and children-at-risk. The objective remained the same and became better worded as “improve quality� when the PDO was revised during the first restructuring. Somewhat similar to the PDO 2 the initial results framework could have definitely been improved in regards to the measurement of progress towards this PDO. Two indicators, one capturing the increase in early detection of disabilities and another capturing the percentage of children-at-risk that receive training, can be mapped to this PDO. During Phase 1 due to implementation difficulties no progress was made towards the targets set for those. However, by the time of the evaluation one of these initial indicators (capturing training for children-at-risk) could be considered to be partially achieved. Nonetheless, efficacy of the project with respect to this objective during Phase 1 is rated as Negligible. 69. The revision of the results framework during the first restructuring in June 2011 made it much easier to track progress towards improvement of the quality of services in Phases 2 and 3. In particular the results framework introduced in Phase 2 included indicators that tracked beneficiary satisfaction and mandated review of performance of programs for PWD and children-at-risk. By the end of Phase 2 54% of PWDs reported to be fully satisfied with the services provided by DSCs (surpassing the goal of 50% goal corresponding to that period in implementation timeline), and by the time of this evaluation close to 94% of beneficiaries reported being fully satisfied with DSC services, equipment, staff behavior and competency. Furthermore, all clients’ records were being digitized and regular reports were produced. Although there is no explicit PDO indicator that addresses the quality of services for children-at-risk during this Phase in same way as services for PWDs are assessed through beneficiary surveys, the development of guidelines for minimum standards of care while working with children-at-risk is addressed by one of the IO indicators. Altogether this leads to the ranking of efficacy in regards to this PDO during Phase 2 as Substantial. 70. Lastly, this PDO in particular benefitted from the revisions to the results framework introduced during Phase 3. Specifically, to balance quality assessment between the disability and children-at-risk components a new indicator to measure success of the services provided to children-at-risk through the percentage of children reintegrated was introduced. By the end of the project 70% of children receiving services at ICPSCs were re-integrated (greatly surpassing the target of 25%) and even though no longer tracked as a PDO indicator close to 25% of those who were not re-integrated 21 received training that lead to their eventual employment and self-employment. And while it might appear that this is the only PDO that does have a PDO indicator that is only partially achieved (“the percentage of DSC staff achieving minimum score on a technical skills assessment�) this is more likely explained by the shortcoming of the results framework that did not properly account for the possibility of multiple different certification processes with different levels of difficulty while setting the target. This oversight resulted in the need to harmonize and aggregate across multiple certifications resulting in a noisier measure than initially anticipated. Thus overall the rating of the efficacy of the project with respect to this PDO in Phase 3 is Substantial. 71. “Strengthening and scaling up of government service delivery mechanisms �: Lastly while strengthening and scaling up of government service delivery mechanisms outlined as the means through which the above objectives of expanded coverage and improve quality of services for PWD and children-at-risk are achieved might not be considered a PDO in itself the project did spend a significant amount of effort on ensuring that this happens. From restructuring in May 2011, the project set out to strengthen government’s capacity to delivery service to PWDs and children at risk. The project contributed to this objective by: (i) supporting service provision directly through DSCs and ICPSCs20 ; (ii) training of concerned government officials and personnel; (iii) establishing MIS for increased efficiency of resource management at JPUF and DSS; (iv) creating institutional guidelines for child protection; and (v) allowing for an increased budgetary allocation for child protection and provision of services to PWDs. In the course of the project government operated centers for disability services and children at risk were established and made operational. More than 3,000 government officials were trained on the rights of PWDs and available services; more than 500 government officials were trained on child protection issues and available services and more than 800 officials were trained on the use of MIS21. Institutional guidelines on standards of services for child protection were established. 3.3 Efficiency 72. At the outset of the project the Economic and Finance analysis using a set of assumptions and national level data on PWD and children-at-risk established that not properly integrating PWD and children-at-risk costed Bangladesh US$1.2-$1.8 billion and US$9 million per annum respectively. The studies find that disability often leads to a vicious cycle of human capital accumulation that results in continued poverty22. Thus provision of services to disabled does target an important economic shortcoming that can help alleviate poverty now and in future generations. Furthermore, there is some evidence that focus on service provision in addition or as opposed to focus on just provision of 20 Reaching total of 103 DSCs covering all 64 districts: 50 DSCs are supported fully and equipment upgrading is provided for the remaining 53 government financed DSCs; and a total of 11 ICPSs with each center catering to more than five districts and thus covering more than 55 out of 64 districts. 21 Based on documentation from DSS and JPUF. 22 Filmer, D. (2008). Disability, poverty, and schooling in developing countries: results from 14 household surveys. The World Bank Economic Review, 22(1), 141-163. 22 disability benefit payments is associated with a smaller negative effect of disability on workforce participation23. Similarly there is evidence that reintegration of children-at-risk can be beneficial for society. For example, a recent study based on US data finds that reintegration of at risk youth early in their life saves society about $2.6-$4.4 million24, and yet another study based on US data presents evidence of quite significant societal gains (close to $2,000 per student annually) from investing in children-at-risk25. 73. The basic economic analysis in Annex 3 concludes that the efficiency of the Project during Phase 1 was Negligible given lack of progress despite rather high transaction costs, Modest during Phase 2 as despite significant achievements the project could still find space for cost reduction and better projection of recurring expenditures, which were undertaken during restructuring in December 2013, and Substantial during Phase 3 as characterized by highest disbursement and highest number of beneficiaries reached through quality service provision. Furthermore, Phases 2 and 3 contributed to an improvement in efficiency by ensuring cancellation of funds no longer required and adjustment of the costings once information became available. 3.4 Justification of Overall Outcome Rating Table 3.3 Phase 1 Phase 2 Phase 3 (2008- May (2011 - Dec (2014 - June 2016) 2011) 2013) Relevance Modest Substantial Substantial Objective Substantial Substantial Substantial Design Modest Substantial Substantial Efficacy Neglible Modest Substantial expand coverage Neglible Substantial Substantial improve quality Neglible Modest Substantial 74. Based on the above assessment, and applying the Bank’s guidance on weighted rating for projects with formally revised objectives and outcome targets, the final outcome rating is subject to a split assessment, according to the three Phases, against the disbursed amounts in each phase. Following the split rating methodology, the final outcome is rated as Moderately Satisfactory. 75. While justified, this might be an underestimation of the overall project outcome, as it is also important to note that the project has significantly contributed to the welfare 23 Jones, M. K. (2008). Disability and the labour market: a review of the empirical evidence. Journal of Economic Studies, 35(5), 405-424. 24 Cohen, M. A., & Piquero, A. R. (2009). New evidence on the monetary value of saving a high risk youth. Journal of Quantitative Criminology, 25(1), 25-49. 25 Heckman, J. J., & Masterov, D. V. (2007). The productivity argument for investing in young children. Applied Economic Perspectives and Policy, 29(3), 446-493. 23 of the highly vulnerable population group in the country. The Government’s allocation of budgetary resources for the services provided through the previously supported project entities reflects the significance and the important contribution that the project has made towards this, which is even more impressive given initial low capacity of implementing agencies. Table 3.4 Phase 1 Phase 2 Phase 3 (July 2008- Jun (July 2011 - (Jan 2014 - 2011) Dec 2013) June 2016) (1) Rating value U MS S (2) (3) (5) (2) Total disbursed 1.89 2.40 13.59 (US $million) (3) Weight 10.6% 13.4% 76% (total disbursed/final disbursed amount of 17.88 USD) (4) Weight value 0.212 0.402 3.80 ((1)x(3)) Final Outcome Rating MS (4.4) Note: HU (1); U (2); MU (3); MS (4); S (5); HS (6) 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development 76. The project targeted the most vulnerable of the poor: people with disabilities and children at risk. Literature on disability indicates that people with disabilities tend to be more likely to be poor. For example, analysis of household survey data from 14 countries shows that having a disability is associated with a 10 percentage point increase in likelihood to be in the two lowest income quintiles26. 77. Similarly provision of child protection services can likely help children from the poorest families to re-integrate back into society and education system. According to the Government’s Project Completion report about 500 of children at ICPSCs have received training and were later engaged in wage and self-employment, which also could be considered as contributing to poverty alleviation. (b) Institutional Change/Strengthening See Section 3.2 26 Filmer, D. (2008). Disability, poverty, and schooling in developing countries: results from 14 household surveys. The World Bank Economic Review, 22(1), 141-163. 24 (c) Other Unintended Outcomes and Impacts (positive or negative) N/A 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops (optional for Core ICR, required for ILI, details in annexes) 78. Third party spot checks were conducted for both Components 1 and 2 in 2014 and 2016. These checks assessed the functionality of service centers, staff skills, their satisfaction and motivation, beneficiary satisfaction. The checks reflected the high levels of beneficiary satisfaction as well as staff competency and satisfaction at the DSCs and the ICPSCs. 79. In terms of infrastructure and facilities in DSCs, both rounds of spot checks showed high levels of functionality with over 80% of the centers having the infrastructure and facilities during both rounds of checks. Similarly, the DSCs scored about 80% in the overall equipment index both in 2014 and 2016. The ICPSCs showed remarkable improvement over the two survey periods. In 2016, all of them were functioning with basic infrastructure and facilities. 80. Staff competency showed a remarkable improvement between the two survey periods both in DSCs and ICPSCs. In DSCs, while in 2014 only 18% of staff had knowledge of disability act and rules, in 2016, 100% of the staff interviewed were aware of at least one act / law related to disability. In terms of job satisfaction, while in 2014 staff satisfaction was only 30%, staff who were fully satisfied or somewhat satisfied with their jobs increased to over 80% in 2016. Similarly, in ICPSCs in 2016 over 98% of the staff were knowledgeable about case assessment and management in comparison to only 46% in 2014. In 2016, all of the staff interviewed had knowledge about the role of ICPSCs for children at risk in comparison to only 61% in 2014. The average motivation of staff was assessed to be 78.9% in 2016. The overall staff competency in ICPSCs increased from 65% to 89.7% in 2016. 81. Overall beneficiary satisfaction levels in DSCs during both survey periods have been over 80%. Over 90% beneficiaries have been satisfied with the services at the DSC including the registration process, behavior of staff, among others. Similarly, during both survey periods over 90% children reported satisfaction with the services at the ICPSCs. In 2016, with the exception of beneficiary satisfaction with the rehabilitation / reintegration plan wherein 82.5% reported satisfaction, in 13 out of 16 parameters over 94% of the beneficiaries reported satisfaction. A summary of the third party spot checks conducted in 2014 and 2016 is given at Annexure 5. 4. Assessment of Risk to Development Outcome 82. Risk to development outcome is assessed as Moderate. As mentioned earlier by the end of the project there were: (i) allocation of 503 million taka (USD 6.29 million) by MoSW to “Protibondhi Sheba O Shanhajji Kendro� Program under JPUF; and (ii) allocation of 205 million taka (USD 2.56 million) by the Finance Division to the “Shei kh Russell Training and Rehabilitation Service Centers� under DSS. These actions serve as a commitment from the government to the continued provision of high quality services to 25 people with disabilities and children at risk. Additionally, open commitment to this goal expressed in the 7th Five Year plan and active participation of Bangladesh in relevant UN committees provides further indications as to the same. However, it needs to be seen if GOB can commit budgetary allocation in the long run, although provision of these services still remain a priority for GOB. Thus, being conservative the risk is assessed as Moderate. 5. Assessment of Bank and Borrower Performance (relating to design, implementation and outcome issues) 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry (i.e., performance through lending phase) Rating: Moderately Unsatisfactory 83. In retrospect, the project in its original design and implementation arrangement though strongly desired by the then Government, could have recognized that (i) the arrangement for delivery of disability services was too complex given low capacity of both the implementing agency (JPUF) and disability NGOs/PDOs; and that (ii) there would have been at least a slowdown in implementation associated with the change of government post December 2008 elections, if not a larger scale change in government’s social welfare policy priorities (see Section 2.1). Further, as it has been acknowledged in the risks that the capacity of both implementing agencies as well as that of MOSW was low, it may have been necessary to ensure capacity enhancement measures such as hiring of PIU staff, especially key fiduciary staff, as a project readiness condition. The need for hiring of core staff in both PIUs has been a key issue raised during 2010 early 2011, as reflected in the ISRs during this period. The change in government, the subsequent change in government’s social welfare policy priorities and lack of key staff in the PIU resulted in almost absent progress prior to the first restructuring. The quality at entry is therefore rated as Moderately Unsatisfactory. (b) Quality of Supervision Rating: Satisfactory 84. Phase 1 (July 2008 – June 2011): During this phase, the project was affected by the undergoing political process in the country resulting in delays. It was marked by changes on both the Government and the Bank side. On the Government side, one of the main implementing agencies was undergoing a reorganization in addition to the overall change in the government’s stance on social welfare policy. On the Bank side the TTL and most of the team changed quite soon after the project became effective. The Bank team nonetheless worked relentlessly on transitioning the dialogue and finding new grounds for consensus with the new counterpart given that both sides remained dedicated to the main objective of the project. They continuously engaged with the government in order to build trust and confidence followed by a discussion on the revised project design. Furthermore, the level of complication of the restructuring added to the already 26 complicated and delicate process that precedes it. Hence the team for example, wasn’t willing to take a risk and push for cancellations at this time (even though it suspected they might be needed based on the capacity and implementation speed). Thus, it took 18 month for restructuring related discussion to conclude. And it was in April 2012 that the first restructuring was declared effective. Furthermore, in regards to the fiduciary issues in 2010, the task team took the initiative and organized orientations for Government and implementing agencies on Bank project cycle and procedures including that on procurement and financial management. 85. Phase 2 (July 2011 – December 2013): After the first restructuring, the project did not progress in the desired pace owing to delays in government clearances (DPP / Revised DPP & the Project Agreement) required after the restructuring. The task team worked closely with the implementing agencies in order to expedite work as soon as requisite government clearances were in place. As noted in the ISR approved in April 2012, the project had made progress in setting up 10 DSCs and 7 ICPSCs. The Bank team during this phase also supported the counterparts by providing opportunities for sharing of international best practices and focusing on relevant staff training. This was also a period when the project finally started to disburse. The task team encouraged by the progress (noted even in MTR from December 2012) but feeling the need to further improve efficiency of implementation undertook a second restructuring (Level 2) in June 2012 (approved by the Board in January 2014) with the goal to primarily focus on core activities and ensure adequate time for their completion. The task team during the second restructuring brought the core focus of the project primarily towards the service centers for persons with disabilities and children at risk, while dropping some activities. A careful assessment of cost savings by the task team due to the delay in project implementation as well as dropping of previously planned activities resulted in partial cancellation of USD 8.8 million in the second restructuring. 86. Phase 3 (January 2014 – June 2016): The third phase of the project was the most encouraging period wherein the project achieved most of the intended outcomes as well as disbursed funds. The task team’s efforts towards bringing the focus of the project to its core activity of service provision for the target population began to pay dividends. Since service provisions in the centers had commenced during the second phase, the task team commissioned two rounds of third party spot checks in order to assess the functionality of the service centers established under the project and the satisfaction levels of the beneficiaries and the staff engaged for service provision. The team was proactive in improving efficiency and resource utilization by undertaking the third restructuring during this period. Furthermore, due to low level of procurement capacity as mentioned earlier, the task team provided close assistance in procurement related matters including hands-on training on bid/proposal evaluation etc. Additionally, the task team worked closely with the implementing agencies to sort out and resolve issues of ineligible expenditures. Lastly, the task team provided very close assistance in the matters of financial management whenever it could including attempts to prevent JPUF fund shortages prior to the closing of the project. 27 (c) Justification of Rating for Overall Bank Performance Rating: Moderately Satisfactory 87. The task team’s performance during supervision resulted in significant progress with the project achieving and surpassing its desired results and disbursement despite the shortcomings of the initial project design. As evident in sections 2 and 3, it was the shortcoming in the project’s quality at entry that hindered the project’s ability to move in the intended pace and direction, and it was the impressive hard work of the bank team during supervision that turned the project around. The team had to work extensively with the implementing agencies to ensure progress. 5.2 Borrower Performance (a) Government Performance Rating: Moderately Satisfactory 88. Phase 1 (July 2008 – June 2011): The Ministry of Social Welfare (MOSW) was the counterpart Ministry in the Government of Bangladesh responsible for implementing this project. The Ministry wanted to strengthen its capacity to develop policies, regulations and service standards, monitor compliance and evaluate outcomes and impact. However, with the advent of a new government in 2009, MOSW stance shifted to the strengthening of government mechanisms for service delivery rather than a model that depended largely on non-government entities. During this phase, the Ministry in coordination with the other relevant Ministries / Departments (ERD, Ministry of Finance) continuously engaged with the task team and worked towards revising the project in light of the new thinking. It must be noted here that the Government’s commitment to provide necessary services to the target population remained the same through this period of change. Most of the time during this phase was spent on relationship building followed by finalizing the revised design and implementation mechanism of the project. 89. Phase 2 (July 2011 – December 2013): During this period, the government worked with the task team to finalize the first restructuring. However, it is important to mention here that the delay in government clearances (signing of Subsidiary Agreement and obtaining Legal Opinion) resulted in the first restructuring becoming effective only in end April 2012, while the board clearance came in June 2011. Despite the delays, this phase also witnessed the starting of operations in the DSCs and ICPSCs for both target populations. The government in consultation with the task team took key decisions pertaining to starting of additional centers due to cost savings, partial cancellation and the second restructuring of the project aimed at focusing activities to core areas. This phase thus became critical to the overall success and efficiency of the project. 90. Phase 3 (January 2014 – June 2016): This phase was the culmination of the second restructuring and was significantly marked by the speedy government clearances and provision of services through the DSCs and ICPSCs. The government also worked with the task team towards finalization of the third restructuring that was primarily aimed at partial cancellation of project financing. 28 91. In conclusion, the Government of Bangladesh through MOSW also established 53 disability service centers on its own apart from their work on the DCAR project, thereby displaying their intentions and commitment. Lastly, the allocation of budget for continuation of these services under Components 1 and 2 to the target population has ensured sustainability of the important initiative. Their performance could have further expedited project achievements, if clearances had been more efficient. The Government’s performance has therefore been rated as Moderately Satisfactory. (b) Implementing Agency or Agencies Performance Rating: Moderately Satisfactory 92. The overall responsibility of project implementation was with the JPUF and DSS. While JPUF is an autonomous institution established under the MOSW, the DSS is a designated Department under the same Ministry. While the status of DSS did not change throughout the project period, JPUF underwent a significant change and restructuring i.e. transitioning from a non-government majority entity to one with majority government representation. 93. Phase 1 (July 2008 – June 2011): The implementing agencies during this phase were affected by the project uncertainties in addition to their own reorganization (in case of JPUF). This was particularly true during the period when the Bank team was in discussions with the MOSW for redesigning the project in line with the revised stance of the then government. The implementing agencies were also engaged with the task team to finalize the first project restructuring (level 1). Therefore very little core project activity happened during this period. 94. Phase 2 (July 2011 – December 2013): During this phase both implementing agencies began project operations. Staff recruitment for both the PIU as well as those for the service centers was conducted. The project became operational with the starting of 7 ICPSCs and 10 DSCs. In order to improve project efficiency and outcomes, the implementing agencies supported the task team in the second restructuring. This restructuring also brought focus on the core activities of the project. 95. Phase 3 (January 2014 – June 2016): Despite the delay, the third phase saw the project achieving most of its results. Both the implementing agencies - JPUF and DSS successfully established 103 centers (50 through project support) and 11 centers respectively for their target populations. Despite shortages of technical human resources, the JPUF has been able to hire 367 staff (in 50 project supported centers) and DSS has a staff strength of 219 in the PIU and the centers. As indicated by the third party spot checks conducted, beneficiary satisfaction with the services rendered has been high. Other results of the spot checks have also been encouraging. 96. Moreover, both agencies have achieved most of the outcomes as per the results framework. On fiduciary issues, both agencies had capacity issues. However, by phase 3 when the bulk of procurement was undertaken, the implementing agencies showed enhanced capacities. As discussed in the section on procurement, there were instances of mis-procurement during phases 1 and 2 that were brought to notice by the task team. The implementing agencies on their part took prompt corrective action. Lastly, in the final 29 phase of the project, the agencies were able to perform well and show high levels of efficiency and effectiveness. The performance has therefore been rated as Moderately Satisfactory. (c) Justification of Rating for Overall Borrower Performance Rating: Moderately Satisfactory 97. Despite initial delays owing to change in Government priorities at the Project outset, the borrower has worked towards realigning the project without losing sight of the target population and its needs. The borrower initiated services for persons with disabilities and children at risk and supported expansion and delivery of services, thus ensuring sustainability of operations. The overall borrower performance has therefore been rated as Moderately Satisfactory. 6. Lessons Learned (both project-specific and of wide general application) 98. It is important for the project team to spend adequate time and resources on preparation to assure high quality at entry, which will be more likely to result in smooth preparation. Furthermore, it might be even more important for projects that are first of a kind or are the first project in a portfolio of a given global practice. For example, the relatively fast preparation of the DCAR project allowed for the first lending project in the SP portfolio in Bangladesh. Nonetheless, poor quality at entry resulted in the need for multiple restructurings and significantly slowed down the implementation for the first five years of the project. Furthermore, given implementation challenges this project posed reputation risks for the World Bank in social protection sector in Bangladesh diminishing the possible “foot in the door� gains. Lastly, low quality at entry did not allow for the learning from this project to happen through impact evaluation and other tools. 99. In low capacity environments it might be better to have simpler design. While the initial project design did build on evidence of successful and innovative interventions, it did not properly tailor them to the context of a very complex challenge (i.e. service deliver to PWD and children-at-risk) with a very low capacity implementing agency. The subsequent restructurings were all an exercise in simplification and streamlining of the design. 100. Spend time on political economy analysis. In a situation where the project team knew that there would be election just after the appraisal, it would have made sense to invest more heavily into a proper political economy analysis and risk assessment. It is unlikely that the team would have been able to foresee the major changes in the government’s stance on social welfare policy, but it is possible that there were signs that pointed to this possibility and proper political economy analysis could have allowed to assess this. At the very least, it would have informed the project of necessary time to build in for implementation delay due to government transition, or would have identified how vulnerable the project design could be to the change of particular counterparts. 30 101. Restructuring even though painful can be beneficial. Despite the fact that all the project restructurings, and in particular the first one, were associated with significant implementation delays and involved investment of time and effort from both the Government and Bank, they are the reason that the project did manage to achieve and surpass what it initially set out to do contributing to the cause of the most vulnerable segments of the society. The team should be applauded for its patience and willingness to persevere through all the complications that come with restructurings and for its use of this tool to make the project more efficient and effective. 102. It pays off to stay on the course. The team started working with the low capacity government agency on a very complicated project. The implementation of the project was not easy, but it did follow an upward trajectory. Most importantly, through the years in which the project was implemented the capacity of DSS and JPUF increased and now both are cited in the NSSS 2015, with DSS playing a major role in the implementation of social security programs and being a counterpart to the new lending project on Safety Net Modernization. 103. Implementation efforts should be recognized. The project persevered and delivered only due to relentless efforts of the team leaders that came after the appraisal, who had to sustain multiple restructurings due to poor quality at entry. The efforts that go into supervision of successful implementation should be recognized in the same way as the ability to deliver a project to the Board. This would also show the increased commitment of the World Bank Group to results and to long term impacts of its work. 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies (b) Cofinanciers (c) Other partners and stakeholders (e.g. NGOs/private sector/civil society) 31 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) Component 1 21.6 11.5 53% Component 2 9.14 5.7 62% Component 3 2.75 0.00 0% Total Project Costs 33.49 17.88 52% (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (USD (USD Appraisal millions) millions) Borrower 0.00 0.00 0.00 International Development 33.49 17.88 52% Association (IDA) 32 Annex 2. Outputs by Component Component 1: Fifty Integrated Disability Service Centers (IDSCs) were built and made operational and equipment was upgraded in addition fifty three IDSCs thus reaching 103 IDSCs in total and covering every single district in Bangladesh. Each IDSC provides a rich number of services including: (i) Identification of different Types of Disability; (ii) Psycho Social Counselling; (iii) Physiotherapy; (iv) Audio Level Test (Audiometry); (iv) Occupational Therapy; (v) Visual Level Test (Optometry); (vi) Speech & Language Therapy; and (vi) Assistive Device Distribution. Each center also has an Autism and Neuro Development Disability Corner (NDDC) with a toy library thus addressing needs of children with disabilities. Furthermore, specifically for children with disabilities each center provides psycho social counseling, group therapy play and a number of other therapy services. Overall through IDSCs the project has covered 235,883 people with 1,163,150 service transactions. Training: The project has also supported training for staff at JPUF and project supported IDSCs through study tours, short training courses, and orientation training. Additionally 3,045 concerned government officials at union, upazilla and district levels trained on rights of persons with disabilities and available services Assistive Devices: JPUF through the Integrated Disability Service Centre (IDSC) is also providing assistive devices like wheel chair, standing frame, crutch, elbow crutch, tricycle, artificial limb, hearing aid etc. among the PWDs. Thirty Two Mobile Therapy Vans: The project has also financed thirty mobile therapy vans, which expanded the coverage of JPUF in particular in rural areas. By July 2016 the project reached more than 91,000 beneficiaries through MTVRs with 124,133 transactions. MIS: MIS for managing transactions at JPUF has been developed by DCAR project. Data has been is currently being entered into the system by JPUF. More than 800 staff between JPUF and DSS has been trained on the use of the MIS (see Component 2). Component 2: Eleven ICPSCs established and made operational: Under component two 11 ICPSCs were build. Then during the second restructuring in 2013 additional four ICPSCs were build and operated at Faridpur, Kushtia, Barguna and Cox’s Bazar districts for more coverage and protection of children at risks. For safe environment for children at risks, DCAR project provided services to 15 categories of children at risk and provided them shelter, food, clothing, medical facilities with psychosocial protection, and education. Each ICPSC comprised of two houses: one for boys and one for girls. According to Government documents each center was operated 33 by a dedicated team of 20-24 staff who were educated and trained in their respective field. Furthermore, to ensure psycho-social protection of residential children two psychologists were appointed for each ICPSC. In addition one Child Protection Policy Specialist was appointed for ensuring child protection issues were adequately addressed. Operational guidelines on child protection issues established: To strengthening the existing Child Protection and development programs, a separate chapter was included in the operational guidelines regarding child protection issues. In addition, a code of conduct has been developed for those who nurture the children under the custody of DCAR project. The operational guidelines and code of conduct were developed not only for DCAR project but also for other similar projects and programs of DSS. Training on child protection services and on use of MIS: For institutional capacity building, training on MIS management had been provided to officials from upazila and zila social services offices, DSS headquarters, MoSW (approximately 800 staff trained). Furthermore, other child protection related training had been conducted for DSS officials and DCAR project staff (approximately 900 staff trained according to the Government’s project completion report). MIS: MIS for managing various allowances has been developed by DCAR project. Data has been is currently being entered into the system by DSS authority. After completion of data entry every official will be able to monitor the allowances distribution and utilization, which will increase the management capacity of DSS. 34 Annex 3. Economic and Financial Analysis The project increased coverage and quality of services provided both to children at risk and people with disabilities. At the outset of the project the Economic and Finance analysis using a set of assumptions and national level data on PWD and children-at-risk established that not properly integrating PWD and children-at-risk costed Bangladesh US$1.2-$1.8 billion and US$9 million per annum respectively. The studies find that disability often leads to a vicious cycle of human capital accumulation that results in continued poverty 27 . Thus provision of services to disabled does target an important economic shortcoming that can help alleviate poverty now and in future generations. Furthermore, there is some evidence that focus on service provision in addition or as opposed to focus on just provision of disability benefit payments is associated with a smaller negative effect of disability on workforce participation 28 . Similarly there is evidence that reintegration of children-at-risk can be beneficial for society. For example, a recent study based on US data finds that reintegration of at risk youth early in their life saves society about $2.6-$4.4 million29, and yet another study based on US data presents quite significant societal gains (close to $2,000 per student) from investing in children-at- risk30. While a full economic and financial analysis was done during preparation building on it was not possible at the ICR stage. The analysis did not follow any of the methodologies outlined in the ICR and instead presented the gains from incorporating people with disabilities into employment and getting children at risk into education. While these are noble goals the project as initially designed and after restructuring cannot claim credit for either of the outcomes as the project did not link PWDs to jobs and neither did it link children at risk to education. Output based nature of the project and lack of impact evaluation limit the ability to properly assess gains from the project and in term conduct a meaningful quantitative assessment of its efficiency. Nonetheless the ICR will present basic economic analysis that could allow for an assessment of project’s efficiency. The analysis looks separately at three project implementation phases: (i) Phase 1: July 2008 – restructuring in May 2011; (ii) Phase 2: restructuring in May 2011 – December 2013; (iii) Phase 3: December 2013- project closing. The ratings for these three respective phases are: (i) Negligible, (ii) Modest, and (iii) Substantial. 27 Filmer, D. (2008). Disability, poverty, and schooling in developing countries: results from 14 household surveys. The World Bank Economic Review, 22(1), 141-163. 28 Jones, M. K. (2008). Disability and the labour market: a review of the empirical evidence. Journal of Economic Studies, 35(5), 405-424. 29 Cohen, M. A., & Piquero, A. R. (2009). New evidence on the monetary value of saving a high risk youth. Journal of Quantitative Criminology, 25(1), 25-49. 30 Heckman, J. J., & Masterov, D. V. (2007). The productivity argument for investing in young children. Applied Economic Perspectives and Policy, 29(3), 446-493. 35 Furthermore, ICR team will also present operational costs per beneficiary. Unfortunately, there are no comparable projects to look at to assess efficiency. Nonetheless, given that the design of this project has greatly informed the design of Bihar Integrated Social Protection project design. Phase 1 of the project produced no outputs and thus cannot be linked to economic returns. Furthermore, it cost the most in terms of supervision. This of course could be explained by need for extensive involvement of the Bank in achieving a mutual agreement with the government on the new project design. Nonetheless, if the initial design had taken into account the risks of changing government properly extensive supervision would not be necessary. Hence efficiency of the project at that time is rated as Negligible. Phase 2 of the project was based on a new design. During this phase despite delays in the approval of the DPP the project managed to contribute to building of 7 new ICPSCs and 20 new DSCs and to upgrades in equipment in the other 23 existing DSCs. Furthermore, during this phase cost saving opportunities were identified, which lead to increased efficiency of the use of project money. And despite upcoming restructuring supervision costs were reduced in comparison to the previous phase. Efficiency of the project at the time is rated as Modest. Phase 3 of the project was focused on a stream lined design with particular attention to activities critical to the achievement of the PDO. This in itself increased the efficiency of the project. During this time the project undertook important activities that resulted in the following outputs:  30 more DSCs (resulting in total of 50 DSCs supported by the project) and upgrading of equipment in 50 government supported DSCs (thus providing services to 370,464 beneficiaries)  Provision of assistive devices  4 more ICPSCs resulting in total of 11 ICPCS supported by the project (where out of more than 6,000 children, 4100 were reintegrated and about 500 received training with some finding subsequent wage and self-employment31)  32 mobile vans (which are crucial given that accessibility is one of the key obstacles to receiving proper healthcare for people with disability32)  MIS that is envisioned to allow officials of both JPUF and DSS to track utilization of funds and thus to improve their management capacity  Training for staff working at DSCs, ICPSCs, and concerned officials at JPUF, DSS and MosW on working with people with disabilities and on child protection  Creation of manuals to ensure improved standards of work with children at risk and people with disabilities. 31 There is no data to allow for an exact attribution of employment status to the training received and thus the ICR team did not quantify this benefit to be conservative. 32 UN World Health Organization (WHO), World Report on Disability: Summary 2011, WHO/NMH/VIP/11.01 available at http://www.refworld.org/docid/50854a322.html [accessed 25 October 2016] 36 Furthermore, project money helped the government to secure a budget allocation to maintain 100+ DSCs, and 11 ICPSCs. This was evidenced by allocation of 503 million taka (USD 6.29 million) by MoSW to “Protibondhi Sheba O Shanhajji Kendro� Program under JPUF and allocation of 205 million taka (USD 2.56 million) by the Finance Division to the “Sheikh Russell Training and Rehabilitation Service Centers� under DSS. Given that only a much smaller allocation existed prior to the closing of project this in itself could be counted as a benefit as it ensures access to necessary care services for people with disabilities and children at risk. Lastly the cost of supervision during this phase was the lowest of the three, which also indicates at more efficient allocation of resources. Thus efficiency of Phase 3 is rated as Substantial. Tables 3.1 and 3.2 present per beneficiary costs for both components. As mentioned above there are no benchmarks for the ICR team to compare those two given variety of services covered by various disability and children at risk interventions. However, these could serve as a useful benchmark for future similar projects. Table 3.1 Cost per beneficiary for Component 1 Disability Service Centers (DSCs) Taka (lakh) Taka USD $ 2215.3 (a) Service provision to DSCs 221,530,000 2,879,890 $ 950.89 (b) DSC Staff trainings 95,089,000 1,236,157 (c)Goods and services for $ 6106.4 DSCs 610,640,000 7,938,320 (d) JPUF management and $ 545 institutional expenses 54,500,000 708,500 (e)Institutional development, $ 400 training and capacity building 40,000,000 520,000 Total number of 370,464 beneficiaries Cost per beneficiary (DSCs $ only: (a)+(b)+(c)) 0.03 2,502.97 32.54 $ Total cost per beneficiary 0.0 2,758.1 35.85 37 Table 3.2 Cost per beneficiary for Component 2 Integrated Child Protection Service Centers (ICPSCs) Taka (lakh) Taka USD 4180 $5,434,000 (a) Provision of services for ICPSCs 418,000,000 233.57 $303,641 (b) Operating expenses 23,357,000 1094.72 $1,423,136 (c ) Good and equipment 109,472,000 (d) Total costs (including staff trainings and institutional capacity 5730.68 $7,449,884 building but excluding MIS and 573,068,000 honorarium travel expenses) Total number of beneficiaries 6,193 Cost per beneficiary (only service 0.7 67495.6 $877 provision for ICPSCS: (a)) Cost per beneficiary (only for 0.9 88943.8 $1,156 ICPSCS: (a)+(b)+(c)) Cost per beneficiary (total: (d)) 0.9 92534.8 $1,203 38 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Burhanuddin Ahmed Consultant GED06 Shaikh Shamsuddin Ahmed Consultant GEDDR AFTEE Susan E. Hirshberg Senior Education Specialist - HIS Iqbal Kaur Sr Social Protection Specialis GSP01 Mohammad Abdullah SARPS - E T Consultant Sadeque HIS Kalanidhi Subbarao Consultant GSP05 Andrea Vermehren Lead Social Protection Special GSP01 Co-TTL (2008) Qaiser M. Khan Program Leader AFCE3 TTL (2008) Supervision/ICR Lead Social Development Maitreyi Das GWAGP TTL (2009-2011) Specialist Jessica Leigh Leino Senior Economist GSP06 TTL (2011-2014) Yoonyoung Cho Senior Economist GSP06 TTL (2014-2016) Burhanuddin Ahmed Consultant GED06 Shaikh Shamsuddin Ahmed Consultant GEDDR Mohamed Ihsan Ajwad Senior Economist GSP03 Md. Mahtab Alam Program Assistant SACBD Co-TTL (2014- Ashiq Aziz Social Protection Specialist GSP06 2016) Gertrude Cooper Temporary GED06 AFTEE Susan E. Hirshberg Senior Education Specialist - HIS Tanvir Hossain Senior Procurement Specialist GGO06 Iqbal Kaur Sr Social Protection Specialist GSP01 Shonali Sen Mandal Consultant GSPDR Aleksandra Posarac Program Leader EACPF Mohammad Abdullah SARPS - E T Consultant Sadeque HIS Kishor Uprety Senior Counsel LEGAM Andrea Vermehren Lead Social Protection Specialist GSP01 Pravesh Kumar Social Protection Specialist GSP06 Srinivas Varadan Social Protection Specialist GSP06 TTL (ICR) 39 Anastasiya Denisova Economist GSP06 Co-TTL (ICR) (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) USD Thousands Stage of Project Cycle No. of staff weeks (including travel and consultant costs) Lending FY07 7.81 42.23 FY08 62.45 380.14 FY09 44.88 76.51 Total: 115.14 498.89 Supervision/ICR FY08 0 0.00 FY09 12.54 85.18 FY10 46.15 127.14 FY11 27.31 122.33 FY12 35.71 132.01 FY13 36.33 221.85 FY14 25.42 135.86 FY15 26.28 116.35 FY16 19.19 109.93 FY17 2.4 11.88 Total: 231.33 1,063.6 40 Annex 5. Beneficiary Survey Results (if any) Third party spot checks were conducted for both components 1 and 2 in 2014 and 2016. These checks assessed the functionality of service centers, staff skills, their satisfaction and motivation, beneficiary satisfaction. The key findings of the third party spot checks conducted to assess the performance of Disability Service Centers and the Integrated Child Protection Service Centers (ICPSC) are given below: 1. Spot check conducted in Disability Services Centers (DSCs) in 2016 a) The infrastructure and facilities: All the centers that were surveyed underwent daily cleaning, have a reception desk, internet facility and electricity connection. 83% had other infrastructure and facilities such a visible sign board and availability of tap water was 72.2%. Overall facility/service index, which is an average of response rates across the parameters, is at 79.0%. Only 50.0% of the DSCs had accessibility ramp. This was primarily due to the fact that several of these centers were operating from rented premises and the scope for construction of ramps was very limited. The DSCs scored relatively high with regard to the overall equipment index: 80.6%. However, only 61.1% of the DSCs reported to having a working IPS/generator, that could lead to inefficiencies in treatment of patients/beneficiaries. b) DSC staff competency and satisfaction: All the center staff that was interviewed (100%) were aware of at least one act/law related to disability as well as the Protibondhi Shurokkha Ain (Disability Security Act 2013). 71.1% knew about the role of JPUF. About 86.7% received on-the-job training and 60% attained certificates/specialized courses. In terms of average staff satisfaction with their jobs, 62.9% fully agreed that they were satisfied while 17.8% were somewhat satisfied. 47.3% of DSC staff was fully satisfied with DSC’s facilities/services while 35.9% were somewhat satisfied. c) Satisfaction levels of Persons with Disabilities: Persons with disabilities (PWDs), have a relatively high level of satisfaction with DSC facilities and an even higher level with DSC Services. Overall, 86.6% were satisfied with DSC facilities, while 94.6% people were satisfied with the services DSCs. 98.4% reported satisfaction with the registration process, 97.6% with the behavior of staff, 95.4% with advice received from the DSC and 94.3% reported satisfaction with the competency of the therapist/technical staff. d) Community Knowledge and awareness about DSCs: About 29.2% of all community respondents had overall awareness about DSC while 55.3% knew that this was the only service provider for PWDs. With regard to community knowledge and awareness, the response rates are for the parameters 9.4% recognized that “DSC staff visited the community� and 19.5% felt that “People inform others about DSC�. In terms of specific knowledge about DSCs, 77.8% knew that there is disability allowance for 41 PWDs, while 34.4% knew that PWDs can access assistive devices at DSCs. About 39.9% have overall awareness about DSCs. e) Impact of Mobile Therapy Vans: 82.4% of DSC representatives said they were very useful and 11.8% felt that it was somewhat useful, indicating high level of acceptability of Mobile Therapy Vans among DSC Staff. In terms of the benefits of this service, DSC representatives were most likely to mention the positive impact of Vans on campaign activities (76.5%), social awareness regarding disabled people (52.9%), and accessibility to remote areas (35.3%). Representatives from all the surveyed (17) DSCs covered by Mobile Therapy Van said that the vans are able to provide all the necessary services available at the centers. However, 59% said they face certain difficulties in providing the service. The main problems have to do with high patient numbers (30.0%), inadequate personnel (30.0%), and problems with the generator in the van (30.0%). 2. Spot check conducted in ICPSCs in 2016 a) The infrastructure and facilities: Among the 11 ICPSCs surveyed in total, all of them (100%) had supply of water, sign board, electricity and internet facilities. 63.6% had access to deep tube well, 27.3% to tube wells and 36.4% to tap/supply water. There is room for improvement with regard to availability of back-up generators, adequate office equipment and furniture. The overall functionality assessment of ICPSCs is 76.1%. b) ICPSC staff competency and motivation: All of the 55 ICPSC Staff (100%) interviewed had knowledge about roles of ICPSC for CAR in that they were able to mention at least 3 roles. All the staff also understood his/her job responsibility (mentioning at least 5 responsibilities). Moreover, 98.2% were knowledgeable about case assessment and management, while 94.6% were knowledgeable about child protection. Overall assessment puts staff competency at 89.7%. The average motivation was assessed at 78.9%. With regard to availability of training, only 30.9% believed that they received enough training in order to accomplish their present work. c) Satisfaction levels of children at risk: Over 90% report to being satisfied with the services provided at the ICPSC. 82.5% reported satisfaction with the rehabilitation/reintegration plan. In 13 out of the 16 parameters for satisfaction, over 94.0% of beneficiaries were satisfied. The satisfaction rate was 100% for services provided at the ICPSC, Center’s facilities, and Accommodation. 3. Spot check conducted in Disability Services Centers (DSCs) in 2014 a) The infrastructure and facilities: At a functional level, the DSCs were found to be moderate with almost 80 percent of surveyed centers were found to meet basic requirements on infrastructure and facilities. All DSCs that were surveyed had electricity and internet connection as well as daily cleaning services and reception desk. Internet connectivity in DSCs located in remote areas was found to be poor and often with disrupted communication and data sharing on customers’ services. The majority had visible sign boards facilitating the provision of information to seekers for services for 42 PWDs. Since the centers were operating from a rented premises that made renovation / construction difficult, only 57 percent of DSCs had accessible ramp facilities for PWDs. Only 48 percent of DSCs reporting adequate access to tap water. 79 percent of IDA and GoB funded DSCs had adequate number of therapy equipment and tools for disability services. However, the spot checks found that some equipment were out of order or had become obsolete and required replacement. b) Staff competency and satisfaction: Overall staff competency across DSCs aggregated in the composite index, was found to be very low with only 39% of staff assessed as competent. Staff knowledge about GoB and JPUF rules and regulations was particularly found to be lacking. Only 18 percent of staff in surveyed DSCs had clear knowledge and understanding about GoB disability act and rules for protection and services for PWDs. Similarly, only 18 percent of staff were able to demonstrate their knowledge about the roles and rules of JPUF. Almost 80 percent of staff including those in technical and non-technical staff category working received some on the job training. Overall staff satisfaction towards their job was low at 30%. Staff expressed a higher level of satisfaction with facilities and treatment offered in DSCs with 81 percent expressing that they are either satisfied or strongly satisfied. c) Satisfaction levels of Persons with Disabilities: beneficiaries positively rated the DSC facilities and services, with almost 80 percent expressing their overall strong or moderate satisfaction with the various DSC facilities and services. Although only 33 of the 210 PWDs received assistive devices, those that did receive these devices expressed a high level of satisfaction with their devices. Over 90% were satisfied or strongly satisfied with the registration process, behavior of staff, competency of therapist / technical staff and the advice they received from the center. d) Community Knowledge and awareness about DSCs: The survey team conducted 40 FGDs with communities within the periphery of IDA funded DSCs and 22 FGDs within the periphery of GoB funded DSCs. 70 percent of respondents had at least basic awareness about DSCs. Only 47 percent FGD participants (50 percent from GoB and 45 percent from IDA financed DSCs) said that they have seen staff from DSCs visit their community to provide information, and identify PWDs and advise them to visit DSCs. The composite index reflected an overall high level of knowledge within the local community members interviewed in the FGDs, with 87 percent of respondents having some knowledge and awareness about NGO and government services for PWDs. Almost all focus group respondents said they were aware of services offered by JPUF. 4. Spot Check conducted in ICPSCs in 2014 a) The infrastructure and facilities: Out of a total of 7 ICPSCs, 4 were found to be functioning at a moderate level, and met the basic requirements for logistics, equipment, facilities and services to the children. Many centers surveyed lacked amenities such as tap water, telephones and appropriate office equipment. 43 b) Understanding of Union Social Workers (USWs): The Union Social Workers are the centers link with the community. They identify children at risk and facilitate their reintegration process when leaving the ICPSCs. Those interviewed in the spot checks generally had a good understanding of their responsibilities and ICPSCs services. About 95 percent of all USWs reported that staff from ICPSCs had visited homes of children at risk in their community at least once and also met the Union Parishad (town union) chairman and other community members. However, about 45 percent interviewed expressed difficulty in appropriately identifying children at risk and following up on their re-integration to the community. Only about 45 percent of USWs could correctly identify children’s eligibility for enrolment with ICPSCs. About 63 percent of USWs confirmed that they knew about ICPSC facilities to serve CARs, but only a few USWs interviewed knew about other institutions serving CARs run by GoB and NGOs specializing in children rehabilitation, protection and development. c) Staff competency and satisfaction: Almost 80 percent of staff holds relevant educational qualification with specialized degrees or have completed related courses such as psychotherapy, counselling, and child psychology. Less than half of the interviewed staff has received on the job training in the relevant areas. Only 46 percent of staff has shown sufficient knowledge about case management and assessment. Only 66 percent of staff had an understanding of the needs of children and services required for them, while only 61 percent of staff were able to articulate different roles of ICPSCs for CARs. Majority of staff was generally satisfied or highly satisfied with their work and quality of services. However, the level of satisfaction was higher for work compared to the quality of services provided by ICPSCs, indicating that there is room for improvement in services. d) Satisfaction levels of children at risk: Children at risk expressed high levels of satisfaction with food, accommodation, entertainment and healthcare facilities, and opportunities for development, although a few children expressed their dissatisfaction with the quantity of clothes they received from the centers. Almost all of the children expressed their high level satisfaction on cleanliness, discipline of the center, entertainment facilities, staff behavior, and quantity and quality of food they were getting. In addition, more than 95% children were highly and moderately satisfied with accommodation, healthcare, non-formal education, life skills training, referral services for vocational training, and psychosocial counselling services. All of them reported satisfaction about the competency of the staff. About 23% of CARs interviewed expressed their dissatisfaction about rehabilitation and reintegration plans. 44 Annex 6. Stakeholder Workshop Report and Results (if any) N/A 45 Annex 7. Summary of Borrower's ICR A. Summary of Project Completion Report (PCR) submitted by Jatiyo Protibondhi Unnayan Foundation (JPUF): Jatiyo Protibondhi Unnayan Foundation (JPUF) is an autonomous entity under Ministry of Social Welfare. It was established in November 1999 under the society’s registration act 1860 headed by an additional secretary. JPUF aims to serve persons with disabilities (PwDs) of Bangladesh. JPUF has implemented one of the components of DCAR project titled as ‘Promotion of Services and Opportunities to the Disabled Person in Bangladesh’. This component aims to strengthen the capacity of JPUF to formulate and implement disability policies and programs, increase the coverage and quality of disability services and programs, and build up public awareness on the rights of persons with disability in Bangladesh through Integrated Disability Service Centers. Key Activities: JPUF has set up 101 Integrated Disability Service Centers (IDSCs) out of which 50 IDSCs have been set up using IDA assistance and 32 Mobile Therapy Rehabilitation Vans (MTRV) which are now being operated by the Project Implementation Unit (PIU) of Head Office all over Bangladesh. The activities in the IDSCs include:  Identification: Types of Disability  Psycho Social Counselling  Physiotherapy  Audio Level Test (Audiometry)  Occupational Therapy  Visual Level Test (Optometry)  Speech & Language Therapy  Assistive Device Distribution There is also an Autism and Neuro Development Disability Corner (NDDC) with a Toy library in each service Centers with the following activities:  Different types of Therapy Services  Psycho Social Counselling  Group Therapy Play 32 mobile therapy vans that are one stop service for persons with disabilities are operational at district- upazila level since September, 2015 to strengthen outreach disability services all over the country. JPUF through the Integrated Disability Service Centre (IDSC) is also providing assistive devices like wheel chair, standing frame, crutch, elbow crutch, tricycle, artificial limb, hearing aid etc. among the PWDs. The total coverage and service transactions in the IDSCs and mobile therapy units is given in the table below. The project has covered until June 2016 235,883 people with 1,163,150 service transactions. 46 Total Registration & Service Transaction Details of PWDs under IDA project (up to June 2016) PwDs + Risk of PwDs PwDs + Risk of PwDs Service New Registration Transaction Under World Bank Femal Male Total Male Female Total IDA funded Project e 50 IDSCs 76,640 67,489 144,129 537,596 401,421 1,039,017 32 MTRVs 59,404 32,350 91,754 68,980 55,153 124,133 Grand Total 136,044 99,839 235,883 606,576 456,574 1,163,150 Sustainability: The Government of Bangladesh has taken initiatives to absorb the Project including all existing Staffs of PIU, 50 IDSCs and 32 MTRVs with all financial liabilities into Government Revenue Budget program. As part of this action, Budget has been sanctioned for FY 2016-2017 from the Ministry of Finance with code: 3900 as Special Help Grant. The Government has committed to extend the service to all sub-district offices. B. Summary of Project Completion Report (PCR) submitted by Department of Social Services (DSS): The Department of Social Services has implemented Component 2 of the DCAR project as part of its Services for Children at Risk (SCAR) initiative. This SCAR project was initiated and approved in 2009 by the Executive Committee of National Economic Council (ECNEC). Before approval MoSW scrutinized and assessed the justification of the project. After that the DPP was sent to Bangladesh Planning Commission (PC) for approval. Concerned sector of PC again scrutinized and presented before Project Evaluation Committee (PEC), an inter-ministerial committee, for recommendation. The committee assessed the project in terms of time, cost, justification, fund availability, concern ministries performances etc. and final recommended for approval. Objectives: The project was mainly designed for addressing two broad objectives a) providing services to the children at risks and b) strengthening institutional capacity of DSS through developing an MIS and necessary training to DSS officials. To achieve these objectives firstly 07 child protection centers and then another 4 centers were established in 11 districts including seven divisional towns. Project restructuring: The project was revised on two occasions involving changes in cost and implementation period of the project. The Services for Children at Risk (SCAR) project being part of the Disability and Children at Risk (DCAR) project was approved by Executive Committee of National Economic Council (ECNEC) on 24 November 2008 47 for a period of January 2009 to December 2013. Financial Agreement was signed on 28 December 2008 and the Loan Agreement became effective on 22 January 2009. The PEC meeting held on 11 May 2011 approved some structural modification of project and made provisions for recruitment of staff instead of engaging NGOs for implementation of field activities. A second revision of the project DPP was necessary and finally completed on 21 July 2014. At the very outset the decision to change implementation modality from NGO sub- contracting to GOB’s own involvement took a long time for which some other relative aspects such as manpower recruitment and their designation change were essential. For these it was necessary to revise the DPP. Due to modality change and project revision the actual implementation started on August 2012. But after starting implementation, though 2nd revision was done, the project ran smoothly up to the end of project period. Key Milestones: Activities under the Services for Children at Risk were completed on June 30, 2016 and inter alia, the following milestones achieved during life of the project:  The children at risks in our society have decreased.  Awareness has been grown among the beneficiaries of the project regarding shelter and services of project.  11 center established in 11 districts.  Operational Guidelines completed and Implemented  Code of Conduct for staff prepared and implemented  About 900 officer and staff received specialized training  More than 800 officer of DSS, JPUF, MoSW received management training in MIS. Achievement against project objectives: The project became fully operational beginning of 2012 with the opening of 7 centers located in 7 divisional districts. Additional 4 centers were added during 2015 in 4 other districts. From the beginning 6245 (3047 boys and 3198 girls) children were admitted in the centers among them 4456 (2226 boys and 2230 girls) were reintegrated in their families/relatives or rehabilitated at other similar organization. At the end of the project 1789 (821 boys and 968 girls) children were residing in the centers. Sustainability: After completion of the IDA credit funding the SCAR project activities will be continuing under Government of Bangladesh (GOB) funding. The project will be operating with the same manpower setup by special allocation of GOB fund of Bangladeshi Taka 2050.00 Lakh for the financial year 2016-2017. Provided the project performance retains the standards as expected and to the satisfaction of the ministry and the stakeholders the project is very likely to be continued under GOB funding. In the near future the project is expected to be funded by GOB as a regular revenue project and be enjoying the routine annual budgetary allocation under the revenue budget of GoB. In addition to continuation of the activities and the centers already in operation a proposal for further expansion to include additional newly created divisions have been submitted to the MoSW. 48 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders 49 Annex 9. List of Supporting Documents World Bank Documents Bangladesh Country Assistance Strategy: 2006 -2009. World Bank. Dhaka, Bangladesh 2006. Country Partnership Strategy of the World Bank Group for Bangladesh FY2016-2020. The World Bank Group. Dhaka, Bangladesh 2016. World Bank. 2008. Bangladesh - Poverty Assessment for Bangladesh : Creating Opportunities and Bridging the East-West Divide. Washington, DC. © World Bank. https://openknowledge.worldbank.org/handle/10986/7886 License: CC BY 3.0 Unported. Disability and Children at Risk (DCAR) Documents “Services for Children at Risk (SCAR) Project. Progress Report: January 2009-2012.� DSS. NFDDP, A/2, Section-14, Mirpur. Dhaka. December 2012. “Statistical Report on Impact Evaluation of 103 Integrated Disability Service Centers and 20 Mobile Therapy Rehabilitation Vans�. JPUF and MoSW. Government of the People’s Republic of Bangladesh. November 2015. Government of Bangladesh Documents Seventh Five Year Plan of the Government of Bangladesh: FY2016-FY2020. Government of the People’s Republic of Bangladesh. November 2015. National Social Security Strategy (NSSS) of Bangladesh. General Economic Division Planning Commission. Government of the People’s Republic of Bangladesh. November 2015. Other Publications Cohen, M. A., & Piquero, A. R. (2009). New evidence on the monetary value of saving a high risk youth. Journal of Quantitative Criminology, 25(1), 25-49. Filmer, D. (2008). Disability, poverty, and schooling in developing countries: results from 14 household surveys. The World Bank Economic Review, 22(1), 141-163. Heckman, J. J., & Masterov, D. V. (2007). The productivity argument for investing in young children. Applied Economic Perspectives and Policy, 29(3), 446-493. Jones, M. K. (2008). Disability and the labour market: a review of the empirical evidence. Journal of Economic Studies, 35(5), 405-424. Zufikar, Ali. “The Economic Costs of Disability in Bangladesh.� Bangladesh Development Studies. Vol. XXXVII, December 2014, No. 4 UN World Health Organization (WHO), World Report on Disability: Summary 2011, WHO/NMH/VIP/11.01 available at http://www.refworld.org/docid/50854a322.html [accessed 25 October 2016] 50