Document of The World Bank Report No: ICR2759 IMPLEMENTATION COMPLETION AND RESULTS REPORT (IBRD-72110 IBRD-78030) ON LOANS IN THE AMOUNT OF US$10.71 MILLION AND US$3.0 MILLION OF ADDITIONAL FINANCING TO THE REPUBLIC OF CHILE FOR A SOCIAL PROTECTION TECHNICAL ASSISTANCE LOAN July 30, 2013 Human Development Sector Management Unit Bolivia, Chile, Ecuador, Peru and Venezuela Country Management Unit Latin American and the Caribbean Region CURRENCY EQUIVALENTS (Exchange Rate Effective June 6, 2013) Currency Unit = Chilean Peso 1.00 = US$ [ 0.0020 ] US$ 1.00 = [ 504.80 ] FISCAL YEAR January 1 – December 31 ABBREVIATIONS AND ACRONYMS AF Additional Financing CAP Circles of Analysis of Practices CAS Commissions for Social Assistance (Comisiones de Asistencia Social) CASEN National Socioeconomic Characterization Household Survey (Caracterización Socioeconomica Nacional) CCT Conditional Cash Transfers CCC Chile Grows with You (Chile Crece Contigo – CCC) CONADI National Corporation for Indigenous Development (Corporación Nacional de Desarrollo Indígena) CS Chile Solidarity (Chile Solidario) FLACSO Latin American Faculty of Social Sciences (Facultad Latinoamericana de Ciencias Sociales) FONADIS National Fund for Disabilities (Fondo Nacional de la Discapacidad) FOSIS Solidarity and Social Investment Fund (Fondo de Solidaridad e Inversión Social) FPS Social Protection Index (Ficha de Protección Social) IEF Ethical Family Income (Ingreso Etico Familiar) INP National Institute of Social Security (Instituto Nacional de Previsión) IPDP Indigenous Peoples Development Plan IPP Indigenous Peoples Plan M&E Monitoring and Evaluation MIDEPLAN Ministry of Planning (Ministerio de Planificación) MIS Management Information System NGOs Non-Governmental Organizations PAD Project Appraisal Document PDO Project Development Objectives QAG Quality Assurance Group RIS Registry of Social Information RUN National Identification Number (Rol Unico Nacional) SERNAM National Women’s Service (Servicio Nacional de la Mujer) SENAMA National Service for the Eldery (Servicio Nacional del Adulto Mayor) SERPLAC Regional Secretaries of Planning (Secretarías Regionales Ministeriales de Planificación y Coordinación) SIGEC Management System for Agreements (Sistema de Gestión de Convenios) SIIS Integrated Social Information System (Sistema Integrado de Información Social) TA Technical Assistance UIFs Family Intervention Units (Unidades de Intervención Familiar) Vice President: Hasan A. Tuluy Country Director: Susan Goldmark Sector Manager: Mansoora Rashid Project Team Leader: Theresa Jones ICR Team Leader: Theresa Jones/Jorge Barrientos CHILE SOCIAL PROTECTION TECHNICAL ASSISTANCE LOAN CONTENTS Data Sheet A. Basic Information B. Key Dates C. Ratings Summary D. Sector and Theme Codes E. Bank Staff F. Results Framework Analysis G. Ratings of Project Performance in ISRs H. Restructuring I. Disbursement Graph 1. Project Context, Development Objectives and Design ............................................... 1 2. Key Factors Affecting Implementation and Outcomes .............................................. 6 3. Assessment of Outcomes .......................................................................................... 11 4. Assessment of Risk to Development Outcome......................................................... 16 5. Assessment of Bank and Borrower Performance ..................................................... 16 6. Lessons Learned ....................................................................................................... 20 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners .......... 21 Annex 1. Project Costs and Financing .......................................................................... 23 Annex 2. Outputs by Component ................................................................................. 24 Annex 3. Economic and Financial Analysis ................................................................. 43 Annex 4. Bank Lending and Implementation Support/Supervision Processes ............ 44 Annex 5. Beneficiary Survey Results ........................................................................... 45 Annex 6. Stakeholder Workshop Report and Results................................................... 46 Annex 7. Summary of Borrower's ICR ........................................................................ 47 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders ....................... 67 Annex 9. List of Supporting Documents ...................................................................... 69 MAP A. Basic Information Social Protection Country: Chile Project Name: Technical Assistance IBRD-72110,IBRD- Project ID: P082037 L/C/TF Number(s): 78030 ICR Date: 08/06/2013 ICR Type: Core ICR GOVERNMENT OF Lending Instrument: TAL Borrower: CHILE Original Total USD 10.71M Disbursed Amount: USD 11.99M Commitment: Revised Amount: USD 12.31M Environmental Category: C Implementing Agencies: Ministry of Social Development Cofinanciers and Other External Partners: N/A B. Key Dates Revised / Actual Process Date Process Original Date Date(s) Concept Review: 02/26/2003 Effectiveness: 06/18/2004 06/18/2004 Appraisal: 10/20/2003 Restructuring(s): Approval: 12/18/2003 Mid-term Review: Closing: 09/30/2007 01/31/2013 C. Ratings Summary C.1 Performance Rating by ICR Outcomes: Satisfactory Risk to Development Outcome: Low or Negligible Bank Performance: Satisfactory Borrower Performance: Satisfactory C.2 Detailed Ratings of Bank and Borrower Performance (by ICR) Bank Ratings Borrower Ratings Quality at Entry: Satisfactory Government: Satisfactory Implementing Quality of Supervision: Satisfactory Satisfactory Agency/Agencies: Overall Bank Overall Borrower Satisfactory Satisfactory Performance: Performance: i C.3 Quality at Entry and Implementation Performance Indicators Implementation QAG Assessments Indicators Rating Performance (if any) Potential Problem Project Quality at Entry No Highly Satisfactory at any time (Yes/No): (QEA): Problem Project at any Quality of No None time (Yes/No): Supervision (QSA): DO rating before Satisfactory Closing/Inactive status: D. Sector and Theme Codes Original Actual Sector Code (as % of total Bank financing) Other social services 100 100 Theme Code (as % of total Bank financing) Administrative and civil service reform 14 14 Indigenous peoples 14 14 Other social development 29 29 Poverty strategy, analysis and monitoring 14 14 Social risk mitigation 29 29 E. Bank Staff Positions At ICR At Approval Vice President: Hasan A. Tuluy David de Ferranti Country Director: Susan G. Goldmark Axel van Trotsenburg Sector Manager: Mansoora Rashid Christopher Chamberlin Project Team Leader: Theresa Jones Theresa Jones ICR Team Leader: Theresa Jones ICR Primary Author: Jorge C. Barrientos Theresa Jones F. Results Framework Analysis Project Development Objectives (from Project Appraisal Document) The objectives of the Social Protection Technical Assistance Loan are to support the efforts of the Government of Chile to implement Chile Solidario and lay the foundation of a national system of social protection. ii Revised Project Development Objectives (as approved by original approving authority) (a) PDO Indicator(s) Original Target Formally Actual Value Values (from Revised Achieved at Indicator Baseline Value approval Target Completion or documents) Values Target Years Indicator 1 : No. of families invited to participate in Chile Solidario Value quantitative or 91,539 225,000 648,014 Qualitative) Date achieved 12/31/2003 12/31/2003 12/31/2012 Comments Monitoring Indicator for the Social Protection Sector Adjustment Loan, (incl. % associated with Chile Solidario. Included for reference only. achievement) Indicator 2 : No. of families participating in Chile Solidario Value quantitative or 91,535 220,000 585,036 Qualitative) Date achieved 12/31/2003 12/31/2003 12/31/2012 Comments Monitoring Indicator for the Social Protection Sector Adjustment Loan, (incl. % associated with Chile Solidario. Included for reference only. achievement) Indicator 3 : No. of families receiving the Bono de Proteccion Familiar Value quantitative or 91,535 220,000 585,036 Qualitative) Date achieved 12/31/2003 12/31/2003 12/31/2012 Comments Monitoring Indicator for the Social Protection Sector Adjustment Loan, (incl. % associated with Chile Solidario. Included for reference only. Data on spending achievement) included in Annex 2. (b) Intermediate Outcome Indicator(s) Original Target Actual Value Formally Values (from Achieved at Indicator Baseline Value Revised approval Completion or Target Values documents) Target Years Indicator 1 : No. of institutions (databases) agreeing to exchange information Value (quantitative 0(0) 46(60) or Qualitative) Date achieved 12/31/2003 01/30/2013 Comments Includes 38 institutions with Registry of Social Information agreements currently (incl. % in force as of 1/30/2013 achievement) iii Indicator 2 : No. of variables incorporated into information system Chile Solidario participant demographic characteristics, status on minimum conditions, data Chile Solidario from social Value participant demographic protection index for (quantitative characteristics and status 80% of families in or Qualitative) on 53 minimum Chile, information conditions associated with participation in programs from databases mentioned in Indicator 1. Date achieved 12/31/2003 01/01/2013 Comments (incl. % achievement) Indicator 3 : No. of institutions adhering to proposed standards Value (quantitative 0 46 or Qualitative) Date achieved 12/31/2003 01/01/2013 Comments (incl. % achievement) Indicator 4 : % of Puente social workers trained Value (quantitative 100% 100% or Qualitative) Date achieved 12/31/2003 12/31/2012 Comments All Puente social workers received some mandatory training. More information (incl. % on types of training and system installed appears in Annex 2. achievement) Indicator 5 : No. of programs included in Chile Solidario monitoring system Since CS monitoring system is part of SIIS, programs of all Value institutions (quantitative 0 mentioned in or Qualitative) Indicator 1. Many institutions have more than 1 program. Date achieved 12/31/2003 12/31/2012 iv Comments (incl. % achievement) Indicator 6 : No. of Program Evaluations Value (quantitative 0 15 11 or Qualitative) Date achieved 12/31/2003 12/31/2003 12/31/2007 Comments Seminars were organized to discuss findings of all program evaluations. (incl. % Although number of evaluations fell short of target, discussions with programs achievement) through working groups covered numerous programs. Indicator 7 : Carry out surveys necessary for impact evaluation Value (quantitative 1 4 4 or Qualitative) Date achieved 12/31/2003 12/31/2003 01/07/2007 Comments (incl. % achievement) Indicator 8 : Creation of 4 supporting units in MIDEPLAN for technical work. Value (quantitative 0 4 4 or Qualitative) Date achieved 12/31/2003 12/31/2003 12/31/2004 Comments During 2006-2007, units were incorporated into operating structure of Executive (incl. % Secretariat for Chile Solidario. achievement) G. Ratings of Project Performance in ISRs Actual Date ISR No. DO IP Disbursements Archived (USD millions) 1 05/26/2004 Satisfactory Satisfactory 0.00 2 05/26/2004 Satisfactory Satisfactory 0.00 3 12/13/2004 Satisfactory Satisfactory 1.51 4 04/29/2005 Moderately Satisfactory Moderately Satisfactory 1.85 5 12/22/2005 Moderately Satisfactory Moderately Satisfactory 4.08 6 11/13/2006 Satisfactory Satisfactory 6.70 7 05/18/2007 Satisfactory Satisfactory 8.02 8 12/01/2007 Satisfactory Satisfactory 9.08 9 05/20/2008 Satisfactory Satisfactory 9.81 10 12/01/2008 Satisfactory Satisfactory 10.60 11 06/30/2009 Satisfactory Satisfactory 10.71 12 12/14/2009 Satisfactory Satisfactory 10.71 13 06/28/2010 Satisfactory Moderately Satisfactory 10.71 14 02/07/2011 Satisfactory Moderately Satisfactory 10.71 v 15 08/10/2011 Satisfactory Moderately Satisfactory 11.71 16 03/13/2012 Satisfactory Satisfactory 11.71 17 10/15/2012 Satisfactory Satisfactory 11.98 H. Restructuring (if any) Not Applicable I. Disbursement Profile vi 1. Project Context, Development Objectives and Design 1.1 Context at Appraisal A. Country and sector issues When the Social Protection Technical Assistance Project was appraised in 2003, Chile had experienced a slow-down in growth, but there were signals that the economy would begin to revive towards the end of the year with growth estimated at over 3 percent. The country continued to emphasize the importance of trade and direct investment to the growth and evolution of its economy. Chile had strong economic fundamentals: a credible monetary policy; rules-based fiscal policy; and a flexible exchange system, providing a strong foundation to support rates of growth above 4 percent in the medium- term. The share of the Chilean population below the poverty line had fallen from 40 percent in 1987 to slightly under 16 percent by 2000. The share in extreme poverty had dropped from nearly 13 percent in 1987 to about 6 percent in 1996, where it had remained. The rapid decline in poverty was mainly associated with economic growth and an increase in targeted social spending. Nevertheless, the Government was concerned that although the share of the population below the poverty line had declined, the share in extreme poverty stagnated. Analytical work suggested that an important determinant of persistent extreme poverty was exclusion from a variety of networks that provided access to public and private risk mitigation instruments. Indeed, the gaps in coverage of social protection programs tended to be most common among the poorest and in rural areas. Although Chile’s social protection institutions were relatively strong and well run, several weaknesses in the social protection system were making it harder for the Government to address exclusion. These included: (i) the inefficient use of available information; (ii) weakness of the targeting instrument; (iii) gaps in monitoring and evaluation of social programs and policies; and (iv)coordination failures at both the national and subnational levels. In May 2002, the Lagos Administration announced the Chile Solidario (CS) initiative, a package of legislative and administrative reforms to Chile’s poverty reduction and social protection policies. The objective was to bring Chile’s 225 thousand poorest households out of indigence poverty. B. Rationale for Bank assistance The Bank had worked with Chile over a long period of time on analyzing trends in poverty and the effectiveness of the Government’s poverty reduction strategy. In addition to poverty assessments, at the time of project preparation there had been a recent analysis of the social protection system (Chile: Household Risk Management and Social 1 Protection). The latter had informed the reforms that were being supported in this Project as well as the accompanying adjustment operation.1 The Government requested the technical assistance loan from the Bank in order to ensure adequate financing and advice during the implementation of the policy reforms that were supported under the adjustment operation. Chile Solidario Although Chile Solidario shares the focus of the classic Conditional Cash Transfer (CCT) Programs in the Latin America on improving the welfare of poor families and the human capital development of children, it differed from them in several important ways. First, it was more narrowly targeted because it was a tool of the government’s social protection strategy oriented to address the needs of extremely poor families. Second, while CCTs generally concentrated on promoting the utilization of health and education services by children and, to a lesser degree, adults, Chile Solidario’s approach was at the same time more personalized and focused on improving the integration and quality of a broader range of services to improve their effective use by C. Higherpoor extremely level families. objectives Third , in to CCTsthe which the project monetary transfer is central and usually large enough to contributed increase significantly the consumption of the beneficiary families. In Chile Solidario the transfer is relatively small and furthermore declines over time. Finally, in many CCTs families stay in the The World program Bank as long asGroup’s Country their children Assistance are attending schoolStrategy and in someforcases Chile setlonger. even out 3 broad goals: The duration of(i) sustaining economic participation growth in CS is limited to and social 5 years. In progress; (ii) promoting order to further clarify someinclusion, especially of these differences, theof components the of CS are described rural population in more detail and vulnerable below. groups; and (iii) modernizing the state, necessary for achieving the previous two objectives. Together with the complementary Social With the aim of linking families to the existing social network, the CS strategy was organized around 3 Protection components: Sector Adjustment Loan, the Project responded to the latter two objectives. It would support Support 1) Psycho-social Chile’s isproposed provided byreform of social a family coach policy, who works directed with the to 2 the yearspoorest family for using households, a methodology and incorporate focused initiativesofto on the achievement modernize specific the state tasks using in order a specially to facilitate designed methodology this and instruments. This was implemented jointly by FOSIS and municipalities. reform, including by supporting better management of information. In addition, Families sign an agreement to investment orwork with their technical assigned family assistance lendingcoach. This is services inthe theonly “conditionality�. area In order was of social protection to support the families, a monetary transfer (Bono de Protección) is provided with a value declining in time. It is in envisaged the Country intended Assistance to finance Strategy. the costs associated with the family’s incorporation into the network of social services and benefits. This component is envisaged as serving as the “Bridge� or Puente between the system and the families. 2) Participating 1.2 Original families ProjectareDevelopment guaranteed access to the existing Objectives and transfers monetary (PDO) that the family Key Indicators (as qualifies approved) for – family allowance, social pension and disability allowance, student retention subsidy, transfer to cover the costs of obtaining identification documentation, and the water subsidy. 3) Participating families were guaranteed preferential access to services, transfers, and benefits The development available objectives at the local level thatof the related were Projectto were the 7todimensions support the (53efforts minimumof conditions the Government in the of Chile areas to implement of identity, Solidario program the Chileemployment, health, education, and to family income, housing, lay the foundation dynamics). Thisof a national access was facilitated system with the of social orientation and protection. Theassistance of the family loan would supportcoach. Government actions in four key Although CS initially targeted extremely poor families, the system expanded later to cover other vulnerable groups, with the required adaptations. These included the elderly living alone (Vinculos), the homeless (Calle), and children of prisoners (Caminos). 1 Social Protection Sector Adjustment Loan (Ln. 7202-CH) 2 1.2 Original Project Development Objectives (PDO) and Key Indicators (as approved) The development objectives of the Project were to support the efforts of the Government of Chile to implement the Chile Solidario program and to lay the foundation of a national system of social protection. The loan would support Government actions in four areas:  Reforms to Chile’s poverty reduction and social protection policies, to reduce poverty and exclusion by creating greater access to social protection and social services for the neediest households (Chile Solidario);  Increase the efficiency and efficacy of social policy through improved targeting and better use of information by integrating, gathering and exchanging publicly and privately held data on households and relevant to social policy, into a national information system, as well as improvements to the country’s primary targeting instrument, the Ficha CAS;  Strengthening regular monitoring and impact evaluation of social programs and policies; and  Increasing the sustainability of reforms to poverty reduction and social protection policies, through clear assignment of roles and division of responsibilities between central government, municipal authorities and civil society. A set of key indicators for the technical assistance project were included in Annex 2 (Results Framework and Monitoring). In addition, the following monitoring indicators for the Social Protection Adjustment Loan (Ln. 7202-CH) associated with Chile Solidario were included for reference: • Number of families invited to participate in Chile Solidario • Number of families participating in Chile Solidario • Number of families receiving the “Bono de Protección Familiar� • Percentage of coverage of social programs and monetary transfers included in Chile Solidario among the neediest families. 1.3 Revised PDO (as approved by original approving authority) and Key Indicators, and reasons/justification There were no revisions to the Project Development Objective or key indictors made during the course of the Project. 1.4 Main Beneficiaries The primary target group for Chile Solidario was the estimated 225 thousand indigent households, a substantial share of whom would participate. Beneficiaries were expected to improve their standard of living, including, but not limited to level of income. If the project were successful, one would expect to see a decline in the share of households below the extreme poverty line, higher coverage of existing social protection policies and programs among the extreme poor, and some re-design of interventions to better address 3 the needs of this group. Other groups besides the extremely poor or those participating in Chile Solidario were expected to benefit from improvements in social protection policies because some reforms would affect a broader segment of the population. 1.5 Original Components Component 1: National Information System for Social Protection. The design and implementation of a national information system for social protection was aimed at addressing limitations of the lack of communication, articulation and dialogue among systems of information managed by different agencies. The basic approach was to use a holistic design for the gathering, handling, and management of data related to the social and economic characteristics of individuals. In the first phase, the design included coverage of all families participating in Chile Solidario. It would then expand to include families participating in all social programs, and finally, cover the total population. The aim was to take advantage of modern technologies in the area of hardware, software and communications to promote the interchange of information on the civil, social, and socioeconomic attributes of the current population and potential beneficiaries of social programs in Chile. The project financed consulting services necessary for the design of the system and the development of the procedures, standards, and rules governing its operation; software applications; computing equipment and data storage/processing services, and staff training. Component 2: Training. There were two training sub-components. The first one covered training of MIDEPLAN and other government officials, at both the central and local levels. The sub-component included activities to help MIDEPLAN progress in the design of a system of social protection, based on the initial experience with Chile Solidario, supported the implementation of the national information system for social protection, and trained staff in the supervision and evaluation of a system of social protection. The second sub-component strengthened the capacity of the social workers and others who work with the families of Chile Solidario. Based on the initial experience implementing CS as well as consultations with both social workers and supervisory staff, a “community of learning� approach was introduced to enhance the cross fertilization of experience among staff. This required a combination of different mechanisms for training and the formalization of knowledge and experience. Learning activities included: (i) structured small group discussions among practitioners on specific themes; (ii) a program of short-term exchanges between different municipalities or regions and contacts with technical experts; and (iii) formal training through courses, seminars, and conferences. All of these activities, as well as a virtual library, were supported through the design and implementation of an appropriate technological platform that operated through a special website. The project financed the design of curricula, the organization of courses, associated travel and materials costs, staff exchanges, national and international technical advice, seminars to discuss issues and problems arising in the course of implementation of Chile Solidario, and supporting technological platforms. 4 Component 3: System for Analysis, Evaluation, and Monitoring of Social Programs and Policies. The goal of this component was to strengthen capacity within the public sector to measure the impact of Chile Solidario and its associated programs. The project financed: (a) the design of an integrated system to monitor the management, processes and partial results of Chile Solidario; (b) the evaluation of the impact of selected aspects of Chile Solidario in addressing the needs of its beneficiaries; and (c) the carrying out of an impact evaluation of Chile Solidario in its entirety. Also included was the design and implementation of a pilot project of citizen accountability. Component 4: Institutional Strengthening of MIDEPLAN. In order for MIDEPLAN to carry out its designated role to coordinate the implementation of the new set of policies, a program of institutional modernization and strengthening for the ministry was required. This process aimed at strengthening MIDEPLAN’s role as the coordinator of social protection policy and programs in Chile. Additionally, several studies relating to issues linked to indigenous families were planned. In addition to consultancy services, the project financed office equipment for the Project Implementation Unit, as well as the refurbishment of office space. 1.6 Revised Components The Project’s components were not revised during implementation. However, with the Additional Financing, activities were incorporated to cover the extension of the social protection system to cover young children through Chile Crece Contigo (Sistema de Protección Integral a la Infancia) – Integrated System of Protection for Young Children or CCC). 1.7 Other significant changes In May 2008 the Government of Chile requested additional financing for the Project. Additional resources were required for activities related to Chile Solidario as well as for the extension of the social protection system to cover young children through the Chile Crece Contigo initiative.2 In fact, without the experience attained in the implementation of CS, particularly the strengthening of coordination arrangements, it would have been difficult to launch CCC. Additional financing was the best available mechanism to maximize development impact and results for the technical support to the scale-up of the social protection system in Chile. The activities proposed to be financed included evaluations for Chile Crece Contigo, training of staff to implement Chile Crece Contigo, the development of supporting materials and training to carry out recent modifications in the methodology for Chile Solidario, and analytical and process studies and were a natural continuation of what had been supported under the original Project. The Board approved the Additional Financing Loan on October 29, 2009; it was signed on January 7, 2010; and became effective on September 3, 2010. 2 CCC covers all children whose mothers receive prenatal care at public health facilities and/or who take their infants to the public health services for check-ups, an estimated 80 percent of children in Chile. 5 In December 2012, at the Government’s request, the Bank cancelled US$1.4 million from the Additional Financing. The main reason was the substantial delays which occurred at the beginning of implementation of the activities due to a combination of factors. These included the relative inexperience (with either public budget or Bank procedures) of the new team at MIDEPLAN who arrived after the new administrative assumed power in March 2010, the pressures they were under to respond to the reconstruction needs after the February 10 earthquake and the fact that any further extension of the closing date was impossible owing to the very short amortization schedule the Government had chosen for the AF loan. 2. Key Factors Affecting Implementation and Outcomes 2.1 Project Preparation, Design and Quality at Entry Project preparation and design were based on the Bank’s broad knowledge of the Chilean poverty situation and the Government’s policies and programs to address it. The Bank had argued in its 2001 Poverty Report that given the persistent rates of extreme poverty the Government should take a more proactive stance in its poverty reduction and social protection policies. Both the Poverty and the Household Risk Management and Social Protection Reports had indicated that despite remarkable gains in eradicating poverty, a new set of policies and interventions – buttressed with technical assistance to improve the use of information, including program monitoring and evaluation, and to strengthen institutional capacity – were required to reach Chile’s poorest groups and increase the effectiveness of poverty reduction and social protection policies. This analytical work provided a solid base to assess the Government’s Chile Solidario initiative and structure achievable development objectives consistent with the country’s needs and ability to implement policy reforms and establish the associated practices and instruments. The rationale for the Bank’s intervention was strong. The development objectives are considered appropriate. They were realistic and responsive to the Government’s priorities. The Project Development Objectives effectively combined a focus on supporting the implementation of a specific initiative (Chile Solidario) and the more ambitious goal of laying the foundations of a national system of social protection. This mirrored the view of Government authorities who believed that by maintaining the strict link between Chile Solidario and social protection reforms, wider changes in the social protection system would eventually take place in a way that was more politically possible and sustainable. The Government commitment to strengthen social protection was maintained throughout the project although reflected differently depending on political administrations. Each of the 4 components was relevant to the achievement of the Project Development Objectives. The project design aimed to use support for the implementation of Chile Solidario to lay the base for activities which would contribute more broadly to the strengthening of the social protection system and this in practice was achieved (see last section in Annex 2 for more detail). The components supported the main elements 6 needed to move toward a more integrated social protection system: a base of integrated social information, including an effective targeting mechanism; a monitoring and evaluation system; and the necessary training and strengthening of institutional capacity. Although the components (and the reforms behind them) were demanding, project design is not considered overly complex. The initial assessment in the PAD that risks were minimal was confirmed, although some predicted difficulties did materialize (challenges in closing the employment gap for the extreme poor and promoting greater participation of civil society). Two risks identified did not materialize. Pending legislation on Chile Solidario was passed and enacted on a timely basis and the targeting performance of Chile Solidario was good. The Quality Assurance Group (QAG) rated quality at entry as Highly Satisfactory. 2.2 Implementation Project implementation was generally satisfactory. Not surprisingly, given the duration of the project, there were several changes in key counterparts and in the institutional arrangements for project activities within first the Ministry of Planning and Cooperation an later in the re-named Ministry of Social Development. An important factor explaining these changes was the development of the social protection system itself and the priorities of different administrations. Initially, there were some difficulties because of a lack of sufficient cooperation between the Social Division of MIDEPLAN and both the Executive Secretariat of Chile Solidario and the Solidarity and Social Investment Fund (FOSIS), the latter responsible for Puente, the family counseling component of CS. These tensions were sorted out over time as the system was consolidated. The project design was sufficiently flexible to accommodate these developments. A factor that positively affected implementation of Chile Solidario, in particular Component 1 was the approval of the Chile Solidario Law (Law 19.949) and the norms issued for regulating its implementation including the interaction among the involved ministries and agencies. The law specifically established the creation of a “national registry of social data.� A key aspect introduced by the norms was the establishment of mechanisms to regulate the access to and privacy of personal data contained in the data bases incorporated into the information system. A factor outside of government control which delayed implementation of the activities under the Additional Financing was the February 2010 earthquake. MIDEPLAN was given an important role in organizing some reconstruction activities which occupied their full attention for several months during the first half of 2010. In addition, at the early stages of implementation of the AF, the government counterparts were somewhat unfamiliar with both government and Bank procedures. 7 2.3 Monitoring and Evaluation (M&E) Design, Implementation and Utilization The assessment covers 3 different aspects of M&E: the Integrated Social Information System (SIIS) which was supported in Component 1 of the Project; the impact evaluation for Chile Solidario, as well as the tracking of the specific indicators for the Project. Design. The Project through Component 1 supported the development of an integral M&E system for the social protection system. The conceptual design of an M&E system for social protection had a broad scope for an integrated system that would enable the availability of common information for the use and management of the inter-institutional network of ministries and agencies. The key design feature was that the integrated information system, in the long term, would become a management and operational tool for all social protection services provided by the public sector. The system would include a common registry and an integrated set of data bases of institutions in the social protection network. Additionally, a key component of the integrated M&E system was the design and implementation of a management information system (MIS) for Chile Solidario to enable its administration, the monitoring of its operations family-by-family, and providing information for evaluating processes and results. At a later stage, other programs also managed by MIDEPLAN were added to the Chile Solidario M&E system. One key design feature was the adoption of a modular approach which enabled a rapid expansion of the system when new programs were added. The Chile Crece Contigo program was incorporated, as well as other programs including support for homeless people, seniors and other vulnerable groups. Basic design features included a user- friendly approach, controlled hierarchical access, daily backup and recovery procedures, and a web based platform. The Bank and the Government agreed on the design of the impact evaluation for Chile Solidario during project preparation. As described in the PAD, final and intermediate objectives were clearly identified. Several options were identified to select the control group, taking advantage of design features and implementation arrangements for CS including its targeting methodology and planned phased roll-out. The design took into account already available data or planned surveys, particularly the main household survey in Chile (CASEN). For the specific project activities, adequate indicators were identified. However, in retrospect, some indicators for some components could have been more sophisticated with more of a focus on progress towards installation of a system. For example, the indicator of number of family coaches trained is very basic and should have focused instead on measurements related to the achievement of competencies (certifications, for example). Instead of tracking only the number of program evaluations, there should have been indicators on the number of programs which agreed on preferential access. Some of this data is presented in Annex 2. Also problematic were indicators presented as ratios for which the definition of the denominator might evolve over time, particularly for Component 1. In the case of training, some indicators proved impossible to track given the quantity and variety of training courses. The results framework recognized that most of the results of Chile Solidario (for example, number of participants and payment of the 8 transfer) were not directly attributable to the technical assistance project. However, several indicators for CS were included for reference because they were monitoring indicators for the complementary Social Protection Sector Adjustment Project and were important for the project context. Implementation. Implementation of the SIIS, including the MIS for Chile Solidario and other programs which were added to the system after 2007, was successful and is described in more detail in Annex 2. There were particular efforts made to ensure and improve data quality. Three surveys were carried out for the impact evaluation. In spite of the problems mentioned under design, MIDEPLAN adequately tracked the activities of the technical assistance project. Utilization. Earlier, the Government of Chile had taken the approach of evaluating social programs in the context of a budgetary system based on results. The M&E system, supported by the TA project, introduced the notion that social programs should start to keep records of their actions and basic data about their beneficiaries. This led to the establishment of a single national registry of beneficiaries of social programs enabling a more rational system for targeting resources to eligible beneficiaries minimizing the fraud and duplication of efforts. Ministries and agencies started to use information contained in the national registry to justify their budgetary request and to facilitate auditing of the use of public resources. This process enabled the establishment of a network of institutions participating in the program with specific programmatic supply of services, sharing annual goals through formal inter-institutional agreements, using a single targeting instrument. The information from the MIS for Chile Solidario was used intensively to monitor performance of municipalities and family coaches, and most importantly the progress of the participating families in the social protection system. The family level specific data on the latest status on the achievement of the minimum conditions guided the work on arrangements for preferential access and identified key bottlenecks. Data was used to make decisions on the level of municipal autonomy in implementing the program as well as to evaluate family coaches. The results of the impact evaluations were extensively discussed in Chile, including the work authored by Bank staff and research by others using the same data sets. The information on the indicators for project activities was used in work planning and to make decisions on reallocations of spending. 2.4 Safeguard and Fiduciary Compliance Safeguards. The project triggered the safeguard policy on Indigenous Peoples and an Indigenous Peoples Plan was the basis for consultations during project preparation, and was disclosed. The application of the safeguards policy promoted more attention to the indigenous population than had been the case up to then in Chile Solidario. Some interventions such as support for self-employment among indigenous participants 9 probably would not have happened otherwise. The IPDP for the original project was completed, with some redefinition of particular studies during project implementation. The work benefited from good technical supervision, close coordination with the Sub- secretary of MIDEPLAN responsible for government policy on indigenous groups and with CONADI (National Corporation for Indigenous Development) and inputs from experts in the area. The main activities included the incorporation of questions in the household survey and in the targeting instrument to identify indigenous families and indigenous beneficiaries of Chile Solidario, training for social workers and supervisors on theory and methodologies related to the indigenous population, and a comprehensive study of the ethnic pertinence of Chile Solidario. The principal results of the latter were that indigenous families in Chile Solidario valued the intervention strategy, that although in general the 53 minimum conditions were valid for indigenous families, some modifications were necessary, and that the intervention strategy needed some adaption in localities with a high concentration of indigenous. The Program followed up on these recommendations, including by adjusting some minimum conditions and adapting the material for the family counseling to the values and beliefs of the indigenous communities. The social assessment done during preparation of the Additional Financing found that the results of Chile Solidario for indigenous participants had been encouraging. Slightly over 10 percent of the families contacted for CS were indigenous. The rate of non- participation for the indigenous was about a third less than the average for Chile Solidario. The share of families who exited the program successfully completing all of the minimum conditions exceeded slightly the program average, as did their ability to maintain the conditions. Poverty data showed a sharp drop in the gap between poverty rates in the indigenous and non-indigenous population between 2003 and 2006 (from a gap of 11.3 to one of 5.7 points). The Government also had taken a series of actions to ensure that the indigenous peoples received social and economic benefits that were culturally appropriate from Chile Crece Contigo, including in response to consultations which had taken place. In order to ensure that the design and implementation of Chile Crece Contigo was culturally appropriate, several analyses had been undertaken and specific materials prepared. The experience with implementation of the IPDP under the original project meant that specific activities related to indigenous populations occurred early on in the development of CCC. These included studies on cultural practices on child rearing, manuals for health workers, and the preparation of special education materials to work with families. Given the measures already taken, there were only three additional actions envisaged under the IPP during implementation of the AF. Most progress was made in the preparation of specially designed educational and other materials for families, service providers, and children in indigenous communities (Mapuche and Rapa Nui) to support implementation of Chile Crece Contigo. In all materials to support early childhood development were prepared in 5 indigenous languages. In line with the policy of cultural integration, the Government distributed these materials to all children, not only the indigenous. Cooperation with CONADI continued to be satisfactory with a focus on supporting self-employment initiatives among indigenous participants in Chile Solidario. 10 On fiduciary issues, there were minor weaknesses, particularly in the area of procurement, which caused the ratings to vary between satisfactory and moderately satisfactory. This was mainly due to the lack of in –depth experience on World Bank procedures and in general a lack of pro-activity on the part of the staff responsible for project administration. In addition, at the beginning of the original Project and during the AF, there were several changes in staffing. Project management strengthened notably at the end of AF implementation. 2.5 Post-completion Operation/Next Phase The social protection system in Chile continued to evolve during the implementation of the AF with the creation of the Security and Opportunities Subsystem known as the Ethical Family Income (Ingreso Etico Familiar-EIF). IEF began to be implemented in 2013 and some design aspects are still being fine-tuned. In several ways, the EIF builds on the experience of Chile Solidario and maintains with some adaptations some of its key features such as the focus on the extreme poor, use of the social protection index for targeting and family coaching. Municipal family intervention units will continue to play a major role in implementing the program. Reflecting the challenge of the employment dimension of CS and in line with the practices of OECD countries, the IEF includes more focus on employment and payments to families linked to achievements in the area of education and employment. In fact, the initial design of Chile Solidario had included a fourth component on employment, but this had not been developed by MIDEPLAN. One of the first initiatives of the Pinera administration was to convert MIDEPLAN into the Ministry of Social Development. Under the new institutional structure it continues to have a lead role in the monitoring, design and implementation of social policy. The information system created under the TA project is now well established and consolidated in all participating ministries and agencies. The newly created Ministry of Social Development has incorporated its management under the new institutional structure supported by the mandate established in the Chile Solidario law. One key difference, however, is that the focus of the new Ministry has shifted more to execution of policies instead of coordination, the focus under Chile Solidario and Chile Crece Contigo. No follow-up operation is currently planned, but the Bank continues to work with the Ministry of Social Development in the context of the Communities of Learning on Conditional Cash Transfers and Labor. 3. Assessment of Outcomes 3.1 Relevance of Objectives, Design and Implementation The Project’s objectives, design and implementation were highly relevant and remain consistent with Chile’s development priorities and with current Bank country and sector assistance strategies. The Government continues to have a focus on the goal of reducing extreme poverty and designing programs to reach that end. With revisions (including some supported by the Project), key instruments supported under the loan such as the SIIS, Registry of Social Information (RIS) and the targeting instrument (Ficha de 11 protección social) continue to be important in the implementation of social policy. The project was responsive to evolving needs originating in the changes in the institutional framework for management of the social protection system, extension of the system, and the specific priorities of different government administrations. 3.2 Achievement of Project Development Objectives The objective of the Project to support the efforts of the Government of Chile to implement Chile Solidario and lay the foundation for a national system of social protection was met to a large extent. Performance as measured by most of the indicators for the specific technical assistance activities was satisfactory, even after considering the shortcomings mentioned earlier in the design of some indicators. Moreover, the Government made substantial progress in implementing Chile Solidario and in strengthening the social protection system. A detailed assessment of the systems approach appears at the end of Annex 2 and is quite positive. Several technical assistance activities were critical to that achievement, particularly the installation of the Integrated System for Social Information (SIIS), the training of family coaches and the undertaking of several studies related to improvements on the supply side, to mention the most important ones. In addition, the Project contributed to the sustainability of reforms to poverty reduction and social protection policies through the strengthening of the role and responsibilities of the municipalities. Details by component are in Annex 2. 3.3 Efficiency   Since this is a technical assistance project, the classic efficiency calculations are not applicable. Efficiency gains that were expected included: reduction of targeting errors, better integration and use of information on households which would enable cases of fraud, misallocations, and overlapping of duplicating benefit payments to be identified and eliminated. Targeting performance was excellent and is covered in 3.5(a) below. The implementation of the SIIS and improved coordination are expected to have resulted in efficiency gains. However, these have not been quantified. The results from the impact evaluations of Chile Solidario show positive impacts on both the final and intermediate objectives identified in the PAD and show interesting changes as longer periods of time were covered, thereby highlighting the effects of the coordination of services and improvements in the quality of programs. The first evaluation which covered 2003-2004, can be considered a measure of the short-term impacts of participation in Chile Solidario, as none of the families had completed their time in the program. These results were primarily related with the take-up of particular programs and were strongly influenced by the performance of the family coaches and the minimum conditions, as the results of the coordination with the social programs were still nascent. In the short-term, the main results for participant families were the following: 12  Significant increase in the take up of monetary transfers and social programs for housing and employment3  Improved education and health outcomes o Higher preschool enrollment (4-6 percentage points), school enrollment for 6-15 year olds (7-9 percent), adult literacy (4 percent higher take-up of adult literacy programs in urban areas and 5 percentage points higher in rural areas) o Households in urban areas were more likely to have taken up complementary programs providing school materials, meals, and dental care o Increased enrollment in public health system (2-3 percent in urban areas and 3 percent in rural areas), increased preventive health visits for children under 6 by 4-6 percentage points in rural areas and for women by 6-7 percent)  Suggestive evidence of the key role that the psycho-social support had in enabling these changes by increasing awareness of social services in the community as well as households ‘orientation towards the future  Evidence that on average Chile Solidario participants have increased their awareness of social services in the community and are more likely to be more optimistic about their future socio-economic situation  No significant effects on household income per capita The results of the evaluation which covered the period 2003-2006 show the medium-term results of the program. Importantly, these results indicate positive effects on employment and income for some groups of families. The biggest impact on employment and income was registered among spouses of heads of household, particularly in rural areas. This evaluation continued to show evidence of increased take-up of monetary transfers. Of particular interest were the results for the family allowance, which in spite of a substantial increase in its coverage during the time covered by the evaluation, continued to show not only significant positive take-up by Chile Solidario participants, but also by families living in those same neighborhoods. Participation in Chile Solidario also raised enrollment in housing and employment programs, among other services. Overall, people who participated in Chile Solidario showed gains in living standards, including more stable housing and, for some groups of participants, more stable work and higher income. Results from the longer-term evaluation4 which is able to incorporate analysis on the influence of the supply-side changes (better coordination and quality) which is forthcoming show important variations in impact from participation in Chile Solidario: 3 This finding relates to the indicator on coverage of social programs and monetary transfers among the neediest families. 4 Forthcoming from Carneiro, Galasso, and Ginja 13  Related to the large increase in the take-up of monetary transfers across the board, the impact is largely driven by those who did have the transfers before entering the program, in other words, those who were socially excluded from the social protection system.   The lack of overall impact on employment and income in the long-term masks very important heterogeneity depending on initial conditions (conditions met before the program) and across socio-demographic groups (age of head), but most importantly across cohorts.   Results show that the long-term impact on employment and housing is large for the cohorts who were exposed to the response of the supply side.     The latter result has potentially interesting policy implications on the design of social protection systems in terms of sequencing and integration of demand and supply side interventions. 3.4 Justification of Overall Outcome Rating Rating: Satisfactory The rating of satisfactory takes into account the high relevance of the objectives, design and implementation of Chile Solidario as well as its role in laying the foundation for a national system of social protection, the successful carrying out of most activities of the technical assistance project and their demonstrated link with the outcomes on the project development objectives. Moreover, as explained in the section above, findings from the impact evaluation show positive results on CS participants. 3.5 Overarching Themes, Other Outcomes and Impacts (a) Poverty Impacts, Gender Aspects, and Social Development Data collected for the impact evaluations indicates that the targeting performance of Chile Solidario was good. Targeting performance was particularly good for the extremely poor. By 2006, the coverage of the program had reached 6 percent of the overall population, but was 20 percent in each of the bottom three ventiles of income distribution:5 Targeting of the extreme poor is likely to have improved after then with the substitution of a national eligibility cut-off score instead of municipal specific ones. Participation in the program had significant results in poverty reduction in rural areas, which increased over time. Taking into account poverty dynamics – the movement in and out of poverty – the impact evaluation suggests that at least in rural areas, Chile Solidario addressed structural poverty, while also reducing the risks of falling into poverty for female-headed households. 5 “El Impacto de Proveer Apoyo Psico-Social a Familias en Extrema Pobreza y Aumentar su Accesso a Servicios Sociales: Evaluando Chile Solidario� by Pedro Carneiro, Emanuela Galasso, and Rita Ginja 14 (b) Institutional Change/Strengthening The Chile Solidario initiative, announced in May 2002 by the Lagos administration, was a package of legislative and administrative reforms to Chile’s poverty reduction and social protection policies focused on bringing Chile’s 225 thousand neediest households out of indigent poverty. Essentially, one of the aims of Chile Solidario was to implement a new model of management of public institutions. In particular, there was a need for better coordination of efforts in order to reflect the multi-dimensionality of poverty and the integration of aspects of both assistance and promotion. Integration was reflected at different levels. At the central level, the Ministry of Finance (Budget Department), MIDEPLAN and FOSIS were involved. In fact, one of the motivations of giving MIDEPLAN a central role in the coordination of Chile Solidario and eventually the social protection system, was to strengthen the Ministry Implementation of CS had an impact on many other public institutions working in the area of social protection for the extreme poor. For those who traditionally worked with this group, it facilitated improvements in targeting and coverage of the extreme poor through the coordination because the CS families were given priority. Early on Chile Solidairo had a significant impact on the Ministries of Health and Education, the agency in charge of child care centers (Junta Nacional de Jardines Infantiles), the agency in charge of remedial support and scholarships in schools (Junta Nacional de Auxilio Escolar y Becas), the Ministry of Housing, and the municipalities. The impact on FOSIS was also great because previously it had not been working primarily with the extreme por. For other programs such as the ones related to employment and more generally activities of the Ministry of Labor, it facilitated the inclusion of these families who had not been attended before. More specifically, the design and implementation of the system of social protection Chile Solidario promoted institutional changes and strengthening in a variety of areas:  Multi-dimensional nature of poverty: CS embodied the idea of a “ventanilla única� or one-stop shop� which extended a bridge (Puente) between the family and their rights. Its explicit ambition was to move away from sectoral to more integrated policies. The one stop shop was Chile Solidario and the instrument was the family coach.  Family Coach: This was an innovative design element and they played an important role as coordinators, integrators, while at the same time reflecting the “promotional� aspects of CS. The coach worked on 2 levels – with the family and also with the network of social programs.  Move away from demand-driven approach. The diagnostic of CS was that families often did not have the minimum instruments necessary (information, resources to get around, etc.) to demand the services they needed and for which they were eligible. This was the motivation for the idea of preferential access. It reflects the premise that the logic of social protection requires that the system develops its capacity to respond in an adequate fashion. In practical terms, this translated into a focus on ensuring the correct functioning of the supply side. The role of the transfer was to provide resources for the families to interact with the network and thus was 15 much smaller than the monetary transfers in the classic CCT programs such as Oportunidades in Mexico or Familias en Acción in Colombia.  Focus on family instead of individual. Part of the dispersion of social programs was explained by a focus on individuals with the disadvantage of ignoring a key asset.  Incorporation of rights focus into the design of social policies. The minimum conditions reflected the establishment of minimum standards that the State should guarantee to all of their citizens. They explicitly recognized fiscal constraints and thus reflected an ethical approach balanced by financial considerations. CS recognized that the idea of rights had to be internalized within the services and government levels in both their day-to-day decisions and overall policy. In order to be a true right, both the precise definition and accountability for their provision to individuals was required.  Integrated Social Information System: CS innovated in the design and implementation of information systems in several dimensions. The SIIS is one of the pillars of the social protection strategy. It provides information on the social transfers and services provided by the State. By facilitating coordination increases the efficiency in the delivery of those services among the agencies providing the services and the institutions managing the social protection system even though they are in different sectors and different levels of government.  Organization and strengthening of a municipal network of local intervention. CS is characterized by the gradual strengthening of the role of the municipality and provision of more autonomy and responsibility for the implementation of the social protection system. This included the transfer of a higher share of monetary resources and of the portion of centrally-hired family coaches to municipalities, as well as investments in training and equipment to enable them to monitor and manage programs at the local level. (c) Other Unintended Outcomes and Impacts (positive or negative) Not applicable. 3.6 Summary of Findings of Beneficiary Survey and/or Stakeholder Workshops Not applicable. 4. Assessment of Risk to Development Outcome Rating: Low to Negligible The rating of low to negligible is based on the fact that the legal and regulatory framework for the intersectoral social protection system in Chile has been maintained through 3 administrations with extensions of the system implemented through additional subsystems (CCC and IEF) meeting the criteria set out in the original law. For details see discussion of legal framework in Annex 2. The objective of the government remains the reduction of extreme poverty. Moreover, the central instruments within the architecture 16 of the social protection system – for example, the Social Protection Index (FPS) and the Integrated Social Information System (SIIS) continue to be maintained and used. The law creating the Ministry of Social Development confirmed the right of the Ministry to request information from other agencies in order to carry out its functions and the agencies are obligated to provide that information in a timely fashion (see Box on Legal Framework in Annex 2). The institutions (MIDEPLAN/MDS, FOSIS, and the municipalities) benefiting from the Project remain closely involved in current social protection initiatives. 5. Assessment of Bank and Borrower Performance 5.1 Bank Performance (a) Bank Performance in Ensuring Quality at Entry Rating: Satisfactory Bank performance in preparing the original operation was highly satisfactory for the following reasons. The technical assistance project supported an initiative to strengthen the social protection system for the extreme poor which was an important policy initiative of the Chilean government and in fact complemented the Social Protection Sector Adjustment Loan. Technical aspects were strong since the assistance provided by the Bank team during preparation and appraisal built on the comprehensive sector work that had been carried out. The availability of the Household Risk Management and Social Protection Report provided a sound basis to work with the team in MIDEPLAN to design the project including key elements to strengthen the social protection system – improving access to social protection and wider social services, measures to improve targeting and the use of information, and strengthening the monitoring and impact evaluation of social programs. Project design was also informed by analysis on poverty trends for which the aforementioned report updated an earlier study done by the Bank. During project preparation the Bank team consulted with a diverse set of municipalities, as well as civil society organizations and think tanks, the findings of which were fed into the social assessment. The original project triggered the indigenous peoples safeguard policy and the Government prepared and disclosed an Indigenous Peoples Development Plan. Reflecting the high capacity of the country, the procurement and financial administration functions of the project were integrated within MIDEPLAN. The Bank also provided strong technical support for the evaluation arrangements for the Chile Solidario program. A minor weakness was the definition of the indicators chosen for the results framework, which could have been better chosen to measure the advances in the social protection system and had other shortcomings. The Quality Assurance Group (QAG) rated the quality at entry of this project as Highly Satisfactory. QAG rated 4 dimensions (strategic relevance and approach; technical, financial and economic aspects; policy and institutional aspects; and Bank inputs and processes as highly satisfactory and none less than satisfactory. 17 Bank performance in preparing the Additional Financing was satisfactory for the following reasons. Additional resources were needed to implement additional activities to scale up the project’s impact and development effectiveness. The additional activities were related to Chile Solidario as well as to the extension of the system of social protection to cover young children through the Chile Crece Contigo initiative. They were a natural continuation of what had been supported under the original project and the additional financing needs were relatively modest. Additional financing was more cost- effective for the borrower compared to a repeater or other new project. The Additional Financing triggered OP 4.10 on indigenous peoples safeguards. A social assessment was prepared and a new Indigenous Peoples Plan was prepared and disclosed. Based on the satisfactory performance under the original project, project implementation, financial management and procurement arrangements remained the same. There was one shortcoming in quality at entry which was the delay in finalizing the Additional Financing. A year lapsed between approval of the Concept Memorandum and negotiations. This delay adversely affected subsequent implementation of the AF and made necessary a series of extensions to the closing date of the original project in order to avoid a gap between the two loans. (b) Quality of Supervision Rating: Satisfactory Bank performance is rated satisfactory for the following reasons. The Bank team focused on development impact by following closely developments in Chile Solidario, its extension to other vulnerable groups, and incorporation of the system for young children, Chile Crece Contigo, as well as the impact evaluation. Project supervision was complemented by a wide range of activities in the context of the CCT Community of Practice facilitated by the Bank of which Chile is a founding member. Fiduciary staff visited regularly and there was adequate follow-up of the safeguards policy related to indigenous peoples, including the preparation of a status report during preparation of the AF. At the request of MIDEPLAN, the Bank included specific expertise in supervision missions, both Bank staff as well as consultants (topics of homelessness and employment). Implementation status reports were prepared twice a year. Reporting of performance on the indicators in the results framework could have been more complete and consistent. The Bank’s technical support was stronger in its areas of comparative advantage or where it was able to identify expertise (overall social protection strategy, homeless schemes, labor market policies, design of targeting instruments and international experience in activation policies) but fell short in the area of information technology. (c) Justification of Rating for Overall Bank Performance Rating: Satisfactory Taking into account the ratings of satisfactory on both ensuring quality at entry and supervision, overall Bank performance is rated satisfactory. 18 5.2 Borrower Performance (a) Government Performance Rating: Satisfactory Government performance is rated as satisfactory for the following reasons. The design and implementation arrangements for both Chile Solidario and Chile Crece Contigo were based on high-quality and timely diagnostic work. In the case of the former it included the results of a 2000 household panel which highlighted the substantial household dynamics behind the overall poverty numbers and the vulnerability of poor households. Studies also showed that existing social benefits and programs were accessed mainly by the poor, not the extreme poor, and that progress in reducing extreme poverty had stalled. Social protection spending was substantial ($800 million per year), but dispersed in 90 programs among 30 institutions with a sector and individual focus. Monetary transfers included few promotional or pro-active measures to add value and empower participants. The political commitment at the highest level to the reduction of extreme poverty and to Chile Solidario (President Lagos) and Chile Crece Contigo (President Bachelet) as well as to the supporting reforms and strengthening of the social protection system is demonstrated by the promulgation of several important laws6 and equally important to the creation of an enabling framework. Especially critical for the latter, was the financing mechanism developed for Chile Solidario in which the Budget Division of the Ministry of Finance was the key player. Resources were allocated to a budget line for Chile Solidario within the overall MIDEPLAN budget, separately for the programs and services needed to respond to the set of minimum conditions, and tied to achievement of specific coverage targets and services for Chile Solidario participants, as reflected in agreements with the Executive Secretariat of Chile Solidario. Another interesting budgetary feature was that in order to accommodate unforeseen economic developments which might increase the share of extreme poor, CS was guaranteed a certain amount of funding and not penalized if it did not need to use all of the resources allocated. There were no issues with counterpart funding considering the global financing made available for CS. (b) Implementing Agency or Agencies Performance Rating: Satisfactory Performance of MIDEPLAN (later renamed the Ministry of Social Development), the implementing agency is rated as satisfactory for the following reasons. The commitment of MIDEPLAN to achieving the development objectives of supporting Chile Solidario was consolidated during implementation of the original project, reflecting the strengthening of the functions of the Executive Secretariat and finally the conversion into 6 Chile Solidario - Law 19.949 (2004), Sistema Intersectorial de Protección Social – Law 20.379 (2009). See Annex 2 for detailed analysis. 19 the Executive Secretariat for Social Protection, which became the main counterpart for the Bank project. Initially there were some problems of coordination between MIDEPLAN and FOSIS, the agency in charge of the implementation of Puente (the name for the family coaching or psycho-social support component of Chile Solidario). Implementation arrangements and staffing were satisfactory, although the visibility and profile of the project within MIDEPLAN varied depending on the priorities of each Minister. There were no major implementation issues, with most processes mainstreamed within the administrative structure of MIDEPLAN, although at times the need to comply with both Bank and Government procedures increased administrative requirements. The use of selected country systems (Chilecompra) was extended during project implementation. Financial management performance tended to be better than procurement performance, the latter generally rated only moderately satisfactory. The monitoring and evaluation arrangements for Chile Solidario were best practice and the information was used extensively in decision-making and resource allocation, with key support being provided by the development of the SIIS (Component 1) as well as the well-functioning budget management system of the Ministry of Finance. The new administration which took over in 2010 had little experience with either government or Bank administrative requirements, which slowed down implementation. That factor, the transformation of MIDEPLAN into the Ministry of Social Development, the decision to cancel several evaluation studies in 2012, and lack of sufficient budget space in 2012-2013 (given shortfalls in execution in 2011), resulted in the cancellation of nearly half of the amount of the AF. Nevertheless, the resources of the AF were used for several high priority tasks, mainly the reform of the targeting instrument with a special focus on measures to combat fraud and error and the baseline and related studies for an evaluation of the parental education component of Chile Crece Contigo. In addition, some of the activities planned for 2012 under the project were carried out using national resources. (c) Justification of Rating for Overall Borrower Performance Rating: Satisfactory Overall Borrower Performance is considered satisfactory in view of the Government and the Implementing Agencies’ performance. 6. Lessons Learned  The experience of Chile with Chile Solidario illustrates that a single initiative or program can serve as a practical and concrete base for the installation of broader social protection system which can be extended to cover other populations.  Bank financing can play a role in financing activities which once validated can be incorporated into the national budget. Examples are initial investments in the SIIS, impact evaluation studies, and training for family coaches. Loan resources were often a more flexible source of financing than were normal budget resources, 20 particularly convenient for new or pilot activities. However, it would have been helpful to be able to depend on country systems to a greater extent.  When considering different policy interventions to reduce poverty and help households manage risks to their income, policy makers often take for granted the critical institutional and information infrastructure necessary to operate social protection programs effectively. A comprehensive, integrated information system is critical in order to ensure that social programs reach those who need them the most, and to guard the system from falling prey to abuse. Without such a system social policies and interventions cannot be effectively targeted, monitored or evaluated. At the same time, the development of an integrated social information platform for a social protection system takes time and is likely to need to be flexible and adaptable to reflect the expansion and modifications of Government strategy over time.  The incorporation of the targeting instrument into the integrated social information platform was a powerful tool in the creation of a national registry of beneficiaries which in turn became a valuable tool for managing a network of social protection services. An additional positive feature was its design on a decentralized basis with online updating possible.  High-quality administrative data bases can be an important source of information not only for the monitoring of social programs, but also their impact evaluation, particularly if they can be linked to survey data.  There are significant challenges in measuring the impact of the installation of a social protection system which incorporates not only interventions with program participants, but also gradual improvements and better coordination with the supply of social services and transfers. The methodologies required are complex and need to incorporate administrative data from a variety of sources as well as the traditional household surveys. And as the methodology becomes more complex, the communication of these results also becomes more difficult.  Efforts to reduce the dispersion of effort and fragmented coverage of social programs of the poor are perhaps best envisaged as a process and need to be sustained over time. The experience of Chile suggests that effective coordination requires mutually-enforcing actions – for example, formal agreements and working groups among institutions and different levels of government, results- based indicators coupled with close supervision and remedial actions when necessary, supportive information and budget mechanisms (see 5.2(a) p. 19) and roles such as family coaches. The combinations of mechanisms and their relative importance are likely to vary depending on the country. 21 7. Comments on Issues Raised by Borrower/Implementing Agencies/Partners (a) Borrower/implementing agencies The implementing agency had no comments on the ICR. (b) Cofinanciers Not applicable. (c) Other partners and stakeholders Not applicable. 22 Annex 1. Project Costs and Financing (a) Project Cost by Component (in USD Million equivalent) Actual/Latest Appraisal Estimate Percentage of Components Estimate (USD (USD millions) Appraisal millions) National Information System for 3.7 2.8 96 Social Protection Training 1.7 1.6 94 Analysis, Evaluation and Monitoring of Social Programs 2.5 2.3 92 and Policies Institutional Strengthening of MIDEPLAN and Project 5.7 5.2 91 Administration Total Baseline Cost 13.6 11.9 88 Front-end Fee IBRD .1 .1 Total Project Costs 13.7 12.0 88 (b) Financing Appraisal Actual/Latest Type of Estimate Estimate Percentage of Source of Funds Cofinancing (USD (USD Appraisal millions) millions) Borrower 4.3 1.9* 44 International Bank for Reconstruction 13.7 12.0 88 and Development *Does not include US$25.4 million in global investment for Chile Solidario 23 Annex 2. Outputs by Component Component 1: National Information System for Social Protection In the context of the efforts through CS of strengthening and improving the organization and management of social policy, the timely availability and use of information was key. It was especially important in order to support the integration of social interventions and sectors. This summary of the outputs of the component relating to the national information system for social protection includes the 3 indicators in the results framework, namely the number of institutions agreeing to exchange information, the number of variables incorporated, and the number of institutions agreeing to the proposed standards. In addition, the section covers broader aspects related to the development of the system including technological, institutional (including the participation of different levels of government), and legal aspects, all of which supported the progress on these indicators. The information tools which were developed to support the social protection system – the Integrated Social Information System (Sistema Integrado de Informacion Social-SIIS) and the Registry of Social Information (Registro de Informacion Social – RIS) surpassed what had been expected at the time of project appraisal. This happened mainly because the development of the information tools reflected, supported, and responded to the concrete changes and operational needs of the also evolving social protection system over the decade (2003-2013). Also discussed in this section are the outputs associated with reforms to the targeting instrument also supported by the Project. The SIIS was implemented gradually and slowly and constantly improved because system development was grounded on the social programs that MIDEPLAN administered through the Executive Secretariat. It includes the Social Protection Index, Chile Solidario, and Chile Crece Contigo. Initially (before the approval of the technical assistance project), the module for Puente was installed in the municipalities. As will become clear in the description that follows, the base for the SIIS was the management information system (MIS) for Puente that early on interchanged information with the Civil Registry and the Institute of Social Security (INP). The first transfers associated with CS (bono de protección) were paid in November 2002 on line through the INP. Under the Project, three phases can be distinguished. During the first phase through December 2004, the SIIS focused on capturing the information needed for the CS target population. Even at this early stage, the development of the SIIS was influenced by a change in focus from centralized administration of databases and primarily descriptive analysis to the aim of giving local managers the information and data on indicators necessary to take the decisions necessary to meet the objectives of the local social protection system. The requirements for CS7 were utilized as the model to design and construct the SIIS. 7 For example, as the first step, requirements included the monitoring of information at the household level of the status on the roughly 50 minimum conditions, and their program exit and entry, data on the family coaches and data consolidations at different levels (by family coach, municipality, region and nation) 24 In the second phase (2006), with the system operating in the 334 municipalities implementing Chile Solidario, the SIIS generated a new module of information for the on-line processing of the revised targeting instrument, the Social Protection Index (FPS).8 This step was crucial to consolidate the SIIS as the technological platform for Social Protection.    This is because at the outset, agencies were reluctant to sign agreements to participate in the information system by exchanging information contained in their data bases. They saw no particular advantage for them when they were invited to participate in the scheme. However, with the inclusion of the FPS with on line data entry decentralized to the municipalities, a major milestone was reached. Since the FPS was a key targeting instrument for a wide range of national subsidies such as social housing, school lunches, water supply subsidies and several other monetary transfers (child allowance, social pension) it gave the Ministry additional leverage for persuading other ministries as it was now a component of the SIIS. In addition, as the social protection system expanded to cover additional groups of the population – elderly individuals living alone, for example – and as more programs gave preferential access to Chile Solidario (school subsidies, for example) use of the Ficha became more extensive. Given the wide territorial coverage and high number of households with the FPS, this component of the SIIS has been transformed since 2007 to be a privileged source of information and operates as the principal support for the rest of the functions included in the SIIS. The last phase began at the end of 2007, when a new module was generated for the registration and monitoring of Chile Crece Contigo. By the end of the TA project some 46 agreements had been signed with ministries and agencies covering more than 60 million records in over 60 data bases incorporated into the SIIS. There were 15,672 users of the system, including all municipalities in Chile. By the end of 2009, nearly 80 percent of the population of Chile was registered in the SIIS. Later the Additional Financing (AF) supported enhancements to the FPS instrument and system software was upgraded by introducing a new data warehousing system and strengthening reporting systems. The main technical and operating characteristics of the SIIS are:  Interoperability which is crucial in order to support decision-making within an intersectoral and interinstitutional social protection system;  Phased Development on a modular basis and scaleable to incorporate new populations and initiatives;  Flexible language utilized by the platform in order to adapt to the characteristics of other information tools, so that it could be easily used by other agencies;  Validation of personal information through a unique national identification number (RUN) with all modules connected online with the Civil Registry; 8 The module related to the FPS was installed in all 345 municipalities in Chile. 25  Guaranteed adherence to secrecy rules through a legal and administrative regulatory framework that needs to be complied with by system users;  Provision of training to prepare people to provide, access, and use the data;  Secure system with controls over passwords and access;  100% web based with user friendly interfaces, easy to learn for users; and developed through open source programs, the latter with the aim of reducing costs. The main outputs financed by the Project included a study on the conceptual, logical and physical design, including the evaluation of the existing databases, implementation of the platform for the Social Protection Index, the implementation of the SIIS, equipment purchases (including for municipalities), training for staff at the central, regional and local levels and implementation of the initial stages of the most recent modifications of the targeting instrument (Ficha Social). The latter included information campaigns and training in addition to the initial registration of households using the new instrument. The chart below provides more information on the development of the SIIS year by year. 26 Area 2004 2005 2006 2007 2008 2009 Stage Nascent Design, Consolidate Inclusion of Social Development of Consolidate RIS Development of Technological Protection Index and Registry of Social Platform and Platform Improvements in Information (RIS) Standards Data Exchange Information Exchange of Inputs for the Installed SIIS Implement Social Expand ways of Consolidate exchanges information construction of in 336 Protection Index on consulting online mechanism of with 3 the SIIS municipalities line in 345 with database to public access institutions – platform were to monitor and municipalities see which social Civil Registry, the CS supervise CS transfers people Agreements with INP and FOSIS agreements with families Achieved were getting. 337 municipalities, (Puente) other interoperability Users were 17 public agencies institutions for 2 additional between levels of municipal and 9 NGOS. Article 6 of preferential institutions government. employees, staff of Law 19.949 access, exchanged programs and Creation of establishing the information; To improve, private institutions. Social periodicity and 13 in total with regularize, and Add module for Protection the form of data 20 databases. protect exchange of Unify systems of municipal benefits System CS exchange. 11 information, prepared information as institutions Improved Regulations on programs Apply standards in exchanged data. technology to Connectivity and expanded and the the Law of interchange Exchange of range of services Transparency information. Information among covered by the Public Institutions social protection Implement Defined system expanded. statistical reports periodicity of Monetary subsidies linking Program for data bases and On-line Warehousing of Homeless (Calle) at data format RIS data national level. structure. Consolidate module Shared platform, no for Program for cost for local actors 27 Area 2004 2005 2006 2007 2008 2009 Homeless (Calle) and develop modules for Portal for social Elderly living alone transfers and (Vinculos) and CCC. services applications and queries No. of Analysis of Data on 53 Variables for CS, and Information variables Information for minimum also for Vinculos, grouped in the CS – requires conditions and Calle and Social following manner: the registration consolidation/anal Protection Index FPS, Programs of ysis at different of information the Intersectoral levels (national, by household regional, Information for CCC System of Social and tools that provincial, includes System of Protection, Non- permit the municipal) plus Registration, monetary social monitoring of basic household Monitoring and services, services, the changes in characteristics Referrals. Also set of subsidies or the minimum measurement assistance without conditions instruments at a monetary different transfer, and development stages monetary transfers and system of alerts. or other assistance involving cash Development of statistical model for Social Protection Index 28 Registry of Social Information (Registro de Informacion Social). The RIS began to be developed in 2008. Within the SIIS, the RIS is the supporting instrument which provides adequate and reliable data on the individuals in the social protection network. It is a database which functions as a unique beneficiary roster. MIDEPLAN was mandated to develop this tool in the 2004 law setting up Chile Solidario. A key step in the progressive achievement of this mandate was the preparation of the regulations necessary to facilitate the best use of the available information. These were finalized in 2008 and permitted the RIS to sign agreements with institutions and regulate the processes of information exchange. The development of these agreements and their extension over time opened up the possibility of a progressive advance in the consolidation of a center of data collection on beneficiaries and potential beneficiaries within the social protection system. The cross checking of databases also helped improve the quality of the registries. Requirements for the development of the RIS included the further analysis of the needs of the inter-operability model, going deeper into the normative bases that would make possible the regulated interchange of information and the technological conditions that would permit the integration of the different information systems. The warehousing by the end of 2008 of the RIS data signified that organized and reliable data for the families or individuals who are or potentially are beneficiaries of social transfers or programs provided by the state was available. A development promoting use of the RIS was the passage of the Law of Transparency which obligated ministries to disclose information on beneficiaries of public programs. It is worth highlighting that the development of the SIIS and the RIS required substantial training to ensure understanding and competency to meet the required standards and norms and also to carryout statistical analysis. Numerous meetings were held with participating institutions as well. Addressing concerns on security and guaranteeing privacy were critical. MIDEPLAN defined the formats for collecting the data; the entries, exits and forms of connection to the SIIS, how queries would be done, the way things were stored and the security standards for the transfer and queries for social information. There was one-on-one work with each entity in order to ensure the timely meeting of these standards. Under the AF the Registry was expanded to cover the Ethical Family Income. . Component 2: Training There were two subcomponents: (1) Training of Government Officials, and (2) Training of Family Coaches. The strategy and outputs of both are discussed in turn. Training of Government Officials The subcomponent generated several outputs. Aside from an assessment of training needs, the principal activities financed by the subcomponent were courses tailored to respond to specific operational needs that arose as the social protection system developed and expanded and, towards the end of the original project, in the design and delivery of a set of courses intended to support the consolidation of the system. Technological tools such as on-line courses were used extensively. The training assessment undertaken 29 during 2005-2006 focused on the demands for training from the regional staff of MIDEPLAN and their teams coordinating Chile Solidario. The main needs identified included: the recent evolution of poverty alleviation policies; gaps and challenges on poverty reduction in 2005; in light of those gaps and challenges, the rationale, vision and performance of Chile Solidario in the region; the politico-communicational positioning of Chile Solidario in the region; fundamentals and methods for work within a network; fundamentals and methods for teamwork; fundamentals and methods for community work in extremely poor localities; and the management and use of information on Chile Solidario. The training model was to be based on competencies. The key operational needs which arose during implementation of the Project included training related to the installation of the new targeting instrument (Ficha de proteccion social – FPS). Over 2000 municipal employees (interviewers) received training on the FPS in 2006. Distance learning courses were developed in several areas including: management of the Programa Calle; integrated local service delivery for monetary transfers, information on the definition and organization of the Social Protection System Chile Solidario, and the legal and integrated management of the programs related to the dimensions of income and employment. During 2006, the training program to develop competencies to manage in a network was executed, including an evaluation process. During 2007-2008, the number of courses offered increased to include in the case of the homeless, courses on how to address handicaps, age and occupational issues. A special online course was developed on topics related to employment and this was updated to reflect the work with employment services. Audiovisual materials were designed on various themes including: Regional Management of CS; work in networks of CS; Chile Solidario from the perspective of work with women, the institutional impact of Chile Solidaro in various programs, and programs with the aged (Vinculos), homeless (Calle), and early childhood (Chile Crece Contigo). In 2008, courses continued to be expanded, mainly for virtual learning, especially to support the program for homeless on several technical issues (mental health, for example). In 2009 the training strategy was extended to support the consolidation of the social protection system. The strategy was organized into two lines of work: (1) a set of 16 competency-based courses delivered virtually through a single technological platform organized in 4 areas of competencies: local management of the social protection system; the management of programmatic resources for the vulnerable population; management of tools to support the Social Protection System; and competencies to provide counseling for CS families, and (2) specialized technical assistance made available to municipalities to provide specialized assistance with the aim of improving their performance in areas relevant to social protection. In that connection, two new courses were developed focusing on management at the local level, one generally on social protection and the other on the management of local networks in social protection. In the case of the first line of work, MIDEPLAN invested in improvements in the technological platform with the objective of putting at the disposal for everyone who had role in the social protection system, a set of resources that constituted a system of lifelong learning with a view to improving management processes and practices. Additional outputs financed under the Project (including the AF) were national meetings related to both CS and CCC. 30 Training of Family Coaches The interaction of the family coach with the participant households is a critical design feature of Chile Solidario. In order for them to contribute to the progress of the households (or other vulnerable groups), they needed to have effective counseling skills as well as be knowledgeable about the network of social services and programs. This subcomponent benefited from substantial prior work analyzing the results of the initial experience with Chile Solidario and an outside assessment of training needs, which led to a comprehensive and flexible training design grounded in a competency and lifelong learning approach, called the Community of Learning Puente. The outside assessment (undertaken after some initial implementation experience) included an analysis of preferences for delivery mechanisms and was informed by the characteristics of the Chile Solidario program, interviews with management, municipal teams, family coaches, and participating families, supplemented by consultations carried out on-line. As described in the PAD, the strategy had several distinguishing features. It stressed teamwork and facilitating fluid communication between the different actors implementing CS. It addressed the needs for initial training, permanent skills upgrading, and feeding back into improvements in the methodology and organization of these activities. This required a combination of different mechanisms for training and the formalization (certification) of their knowledge and experience. The system integrated 4 types of learning activities, supported by a common easily accessible computerized platform:  Circles of Analysis of Practices of structured communication and interchange of experiences among practitioners at different levels and with different responsibilities.  An “Agency� which facilitated exchanges within CS, including visits to other locations, apprenticeships, contacts with outside experts and agencies.  An “Institute� which focused on providing technical assistance to the family coaches through courses, seminars, conferences, and round tables, using both residential and virtual spaces.  A “University� through which the Faculty of Social Sciences of the University of Chile opened a post-graduate degree for a social work specialty for the family coaches with the objective of improving theoretical and practical knowledge. In March 2004 the Community of Learning was launched with a pilot phase until July. The academic and operational definition of each component of the training system and their annual operational plan were prepared and validated in the process with several municipal UIFs. The Community of Learning worked during 2004 and 2005 with the following results:  303 Circles of Analysis functioning with 1.837 participants representing 79% of the family coaches.  1.776 participants in 103 clinics on the minimum conditions  379 participants in 60 apprenticeships  1.056 participants in 44 workshops for municipal Family Intervention Units (UIFs)  185 face-to-face courses with 3.743 participants 31  21 graduate level university courses with 178 participants  Computerized platform functioning The chart below summarizes the main activities during the implementation of the original loan. In the Project Paper for the AF, in addition to the continuation of training for government staff related to the social protection system and in line with the legislated transfer of more responsibility for the implementation of Chile Solidario from MIDEPLAN to municipalities, $250,000 was to be made available to municipalities to carry out capacity building activities to support activities to support strengthening of management in areas related to the social protection system. However, the new Executive Secretariat of CS which entered in 2011 decided on a different strategy of support to municipalities which focused on the design and development of a portal which would operate as a “one-stop shop� for social protection interventions accessed at the municipal level. This was related to the development under the AF of a platform designed for the management of social programs which aimed to improve the effectiveness of basic business processes, which was supported under the AF, together with the associated training. The platform would be used for the Ethical Family Income. 32 2005 2006 2007 2008 2009 Evaluated activities and Carried out Continued to strengthen Redesign of targeting Review, approval and systemized learning. qualitative capacity of family coaches. instrument (Social implementation of re- study on Protection Index) and designed CS. Design of Training and training results. Courses and internships lessons from Competency Certification organized by the University implementation Changes transmitted through for family coaches. Joint Incorporated work by FOSIS Training training criteria of Chile. motivated a redesign of “updating workshops� with Unit and FLACSO to in evaluations CS to take into account the family coaches. define competencies and and ratings of Improvements needed in different family types. gaps. Identified UIFs methodology of Chile An important new tool methodology to guarantee Solidario identified, for With exception of incorporated was the group service quality. Decided example, more training on workshop – a concrete technical and operational differentiation of attention competencies through expression of the community orientations to guarantee depending on family type. technical assistance dimension of counseling – delivery. provided by FOSIS, complementing and linking Curriculum of courses training suspended to the minimum conditions. Updated courses by developed according to pending finalization of Focus on abilities and skills identifying gaps and redesigning delivery preferences of staff as well new design. for social integration during mechanisms and the as the definition of the the counseling process. technical assistance competencies they needed New design implied delivered by FOSIS teams for counseling. Training focused on generation of family to the municipal UIFs. providing family coaches the profiles, revision of 40 semi-presencial versions minimum conditions tools, capacity, and 1835 participants in 304 of courses (counseling and (from 53 to73) competencies to help their CAP self-help, mediation, counseling when they come systematic family analysis, Working group upon family structures and 8,848 hours of training family conflict and (SERNAM, SENAMA, compositions with particular delivered through different violence, crisis intervention, CONADI, FOSIS, y characteristics of components of Community of Learning; 9992 hours team management, work in FONADIS) analyzed vulnerability. including activities outside and with networks, social emerging profiles. of Community of Learning interventions supporting 33 2005 2006 2007 2008 2009 young children and Given decision to Developed 90% of the Training and supporting community focus). include group sessions, teaching materials to materials developed on use experts needed to communicate the new of CAPS. 1500 users of the system develop content on contents developed for the had to take at least one special themes. program and the updating 2 Studies carried out: How CAP made family coaches course. course. more effective and what conditions facilitated an 100% of UIFs participated in effective family coach the process of updating, attending the training as well 31 Clinics delivered on as taking the module minimum conditions available in the Platform “Learning Together� 17 group internships with 245 participants 20 individual internships with 40 participants 34 Component 3: System for Analysis, Evaluation and Monitoring of Social Programs and Policies Design of an integrated system of monitoring of management, processes and the partial results of Chile Solidario System Evaluations of programs that, forming part of the social protection network, provide benefits and transfers to the users of Chile Solidario. Specific studies include a process evaluation of CS, analysis of ethnic pertinence of CS, and advice in the area of disability and gender. Other outputs under this activity supported the feature of Chile Solidario which provided preferential access of beneficiaries to existing monetary transfers and social services, particularly those relevant to the achievement of the minimum conditions. Since the studies were focused on the objective of preferential access, the findings became inputs into the design and management of the agreements between the Executive Secretariat of Chile Solidario and other agencies. These studies of the “supply side� were important not only because they provided information on the results of the programs for the participants, but also because it enabled the Executive Secretariat to have a more precise understanding of how services functioned in practice, what were the specific processes and procedures within the program, and how the actual operation compared with the formal design. The discussion below describes first the studies which were carried out and then the related changes in the way the agreements with other agencies were managed. The first set of program evaluations was carried out in 2005. These included the following programs: Pre-school Program, the School Subsidy for Retaining Students, and the Housing Program Chile Solidario, Wage Subsidy, and the training program for domestic workers. In addition to the in-depth evaluation of each program, the studies included common themes related to preferential access including flexibility/adaptability, social targeting, sustainability, sequencing, and management within a network. In 2006, a set of evaluations of 6 programs related to the dimensions of work and income were carried out. Prior to the beginning of the technical assistance project, MIDEPLAN had started preparing and signing agreements with various agencies to ensure preferential access to the services related to the minimum conditions. During 2004-2006, the main activities were: design of a standard instrument for the periodic reporting by MIDEPLAN regional offices of the implementation of these agreements, including factors favoring or impeding each program and tracking of progress on coverage, pertinence and effectiveness; a national review of these programs; and the incorporation of the conclusions from the monitoring reports into the agreements. At the end of 2006, as part of the functional re- design of the Executive Secretariat, the function of monitoring the agreements was formalized into the Management Unit for Preferential Access whose principal role was the management and monitoring of the agreements which transferred resources for the 35 implementation of both Chile Crece Contigo and Chile Solidario. The specific functions of the Unit were:  Carry out periodic working sessions with the institutions involved (whether collaboration or transfer of resources);  Coordination of the national working groups for each one of the dimensions in the Social Protection System;  Review, supervise, and approve the monitoring reports submitted as part of the requirements of the agreements which include transfers; and  Identify new supply to be included in the Opportunities Map of Chile Solidario and Chile Crece Contigo. As part of the agreements the programs themselves had to send periodic reports and all of the agreements were in the system of monitoring of programs and social transfers of agencies with agreements with Chile Solidario. During 2007-2008, the new system was implemented. Programs with an agreement for preferential access had a methodology and system of monitoring and evaluation designed for their actions. Part of the responsibilities of the partner agency set out in the agreement was sending reports to the Executive Secretariat, explaining what had been done, who were the beneficiaries covered by the program, and spending levels. Those responsible for reporting were identified. These reports were reviewed by the Management Unit for Preferential Access. Each agreement linked with the organization of committees or technical groups at the national as well as the regional level constituted by representatives of MIDEPLAN and the institution in order to define coverage, agree on actions, report on results, identify bottlenecks, etc. These groups were critical in order to monitor execution, analyze advances and provide feedback. Also critical was the fact that the work of the Management Unit for Preferential Access related to each of the agreements was used to inform the budgetary process, specifically the definition of resources that would be made available and the negation of budget line items with the Ministry of Finance and in the National Congress. The Unit of Territorial Management within the Executive Secretariat played an important role. They were responsible for accompanying the process in the regions. When problems arose technical coordination groups were activated at either the central or regional level. To sum up, the monitoring and evaluation of the actions under the agreement was the responsibility of the agency itself that got the resources and they informed the Executive Secretariat. It is worth mentioning that most of these programs already had their own monitoring and control systems which they took advantage of. Starting in the second half of 2008, the Executive Secretariat incorporated the assignment and monitoring of investment supporting municipal actions and the fund to support the detection of gaps into the system. In 2009, the emphasis was on improving the instruments used to monitor the programs and social transfers which had an agreement with CS. The management of all of the agreements was institutionalized within MIDEPLAN in a management system (Sistema de Gestión de Convenios, SIGEC) in order to standardize procedures and mechanisms. 36 Another output under this component was a study that evaluated the installation of a pilot on citizen participation for CS. However, in part because there were delays in carrying out the study, the pilot was not judged necessary in view of the fact that CS had in the meantime developed standard mechanisms of consultations with participants and also was involving NGOs directly in the implementation of work with other vulnerable groups such as the homeless. Carry out Impact Evaluation for Chile Solidario Three rounds of a panel survey were carried out under the project for the impact evaluation. In addition, the analysis incorporated the use of the rich set of administrative databases including the MIS of Chile Solidario, the Social Protection Index data, and the information on participation in the programs of preferential access. Several seminars were held during the life of the project to discuss and disseminate the results of the impact evaluations. During the time covered by the AF, although no new surveys were undertaken, the impact evaluation of CS was updated to take into account more administrative data in order to assess the long-term impact of the program (see Section 3.3 in the main document). In addition, the AF financed evaluation studies related to CCC. Component 4: Institutional Strengthening of MIDEPLAN and Project Administration This component was intended to support a program of institutional modernization and strengthening for MIDEPLAN in order for it to carry out its designated role to coordinate the implementation of the social protection system. The main initial output was the creation in 2004 of several operating units in MIDEPLAN to support the operation of Chile Solidario. In 2005, MIDEPLAN requested changes in this arrangement to reflect internal changes in the design of the support for Chile Solidario, which involved a larger role for the cabinet of the Minister of MIDEPLAN. In 2006, with the new administration, the definitive design of the strengthened Executive Secretariat was put in place with the support of the Project. From then on, the Executive Secretariat was the main counterpart for the Project. The following units were created:  Management of Access to the System (Social Protection Index)  Management of Monetary Transfers  Management/Analysis of Rights (Political-Technical Guidance for Management of the Supply of Services)  Integrated Social Information System (SIIS)  Management of Territorial Network  Institutional Strengthening (Studies, Systematizations, Training)  Internal Administration Given the consolidation of the Executive Secretariat in MIDEPLAN, the focus turned to the regions. In 2006 a study “Systemization and Analysis of Regional Management, as 37 applied to Chile Solidario� was carried out. The study identified gaps in the management of coordination arrangements at the regional level in the area of social protection. Using the results of the study as a guide, in 2007 efforts were made to strengthen the areas of social protection in the MIDEPLAN regional offices. This included the distribution of responsibilities within the offices, designing and making available instruments to orient the work of the Regional Technical Committees of Social Protection, and putting in place technical criteria to support the installation of the programs of family coaching within the municipalities, and to enable the regions to supervise the projects and use of resources which were being transferred increasingly to the municipalities. In 2004, only 7 percent of the resources managed by Chile Solidario were transferred to municipalities. By 2009, this share had risen to 40 percent. Another output of this component was a series of studies related to the implementation of Chile Solidario including the needs and aspirations of CS families who had finished family counseling and a study of families who did not join or who dropped out of CS. Other outputs included the design of the public portal for social services and transfers, workshops related to other vulnerable populations (homeless, children of prisoners), printing of communication and educational materials, and studies of programmatic needs in the area of family dynamics. Under the AF, the project financed the design of the psychosocial support and employment counseling for the Ethical Family Income. Chile Solidario as a base for a Social Protection System The Project Development Objective of the Project was to support the efforts of the Government of Chile to implement Chile Solidario and lay the foundations of a national system of social protection. Building on the detailed outputs and achievements of the four components of the Project described in the preceding sections, the purpose of this section is to relate them more directly to Bank benchmarks for a systems approach. These are drawn from the World Bank’s recent Social Protection and Labor Strategy 2012-2022: Resilience, Equity and Opportunity. The core focus of the strategy is to move Social Protection and Labor from isolated interventions to a coherent, connected portfolio of programs. The argument is that in order to effectively mitigate risks and foster opportunity, social protection and labor programs need to be able to work together as a portfolio of complementary initiatives, under a systems approach. Social Protection and Labor systems are “portfolios� of coherent programs that can communicate with each other, share common administrative subsystems, and work together to respond to risks and to deliver resilience, equity, and opportunity to the population. The expected results are stronger social protection and labor systems that can enhance the rights of citizens, particularly the poor. These systems offer efficiency, through better designed tax/financing arrangements, economies of scale, and common platforms such as registries. They impart effectiveness, through harmonized incentives and fewer unintended consequences on behaviors. Beginning with Chile Solidario, the detailed information on budget provides evidence of the steady increase in spending for a connected portfolio of programs. The box gives 38 examples of programs which received transfers under the Chile Solidairo budget line, a subset of those which provided preferential access to Chile Solidario. Budget for Chile Solidario by Year and Dimension (US$ millions) Component 2002 2003 2004 2005 2006 2007 2008 2009 2010 Psycho-social Support 3.2 6.4 7.8 9.0 10.4 11.7 15.2 18.2 21.9 Monetary transfers .4 8.3 14.7 23.8 23.8 31.9 29.8 31.9 36.6 CS Supply-Identity .3 .4 .4 .6 .2 .1 .4 CS Supply-Health 11.3 13.1 8.9 9.3 9.6 9.5 10.3 CS Supply-Education 6.6 13.7 17.8 22.3 30.1 34.5 37.7 CS Supply-Family .5 .7 .4 .5 .5 .5 .5 Dynamics CS Supply – Housing 2.8 15.6 20.6 18.9 13.8 22.0 22.4 CS Supply-Work and 12.3 30.7 37.1 36.2 39.4 36.5 42.3 Income Total 3.6 14.7 56.2 106.9 119.4 131.4 138.7 153.3 172.1 Dimension Programs Involving Resource Transfers providing Preferential Access to CS families Education School Feeding Program School-based Dental Services Middle School Student Scholarships Life Skills Program Child Care Centers (including infant care and extended hours) Program “Know your Child� and “Program of Improvement of early childhood� Family National Program for Support to Family Dynamics Dynamics Health Family Health Plan Programs of Technical Assistance for persons with Disabilities Work and Support for Microenterprises Income Program to Support Youth Labor Market Insertion Hiring Subsidy Program of Employment Support for Chile Solidario Beneficiaries Creation of Microenterprises (Urban) Program for Development of Employment Competencies for W omen Program to Support Own-Consumption Production Housing Housing Program Chile Solidario (Title regularization) Housing Program Chile Solidario (Housing Improvements) Note: This table only includes the programs which received resource transfers under Chile Solidairo. There were other programs that provided preferential access to Chile Solidairo participants which did not receive resource transfers. The last chart benchmarks the current social protection system in Chile against the criteria set out in the World Bank strategy document at the three levels of engagement (administrative, program and policy), summarizing the information presented in this document. Of particular importance are the legal framework for the intersectoral social 39 protection system (described in more detail in the box) and the specific mechanisms and administrative arrangements which now exist to promote coordination including inter- institutional agreements, national budgeting procedures, an integrated social information system (SIIS) and registry of beneficiaries (RIS) through which information is exchanged among nearly 40 public and private agencies. The long standing proxy means instrument is used for targeting a wide range of programs. Both instruments and programs have been adapted over time as conditions have changed and lessons learned. The Chilean experience also suggests that in order to bring about better coherence among social programs and policies, specific mechanisms may be helpful. There are some examples included in the program section of the chart. They include the role of family counseling (and specifically the role of the family coach) in the case of CS, support from the budget process and the role of local actors where much of the coordination for vulnerable groups needs to work in practice. Chile has in recent years evolved towards a system in which all eligible persons have a feasible chance of entry (in contrast to the previous rationing). Most social programs include well-designed monitoring systems. The Budget Division of the Ministry of Finance ensures that evaluation results are taken into account in spending decisions and organizes an annual schedule of program evaluations. Program monitoring indicators and evaluation results are accessible to the public on the website of the Budget Division. Programs must also comply with requirements of the Law of Transparency, in which the SIIS has played an important role for a variety of programs. Finally, the existence of social programs and transfers and the wide coverage of the targeting instrument enabled the Government to quickly respond to the need of the poor during the previous financial and food price crises by providing lump sum transfers to specific target groups. 40 Levels of Systems Model for Social Protection and Chile Engagement Labor Administrative Build basic management, information, and “Nuts and bolts� subsystems: service delivery tools. Focus on developing “nuts and bolts� (1)Use of RUN as unique identifier subsystems that facilitate core business (2)RIS (38 agencies) processes of SPL programs: (3) Social Protection Index online (1)Beneficiary identification systems (4) Government system of monitoring and(2) registries indicators and evaluations. (3)Targeting schemes Results reported to Congress and (4)M&E arrangements considered in budget decisions, (5)Contracting and payment arrangements including closing of programs. for providers (5) Inter-Agency Agreements in AIM: Building basic subsystems to Management System support one or more programs. (6) Budgetary mechanism for resource allocation (7) Integrated Social Information System (SIIS) including RIS and Social Protection Index. Program Focus on design and implementation issues (1)Use of Social Protection Index through a synchronization across programs across social assistance, labor, housing, or integration of similar programs. and pension programs. Only Step needed for a systems approach is instrument to determine level of better synchronization of programs – with vulnerability of potential beneficiaries. beneficiary databases communicating with (2) Structured access to set of each other to be able to identify overlaps programs and benefits, i.e. family and gaps in benefits, and coordinated counseling in CS and the Program of eligibility parameters across different types Development Support in CCC, which of programs. function as “one stop shop� AIM: Improving design of existing (ventanilla única) for participants. programs and harmonizing across Mechanism that facilitates portfolios of programs. harmonization and coherence among programs. (3) Annual revison (at regional level) of how programs performed for CS participants to decide whether to eliminate, expand, or include new ones. Linked to budget discussion. (4) Municipalities coordinate implementation of different programs and referral systems at local level. Policy Policy coherence and general strategy. Evolving Legal Framework as scope of AIM: Ensuring policy coherence across System grew to cover extremely poor programs and levels of government. families, other vulnerable groups and young children Law No. 19.949 F(2004) creates Social Protection System CS Law No. 20.379 (2009) creates Intersectoral Social Protection System and Subsystem for Protection of Infants (CCC) Refers municipal role. Law No. 20.595 (2012) creates Subsystem Security and Opportunities and Family Ethical Income. 41 Systemic Approach: Evolving Legal Framework in Chile The evolution of the legal framework for social protection in Chile shows progress and maintenance of a systems approach. Law 19.949 (2004) created the Chile Solidario system. It includes: guaranteed access for participants to specific existing monetary transfers and refers to preferential access to other programs: provides legal basis for agreements with municipalities and public and private agencies; specifies that national coverage of program will be set in accordance with Budget Law and that targeting will use the Ficha (later the Social Protection Index); mandates creation of the Registry of Social Information (RIS) which would be made available to municipalities and public agencies; and requires MIDEPLAN to evaluate the program. Law 20.379 (2009) created the Intersectoral Social Protection System: a management model comprised of social actions and services executed and coordinated by distinct public agencies for the most vulnerable population and requiring the coordinated action of these agencies to improve their living conditions. Law mandates external evaluation of system and subsystems, according to the instruments of the Budget Division of the Ministry of Finance. It mandates that each subsystem has an MIS and includes provisions on management tools, specifically the Social Protection Index and the RIS. It is comprised of distinct subsystems: Chile Solidario, and CCC. Requirements for other subsystems to be incorporated:  Attend a defined group of families or persons, identified by the Social Protection Index  Have a core service, in other words a particular program that provides entrance to the system (i.e. in the case of Chile Solidario this was Puente)  Provide services or transfers that have been created by law  Carry out actions that require coordinated, intersectoral management by distinct public agencies, and which is executed normally by the municipality  Respond to criteria of relevance in their services  Include procedures of measurement and evaluation, at the least results such as coverage, targeting, and quality (including MIS mentioned earlier)  Include mechanisms of gradual exit  Include a regional analysis of the social impact of the subsystem and its necessary adjustments to local conditions. President determines entry of new subsystems based on proposal of the Interministerial Committee and report prepared by MIDEPLAN. Law 20.530 (2011) establishing the Ministry of Social Development maintained the central role of the RIS by confirming that the Ministry has the right to “solicit from other ministries, services or public agencies, the supply of available information needed to carry out its functions. Agencies are obliged to provide this information in a timely fashion. Law 20.595 (2012) creates subsystem of Security and Opportunities of the Ethical Family Income. 42 Annex 3. Economic and Financial Analysis Not applicable. 43 Annex 4. Bank Lending and Implementation Support/Supervision Processes (a) Task Team members Responsibility/ Names Title Unit Specialty Lending Supervision/ICR Daniel Jorge Arguindegui Procurement Specialist LCSPT Jorge C. Barrientos Consultant LCSHS-DPT Martha Raines Burt Consultant LCSHS-DPT Pedro Carneiro Consultant DECIE Emanuela Galasso Senior Economist DECPI Maria Lucy Giraldo Senior Procurement Specialist LCSPT Ana Maria Grofsmacht Procurement Specialist LCSPT Febe Mackey Consultant LCSPS Mariana Margarita Montiel Senior Counsel LEGLE Demetra Nightingale Consultant LCSHS-DPT Truman G. Packard Lead Economist EASHS Marcela Ines Salvador Social Protection Specialist LCSHS-DPT Morag N. Van Praag Senior Finance Officer CTRDM - His (b) Staff Time and Cost Staff Time and Cost (Bank Budget Only) Stage of Project Cycle USD Thousands (including No. of staff weeks travel and consultant costs) Lending FY04 15 105.86 FY05 0.00 FY06 0.00 FY07 0.00 FY08 0.00 Total: 15 105.86 Supervision/ICR FY04 1 18.18 FY05 19 131.90 FY06 13 123.86 FY07 13 95.80 FY08 19 125.72 FY09 26 120.73 FY10 16 92.76 FY11 0 0 FY12 6 53.58 FY13 10 89.18 Total: 123 851.71 44 Annex 5. Beneficiary Survey Results Not applicable 45 Annex 6. Stakeholder Workshop Report and Results Not applicable 46 Annex 7. Borrower's ICR 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 Annex 8. Comments of Cofinanciers and Other Partners/Stakeholders Not applicable 68 Annex 9. List of Supporting Documents Proyecto de Asistencia Técnica Banco Mundial – Chile Solidario: Informe sobre Estado de Avance según Indicadores de Logro: Informe periodo hasta Diciembre 2009 Apuntes del Sistema Integrado de Información Social: Gestión de la Información Social del Estado Aportes del Sistema de Protección Social Chile Solidario a la Política Publica de Chile Trayectoria institucional, arreglos operacionales y alcances del Sistema Chile Solidario (Informe 2009) Alleviating extreme poverty in Chile: the short term effects of Chile Solidario by Emanuela Galasso Evaluating the Chile Solidario program: results using the Chile Solidario panel and the administrative databases by Fernando Hoces de la Guardia, Andres Hojman, Osvaldo Larranaga El Impacto de Proveer Apoyo Psico-Social a Familias en Extrema Pobreza y Aumentar su Accesso a Servicos Sociales: Evaluando Chile Solidario by Pedro Carneiro, Emanuela Galasso, and Rita Ginja 69 IBRD 33386R 85°W 80°W 75°W 70°W 65°W 60°W PERUTacna To To La Paz To Arica XV Corque Atta BO LIVIA And cama To es M Iquique Uyuni I 20°S Desert ountains Tocopilla CHILE Calama II Antofagasta To PA R AGUAY Salta To Tolar Grande 25°S 25°S To Fiambalá Copiapo Nevado Ojos III del Salado PAC I F I C OC EAN Vallenar (6,880 m) La Serena REGIONS OF CHILE To I. TARAPAC� Ovalle IV Rodeo 30°S 30°S II. ANTOFAGASTA To III. ATACAMA V UR Villa Nueva Valparaíso IV. COQUIMBO SANTIAGO UG V. VALPARA�SO RM UA RM. REGIÓN METROPOLITANA VI Rancagua Y DE SANTIAGO VI. LIBERTADOR GENERAL Talca VII To A RGENT INA 35°S BERNADO O’HIGGINS Bardas 35°S Blancas VII. MAULE Concepcíon Chillán VIII. BIO B�O VIII To IX. LA ARAUCAN�A Victoria Las Lajas X. LOS LAGOS Temuco IX XI. AISÉN DEL GENERAL CARLOS Valvidia IB�ÑEZ DEL CAMPO XIV To XII. MAGALLANES Y DE LA Osorno Nahuel Huapí ANT�RTICA CHILENA 40°S Puerto Montt 40°S XIV. LOS RÌOS* tains XV. ARICA-PARINACOTA* X * Regions approved by cabinet and Puerto Quellión AT LA NT I C Moun parliament, will become effective on October, 2007. O CEA N This map was produced by the Map Design Unit of The World Bank. Puerto Aisén The boundaries, colors, denominations and any other information To Paso shown on this map do not imply, on the part of The World Bank Coihaique Río Mayo Group, any judgment on the legal status of any territory, or any 45°S Andes endorsement or acceptance of such boundaries. XI Puerto C HI LE Yungay 0 200 400 600 Kilometers SELECTED CITIES AND TOWNS 0 200 400 Miles REGION CAPITALS XII To Río Gallegos 50°S NATIONAL CAPITAL Strait of RIVERS Manatiales Magellan MAIN ROADS Punta To FALKLAND ISLANDS (ISLAS MALVINAS) San Sebastián Arenas A DISPUTE CONCERNING SOVEREIGNTY OVER THE ISLANDS EXISTS BETWEEN ARGENTINA WHICH CLAIMS RAILROADS THIS SOVEREIGNTY AND THE U.K. WHICH ADMINISTERS THE ISLANDS. REGION BOUNDARIES INTERNATIONAL BOUNDARIES Cape Horn 80°W 75°W 70°W 65°W 60°W 55°W 50°W JUNE 2007