DO NOTE CITE. FOR INTERNAL DISCUSSION. 69938 March 3, 2010 Document of the World Bank DO NOTE CITE. FOR INTERNAL DISCUSSION. ABBREVIATIONS AND ACRONYMS AAA Analytical and Advisory Activities AIN-C Atención Integral a la Niñez en la Comunidad/ Child integrated care at the community AMHON Asociación de Municipalidades Hondureñas/Honduran Association of Municipalities ARI Acute Respiratory Infection BCG Bacile Calmette-Guerin CBO Community Based Organization CESAMO Centro de Salud con Médico y Odontólogo/Health Center with doctor and dentist CESAR Centro de Salud Rural/ Rural Health Center CMI Clínica Materno-Infantil/ Mother and child clinic DHS Demographic and Health Survey DFID UK Department for International Development DPT Diphtheria Pertusis and Tetanus GDP Gross Domestic Product GTZ German Society for Technical Cooperation HR Human Resources IADB Inter-American Development Bank LAC Latin America and the Caribbean MDG Millennium Development Goal MSPAS Ministerio de Salud Pública y Asistencia Social NGO Non Governmental Organization ORS Oral Rehydration Solution PAHO Pan-American Health Organization PEC Programa de Extensión de Cobertura PCN Pre-natal care PROHECO Proyecto Hondureño de Educación Comunitaria/Honduran Project of Community Education TT Tetanus Toxoid TTL Task Team Leader UNDP United Nations Development Program UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund USAID United States Agency for International Development WDR World Development Report WHO World Health Organization 2 DO NOTE CITE. FOR INTERNAL DISCUSSION. ACKNOWLEDGEMENTS This report was prepared by María E. Bonilla-Chacín (Sr. Economist) with the help of Sarah Berger and under the direction of Helena Ribe (Sector Manager, LCSHS), Keith Hansen (Sector Manager) and Sajitha Bashir (Sector Leader, LCSHD). This report was based on background papers commissioned for this AAA. These background papers were prepared by the following consultants Lucrecia Méndez, Mark Payne, ESA Consulting, Rosa Inés Ospina, y Ana Etchegaray. At its inception this paper also received contributions from Christine Lao Pena and Laura Rawlings. Finally, the paper also benefited greatly from contributions made by Gastón Blanco (Social Protection Specialist) and Marcelo Bortman (Sr. Public Health Specialist). The report also benefited from comments from Maureen Lewis, Ariel Fiszbein, Ian Walker, and Margaret Grosh. The findings, interpretations, and conclusions expressed herein are those of the authors and do not necessarily reflect the views of the Executive Directors of the International Bank for Reconstruction and Development/The World Bank or the governments they represent. We would like to thank the Guatemalan and Honduran government and their Ministries of Health for their participation and engagement in the political dialogue throughout the execution of this project. The team is also grateful for the financial support of the DFID trust fund, allocated for work on “Governance and Poverty Reduction,� and the Oslo-Norway Trust Fund, granted for research on school-based management, which funded activities for the completion of this case study. Vice President Pamela Cox Sector Director Evangeline Javier Country Director Laura Frigenti Sector Manager Helena Ribe Sector Leader Sajitha Bashir TTLs-Regional Study Maria Eugenia Bonilla-Chacín, Sajitha Bashir 3 DO NOTE CITE. FOR INTERNAL DISCUSSION. EXECUTIVE SUMMARY 1. In the last two decades, Guatemala and Honduras have made significant advances in health outcomes and in access to health services. Since the end of the civil war in 1996, infant and child mortality in Guatemala have decreased more than 50 percent. There has also been a notable decrease in child malnutrition. Honduras, benefiting from a longer period of relative peace and stability, has likewise made significant advances in the reduction of infant and child mortality as well as malnutrition. Finally, in both countries, the coverage of health services has similarly increased. 2. However, large regional, income and ethnic inequalities remain in health outcomes and in access to health services. These differences are particularly large in Guatemala where indigenous children are 1.5 times more likely to die before they reach five years of age than are non-indigenous children and are nearly twice as likely to be malnourished. In Honduras, income and regional differences are also present. In addition, despite progress, some health outcomes are still poor when compared to the regional averages, as with maternal mortality in both Guatemala and Honduras, where the estimated rates are more than double the regional rate. 3. This health study, conducted as part of the World Bank study on “Strengthening Accountability in Social Service Delivery in Central America�, examines the alternative models of health service delivery implemented in Guatemala and Honduras. Following the framework of the 2004 World Development Report, this study encompasses both analysis and active participatory engagement around the core themes of accountability and incentive structures, based on the hypothesis that these are key determinants of performance. Specifically, this study documents the development of alternative models of health service delivery and the accountability mechanisms and incentives developed in these models. 4. Since the 1990s Guatemala and Honduras have implemented innovative health reforms aimed at increasing access to services in rural remote areas; these reforms share some similarities. In both countries, the Ministry of Health contracted out the provision of a basic package of health services to non-governmental organizations (NGOs), community based organizations (CBOs) or to lower levels of government. These contracts or agreements specify the population to be served in rural and often isolated areas. The contracts also establish performance based incentives and set a per capita payment as payment mechanism. In Honduras, for instance, a large percentage of the per capita payment to providers depends upon the achievement of predetermined health coverage targets. In Guatemala, the disbursement of the per capita payment is not based on results; however, the contract renewal is, and the provider’s health personnel can receive small incentive payments based on results. 5. The alternative models of health service delivery in both countries also differ in many features. The PEC in Guatemala is a long standing coverage extension program, whereas, the program in Honduras was developed more recently to contract out services to local agents. More specifically, in Guatemala, the Ministry of Health and 4 DO NOTE CITE. FOR INTERNAL DISCUSSION. Social Assistance enters into agreements with NGOs to provide, through mobile teams, mainly preventive and promotional health and nutrition services in rural isolated areas with no access to health facilities. In Honduras, the Secretary of Health signs performance-based contracts with NGOs, community organizations, and mancomunidades (groups of municipalities) for the provision of a package of primary health care services in rural areas. These alternative providers manage a network of primary health care facilities and thus the package of services they provide is more comprehensive than the one provided in Guatemala. 6. These models have strengthened accountability and incentive structures in the service delivery chain. This has been done through: (i) clearly delineating the responsibilities of providers and Governments in the provision of services; (ii) establishing sanctions for non-compliance with the contracts as well as rewards for the provision of services. In the past, the alternative health care model in Guatemala created a social audit that aimed at strengthening the accountability of providers’ vis-à-vis beneficiaries. In Honduras, the alternative providers agreed to perform client satisfaction surveys. 7. These alternative models have contributed to the expansion of health services in rural areas; in addition, the alternative providers seem to be more productive, although not necessarily less costly, and offer higher quality services. In Guatemala in 2001, the population in the catchment areas of NGOs had the same coverage of basic health services as households in the catchment areas of health facilities.1 In addition, people tend to be more satisfied with the services provided by these NGOs. Finally, the NGOs are more productive than traditional providers but their unit costs are higher.2 In Honduras, evidence from descriptive studies and patient exit surveys implemented in 2007 and 2008 shows higher quality in the services provided by the alternative models of service delivery in terms of compliance with protocols and availability of pharmaceuticals.3 The studies also found that these models are more productive. However, the evidence in terms of cost is mixed. 8. At their inception, both alternative service delivery models benefited from broad Government support. The end of the civil war in Guatemala and the ambitious targets of the Peace Agreements set the stage for the implementation of the alternative model of service delivery in Guatemala, the Programa de Extensión de Cobertura (PEC). In Honduras, the alternative models of service delivery also benefited from Government support at its inception. 9. However, at their inception and during their implementation, both models have also faced major opposition within the Government and from health workers unions. Despite the longevity of the PEC in Guatemala, Government support has been variable after its adoption and thus its progress has been uneven. This program faces 1 Danel & La Forgia. (2005) 2 Idem 3 Pena and Garcia-Pardo (2007) and Secretaría de Salud and Measure Evaluation/PRODIM Consultores. 2009 . 5 DO NOTE CITE. FOR INTERNAL DISCUSSION. large opposition both within and outside the Ministry of Health but lately has benefited from some support given the need to sustain the supply of services to complement the conditional cash transfer program, Mi Familia Progresa. In Honduras, the alternative models have faced strong opposition from health workers’ unions and professional associations. Nonetheless, they have also received the strong support of the Honduras Association of Municipalities and the beneficiaries.4 10. Despite their positive features, many weaknesses persist in the implementation of these alternative models that limit their possible impact. Some of these weaknesses include: (i) poor integration of the alternative models of service delivery into the rest of the health systems; (ii) weak monitoring and evaluation of these models and no mechanisms to verify the information rendered by providers; (iii) in Guatemala, small result-based incentives received by NGO personnel relative to overall remuneration, thus limiting any possible behavior change; (iv) insufficient and delayed payments to NGOs, which has generated disincentives for the provision of services; and (v) in Honduras, results-based incentives received by the provider organization and not the health personnel, also limiting its impact. Possible Ways Forward 11. A systematized monitoring and evaluation system, and more importantly a system to verify providers’ information on results is urgently needed in both countries. As with traditional models of service delivery, the monitoring and evaluation systems for the alternative models are weak and there are no mechanisms to verify the information rendered by providers. As results-based models generate incentives to over- report results; the verification of this information is essential. Guatemala had previously designed systems to monitor providers and verify information: the technical and social audits. However, the budget for these two systems stopped in 2008 when external funds for these audits stopped. Finally, the nominalization of beneficiaries, would, among other things, facilitate the supervision and audit of these programs. 12. A better integration of these models with the rest of the health systems will be needed to ensure faster progress in meeting the Millennium Development Goals, particularly in terms of maternal mortality. This is a major challenge in Guatemala given its limited fiscal space. The itinerant health teams offer mainly preventive and promotional services; to ensure a continuum of care, particularly regarding delivery care, the integration of these teams to the rest of the system becomes vital. However, the PEC offers services to rural and often isolated areas with low population density and no access to the fixed network of health facilities. Thus, better integration will require not only better referral and logistic systems, but also an increased supply of facilities that offer emergency obstetric care. This will be costly; however, the Guatemalan Ministry of Health is currently working on improving the referral system and increasing this supply of services with the support of the World Bank. In Honduras, the alternative models of service delivery offer delivery care. However, a better integration with the rest of the system is still necessary for the provision of a continuum of care. 4 Payne (2009). 6 DO NOTE CITE. FOR INTERNAL DISCUSSION. 13. The size, the recipient, and the probability of receiving a results-based incentive can each determine whether an incentive for increased coverage of services and/or quality will work.5 If the incentive is small, or unlikely to be obtained, or not received by the frontline providers whose behaviors these incentives try to modify, the expected impact might not materialize.6 In Guatemala, the performance incentives health personnel receive are very small and difficult to obtain. In Honduras, the performance based incentives are received by the provider organization and not by the frontline providers. These features of the alternative models of service delivery in Guatemala and Honduras might therefore limit their potential benefits. 14. There is an urgent need to evaluate these programs. There have not been many evaluations done of these models. To better learn from them and correct any unintended negative impact; it is important to rigorously evaluate them. 15. The results-based incentives created in these models can be replicated throughout the rest of the health system. In Honduras, the Government enters into results based agreements not only with NGOs and CBOs but also with municipalities. Countries as diverse as Argentina, Afghanistan, and Rwanda have experimented with results-based financing within the public sector. Although these experiences are relatively new, the evidence available shows promising results.7 5 See Eichler, Levine, and the Performance-Based Incentives Working Group (2009) 6 Idem. 7 Idem. 7 DO NOTE CITE. FOR INTERNAL DISCUSSION. Contents I. INTRODUCTION ...................................................................................................... 9 II. CONTEXT ............................................................................................................... 10 Health Sector Overview ............................................................................................ 10 III. Health Alternative Service Delivery Models .......................................................... 13 A. Characteristics of the models ............................................................................... 14 B. Accountability mechanisms ................................................................................. 25 C. Results and Outcomes .......................................................................................... 26 C. Politics of Adoption and Institutionalization of these models ............................. 30 D. Opposition and support to these programs ........................................................... 33 IV. IMPROVING THE FUNCTIONING OF THESE MODELS................................ 35 V. REFERENCES......................................................................................................... 39 8 DO NOTE CITE. FOR INTERNAL DISCUSSION. I. INTRODUCTION 1. Since the mid-1990s, Guatemala and Honduras have made innovative reforms in health service delivery. These reforms generated alternative service delivery models that strengthened incentive structures and accountability. In the health sector, governments contracted non-governmental organizations (NGOs), community-based organizations (CBOs) and lower levels of Government to provide a basic package of health services to rural, often isolated communities. In both Guatemala and Honduras, these contracts have included performance-based incentives in which the provider is paid the agreed amount or receives bonus payments if he or she achieves the pre-defined and agreed upon service delivery targets. These reforms were mainly aimed at improving access to services in rural areas; however, by strengthening incentive structures and accountability across all actors in the service delivery chain, the reforms could also improve the efficiency and quality of health service delivery. Objectives 2. The purpose of the AAA is to suggest improvements in the delivery of services in Central America by analyzing alternative models of service delivery, their features and particularly the accountability relationships and incentives structures they created. Following the framework of the 2004 World Development Report (see Analytical Framework in Annex 1), this AAA encompasses both analysis and active participatory engagement around the core themes of accountability and incentive structures, based on the hypothesis that these are key inputs to improving service delivery. Specifically, the AAA will document the development of alternative models of health service delivery and the accountability mechanisms and incentives developed in these models. 3. This study is based on two case studies analyzing alternative models of service delivery. These case studies are: (i) Honduras-health: Decentralized models of health service delivery; and (ii) Guatemala-health: Coverage Extension Program (PEC). For the development of each case study, the following three assessments were commissioned:  An institutional analysis of the alternative and traditional service delivery models. This background study described and assessed the institutional arrangements and incentive structures of the alternative service delivery models and compared them with the traditional ones. These analyses also summarized what is known about the results of these models. The institutional analyses of PEC in Guatemala and the alternative models in Honduras were: a. Análisis Institucional del Programa de Extensión de Cobertura by Lucrecia Méndez Pérez, June 2009. b. Rendición de Cuentas y los Servicios Sociales en Centro América el Caso del Sector Salud en Honduras by ESA Consultores.  A political economy analysis of the adoption and institutionalization of these alternative models of service delivery. These background pieces were: 9 DO NOTE CITE. FOR INTERNAL DISCUSSION. a. Political Economy of Innovation in Guatemala Health by Mark Payne, 2009. b. Political Economy of Innovation in Honduras Health by Mark Payne, 2009.  A qualitative analysis of the alternative models of service delivery. These analyses were based on focus group discussions with beneficiaries, providers, official from the Ministries of health, and other stakeholders of the alternative and traditional models of service delivery. These background pieces were: a. Análisis Cualitativo sobre la Gobernabilidad del Sector Salud en Guatemala by Rosa Inés Ospina y Ana Etchegaray.8 May 2009. b. Análisis Cualitativo sobre la Gobernabilidad del Sector Salud en Honduras by Rosa Inés Ospina y Ana Etchegaray.9 April 2009. 4. In the following sections, the study explores what we know about the alternative models of health service delivery in Guatemala and Honduras. The next section of the study presents the context in which these models were developed. The third section describes the alternative service delivery models in the two countries: characteristics, accountability mechanisms created, impact, and political economy context in which they were adopted and implemented. The last section explores possible options for improving service delivery in general and the alternative service delivery models in particular. II. CONTEXT 5. Since the end of the civil wars in the region, Central America has enjoyed relative peace and political stability. In December of 1996, the Peace Accords in Guatemala marked the end of almost four decades of civil war. In the social sectors, these Accords set specific goals and targets to be achieved by the year 2000. In health, the Peace Accords aimed at increasing access to services, increasing the share of domestic product allocated to the sector, and allocating 50 percent of the health budget to preventive services and services aimed at halving maternal and child mortality. These Peace Accords set the stage for the development of the alternative models of service delivery in Guatemala. Honduras did not suffer a civil war and thus enjoyed until early 2009 relative stability for much longer time. Health Sector Overview 8 This qualitative study is based on the results of six focus group discussions: two with beneficiaries of PEC, two with providers and two with District Directors and chiefs of areas in San Pedro Carchá in Alta Verapaz and in Jalapa. 62 people participated in these discussions. 9 This qualitative study in Honduras is based on the results of five focus group discussions: one in the community of El Guate in Francisco Morazán with emphasis in the decentralized models; one in los Cedros in Francisco Morazán with an emphasis in the decentralized models; two in Taulabé in Comayagua , one with a focus on the decentralized model and one in the traditional model; and one in Tegucigalpa. Community members, providers, health facility staff, and personnel from the ministry participated in these discussion. A total of 102 people participated in the focus groups discussions. 10 DO NOTE CITE. FOR INTERNAL DISCUSSION. 6. In 1995, prior to the alternative models of service delivery in Guatemala, and years before they started in Honduras, health outcomes and access to health services in both countries were poor compared to other countries in the region (see Table 1). This was particularly the case in Guatemala where infant and child mortality rates were higher than most countries including those with a lower income such as Nicaragua and Paraguay. These average health indicators also concealed large inequalities across income, locality, and ethnic groups. The alternative models of service delivery were established as a strategy to improve access to basic services in rural areas and thus improve outcomes and diminish inequalities. Table 1: Infant and child mortality in Latin America 1995 1995 Country Name Under-5 Infant Argentina 25 23 Belize 32 26 Bolivia 105 76 Brazil 42 37 Chile 14 13 Colombia 31 25 Costa Rica 16 14 Ecuador 43 34 El Salvador 46 37 Guatemala 64 49 Honduras 49 39 Mexico 45 36 Nicaragua 53 41 Panama 30 23 Paraguay 37 31 Peru 65 50 Suriname 47 36 Uruguay 20 18 Venezuela, RB 28 23 Source: World Bank Development Data Platform 7. Health and nutrition indicators in Guatemala and Honduras, however, have improved significantly since the 90s. As seen in the graphs below, neonatal, infant, and under five mortality rates have markedly decreased in both countries. Malnutrition rates have also decreased. In Guatemala, the decrease in chronic malnutrition rates between 1987 and 2008/2009 was about 14.5 percentage points.10 In contrast, in Honduras, the percentage of children five years old or younger chronically malnourished decreased 18 percentage points between 1991 and 2006.11 10 Monteith, R. Stupp, P. and Mc Cracken, S. (2005) and ENSMI-2008/09. 11 Idem. 11 DO NOTE CITE. FOR INTERNAL DISCUSSION. Graph 1: Guatemala - Trends in neonatal, Graph 2: Honduras – Trends in neonatal, infant, and under five mortality infant, and under five mortality 60 48 50 45 36 34 40 30 30 23 19 19 20 14 10 0 1991-1995 1996-2000 2001-2006 Neonatal Infant Under five Source: Monteith, R. Stupp, P. and Mc Cracken, S. Source: Monteith, R. Stupp, P. and Mc Cracken, S. (2005) and ENSMI-2008/09 (2005) and Honduras DHS 2005/06 8. Despite improvements in health outcomes and in access to services, some health indicators like maternal mortality remain poor. In 2005, UNICEF, WHO, UNFPA, and the World Bank estimated the maternal mortality rate in Guatemala at 290 per 100,000 live births and in Honduras at 280 per 100,000 live births, both rates more than twice as high as the Latin America and the Caribbean average estimated rate of 130 per 100,000 live births. 9. Similarly, large regional, income and ethnic inequalities persist in health outcomes and in access to health services. Data from the Guatemala ENSMI-2008/09 show significant inequalities in health outcomes and access to health services, particularly between the indigenous and non-indigenous population. In 2008/2009, while child mortality among the indigenous population was 51 per 1,000 live births, among the non-indigenous it was 33 per 1,000 live births (see Graph 1). Additionally, the percentage of indigenous children chronically malnourished (58.6 percent) was twice as high as that of non-indigenous children. These inequalities can also be observed in access to basic health services. Indigenous women in 2008/2009 were two times more likely to deliver a baby without the assistance of a doctor than non indigenous women. 10. In 2000, the Ministry of Public Health and Social Assistance (Ministerio de la Salud Pública y Asistencia Social- MSPAS) in Guatemala estimated maternal mortality to be 153 per 100,000 live births.12 However, this average hides large regional differences. For example, in Alta Verapaz, the maternal mortality rate is estimated to be 302 per 100,000 live births; in Solola 297; in Huehuetenango 255; in Totonicapán 219; and in Izabal 217. The same MSPAS report estimates that the risk of maternal mortality was three times greater among the indigenous population, 211 per 100,000 live births, relative to the rest of the population, 70 per 100,000.13 11. In Honduras, between 2001 and 2006, child mortality (children under five) in San Pedro Sula was noted at 23 per 1,000 live births and in Copán, 61 per 1,000.14 12 This is a much lower estimate than that of the UNICEF, WHO, UNFPA, and World Bank 2005 of 186 per 100,000 in 2005. 13 Méndez Pérez (2009). 14 Demographic and Health Surveys. 2005/2006. Honduras. 12 DO NOTE CITE. FOR INTERNAL DISCUSSION. Furthermore, the graphs below (Graphs 3 and 4) show socio-economic differences in the percentage of children chronically malnourished and in women’s access to maternal services in the country. As noted in Graph 3, the percentage of malnourished children is almost 9 times greater among children in the poorest quintile of households than among the richest quintile. Graph 3: Honduras, percentage of children Graph 4: Honduras, percentage of women chronically malnourished by wealth quintiles delivering at a health facility and of pregnant women receiving prenatal care across wealth deciles 50 100 40 80 30 60 20 40 10 20 0 0 Poorest II III IV Richest Poorest II III IV Richest Deliver in facility pre-natal care Source: Honduras DHS 2005/06 Source: Honduras DHS 2005/06 III. Health Alternative Service Delivery Models 12. In Guatemala the Coverage Extension Program (Programa de Extensión de Cobertura- PEC) was initiated in 1997 with the aim of extending coverage of health and nutrition prevention and promotion services to poor, rural and largely indigenous populations through the contracting of NGOs. This program was created in the post-civil war period as a strategy to meet the ambitious goals set in the Peace Accords of 1996. Given that many NGOs were already on the ground offering services, the PEC formalized and extended these existing practices. This program received support from many donors, particularly the IADB but also PNUD, BID, PAHO, GTZ, European Union, World Bank, and USAID.15 13. The health alternative models in Honduras began in 2004 as part of the Government’s efforts to decentralize the management and provision of basic health services and decrease inequalities in access to services. In that year the Secretary of Health, with support from the World Bank, began piloting decentralized models of service delivery that are currently operational. These models are established through contractual agreements between the Secretary of Health and CBOs, NGOs, and mancomunidades or municipal associations. This was a continuation of a process of reform which began in the early 1990s focused on enhancing the role of local governments and communities. Since 2001, there have been different pilot projects on decentralized management of health facilities, all supported by international donors. These pilot projects have received financial support from the Swedish Government funded Access Project and from the Inter-American Development Bank, which financed 15 Méndez Pérez (2009). 13 DO NOTE CITE. FOR INTERNAL DISCUSSION. a pilot that supported the provision of health services through the use of mobile health teams. The models that are currently functioning receive support from the Italian Government’s debt relief, the World Bank, and USAID.16 A. Characteristics of the models 14. The alternative service delivery models in Guatemala and Honduras are characterized by the use of contracts with public and private health care providers. In both countries, the Ministry of Health contracts a third party to provide services to a previously defined population. Through these contracts/agreements, these alternative models establish a clear division of responsibilities between the Ministry of Health, which is now in charge of financing and supervising the contracts, and the third party, which is now responsible for providing services. 16. In both countries these contracts/agreements share the following features (see Table 2): (i) they benefit poor and remote rural communities with limited or no access to services; (ii) the services contracted are primary health care services, including health promotion, preventive services, and a few curative services; (iii) these contracts/agreements establish clear rewards and sanctions in case of non-compliance; and (iv) the provider payment mechanism established by the contracts is a capitation payment, giving the provider an incentive to increase efficiency. Table 2: Comparison of health alternative models of service delivery in Guatemala and Honduras Characteristic Guatemala Honduras Eligibility criteria Poor and remote rural areas Poor rural areas without access to health services Service Providers NGOs NGOs, CBOs, Mancomunidades Method of provision Itinerant teams Networks of health centers: CESAR, CESAMO, CMI, and itinerant teams Services provided Maternal and child health Maternal and child services: services: preventive and Preventive, promotion, and promotion services and the curative. All services normally control of childhood provided in primary health care prevalent diseases. Does facilities, including deliveries. not include institutional deliveries Infrastructure - Often the provider uses the infrastructure of the Ministry of Health or rents space that is then habilitated to provide services. Monitoring and Supervision of Until recently, there was a Regional level staff from the contracts technical and a social audit Secretary of Health monitors designed to verify contract contracts every three months. 16 ESA Consultores (2009). 14 DO NOTE CITE. FOR INTERNAL DISCUSSION. compliance, provide Central level monitors contracts technical assistance to on a yearly basis. providers and ascertain consumer satisfaction. Providers’ actions are supervised by the Health Area Directorate. Not clear if they verify the information provided. Human Resources All health providers are Managers can contract their contracted by NGOs own personnel. However, some managers work with personnel from the Ministry of Health and also with contractual personnel. In some cases providers contract a firm that provides health personnel services and not individual doctors or nurses. Financing Ministry of Health Ministry of health and also co- payments Payment mechanism Per capita Per capita Fee for services for deliveries Incentives for results Personnel incentives based The total payment to the on the results of NGOs provider depends on achieving evaluations. results (up 30% of the per Small incentive based on capita payment is based on the results of the social results, the rest is a fixed audit payment). In addition, at the end of the year the provider can receive as incentive payment 1% of the total cost of the contract if he reaches a score of 90% in the yearly performance evaluation. Contract/agreement renewal Depends on the results Depends on the results achieved. achieved. 15. While these two models have many common characteristics, they also differ in many aspects. In Guatemala, the PEC is a long standing extension of coverage program that benefits millions. The decentralized models of service delivery in Honduras are relatively new schemes aimed at improving the efficiency of the services provided by contracting provision to local agents. Specifically, in Guatemala, the PEC signs agreements (convenios) directly with NGOs to provide, through mobile teams, a basic package of services to isolated rural communities. There is a second alternative model of service delivery in Guatemala where NGOs are responsible for managing itinerant teams linked to public health facilities. This study focuses on the first model which represents the majority of the contracts with NGOs. As seen in Table , in 2001 the program offered services to about 3 million people. By 2007, the Government had contracts with 85 15 DO NOTE CITE. FOR INTERNAL DISCUSSION. NGOs working in 26 health areas in 20 of the country’s departments, serving an estimated 4.1 million people (see Table 7). Table 3: Evolution of the Extension Coverage Program (PEC) 2001 2002 2003 2004 2005 2006 2007 Municipalities 164 164 184 168 184 198 439 Jurisdictions 288 290 303 316 376 402 439 Centros de Convergencia 2.501 2.700 3,945 3,062 3,989 4,436 4,436 Population Covered (convenio) 3.0 3.2 3.6 3.2 3.8 4.1 4.2 Agreements/Convenios 154 161 155 199 196 381 408 Providers 89 88 92 102 93 83 85 Source: Programa de Extensión de Cobertura – MSPAS Guatemala as appeared in Méndez (2009). 16. In Honduras, the Secretary of Health has contracts with NGOs, community based organizations, and mancomunidades, which are groups of municipalities. At the moment, the Secretary of Health is managing 28 contracts, 7 of them with mancomunidades. The alternative providers in Honduras manage a network of primary health care facilities, and although some of these networks also provide services through mobile teams, this is not their only method of provision. Of the 28 contracts on implementation, only two contracts provide services through mobile teams only.17 17. There are also differences in the payments to providers and in the contract evaluation. In Guatemala, although the financial transfers to providers are based on capitations, the disbursement does not depend on results; however, the renewal of the contracts does. Each trimester the NGOs are evaluated based on the achievement of predetermined targets of 28 indicators (see Table 4). Depending on the scores received by each NGO in these evaluations, agreements can be canceled, their continuation can be made conditional, or they can continue but the NGOs must implement a performance improvement plan. NGOs that obtain a score higher than 81 receive a written recognition of their performance. Table 5 presents the results of these evaluations in 2006 and 2007. In 2006 seven agreements were eliminated due to poor performance and the continuation of 23 agreements was conditional on improvements. Only 87 agreements received recognition. In 2007 there was an improvement as only 5 agreements were eliminated and 16 made conditional. The number of NGOs receiving a written recognition increased to 235. Table 4: Guatemala Health Indicators from PEC Indicator Annual Target % of pregnant women with one pre-natal control 75% % of pregnant women that received pre-natal control within 12 weeks of pregnancy 21% % of pregnant women with three pre-natal controls 40% % of women that receive a postnatal control in the first 40 days after delivery 44% % of newborns that receive a postnatal control in the first 28 days after birth. 44% % of women 15 to 49 years old that have received micronutrients. 17% 17 ESA Consultores (2009). 16 DO NOTE CITE. FOR INTERNAL DISCUSSION. % of women 15 to 49 years old that are new users of family planning methods. 24% % of women 15 to 49 years that use family planning that have been re-stocked each 60% trimester with methods. % of women that have taken a sample of PAP/IVAA. 10% % of women that have received the results of their PAP/IVAA. 10% % of women of fertile age that have received the 3rd doses of TT. 90% % of one year olds that have received the BCG vaccine. 95% % of one year olds vaccinated with OPV-3 95% % of one year olds vaccinated with PENTA-3 95% % of one year olds vaccinated against measles 95% % of one year olds with at least 2 medical controls 90% % of children 1-5 years old with at least one medical control 90% % of children under five with pneumonia that have received treatment 100% % of children under five with diarrhea that have received ORS 100% % of children under 2 years that have received at least one weight control during 55% the trimester % of children under 2 years with at least 2 continuous weight controls 55% % of children 2 to 3 years old with at least one weight control per trimester 70% % of children under 1 that have received vitamin A supplements after their 6 85% months of age % of children 1-5 years old that have received a second doses of Vitamin A 55% % of children under 1 that have received iron sulfate for three moths 60% % of children 1 to 5 years old that have received iron sulfate for 3 moths 60% % of coverage of canine vaccine 80% % of active traditional birth attendants that have been trained 100% Source: MSPAS as appeared in Méndez (2009) Table 5: Guatemala Results of contract evaluation with providers in 2006 and 2007 Number of Number of jurisdictions jurisdictions in Evaluation Classification in 2006 2007 range 7 5 Less than 50 Agreement eliminated points 23 16 50 to 60 pointsConditional 246 175 61 to 80 pointsContinue providing services but must implement action plan to improve performance 87 235 81 to 100 Continue providing services. The NGO points receives a written recognition. 1 Non classified New agreement PNUD USAID 1 Non classified Intervened?? Source: Unidad de Supervisión y Monitoreo del PEC/MSPAS as appeared in Méndez (2009). 18. Even though the health indicators and the targets to be achieved by NGOs have remained the same for many years, on average, only a few of these targets are routinely achieved (Table ). These targets are not coordinated with NGOs and do not vary to reflect heterogeneous jurisdictions. The lack of progress might indicate a lack of 17 DO NOTE CITE. FOR INTERNAL DISCUSSION. incentive to providers to achieve them, lack of provider capacity, or an insufficient financing of the services. Table 6: Guatemala PEC health indicators, results and targets 2002-2008 DHS PEC PEC PEC PEC Indicator 2002 2005 2006 2007 2008 Target % of pregnant women with one pre-natal control 51.10% 72.82% 61.50% 64.88% 69.51% 75.00% % of pregnant women that received pre-natal control within 12 weeks of pregnancy 0.00% 14.63% 18.63% 22.40% 22.70% 21.00% % of pregnant women with three pre-natal controls 4.80% 21.45% 22.39% 30.15% 34.94% 40.00% % of women that receive a postnatal control in the first 40 days after delivery 20.30% 47.76% 39.50% 51.09% 55.32% 44.00% % of newborns that receive a postnatal control in the first 28 days after birth. 43.20% 36.61% 31.35% 43.82% 50.07% 44.00% % of women 15 to 49 years old that are new users of family planning methods. 43.30% 14.83% 11.88% 13.90% 16.63% 24.00% % of women of fertile age that have received the 3rd doses of TDA. 0.00% 0.00% 19.31% 28.55% 34.58% 90.00% % of one year olds that have received the BCG vaccine 97.30% 0.00% 0.00% 86.88% 86.80% 95.00% % of one year olds vaccinated with PENTA-3 0.00% 0.00% 0.00% 80.83% 82.51% 95.00% % of one year olds vaccinated against measles 0.00% 0.00% 79.11% 71.25% 75.10% 95.00% % of children under 1 that have received vitamin A supplements after their 6 months of age 0.00% 0.00% 63.61% 76.43% 77.12% 85.00% % of children 1-5 years old that have received a second doses of Vitamin A 0.00% 0.00% 13.19% 20.62% 31.75% 55.00% Source: PEC Management Information System and DHS 2002 as in Méndez (2009) 19. In Guatemala, the health personnel contracted by NGOs—not the NGO or provider organization as is the case in Honduras—are eligible to receive incentive payments also based on the trimester evaluation. If the NGO receives a score of 81 or higher in these evaluations, the provider personnel are eligible to receive the incentives shown in Table 3. These personnel are eligible for up to five incentive payments per year. As seen below, these incentive payments represent a very small percentage of the total remuneration. Finally, in the past, providers could also receive an incentive based on the results of the social audit. 18 DO NOTE CITE. FOR INTERNAL DISCUSSION. Table 3: Guatemala Incentives by type of personnel working in PEC 2009 Amount of incentive Maximum amount Maximum amount as Personnel per year % of total salary Ambulatory Doctor Q500.00 Q2,500.00 2.6% Ambulatory Nurse Q500.00 Q2,500.00 3% Institutional Q500.00 Q2,500.00 7% facilitator Information Assistant Q200.00 Q1,000.00 4% Accountant Q250.00 Q1,250.00 4% Health and Nutrition Q200.00 Q1,000.00 3.3% Educator Source: Approved budget for an NGO working in PEC 2009 as appeared in Méndez (2009). 20. In 2010, the MSPAS will pilot, with the support from the World Bank, the organization of institutional mobile health teams to provide a similar package of health services in areas where there is a health facility. These institutional mobile teams will be linked to a fixed facility and will be organized and supervised by the Departmental Health Directorates who will enter into performance agreements with the central MSPAS. A fifth of the per capita payment to the departmental health directorate will be based on performance.18 21. In Honduras, the transfer of the capitation payments to providers depends on the achievement of specific and pre-determined coverage targets; ensuring that the provider has an incentive to increase the supply of services. Providers receive payments on a monthly basis. However, if the provider receives a score lower than 85 percent in the contract evaluation, the payments in the last month of each trimester can be reduced from 2% to as much as 30 percent (see Table ). Table 8: Honduras decentralized health models: Deductions in transferences according to NGO evaluation Qualification ranges of trimester % of per capita to be paid each monitoring trimester 85 to 100% 100% 75 to 84% 98% 65 to 74% 96% 60 to 64% 94% Less than 59% 70% Source: Secretaría de Salud. Management agreement with “Madre Feliz� Association 22. The provider evaluation depends on the achievement of service coverage targets specified in the contract (see Table for an example of indicators and targets). In addition, if the provider scores more than 90% in this evaluation, they receive an incentive at the end of the year, representing an additional 1% of the total amount of the capitation payments. Finally, deliveries and transport for women to the clinics are paid 18 World Bank (2009). 19 DO NOTE CITE. FOR INTERNAL DISCUSSION. separately. The NGO and the personnel assisting deliveries receive a fee per service, generating an incentive to increase institutionalized deliveries. Table 9: Honduras- decentralized health service delivery model—Example of contract indicators and targets 2008 Indicator Population Target Pre-natal care coverage 90% Puerperal coverage (before 10 days) 80% Coverage of pentavalent immunization for infants 95% BCG coverage for infants 95% Measles immunization for children 11 to 24 months 95% Coverage of DPT2 in 18 months old children 95% Health services for children under five 80% Source: Secretaría de Salud. Management agreement with “Madre Feliz� Association 23. In Honduras, very few providers receive a poor score. It is not clear whether this reflects good performance on the ground, a limited list of indicators, or lack of verification of the information rendered by providers. For instance, data from the Unit of Coverage Extension and Financing of the Health Secretariat from May 2009 indicates that of the 18 alternative providers for which evaluation was available at the time, only one had an evaluation lower than 70 percent and 14 had an evaluation equal to or higher than 85 percent.19 The performance indicators and targets to be achieved are determined by the Secretary of Health. 24. The services provided by these alternative models also differ in the two countries. Given that the PEC in Guatemala is an extension of coverage program, the basic package of services it provides includes mostly health promotion, preventive services, and some curative services; it does not include institutional deliveries. Annex 2 presents the list of services contracted to the NGOs. The basic services contracted in Honduras include preventive, promotion, and curative services, including institutional deliveries. These services are the services normally provided by the primary health services (CESAR y CESAMO) and by the maternal and child clinics (CMIs). Annex 3 list the services provided by these primary health care centers. 25. In both countries, the per capita payment was based on an estimation of the cost of the services provided. In the case of Guatemala, this per capita payment varies across jurisdictions according to geographical access, a health index, and an index of unsatisfied needs. This payment varies from about US$7.10 (54 Q) to US$10.50 (80Q). NGOs complain that this amount is rarely revised and that the estimated population per jurisdiction has also not been revised in four years, even though this population has increased20. In Honduras the per capita payment varies between US$15 - US$18 depending on the size of the population and ease of geographical access; in addition, the 19 ESA Consultores (2009). 20 Méndez Pérez (2009). 20 DO NOTE CITE. FOR INTERNAL DISCUSSION. provider receives a fee of about US$ 84-106 (Lempira 1,600-2,000) per delivery and about US$ 42-79 (L. 800-1.500) per delivery transportation21. 26. In both countries the provider organization directly hires health personnel. However, in Honduras some providers are assigned the management of facilities where there are already civil servants working. In these cases, the Ministry of Health directly pays the salary of these personnel and subtracts these salaries from the contracted amount22. In some cases alternative providers in Honduras contract a firm that provides health personnel services and not individual doctors or nurses. 27. The requirements for health care personnel are also different in the two countries. In Guatemala, each NGO must hire one doctor or ambulatory nurse and one “institutional facilitator� per basic health team. These teams cover 10,000 people. In addition, each health team should also have “community facilitators� and traditional birth attendants who receive small stipends for services rendered. In areas with high maternal mortality rates the NGOs must also hire an auxiliary nurse qualified in maternal and neonatal services. In addition, in areas with high malnutrition rates health and nutrition educators23. 28. In Honduras, the technical team to be contracted by the provider organization depends on the number and type of facilities to be managed by the organization and the number of beneficiaries. In general, the personnel required are more numerous than in Guatemala. For instance, the teams must have one promoter per 2,000 people, one doctor per 5,000-7,000 people, one dentist per CESAMO24 or per 10,000 people, one qualified nurse per contract, one auxiliary nurse per 2,500 to 3,000 people, one lab technician per CESAMO and an administrative unit per contract. These administrative units must have as a minimum: one doctor or nurse to coordinate the service network, one promotion coordinator, and one statistician25. 29. In addition, one of the advantages of alternative models in Honduras is the presence of a contractual covenant that forces the provider to have the required health personnel at all times and to have the facilities open from 8am to 4pm Monday to Fridays and from 8am to 12pm on Saturdays. In contrast, in the traditional model the health facilities and particularly the CESARs (rural health centers) are only open five days a week and are often not available 12 months of the year due to vacations and lack of resources to substitute for personnel on leave.26 Finally, another advantage of the alternative models in Honduras is that the contractual doctors can work 8 hours a day and not only 6 hours as stipulated in the Doctors Statute for civil servant doctors. These latter characteristics of the models represent a great advantage in the provision of services but they can also represent a source of tension with the unions. 21 ESA Consultores (2009). 22 ESA Consultores (2009). 23 Méndez Pérez (2009). 24 Health center with a doctor and a dentist. 25 ESA Consultores (2009). 26 Secretaría de Salud and Measure Evaluation/PRODIM Consultores (2009). 21 DO NOTE CITE. FOR INTERNAL DISCUSSION. 30. In Guatemala health personnel under alternative delivery models have fewer benefits than personnel under the traditional model; this is a source of contention against these models. In Guatemala, although based salaries for contracted NGOs are similar to those of civil servants; the former do not receive the following benefits: (i) end of the year bonus or aguinaldo equivalent to a month salary; (ii) bonus 14 paid in July of each year also equivalent to a month salary; (iii) vacations; (iv) employers compensation; or (vi) social security.27 31. In Honduras, data from a health facility staff survey indicate that health personnel working in facilities managed by mancomunidades or municipalities receive on average lower remunerations than other public sector personnel.28 Their perception of how their wages relate to market wages is different from other public sector workers. The same staff survey indicated that 38.1 percent of all health workers interviewed perceived that they receive less than market wages (but the proportion decreases to 27.1 percent in private facilities and increases to 33.3 percent in “municipal� units); 37.2 percent reported earning comparable wages (but 54.2 percent and 58.3 percent in private and “municipal� facilities respectively); and 23 percent believe they earn above market wages (16.9 percent and 8.3 percent) (see Table 10).29 Table10: Honduras Mean gross monthly wage by type of facility and profession (in Lempiras) Medicine Nursing Other Managemen Other Total Technical t National Hospitals 30.875 13.456 19.459 8.448 9.742 14.242 Regional Hospitals 14.500 11.330 10.683 6.977 9.874 Local Hospitals 37.000 9.333 10.552 7.810 6.467 12.876 IHSS facilities 35.000 13.500 12.643 10.222 8.850 13.469 CESAMOs 20.227 9.932 12.411 10.187 10.035 11.436 CESARs 32.050 8.456 4.740 NA 9.280 9.546 CMIs 29.700 8.001 NA 2.900 4.500 8.365 CLIPPERs 30.000 10.141 9.682 8.100 NA 10.451 "Municipal" units NA 5.875 6.159 5.500 NA 6.010 Private Clinics 5.321 4.747 27 Méndez Pérez (2009). 28 This staff survey was included in a Public Expenditure and Service provision survey in Honduras (World Bank, forthcoming) implemented in 2009. This surveyed interviewed 358 health facility staff in the following four departments: Francisco Morazán, Yoro, Copán, y Olancho. 29 World Bank. (forthcoming). Honduras Public Expenditure and Service Provision Survey. 22 DO NOTE CITE. FOR INTERNAL DISCUSSION. 12.861 6.438 5.145 6.258 Total 25.534 9.027 10.514 7.994 8.008 10.275 Source: World Bank (forthcoming): Public Expenditure and Service Provision Survey - Staff survey. 32. The systems set up to monitor and evaluate contract compliance are also different in both countries. In Guatemala, the government set up technical teams to audit the work of the NGOs on a regular basis. These technical teams were based in the regional offices of the Ministry of Health. These teams visited the beneficiary communities to verify the financial and service information provided by NGOs, including the verification of the data on 28 coverage indicators. The renewal of the contracts depended on the results of these technical evaluations. In addition, in 2006 the Ministry set up a social audit as a feedback mechanism to providers. This audit collected the perceptions of a few community members on the services provided. Based on the results of this audit, providers could receive an incentive payment at the end of the year. 33. The technical teams in Guatemala charge of monitoring contract compliance were eliminated in 2008. The evaluation of the NGOs is now done by the Health Area Directorates. It is not clear, however, whether the Health Areas Directorates verify the authenticity of the information provided by the NGOs. For instance, in a focus group analysis of beneficiaries and providers commissioned by this AAA, beneficiaries noted that in many instances the health authorities receive the reports from the NGOs to process their payment without real monitoring or control of what is done in practice; thus some NGOs over-report.30 In these focus groups, officials from the Ministry of Health also indicated lack of resources to do adequate monitoring and evaluation of the providers’ performance.31 34. The Guatemalan Ministry of Health, with the support of the World Bank, is planning to expand the package of health services offered by PEC to include child growth monitoring at the community level (AIN-C) and create institutional mobile teams as discussed before. The MSPAS will contract an external firm to verify the information rendered by the institutional providers. This firm will verify that the services were provided with the agreed quality standards to the target population, and that the targets were reached32. 35. In Honduras, teams from the regional departmental health office supervise the alternative models every three months. An annual evaluation of the contracts takes place at the end of the year. This evaluation is the responsibility of the Coverage Extension and Financing Unit of the central level of the Ministry of Health. It is also not clear whether this supervision also verifies the information reported by providers. 36. There is no roster of beneficiaries of the alternative service delivery models, making difficult the supervision of providers and the verification of information. If 30 Ospina and Etchegaray (2009). 31 Idem. 32 World Bank (2009). 23 DO NOTE CITE. FOR INTERNAL DISCUSSION. the reports on results are nominalized, with the name and identification of people receiving services, the time and place where these services were received, and the location of their clinical history it would be relatively simple to verify whether these services were actually provided. The nominalization would help ensure that the provider receives payment for the actual number of beneficiaries under its catchment area, and not for an estimated population. Finally, the nominalization would ensure that the payment for performance would be based on the results achieved and not a measure based on an estimated population. To ensure that there is no misreporting of beneficiaries the roster should also be audited. 37. In Guatemala, most of the information needed for a complete nominalization is available. The NGOs implement a population census in their jurisdictions and have all the information needed to generate a complete roster of beneficiaries: name, last name, address, characteristics, whether they have received services, etc. However, this information has not been audited and it is not used in the contracts and thus the per capita payment is based on an estimated population. Additionally, it is not clear whether this information is included in the results that are reported. The MSPAS is planning to generate a roster of beneficiaries for the institutionalized mobile teams. 38. In Honduras, the situation is similar; the contract for alternative providers establishes as one of their responsibilities the implementation of a population census. However, the information to collect is not specified and the data collected not audited. The reports that providers render do not include this information either. 39. Finally, the PEC in Guatemala is not fully integrated to the rest of the health system. As aforementioned, the PEC was designed with the aim of offering preventive and promotion health and nutrition services to communities with no access to the network of health facilities. These services have been proven to be cost-effective, especially with regard to improving child and infant mortality. 33 However, to improve maternal health and decrease maternal mortality, women should have access to facilities that offer emergency obstetric care in case of complications. 40. Thus, to improve maternal health the mobile teams need to be better integrated with the rest of the health system. This will require not only better referral and logistic systems but also an increase in the supply of facilities that offer basic emergency obstetric care. This will be costly. The Government is currently re-designing its health care network to ensure a working referral and counter-referral system. The Ministry of Health, with the support of the World Bank, is also transforming health centers in municipal capitals in facilities that can offer integral maternal and health services called CAIMI (Centro de Atención Integral Materno Infantil). This is not the case in Honduras as most of the alternative providers manage a small network of health facilities, many of them public facilities that were previously closed for lack of personnel. 33 Lancet Child Survival Series. 2003. Volume 36, number 9351 24 DO NOTE CITE. FOR INTERNAL DISCUSSION. B. Accountability mechanisms and incentive structures 41. These alternative models improve the long route of accountability by strengthening the compact between frontline providers and policymakers. This is achieved through: (i) clearly delineating responsibilities of providers in the contracts with NGOs or with sub-national governments for the provision of health services; (ii) establishing clear sanctions for non-compliance with the contracts; (iii) establishing positive incentives in contracts to ensure the provision of services;34 and (iv) collecting, monitoring and evaluating information on performance, contract compliance, and service delivery outcomes. These characteristics distinguish these alternative models from the traditional way of delivering health services. 42. These models have also tried to strengthen the accountability of frontline providers to the beneficiaries, although with less success. In Guatemala, at the end of the year, providers receive a bonus payment based on the results of a social audit. This social audit collects information from beneficiaries on the quantity and perception of quality of services provided by NGOs. The social auditor collects information directly from beneficiaries and from organized members of the community. This auditor is usually a member of a community organization, although does not necessarily use the services. The audits consist of two sets of interviews per year, one to members of community organizations and another one to service users. The results of the audits are then discussed with the providers. Although some jurisdictions still implement these audits, they have not been financed since 2008.35 43. In Honduras, the feedback from the community to frontline providers is not systematic, although in the contracts, the providers agree to implement beneficiary surveys. It is not clear whether the information collected is used as feedback to providers. In addition, as these surveys are done by the same provider organization, there is a risk that the results of these surveys could be misreported if at all reported. Communities in Honduras are largely involved in volunteer work in programs like AIN-C or in vaccination campaigns. Community involvement has also been present through participation in health committees and via consultations, for instance, in deciding the location of facilities, in providing feedback through health committees, user surveys, exit polls, and suggestion boxes. However, their involvement in the actual management and oversight is limited. Preliminary results of a public expenditure and service provision survey shows that alternative facilities managed by municipalities are not more likely than other facilities to have a user committee (see Table 11). Finally, approximately, 50 percent of the alternative models in Honduras involve the management of health services by community organizations. Partial information from the qualitative governance study indicates that even in these cases communities are not directly involved on the oversight of these organizations; limiting thus the accountability of the provider vis-à-vis communities. 34 In Honduras, for example, contracts offer an incentive to providers upon achievement of previously specified service coverage targets (about 1% of the total per capita payments); and in Guatemala, providers receive an additional funds based on the results of a social audit. 35 Méndez Pérez (2009). 25 DO NOTE CITE. FOR INTERNAL DISCUSSION. Table 11: Presence of Users’ Committees by Facility Type and Management Model in 4 departments in Honduras 2009 Public Managed by Managed by Managed by Management Mancomunidad NGO, Civil or Private entity or Municipality Community Association National Hospitals 80.0 - - - Regional Hospitals 100.0 - - - Local Hospitals 33.3 - - - IHSS Hospitals 100.0 - - - CESAMOs 69.0 100.0 100.0 - CESARs 78.9 100.0 100.0 - CMIs 62.5 50.0 0.0 - CLIPPERs 0.0 - - - IHSS clinics - - 0.0 0.0 “Municipal� facilities - 83.3 - - Private clinics - - 0.0 14.3 Total 68.5 76.9 75.0 14.3 Source: World Bank (forthcoming). Honduras Public Expenditure and Service Provision Survey 44. Both models have used performance-based incentives to strengthen incentive structures and the “compact� between providers and policymakers. In Guatemala, the entire payment to providers does not depend on performance; however, their renewal does. In addition, providers can receive small incentives upon reaching certain targets. In the alternative service delivery models in Honduras, the contracts are performance-based as the payment to providers depends upon reaching predetermined targets. Contracts or agreements that establish performance-based incentives, like the ones in Honduras, partly overcome the need to fully specify the transactions between providers and patients as they reward providers upon results achieved. By specifying proxies for performance that can be easily monitored and represent a group of good behaviors, these contracts pay against specified targets for good performance and not for the inputs used in providing services or for the transactions involved.36 C. Results 45. The alternative models of service delivery were the strategy used by Guatemala and Honduras to extend the coverage of health services. The Programa de Extensión de Cobertura in 2007 offered services to an estimated 4.3 million people, about 33% of the Guatemalan population. When the program was initiated in 1997 only an estimated 490,000 people were receiving services from the contracted NGOs.37 In Honduras, today there are about 129 health facilities in 11 out of the 18 provinces (departamentos) under alternative management, providing services to about 719,000 people, about 11% of the 36 Idem. 37 La Forgia et al (2005). 26 DO NOTE CITE. FOR INTERNAL DISCUSSION. population.38 Although the alternative providers manage previously existing facilities, many of these facilities were not functioning before. In addition, close to 109,000 people receive services through mobile clinics also managed through results based contracts. 46. Despite these efforts, access to services remains an issue in both countries. A large percentage of the population remains without access to basic health care services. For instance, as seen in Graph 5, in Guatemala about 6 percent of the population remains without access to any health service. Similarly, as seen in Error! Reference source not found. basic services such as pre-natal and post-natal care in Honduras have increased considerably in the last years; nonetheless, by 2005/2006 a little more than 25 percent of pregnant women were still not receiving post-natal care. Graph 5: Guatemala, estimated number of Graph 6: Honduras, trends in percentage of people covered by health services, 2002-2007 women receiving pre and post-natal care, live births in the 5 years previous to the survey 100 87.7 83.9 92 82.6 80 73.5 60 40 33.6 34.3 34 20 0 1991/1992 1996 2001 2005/06 Pre-natal Post-natal Source: MSPAS as appeared in Méndez (2009) Source: Memorias de Estadísticas vitales y Vigilancia epidemiológica. MSPAS. 47. There is evidence in Guatemala indicating that the PEC has increased coverage of basic health services among other positive results. The results of a household and a provider survey implemented in 200139 in Guatemala found that the coverage of health services of women and children in the catchment areas of contracted NGOs was similar to those found in the catchment areas of the traditional model (see Table 12). These surveys also found that users tended to report greater satisfaction with the NGOs model. Finally, analysis from these surveys also showed that the NGOs in general are more productive40 than the traditional providers but also more costly; this is partly because these NGOs have to reach poor remote areas, incurring higher operational costs. Table 12: Immunization Rates and PNC use and services provided, Guatemala PEN and traditional model, 2001 Immunization rates PNC use and services provided Vaccine Traditional Direct PSS Traditional Direct PSS 38 These alternative models are currently operating in these provinces: Copán, Comayagua, El Paraíso, Francisco Morazán, Lempira, Santa Bárbara, Intibucá, La Paz, Choluteca, Valle y Yoro. 39 Danel & La Forgia (2005). 40 Productivity is here defined as the average monthly provider volume of services per health worker providing care. 27 DO NOTE CITE. FOR INTERNAL DISCUSSION. Percentage with at least all vaccines 73 74.2 1 PN 75 78 Average number of No vaccine 5.2 5.7 PNC visits 2.9 2.8 Percentage who DPT3 80.1 80.7 received TT 61.9 55.1 Percentage who Polio 3 81 81.7 received iron 57.7 57.8 Percentage who Measles 81.7 82 received folate 52 52.6 Women with more than 1 PNC visit percentage BCG 93 87.7 with one trimester PNC 32.3 29.1 Source: Danel & La Forgia, 2005 48. In Honduras, the results of a patient exit survey41 implemented in 2007 showed that the perceived quality of health services measured in terms of waiting time, bathroom cleanliness, and access to medicines was higher in the alternatively managed health facilities than in the traditionally managed ones (see Table 4). Some of these results were corroborated by a facility survey implemented as part of the same study. The study found significant differences in the availability of medicines and other supplies between alternative and traditional health facilities. As in Guatemala, this latter survey also found that productivity, measured in terms of services per hour, is also higher in the alternative model but unit costs are also higher. The study concluded that medicines and other supplies are likely increasing the average unit cost for alternative facilities. In addition, these facilities spend more on administrative costs, as well as other categories of expenses that traditional models do not cover as these functions are performed at the central level. The study sampled 10 alternative health facilities, of the 25 of the models existing in 2007, and 10 traditionally managed facilities. The traditional facilities were selected so that they matched the alternative ones in terms of population, geographic location (rural/urban), access, number of personnel and type of services provided. Table 4: Patients perception of waiting times, bathroom cleanliness and availability of medicines Alternative Traditional Satisfied with waiting time 87 66 Clean Bathrooms 50 0 Received medicines needed 91.6 68.1 Source: Garcia Pardo, A. and C. Peña. 2007. 49. A 2009 descriptive study financed by USAID42 in Honduras compared 8 decentralized health facilities with 8 traditional ones in terms of production, quality, cost- efficiency, and cost-effectiveness. As seen in the table below, the study found that alternative or decentralized health facilities provided more services than traditional ones, even though they offered services to similar population sizes. Only in the case of family planning, antenatal care, and in services not included in the prioritized programs did these decentralized facilities have lower levels of production. 41 Garcia Prado, A. and C. Peña. (2008). 42 Secretaría de Salud, MEASURE Evaluation/PRODIM consultores. (2009). 28 DO NOTE CITE. FOR INTERNAL DISCUSSION. Table 14: Annual production of services by type of services and by type of facility management, 2007 Decentralized Centralized Difference Total 70128 59008 11120 Family Planning 3979 5605 -1626 Antenatal care 5894 6864 -970 Deliveries 1318 462 856 Postnatal care 1388 893 495 Growth monitoring 10529 5838 4691 Vaccinations 21545 17654 3891 Acute diarrhea 1272 873 399 ARI 4349 3276 1073 Pneumonia 2190 919 1271 House visits 3391 42 3349 Others 14273 16582 -2309 Source: Secretaría de Salud and Measure Evaluation/PRODIM Consultores. 2008. 50. In contrast to the previous study, the latter study found that unit costs in decentralized health facilities are lower than unit costs in traditional ones43. Although the total cost of alternative facilities is higher (about Lempiras 1.6 million higher or US$ 89 thousands), unit costs are lower as these facilities have much higher production. These different results are partly due to different definitions of costs used in the two studies. This more recent study included all direct and indirect cost of both services and thus included in the traditional facilities the administration cost incurred by the central level. As this study found that unit costs are lower in alternative facilities, these facilities were deemed more cost-effective than traditional ones. The traditional facilities were selected so that they matched the alternative ones in terms of an index of indicators of supply and demand of services. 51. In terms of quality, this study found that compliance with protocols in the treatment of acute diarrhea and pneumonias for children under five was higher in decentralized facilities than in traditional ones. While in alternative facilities these protocols were followed 81.5 percent of the time in the case of pneumonia and 69.5% in the case of diarrhea treatment, in the traditional ones the protocols were followed only 32.5% and 37.8% respectively. Nevertheless, this study found that in general very few facilities, traditional and alternative, comply with the protocol for the use of partograms. 52. Finally, this study also compared health facilities44 by type of managers: NGOs, CBOs, and mancomunidades. The study found that services managed by CBOs had higher production in six of the ten services evaluated, NGOs in three of the ten, and the manconumidades surveyed in only one (see Table ). However, the mancomunidades had the lowest unit costs in all services, while the CBOs had the higher cost in eight of ten services. 43 Idem. 44 This comparison was only made in terms of CESAMOs, health facilities with doctors, as this was the only type of facility common among the decentralized models assessed. 29 DO NOTE CITE. FOR INTERNAL DISCUSSION. Table 15: Production in decentralized CESAMOs by type of services and type of manager CBOs NGOs Mancomunidad Total 13800 9908 12967 Family Planning 850 400 858 Antenatal care 952 472 933 Postnatal care 197 86 157 Growth monitoring 2628 1749 1051 Vaccinations 5246 2413 4557 Acute diarrhea 150 186 161 ARI 859 984 766 Pneumonia 791 77 146 House visits 600 143 476 Others 1527 3398 3862 Source: Secretaría de Salud and Measure Evaluation/PRODIM Consultores. 2008. 53. There is some evidence indicating that the performance based incentives in these alternative models have so far benefited mainly the providers’ managers and not the health workers, possibly limiting the effects of these models. In Honduras, in focus group discussions commissioned for this study45, employees from two alternative models managed by CBOs highlighted as one of the main disadvantages of these models that the incentives received by the provider do not benefit workers, contractual or civil servants, but are maintained by the organization46. 54. In summary, even though the coverage of basic health services seems to have increased and there is some evidence of possible positive effects of these models, they merit and require further evaluation. It is difficult to measure the effect of a heath reform on outcomes, as these effects take time to materialize. To date, there is no evaluation that has measured the effects of these reforms on health outcomes, in absolute numbers and when compared to the traditional model. Additionally, existing evidence comes mainly from descriptive studies that cannot uncover causality. There is therefore still need to implement rigorous evaluations to measure the effects of these health models. This might be difficult in the case of PEC, as this program has been in place for more than 10 years, there is no baseline, and the program is not likely to expand. In Honduras there is an opportunity to evaluate the new decentralized models. These models have been recently developed and are in the process of expansion; thus control groups can be identified47. C. Politics of Adoption and Institutionalization of these models 45 Ospina and Etchegaray (2009). 46 Ideam. 47 The new health project in Honduras proposes to scale up these decentralized models. Team members of the AAA are then working with the project team on these issues. 30 DO NOTE CITE. FOR INTERNAL DISCUSSION. 55. In Guatemala, the ambitious goals of the Peace Agreements in the health sector helped make access to the poor a priority program for the government. This context facilitated the adoption of the PEC as a strategy to meet these goals. Soon after its implementation, the original pilot program was quickly extended given the urgency to respond to the needs of poor rural and largely indigenous population. Although this program has been in place for more than ten years, the favorable environment that existed during its adoption does not exist anymore. Thus, despite having survived several administrations, its progress has been uneven. 56. In Honduras, the decentralized models of service delivery were the result of a broader and relatively continuous process of reform which began in the early 1990s. This reform focused on establishing primary health care centers and enhancing the roles of regional administrative units and local governments. This previous reform process facilitated the establishment of these alternative models that further increased the participation of municipalities and community based organizations in the delivery of health services. 57. The legal framework also facilitated the adoption of PEC in Guatemala. At the time of its adoption, the Ministry of Health and Social Assistance enacted a new legal instrument, convenios, which provided a new type of public-private contract that avoided the cumbersome processes and procedures required under the Public Contract Law48. 58. In Guatemala the adoption and institutionalization of these models were facilitated by the strong leadership of the Ministry of Health. In Guatemala the decision to introduce the PEC was strongly supported by the Office of the President and the Ministry of Public Health and Social Development in part because of the institutional weakness of the ministry as at the time there were difficulties in executing the budget in primary care due to scarcity of resources and particularly due to serious problems in recruiting trained personnel to work in rural areas.49 This strong leadership has not always been present, explaining the different paces of the program progress across almost four administrations. 59. In Honduras, the Ministry of Health has also been one of the main supporters of the alternative models. However, the instability of the Ministry of Health leadership and the susceptibility of its staff to turnover have limited the consistency of its influence and support to the models. 60. Donor support has also been key in the conceptualization and implementation of these alternative models; this has sometimes deterred their institutionalization. The Programa de Extensión de Cobertura (PEC) in Guatemala was originally developed with support from the IADB through its Health Services Improvement Program. Although the financing of NGOs is now fully financed by the Ministry’s budget, the PEC’s support systems (technical and social audits) were not and thus their financing stopped. Finally, in Honduras, the alternative health models have received support from many actors. The 48 Repetto and Zapata, 2005 and La Forgia et al., 2005 as appeared in Payne (2009). 49 La Forgia et al. (2005). 31 DO NOTE CITE. FOR INTERNAL DISCUSSION. first support was received from the Swedish-financed Access Program which was originally managed by PAHO. Then in 2003, with the support of the IDB, pilot programs were implemented for the delivery of basic health services through mobile teams, as in Guatemala. During this same period the alternative models as they are known today also started on a pilot basis with the support of the World Bank’s PRSS project, the Government of Japan, funds from the Italian Government’s debt relief and also from USAID which offers training to provider’s organizations in management aspects.50 61. The PEC was based on a practice on the ground, an aspect that also facilitated its institutionalization. After years of civil war, many NGOs, often with external support, were providing health services in rural areas in Guatemala. This facilitated the rapid creation of the program. Nonetheless, at the time when the PEC was designed, the number of NGOs or the types of services they were providing were unknown to the Government.51 62. In Honduras, the alternative models were not based on practices on the ground. However, the agreements with mancomunidades are in part the result of a long decentralization process that had increased the municipalities’ responsibilities in the social sectors over time. 63. The institutionalization of both health alternative models of service delivery is now almost complete. The Programa de Extesión de Cobertura is integrated with the Ministry of Health and Social Assistance. Today, the financial transfers to NGOs are fully covered by the ministry’s budget. However, this institutionalization has not been complete. Both the technical and social audits were recently developed with donor support. In 2008 when this support finished, the Government did not continue to finance these two audits. 64. In Honduras, the alternative models of service delivery until recently were financed by donor funds alone. However, they are now funded by the Secretary of Health’s budget, which now receives additional funds due to debt relief from the Italian Government. This Government set as a condition for debt relief that these funds be used to finance a package of basic services, and thus these funds now finance the alternative health care models. Finally, although these alternative models are managed by the Secretary of Health, they are managed by an external unit, the UECF or Coverage Extension Management Unit and Financing. Despite this progress, the alternative health service delivery model is still in a nascent stage in Honduras. 65. In summary, despite this institutionalization and many years of implementation, the sustainability of the health models is still not certain. The PEC in Guatemala has been in place for many administrations; however, depending on the administration, its development has been uneven. For instance, the new Guatemalan administration maintained this program, but it has not ensured the financing of the social and technical audits. In addition, in 2008 a new problem emerged for the program in the form of long 50 Payne (2009). 51 Idem. 32 DO NOTE CITE. FOR INTERNAL DISCUSSION. delays in the transfer of resources to NGOs. These delays were due to a change in the budget law enacted in December 2007 which required NGOs’ expenditures to be itemized according to existing budget categories and entered into the Integrated System of Accounting. This cumbersome procedure and the resulting delays could be partly attributed to the desire of the Ministry of Finance to control the outflow of resources and to critical attitudes towards the program by some in the government, including suspicions of misuse of resources by NGOs52. As a result of these delays, numerous communities across the country did not receive health care for many months because the employees of NGOs were on strike since they had not received their salaries. 66. In Honduras, both major political parties have supported the alternative models of service delivery. However, in some instances this has not been enough to counteract unions’ opposition. For instance, in San Pedro Sula a project of universal insurance managed by the municipality was stopped by the opposition of SITRAMEDYS, the main health workers unions. D. Opposition and support to these programs 67. The health alternative models of service provision have faced opposition from professional associations, unions, and other influential actors. In Guatemala, unions have opposed the PEC; however, these unions are relatively weak and are not the main forces opposing contracts with NGOs for the provision of services. The unions consider this program as a step towards the privatization of public health services and also oppose the fact that NGO workers cannot be unionized.53 68. The PEC program, however, has received and continues to receive much opposition inside the Ministry of Health and from other actors in the government. As mentioned before, the PEC provides a limited package of services and is also not well integrated with the rest of the health sector. As a consequence, many members of the current government and other politicians want resources to be shifted away from the PEC to the traditional model or at least to place greater priority on strengthening the traditional health centers. For instance, during the 2007 presidential electoral campaign both the winning political party, Unidad Nacional de la Esperanza, and the party placed second, Partido Patriota, called for a review of the effectiveness of the PEC program arguing that administrative costs were too high and that for the same amount of resources a doctor could be offering services 24 hours.54 However, key barriers to strengthening the traditional model and/or enlarging the package of services offered by PEC, expanding its coverage, and improving its integration with the rest of the system are the dispersion of the beneficiary population and the limited public resources for health and in general the limited revenues collected by the Government; in 2006 these revenues only amounted to 13% of GDP55. 52 Payne (2009). 53 Payne (2009). 54 La Prensa Libre, August 28, 2007 and El Periódico, August 23, 2007 as appears in Payne (2009). 55 World Bank Development Data Platform. 33 DO NOTE CITE. FOR INTERNAL DISCUSSION. 17. In contrast, in Honduras, the unions and professional associations are stronger and represent the main actors opposing these models. The union that has been most vocal against these models is SITRAMEDHYS, the union of non-professional health workers; this union has about 10,000 members who are among the 16,024 members of the Secretary of Health. Unions and professional associations argue that these models imply a privatization of health services and that all health centers should conform to the same hierarchy, under the supervision of the regional director. They also oppose the models due to their more limited job security for employees, who are hired on a contractual basis and can be fired. Finally, unions also fear loss of power as these models expand in number because most of the health workers in these models are not unionized. But, in contrast to the case of PROHECO (Honduran Project for Community Education), the alternative models of service delivery have also important supporting actors56. 69. In Guatemala, the Government has recently helped dissipate some opposition to PEC as this program is central for the success of the Conditional Cash Transfer program “Mi Familia Progresa�. This program transfer resources to families under the condition that children and pregnant woman utilize basic health services. The PEC is thus central for compliance with this conditionality. As a result, the Ministry of Finance recently approved a 40 percent bridge payment to NGOs, temporarily eliminating delays in payments.57 70. In Honduras, the union opposition to the alternative health delivery models is counterbalanced by the strong support from mayors and rural communities’ demand for the models. Some of the decentralized models of health service delivery include results based contracts with mancomunidades. These models therefore involve the devolution of health provision to local governments and thus represent a decentralization of health responsibilities to lower levels of governments. These models have then received strong support from mayors and the Honduras Association of Municipalities (AMHON) who have allied with the Ministry of Health to support their expansion and consolidation. The union’s opposition is also counterbalanced by increasing demand from rural communities who would also like to benefit from these models, having heard about them from beneficiary communities.58 71. In summary, these health alternative models of service delivery constituted the strategy used by Governments to increase access to a basic package of health services in remote rural areas. Limited evidence shows that the coverage of basic health services in the catchment areas of NGOs in Guatemala is similar to that of the catchment areas of traditional facilities. There is also evidence from both countries that alternative providers are more productive and efficient than traditional providers. However, these models also have weaknesses. The alternative models are not well integrated with the rest of the health system. The system to monitor contracts will need 56 Payne (2009). 57 Payne (2009). 58 Idem (2009). 34 DO NOTE CITE. FOR INTERNAL DISCUSSION. to be strengthened. These models have also faced strong opposition but have also some supporters. IV. IMPROVING THE FUNCTIONING OF THESE MODELS 72. Better integrating these programs with the rest of the health system will be essential. The PEC aims at providing a package of preventive and promotion services to communities without access to health services. Although these are cost-effective services that respond to the main burden of disease in rural areas, to improve maternal health and reduce Guatemala’s high maternal mortality rate the integration of the mobile teams to the rest of the health network becomes vital. This integration is complex and costly as it requires not only a better referral and logistical system but also an increased supply of health facilities that offer emergency obstetric care. Despite budget constraints, the MSPAS is redesigning its health network to ensure a working referral and counter- referral system where mobile teams would constitute the first step in the system. In addition, with the support of the World Bank, the MSPAS is transforming health centers in municipal capitals in facilities that offer integral maternal and health services called CAIMI (Centro de Atención Integral Materno Infantil). This is not exactly the case in Honduras as most of the alternative providers manage a small network of health facilities, many of them public facilities that were previously closed for lack of personnel. However, to ensure a continuity of care and thus access to secondary and tertiary care these alternative models should also be better integrated to the rest of the system. 73. There are several experiences with results-based or performance-based financing of services in developing countries; these experiences are relatively recent but evidence on its potential benefits is increasing. The incentive payments in Guatemala and the results-based financing scheme in Honduras are part of a group of recent similar experiences in countries as diverse as Argentina, Panama, Afghanistan, and Rwanda. The evidence on the benefits from these schemes is still limited but increasing. A recent impact evaluation of the Rwanda experience shows a significant increase in institutional deliveries, in the utilization of preventive child care services, and in the quality of prenatal care measured by a quality score59. In Argentina, the Plan Nacer maternal and child insurance program is a results based scheme between the federal and provincial governments. An impact evaluation of this program is still on-going. However, preliminary results of this impact evaluation, based on administrative data from Misiones Province, indicate that this program has had a significant effect on children’s height and thus on their health and nutrition60. 74. Monitoring performance and verifying information: When payments are based on results, there is a high risk that providers will over-report results to receive incentive payments. Under these circumstances, monitoring performance and validating the information providers submit to health authorities becomes critical. In neither of the two countries under study is there a system that systematically monitors performance and 59 Basinga et al. (2010). 60 Presentation by Paul Gertler from University of California Berkeley, the Federal Ministry of Health of Argentina, and the Presidency on preliminary results of this impact evaluation. 35 DO NOTE CITE. FOR INTERNAL DISCUSSION. validates the information provided. In Guatemala, the technical teams that existed until 2008, used to do this verification. In the next months, though, the MSPAS will contract an external firm to verify the information rendered by some of the alternative providers. 75. Recognizing the risk of misreporting, many of the international experiences with results-based financing have included an independent mechanism to verify information and monitor providers. These mechanisms are often different from the normal supervision carried out by the Ministries of Health. In Argentina, the Maternal and Child Insurance program (Plan Nacer) contracts an independent audit firm that validates the beneficiary register or roster and validates the information on target indicators61. An external audit firm with similar responsibilities will be contracted by the Ministry of Health in Panama to monitor results-based agreements with NGOs and with teams organized by the regional offices of the Ministry of Health 62. In Afghanistan, an independent firm carries out four annual assessments of the performance of health facilities and three semiannual assessments in a few regions. These annual assessments are based on a survey of a random sample of facilities63. In Rwanda in an earlier results- based financing pilot program the verification of the achievement of service coverage and quality targets was validated by community organizations. Today data is entered via internet and data validation is done at the district level by specially trained data agents who performed random audits64. 76. The nominalization of beneficiaries facilitates the supervision of providers and the verification of the information they render. If the results reports are nominalized and include the name, last name, identification of beneficiary, services provided, site, and location of clinical history, the supervision and audit of information rendered by providers would become relatively simple. Thus, the importance of not only generating a beneficiary roster but also of nominalizing the reports on results. To ensure that there is no misreporting of beneficiaries, the roster of beneficiaries should also be audited by an external firm. The beneficiaries of the Plan Nacer in Argentina are nominalized. The roster of beneficiaries is audited by an external firm and the health results are also nominalized and audited. 77. Both the size of the incentives and their recipients are important to achieve results. Limited international evidence shows that too much of a financial risk or too little of it might not generate the intended results.65 One of the reasons given for the failure of a performance-based pilot in Uganda was the small size of the benefit (Lundberg et al in Eichler and Levine, 2009). The impact evaluation of the Rwanda results-based incentive scheme, did not find any impact of the scheme in prenatal care service utilization where the incentive was the lowest (US$ 0.09); in contrast, institutionalized deliveries had the highest incentive to the provider (US$ 4.59), resulting 61 República de Argentina. 2008. 62 República de Panamá. 2009. 63 Sondorp et al. (2009). 64 Rusa, Schneidman, Fritsche, and Musango (2009). 65 Eichler et al. (2009). 36 DO NOTE CITE. FOR INTERNAL DISCUSSION. in a significant increase in the probability of these deliveries66. In Guatemala the incentives received by personnel for good performance are small and do not seem to be providing the wanted incentive (see Table 3). 78. It is also important for results-based agreements to ensure that the frontline provider also receives incentives to perform. This is not the case in Honduras as discussed before. If incentives do not reach frontline providers the benefits of these models might be limited and might increase opposition against these decentralized models. For instance, in a hospital reform in Costa Rica, it was not possible to distribute financial incentives to individuals due to union resistance. In consequence, performance rewards retained by management did not change staff motivation (Garcia Pardo and Chawla, 2006). Anecdotal evidence from some facilities in Afghanistan also showed that when bonus payments to facilities stayed with management, the personal motivation of health workers did not increase as much67. 79. Evaluating these programs. Both service delivery models under study created alternative mechanisms to manage and finance providers. There is thus an urgent need to evaluate them and learn from what is working and what is not. The program in Guatemala has been in place for many years; the program in Honduras is relatively recent. Despite being in operation for a while, there have been very few evaluations of these programs. At the moment it would be very difficult to do an experimental or quasi- experimental evaluation in Guatemala. An evaluation was performed in 2002 when the program was relatively recent. This evaluation was based on household surveys that compared health indicators in the catchment areas of the contracted NGOs and health facilities68. Since then, we are not aware of any other quantitative evaluation. In Honduras, as these programs are likely to be extended, there is an opportunity to rigorously evaluate them. Finally, as the incentive structures these models created are new; piloting and rigorously evaluating small changes in the size and type of incentives as well as their recipients will also be needed. 80. Aspects of these programs could be replicated in the traditional service delivery models. Contracts or agreements with public providers and particularly performance based agreements are possible and could produce the same results. There are examples of performance-based agreements with lower levels of Government or decentralized offices of the Ministries of Health, as with MOH and mancomunidades in Honduras, between the Ministry of Health and provinces in Argentina for subsidized maternal and child insurance, between the Ministry of Health and administrative districts as in Rwanda, or between the center and decentralized offices of the Ministry of Health as in Panama. Some of these programs have already started to produce encouraging results. 81. In Argentina, the Plan Nacer is a Federal Government Program that subsidizes a mother and child insurance scheme. The Federal Government enters into results based agreements with provincial governments and finances a per capita payment based on the 66 Basinga et al (2010). 67 Sondorp, Palmer, Strong, and Wali. (2009) 68 Danel et al. (2005). 37 DO NOTE CITE. FOR INTERNAL DISCUSSION. number of women included in the insurance scheme at the provincial level. The disbursement of part of this payment to the provinces depends on whether the province achieved pre-determined targets of indicators (up to 10 indicators) for the coverage and quality of maternal and child services. The verification of the information provided by the provinces as well as the health providers contracted by them to provide services is done by an external audit firm in a concurrent basis.69 82. In Rwanda, the Ministry of Health enters into performance based agreements with the administrative districts, the districts committees enters into performance based agreements with health facilities’ management and with the health facilities’ personnel. These agreements are entered with both public health facilities and private confessional facilities. The facilities are then paid for the quantity of services provided as there is a standard fee for 14 basic services. Each basic fee is adjusted by an index of quality.70 83. Following these encouraging examples, the Guatemala government, with the support of the World Bank, will organize in the coming months performance based agreements between the central MSPAS and the Departmental Health Directorates. The objective of these agreements is the provision of services through institutionalized mobile teams linked to fixed health facilities. 69 República de Argentina. (2008). Plan Nacer: Proyecto de Inversión en Salud Materno Infantil Provincial. Manual Operativo. 70 Rusa, Schneidman, Fritsche, and Musango (2009). 38 DO NOTE CITE. FOR INTERNAL DISCUSSION. VI. REFERENCES Basinga, P., Gertler, P., Binagwaho, A., Soucat, A., Sturdy, J., and C. Vermeesch. “Paying Primary Health Care Centers for Performance in Rwanda�. In World Bank Policy Research Working Paper. N 5190. The World Bank, January 2010. Brinkerhoff. 2004. “Accountability and Health Systems: toward conceptual clarity and policy relevance�. Health Policy and Planning:19.6: 371-379. Cuellar-Marchelli, Helga. 2008. “La Estrategia de El Salvador y sus desafíos.� FUSADES. San Salvador, El Salvador. Daniel, I and G. La Forgia. 2005. “Contracting for Basic Health Care in Rural Guatemala Comparison of the Performance of Three Delivery Models�, in La Forgia. Health System Innovations in Central America: Lessons and Impact of New Approaches. The World Bank. De Fuentes, Elda Zulema. 2009. “EDUCO: Una Historia de Éxitos y resultados.� San Salvador, El Salvador. Di Gropello, Emanuela. 2006. “A Comparative Analysis of School-based Management in Central America. World Bank Working Paper Series No. 72. Demographic and Health Surveys. 2005/2006. “Honduras.� Eichler, R., Auxia, P., U. Antoine, B. Desmangles. 2007. “Performance-Based Incentives for Health : Six Years of Results from Supply-Side Programs in Haiti.� Center for Global Development Working Paper # 121. Eichler, R., Levine R. and the Performance Based Incentives Working Group. 2009. Performance-Based Incentives for Global Health: Potential and Pitfalls. Center for Global Development. Washington, DC. ESA Consultores. 2009. “Rendición de Cuentas y los Servicios Sociales en Centroamérica. El Caso de Honduras: Salud.� Tegucigalpa, Honduras. Garcia Prado, A. and C. Peña. 2008. “Assessing the Purchasing and Provision of Basic Health Services in Honduras: A Comparison of Traditional and Alternative Service Delivery Models.� Getler, P., Patrinos, H., and M. Rubio-Codina. 2007. “Impact Evaluation for School- based Management Reform�. Doing Impact Evaluation No. 10. World Bank. Jimenez, Emmanuel and Yasuyuki Sawada. 1999. “Do Community-Managed Schools Work? An Evaluation of El Salvador’s EDUCO Program.� The World bank Economic Review. Vol 13. No. 3: 415-441. Washington, D.C. Jimenez, Emmanual and Yasuyki Sawada. 2008. “Does Community Management help keep Children in Schools? Evidence Using Panel Data from El Salvador’s EDUCO Program.� The World Bank. Washington, D.C. La Forgia, G., P. Mintz, and C. Cerezo. 2007. “Is the Perfect the Enemy of the Good? A Case study on Large-Scale contracting in Basic Health Services in Guatemala� in 39 DO NOTE CITE. FOR INTERNAL DISCUSSION. La Forgia. Health System Innovations in Central America: Lessons and Impact of New Approaches. The World Bank. Médez Pérez, Lucrecia. 2009. Análisis Institucional del Programa de Extensión de Cobertura. Monteith, R. Stupp, P. and Mc Cracken, S., 2005. Reproductive, Maternal, and Child health in Central America. Trends and Challenges facing women and children. Division of Reproductive Health. Center for Disease Control and Prevention. USAID. August 2005. Opsina, Rosa Inés and Ana Etchegaray. 2008. “Análisis cualitativo sobre la gobernabilidad en el sector de salud en Guatemala.� Payne, Mark. 2009. “Political Economy of Reform Innovations in Health and Education in El Salvador, Honduras, and Guatemala.� Public Expenditure Review: Honduras. 2007. The World Bank Group. Washington, D.C. Secretaría de Salud and Measure Evaluation/PRODIM Consultores. 2009 Sondorp, E., N. Palmer, L. Strong, and A. Wali. 2009. “Paying NGOs for Performance in a Postconflict Setting� in Eichler, R., Levine R. and the Performance Based Incentives Working Group. 2009. Performance-Based Incentives for Global Health: Potential and Pitfalls. Center for Global Development. Washington, DC. UNICEF, WHO, UNFPA, and the World Bank. 2007. Maternal Mortality in 2005: Estimates of UNICEF, WHO, UNFPA, and the World Bank. World Health Organization, Geneva. World Bank. 2003 “Making Services Work for Poor People�. World Development Report 2004. A Co-publication of the World Bank and Oxford University Press. Washington, D.C. World Bank. 2005. “Key Issues in Central America Health Reforms: Diagnosis and Strategic Implications�. Volumes I and II. Washington, D.C. World Bank. 2008.“Transparency and Accountability in the Education Sector: Honduras Institutional Governance Review. World Bank. Washington, D.C. World Bank. 2009. Project Appraisal Document on a proposed loan for US$ 122 million to the Government of Guatemala: Expanding Opportunities for Vulnerable Groups Project 40 DO NOTE CITE. FOR INTERNAL DISCUSSION. Annex I: Analytical Framework 1. Since the mid-1990s, Central American countries have made innovative reforms in service delivery. These reforms generated alternative service delivery models that strengthened incentive structures and accountability, the obligation of providers and policymakers to take responsibility for their decisions and actions 71. In the health sector, the alternative models of service delivery aimed to improve the accountability relationship between providers and policymakers through contractual agreements and performance-based incentives. These reforms tried to ensure that the incentives faced by providers were aligned with the delivery of quality services. 2. To understand these alternative service delivery models, this study follows the World Development Report (WDR) 2004 framework on service delivery 72. This framework explains service performance through the following three accountability relationships: “voice� between citizens/clients and politicians/policymakers, “compact� between policymakers and providers, and “client power� between providers and clients. If any of these relationships is not working, the services provided will not meet the needs or expectations of patients and the public in general. Thus to improve service delivery community members have two different routes; a “long route� by exercising pressure to their elected officials for them to ensure that providers offer quality services, and a “short route� by increasing their power over providers. 3. Contracting arrangements with public and private health providers strengthen the compact between policymakers and providers. If citizens through their votes can pressure policymakers to improve services, services might not necessarily work if policymakers cannot make providers accountable for the provision of quality services73. Contracts or agreements between policymakers and providers can clearly establish the rights and responsibilities of each party to the contract and can set rewards or sanctions in case of non-compliance. Through these mechanisms each party could potentially be held accountable for its actions. If services are not provided or are provided with low quality, the contracts can specify sanctions including their non renewal. 4. In the health sector where most transactions between patients and health providers are difficult to monitor, contracts for services can be very complex and costly to supervise. Only for services that have been standardized, such as most preventive services (e.i. immunization, vitamin supplementation, pre-natal control, etc.), can these contracts be more clearly and fully specified74. This is more difficult in the case of curative care. In Central America, governments have experimented with contracts for the provision of primary care, mainly preventive and promotion services but also with basic curative care. Figure 1: Accountability Relationships 71 Brinkerhoff. 2004. “Accountability and Health Systems: Toward conceptual clarity and policy Relevance.� Health Policy and Planning. Washington, D.C.19.6:371-379. 72 World Bank. 2003. World Development Report 2004: Making Services Work for Poor People. 73 Idem. 74 Idem. 41 DO NOTE CITE. FOR INTERNAL DISCUSSION. Politicians & Policymakers •Budget planning & execution (e.g. resource allocation) •Civil service (e.g. HR Co Management) •Voting •Intergovt institutional cs mp •Taxes arrangements (e.g. fiscal liti •Direct requests ac relations, decentr.) •Parliamentary /Po •Procurement t/ representation •Regulatory system (e.g. Po ice •Etc… laws, norms, standards) •Formal controls (e.g. lic Vo Performance audit ies •Etc… Service Citizens Client power Providers •Social accountability mechanisms •Citizen scorecards •Vouchers •Etc… Source: World Bank (2003). 5. Performance-based incentives strengthen incentive structures and the compact between providers and policy-makers. These incentives are monetary payments that are provided on the condition that predetermined performance targets are met or that some indicators for performance change75. Contracts or agreements that establish performance-based incentives partly overcome the need to fully specify the transactions between providers and patients as they reward providers upon results achieved. By specifying proxies for performance that can be easily monitored and represent a group of good behaviors, these contracts pay against specified targets for good performance and not for the inputs used in providing services or for the transactions involved76. 6. In the alternative service delivery models in Honduras, the contracts are performance- based as the payment to providers depends upon reaching predetermined targets. In Guatemala, the entire payment to providers does not depend on performance; however, their renewal does. In addition, providers can receive small incentives upon reaching certain targets. 75 Eicheler, Levine, and the Performance-Based Incentives Working Group (2009). 76 Idem. 42 DO NOTE CITE. FOR INTERNAL DISCUSSION. ANNEX 2: Guatemala Basic Health Services Services Components Integral care to women Prenatal care and newborn: Clean and safe delivery in the community Puerperium care Care in case of complications during pregnancy, delivery and puerperium Newborn care Care to women during fertile years Reference and counter-reference Child integral care General care Micronutrient supplementation Growth monitoring Immunizations Control to childhood prevalent diseases Care requested for Care demanded morbidity and urgencies Control of vector transmitted diseases Zoonosis control Environmental care Environmental care promotion Promotion and education Health promotion and education. for self-health care Source: Programa de Extensión de Cobertura/MSPAS. 43 DO NOTE CITE. FOR INTERNAL DISCUSSION. ANNEX 3: Honduras package of services provided in primary health care facilities Línea de Servicios ofertadas de acuerdo al tipo de Unidad de Salud ACTIVIDADES DE PROMOCION CESAR CESAMO CMI Información, Educación y comunicación individual, familiar y    comunitaria Organización y fortalecimiento de grupos de auto apoyo   X (diabéticos, hipertensos, depresivos, embarazadas etc.) comités de salud y voluntarios de salud Elaboración y actualización de mapas de riesgos basados en   X los censos, encuestas y otros medios Operativos de limpieza y destrucción de criaderos   X Ferias de la Salud   X Promoción del manejo adecuado de alimentos y desechos   X Promoción de los servicios de salud   X Promoción de municipios, comunidades y viviendas   X saludables Promocionar la salud mental a través de la participación   X activa, toma de responsabilidades y solidaridad ACTIVIDADES DE PROMOCION CESAR CESAMO CMI Consejería y oferta de métodos de Planificación Familiar    Vacunación de acuerdo a normas de la SS    Detección del cáncer de cérvix y mama   X Detección del cáncer de próstata X  X Atención integral al menor de 2 años en la comunidad (AIN-   X c) Detección de sintomáticos   X Vacunación canina y control de foco rábico   X Operativos de limpieza y destrucción de criaderos   X Visitas domiciliar a familias de riesgo   X Aplicación de flúor y sellantes a preescolares y escolares   X Apoyar las actividades de Escuela Saludable   X ACTIVIDADES ASISTENCIALES (CURACIÓN) CESAR CESAMO CMI Control prenatal y puerperal de acuerdo a normas de la SS    Atención de parto normal y emergencias obstétricas y en X X  menores de 5 años Atención Integral de las Enfermedades Prevalentes de la   X Infancia (AIEPI) Atención de la morbilidad infantil no incluida en el AIEPI   X Atención y/o estabilización de emergencias    Atención de la morbilidad gral. en escolares, adolescentes,   X adulto y adulto mayor Atención de problemas de Salud mental de acuerdo a X  X normas Diagnostico, tratamiento y seguimiento de pacientes con  X enfermedades transmisibles y transmitidas x vectores Diagnostico y Tratamiento odontológico en escolares y X  X embarazadas Atención de Urgencias dentales X  X Atención del Paciente convulsivo X  X 44 DO NOTE CITE. FOR INTERNAL DISCUSSION. ACTIVIDADES DE VIGILANCIA CESAR CESAMO CMI Notificación e Investigación de la Mortalidad Materna e    infantil. Mantenimiento de la red fría    Vigilancia de las enfermedades de notificación obligatoria.   X Notificación e Investigación de Enfermedades   X Inmunoprevenibles por vacunación (enfermedad febril eruptiva) Vigilancia de Agua para consumo humano.   X Detección y notificación de enfermedades transmitidas por   X Vectores (Chagas, Malaria, Leptospirosis, Leishmaniasis , dengue clásico y hemorrágico, rabia canina etc) Vigilancia de la Diarrea en mayores de 15 años (Hisopado   X rectal). Notificación de casos de VIH/Sida/ TB.    Vigilancia de casos o eventos que pueden constituirse en un    problema para la salud que no se vigilan en el telegrama epidemiológico. SERVICIOS DE APOYO CESAR CESAMO CMI Exámenes de laboratorio: Prueba de embarazo, VDRL,   hematozoario, orina, heces, hematológico, baciloscopia, Gram, prueba de dengue, citologia vaginal, urocultivos. Traslado de emergencias   45