Report No: AUS0000853 External Assessment of Quality of Care in the Health Sector in Colombia World Bank & IFC Final July 17, 2019 1|Page © 2019 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved This work is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because the World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: “World Bank. 2019. External Assessment of Quality of Care in the Health Sector in Colombia. © World Bank.� All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. 2|Page Table of Contents Abbreviations ................................................................................................................................................ 5 Executive Summary....................................................................................................................................... 6 1 Introduction ............................................................................................................................................. 19 1.1 Context and Positioning for Quality Assessment in the Health Sector ............................................. 19 1.2 Methods ............................................................................................................................................ 21 2 Summary Findings .................................................................................................................................... 25 2.1 Quality Results .................................................................................................................................. 25 2.3 Quality Definition, Guidelines and Standards ................................................................................... 35 2.4 Quality Measurement & Reporting................................................................................................... 37 2.5 Quality Improvement Capacity & Activities ...................................................................................... 40 2.6 Inputs – Staff, Facilities, Equipment, Supplies .................................................................................. 42 2.7 Accountability & Governance ........................................................................................................... 45 2.8 Policies & Incentives to Support Quality of Care .............................................................................. 52 2.9 Focus on Hospital Quality Assessment ............................................................................................. 54 3 Recommendations ................................................................................................................................... 59 3.1. Strengthen capacity for quality improvement in the health sector ................................................ 60 3.2. Strengthen the quality ecosystem in the health sector ................................................................... 61 3.3 Improve the rigour and use of information on quality of care ......................................................... 62 3.4. Develop and implement a primary health care-focused integrated care model to support the management of individuals with complex, chronic health care needs .................................................. 64 3.5. Improve accountability & contracting mechanisms to incent for quality in the health sector ....... 65 Acknowledgements..................................................................................................................................... 70 Appendix 1: Interviews, Site Visits and EPS Survey .................................................................................... 71 A1.1 List of Quality Standards ................................................................................................................. 71 A1.2 Key Informant Interviews ............................................................................................................... 80 A1.3 Site Visits Conducted ...................................................................................................................... 83 A1.4 Documents Reviewed for Quality Assessment Tool ....................................................................... 85 A1.5 Databases Reviewed for Quality Assessment Tool ......................................................................... 90 A1.6 EPS Survey....................................................................................................................................... 92 Appendix 2: Detailed Analysis of Accountability Mechanisms for Quality of Care in Colombia ............... 96 Appendix 3: IFC Survey of Quality Assurance Standards in Selected Colombian Hospitals .................... 100 A3.1 Introduction .................................................................................................................................. 100 3|Page A3.2 Survey Approach & Methodology ................................................................................................ 101 A3.3 Survey Results ............................................................................................................................... 103 A3.4 Conclusions & Recommendations ................................................................................................ 109 A3.5 Survey Questionnaire ................................................................................................................... 110 References ................................................................................................................................................ 112 4|Page Abbreviations Abbreviation Spanish English CPGs Guías de Práctica Clínica Clinical Practice Guidelines Territorial Entities (municipalities, health DMDS Dirección Municipal y Distrital de Salud districts) Health Promotion Entity (Health Insurance EPS Entidad Promotora de Salud Enterprise) IETS Instituto de Evaluación Tecnológica en Salud Institute for Health Technology Assessment IFC Corporación Financiera Internacional International Finance Corporation INCAS Informe Nacional de Calidad en la Atención en Salud National Report on Health Care Quality IPS Institución Prestadora de Servicios de Salud Health Provider Institution LMIC Países con ingresos bajos y medios Low- and Middle-Income Country MPS Ministerio de Protección Social Ministry of Social Protection MSPS Ministerio de Salud y Protección Social Ministry of Health and Social Protection NCD Enfermedades No Trasmisibles Non-communicable Diseases NQPS Política y Estrategia de Calidad Nacional National Quality Policy and Strategy Advisory Office of Planning and Sectoral OAPES Oficina Asesora de Planeación y Estudios Sectoriales Studies Organización para la Cooperación y el Desarrollo Organization for Economic Cooperation and OECD Económico Development Programa de Auditoria para el Mejoramiento de la PAMEC Audit Program for Quality Improvement Calidad PDSP Plan Decenal de Salud Pública 10-Year Public Health Plan PIC Plan de Intervenciones Colectivas Collective Intervention Plan PND Plan Nacional de Desarrollo National Development Plan PNMCS Plan Nacional de Mejoramiento de la Calidad en Salud National Health Quality Improvement Plan PTD Plan Territorial de Salud Territorial Health Plan QA Garantía de Calidad Quality Assurance QI Mejora de Calidad Quality Improvement RC Régimen Contributivo Contributory Regime REPS Registro Especial de Prestadores de Servicios de Salud Special Registry of Health Care Providers ReTHUS Registro Único Nacional del Talento Humano en Salud National Registry of Health Human Resources RIPS Registro Individual de Prestación de Servicios de Salud Register of Individual Health Care Services RS Régimen Subsidiado Subsidized Regime SGSSS Sistema General de Seguridad Social en Salud General System of Social Security in Health Sistema Integrado de Información de la Protección SISPRO Integrated Health Information System Social SOGC Sistema Obligatorio de Garantía de Calidad Mandatory System of Quality Assurance WHO Organización Mundial de la Salud World Health Organization 5|Page Executive Summary This report presents findings from an assessment of quality of care in the health sector in Colombia, using a novel methodology developed by the World Bank Group to assess government oversight, promotion and stewardship of quality of care in the health sector This assessment of quality of care in the health sector in Colombia examines how well governments strategize, plan and measure quality; set standards of care; build capacity for quality improvement in the sector; ensure adequate resources are available and well distributed to support quality results; hold organizations accountable for quality results; apply quality policies consistently; and achieve superior quality of care results. This methodology evaluates these eight domains using 49 criteria based on 171 standards. The assessment is based on quantitative analytics, key informant interviews, over 30 visits of public and private healthcare facilities, a review of key policy documents and a survey of a sample of health insurance companies. The assessment is a joint initiative from the World Bank and the International Finance Corporation (IFC) and was undertaken for the first time in Colombia. It took place from January to June 2019. Outstanding improvements in healthcare coverage have been realized since 1993, yet quality of care remains a challenge and the key to the sustainability of the health sector in Colombia Colombia has made important progress in promoting a quality agenda across its health care system, in addition to a successful expansion of Universal Health Coverage. Clear prioritisation of quality and outcomes dates back to 2006 with the creation of the mandatory system for guarantee of quality of care (Sistema Obligatorio de Garantia de la Calidad en Salud). Since then, the country has developed a wealth of strategies and policies aimed at embedding quality governance throughout the health sector, including quality standards for infrastructure and for clinical practice, provider licensing, inspection and accreditation, and targets for improvement in health outcomes. Colombia now has a detailed list of quality indicators as well as a formidable repository of data in its Integrated Health Information System warehouse (SISPRO), with many indicators made publicly available and disaggregated by health insurance company and healthcare provider. Named individuals with official responsibility for quality are in place throughout the health system, working alongside quality committees in most hospitals. As a result of these efforts, there have been notable improvements in health outcomes. Healthcare acquired infections have become less frequent, there have been improvements in patient satisfaction, and waiting times are now comparable to Organization for Economic Cooperation and Development (OECD) peers -- even if they remain an issue of concern in Colombia. Still, quality of care is largely seen across the sector as the change agenda required to turn significant investments in the health sector into a more effective, financially sustainable system meeting the demands of patients and citizens. The National Development Plan for the period 2018-2022, approved by the Colombian congress in May 2019 emphasizes the importance of quality of care as the core strategy to improve the financial sustainability of the health sector. 6|Page Yet, significant concerns persist about improvements in quality of care not meeting investments and expectations in the sector. A landmark study from The Lancet Global Health Commission on High Quality Health Systems (2018) estimates that over 22,000 Colombians die each year due to poor quality of care, with around another 12,000 dying due to poor access to, or utilization of, services. Although Colombia now provides coverage of healthcare services for a greater share of the population than many other OECD countries (94% to 96% since 2010), health outcomes and quality of care continue to be worse than most other OECD countries. Colombia’s maternal mortality rate is higher than all other OECD countries and some 25% higher than that of Mexico, which had the next highest rate. Colombia’s years of life lost due to pneumonia (a relatively easily treatable clinical condition) also exceed almost all other OECD countries, except for Lithuania and Mexico. For several key indicators, results are worse in rural areas, public hospitals and regions with lower income. The proportion of pregnant women with four antenatal visits was 63% in rural areas compared to 73% in urban areas in 2018. Screening mammography rates for women aged 50 to 69 years are lowest in regional departments with lower income, as low as 1.3% compared to a national average of 10.1%. In- hospital mortality rates per 1,000 admitted patients within 72 hours of admission were higher in rural compared to urban facilities (6.22 vs. 1.35 in 2014). The average waiting time for an appointment with a general physician was slightly higher in public compared to private sector facilities (2.9 vs 2.6 days in 2014). This study also found large rural-urban differences in specialist wait times, such as 38.7 vs 7.4 days for a pediatric consultation and 20.1 vs 14.1 days for general surgery (2016). A more ambitious and consistent approach to measuring, supporting and improving quality at hospital- and clinic- level will be key to unlocking the next wave of quality gains in Colombia’s health care system and ensure its financial sustainability The wealth of policies and institutions at system-level - described throughout this report - shows that there is no shortage of top-down efforts to drive quality gains. This is not always matched, however, by focused and effective bottom-up activity. In Colombia’s highly decentralised health care system, greater thought now needs to be given on how to support hospitals, primary care clinics and other health care providers to continuously improve quality of care and health outcomes. This report offers a balanced set of ten recommendations grouped under five themes on how to do that, whilst strengthening drivers of change at system-level to ensure effective, safe and patient-centred health care for all Colombians. Recommendations were mapped against current priorities of the government of Colombia to improve quality of care and discussed extensively with the government of Colombia during the completion of this assessment. 7|Page Key findings from the quality assessment of the health sector in Colombia (see summary in table S3) • There is ample room to improve quality of care results in the health sector, with large variations in quality found between the public and the private sector, by geography, urban and rural settings, and worst quality affecting the most vulnerable populations. As measured, satisfaction rates are high and waiting times are low compared to OECD countries, however, measures of care effectiveness suggest weaknesses in care processes for Non-Communicable Diseases (NCDs), efficiency indicators suggest over-use of available resources and low availability of integrated care indicators show further weaknesses in quality of care. • The national quality strategy needs to be more explicit about targets, plans, activities, roles and responsibilities, and accountability for results. There are few mechanisms to ensure proper alignment and cascading between national quality strategy and strategies of health insurance companies, department level health authorities and healthcare providers. • The rich data sources in the government’s integrated information system for the sector are rarely used for quality improvement activities. Information is not fed back to front-line providers who need it the most. Some data collected are not easily accessible and information that is published is mostly out-of-date. Data are released publicly with little context on how to interpret the results or what actions to take for improvement. • Capacity for continuous quality improvement is unevenly distributed in the sector yet there is a tremendous amount of energy and enthusiasm for quality improvement in the sector. There are wide variations in the number of healthcare professionals trained in quality between different healthcare providers and health insurance companies. There is a lack of mechanisms for shared learning between peers, and limited information on benchmarks for quality. The accreditation program is rigorous but has very low enrolment. Use of decision- support tools to guide clinical decision making is limited and should be expanded. • Mechanisms to verify clinical skills of providers are weak. There is no core standardized curriculum for medical graduates across the country. There are important disparities in human resource distribution, especially in rural and remote parts of the country. • The current model of care is fragmented. This is an important barrier to quality care for a country where most of the burden of disease comes from non-communicable diseases requiring care integration and coordination to be effective. There is poor communication between primary care and specialist services. Patients seek care from multiple providers (e.g. primary care, laboratories, specialty care) in different sites, due to the nature of contracting for different services by health insurance companies. This is an impediment to delivering quality care for an increasing number of patients with multiple chronic conditions. • Accountability mechanisms need to be strengthened and accountability for care integration is weak and of concern. Although there are examples of incentives for quality in contracts between few health insurance companies and healthcare providers, this mechanism is generally weak. There are only a few examples of financial accountability for quality. Consumers in theory hold health insurance companies accountable for quality, but lack the quality, timely information to do so. 8|Page Recommendations To improve quality of care in the health sector, action could be taken by government, health insurance companies, local health authorities and care providers to: (1) strengthen capacity for continuous quality improvement in the health sector; (2) strengthen the quality ecosystem in the sector; (3) improve the rigor and use of information on quality to stimulate patient choice and peer learning for healthcare providers; (4) develop and implement an integrated care model to support the management of individuals with complex, chronic health care needs; and (5) improve accountability and contracting mechanisms to incent for quality. All five themes and related 10 recommendations are summarized in Table S1 at the end of this section, which also presents for each recommendation a preliminary assessment of resources requirements, difficulty of execution and potential impact, each rated on a scale of + to +++. We also indicate whether the expected time horizon for achievement of the impact is short-, mid-, or long-term. Finally, we point to the quality dimensions (such as patient safety, care effectiveness or care integration) each recommendation contributes to. These scores are intended to encourage the Ministry of Health and Social Protection of the government of Colombia to consider implementation, risk management and cost effectiveness considerations as they plan for implementation. It is important to note that this report proposes a cohesive package of interventions which are all necessary and important to achieve sustainable change in quality of care in the sector. If these recommendations should be carefully sequenced and planned by the government, the implementation of all ten recommendations over time will be necessary to achieve impact at scale and make a commitment to quality a cornerstone of the financial sustainability of the sector. It is also important to acknowledge that all actors in the sector have an implement role to play in implementing the recommendations proposed for consideration: table S2 makes the point that not only the government, but also departmental health authorities; health insurance companies; healthcare providers; and patients and citizens all have to be fully engaged and lead the quality revolution that is required to ensure that a high quality health system in Colombia delivers quality results for all Colombians. 1. Strengthen capacity for quality improvement in the health sector An initial and critical step is to invest in training healthcare professionals to augment their capacity for continuous Quality Improvement (QI) at different levels of the system: from the frontline of healthcare services to departmental health authorities to health insurance companies to government level. Launching a national quality campaigns for cancer or diabetes would mobilize the sector in the short term and generate support and engagement from healthcare providers and possibly patients if properly engaged. Quality improvement collaboratives and communities of practice could also play an important role to promote shared learning among healthcare providers, territorial health authorities and health insurance companies and would be transformative. It would also be important to develop and implement a suite of clinical decision support tools, which help healthcare providers comply with clinical best practices. 9|Page 2. Strengthen the quality ecosystem in the health sector The government could consider updating and expanding the scope of the national system of mandatory guarantee for quality of care to strengthen system capacity for quality improvement and enhance the quality culture in the sector. This includes reforming the accreditation process to implement it at scale using a more stepwise approach to accreditation, but also augmenting the current regulatory system with initiatives to engage and empower patients and citizens so that they participate in planning, design and accountability of health care services and demand better quality healthcare. Another priority would be to invest in improving the clinical competence of healthcare professionals before and after graduation, for example by developing a core curriculum for medical doctors and nurses to improve their level of clinical competence. A strategy for reducing regional disparities in supply of health professionals is needed, with an emphasis on expanding health professional education in rural areas, for example through rural education programs, at both the undergraduate and postgraduate (i.e. residency training) levels. 3. Improve the rigor and use of information on quality of care Colombia has a wealth of information on quality of care, but performance indicators are often not timely, not tailored to the needs of specific information users and of uneven reliability. There is limited guidance on how to use information and suggested targets for improvement. As a result, providers are not using information optimally for improvement, and patients are not using it routinely to choose their healthcare insurer or provider. Patient choice of healthcare provider and insurer could be enhanced by providing patients with localized, pertinent, accessible information on quality of care they encounter (e.g. provider star rating system, rankings of providers). For healthcare providers, the next generation of Colombia’s publicly reported quality measurement system should include quality scorecards at every level of the system. Benchmarks for quality results should be developed to allow healthcare providers (e.g. public hospitals) to compare quality results and learn from each other. Colombia should also establish a new generation of quality indicators, particularly for care integration and quality inequities. Finally, Colombia should put substantial effort in improving data currency (with quarterly to real-time data being the norm), data quality through regular audits and better dissemination strategies for various information users including the public. 4. Develop and implement a primary health care-focused integrated care model to support the management of individuals with complex, chronic health care needs To adapt to the increasing prevalence of non-communicable diseases, action by departmental health authorities, health insurance companies and government is needed to develop and roll out a new integrated care delivery model while strengthening primary health care. Specific options to consider in the design of an integrated model include: using integrated care pathways for chronic diseases; embedding visiting specialists in primary health care settings; interdisciplinary case management for complex patients; use of patient navigators; telemedicine or e-consult systems to provide primary care providers with greater access to specialists; incentives to co-locate different services (e.g. labs, imaging, rehabilitation) to minimize patient travel; use of home monitoring devices with two-way communication 10 | P a g e with providers; and shared information systems accessible to all members of an interdisciplinary care team. Incentives and innovation in payment systems should promote integration of care around specific medical conditions such as cancer. To achieve it, it is advisable to expand the scope of the High Cost Account program to other chronic conditions like cancer, COPD, mental illness, other rheumatic diseases, and multi-morbidities, as well as to index episodic conditions (e.g. myocardial infarction, stroke or appendicitis); and increase the emphasis on outcomes-based accountability and not just process compliance. Scaling up of self-management support (SMS) approaches should also be encouraged for patients with chronic conditions. 5. Improve accountability & contracting mechanisms to incent for quality in the health sector Enhancing patient participation and engagement in local governance and accountability processes is advisable. Patient empowerment and patient engagement is key to design, plan and manage services in a way that is person-centered, better meets the expectations of patients, families and communities and ensures better financial sustainability of the sector. Other potential interventions to augment demand for quality of care include: a national commitment to a quality guarantee; charters of rights for patients; expanding opportunities for recourse, complaints and participation in local governance systems. Accountability mechanisms to hold health insurance companies responsible for the implementation of integrated care models, and improvements in person-centeredness and efficiency of service delivery should be strengthened, through processes such as better licensing and accreditation mechanisms for health insurance companies. Payment and contracting mechanisms between the Ministry and health insurers need to be strengthened. Examples include incentives to prevent patients from developing high- cost conditions such as renal failure, through prevention and good chronic disease management or ex- post risk adjustment mechanisms for high cost patients or targeted groups of patients such as cancer patients, using the mechanism for high cost patients accounts. 11 | P a g e Immediate next steps and further analytical work required Short-term recommendations suggested cover the following: • Update and expand the scope of the national system of mandatory guarantee for quality of care including clinical competence of healthcare providers and patient engagement and empowerment • Implement an incentive regime for quality cancer care and other medical conditions through ex- post adjustment mechanisms for capitation payments (UPC) paid to health insurance companies • Develop mechanisms to benchmark quality of healthcare providers and insurance companies such as an observatory of public hospitals or a ranking mechanism for quality of care provided by health insurance companies • Strengthen system capacity for continuous quality improvement and learning & launch quality improvement campaigns for specific conditions such as cancer Further analytical work is suggested to better understand variations and drivers of quality of care and develop more targeted strategies and plans to improve quality of care for remote and rural areas and for most vulnerable populations, who are the most affected by poor quality and unsafe care. 12 | P a g e Table S1: Recommendations Based on Colombia Assessment Study on Quality of Care Recommendations Resources Difficulty of Potential Time Horizon for Main quality required execution Impact Impact (Short, Mid Dimensions Affected (+ to +++) (+ to +++) (+ to +++) or Long Term) Strengthen capacity for quality improvement in the health sector a. 1. Strengthen system capacity for continuous quality ++ ++ +++ Short Term Effectiveness improvement (QI) and learning & launch national QI Efficiency campaigns for specific conditions (e.g. cancer) Patient safety Strengthen the quality ecosystem in the health sector b. 2. Update and expand the scope of the national system of + ++ ++ Short Term Patient safety mandatory guarantee for quality of care Effectiveness Person-centeredness c. 3. Improve pre-service and continuous education for ++ +++ +++ Mid Term Effectiveness clinicians with a focus on improving clinical competence and Patient safety QI skills, and building rural training capacity Improve the rigour and use of information on quality of care 4. Augment choice of patients by providing them with + + ++ Mid Term Person-centeredness localized, pertinent information on quality Equity a. 5. Develop transparent mechanisms to benchmark quality of ++ ++ +++ Short Term Effectiveness providers and insurance companies Timeliness Equity b. 6. Improve data currency and data quality and better +++ +++ +++ Mid Term Effectiveness disseminate quality, timely information to healthcare Care integration providers, insurance companies and the public Timeliness Develop and implement a primary health care focused integrated care model to support management of individuals with complex, chronic health care needs e. +++ d. 7. Develop and roll out a new integrated care delivery model f. +++ g. +++ h. Long Term i. Care integration while strengthening primary health care j. Person-centeredness k. Equity l. 8. Expand the scope of the High Cost Account program to m. ++ n. ++ o. +++ p. Short Term q. Care integration other chronic diseases and scale up self-management r. Person-centeredness support mechanisms for patients with chronic conditions Improve accountability and contracting mechanisms to incent for quality s. 9. Enhance patient participation and engagement in systemt. + u. ++ v. +++ w. Mid Term x. Effectiveness planning and accountability processes y. Care integration z. Timeliness aa. 10. Strengthen outcomes-focused accountability bb. ++ cc. ++ dd. +++ ee. Mid Term ff. Effectiveness mechanisms through improved regulation, payment systems gg. Efficiency and incentives such as ex-post adjustment mechanisms for hh. Equity capitation payments ii. Person-centeredness + low ++ medium +++ high 13 | P a g e Table S2: Implications of Recommendations for System Stakeholders: Who is Involved Recommendations Government Department Health Healthcare Academic Patients, level health insurance providers centers citizens and authorities companies their representative groups Strengthen capacity for quality improvement in the health sector jj. 1. Strengthen system capacity for continuous quality FS M or P M P M or P I improvement (QI) and learning & launch national QI campaigns for specific conditions (e.g. cancer) Strengthen the quality ecosystem in the health sector kk. 2. Update and expand the scope of the national FSM M P P I I system of mandatory guarantee for quality of care ll. 3. Improve pre-service and continuous education for FS P P P M I clinicians with a focus on improving clinical competence and QI skills, and building rural training capacity Improve the rigour and use of information on quality of care 4. Augment choice of patients by providing them with FSM P P P I C localized, pertinent information on quality c. 5. Develop transparent mechanisms to benchmark FSM P P P P I quality of providers and insurance companies d. 6. Improve data currency and data quality and better FSM P P P I I disseminate quality, timely information to healthcare providers, insurance companies and the public Develop and implement a primary health care focused integrated care model to support management of individuals with complex, chronic health care needs 7. Develop and roll out a new integrated care FS M or P M M or P I P delivery model while strengthening primary health care 8. Expand the scope of the High Cost Account FSM P P P I P program to other chronic diseases and scale up self- management support mechanisms for patients with chronic conditions Improve accountability and contracting mechanisms to incent for quality 9. Enhance patient & citizen participation and FS M M P I P engagement in system planning and accountability processes 10. Strengthen outcomes-focused accountability FSM P P P I or P I mechanisms through improved regulation, payment systems and incentives such as ex-post adjustment mechanisms for capitation payments F: provide funding or financial incentives S: set strategic direction, standards, policies M: manage program P: participate in program or support implementation I: be informed about activity C: make informed or strategic choices 14 | P a g e 0 10 20 30 40 50 60 70 72 74 76 78 80 82 84 86 100 120 140 160 0 20 40 60 80 Mexico Colombia Japan Lithuania Mexico Switzerland Colombia Latvia Spain Slovak Republic Iceland Italy Poland Luxembourg Luxembourg Latvia Chile Australia Turkey Turkey Israel Estonia Estonia Norway Czech Republic United States France United Kingdom Hungary Korea Chile France Sweden Portugal Korea Iceland Japan Czech Republic Canada United States Portugal Ireland Hungary Austria Slovak Republic Korea New Zealand United Kingdom Belgium Netherlands Canada Canada Belgium Austria Germany Finland Finland Greece Greece (2015) Greece Sweden Slovenia New Zealand Denmark Portugal Slovenia Israel United Kingdom Slovenia Australia Germany Japan Maternal Mortality OECD (2016) Australia Denmark Life Expectancy at Birth OECD (2016) Spain Spain Chile Ireland Netherlands Czech Republic New Zealand Italy United States Switzerland Norway Poland Italy Belgium Turkey Netherlands Germany Estonia France Sweden Slovak Republic Figure S1: Comparisons of Health Outcomes Across OECD Countries Norway Poland Hungary Pneumonia: Years lost, /100 000 population, aged 0-69 years old OECD Luxembourg Switzerland Colombia Austria Israel Mexico Finland Ireland Lithuania Iceland Denmark Latvia 15 | P a g e Figure S2: Variations in Selected Indicators by Region, Public-Private or Rural-Urban Status Source: based on data from the National Quality of Health Care Observatory 16 | P a g e Table S3 below presents the detailed results of the findings from the quality results assessment, for eight dimensions of quality of care further described in the report. The scores reflected in the table below correspond to a general scale from 1 to 4, with a score of 1 for minimal or no activity; a score of 2 for <50% of desired activities, elements or results in place ; a score of 3= for >=50% of desired activities, elements or results in place; and finally a score of 4 if implementation is optimal or at benchmark. Table S3: Summary of Results from Quality Assessment Tool Quality Results Patient experience 3 Effectiveness - prenatal care 2 Effectiveness - care of children 3 Effectiveness - infectious diseases 2.5 Effectiveness - NCDs 2 Safety - hospital care 2 Timeliness - specialty services 2 Efficiency - avoidance of waste 2 Equity - maternal/child health 2 Integrated 1.5 Quality Strategy & Planning National strategy is in force with national goals & priorities. 3.5 Strategy is cascaded to all regions & institutions and different diseases. 2 Strategy is actively managed. 3 Strategy has stakeholder mapping & engagement. 2 Quality Definition, Guidelines and Standards Definition of quality exists. 4 Clinical practice guidelines exist. 4 Standards for physical infrastructure exist. 4 Standards for supply, distribution of human resources. 2 Planning standards for supply, distribution of health facilities. 2 Quality Measurement & Reporting Data collection system and quality indicators exist. 3.5 Data quality assurance mechanisms established. 2.5 Strong infrastructure for reporting and disseminating data exists. 3 Strong analytical capacity exists. 2 Strong knowledge exchange mechanisms exist. 2 17 | P a g e Quality Improvement Capacity & Activities Critical mass of staff expertise in quality management in system. 2.5 Formal quality structures and teams exist. 4 Demonstrated use of QI methods (process maps, PDSA cycles, etc.) 2 National campaigns and other methods to share learnings. 2 Broad participation in a strong accreditation program. 2 Widespread use of decision support tools. 2 Inputs – Staff, Facilities, Equipment, Supplies Physician supply 3 Nursing supply 3 Regional variations in supply of health professionals 2 Workforce management capacity 3 Availability of drugs 4 Hospital bed capacity 3 Model of care 2 Accountability and Governance Safe 2.5 Effective 2 Patient centered 2 Timely 3 Efficient 2 Equitable 2 Integrated 1.5 Policies & Incentives to Support Quality Quality of care is central to Universal Health Coverage policies 4 Patient rights legislation and patient empowerment 4 Alignment across quality of care policies 2 Comprehensiveness of quality of care policies 3 Definition of roles and responsibilities for implementation of quality of care policies 2 18 | P a g e 1 Introduction 1.1 Context and Positioning for Quality Assessment in the Health Sector Quality of health care is a major concern for national health systems around the world . The Lancet Global Health Commission on High Quality Health Systems estimates that 5 to 8 million lives are lost annually due to poor quality of care. (Kruk, Gage, Arsenault, & Jordan, 2018). Health providers in low- and middle-income countries (LMICs) provide less than half of recommended evidence-based care. One- third of patients experience disrespectful care, short consultations, poor communication or long wait times. Poor experience in turn leads to lack of trust and reluctance to seek care when needed. A recent joint OECD-World Bank-WHO report provides further evidence of quality gaps (Kieny, et al., 2018). Only 1% to 31% of patients with hypertension in LMICs are treated to a desired target. One in ten patients in LMICs suffers a healthcare associated infection when hospitalized. Medication errors are widespread, and direct costs associated with them is estimated at $42 billion. Many LMICs have moved decisively to expand universal health insurance, which is a laudable goal. However, providing good access to poor quality will not yield sufficient results; a focus on improving quality is essential for achieving better health care outcomes (National Academies of Sciences, Engineering, and Medicine, 2018). Ministries of Health around the world are struggling with how to ensure good quality within their health systems. This is especially the case when governments pay for health services, but do not directly manage health care organizations. Governments may have even great difficulty exerting influence in private sector facilities that they do not fund. Even when services are directly run, governments may not be able to ensure quality, if the chain of accountability from upper to lower management is weak. Ultimately, most of the decisions that affect quality are made at the front-line of clinical care by doctors, nurses and other providers, who have a distant relationship with governments with many intermediary levels of governance in between. Medical doctors specifically have a long tradition of clinical autonomy, making it challenging to influence provider practice both at the level of local institution as well as government. Current scientific literature and guidelines now provide a good picture of what a quality-focused health system looks like. For example, a landmark study entitled Quality by Design analyzed leading health systems around the world which are widely regarded as having superior outcomes. (Baker GR, 2008) Examples include the Jonkoping health system in Sweden, Intermountain Health and Kaiser Permanente in the USA and selected Primary Care Trusts in the UK. These organizations have strong leadership and vision for quality; robust measurement systems; and strong quality management capacity to implement changes and get staff and patients engaged in the improvement process. The WHO’s Handbook for National Quality Policy and Strategy (NQPS) re-affirms these key principles for high-quality health systems (WHO, 2018). Governments can set clear goals and priorities, engage stakeholders, establish good governance for quality, encourage the use of improvement methods, build information systems and track quality indicators. Another recent paper on government stewardship for quality proposes the following categories to assess the strengths and weaknesses of government quality stewardship: strategy, intersectoral collaboration, governance and accountability, health system design, policies and regulation. (Chan & Veillard, 2019). 19 | P a g e The World Bank and IFC have developed a common methodology to assess quality of care in the health sector including how national governments fulfil their stewardship role in managing healthcare quality. The methodology was developed by the World Bank and the IFC teams of the World Bank Group. The WBG quality assessment methodology evaluates the enabling environment and factors for quality of healthcare across the private and public healthcare sectors. It does this through a review of the ministry of health strategy and stewardship role, accountability mechanisms for quality in the sector, various regulatory aspects, and mechanisms for quality improvement such as the accreditation system. In addition, the WBG Quality Tool integrates the IFC Health Quality Assessment Tool. This tool assesses quality at hospital level and intends to (i) validate hypothesis derived from the WBG Quality Tool (e.g. the accreditation body needs to conduct trainings and improve communication on interpretation of the different standards), and (ii) shine a light on specific issues hospitals are facing in delivering quality. This assessment is instrumental to the Maximizing Financing for Development strategy for Colombia in the health sector. It is a deliberated and coordinated intervention by the WB and IFC to promote quality for better involvement of the private sector in a country where about 96% of the population is covered by social health insurance while 80% of care provision is private. The approach includes recommendations for better stewardship and management of quality of care for public and private healthcare providers. This Quality Assessment Tool is piloted in Colombia for the first time in early 2019, to help understand where the greatest quality gaps exist and inform future policymakers of priorities for future health care reforms. The Tool aims to provide a holistic view of the strengths and weaknesses of quality of care in the Colombian health care system. Discussions between the World Bank Group and the government of Colombia are currently under way regarding the terms of a possible credit operation. It is expected that priority results to be identified as part of the loan preparation will be informed by the areas for improvement identified by the Tool. Use of this systematic approach is meant to avoid missing any important areas of system weakness that need to be addressed. Colombia may benefit particularly from this assessment, given the highly decentralized nature of its health care system which makes stewardship of quality inherently challenging. Colombia’s Social Security System for Health (Sistema General de Seguridad Social en Salud, SGSSS) guarantees citizens universal health coverage through a managed competition design. Citizens belong to one of three insurance mechanisms: the contributory regime (Regimen Contributivo, RC) for employed persons, financed by payroll contributions, the subsidized regime (regimen subsidizado, RS) for low-income or unemployed persons. (MPS, 2004) and the regime for special / exception groups. Citizens in these regimes select a health insurance company (Entidad Promotora de Salud, EPS) to provide services for them, and in theory, exert influence on quality by choosing an EPS that they feel best meets their needs. Health insurance companies, in turn, purchase care from healthcare service providers, such as primary care clinics, hospitals, laboratories and imaging facilities. Health insurance companies negotiate with healthcare providers on the terms for their contract, reflecting price, volume, and in theory, quality. Territorial entities (Dirección Municipal y Distrital de Salud, DMDS) are responsible for conducting inspections on healthcare providers, and municipalities oversee the quality of public health and health promotion services. Within this complex structure, each accountability relationship must be functioning well, and failure at any level of accountability may decrease in the government’s ability to influence change. 20 | P a g e Expectations for quality in Colombia will likely increase, as life expectancy and education levels rise; use of the tool to identify opportunities to improve quality will help meet this growing demand. Maternal and child mortality have decreased steadily (see section 2.1), while the burden of illness of NCDs has increased. NCDs are now the main causes of death and disability, increasing from 46% of total disability- adjusted life years lost (DALYs) in 1990 to 69% in 2017. Post-secondary education has also expanded dramatically in the past two decades, with the number of students quadrupling during this period (OECD, 2016). A well-educated citizenry will likely have increased demand for good quality. It is anticipated that investing in activities to strengthen quality will help the government meet this growing demand from citizens for quality. 1.2 Methods The Quality Assessment Tool evaluates government stewardship of quality across 8 domains. These domains are illustrated in the graphic below: Figure 1: Domains of Quality Assessment Tool 21 | P a g e This framework reflects key components of high-performing health systems as noted in section 1.1. The approach is summarized in the following related key questions below: • Strategy: is there a clear strategy for improving quality? Are there indicators and targets? A clear description of activities that will lead to improvement? • Measurement: is quality being measured? Are measurements accurate? Repeated continuously? Are results fed back to teams in a way that are understandable? • Standards: are there clear guidelines and practice standards for clinicians? • Quality improvement capacity: are there people in the health system skilled at managing and improving quality? Can they analyze a root cause of a quality problem? Observe a process and find out where the bottlenecks or errors occur? Manage a quality improvement team? Know how to test and refine ideas for improvement before jumping to full implementation? Do organizations have a culture of learning to facilitate the quest for improvement? • Inputs: what is the supply of facilities, staff, drugs, and supplies? • Accountability: is there any methods used to ensure accountability? Are there expectations, and consequences for not meeting them? • Policies: are there policies which support quality of care, such as universal health insurance, licensing and certification of personnel, patient rights legislation, policies to encourage healthy behaviors? • Results: how does the country perform according to a range of person-centered quality indicators, covering maternal and child health, non-communicable and communicable diseases and acute care? Different elements of the WHO’s NQPS can also be mapped to this framework, as presented in table 1 below. Table 1: Comparison between WHO’s NQPS guidelines and Quality Assessment Tool WHO National Quality Planning & Strategy guidelines Quality Assessment Tool National health goals and priorities Quality Strategy and Planning Local definition of quality Quality Definition, Guidelines & Standards Stakeholder mapping and engagement Quality Strategy & Planning Situational analysis – state of quality Quality Strategy & Planning Governance and organizational structure for quality Accountability & Governance Improvement methods and interventions Quality Improvement Capacity & Activities Health management information systems and data Quality Measurement & Reporting systems Quality indicators and core measures Quality Measurement & Reporting 22 | P a g e The methodology proposed elaborates on additional areas not mentioned explicitly as a separate category of the NQPS, including inputs such as human resources and infrastructure, and policies to support quality. Within each domain, there are four to seven more detailed criteria; within each criterion, there are a set of more specific standards which contribute to the score of the criterion. For illustrative purposes, the domain for strategy contains the following four assessment items: • National strategy is in force with national goals & priorities. • Strategy is cascaded to all regions & institutions and different diseases. • Strategy is actively managed. • Strategy has stakeholder mapping & engagement. Within the first assessment item for national strategy being in force, micro-level criteria have been developed to help arrive at a score. These include: • There is a national plan to improve health care quality. • The plan is up-to-date (i.e. currently in force). • There are clearly defined priorities for improvement at national level. • The plan has baseline data on key measures, used to assess progress. • There are clearly defined numeric targets for improvement compared to baseline, with time frame for completion. In total, there are 8 domains, 49 criteria and 171 standards in the assessment tool. Within each assessment item, a score between 1 and 4 is assigned, based on the degree to which all the elements of an ideal level of implementation were achieved. For any given concept, a number of questions regarding implementation may be considered, such as the level of sophistication of a program and the number of program features; the spread of the activity or program to all facilities, regions or target populations; the inclusion of all clinical domains; and continuity over time (e.g. whether the activity is done on a regular, frequent basis or a one-time or ad-hoc basis). For example, in the criteria for clinical practice guidelines (CPGs), we examined whether the guidelines were well-constructed (e.g. appeared to reflect a thorough evaluation of the evidence with good processes for conducting the review); included a broad range of clinical topics; and were updated regularly. Scoring generally follows the following rubric: level 4 represents an ideal degree of implementation, or close to 100%; level 3, more than 50% but short of ideal; level 2, some implementation but less than 50%; level 1, little or no activity. In situations where one attribute scored a certain level (e.g. 4) and another attribute scored differently (e.g. 3), an intermediate rating was set (e.g. 3.5). The assessment methodology uses a mixed methods approach, involving data analytics, document reviews, key informant interviews, site visits to health care facilities and on-line surveys of health insurance companies. Documents for review were identified either during key informant interviews, or through a literature search. Different interview grids for key informant interviews were developed for each domain of the model, and separate grids were created for primary care and hospital site visits (see Appendix 1.2 for list of interviews and 1.3 for site visits). Site visit selection was designed to include a mix 23 | P a g e of public and private sector institutions; hospitals and primary care facilities; and rural and urban communities. A web-based survey of health insurance companies was also administered (see Appendix 1.6). The assessment team conducted a total of 42 key informant interviews, 28 site visits to hospitals and primary care sites, and surveyed 13 health insurers. An assessment of hospitals using the IFC hospital quality assessment methodology was carried out concurrently in a sample of eleven public and private hospitals part of the total 28 site visited. This assessment methodology offers a more detailed assessment of structure, human resources, equipment and clinical and managerial processes for hospital quality and is based on site visits and document reviews. The assessment was carried out from February 4 to May 31, 2019. The survey of health insurance companies was pre-tested in March 2019 and rolled out from April 10 to May 10, 2019. The IFC hospital quality assessment tool was carried out from March 20 to April 5, 2019. Recommendations were discussed with the government at different stages of the advancement of the assessment and mapped against the current priorities of the government for quality improvement in the health sector. Limitations of the approach include generalizability of information from site visits, and responder biases. Although every attempt was made to reach a diversity of sites, resources did not permit a sampling of institutions that would be statistically representative. Furthermore, rural sites were limited to the Boyacá region, due to security clearance issues. Responder bias may include situations where an individual overstates the level of quality. This was mitigated by asking for corroborating information. For example, during site visit interviews, quality staff stated that they were using common QI tools effectively. However, review of actual documentation revealed some errors and gaps in how these tools were used. While corroborating information is useful, it was not always available, resulting in potential bias of results. 24 | P a g e 2 Summary Findings 2.1 Quality Results This section evaluates Colombia’s results on a variety of quality indicators across seven dimensions of quality1. It also considers changes in quality over time as well as comparisons to international benchmarks where available. Over a period of 25 years since the 1993 health reform ( ‘Ley 100’), Colombia has made great strides to reach universal health coverage across its three key dimensions: population coverage, services and financial protection. The Colombian health system provides nearly 95% of its population with health insurance.2 Service coverage is comprehensive, with a benefits package that excludes only 57 services or technologies for primarily cosmetic or unproven health benefits (MSPS, Resolución 000244, 2019). Financial protection is high, with out-of-pocket expenditures at only 15.5% of the total health expenditures which is very low compared to other Latin American countries3. However, although Colombia generally provided greater coverage for a larger percentage of the population than many other OECD countries, Colombia’s health outcomes continue to fall below its peers. Colombia’s life expectancy at birth is lower than all OECD countries except Mexico, Latvia and Lithuania. Colombia’s maternal mortality rate in 2016 was higher than all other OECD countries and over 25% than that of Mexico which had the next highest rate. Colombia’s years of life lost due to pneumonia also stood above other OECD countries in 2015, with the exception only of Lithuania and Mexico (see Figure 2). A recent study estimates that, in Colombia, 33,917 deaths per year are attributable to the health care system; of these, 65% (22,080 deaths per year) are due to use of poor quality of care while the other 35% (11,836 deaths per year) are due to non-care utilization or poor access (Kruk, Gage, Arsenault, & Jordan, 2018). While comparisons with other OECD countries are not available, Colombia is estimated to have fewer deaths per 100,000 population (46) due to poor quality of care compared to Latin American peers like Brazil (74) and Mexico (56). Nonetheless, the numbers are staggering and point to a need for better understanding of quality of care in the country. 1 The Institute of Medicine recognizes the first six dimensions of quality. A recent WHO-OECD-World Bank report examines a seventh dimension: integration. 2 As of April 2019. See: https://www.minsalud.gov.co/proteccionsocial/Paginas/cifras-aseguramiento-salud.aspx 3 See data from Así Vamos en Salud at: https://www.asivamosensalud.org/indicadores/financiamiento/gasto-de-bolsillo-en- salud-en-colombia-periodo-2008-2014 25 | P a g e 0 10 20 30 40 50 60 70 72 74 76 78 80 82 84 86 0 20 40 60 80 100 120 140 160 Colombia Japan Mexico Lithuania Mexico Switzerland Colombia Latvia Spain Slovak Republic Iceland Italy Poland Luxembourg Luxembourg Latvia Chile Australia Turkey Turkey Israel Estonia Estonia Norway Czech Republic United States France United Kingdom Hungary Korea Chile France Sweden Portugal Korea Iceland Japan Czech Republic Canada United States Portugal Ireland Hungary Austria Slovak Republic Korea New Zealand United Kingdom Belgium Netherlands Canada Canada Belgium Austria Germany Finland Finland Greece Greece (2015) Greece Sweden Slovenia New Zealand Denmark Portugal Slovenia Israel United Kingdom Australia Germany Slovenia Japan Maternal Mortality OECD (2016) Australia Denmark Life Expectancy at Birth OECD (2016) Spain Chile Spain Netherlands Czech Republic Ireland Italy United States New Zealand Switzerland Norway Poland Italy Belgium Turkey Netherlands Germany Estonia Figure 2: Health Outcomes Comparisons Across OECD Countries France Sweden Slovak Republic Norway Poland Hungary Pneumonia: Years lost, /100 000 population, aged 0-69 years old OECD Luxembourg Switzerland Colombia Austria Israel Mexico Finland Ireland Lithuania Iceland Denmark Latvia 26 | P a g e Timeliness Wait times remain comparable to those in many high-income countries but have increased for different medical specialties and remain an issue of concern. Average waiting times have seen a modest increase from 2011 to 2018, for example for general practitioners (from 2.7 to 3.4 days) and gynecologist consultations (2.7 to 10.1). Wait times for pediatric consultations were 8.8 days, for internal medicine 12 days. Among 5 of 19 OECD countries reporting data, at least one half of residents waited four or more weeks to see a specialist, suggesting an average wait time of at least one month.4 Safety Colombia has seen improvements in available patient safety indicators. Between 2011 and 2014, the number of nosocomial infections per 100 hospitalization days decreased from 1.41 to 0.83. Similarly, in- hospital mortality rate after 48 hours of admission decreased from 17 in 2011 to 13.1 in 2014. One caution is that these data do not undergo the same data quality audits as he High Cost Account program (see discussion on data quality section 2.4). Effectiveness Effectiveness in maternal care has improved, but there is ample room for further progress. The maternal mortality ratio has declined by almost half, from 68.8 in 2011 to 36.1 per 100,000 births in 2018. However, as noted above, maternal mortality still compares poorly to other OECD countries. Process measures give insight into how this outcome could be improved. Only 53% of women in 2017 had a first prenatal visit in the first trimester and only 88% received 4 prenatal visits during pregnancy in 2015. During prenatal care consultations, in 2018, only 68% of pregnant women had their blood pressure measured, blood drawn, or urine sample taken for testing. These are key services recommended by evidence-based clinical guidelines that should be undertaken in all antenatal consultations. Furthermore, indicators of low value care services such as high caesarian section rates were very high at 44% in 2018. Effectiveness measures in care of children have improved over time. Vaccination rates have increased between 2011 and 2018, for DPT vaccine (from 86% to 93%), BCG (83% to 89%) and pneumococcus (46% to 94%). Outcomes have also improved from 2011 to 2016. Infant mortality has declined steadily from 12.2 to 8.9 per 1000 births as has under 5 mortality (15.9 to 13.8). Available effectiveness measures for non-communicable disease care show poor results. For cancers, Pap screening was undertaken in the previous year for 51% of women between the ages of 25 and 69, while 30% of women between the ages of 50 and 69 were screened for breast cancer using mammography in the past two years (2017). By comparison, rates of breast cancer screening with mammography in OECD countries range from 42% in Hungary to 84% in Finland and Pap smear screening rates range from 35% in Hungary to 82% in Sweden in 2016. Mortality rates have increased from 2011 to 2017 for hypertension (from 13.1 to 17.9 per 100,000 population) and diabetes (from 3.4 to 4.7). 4 From the OECD Health Statistics database: https://stats.oecd.org/index.aspx?DataSetCode=HEALTH_STAT# 27 | P a g e Changes to effectiveness of care for infectious diseases are mixed. Malaria mortality has decreased from 23 in 2011 to 7 in 2018. However, tuberculosis care has worsened as the percentage of patients successfully completing treatment dropped from 66% to 33% between 2011 and 2018. In similar contrast, in 2014, the rate of adherence to antiretroviral treatment for HIV was relatively good at 89%, but vertical HIV transmission from mother to child was far too high, at 47%. Patient Experience Overall rates of satisfaction with health services received are generally high in Colombia and have improved modestly from 70% in 2014 to 73% in 2017 (MSPS, 2017). Satisfaction rates were higher in 2017 among beneficiaries of the subsidized regime (77%) than the contributory regime (67%). A large majority of beneficiaries (80%) would recommend their insurer (EPS) to their friends and family for affiliation. Satisfaction with facility infrastructure was relatively high (74%) but low for waiting times (51%) and insurance approval processes (47%). A relatively high number of patients report receipt of respectful treatment by providers (77%) in 2017 but this value decreased from 80% in 2016. Efficiency and Integration There are few indicators of equity, care integration and efficiency. The proportion of surgeries that are cancelled is 6.8% (2013 figures). This measures inefficiency but represents only a small example of waste in the system (MSPS, 2015). Rates of readmission within 15 days were at 1.35% in 2014; such rates should be lower if there is good coordination between the hospital and primary care at discharge. However, readmission rates by disease are more useful as an indicator for making fair comparisons. The development of better indicators for these dimensions should be considered. A focus on inequities and variations in quality results There are significant differences that are statistically significant across quality of care indicators when we compare urban and rural areas, public and private facilities as well as across departments. Variations in Effectiveness Indicators of effectiveness point to significantly greater weaknesses in public facilities as compared to private ones, in rural facilities as compared to urban ones and in departments with higher poverty rates. The percentage of pregnant women who received 4 antenatal care consultations is significantly lower in public compared to private facilities (63% vs 73% in 2018). Similarly, the percentage of pregnant women who received their first antenatal consultation within the first twelve weeks of gestation was lower in rural than urban areas (7% vs 9%). When comparing across departments those with the higher rates of poverty show much lower rates of delivery of screening mammographs for women between the ages of 50 and 69 in 2018; the departments of Vichada, Vaupés and Guainía have rates of screening that are one tenth those of the country average (1.1%, 1.6% and 2.6% respectively compared to 10.3% for the country average) (see figure 3). The differences in these rates can be nearly twice as high within departments such as Bogotá, Bolivar and Valle del Cauca. 28 | P a g e Figure 3: Geographic variations in proportion of women aged 50-69 who received a screening mammograph, 2018 Source: Based on data from National Quality of Health Care Observatory Variations in Safety Available data for health care safety show mixed results when comparing urban and rural facilities as well as those in departments with higher poverty rates. In-hospital mortality rates per 1000 admitted patients within 72 hours of admission was significantly higher in rural compared to urban facilities (6.22 vs 1.35, in 2014). This contrasts however, with in-hospital infection rates compared across departments, where those with higher poverty rates such as Amazonas, Magdalena, Vaupés and Vichada report infection rates as low as 0.3 per 100 admitted patients, whereas wealthier departments such as Santander, Valle, Antioquia and Bogotá report rates of 2.14, 1.78, 1.49 and 1.31. This is most likely due to underreporting of patient safety adverse events rather than safer care. Variations in Timeliness Available measures for timeliness suggest low availability of specialized care providers in public sector and rural facilities and for departments with higher rates of poverty. The average waiting time for an appointment with a general physician was slightly higher in public compared to private sector facilities (2.9 vs 2.6 days, 2014 data). Differences in wait times were more pronounced between rural and urban areas, for general physicians (6.4 vs 2.7 days), gynecologists (16.3 vs 8.0 days), general surgery (20.7 vs 14.1 days), and pediatrics (38.8 vs 7.5 days). In 2018, there was a 15-fold difference between departments in the time required to obtain an internal medicine consultation (1 day in Vaupes compared to 15 days in Guainía; see Figure 4). 29 | P a g e Figure 4: Geographic variation in wait times for a general medicine appointment in days, 2018 Source: Based on data from National Quality of Health Care Observatory Variations in Efficiency There is limited data on health care efficiency for Colombia; data on C-sections suggest greater inefficiencies in private sector facilities. WHO (2015) guidelines suggest that C-section rates should be between 10% and 15%, and OECD data shows that seven-member countries have been able to limit their rates to under 20%5. Colombia’s rate is second-highest among OECD countries, with private facilities having higher rates than public ones (49% vs 42%). Information from the 2017 Atlas of Health Variations published by the Ministry shows large variations in utilization of different procedures. While there are no clear benchmarks for appropriate use of these services, the variations suggest either over- or under- utilization in some regions. For example, there is a 16-fold difference between departments in the standardized rate of CT scans per 100 residents (7 in Meta vs 0.4 in Vaupés), suggesting either over- or under-utilization in some regions. The rates in higher-income departments such as Bogotá and Antioquia (4.9% and 3.6% respectively) are higher compared to Vichada (0.7%), a lower-income department. 5 https://data.oecd.org/healthcare/caesarean-sections.htm 30 | P a g e Figure 5: Geographic variations in C-section rates, 2017 Source: Based on data from National Quality of Health Care Observatory Synopsis: Colombia has made progress in reducing hospital infections, vaccinations and outcomes of maternal and child health. There have been modest improvements in patient experience, and wait times are comparable to OECD peers. Yet, major gaps in quality remain. Over 22,000 patients were estimated to die from poor quality care in 2016 while 12,000 deaths were attributable to non-utilization or poor access to services. Many evidence-based practices for prenatal care and cancer screening are not well implemented. Variations in quality are common. 31 | P a g e Table 2: Ratings for Section on Quality Results Patient experience 3 Effectiveness - prenatal care 2 Effectiveness - care of children 3 Effectiveness - infectious diseases 2.5 Effectiveness - NCDs 2 Safety - hospital care 2 Timeliness - specialty services 2 Efficiency - avoidance of waste 2 Equity - maternal/child health 2 Integrated 1.5 2.2 Quality Strategy & Planning A key stewardship role for health ministries is to develop strong national quality strategies and plans. This is consistent with the WHO’s NQPS guidelines. The plan should have clear priorities, targets, and details on the activities to be implemented. There should be a clear description of current gaps in the system to be addressed. The planning process should be done with strong stakeholder engagement, and there should be a “cascading� of the plan to different actors and regions, whereby the responsibilities of each stakeholder are clear. Lastly, there should be an organizational unit dedicated to the development and implementation of the plan for quality of care. Three documents taken together form the elements of a national “quality plan�: The National Development Plan, 2018-2021 (PND); the 10-Year Public Health Plan 2012-2022 (PDSP); and the National Health Quality Improvement Plan 2016-2021 (PNMCS). The PND, approved by Congress through legislation, covers multiple sectors (e.g. economic growth, education) in addition to health care, with projected investments of COP$550 trillion (US$180 billion) over the next four years. The PDSP outlines goals for improvements in public health indicators. The PNMCS is a strategic plan for QI, which outlines high-level activities planned by government to support desired improvements specified in the PDSP (MSPS, 2016). Plans also exist at a regional and institutional level. Municipalities are required to formulate a Development Plan covering all aspects of public infrastructure, with a component on health. For example, Cali’s plan sets targets for reducing infant mortality, with more specific targets for immunization rates and adoption of guidelines for treating childhood illnesses (Alcaldía de Santiago de Cali, 2012). Furthermore, each territorial entity prepares a Territorial Health Plan (PTD) every three years*, which should align with the national PDSP. Each municipality must also develop a yearly Collective Intervention Plan (PIC) with health promotion and prevention activities in support of the municipal Development Plan and the national PDSP. Finally, the Audit Program for Quality Improvement (Programa de Auditoria para el Mejoramiento * For more Information: www.minsalud.gov.co/salud/publica/epidemiologia/Paginas/planes-territoriales-de-salud.aspx 32 | P a g e de Calidad, PAMEC) is a quality improvement plan which must be developed by each Health Provider Institution (Institución Prestadora de Servicios de Salud, IPS), EPS and territorial entity.* PAMEC is one of the four components of the Mandatory System of Quality Assurance (Sistema Obligatorio de Garantía de Calidad, SOGC). Each organization audits its own weaknesses in quality and develops an improvement plan, which is monitored by the territorial entities and Supersalud (Superintendencia Nacional de Salud), the Ministry’s agency responsible for overseeing compliance with regulations. Although legislation mandates that a strategy should be developed at national, regional and institutional level, the plans are not closely linked. Ideally, the national plan “cascades� down to different parts of the health system. Regions set local goals that contribute to the national goal, and health facilities set goals that contribute to regional goals. Units and staff within facilities have goals that contribute to the health facility goal. In practice, however, interviewees noted that there was confusion over how all plans fit together. Part of this confusion relates to a perceived complexity of regulatory requirements for planning. Interviewees in DMDS stated that they had set locally adjusted targets, but that there was no process of verifying whether local targets, if accomplished, would add up to the national target. We could not find a dashboard comparing plans of each DMDS and the extent to which each entity had met targets. Interviewees themselves asked for a clear strategy map, showing how local priorities linked to national priorities. Colombia endorses the “triple aim� and has a broad array of 10 specific priorities for population health; to maximize impact, the Ministry may wish to consider narrowing the focus of quality improvement activities to only one or two topics at a time. The PND explicitly endorses the “Triple Aim� promoted by the US-based Institute for Healthcare Improvement (IHI), of improved health for populations, improved care for individuals, and optimal resource use (Departamento Nacional de Planeación, 2019). The PDSP outlines eight specific priorities: environmental health; NCDs; mental health; nutrition and food security; sexual and reproductive health; infectious diseases; public health in emergencies and disasters; and occupational health. There are also two cross-cutting priorities: better meeting the needs of vulnerable populations and improving governance and management capacity of health districts (MSPS, 2013). One criticism voiced by some interviewees is that the ten priorities of the PDSP are so broad that they essentially cover all quality issues within the health system. From a quality perspective, it may be helpful to narrow the focus to a smaller number of topics for national, coordinated action at any given moment in time. Targets for improvement are well defined in some areas (e.g. health prevention) but not in others. Furthermore, there are sub-indicators associated with each of these priority areas, each of which have specific goals. For example, there are goals for mortality from childhood malnutrition, prevalence of undernutrition, and reproductive health, and there are goals for maternal mortality and teen pregnancy rates. A weighted index is constructed for each of the eight priorities, and a dashboard is available which allows planners to monitor progress. However, this type of target-setting is missing in other areas. For example, although patient experience is acknowledged within the triple aim, we could not find targets *For more Information: www.minsalud.gov.co/salud/Paginas/auditoria-parael-mejoramiento-dela-calidad.aspx 33 | P a g e specifying desired improvements in different components of patient experience, such as communication, courtesy, responsiveness and shared decision-making. Current planning documents describe general strategic directions but do not contain the precise implementation details necessary for quality improvement. In the health section of the PND, areas of action include: strengthening technical capacity for managing quality in territorial entities; strengthening supervision by SuperSalud; eliminating differences between the CR and SR to promote equity. The PMNCS proposes greater technical assistance to provider organizations to improve quality; activities to improve shared learning between organizations; strengthening accreditation; and policies and training on humanization of health care. Details on how to accomplish these strategies remain to be developed. There are organizational units for planning for managing a quality strategy. At the Ministry of Health and Social Protection, the Directorate of Promotion and Prevention in Health is responsible for the implementation of the PDSP, while the Directorate of Epidemiology and Demography monitors and evaluates the fulfillment of its goals. The Advisory Office of Planning and Sectoral Studies (OAPES) of the Ministry of Health and Social Protection is responsible for development and monitoring of the National Development Plan. The Ministry’s Office of Quality is the steward of the quality plan (PNMCS). At subnational level, territorial entities and municipalities formulate and implement the PTD and PIC. There are some forums for stakeholder engagement in development of the plan, but these could be strengthened. According to interviews, the OAPES holds annual meetings of national stakeholders to discuss progress in national plans, bringing together academics, professional associations and provider organizations. There is also a planning meeting for territorial entities. While these activities are laudable, it appears that there are ample opportunities to ensure greater communication and engagement at various levels of the systems. During site visits, interviewees were generally aware of the major priorities of the PDSP but not of the PNMCS. The role of different stakeholders in improving quality is still unclear. Section 6 of the PNMCS describes roles and responsibilities of the Ministry, territorial entities, healthcare providers and health insurance companies. However, many of the tasks are generic and apply to all parties (e.g. promote a culture of quality, align planning activities, develop and apply methodologies, etc.). Synopsis: There is a plethora of detailed strategy documents related to health quality at multiple levels of the system. However, priorities are still broad, implementation strategies are vaguely defined and the relationship between planning activities carried out at the national, regional and organizational level is not clear. There are opportunities to engage stakeholders by clarifying their roles and responsibilities, and by ensuring greater transparency on how accomplishing their goals will contribute to national-level goals. 34 | P a g e Table 3: Ratings for Quality Strategy and Planning National strategy is in force with national goals & priorities. 3.5 Strategy is cascaded to all regions & institutions and different diseases. 2 Strategy is actively managed. 3 Strategy has stakeholder mapping & engagement. 2 2.3 Quality Definition, Guidelines and Standards Having a clear definition of quality, standards and guidelines is an important prerequisite to creating a high-quality health system. The WHO NQPS emphasizes that each country should develop its own definition of quality. Standards and guidelines are also necessary; they serve as a more detailed, working definition of quality. Standards help identify the norms of what types of services and infrastructure should be available, and clinical practice guidelines describe what services providers should give to patients. Guidelines and standards should be responsive to local context. Colombia fulfils the requirement of the WHO NPQS that each country should develop its own definition of quality. Colombia’s SOGC, established by Decree 1011 (MPS, 2006) identifies five dimensions of quality: accessibility, timeliness, safety, pertinence (medical necessity) and continuity. The decree defines quality of health care as “…the provision of health services to individual and collective users in a manner that is accessible and equitable, at an optimal professional level, taking into account a balance between benefits, risks and costs, with the purpose of achieving the adherence and satisfaction of these users�. It should be noted that two of the Institute of Medicine’s six dimensions of quality – patient experience, equity - are mentioned implicitly in the latter definition, but effectiveness overlaps partially with pertinence (medical necessity). Colombia has a well-established system of clinical practice guidelines (CPGs) . CPGs define what interventions health care providers should deliver, in terms of tests, drugs, other treatments and advice provided. Guideline development is the responsibility of the Institute for Health Technology Assessment (IETS), a non-profit agency*. The MSPS website has a section compiling a national repository of CPGs†. A review of these guidelines found examples for a wide range of clinical conditions spanning maternal and child health, NCDs, infectious diseases and patient safety in hospitals. Most were found to be up to date (i.e. developed and validated within the last five years). Colombia has good basic standards for health facilities, regarding equipment, supplies, physical space, and available personnel. The SOGC has four components: licensing (habilitación), accreditation, health information systems, and auditing for quality Improvement. Licensing (habilitación) identifies minimum standards for each IPS and individual health professional (MSPS, 2014). These standards, updated in resolution 2003 in 2014, identify basic criteria for qualifications of staff and the type of equipment, facilities, physical space and staffing available in health facilities. Interviewees did suggest, however, that * For more Information, visit: http://www.inahta.org/members/iets † To review repository, visit: http://www.gpc.minsalud.gov.co/gpc/SitePages/buscador_gpc.aspx 35 | P a g e some standards which have little contribution to quality should be eliminated – such as regulations on the inside corner trim of interior spaces. Colombia, through its accreditation program, has well-developed aspirational standards for processes and structures needed to promote quality and safety. Within the SOCG, licensing (habilitación) is considered a minimum, mandatory standard while accreditation is an aspirational, voluntary standard. Accreditation contains 158 standards with specific criteria, covering topics such as care processes and management of quality, health information, technology, physical environment, and human resources (Ministerio de la Protección Social, 2011). Under the largest section for care processes, specific clinical best practices are evaluated, such as those for preventing hospital-acquired infections (standard 39), communication and patient engagement (standard 30). Colombia does not have clear planning standards for the supply of health facilities and staff. According to interviews, there are no benchmarks for the number of hospital beds, physicians or nurses per population. Each territorial entity develops its own plan for public hospital capacity and submits it to the Ministry of Health and Social Protection for approval. Private investors can build new facilities with no prior authorization from the Ministry. Synopsis: Colombia has a clear definition of quality, a strong set of clinical practice guidelines, and comprehensive minimum requirements for infrastructure in each health facility. It does not, however, appear to have a clear plan for supply and distribution of health facilities and personnel in the country. Table 4: Ratings for Quality Definition, Guidelines and Standards Definition of quality exists. 4 Clinical practice guidelines exist. 4 Standards for physical infrastructure exist. 4 Standards for supply, distribution of human resources. 2 Planning standards for supply, distribution of health facilities. 2 36 | P a g e 2.4 Quality Measurement & Reporting Quality measurement is critical for any health system, as it helps identify areas of weakness, set priorities, support learning, and verify that improvement activities have resulted in better results. This section examines the extent to which Colombia has a strong system for measuring quality reliably, providing feedback to providers and managers, and using the data effectively to improve quality. Over the past several years, Colombia has developed an impressive array of databases and data sources. As noted above, the SOGC’s third pillar is the information system for quality. The Office of Information Technology and Communications (OTIC) within the MSPS is responsible for overseeing the health information infrastructure in the country. It manages SISPRO, a national data warehouse which houses data on supply and use of health services, quality of care, insurance status, financing and health promotion. Examples include: • REPS*, a registry of health care organizations. • RETHUS †, a registry of health professionals and the services they are authorized to provide. • RIPS, (MPS, 2009) a database of utilization information about each health care service provided, including visits to primary care physicians, dentists, diagnostic imaging centers, emergency departments and hospitalizations. Colombia also conducts national patient satisfaction surveys to compare performance between Health insurance companies (MSPS, 2017). Colombia has also mandated the collection of a large basket of quality indicators. First, health care organizations and health districts periodically submit information on 55 quality indicators, as required by Resolution 256/2016 (MSPS, 2016). This list includes patient safety (e.g. falls, pressure ulcers, procedure- related infections), effectiveness (e.g. childhood mortality), timeliness (e.g. wait times for specialist visits, advanced tests) and patient satisfaction. Second, data on individual persons related to public health and prevention are submitted by healthcare providers to health districts by resolution 4505/2012 (MSPS, 2012). Examples include vaccinations, communicable diseases, prenatal visits, family planning, Pap smears and mammography screening. (MSPS, 2013) Third, under the High Cost Account program, as specified by decree 2689/2007 (MPS, 2007), healthcare providers must submit monthly data on quality indicators for selected high-cost diseases such as diabetes, renal failure and rheumatoid arthritis. The quality of data collected is good for the High Cost Account program, but other data sources are not subject to as rigorous a quality assurance process. Data are collected usually at the level of healthcare providers based on information recorded by health care providers. At some site visits, staff were observed entering clinical information in certain templates within their electronic medical record (EMR) and the EMR automatically prepared data for submission. In most instances, data quality assurance consists of monitoring incoming data for results that appear implausible (e.g. a male receiving a service intended for * Visit: https://www.sispro.gov.co/central-prestadores-de-servicios/Pages/REPS-Registro-especial-de-prestadores-de-servicios- de-salud.aspx † Visit: https://www.minsalud.gov.co/salud/PO/Paginas/registro-unico-nacional-del-talento-humano-en-salud-rethus.aspx 37 | P a g e females). Numerous interviewees raised concerns that this process is insufficient for assuring data quality. In contrast, the High Cost Account program has a structured system of random monthly audits of patient care which are conducted by an auditor who may be external or internal to the organization. Auditors receive training and certification from the Ministry. Information is also published on data quality from each insurance company for the sake of transparency (Cuenta de Alto Costo, 2018). Interviewees expressed confidence in this system and suggested its use for other types of data. Information is disseminated via a public reporting website and national reports; however, there are opportunities to improve the user-friendliness and timeliness of these tools and to guide users on how to use them for quality. The MSPS has also published two versions of the National Report on Health Care Quality (Informe National de Calidad de la Atencion en Salud – INCAS) in 2009 (MPS, 2009) and 2015 (MSPS, 2015). These reports contain information similar to what is on public websites but include text to help the reader interpret and contextualize the raw data. Another key publication is the quality ranking of health insurance companies, published by the High Cost Account program (Cuenta de Alto Costo, 2018). This report focuses on indicators for the high cost diseases, and for each indicator, health insurance companies are given a colour code (red, yellow and green) representing their relative performance. In addition, some of the data in SISPRO can be accessed through a public reporting website, the national quality observatory (Observatorio Nacional de Calidad en Salud).* Here, the user can select a topic (e.g. patient safety) and obtain information on different indicators, broken down by healthcare facility. Figure 6: Example of Indicator Report from National Observatory of Health Quality * The quality observatory is accessible at: http://calidadensalud.minsalud.gov.co/Paginas/Indicadores.aspx 38 | P a g e While the quality observatory is an excellent resource that is the envy of other middle and even high- income countries, opportunities for improvement were observed. First, some key indicators are not available on the website (e.g. control of blood sugar for diabetes). Second, there is a lengthy time lag with many indicators being more than one year out of date. Third, there were issues of user-friendliness resulting in error messages produced for simple queries. The language of the website is highly technical, making it difficult for a lay person to navigate. Information on quality is not flowing back to providers to help them improve quality of care. This observation was noted during site visit interviews. Providers require micro-level information on the performance of their own practice, in real time if possible. As noted above, the quality observatory does not provide timely feedback and many indicators are missing. Interviewees noted that some information may be sent to health insurance companies and management in healthcare providers, but do not necessarily get used by front-line providers. There is an opportunity to improve knowledge translation – the existence of information and guidance on how to use quality data for improvement. Documents reviewed, such as the quality reports described above and indicator metadata documents tend to focus on questions on how the indicators are defined, and description of differences (e.g. between organizations). We could not find detailed information on what an individual should do differently based on this information. For example, in the quality observatory, a health professional can look up information about his/her hospital’s performance, but there is no guidance on what the desired level of quality should be, nor on what steps to take or what ideas to implement to improve performance. There is limited analytical capacity to do more sophisticated analyses. Interviewees noted that the staff at OTIC tended to be mostly composed of individuals skilled at databases management but did not have backgrounds in biostatistics or epidemiology and did not produce more advanced analyses. Suggestions for increased research activities included better benchmarks for quality, and deeper analyses on the predictors of quality and impact of different policies and programs on quality. Synopsis: Colombia has developed an impressive array of data collection tools, standardized quality indicators, a centralized database, and a public reporting website. Areas for improvement include improving data quality; feedback of data to health professionals and healthcare providers in a timelier fashion; user-friendliness of web-based reports on the performance of health insurance companies for the benefit of the public; clearer benchmarks for performance; and better analytical capacity to do research on factors influencing quality and the impact of different policies on quality. Table 5: Results for Quality Measurement & Reporting Data collection system and quality indicators exist. 3.5 Data quality assurance mechanisms established. 2.5 Strong infrastructure for reporting and disseminating data exists. 3 Strong analytical capacity exists. 2 Strong knowledge exchange mechanisms exist. 2 39 | P a g e 2.5 Quality Improvement Capacity & Activities The presence of leaders, managers and staff throughout the system with strong knowledge of quality improvement techniques is a hallmark of a high-performing health system. This section examines whether a health system has formal quality committees and structures; strong, expert-level staff with the skills to manage quality and implement improvements; demonstrated use of QI tools and methods; use of decision support tools; and accreditation programs. Colombia has a basic infrastructure for quality within health care institutions, including quality committees, staff and basic policies. All hospitals and clinics visited had at least one person identified as having responsibility for quality and at least one quality committee. This is likely in response to Decree 3518/2006 mandating the creation of infection control and public health surveillance committees, for monitoring reportable diseases (MPS, 2007). Sites visited were also able to show policies and procedures for infection control, dispensing of medications and other safety practices. Variations were observed in level of quality management expertise. There are many international formal training programs and certifications in quality in existence,* which have a set curriculum of defined skills (e.g. root cause analysis, process mapping and redesign, managing a Quality Improvement project, etc.). We asked staff during site visits about the types of qualifications they had, including Colombian equivalents of international programs. Some private sector hospitals had extensive quality departments (e.g. over 30 staff in one case) with at least some individuals trained at a master’s degree level in quality. Many other sites, however, reported that staff had learned quality improvement techniques through self- training. There were wide variations in use of quality improvement methods between healthcare providers visited. One large private hospital system, for example, demonstrated well-developed quality dashboards for its leadership, and used sophisticated analytical tools such as failure modes effects analysis to anticipate and prevent patient safety failures. Another was using software to help teams document their use of QI tools. At other sites, the examples of tools requested (e.g. process maps, Ishikawa diagrams / root cause analyses, PDSA cycles) did not actually match the description of the tool or were used improperly or incompletely. We also requested examples of successful quality improvement projects. Many sites were able to provide at least one example, but some sites defined their activities as “implementation of a tool� rather than an actual improvement in a quality indicator (e.g. reduction of patients falls). These challenges aside, interviews with quality staff in health care institutions enthusiastically supported greater activities to increase level of expertise and training. These findings suggest that elements of a quality culture are in place, with openness to identifying problems and team- based problem solving, but there are major opportunities to strengthen this culture. Another weakness is the extent to which quality improvement activities and knowledge is shared. In other jurisdictions, this is often done through “quality improvement collaboratives�, a structured method *Examples include Lean/Six Sigma green or lack belt, IHI Improvement Advisor, certifications from American Society of Quality or National Association for Healthcare Quality, or Patient Safety Officer program of Canadian Patient Safety Institute. 40 | P a g e where quality teams working on a common topic meet periodically to share information on implementation challenges and possible solutions. “National campaigns� are a more intensive, coordinated version of such initiatives. Less structured models include “communities of practice� which create regular discussion forums through a variety of means. Interviewees at site visits generally stated that outside of their organizations, there is a lack of formal structures to support this type of sharing of knowledge. Colombia has a well-developed and comprehensive accreditation system, but the program suffers from low participation. Accreditation is managed by ICONTEC, an arms-length organization which runs the program based on fees charged. This program is recognized by the International Society for Quality in Healthcare (ISQUA). One of the challenges with the program, however, has been the low participation rate, as less than 50 healthcare providers have been accredited to date. Interviewees noted several challenges, including: the cost of accreditation (relatively high for small hospitals); limited resources to comply with physical infrastructure requirements; heavily labour-intensive processes; and little benefit of formal accreditation, other than prestige; little financial incentive to do so, or compensation for costs of going through the process. Some interviewees stated that they could get the benefits of accreditation by adopting the standards without formally obtaining accreditation. Decision support tools can help providers increase their adoption of best practices; there are some examples of their use, but ample opportunity to expand their application. One of the main barriers to adoption of CPGs is that providers may be unaware of their details, or forget to implement them, given their complexity. Decision support tools remind providers what they should do in different situations. Such actions may include questions to ask, items to check, or drugs, treatments or advice to give to patients. Examples used in Colombia include the WHO surgical checklist. We saw examples of protocols for different emergency situations posted in hospitals. Some primary care EMRs automatically calculated malnutrition indicators based on the weight and height entered and provided warnings when indicators were in an unacceptable zone. These are positive developments, but there was ample room to increase the use of decisions support tools found in other countries. We did not observe the use of standard order sets in hospitals, which provide in advance the recommended diet, activity, drugs and investigations for patients with a particular admission diagnosis. Most hospitals did not have automatic systems to alert providers of potentially dangerous drug interactions or contraindications. Primary care sites did not use flowsheets for chronic disease management, which serve as checklists for items to review at each visit. In general, involvement of clinicians in the definition of practice guidelines, standards, decision support tools and verification of adherence to guidelines is low. Synopsis: Colombia has many structures in place for quality improvement capacity, including formal committees and quality managers. However, there are variations between sites in the level of training of these individuals and the extent to which quality methods are used. There are few opportunities for healthcare facilities to benchmark and learn from one another. There is a very strong desire from frontline healthcare professionals to strengthen their benchmarking and continuous quality improvement skills across the system. 41 | P a g e Table 6: Quality improvement capacity & activities Critical mass of staff expertise in quality management in system. 2.5 Formal quality structures and teams exist. 4 Demonstrated use of QI methods (process maps, PDSA cycles, etc.) 2 National campaigns and other methods to share learnings. 2 Broad participation in a strong accreditation program. 2 Widespread use of decision support tools. 2 2.6 Inputs – Staff, Facilities, Equipment, Supplies The availability of physical infrastructure, drugs and human resources in the health system is a basic prerequisite for quality; the adequacy of these inputs is assessed in comparison with peer countries . Making conclusions about the adequacy of these inputs based on statistics such as ratio of providers per population is challenging. There is no clear evidence-based benchmark for ideal supply, and each country’s choice depends on its model of care, and on the extent to which a country can task-shift between healthcare professionals and operate more efficiently. Nonetheless, for the purpose of comparison, we use the following approach to set a conservative benchmark: examine the supply of inputs in high-income OECD countries which are generally regarded has having well-functioning health systems while having relatively lower ratios of inputs to population. This section also explores the model of delivery of care, and whether it is designed to provide comprehensive primary care with good continuity of provider and smooth transitions and coordination between primary care, hospitals, and specialist services. Colombia has made major investments in improving the supply of health human resources; however, the supply of doctors, nurses and hospital beds remains low compared to OECD countries. According to the RETHUS database*, the number of doctors has increased by 34% over just six years, to a level of 128,354 as of 2017. Nurses have increased by 47%. Despite these increases, Colombia has a physician to population ratio of 2.1 per 1000, which is lower than the OECD median of 3.2 and lower than the conservative benchmark of 2.6†. This ratio is, however, comparable to other Latin American countries with similar income levels‡. For nursing, Colombia has both nurses and auxiliary nurses (1.26 and 5.16 per 1000) but relies more on the latter compared to other countries. When both types of nurses are grouped together, the total nursing professional to population ratio again is in the lower end of OECD countries (benchmark 8 per 1000)§. Hospital bed capacity, at 1.7 beds per 1,000, is also at the lower end of OECD countries and lower than the benchmark of 2.5.** * The RETHUS database is accessed at: https://www.datos.gov.co/Salud-y-Protecci-n-Social/Saludatos-Estimaci-n-de-los- Indicadores-para-el-se/5atd-7cac † Example: select countries with strong health systems include Japan (2.4 doctors/1000); USA 2.6, UK 2.7, UK 2.8. ‡ Visit WHO Observatory at: https://www.who.int/gho/health_workforce/physicians_density/en/ § Visit: https://data.oecd.org/healthres/nurses.htm#indicator-chart ** Basket of 5 countries: Sweden 2.2 Denmark 2.3 Canada 2.5 UK 2.6 New Zealand 2.7. The OECD median is 3.8. 42 | P a g e Availability of medications is generally good and is guaranteed by law. The Constitution of 1991 guarantees universal health coverage for essential medicines. There is a standard benefit plan which defines drugs and services which providers should offer; medicines, services and technologies that are not in the benefit plan may still be provided through the MIPRES tool (MSPS, 2017). Regulation 1751 of 2015 (Congreso de Colombia, 2015) confirms that drugs will be provided, except those lacking in clinical evidence of effectiveness or safety. One issue noted in this regulation is that there is a national mechanism for setting prices of drugs, for cost containment purposes. Although most site visit interviewees did not raise concerns about drug availability, some did report occasional shortages of basic drugs such as oxytocin or lidocaine; it is not clear if this is a supply chain issue or due to the price being set too low to attract reliable suppliers. Basic human resource management procedures are in place, but there are numerous concerns about high staff turnover, occupational stress and burnout. Basic tools such as job descriptions and staff evaluation mechanisms were in place in most sites visited. However, interviewees raised concern about high staff turnover. Reasons for this are not clear, but one issue is that in rural areas there is high turnover as new graduates are required to do a year of mandatory public service in these communities and usually do not stay afterwards. Regarding occupational stress, the Ministry has recently issued guidelines on avoiding burnout, and recognizes this as a major concern that also affects patient safety (MSPS, 2015). These guidelines propose indicators for future measurement, but baseline statistics are not yet available. Human resources policies have established minimum standards for health professionals, but confidence in medical skills is weak, likely due to problems with lack of standard medical curriculum, standard evaluation of skills, and verification of skills over a health professional’s career . The Habilitación program acts as a form of credentialing to ensure that health professionals have minimum standards for previous training. Providers who wish to work in a territorial entity must present their qualifications, be registered within the territory, and renew this registration every five years. Many interviewees, however, expressed concerns. First, there is no standardized examination and standardized curriculum for recent graduates of medical schools, and therefore no way of measuring clinical competency reliably. Second, there are no mandatory requirements for maintenance of competence, and no process to re-evaluate whether health professionals have maintained their skills and knowledge in the face of expanding medical knowledge. One example of poor clinical skills noted by some site visit interviewees is that primary care providers referred excessive numbers of cases to specialists. A formal survey of health care managers noted that 59% did not have confidence in the clinical skills of generalist physicians in their facility, and 27% lacked confidence in their specialists (Universidad Javeriana, 2013). 43 | P a g e Figure 7: Density of Health professionals per 10,000 residents in Colombia, 2011 to 2016 Source: ReTHUS database (MSPS, 2018) There are major regional imbalances in distribution of health professionals, which have persisted over time. As noted in the figure below from MSPS, there is a six-fold difference in the health provider to population ratio between the region of Bogotá and Chocó, which has the lowest supply. Policies, such as the social service requirement of new graduates to spend one year in an underserviced area, do not appear to have a long-term impact on this distribution problem. Staff interviewed cited numerous problems with the current model of care, such as poor communication between primary care and other sectors. Interviewees noted that when patients are sent to a specialist, other health workers (e.g. dietician or mental health), or admitted to hospital, information about the visit is not sent back to the primary care physician. Some exceptions were noted; accredited hospitals generally provided information about treatment plans, and information on pregnancy care is usually provided. Also, in some cases a patient is provided with a summary of the diagnosis and treatment plan. If the patient brings this to the visit, then the doctor is aware, but this does not always happen. In general, specialists and primary care doctors work in separate locations, do not share records and cannot see each other’s health information. One exception was a network in Bogota which had visiting specialists within a primary care clinic, who could access the same EMR. Joint case management of patients with difficult-to-treat medical conditions generally does not occur. Interviewees also described inconveniences for patients in obtaining lab, diagnostic imaging and specialist services. First, some services can be obtained only if ordered by a specialist. Second, patients may need to wait for the EPS to approve services ordered by the physician; depending on the EPS, the delay ranges from minimal to five months. Third, interviewees noted that the EPS may have contracts with different providers for different tests, resulting in patients having to travel to multiple sites. Fourth, sometimes tests are repeated by specialists who do not trust the accuracy of tests performed in a primary care site. Lastly, there is a gatekeeper system where patients see specialists only through referral from primary care. However, some the health insurance companies with financial problems do not have a 44 | P a g e contract with a healthcare provider for specialty care, and the healthcare provider then resorts to sending the patient to the nearest emergency department for treatment. Synopsis: Colombia has a supply of human resources and hospital infrastructure comparable to middle- income South American countries but less than typical OECD countries. There are concerns about the system’s ability to ensure staff have appropriate clinical skills. Existing resources are not used optimally; health professionals are concentrated in urban regions like Bogota, and problems with the model of care delivery need to be addressed, including poor coordination and a disjointed patient journey due in part to the way the health insurance companies contract their services. Table 7: Results for Inputs – Staff, Facilities, Equipment, Supplies Supply of health providers 3 Regional variations in supply of health professionals 2 Human resource management capacity 2 Availability of drugs 4 Hospital bed capacity 3 Model of Care 2 2.7 Accountability & Governance Colombia uses a mix of most of the different accountability tools seen worldwide and has a chain of accountability from the Ministry of Health to regional governments, insurers, providers, health professionals and to the population. While the web of accountability relationships in Colombia is highly complex, we identified licensing/inspection, accreditation, public reporting, patient and community complaints and insurance-based supervision as the main accountability tools of the system. Our assessment evaluates these tools in terms of their effectiveness, efficiency and reach across seven dimensions of healthcare quality: patient safety, effectiveness of care, patient centeredness, timeliness, efficiency, care integration and equity. Patient Safety Inspection through licensing (habilitación) is an important method of accountability for patient safety. As noted in section 2.3, licensing examines structures and inputs such as policies, committees, infrastructure and supplies for infection control. DMDSs undertake periodic licensing inspections and assessments to determine if an IPS can operate. Weaknesses in the licensing (habilitación) program include infrequent inspections, conflicts of interest, misalignment with local realities, and reluctance to sanction particularly in areas with only one provider. First, healthcare providers generally receive a full inspection only once every four years. Second, territorial entities often own the facilities they license, and therefore have a conflict of interest, with no incentive to close services or complicate the licensing process. Hence, there are serious doubts about the transparency, fairness and technical rigor of this process. Third, health insurance companies and healthcare providers interviewed also felt that licensing criteria were not aligned with local realities. Public 45 | P a g e hospitals tend to have older infrastructure that does not meet basic criteria (for example, space standards or anti-seismic protections). Fourth, interviewees raised concerns about facilities which are the sole provider in a region for the subsidized regime; they cannot be closed, and as a result, they may be permitted to operate with substandard levels of patient safety. Because of weaknesses in licensing (habilitación), some insurance schemes have developed parallel inspection systems; this creates a duplication of efforts. Some leading health insurance companies have developed systems where they undertake their own safety assessments for each healthcare provider under contract. Different health insurance companies are each inspecting facilities which leads to overlapping efforts. This also places a burden on healthcare providers as the insurers are making multiple inspections and each applies a somewhat different inspection standard. Most health insurance companies include expectations for safety in their contracts, albeit with weak consequences for missing expectations. According to a survey of health insurance companies, performance indicators used in contracts with healthcare providers include hospital falls (93% of health insurance companies having responded to the survey), infections, and surgical complications (71%). In most cases, failure to meet an expectation would result in a request for a quality improvement plan (91%) or a meeting to discuss results (71%). The accreditation system is a potential tool for accountability for patient safety but is not used for this purpose. As noted in sections 2.3 and 2.5, accreditation sets rigorous standards of care, but suffers from low participation. Most providers see it as costly and cumbersome. There is a concern expressed that accreditation requires expensive upgrades that are not clearly related to quality such as raising ceiling heights or increasing the number of elevators. Furthermore, there is a perception in the market that accredited providers, are not achieving better outcomes and so health insurers do not see the rationale for paying higher prices for their services. As such, it appears that the main rationale for accreditation is that it has been linked to the loss of ability to operate for providers linked to universities. The audit program for quality improvement (PAMEC) is not effective. Under the SOGC, healthcare providers carry out a self-assessment of their weaknesses (using accreditation standards for comparison) and generate a PAMEC, which is shared with health insurance companies and SuperSalud. However, interviewees stated that there were no consequences if the goals of the PAMEC were not achieved. Timeliness Providers are held accountable for meeting wait time targets set through regulation through reporting to Supersalud and public reporting mechanisms. Supersalud, can penalize care providers and insurance companies for missing targets. Supersalud also processes complaints filed by users, frequently related to waiting times, including specialty consultations. Insurers and providers interviewed, mentioned that they were unaware of instances of legal consequences from Supersalud for not meeting targets. Public reporting of wait times is another mechanism of accountability, through the Observatorio public website, but its usefulness seems limited (see below on effectiveness). 46 | P a g e The court system of tutelas allows citizens to hold providers accountable for quality; most of these complaints are for wait times. An Ombudsman’s Office (Defensoría del Pueblo) independently tracks waiting times and complaints and helps patients file ‘tutelas’, judicial actions to exercise their rights. This Office publishes periodic reports on tutelas (Defensoría del Pueblo, 2017) which garner wide media coverage, holding poor performing insurers or providers accountable. Contract clauses between health insurance companies and healthcare providers include accountability for wait times, as noted by 86% of respondents to the survey of health insurance companies. In case of poor quality in general (for wait times or other dimensions of quality), health insurance companies may cancel contracts (71%), refrain from renewing contracts (64%) or withhold funds (43%). Patient Advocacy Groups* for specific conditions that require high-cost technologies help hold providers accountable for timely access to care. These groups actively engage the media about their concerns. Researchers can also be influential; when long delays were shown to decrease child leukemia survival (Suarez, 2015), the government issued a policy to accelerate diagnosis and treatment initiation. There are opportunities to improve accountability for timeliness of care. There is large variation in the times for appointment when the information system is analyzed, and it is commonly known that the insurance companies and providers have developed ways to reduce waiting times on paper, resulting in unreliable information. This suggests that improved standards for measuring waiting times, and an increased focus on auditing is warranted. Effectiveness Primary care providers are accountable to territorial entities and health insurers, through contracts, for measures of effective service delivery. At site visits, for example, primary care sites noted financial penalties for missing targets for indicators such as immunization rates. According to the Health insurance company survey, 86% of insurer respondents included targets for management of diabetes, hypertension, adequate prenatal care and HIV, within their contracts with healthcare providers. The High Cost Account system is working relatively effectively. The High Cost Account (Cuenta de Alto Costo), established and managed by health insurance companies, has been used to track health outcomes of high cost illnesses for the purpose of ex-post risk adjustment between all payers in the country. This has given insurers an incentive to effectively manage these diseases, and so has strengthened the accountability for the outcomes of diseases. The High Cost Account program also publishes public reports of rankings of health insurance companies on a variety of detailed process and outcome measures of effectiveness, which in theory encourages lower performers to match the results of leaders (Cuenta de Alto Costo, 2019). Health insurance companies have audit mechanisms for adherence to CPGs, but these are fragmented, do not necessarily align with national CPGs, and are reported to be ineffective. Health insurance companies have developed auditing mechanisms for service provision in Colombia that are parallel to those implemented through the continuous improvement strategies outlined in the Decree 1011. These *Concerns about their receiving funding from drug companies prompted them to sign a covenant for transparency to dispel suspicions that they were acting in the interest of these companies 47 | P a g e auditing mechanisms are inefficient because each insurance company contracts health providers with the expectation of delivery of care through different clinical guidelines which are not required to align with national guidelines produced by IETS, nor is there a requirement to ensure those applied are based on the latest scientific evidence. Furthermore, interviewees at health facilities and health insurers agreed that adherence to clinical practice guidelines is not adequately verified and is very low, particularly among medical specialties. Medical schools are not held accountable for producing graduates that can deliver effective health care. Large gaps in accountability mechanisms for health care effectiveness are found lower in the supply chain as medical schools are not supervised in their development and application of medical curricula as the legal autonomy of universities restricts the ability of the Ministry of Health and Social protection to hold them accountable for clinical competence of medical doctors and nurses during their pre-service training. A more proactive involvement of clinicians, not only in developing and adapting CPGs but also in improving quality of care and in managing unwarranted variability in usage patterns would be desirable, as it goes in line with the profession’s commitment to self-regulation that was enshrined in Law 1751. Efficiency Although efficiency is considered as a key component of quality of care in Colombia, our assessment finds limitations in the availability of explicit expectations, comprehensive accountability tools or designated actors to ensure it. The main tools to ensure efficiency are the health insurer contracts with health care providers. Insurers are under substantial pressure to reduce costs because the prices (premiums) they charge are regulated. This cost pressure is not necessarily translating into oversight over providers’ efficiency. Indications are that some insurers, particularly those with low financial solvency, are responding to cost pressures by introducing measures to reduce patients’ access to effective treatments and medication, or are contracting with lower quality, and so lower cost, providers. Few insurers have the systems needed to hold providers to account for efficiency . To reduce waste by providers, insurers need (at a minimum) to collect detailed, audited data on providers’ expenditures and treatments on an individual patient basis. Relatively few insurers have these systems in place. Many insurers pay providers through overall budgets for specified types of care. This makes monitoring of waste more difficult. Insurers have developed a tool for service utilization review to justify payment denials on grounds of absence of medical necessity, however this system is not based on an assessment of standardized national evidence-based clinical guidelines, it creates high transaction costs and is believed to lead to opportunistic behavior by health care providers. Outside of the insurers contracts, a number of systems are in place to encourage and support better efficiency of providers, but indications are that they are not working effectively. The Ministry of Health has introduced an information system to track pharmaceutical prescription practices, called MIPRES, which aims to hold physicians accountable for their prescription practices of medicines not currently included in the benefits package. Although this system publicly reports prescription practices, it only covers a limited set of medicines and results are not widely communicated to patients, territorial entities or insurance companies, to effectively hold physicians accountable. Accordingly, it is urgent for clinicians to control among themselves for unwarranted variations in over testing/overtreatment (under fee-for- 48 | P a g e service payment models) and undertesting/undertreatment (under prospective payment models), following the principle of professional autonomy, which must be paralleled with the commitment to self- regulation that is stated in Law 1751. Patient-centeredness There is some accountability for patient experience in contracts between health insurance companies and healthcare providers. Half of respondents to the EPS survey report including patient experience expectations in their contracts with hospitals, and 72% included it in contracts with primary care facilities. Public reporting is one form of accountability for patient experience. As noted in section 2.4, the Ministry of Health and Social Protection conducts patient satisfaction surveys annually and releases reports comparing health insurance companies. (MSPS, 2018). Information is also available on the quality observatory of the Ministry of Heath and Social Protection. The instrument measures cordialness of treatment but does not yet include questions such as care co-ordination, respect for individual values, receipt of clear information and education related to care, or emotional support, which would provide a more comprehensive assessment of person-centeredness of the system. Public reporting, in theory, can help people choose their insurer based on quality, thereby holding insurers accountable; in practice, however, this mechanism is weak because publicly reported information is difficult to use. The health care system of Colombia was designed to allow people to freely choose their insurer based on quality of care. Theoretically this should create a direct channel of accountability whereby insurers feel compelled to improve quality to avoid losing customers. In a recent survey, 20% of Colombians considered switching EPS, due to concerns with wait times; this suggests they are ready to exercise this form of accountability. However, as noted in section 2.4, public reporting websites and reports are highly technical and difficult for the user to interpret. Information is at least a year out of date. There are no clear benchmarks for good performance. We did find one example of a newspaper that distilled information in a user-friendly format*; such formats could be marketed more aggressively to the target audience. Inadequate risk adjustment in remuneration may be discouraging health insurance companies to aim for the best result in public reports. Some interviewees from top-ranked health insurance companies noted that funding formulas for patients do not fully account for the complexity, in terms of current diseases and severity and costliness of each disease. As a result, it is reported that having a high score on public reporting has generated adverse selection whereby the sickest patients with expensive diseases seek care from the best provider, who is then paid at a rate that does not reflect the cost of treatment for such a patient. This gives insurers the incentive not to improve outcomes or to not report achievements. Users associations and patient-advocacy groups are an increasingly important mechanism for person- centeredness. User associations are created for insurers and public hospitals. Insurers report that user associations are very active and are routinely listened to when discussing quality matters. A second * https://www.eltiempo.com/vida/salud/ranking-de-las-mejores-eps-de-colombia-en-2018-196510 49 | P a g e mechanism of public participation in quality of care decisions is through patient engagement in public hospitals boards of directors. Integration There is no clear mechanism to ensure accountability for co-ordination of care. Territorial entities and Health insurance companies are expected to act as the coordinators of care across facilities and stages of the disease (MSPS, 2016). However, patients are not ensured a seamless flow of their care due to fragmented delivery systems, poor interoperability in information systems and poorly designed referral pathways. A policy (MSPS, Resolución 1441, 2016) establishes the concept of integrated provider networks (Modelo Integral de Atención en Salud – MIAS), but implementation remains elusive. Difficulties in setting up such networks lie in the autonomy of the DMDSs which are implicitly in charge of the organization of the public provider network, but given the multitude of contracting mechanisms, this does not necessarily result in integrated care for enrollees within the territory. Bundled payment models for services across facilities could act as a financial incentive for integrated care, but this has not been implemented in most settings. Health insurance companies are moving away from fee-for-service remuneration models towards prospective payments where they get a set fee for a package of services (e.g. a hospital stay). However, these models are based on an individual venue of care, or individual specialty. This can reinforce fragmentation of care, as there is an incentive to shift costs to others. Equity Accountability mechanisms to ensure equity within facilities include patient advocacy groups but other formal mechanisms could be strengthened. At the facility level, differential treatment of patients by income level or insurance regime was mentioned by providers as a key focus for continuous improvement, likely reflecting variations within and across facilities. Patient advocacy groups and representation on boards can help hold facilities accountable for providing patients with the same quality of care irrespective of income, race or gender. However, equity indicators do not appear in the list of indicators in resolution 256, 4502 or 2699 (high- cost conditions); without formal indicators and targets, it is difficult to set expectations for improvement. Yet, equity is acknowledged as a concern in the National Development Plan; the government wishes to close the gap in services offered between the subsidized and contributory regimes and address regional inequalities in quality and access to care. We identified nine key binding constraints to accountability for quality of care in Colombia: • The availability of relevant and timely information about quality of care is limited for patients, which reduces the pressure they can impose on the system to drive improvements. • Weak accountability tools to ensure care coordination among health care providers and within health insurance networks result in poor person-centered care, fragmented care, inefficiencies, untimely and ineffective care for many patients. 50 | P a g e • The use of facility and insurer benchmarking on quality of care is limited and fragmented, reducing its potential impact to inform patient choice and performance-based contracting mechanisms. • Per-capita allocations to insurers are not adequately adjusted for risk, creating a disincentive for insurers to invest in quality improvements that will attract high-risk, high-cost patients. • Accountability mechanisms from the Ministry and SuperSalud to health insurance companies are weak, as no clear tools have been developed for specific aspects of quality of care such as care coordination, effectiveness or person centeredness. This has led to weak incentives to ensure and improve the quality of care their customers receive. • Insurers in a poor financial position undermine quality of care. They face difficulties in contracting good-quality providers, and so provide a ready market for poor quality providers. • Inefficiencies are arising from a lack of co-ordination and accountability mechanisms between insurers. These include multiple inspections and inconsistent treatment protocols. This all leads to unnecessary costs that do not promote quality of care. • Local governments that undertake licensing are often under-funded and are often not independent of the providers they are overseeing. This reduces their ability to ensure appropriate oversight especially in the context of monopolistic local markets. • Facility licensing and accreditation, key accountability tools for quality of care, rely on onerous and often irrelevant standards used primarily to ensure patient safety. These onerous standards create barriers for the implementation of these tools and are limited in the impact they can have on other dimensions of quality of care such as effectiveness or patient centeredness. Synopsis: Colombia has developed numerous tools for holding health care providers accountable for quality, including inspection programs; public reporting; complaints processes, community engagement and public courts; accreditation; and contracting. Yet despite the existence of these activities, accountability for quality is still weak, due to flaws in the design or deployment of each of these tools. Among dimensions of quality, accountability mechanisms were strongest for “timeliness� and weakest for “integrated�, but there is room to improve accountability in all domains. 51 | P a g e Table 8: Results for Accountability and Governance Safe Avoiding injuries to patients from the care that is intended to help them. 2.5 Providing services based on scientific knowledge to all who could benefit Effective and refraining from providing services to those not likely to benefit (avoiding underuse and overuse). 2 Providing care that is respectful of and responsive to individual patient Patient centered preferences, needs, and values and ensuring that patient values guide all clinical decisions. 2 Reducing waits and sometimes harmful delays for both those who receive Timely and those who give care. 3 Avoiding waste, in particular waste of equipment, supplies, ideas, and Efficient energy. 2 Providing care that does not vary in quality because of personal Equitable characteristics such as gender, ethnicity, geographic location, and socioeconomic status 2 Integrated Providing care that is coordinated across provider levels and specialties 1.5 2.8 Policies & Incentives to Support Quality of Care Policies and regulations are important for encouraging providers, managers and the public to act or behave in certain ways that will improve quality. Policy instruments may be used to create financial incentives, set directives to undertake certain actions, enforce adherence through use of penalties, forbid certain actions, establish new organizations with certain functions, or fund new initiatives. The section below, describes available policies for quality of care, their strengths and weaknesses. The Colombian General System of Social Security in Health (Sistema General de Seguridad Social en Salud, SGSSS) provides almost universal insurance coverage. Colombia’s constitution was amended in 1991 to ensure that access to health care was a universal right for its residents. The health reform of 1993 (Ley 100) and the subsequent Sentence T760 of 2008 that called for the equalization of the right to health for all residents, provides the legal basis for a commitment to Universal Health Coverage. Central to the design of the health system, to achieve this aim, is the expectation of quality of care whereby all residents, given the right to health care are ensured a capitated payment to their insurer of choice that is to be selected on the citizens’ perception of quality of care to be received. Since 1993, the proportion of the population with some form of coverage has increased to 95% (OECD, 2016). In addition, out-of-pocket expenditure as a percentage of current health expenditure declined from an estimated 44% in 1993 to less than 20% in 2016, significantly lower than the Latin American average of over 30%. In addition to the legal basis of the right to health defined in the constitution and health reform in the early 1990’s, Colombia has established a system of judicial claims to ensure the right to access and quality of health services. Among Latin American countries, Colombia stands in similar ranking to Brazil, with the highest number of judicial claims (tutelas) filed by residents who primarily demand access to specialist health services and timeliness in the receipt of services. Although system of judicial claims (tutelas) is currently backlogged and responses are often delayed, the continued policy of its use continues to assure residents of the country of their right to health and quality of services. 52 | P a g e In addition to quality of care-related plans, Colombia has several other laws and regulations which promote quality, safety and patient experience. For example, Law 23 of 1981 (Congreso de Colombia, 1981) and Resolution 1995 of 1999 (Ministerio de Salud, 1995) establish rules on the management of medical records, confidentiality and access to information and informed consent. The Ministry of Health and Social Protection has also issued policy guidelines for patient safety (MPS, 2008), which includes investigation of critical incidents. Quality of care policies are fragmented across different national and subnational plans and a decree. The section on quality strategy describes three key policy instruments that define quality standards for health service delivery: The National Development Plan (2018-21), the Public Health Plan (2012-22) and the National Quality Improvement Plan (2016-21). These documents lay the basis for a policy and goals on health service delivery quality at the national level in Colombia but provide fragmented directives. At the subnational level the Territorial Health Plans and Collective Intervention Plans provide further directions and goals on quality of care. The documents are complemented by the Decree (1011) on the SOGC that defines mechanisms and responsible actors for the implementation of quality improvement policies. Across these policy instruments, there is a variation of goals, strategies and responsible actors that create duplications and gaps in quality directives for health insurance companies and healthcare providers. Although policies are fragmented, overall across the different policy tools, quality of care has been defined and a set of standards developed. The SOGC defines quality of care and identifies licencing (habilitación), accreditation, auditing and information systems as central mechanisms for quality assurance. Additionally, clinical practice guidelines define standards for effective care and a large array of databases define expectations of measurement through a define set of 55 quality indicators. The definition of quality is comprehensive, however, the tools and standards to ensure policy implementation, are not. As mentioned in the section on quality measurement infrastructure, information systems are comprehensive but policies to ensure the quality of data and its utilization for quality improvement are weak. Additionally, as mentioned in the section on accountability, quality assurance tools such as licensing, accreditation and auditing are not always consistently implemented or having the desired quality improvement impact. Policies, standards and regulations have been implemented by health facilities who have developed quality infrastructure that includes committees, staff and policies. At the facility level, investments in quality of care that include the availability of specialized staff for quality measurement and implementation of quality improvement projects was found throughout facility visits. Incentives for quality improvement at the facility level, however, are both limited and mixed due to fragmented and restricted expectations from existing policies. Policies, often through information systems and legal standards, are currently guiding facilities to focus on the timeliness of service delivery and patient safety but provide little guidance on use of clinical practice guidelines and care coordination, for example. As mentioned in the accountability section, existing policies do not currently provide comprehensive guidance on the roles and responsibilities of key actors in the health system, to ensure quality of care. Responsibility for the availability and distribution of key inputs necessary for quality of care, such as 53 | P a g e infrastructure, medicines and human resources is weakly defined by policies setting these responsibilities at the national and subnational levels as well as with health insurers and care providers. Responsibilities for implementation of licensing programs to ensure patient safety are defined but conflicts of interest are a barrier to their effectiveness. Responsibility for ensuring care effectiveness is not clearly defined and generally lies with insurers, providers and provider training programs. Finally, tools and standards defined through the different policy documents do not outline responsibility or mechanisms for key areas of policies on patient-centredness, health care integration and safety. Synopsis: Colombia has developed policies for quality of care, but the goals, standards and implementation processes are currently fragmented across different documents. Policies on quality of care provide guidance on several key aspects of quality of care but often lack a clear definition of actors responsible to implement policies and standards. Table 9: Results for Policies & Incentives to Support Quality Quality of care is central to Universal Health Coverage policies 4 Patient rights legislation and patient empowerment 4 Alignment across quality of care policies 2 Comprehensiveness of quality of care policies 3 Definition of roles and responsibilities for implementation of quality of care policies 2 2.9 Focus on Hospital Quality Assessment A parallel IFC mission using a more detailed assessment tool provided additional information about hospital quality. The IFC tool has developed a structured and evidence-based survey methodology which evaluates quality across a series of detailed criteria, based on site visits, interviews with management and front-line staff, and review of quality assurance policies and procedures. Areas visited included operating room, intensive care unit, patient wards, emergency department, lab, radiology, and sterilization units. The survey questionnaire was developed in collaboration with the US-based Joint Commission International (JCI) and incorporates core international standards based on JCI and international standards. The survey covered 11 public and private hospital facilities across three regions of Colombia – Bogota, Barranquilla, Cartagena, and Cali and a meeting with ICONTEC (see Appendix 1.2 for full listing). There were a number of strengths regarding use of key international patient safety best practices, but some variation in their application. In particular, there were good practices for patient identification; management of high-alert medications; safe surgery (e.g. use of WHO checklist); basic infection control (e.g. hand hygiene procedures); falls prevention (e.g. falls risk assessment); and patient ethics and rights (e.g. informed consent, use of patient bill of rights). Some gaps in application of these concepts were noted in some institutions. For example, there were some instances of identifying patients by room instead of name, and different colour code systems for types of patient risks used in different hospitals, 54 | P a g e which can lead to confusion. Some hospitals had falls risks (e.g. slippery floors, hazardous stairways). Some hospitals did not publicize patient rights or had informed consent procedures that were not specific enough to the procedure done. Several key practices on medication management are in place, with some gaps noted. Most hospitals had clinical pharmacists present, conducted regular audits, a clear drug formulary, and a medication management plan and use of medication management forms to track timing and dose of each drug administered. There were gaps in certain practices (e.g. measures to deal with look-alike, sound-alike drugs) and there were some shortages of basic drugs noted (e.g. lidocaine, oxytocin in some regions). Although a broad range of quality improvement activities are in place, there are variations in the level of training in quality among managers and in the application of quality tools; staff expressed a desire for greater support in implementing quality improvement activities. Consistent with observations in section 2.5, there is a quality manager, quality team and quality improvement plan in each hospital. Each site collected data and reported indicators on incidents (e.g. near-misses, sentinel events), outcomes (eg mortality rate, infection rate), and patient and employee satisfaction. Incident reporting systems were in place, with root cause analysis conducted and preventive measures taken. Numerous methods for tracking patient concerns are used, including suggestion boxes, surveys and web interfaces. Regarding areas for improvement, the qualifications of quality managers varied; some had formal training while others were self-taught. Incidents were addressed at different levels of thoroughness in different hospitals. Some flow charts are not developed up to the required level. Some documents lack proper implementation or are not understood by staff. Some staff pursuing accreditation felt they did not have enough internal knowledge or capacity and had to rely on external consultants. Overall, many interviewees felt there was inadequate external support for their QI efforts and expressed a desire for this gap to be addressed. Infection control programs and officers are well-established, as well as basic procedures and amenities for sanitation, but gaps were noted in some sites. Most hospitals had handwashing programs, isolation areas, properly organized sterilization units and formal procedures for waste management including biomedical waste. All hospitals had sharps containers. Staff were generally observed wearing masks and other protective means when required. Indicators on infections are collected and analysed. Areas for improvement in some sites include the following: biomedical waste containers overfilled; empty hand sanitizer stations; inconsistent infection control requirements in sterile areas; ceiling or wall materials not meeting basic requirements (e.g. have pores which can harbour microorganisms) and lack of infection prevention in kitchen areas. Basic management and human resource practices are in place. Each site had an organizational chart, mission and vision, and necessary licenses displayed. Under human resources management, there are job descriptions, orientation programs for new staff, an assessment program for staff, and probation / training periods for new employees. 55 | P a g e Synopsis: Based on our observations, the overall development of healthcare quality assessment standards in Colombia seemed good – even compared with other middle-income countries; areas for improvement include addressing variability in the implementation of different quality standards, and variability in training and knowledge of quality concepts. Specifically, quality requirements are consistent with international practices, quality regulations are clear, and staff awareness of these required practices and regulations is high. For each of the areas of quality management listed above (e.g. medication management, infection control, safety practices, falls prevention), there were gaps noted in implementation. While there is good basic knowledge of quality concepts, there is variation in the more detailed use and application of quality tools. These findings are consistent with those in other sections of the report, including observations made in other sectors such as primary care. 2.10 Summary of Key Assessment Findings Colombia has made important progress in promoting a quality agenda. Interest in quality dates back to 2006 with resolution 1011 creating the SOGC. Since then, the country has developed a remarkable wealth of planning documents for quality throughout the system. There are clear targets for improvement in health outcomes. There is now a formidable repository of data in SISPRO and a detailed list of quality indicators, and some of this data is available by IPS and EPS on a public website. There are standards for quality for both infrastructure and clinical practice. There is a basic licensing and inspection program (habilitación), and a rigorous accreditation program. There are individuals with official responsibility for quality throughout the health system along with quality committees. In the past several years, there have been notable improvements. Universal health coverage is solidly entrenched. Outcomes have improved in maternal mortality, infant mortality and vaccination rates. While these accomplishments are commendable, there are major opportunities for improvement in all areas of the framework. Highlights are as follows: • There is ample room to improve quality of care results in the health sector, with large variations in quality found between the public and the private sector, by geography, urban and rural settings, and worst quality affecting the most vulnerable populations. As measured, satisfaction rates are high and waiting times are low compared to OECD countries, however, measures of care effectiveness suggest weaknesses in care processes for NCDs, efficiency indicators suggest over-use of available resources and low availability of integrated care indicators show further weaknesses in quality of care. • The national quality strategy needs to be more explicit about targets, plans, activities, roles and responsibilities, and accountability for results. There are few mechanisms to ensure proper alignment and cascading between national quality strategy and strategies of health insurance companies, department level health authorities and healthcare providers. • The rich data sources in the government’s integrated information system for the sector are rarely used for quality improvement activities. Information is not fed back to front-line providers who need it the most. Some data collected are not easily accessible and information 56 | P a g e that is published is mostly out-of-date. Data are released publicly with little context on how to interpret the results or what actions to take for improvement. • Capacity for continuous quality improvement is unevenly distributed in the sector yet there is a tremendous amount of energy and enthusiasm for quality improvement in the sector. There are wide variations in the number of healthcare professionals trained in quality between different healthcare providers and health insurance companies. There is a lack of mechanisms for shared learning between peers, and limited information on benchmarks for quality. The accreditation program is rigorous but has very low enrolment. Use of decision- support tools to guide clinical decision making is limited and should be expanded. • Mechanisms to verify clinical skills of providers are weak. There is no core standardized curriculum for medical graduates across the country. There are important disparities in human resource distribution, especially in rural and remote parts of the country. • The current model of care is fragmented. This is an important barrier to quality care for a country where most of the burden of disease comes from non-communicable diseases requiring care integration and coordination to be effective. There is poor communication between primary care and specialist services. Patients seek care from multiple providers (e.g. primary care, laboratories, specialty care) in different sites, due to the nature of contracting for different services by health insurance companies. This is an impediment to delivering quality care for an increasing number of patients with multiple chronic conditions. • Accountability mechanisms need to be strengthened and accountability for care integration is weak and of concern. Although there are examples of incentives for quality in contracts between few health insurance companies and healthcare providers, this mechanism is generally weak. There are only a few examples of financial accountability for quality. Consumers in theory hold health insurance companies accountable for quality, but lack the quality, timely information to do so. 57 | P a g e Table 10. Summary of Strengths and Weaknesses from Key Assessment Findings Domain Strengths Weaknesses Planning & Multitude of national, regional planning Priorities widespread Strategy documents; 10 priorities; targets for Implementation details need further development public health; planning bodies Standards Good clinical practice guidelines, facility Standards for planning density lacking standards, quality definition Measurement Good list of quality & public health Data quality gaps indicators Lack of benchmarks SISPRO database Data not fed back to providers Public reporting through quality Data out of date observatory QI capacity Quality committees widespread Wide variations in QI staff capacity Rigorous accreditation program Low use of decision support tools Low participation in accreditation Few opportunities to share learnings Very few documented case studies of improvement Inputs Wide range of health professionals No standard medical curriculum Recent increase in supply of staff No standard examination of graduates Weak system to monitor competency Weak model of care, poor coordination between primary, specialist care Governance & Habilitación system for basic inspection Patients lack info needed to choose EPS based on Accountability Some examples of community quality accountability, financial incentives, use Duplicate inspections by Health insurance companies of contracting Contracting for quality could be strengthened Weakness particularly for coordination Policy Strong universal health coverage Safeguards for privacy & confidentiality 58 | P a g e 3 Recommendations The following recommendations address the key areas for improvement identified in the preceding section summarizing findings from this assessment. Overall, we found that a more ambitious and consistent approach to measuring, supporting and improving quality at hospital- and clinic- level will be key to unlocking the next wave of quality gains in Colombia’s health care system. To do this, action should be taken by government, health insurance companies, local health authorities and care providers to: (1) strengthen capacity for continuous quality improvement in the health sector; (2) strengthen the quality ecosystem in the health sector; (3) improve the rigor and use of information on quality of care in order to stimulate choice by patients and peer learning by healthcare providers; (4) develop and implement an integrated care model to support the management of individuals with complex, chronic health care needs; and (5) improve accountability and contracting mechanisms to incent for quality. The 10 recommendations are summarized in table 11 at the end of this section, which also presents for each recommendation a preliminary assessment of resource requirements, difficulty of execution and potential impact, each rated on a scale of + to +++. We also indicate the expected time horizon for achievement of impact of short-, medium-or long-term. These scores are intended to encourage the Ministry of Health and Social Protection of the government of Colombia to consider these perspectives as they plan for implementation. Detailed planning will be required to estimate resources, map out activities, and manage risks related to implementation. Specifically, variations in quality of care among departments, urban and rural locations, and public and private facilities call for targeted interventions addressing specifically the needs of the most vulnerable populations, who tend to experience lower quality care and whose health outcomes are worse than most of the population. It is important to note that this report proposes a cohesive package of interventions which are all necessary and important to achieve sustainable change in quality of care in the sector. If these recommendations should be carefully sequenced and planned by the government, the implementation of all ten recommendations over time will be necessary to achieve impact at scale and make a commitment to quality a cornerstone of the financial sustainability of the sector. It is also important to acknowledge that all actors in the sector have an implement role to play in implementing the recommendations proposed for consideration: table 12 makes the point that not only the government, but also departmental health authorities; health insurance companies; healthcare providers; and patients and citizens all have to be fully engaged and lead the quality revolution that is required to ensure that a high quality health system in Colombia delivers quality results for all Colombians. 59 | P a g e 3.1. Strengthen capacity for quality improvement in the health sector An initial and critical step to build up capacity for quality improvement in the health sector is to invest in augmenting the capacity of healthcare professionals for continuous quality improvement (QI) . This would entail developing a critical mass of healthcare professionals with advanced quality improvement skills within public and private health care facilities, health insurance companies and territorial health authorities, who are able to lead quality improvement initiatives and manage change, bring order to poorly designed or confusing processes, and incorporate patient preferences into healthcare design and planning. Targets could be set to define the adequate number of quality improvement experts for each different category of health care provider, health insurance companies and territorial entities. A standard curriculum could be developed based on international methods, and skills would include root cause analysis of quality problems; process mapping and redesign; team leadership, conflict resolution and physician and stakeholder engagement; and use of Plan-Do-Study-Act cycles. Training could include a practical component where participants must demonstrate effective use of QI techniques in a QI project, as is done in other similar programs.* In addition, launching a national quality campaign for specific clinical conditions such as cancer or diabetes, as well as quality improvement collaboratives and communities of practice to promote shared learning among healthcare providers, territorial health authorities and health insurance companies would be transformative. Organizational structures should be developed to encourage QI teams at different sites working on the same topic to learn together and share knowledge and experience. A classic method for doing so is the “Learning Collaborative� (IHI, 2003) where QI teams meet every three months to learn from experts, present results and discuss implementation barriers and solutions. National or regional quality campaigns are a more expansive option. Healthcare providers, health insurance companies, territorial health authorities and the Ministry of Health and Social Protection would work simultaneously on common quality problems, tackling it from multiple angles (e.g. local and regional QI teams, public reporting, changes to policies). A national campaign increases opportunities for shared learning and can build excitement around a specific topic (e.g. patient safety, cancer, pain management, chronic disease management). Multiple concurrent collaboratives in different regions could take place in support of the campaign. Another approach would be to engage clinicians in reducing over-testing and/or over-treatment through a physician-led campaign like the Choosing Wisely initiatives that have been launched in 18 countries around the world. Finally, while collaboratives and campaigns may exist over shorter periods of time, communities of practice can be used to allow individuals interested in a common topic to share ideas on a continuous basis. The Project Echo approach takes this idea further and connects providers in remote sites regularly with central experts to help them troubleshoot difficult clinical cases or management issues.† Additionally, it would be important to develop and implement a suite of clinical decision support tools, which support providers in the consistent implementation of clinical best practices. Examples include flowsheets for management of non-communicable diseases (e.g. hypertension, diabetes, ischemic heart disease, COPD); standard order sets for hospital admissions; drug titration protocols; and common *For example, the IHI Improvement Advisor program (www.ihi.org/education/InPersonTraining/improvement- advisor/Pages/default.aspx) and the IDEAS program in Ontario, Canada (www.ideasontario.ca). † For more information visit: www.echo.unm.edu 60 | P a g e discharge planning tools. Developing national versions of these tools can avoid duplication of efforts by healthcare providers. The government could also mandate that electronic medical records vendors embed these tools into their software as a vendor requirement. 3.2. Strengthen the quality ecosystem in the health sector The government should consider updating and expanding the scope of the SOGC to strengthen a culture of quality improvement. The current presidential decree dates from 2006 and includes four pillars as described above: certification, accreditation, audit of local continuous quality improvement plans, and quality measurement. Expanding the scope of the set of instruments aiming at strengthening the culture for quality would be useful and should focus on clinical competence of healthcare professionals and patient engagement and empowerment. Such a culture would include transparency, encouragement to report defects without fear of reprisal, team-based problem solving, and engaging patients in the design of care. Investments in quality training and capacity described above will contribute to this culture, and policies such as mandatory disclosure of critical incidents to families and legal protection of workers who report defects in good faith may contribute to this desired culture. Developing a national policy for quality of care collaboratively with healthcare professionals, patients and health insurance companies would also be useful. An important intervention would consist of reforming the accreditation process to implement it at scale using a more stepwise approach. Facilities could receive a “star rating� from one star (basic standards are satisfied) to five stars (the hospital provides outstanding patient centered care, ensures co-ordination of care, and has good outcomes). Examples of such systems exist in France, Canada and Australia. The accreditation results could be used to inform health insurance companies, local governments and healthcare users for quality of care. A system with gradations would allow sites on their way towards full accreditation to get recognition for their efforts and more specific feedback for improvement. Such an approach could be implemented by local governments or in a coordinated effort by health insurers so that it avoids conflicts of interest and duplicated efforts. The government of Colombia could also consider setting up an arms-length agency for quality that would function as an accelerator for the improvement of the quality of health care. Many countries have created government arms-length agencies responsible for quality of care to achieve various policy objectives such as: reporting independently to the public or to parliament on quality of care (e.g. Bureau of Health Information in New South Wales, Australia), Agency for Health Research on Quality in the United States or Canada (Health Quality Ontario) but also augmenting capacity for QI or advising government on best use of evidence in decision-making to improve quality of care, such as quality-based payment systems (Health Quality Ontario, Canada). A second priority would be to invest in improving the clinical competence of healthcare professionals before and after graduation. First, there should be a standard method to evaluate the skills of health professionals, ideally using methods of structured observations of clinical examinations where candidates are observed examining a patient actor and coming up with a diagnosis and treatment. This could be applied to new graduates, foreign graduates and for retesting existing doctors and nurses. Clearer 61 | P a g e common standards for medical school curricula could also be developed, which specify the competencies that each specialist or care provider must demonstrate. Continuing education activities could be strengthened beyond the traditional model of lectures, employing tools such as clinical vignettes (case studies) delivered on electronic devices or online*, or telemedicine-based clinical mentorship programs such as Project Hope†. Topics for skills development initiatives should include priority conditions (e.g. non-communicable diseases), quality improvement skills and culture as described above, and patient- centered care (respectful care, good communication and counseling, culturally competent care). A strategy for reducing regional disparities in supply of healthcare professionals is needed, with an emphasis on expanding health professional education in rural areas. As noted in section 2.6 on inputs, there are wide variations in the supply of health providers, and the current system of mandatory service in rural areas for new graduates is ineffective as most do not stay afterwards. One of the most effective strategies for reducing such disparities is rural education, at both the undergraduate and postgraduate (i.e. residency training) levels (O'Sullivan, et al., 2018). Examples of rural medical schools include the James Cook University Medical School in Australia and Northern Ontario School of Medicine in Canada. Such programs prepare students for the rigors of practicing in areas with little specialist backup and draw students with rural backgrounds who are more likely to stay in rural areas. Offering to rural medical doctors the opportunity for faculty appointments and research activities may also contribute to higher retention rates. 3.3 Improve the rigour and use of information on quality of care Colombia has a wealth of information on quality of care, but often performance indicators are not timely, not tailored to the needs of specific information users and of uneven quality, leading to limited use of data for quality assessment, benchmarking and quality improvement. Worse, the lack of guidance in interpreting data on healthcare quality limits the ability of patients to choose their healthcare insurer or healthcare provider. This is a problem in a managed-competition system which depends, in theory, on public availability of quality data to support user choice of insurer and provider and spur quality-driven competition. Further effort is needed, therefore, to improve the rigour and use of information on the quality and outcomes of care. Patient choice should be enhanced by providing patients with localized, pertinent, accessible information on quality of care they encounter (e.g. provider star rating system, rankings of providers). A key recommendation of the landmark 2018 Lancet Commission on high quality health systems (Kruk, Gage, Arsenault, & Jordan, 2018) is to ignite demand to ensure that citizens are aware of current quality gaps and insist on better services from their care providers and insurers. Citizens can exercise this choice, for example, when they select an insurance company or primary care provider for the first time, or choose where to obtain elective surgery, maternity care or other specialized services. Citizens will need easy-to- interpret public reports, such as the system developed in Tanzania (Yahya & Mohamed, 2018) and other countries where complex information about quality is synthesized into a simple five-star rating. The * For example, see www.humandx.org. † Visit projecthope.org. 62 | P a g e Ministry of Health and Social Protection is already developing such a system for public hospitals, which is a positive development. Mobile apps could be developed with associations of patients and users, to improve user-friendliness of this type of information. Availability of this type of information could also empower patient advisory groups and advocates to engage in conversations with their providers, system planners and policy makers about quality of care, which in turn will help build trust, resilience and sustainability in the sector. The next generation of Colombia’s publicly reported quality measurement system should include quality scorecards at every level of the system. In such a system, each entity – health insurance companies, healthcare providers, health professionals, departmental health authorities – would have a scorecard on key quality measures spanning multiple dimensions of quality (e.g. outcomes and clinical processes, patient centredness, wait times, safety). The scorecard would have a level of detail appropriate to the entity; for example, front-line healthcare providers need indicators with micro-level clinical detail, while territorial entities will need to track broader population outcomes. Scorecards would be aligned between entities of the same peer group (e.g. all tertiary hospitals would have the same scorecard). Targets for improvement would be embedded in the scorecard, and the national quality measurement system would allow the government to verify whether all the targets, if achieved, would lead to successful implementation of national goals such as stated in the National Development Plan and Ten-Year Public Health Plan. Implementation of this system will require a consensus on the appropriate indicators to measure, as well as an accepted feedback system that encourages open, transparent comparisons on a regular basis. Further, benchmarks for quality results should be developed to enhance the capacity of healthcare providers (e.g. public hospitals) to compare quality results and learn from each other. Benchmarks identify the level of a specific indicator that would be reasonable to aim for, based on factors such as the best performance achieved to date, or a theoretical best. Funding and staff for research may be required to identify which organizations have the best performance, verify data quality, ensure that good results were due to good quality care and assess to what extent the level of quality is achievable elsewhere. Benchmarks can be used to influence the local targets set by health providers. Colombia should also continue and extend its participation in international benchmarking efforts for health system quality, such as the OECD Health Care Quality Indicators project or the Primary Health Care Performance Initiative. Benchmarking itself internationally will be another mechanism to strengthen Colombia’s quality governance and capacity for improvement, drawing from the Canadian Institute for Health Information (2013), United Kingdom, Nordic countries, the Netherlands and USA. Colombia should establish new quality indicators, particularly for care integration and inequities. These indicators are will be needed in the recommendations below for stronger accountability and contracting mechanisms for integration. There are multiple options to consider based on examples used in other countries, such as: readmission rates for specific conditions*; continuity of care; patient’s perceived level of coordination between the primary care doctor and specialist; patient’s perception of time wasted due * Currently, readmission rates are reported for all diagnoses; however, this is uninterpretable as rates could differ depending on the types of diagnoses seen at a hospital. 63 | P a g e to unnecessary travel between providers; timeliness of key documents for transitions of care (e.g. hospital discharge summaries); and use of discharge plans. For equity, Colombia may consider indicators such as the gap between most and least disadvantaged group in society (e.g. remote rural vs urban, or high vs low income), calculated for a small number of high priority indicators. Finally, Colombia should put substantial effort in improving data currency (with quarterly to real time data being the norm), data quality through regular audits and better dissemination strategies for various information users including the public. Real-time data is particularly important for front-line providers working with micro-level indicators who need instant feedback on whether the ideas for improvement they are testing are making a difference. The system of data quality audits for the High Cost Account program has been well-received by stakeholders and could be extended to other data sources. Information on how to interpret data and what actions to take should be provided for the reporting platforms mentioned above: the star-rating system for citizens to select insurers or providers; and the common quality dashboards at each level of the system. For example, local communities or patient advisory groups could be armed with questions to ask to managers about what they are doing to improve patient experience, safety or wait times. Managers and care providers could be equipped with practical ideas on how to improve key indicators in their scorecards. On patient experience, for example, such ideas could include customer service training; alternatives to medical jargon; use of techniques such as “teach-back� to verify that patients understand instructions; or more flexible visitation policies. 3.4. Develop and implement a primary health care-focused integrated care model to support the management of individuals with complex, chronic health care needs Adequate service delivery design is an important pre-condition to deliver quality services that meet the evolving health needs of the population. To adapt to the increasing prevalence of non-communicable diseases, care must be proactive, continuous, coordinated and personalized for people with complex health care needs. At the same time, high-quality episodic care must remain available for individuals with acute health care needs, such as injuries or short-lived infections. Providing this type of flexible and responsive care requires an integrated delivery model, with high-performing primary health care at the centre. Action is therefore needed to develop and roll out a new integrated care delivery model while strengthening primary health care. Colombia can choose from a range of options based on international evidence and experience to design an integrated care model that offers patients good coordination, communication, convenience and continuity. Recent guidelines from the WHO Europe office (2016) emphasize the importance of creating alignment and collaboration across multiple healthcare services, providers and settings that patients encounter. Specific options to consider in the design of an integrated model include: creating integrated care pathways for individuals with chronic conditions (Utens, et al., 2012); embedding visiting specialists in primary care settings; interdisciplinary case management for complex patients (Dieterich, et al., 2017); individualized care planning; use of patient navigators; telemedicine or e-consult systems to provide primary care providers with greater access to specialists 64 | P a g e (Liddy , et al., 2016); incentives to co-locate different services (e.g. labs, imaging, rehabilitation, etc.) to minimize patient travel; use of home monitoring devices with two-way communication with providers (Clarke, et al., 2011); and shared information systems accessible to all members of an interdisciplinary team. It is also advisable to expand the scope of the high cost account program (Cuenta de Alto Costo) to other chronic conditions like cancer, COPD, mental illness, other rheumatic diseases and multi-morbidities, as well as to index episodic conditions (e.g. myocardial infarction, stroke or appendicitis); and increase the emphasis on outcomes-based accountability and not just process compliance. Putting the focus on outcomes creates incentives for integrated care and for healthcare providers to develop innovative models of integrated care to increase the likelihood of achieving better health outcomes. It also requires refined mechanisms for risk adjustment of payment to incent for care integration, quality and safety, and avoid cream skimming by providers. Integrated care for chronic conditions and cancer must be paralleled by changes in payment mechanisms, towards paying for expected services to be provided to a given population but adjusted for health outcomes achieved and adding a risk adjustment method. Paying for care bundles including for complex populations should be tested to encourage a holistic approach to care integration and distribute risks more fairly across providers. A similar option would be to provide incentives for integrated care, such as exchange of information between primary care and specialists; providing tests, imaging and other services in a single place; or case management of complex cases. Finally, scale up of self-management support (SMS) approaches should be encouraged for patients with chronic conditions. Self-management support is the care and encouragement provided to people with chronic conditions and their families to help them understand their central role in managing their illness, make informed decisions about care, and engage in healthy behaviors. Health care educators with training in SMS act as a coach to patients to help them set small but doable goals for improvement, which gradually builds up the patient’s confidence to make more substantive changes. Implementation of such a program would require training personnel in primary health care sites and adopting a curriculum, which can be based on well-established international programs.* Consideration should also be given to more recent tools for self-management provided through digital health platforms, including on-line tools, reminders and interactions with providers and telehealth monitoring (Cahn, Akirov, & Raz, 2018). 3.5. Improve accountability & contracting mechanisms to incent for quality in the health sector A first recommendation would be to ignite demand for quality care by enhancing patient participation and engagement in local governance and accountability processes. Patient empowerment and patient engagement is key to design, plan and manage services in a way that is person-centered, better meets the expectations of patients, families and communities and ensures better sustainability of the sector. As argued above, a priority would be to provide better choice to system users by improving access to relevant, localized, current information on quality of care. In addition, investments could be made to better inform * For example, www.selfmanagementresource.com 65 | P a g e community representatives about standards of quality of care that should be expected, and how to engage with local health care providers on quality of care, as was done in Guatemala with the example of healthcare defenders*. In addition, engaging with patients in local accountability mechanisms has proven important in the implementation of strong primary health care models: communities with stronger local governance mechanisms and patient engagement were able to reduce avoidable mortality faster in Brazil compared to communities with weaker local governance (Hone, et al., 2017). Other potential interventions to augment demand for quality of care include: a national commitment to a quality guarantee; charters of rights for patients; expanding opportunities for recourse, complaints and participation in local governance systems. Accountability mechanisms to hold health insurance companies responsible for quality and related incentives should be developed and implemented at scale. Currently Colombia has developed but has not widely implemented a system of health insurance company licensing that includes entry requirements of financial solvency. Additional standards could be considered, including standards for claims management; prompt payment to providers; fraud management; and maintenance of financial reserves and avoidance of deficits in any given year. The Health Superintendence (Supersalud) is responsible for overseeing health insurance companies. A clearer process is required for dealing with insurers that become fiscally insolvent, which identifies the root causes of insolvency (fiscal mismanagement, fraud, poor disease management, or capitation rates which are too low for the case-mix severity of patients) and implements the appropriate response to the root cause (e.g. replace management, fraud prevention procedures, disease management programs, or payment adjustments). Colombia is also developing an accreditation process for health insurance companies, and accreditation is generally considered as a set of standards above the minimum bar set by licensing standards. These standards could examine the extent to which an insurer acts like a “managed care� organization, which has data collection, information systems to track and deter inappropriate use; encourage health promotion activities; and maximize secondary prevention for patients with non-communicable diseases like diabetes and hypertension. Payment and contracting mechanisms between the Ministry of Health and health insurers also need to be strengthened. First, it is important to set an appropriate payment to insurers who take on high-cost patients, to avoid situations where insurers are under-resourced to handle them, or where insurers with good quality are penalized financially for attracting high-cost patients. At the same time, there must be incentives to prevent patients from developing high-cost conditions such as renal failure, through prevention and good chronic disease management. Several options exist to help achieve this balance. First, risk adjustment mechanisms could include the burden of high cost illnesses to define the per-capita allocation to health insurers from the ADRES for new subscribers to an insurer. In addition, ex-post risk adjustment mechanisms for capitation payment (UPC) for high cost patients or targeted groups of patients such as cancer patients should be expanded for health insurance companies, using the mechanism for high cost patients accounts. Also, when a patient develops a potentially avoidable complication, there could be a delay of when an insurer will receive an increase in per-capita payment for higher case-mix, to create an incentive to prevent such complications from occurring. Further, there could be a common, minimum set of guidelines for payment for good performance on quality indicators for all providers. This * See website of the Procurator of Human Rights, Health Defender: https://www.pdh.org.gt/derecho-a-la-salud/ 66 | P a g e would reduce the administrative burden of providers having to deal with multiple different indicators and targets from different insurers. An emphasis could be placed on certain process measures of care, for which providers are clearly responsible, as well as indicators for coordination and communication between providers. The choice of such standardized measures could be linked to key objectives in the National Development Plan. There could also be clearer guidelines on what interventions a health insurer should undertake if quality is suboptimal. Lastly, in situations where a public provider has a monopoly, a clearer separation of the ownership and oversight function of regional entities will need to be considered, to avoid conflicts of interest. If a monopoly provider consistently performs below expectations even after investments in building quality improvement capacity, guidelines could be established to encourage private sector competitors or mixed (public-private) models of care delivery. 67 | P a g e Table 11. Recap table recommendations Colombia assessment study on quality of care Recommendations Resources Difficulty of Potential Time Horizon Main quality requirements execution Impact for Impact Dimensions (+ to +++) (+ to +++) (+ to +++) (Short, Mid or Affected Long Term) Strengthen capacity for quality improvement in the health sector mm. 1. Strengthen system capacity for continuous quality ++ ++ +++ Short Term Effectiveness improvement (QI) and learning & launch national QI campaigns Efficiency for specific conditions (e.g. cancer) Patient safety Strengthen the quality ecosystem in the health sector nn. 2. Update and expand the scope of the national system of + ++ ++ Short Term Patient safety mandatory guarantee for quality of care Effectiveness Person- centeredness oo. 3. Improve pre-service and continuous education for clinicians ++ +++ +++ Mid Term Effectiveness with a focus on improving clinical competence and QI skills Patient safety Improve the rigour and use of information on quality of care 4. Augment choice of patients by providing them with localized, + + ++ Mid Term Person- pertinent information on quality centeredness Equity e. 5. Develop transparent mechanisms to benchmark quality of ++ ++ +++ Short Term Effectiveness providers and insurance companies Timeliness Equity f. 6. Improve data currency and data quality and better disseminate +++ +++ +++ Mid Term Effectiveness quality, timely information to healthcare providers, insurance Care integration companies and the public Timeliness Develop and implement a primary health care focused integrated care model to support management of individuals with complex, chronic health care needs pp. 7. Develop and roll out a new integrated care delivery model +++ +++ +++ Long Term Care integration while strengthening primary health care Person- centeredness Equity qq. 8. Expand the scope of the High Cost Account program to other ++ ++ +++ Short Term Care integration chronic diseases and scale up self-management support Person- mechanisms for patients with chronic conditions centeredness Improve accountability and contracting mechanisms to incent for quality rr. 9. Enhance patient participation and engagement in system + ++ +++ Mid Term Effectiveness planning and accountability processes Care integration Timeliness ss. 10. Strengthen outcomes-focused accountability mechanisms ++ ++ +++ Mid Term Effectiveness through improved regulation, payment systems and incentives Efficiency such as ex-post adjustment mechanisms for capitation payments Equity Person- centeredness 68 | P a g e Table 12. Implications of Recommendations for System Stakeholders: Who is Involved Recommendations Government Department Health Healthcare Academic Patients, level health insurance providers centers citizens and authorities companies their representative groups Strengthen capacity for quality improvement in the health sector tt. 1. Strengthen system capacity for continuous quality FS M or P M P M or P I improvement (QI) and learning & launch national QI campaigns for specific conditions (e.g. cancer) Strengthen the quality ecosystem in the health sector uu. 2. Update and expand the scope of the national FSM M P P I I system of mandatory guarantee for quality of care vv. 3. Improve pre-service and continuous education for FS P P P M I clinicians with a focus on improving clinical competence and QI skills, and building rural training capacity Improve the rigour and use of information on quality of care 4. Augment choice of patients by providing them with FSM P P P I C localized, pertinent information on quality g. 5. Develop transparent mechanisms to benchmark FSM P P P P I quality of providers and insurance companies h. 6. Improve data currency and data quality and better FSM P P P I I disseminate quality, timely information to healthcare providers, insurance companies and the public Develop and implement a primary health care focused integrated care model to support management of individuals with complex, chronic health care needs 7. Develop and roll out a new integrated care FS M or P M M or P I P delivery model while strengthening primary health care 8. Expand the scope of the High Cost Account FSM P P P I P program to other chronic diseases and scale up self- management support mechanisms for patients with chronic conditions Improve accountability and contracting mechanisms to incent for quality 9. Enhance patient & citizen participation and FS M M P I P engagement in system planning and accountability processes 10. Strengthen outcomes-focused accountability FSM P P P I or P I mechanisms through improved regulation, payment systems and incentives such as ex-post adjustment mechanisms for capitation payments F: provide funding or financial incentives S: set strategic direction, standards, policies M: manage program P: participate in program or support implementation I: be informed about activity C: make informed or strategic choices 69 | P a g e Acknowledgements This report was written by a team of World Bank Group staff and consultants led by Jeremy Veillard (World Bank). The team was composed of: Ben Chan (methodological lead), Manuela Villar Uribe, Ian Forde, Ramon Abel Castaño, Lenis Urquijo, Alexandra Porras, Roberto Iunes (World Bank) and Andrew Myburgh, Anais Furia and Iuliia Khalimova (IFC). Dr. Don Berwick, President Emeritus of the Institute for Healthcare Improvement, was senior advisor to the World Bank Group, providing input into all aspects of the report. The report on the IFC Survey of Quality Assurance Standards in Selected Colombian Hospitals (Appendix 3) was written by IFC consultant Iuliia Khalimova. Logistic support for the report, mission, and organization of site visits was provided by Juan Pablo Toro Roa and Ana Maria Lara. All individuals listed above participated in the missions, key informant interviews and site visits. The EPS survey was administered by Lenis Urquijo and Alexandra Porras. Additional support from IFC was provided by Elena Sterlin (Senior Manager, Head of Health and Education), Issa Faye (Direct of Sector Economics), Deepa Chakrapani (Manager, Sector Economics), and Tania Lozansky (Senior Manager, MAS Advisory). The Quality Assessment Tool was developed by a World Bank & IFC joint team composed of Ben Chan (methodological lead); Jeremy Veillard; Andrew Myburgh; and Manuela Villar Uribe. The team would like to thank the leadership of the Ministry of Health and Social Protection of Colombia for their insights and feedback into the development of the report: Dr. Juan Pablo Uribe Restrepo, Minister of Health and Social Protection; Dr. Iván Darío González Ortiz & Diana Isabel Cardenas Gamboa, Vice Ministers at the Ministry of Health and Social Protection; and Dr. Olga Lucia Giraldo Velez, Director, Quality Office and focal point of the Ministry of Health and Social Protection for this study. The team thanks Dr. Teresa Tono for extensive comments on the application of the Quality Assessment Tool in Colombia. The team also thanks all participants in interviews, site visits and surveys listed in Appendix 1 for their contributions. 70 | P a g e Appendix 1: Interviews, Site Visits and EPS Survey A1.1 List of Quality Standards There are eight domains in the quality assessment tool, and multiple assessment criteria for which a score of 1 to 4 is given, depending on the extent to which all desired aspects of the criteria are met. The scores for a criterion, in turn, are based on several standards, listed and numbered below. Domain 1: Quality Strategy and Planning Criteria 1a: There is a national strategy document for quality. 1.1 There is a national plan to improve health care quality. 1.2 The plan is up-to-date (i.e. currently in force). 1.3 There are clearly defined priorities for improvement at national level. 1.4 The plan has baseline data on key measures, used to assess progress. 1.5 There are clearly defined numeric targets for improvement compared to baseline, with time frame for completion. Criteria 1b: The strategy is fully cascaded. 1.6 There is a cascade of indicators at different levels of the system, with sub-targets for improvement linked to high-level targets. Sub-targets may be at the regional, facility, team, or provider group level. 1.7 The plan has considered any gaps in information about quality and includes activities to invest in improved data and reporting. 1.8 Improvements in physical infrastructure to support different components of the plan have been identified for each sub-target. 1.9 Improvements in staff (increase in number and/or skills of staff, training, recruitment, etc.) to support the plan to have been identified, for each sub-target. 1.10 The plan has considered any changes in policies or regulations necessary to support improvements. 1.11 Other improvement activities have been clearly identified, to support each sub-target. 1.12 Funding and other resources have been allocated to different improvement activities, staff enhancements and infrastructure. Criteria 1c: There is stakeholder engagement in the plan. 1.13 There are persons, units, departments or other entities clearly assigned as being responsible for implementing the activities defined in the four rows above, and responsible for reaching the sub- target. 1.14 The plan specifies related stakeholder engagement activities & a communications plan. Criteria 1d: The Strategy is actively managed 1.15 An implementation/oversight/project management team has been identified to oversee the quality plan. 1.16 The plan identifies other committee structures or consultation bodies needed to oversee subcomponents of the plan. 1.17 There is a process for reviewing progress towards targets & corrective actions. 71 | P a g e Domain 2: Quality Definition, Guidelines & Standards Criteria 2a: There is a nationally recognized definition of quality. 2.1 A quality definition exists. 2.2 The definition includes key dimensions of quality (e.g. safety, patient-centredness, effectiveness) although it may use different terminology. 2.3 The quality definition is officially approved by the government or designated agency. Criteria 2b: Clinical practice guidelines exist. 2.4 CPGs and are up to date – antenatal care. 2.5 – same, for childhood conditions under age 5. 2.6 – HIV 2.7 – TB, malaria, other infectious diseases 2.8 – NCDs – hypertension, diabetes 2.9 – NCDs – cancer 2.10 – NCDs – mental health /addictions 2.11 – NCDs – other conditions 2.12 – Hospital – safe medication practices 2.13 – Hospital – infection control 2.14 – Hospital – management of common conditions (e.g. acute coronary syndrome, heart failure) 2.15 – Hospital – general safety (e.g. prevention of falls, ulcers, venous thromboembolism) 2.16 – Hospital – discharge & follow-up 2.17 – Hospital – obstetrical care Criteria 2c: Standards for physical infrastructure exist 2.18 Basic standards (water, electricity, toilets) 2.19 Waste disposal standards 2.20 Management & internal governance standards 2.21 Standards for IT resources 2.22 Standards for staffing level for health facilities 2.23 Standards for supply chain management – drugs, supplies Criteria 1d: Standards for supply, distribution of human resources exist 2.24 Planning standards exist for doctors 2.25 Planning standards exist for nurses 2.26 Planning standards take into account age, gender, disease prevalence, socioeconomic status, other special need Criteria 2e: Planning standards for supply, distribution of health facilities exist 2.27 Planning standards for hospitals 2.28 Planning standards for lab and diagnostic imaging 2.29 Planning standards for primary care sites 72 | P a g e Domain 3: Quality Measurement & Reporting Criteria 3a: Data collection system and quality indicators exist. 3.1 An organization is responsible for setting data standards. 3.2 An organization is responsible for managing quality indicators. 3.3 Clinical data are routinely collected within the country. 3.4 There are information systems to archive routinely collected data. 3.5 Indicators exist for: antenatal care; obstetrics; childhood growth & monitoring; immunizations; childhood conditions; family planning; HIV; TB; malaria; non-communicable diseases; wait times; patient experience; hospital adverse events & nosocomial infections; readmissions; hospital mortality Criteria 3b: Data quality assurance mechanisms established. 3.6 An organization is responsible for verifying data quality. 3.7 There are basic checks for completeness of data, obvious data entry errors. 3.8 There are advanced data quality audit mechanisms (e.g. chart audits, patient interviews, etc.). Criteria 3c: Strong infrastructure for reporting & disseminating data exists 3.9 Data are fed back to individual institutions 3.10 Data is fed back in a timely fashion (ideally real-time) 3.11 Comparisons are available between peer organizations 3.12 Data are made available to the public 3.13 Data are easily accessible to the public (e.g. easy to find, read; evidence of its use) 3.14 Information is detailed enough to support quality improvement by clinicians Criteria 3d: Strong analytical capacity exists 3.15 Organizations responsible for analysis have advanced statistical expertise 3.16 Evidence of sophisticated analytical techniques Criteria 3e: Strong knowledge exchange mechanisms exist 3.17 Targets for indicators clearly specified 3.18 Actions for improvement clearly specified 3.19 Information on targets and actions to take are broadly disseminated 3.20 Training on how to use info for improvement is available 73 | P a g e Domain 4: Quality Improvement Capacity & Activities Criteria 4a: There is a critical mass of staff expertise in quality management in the system 4.1 Facilities have designated staff in charge of quality. 4.2 Hospitals have staff with high-level QI skills (e.g. international certification, etc.) 4.3 Primary care sites have access to staff or coaches with high-level QI skills Criteria 4b: Formal quality structures, committees, teams exist. 4.4 Primary care sites have quality committees 4.5 Hospitals have quality committees, infection control, etc. 4.6 Governing boards of institutions have quality dashboards for regular review Criteria 4c: Demonstrated use of QI methods (process maps, PDSA cycles, etc.) 4.7 Hospitals have QI projects that have demonstrated improvements in outcomes. 4.8 Hospitals have QI projects that have demonstrated use of core QI tools (e.g. process maps, PDSAs) 4.9 Primary care sites have QI projects that have demonstrated improvements in outcomes. 4.10 Primary care sites have QI projects that have demonstrated use of core QI tools (e.g. process maps, PDSAs) Criteria 4d: National campaigns and other methods to share learnings 4.11 There are examples of QI collaboratives/campaigns aimed at sharing experiences in implementing QI. 4.12 There are examples of national campaigns for quality. 4.13 There are other forums for exchanging experiences (e.g. communities of practice, Project Echo) Criteria 4e: Broad participation in a strong accreditation program 4.14 Accreditation standards are internationally accredited. 4.15 Accrediting organization is independent. 4.16 Standards include clinically relevant practices e.g. related to patient safety, patient experience, patient flow. 4.17 There is strong participation in accreditation. Criteria 4f: Widespread use of decision support tools 4.18 WHO surgical checklist is used. 4.19 Evidence of use of protocols, pathways, algorithms for emergencies, major clinical conditions. 4.20 IT systems can generate reminders, warnings (e.g. of drug interactions). 4.21 Hospitals use standard order sets. 4.22 Primary care sites use flowsheets or similar instruments. 74 | P a g e Domain 5: Inputs – Staff, Facilities, Equipment, Supplies Criteria 5a: Supply of Health Professionals 5.1 Supply of physicians consistent with benchmark for peer countries. 5.2 Supply of nurses consistent with benchmark for peer countries. Criteria 5b: Regional variations in health professional supply 5.3 Variations in supply between regions minimized (avoid having regions with < 50% of median supply). Criteria 5c: Human resource management adequate (i.e. maintenance of clinical competency, workload, turnover) 5.4 Reliable methods to measure and ensure clinical competence. 5.5 General human resource practices to avoid burnout, turnover. Criteria 5d: Availability of Drugs 5.6 Guarantees of access to drugs in place for low-income populations. 5.7 Drugs well stocked in hospitals and primary care settings. Criteria 2e: Hospital bed capacity 5.8 Supply of hospital beds consistent with benchmark for peer countries. Criteria 2f: Model of care 5.9 Good continuity of primary care for patients (ability to see same provider). 5.10 Good coordination of referrals & care between primary care, specialists. 5.11 Good coordination of labs, imaging, special services. 5.12 Good coordination at transitions (e.g. hospital admission, discharge). 5.13 Sharing of health information between providers. 75 | P a g e Domain 6: Accountability & Governance The desired impact of accountability mechanisms is measured across a matrix which considers seven dimensions of quality and five different mechanisms. In total, there are 35 combinations of mechanism and dimension. Criteria Accountability Mechanism Dimension of Quality 6.1.1 Licensing / Inspection Safe 6.1.2 Licensing / Inspection Effective 6.1.3 Licensing / Inspection Patient-centred 6.1.4 Licensing / Inspection Timely 6.1.5 Licensing / Inspection Efficient 6.1.6 Licensing / Inspection Equitable 6.1.7 Licensing / Inspection Integrated 6.2.1 Accreditation Safe 6.2.2 Accreditation Effective 6.2.3 Accreditation Patient-centred 6.2.4 Accreditation Timely 6.2.5 Accreditation Efficient 6.2.6 Accreditation Equitable 6.2.7 Accreditation Integrated 6.3.1 Public reporting Safe 6.3.2 Public reporting Effective 6.3.3 Public reporting Patient-centred 6.3.4 Public reporting Timely 6.3.5 Public reporting Efficient 6.3.6 Public reporting Equitable 6.3.7 Public reporting Integrated 6.4.1 Patient & community accountability Safe 6.4.2 Patient & community accountability Effective 6.4.3 Patient & community accountability Patient-centred 6.4.4 Patient & community accountability Timely 6.4.5 Patient & community accountability Efficient 6.4.6 Patient & community accountability Equitable 6.4.7 Patient & community accountability Integrated 6.5.1 Contracts with insurers Safe 6.5.2 Contracts with insurers Effective 6.5.3 Contracts with insurers Patient-centred 6.5.4 Contracts with insurers Timely 6.5.5 Contracts with insurers Efficient 6.5.6 Contracts with insurers Equitable 6.5.7 Contracts with insurers Integrated 76 | P a g e For each unique combination of accountability mechanism and quality dimension, the following criteria are used to assess the strength of the accountability relationship. Criteria a. Criteria exist, expectations are clear b. Performance is accurately measured, free from bias c. Appropriate feedback given d. There is an incentive to respond to feedback, concerns e. Avoid duplication f. Program is applied consistently g. Standards are not overly onerous h. Accountability extends to all in target group Criteria a-d measure effectiveness of the accountability mechanism; e-g measure efficiently; h measures reach. 77 | P a g e Domain 7: Policies & Incentives to Support Quality Criteria Criteria 7a: Quality of care is central to Universal Health Coverage policies 7.1 National policies to reach and ensure universal health coverage include quality of care within definition of coverage 7.2 The vast majority of the population has health insurance. 7.3 Co-payments are minimal compared to benchmark countries. 7.4 Citizens have coverage for services in multiple sectors (hospital, primary care, prevention, rehabilitation) all major clinical conditions. 7.5 The plan excludes services deemed of no benefit, according to the literature. Criteria 7b: Patient rights legislation and patient empowerment 7.6 The right to health is defined in legislation 7.7 Legal pathways are defined to ensure patient ability to demand quality of health care 7.8 Policies exist on confidentiality of info 7.9 Policies exist on informed consent 7.10 Policies exist on patients right to access their medical record 7.11 Policies exist on mandatory investigation of critical incidents Criteria 7c: Quality of care policies are aligned 7.12 National health policies and plans integrate quality of care in their definitions and actions. 7.13 Quality of care policies at the national level (i.e. National Health Policy/Plan, National Development Plan, National Quality Policy/Plan/System) are aligned in their definition of Quality 7.14 Quality of care policies and plans at the national level include same or similar definitions of quality of care as those defined by subnational or sub-system levels. Criteria 7d: Quality of care policies are comprehensive 7.15 Quality of care policies define expectations for health care effectiveness, safety, person- centeredness, timeliness, equity, health care integration and efficiency 7.16 Health care policies define and set the basis for the development of adequate tools to ensure (through measurement and supervision) the assessment of the comprehensive definition of quality of care Criteria 7e: Roles and responsibilities for implementation of quality of care policies are clearly defined 7.17 Quality of care policies define responsibilities to system actors for the implementation of quality of health care delivery actions defined within policies 7.18 Quality of care policies define responsibilities to system actors for development of measurement tools to assess quality of health care delivery 7.19 Quality of care policies define responsibilities to system actors for the assess quality of health care delivery using defined tools 7.20 Quality of care policies define responsibilities to system actors reporting results of assessment of quality of health care delivery 78 | P a g e Domain 8: Quality Results Criteria 8a: Patient Experience 8.1. Global patient experience compares favourably to international benchmarks. 8.2. Specific patient experience indicators (e.g. communication, courtesy, shared decision-making, etc.) compare favourably to benchmarks. Criteria 8b: Effectiveness – Prenatal Care 8.3. Process indicators (e.g. ANC4, first prenatal visit) compare favourably to benchmarks. 8.4. Outcomes indicators (e.g. MMR) compare favourably to benchmarks. Criteria 8c: Effectiveness – Childhood Care 8.5. Process indicators (e.g. immunizations, treatment of IMCI conditions) compare favourably to benchmarks. 8.6. Outcome indicators (e.g. U5 mortality) compare favourably to benchmarks. Criteria 8d: Effectiveness – Infectious Diseases 8.6. Process indicators (e.g. TB, HIV treatment plan adherence) compare favourably to benchmarks. 8.7. Outcome indicators (e.g. TB, HIV mortality or other indicators) compare favourably to benchmarks. Criteria 8e: Effectiveness – NCDs 8.8. Process indicators (e.g. regular follow-up, testing) compare favourably to benchmarks. 8.9. Outcome indicators (e.g. blood sugar, blood pressure control, overall mortality rates) compare favourably to benchmarks. Criteria 8f: Safety 8.10. Process measures (e.g. hand hygiene, use of WHO checklist, VTE prophylaxis, patient risk assessments) compare favourably. 8.11. Adverse event rates (e.g. nosocomial infections, avoidable complications, etc.) compare favourably to benchmarks. Criteria 8g: Timeliness 8.12. Wait times for different services compare favourably to international benchmarks, yet remain a concern in the context of Colombia Criteria 8g: Equity 8.13. Gaps between urban and rural, low and high income minimized, for selected outcomes in above areas compare favourably to benchmarks. Criteria 8h: Efficiency 8.14. Information on any available efficiency indicators (e.g. unnecessary length of stay, inappropriate use of services, etc.) compares favourably to benchmarks. Criteria 8i: Integrated 8.15. Indicators of good integration (e.g. patient-reported smooth transitions, coordination of care) compare favourably to benchmarks. Note: In some instances, limited data were available for standards within a criterion, and the lowest possible score was assigned to the criteria. 79 | P a g e A1.2 Key Informant Interviews Table A1.2 List of Key Informant Interviews Name Position Organization or Department Ministry of Health and Social Protection Dr. Juan Pablo Uribe Restrepo Minister (MHSP) Ministry of Health and Social Protection Dr. Iván Darío González Ortiz Vice Minister (MHSP) Directorate for the Development of Dr. Luis Gabriel Bernal Pulido Director Human Talent in Health, MHSP Directorate of Provision of Services and Dr. Samuel García de Vargas Director Primary Care, MHSP Directorate of Epidemiology and Dr. Sandra Lorena Girón Vargas Director Demography, MHSP Dr. Olga Lucia Giraldo Velez Chief Quality Office, MHSP Dr. Wilson Melo Velandia Chief Planning Office, MHSP Dr. Dolly Esperanza Ovalle Office of Information and Communication Chief Carranza Technology, MHSP Directorate of de Promotion y Prevention, Dr. Harold Mauricio Casas Cruz Subdirector MHSP Subdirectorate of Infrastructure in Health, Dr. Augusto Ardila Subdirector MHSP Dr. William Jiménez Herrera Coordinator Planning group, MHSP Office of Information and Communication Ing. Patricia Delgado Rodriguez Coordinator Technology, MHSP Knowledge Management and Information Dr. Angélica Molina Rivera Coordinator Sources Group, MHSP Group of Management of Knowledge and Dr. Rafael Borda Rivas Coordinator Information of Human Talent in Health, MHSP Office of Information and Communication Ing. Luz Emilse Rincón Medrano Specialist Technology, MHSP Dr. Giselle Arias Leon Specialist Planning group, MHSP Dr. Javier Bohorquez Gelvez Specialist Planning group, MHSP Office of Monitoring Methodologies and Dr. Daniel Pinzón Fonseca Chief Risk Analysis, Supersalud 80 | P a g e Name Position Organization or Department Directorate for the Supervision of Dr. Manuel Godoy Cubillos Director Economic Risks, Supersalud Director of Social Dr. Lydia Carolina Suarez Vargas National Planning Department Development Institute of Technological Evaluation in Dr. Adriana María Robayo García Director Health, IETS Institute of Technological Evaluation in Dr. Carlos Pinzón Florez Technical Subdirector Health, IETS Administrative Assistant Institute of Technological Evaluation in Dr. José Luis Gutierrez Noreña Director Health, IETS Dr. Carlos Edgar Rodriguez National Director of The Colombian Institute of Technical Hernández Accreditation in Health Standards and Certification (ICONTEC) Former Minister of Dr. Augusto Galán Sarmiento Así Vamos en Salud Health, Director Former Vice-Minister Cancer Research and Treatment Center Dr. Fernando Ruiz Gómez Scientific Medical (CTIC) Leader Departmental Secretary of Health of Dr. Carlos Mario Montoya Serna Secretary Antioquia Directorate of Quality and Services Dr. Elizabeth Quintero Cardona Director Network of Antioquia Dr. Anabelle Arbelaez Velez Undersecretary District Secretary of Health of Bogotá Technical Director, Departmental Secretary of Health of María Victoria �vila Ramos Health Service Delivery Boyacá Departmental Secretary of Health of Dr. Libia Perilla Valle Specialist Boyacá Dr. Teresa Tono Ramírez Chief Executive Officer Organization for Health Excellence (OES) Insurrance Vice- Colombian Association of Integral Dr. Nelcy Paredes Cubillos president Medicine Companies (ACEMI) Colombian Association of Integral Dr. Fabián Cardona Medina Health Vice-president Medicine Companies (ACEMI) Epidemiology and Public Health Dr. Mónica Zúñiga Nuñez Coordinator Department, ACEMI Association of Health Insurance Dr. Miguel Uprimmny Yepes Health Leader Companies in Colombia (GESTARSALUD) 81 | P a g e Name Position Organization or Department Insurance Department - National Dr. Eileen Patricia Guzman Tovar Coordinator Association of Family Compensation Funds (ASOCAJAS) Dr. Juan Gonzalo Lopez Casas President Salud Total EPS Subdirectorate of Insurance - Dr. Mauricio Serra Tamayo Subdirector COMPENSAR - EPS Dr. Gabriel Mesa Nicholls Chief Executive Officer SURA - EPS Inter-institutional Observatory of Adult Dr. Martha Lucía Gualteros Reyes President Cancer (OICA) Sr. Francisco Marín Board Member ASMETSALUD - EPS 82 | P a g e A1.3 Site Visits Conducted Table A1.3 Site Visits for Quality Assessment Tool Name of Site Type of Site Location Main contact person Hospital Universitario Dr. Orlando Jaramillo Jaramillo, Hospital Bogotá Mayor - MEDERI CEO Specialized Centro de Diagnóstico Dr. Edgar Humberto Cortés Ostos, Diagnostic Bogotá Especializado - Colmédica Manager Center Centro de Atención Prioritaria en Salud CAPS PHC Bogotá Dra. Patricia Lozano San Benito Dra. Sandra Torres, Quality Specialist E.S.E. Hospital Salazar Hospital Villeta Dr. Yair Rocha, Medical leader E.S.E. Hospital General de Dra. Liliana Bermudez, Director of Medellín. Luz Castro de Hospital Medellín Quality and Planning Gutiérrez. Dra. Martha Lucía Vélez, Manager Dra. Adriana Posada Dra. Luisa Gómez Dra. Luz Alzate Dr. Carlos Maldonado E.S.E. Hospital Manuel Uribe PHC Medellín Dra. Sandra Bustamante �ngel Dra. Angélica Calle Dra. Carmen Mora Dra. Marlly Muñoz Dra. Diana Saldarriaga Dra. Mónica Jiménez Dra. María Victoria Restrepo, Chief Hospital Pablo Tobón Uribe Hospital Medellín Quality Office Dra. Yeny Patricia Pineda Dr. Hector Darío Cano Arango, E.S.E. Hospital Santa Director PHC Copacabana, Antioquia Margarita de Copacabana Dra. Carolina Montoya Toro Dr. Omar Alberto López Gómez E.S.E. Hospital San Rafael de Dr. Diego Alfonso Montoya PHC Girardota, Antioquia Girardota Grajales, Director Dra. Maricela Guerrero Tibatá, E.S.E. Centro de Salud Fe y Director PHC Soracá, Boyacá Esperanza, Soracá (Boyacá) Lic. Verónica Fonseca Bohorquez Lic. Elba María Lozano Lozano E.S.E. Centro de Salud de Lic. Maryi Mildred Borda �lvarez, PHC Cómbita, Boyacá Cómbita (Cómbita, Boyacá) Sister Dra. Lyda Marcela Pérez R, Director E.S.E. Hospital San Rafael Hospital Tunja, Boyacá Dra. Sandra Liliana Bello 83 | P a g e Name of Site Type of Site Location Main contact person Dra. Ayda Patricia Medina Dra. Angélica Rojas Dr. Henry Corredor Dr. José Ramón Merchán Ruiz, Director E.S.E Hospital San Vicente Dra. Pilar Torres PHC Paipa, Boyacá de Paul Dra, Rubiela Vargas Dra. Edna Jimena Holguin Dr. Juan Carlos Castellanos Dra Flor Cárdenas Pineda, Manager Dr. Cristian López E.S.E. Hospital Regional de Hospital Duitama, Boyacá Dr. Emerson González Duitama Dr. Fabián Goyeneche Dra. Jeaneth Villate Dra. Claudia García, Manager Dr. Saúl Rodríguez Dra. Andrea Medina E.S.E. Salud del Tundama Hospital Duitama, Boyacá Dra. Lorena López Dra. Paola Garzón Dra. Edna Dueñas Dra. Lina Patarroyo E.S.E. Hospital Simón Hospital Bogotá Dra. María Eugenia Rodríguez Bolívar. Centro de Atención Prioritaria en Salud CAPS PHC Bogotá Dra. María Eugenia Rodríguez SUBA 84 | P a g e A1.4 Documents Reviewed for Quality Assessment Tool Abbreviations MSPS Ministerio de Salud y Protección Ministry of Health and Social Protection Social MPS Ministerio de la Protección Social Ministry of Social Protection DNP Departamento Nacional de National Planning Department Planeación Supersalud Superintendencia Nacional de Salud National Health Superintendent Quality Strategy and Planning National Planning Documents 1. MSPS. (2016). Plan Nacional de Mejoramiento de la Calidad en Salud, 2016 . Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/Plan-nacional-de- mejoramiento-calidad.pdf 2. MSPS. (2013). Plan Decenal de Salud Pública 2012-2021. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VS/ED/PSP/PDSP.pdf 3. MSPS. (2018). Informe de Gestión Avance del Plan Decenal de Salud Pública 2012-2021. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VS/ED/informe-gestion-plan-decenal- salud-publica-2018.pdf 4. MSPS. (2013) Sistema de Monitoreo y Evaluación al Plan Decenal de Salud Pública, 2012 – 2021. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/Documentos%20y%20Publicaciones/Sistema%20de%20Seguimiento%20y% 20Evaluaci%C3%B3n%20del%20Plan%20Decenal%20De%20Salud%20P%C3%BAblica%20-%20PDSP%20Co lombia%202012%20-%202021.pdf 5. DNP. (2019). Plan Nacional de Desarollo. Bogotá: Departamento Nacional de Planeación. Retrieved from www.colaboracion.dnp.gov.co/CDT/Prensa/BasesPND2018-2022n.pdf 6. DNP. (2019). Proyecto de Ley, por la cual se expide el Plan Nacional de Desarrollo 2018 – 2022 “Pacto por Colombia, Pacto por la Equidad�. Bogotá: Departamento Nacional de Planeación. Retrieved from https://colaboracion.dnp.gov.co/CDT/Prensa/Articulado-Plan-Nacional-de-Desarrollo-2018-2022-Pacto- por-Colombia-Pacto-por-la-Equidad.pdf Regional and Local Plans 7. Alcaldía de Santiago de Cali. (2012). Plan de Desarrollo del Municipio de Santiago de Cali. Cali: Alcaldía de Santiago de Cali. Retrieved from http://www.cali.gov.co/planeacion/publicaciones/44418/plan_desarrollo_municipal_2012_2015/ 8. Supersalud. (2016). Circular externa 012 de 2016 Programa de Auditoría para el Mejoramiento de la Calidad de la Atención PAMEC. Bogotá: Superintendencia Nacional de Salud. Retrieved from https://docs.supersalud.gov.co/PortalWeb/Juridica/CircularesExterna/CIRCULAR%20EXT%20000012.pdf. 85 | P a g e Quality Results and Quality Measurement and Reporting sections National Reports 9. MPS. (2009). Informe Nacional de Calidad de la Atención en Salud. Bogotá: Ministerio de la Protección Social. Retrieved from http://calisaludable.cali.gov.co/secretario/2010_incas/MPS_INCAS_FINAL.pdf 10. MSPS. (2015). Informe Nacional de Calidad de la Atención en Salud. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/DIJ/informe-nal-calidad-atencion- salud-2015.pdf 11. MSPS. (2015). Informe de la calidad de la atención de las EPS. Versión corta, año 2015. Bogotá: Ministerio de Salud y Protección Social. Retrieved from http://minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VP/DOA/RL/informe-calidad-atencion- eps.pdf 12. Cuenta de Alto Costo. (2018). Ranking de las EPS con la mejor calidad de datos en las enfermedades de alto costo. Bogotá: Cuenta de Alto Costo. Retrieved from https://cuentadealtocosto.org/site/images/Publicaciones/2018/evento/Ranking_EPS_calidad_de_datos_ 2018.pdf 13. MSPS. (2017). Encuesta Nacional de Evaluacion de los Servicios de las EPS. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/Informe-encuesta- satisfaccion-eps-2017.pdf 14. MSPS. (2018). Sistema de Evaluación y Calificación de Actores: Ranking de Satisfacción EPS 2018. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/Ranking-satisfaccion-eps- 2018.pdf National Decrees and Regulations Regarding Data Collection 15. MPS. (2006). Decreto 3518. Por el cual se reglamenta el Sistema de Vigilancia en Salud Pública. Bogotá: Ministerio de la Protección Social. Retrived from https://www.minsalud.gov.co/Normatividad_Nuevo/DECRETO%203518%20DE%202006.pdf 16. MSPS. (2012). Resolución 4505, por la cual se establece el reporte relacionado con el registro de las actividades de Protección Específica, Detección Temprana y la aplicación de las Guías de Atención Integral para las enfermedades de interés en salud pública de obligatorio cumplimiento. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.cruzblanca.com.co/Descargas/malla- validadora/resolucion-4505-2012.pdf. This policy establishes a national set of public health indicators. 17. MSPS. (2012). ABECÉ Resolución 4505. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VS/ED/GCFI/abece-resolucion- 4505.pdf 18. MSPS. (2016). Resolución 00256. Bogotá: Ministerio de Salud y Protección Social. Retrieved from http://www.acreditacionensalud.org.co/sua/Documents/Resolución%200256%20de%202016%20SinfCali dad.pdf. This policy establishes a national set of quality indicators. 86 | P a g e 19. MSPS. (2017) Fichas técnicas de los indicadores de calidad de la Resolución 0256 de 2016. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/fichas-tecnicas-indicadores- resolucion-0256-2016.pdf 20. MPS. (2007). Decreto 2699. Bogotá: Ministerio de la Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/DIJ/Decreto-2699-de-2007.pdf. This policy establishes the High Cost Account (Cuenta de Alto Costo), including indicators to track quality of care for expensive conditions. Technical documents 21. MSPS. (2015). Guía Metodológica de Registros, Observatorios y Sistemas de Seguimiento en Salud ROSS. Colombia. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VS/ED/GCFI/ROSS_001.11.2013.pdf 22. MPS. (2009). Linamientos Técnicos para el Registro de los Datos del Registro Individual de la Prestacion de Servicios de Salud. Bogotá: Ministerio de la Protección Social. Retrieved from https://www.minsalud.gov.co/Lineamientos/Lineamientos%20técnicos%20IPS.pdf. Inputs Human resources 23. Pontificia Universidad Javeriana, Cendex. (2013). Estudio de disponibilidad y distribución de la oferta de médicos especialistas. Bogotá: Universidad Javeriana y Centro de Proyectos para el Desarrollo (Cendex). Retrieved from: www.minsalud.gov.co/salud/Documents/Observatorio%20Talento%20Humano%20en%20Salud/Disponibi lidadDistribuciónMdEspecialistasCendex.pdf. 24. MSPS. (2015). Prevenir el cansancio en el personal de salud. Bogotá: Ministerio de Salud y Protección Social. Retrieved from: www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/Prevenir-el-cansancio-personal- salud.pdf 25. MSPS. (2018). Información del talento humano en salud para la gente. RETHUS. Registro Único Nacional de Talento Humano en Salud. Bogotá: Ministerio de Salud y Protección Social. Retrieved from www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/INEC/INTOR/rethus-observatorio.pdf Model of Care 26. MSPS. (2016). Resolución 1441. Bogotá: Ministerio de Salud y Protección Social. Retrieved from http://achc.org.co/documentos/prensa/res-1441-16%20habilitacion%20redes%20integradas%20ips.pdf 27. MSPS. (2016). Política de Atención Integral en Salud. Bogotá: Ministerio de Salud y Protección Social. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/modelo-pais-2016.pdf 87 | P a g e Quality Definition, Standards and Guidelines 28. MSPS. (2019). Lista de guías de práctica clínica aprobadas en Colombia. Bogotá: Ministerio de Salud y Protección Social. Retrieved from http://gpc.minsalud.gov.co/gpc/SitePages/buscador_gpc.aspx 29. MPS. (2011). Promoción de la cultura de seguridad del paciente. Bogotá: Ministerio de la Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/Guia_promocion_seguridad_p aciente.pdf. 30. MPS. (2006). Decreto Numero 1011. Bogotá: Ministerio de la Protección Social. Retrieved from https://www.minsalud.gov.co/Normatividad_Nuevo/DECRETO%201011%20DE%202006.pdf. This decree establishes the Mandatory System of Quality Assurance. Quality Improvement Capacity & Activities 31. MPS. (2011). Manual de Acreditación en Salud Ambulatorio y Hospitalario Colombia. Bogotá: Ministerio de la Protección Social. Retrieved from http://www.acreditacionensalud.org.co/Documents/Manual%20AcreditSalud%20AmbulyHosp2012.pdf Reference 1 also used in this section. Governance and Accountability 32. MPS. (2004). Sistema de Seguridad Social en Salud: Régimen Contributivo. Bogotá: Ministerio de la Protección Social. Retrieved from www.minsalud.gov.co/Documentos%20y%20Publicaciones/GUIA%20INFORMATIVA%20DEL%20REGIMEN %20CONTRIBUTIVO.pdf 33. Defensoría del Pueblo. (2017). La tutela y el derecho a la salud, 2016. Bogotá: Defensoría del Pueblo . Retrieved from https://www.queremosdatos.co/request/396/response/773/attach/4/Tutelas%202016.pdf. 34. MSPS. (2016) Sistema de evaluación y calificación de actores 2016. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/sistema-evaluacion- calificacion-actores-2016.pdf 35. MSPS. (2014). ABC Habilitación de prestadores de servicios de salud. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VS/PSA/abc-habilitacion- prestadores.pdf 36. MSPS. (2014). Resolución 2003 de 2014, por la cual se definen los procedimientos y condiciones de inscripción de los Prestadores de Servicios de Salud y de habilitación de servicios de salud. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/Normatividad_Nuevo/Resoluci%C3%B3n%202003%20de%202014.pdf. Policy describing updated standards for inspection / licensing system (habilitación). 88 | P a g e Quality Policy and Incentives Policies on universal health coverage 37. Congreso de Colombia. (2015). Ley estatutaria 1751 del 16 de febrero de 2015 por el cual se regula el derecho fundamental a la salud y se dictan otras disposiciones. Bogotá: Congreso de Colombia. Retrieved from https://www.minsalud.gov.co/Normatividad_Nuevo/Ley%201751%20de%202015.pdf 38. MSPS. (2019). Resolución 0244 Por la cual se adopta el listado de servicios y tecnologías que serán excluidas de la financiación con recursos públicos asignados a la salud. Bogotá: Ministerio de Salud y Protección Social. Retrived from https://www.minsalud.gov.co/Normatividad_Nuevo/Resoluci%C3%B3n%20No.%20244%20de%202019.p df 39. MPS. (2004). Guía informativa del régimen contributivo. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/Documentos%20y%20Publicaciones/GUIA%20INFORMATIVA%20DEL%20R EGIMEN%20CONTRIBUTIVO.pdf Policies on Licensure 40. MSPS. (2018). Política nacional de talento humano en salud. Bogotá: Ministerio de Salud y Protección Social. Retrieved from https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VS/TH/politica- nacional-talento-humano-salud.pdf 41. Congreso de Colombia. (2007). Ley 1164 de 2007. Por la cual se dictan disposiciones en materia del Talento Humano en Salud. Bogotá: Congreso de Colombia. Retrieved from https://www.minsalud.gov.co/Normatividad_Nuevo/LEY%201164%20DE%202007.pdf 42. Congreso de Colombia. Ley 1917 del 12 de julio de 2018. Por la cual se reglamenta el sistema de residencias médicas. Bogotá: Congreso de Colombia. Retrieved from http://www.suin- juriscol.gov.co/viewDocument.asp?ruta=Leyes/30035417 Policies on privacy, confidentiality, disclosure, consent 43. Congreso de Colombia. (1981). Ley 23 de 18 de febrero de 1981. Normas en materia de ética médica. Bogotá: Congreso de Colombia. Retrieved from www.mineducacion.gov.co/1621/articles- 103905_archivo_pdf.pdf 44. MPS. (2008). Lineamientos para la implementación de la política de seguridad del paciente. Bogotá: Ministerio de la Protección Social. Retrieved from https://www.minsalud.gov.co/Normatividad_Nuevo/Resoluci%C3%B2n%200112%20de%202012%20-%20 Documentos%20de%20apoyo%202.pdf. 45. Ministerio de Salud. (1999). Resolución 1995 de 8 de julio de 1999, manejo de historia clínica. Bogotá: Ministerio de Salud. Retrieved from www.minsalud.gov.co/Normatividad_Nuevo/RESOLUCI%C3%93N%201995%20DE%201999.pdf 89 | P a g e A1.5 Databases Reviewed for Quality Assessment Tool Health Quality Observatory (Observatorio de calidad en salud) http://calidadensalud.minsalud.gov.co/Paginas/Indicadores.aspx http://calidadensalud.minsalud.gov.co/indicadores/Paginas/Densidad-estimada-profesionales- medicina-enfermer%C3%ADa.aspx http://calidadensalud.minsalud.gov.co/Paginas/ConsultaDescargaIndicadores256.aspx Public health expenditures data https://datosmacro.expansion.com/estado/gasto/salud/colombia https://www.minsalud.gov.co/proteccionsocial/Financiamiento/Paginas/indicadores_generales. aspx https://observatoriosocial.cepal.org/inversion/es/paises/colombia Human resources (RETHUS database) https://www.datos.gov.co/Salud-y-Protecci-n-Social/Saludatos-Estimaci-n-de-los-Indicadores- para-el-se/5atd-7cac https://www.minsalud.gov.co/salud/PO/Paginas/registro-unico-nacional-del-talento-humano-en-salud- rethus.aspx Service Coverage Data https://www.datos.gov.co/Vivienda-Ciudad-y-Territorio/Cobertura-Servicios-P-blicos/qhgd-jun6 Hospital Bed Capacity Data (pediatrics) https://www.datos.gov.co/Salud-y-Protecci-n-Social/N-mero-de-camas-hospitalarias-pediatras- por-depart/63zs-yhcj Hospital bed capacity – international comparisons https://www.indexmundi.com/g/r.aspx?c=co&v=2227&l=es 90 | P a g e Repository of Clinical Practice Guidelines http://www.gpc.minsalud.gov.co/gpc/SitePages/buscador_gpc.aspx Database of Health Care Organizations (REPS) https://www.sispro.gov.co/central-prestadores-de-servicios/Pages/REPS-Registro-especial-de- prestadores-de-servicios-de-salud.aspx OECD – International Comparisons Database https://stats.oecd.org/index.aspx?DataSetCode=HEALTH_STAT# WHO – Health Workforce Database https://www.who.int/gho/health_workforce/physicians_density/en/ 91 | P a g e A1.6 EPS Survey An on-line survey was carried out between April 10th and 30th, 2019, to which the following 13 Health insurance companies responded: Table A1.6.1: Respondents to EPS Survey EPS Number of subscribers Servicio Occidental de Salud 895,166 Saludvida 1,168,99 Saludvida-Subsidiado 1,536,200 Comfamiliar Cartagena 160,000 Aliansalud 235,000 Cruz Blanca 385,000 Comfasucre 112,573 Salud Total 3,097,000 Anónimo 320,386 EMSSANAR 1,921,809 EMSSANAR Subsidiado 1,801,874 Medimas 3,800,000 Nueva EPS 4,633,684 Total 20,067,920 All Health insurance companies responding to the survey were from the private sector, and respondents collectively served almost half of the Colombian population. The number of subscribers to each EPS ranged from 112,573 to 4,633,684. The total number of subscribers served by the responding Health insurance companies, 20,067,920, represented 44% of the Colombian population. Among responding Health insurance companies, 79% provided care to the subsidized regime and 86% to the contributory regime. On average, each EPS contracted with 255 healthcare providers for hospital services, 390 healthcare providers for primary care and 322 healthcare providers for other services. The distribution of the population served by these Health insurance companies by area of residence is similar to the national population: 29% reside in rural areas and 71% in urban areas. All regional departments of Colombia were served by at least one of the Health insurance companies responding to the survey. Health insurance companies requested information from healthcare providers on a wide range of quality indicators of hospital care, such as patient experience, waiting times, hand washing rates, surgical site infections and others. Healthcare providers also asked primary care healthcare providers to submit information on wait times and patient experience, as well as management of NCDs, immunization rates and maternal and child health. For each indicator, there were variations between Health insurance companies in the extent to which data were collected from all, some, or none of the healthcare providers contracted; in general, the most common response was “some� (see Table A1.6.2, A1.6.3 below). 92 | P a g e 100% of the responding Health insurance companies indicated that quantitative performance goals were established for healthcare providers, for at least some indicators. In case of non-compliance, 93% indicated that they would require a quality improvement plan; 71% would meet with the IPS to discuss the results; and 36% would withhold funds. All Health insurance companies require subscribers to obtain authorization for certain services, and 75% of Health insurance companies stated this took less than a day. Regarding contracting modalities for hospital care, 100% of Health insurance companies reported that payment is based on the volume of services provided (i.e. fee for service) and to a lesser extent, capitation (31%). For ambulatory care, the prevalent payment modality is capitation (100%) followed by fee-for- service (85%). Table A1.6.2: Information collected by health insurance companies from hospital healthcare providers Data collected from: Indicator all IPS some no healthcare healthcare providers providers 14% 71% 14% Patient experience survey data 36% 50% 14% Wait times – emergency department 36% 64% 0% Wait times – advanced diagnostic imaging (e.g. CT, MRI) 36% 64% 0% wait times – specialist consultations 14% 43% 36% wait times – other (please specify)__________________ 36% 64% 0% Hand hygiene rates 29% 64% 7% Surgical site infections 29% 57% 14% Other health care acquired infections 29% 50% 21% Surgical complication rates 14% 29% 57% Medication errors 29% 71% 0% Falls in hospital 29% 64% 7% Pressure ulcers in hospital 21% 64% 14% Critical incidents resulting in unintended harm to patient 43% 57% 0% Obstetrical complication rates 57% 43% 0% Mortality rates 43% 57% 0% Length of stay Excess length of stay (i.e. average # of hospital bed days above what is 29% 57% 14% expected, for a given diagnosis) Inappropriate referrals or utilization (e.g. use of emergency department 7% 50% 43% for minor conditions, or use of tertiary hospitals for uncomplicated hospitalizations or procedures) Inappropriate use of health care services (If yes, please specify: 7% 64% 0% ______________) 36% 43% 7% Other quality indicators 93 | P a g e Table A1.6.3: Information collected by Health insurance companies from Primary Care Healthcare providers Data collected from: Quality indicator all IPS some no healthcare healthcare providers providers 29% 50% 21% Patient experience survey data 43% 50% 7% Information on complaints from patients & family about the IPS 36% 64% 0% Wait times – visit with physician 36% 64% 0% Wait times – specialist consultation 57% 43% 0% Immunization rates 50% 50% 0% Cancer screening rates (e.g. Pap, mammography) Lifestyle indicators (e.g. rate of smoking, heavy alcohol intake, poor diet or 14% 36% 50% exercise) % of patients who have had a healthy lifestyle intervention (e.g. diet or 14% 36% 50% smoking cessation counselling) 57% 43% 0% Appropriate management for diabetes (e.g. blood sugar controlled) 57% 43% 0% Appropriate management for hypertension (e.g. BP well controlled) 50% 50% 0% % of pregnant women getting appropriate antenatal care 57% 43% 0% % of HIV patients compliant with medications & in good control of disease 50% 36% 14% TB treatment completion rate 0% 50% 50% Medication errors Inappropriate use of health care services (If yes, please specify: 0% 21% 64% ______________) 21% 21% 36% Other quality indicators (please specify: _____________) No EPS offered incentives for accreditation, but Health insurance companies did take quality in consideration when contracting with healthcare providers. In case poor quality, Health insurance companies have the ability to cancel the contract (71%), refrain from renewing the contract with the IPS (64%) or penalize by withholding of funds (43%). Many hospital quality indicators are used in contractual processes with IPS. These included hospital falls (93%), waiting times (86%), mortality rates (86%), length of stay and rates of infections and surgical site complications (71%), among others. At the primary care level, indicators such as immunization coverage, early detection of cancer and waiting times for consultation with a general practitioner are considered (93%) as well as appropriate management of diabetes, hypertension, adequate prenatal care and adequate control to patients with HIV (86%). 93% of the EPS respondents stated that they have personnel with special training in quality management, on average 25 employees, who generally provide technical support to the IPS for quality improvement. 86% of Health insurance companies had organized quality improvement campaigns with their healthcare providers on topics such as promotion of the culture of quality, patient safety, wait times, adherence to 94 | P a g e treatments. However, when asked about specific degree of improvement, only one was able to quote a specific percentage improvement in an indicator. All Health insurance companies surveyed provide information to their contracted IPS about their performance in quality. This information is provided quarterly (36%) or monthly (29%). 64% of Health insurance companies indicated that they provide the IPS with benchmarks for comparison against other healthcare providers. In general, this information is directed to the middle management levels (85%) and in no case to front-line providers. Health insurance companies were asked about their perception of the biggest problems with quality among the healthcare providers with which they contract. Responses included problems with information systems, non-compliance with promotion and prevention programs, waiting times and high staff turnover. Health insurance companies were asked to give a priority for different options for investment in the health system to improve quality. Their responses, in order, are as follows: 1. Improved quality measurement infrastructure. Ensure that all healthcare providers are able to collect data on most of the quality indicators mentioned above. Provide healthcare providers with comparisons on how they are performing compared to peers. (7 votes) 2. National quality improvement campaigns. Invest in training of staff / managers in specialized quality management skills, and support quality improvement teams in healthcare providers to make improvements in targeted areas (e.g. patient safety, reducing wait times, reducing inappropriate use). (6 votes) 3. Develop community accountability mechanisms. Healthcare providers would be required to report their results on quality (e.g. wait times, patient safety issues, patient experience) to public forums or community councils and explain what they are doing to improve. (4 votes) 4. Greater financial accountability mechanisms. Develop national requirements that Health insurance companies base at least some portion of their payments to healthcare providers on quality. (3 votes) 5. National program for verifying and strengthening clinical skills of health care providers. (2 votes) 6. Expanded accreditation program for hospitals. Make accreditation a mandatory requirement, to be phased in over time. (1 vote) Other suggestions for national programs to improve quality included strengthening of information systems, incentives for payment for results, development of competencies and skills in quality at the undergraduate level, promote intersectoral collaboration, and centers of excellence for pathologies of high impact on public health. 95 | P a g e Appendix 2: Detailed Analysis of Accountability Mechanisms for Quality of Care in Colombia The table below summarizes the players of the chain of accountability in the health sector and how they interact in the accountability chain. It allows the reader to separately analyze how different actors hold each other accountable. An arrow pointing from actor A to B indicates that B is accountable to A for delivering on certain expectations, and that A can enact certain mechanisms to ensure that B meets these expectations. These mechanisms are described in the last column. 96 | P a g e Accountability mechanism Patients, users and the public Health professional bodies Health professionals Territorial Entities Supersalud Congress Ministry EPS IPS |= => Patients pressure elected officials to improve quality at time of elections. Patients file lawsuit in a special court (tutela) with the help of Office of the Ombudsman ( Defensoría del pueblo y personerías), = == == == == => if they feel they are denied access to care. EPSs face penalties (desacatos), including a short stay in prison, for not complying with tutela orders. Choice of EPS: users can switch to another EPS. Information from EPS comparison reports, and media stories based on these = == == == == => reports, can in theory guide these decisions. = == == == == == => = == == == == == == == => Patients can file malpractice lawsuits through the judicial system. Patient's bill of rights is exhibited in facilities. Patient advocacy groups, user associations, veedurías (non-profit advocacy = == == == == == => groups) can advocate for fulfilment of these rights. = == == == == == => Patient appointees on hospital boards in public hospitals help hold executives accountable for quality. == == == == == == == => Users' associations in hospitals advocate for patient quality. = => Ministry files annual reports to Congress. Office of Ombudsman also files annual reports on tutelas. == => Supersalud files annual reports to Congress. Ministry influences EPSs through public reporting and ranking of EPSs according to performance, based on data analysis on |= == == => SISPRO. Ministry also attempts to promote integrated model of care, clinical practice guidelines. == => Supersalud grants financial licenses to EPSs and handles complaints from users. 97 | P a g e Patients, users and the public Accountability mechanism Health professional bodies Health professionals Territorial Entities Supersalud Congress Ministry EPS IPS The Ministry sets standards for habilitación, the system of licensure and inspection of facilities. (Departments and districts apply and enforce; see below). |= == == == => The Ministry controls investments of capacity in public hospitals. The Ministry encourages accreditation (although it is voluntary and not associated with incentives). Authorization for tertiary care and high-cost providers directly controlled by Ministry, but the rest of the process is delegated to departments and districts. |= == == == == == => Professionals have to sign-up in the human resources registry (RETHUS). Prescription patterns for technologies outside the standard benefits package is monitored through MIPRES. |= == => Territorial entities apply and enforce habilitación; in particular, they inspect IPSs and can revoke licenses for failure to comply. Under habilitación, health professionals must prove that they have the necessary qualifications in order to obtain a license to |= == == == => practice. EPSs may conduct clinical audits of IPSs for adherence to practice guidelines. EPSs may do supplemental inspections in addition to those of habilitación. |= => EPSs may do their own private rankings of IPSs and can choose to contract with IPSs with better quality. EPS can embed their information systems into a provider for direct control. EPSs may offer complementary payments based on clinical outcomes and process indicators. EPSs may choose to contract with providers with better quality or credentials. |= == == => EPSs may conduct clinical audits of adherence to clinical practice guidelines. |= == => Clinical audits take place within individual institutions to monitor adherence to CPGs, adverse events, etc. 98 | P a g e Note also: The MoH is accountable to all the actors in the system through the annual event open to the public and with wide media coverage, called "Rendición de Cuentas". Trade associations (ACEMI, ACHC, ACSC, ACESI, etc), patient advisory groups and user associations hold other actors accountable through formal mechanisms of participation and deliberation, and through mass media and social media. Legend: CPG: Clinical practice guidelines EPS: Entidades Promotoras de Salud – Health care insurers IPS: Institución Prestadora de Servicios de Salud – Health care providers (e,g, hospitals, clinics, labs) MiPRES – Registry for technologies not covered under standard benefits package. RETHUS: Registro del Talento Humano en Salud, a human resources registry SISPRO: Sistema Integrado de Información de la Protección Social, the national Health data repository SuperSalud – Health Superintendency, the national regulatory agency overseeing compliance with government policies 99 | P a g e Appendix 3: IFC Survey of Quality Assurance Standards in Selected Colombian Hospitals A3.1 Introduction Context Around the world, most patients admitted to hospital assume that they are in safe hands. Unfortunately, this assumption is not always true. Patient Safety is now a serious global public health issue. Estimates show that in high income developed countries about one in 10 patients is harmed while receiving hospital care. In the USA, by some estimates, medical errors account for more deaths annually than road accidents and breast cancer combined. The situation is even worse in low- and middle-income countries (LMICs), which account for 66% of the global burden of adverse events and DALYs lost from unsafe care. For example: ▪ the rate of surgical site infections in LMICs is 6.1%, compared with 0.9% in the US. ▪ the median mortality rate in ERs is 1.8%, which is 45 times higher than in the US. ▪ A mother having a C-section in a LMIC is ten times more likely to die compared with the Netherlands. According to the WHO and Lancet Commission, poor quality healthcare is now a bigger barrier to reducing mortality than increasing access. Recent WHO research found that 5.7 million deaths were due to use of poor-quality services, compared to 2.9 million deaths due to non-utilization of services. From a private investment perspective, Quality Assurance (QA) and patient safety are essential aspects of risk management. International investors are increasingly nervous of investing in healthcare companies (particularly in emerging markets) where QA and patient safety procedures are under-developed. This is because it is difficult to assess the level of risk to patients. This concern especially relates to inpatient facilities, and particularly those providing higher risk services such as complex surgery, maternity, cancer and pediatrics. Even a single incident can result in a large financial claim and ruin a hospital’s reputation. Increasingly, international investors invite independent QA/patient safety experts to support this aspect of their due diligence. Patient safety is recognized as a global challenge that requires knowledge and skills in multiple areas, including human factors and systems engineering. Patient safety has received attention by international health organizations. In 2004, the WHO launched the World Alliance for Patient Safety, targeting the following issues: prevention of healthcare-associated infections, hand hygiene, surgical safety, and patient engagement. Purpose of Report The main purpose of this report is to objectively assess the effectiveness of Colombia’s National Quality Improvement Program and the impact of its national accreditation system at the level of the medical 100 | P a g e facilities, using a structured and evidence-based survey methodology developed by the IFC. In particular, this approach aims to identify the main gaps that would need to be addressed to increase the effectiveness of QI efforts. About IFC This project was supported by the IFC. IFC is the private sector arm of the World Bank and is one of the largest investors in healthcare in emerging markets. In addition to financial support, IFC also provides advisory services including Healthcare Quality Standards Assessment and Improvement. A3.2 Survey Approach & Methodology Site visits We surveyed 11 public and private hospital facilities across three regions of Colombia – Bogota, Barranquilla, Cartagena, and Cali (see Table A3.2.1) and conducted a meeting with iContec, the accrediting body. The project was undertaken from March to April 2019. Each survey involved a half-day site visit which included: ▪ A tour of each hospital, including physical observations and discussions with departmental staff. Areas visited included the operating room, intensive care units for adults and children, sterilization units, wards for children and adults, radiology departments, maternity departments, laboratories, kitchen, laundry and emergency departments. ▪ Structured interviews with members of the company’s team and senior management and clinical staff, using questions from the IFC Healthcare Standards guide. ▪ High-level review of QA documentation as provided by sites. Table A3.2.1 Hospitals Visited for IFC Quality Tool Deployment Hospital name Location Ownership Hospital Nino Jesus Barranquilla Public Clinica Portoazul Barranquilla Private ESE Hospital Departemental de Sabanalarga Sabanalarga Public Nuevo Hospital de Bocargande Cartagena Private Hospital San Rafael Bogota Private Hospital Santa Clara Bogota Private Kennedy Hospital Bogota Public Hospital Mederi Bogota Private Clinica Farallones Cali Private Hospital Universitario del Valle Evaristo Cali Public Garcia Hospital Primitivo Iglesias Cali Public 101 | P a g e Survey Questionnaire We used a survey questionnaire based on the newly revised IFC Healthcare Standards guide. This was developed in collaboration with Joint Commission International (JCI) and incorporates “core� international standards based on JCI and international standards (See Table A3.2.2). Specifically, 25 “core� questions were taken from the Guide and used during each site visit (see section 3.4). Feedback and reporting Preliminary results of the survey were presented Informally on-site at the end of local facility surveys, and formally to the Deputy Minister and his team in Bogota in April 2019. This report sets out the main results and conclusions of the survey. Consultant The review was undertaken by Julia Khalimova who is IFC’s Specialist in Healthcare International Standards & Accreditation. She has extensive international experience of healthcare quality assessment and quality improvement including in Europe, Latin America, Asia, and Africa. Table A3.2.2: Standards used in assessment of hospital quality Areas Standards International Patient Safety Goals • Identify patients correctly • Improve effective communication • Improve the safety of high-alert medications • Ensure safe surgery • Reduce the risk of healthcare associated infections • Reduce the risk of patient harm resulting from falls Ethics, Patient & Family Rights • Rights of patients and families • Informed consent • Organ donation • Fertility treatment • Termination of pregnancy services (where legally allowed) • Clinical research Medication Management & Use • Organization and Management • Selection and procurement • Storage • Prescribing and transcribing • Preparing and dispensing Quality Improvement & Patient Safety • Oversight of Quality and Patient Safety Activities • Use of data to improve quality Prevention & Control of Infections • Prevention & Control of Infections Governance, Leadership & Direction • Governance • Senior manager responsible for operations • Leadership for quality and patient safety • Evidence-based clinical guidelines • Departmental service specification 102 | P a g e Areas Standards Facility Management & Safety • Facility compliance • Facility management oversight and program • Safety and security • Hazardous materials plan • Disaster preparedness • Fire safety • Medical technology • Utility system Staff Qualifications & Education • Appropriate staff numbers and qualifications A3.3 Survey Results International Patient Safety Goals Key findings include: Goal 1. Patient Identification Good practices: ✓ There is a procedure to identify patients. ✓ Patients are identified with at least 2 x identifiers. ✓ ID/wrist-bands are used. ✓ Colour coding in ID/wrist-band addresses various risks. ✓ There is a process for identification of neonates. ✓ Patients are identified before surgery and other therapeutic procedures including blood, transfusion, injections, procedures in radiology. Areas for improvement: ✓ Some hospitals identify patients by writing on a board on their bed, which is identification by location, significantly increasing the risk of wrong patient/wrong-site surgery. ✓ No consistent system of colour codes for ID bands, so different hospitals use different colours for certain type of risks. Goal 2. Improve effective communication Good practices: ✓ In many hospitals visited IT system helps to fully exclude verbal orders, which are prone to errors. ✓ In hospitals which do use verbal orders, there is a procedure to address effective communication. Goal 3. Improve the safety of high-alert medications Good practices: ✓ Most hospitals visited had a list of high-alert medications. ✓ Staff interviewed were able to provide examples of these. ✓ High-alert medications were observed to be properly labelled in most of the hospitals visited. 103 | P a g e Areas for Improvements: ✓ Understanding of key terms - some important terms are not clearly understood by all staff. For example, “Look-alike� and “Sound-alike� (LASA) terms were not known to some who we interviewed, even though the abbreviation “LASA� is known and used. This may indicate a lack of formal training, or reliance on self-training. Goal 4. Ensure safe surgery Good practices ✓ There are procedures for addressing safe surgery. ✓ There are processes for surgical site identification. Surgeons in most hospitals were able to describe the process clearly. ✓ The WHO Safe Surgery Checklist is used, or hospitals develop their own, adapted from the WHO tool to ensure that all documents, equipment, etc are in place. The checklist was displayed on the boards in operating rooms in several hospitals visited. ✓ Time-out procedures are used. Goal 5. Reduce the risk of healthcare associated infections Good practices: ✓ Staff in all hospitals visited were aware of hand-washing guidelines. ✓ Hand hygiene posters were observed in most hospitals visited. ✓ Wash basins, hand disinfection bottles were observed in consultation rooms, inpatient wards, and nurse stations in most hospitals - and they had clearly been used. ✓ Soap and water were available in all toilets that were inspected. Shortcomings ✓ In some hospitals, sanitizers, though present in the halls/wards, were empty or not functioning. Goal 6: Reduce the risk of harm resulting from patient falls Good Practices: ✓ There are procedures for addressing patient falls. ✓ Patients are assessed from the moment of admission. ✓ Fall-risk assessment forms have been developed and are used. ✓ Patients are correctly assessed, based on circumstances that could potentially affect a patient's stability, e.g. medical conditions, procedures and medications. ✓ Patients with increased fall risk have a special colour wrist band for easy identification. ✓ Bed rails are used. ✓ Signage - ‘Slippery Floor’, ‘Wet Floor’ signs were present and used. Shortcomings: ✓ Some hospitals have not identified areas with a risk of a fall or fall-from-height accident. These include landings with low walls/rails, narrow/spiral staircases with gaps in the rails, and lack of rails in some staircases. ✓ Slippery floor tiles/stairs in some hospitals. ✓ Wrist bands colour for patients with high risk of falls differ from hospital to hospital. 104 | P a g e Ethics, Patient & Family Rights Good practices: ✓ Patients are informed about the process of granting consent in a manner they understand. ✓ Patients give informed consent consistent with the organization’s written policies and procedures. ✓ Consent forms have been developed based on national guidelines and implemented. ✓ Patients’ Rights are identified and publicly displayed in many hospitals visited. Staff members interviewed were able to provide examples of patients’ rights. Areas for improvement: ✓ Some hospitals visited did not display patients’ rights. ✓ Lack of formalized training on communicating contents of Informed Consent in some hospitals. ✓ Lack of clear understanding of Code of Ethics in some facilities. ✓ Informed consents forms in most hospitals are too general and do not describe specifics of the procedures in the written form. ✓ In some hospitals, cameras were observed in staff changing rooms, which may unnecessarily infringe privacy. We discussed certain services that involve particular scrutiny of ethical issues. During these discussions staff described a lack of clarity in legislative requirements regarding termination of pregnancy and organ donation. Medication Management and Use Good practices: ✓ Medication Management Plan and Pharmaceutical Formulary developed in most hospitals visited. ✓ Clinical Pharmacists present in most hospitals visited. ✓ Regular audits performed. ✓ Medication Management forms developed and used in most hospitals visited. These contain all the required information (ie name of patient, name of the medication, time, and dosage). Areas for improvement: ✓ Although staff demonstrated some knowledge of specific terms, there is lack understanding of certain safe medication management practices (e.g. for handling look-alike, sound-alike drugs). ✓ Some hospitals lack processes to address shortages of specific medications. (We were informed about shortages of Lidocaine and Oxytocin in some regions). Quality Improvement & Patient Safety Good practices: ✓ There is a designated quality manager and a quality team in every hospital visited. ✓ Quality Improvement training is undertaken regularly in every hospital visited. ✓ Staff members understands their role in quality improvement work in most of the hospitals visited. 105 | P a g e ✓ Every hospital visited developed and implemented a Quality Improvement Plan or a similar document. ✓ Every hospital visited was collecting data; measuring, analysing and reporting quality indicators relating to Incidents (eg near-misses, sentinel events), outcomes (eg mortality rate, infection rate), and patient and employee satisfaction. ✓ An Incident Reporting System is developed and implemented in every hospital. The definition of an incident is defined, data collected, classified (based on seriousness of the event) and root-cause analysis is performed. Preventive measures are taken. Staff were able to provide examples of the most recent incidents, analysis and actions taken based on the results. ✓ Most hospitals visited developed flow charts for most important processes. ✓ There are procedures for addressing complaints. Complaints are collected in several ways (eg suggestion boxes, website, satisfaction surveys) and addressed. Based on analysis of complaints hospitals state that expectations of patients are growing. For example, patients reportedly used to complain about ineffective treatment and lack of attention from medical staff; now they complain about compromising privacy if a doctor does not close the door properly. ✓ Internal audits relating to quality assurance work are carried out in every hospital visited. Areas for improvement: ✓ Differences in qualifications of Quality Managers in different hospitals, with some quality managers self-trained. ✓ Some processes are not properly developed. Incidents are addressed differently in different hospitals. Some flow charts are not developed up to the required level. Some documents lack proper implementation or are not understood by staff. ✓ Inconsistent quality related documentation – ranging from good, well-structured to brief and superficial. Prevention & Control of Infections Good practices: ✓ There is a formal Infection Control Program in most hospitals visited. ✓ Infection Prevention and Control programme is evidence-based with references. ✓ There is a designated Infection Control Officer (or similar) in every hospital visited. ✓ Hand-washing Programs are developed and properly implemented in most hospitals visited. ✓ Staff were observed wearing masks and other protective means in most hospitals. ✓ Sharps and needles are collected in designated puncture-proof containers. There is a formal process to address needle-stick injuries in most hospitals. ✓ There are formal procedures for waste management. Waste segregation is implemented, and color- coded bags are used to indicate the type of waste. Bio-medical waste is collected in designated red bags and managed in accordance with requirements in most hospitals visited. ✓ There is a formal procedure addressing the handling of linen, including contaminated, in most hospitals visited. ✓ There are designated isolation areas and policies for infectious patients in most hospitals. ✓ Surgical scrub areas are present in every hospital visited. ✓ Staff changing rooms for operating theatre staff are present in all hospital visited. ✓ Sterilization units in most hospitals are properly organized. ✓ Indicators (infection rate and others) are collected and analysed. 106 | P a g e Shortcomings: ✓ Some staff changing rooms did not have functioning showers (showers were broken, shower cabins cluttered). ✓ In some hospitals overfilled bio-medical waste containers were observed. ✓ Flow of patients/staff in some operating theatres does not ensure necessary separation and may increase the risk of cross-contamination. ✓ Inconsistent infection control related requirements were observed for visitors in “sterile� areas, such as operating theatres. Some hospitals ask to wash hands and wear a mask; others ask to change clothes completely and remove all jewelry etc. ✓ Wall/ceiling/floor materials in some hospitals do not meet requirements (have seams, pores etc, which can potentially harbour microorganisms). ✓ Some kitchen areas do not meet basic requirements for infection control. Governance, Leadership and Direction Good practices include: ✓ All necessary licenses are obtained, and certificates publicly displayed in all hospitals visited. ✓ A formal organizational structure was created in all hospitals visited. ✓ Clinical guidelines and protocols are adopted in all hospitals visited. ✓ Mission, vision created and publicly displayed in most hospitals visited. ✓ Awareness of quality and patient safety work is generally high among top-management in most hospitals visited. Shortcomings: ✓ Management in some hospitals did not fully understand the essence of quality improvement work and perceived some aspects of it as “red tape�. Facility Management and Safety Good Practices include: ✓ Staff are aware of national requirements relating to facility maintenance. ✓ Most private hospitals have well-maintained both buildings (both inside and outside), meeting basic international requirements regarding layout and size. ✓ Most hospitals visited have adequate signage making navigation for patients easier. Warning signs (eg in Radiology or areas with limited access) were observed in most hospitals visited. ✓ There is a legislation requirement to include 5% of the budget for physical upgrade and refurbishment and all hospitals visited are aware of it. ✓ Kitchen areas in private hospitals (where present and not outsourced) are neat and tidy, and properly designed. ✓ Laundry areas in private hospitals (where present and not outsourced) are neat and tidy, and properly designed. ✓ Sterilization units in most hospitals visited are properly designed from the point of view of layout and size. 107 | P a g e ✓ There are formal processes for medical equipment maintenance in every hospital visited. Critical equipment is identified and processes for substitute in case of failure created. ✓ There is a formal policy regarding the safety and security of the facility in most hospitals. ✓ Visitors/patients are identified. Staff, vendors, visitors wear badges in most hospitals visited. ✓ There are physical security measures in place including cameras, and biometric access to wards and theatre in some hospitals visited. ✓ Aspects of a hazard material management program exists in all hospitals visited. ✓ Waste is collected according to the adopted policies. Plans of waste flow for each type of waste are displayed on the walls in all hospitals visited. ✓ Hazardous waste and materials are managed according to developed procedure. ✓ Staff was aware of national requirements regarding disaster preparedness in all hospitals visited. ✓ Staff are aware of national requirements regarding Fire Safety in most hospitals. Fire drills are performed in most hospitals, and in some day drill and night drill. Fire Extinguishers and other fire fighting means present, regularly checked. ✓ Fire exits are present, easily accessed, and properly marked in all hospitals visited. Areas for improvement: ✓ Condition of some public hospitals buildings is substandard, which may compromise the quality of care, especially in areas requiring increased control over infection prevention (operating theatre, ICU): peeling paint, leakages on the ceiling/walls, broken floor with cracks and dust in them. Shabby wooden rails on the walls presenting increased risk of micro-organisms growth. ✓ Some light sockets were incomplete. ✓ Some holes in walls observed. ✓ Loose wires and cables on the floor that present a trip and fall hazard, hampers cleaning. Some cords are not isolated (open). ✓ Loose ceiling panels or lack of such and holes in the ceiling exposing pipes and electrical cords in some areas of several public hospitals. ✓ Broken steps in some staircases. ✓ Fragmented buildings layout at some hospital – with “patients carried by ambulance car�. ✓ Narrow corridors, including some ICU areas. ✓ Inadequate layout of some operating theatres of some public hospitals (inadequate segregation of surgical patients; flows of staff/patients before/after surgeries mixed). ✓ Some kitchens in public hospitals do not meet requirements regarding size, layout presenting increased risk from the point of view of infection control. ✓ Some equipment in some public hospitals visibly unclean/ shabby. ✓ Hazardous materials program in some hospitals is not structured to the required level. Staff not always able to provide examples, sometimes were not aware of the specific terms relating to this area. ✓ In two hospitals expired fire extinguishers were observed. ✓ Construction work sites observed in several hospitals were not isolated, with patients exposed to dust and noise. 108 | P a g e Staff qualifications and Education Good Practices ✓ Job descriptions developed. ✓ General orientation program developed in most hospitals. ✓ There is an assessment program for staff. ✓ There is a program to verify diplomas/qualifications. ✓ New employees, medical, nursing, and pharmacy interns/students work under supervision through their probation /training period. A3.4 Conclusions & Recommendations Positive observations This was a complex project which surveyed a wide range of QA practices among diverse hospital facilities across several regions of Colombia. We obtained a high level of local cooperation during this project. Based on our observations, overall the development of healthcare QA standards in Colombia is good – even compared with other middle-income countries. Specifically: ✓ National requirements regarding QA are generally consistent with internationally recognized practices. ✓ Regulations regarding QA are generally clear and comprehensible. ✓ Staff awareness of QA standards and practices is generally high. For example, most staff have some level of familiarity with and understanding of QA terminology, and were able to provide examples of specific QA processes/terms/techniques. ✓ Staff are also generally aware of regulatory requirements and were able to quote relevant laws. Areas of improvement Shortcomings mainly relate to variations across the system. These include, for example: ✓ Variable levels of implementation of standards across hospitals, including variable levels of compliance. ✓ Variable levels of training in, and depth of understanding of QA practices. Some QA personnel had an advanced of understanding through formal training, while others appeared to be self-taught. Some staff see limited value regarding certain QA practices, perceiving them as “red tape�. Many hospitals seeking accreditation feel the need to engage external consultants to guide them through the process. ✓ Variable reported levels of understanding at the level of local health authorities which are charged with QA oversight. A common complaint a hospital level was a perceived lack of support from the regulator for quality improvement. There is also a perceived lack of clarity in some areas – including aspects of Termination of pregnancy and Organ Donation. This leave local hospitals feeling exposed and unwilling to undertake such services. 109 | P a g e Potential areas for IFC/WB support Areas of future IFC/WB support may include: ✓ Formal training in QA methods at both regulator and local levels – in order to implement consistent approaches country-wide. ✓ Capacity-building at the level of regulator/MoH regarding QA oversight and support. ✓ Clarification of regulations regarding sensitive (or omitted) areas – eg termination of pregnancy and organ donation. These measures might include development of a national QA “Centre of Excellence� (or similar) at the central regulator/MoH, supported by highly trained local regional “QA champions�. A3.5 Survey Questionnaire 1. The hospital develops and implements written documents that improve the accuracy of patient identification. 2. Written documents which improve the accuracy of verbal and telephone communications are developed and implemented. 3. The hospital implements actions to improve the safety of high-alert medications such as, but not limited to: specific storage, prescribing, preparation administration and monitoring processes. 4. The organization uses a check-list (or other similar document) to verify that: the correct site, correct procedure, correct patients are identified; all documents (e.g. informed consent) are on hand, correct and functional; and equipment needed is on hand, correct for purpose and functional (Usually the WHO surgical safety check-list is used). 5. The organization implements a hand-hygiene program, as part of an Infection Prevention and Control Program. 6. The hospital develops and implements procedures aimed at reducing the risk of patient harm resulting from falls. 7. The organization establishes ethical and legal norms which guide staff conduct. 8. The organization has a clearly defined informed consent process described in written documents. 9. The organization supports patient and family choices to donate organs and other tissues. The organization provides information to patients and families on the donation process, and on the way organ donation is organized. 10. The organization has written procedures regarding reproductive health technologies. The procedures take into consideration legal and ethical considerations. 11. The organization has written procedures regarding termination of pregnancy. The procedures take into consideration legal and ethical considerations. 12. Prior to commencement the purpose of any clinical trial and its benefits are clear. Ethical issues relating to the trial are also fully considered. An ethics committee including external experts many be formed to approve trials. 13. A medication management plan (or similar) identifies how medication is organized and managed throughout the organization. There has been one documented review of the medication management system within the previous 12 months. 14. The organization has a written Quality improvement and patient safety plan (or similar). The plan is updated at least annually. 15. The definition of an incident is established by the organization. 16. There is a process to report, analyze and prevent incidents. 110 | P a g e 17. The organization has a written and implemented infection control and prevention program. 18. Organization and department leaders collectively determine clinical practice guidelines to be used. 19. The organization has a comprehensive facilities management program. The documents are fully implemented and regularly reviewed (at least once per year). 20. A hazardous material management program exists. 21. The organization develops a disaster preparedness plan which describes its response to likely disaster (see page 106 for full requirement). 22. The organization has a fire safety plan to ensure that all occupants of the facility are safe from fire and smoke. 23. The organization identifies the areas and services at greatest risk when power fails and water is contaminated or interrupted. 24. There is a formal process for recruitment and evaluation of new staff (including diploma verification). 25. Each staff member has a job description. 111 | P a g e References Alcaldía de Santiago de Cali. (2012). Plan de desarrollo del Municipio de Santiago de Cali. Cali: Alcaldía de Santiago de Cali. Récupéré sur http://www.cali.gov.co/planeacion/publicaciones/44418/plan_desarrollo_municipal_2012_201 5/ Baker GR, M.-M. A. (2008). High Performing Healthcare Systems: Delivering Quality by Design. Toronto: Longwoods. Récupéré sur www.longwoods.com/publications/books/571 Chan, B., & Veillard, J. (2019). Stewardship of quality of care in health systems. Public Administration & Development, 39, 34-46. Congreso de Colombia. (1981). Ley 23 de 18 de febrero de 1981. Normas en materia de ética médica. Bogota: Congreso de Colombia. Récupéré sur www.mineducacion.gov.co/1621/articles- 103905_archivo_pdf.pdf Congreso de Colombia. (2015). Ley estatutaria 1751 del 16 de febrero de 2015 por el cual se regula el derecho fundamental a la salud y se dictan otras disposiciones. Bogota: Congreso de Colombia. Récupéré sur https://www.minsalud.gov.co/Normatividad_Nuevo/Ley%201751%20de%202015.pdf Cuenta de Alto Costo. (2018). Ranking de las EPS con la mejor calidad de datos en las enfermedades de alto costo. Bogota: Cuenta de Alto Costo. Récupéré sur https://cuentadealtocosto.org/site/images/Publicaciones/2018/evento/Ranking_EPS_calidad_d e_datos_2018.pdf Defensoría del Pueblo. (2017). La tutela y el derecho a la salud 2016. Bogota: Defensoría del Pueblo . Récupéré sur https://www.queremosdatos.co/request/396/response/773/attach/4/Tutelas%202016.pdf Departamento Nacional de Planeación. (2019). Plan Nacional de Desarollo. Bogota: Departamento Nacional de Planeación. Récupéré sur colaboracion.dnp.gov.co/CDT/Prensa/BasesPND2018- 2022n.pdf IHI. (2003). The Breakthrough Series: IHI’s Collaborative Model for Achieving Breakthrough Improvement. Cambridge, MA: Institute for Healthcare Improvement. Récupéré sur http://www.ihi.org/resources/Pages/IHIWhitePapers/TheBreakthroughSeriesIHIsCollaborativeM odelforAchievingBreakthroughImprovement.aspx Kieny, M.-P., Evans, T. G., Scarpetta, S., Kelley, E. T., Klazinga, N., Forde, I., . . . Donaldson, L. (2018). Delivering Quality Health Services: A Global Imperative for Universal Health Coverage. Washington DC: World Bank Group. Récupéré sur 112 | P a g e www.worldbank.org/en/topic/universalhealthcoverage/publication/delivering-quality-health- services-a-global-imperative-for-universal-health-coverage Kruk, M., Gage, A., Arsenault, C., & Jordan, K. (2018). High-quality health systems in the Sustainable Development Goals era: time for a revolution. Lancet Global Health, e1196-e1252. doi:doi: 10.1016/S2214-109X(18)30386-3 Ministerio de la Protección Social. (2007). Saludatos - Estimación de los Indicadores para el seguimiento del Talento Humano en Salud THS. Bogota: Ministerio de la Protección Social. Récupéré sur https://www.datos.gov.co/Salud-y-Protecci-n-Social/Saludatos-Estimaci-n-de-los-Indicadores- para-el-se/5atd-7cac Ministerio de la Protección Social. (2011). Manual de Acreditación en Salud Ambulatorio y Hospitalario Colombia. Bogota: Ministerio de la Protección Social. Récupéré sur http://www.acreditacionensalud.org.co/Documents/Manual%20AcreditSalud%20AmbulyHosp2 012.pdf Ministerio de Salud. (1995). Resolución 1995 de 8 de julio de 1999, manejo de historia clínica. Bogota: Ministerio de Salud. Récupéré sur www.minsalud.gov.co/Normatividad_Nuevo/RESOLUCI%C3%93N%201995%20DE%201999.pdf Ministerio de Salud y Protección Social. (2012 - 2021). Plan Decenal de Salud Pública. Bogota: Ministerio de Salud y Protección Social. Récupéré sur https://www.minsalud.gov.co/salud/publica/epidemiologia/Paginas/planes-territoriales-de- salud.aspx MPS. (2004). Sistema de Seguridad Social en Salud: Régimen Contributivo. Bogota: Ministerio de la proteccion social. Récupéré sur www.minsalud.gov.co/Documentos%20y%20Publicaciones/GUIA%20INFORMATIVA%20DEL%20 REGIMEN%20CONTRIBUTIVO.pdf MPS. (2006). Decreto Numero 1011. Bogota: Ministerio de la Protección Social. Récupéré sur https://www.minsalud.gov.co/Normatividad_Nuevo/DECRETO%201011%20DE%202006.pdf MPS. (2007). Deecreto 2699. Bogota: Ministerio de la Protección Social. Récupéré sur https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/DIJ/Decreto-2699-de- 2007.pdf MPS. (2007). Deecreto 3518. Bogota: Ministerio de la Protección Social. Récupéré sur https://www.datos.gov.co/Salud-y-Protecci-n-Social/Saludatos-Estimaci-n-de-los-Indicadores- para-el-se/5atd-7cac MPS. (2008). Lineamientos para la implementación de la politica de segurided del paciente. Bogota: Ministerio de la Proteccion Social. Récupéré sur https://minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/LINEAMIENTOS_IMPLEME NTACION_POLITICA_SEGURIDAD_DEL_PACIENTE.pdf 113 | P a g e MPS. (2009). Informe Nacional de Calidad de la Atención en Salud. Bogota: Ministerio de la Protección Social. Récupéré sur http://calisaludable.cali.gov.co/secretario/2010_incas/MPS_INCAS_FINAL.pdf MPS. (2009). Linamientos Tecnicos para el registro de los datos del registro individual de la prestacion de servicios de salud. Bogota: Ministerio de la Proteccion Social. Récupéré sur https://www.minsalud.gov.co/Lineamientos/Lineamientos%20técnicos%20IPS.pdf MSPS. (2012). ABECÉ Resolución 4505. Bogota: Ministerio de Salud y Protección Social. Récupéré sur https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VS/ED/GCFI/abece- resolucion-4505.pdf MSPS. (2013). Plan Decenal de Salud Pública. Bogota: Ministerio de Salud y Protección Social. Récupéré sur https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VS/ED/PSP/PDSP.pdf MSPS. (2013). Resolución 4505. Bogota: Ministerio de Salud y Protección Social. Récupéré sur https://www.cruzblanca.com.co/Descargas/malla-validadora/resolucion-4505-2012.pdf MSPS. (2014). ABC Habilitacion de prestadores de servicios de salud. Bogota: Ministerio de Salud y Protección Social. Récupéré sur https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/VS/PSA/abc-habilitacion- prestadores.pdf MSPS. (2015). Informe Nacional de Calidad de la Atención en Salud. Bogota: Ministerio de Salud y Protección Social. Récupéré sur https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/DIJ/informe-nal-calidad- atencion-salud-2015.pdf MSPS. (2015). Prevenir el cansancio en el personal de salud. Bogota: Ministerio de Salud y Protección Social. Récupéré sur www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/Prevenir-el-cansancio- personal-salud.pdf MSPS. (2016). Plan Nacional de Mejoramiento de la Calidad en Salud, 2016 . Bogota: Ministerio de Salud y Proteccion Social. Récupéré sur https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/Plan-nacional-de- mejoramiento-calidad.pdf MSPS. (2016). Política de Atención Integral en Salud. Bogota: Ministerio de Salud y Protección Social. MSPS. (2016). Resolución 00256. Bogota: Ministerio de Salud y Protección Social. Récupéré sur http://www.acreditacionensalud.org.co/sua/Documents/Resolución%200256%20de%202016%2 0SinfCalidad.pdf MSPS. (2016). Resolución 1441. Bogotá: Ministerio de Salud y Proteccion Social. Récupéré sur http://achc.org.co/documentos/prensa/res-1441- 16%20habilitacion%20redes%20integradas%20ips.pdf 114 | P a g e MSPS. (2017). Encuesta Nacional de Evaluacion de los Servicios de las EPS. Bogota: MSPS. Récupéré sur https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/Informe-encuesta- satisfaccion-eps-2017.pdf MSPS. (2017). What is the Mipres app. Récupéré sur Ministerio de Salud: https://www.minsalud.gov.co/English/Paginas/What-is-the-Mipres-app.aspx MSPS. (2018). Información del talento humano en salud para la gente. RETHUS. Registro ünico Nacional de Talento Humano en Salud. Bogota: Ministerio de Salud y Protección Social. Récupéré sur www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/INEC/INTOR/rethus- observatorio.pdf MSPS. (2018). Sistema de Evaluación y Calificación de Actores: Ranking de Satisfacción EPS 2018. Bogota: Ministerio de Salud y Promocion Social. Récupéré sur https://www.minsalud.gov.co/sites/rid/Lists/BibliotecaDigital/RIDE/DE/CA/Ranking-satisfaccion- eps-2018.pdf MSPS. (2019). Resolución 000244. Bogota: MInisterio de Salud y Promocion Social. OECD. (2016). Education in Colombia. Bogota: Ministerio de Educación de Colombia, OCDE. Récupéré sur www.oecd.org/education/school/Education-in-Colombia-Highlights.pdf OECD. (2016). OECD Reviews of Health Systems: Colombia. Paris: OECD. Récupéré sur http://www.oecd.org/publications/oecd-reviews-of-health-systems-colombia-2015- 9789264248908-en.htm Self-Management Resource Center. (s.d.). Help your community take charge of its health. Récupéré sur www.selfmanagementresource.com Suarez, A. (2015). A Strategy to Improve Treatment-Related Mortality and abandonment of Therapy for Childhood ALL in a Developing Country Reveals the Impact of Treatment Delays. . Pediatric Blood Cancer. Universidad Javeriana y Centro de proyectos para el desarrollo Cendex. (2013). Estudio de disponibilidad y distribucion de la oferta de medicos especialistas. Bogota: Universidad Javeriana y Centro de proyectos para el desarrollo Cendex. Récupéré sur www.minsalud.gov.co/salud/Documents/Observatorio%20Talento%20Humano%20en%20Salud /DisponibilidadDistribuciónMdEspecialistasCendex.pdf. WHO. (2018). Handbook for National Quality Policy and Strategy. Geneva: World Health Organization. Récupéré sur https://apps.who.int/iris/bitstream/handle/10665/272357/9789241565561- eng.pdf?ua=1 115 | P a g e