Report No: AUS0000313 . Nigeria Resource Tracking in Health in Nigeria Volume II . March 26, 2018 . HNP . . 1 . © 2017 The World Bank 1818 H Street NW, Washington DC 20433 Telephone: 202-473-1000; Internet: www.worldbank.org Some rights reserved This work is a product of the staff of The World Bank. The findings, interpretations, and conclusions expressed in this work do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. The World Bank does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of The World Bank concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this work is subject to copyright. Because The World Bank encourages dissemination of its knowledge, this work may be reproduced, in whole or in part, for noncommercial purposes as long as full attribution to this work is given. Attribution—Please cite the work as follows: “World Bank. 2018. Resource Tracking in Health in Nigeria. © World Bank.” All queries on rights and licenses, including subsidiary rights, should be addressed to World Bank Publications, The World Bank Group, 1818 H Street NW, Washington, DC 20433, USA; fax: 202-522-2625; e-mail: pubrights@worldbank.org. 2 RESOURCE TRACKING IN PRIMARY HEALTH CARE IN NIGERIA: CASE STUDY FROM NIGER AND EKITI STATES VOLUME II March 26, 2018 GHN07 Bernard Gauthier, Obert Pimhidzai, Karima Saleh 3 TABLE OF CONTENTS Acknowledgements ........................................................................................................................................ 9 About the authors ........................................................................................................................................ 10 Abbreviations ............................................................................................................................................... 12 CHAPTER 1: Introduction .......................................................................................................................... 14 1.1 Context and Objectives .................................................................................................................... 14 1.2 Methodology and Data..................................................................................................................... 16 1.3 Main findings ................................................................................................................................... 19 1.4 Organization of the report ................................................................................................................ 25 CHAPTER 2: Background: Policy and Institutional Context in the Health Sector ..................................... 27 2.1 Country Context............................................................................................................................... 27 2.2. Health indicators .............................................................................................................................. 28 2.3 Socio-economic and health indicators in Ekiti and Niger States ..................................................... 30 2.4 Health System .................................................................................................................................. 31 2.5 Health Expenditure .......................................................................................................................... 32 2.6 Policy reforms toward PHC services ............................................................................................... 34 2.7 Financing of the health sector and PHC Services ............................................................................ 36 2.8 Resource Flows toward PHC facilities ............................................................................................ 38 2.9 Governance and Accountability in the Health system ..................................................................... 40 CHAPTER 3: Methodology......................................................................................................................... 43 3.1 Rationale of the new PETS approach .............................................................................................. 43 3.2 Overview the new PETS approach .................................................................................................. 44 3.3 Survey instruments and implementation .......................................................................................... 45 3.4 Sampling strategy ............................................................................................................................ 50 3.5 Lessons learnt and challenges .......................................................................................................... 51 CHAPTER 4: Financing of Primary Health Care and Resource Flows toward PHC Facilities .................. 53 4.1 Public Expenditures toward PHC facilities...................................................................................... 53 4.2 Federal level..................................................................................................................................... 54 4.3 FMOH allocation vs release of resources ........................................................................................ 55 4.4 State level......................................................................................................................................... 57 4.5 SMOH allocation vs release of resources in both states .................................................................. 58 4.6 LGA level ........................................................................................................................................ 60 4.7 LGA budget allocation vs release of resources in both states .......................................................... 61 4.8 Transfers .......................................................................................................................................... 64 4 4.9 Overall Public Resources for health and PHC services ................................................................... 66 4.10 Resources available at PHC facility level ........................................................................................ 67 4.11 Overall resources available at PHC level......................................................................................... 72 CHAPTER 5: Resource availability at primary health care facilities .............................................................. 74 5.1. Overview of methodology for estimating the resource envelop of PHC ......................................... 74 5.1.1. Defining the resource envelop ................................................................................................. 74 5.1.2. Valuation of resources ............................................................................................................. 75 5.2. Main Findings .................................................................................................................................. 76 5.2.1. Sources of Financing................................................................................................................ 76 5.2.2. Adequacy of resources............................................................................................................. 80 5.3. Financing for drugs and their availability at PHCs .......................................................................... 84 5.4. Summary .............................................................................................................................................. 90 CHAPTER 6: Performance Contingent Incentives To Motivate Behavioral Change ..................................... 91 6.1. Status of Health Care Expenditure Tracking in Nigeria ....................................................................... 91 6.2. Motivating Reporting ....................................................................................................................... 93 6.3. Behavioral Interventions Design...................................................................................................... 95 6.4. Data .................................................................................................................................................. 96 6.5. Implementation Details .................................................................................................................... 97 6.6. Results from the RCT (treated as two independent experiments).................................................. 100 6.6.1. Summary ................................................................................................................................ 100 6.6.2. Part I of RCT: Social Recognition (vs. Comparison) ............................................................ 101 6.6.3. Part II of RCT: Lottery (vs. Social Recognition) ....................................................................... 103 6.6.4. Analytical note on the Data ................................................................................................... 106 6.7. Insights from the Post-RCT Qualitative Study .............................................................................. 107 6.8. Conclusion & Learnings for Future Implementation of Incentive Measures ................................ 114 CHAPTER 7. Continuous Pets Approach: Evaluation Of The Methodology ............................................... 119 7.1. Introduction.................................................................................................................................... 119 7.2. The continuous PETS approach ..................................................................................................... 120 7.2.1. Rationale ................................................................................................................................ 120 7.2.2. Comparison with traditional PETS: pros and cons of the approaches ................................... 120 7.3. Assessment of survey data ............................................................................................................. 124 7.4. Lessons learned and way forward .................................................................................................. 141 7.5. Conclusion ..................................................................................................................................... 147 5 FIGURES 2.1 Map of Nigeria with highlighted States in the study ……………………………………… 22 2.2 Life expectancy, infant and mother mortality: Nigeria and selected African countries……24 2.3 Health System and levels of care in Nigeria………………………………………….……..27 2.4 Total health expenditures by sources (per capita $ and % of GDP) 2005-2014…………….27 2.5 Public health expenditure vs GDP per capita 2014………………………………….………29 2.6 Nigeria health sector fund flows toward PHC facilities…………………….………………34 3.1 Overview of PETS study survey structure……………………………………………….…41 3.2 Overview of PHC PETS survey instruments and protocol…………………………………43 4.1 Plannes vs Actual expenditures of overall budget by LGA, in Ekiti and Niger (2015)…….56 4.2 PHC official budget and actual expenditure in LGA budget, in Ekiti and Niger (2015……58 4.3 Public resources in the health sector toward PHC services, by levels (2015)………………66 5.1 Composition on PHC resource by state……………………………………………………..73 5.2 Composition on PHC revenues by source…………………………………………………..74 5.3 Composition on PHC non-wage resources………………………………………………….75 5.4 Composition on PHC discretionary revenues by source …………………………………...75 5.5 PHC resources per user and per capita by state……………………………………………..76 5.6 PHC budget allocation and execution by state………………………………………………79 5.7 Disaggregation of drugs value by financing source …………………………………………82 5.8 Disaggregation of drugs value by procurement source………………………………………82 5.9 Drugs stock out rates…………………………………………………………………………83 5.10 Drugs replenishment rates……………………………………………………………………83 6.1a & b Fraction of men and women, respectively, with no formal schooling in Nigeria by region..88 6.2 Average Social Recognition Effects in Weeks 1-4 in Ekiti and Niger ……………………....96 6.3 Results in Ekiti for Specific Questions in Part I of the RCT…………………………………99 6.4a & b 101. Performance Scores in Ekiti and Niger, respectively, throughout the period of the Intervention…………………………………………………………………………………101 6.5 Some key attributes and relationships for primary health care service delivery …………..103 7.1 Weekly average cash receipts from IRG over the 29 weeks in Ekiti and Niger (control PHCs) 128 TABLES 1.1 Structure of the Sample………………………………………………………………..……12 2.1 Health Outcomes Indicators……………………………………………………………..….23 2.2 Socio-economic characteristics of Ekiti and Niger compared to Nigeria…………….…..…25 6 2.3 Types of facilities and characteristics by level of care………………………………..…….26 2.4 Health expenditures in Nigeria vs Africa and low Middle Income Country………………..28 3.1 Overview of Administrative Level questionnaires…………………………………………..44 3.2 Overview of Federal Level questionnaires…………………………………………………..44 4.1 Federal Government Budget (2015 and 2016)………………………………..………..……49 4.2 Structure of the Federa Health Budget (2015 and 2016)……………………………..……..50 4.3 Actual Expenditure of Federal Health Budget by categories (2015)………………….……51 4.4 State Government Budget-Ekiti and Niger (2015 and 2016)……………………………….52 4.5 Structure of the State Health Budget: Ekiti and Niger (2015 and 2016)……………………53 4.6 Actual expenditure of State Health Budget (2015 and 2016-Q1)…………………………..54 4.7 Structure of LGAs Budget: Ekiti and Niger (2015) and 2016)……………………………..55 4.8 Financial and In-kind contributions by main International donors to FMOH vertical programs (2015 and 2016) (in million N)…….………………………………..…….……………….59 4.9 Transfers from International donors/NGO and between levels of governments (in million N)……………………………………………………………………………….60 4.10 Budgeted and actual expenditure for health and PHC services by level of government, $ per capita (2015)………………………………………………………………………….61 4.11 Arrival of public resources in PHC facilities ($ per capita)………………………………..62 4.12 Arrival of Public Resources in PHC facilities (2015)..…………………………………….63 4.13 Public resources per capita in LGA and PHC facilities ($ per capita, 2015)………………64 4.14 Actual contributions to PHC operations (2015)……………………………………………69 5.1 Share of facilities receiving support by source of revenue…………………………………74 5.2 Contribution to PHC resources by source and type of support…………………………….74 5.3 Availability of basic inputs by state and PHC input performance ranking………………...79 5.4 Disaggregation of average resources per PHC facility by source and type ………………80 5.5 Disaggregation of internally generated revenues by type …………………………………81 5.6 Composition of PHC expenditure (per 6 months of survey)………………………………82 5.7 Comparison of drug prices ………………………………………………………………...85 5.8 Comparison of most frequently purchased drugs in Ekiti and Niger ……………………...86 6.1 Sequential 2-Arm Study Design by Week ………………………………………………………………………93 6.2 Selection criteria for facilities involved in qualitative study in Ekiti and Niger ………………96 6.3 Performance Scores by State and Treatment Arm in Part I of the RCT ………………………….98 6.4 Results in Ekiti for Specific Questions in Part I of the RCT ………………………………………….99 6.5 Performance Scores by State and Treatment Arm in Part II of the RCT ……………………….101 7 6.6a & b Likelihood of Any Transactions by State and Treatment Arm, in Parts I and II of the RCT Respectively 103 7.1 Quality of data: PHC Financial support ……………………………………………………………………….123 7.2 Quality of data: PHC in-kind support ………………………………………………………………………….124 7.3 Completeness of reporting: PHC Cashbook receipts ……………………………………………………127 7.4 Mean & frequency of the checklist of the cash book variables, for the control PHCs, with Additional variables......………………………………………………………………….129 7.5 LGA: Completeness of reporting: LGA financial resources……………………………..130 7.6 Consistency of reporting: LGA financial resources……………………………………..131 7.7 Completeness of reporting: LGA in-kind items …………………………………………132 7.8 LGA Completeness: Drug reception 134 7.9 LGA Completeness: Budget and Expenditure …………………………………………..135 7.10 Matching of Reported Transfer and Reception of Resources from LGA to PHC, by state and month………………………………………………………………………………...137 8 Acknowledgements Within the Government of Nigeria, this task was conducted with the support and leadership form the Nigeria Federal Ministry of Health (FMOH). A task force was set up under FMOH leadership and included members from the National Primary Health Care Development Agency (NPHCDA), National Health Insurance Scheme (NHIS), and representation from the State Ministries of Health (SMOH) of Niger and Ekiti States. It also included representation from key development partners, including World Health Organization, and DFID financed PATH2 (Consulting firm: Abt Associates). We would like to acknowledge the leadership of the Director of the Department for Health Planning, Research and Statistics at the FMOH. We acknowledge the commitment and engagement and guidance provided by: Francis Ukwuije and Banji Filani (FMOH); Olubajo Lekan and Lami (NPHCDA). We appreciate close collaboration with the SMOH and State Primary Health Care Development Agencies (SPHCDA) and other State and local government agencies in the two states of Ekiti (Honorable Commissioner Dr. Rotimi Ojo, Dr. Folakemi Olomojobi, Mrs. A. Arogundade (SMOH), and Pharm. Abike Adewole (SPHCDA)] and Niger [Honorable Commissioner Dr. Mustapha Jibrin, Dr. Ibrahim Tiffin, Dr. Joseph Kolo (SMOH) and Dr. Shehu Yabagi (SPHCDA)]. From the World Bank Group, this task was led by Karima Saleh, Senior Economist (Health), Global Practice for Health, Nutrition and Population. The co-authors of volume 2 of this report are (on alphabetical order): Bernard Gauthier (Consultant), Obert Pimhidzai and Karima Saleh (World Bank). The co-authors of Chapter 6 on Performance Contingent Incentives are (on alphabetical order): Julian Jamison, Nina Mazar, Owen Ozier, Shomikho Raha (World Bank), and Temilade Sesan (Consultant), while, Varun Gauri and Karima Saleh provided guidance to this team. Owen Ozier also provided guidance on study survey sampling. Gil Shapira provided guidance on study survey instruments. Kara Adamon helped with drugs and medicines data analysis. The other extended team members were: Bernard Couttolenc, Alberto Gonima, Cyprian Kamaray and Paul Angbazo (Consultants). Research analyst on the team was Egon Tripodi, and other research assistants included: Michael Olugbile, Ghazia Aslam, and Samuel Berube (Consultants). The data collection firm was Hanovia Medical Ltd, Nigeria. We appreciate the support from the program assistants, Essienawan Ekpenyong Essien, who helped make our mission and country engagement easier, Shafali Rajora, Amba Denise Sangara, and Karine Noelle Mouketo-Mikolo who provided administrative and report formatting support. We are grateful to the peer reviewers who provided valuable comments. The peer reviewers were: Pia Schneider, Waly Wane, Tiago Peixoto, and Christophe Rockmore. We also acknowledge the earlier peer reviewers who provided guidance to the team at concept note stage: Deon Filmer and David Evans. We are grateful to the Nigeria Country Management team: Rachid Benmessaoud (Country Director for Nigeria) who chaired the decision meeting for this report and provided valuable directions to the team, to Marie Francoise Marie-Nelly (previous Country Director for Nigeria) for her guidance and commitment to this topic, and to Indira Konjhodzic (Country Program Coordinator) for her support. We are also grateful to Trina Haque (Practice Manger, Global Practice on Health, Nutrition and Population) for her commitment and support. We appreciate the guidance received from various World Bank staff: Ayodeji Oluwole Odutolu, Andreas Seiter, Adewunmi Adekoya, and Ismaila Ceesay. From WHO: Ogo Chukwujekwu, and from PATH2: Benson Obonyo. We are grateful to and appreciate the extended consultation with and support from Hong Wang (BMGF). The financing for this study came from the Bill and Melinda Gates Foundation (BMGF). The Trust Fund was managed by the Global Practice on Health, Nutrition and Population of the World Bank Group. This task was financed under Project #: P152141. 9 About the authors Karima Saleh was the task team leader on this task. She is a Senior Economist (Health) at the World Bank. She has over 20 years of experience working in more than 20 countries, with work experience (including fieldwork) on health financing and service delivery policy and planning in low- and lower-middle-income countries. She holds a PhD in health economics from the Johns Hopkins University. She was part of the core team that developed the 1993 World Development Report, Investing in Health (Washington, DC: World Bank, 1993). Among her recent publications are The Health Sector in Ghana: A Comprehensive Assessment (Washington, DC: World Bank, 2013), Health Financing in Ghana (Washington, DC: World Bank, 2012), and Health Financing in the Republic of Gabon (Washington, DC: 2014). Bernard Gauthier is professor of economics in the Department of Applied Economics at HEC Montreal in Canada. His research covers issues of governance and corruption, the efficiency and equity of public resource allocation, and the quality of public services. In the last two decades, he has conducted applied research work using micro level surveys in social sectors, in particular Public Expenditure Tracking Survey (PETS) and Quantitative Service Delivery Survey (QSDS). He has also contributed to the development of the Service Delivery Indicators (SDI) in education and health. Obert Pimhidzai is a Senior Economist from the Poverty and Equity Global Practice of the World Bank, leading the Practice’s work program in Cambodia, Lao PDR and Vietnam. Since joining the Bank in 2009, Obert worked in Kenya, Uganda, Mongolia and Nigeria, focusing on poverty measurement and welfare monitoring, using poverty diagnostics to inform policy, undertaking service delivery assessments and resource tracking surveys. His current research focus is on ethnic minorities poverty, distributional effects of fiscal policy, welfare impact of infrastructure investments, and equity in service delivery. He has a PhD in Economics from the University of Cape Town. 10 CHAPTER 6 Julian C. Jamison is a behavioral economist whose research uses theoretical modeling, laboratory experiments, and field experiments to explore individual preferences broadly construed, as well as how such characteristics influence decisions, behaviors, outcomes, and well-being across a range of applications – especially in health and finance. Julian is currently a Senior Economist in the eMBeD (Mind, Behavior, and Development) Unit at the World Bank, as well as a Research Associate at Innovations for Poverty Action and an Associate Editor at Behavioral Science & Policy. His research has appeared in academic journals spanning disciplines from economics to information technology to linguistics to medicine to mathematics to philosophy, as well as online at the New York Times, Wall Street Journal, Business Week, Foreign Policy, Bloomberg, Forbes, and Foreign Affairs. Previous professional affiliations include Northwestern’s Kellogg School of Management, the Federal Reserve Bank of Boston, and the US Consumer Financial Protection Bureau, in addition to visiting positions at UC Berkeley, Yale, and Harvard. Julian holds BS and MS degrees in mathematics from Caltech and a PhD in economics from MIT. Shomikho Raha is a Senior Public Sector Specialist in the Governance Global Practice of the World Bank, presently working in the Africa Region. Prior to joining the World Bank, he worked for the Government in India, served as senior adviser to the Asia Foundation and Governance Adviser for DFID (U.K.), covering reform of public administration and public services in health, education and water-sanitation. His work relates to the political economy of public policy reform, human resource management for organizational performance and transparency and accountability initiatives. Raha has published peer-reviewed articles, book chapters and policy reports, which include co-authoring the World Bank flagship publication “Opening the Black Box: The Contextual Drivers of Social Accountability”. He is a member of the WHO’s new Health Systems Governance Collaborative initiative. He holds a PhD, MPhil and MA from Trinity College, Cambridge University with a focus on politics and economic history. Nina Mazar is an Associate Professor of Marketing & Co-Director of BEAR at the Rotman School of Management, University of Toronto, Co-Founder & Chief Scientific Advisor of BEworks Inc., and former Senior Behavioral Scientist at eMBeD, The World Bank. A behavioral scientist, Nina has been named one of “The 40 Most Outstanding B-School Profs Under 40 In The World” (Poets&Quants, 2014). With her focus on behavioral economics, Nina investigates how expectations, emotions, peers, and random cues in the environment affect how we think about products, money, investments, and morality, and their implications for welfare, development, and policy. Nina has published her research in leading academic journals. Popular accounts of her work have appeared among others on NPR, BBC, and in the New York Times. Nina also serves as advisor to a number of governments around the world. Speaking engagements include the Canada Revenue Agency, European Commission, Google Venture, OECD, TEDxToronto, and Toyota. Owen Ozier is an Economist in the World Bank's Development Research Group, Human Development and Public Services Team. He received his Ph.D. in Economics from the University of California at Berkeley in 2010. He has conducted research in Sub-Saharan Africa for fifteen years; he studies education, health, labor markets, and economic decisions in Africa and around the world. 11 Abbreviations ACT Arteminisin-based Combination Therapy ANC Antenatal Care BHCPF Basic Health Care Provision Fund CEO Chief Executive Officer CHEW Community Health Extension Worker CHO Community Health Officer CMS Central Medical Stores CPETS Continuous Public Expenditure Tracking Survey DHS Demographic Health Survey DPHC Department of Primary Health Care DRF Drug Revolving Fund EHO Environmental Health Officer FBO Faith Based Organizations FCT Federal Capital Territory FMOH Federal Ministry of Health GHE Government Efforts in Health Financing HCP Health Coverage for Poor HIV/AIDS Human Immunodeficiency Virus Infection & Acquired Immune Deficiency Syndrome IGR Internally Generated Revenues IMCI Integrated Management and Childhood Illness ITN Insecticide Treated bed Nets LG Local Government LGA Local Government Administration LMI Lower Middle Income M&S Monitoring and Supervision M&V Monitoring and Verification MCH Maternal and Child Health MDG Millenium Development Goals MEP Malaria Elimination Program MNCH Maternal, Newborn and Child Health MSH Management Science for Health MSS Midwives Service Scheme MTFF Medium Term Fiscal Framework MTSS Medium Term Sector Strategy NCD Non Communicable Diseases NG Nigerian Niara NGO Non-Governmental Organization NHA National Health Act NHIS National Health Insurance Scheme NHIMS National Health Information Management System NHIS Natinal Health Insurance Scheme NHP National Health Policy NHSDP National Health Strategic Development Plan NPHCDA National Primary Health Care Development Agency NPI National Program on Immunization NSHIP Nigerian States Health Investment Project OIC Officer In Charge OPD Outpatient PBF Performance Based Financing 12 PEP Polio Eradication Program PETS Public Expenditure Tracking Survey PHC Primary Health Care PHCUOR Primary Health Care Under One Roof PMV Patent Medicine Vendor QSDS Quantitative Service Delivery Survey QSTART Quality of Service Assessment and Resouce Tracking RCT Randomized Control Trial RDA Rapid Data Assessment RIE Randomized Impact Evaluation RT Resource Tracking SDI Service Delivery Indicators SMOH State Ministry of Health SOML Saving One Million Lives Initiative SPHCDA State Prtimary Health Care Development Agency SSHDP State Strategic Health Development Plan STI Sexually Transmitted Infections TB Tuberculosis TGE Total Government Expenditure THE Total Health Expenditure USD United States Dollar VAT Value Added Taxes VCT Voluntary Counselling and Testing WDI World Development Indicators WHO World Health Organization WMHP Ward Minimum Health Package 13 CHAPTER 1: Introduction With health outcomes worse than the average for African countries despite large investments in the health sector during the last decades, Nigeria’s Primary Health Care (PHC) services faces a number of significant challenges. Efficiency and equity concerns draw specific attention to PHC services as many of the most cost- effective preventive and curative health interventions can be offered at that level, with the poor being more likely than the rich to seek care in PHC facilities. The Government of Nigeria has developed a new result oriented National Strategic Health Development Plan (NSHDP) which focuses on PHC services. The implementation of the NSHDP requires ensuring efficient use of resources as well as accountability of expenditure and for results throughout the health care system, especially at the PHC level. This study aims to contribute to a better understanding of the linkages between public expenditure and results in the Nigerian health sector by assessing resources effectively channelled toward PHC services and identifying institutional and financial constraints. It seeks to propose an innovative continuous Public Expenditure Tracking Survey (PETS) to record and aggregate information on resource flows in real time and over time, instead of the more established practice of generating such data through traditional (retrospective) PETS. Over the years, the government of Nigeria has not been able to reliably track information on how much it spends on PHC services from the various levels of governments. This is partly due to too broad classifications of expenditure in the health sector to capture such granular information, the fragmented nature of these flows toward PHC services and the absence of a system to aggregate them in the context of a highly-decentralized federation. The need to devise alternative means of tracking income and expenditure in the health sector is especially great at the PHC facility level, for which no one – including state and local governments – has reliable information on income and expenditure streams from governments, as well as from other sources, including donors and Internally Generated Revenues (IGR). One of the central issues identified by previous studies is the weakness of monitoring and accountability structures at the PHC facility level – a gap that the current Resource Tracking (RT) study introducing the ‘continuous PETS’ attempted to address. This report presents the results of the RT study in the Nigerian health sector implemented in two states (Ekiti and Niger) in 2015 and 2016 using a continuous Public Expenditure Tracking Survey (PETS) approach. This RT exercise is part of a study on Quality of Service Assessment and Resource Tracking (QSART) initiated by the Bill and Melinda Gates Foundation and the World Bank, which aims at providing robust evidence on the quality of primary health care services and resource flows in the primary health sector in selected states in Nigeria. 1.1 Context and Objectives With only 2% of the world’s population, Nigeria has 10% of the world’s maternal deaths, 10% of the world’s under-five mortality and 25% of the world malaria burden. Furthermore, Nigeria has not been able to achieve the health MDG goals, falling short by 90% of reducing infant mortality rate by two-third, while reduction in under-five mortality fell short by 28% (MDG Report 2015). Despite a significant increase of government health expenditure (GHE) to 8.2 % of total government expenditure in 2014 from 5.9% in 2000, public health expenditure in Nigeria remains much below what is 14 observed in comparable countries, representing only 1% of GDP, less than one third of what is observed on average in other African and Lower-Middle-Income (LMI) countries. Furthermore, total health expenditure (THE) is 43% below African region average (216 US$ (PPP) per capita vs 308 $ (PPP), with government financing representing less than 25% of THE (in PPP) compared to close to 60% on average in the Africa region (NHA 2016). In low resource settings, such as Nigeria, a well-functioning public expenditure and service delivery system is a critical determinant of both quantity and quality of care. However, Nigeria’s health system organization and financing is fragmented, with funding originating from different sources and flowing to different levels using various allocations. Furthermore, information systems for monitoring the funding and delivery of health services are weak, as studies undertaken in Nigeria in recent years have indicated (Couttolenc 2013, World Bank 2014). Poor information and systems contribute to poor accountability of service providers to various levels of government and to society in general. As part of the first component of QSART, the Service Delivery Indicators (SDI) survey implemented in the spring of 2013, allowed measuring and benchmarking quality of services among 1200 PHC facilities across 6 states in Nigeria. It showed important deficiencies in service delivery at the PHC level. The findings of the small retrospective RT module implemented as part of the SDI at the health facility level highlighted the poor information and record keeping practices at the PHC level, with data by sources of financing and expenditure classification, including internally generated revenue, being mostly unavailable (World Bank 2014). Under this second component of QSART, the retrospective RT situation analysis report in the fall of 2013 indicated that information on PHC revenues and expenditure is also unknown at the district or state levels (Couttolenc 2013). This lack of information on resource allocation and usage at PHC level is associated with weak capacity and accountability mechanisms, compounded by challenges present in the Nigeria health system. As other African countries, most of the resources transferred to PHC facilities are in-kind (e.g. drugs, vaccines, medical consumables, health equipment). Furthermore, most PHC facilities do not keep records on resource reception and usage, and routine health information systems are poorly developed and used. In such context of weak information and accountability systems in which the health system is unable to generate reliable information on PHC financing and spending, a traditional retrospective PETS based on available records, would have faced important challenges. A different PETS approach was hence devised making use of a continuous data collection method. This continuous PETS approach sought to collect information on resources and activities on a prospective, real time basis, instead of retrospectively. It entailed tracking public expenditure through the various layers of the healthcare system toward PHC services and implementing real time information recording and reporting tools and capacity building at the PHC and Local Government Administration (LGA) levels, to allow continuous data recording and verification to assess resource flows from the various layers of governments and vertical health programs to the PHC units. This study’s overall objective is to strengthen the performance, accountability and monitoring systems of PHC activities in order to produce better PHC services and ultimately improve health outcomes. More specifically, the main objectives of the study are: (i) To better understand the financing flows and identify bottlenecks in transferring and receiving resources through the health administrative system down to the PHC facility level. (ii) To assess the level of resources allocated toward PHC services actually reaching these PHC facilities, in particular, to determine the level of resources available at PHC level by state, including from internally generated revenues; 15 The study also aims to assess if the system developed at the PHC facility to strengthen and monitor resource flows on a continuous basis is able to: (a) improve resource flow monitoring, (b) allow a better understanding of where resources are spent, and (c) demonstrate who benefits from these resources. Ultimately, the study should propose plan of actions to improve efficiency, effectiveness and equity of the public resources. 1.2 Methodology and Data The continuous PETS study in Nigeria part of the second component of QSART has two main components: i) a tracking of public expenditure through the various layers of the healthcare system with data collection and monitoring of resources during a 6 month period at the PHC and LGA levels, and ii) an incentive intervention at the facility level to test the effectiveness of different types of external incentives (monetary and social) to improve reporting of internally generated revenues by facilities (i.e. cash book). (i) Continuous PETS study PETS were developed in the last two decades to collect micro-level information on service delivery systems and service provider performance. These surveys were implemented in response to perceived problems in public expenditure management and service delivery. PETS have proved to be powerful instruments at identifying several bottlenecks, inefficiencies and wastages in service delivery, in particular, problems of efficiency, equity, leakage, delays, record keeping, information, user fees and decentralization. PETS methodology normally involves collecting retrospective data on at least a full fiscal year at the various administrative level in the service delivery chain. The continuous PETS approach developed in Nigeria in the context of weak governance environment and poor data recording and reporting, especially at the PHC level, has entailed implementing a program of information recording and reporting at the PHC and Local Government Administration (LGA) levels, allowing continuous data recording and verification. Survey tools were devised to register and capture transactions as they happened, each time a transaction was made. Officers in Charge (OICs) (or those responsible for recording information such as monitoring and evaluation officers) at PHC facilities were trained to make use of these new information tools and practices as well as district health officers/M&E officers (responsible for data consolidation and analysis) at the LGA level. Weekly visits by enumerators served to supervise and observe the uptake of these RT tools. The continuous PETS has sought to allow continuous data recording and verification of in-kind and financial resources reception, expenditure and activities within the PHC sector in 2 selected States in Nigeria, Ekiti and Niger. The continuous PETS survey was implemented over 6 months, between October 1 2015 and March 30 2016, covering Q4 of FY 2015 and Q1 of FY 2016 among 140 public PHC facilities (65 in Ekiti and 75 in Niger), as well as 41 LGAs (15 in Ekiti and 26 in Niger). Despite that the private sector, through private for-profit and private not for-profit health facilities, in particular faith-based organizations (FBO) and NGOs, plays a large role in the provision of health care in Nigeria, this study focuses solely on the public sector. 16 Retrospective PETS tools were also used to collect information in federal and state level line-ministries (e.g. Ministry of Health, Primary Health Care Development Administration, PHCDA) to assess overall resources budgeted and transferred toward PHC services. Furthermore, surveys of patients exiting PHC facilities were conducted weekly during the survey period to collect information on patients’ characteristics, facility usage, costs of services and satisfaction with services. Table 1.1 presents the various sample units among which data were collected. Table 1.1. : Structure of the Sample Units Number Federal Health Ministry 1 National PHCDA 1 State Health Ministries 2 State PHCDA 2 Local Government Administrations (LGAs) 41 Health facilities 140 Patients 3454 In addition to primary data generated by the survey, the study made use of secondary budget data to assess resources officially budgeted for PHC programs. Five main elements were assessed in the PETS study: i. Resources (inputs) mobilized for PHC programs: their sources and importance (including user’s contributions and other sources); ii. Ways (process) in which these resources are mobilized: programs and allocation mechanisms used and various channels toward PHC facilities; iii. Ways (process) in which these resources are used: expenditure centres and usage at central, decentralized, and PHC facility levels; iv. Inefficiencies and inequities in resource allocation within the health system toward PHCs v. Results obtained by these resources: intermediate and final outputs, and health outcomes. Ultimately, the study sought to enhance systems of recording and reporting at the PHC level, to promote better systems of monitoring, accountability and governance on resource flows and usage. (ii) Incentive intervention at the facility level to improve data recording. Reforming resource management in health care is an ambitious task. The information and knowledge that accompanies the RT activities, including supervision, are potentially important factors in improving data recording. However, it may not be sufficient to change attitudes and instill a desire to change behavior particularly at the facility level where RT practices have been historically below par. In such context, in addition to the continuous PETS study, a randomized controlled trial was implemented in both states to compare the effectiveness of two approaches to improve the record keeping by facility staff 17 through extrinsic incentives. More specifically, the intervention tested the effectiveness of lottery tickets as a material reward in comparison to social recognition, to increase extrinsic motivation of health facility workers to improve the quality of their record-keeping of the ‘cash book’ form relating to drugs. The motivation for the intervention draws from the nascent literature studying how to motivate agents engaged in pro-social activities and a growing body of evidence suggesting that “nudges” to influence behavioral choices and individual motivations can be used to address problems in public service delivery. For health facility staff, low supervision and lack of intrinsic motivation to perform mundane administrative tasks provide little incentive for proper RT. Behavior change with regard to record keeping can be driven by both intrinsic and extrinsic motivations. While the former motivates positive change by making a task “inherently interesting or enjoyable”, the latter in particular, tends to be effective in situations where people show little interest in potentially useful activities (like RT) and this was to be used in the intervention. Lottery tickets as a reward for good performance, for example, can be particularly attractive by scaling the use of more valuable material reward incentives. In addition, individuals crave status, and therefore can be positively motivated by public and peer recognition. In both states, the intervention compared the effectiveness of a non-monetary incentive, i.e. social recognition, with an incentive that improves the chances of the facility (as a collective of individuals) winning an attractive prize at a lottery held at the end of the program period. Two groups in both states were testing two different program interventions against one another, based on the extent to which scores improve during weekly observations at the facilities visited by enumerators during the study period of 2 months. By randomizing facilities into different treatments, the intervention design sought to isolate the causal effects of approaches that have been identified as promising methods to encourage such behavior. Two experiments were conducted in the sampled 140 medical facilities in Ekiti and Niger, using the cashbook form that was introduced as part of the PETS to record daily receipts and expenditures (in cash or bank transactions) on 7 specific revenue streams and 20 expenditure lines. The effectiveness of social recognition to incentivize the filling of the cashbook was tested in a first experiment (weeks 1-4) by assigning half of the facilities in each state to the intervention. In those sites, enumerators that weekly visited the facilities scored the quality of the cashbook filling using a standard protocol. These scores were converted to a number of stars displayed in a public place on a Certificate of Excellence for patients to see.1 The other half of facilities simply followed the standard procedure in which the enumerators weekly visited and scored the quality of the cashbook filling, but without sharing the outcome with facility administrators and the public. In a second experiment (weeks 5-8), the effectiveness of a lottery was compared with that of the social recognition approach. This was done by assigning half of the facilities that had experienced the standard procedure in the first experiment to a lottery treatment where enumerators’ weekly score was converted to number of lottery tickets towards a raffle at the end of the four week- period. The other half of facilities continued to receive the social recognition incentive. In the context of weak resource management practices and accountability, the intervention allows to provide quantitative experimental evidence, paired with qualitative descriptive evidence, on the effects of two performance-contingent incentives designed to encourage health facilities to improve the quality with which they report IGR (user fees). 1 In addition, the staff of the best-performing facility that participated in filling the cashbook were commended and posed for photographs with the Permanent Secretary of Health in a special ceremony at the end of the four-week period. 18 1.3 Main findings The study allows shedding light on the specificities of the Nigerian health system’s institutional arrangement that have consequences on the performance of the health system and especially the PHC services. It also allows assessing the level of public resources officially and in practice allocated toward PHC services from the three levels of government in the two states, and estimating the shares and levels of resources actually available at the PHC level from governments, as well as from other sources, including international donors, NGOs and IGR. The study also identifies income composition at the PHC facility level in terms of wage and non-wage contributions and adequacy of resources available at PHC facilities in both states. One of the major contribution of the continuous PETS in Nigeria is to design and introduce relatively simple- to-fill-out forms (i.e. Excel spreadsheets) to track various revenue (e.g. financial and in-kind items, user fees, drug sales) and expenditure flows (i.e. purchases of drugs, equipment, materials, supplies, facility maintenance), and to teach staff at the facility level to fill out these forms on a daily basis. The study also allows testing two behavioral interventions potentially improving the incentives and performance in resource management at the facility level, a social recognition and a material prize lottery. Institutional arrangements The health system in Nigeria is hampered by important challenges relating to governance and accountability in the use of resources and services. Nigeria operates within a system of fiscal federalism characterized by extensive intergovernmental fiscal decentralization through fiscal autonomy and responsibility attributed to subnational (State and Local) governments. In particular, the states and LGA have independent revenue sources recognized by the constitution and the states’ budget are not subject to federal control and scrutiny. Health, according to the Nigerian Constitution, is a shared responsibility of the federal, state and local governments. PHC services, in particular, are the main responsibility of LGAs, with federal and state administrations participating in the management and financing of these services. These arrangements create difficulties in the coordination and management, with regard to, health care policies as expenditure decisions of the three tiers of government are taken independently, including in the health sector. This implies that the FMOH has no significant influence on funds allocated by subnational governments to secondary and primary health care services– except for special programmes and projects funded through some agencies. Similarly, state governments have limited control over how LGAs allocates resources toward primary health services (even though the constitution gives the state supervisory role over the LGAs). There is minimal coordination between these agencies and levels. In particular, there are no requirements for reporting public budget and expenditures to federal government or between administrative levels. There is no mechanisms or organization responsible for collecting health expenditure data across budget categories, tiers of government or types of services. This lack of transparency and accountability in the government system, and in particular, in health expenditure, combined with poor coordination between levels of government, have greatly weakened the governance and effectiveness of the system as a whole. 19 Because of system fragmentation and poor information systems, there is a paucity of reliable data across the country on government activities and financing of the health system. Budgetary information is often not readily available, consolidated or comparable. Resource Flows toward PHCs The institutional features of the Nigerian health system have influenced the RT strategy and led to the development of the continuous PETS approach. Still, the RT exercise was made difficult due to the paucity of official information at various levels, but also to the fact that mainly in-kind resources from multiple sources and allocated through different channels, are transferred to PHC facilities. The continuous tracking exercise over the 6-month period in both states in addition to retrospective survey at the other administrative levels and analysis of the official budgets at each level has allowed assessing the share of the health budget that is allocated and that reaches PHC facilities and available for primary care service delivery. It has also allowed estimating overall resources available for PHC services at the facility level, from all sources, including IGR. Based on the analysis of the budgets of the federal, state and local governments, allocation mechanisms and the data collected through the survey among PHC facilities and administrative units, the study shows that financing of PHC services in Nigeria, other than personnel, is mainly realized through discretionary financing at the LGA level. The federal government does not provide (conditional) cash transfers toward lower government levels other than through MOH vertical program direct interventions. However, the importance of these interventions is relatively small, as well as support by states, and from private and foreign donors toward PHC facilities. Overall, PHC services in Nigeria are financed through three main sources: (i) public expenditures at various government levels; (ii) international donors; (iii) private financing and internally generated funds within PHC facilities. In the context of the fiscal federalism system in place in Nigeria, measuring public health expenditures as well as the other components of overall health expenditure presents several hurdles, in particular regarding PHC services. Given the shared responsibility of the federal, state and local governments in financing PHC services and the absence of mechanisms or organization responsible for collecting health expenditure data across budget categories, tiers of government or types of services. -- measuring officially allocated health expenditure (financial and in-kind) and actual resources available at the PHC facility level involves measuring budget and expenditure at each level (federal, state and LGA), then accounting for transfers between governments, as well as support from external sources, including donor agencies and IGR (user fees). For the two FY examined in the study, 2015 and 2016, the overall federal health budget remained stable at about 250 billion Naira but decreased in relative share within the federal budget (from 5.1 to 4.1%). During the period, the PHC budget (including the main agencies and administrations involved in PHC services at the federal level, that is NPHCDA and vertical programs) increased by close to one third during the period (from 14,5 billion N to 20,4 billion N). However, the PHC budget’s execution rate, in the context of important decrease in revenues of oil production, was only about 60% of the budgeted envelope for FY 2015. Ultimately, the actual spending for PHC services was hence about half the share of the officially allocated federal health budget during that year (3.1 % compared to 6.1%). At the state level, in Ekiti, the health budget in 2015 represented a relatively larger share of the overall state budget than in Niger (6.2% vs 3.1%). In face of decreasing oil revenue share among states, the overall state budget in Ekiti decreased by about 17% between 2015 and 2016 with the health sector budget taking the most severe blow with a decline of 76% over the period, dropping to a meager 0.7% of the overall state budget. In 20 Niger during the same period, the reverse situation was observed with the overall state budget decreasing by a mere 2%, and the health sector budget increasing by a quarter, reaching about 4% of the state budget. With regard to PHC services, official PHC expenditure in Ekiti represented 6.1% of the state health budget in 2015 and increased to 16.6% in 2016 in the context of a severely shrinking health budget. In per capita terms however, the PHC budget dropped by close to four fifth compared to the previous year, from 131 N in 2015 to 34 N in 2016. In Niger, the official PHC budget in 2015 represented a stunning 43.1% of the state health budget. However, in both states, apart from personnel cost, most expenditure components of the budget were hardly implemented. In particular, the execution rate of the SPHCDA budget in Ekiti was only 10.8% of the official budget (3.9%.for the first quarter of 2016). In Niger, higher execution rates were observed but represented only about one-fifth for the health budget and one-third of the approved budget for PHC services. At the LGA level, where most of the funding for PHC activities is observed, the budget for LGA’s Health Departments which support PHC activities in the districts was much larger compared to the state level health budget, especially the one devoted to PHC activities (SPHCDA), by a factor of 17 to 1 in Ekiti, and of 9 to 1 in Niger (for the entire state health budget the ratio is 2.6 to 1 in both Ekiti and Niger). A higher execution rate of the budget was observed across LGAs in both states compared to federal and state budgets. Still, the execution rate of the PHC budget in 2015 across LGAs was about 60% of approved budget (58% in Ekiti and 62% in Niger), with very low rate of actual expenditure of capital and overhead budgets (less than 5% in both states). Aggregating public resources available for health from all levels of government as well as international donors/NGOs, official public health resources per capita totaled 25.4 $ in Ekiti, and 18.3 $ in Niger. Actual expenditure for health are lower, about 18 $ per capita in Ekiti and 15$ in Niger. When excluding human resources, actual expenditures for health represent about 2 $ per capita in Ekiti and 3$ in Niger. With regard to PHC service expenditure from all levels of governments, total actual health expenditure was about 6 $ per capita in both states and represented about 1.5 $ per capita in Ekiti and 1.8 $ per capita in Niger excluding human resources. At the PHC facility level, public resources available were estimated using the information of reported resources received at the PHC level collected over the 7-month survey period within the representative sample facilities and using sampling weights, to derive values at the state level. Public resources available at PHC facility level for service delivery, excluding human resources, amount to about 7 cents per capita in Ekiti and 4 cents in Niger. When human resources are accounted for, public resources made available for PHC services amounts to 3$ per capita in Ekiti and 4$ in Niger. Of these public resources, about 98% (99% in Niger) comes from the local government level and 2.3% from the state level in Ekiti (0.8% in Niger), while the federal level contribution is negligible. The study also allowed assessing the shares of the budget reaching PHC facilities. While budget officially allocated to PHC activities represents about half of the total health budget) in both states (excluding and including salaries, the share of non-wage resources reaching PHC facilities and available for service delivery is estimated at less than 1% of the official non-wage PHC budget in both states. Including wages, the share of the official health budget reaching PHC facilities is only 12.4% of the official health budget in Ekiti (22.7% in Niger). 21 Given such low support from government sources, PHC facilities rely to a large part on IGR (user fees), especially in Niger, and donor contribution (especially in Ekiti) to finance their activities. Excluding human resources, government transfers account for only about 4.7% of PHC facilities’ revenues, while the large majority of non-wage revenues derives from user fees (67%) and donors’ direct and indirect transfers (about 18%)2. Important differences are observed between the two states as PHC centers in Niger rely at 85.8% on user fees (41 cents per capita) to finance their non-wage expenditures, compared to 36.6% in Ekiti (4.5 cents per capita). Donors contribute to 54% of non-wage revenues in Ekiti compared to only 14% in Niger. In per capita terms, user fees contribute 27 cents on average across the two states, donors contribute about 0.06$ per capita on average in both states in direct and indirect support other than wages, while government provides less than 1 cent per capita to PHC health centers. Including wages, however, government transfers account for about almost all PHC revenues of the 4$ per capita of overall revenues of PHCs centres in both states. The second largest source of PHC revenues is user fees, which accounts for 6.1% of total revenues. Availability of Resources at PHC level The PHC analysis in chapter 5 centers on the relative contributions of all types of revenue sources (government, donors and internally generated revenues (IGRs), to PHC facilities’ overall resource envelope. It also looks at their composition in terms of wage and non-wage contributions and what that means for adequacy of resources available at PHC facilities or the prioritization of getting resources to front line health delivery centers. The chapter shows that the government is the most important source of resources of public primary health care facilities. However, its support to facilities almost entirely comprises of wages and salaries. During the period of the survey, none of the facilities in the sample received cash from the government in Ekiti and only one percent received such support from the government in Niger. Outside donor provided support, government’s spending of its own resources to primary health care is very limited. Adjusted for their contribution through government channels, donor support constitutes 12 and 9 percent of support to PHC facilities. Donors – international agencies in particular- provide less of their support directly to facilities, which is reflected in such support accounting for less than 1 percent of PHC facilities’ resources. The largest value of non-government support is made by international agencies and NGOs to the government instead, especially through vertical programs and other projects. A combination of the shortfall in non-wage spending and that a smaller share of government’s health budget is devoted to primary care, are the reason why resources at PHC facilities are inadequate, and spending is less responsive to gaps in availability of drugs, medical equipment and infrastructure. Facilities do not receive any discretionary funding from the government and that coming from non-government sources is negligible. They try to make up for the short fall using IGR but this also is not adequate. While user fees provide a relatively significant contribution in Niger, they have a much smaller importance in Ekiti. In the former, user fees accounted for 12 percent of resources, making them the second most important source of revenues at the facility level. However, they did not contribute much in Ekiti where they made up only 3 percent of revenues received. 2 Direct donor contributions are resources directly allocated by donors to PHC facilities reported by PHC facilities through the survey modules. Donors’ indirect contributions to PHCs are support provided by donors through federal and state transfers, in particular cash and in-kind support by donors to FMOH vertical programs and SPHCDA for PHC services (for non-salary purposes). 22 Based on actual revenues from all sources, the resource envelop of PHC facilities is quite low, when considering the demands against these facilities. Resources at PHC facilities (wages and salaries included), range from USD 15 to USD 22 per user, based on prevailing exchange rates at the time. The amounts are much lower in per capita terms as the resources PHCs receive and generate amount to N 235 per capita in Ekiti and N 533 per capita in Niger, thus equivalent to between USD 1 – 2.5 per capita. The amount of resources received per capita in Ekiti and Niger, indicates that only a small share of public spending on health reached PHC facilities and points to lack of prioritization of primary health care. Public spending is either concentrated on curative rather than preventative care or a greater share of this spending is not devoted to frontline service provision. Excluding user charges and direct non-government support, shows that government supported public PHC facilities to the tune of NG 225 and NG 465 per capita in Ekiti and Niger respectively, that is equivalent range from USD 1 to USD 2 per capita. This pales in comparison to the estimate of USD 16 and USD 19 government per capita spending in Ekiti and Niger respectively. Lack of basic infrastructure, equipment and drugs at primary health facilities attests to the inadequacy resources at Primary Health Care facilities. On average, facilities only had 2 of the six basic medical equipment in the minimum equipment SDI indicator (sphygmonometer, thermometer stethoscope, weighting scale, sterilizer and refrigerator). They had less than 5 percent of tracer drugs available and only 21 percent had the minimum infrastructure (electricity, improved water source and improved toilets). The lack of basic medical equipment was especially glaring. Despite the poor availability of medical supplies and equipment, facilities barely received them from the government or non-government sources. In both states, less than 3 percent received medical equipment of any kind. Regression results of aggregate input availability indicators show these to be significantly correlated with receipt of drugs from NGOs or directly from the government and amount of discretionary funding received by facilities, which is mostly internally generated revenues. Without direct cash support and with low government spending on non-wage health expenditures, out-of- pocket spending is the most important source of financing for drugs, medical supplies and other expenses. Facilities generated most of their internal revenues from selling drugs, representing up 75 percent of IGRs in Ekiti and 85 percent in Niger. However, the procurement sources of drugs are starkly different. Public procurement accounted for a greater share drugs purchases and supplies in Ekiti (about 90 percent of the value) and the private sector for a much larger share in Niger (80 percent of the value). With limited contribution from the government, the supply of drugs is inadequate, irrespective of how they are procured. The share of tracer drugs available each month barely exceeded 6 percent in Ekiti and was less than 13 percent in Niger. Randomized Controlled Trial: Performance Contingent Incentives With regard to the performance contingent incentive experiment conducted in both states to compare the impact of two types of external incentives: social recognition vs lottery, the results in Chapter 6 show that despite parallel program implementation in both states, the two states exhibit substantially divergent impact for both experiments. Experiment 1’s results showed that in Ekiti state, social recognition significantly improved data recording in comparison to the standard data recording procedure without the intervention (our comparison group). In Niger, the social recognition incentive did not have significant impact on data recording behavior. A number of institutional and demographic differences between the two states could be responsible for the differential efficacy of the intervention. 23 Experiment 2’s results regarding the impact of a lottery in Niger showed no significant effect. That is, there was no significant difference between the effectiveness of the lottery and social recognition incentives. In Ekiti, specificities in the study design and the results trends during Experiment 1, made it impossible to draw conclusions between the lottery and social recognition interventions. The results suggest that the same incentive schemes can produce different impacts according to contextual factors at the state, community and facility levels, even when health services are coordinated by a single federal ministry of health operating under the same overall organizational structure. Evaluation of the CPETS Approach Because of capacity and budget constraints, the evaluation of the new continuous data recording intervention could not be performed using a rigorous randomized impact evaluation (RIE) at the time of the survey implementation. Still, while not benefiting from the rigorous RIE methodology, Chapter 7 seeks to be a first step trying to analyze ex-post the successes and drawbacks of this first CPETS approach, in order to highlight lessons learned and propose recommendations for future tracking survey. After reviewing the main rationale of the new continuous approach, the chapter reviews some of the potential pros and cons of the continuous vs traditional retrospective PETS approach. It then assesses the performance of the tools implemented in Nigeria, analyzing the survey data quality along two dimensions: (i) frequency of entry (i.e. were the instruments used to record the resources and activities) in order to measure capacity, (ii) quality of entry, that is was the information collected consistent and potentially accurate. Furthermore, another key objective of the CPETS was to build capacity and incentives to accomplish data recording activities and in particular to promote the sustainability of these practices in particular through the institutionalization of the CPETS recording tools following the project completion. The analysis of data quality is performed on questionnaires addressed to PHCs and LGAs, being the two levels for which continuous data tools and protocols were applied (i.e. health staff and officials being the main data recorders). At the other administrative levels, survey tools administered consisted of traditional retrospective PETS instruments with data collection performed by enumerators. Potential data triangulation between reported transfers by LGA toward PHC facilities vs reported reception of resources at the facility level by PHC is examined to assess data consistency. This evaluation analysis of the CPETS implementation in Nigeria is inconclusive on both dimensions of data quality and sustainability. While greater record does exist for the period examined especially at the LGA and PHC levels, resulting from the continuous data collection exercise, it is unclear how to evaluate quality, consistency and sustainability of these data. The analysis of frequency of reporting and consistency clearly show highly un-frequent transactions recorded and low levels of valued transfers and activities across PHCs. We could suspect underreporting, especially for IGR and clear mismatch between PHC and LGAs, hinting toward underreporting of sending by LGA. However, the absence of adequate benchmarking information precludes such conclusions. Furthermore, it is unclear, in both states’ facilities and LGAs, if the instruments were adopted following the completion of the project and if data availability and quality has improved. A review a few months after the end of the survey to measure tools’ utilisation and presence of records would be necessary to answer these fundamental questions. Various of the challenges and ultimate shortcoming faced by the CPETS are similar to the ones facing traditional PETS. However, the extended length of the survey period and the introduction of a new layer of 24 agents (government officials) external to the survey firm with potentially weak incentives to self-recording, adds to the challenge of the approach. Various methodological choices made in the Nigeria CPETS (especially the wide scope and census approach vs sample) added layers of complexity to the design and customization of the continuous data recording instruments. The sheer number of items, especially medical supplies and drugs in particular, with wide variety of formats, dosage and packaging, made it especially complex to codify and exhaust possibilities of potential item reception in LGAs and PHCs in each state. This was unavoidable as the cash equivalent of these drug receipts had to be conducted to understand the cost of oprations of a primary health care facility. Howver, for the health staff and officials, it represented an important task of recording resources across this universe of items. In future CPETS, continuous real-time quality control procedures need to be implemented. These quality control systems would take advantage of being put in place in the context of electronic data entry at each level, allowing continuous monitoring of data captured, with criterions to inform on slacks and under reporting. Given the added complexity of un-frequent transfers, from many various sources and levels, and benchmarking difficulties in the presence of self-reporting by health agents, an additional verification and benchmarking mechanism could take the form of a controlled shipment or “stress test”. Such mechanism could seek to test the proper recording of some invoices of goods or finances through the supply chain system, for instance a control shipment or an already planned shipment in coordination with a specific donor or administrative level. Given the reliance on health staff and government officials in the continuous data recording, training and capacity building of these first line agents need to be recognized and supported. Proper identification and participation of the facility staff and district officials that will become responsible at each data recording levels of using the instruments and inclusion in the training and capacity building activities are an essential component of successful CPETS. Future CPETS should include a review of capacity building, development and tool adoption to promote sustainability. This would involve analyzing if the PHC personnel and LGA officials have continued daily tracking and information collection following the survey period institutionalization of the tools introduction to the effect and to assess the quality of information generated and available. 1.4 Organization of the report The report organized as follows: Chapter 2 presents an overview of the health sector in Nigeria. It describes the institutional structure and policy context within the sector as well as the financing structure and flow of funds for PHC services. Chapter 3 presents an overview of the continuous PETS methodology used in the health sector, as well as its rationale, and presents the sampling strategy and instruments. Chapter 4 examines financing of primary health care services in Ekiti and Niger states. Based on the budgets of the federal, state and local governments and on the data collected through the survey among PHC facilities and administrative units, it assesses the resources effectively available at each level down to PHC facilities, the main service delivery point for the population. Chapter 5 examines resource envelopes available among PHC facilities. The analysis centers on the relative contributions of all types of revenue sources (government, donors and internally generated revenues (IGRs), to PHC facilities’ overall resource envelope. It also looks at their composition in terms of wage and non-wage contributions and what that means for adequacy of resources available at PHC facilities or the prioritization of getting resources to front line health delivery centers. Chapter 6 presents the results of performance contingent incentive experiments conducted in both states to compare the impact of two types of external incentives: social recognition vs lottery. Finally, chapter 7 presents an ex-post evaluation of the successes and drawbacks 25 of this first CPETS approach examining data quality, in order to highlight lessons learned and propose recommendations for future tracking survey. 26 CHAPTER 2: Background: Policy and Institutional Context in the Health Sector This chapter presents an overview of the health sector in Nigeria. It describes the institutional structure and policy context within the sector as well as the financing structure and flow of funds for PHC services. Four main components are examined: (i) health outcomes, (ii) organizational structure and responsibilities in the health system, (iii) policy challenges and reforms toward PHC services, (iv) financing of the health sector and PHC services; and (v) governance and accountability gaps in PHC services. 2.1 Country Context Situated in West Africa and with an estimated population of 182.2 million individuals (2015) divided into over 250 ethnic groups, the Federal Republic of Nigeria is the most populous African country. Children under 15 years represent 43 % of the population while adolescent aged 15-24 years’ account for 19.4% (2015). Its economy is mainly based on agriculture, oil extraction and services with GDP shares of 22%, 14.4% and 52.3% respectively (2013). The most recent estimate of poverty headcount ratio at national poverty lines in Nigeria is 46% (2009) with US$ 3200 GDP per capita income (2014). The adult literacy rate is 59.9%. Only 68.5% of the population has access to improved water and 29% to sanitary facilities (WDI, DHS). Figure 2.1: Map of Nigeria with highlighted States in the study Since independence in 1960, the country went through a civil war in the late sixties followed by three decades of military governments with intermittent short-lived elected government. Uninterrupted democratic government began on May 29, 1999 and in 2015, there was successful transition from a political party which has been in power for 16 years to an opposition party. Nigeria operates a federal system of government comprising of a Federal government, 36 states and a Federal Capital Territory (FCT) as well as 774 local Government Areas (LGAs). Each tier of government (federal, state and local) has an executive, legislative and judiciary arms. 27 2.2. Health indicators Health indicators in Nigeria are low and even below what would be expected at the equivalent level of GDP. Life expectancy at birth is 53 years. Infant and maternal mortality rates are among the highest in the world at 72 per 1,000 live births and 820 per 100,000 births respectively (See table 1). While some progress has been observed in the last decades, Nigeria remains one of the worst performers in Africa regarding health outcomes (See Figures 1-3). The country has not achieved its MDG indicator 4 goal of reducing by two-third infant mortality rate falling short by 90%, while reduction in under-five mortality fell short by 28% (MDG Report 2015). Table 2 1: Health Outcomes Indicators 2000 2005 2010 2013 2014 GDP per capita (current US$) 377.5 804.0 2315.0 2979.8 3203.3 Life expectancy at birth (years) 47 49 51 52 53 Maternal mortality ratio per 100,000 births 1170 946 867 821 820 Infant mortality rate per 1000 live births 112 97 82 74 72 Sources: WDI and WHO While the major causes of morbidity have been a result of communicable diseases, non-communicable diseases such as diabetes, cardiovascular diseases are also becoming more prevalent. Some improvements have been observed regarding the prevalence of HIV among pregnant young women aged 15-24 years which has slightly declined from 5.4% in 2000 to 4.1% in 2010. However, with an incidence rate of 338 per 100 000 people, tuberculosis is still highly prevalent with no appreciable results observed over the last decade (MDG 2015). Immunization coverage has increased over the last decades but remains generally low, with for instance the proportion of one-year old children immunized against measles at 63% in 2014 from 46% in 1990 (MDG 2015). 28 Figures 2.2: Life expectancy, infant and mother mortality: Nigeria and selected African countries 29 2.3 Socio-economic and health indicators in Ekiti and Niger States The two states examined in this study, Ekiti and Niger, present quite different socio-economic characteristics and health indicators (see table 2). Niger state, established in 1976, and located in the North Central area, enjoys a GDP per capita of USD 1687 compared to USD 317 in Ekiti, located in the South West, and amongst the newest states, having been carved out of Ondo3 state in 1996. However, there is a sharp contrast in health outcomes and other socio economic characteristics of Niger State when compared to Ekiti States. Indeed, Ekiti is one of the best performers with regard to health status, and Niger is a below average performer (see table 2.2). Table 2.2: Socio-economic characteristics of Ekiti and Niger compared to Nigeria Nigeria Ekiti Niger Year of state establishment 1960 1996 1976 Population census 2006, Million 140,4 2,45 3,86 Location South West North Central 2 Land size, km 923768 5436 68925 GDP per capita 2011, PPP US$ 2300 1187 1518 Poverty head count 2009, % 46 42,3 63,9 Antenatal care coverage (4+ visits), % 45 95,4 44 Skilled attendace at delivery, % 34 89,8 53,9 Tubercolosis immunization coverage, % 61,7 96,3 41,8 DPT immunization coverage, % 47 93,4 23,9 Stunting prevalence (moderate/severe) 34,8 13,6 46,6 Children under 5 sleeping under ITNs, % 29 26,5 9,8 Use of improved drinking water, % 61 74,4 69,6 Use of improved sanitation facilities, % 31 37,7 37,6 Adult literacy 2013, % 59,9 93,4 40,8 Source: Cottolenc 2014, DHS 2013 Population density in Niger is about 8 times lower than in Ekiti (56 and 451 inhabitants per square kilometer, respectively) and despite the relatively high GDP per capital 63.9 percent of the population in Niger lives below the poverty line of 1US$ a day. This is above the national poverty rate of 54.4 percent and much greater than the poverty rate in Ekiti of 42.3 percent, and this despite income per capita five times that of Ekiti.4 Furthermore, adult literacy rate in Niger at 41.7% is among the lowest in the country and much below that observed in Ekiti (74.4%). About 61.1% of the population in Niger has access to an improved source of drinking water compared to 67% in Ekiti. Furthermore, large differences are found across the two states in access to and use of health services. For example, the proportion of people not using pre-natal care services because care is not available, too far or too costly, varies between 2.3% in Ekiti State and 28.2% in Niger; the proportion of people not using post-natal care are generally higher and exhibit less variability with 41.4% in Ekiti and 59.9% in Niger (DHS 2013). 3 Ondo is an oil producing state. 4 Kogi is the poorest state in Nigeria with 88.6 percent of the population living with less than 1$ a day (UNDP 2010) 30 2.4 Health System According to the 1999 Constitution of the Federal Republic of Nigeria, health is a shared responsibility of all the three tiers of governments with the federal, states and local governments having a role in the provision and maintenance of health services. Within the public health sector, the federal government, through the Federal Ministry of Health (FMOH), is responsible for overall policy-making, sector planning, coordination and regulation. It is also responsible for the provision of tertiary care level services through 59 tertiary hospitals (including 26 federal teaching and specialized hospitals). The State governments, through State Ministries of health (SMOH), are responsible for the provision of secondary care services through 855 general hospitals, providing general medical and laboratory services as well as specialized health services (such as surgery, pediatrics, obstetrics and gynecology). Some states have also established state-owned tertiary specialized and university hospitals. State governments also coordinate and support Primary Health Care (PHC) services offered at Local Government level. Local Governments (LGAs) are responsible for the provision of primary health care services. PHC services are offered through different categories of health centers and community health services provided at the district or ward levels, ranging from Comprehensive Health Centers to Health Posts, Clinics and Dispensaries. These facilities typically provide general health services including preventive, curative, health promotion, and pre- referral care. The health centers are generally staffed by nurses, CHOs, community health extension workers (CHEWs), junior CHEWs, and environmental health officers. Table 2.3 presents the levels of care, types and characteristics of facilities Table 3: Types of facilities and characteristics by level of care Level Facilities Size Services Typical staffing Tertiary Teaching hospitals 100-500 Referral and specialized Physicians, nurses, Specialized hospitals beds inpatient, outpatient and lab/radiology Medical centers diagnostic services, training technicians Secondary General hospitals 50-250 beds Inpatient and outpatient Physicians, nurses, care in basic medical midwives, lab & specialties, lab services pharmacy, CHOs Primary Health centers Clinics, 0-30 beds Health prevention and Nurses, CHOs, Dispensaries Health promotion, basic curative CHEWs, EHOs posts care Source: Couttolenc 2014 Note: CHO= Community Health Officer; CHEW= Community Health Extension Worker; EHO= Environmental Health Officer. Other ministries, including the Ministry of Defense, the Ministry of Education, and the Ministry of Interior Affairs operate their own health facilities and services, providing care for armed forces personnel and their family, students, police force and prison inmates respectively but with little coordination with the main FMOH/SMOH system. The private sector also plays a large role in the provision of health care in Nigeria through private for-profit and private not for-profit health facilities. These include hospitals, clinics, maternity homes, faith-based organizations (FBO) and NGOs run facilities as well as traditional medicine practitioners and patent medicine vendors (PMVs). Since the mid-1980s, private for-profit health facilities have proliferated and together with 31 FBO facilities are reported to provide as much as 80% of health services in the country (NHA, 2009, p. 2) (FMOH, 2007; DFID, 2000; Larbi et al, 2004). Figure 4 presents an overview of the health system in Nigeria with its various levels of care and jurisdiction. While the private sector makes an important contribution to the health sector in Nigeria, this study focuses solely on the public sector. Figure 2.3: Health System and levels of care in Nigeria Source: Ojo, Jimoh and Nwali (2009) The National Health Insurance Scheme (NHIS), launched in the late 1990s but actually established in 2005, sought to gradually institute universal health insurance in Nigeria. However, coverage extension has progressed slowly in the country, focusing on specific population groups. It currently reaches only about 5% of the population, mainly public sector employees. 2.5 Health Expenditure Level of health expenditure in Nigeria has increased over the last decade, but remains much below what is observed in other African and Middle-Income Countries (MIC) (see Figure 5 and Table 4). Figure 2.4: Total Health Expenditure by sources (per capita $ and % of GDP) 2005-2014 32 Total Health Expenditure (THE) per capita amounts to USD 118 in 2015 (216 in PPP), which is one fifth below what is observed on average in other African countries and MIC (43% below in PPP). Relative to the size of the economy, THE represented only 3.7% of GDP in 2015, compared to about 6% on average in Africa and MIC (see Table 2.4). Households, through mostly out-of-pocket payments to health providers, are the main contributor of THE in Nigeria, representing about 70.3 percent of THE, while public financing accounted for about 18.7%. The remaining 11% comes from firms through the NHIS, but also from a few private insurers or their own coverage plans, as well as from International donors in the form of grants, financial and technical assistance by bilateral and multilateral agencies and international NGOs/foundations. (WHO NHA 2016). Table 2.4: Health expenditures in Nigeria vs Africa and Low Middle Income Country Low middle African income Nigeria region country 2000 2005 2010 2014 THE per capita current US$/PPP 17/103 53/243 80/174 118/216 147/308 143/328 GHE per capita current US$/PPP 6/34 15/71 21/46 30/55 87/180 79.3/175 THE as % of GDP 2.8 4.1 3.5 3.7 6.0 5.6 GHE as % of THE 33.5 29.2 26.2 25.1 50.8 52.0 GHE as % of GDP 0.9 1.2 0.9 0.9 3.0 3.0 GHE as % of Total Govt Exp. 5.9 7.3 5.7 8.2 9.8 9.9 Source: NHS Government´s effort in health financing (GHE), measured by the proportion of health spending on total government expenditure (TGE), increased significantly in the early 1980s, to reach 9.2% in 2007, but declined thereafter to 8.2 % in 2014. This level of effort is lower than that of most countries in the African Region and many Lower-Middle-Income Countries with about 9.8% of government expenditures (See Table). As a proportion of GDP, government expenditure in health in Nigeria, after reaching a maximum of 1.2% in 2005, only represented less than 1% of GDP in 2014. This expenditure level, from three tiers of government and several semi-autonomous agencies and programs, is less than one third of what is observed in other African and MIC with about 3% of GDP (see Table 4 and Figure 6). 33 Figure 2.5: Public Health Expenditure versus GDP per capita 2014 2.6 Policy reforms toward PHC services The public health system in Nigeria suffers from a number of shortcomings and various problems in particular plague PHC services. USAID Health Systems 20/20 project undertook a large PHC facility level survey in 31 states (25,000 facilities) in 2008 to map out the supply of selected services, especially HIV/AIDS, tuberculosis, malaria, and MCH. It was concluded that: “The coverage of most key preventive and curative health services is relatively low in Nigeria”, and shows important disparities across region s, states, urban-rural settings and socioeconomic strata (USAID, 2009: Health Sector Assessment). Governance is weak at all levels, including poor accountability mechanisms and weak institutional arrangements for channeling advocacy and participation. With the view of improving PHC services and health outcomes, several federal and state policy reform initiatives have been enacted in the last decades, including: • The National Health Policy (NHP) of 2004 and 2008 have sought to “strengthen the national health system […] to provide effective, efficient quality, accessible and affordable health services that will improve the health status of Nigerians through […] Millennium Development Goals.” They aimed at reducing by 2015 under-five mortality and maternal mortality, and halt and reverse the spread of HIV/AIDS, malaria and other diseases. (Couttolenc, 2014, p.18); • Launched in November 2010, the National Strategic Health Development Plan (NHSDP) for 2010-2015 recognized the slow pace of progress in health indicators in Nigeria, reaffirmed the Government´s commitment to the Primary Health Care (PHC) approach, increases budget allocations to health by at least 25% annually, and sets ambitious goals for reducing infant and maternal mortality (by ½ and 1/3 by 2015), among others; • Also presented in 2010, the Nigeria Vision 20:2020 proposed a comprehensive push for accelerating Nigerian development and improving the well-being of Nigerians, by (a) Eradicating extreme hunger and poverty, (b) Guaranteeing food security, (c) Enhancing access to quality / affordable healthcare, (d) 34 Sustaining access to potable water and basic sanitation, (e) Making housing accessible and affordable, (f) Developing human capital through education etc. Key activities in the health sector included establishing at least one general hospital in each LGA with specialized care in the four major basic specialties (surgery, pediatrics, obstetrics and gynecology); • Launched in September 2012 with World Bank and other international partner supports, the Saving One Million Lives Initiative (SOML) aimed at drastically reducing child mortality by 2015 through: (i) improving Maternal, Newborn and Child Health through delivering an integrated package of MNCH interventions at PHC clinics; (ii) improving routine immunization coverage to eradicate polio and reduce the burden of Communicable Diseases; (iii) scaling up access to essential medicines and commodities; (iv) controlling malaria by expanding utilization of bed nets and access to medicines; (v) improving child nutrition, and (vi) strengthening services logistics and supply chain innovations and technology; • The National Health Act (NHA) of 2014 has sought to respond to fundamental crisis of financing health at the PHC level with the establishment of the Basic Health Care Provision Fund (BHCPF). In principle, the fund will provide states and local governments an access to flexible funds to finance PHC activities and it holds the promise of additional resources for the health sector through allocations to the NHIS and NPHCDA. There are ongoing efforts to operationalize the BHCPF; • In the context of poor quality and performance of primary health care services due to myriads of problems including inadequate funding from LGAs, the concept of Primary Health Care Under One Roof (PHCUOR) was introduced. It aimed at integrating all PHC services under one authority with the establishment of State Primary Health Care Development Agency (SPHCDA). Through a combination of financing from LGA and State contribution, the SPHCDA is supposed to directly finance primary health care within the state and among other responsibilities including recruitment, allocation and management of PHC health workers. In practice however, the PHCUOR concept has not yet been fully implemented in many states and SPHCDA still have very limited roles in PHC activities; • Various other initiatives have been developed in recent years to improve the health system performance and level of funding, including at the state level. Some states for instance are implementing an innovative Performance Based financing (PBF) scheme through a World Bank funded Nigerian States Health Investment Project (NSHIP) which initially started in three States of Adamawa, Ondo and Nasarawa but is now being extended to all other states in the North East; • In Niger in particular, the State Strategic Health Development Plan (SSHDP) for 2010-2015 defines ambitious health targets for the horizon 2020, including the reduction by 2/3 of infant mortality, reduction by ¾ of maternal mortality. Service related targets are also indicated, such as immunization coverage and ambulance availability. These are to be achieved through among other provisions the establishment of a functional State Primary Health Care Development Agency (SPHCDA) by 2010 and budgetary allocation for health sector of 15% of the overall State Budget”. (Couttelenc, 2014, p.28). Despite these various ambitious goals and policy reform initiatives, improvements in health services and outcomes have been slow and PHC service financing faces important constraints. Sources of financing for the health sector and PHC services in particular are discussed in the following section. 35 2.7 Financing of the health sector and PHC Services PHC facilities in Nigeria, which are at the frontline of PHC service provision, depend on a complex logistical and administrative system that is supposed to ensure their adequate operation. They do not have specific budget lines in the national, state or local budgets and are recipient of public resources only when higher administrative levels decide so. The three levels of government participate in the financing, operations and management of PHC services and health facilities. In addition, PHC services are supported by external funding from NGOs, bilateral and multilateral agencies, as well as direct household expenditures (user fees). As conferred by the constitution, each levels of government have specific revenue collection responsibilities.5 In addition to their own revenue sources, state and LGA revenues derive from transfers from other levels of government. Federal transfers toward states and LGA come from three types of accounts maintained by the federal government: The Federation Account,6 VAT pool account7 and Excess Crude Account.8 Wit h regard to PHC services, there is no direct specific cash transfers from the federal level to other levels of governments or PHC facilities. The support of FMOH towards PHC takes the form of in-kind transfers including financing the construction of PHC facilities and allocation of medical equipment through the NPHCDA. In addition, it implements several vertical health programs which channel in-kind resources toward PCH services (sometimes through the SMOH/SPHCDA or directly to facilities), in particular the following:9 • National Program on Immunization (NPI); • Adolescent and Sexual Reproductive Health services; • National Malaria Elimination Program; • National TB and Leprosy control program; • HIV/AIDS program; 5 The federal government is responsible for the collection of major taxes and levies (such as corporate income tax, petroleum profit tax, mining rents and royalties), customs and excise duties, value added tax, etc. as well as proceeds from the sale of crude oil. State governments collect especially personal income taxes and capital gains tax payable by individuals. Local governments, for their part collect a variety of local taxes (including property taxes; market and registration fees, etc.) (PER, 2011, p 5-6). 6 The sources of the revenue into the federation account are mineral revenue (e.g crude oil sales, royalties from Oil companies) and non-mineral revenue (e.g non-oil taxes e.g company tax; custom duties etc.). A Federation Account Allocation Committee (FAAC) shares accruals to the Federation Account on a monthly basis using a fixed formula. The nine oil-producing states (which Ekiti and Niger are not part of) are entitled to 13 percent of the amount due for sharing. Oil producing states are Abia, Akwa Ibom, Bayelsa, Cross River, Delta, Edo, Imo, Ondo and Rivers. The remaining amount is shared among the three tiers of government as follows: 52.68 percent for the federal government, 26.72 percent for states and 20.6 percent for local governments. The allocation formula used to determine the proportion that each State gets from the 26.72 share is based on the following criteria: equality of states (40%), population (30%), landmass and terrain (10%), social development factor (10%), and internal revenue generation effort (10%). States with low population, a small landmass, low social development factor such as primary school enrolment and low IGR effort receive less transfer (PER 2011, p5). Kano state gets the highest allocation and Ekiti the second lowest. The LGA share of 20.6% of the federation account is shared among the 774 LGAs using the same formula as among states. 7 Similarly, for the VAT pool account, the FAAC shares funds amongst the three tiers of government as follows: 15 percent for the federal government, 55 percent for states and 30 percent for local governments. The horizontal allocation among states is based on three criteria: equality of states (50%), population (30%) and derivation (20%). States such as Ekiti and Bayelsa with low population and low level of commercial and industrial activities receive less allocation. (PER 2011, p.5) 8 The Excess Crude Account (ECA) is used to save oil revenues above a base amount derived from a defined benchmark price. When the price of oil is above a benchmark price used in the federal budget, the excess revenues are saved into the ECA. Funds kept in the ECA are shared amongst all the three tiers of government when either oil prices or production levels fail below budgeted levels by a certain margin. In practice, withdrawals have been made on an ad hoc basis, under the direction of the National Economic Council which comprises of the President, the Vice-President, and State Governors. (PER, 2011, p7) 9 Federal activities in the states also relate to Training, Monitoring & Evaluation and Supervision of public programs and facilities undertaken by NPHCDA. 36 • Midwives Service Scheme (MSS); • SURE-P Maternal and Child Health (MCH) program.10 At the state level, the establishment of SPHCDA has provided an opportunity for States to contribute to PHC services. However, in Ekiti and Niger, as in most states of the country, the SPHCDAs have minimal roles in PHC activities not being yet fully functional.11 They do not currently provide any direct support (financial or in-kind) to PHC facilities, other than through the support provided to some federal level vertical health programs. These in-kind supports channeled through the SPHCDA to PHC and LGAs include for instance provision of vaccines (immunization program); ARV drugs and testing kits (HIV/AIDS program); provision of TB Directly Observed Therapy Short-course DOTS Kits, equipment and lab reagents (TB & Leprosy Control program); and the provision of treated bed nets (Malaria program). In addition, in both states, the SMOHs have established Drug Revolving Fund (DRF) programmes providing drug seed stocks to PHC facilities. In Ekiti, the DRF programme is managed by the state Central Medical Store and in Niger by the SPHCDA. Furthermore, the SMOH in Ekiti has established a Free Health Care programme for maternal and child health (MCH), through which drugs and medical consumables are provided along with direct cash transfers to selected PHC Facilities. At the LGA level, the Department of Primary Health Care (DPHC) is responsible for primary health services.12 Its budgetary allocation in the LGA budget13 includes allocation for salary payments to all health workers in the LGA, including PHC workers. It generally also includes allocations for overhead and capital expenditures for PHCs.14 Some of these (nonwage) recurrent inputs are procured directly by the DPHC, while others (i.e. 10 The FMOH, through the SURE-P program, provides in-kind support towards maternal and child health care in all states including by supporting various health workers at PHC facility level. In 2013, for instance, a total of 24 facilities were supported by the SURE- P MCH programme at the level of ₦314,735,106 in each of 22 States plus federal capital territory in the North Central, South South, South East and South West Zones and 32 facilities were suppor ted at the level of ₦410,287,690 in each of 14 States in the North East and North West Zones & Ebonyi State. 11 While under the concept of PHCUOR, the SPHCDAs in principle, through financing from States and LGAs, is supposed to directly finance primary health care services and be responsible for PHC operations --including recruitment, distribution and management of PHC health workers-- in practice, in both states the agencies have currently minimal roles. In both states, SPHCDA only current human resources role consist only in the posting and redeployment, discipline or promotion of the PHC department director in each LGA, while all other health workers are posted (or redeployed) and promoted or disciplined at LGA level. The agency is yet to do any recruitment of health workers; the last recruitment of PHC workers having being done by LGA service commission. SPHCDA does not provide supervision activities. 12 The sources of funding for LGAs’ budget, including for PHC activities, come from various sources, in particular: (i) Monthly Statutory allocation from Federation Account and VAT account; (ii) occasional sharing from excess crude account; (iii) sharing from State’s IGR; (iv) LGA own IGR (from local taxes, fees, etc.); (v) transfers from federal government part of special intervention programs (i.e. SURE-P, MDG); (vi) Donor contributions. In Niger, the revenues accruing to each LGA from the federal account and VAT are pooled together in a joint LGA account for all the 25 LGAs at the state level. 10% of the state IGR is also transferred to the joint LGA account. The LGA joint account in Niger is used to pay salaries to all LGA workers in the state, including PHC workers. The salary amounts are transferred to the Ministry of LGA for direct payments to workers (and deduction of the salary account of each LGA). The remaining funds are shared among the 25 LGA using a formula. It is from this net allocation to each LGA combined with LGA own IGR that the PHCD receives his actual allocation, used to cover overhead expenses and capital projects. In Niger, following the fall in oil price, net allocation to LGAs have been less than N10 million on the average in recent years. In Ekiti, the LGA joint account is joint with (part of) the State account. Salaries of both state and LGAs workers, including PHC health workers, are paid from the State’s joint account. The unified payment scheme is domiciled in the State’s Accountant General O ffice from which salaries are paid directly to all civil servants. The net budget allocation to 13 LGAs do not have unfettered access to their budget as their funds are disbursed through the State government-managed Joint Allocation Account Committee (JAAC). 14 However, in recent years in the 2 states examined, LGA transfers to PHC facilities only covered salaries. Hence, in Ekiti, there were no overhead budgeted for PHCs, nor for supervision by DPHC. As a result, most facilities within LGAs have not been visited during the last three years. Furthermore, maintenance and fuel cost for ambulances in each LGA were limited. Funding requests by LGA PHC departments were generally not supported. The situation in Niger is similar. 37 drugs, vaccines, medical supplies) are procured at the state or national level and distributed through vertical programs. In addition, selected PHC facilities are also financed by direct cash transfers from the National Health Insurance Scheme (NHIS). These quarterly basis payments in the form of capitation fees are based on the number of NHIS registered individuals with those facilities, for instance part of the Health Coverage for the Poor (HCP) focusing on Maternal and Child Health (MCH).15 Contributions to PHC services and facilities also come from bilateral and multilateral development partners as well as NGOs.16 Donors support vertical programs either directly or through projects.17 Finally, PHC activities are also financed through Internally Generated Revenues (IGR) at PHC facility level, derived mainly from out of pocket expenditures by patients. These include user fees, drug sales mark-up, laboratory test fees, payment for birth certificates, etc. The income generated at the facilities are use d to support minor facility maintenance, cleaning, allowance for volunteer staff etc. 2.8 Resource Flows toward PHC facilities Given these multiple sources of financing and actors, expenditure flows toward PHC facilities within the Nigerian public health system are complex and very fragmented. This is due in part to the federal nature of the country - which involves the co-responsibility and co-financing of PHC services by the three levels of government, but also to the fact that most transfers toward PHC services are in-kind, instead of in-cash. In this context, we observe many horizontal flows -- across ministries and agencies at the same level, as well as vertical flows -- across levels of governments—with resources disbursed by different sources and agencies through multiple programs. Figure 7 summarizes the main resource flows (capital, salaries, financial and in-kind nonwage recurrent expenditures, including drugs and vaccines) within the health system toward PHC facilities. Figure 2.6: Nigeria Health Sector Fund Flows toward PHC facilities 15 NHIS has been charged with managing enrollment of pregnant women to public facilities at the state level. Under the NHIS-MDG- MCH programme, direct cash transfers is provided to selected PHC facilities to promote access to health care services for pregnant women and children under five registered. Facilities are paid a per capita amount by NHIS through its accredited HMOs. 16 It should be noted that although donors contribute to funding of PHC activities at all levels of government, including primary health care interventions, their contribution is generally of-budget, not being routinely captured in government budgets. However, when a donor funded program requires a government counterpart funding, the counterpart fund appears in the government budget under capital expenses (instead of expenditure). The failure to routinely capture donor contribution in the budget has given room to potential double ‘dipping’ i.e claiming to have spent budgeted funds on projects implemented with donor funds without the counterpart expendit ure. Also, because counterpart funding is captured under capital expenditure, payments of counterpart funding has always been a problem. Most LGA only have resources to cover overhead expenses. 17 At LGA level, vertical programs (i.e. immunization, malaria, HIV/AIDS, TB/Leprosy, Family Planning) are mostly donor-funded. For example, WHO and UNICEF provide the logistics support (including training) for immunization programs, while are vaccines are supplied by the NPHCDA (originating from donors, such as GAVI and World Bank, and from Federal government budgetary provisions). However, some LGA provides counterpart funding to support the conduct of the immunization outreach activities. 38 As represented, PHC facilities receive resources (mainly in-kind) from (a) federal MOH via NPHCDA to state SPHCDA, SMOH via SPHCDA and LGA DPHC, (b) vertical program grants e.g. MDG fund/Global Fund (via state MOH, or via LGA DPHC), and (c) internally generated revenue (NHIS, user fees). More specifically, at the federal level, as discussed above, the NPHCDA procures equipment and finances the construction and renovation of PHC facilities. There are no financial transactions between the FMOH, the States and LGAs. However, several vertical programs funded through federal agencies channel resources to State and Local governments and sometimes directly at PHC facilities through the NPHCDA. Furthermore, several of the programmes receive in-kind support from international development partners, which trickled down to States and LGAs.18 Among the main programs are: (i) The National Program of Immunization (NPI). The NPHCDA (with support from UNICEF, World Bank and GAVI Alliance Funds), provides vaccines to selected PHC Facilities for immunization. Vaccines are purchased centrally on behalf of NPHCDA by UNICEF and then channeled through NPHCDA to State PHCDA and to LGA for onward delivery to the PHC facilities. Furthermore, funds for logistics support (e.g. transport, field worker allowance etc.) are generally provided to the SPHCDA to support immunization activities; (ii) HIV/AIDS Control Program: A division under the department of public health at the FMOH receives support from World Bank, Global Fund and other development partners distributes in-kind resources 18 Examples are HIV-AIDS, tuberculosis and malaria (Global Funds); tuberculosis/leprosy (Damien Foundation); and malaria elimination programme (World Bank). 39 such as HIV rapid test kits through the SMOH/ SPHCDA and then to PHC facilities (as well as directly to private NGOs or FBOs); (iii) The Malaria elimination program (MEP) supported by Global Fund, DFID, USAID and other partners; (iv) The Midwives Scheme (MSS) is a program run by NPHCDA to train and maintain a number of midwives in the states. Resources are channeled through SPHCDA; (v) The Polio Eradication Program is managed by WHO on behalf of NPHCDA. In-kind resources are distributed to State governments and then facilities; (vi) The Health Coverage for the Poor (HCP) program focusing on Maternal and Child Health. Financial resources flow from FMOH to NHIS, and then directly to public PHC providers; The SURE-P program, provides in-kind support towards maternal and child health care. It procures in-kind resources as well as healthcare workers in public PHC facilities.19 At the state level, SPHCDA supports in-kind resources provision to PHC facilities through the various vertical programmes. Also, the SMOH provides drug seed stocks part of the DRF programmes to PHC facilities (through the state Central Medical Store in Ekiti and the SPHCDA in Niger). In addition, in Ekiti, part of a Free Health Care programme for MCH services, the SMOH channels in-kind drugs and medical consumables along with direct cash transfers to selected PHC facilities. At the LGA level, DPHCs allocate overhead, capital expenditures to PHC facilities as well as in-kind resources procured at the LGA level and distributed through vertical programs. Salary payments for PHC workers just like other LGA workers are centrally paid through e-payment in both Ekiti and Niger States. The NHIS receives funds from federal sources (FMOH) and provides per capita financial transfers to selected PHC facilities for MCH PHC services under the NHIS MCH programme. These cash contributions directly to selected PHC facilities in the form of capitation fees are based on the number of NHIS enrolees in the facilities (for instance part of the Health Coverage for the Poor (HCP) program for MCH services. NGOs and bilateral and multilateral development partners channel financial and in-kind support resources through vertical programs through the FMOH (as well as NACA or NPHCDA) and SMOH, as well as to LGA health departments. 20 Support is also channelled directly to the health facilities. Finally, IGR through out of pocket payments by patients are collected directly in-cash at the facility level. 2.9 Governance and Accountability in the Health system The performance of the health system in Nigeria and especially the PHC services is hampered by important challenges relating to governance and accountability in the use of resources and services. 19The SURE-P program procures in-kind resources, in particular: (a) 4 midwives per PHC facility, 2 CHEWs per facility, (b) Drugs, consumables, maternal care kits and medical equipment; (c) Infrastructure Development - renovation of PHC Facilities, construction of bore holes, construction of Midwives accommodation; and (d) training/capacity building for health care workers. 20For instance, funds originating from the Global Fund or GAVI Alliance are channeled through NACA and the NPHCDA, where they are used to procure goods or services (eg. vaccines, training) that are transferred in-kind to States and then to health facilities. Bilateral agencies contributing to these programs include USAID/MSH; DFID/SuNMaP. 40 Nigeria operates within a system of fiscal federalism characterized by extensive intergovernmental fiscal decentralization through fiscal autonomy and responsibility attributed to subnational (State and Local) governments. States enjoy considerable independence with respect to their economic and fiscal policies as the Nigerian Constitution grants the Federal Government very limited and indirect influence over subnational government. In particular, the states, as discussed, have independent revenue sources recognized by the constitution and the states’ budget (as well as development plans, accounts, procurement practices, etc.) are not subject to federal control and scrutiny (PER 2011). While these arrangements create opportunities for states to exert autonomy over policies in line with their own preferences, priorities and capabilities, however, as for other fiscal and social policies, they create difficulties in the coordination and management with regard to health care policies. Indeed, expenditure decisions of the three tiers of government are taken independently, including in the health sector. Each tier of government decides independently on what to do with the fund allocated to it and the federal government has no constitutional power to compel other tiers of government to spend in accordance with its priorities. This implies that the FMOH has no significant influence on funds allocated by subnational governments to secondary and primary health care services– except for special programmes and projects funded through some agencies. Similarly, state governments have limited control over how LGAs allocates resources toward primary health services (even though the constitution gives the state supervisory role over the LGAs). In addition, they are no reporting requirements about budget and expenditure to the federal government from state and local governments, as none of these tiers of government – federal, state and local government - has any obligation to share information on their budgets. This lack of transparency and accountability in the government system, and in particular in health expenditure, combined with poor coordination between levels of government, have limited the levels of integration within the health system and has led to overlapping responsibilities among government levels and agencies. These have greatly weakened the governance and effectiveness of the system as a whole. Furthermore, discordance between fiscal capacity of the various levels of government and their expenditure responsibilities conflicting patterns of decentralization for some functions and centralization for others have not allowed to promote efficiency. 21 In this context, information on health services, resource use and health outcomes is scattered across many government levels, departments and facilities, and no system is in place to consolidate this information and ensure its reliability and availability. While some reporting tools are in place, they are generally un- standardized and submitted to different administrative entities. Lack of management tools, lack of incentives for performance, and weak planning and budgeting systems, among others, are frequently mentioned by facility managers as obstacles to facility and service management and effective governance (reference). As a consequence of system fragmentation and poor information systems, there is a paucity of reliable data across the country on government activities and financing of the health system. Hence, for instance, the HMIS systems in Ekiti and Niger State are weak, like in all states of the country, facing issues of accuracy, timeliness and completeness of data. Specific programs or projects (i.e. NMEP, HIV-AIDS, TB/Leprosy) financed by 21 For instance, facilities are responsible for spending on goods and services but resource allocation decisions are centralized; with the exception of drugs in Ekiti, procurement is fully decentralized to facilities of all sizes, contributing to loss of scale economies and non- transparent nor standardized procurement practices. 41 development partners, tend to have their own data collection and reporting systems, while data for planning and intervention designs are often collected through ad-hoc surveys.22 As a result of this fragmentation, information is often not readily available, consolidated or comparable. Budget reports do not reflect financial performance because important financial flows are not captured in the budget, and other financial and audit reports are equally narrow in scope and coverage. Currently, available data do not permit assessing the level, trends and distribution of health expenditure – between different tiers of government, geographical areas or types of expenditure - or its efficiency. In particular, no reliable estimate is available on consolidated health expenditure at state or LGA levels and even less at the PHC facility level. These features of the Nigerian health system, which have far-reaching consequences on the efficiency of PHC services, will influence the strategy to conduct the PETS study presented in the next chapter. 22Ekiti appears to have a relatively better HMIS than Niger. This can be attributed in part to DFID PATHS program implemented in Ekiti between 2005-2009. Initiatives part of the program included strengthening HMIS and the DRF system. (reference) 42 CHAPTER 3: Methodology The purpose of a resource tracking exercise such as this one is to measure the adequacy and efficiency of use of public resources. In order to serve this purpose, an ideally designed public expenditure tracking survey (PETS) should generate data needed to make an assessment of public spending on six aspects (Dehn, et. al., 2003; Gauthier and Ahmed, 2012), namely resource mobilization, resources allocation, resource utilization, resource reconciliation, resource productivity and resource targeting. The resource mobilization aspect looks at the level of resources mobilized in a sector and programs, the sources of these resources and their relative importance. The resource allocation aspect looks at how resources are distributed across different functions and the allocation mechanisms used and various channels toward different programs, levels and users. An analysis of resource utilization considers the way resources are used, looking at expenditure centers and usage at central, decentralized and service provider levels, execution, bottlenecks, incentives and behaviors. Resource reconciliation is essentially a triangulation of resources transferred and received at various levels by comparing official budget allocations versus release of funds, actual expenditure and reception of resources across different programs and levels and quantifying the resources that reach final users. Resource productivity is looked at in terms of outputs and services across various categories of users, Finally, resource targeting is about whether resources are benefiting the intended groups and individuals or are achieving the intended results and outcomes. This public expenditure tracking framework could be applied with a narrow focus on a specific program or a broader focus on an entire sector. This study focuses mainly at resources for the primary health care system as a whole. The data needs for assessing all aspects of the public expenditure tracking framework for the primary health care system in Nigeria are rather immense. As shown in the previous chapter, resources for primary health care are mobilized from various sources at the Federal, State, Local government and primary health care facility levels. To assess resource mobilization, allocation, utilization and make a resources reconciliation requires collecting data at all levels of government and at the primary health care facilities too. On the other hand, assessing resource productivity and targeting requires data at both the facility and individual levels. Thus in selecting the methodology for the public expenditure tracking survey (PETS), key decisions should be made on which components should be assessed, depending on the data gaps and key policy questions that need to be addressed. In the context of Nigeria, knowledge on resource mobilization, allocation and utilization in the primary health care system is lacking. Closing this knowledge gap is the primary focus of the study, and drove its methodological design. 3.1 Rationale of the new PETS approach Getting a good understanding of resource mobilization and utilization at the facility level – the front line for service delivery – is key to understanding the adequacy and efficiency of use of resources for primary health care. However, one encounters a unique challenge in doing so in Nigeria, stemming from the way resources are transferred to health facilities. Unlike in some countries where PETS have been implemented, Nigeria does not have a specific capitation grant primary health care. Instead, resources are mainly transferred to facilities in-kind, with no obligation for maintaining records of transactions. Because of this, the traditional retrospective PETS is not suitable for comprehensively accessing resource mobilization, allocation and utilization for the primary health care system, let alone to gather credible information to undertake resource reconciliation. One encounters two specific obstacles when undertaking a retrospective PETS for primary health care finances in Nigeria. The most obvious of the two is the absence of financial records at primary health facilities which a traditional PETS heavily relies on. But even if records were available, a bigger challenge would be quantifying the in-kind resources received at the facility level, given that (i) a variety of supplies are made in the health sector and (ii) these come in different units and forms. Valuation is thus a challenge, especially when resource reconciliation is one of the objective since that rules out imputing “actual” values of s upplies received at the facility level based on the transfer values reported by the sending centers. 43 Previous PETS in Nigeria thus fell short on assessing resource mobilization and utilization down to the primary health facility and instead, focused only on quantifiable aspects that could be achieved through the traditional approach. While it gathered financial information at the LGA level, a World Bank PETS study in 2002 (das Gupta, M., V. Gauri, et al., 2004) neither collected financial information nor attempt to quantify the amount of in-kind supplies received at the health facility level for example. Rather, it collected more detailed information about staffing including staff perception, timeliness of salary payments and training. This is the kind of information that could be captured in a retrospective PETS approach. The SDI survey implemented in 12 states of Nigeria in 2013 made a bold attempt by including a health facility financing module, but this provided incomplete financial information and highlighted the need for a new approach. Only 15 percent of the facilities had financial records. Most facilities did not have cash transactions. About 54 percent of facilities did not receive any cash support from any source in the 12 months prior to the survey. Facilities instead received government support in the form of in kind supplies and no estimate of value of the items received were provided in 83 percent of the cases. Most interviewees responded “don’t know”. Estimating the quantity of items received for a broad group of items such as drugs was not feasible, as it amounted to adding apples and oranges. These gaps highlighted the need for a new approach of capturing resource information at the facility level. One tailored to capturing highly disaggregated in-kind supplies and assigning a monetary value to them. 3.2 Overview the new PETS approach This study used a prospective PETS, rather than the traditionally used retrospective PETS in order to address the challenge of assessing resource mobilization and utilization in a sector dominated by in-kind supplies or highly irregular cash transactions with no accountability requirements. The main feature of the prospective PETS is capturing receipt and use of resources at primary health facilities continuously, by adopting a diary approach. This approach is borrowed from the poverty measurement literature, where diaries are widely used and were once referred to as the golden standard for capturing household consumption data (Beegle, et. al., 2012). A diary approach addresses two main challenges, namely the lack of financial information and the difficulty of capturing highly disaggregated receipts of in-kind supplies. In this case, the diaries serve as records and delivery sheets for in-kind supplies, making recording of detailed information more feasible at the point of delivery, as opposed to recalling all that information over a long reference period. Diaries that form the core of the primary health care facility instrument required information to be captured by target respondents (i.e. facility heads) instead of professional enumerators. Misreporting – be it intentional or unintentional - and respondent fatigue were therefore of major concern. Respondents could deliberately “hide” transactions for example, or may be careless in their reporting. Intentional misreporting usually happens when respondents think they could be sanctioned based on information they provide. This was observed early in Ekiti when some facilities originally under-reported drug purchases from provide vendors because that was against regulations. This type of misreporting is not unique to the prospective PETS. Without bank statements or audited financial records, it would still be a concern in a retrospective PETS. The study addressed this issue by repeatedly assuring and convincing facility heads that information they provided would remain anonymous and that it was purely for research purposes. Other reporting errors can be minimized through training, providing incentives to respondents and meticulous supervision. Facility heads and their backups were trained along with enumerators for a week on how to complete the forms. An experiment to provide incentives for regular recording of the instruments was also conducted to promote regular completion of the diaries (see Chapter 5). In addition, trained enumerators paid regular visits to health facilities twice a week – once at the beginning of each week and another during any other day of the week to monitor recording of the diary, probe and assess consistency of recorded information and also cross check with whatever records could be found. Cross-validation was also done using information from the exit surveys to assess any underreporting of own generated revenues. 44 Even, with detailed information on in-kind supply recorded, a major challenge concerns valuation of the recorded supplies. A drug market survey was conducted monthly in each LGA to collect drugs prices, which would later be used to assign monetary values to drugs and non-medical pharmaceuticals which facilities received. Details on the valuation procedure are provided in Chapter 4. Prices for other in-kind supplies such as medical equipment and prices of vaccines (they are centrally procured) were obtained from a survey at the federal level. Specific details about the market surveys are provided in the description of the survey instruments below. Regular financial information and transfers to health facilities were also recorded at the LGA level on a monthly basis using a separate LGA instrument. This included highly disaggregated transfers of drugs and vaccines made by the LGA to each primary health care facility under their jurisdiction. This to some extent allowed for a reconciliation of reported transfers and actual transfers received by health facilities at the State level. Information from the State and Federal level ministries and agencies was captured using the traditional retrospective survey given that official records – budgets – are reported. Data was collected over a period of 6 months, from the beginning of October 2015 to April 2016 in both Ekiti and Niger. This covered the last quarter of fiscal year 2015 and the first quarter of fiscal year 2016. This was done to ensure that data collection captures variations in morbidity patterns in Nigeria, as well as financial transfers across quarters. Prior to this, a pilot data collection exercise for the facility and LGA questionnaires was conducted in August 2015 in Abuja. No pre-testing of the state and federal level questionnaire was conducted since concerned agencies were the solitary respondents, thus no out of sample respondents were available for those instruments. An overview of the set of survey instruments is provided below. 3.3 Survey instruments and implementation Data was collected at four different levels – using seven different survey instruments which we refer to as modules. The four levels are (i) primary health facility, (ii) LGA, (iii) State and (iv) Federal levels (see). The set of seven instruments included a single multi-section instrument each for the primary health facility level and the LGA levels. Two instruments – one for the State Primary Health Care Development Agency (SPHCDA) and State Ministry of Health (SMoH) were used to collect data at the State level. The Federal level had three instruments, one each for the Federal Ministry of Health (FMoH), National Primary Health Care Development Agency (NPHCDA) and National Health Insurance Scheme (NHIS). An overview of the survey implementation structure is provided in figure 1 below. 45 Figure 3.1 : Overview of PETS study survey structure Module 6: Module 7: Module 5: National Primary Health National Health Federal Ministry of Care Development Insurance Scheme Health (FMoH) Agency (NPHCDA) (NHIS) Module 4 Module 3: State Ministry of Health (SMoH) State Primary Health Care Development Agency (SPHCDA) Module 2 Local Government Agency (LGA) Module 1 Primary Health Care Facility (PHCs) The innovation in data collection was mainly in the primary health care facility questionnaire. This was also the most complex of the survey instruments used in this study. The instrument captured resource mobilization at the primary health facility level in four sections depending on the type of resources. Section A of the instrument is a monthly form for recording external financial support, from all levels of government and non- state instructions including private donors. In addition to questions on the amount received, the section included questions on whether revenues received were earmarked and their purpose. Section B has 2 parts - B.1 for all in-kind supplies other than drugs, and B.2 for all drugs received – captured in high detail, in other words by specific items, quantities and units received as well as date of delivery. Thus an item received on multiple occasions would have as many entries as the number of times it was delivered, each entry recording the specifics of the delivery on that date. Section A and B were presented as three different monthly sheets (one each for Section A, B.1 and B.2) for the facility head to record as and when the relevant transaction/delivery happened. Enumerators collected these forms for data entry at the end of every month. A cash book was also introduced (Section C) for facility heads to record daily receipts and expenditures (in cash or bank transactions) on seven specific revenue streams and 20 expenditure lines. Thus internally generated revenues were collected from the section to complete the picture of resource mobilization. In addition, expenditure information provides a good picture of utilization of resources at the primary health facilities. The cash book was as weekly sheet for daily entry of each of the listed items every day of the week. Respondents were instructed to write “0” for the day when they did not transact on a particular line item. Thus the form could not be left blank. Enumerators collected this form at the beginning of each successive week when they gave facilities a fresh form for the new week. Linked to the primary health care facility questionnaire is a market survey for capturing drug prices for all drugs found in facilities. Enumerators from the survey firm carried out the market survey monthly using Section D.2 of the facility instrument designed to capture prices of drugs at high level of specificity, including drug country of origin, package and dosage details. The market survey covered the Central Medical Stores (CMS) in Ekiti and Niger, as well as pharmacies and patented medical vendors (PMVs) in each LGA. This price data was used for determining a fair value of drugs received by facilities and to monetize in kind resources 46 mobilized. Enumerators also recorded additional information on tariffs and user chargers for specific services in a separate form (D.1). Additional information on resource adequacy and availability is obtained from a monthly drugs stock take (Section E), done by enumerators at the beginning of every month. This gave a snapshot of the drugs commonly out of stock and those that were never delivered. Enumerators also recorded information on outputs in Section F using two sources of information. First they worked with the M&E officers in the SMoH to download information from the National Health Information Management System (NHMIS). They also manually corroborated this data with the monthly NHMIS summary sheet at each facility and where information was incomplete, they aggregated it from the various daily registers– antenatal care (ANC), outpatient (OPD), attendance registers. In Nigeria, vaccines are mainly stored in the cold-chains in local government authority warehouses. The common practice if for health facilities to get vaccines on vaccination days and return unused ones at the end of the day. A previous assessment at the early stage of this study concluded that vaccines received by facilities could be accurately estimated from the number of shots/vaccination recorded in the NHMIS, hence this was used as the source of information for quantifying supply of vaccines. Lastly, the health facility instrument contained a service readiness section for collecting information on availability of infrastructure, medical equipment and staff. Enumerators administered this instrument at each facility twice, once at the beginning of the survey period and then at the end. Questions about staff included staff qualifications, the grade of each staff and monthly salaries (these are standardized by grade), which were later used to estimate government salary obligations to health facility staff on government payroll. At the end of the survey, a staff questionnaire was developed to record additional information related to staff, including salary arrears. 47 Figure 3.2 : Overview of PHC PETS Survey instruments and Protocol Section A: Section B Section C: Section D.1: Section E: Section F: Financial (Cash) In-kind support Facility Cash Tariff/User Drug Stocks Outputs Support Book fees Recorded by: OIC Recorded by: Facility Recorded by: Recorded by: Recorded by: Recorded by: Frequency: Monthly Officer in Charge (OIC) form-daily update OIC/Accountant Enumerator Enumerator Supervisor/Enu Frequency: Monthly Respondent: OIC merators Verification: Enumerator Frequency: Weekly Respondent: OIC form-Daily update Form – Recorded Frequency: Frequency: ONCE Recorded ONCE Frequency: Verification: Section B.2: In- DAILY at the beginning of Enumerator A MONTH Recorded kind supplies Verification: EACH MONTH Source Description Enumerator ONCE every Medical Consumables and Other third week of Federal MOH Laboratory/Medical/Dent next Month by al Equipment and INTERNAL Section D.1: the Supervisor Supplies Tariff/User Fees Section E: Drug Other Federal funds GENERATED from Bedding and linen Registration card Stocks REVENUE FROM HMIS/DHIS at Medical records Consultations State MOH USER FEES State Level, Food and Rations Sale of drugs EXPENDITURES Laboratory tests Enumerator Other state funds Uniforms & clothing BY CATEGORY Other (medical verifies with Sanitary and Cleaning supplies, etc..) Local Government funds Materials, Supplies and NHIMS Services registers at the NHIS Capitation (Formal/Inf.) Utilities (Electricity, Water & Sewerage) facility NHIS MDG Capitation Section D.2: Transport (e.g fuel) and Communication Drug Price International Donors/NGOs Market Survey Sure-P personnel suport [Monthly Form – Private Donors Recorded ONCE A MONTH by Enumerator) Section B.2: Community Drugs received Other Cash Receipts In-kind (specify)_ 48 Table 3.1: Overview of Administrative Level questionnaires Panel A: Overview of SMoH questionnaire Panel B: Overview of LGA Questionnaire Section 0. Identification Part I - Monthly Section 1. Financial and in-kind resources Section 0. Identification i. Financial resources - Record State MoH budgeted Section 1. Financial and in-kind resources and actual for the Fiscal Year 2015 by Source i. Financial resources -Record LGA Revenue for ii. In-kind resources from the Federal level the Fiscal Year 2015 by Source Section 2. Non- governmental sources of support for ii. Non-monetary Resources 2015 Section 2. Non- governmental sources of support for i. Financial resources 2015 ii. Material resources (Non- governmental) i. Financial resources Section 3. State MoH Expenditures for FY 2015 ii. Material resources (Non- governmental) Section4.Transfers from SMoH administration to Section 3. LGA and PHC Expenditures SPHCDA and LGAs Section4.Transfers from health LGA i. SMoH resources sent to LGAs administration to PHC Health Facilities ii. Financial resources i. Resources the LGA Sends to its Health PHC iii. In-kind resources Facilities (Material) Section 5. Supervision Part II – One time only i. Supervision from the State/Federal government Section 5. Budget ii. Supervision to LGAs Section 6. Supervision i. Supervision from the State and Federal government ii. Supervision of the health facilities iii. Perception of problems Table 2.2: Overview of Federal Level Questionnaires Panel A: Overview of FMoH questionnaire Panel B: Overview of NPHCDA Questionnaire Section 0. Identification Section 0. Identification Section 1. Financial resources Section 1. Financial resources i. Financial resources - Record FMoH budgeted ii. Financial resources - Record NPHCDA and actual for the Fiscal Year 2015 by Source budgeted and actual for the Fiscal Year 2015 Section 2. Non- governmental sources of support for by Source 2015 iii. In-kind resources from the Federal level i. Financial resources Section 2. Non- governmental sources of support for ii. Material resources (Non- governmental) 2015 Section 3. Federal MoH Expenditures for FY 2015 i. Financial resources Section4.Transfers from FMoH administration to ii. Material resources (Non- governmental) NPHCDA and SMoHs Section 3. Federal MoH Expenditures for FY 2015 i. Financial resources Section4.Transfers from FMoH administration to ii. In-kind resources NPHCDA and SMoHs Section 5. Supervision i. Financial resources i. Supervision from the Federal government ii. In-kind resources ii. Supervision to SMoH Section 5. Supervision i. Supervision from the Federal government ii. Supervision to SPHCDA Survey instruments for the State and Federal levels are traditional PETS instruments. At the State level, entries for transfers to each of the LGAs were also made. An example is shown for the SMoH questionnaire in Table 3.1 Panel A, while Table 2 shows an overview of two of the three Federal level instruments. The LGA questionnaire is traditional too, except that it was designed for monthly data collection and that in- 49 kind supplies of drugs and vaccines to facilities were much more detailed i.e. by drug or vaccines for each facility (see Table 3.1, Panel B). In addition, payments received from revenues generated at the facility level were also captured. The LGA questionnaire not only captured budget and expenditure for primary health care, but for overall health expenditures and non-health expenditure as well. This provides a picture of resource allocation at the LGA level, reflecting the prioritization of expenditures between health and other sectors and within health, primary health expenditures, non-wage and wage expenditures. All the instruments at the administrative level captured information on budgeted, releases and actual expenditures as required to fully understand resource allocation in the sector. 3.4 Sampling strategy Determination of sample posed trade-offs between accurately assessing resource mobilization/utilization aspects and resource reconciliation. Assessing these components all require data collection at multiple levels, but have different units of analysis. This had implications for the optimal sampling strategy, and meant that priority had to be given to some components of the PETS framework before deciding on the optimal sampling strategy. The resource mobilization and utilization components require understanding financing, flows and efficiency in use of funds at the primary health care facilities, which become the primary unit of analysis. In order to make statements on the overall level of financing of PHCs in each state, a sample of facilities representative at the state level is required to provide an overall picture of total finances that reach the facilities, their sources and how they are utilized. This implies that a random sample with few rather than all facilities selected in most LGA will suffice. With a limited sample size to fit a given budget, the key is to select these facilities from as many LGAs in order to capture heterogeneity across LGAs and offset the power reducing influence of high intra-LGA correlation in outcomes. Resource reconciliation on the other hand would focus on the leakage of resources meant for primary health care financing. The unit of the analysis in this case is the LGA, and optionally, higher administrative level. The required data includes capturing all the financial flows for primary health care financing coming through the LGAs, recording LGA disbursements to primary health care facilities and verifying receipts of the resources by health facilities. In a prospective PETS, this verification process requires recording disbursement of funds and supplies for every facility within the LGA in order to estimate the total value of verified resources facilities received and compared with reported disbursements at the LGAs. If a gap exists, that could be attributed to leakages. The desired sampling approach is therefore one in which facilities are clustered by LGAs and followed by a census of facilities within the selected LGAs. Properly done, the clustered approach can also give a representative primary health care facility sample. Such a sample is likely to be low powered, given the high intra-cluster correlation of financing within LGAs, thereby compromising the ability to make inference of primary health facility level outcomes if the sample size is not expanded significantly. The analysis can also be done at the state level, in which case all LGAs would need to be surveyed. In a prospective PETS it can be assumed that LGAs would alter their behavior to minimize leakage to designated survey facilities during the survey period. If that were true, surveying only a few facilities within the LGA would provide biased estimates of leakage. Thus unbiased estimates are most likely to be obtained for assessing resource mobilization and utilization as opposed to leakages. It could also be the case that undertaking a prospective PETS alters behavior in terms of leakage in general. The sampling strategy therefore focused on obtaining credible estimates of resource mobilization and utilization at the primary health facility level. The final sample was chosen based on two considerations – one is to cover LGAs in each State (17 and 25 LGAs in Ekiti and Niger respectively) then selecting a few facilities within each and secondly to undertake a narrow scope of resource reconciliation thus a facility 50 “census” of one LGA per state was conducted. Within these principles, 65 facilities (out of 282 public facilities) and 75 (out of 1209 public facilities) facilities were surveyed in Ekiti and Niger respectively. In Ekiti, samples 3 facilities per non-census LGA and a minimum of 4 per “large” LGA were selected, all 12 facilities from Efon LGA were selected. In Niger, 2 facilities per LGA were selected, except for two LGAs namely in Katangora where all 28 facilities were surveyed and Shiroro, the biggest LGA in the state by facility count, where 5 facilities were sampled. The sample size is deemed adequate given that data is collected in multiple rounds over a period of six months. Given that some transactions are irregular (e.g. financial support), this suggested a tradeoff in favor of a longer period of observation (see McKenzie 2012, for example) rather than a larger number of facilities, so the current approach seems justified conceptually. The sample is only representative at the State level and therefore no rural urban disaggregation is done in the analysis. 3.5 Lessons learnt and challenges The implementation of the prospective PETS was not without its challenges. Surprisingly, getting disaggregated data at the Federal level was a tough challenge and took longer than anticipated. There were also some challenges specific to the implementation of continuous data collection at the facility level Some of these and the key lessons learnt are briefly discussed below: (i) The first challenge, alluded to earlier, is reliability of the diaries. Early weeks showed evidence of under-reporting which had to be retrospectively re-entered after the second week when a supervision team from the government, well vested in the actual practices at facilities, brought that as an issue. A key lesson learnt on that is an insider is needed to spot anomalies and assess reliability of the data. Complete independence in this case, might come at the risk of ignorance. (ii) Creating unique identifiers for drugs and matching them was a complex task. The instrument started with what been considered a long list of drugs expected to be found at the facilities. But each month new drugs appeared on the list, some just once for the entire survey period. This was accommodated by switching to a flexible – drop down data entry process using names of drugs. The consequence is that unique identification codes could only be generated at the end, resulting in a costly process of re-entering and matching the data. This will remain a core challenge for implementing this type of survey. Two options can be considered for future studies. One is to stick to a long list (e.g, 200 item list) which then omits the odd items delivered. This will likely not affect overall estimates of value of quantities received, but nonetheless means the data is not 100 percent complete. The other alternative is giving facilities tablets with the most detailed list of drugs coded into them, by category for easy identification – which respondents can use to record the drugs using a drop down menu. The same list can then be use in the market survey. The tablets can be given to respondents at the end of the study as incentives for judicious recording, but before then, various accountability challenges emerge concerning the security, loss/damage and proper use of the tablets. It also means the tools are not easily institutionalized. (iii) The few facilities with multiple cash collection points were another challenge, given the devolved responsibilities for data entry. One option is to identify these facilities early and give them multiple diaries, one for each revenue collection unit for the responsible persons to complete. A related challenge is when facilities have their own cash records, making the diaries a duplication of effort. Future teams can assess such cash books and explore the possibility of a dedicated set of enumerators instead of facility heads, transferring this information to the diaries (iv) Data collection over 6 months seems beyond the optimal for quality data collection. The rate of errors increased and quality of supervision declined after the first four months. (v) Lastly, additional cross validation efforts – exit interviews and facility observations – took important time away from other core supervision activities, which may have affected overall data 51 quality control. Future implementation should strike a balance between the comprehensiveness of information collected and the quality of that information. 52 CHAPTER 4: Financing of Primary Health Care and Resource Flows toward PHC Facilities This chapter presents an analysis of financing of primary health care services in Ekiti and Niger states. The objective is to provide an estimate of the share of the health budget that is allocated and that reaches PHC facilities and available for primary care service delivery in the 2 states. Based on the budgets of the federal, state and local governments and on the data collected through the survey among PHC facilities and administrative units, it assesses the resources effectively available at each level down to PHC facilities, the main service delivery point for the population. As discussed in previous chapters, financing of PHC services in Nigeria, other than personnel, is mainly realized through discretionary financing at the LGA level. There are no conditional health transfers from the federal government toward lower government levels. Still, the Federal MOH intervene through vertical programs, but the importance of these interventions, as we will observe, is relatively small as well as support state as well as from private and foreign donors. The resource tracking exercise is made difficult due to paucity of official information but also to the fact that mainly in-kind resources from multiple sources and allocated through different channels are transferred to PHC facilities. This chapter is organized as follows. The first section assesses budgetary resources at the federal, state and LGA levels officially allocated in FY2015 and FY2016 toward the health sector and specific share devoted to PHC service. It also accounts for execution rates and evaluates actual expenditures toward PHC services at each administrative level. The second section accounts for resource transfers across levels and resources received from donors and NGOs. Section 3 estimates overall public resources actually allocated toward PHC services in both states. The fourth section estimates overall public resources available for PHC services at each level and that ultimately reaches primary health centres.23 We account for various types of expenditure and sources of financing at each level, including NGOs and international donors as well as PHCs’ own internally generated funds. 4.1 Public Expenditures toward PHC facilities PHC services in Nigeria are financed through three main sources: (i) public expenditures at various government levels; (ii) international donors; (iii) private financing and internally generated funds within PHC facilities. As discussed in chapter 2, in the context of the fiscal federalism system in place in Nigeria, measuring public health expenditures as well as the other components of overall health expenditure presents several hurdles, in particular regarding PHC services. Health according to the Nigerian Constitution is a shared responsibility of the federal, state and local governments. PHC services in particular are the main responsibility of LGAs, with federal and state administrations participating in the management and financing of these services. At these various levels, numerous ministries, agencies and departments channel resources toward the sector and the constitution provides for specific revenue source for each level of government. There is minimal coordination between these agencies and levels. In particular, there are no requirements for reporting public budget and expenditures to federal government or between administrative levels. There is no mechanisms or organization responsible for collecting health expenditure data across budget categories, tiers of government or types of services. In this context, measuring health expenditure toward primary health care involves measuring budget and expenditure at each level (sections 4.1.1-3), and accounting for transfers between governments, as well as support from external sources, including donor agencies (section 4.2). We then estimate overall resources officially allocated and actual expenditure reported for PHC services 23 Our focus is on PHC facilities and hence PHC services provided through community services or at hospital level at the district, state or federal levels are excluded from the estimation of PHC resources. 53 (section 4.3) and measure actual resources available at the PHC facility level using survey data, accounting for external source financing and user fees (section 4.4). 4.2 Federal level At the federal level, public spending in the health sector involves budget preparation by the executive arm of government, its passage into an Appropriation Act by the Parliament and ascent by the President (preparation and approval). According to the Budget office of the Ministry of Finance, the total budget for the federal government is about 4,873 billion N and 6,061 billion N for FY 2015 and FY 2016 (Table 4.1). Table 4.1: Federal Government Budget (2015 and 2016) FY 2015 FY 2016 Health/ Health/ Budget Federal Budget Health Budget Federal Budget Health Total Total Million N % Million N % % Million N % Million N % % Total 4,873,403 100.0 249,124 100.0 5.1 6,060,677 100.0 250,062 100.0 4.1 Debt 953,620 19.6 -- -- -- 1,475,320 24.3 -- -- -- Service Statutory 375,616 7.7 -- -- -- 351,370 5.8 -- -- -- transfers Recurrent 2,912,322 59.8 226,440 90.9 7.8 2,646,389 43.7 221,412 88.5 8.4 Personnel 2,394,279 49.1 221,902 89.1 9.3 - - 217,472 87.0 - Overhead 518,043 10.6 4,538 1.8 0.9 - - 3,940 1.6 - (a) Capital 631,845 13.0 22,684 9.1 3.6 1,587,598 26.2 28,650 11.5 1.8 Sources: Appropriation act 2015 and bill 2016, complemented with PETS NG data collected from Accountant General office. Note: "-" not available; "--" not applicable. (a) The Nigerian public accounting system does not follow the standard definition of capital and its distinction from recurrent. Operational budget items are often classified as capital instead of recurrent expenditure, resulting in the inflation of capital expenditure in the budgets. As observed in Table 3.1, the share of health expenditures in the federal budget declined between 2015 and 2016 from 5.1% to 4.1%. However, the debt service of the federal government has increased by 5 percentage points from close to one-fifth (2015) to one-quarter (2016) of the overall budget, and represent about four times the federal health budget.24 Table 4.2 presents the structure of the Federal Health budget and specifically PHC activities. These include the main agencies and administrations involved in PHC services at the federal level, that is NPHCDA and vertical programs from FMOH headquarter budget.25 24 Noteworthy, the share of capital expenditures in the federal budget has double between 2015 and 2016, increasing from 13% to 26% of the budget. This is associated with governmental measures taken by recent governments to reduce recurrent expenditure while boosting capital expenditure. Furthermore, recurrent expenditures in 2015 were unusually high because of severance packages allocated to all elected and appointed political office holders in 2015, while newly elected ones also received upfront allowances, but these payments are not expected in 2016. 25 For both fiscal years, the share of the federal health budget devoted to PHC services in Table 4.2 is composed of the NPHCDA budget and of the FMOH Head Quarter (HQ) vertical programs budget. There are two caveats. In general, part of the NHIS capitation budget is directed to PHC facilities through in particular the MCH programme. However, for both years studied, the MCH programme was not implemented in any state because of incapacity by state level governments to meet their budgetary counterpart responsibilities. Furthermore, FMOH HQ vertical program’s budget finances PHC services at all levels, including those performed in secondary and tertiary hospitals. We could not distinguish the share of vertical program supporting PHC services provided in PHC facilities specifically and impute all vertical program resources potentially targeting PHC services (see note c in table 4.2) as being directed to PHC facilities. This assumption leads to a potential overestimation of the share of federal budget devoted to PHC facilities in section 4.2. 54 Table 4.2: Structure of the Federal Health Budget (2015 and 2016) FY 2015 FY 2016 Health NPHCDA FMOH Health/ PHC(b)/ Health NPHCDA FMOH Total/ PHC/ Budget Budget HQ VP(c) Total Health Budget Budget HQ VP(c) Health Health Million N % % Million N % % (a) Total 249,124 12,030 2,536 5.1 6.1 250,062 17,745 2,628 4.1 12.3 Recurrent 226,440 2,066 - 7.8 0.9 221,412 2,050 - 8.4 3.2 Personnel 221,902 1,860 - 9.3 0.8 217,472 1,846 - - 3.0 Overhead 4,538 206 - 0.9 5.0 3,940 204 - - 14.7 Capital 22,684 9,964 - 3.6 46.3 28,650 15,695 - 1.8 73.4 Sources: Appropriation Acts 2015 and 2016, Federal Ministry of Health, Federal Government of Nigeria, National Assembly. Note: "-" not available. (a)Debt service and statutory transfers do not appear in the table. (b) PHC budget = National Primary Health Care Development Agency (NPHCDA) + FMOH Vertical Programs. (c) FMOH HQ Vertical programs include: MDG- IMNCH, MDG HIV/AIDS, MDG National Tubercolosis control, MDG Malaria program, Procurement of mosquito net hangers for effective use of Long Lasting Insecticide Treated Nets (LLINS). (d) Population estimates are used from the Population Division of the United Nations for 2015 and 2016; exchange rate: 1$=199N. We observe in Table 3.2 that PHC activities, which represented 6.1% of the federal health budget in FY2015 double in percentage point to 12.3% in 2016. This increase is mainly explained by an increase in NPHCDA capital expenditures in 2016.26 Hence, while the overall federal health budget remained stable at about 250 billion N during the two years but decreased in relative share within the federal budget (from 6.1 to 5.1%), the PHC budget has increased by close to one third in Naira terms during the period (from 14,5 billion N to 20,4 billion N). 4.3 FMOH allocation vs release of resources In the context of important decrease in oil prices and revenues of oil production in Nigeria since November 2014, the federal government faced problems of over expenditure commitments and budget overruns. In this context, the federal budget process as a mechanism for setting resource priorities, and controlling the use of those resources to attain policy objectives, has been steadily undermined in recent years. Insufficient revenues to meet increasing budget during economic slowdown has resulted in continuous forced expenditure seizure, lack of efficient controls over budgetary commitments, forcing fiscal adjustment from the federal to lower levels of government and irregular budgetary controls. The excess of budget commitments over available resources has undermined the meaningfulness of budget numbers, by giving rise to ineffective budget formulation, execution, utilization and unfavorable impact on budget beneficiaries. Table 4.3 presents the rate of execution of the overall health budget and PHC budget components with respect to official budget as well as released revenues toward the health ministry for FY 2015. 26 This increase in capital expenditure in 2016 is explained in part by the renovation, construction and provision of equipment at PHC facility level. For instance, 110 PHC facilities (one for each senatorial district) are to be restored (including with all essentials inputs to deliver minimum package of care) part of the Rapid Results Initiative (RRI) supported by the World Bank. Furthermore, it should be noted that in Nigeria, recurrent expenditure items such as drugs, vaccines, contraceptives, etc. are generally classified as capital expenditures, resulting in the inflation of capital expenditure in the budgets. 55 Table 4.3: Actual expenditure of Federal Health Budget by categories (2015) FY 2015 Total Total FMOH HQ PHC/ Health PHC NPHCDA Health PHC(c) Vertical (a) Health % Budget Budget $ per capita(b) Total, Mil. N 297,249 9,041 8,372 669 3.0 8.2 0.2 Exp./Budget, % 119.3 59.7 69.6 26.4 -- -- -- Recurrent, Mil. N 285,644 1,896 1,896 - 0.7 7.9 0.1 Exp./Budget, % 126.1 90.8 91.8 - -- -- -- Personnel, Mil. N 279,766 1,753 1,753 - 0.6 7.7 0.0 Exp./Budget, % 126.1 94.2 94.2 - -- -- -- Overhead, Mil. N 5,878 143 143 - 2.4 0.2 0.0 Exp./Budget, % 129.5 62.4 69.4 - -- -- -- Capital, Mil. N 11,605 6,476 6,476 - 55.8 0.3 0.2 Exp./Budget, % 51.2 61.6 65.0 - -- -- -- Sources: PETS NG 2016, modules 5, 6, and 7 collected from the Office of Accountant General of the Federation, Ministry of Finance. Note: "-" not available; "--" not applicable. (a) FMOH HQ Vertical programs include: MDG-IMNCH, MDG HIV/AIDS, MDG National Tubercolosis control, MDG Malaria program, Procurement of mosquito net hangers for effective use of Long Lasting Insecticide Treated Nets (LLINS). (b) Population estimates are used from the Population Division of the United Nations. (c) We assume that the PHC budget is composed of NPHCDA and FMOH HQ vertical programs with PHC component. Normally some of the NHIS budget is devoted to PHCs for maternal care (MCH) capitation but for the FY 2015, but the States could not meet up with their counterpart responsibilities hence the MCH programme was not executed in any of the states in 2015. A breakdown of FMOH HQ Vertical programs into capital and recurrent is not available, and so these are not counted in the We observe in Table 3.3 that execution rate of the health budget was higher than budgeted in 2015 by 19%.27 However, the PHC budget’s execution rate is only about 60% of the budgeted envelope. This low expenditure level is mainly driven by very low financing of vertical programs through FMOH HQ budget of only one fourth of the official budget envelope in 2015. The NPHCDA component shows slightly higher expenditure level at 70% of the official allocation. The execution rate of the capital budget is especially low, at about half of the allocated budget. In particular, capital expenditure of FMOH HQ’s vertical programs shows much lower execution rates than other budget categories, with actual expenditures of only 7.9% of the officially allocated budget. Ultimately, given the low spending level of PHC activities, the actual spending for PHC services during FY2015 is about half the share of the officially allocated health budget (3.1 % compared to 6.1%). With regard to the 2016 budget, it is worth noting that the current federal administration came on board in May 2015, but due to the delay in constituting Federal Executive Councils, the 2016 budget was presented late to the National Assembly and was approved in May 2016, 6 months after the beginning of the year. Hence, the 2016 budget was not yet approved at the completion of PETS data collection. However, the FMOH, as other ministries, made use of the 2015 budget until the second quarter of 2016. 27 This is potentially explained by additional mid-year release to the health budget. 56 4.4 State level At the state level, budget preparation also goes through the same process as at the federal level with a budget preparation phase by the executive arm of government, its passage into an Appropriation Act by the Parliament and final ascent by the Executive Governor (preparation and approval). In Ekiti, part of the budgeting process also includes the development of a three-year Medium Term Sector Strategies (MTSS) and Medium Term Fiscal Framework (MTFF) from which the budget for the year is derived. In Niger State by contrast, the budget process does not include the development of MTSS or MTEF. Table 4.4 presents the official State Budget in Ekiti and Niger for FY 2015 and FY 2016. Table 4.4: State Government Budget - Ekiti and Niger (2015 and 2016) FY 2015 FY 2016 Budget State Budget Health Health/ Budget State Budget Health Health/ Ekiti Million N % Million N % Total % Million N % Million N % Total % Total 80,784 100 5,048 100 6.2 67,044 100 493 100 0.7 Recurrent 48,816 60.4 3,416 67.7 7.0 42,121 62.8 300 60.76 0.7 Personnel 8,613 10.7 3,307 65.5 38.4 7,933 11.8 286 58.01 3.6 Overhead(a) 40,203 49.8 109 2.2 0.3 34,188 51.0 14 2.74 0.0 Capital 31,966 39.6 1,632 32.3 5.1 24,923 37.2 193 39.15 0.8 Niger Total 86,610 100.0 2,677 100 3.1 85,092 100 3,375 100 4.0 Recurrent 48,188 55.6 458 17.1 1.0 41,434 48.7 398 11.8 1.0 Personnel 35,456 40.9 338 12.6 1.0 31,502 37.0 308 9.1 1.0 (a) Overhead 12,732 14.7 120 4.5 0.9 9,932 11.7 90 2.7 0.9 Capital 38,422 44.4 2,219 82.9 5.8 43,658 51.3 2,977 88.2 6.8 Source: PETS NG 2016, modules 3 and 4 reflecting collection of retrospective data from the Ministry of Health and from the Ministry of Budget and Economic Planning. Data on Hospital Management Board (in SMOH) for Ekiti is complemented from the 2016 approved budget of the Ekiti State government. Notes: "Other PHC Programs" include the Drug Revolving fund and SACA in Ekiti; the Essential Drug Program, the TB and Leprosy Control Program, and the Public Health Program in Niger. (a) Overhead includes transfers to other funds. We observe in Table 4.4 that in Ekiti, the health budget represented a relatively larger share of the overall state budget than in Niger (6.2% vs 3.1%, respectively in 2015).28 Between 2015 and 2016, in face of decreasing oil revenue share among states in Nigeria, the overall state budget in Ekiti decreased by about 17%. The health sector budget took the most severe blow with a decline of 76% over the period, dropping to a meager 0.7% of the overall state budget.29 This shrinking health budget is mainly associated with an 88% decrease in the health capital budget in 2016. In Niger during the same period, the reverse situation was observed. While the state budget decreased by a mere 2%, the health sector budget increased by a quarter, reaching about 4% of the state budget. 28 It should be noted that Niger’s data does not include funds for hospital management board (HMB) which accounts for 3.6% of the State budget in Ekiti. 29 Excluding HMB funds for which data are not available in 2016. Note that the health budget in Ekiti excluding the HMB funds is 2092 million N in 2015 and 493 million N in 2016. 57 Table 4.5 presents the structure of the State health budget and PHC components in both states, including in particular the SPHCDAs, which are the state agencies under the SMOHs responsible for PHC services at the state level. Table 4.5: Structure of the State Health Budget: Ekiti and Niger (2015 and 2016) FY 2015 FY 2016 Other State Health State Health/ PHC/ State Health State Health/ PHC/ PHC Total (SMOH) PHCDA Total Health Total (SMOH) PHCDA Total Health Programs Ekiti Million N % % Million N % % Total 80,784 5,048 312 4 6.2 6.3 67,044 493 82 0.7 16.6 Recurrent 48,816 3,416 28 - 7.0 0.8 42,121 300 27 0.7 9.0 Personnel 8,613 3,307 25 - 38.4 0.8 7,933 286 23.4 3.6 8.2 Overhead(a) 40,203 109 3 - 0.3 2.9 34,188 14 3.6 0.0 26.6 Capital 31,966 1,632 284 - 5.1 17.4 24,923 193 55 0.8 28.5 Niger Total 86,610 2,677 729 425 3.1 43.1 85,092 3,375 599 4.0 17.7 Recurrent 48,188 458 272 - 1.0 59.4 41,434 398 248.6 1.0 62.5 Personnel 35,456 338 266 - 1.0 78.7 31,502 308 242.4 1.0 78.7 (a) Overhead 12,732 120 6 - 0.9 5.0 9,932 90 6.2 0.9 6.9 Capital 38,422 2,219 457 - 5.8 20.6 43,658 2,977 350 6.8 11.8 Source: PETS NG 2016, modules 3 and 4 reflecting collection of retrospective data from the Ministry of Health and from the Ministry of Budget and Economic Planning. Data on other PHC programs and Hospital Management Board (in SMOH) for Ekiti is complemented from the 2016 approved budget of the Ekiti and Niger State government. Notes: "-" not available. "Other PHC Programs" include the Drug Revolving fund and SACA in Ekiti; the Essential Drug Program, the TB and Leprosy Control Program, and the Public Health Program in Niger. (a) Overhead includes transfers to other funds. (b) Population projection based on the 2006 population census with annual population growth assessed at 3.1%; exchange rate: 1$=199N. In Table 4.5, we observe that official expenditure for PHC activities in Ekiti represented 6.3% of the state health budget in 2015 and that in 2016 its share increased to 16.6% in the context of a severely shrinking health budget. In per capita terms, the PHC budget dropped by close to four fifth compared to the previous year, from 131 N in 2015 to 34 N in 2016 (not represented). In Niger, the official PHC budget in 2015 represented a stunning 43.1% of the state health budget (excluding HMB funds, for which no data are available). 4.5 SMOH allocation vs release of resources in both states The execution of state budgets proceeds through the release of funds from the Federation Account, the major source of funds for all tiers of government, by the Ministries of Finance (MoF) in both states, which grants Authority to Incur Expenditure (AIE). There is also the granting of clearance by the Due Process Office (DPO) and procurements of goods and services (including labour services) by font-line service providers, Ministries, Departments and Agencies (MDAs), LGAs and FCT. Table 4.6 presents the rate of execution of the health and PHC budget components with respect to official state budget for FY 2015 and the first quarter of FY2016, in both states. 58 Table 4.6: Actual expenditure of State Health Budget (2015 and 2016-Q1) FY 2015 FY 2016 Q1 Other Health/ PHC/ Health/ PHC/ State Health State State Health State PHC Total Health Total Health Total (SMOH) PHCDA Total (SMOH) PHCDA Ekiti Programs % % % % Total (Mil. N) 41,026 2,469 34 2 6.0 1.5 34,209 55 3 0.2 5.8 Exp./Budget (%) 50.8 48.9 10.8 63.8 -- -- 51.0 11.1 3.9 -- -- Recurrent (Mil. N) 35,196 2332 20 - -- -- 14,747 45 2 -- -- Exp./Budget (%) 72.1 68.3 71.4 - -- -- 35.0 14.9 8.0 -- -- Personnel (Mil. N) 8,464 2328 18 - -- -- 1,827 44 2 -- -- Exp./Budget (%) 98.3 70.4 71.8 - -- -- 23.0 15.4 7.4 -- -- Overhead (Mil. N) 26,731 5 2 - -- -- 12,920 0.4 0.4 -- -- Exp./Budget (%) 66.5 4.2 68.8 - -- -- 37.8 3.0 12.2 -- -- Capital (Mil. N) 5,830 137 13 - -- -- 19,461 10 1 -- -- Exp./Budget (%) 18.2 8.4 4.7 - -- -- 78.1 5.2 1.8 -- -- Niger Total (Mil. N) 64,232 558 267 0 0.9 47.9 13,395 160 62 1.2 39.0 Exp./Budget (%) 74.2 20.8 36.7 0.0 -- -- 15.7 4.7 10.4 -- -- Recurrent (Mil. N) 42,289 553 267 - -- -- 9,103 160 62 -- -- Exp./Budget (%) 87.8 120.7 98.2 - -- -- 22.0 40.1 25.0 -- -- Personnel (Mil. N) 30,549 469 266 - -- -- 6,975 119 61 -- -- Exp./Budget (%) 86.2 138.7 100.0 - -- -- 22.1 38.6 25.0 -- -- Overhead (Mil. N) 11,740 84 1 - -- -- 2,128 40 2 -- -- Exp./Budget (%) 92.2 70.2 20.7 - -- -- 21.4 45.0 24.9 -- -- Capital (Mil. N) 21,944 5 0 - -- -- 4,292 0 0 -- -- Exp./Budget (%) 57.1 0.2 0.0 - -- -- 9.8 0.0 0.0 -- -- Source: PETS NG 2016, modules 3 and 4 reflecting collection of retrospective data from the Ministry of Health and from the Ministry of Budget and Economic Planning. Data on other PHC programs and Hospital Management Board (in SMOH) for Ekiti is complemented from the 2016 approved budget of the Ekiti and Niger State government. Note: "-" not available; "--" not applicable. Data on budget release is used as proxy of actual utilization due to unavailability of such data. (a) Population projection based on the 2006 population census with annual population growth assessed at 3.1%; exchange We observe that in both states, official budgets typically have low implementation rates (about ½ of the official budget in Ekiti and ¾ in Niger). We see that apart from personnel cost (for salaries and wages of employees), most expenditure components of the budget are hardly implemented, especially for capital projects which budgetary are abysmally low. This could be in part explained by insufficient release of revenues accruing from Federation account in recent years as well as lower autonomous revenues. In both states but especially in Ekiti, the SPHCDA budgets have much lower execution rates than observed across the state budget. Indeed, the execution rate of the SPHCDA budget in Ekiti was only 10.8% of the official budget. For the first quarter of 2016, execution rate 3.9%. In Niger, higher execution rates were observed. Still, actual expenditures only represented about one-fifth for the health budget and one-third of the approved budget for PHC services put forward by SPHCDA.30 30 These low execution rates observed in the health sectors in both states are likely due to very low release rates, especially of capital and overhead expenditures compared with other budget categories and sectors, as health agencies in both states reported spending all resources released by the Treasury. 59 4.6 LGA level Turning now to local government budgets, just like the other tiers of government, LGAs prepare annual budgets, which contain the details of estimated revenues and proposed expenditures. The budget process, as for the federal and state levels, includes budget preparation by the executive arm of government, its passage into an Appropriation Act by the Parliament and ascent by the executive. The budget process typically starts in the last quarter of the outgoing year and ideally should have been completed by March of the following year. The budgeting process includes call for proposals to each department and consultation with the LGA stakeholders including traditional and community leaders. As mentioned, in Ekiti, part of the budgeting process also includes the development of a three-year Medium Term Sector Strategies (MTSS) and Medium Term Fiscal Framework (MTFF). Within the principle of separation of powers, while the executive arm of government prepares the budget estimates, approval of the budget of LGAs in Ekiti is the responsibility of the LGA Council made up of elected councillors. In contrast, the budgeting process in Niger does not include development of MTSS or MTEF and the approval of the budget is done by the State House of Assembly, rather than individual LGA councils. Table 4.7 presents the aggregate LGA overall and health budget in each state for FY2015 and FY2016. Table 4.7: Structure of LGAs Budget: Ekiti and Niger (2015 and 2016) FY 2015 FY 2016 LGA PHC/ PHC LGA PHC/ PHC PHC(b) PHC(b) Budget Total Budget Budget Total Budget $ per $ per Ekiti Millions N % Millions N % capita(c) capita(c) Total 31,278 5,424 17.3 8.68 21,542 3,192 14.8 5.11 Recurrent 17,044 4,418 25.9 7.07 12,296 2,799 22.8 4.48 Personnel 16,836 4,347 25.8 6.96 12,138 2,755 22.7 4.41 (a) Overhead 208 71 34.1 0.11 157 44 27.9 0.07 Capital 14,234 1,005 7.1 1.61 9,247 392 4.2 0.63 Niger Total 55,769 6,870 12.3 6.47 42,109 6,503 15.4 6.12 Recurrent 38,491 6,298 16.4 5.93 31,424 6,016 19.1 5.66 Personnel 13,679 5,642 41.2 5.31 12,397 5,576 45.0 5.25 (a) Overhead 24,812 656 2.6 0.62 19,027 439 2.3 0.41 Capital 17,277 572 3.3 0.54 10,686 487 4.6 0.46 Source: PETS NG 2016, module 2 based on additional collection of retrospective data by the state ministry of LGAs in Niger and by the single LGAs in Ekiti. Notes: (a) Transfers to other funds are presented as part of overhead. Overhead also includes charge on consolidated fund. (b) Health department. (c) Population projection based on the 2006 population census with annual population growth assessed at 3.1%; exchange rate: 1$=199N. As observed, the budget for LGA’s Health Departments which support primary care activities in the districts, represent in both states about one-sixth of the overall local government budget and are quite comparable at about 15% in 2016. When compared to the state level health budget, especially the one devoted to PHC activities (SPHCDA), we observe that the local health budget is much larger, by a factor 60 of 17 to 1 in Ekiti, and of 9 to 1 in Niger (for the entire state health budget the ratio is 2.6 to 1 in both Ekiti and Niger). 4.7 LGA budget allocation vs release of resources in both states Figure 4.1 presents the budgeted and actual overall expenditures for each LGA in both states for FY 2015 using data collected from States’ Ministry of Local Government Authorities. Figure 4.1: Planned vs. Actual Expenditures of overall budget by LGA, in Ekiti and Niger (2015) Source: PETS NG 2016, module 2 based on additional collection of retrospective data by the state ministry of LGAs in Niger and by the single LGAs in Ekiti. Each bar in the histograms of Figure 4.1 represents the percentage of planned and executed expenditure for each of the LGA, by state.31 Budget execution rates of overall expenditure by LGA are also represented (green dotted line), as well as average execution rate across LGAs in each state (orange line). 31 The LGA of Kontagora in Niger was excluded because of an outlying level of expenditure. 61 As observed, as for federal and state budgets, officially approved budgets typically have low implementation rates across LGAs in both states.32 We observe in particular low level of execution of LGA budgets in 2015 of about 45% of the approved budget in Ekiti, and about 37% of the official budget in Niger. Examining specifically the primary health care budget at the LGA level, Figure 4.2 (right hand sides) present the execution rate of the health (PHC) budget in 2015 across LGAs in each state. It shows the official budget and actual expenditure for PHC services (in million N, left scale), as well as the execution rate (in %, right scale) for PHC services and by categories of expenditures (personnel, capital, overhead) across LGAs in both states.33 The left-hand side of Figure 4.2 (pie charts) show the shares of expenditure for PHC services as well as by categories in the official budget (top chart) and in actual expenditure (bottom chart) across LGAs in both states. We observe that in both states, the execution rate of the PHC budget in 2015 across LGAs is relatively similar at about 60% of approved budget (58% in Ekiti and 62% in Niger), with very low rate of actual expenditure of capital and overhead budgets (less than 5% in both states). Personnel expenditures on the other hand were executed at a rate of about 66% across LGAs in Ekiti and 72% in Niger. 32 This is potentially mainly due to insufficient release of revenue accruing from Federation account in recent years. As mentioned above, in addition to LGA own resources, annual budgetary allocation from federal sources (as well as from state sources) is determined through a funding formula at the Federal Ministry of finance level, based on historic allocation. Annual allocations to LGAs are announced around October, about 3 months before the beginning of the fiscal year in January. 33 The LGAs of Ekiti West and Oye in Ekiti state, as well as Gurara and Shiroro in Niger state were excluded because of outlying level of expenditures. 62 Figure 4.2: PHC official budget and actual expenditure in LGA budget, in Ekiti and Niger (2015) Source: PETS NG 2016, module 2 based on additional collection of retrospective data by the state ministry of LGAs in Niger and by the single LGAs in Ekiti. 63 4.8 Transfers Having assessed official and actual health and PHC expenditure at each level of government, we now account for transfers across government levels for PHC purposes as well as contributions from NGOs and International donors from health purposes. This will allow estimating, in section 4.3, overall resources allocated and reaching PHC facilities by aggregating contributions received by each tiers of government as well as net transfers across levels. Financial and In-kind contributions of international donors to FMOH vertical programs for PHC activities are presented in Table 3.8. We account for reported health sector transfers by major international donors to the federal government which were collected directly from donors.34 It should be noted that official budget data from donors were not available. Table 4.8: Financial and In-Kind contributions by main International donors to FMOH vertical programs (2015 and 2016) (in million N) FY 2015 Q1 2016 Financial In kind Financial In kind Glaxosmithkilne - 40.583 - 0.000 PharmAccess 0.075 - 0.000 - (a) SPRING - 0.980 - 0.000 The Global Fund 58.930 79.077 1.173 1.863 UNICEF 0.211 0.029 0.055 0.000 United consortium - 0.015 - 0.003 WHO 0.148 1.346 0.000 0.000 Total 59.364 122.030 1.228 1.866 Source: PETS NG 2016, module 6 based on information collected by the FMOH. Note: "-": not available. (a) SPRING: Strengthening Partnerships, Results and Innovations in Nutrition Globally. We observe that in FY 2015, about ¾ of international donor contribution to FMOH vertical programs was from the Global Fund through financial and direct in-kind support with most of the balance coming from the multinational Glaxosmithkilne through in-kind vaccine support. Accounting for international donor contributions as well as from NGOs and between levels of governments, Table 3.9 summarizes resource transfers for PHC services received at each government level in each state, for FY 2015 and the first quarter of FY2016. 34 The World Bank group contributes to FMOH vertical programs through budget support to the Ministry of Finance. Hence, specific contributions could not be identified without potential double counting. 64 Table 4.9: Transfers from International donors/NGOs and between levels of governments (in million N) FY 2015 FY 2016 Q1 (b) (a) (b) Federal State LGA Total Federal State LGA(a) Total Ekiti Federal - In kind -- 571.24 0.01 571.25 -- 199.23 0.00 199.23 State - In kind -- -- 40.50 40.50 -- -- 43.26 43.26 LGA- In kind(c) -- -- 2.09 2.09 -- -- 0.00 0.00 International donors/NGOs(d) Financial 203.47 61.62 0.00 265.09 4.21 46.22 29.44 79.87 In kind 418.25 257.17 0.00 675.43 6.40 36.65 1.58 44.63 Total 621.72 890.04 42.60 1554.35 10.60 282.10 74.29 366.99 Niger Federal - In kind -- 1614.73 0.53 1615.26 -- 378.20 0.00 378.20 State - In kind -- -- 90.56 90.56 -- -- 108.54 108.54 (c) LGA- In kind -- -- 0.00 0.00 -- -- 0.00 0.00 (d) International donors/NGOs Financial 346.05 116.20 0.00 462.25 7.16 526.10 82.71 615.97 In kind 711.34 938.00 3.75 1653.09 10.88 29.27 0.00 40.14 Total 1057.38 2668.93 94.85 3821.16 18.03 933.57 191.25 1142.86 Source: PETS NG 2016, modules 2, 3, 4, 5, 6, and 7 that are based on retrospective data collected from the Ministry of Health - including data collected from the single vertical programs by the Ministry - and the Ministry of Budget and Economic Planning, and retrospective data by the state Ministry of LGAs in Niger and by the single LGAs in Ekiti. Notes: "--" not applicable. (a) Data for LGA in 2015 only refers to the fourth quarter. We impute a yearly value multiplying by four the quarterly value. (b) Per capita expenditure computed at the state level assuming equal spreading of resources spent from the federal level. (c) Tranfers to LGAs from the Local Government level may represent transfers from (NACA/SACA/LACA). Because this was specified in the survey as a different source (presented in the table as Federal) we report this information separately. (d) International donors/NGOs include Glaxosmithkilne, SPRING, PharmAccess, The Global Fund, Unicef, WHO. (e) Population projection based on the 2006 population census with annual population growth assessed at 3.1%; exchange rate: 1$=199N are used to impute Federal level transfers to each State. NGOs contributions were collected using data on reported transfers received for PHC services at each government level. This includes financial transfers to vertical programs from NGOs at the federal level, as well as financial and in kind transfers to state level. In addition, at the state level, data are based on SMOH and SPHCDA reported reception of resources from international donors and NGOs. The former covers transfer for general health purposes, while the later is specifically for PHC services. For federal to state transfers, data are based on reception of resources reported by SMOH and SPHCDAs, as well as LGAs. We observe in Table 3.9 that both Ekiti and Niger receive relatively comparable transfers and contributions (3.3 $ per capita in Ekiti, 3.5$ per capita in Niger for FY 2015). Of these transfers and contributions, about 2/3 were received by state level governments and the balance by the federal level vertical programs. The share of LGA reception of transfers and contributions is about 2% from 2015. In the first quarter of FY2016, we observe financial contributions from International donors to each level of government, mainly to State governments, but also to LGA (about 14%). 65 4.9 Overall Public Resources for health and PHC services Having estimated public resources officially allocated and actually spent at each level of government as well as transfers and contributions, we now estimate overall public resources available at each level of government, for health and specifically for PHC services. Table 3.9 presents officially budgeted, as well as actual level of expenditure, in per capita terms for FY 2015 in each state by level of government, as well as international donor/NGO contributions, and overall. The first set of columns present the value of resources per capita excluding personnel (salaries) and the second set of columns present total resources (including personnel), for each tiers of government, and each state Table 4.10: Budgeted and Actual Expenditure for health and PHC services by level of government, $ per capita (2015) Ekiti Niger Health PHC Health PHC Resources Resources Resources Resources (excluding Total (excluding Total (excluding Total (excluding Total salaries) salaries) salaries) salaries) International donors/NGOs Budget(c) 1.506 1.506 1.290 1.290 2.483 2.483 1.491 1.491 Actual 1.506 1.506 1.290 1.290 2.483 2.483 1.491 1.491 Federal Budget 0.751 6.871 0.280 0.402 0.751 6.871 0.280 0.402 Actual 0.482 8.198 0.183 0.249 0.482 8.198 0.183 0.249 State Budget 2.788 8.083 0.460 0.500 2.202 2.520 0.436 0.686 Actual 0.226 3.953 0.025 0.054 0.084 0.525 0.001 0.252 LGAs Budget 1.724 8.685 1.724 8.685 1.156 6.469 1.156 6.469 Actual 0.042 4.669 0.042 4.669 0.139 4.008 0.139 4.008 (a) Total Budget 6.768 25.145 3.754 10.876 6.593 18.343 3.364 9.048 Actual 2.256 18.327 1.539 6.262 3.189 15.215 1.814 6.000 Source: PETS NG 2016, modules 2, 3, 4, 5, 6, and 7 that are based on (i) retrospective data collected from the Ministry of Health and the Ministry of Budget and Economic Planning, retrospective data by the state ministry of LGAs in Niger and by the single LGAs in Ekiti, and retrospective budget data from Appropriation Act 2015, Federal Ministry of Health, National Assembly. Notes: (a) Per capita expenditure computed at the state level assuming equal spreading of resources spent from the federal level. (b) Population projection based on the 2006 population census with annual population growth assessed at 3.1%; exchange rate: 1$=199N. (c) Missing budget information, we assume budget to coincide with actual for the entire FY 2015. (d) International donors/ NGOs represents (financial and in-kind) transfers from these organizations to FMOH vertical programs as well as in kind transfers to SMOH, and SPHCDA. This figures likely underestimate donors contributions that are otherwise included in public budgets and are difficult to isolate from the pool of public resources. Table 4.10 summarizes and aggregates official and actual transfers at each level in per capita terms to assess overall contribution to PHC services at each level. The table is based on figures presented in Tables 4-1- 4.10, summed across federal, state and LGA levels. As discussed earlier, under International donors/NGOs, we identify – without double counting – both financial and in-kind contributions from donors flowing into FMOH vertical programs, as well as financial 66 and in-kind resources transferred to SMOH, SPHCDA and LGAs. It should be noted that reported contributions from donors may not be exhaustive, and may underestimate donors’ contributions. Indeed, some of these contributions cannot be distinguished from general public resources in the budgets. Furthermore, actual transfers from donors were imputed for budget figures, given that officially budgeted contributions from international donors were not available. We acknowledge that our assumptions can underestimate officially budgeted contributions. Aggregating resources at each level, we observe that overall official public resources for health, including personnel, total 25.1 $ per capita in Ekiti, and 18.3 $ in Niger.35 Actual expenditure for health, including human resources, is about 18 $ per capita in Ekiti and 15$ in Niger. Excluding human resources, actual expenditures for health represent about 2 $ per capita in Ekiti and 3$ in Niger. With regard to PHC services, actual total expenditure for health is about 6 $ per capita in both states. Excluding human resources, actual expenditure is about 1.5 $ per capita in Ekiti and 1.8 $ per capita in Niger. 4.10 Resources available at PHC facility level We now turn to assessing resources available at PHC facility level for health care delivery. Table 3.10 presents the reception of resources in per capita terms at the PHC level. Table 4.11: Arrival of public resources in PHC facilities ($ per capita) Ekiti Niger Financial Financial Share of Share of In kind excluding Salaries Total In kind excluding Wages Total source source salaries salaries $ per capita(a) % $ per capita(a) % Federal 0.002 -- -- 0.002 0.1 0.000 -- -- 0.000 0.0 State 0.071 -- -- 0.071 2.3 0.035 -- -- 0.035 0.8 LGAs 0.001 0.000 3.060 3.061 97.7 0.004 0.005 4.119 4.128 99.2 Total 0.074 0.000 3.060 3.135 100.0 0.039 0.005 4.119 4.163 100.0 Source: PETS NG 2016, module 1. Note: "--" not applicable. We propose a 12 months projection based on the 7 months covered from the PETS between October 2015 and April 2016. (a) Population projection based on the 2006 population census with annual population growth assessed at 3.1%; exchange rate: 1$=199N. Table 4.11 is a summary of PHC revenues from public sources of each tier of government. For comparability purposes, we present a projection for 2015 using the information of reported resources received at the PHC level collected over the 7-month survey period. Using sampling weights, we derive values at the state level based on our representative sample. Furthermore, we make use median reported salary in each job category collected from the staff roster at the facility level and account for salary arrears. 36 We observe in Table 4.11, excluding human resources, that public resources amounting to about 7 cents per capita are available in Ekiti (4 cents in Niger). When human resources are accounted for, public resources made available for PHC services amounts to 3$ per capita in Ekiti (4$ in Niger). Of these public resources, about 98% (99% in Niger) comes from the local government level and 2.3% from the state level in Ekiti (0.8% in Niger), while the federal level contribution is negligible. 35 HMB budget, which is accounted for in Ekiti, but not in Niger because of data unavailability, represents about 5 $ of official per capita expenditure and 3 $ of actual per capita expenditure. 36 In Ekiti, arrears were particularly important (about three months of salary), while they were negligible in Niger. We apply a correction assuming that the information collected as of March 2016 corresponds to what was owed for the whole FY 2015. 67 Having estimated public and international donor/NGO resources at each level, we now estimate the percentage that ultimately reaches local PHC facilities. Using survey data on reception of resources among PHC facilities during the survey period and resources reaching each level, Table 3.10 and Figure 4.3 summarize official and actual health expenditures at the federal, state and LGA levels, as well as resources received at the local level, in each state, for FY2015. It should be noted that we cannot distinguish the source of funds — public versus international donors-- transferred by the federal government through the various vertical programs toward PHC facilities, nor at state level. Indeed, once allocated to public treasury, the support provided to federal and state levels by international partners are fungible and hence undistinguishable at the reception side from domestic resources. Given this fungibility, public resources presented in Table 3.10 include these “indirect transfers” from international donors channeled through Federal and State agencies and that cannot be differentiated for domestic budget sources. It should be also noted that we assume that international donor (and NGO) contributions are not accounted in official federal and state budget figures, hence avoiding double counting. Table 4.12: Arrival of Public Resources in PHC facilities (2015) Official Health budget Official PHC budget Actual PHC expenditure Resources received by PHCs Excluding Excluding Excluding Excluding Total Total Total Total salaries salaries salaries salaries Ekiti Million N 4,227 15,703 2,344 6,792 156 3,105 46 1,957 $ per capita 6.768 25.145 3.754 10.876 0.249 4.973 0.074 3.135 % Health budget excluding salaries 100.0 -- 55.5 -- 3.7 -- 1.1 -- % Health budget including salaries 26.9 100.0 14.9 43.3 1.0 19.8 0.3 12.5 Niger Million N 7,002 19,482 3,573 9,609 343 4,789 47 4,422 $ per capita 6.593 18.343 3.364 9.048 0.323 4.509 0.044 4.163 % Health budget excluding salaries 100.0 -- 51.0 -- 4.9 -- 0.7 -- % Health budget including salaries 35.9 100.0 18.3 49.3 1.8 24.6 0.2 22.7 Sources: PETS NG 2016, module 1 as well as the modules 3, 4, 5, 6, and 7 that are based on (i) retrospective data by the Ministry of Health and by the Ministry of Budget and Economic Planning, (ii) Appropriation Acts 2015 and 2016, Federal Ministry of Health, Federal Government of Nigeria, National Assembly, (iii) retrospective data by the State Ministry of LGAs in Niger and by the single LGAs in Ekiti. Note: "--" not applicable. Population projection based on the 2006 population census with annual population growth assessed at 3.1%; exchange rate: 1$=199N. Reception of resources at the PHCs should represent all facilities in the state: reception of each facility adjusted by sampling weights. Moreover, based on the 7 months of the study we compute monthly average reception and impute 12 times this amount to 2015. Budgeted transfers from international donors, to different levels of government, for health and PHC budget are counted in using actual transfers. This likely underestimates the budget. In Table 4.12, figures are presented in million Naira, in $ per capita and in % of the health budget. The table is composed of four pairs of columns summarizing information presented in more details in previous tables of the chapter. The first pair of columns aggregates official public expenditures budgeted for health by the three tiers of government, excluding and including salaries; the second pair of columns does the same for official public expenditures budgeted for PHC operations; the third pair of columns are the actual expenditures for PHCs from all levels of government; while the fourth pair of columns shows all public resources received in PHC facilities. Ideally, in the absence of leakages, and with perfect reporting at various levels, the last two set of columns should be identical. 68 As observed in Table 4.12 and Figure 4.3, human resources are the most important resources to reach PHC facilities in both states. We observe that official budget allocated to PHC activities is about half of the total health budget (excluding and including salaries) in both states. However, actual expenditure for PHC services is much smaller, especially for non-wage expenditures. Indeed, actual expenditure excluding salaries falls to less than 4% of the official health budget in Ekiti and to 5% in Niger. As for non-wage resources ultimately reaching PHC facilities, in both states, the actual share is about only 1% of the official non-wage PHC budget (Figure 4.3). Taking wages into account, the share of actual expenditures reaching PHC facilities is about 19.8 % of the official health budget in Ekiti (24.6% in Niger). As for the share of total resources (including human resources) officially allocated to PHC services and reaching PHC facilities, it is only 12.5% of the official health budget in Ekiti and one-third of the official PHC budgetary envelope. This is explained in part by important arrears to PHC personnel accumulated in 2015 and 2016. In Niger, given the quasi absence of arrears observed among PHC personnel, the share of total resources officially allocated to PHC services and reaching PHC facilities is 22.7% of the official health budget and about half of the official PHC budget. 69 Figure 4.3: Public Resources in the Health sector toward PHC services, by levels (2015) Sources: PETS NG 2016, module 1 as well as the modules 3, 4, 5, 6, and 7 that are based on (i) retrospective data by the ministry of health and by the ministry of budget and economic planning, (ii) Appropriation Acts 2015 and 2016, Federal Ministry of Health, Federal Government of Nigeria, National Assembly, (iii) retrospective data by the state ministry of LGAs in Niger and by the single LGAs in Ekiti. Note: Health budget in Niger does not include Hospital Management Board (HMB) funds due to missing data. Therefore, the shares of budget for PHC, expenditure for PHC and reception for PHC, compared to the health budget, are overestimated In both states, public resources arrival rates per capita in PHC facilities vary considerably among LGAs. Table 4.13 presents official and actual per capita public expenditure across LGAs in both states, as well as resources reaching PHC facilities in each LGA. 70 Table 4.13: Public resources per capita in LGAs and PHC facilities ($ per capita, 2015) Total (excluding salaries) Total LGA Health Public reception LGA Health Public reception at the PHC at the PHC Official budget Actual release level(a) Official budget Actual release level(a) EKITI State(c) 1.72 0.04 0.07 8.68 4.67 3.13 ADO 0.90 0.01 0.25 4.68 4.30 3.34 EFON 1.68 0.00 0.29 8.97 6.04 4.57 EKITI EAST 0.00 0.00 0.18 0.00 0.00 2.70 EKITI SOUTH WEST 1.90 0.00 0.39 3.75 1.51 5.20 EKITI WEST(b) 5.26 6.35 0.28 10.48 12.66 4.16 EMURE 1.53 0.09 0.36 18.74 7.12 1.12 GBONYIN 11.02 0.08 0.29 18.44 7.66 4.44 IDO/OSI 0.92 0.04 0.04 9.93 6.64 3.76 IJERO 1.08 0.01 0.27 6.90 5.25 3.21 IKERE 0.73 0.02 0.37 10.84 7.46 1.63 IKOLE 2.36 0.00 0.22 10.94 5.12 2.31 ILEJEMEJE 2.57 0.00 0.18 13.57 0.00 4.50 IREPODUN/IFELODUN 1.74 0.38 0.19 23.04 12.12 1.97 ISE/ORUN 1.86 0.09 0.17 14.23 7.92 3.91 MOBA 1.07 0.01 0.11 5.46 3.95 2.78 OYE(b) 0.49 0.02 0.33 7.39 6.92 4.45 NIGER State(c) 1.16 0.14 0.04 6.47 4.01 4.16 AGAIE 0.56 0.09 0.21 7.84 4.79 6.94 AGWARA 1.88 0.01 0.25 15.25 8.82 2.53 BIDA 0.60 0.29 0.23 4.22 2.77 1.22 BORGU 1.26 0.00 0.82 5.24 3.60 12.09 BOSSO 0.24 0.03 0.19 7.01 5.95 4.00 CHANCHAGA 3.25 0.17 0.56 10.89 7.78 6.79 EDATI 0.94 0.03 0.54 5.27 3.11 1.67 GBAKO 0.76 0.95 0.34 6.12 5.84 2.90 GURARA(b) 1.27 6.32 0.53 4.79 12.53 5.28 KATCHA 1.04 0.00 0.20 11.38 10.60 5.00 KONTAGORA 1.42 0.00 0.32 5.99 2.41 1.81 LAPAI 0.76 0.00 0.66 8.58 7.82 4.20 LAVUN 0.86 0.06 0.28 5.65 3.90 1.37 MAGAMA 0.87 0.03 0.30 5.39 4.01 2.77 MARIGA 0.99 0.07 0.34 4.76 3.44 3.19 MASHEGU 0.86 0.01 0.17 2.86 2.08 1.18 MOKWA 0.82 0.18 0.40 4.15 1.68 1.60 MUNYA 1.06 0.44 0.66 8.75 8.14 7.84 PAIKORO 0.46 0.01 0.27 5.45 1.93 3.77 RAFI 0.30 0.10 0.82 13.22 3.30 4.71 RIJAU 0.87 0.01 0.32 7.55 3.64 2.61 SHIRORO(b) 8.82 0.00 0.50 13.67 0.00 2.75 SULEJA 1.17 0.06 0.43 4.55 2.87 1.38 Sources: PETS NG 2016, module 1 as well as modules 3 and 4 based on retrospective data collected by the Ministry of Health and by the Ministry of Budget and Economic Planning. Notes: (a) Computed using sampling weights in order to make data representative of the state level. We propose figures for 2015 that we construct using the monthly average value of transfers over the 7 months of the PETS NG survey. In Niger, only 23 out of 25 LGAs were covered from the survey. (b) Excluded from other computations. (c) State average. (d) Population projection based on the 2006 population census with annual population growth assessed at 3.1%; exchange rate: 1$=199N. 71 Note that for figures for columns 1, 2, 4, and 5 in Table 4.13 are based on data collected from LGAs and computed in $ per capita terms. Columns 3 and 6 present estimated resources arriving at PHC within each LGAs based on sample weights. As observed, the official overall health expenditure per capita at the LGA level is 1.72 $ per capita in Ekiti (1.16$ in Niger). Of this amount, we estimate that only 7 cents reach PHC facilities in Ekiti (4 cents in Niger). With regard to human resources, which are the principal resources made available to health centers, the official health expenditure (including salaries) is about 8.68 $ per capita (6.47$ in Niger). Of this amount, about 4.67$ per capita (4.01 in Niger) is actually released and final users in health facilities ultimately receive about 3.13 $ per capita in Ekiti (4.16$ in Niger). 37 4.11 Overall resources available at PHC level The low level of execution of the budget and high level of retention of resources at each level of government translate into low level of public resources reaching local health centers. Table 4.14 presents the contribution of various actors financing primary care in Nigeria, including government, donors and households who pay the user fees. Table 4.14 presents a projection on an annual basis of PHC facilities’ revenues (share and per capita $) from different sources using the 7-month data collected through the PETS between October 2015 and November 2016.38 In addition to overall PHC facilities revenues from government sources, table 4.14 presents user fees and donor contribution for which we distinguish between direct and indirect contributions. Direct donor contributions are resources directly allocated by donors to PHC facilities reported by PHC facilities through the survey modules. Donors’ indirect contributions to PHCs are support provided by donors through federal and state transfers, in particular cash and in-kind support by donors to FMOH vertical programs and SPHCDA for PHC services (for non-salary purposes). This distinction between direct and indirect donor support is important given that governments’ transfers to PHC facilities include domestic public resources mixed with donors’ contributions to federal and state government’s health programs. These resources reaching PHC facilities through government programs are not separable by recipient facilities between the public and donor component. Because transfers from donors, other than salaries, largely exceed government contributions from domestic resources, it is pertinent to present them separately to assess the net contribution of individual sources (and we ensure possible double-counting of indirect transfers from donors and government transfers). We observe in Table 4.14 that excluding human resources, government transfers account for only about 4.7% of PHC health centers’ revenues, while the large majority of non-wage revenues derives from donors direct and indirect transfers, contributing about 92.4% of health centers revenues (96.5% in Ekiti). In per capita terms, donors contribute about 0.06$ per capita on average in both states in direct and indirect support, while government provides less than 1 cent. Including human resources, domestic government transfers account for about almost all of PHC revenues of the 4$ per capita of overall revenues of PHCs in 37 PHC spending per capita estimated relate to resources received at the PHC facility level only. The estimated per capita figures hence exclude potential spending for PHC services provided through community services or at hospital level at the district, state or federal levels. The estimated PHC per capita spending and share of PHC resources reaching facilities then potentially underestimate the actual share of PHC resources reaching the population. Furthermore, it should be noted that our estimates of resources allocated for PHC activities at the national level tend to overestimate the resources allocated to PHC services given that part of FMOH vertical program are allocated toward care provided at hospital level. 38 For donors’ indirect contributions, we use rep orted data for the complete year. 72 both states: in fact, the second largest source of PHC revenues is user fees, which accounts for just 6.1% of total revenues. Health centers do rely on user fees to finance their activities. These account for 85.8% of non-salary revenues in Niger, but only 36.6% in Ekiti. The impact in terms of access could however be significant, as it will be discussed in the next section. Table 4.14: Actual contributions to PHC operations (2015) Ekiti Niger Both % of total revenues (excluding salaries) User fees 36.6 85.8 67.2 Donors - direct 3.0 5.0 4.7 Donors - indirect 50.6 7.6 13.4 Government 9.8 1.7 4.7 % of total revenues User fees 1.4 8.9 6.1 Donors - direct 0.1 0.5 0.4 Donors - indirect 2.0 0.8 1.1 Government 96.5 89.8 92.4 (a) Total revenues (excluding salaries) $ per capita User fees 0.045 0.407 0.270 Donors - direct 0.004 0.024 0.016 Donors - indirect 0.062 0.036 0.046 Government 0.012 0.008 0.009 Total 0.123 0.474 0.342 Total revenues - $ per capita (a) User fees 0.045 0.407 0.270 Donors - direct 0.004 0.024 0.016 Donors - indirect 0.062 0.036 0.046 Government 3.072 4.127 3.730 Total 3.183 4.593 4.063 Source: PETS NG 2016, module 1 and 2, as well as module 3 and 6 based on retrospective data collected from the single vertical programs at the Ministry of Health as well as the Ministry of Budget and Economic Planning and retrospective data by the state Ministry of LGAs in Niger and by the single LGAs in Ekiti Note: Direct donors' contributions refer to the revenues observed at the PHC level as coming directly from donors, while indirect contributions are imputed using information on transfers from donors to Federal, State and LGA level of government directly for PHC facilities to determine the share of government expenditure (excluding salaries) likely sourcing from donors. (a) A 12- month projection based on the 7 months covered from the PETS between October 2015 and April 2016 is used. Population projection based on the 2006 population census with annual population growth assessed at 3.1%; exchange rate: 1$=199N. 73 CHAPTER 5: Resource availability at primary health care facilities Both the amount, sources and composition of support to PHC facilities affect their ability to deliver high quality services. Having looked at PHC financing from the government, this chapter makes a deeper dive at the resource envelope of PHC facilities. The analysis centers on the relative contributions of all types of revenue sources (government, donors and internally generated revenues (IGRs), to PHC facilities’ overall resource envelope. It also looks at their composition in terms of wage and non-wage contributions and what that means for adequacy of resources available at PHC facilities or the prioritization of getting resources to front line health delivery centers. 5.1. Overview of methodology for estimating the resource envelop of PHC 5.1.1. Defining the resource envelop The study measured the resource envelope of PHC facilities, meaning all revenues and forms of support for which primary health care facilities are the direct recipients or spending units. This is estimated primarily using data from the PHC facility instrument of the survey, designed to capture facilities’ direct receipts of cash and supplies or payments made to providers of services (e.g. salaries of health workers) at facilities by the government or other stakeholders. The PHC facility prospective PETS instrument comprised a number sections meant to capture direct receipts by PHC facilities from all sources as they occurred. Section A captured all monetary payments to facilities by government and non-government agencies as those payments were made. Section B had two parts to record supplies to PHC facilities upon delivery. In the first part (B.1), respondents recorded all in-kind supplies other than drugs. This included payments of operating expenses on behalf of the facility and any civil works at the facility in addition to supply of inputs and equipment. In the second part (B.2), respondents recorded all receipts of drugs, both given to the facility free of charge and those paid for by the facilities. In Section C, they recorded daily, all cash receipts from internally generated revenue sources and all cash payments made by the facility. The study also captured other complimentary information, including an updated staff roaster with detailed information on the grade levels of each staff and monthly records of vaccinations made at the facility. The financing envelope is thus estimated from the lens of resources flowing directly to facilities and payments made in lieu of direct services provided to facilities to support their core functions. The study thus measures PHC facility level spending. We focused on PHC facilities spending because they are the focal delivery point for the Ward Minimum Health Package (WMHP), which is the government definition of the priority of interventions that PHC facilities in each ward should deliver at a minimal cost. This package has six components listed below: (i) Child Survival - provision of IMCI, routine immunization and maintaining records at PHC facilities. (ii) Maternal and newborn care – provision of ANC, delivery, postnatal care and family planning services, (iii) Nutrition – having health and education materials, adequate equipment for food demonstration, community and health facility based growth monitoring, adequate provision of food supplements. (iv) Control of communicable diseases of public importance (Malaria and STI/HIV/AIDS, TB). Implementation at the PHC facility level would be through provision of drugs (and INTs for malaria), basic laboratory infrastructure for testing and early detection and VCT for HIV/AIDS. (v) Control of non-communicable diseases (NCDs) – All PHC facilities should have basic equipment for screening and early diagnosis of NCDs, information and education campaign materials on NCDs displayed and trained health workers on case management of NCDs. 74 (vi) Health education and community mobilization – at least 2 health workers in each ward are trained as health educators and every PHC facility having information and education campaign materials displayed. Transportation is provided at the LGA level. The government defined the minimum health resources needed to provide this package, covering a list of minimum requirements for manpower, essential drugs, equipment, infrastructure and services for PHC facilities. The focus on PHC facilities spending in this chapter thus provides a picture of how these inputs are financed and estimates the amount of resources available at PHC facilities to pay for these minimum inputs. 5.1.2. Valuation of resources The resource envelop is estimated from a valuation of the three types of support received by facilities. These are: (i) direct cash (bank) payments to facilities, (ii) in-kind supplies delivered to the facilities and (iii) indirect support through payments to providers on behalf of facilities. The first type of support comprises own generated revenues and financial payments to facilities by government agencies, non-government agencies, the community and private individuals, including patients. Their value is easy to quantify, since the amounts received are directly recorded as either financial/cash support in Section A or internally generated revenues in the daily cash book (Section C) of the PHC instrument. However, actual payments for the other two forms of support are made other “actors” instead of the facilities themselves. Therefo re, the monetary value of in-kind support was either not captured at the point of receipt at the facility level, e.g. drugs (Section B.2) or where an attempt was made (all other in-kind support in Section B.1), the values are rarely reported largely because respondents did not know the values. Values for these types of resources are therefore imputed to estimate the overall resource envelope of facilities. Monetary values of in-kind supplies are estimated based on their market value. While following the same principle, the valuation procedures differed in complexity between in-kind supplies other than drugs and vaccines, and the valuation procedures for drugs and vaccines. The former was the simplest to impute. At the end of the survey period, the survey team compiled a detailed list of all items received by facilities in- kind (i.e. items captured in Section B.1) and collected their market prices. This multiplied by the quantity received to obtain their monetary value. The same market price was applied for facilities in Ekiti and Niger. Drugs are valued based on the same principle, though slightly more complicated. First a distinction is made between drugs received free of charge and those paid for by the facilities. The calculation excludes the later from the resource envelop to avoid double counting, since they are expenditures against direct cash revenues (internally generated or cash support) received by facilities. Drugs received from the DRF but paid for by the facilities are thus excluded when calculating the resource envelop, while any free drugs from the DRF are included (note, the instrument offered separate response options for “Paid DRF” and “Free DRF”). The value of drugs received for free was not directly recorded, so this is imputed based on the market price for each of the drugs facilities received. The valuation of drugs is primarily based on price data from the market price survey, which was part of the PHC facility module (Section D.2). The imputation process dealt with two main challenges. One is the large variance in the price of the same drug depending on whether it is generic or patented, country of origin and its packaging. Although the market price survey instrument captured these attributes, similar information was not recorded in the drugs delivery sheet (Section B.2) to simplify recording requirement for respondents. The second is that the price for certain drugs, generally provided to facilities free of charge, may not be available locally. Considering these, the imputation of the value of drugs followed the following procedures: 75 • Step 1: For each drug dosage and format, find the most commonly available packaging and country of origin from the monthly stock take instrument (Section E) and use these as the defined attributes for the drug, dosage and format received by the facility. This assumes that the most commonly purchased format and packaging of drugs are reflected in the drug stocks facilities hold. • Step 2: From the market survey, calculate the median unit price for each drug, dosage, format, country of origin and packaging type. Initially, we calculated these prices separately by vendor type, but coverage was low as prices for some drugs were available through only one source. When the price is available from one source, we calculated the unit price from this sole source, otherwise it would be averaged over all sources for which prices are available. The same average is used for Ekiti and Niger, on the assumption that the same drug is “worth” the same to the facility irrespective of location. • Step 3: Calculate the imputed value for each drug as the quantities received multiplied by the median unit prices • Step 4: For not commonly available drugs on the market, e.g. certain ACT supplied by the donors, unit prices from larger pharmacies in Abuja were used instead, given challenges in finding prices among local pharmacies in Ekiti and Niger. Values for salaries and vacancies were imputed from other sources of data. We imputed staff salaries costs from a combination of the staff roster and monthly salary as reported by the respondent. This value captured government obligations to paid staff employed at each facility, hence it is inclusive of unpaid salaries which were a major issue in Ekiti. Vaccinations are imputed from (i) the number of vaccination shots for each vaccine provided by the facility as recorded in the monthly outpatient attendance summary sheet and (ii) cost norm per vaccination shot. The imputed values from these sources were added to the direct cash receipts to come up with the resource envelope of PHC facilities. Key findings in terms of sources of financing and size of the resource envelope are provided below. These results are all generated applying sampling weights. 5.2. Main Findings 5.2.1. Sources of Financing Primary health care facilities mostly receive external support in kind, making user fees a core source of discretionary funds Public PHC facilities in Ekiti and Niger received resources from multiple sources (Error! Reference s ource not found.). All of them received support from the government, mainly because it pays salaries of their health workers. In addition, the government provided vaccines to about 89 percent of the facilities in both states, and drugs to 61 and 44 percent of facilities in Ekiti and Niger respectively. In all, about 96 percent of facilities in both states received in-kind support from the government. Non-governmental actors – international agencies, NGOs, communities and private donors – also contributed, with 43 and 27 percent of PHC facilities in Ekiti and Niger respectively, benefiting from their support. Most PHC facilities have user fees too. More so in Niger where 98 percent of facilities had internally generated revenues compared to 89 percent in Ekiti. Out-of-pocket spending is an engrained part of health financing in both states. 76 Table 5.1: Share of facilities receiving support by source of revenue Ekiti Niger Government Non- IGR Government Non- IGR government government 89.1 98.1 Direct cash transfers 0.0 35.0 1.1 2.8 - - Salaries and wages 100.0 0.0 100.0 0.0 - - In kind 95.9 19.0 95.8 24.6 - - Vaccines 88.7 0.0 89.4 0.0 - - Drugs 61.1 17.2 43.6 18.6 - - Medical supplies 5.7 1.8 1.2 4.0 - - Medical equipment 2.8 0.0 0.0 2.9 - - Other operational expenses 0.0 0.0 2.2 0.0 89.1 98.1 Any type of support 100 42.6 100 26.5 Government support to facilities is mostly in kind. During the period of the survey, none of the facilities in the sample received cash from the government in Ekiti and only one percent received such support from the government in Niger (Error! Reference source not found.). Instead, the government makes direct p ayments to providers (Error! Reference source not found.). The most important contribution is direct payment of salaries and wages to health workers, constituting 85 to 89 percent of government resources reaching PHC facilities in Ekiti and Niger respectively. The second most important is direct provision of vaccines and drugs which make up for 15 percent and 11 percent of support facilities received from the government in Ekiti and Niger respectively. These two types of support account for 98 percent of government support to PHC facilities. Table 5.2: Contribution to PHC resources by source and type of support Ekiti Niger Government Non-government Government Non-government Direct cash support 0.0 21.1 0.1 3.5 Salaries and wages 85.5 0.0 88.5 0.0 In kind 14.5 78.9 11.4 96.5 Vaccines(a) 12.6 0.0 9.6 0.0 Drugs(b) 1.7 77.9 1.7 95.3 Medical supplies 0.0 0.9 0.0 1.1 Medical equipment 0.2 0.0 0.0 0.1 Other operational expenses 0.0 0.0 0.0 0.0 All 100 100 100 100 Non-government actors provide both cash support and in-kind support, but this varies by state. Roughly 17 and 18 percent of facilities in Ekiti and Niger respectively, received in-kind drug supplies from donors. A considerable number of facilities in Ekiti (35 percent) also received cash from donors and well-wishers (Error! Reference source not found.). This mostly came from international donors who provided cash to 2 4 percent of facilities in the state, followed by the community (9 percent of PHCs) and then private donors (4.6 percent of PHCs). However, less than 3 percent of facilities in Niger received cash support – all from private donors. In terms of value, in-kind support made up a greater share than cash support even in Ekiti. They constituted 78 percent of direct non-government support to PHC facilities in Ekiti, with cash support 77 making up the remaining 21 percent (Error! Reference source not found.). Cash support barely registered a s a share of non-government support in Niger, dwarfed by the 97 percent contribution of in-kind support in the total value of direct non-government support to PHC facilities in the state. Figure 5.1: Composition of PHC resource by state Ekiti Niger 3% 11% 12% 14% 83% 77% Cash (including IGR) Wages and salaries Cash (including IGR) Wages and salaries Direct supplies Direct supplies Facilities thus have limited discretionary financing as cash revenues are a small share of their overall resources. The bulk of resources to PHC facilities go towards fixed costs and never pass through the facilities (Error! Reference source not found.). On average, wages and salaries constitute 83 percent and 7 7 percent of resources of PHC facilities in Ekiti and Niger respectively, and direct supplies (i.e. in-kind supply delivered directly to facilities) constitute between 11-14 percent. Cash (IGR and external cash receipts) as a share of revenue is higher in Niger, at 12 percent, but very small in Ekiti (3 percent) despite 90 percent of facilities there also charging user fees to patients and a higher share of their facilities receiving direct cash support from donors. This is a manifestation of the small sizes of cash donations made to facilities. For instance, the average cash donation to recipient facilities in Ekiti was around NG 14 000. The small cash contribution to overall resources in Ekiti even when IGR is included is also attributable to lower utilization rates for public PHCs in the state, meaning fewer transactions and consequently lower revenue being collected from user fees. The government is the predominant source of revenues for facilities The government is the major source of resources to facilities overall, owing to the dominance of wages and salaries in PHC facilities’ resources. Resources to PHC facilities coming through the government accounted for 97 and 87 percent of resources at PHC facilities in Ekiti and Niger, respectively ( Error! Reference s ource not found.a). This share includes donor support coming through government channels. It overstates government’s own contribution. The analysis presented in the previous chapter suggests that donor support accounted for 74 percent of actual government non-wage resources for PHC facilities in Ekiti and 78 percent in Niger. The bulk of in-kind support attributed as government contribution at the facility level, mostly vaccines and drugs, amounting to 14 and 10 of PHC facilities resources in Ekiti and Niger respectively, is indirect donor support. Adjusting for this suggests that government support at the facility level is 85 percent in Ekiti and 79 percent in Niger. That still makes government the dominant contributor of resources at PHC facilities, but nearly all of it comprising salaries and wages. 78 Figure 5.2: Composition of PHC revenues by source (a) Unadjusted for indirect donor support (b) Adjusted for donor support to government 100 3 100 3 12 12 90 90 12 1 9 Contribution to PHCs resources Contribution to PHCs resources 80 80 70 70 60 60 50 97 50 87 85 40 40 79 30 30 20 20 10 10 0 0 Ekiti Niger Ekiti Niger Government Non-government IGR Government Non-government IGR Adjusted for their contribution through government channels, donor support constitutes 12 and 9 percent of support to PHC facilities. Donors – international agencies in particular- provide less of their support directly to facilities, which is reflected in such support accounting for less than 1 percent of PHC facilities’ resources. The largest value of non-government support is made by international agencies and NGOs to the government instead, especially through vertical programs and other projects. Taking this into account reveals a much larger contribution of donors to resources received at PHCs. In Ekiti, non-government sources make the second largest contribution to resources received by PHC facilities once donor indirect support is included. User fees provided a significant contribution in Niger but less so in Ekiti. In the former, user fees accounted for 12 percent of resources, making them the second most important source of revenues at the facility level. However, they did not contribute much in Ekiti where they made up only 3 percent of revenues received. The role of IGR is however amplified when considering the composition of non-wage resources received by PHC facilities, and their sources of discretionary financing. User charges make up 52 percent of non- wage revenues of PHC facilities in Niger for example. With few facilities receiving direct cash support in both states, IGRs are also the most common form of cash revenues they had over the period of the survey. IGRs contributed to 97 percent of discretionary funds for facilities in Niger and almost all (99 percent) of discretionary financing for PHCs in Ekiti. Even though their contribution to the overall resource envelope (including wages) might be smaller, IGRs are an important source of financing for none wage expenses, particularly when it comes to plugging gaps not covered by in-kind supplies from the government and non- government actors. 79 Figure 5.3: Composition of PHC non-wage resources Figure 5.4: Composition of PHC discretionary revenues by source 100 100 3 1 90 90 Share of non-wage resources 80 80 47 70 70 60 83 60 50 50 97 99 40 40 30 30 52 20 20 10 17 10 0 0 Ekiti Niger Ekiti Niger Direct Cash IGR Inkind supplies IGR Direct Cash 5.2.2. Adequacy of resources The amount of resources at PHC facilities are inadequate to provide high quality of care at this level Based on actual revenues from all sources, the resource envelop of PHC facilities is quite low, when considering the demands against these facilities (Error! Reference source not found.). PHC facilities in E kiti and Niger respectively had a resource envelope of NG 1.9 million and NG 2.2 million per facility on average during the 6 months period of the survey. Based on utilization rates, these amounts respectively translate to NG 3,040 and NG 4,386 per user over six months. This means resources at PHC facilities (wages and salaries included), range from USD 15 to USD 22 per user, based on prevailing exchange rates39 at the time. The amounts are much lower in per capita terms as the resources PHCs receive and generate amount to N 235 per capita in Ekiti and N 533 per capita in Niger, thus equivalent to between USD 1 – 2.5 per capita. Figure 5.5: PHC resources per user and per capita by state ₦ 5,000 ₦ 600 4,387 533 ₦ 4,500 ₦ 4,000 3,829 ₦ 500 466 3,469 422 Amount per capita ₦ 3,500 2,939 3,040 ₦ 400 Amount per user ₦ 3,000 2,583 ₦ 2,500 ₦ 300 227 235 ₦ 2,000 200 ₦ 1,500 ₦ 200 ₦ 1,000 ₦ 100 ₦ 500 ₦- ₦- Government Government Overall Government Government Overall support support - resources support support - resources adjusted adjusted Ekiti Niger Ekiti Niger 39 Exchange rate applied at 1 USD : NG 199 80 Notes: Government support - adjusted means government support excluding donor support coming through government channels. The amount of resources received per capita in Ekiti and Niger, indicates that only a small share of public spending on health reached PHC facilities and points to lack of prioritization of primary health care. Public spending is either concentrated on curative rather than preventative care or a greater share of this spending is not devoted to frontline service provision. Excluding user charges and direct non-government support, shows that government supported public PHC facilities to the tune of NG 225 and NG 465 per capita in Ekiti and Niger respectively, that is equivalent range from USD 1 to USD 2 per capita. This pales in comparison to the estimate of USD 16 and USD 19 government per capita spending in Ekiti and Niger respectively. Another indicator of the low prioritization of PHC facility spending is that PHC facilities receipts from the government in Ekiti and Niger, range between 6 and 11 percent of government spending on health in these states. Lack of basic infrastructure, equipment and drugs at primary health facilities attests to the inadequacy resources at Primary Health Care facilities (see Table 5.3). On average, facilities only had 2 of the six basic medical equipment in the minimum equipment SDI indicator (sphygmonometer, thermometer stethoscope, weighting scale, sterilizer and refrigerator). They had less than 5 percent of tracer drugs available and only 21 percent had the minimum infrastructure (electricity, improved water source and improved toilets). The lack of basic medical equipment was especially glaring. Even the top 20 percent performing facilities in terms of input availability averaged between three and four out the six basic medical equipment PHCs must have. Sterilizing equipment and refrigerators are the most uncommon, especially in Niger for the later. The primary difference between the top 20 percent performing facilities in terms of input availability is they had most of the infrastructure in place (90 percent of them had all three basic infrastructure) and they were much more likely to have a thermometer (81%), weighing scale (61%), sphygmonometer (91%) and a refrigerator (45%). Table 5.3: Availability of basic inputs by State and PHC input performance ranking Input availability State rankinga Bottom Top 20 All Input indicator Ekiti Niger 80 percent percent facilities Share with electricity 49.5 29.1 25.0 93.3 33.0 Share with improved water source 77.2 90.7 86.5 100.0 88.1 Share with functioning improved toilet 70.9 45.4 44.0 97.1 50.2 Share with functioning thermometer 46.1 47.9 43.1 81.2 47.5 Share with functioning stethoscope 59.2 70.0 68.9 60.7 68.0 Share with functioning sphygmonometerr 55.7 54.8 50.2 91.0 55.0 Share with functioning weighting scale 50.4 24.4 25.1 61.1 29.3 Share with functioning sterilizer 4.1 2.9 2.7 6.7 3.1 Share with functioning refrigerator 37.2 1.9 3.8 45.2 8.6 Average number of basic medical equipment (out of 6) 2.5 2.0 1.9 3.5 2.1 Average number of basic infrastructure available (out of 3) 2.0 1.7 1.6 2.9 1.7 Share with all minimum infrastructure needed 37.5 17.2 11.8 90.4 21.0 Share with all minimum equipment 0.4 0.0 0.0 0.6 0.1 Share of priority drugs available 2.8 5.0 4.4 5.6 4.5 81 Notes: Input availability ranking is based on principal component analysis of three core SDI indicators namely share of priority drugs available, having minimum medical equipment and minimum infrastructure. PHCs were then ranked based on the first principal component from this analysis. Lack of prioritization of non-wage expenditures is the key driver of low PHC financing Poor availability of inputs is likely to persist as non-wage expenditures are not highly prioritized. The current financing structure directs few resources towards purchasing drugs and medical consumables, let alone equipment and PHC facilities infrastructure and amenities, despite these being essential for the delivery of the minimum ward health package. Put together, these categories only constitute less than 3 percent of external resources transferred to facilities. Direct supplies to facilities mainly comprise of vaccines which account for 12 and 8 percent of external support to PHC facilities in Ekiti and Niger respectively. In contract, facilities received just under NG 40,000 worth of drugs. Thus drug supplies only made up about 2 percent of external resources (i.e. excluding IGR) of PHCs facilities (Error! Reference s ource not found.). Despite the poor availability of medical supplies and equipment, facilities barely received them from the government or non-government sources. Less than 8 percent of PHCs in Ekiti and about 5.2 percent in Niger, received medical supplies over the 6-month period of the survey. In both states, less than 3 percent received medical equipment of any kind. Only about a million worth of supplies and equipment were delivered to all facilities in Ekiti and barely a quarter of that in Niger. Facilities that received this direct support are among the top performers in terms of input availability. It is not surprising then, that regressions40 of aggregate input availability indicators show these to be significantly correlated with receipt of drugs from NGOs or directly from the government and amount of discretionary funding received by facilities, which is mostly internally generated revenues. Table 5.4: Disaggregation of average resources per PHC facility by source and type Ekiti Niger Government Non- Total Share in Government Non- Total Share government total government revenues in total 1,652 Financial 0 1,652 0.1 1,724 478 2,202 0.1 1,641,619 Salaries 1,641,619 0 82.7 1,346,798 0 1,346,798 77.3 284,321 In kind 278,163 6,158 14.3 172,903 13,198 186,101 10.7 241,009 Vaccines) 241,009 0 12.1 146,249 0 146,249 8.4 39,670 Drugs 33,583 6,087 2.0 26,446 13,037 39,483 2.3 96 Medical supplies 25 71 0.0 51 150 201 0.0 3,546 Medical equipment 3,546 0 0.2 0 11 11 0.0 Other operational expenses 0 0 0 0.0 156 0 156 0.0 IGR - - 58,027 2.9 - - 207,643 11.9 Total 1,919,782 7,809 1,985,618 100.0 1,521,425 13,676 1,742,743 100.0 40 Two types of regressions were estimated. One is an ordered probit with availability of drugs, minimum infrastructure and minimum equipment added to form an aggregate score ranging from 0 to 3. Another is a quintile regression (at the 80th percentile) of the first component from the principle component analysis of these three variables. Both types of regressions were estimated in two specifications, one with binary variables for having received a particular type of support, another with the sine-inverse hyperbolic transformation of the amounts received. The results in both specifications are qualitatively similar. 82 Figure 5.6: PHC budget allocation and execution by state 0.6 0.8 (a) Budget allocation Ekiti (b) Budget execution Niger 0.7 0.5 Ekiti 0.6 Niger Share in budget 0.4 0.5 0.3 0.4 0.2 0.3 0.2 0.1 0.1 0 0 PHC budget PHC non-wage Health non- Health non-wage PHC non-wage PHC wage actual share in health budget share in wage budget actual share in actual share in share in PHC budget health non- share in health health non-wage PHC non-wage wage budget wage budget budget budget budget Low non-wage resources at the facility level reflect lack of prioritization of non-wage health expenditures at all levels of the government. Firstly, the budget allocation towards non-wage health expenditures is low, with just above a quarter of the health budget in Niger and Ekiri allocated non-wage expenditures (Error! R eference source not found.a). Of this allocation, less than half is allocated to PHC non-wage expenditures, implying that less than 14 percent of the health budget is allocated to PHC non-wage expenditures. The reported budget allocations are inclusive of support received from donors. If this is excluded, government’s own revenues allocation towards non-wage PHC spending is reduced to around 17 percent of the health budget allocation by the government (all levels) in the two states. Poor budget execution of non-wage spending for health compounds the inadequacy from originally low allocations (Error! Reference source not found.b). Only 10 percent of the government’s own non-wage b udget for PHCs is executed in Ekiti. In Niger, the execution rates for PHC non-wage expenditures is 17 percent. These exactly mirror poor budget execution of non-wage health expenditures overall in the two states. Largely being discretionary, non-wage spending bear a disproportionate burden of budgetary adjustments, in contrast to the wage budget with a much higher budget execution rate. Clearly, the government funding model of non-wage health expenditures characterized by lack on conditional transfers and largely depended on discretionary LGA budgets and donor funding, has resulted in significant gaps in non-wage financing for PHC facilities. Internally generated revenues are consequently the major source of non-wage expenditures. Without direct cash support and with low government spending on non-wage health expenditures, out-of- pocket spending is the most important source of financing for drugs, medical supplies and other expenses. Facilities generated most of their internal revenues from selling drugs with 83 and 99 percent of facilities in Ekiti and Niger respectively, selling drugs to patients. These drug sales made up 75 percent of IGRs in Ekiti and 85 percent in Niger. More than half of facilities in both states charged consultation fees, which contribute 3 percent of IGRs in Niger and up to 6 percent in Ekiti. All other fees provide 16 percent of IGRs in Ekiti and 7.6 percent in Niger. 83 Table 5.5: Disaggregation of internally generated revenues by type Ekiti Niger Percentage Amount collected by Share Percentage Amount collected by Share of PHCs facility of total of PHCs facility of total with Average Total IGR with Average Total IGR revenue (Naira) (Naira) (%) revenue (Naira) (Naira) (%) IGR source source source Drug sales 83.1 43,459 12,255,461 75.0 98.7 175,825 212,747,986 84.9 Consultations 58.5 3,626 1,022,659 5.9 51.3 7,949 9,618,637 3.3 Laboratory tests 10.8 1,631 460,071 3.1 18.4 7,023 8,498,270 4.2 Others* 80.0 9,310 2,625,522 15.9 63.2 16,845 20,382,910 7.6 All sources 86.2 58,027 16,363,713 100.0 98.7 207,643 251,247,804 100.0 Notes: *Other fees include inpatient bed charges, discharge fees, one off registration fees and charges for specialized services when offered. PHC facilities spent some of their discretionary funds on administrative expenses as well, but much less on medical supplies and equipment. On the top of the list of administrative expenses are salaries and stipends paid to temporary workers and volunteers - a practice more common in Niger where close to 28 percent of facilities incur such expenditures. Transportation is the next major expense, incurred by 80 percent of facilities in Ekiti and 58 percent of facilities in Niger. Unlike drugs, PHC facilities only spent little of their cash revenues buying supplies and equipment. Less than 5 percent of PHC facilities in Ekiti bought medical supplies and only 15 percent did so in Niger. On average PHCs spent just NG 53 and NG 145 on medical supplies in Ekiti and Niger respectively. Along with infrastructure, medical supplies and equipment are a grossly neglected area, with neither the government, donors nor the PHC facilities themselves spending a significant amount of resources on these items. This explains the poor availability of equipment and poor access to amenities in these areas. Table 5.6: Composition of PHC expenditure (per 6 months of survey) Ekiti Niger Share of Amount collected by Share Share of Amount collected by Share of facilities facilities of total facilities facilities total (%) with Average Total (%) with Average Total Expenditure expense (Naira) (Naira) expense (Naira) (Naira) Drug purchase 58.5 33,970 9,579,664 58.9 98.7 126,749 153,366,465 75.9 Medical supplies 4.6 63 17,733 0.1 14.5 172 208,185 0.1 Salaries and wages 9.2 3,354 945,800 11.5 27.6 6,711 8,119,750 7.8 Transportation 80.0 3,570 1,006,814 8.1 57.9 6,475 7,835,337 5.2 Food and rations 6.2 694 195,741 1.0 36.8 4,119 4,983,600 2.8 OPD cards 10.8 797 224,720 1.3 32.9 2,817 3,408,745 2.3 Other expenditures 75.4 9,919 2,797,233 19.1 72.4 11,165 13,509,993 5.9 All expenditures 92.3 52,359 14,765,256 100.0 98.7 158,010 191,192,134 100.0 5.3. Financing for drugs and their availability at PHCs Drugs are mainly purchased out of user fees contributing to their limited supply at facilities. Much of the internally generated revenues come from drug sales and are spent on procuring drugs. Facilities in Niger spent NG 115 000 buying drugs, well above the NG 26 000 worth of drugs they received from the 84 government and NG 13 000 worth of drugs received from non-government actors. In Ekiti, facilities spent as much buying drugs as the value of drugs they received from the government (around NG 33,000), with non-governmental actors providing an average of NG 4,756 worth of drugs per facility. In both states, IGRs are clearly an integral source of drug financing, as their systems are designed to finance drugs through user fees as further illustrated in Figure 5.7. However, the procurement sources of drugs are starkly different. Public procurement accounted for a greater share drugs purchases and supplies in Ekiti (about 90 percent of the value) and the private sector for a much larger share in Niger (80 percent of the value). In both states, direct supplies from donors accounted for less than 10 percent of the value of drugs, with a higher contribution in Ekiti than in Niger (Figure 5.7). Figure 5.7: Disaggregation of drugs value by financing source 100 5.1 Share in value of drugs supplied (%) 90 80 70 74.3 60 50 40 5.6 30 1.7 20 4.4 10 7.0 0 Ekiti Niger Federal Gvt State Gvt Local Gvt Non-government DRF (Paid) Other own purchases Figure 5.8 Disaggregation of drugs value by procurement source 100 5.6 4.4 Share in value of drugs supplied (%)_ 90 5.1 80 70 60 74.3 50 89.3 40 30 20 10 21.3 0 Ekiti Niger Public Private sector Donors 85 Purchases from the private sector were more responsive to drug stock outs than public procurement Figure 5.9: Drugs Stock out rates 14 Out of stock drugs in October ever Availability of tracer drugs at the beginning of each replenished during the survey month by state 70.0 12 Share of out of stock drugs replensished (%) Share of tracer drugs available (%) Ekiti Niger 62.1 60.0 10 51.7 50.0 8 39.5 40.0 37.0 6 30.0 4 20.0 2 10.0 0.0 0 Ekiti Niger Oct Nov Dec Jan Feb Mar Apr Tracer drugs Non-tracer drugs 2015 2016 With limited contribution from the government, the supply of drugs is inadequate, irrespective of how they are procured. The share of tracer drugs available each month barely exceeded 6 percent in Ekiti and was less than 13 percent in Niger (Figure 5.9). There was little variability across months, partly reflecting a low replenishment rate. The share of out of stock tracer drugs ever purchased or received was a 52 percent in Ekiti and 62 percent in Niger (Figure 5.10). Purchases from the private sector were thus more responsive to drug stock outs than public procurement, but the response was not adequate at all. Out of the 46 tracer drugs, stock were never maintained for over half of them for the duration of the survey. Figure 5.10: Drugs replenishment rates Tab.Arthemeter +Lumefantrine 21 5 16 Delivered Purchased Syr Vitamin C 10 Cap. Amoxycillin 500Mg 22 3 Inj Arthemeter 80Mg 11 10 Susp Metronidazole 21 12 Tab. Co-Trimoxazole 14 18 Tab. Vit. B Complex 24 19 Gentamyicin Eye/Ear Drops 12 23 Tab. Paracetamol 500Mg 36 24 0 10 20 30 40 Number of times drugs was purchases or delivered in all facilities 86 No specific procurement channel performs consistently better on unit prices of drugs. The volume of purchases, rather than procurement channel appeared to be a more important determinant of prices. Procurement prices are analyzed for drugs where a price could be obtained for both states and for both CMS and private procurement. The available data on drugs does not allow to distinguish between generics and originator drugs. That 70% of drugs in the study are manufactured in Nigeria, with China and India accounting for another quarter of drugs suggests that most drugs captured are likely generics or branded generics, whose prices would be expected to be significantly lower than that of originator drugs. Table 5.7: Comparison of drug prices Ekiti Niger Median Mean Median Mean Median Mean Median Mean Dosage/ private private public public private private public public drugitem Drug Form Strength price price price price price price price price Amoxicillin Capsule Or Tablet 500 Mg 1.6 1.8 0.6 0.7 1.3 1.3 2.4 4.6 Cimetidine Capsule Or Tablet 200 Mg 2.9 2.9 3.5 3.5 2.2 3.5 5.5 4.1 Erythromycin Capsule Or Tablet 250 Mg 3.8 5.5 1.2 1.2 2.6 3.7 3.6 3.6 Folic Acid Capsule Or Tablet 2 Mg 1.7 2.9 0.4 1.0 1.7 1.9 1.3 1.4 Magnesium Trisilicate Capsule Or Tablet 500 Mg 2.5 2.2 1.3 1.3 1.3 1.4 1.1 1.1 Vitamin B Complex Capsule Or Tablet 20 Mg 1.6 1.5 0.4 0.4 0.7 0.9 0.7 0.5 AVERAGE 2.3 2.8 1.2 1.3 1.6 2.1 2.4 2.6 Prices of these drugs were compared to a standard International Reference Price (Management Sciences for Health (MSH) 2014 prices). Most drugs examined have unit prices between 1 and 5 times the IRP both for public and private procurement (Table 5). While in Ekiti, private prices are about twice as high as public prices, in Niger, private prices are lower that public prices by 20 to 30%. Also, the average public price in Niger is twice as high as the average public price in Ekiti and at similar level as the average private price in Ekiti. This suggests that high price levels are not inherent to the channel, but rather high volumes probably drive down the unit price. Drugs procured from both public and private sources focus on the same therapeutic areas and point to poor adherence to treatment protocols The Top 5 drugs procured in each state are very similar and prioritize the same therapeutic areas regardless of the procurement method ( 87 Table 5.8). In Ekiti the 5 most purchased drugs are Co-Trimoxazole, Multivitamin, Vitamin C, Vitamin B Complex and Folic acid and in Niger, the list consists of Co-Trimoxazole, Vitamin C, Vitamin B Complex, Ferrous Sulphate and Gentamycin (followed by multivitamin). Antibiotics are the top category overall, accounting for 62% of “top 80” value in Ekiti (almost all of it co -trimaxozole) vs for 51% in Niger, with co-trimaxozole accounting for over 75% of antibiotic value. Vitamins (B, C, multi) account for 22% of “top 80” value in Ekiti vs 35% in Niger. 88 Table 5.8: Comparison of most frequently purchased drugs in Ekiti and Niger EKITI NIGER TA Drug Items Form Dosage Total Value % share TA Drug Items Form Dosage Total Value % share ANTIBIOTICS 42,797,062 62.0% ANTIBIOTICS 81,822,765 51.2% Amoxicillin Capsule Or Tablet 500 Mg 222,272 0.3% Amoxicillin Capsule Or Tablet 500 Mg 3,372,900 2.1% Ampicillin Capsule Or Tablet 250 Mg 135,135 0.2% Ampicillin And Cloxacillin Capsule Or Tablet 500 Mg 487,795 0.3% Co-Trimoxazole Capsule Or Tablet 480 Mg 41,895,000 60.7% Co-Trimoxazole Capsule Or Tablet 200 Mg 350,000 0.2% Erythromycin Capsule Or Tablet 250 Mg 179,550 0.3% Co-Trimoxazole Capsule Or Tablet 480 Mg 61,711,650 38.6% Erythromycin Syrup/Suspension Not Applicable 143,485 0.2% Co-Trimoxazole Syrup/Suspension Not Applicable 110,070 0.1% Gentamycin Injection Not Applicable 221,620 0.3% Doxycycline Capsule Or Tablet Not Applicable 140,100 0.1% VITAMINS AND MINERALS 15,518,133 22.5% Gentamycin Injection Not Applicable 15,650,250 9.8% Multivitamin Capsule Or Tablet Not Applicable 6,387,540 9.2% VITAMINS AND MINERALS 56,530,100 35.4% Vitamin B Complex Capsule Or Tablet 20 Mg 712,187 1.0% Multivitamin Capsule Or Tablet Not Applicable 2,022,400 1.3% Vitamin B Complex Injection Not Applicable 2,982,429 4.3% Multivitamin Syrup/Suspension Not Applicable 2,174,930 1.4% Vitamin C Capsule Or Tablet Not Applicable 2,522,520 3.7% Vitamin B Complex Capsule Or Tablet 20 Mg 1,801,050 1.1% Vitamin C Syrup/Suspension Not Applicable 2,913,458 4.2% Vitamin B Complex Injection Not Applicable 17,607,225 11.0% Vitamin C Capsule Or Tablet 20 Mg 2,114,350 1.3% Vitamin C Capsule Or Tablet Not Applicable 3,384,500 2.1% Vitamin C Syrup/Suspension Not Applicable 27,425,645 17.2% DRUGS AFFECTING THE BLOOD 3,180,722 4.6% DRUGS AFFECTING THE BLOOD 15,770,020 9.9% Folic Acid Capsule Or Tablet 20 Mg 3,180,722 4.6% Blood Tonic Syrup/Suspension 200 Mg 101,700 0.1% Ferrous Sulphate Capsule Or Tablet 200 Mg 15,668,320 9.8% GASTRO INTESTINAL 3,150,000 4.6% GASTRO INTESTINAL 2,742,670 1.7% Magnesium Trisilicate Capsule Or Tablet 500 Mg 3,150,000 4.6% Magnesium Trisilicate Capsule Or Tablet 500 Mg 1,800,000 1.1% Magnesium Trisilicate Syrup/Suspension 200 Mg/100 Ml 251,790 0.2% Metoclopramide Not Applicable Not Applicable 70,000 0.0% Ors(Lo)With Zinc Satchet Not Applicable Not Applicable 620,880 0.4% OTHER 4,430,271 6.4% OTHER 2,992,130 1.9% Antimalarials Artesunate Capsule Or Tablet 140 Mg 1,094,038 1.6% Amoebicide Metronidazole Syrup/Suspension Not Applicable 1,303,645 0.8% Arthemeter Injection Not Applicable 433,818 0.6% Anti Allergics Chlorpheniramine Capsule Or Tablet 200 Mg 532,350 0.3% Nonsteroidal anti-Inflammatory Diclofenac Capsule Or Tablet 50 Mg 99,750 0.1% Chlorpheniramine Injection Not Applicable 182,000 0.1% Ibuprofen Capsule Or Tablet 200 Mg 1,050,000 1.5% Nonsteroidal anti-Inflammatory Ibuprofen Capsule Or Tablet 200 Mg 647,000 0.4% Dessings and devices Disposable Gloves Not Applicable Not Applicable 626,400 0.9% Antacid and antihistamine Cimetidine Capsule Or Tablet 200 Mg 109,920 0.1% Anti allergics Chlorpheniramine Injection Not Applicable 428,584 0.6% Misc. 4.3% Dextrose Saline Injection Not Applicable 217,215 0.1% Oxytoxic Ergometrine Injection Not Applicable 154,140 0.2% Oxytocin (Pitocin) Injection Not Applicable 85,365 0.1% Antacid and antihistamine Cimetidine Capsule Or Tablet 200 Mg 167,480 0.2% Anthelmintics Albendazole/Mebendazole Syrup/Suspension Not Applicable 88,834 0.1% steroid Hydrocortisone Injection Not Applicable 78,750 0.1% Analgesics Paracetamol Injection Not Applicable 64,554 0.1% Amoebicide Metronidazole Syrup/Suspension Not Applicable 58,558 0.1% TOTAL 69,076,188 100.0% TOTAL 159,857,685 100.0% 89 The heavy emphasis on a few drugs could indicate a lack of treatment protocols and a mismatch between treatment and patient conditions in both states. Heavy emphasis on a few drugs or drug categories drugs would suggest that they are heavily prescribed, potentially inappropriately regarding the patients’ ailments. In both states, the top 5 drugs account for ~ 85% of total value of the “top 80”. Vitamins are the second most important category of drugs in the “top 80”. This seems excessive unless a need for specific targeting of malnutrition exists. Use of some drugs do not reflect international guidance or best practices. Co- Trimoxazole, the top drug in both states (75% of antibiotics value out of the top 80 in Niger and almost 100% in Ekiti) is a relatively outdated antibiotic. Antimalarial monotherapies appear in the “top 80” in Ekiti, when, per WHO guidance, they should not be used anymore. 5.4. Summary The government is the most important source of resources of public primary health care facilities. However, its support to facilities almost entirely comprises of wages and salaries. Outside donor provided support, government’s spending of its own resources to primary health care is very limited. A combination of the shortfall in non-wage spending and that a smaller share of government’s health budget is devoted to primary care, are the reason why resources at PHC facilities are inadequate, and spending is less responsive to gaps in availability of drugs, medical equipment and infrastructure. Facilities do not receive any discretionary funding from the government and that coming from non-government sources is negligible. They try to make up for the short fall using IGR but this also is not adequate. Performance of facilities in terms of inputs availability is primarily dependent on having discretionary financing. Receiving direct in-kind support was another determining factor. Thus, the best performers are those facilities that raised more in IGR, received donor supplies of drugs, were in LGA that delivered drugs or were lucky enough to get cash support from the government. Nonetheless, purchase of medical equipment and upgrading of facilities infrastructure are totally neglected, being neither financed from discretionary facility funds nor by the government. IGRs were more crucial for financing drugs, through procurement channels differ. Niger which relied more on private purchases of drugs replenished out of stocks more often than Ekiti which depended heavily on public procurement. The cost of drugs depended on the predominant method of procurement in each state, with procurement from private sector being cheaper in Niger, but more expensive in Ekiti. Despite the procurement channel, the prioritized therapeutic areas were the same. Thus, public procurement of drugs doesn’t seem to offer any particular advantages over private procurement, which instead seemed more responsive to drug stock outs. 90 CHAPTER 6: Performance Contingent Incentives To Motivate Behavioral Change 6.1. Status of Health Care Expenditure Tracking in Nigeria Previous chapters have attempted the best estimates to date regarding the financing of primary health care in Nigeria. However, historically the government of Nigeria has not been able to reliably track information on how much it spends on primary health care. This is partly because classifications of expenditure in the health sector tend to be too broad to capture such granular but vital information, but also because the funds that flow to primary health care in the country are heavily fragmented, with no system in place to aggregate them into central databases. This makes it important to devise alternative means of tracking income and expenditure in the sector. In an effort to help Nigeria with that development, the World Bank introduced an innovative continuous Public Expenditure Tracking Survey (PETS) in two states (Ekiti and Niger), which records and aggregates information on resource flows in real time, over time, rather than follow the more established practice of retrospectively generating such data. The need is especially great at the facility level, for which no one – including local government and state authorities – has reliable information on income and expenditure streams. One of the central issues identified by previous studies is the weakness of accountability structures at the facility level – a gap that the current World Bank-financed Resource Tracking (RT) Study introducing the ‘continuous PETS’ is attempting to address. The main contribution of the PETS has been to design and introduce relatively simple-to-fill-out forms (excel spreadsheets; RT tools; see Chapter 2 for a detailed description of all newly introduced forms in each fo the instrument sections) to track various income and expenditure streams such as, for example, a cash book Form C (for the recording of cash receipts from user charges e.g., consultation fees, laboratory tests, and drug sales and expenditures such as due to purchases of drugs, equipment, materials, or supplies, wages, and facility maintenance), and to teach staff at the facility level to fill out these forms on a daily basis. Weekly visits by enumerators served to supervise and observe the uptake of these RT tools. The underlying idea here was that capacity building is one of the major first areas needing intervention on the road to primary health care sector reform. Importantly, any policy change that improves performance may not do so homogeneously : both the impact of the policy and the possible heterogeneity of that impact are of interest. Recent work shows that the same intervention can have different effects in the hands of different implementers – a regularity across low- income, middle-income, and high-income countries (Bold et al., 2013 ; Banerjee, Karlan, and Zinman, 2015 ; Allcott and Mullainathan 2012). This project attempted to both acknowledge and further test such a hypothesis. The RT project takes place in two very different states : Ekiti and Niger. As can be seen in Figures 1a and 1b here (see also chapter 1.2), Ekiti is one of the most educated states in Nigeria, with one of the lowest rates of adults with no formal education of any state in Nigeria ; Niger is the opposite. We find, however, that the different institutional and governance contexts in both states can also have an impact on the effectiveness of the very same intervention. 91 Figures 6.1a and 6.1b. Fraction of Men and Women, respectively, with no formal schooling in Nigeria by region Fraction of Men with No Formal Schooling (DHS 2013) 100 80 60 40 20 0 Lagos Rivers Borno Zamfara Bauchi Ebonyi Yobe Jigawa Taraba Oyo Kogi Cross River Akwa Ibom Edo Niger Average Adamawa Kwara Ekiti Bayelsa Gombe Kano Ogun Plateau Ondo Osun Sokoto Kaduna Nasarawa Enugu Delta Imo Benue Kebbi Anambra Katsina Abia FCT-Abuja Non-Study Areas Study Areas Average Fraction of Women with No Formal Schooling (DHS 2013) 100 80 60 40 20 0 Lagos Rivers Ebonyi Zamfara Bauchi Jigawa Taraba Oyo Yobe Average Adamawa Kwara Kogi Bayelsa Edo Borno Niger Cross River Akwa Ibom Kano Ogun Ondo Ekiti Gombe Plateau Sokoto Kaduna Nasarawa Osun Imo Benue Delta Kebbi Enugu Katsina Abia FCT-Abuja Anambra Non-Study Areas Study Areas Average Information and knowledge are without doubt important factors in reforming resource management in health care. However, it is likely not sufficient to change attitudes and instill a desire to change behavior particularly at the facility level where resource tracking practices have been historically below par. Based on extended field-visits in Ekiti and Niger it has been found that record-keeping is generally viewed as a mundane task, with no broader pro-community mission acting as a motivating factor to undertake the task. There is no existing incentive to track rescources systematically. In fact, there may even be perverse incentives to further highlight a situation of scarcity – with the assumption that under-reporting of resources could lead to more assistance especially from development partners. In addition, there is a structural weakness to enforce accountability. Very rarely does a fear of supervision exist that makes facility officers ensure records are maintained, to protect themselves from corruption allegations. It was therefore of interest to the team to test how to incentivize filling of the instrucments of the PETS study to learn how in addition 92 to the more structural changes and knowledge transfer that occured through the PETS, one could increase the motivation of staff at the facility level and improve the reporting primarily of the their income and expenditures. This chapter 5 describes the testing and results of two incentives to increase motivation: material/economic and social-recognition incentives. 6.2. Motivating Reporting There are various ways to motivate human behavior when it falls short of expectations. One of the most basic distinctions is between interventions to improve intrinsic motivation (which according to Ryan and Deci, 2000, p. 55, “refers to doing something because it is inherently interesting or enjoyable”) and extrinsic motivation (“which refers to doing something because it leads to a separable outcome”). Rewards—such as a job promotion, money, a sticker, grades, or candy—are examples of extrinsic motivators. Social and emotional incentives like praise, attention, and status, or social recognition rewards (e.g. “employee of the month”) are also extrinsic motivators since they are bestowed on the individual by another person. Extrinsic rewards are often used to impact someone who shows little interest in a potentially useful activity (see, e.g., the use of monetary rewards to motivate blood donations; Lacetera, Macis, and Slonim, 2013). Financial and economic incentives in particular are commonly used in the labor market because of their standard direct price effect and ease of implementation (Arrow, 1972; Gneezy, Meier, Rey- Biel, 2011). However, they are relatively expensive, making them sometimes less sustainable. Thus, cheaper social incentive-motivators have become a topic of increased study. Lotteries as motivator to improve performance Lottery tickets, or raffles, have recently been tested as a way to scale the use of more valuable monetary reward incentives, to increase the standard direct price effect, which makes the incentivized behavior change more attractive. The idea here is that humans’ tendency for wishful thinking (Krizan and Windschitl, 2007) together with the overweighting of small probabilities (i.e. humans tend to overestimate the likelihood of small probability events occurring; see Kahneman and Tversky, 1979) makes them focus on winning the attractive grand prize rather than on the uncertainty. Therefore, a lottery may make a more effective motivator for behavioral change than a smaller certain prize, given a particular expected value. For example, in recent randomized controlled field experiments researchers found financial lotteries to improve adherence to medications such as warfarin (to lower risks of heart attacks; Kimmel et al., 2012 ; Volpp et al. 2008a) as well as to promote weight loss (Volpp et al. 2008b). Similarly, in a different domain, Naritomi (2015) showed that offering consumers incentives to collect Value Added Taxes (VAT) receipts in the form of, among other methods, lottery tickets increased consumers’ likelihood to ask for receipts for their purchases, thereby acting as a monitoring device that improved firm-level tax revenue reporting and compliance. Finally, in an ongoing study Ideas42 (unpublished work) used lotteries in India to improve road-safety: Valuable prizes were announced to be awarded to qualifying drivers. Specifically, all motorists in a province were automatically entered into the draw, but in order to stay eligible for the prize, they had to ensure that they did not speed, drink & drive, or use cell phones while driving. In addition, they needed to be mindful of pedestrians and make sure everyone in the car was buckled up. Preliminary data suggests that the period with the lottery saw a dramatic decline in road deaths per day. Recognition as a motivator to improve performance The idea that individuals are motivated by public and peer recognition is not new. The notion that individuals crave status has long been studied and has more recently been formalized (Moldovanu, Sela, and Shi, 2007; Besley and Ghatak, 2008). Compared to monetary rewards, recognition is also less likely 93 to have the indirect psychological effect of crowding out the intrinsic motivation of the recipients (Ryan and Deci, 2000). In fact, there is an argument that recognition targets and enhances intrinsic motivation (Neckermann and Bruno, 2008). Incentives or awards that are not primarily financial may affect behavior through several mechanisms: (a) by conferring other peoples’ recognition; (b) by enhancing social visibility, and (c) by facilitating positive influence by peers on performing particular tasks. There is consistent empirical evidence to show that employees value recognition. For example, Nelson (2001) reviewed studies that identified ‘appreciation’ and ‘recognition’ as being more important than traditional incentives such as ‘good wages’ and ‘job security’ or ‘promotion opportunities.’ This observation remains true in developing countries with financial constraints as well. A number of studies on developing countries suggest that non-financial incentives such as ‘recognition’ are important to drive employee behavior (Mathauer and Imhoff 2005; Stilwell, 2001). Specifically regarding the health sector, Mathauer and Imhoff (2006) engaged in detailed semi-structured qualitative interviews with staff from healthcare facilities from public, private, and NGO facilities in rural areas of Benin. They concluded that health workers highly valued recognition and appreciation from superiors and colleagues as well as patients. Furthermore, this valuation did not differ by the type of institution (private, public, or NGO). A meta-analysis by Stajkovic and Luthans (1997) showed that social rewards (recognition and attention) had a big impact on employee performance. Stajkovic and Luthans (2001) reinforced this finding after a quasi-experiment in a large company. Peterson and Luthans (2006) similarly found that non-monetary incentives that included formal recognition and appreciation had a significant impact on employee performance and business outcomes in 21 stores in a fast-food corporation. While financial incentives initially had a greater effect, over time the financial and non-financial incentives had an equally significant impact. Larkin (2011) used observational data to study a non-linear incentive scheme that provided employees of a software firm with a “gold star” and company-wide recognition if they meet an annual performance threshold. These employees gave up about 27,000 USD worth of commission to obtain the non-financial reward. The reward included a gold star on their name card, company-wide recognition, an email from the CEO, and a weekend trip (which cost much less than 27,000 USD). While the above studies show that positive recognition can incentivize performance, there is evidence that negative recognition can also positively affect behavior change. For example, Chetty et al. (2014) conducted a study in Bangladesh where they promised to reveal information of firms’ tax payment behavior to neighboring firms with the intent to change tax behavior. The authors found that in the neighborhoods where some firms were already complying, the promise of exposing information about all firms’ tax payment behavior led to a positive behavioral response and an increase in tax compliance. Firms who deviated from the norm in their cluster, and who were therefore at greatest risk of “negative” information revelation relative to their peers, acted most strongly. Finally, in a field experiment run in collaboration with a public health organization, Ashraf, Bandiera, and Jack (2014a) randomized 800 community agents hired to sell condoms in urban compounds into four monetary and nonmonetary performance contingent reward treatments. Agents who were assigned to the nonmonetary reward treatment—namely, stars for performance plus a public ceremony for top performers—sold twice as many condoms as agents who were offered a financial margin on each pack sold. Together these findings suggest that both external incentives in terms of financial and social/public rewards can be effective motivators of behavioral change. Based on these insights and a growing body of evidence suggesting that “nudges” to influence behavioral choices can be used to address problems in public service 94 delivery (World Bank, 2015; Thaler and Sunstein, 2008; Ashraf, 2013; see also lessons drawn from different field-experiments of Nava Ashraf and colleagues 2014a, 2014b, 2016), we designed a randomized control trial (RCT) in which we tested in the field the effectiveness of two different types of external incentives (lottery with material reward vs. social recognition) to improve the performance (i.e. quality) of record-keeping at medical facilities (particularly of the ‘cash book’ Form C; see Chapter 2 for a more detailed description of Section C) in Nigeria. The practicality of the interventions beyond the study period was an important consideration in the choice and design of the interventions. 6.3. Behavioral Interventions Design In light of the motivation and goals described above, the team devised two behavioral interventions that could be tested in two different states in Nigeria. The first would be a low-cost “nudge,” where the incentive for good performance would be purely social recognition, the second would be raffle tickets for a material prize lottery. These two interventions are described in more detail below. The sample within which this experiment took place was the same as the continuous PETS and RT sample of 140 facilities across Ekiti and Niger states, described in more detail in Chapter 3. This relatively modest sample size led to a key design decision. We ruled out differing programs by state, as comparability of interventions across states was seen as paramount and was highly desired by government partners. Among the remaining possible options: (1) we could have chosen a three-arm study, in which each of the two interventions could be simultaneously compared with each other and with a comparison group, or (2) a sequential two-arm study, in which one intervention would be measured against the comparison group, and after a certain number of weeks, the comparison group would begin the second intervention. Each of these options was accompanied by potential risks. The simultaneous three-arm study option 1 would have substantially lower statistical power, while in the sequential two-arm study option 2, the evaluation of the second intervention would depend on the patterns shown during the first intervention. If the first intervention did nothing, the second could be interpreted as a simple RCT. However, if the first intervention had an effect, the second would be measurable only in a difference-in-difference framework, which would in turn depend on the defensibility of the common trend assumption given our data. Acknowledging these risks, the team opted for the latter design: a sequential two-arm study. From January to April, 2016, the team tested the two interventions in sequential RCTs. The first (Social Recognition) intervention was tested in a straightforward RCT, while the second (Lottery) was tested in order to maximize the possibility of additional conclusions above and beyond the initial RCT, given the logistical and sample size constraints. The study leveraged the ongoing PETS and RT study of 140 facilities across the two states. Accordingly, we randomly assigned the 140 facilities to two arms per state for the purposes of this RCT, to find out whether either of two motivation programs could improve record-keeping. To maximize the study’s statistical power, we assigned approximately half of the facilities to each arm in each state (see Table 1). To ensure that geography would not confound analysis, we stratified this randomization by local government authorities (LGA), meaning that we randomized the facilities to arms within LGA. The “arms” in question were variations in the incentive design based on scoring the facilities’ record- keeping qualities. The record in question was the cash book Form C, a simple excel balance sheet (designed as part of the PETS RT Study) that tracked the cash receipts from user charges as well as expenditures, by category, on a daily basis. The quality of those records could be assessed by enumerators who visited the facilities on a weekly basis. The precise design of the evaluation is described below and is shown in Table 1 following. Part I of RCT: for the first four weeks of the incentives study, the two study arms being compared were 95 A “Comparison”: Records were scored weekly by enumerators; however, scores were not shared with facility administrators. B “Social Recognition”: Records were scored weekly by enumerators; scores were then converted to a number of stars between 0 and 5. The number of stars was displayed in a public place on a Certificate of Excellence (for a picture of the certificate, see Appendix Figure A1) for anyone visiting the facility to see. Furthermore, at the end of the experimental period the best-performing facility and its accounting staff (i.e. all that participated in completing PETS RT forms) were commended and posed for photographs and an honorable handshake with the Permanent Secretary of Health in a special ceremony. Part II of RCT: for the fifth through eighth weeks of the incentives study, the two study arms being compared were C “Lottery”: Records were scored weekly by enumerators; scores were then converted to a number of tickets toward a lottery between 0 and 5. Each ticket increased the probability that the each of the accounting staff at that facility (i.e. all that participated in completing PETS RT forms) would win a mobile smartphone prize in a random draw at the end of the experimental period. The draw was done during the Social Recognition ceremony, with the actual phones delivered to the staff of the winning facility at a separate, later ceremony. B “Social Recognition”: as above Table 6.1. Sequential 2-Arm Study Design by Week Week Week Week Week Week Week Week Week Week 0 1 2 3 4 5 6 7 8 1st half of facilities in Baseline Social Recognition Social Recognition each state 2nd half of facilities in Baseline Comparison Lottery each state Thus, at the fifth week of the study, those in the Comparison group began to receive the Lottery incentive. The randomized design permits us to attribute any differences in performance over the first four weeks (post-baseline) to the Social Recognition scheme. During the fifth through eighth weeks, if the performance levels in the comparison group had not risen too much, and the common trends assumption (or another identification strategy) were clearly borne out by the first four weeks of post-baseline data, the same randomized design would permit us to attribute differences in performance over the fifth through eighth week to the relative impacts of the Lottery and Social Recognition scheme. 6.4. Data To measure performance and quality of work by health facility staff, we gathered a weekly dataset on the quality of record-keeping. We focused on the quality of records in one of the key forms introduced in 2015, called “Form C,” which recorded cash transactions (cash receipts from user charges and expenditures) at the facility level. The “checklist,” which assesses the quality of the records, is based on an introductory question, two auxiliary questions, and ten key scoring questions, shown below: 96 Introductory question 1. Is the Form C kept in the facility? Auxiliary questions 1. Number of days in this week the facility is open 2. Are all entries on Form C zero? Main questions: 1. Prior to arrival: Had the facility staff completely filled the Form C before you arrived? 2. Treasurer/Officer-In-Charge (OIC) check: Did Facility OIC/Treasurer check Form C for inaccuracies and incompleteness? 3-7: Completeness on each day. For how many days… 3. …is the 'Balance forward' section of Form C complete? 4. …is the 'Cash receipts from user charges' section of Form C complete? 5. …is the 'Cash receipts from drug sales' section of Form C complete? 6. …is the 'Expenditure' section of Form C complete? 7. …is the 'Expenditure from drug sales' section of Form C complete? 8. Consistency Cross-Check: Are the drug purchases and sales records appearing on Form C consistent with the drug purchases and sales records appearing on 2? 9-10. Documentation. Can the staff provide receipts/invoices or other documentation…? 9. …to substantiate the data for cash receipts from user charges? 10. …to substantiate the data for cash receipts from expenditure? Based on the answers to these questions, the checklist produces a weekly score for each facility. Questions 1, 2, and 8, above, are scored on a binary basis: the answer is either “yes,” for a score of 1, or “no,” for a score of zero, each week. Questions 3-7 are scored as fractions from zero to one, depending on the number of days that various conditions are satisfied as a fraction of the days that the facility is open. Questions 9 and 10 are scored zero when there is no documentation; one when all relevant transactions have documentation; and 0.5 when only some of the transactions have documentation. Note that question 8 pertains to “Form B2” (see Chapter 2 for a detailed description of Form B2), which records dates and quantities for any drug shipments that the facility receives, whether locally purchased or regionally distributed; this allows an enumerator to check Forms B2 and C for consistency on at least the arrival of drugs in the facility on a given day. 6.5. Implementation Details The implementation of the incentive interventions was undertaken between end-February and mid-April 2016. Enumerators already making weekly visits to facilities for the larger RT Study were trained on the different arms of the incentive intervention. Enumerators on average visited 4-5 facilities and these could include both sample facilities from the control group (in later weeks, the Lottery group) as well as the Social Recognition group. The treatment for facilities in the Lottery group began only halfway into the incentive period: from Weeks 1-4, their performance was assessed and scored by enumerators, but those scores were not revealed to facility staff (“Comparison” treatment). Scores were shared from week 5 onward, in addition to presenting facility staff with a certain number of lottery ‘tickets’ each week that corresponded to their scores for the week. The assumption underlying this treatment was that each of the staff would be motivated to work better at filling out the cash book Form C because of the prospect of winning a grand prize – a smartphone for each staff member of the winning facility within a state – at the end of the intervention period. Central to the design was the notion that each facility could improve its chances of winning the grand prize by 97 accumulating as many tickets as possible over the period (a maximum of 20 over 4 weeks; quality- contingent financial reward). Facilities in the Social Recognition group, on the other hand, received weekly certificates from the beginning of the incentive period, with star ratings that corresponded to their scores for each week. Importantly, staff were expected to display the certificate for each week in a prominent spot at the facility, so that their progress (or the lack of it) would be visible to all. The assumption here was that staff would be motivated to work better at filling out the cash book Form C because of the desire to earn stronger community approbation (quality-contingent social reward). A key element of the incentive design was the expectation that community members would engage with the certificates to a degree sufficient to provide feedback (mainly in the form of recognition) to facility staff, although the intervention did not directly communicate with community members. After the completion of the RCT, using a purposively selected sample of 17 facilities across Ekiti and Niger, the qualitative component of the study assessed the implementation of the Lottery and Social Recognition schemes, with attention to contextual features in each state that might help to explain particular outcomes. Fieldwork was conducted over a period of two consecutive weeks, first in Ekiti, and then in Niger. This followed on from two weeks of fieldwork visiting fewer facilities to better understand the local service delivery context in each state undertaken in August-September 2015 in order to help with the design of the interventions. The in-depth nature of the qualitative inquiry on understanding the effect of the interventions made it both impractical and undesirable to involve all 140 facilities in the larger RT Study sample; hence it was expedient to select a few cases with the aim of arriving at a sample that could be considered representative in light of the key variables that the study was concerned with. Three variables (DRF or “drug -revolving fund” status41, type of incentive intervention, and direction/degree of change in resource tracking behavior) were eventually decided upon, resulting in the sample outlined in Table 6.2. 41 The introduction of the Drug Revolving Fund (DRF) in Ekiti state in the early 2000s marked a departure from the free health system that had operated in the state up until then. Ushered in on the platform of the Partnership for Transforming Health Systems programme funded by by the UK’s Department for International Development and run in collaboration with the State Ministry of Health, the Fund was instituted with clear operational guidelines laid out in a ‘procedures guide’ document made available to stakeholders. 98 Table 6.2. Selection criteria for facilities involved in qualitative study in Ekiti and Niger Selection criteria Ekiti Niger 1. Non-DRF, Recognition, Most improved Erekesan CHC PHC Tungan Madaki after 8 weeks 2. Non-DRF, Recognition, Least improved Iloro Health Centre No easily accessible facility after 8 weeks in this category* 3. DRF, Recognition, Most improved after Model Health Centre Ilogbo PHC Barkuta (also the social 8 weeks recognition winner) 4. DRF, Recognition, Least improved after Irapa Health Centre Zollegi Health Post** 8 weeks 5. Non-DRF, Lottery, Sharpest change Igbo-Olofin Health Centre PHC Shakwata between Weeks 4 and 8 6. Non-DRF, Lottery, Smallest change Idemo Health Post PHC Sabon Gari between Weeks 4 and 8 7. DRF, Lottery, Sharpest change between Oke-Oja BHC BHC Tegina Weeks 4 and 8 8. DRF, Lottery, Smallest change between Surulere BHC PHC Lambata Weeks 4 and 8 9. Lottery-winning facility Osin BHC PHCC Chukuba *This is the case going by the official list of DRF and non-DRF facilities consulted prior to fieldwork; however, fieldwork revealed the counterpart DRF facility (Zollegi Health Post) to be, in reality, a non- DRF facility, making it more suited to this second category than the fourth. **Refer to above explanation. It is important to note a few caveats in the case of Niger. While it was methodologically and practically feasible to select a clear ‘winner’ in each category in the case of Ekiti, the much larger geographical area in Niger and restrictions on the time available for fieldwork made it expedient to overlay the pre-determined selection criteria with the practical considerations of distance (from the capital) and accessibility. Consequently, the Niger sample was made up by selecting reasonably accessible locations from a range of facilities in each category that had recorded either relatively large or relatively small (or negative) improvements in resource tracking performance over the intervention period.42 The priority in both states was to interview the member of staff responsible for completing the RT intervention forms in each facility. This was usually the officer-in-charge (OIC), especially as they were the ones who were required to attend the PETS training at the start of the intervention. However, especially in the Ekiti DRF facilities, responsibility for filling the intervention forms was sometimes ceded to the dedicated ‘DRF Officer’ in a facility, in the process leveraging pre-existing institutional capacity. Questions were designed to tease out staff capacity, sources of motivation for staff at various levels (including the 42 The final round of selection was done in conjunction with the Niger State Team Leader (STL) from the firm implementing the RT Study, who has a very good overview of the location and performance of the programme facilities. Access to the OICs in the first instance was negotiated by the STL, in Niger as well as in Ekiti. 99 incentives introduced by the intervention), as well as leadership and group characteristics that might have contributed to shaping the outcomes of the intervention in each facility (see Annex 5.10.2. for the complete question guide). Wherever practical, the chairman of the facility health committee was also interviewed, to gain insight into the implementation and effects of the intervention from the perspective of community members with a stake in the facility’s performance (see Annex 5.10.2. also). The sample included committee chairmen for only four facilities in Ekiti – Iloro, Irapa, Idemo, and Osin – partly because availability was limited for the group, especially in Niger where the role of the equivalent village development committees in facility administration (particularly drug accounting) is more variable than in Ekiti. Lastly, community members/clients were interviewed in a selection of Social Recognition facilities (two in Iloro, two in Tungan Madaki, and one in Barkuta), primarily to assess the extent to which they understood and engaged with the intervention. It was more difficult to locate clients for interview in the other Social Recognition facilities due to (sometimes acute) low patronage levels in those facilities. In addition, to gain a good understanding of the context of the intervention, key informant interviews were conducted with the Project Field Coordinator and the state team leaders of the implementing firm, as well as supervisors and enumerators in Ekiti and Niger (see Annex 5.10.2. for the list of enumerator prompts). 6.6. Results from the RCT (treated as two independent experiments) 6.6.1. Summary Social recognition significantly improved resource tracking in comparison to our control condition in a 4- weeks experiment in Ekiti state, but did not have the same effect in Niger (see Figure 2). A number of institutional and demographic differences between the two states could be responsible for the differential efficacy of the intervention. Due to our sequentail 2-arm study design and the trends of our results during the social recognition intervention in weeks 1-4, we are not able draw conclusions about the effectivenss of the lottery intervention in comparison to the social recognition intervention in weeks 5-8 in Ekiti. In Niger, we continued to not have an effect (i.e. there was no significant difference between the effectivenss of our lottery incentive and the effectiveness of our social recognition incentive). In what follows we present the results in more detail. Figure 6.2. Average Social Recognition Effects in Weeks 1-4 in Ekiti and Niger 100 6.6.2. Part I of RCT: Social Recognition (vs. Comparison) We begin with results from the first part of the RCT, reporting separately in Ekiti and Niger. Conditional on having any non-zero transactions (one of the auxiliary questions on the checklist), we compare the checklist scores in the Social Recognition arm to those in the Comparison arm during Weeks 1 – 4. The results are displayed in Table 6.3. Table 6.3. Performance Scores by State and Treatment Arm in Part I of the RCT State : Ekiti Niger Outcome : Score Pct Score Pct Social vs Comparison 0.135*** -0.0230 (0.0494) (0.0321) Constant 0.467*** 0.662*** (0.133) (0.0460) Observations 182 277 Weeks 1-4 1-4 Note: Standard errors in parentheses; * p<0.10, ** p<0.05, *** p<0.01 As shown in the table, the Social Recognition of having the checklist performance posted in public on a weekly basis improved the checklist score by just over 13.5 percentage points in Ekiti State: a roughly 29- percent increase over baseline. In Niger State, there was no detectable effect of the Social Recognition scheme. The outcome in the table above is a scoring scale that runs from a minimum percentage score of zero to a maximum of one. While the “score” on the checklist is the basis for the social incentive, it may not translate or compare meaningfully to other contexts. The impact in Ekiti can be broken down into effects on each of the ten key components of the score, as shown in Table 6.4 and Figure 6.3 below. 101 Table 6.4. Results in Ekiti for Specific Questions in Part I of the RCT Social Recognition Effect Constant Obs Coeff. SE Coeff. SE . Had facility staff completely filled Form C (1) before you arrived? 0.205** (0.0972) 0.149 (0.250) 182 Did OIC/Treasurer check Form C accuracy 0.262** (2) and completeness? * (0.0872) 0.0468 (0.233) 182 0.200** (3) Fraction of days complete: Balance Forward * (0.0542) 0.325** (0.151) 182 Fraction of days complete: Cash Receipts (4) from User Charges 0.182** (0.0711) 0.329* (0.194) 182 Fraction of days complete: Cash Receipts (5) from Drug Sales 0.186** (0.0703) 0.324 (0.194) 182 0.202** (6) Fraction of days complete: Expenditure * (0.0609) 0.268 (0.161) 182 Fraction of days complete: Expenditure from (7) Drug Sales 0.203** (0.0781) 0.227 (0.208) 182 Drug purchases and sales: Form C consistent 0.829** (0.0964 (8) with Form B2 0.0533 (0.0332) * ) 182 Documentation substantiates cash receipts 0.686** (9) from user charges 0.0709 (0.0750) * (0.194) 182 (10 Documentation substantiates cash receipts 0.524** ) from expenditure 0.0876 (0.0723) * (0.188) 182 102 Figure 6.3. Results in Ekiti for Specific Questions in Part I of the RCT As shown above, Social Recognition motivated staff in Ekiti to complete all sections of the forms, to do so prior to enumerator arrival, and to check the forms for their accuracy and completeness. These changes on the first seven checklist items are all near 20 percentage points, which in some cases is a very large fraction of the Comparison group value: almost no Comparison group facilities, for example, had the form checked by the treasurer or officer-in-charge prior to the enumerator’s weekly visit. The Social Recognition, however effective at encouraging staff to complete Form C, did not change the likelihood that documentation was available to substantiate the sections of the form that pertained to cash receipts. It also did not change the likelihood that the different forms relating to drug acquisition were in agreement with one another. 6.6.3. Part II of RCT: Lottery (vs. Social Recognition) Next, we examine the difference between the Social Recognition group and the Lottery group during Weeks 5 – 8, the second part of the RCT. Table 6.5 below shows the main analysis. 103 Table 6.5. Performance Scores by State and Treatment Arm in Part II of the RCT State : Ekiti Niger Outcome : Score Pct Score Pct Lottery vs Social 0.0171 -0.0241 (0.0252) (0.0394) Constant 0.900*** 0.719*** (0.0665) (0.0581) Observations 202 272 Weeks 5-8 5-8 Standard errors in parentheses * p<0.10, ** p<0.05, *** p<0.01 The analysis shows that there is no statistically significant difference between the Lottery and Social Recognition interventions in either state. In both states, this could be either because the Comparison group in weeks 1-4 was slowly catching up to the Social Recognition group anyway, or it could be because the Lottery treatment has the same effect as the Social Recognition intervention originally did. In Niger state, because the Social Recognition arm initially had no effect, we can conclude that neither intervention had an effect. In Ekiti state, because the Social Recognition arm had an effect, the interpretation depends on controlling for trends in scores at the end of the first experiment. A visualization of those trends in both states is provided in Figures 4a and 4b below. 104 Figures 6.4a and 6.4b. Performance Scores in Ekiti and Niger, respectively, throughout the period of the intervention Although, in Ekiti, the trends in the two experimental groups were parallel between weeks 3 and 4, the trend cannot continue for the Social Recognition group because it is already so close to the maximum score (ceiling effect). The week 5 score for the Lottery group appears to be very close to what the existing trend would predict, but it is above a longer-term trend. So, while we cannot rule out the possibility that there was an effect of the Lottery intervention (ignoring trends), we also cannot rule out the possibility that the Comparison group was slowly improving scores and would have performed exactly as well as we observe in the data, even in the absence of the intervention. 105 In essence, the conditions for the second experiment to be easily interpretable were not met: because one of the groups was so close to the maximum score at the end of the fourth week, we are unable to comprehensively account for trends in both groups. 6.6.4. Analytical note on the Data The analysis shown here is conditional on non-zero transaction data : that is, that there is any information to record on Form C, and thus for which the checklist can assess completeness and accuracy. Any changes in the rates of non-zero transaction data that are induced by the intervention could pose a potential threat to validity. Below, we test for changes in the rates of non-zero transaction data on Form C, in both experiments and both states, and find no significant difference between experimental study arms. Tables 6.6a and 6.6b. Likelihood of Any Transactions by State and Treatment Arm, in Parts I and II of the RCT respectively State: Ekiti Niger Outcome: Non-zero Non-zero Social vs Comparison -0.0182 -0.0124 (0.0924) (0.0461) Constant 0.745*** 0.939*** (0.233) (0.0712) Observations 253 290 Weeks 1-4 1-4 Standard errors in parentheses * p<0.10, ** p<0.05, *** p<0.01 State: Ekiti Niger Outcome: Non-zero Non-zero Lottery vs Social 0.0711 0.0313 (0.0916) (0.0533) Constant 0.596** 0.851*** (0.241) (0.0889) Observations 256 275 Weeks 5-8 5-8 Standard errors in parentheses * p<0.10, ** p<0.05, *** p<0.01 106 6.7. Insights from the Post-RCT Qualitative Study This section summarizes the key findings and explanations for reading the data results, undertaken as outlined above through field level interviews at a select sample of facilities in both Ekiti and Niger. The analysis suggests there are three overarching factors driving the facility level results in response to the incentive interventions: (a) the (in)effectiveness of the same incentive in Niger or Ekiti is driven by the provincial-level and local-level institutional and governance context; (b) incentives can be especially powerful behavior change mechanisms when they provide regular performance feedback to facility staff, with the recognition intervention scoring higher than the lottery intervention on this count; and (c) facility- level factors can further influence the degree of effectiveness of incentive interventions in practice. This section is organized around these major findings. 1. The governance and institutional context determines the effectiveness of the same incentive interventions There are four aspects of the broader health system context in Ekiti and Niger states that were found to have a bearing on the effectiveness of the resource tracking and incentive interventions, namely: the strength of the facility-government compact in each state and its impact on accountability at facility level; the main source of financing (i.e., public or private) for drug procurement in each state; the nature and strength of the facility-community engagement in each state; and human resource management practices at state and local government level. If the institutional arrangements found in Ekiti and Niger for primary health care are mapped to an ‘accountability triangle’ framework for service delivery popularized by the World Development Report 2004, each state therefore had a different state-provider compact (in levels of supervision, financing, mechanisms in use for drug procurement) and the nature of citizen engagement at facility level was also different (see Figure 5). In neither state were health services a key political priority with strong citizen ‘voice’, but in Ekiti much more so than Niger, legitimacy of the state government was much stronger around health care. These factors are examined in turn below. Figure 6.5. Some key attributes and relationships for primary health care service delivery Source : Adapted from World Development Report, 2004 107 One of the overall findings from the qualitative study is that Ekiti and Niger have broad differences in the way that drugs, the central resource in primary health care facilities, are administered. Since the early 2000s when the DRF was introduced to Ekiti by a donor program, it has been the default drug procurement scheme in the state.43 The DRF has proven to be so self-sustaining that frontline health workers in the state actually consider it superior to the Free Health system introduced more recently, mainly because the fee-paying structure of the former encourages a steady supply of drugs, and the public procurement system employed actually ensures lower prices than those offered by private suppliers. The reverse is the case in Niger, where the few facilities running the state’s limited version of the DRF hinge the viability of their op erations on the lower-cost, higher-margin private drugs often sourced from local patent medicine vendors. The element of DRF implementation and supervision in Ekiti therefore makes for a better regulated primary health care environment in the state relative to Niger. The fact that reporting requirements for the Free Health program (which is limited to a handful of facilities per LGA) were modelled after the DRF illustrates the entrenched nature of the DRF in the state and the continued exercise of government oversight even in facilities that may have temporarily lost their DRF status. Outside of drug procurement, however, both states have similarly low levels of government investment in facility-level operations. Common issues that staff resolve on their own, often without financial or other support from the local or state government, include building repairs, security provision, and bush clearing around the facility (mostly an issue in Ekiti). Especially in Niger, support for these basic services is usually more forthcoming from community members, mobilized by the village development committees (or facility health committees in the case of Ekiti) – though the extent to which this occurs varies across communities. This variability (coupled with the drug procurement challenges peculiar to Niger) has resulted in a fragmented primary health care system in both states, with quality of care depending more on the characteristics of individual communities and facility staff than the application of any set of standards stipulated by government authorities. 1a. Supervision from local or state authorities is different in both states and has effect on RT and incentive interventions In Ekiti, there is a strong sense of responsibility among facility staff towards the local and state government, even if there remains ample room for improvement in the area of oversight by relevant government departments. Indeed, despite the reality of infrequent and unpredictable supervision visits by state and local government (LG) officials in Ekiti, anticipation of those visits plays a significant role in motivating facility staff to keep records of all kinds, signaling that even the prospect of supervision may be sufficient to keep facility staff on their toes. There is evidence that this fear-induced legitimacy went some way in encouraging staff compliance with the requirements of the intervention, even before the introduction of the incentives: some of the best- performing staff spoke of being exceptionally wary of slip-ups during the intervention due to a heightened awareness of government surveillance. In Niger, supervision visits by government authorities to facilities are far less frequent and less substantive than in Ekiti, indicating a detachment of local and state government authorities from their monitoring obligations in particular and a weak facility-government compact in general. In interviews conducted for 43From its inception, the DRF was designed to be a ‘unified’ scheme – that is, to ultimately operate in all primary (and secondary) health care facilities in the state, and to enable full recovery of the cost of the drugs injected into the facilities by the Fund. The first batch of drugs transferred from the Central Drug Store to each facility was free, but subsequent purchases were expected to be funded by proceeds from successive quarterly cycles. Host communities were central to the program design from the start: though seeded with state funds, the drugs in each facility were explicitly designated as community property, and a rather sophisticated local governance arrangement was put in place to reflect this. The governance structure consisted of ‘facility health committees’ comprising facility staff and community members, with stipulations for membership that were intended to facilitate equitable representation of relevant demographics in each community 108 this study, the relationship between local governments and their facilities is characterized by apathy on the part of the former and ambivalence on the part of the latter. The nature of this tacit compact between the two is perhaps illustrated most vividly by the arms-length approach employed by local government authorities in overseeing the state’s DRF program for drugs procurement. Such weak engagement, including any financial or in-kind transfers to facilities beyond salaries, essentially incentivizes facilities to set up coping mechanisms outside of the formal system. Conversely, facilities do the bare minimum required of them by their local governments and no more, propelled by a mix of the tacit knowledge that the latter will likely not ask questions and an unspoken fear among staff that they might be implicated by written records of their private drug purchases. Whereas OICs in Ekiti defer to government policy and provision even when it does not seem to be forthcoming, OICs in Niger subvert formal processes and work out their own way of doing things for which they feel justified and do not consider themselves accountable to the government. OIC and key informant responses from the state indicate that this lack of accountability of facility staff to government authorities constitutes a major disincentive for the former to engage in reporting and in resource tracking activities. 1b. A functional public drug procurement mechanism used by facilities in Ekiti, unlike in Niger, has corresponding implications on the relevance to track resources for facility staff, which made the incentive interventions likewise more relevant in one state than the other The facilities in the Ekiti sample have a relatively limited and predictable range of options for sourcing their drugs. The main sources are a largely functional DRF system.44 An ailing ‘Free Health’ program was additionally introduced by a previous government in selected facilities.45 Notably, the practice of dealing in private drugs occurs to a very limited degree in Ekiti, at least partly because government oversight of the official drug procurement systems is very strong and prohibitive of ‘sharp practices’. (On the demand side, low patient traffic in Ekiti also likely contributes to the inertia of staff to engage in private drug enterprise – the next section discusses this in more detail.) The considerable influence of government authority in the state extends beyond DRF facilities: even in Free Health centers that have been plagued by acute drug shortages since the current state government administration took over, staff are mostly making do without drug stocks and await government orders to revert to the DRF rather than take matters into their own hands. The DRF in Niger, on the other hand, is a caricature of the original design, and lags far behind Ekiti in its degree of government regulation and standardization. Weak state control has led to variations within the scheme and outside of it, with many so-called DRF facilities needing to supplement their official drug supplies (and incomes) with (often cheaper) drugs from private sources. Many OICs have struck up rather sophisticated arrangements with private suppliers, including the establishment of credit lines and ‘home service’ agreements that enable them to operate more easily within the financial and geographical constraints they face. (Again, as in Ekiti, exceptions can exist in facilities where community patronage is so low as to make private drug enterprise non-viable.) In the absence of strict state monitoring systems in the state, facility staff arbitrarily devise their own set of operating rules: they unilaterally decide, for instance, the profit margins on DRF supplies received on credit from the LG; when and how to pay back the principal to the LG; whether or not to lodge the monies realized in dedicated bank accounts; and how 44 At state level, procedures for price setting and profit calculation (to ensure a stream of petty cash for facilities) were very important components of the DRF scheme. Sound accounting and resource tracking provisions were also built into the scheme, promoting accuracy and transparency of record keeping at facility level, though in practice facilities tend to focus on a subset of the RT toolkit introduced by the program. Generally, the provisions in the guide document are not fully implemented in practice – there is, for example, a provision for waiving or deferring treatment charges for indigent clients that is rarely applied in practice. Nonetheless, the implementation of the Fund is substantive enough that many of the principles underlying its design are still apparent. 45 A third official drug procurement system, the National Health Insurance scheme, exists in Ekiti, but none of the facilities in the sample is involved in the scheme. A few facilities, however, fall outside the purview of these main systems most often due to unresolved malfunctions in their DRF accounts. Although such facilities remain classed as DRF centers in official record, in reality they do not have a functioning DRF. 109 to keep records of all of these transactions. Local governments seldom ask any questions of facilities other than that the latter periodically remit the cost of the ‘DRF drugs’ given to them on credit, not minding that these sums are sometimes made up by the sale of cheaper drugs from private sources. This pattern is also evident in quantitative data. See Chapter XX, Section X, on the nature of the drug purchases recorded on the newly introduced forms, and how they vary systematically between Ekiti and Niger. In short, government oversight that is more likely on functional DRF facilities holds them to a minimum standard of record keeping that make both the RT interventions and the incentives relevant to facility staff, even if those records are seldom checked by government authorities. Instructively, the cases in which staff were most reluctant to engage with the intervention were in facilities where drugs are wholly purchased from private sources and there is no government input, either in terms of funding or oversight. This was the case irrespective of state, and regardless of type of incentive applied. 1c. A preponderance of private drug enterprise and a credit regime in Niger create conditions for facility-community engagement that can enable responsiveness and a degree of accountability at community level but do not support RT objectives at government level. In general, community patronage of Niger facilities is higher than that in Ekiti. Several factors seem to contribute to the relative advantage of Niger in this regard. Given the geographical spread of the state, facilities are often remote and far from any major source of competition, sometimes including patent medicine vendors. More importantly, however, remote facilities in the state most often began life several decades ago as community self-help projects. Even where such facilities have since been rebuilt by the government, the legacy of ownership is still apparent in the communities. (The self-help inclination of many of the Niger facilities may partly explain their apparent disconnection from government provisioning and oversight). The issue of community patronage is closely linked to drug resourcing patterns: in one exceptional case in Niger, and in several facilities in Ekiti, the incentive for staff to stock privately purchased drugs has been stymied by perennially low patient traffic, even when no alternative drug procurement pathways exist for the facility. Extremely low levels of client patronage often act as a disincentive to stock drugs in the facilities, much less track their use. This link between patronage levels and drug resourcing partly helps to explain why privately funded facilities in the Niger sample were more committed to keeping records, at least pre-intervention, than their counterparts (i.e. similarly funded facilities) in Ekiti. The most significant driver of relatively high community patronage in Niger however seems to be the willingness and ability of many facilities in the sample to accommodate clients’ demand for credit – a demand-side feature which is conspicuously absent in Ekiti. OICs in Niger make this allowance even though they know from experience that there will sometimes be cases of overly delayed payment and even eventual default. The complication introduced by the prevalent credit regime in the state makes official RT practices seem more elusive to the OICs, many of whom are content to just keep facility finances afloat even if they do not know the specifics of their cash flow or profit margins. However, it is the very dearth of official oversight and sanctioning that enables the facilities to be flexible enough to adapt to the expressed needs of community members at the local level. For example, many OICs in the state are able to accommodate the credit needs of community members while meeting their repayment obligations to the LG because they independently determine their (often low-cost) sources and (relatively high) profit margins without interference from the government. The RT tools of ‘Form C’ and the incentive intervention, therefore, found traction in Niger facilities, where the OIC was less concerned in reporting to state/local authorities, but instead more inclined to improve data around the credit owed by patients or to further the existing link with the community through the social recognition incentive intervention. This is unlike the case in Ekiti, where the high degree of legitimacy 110 ascribed to the state by facility staff is a key driver for interest in RT tools, which is largely independent of the nature of facility-community relations, and vice versa. 1d. Current HR management practices of regular transfers in Ekiti, unlike in Niger, can incentivize facility OICs to improve accountability on resource tracking, including engaging more with the related incentive interventions In Ekiti, frequent, seemingly arbitrary staff transfers create conditions where incoming OICs are reluctant to take over from outgoing ones until they are certain that the DRF accounts they are about to inherit are in order. OICs in Ekiti facilities, unlike those in Niger, expressed that their motivation to adopt RT practices were in order to ‘insure’ their own reputations, and protect themselves from the acts of previous OICs with regard to DRF accounts. Since these transfers, however, sometimes take place without a process of official handover from old to new staff, a major reason why some officially designated DRF centers no longer run the program is also due to the strict accountability requirements of the DRF. Indeed, the only two facilities in the Ekiti sample that have private drug purchase arrangements are actually designated DRF centers that have had problems with the transfer of accounts and records from one set of workers to the next. In one case, all six workers in the facility had been newly transferred to the station just before the start of the intervention, allowing no form of continuity from the previous set of workers. The situation appears to be very different in Niger, where many of the staff in the sample have been working in the same facility for years, and sometimes decades. Combined with an institutional culture of weak or no supervisions from government authorities and ahigh preponderance of private purchase of drugs that create limited incentives for RT reporting, the fact that many of the OICs in the sample had been in the same facilities for long numbers of years additionally created greater inertia for change, making the incentive interventions less effective in Niger than in Ekiti. 2. The ability of incentives – especially the social recognition intervention – to provide regular feedback on RT performance to facility staff is a key driver of behavior change. 2a. Regular feedback on RT performance gives impetus to facility staff to build their personal capacity on the job, independently of the potential for public recognition. The weekly certificates issued to facilities on the recognition scheme proved particularly capable of providing immediate feedback to staff on their RT performance. The initial response most commonly triggered by the feedback was that staff sought further training, usually from their enumerators, to address their knowledge deficits. The effect on performance was sometimes dramatic: in one case, the facility’s ratings immediately jumped from 1 to 5 stars and stayed there throughout the incentive period. Thus, the certificates succeeded in applying a quick and lasting corrective to whatever knowledge or interest deficits existed for individual staff prior to introduction of the incentive. 2b. The prospect of community (dis)approbation invited by public display of certificates on the recognition scheme gives an added boost to staff motivation, whether or not community members are fully engaged with the intervention. Beyond the level of personal feedback described above, the public component of the recognition scheme, which required facilities to prominently display their weekly certificates, also enabled a more community- oriented interpretation of the feedback received. Indeed, the sense of personal shame experienced by staff with initially low performance seems to have been overwhelmed by the shame they felt at being publicly exposed, especially in centrally located facilities with relatively high patient traffic. It is important to note that this sense of public shame was independent of the degree to which community members actually took interest in the certificates – in at least one facility, it was evident that community engagement with the 111 particulars of the scheme was very limited in reality. This indicates that the public display aspect of the certificate scheme did play a significant role in challenging staff to work harder, and is quite distinct from the aspect of personal improvement reflected by improved scores. It also indicates a certain robustness to the behavior change assumptions underlying the design of the recognition scheme, given that those assumptions seemed to inform change in practice even though there was very little engagement with, or feedback from, the community in reality. 2c. The social recognition incentive intervention far outperformed the lottery intervention in its capacity to provide feedback on RT performance and progress to facility staff. The capacity of the lottery scheme to provide continuous feedback to facility staff is directly linked to the degree to which those staff understood the design of the incentive. As there was a lot of variation in levels of understanding of the incentive design, there was a corresponding variation in the effectiveness of the lottery scheme as a feedback mechanism. The strongest example of the operation of a feedback mechanism within the lottery group was found in a facility in Niger where the OIC has been running the private equivalent of a drug revolving fund for over a decade. This OIC seemed to value the feedback he received (and the resulting improvements in performance) not just for the sake of the incentive or the intervention, but for the transparency it could bring to his more rudimentary pre-intervention RT practices. The recognition scheme, on the other hand, exhibited a high self-correcting tendency and a high potential to draw on the desire for public approbation to drive sustained engagement with the incentive and the explicit feedback mechanism it activates. It is particularly instructive that most staff responded enthusiastically to the tacit logic of social recognition even when they had initially been convinced about their desire for material reward at the start of the intervention. In Niger particularly, most of the staff in the recognition facilities seemed appreciative of the incremental learning process they went through during the intervention and claimed that the prospect of social recognition was just a bonus on top of the superior knowledge and experience they had gained. Nonetheless, the deliberate manner in which those staff tracked their progress and shared it with community members suggests that they also placed a high value on the recognition element, again highlighting the robustness of the premise of the intervention design. 3. Facility-level factors can further influence the degree of effectiveness of incentive interventions in practice 3a. The OIC’s role could have a significant influence on the implementation and outcomes of the intervention, regardless of human resource capacity at facility level and type of incentive applied Staff numbers vary widely across the facilities in both states, but the facilities in Ekiti generally have higher staff strengths: one of the better-staffed facilities in the state had 32 workers, for instance, while the highest- staffed facility in Niger had only a quarter of that number. Levels of qualification, especially of OICs, also tended to be higher in Ekiti, and it was more common to find junior staff in specialized roles such as Medical Records Officer and Dental Technician. The implications for RT are evident: reasonable staff strengths can be an enabling factor for improving RT practices in a facility, as the ability of OICs to delegate or coordinate responsibilities in larger facilities apparently gives a boost to RT in those centers. Conversely, examples from smaller facilities suggest that extremely low staff strengths can pose significant challenges to daily resource tracking, as the few (or in some cases, lone) staff available may not even attend work daily in the first place. Nonetheless, there are other facilities, especially in Niger, with extremely low staff strengths but extremely high leadership commitment, in which OICs effectively combine their RT responsibilities with many other functions – sometimes with very little help from junior staff. This indicates that, while staffing and other supply-side deficits can pose a real challenge to RT operations, OIC leadership can compensate for these deficits to a considerable extent. 112 Within the lottery group, the highest-performing facilities mostly had OICs who took responsibility for leading or delegating RT responsibilities during the intervention. This was complemented in some cases by the ability of lower-level staff to take the initiative on RT tasks. This was especially true for Ekiti DRF facilities where lower-level staff had existing responsibilities to manage the DRF records. In such cases, it was lower-level staff who tended to support OICs in meeting the RT requirements of the intervention, rather than the other way around. Unsurprisingly, high levels of pre-intervention OIC leadership correspond to greater ability to rally their team around the intervention, with extremely positive results. Facilities in this group demonstrate that high levels of leadership can give a boost to teamwork, which can compensate for low levels of individual initiative or willingness to engage with the intervention. When the OIC was interested in winning the lottery prize, there were instances also of the OIC personally mobilizing facility staff, since the prize of the smartphone was given to each individual engaged in RT tasks. A comparable degree of OIC leadership was also found in high-performing facilities within the recognition group. Here again, an appreciable degree of initiative was evident among some lower-level staff; however, tactful intervention by the OIC was still necessary to encourage those staff to follow through on the improvements they had determined to make based on feedback received from their weekly certificates. In such cases, it was enough for OICs to rally behind the resolve of lower-level staff for the desired changes to be implemented. Importantly, this more subtle form of leadership is no less effective in its ability to galvanize team action towards the common goal of improved RT performance. From the perspective of one OIC, this task is made easier by the built-in capacity of the recognition incentive to emphasize collective responsibility over individual reward. 3b. The way incentive interventions are communicated and interpreted is fundamental to their effectiveness in driving behavior change, over and above the educational level and technical capacity of staff Despite receiving similar levels of technical training and enumerator instruction over the period of the intervention, there were variations in staff understanding of the intervention and incentives across the facilities in the sample, which also reflect variations in how effectively enumerators communicated to facility staff. Two examples from Ekiti illustrate the possible role of education in determining message communication and reception. In one facility, the OIC, a Community Health Officer with a university diploma, understood that her facility had been randomly selected to participate in the intervention (as opposed to being singled out for any particular reason) and that what was being measured was not their performance in generating income (which she considered out of her control) but their performance in recording all transactions that occurred, regardless of size. Understanding that the assessment was judgement-free, the OIC was motivated to concentrate on the area of RT targeted by the intervention and not on all other constraints faced at the facility. This contrasts with the experience of another OIC (a health assistant) who, having had exceptionally low levels of understanding of the intervention throughout the period, conflated the narrow aims of the intervention on RT practices with broader facility objectives and took the facility’s consistently low performance as evidence that the assessment was unfair. The case of another facility in Niger demonstrates that when high levels of understanding are complemented by innovative messaging, the impacts on collective RT behavior can be even more dramatic. The OIC in Niger also holds a Diploma in community health, and had clearly and correctly communicated the key messages of the intervention and incentive (the lottery in this case) to his staff. However, what resulted in exceptional motivational levels among staff was the OIC's invocation of team spirit in communicating the stakes of the intervention: work together as a team to score highly and the whole team wins; do the opposite and the whole team loses. The OIC essentially employed peer pressure in motivating his staff: no one wants to be seen as the person that lets down the team. The lesson here, especially when set against the background of the communication failures observed in other facilities in both states, is that the communication or presentation of the key messages of the intervention is at least as important as those messages in themselves. 113 The messaging to community members and their consequent engagement with the intervention was more nebulous, especially on the recognition scheme: the weekly certificates were generally interpreted as a sign that their facility (and, by extension, their community) had been recognized at higher levels, and this conferred a sense of importance on them even if they did not quite know the meaning of the star ratings or how the ratings related to facility performance. In Niger, especially, some OICs took it upon themselves to translate the certificates into terms which, however vague, were easy for community members to relate to. There is some evidence that this OIC-mediated communication was considered more instrumental to community engagement than the actual certificates, given communities were not directly informed by the intervention about the certificates. To further their client recruitment objectives, OICs at a few facilities impressed upon community members that continued receipt of the certificates was dependent upon increased client patronage. OICs who adopted this strategy, not part of the incentive design on the recognition intervention, claimed that it did swell client numbers during the period. Such latitude for reinterpretation of the incentive aims, though unintended, can have beneficial (and arguably more fundamental) impacts beyond the scope of the intervention. Especially with the recognition intervention, even where a strong feedback effect of the certificates on staff performance is not evident, staff may still be able to convert specific aspects of the scheme into broad social capital and, ultimately, greater client engagement, thereby potentially tapping into a more intrinsic incentive for them. 6.8. Conclusion & Learnings for Future Implementation of Incentive Measures In this chapter, we provide quantitative experimental evidence, paired with qualitative descriptive evidence, regarding the effects of two performance-contingent incentives designed to encourage health facilities in Nigeria to improve the quality with which they report their resources. Specifically, we analyze the effects of a public social recognition scheme and a material lottery scheme over a 2-month period in Ekiti and Niger states. As has been shown, despite being part of the same country and despite parallel program implementation, the two states exhibit substantially divergent results. For the quantitative analysis, we randomized 140 facilities into two subgroups, stratified by state. For the first four weeks, one group received public stars depending on their performance, while the other group served as a control and was only monitored. In the second four weeks-period, the latter group switched to a chance of receiving a material reward (a smartphone), while the former group continued with the social reward. The summary of the results is that the social recognition intervention had a substantial and statistically significant positive impact on performance in Ekiti but essentially no effect in Niger. The positive outcome in Ekiti was seen across multiple sub-dimensions, especially regarding completeness of the target form C, albeit with less effect on the existence of substantiating documentation. This is encouraging given the relatively low cost of such an intervention, although due to a lack of information we are not able to estimate any cost-effectiveness. Also, it is important to note that there is a suggestive upward trend in performance for the control group, perhaps due to the mere fact of monitoring, also known as Hawthorne effect. Because we don’t have an unobserved control group we cannot draw any strong conclusions. It is also possible that the social recognition intervention may be significantly more effective than the control group “only” in the first four weeks of the treatment and if we had run the interventions for longer that any differences between our social recognition intervention and control conditions would have disappeared. If that were the case, it could simply be that instead of a true “social recognition” effect, the display of certificates made more salient earlier to facility workers the fact that they were being observed. We do not observe any difference between the lottery and the social recognition in either state, although given the results described above this yields two different interpretations. In Niger, this implies that neither intervention, as implemented, induced a noticeable change in average behavior. In Ekiti, by week 5, most 114 facilities in both groups were already performing fairly well, and their scores appear to have been trending upward at different rates, in part because the Social Recognition arm was already scoring near the maximum. The initially slower but still upward trend in the Comparison group in Ekiti over the first four weeks of the experiment may have been due to spillovers from the Social Recognition arm, or may have been due to a process of learning about how to fill out the forms correctly or as mentioned above, the Hawthorne effect. Whatever the reason, the differential trends prevent us from distinguishing whether the Lottery would have had an impact initially, and if so whether that impact would have been larger or smaller than the impact of the Social Recognition treatment. This limitation is partly due to the relatively small sample size, which drove the design decision that ultimately did not allow a direct concurrent comparison between all treatment arms. The combination of the quantitative and qualitative analyses strongly suggests that contextual factors at state, community and facility level – notably, drug procurement pathways, human resource management, demand-side dynamics, and institutionalized RT practices (whether these are formal or informal) – may play a key role in determining the viability and effectiveness of RT interventions in particular facilities. One key conclusion from the experience of implementing the incentive interventions in Niger and Ekiti is that the same incentive intervention can provide different results in the same country if the institutional context of the service provision is different. Detailed institutional assessments at the sub-national level for PHC services can inform important tweaks in designing incentive interventions to motivate behavior change, even within the uniformity that can be presumed when health services are driven by a single federal ministry of health operating under the same overall structure. The study results and the qualitative analysis suggest that particular characteristics of the incentive intervention, such as regular high-level feedback on performance (as provided by the scoring metric here), can be especially important – even at times more so than the final reward (whether a lottery award or social recognition). The effectiveness of the recognition intervention in delivering weekly certificates of performance seemed to be easily comprehensible (perhaps more than the lottery tokens) and had an effect on personal motivations to improve capacity, with the additional attribute of such publicly displayed certificates creating a fear of public shame among facility staff. This may not simply be feedback on performance per se, but in fact an inducement for staff to gather information regarding what precise tasks they are expected to carry out. Furthermore, the variation found at the facility level also suggests communication of the intervention during field implementation can be paramount, especially in harnessing the role of the leadership at facility-level, here the OICs. While the limited timeframe as well as budgetary constraints for implementation determined the intervention design, and although field enumerators were trained together on the incentive interventions, the importance of better training enumerators on introducing the incentive interventions at the facility remains a key takeaway for future implementation and expansion of similar incentive schemes for behavior change. In addition, the links between facilities and host communities also need to be strengthened and expanded to incorporate RT supervision and accountability. The function of the facility health committees in both states is presently limited to mobilizing community members to patronize the facilities in greater numbers. The narrowness of this function was brought into relief during the intervention: the involvement of the majority of health committee members with the intervention was very limited and thin on specifics – some committee chairmen in the recognition group did not know why their facilities’ certificates were changed weekly, for instance, and they showed no interest in knowing why. Indeed, most committee members were only interested in the scheme to the extent that the certificates could bolster their primary objective of getting more people to visit their facilities. There remains scope, differently for both Ekiti and Niger, to extend the intervention to further empower community members of health facilities or inform communities directly on the RT interventions. This would not only require approval from government authorities at all 115 levels, but would also require the time and budgets for wider advocacy campaign prior to the implementation of the incentive interventions. While the Social Recognition intervention arm did attempt (though we were not able to confirm its success) to bring in the wider engagement of the community through public display of the weekly certificates, the potential for community engagement to drive the incentive measure can be explored further differently for both Ekiti and Niger. Finally, it would have been valuable to examine any evidence of “goal displacement” in those facilities that improved their RT performance during the social recognition internvention (e.g., has the act of filling in form C replaced, or reduced the time spent on, activities that are more essential for health outcomes such as talking with patients or hygiene measures). Given that performance was evaluated in terms of numerical outputs, facility staff may have had an incentive to maximize those outputs, regardless of whether maximizing them was the preferred path for achieving desired social outcomes. While we did not observe evidence for goal displacement in the qualitative data, we did not collect time-use data explicitly to confirm this observation (although the staff in charge of filling out the RT is likely non-medical and thus, not the same as the medical staff that cares for the patients). Future work may systemtatically examine such possibilities. In spite of the observations above as key lessons from this work to help future interventions and research, what remains clearly the case in Ekiti is that the Social Recognition intervention significantly improved the quality with which facilities filled out one of the RT tools (the Cash Form C) and sustained the improved performance over the intervention period of four weeks. Naturally, this analysis cannot tell us anything about long-term effects of similar approaches and in particular whether there is potentially either adaptation to the recognition intervention or, on the flipside, habit formation regarding the desired activity. Future work may be able to speak to these questions. 116 References for Chapter 6 Allcott, Hunt, and Sendhil Mullainathan (2012), External Validity and Partner Selection Bias, NBER Working Paper 18373, Cambridge, MA. Arrow, Kenneth J. (1972), Gifts and Exchanges, Philosophy and Public Affairs, 1(2), 343–362. Ashraf, Nava (2013), Rx: Human Nature: How Behavioral Economics Is Promoting Better Health Around the World, Harvard Business Review, 91(4), 119–125. Ashraf, Nava, Oriana Bandiera, and Kelsey Jack (2014a), No Margin, No Mission? A Field Experiment for Public Services Delivery, Journal of Public Economics, 120, 1–17. Ashraf, Nava, Oriana Bandiera, and Scott S. Lee (2014b), Awards Unbundled: Evidence from a Natural Field Experiment, Journal of Economic Behavior & Organization, 100, 44–63. Ashraf, Nava, Oriana Bandiera, and Scott S. Lee (2016), Do-gooders and Go-getters: Career Incentives, Selection, and Performance in Public Service Delivery, Unpublished paper, London School of Economics. Banerjee, Abhijit, Dean Karlan, and Jonathan Zinman (2015), Six Randomized Evaluations of Microcredit: Introduction and Further Steps, American Economic Journal: Applied, 7:1, 1-21. Besley, Timothy J., and Maitreesh Ghatak (2008), Status Incentives, American Economic Review, 98(2), 206–211. Bold, Tessa, Mwangi Kimenyi, Germano Mwabu, Alice Ng’ang’a, and Justin Sandefur (2013), Scaling Up What Works: Experimental Evidence on External Validity in Kenyan Education, CGD Working Paper 321, Washington, DC. Chetty, Raj, Mushfiq Mobarak and Monica Singhal (2014), Increasing Tax Compliance through Social Recognition, Policy Brief. Gneezy, Uri, Stephan Meier, and Pedro Rey-Biel (2011), When and Why Incentives (Don’t) Work to Modify Behavior, Journal of Economic Perspectives, 25(4), 191–210. Kahneman, Daniel and Amos Tversky (1979), Prospect Theory: An Analysis of Decision under Risk, Econometrica, 47 (2), 263–291. Kimmel et al. (2012), Randomized trial of lottery-based incentives to improve warfarin adherence, American Heart Journal, 164(2), 268–274. Krizan, Zlatan, and Paul D. Windschitl (2007), The influence of outcome desirability on optimism. Psychological Bulletin, 133(1), 95–121. Lacetera, Nicola, Mario Macis, and Robert Slonim (2013), Economic rewards to motivate blood donations, Science, 340(6135), 927–928. Larkin, Ian (2011), Paying $30,000 for a gold star: An empirical investigation into the value of peer recognition to software salespeople, Mimeo. 117 Mathauer, Inke and Ingo Imhoff (2006), Health worker motivation in Africa: the role of non-financial incentives and human resource management tools, Human Resources for Health, 4(1), 24. Moldovanu, Benny, Aner Sela, and Xianwen Shi (2007), Contests for Status, Journal of Political Economy, 115(2), 338–63 Naritomi, Joana (2015), Consumers as Tax Auditors, Working Paper, Harvard University. Neckermann, Susanne and Bruno S. Frey (2008), Awards as incentives, SSRN Working Paper 1021208. Nelson, Robert B. (2001), Factors that encourage or inhibit the use of non-monetary recognition by U.S. managers, Dissertation, Claremont Graduate University, available at http://novascotia.ca/psc/pdf/employeeCentre/recognition/toolkit/step1/BobsPhDDissertation.pdf. Accessed July 12, 2016. Peterson, Suzanne J. and Fred Luthans (2006), The impact of financial and nonfinancial incentives on business-unit outcomes over time, Journal of applied Psychology, 91(1), 156. Ryan, Richard M., and Edward L. Deci (2000), Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions, Contemporary Educational Psychology, 25(2000), 54–67. Stajkovic, Alexander D. and Fred Luthans (1997), A meta-analysis of the effects of organizational behavior modification on task performance, 1975–95, Academy of Management Journal, 40(5), 1122–1149. Stajkovic, Alexander D. and Fred Luthans (2001), Differential effects of incentive motivators on work performance, Academy of Management Journal, 44(3), 580–590. Stilwell, Barbara (2001), Health worker motivation in Zimbabwe, Unpublished paper/internal report for the Department of Organization of Health Care Delivery. Thaler, Richard and Cass R. Sunstein (2009), Nudge: Improving Decisions About Health, Wealth, and Happiness, Penguin Books, New York. Volpp Kevin G., George Loewenstein, Andrea B. Troxel, et al. (2008a), A test of financial incentives to improve warfarin adherence, BMC Health Services Research, 8(1), 272–277. Volpp Kevin G., George Loewenstein, Andrea B. Troxel, et al. (2008b), Financial incentive-based approaches for weight loss: a randomized trial, JAMA, 300, 2631–2637. World Bank (2004). Making Services Work for Poor People, World Development Report, Washington, D.C. World Bank (2015). Mind, Society and Behavior, World Development Report, Washington, D.C. 118 CHAPTER 7. Continuous Pets Approach: Evaluation Of The Methodology 7.1. Introduction Especially for a new tool or new adaptation of a tool, such as this new Continuous Public Expenditure Tracking Survey (CPETS), it would be useful to understand the capacity of the approach put forward at reaching its various objectives. These were especially (i) to devise useful data collection instruments able to capture relevant information on various dimensions of resource availability and usage (ii) to improve the incentive to collect the relevant information within the health system (iii) to generate useful data and information on resource availability for PHC services in the health sector in Nigeria. Ideally, an evaluation of the project and survey tools would have entailed designing a Randomized Impact Evaluation (RIE) of the CPETS intervention. This would have required to randomize the intervention between treated and non-treated (control) facilities with similar characteristics and to assess the intervention sometimes after (for instance using a retrospective PETS) on various components to measure the difference in trends between the two groups of facilities, before and after the CPETS intervention. While the RIE was considered in the initial design of the CPETS, because of capacity and budget constraints, the evaluation of the new continuous data recording intervention could not be performed using a rigorous RIE at the time of the survey implementation. A smaller RIE was also considered involving evaluating training and governance interventions as well as the main intervention, on a smaller sample of facilities. However, ultimately a choice of evaluating external incentive mechanisms (monetary and nonmonetary), was selected which however, could not substitute for a RIE of the CPETS intervention itself, as the experiments were implemented on the base intervention (see chapter 6). This chapter, while not benefiting from the rigorous RIE methodology, is a first step to try analyzing ex- post the successes and drawbacks of this first CPETS approach, in order to highlight lessons learned and propose recommendations for future tracking survey. This assessment of the tools focuses on specific dimensions of data quality generated by the new survey methodology. In particular, we seek to identify potential advantages of this approach, in particular are the new instruments able to collect the expected information, and were there shortcomings in the implementation of the tools and protocols? Ultimately, the assessment seeks to identify lessons learned and ways forward for future tracking survey using a continuous approach. The chapter is organized as follows. In section 2, after reviewing the main rationale of the new continuous approach, we review some of the potential pros and cons of the continuous vs traditional retrospective PETS approach. We then assess in section 3 the performance of the tools implemented in Nigeria, analyzing the survey data quality along two dimensions: (i) frequency of entry (i.e. were the instruments used to record the resources and activities) in order to measure capacity, (ii) quality of entry, that is was the information collected consistent and potentially accurate. We perform such analysis at the PHC level on the financial and non-financial resources forms, including medicines. At the LGA level, we especially examine financial and non-financial resources, as well reporting on budget and expenditures. We look at potential triangulation of data recorded with other sources, in particular through district level reported transfers vs facility level reported reception of resources. Section 4 discusses lessons learned and proposes ways forward for future tracking survey using a continuous approach. Section 5 concludes. 119 7.2. The continuous PETS approach 7.2.1. Rationale At the basis, a PETS is an ad-hoc survey tool that seek to track and assess resources levels and usage across administrative levels and service delivery units, across various programs or budget and period of time. It is implemented in the context of incapacity of routine data management systems and governance mechanisms to provide sufficient information and accountability of resources use. It has also been used as a verification mechanism to assess the quality of routine information systems. While PETS have been successful in various countries in the past at identifying inefficiencies and inequities in public management systems, they have faced challenges in implementation due to the often complexity of the system and flows they seek to analyse, and the poor quality of records often encountered in these weak governance contexts.46 This situation of poor quality of records was perceived as extreme in the Nigerian health system, as reported by health sector surveys and previous PETS studies performed in the health sector.47 In such context of absence of adequate records on which to base a tracking exercise of resource mobilization, transfers and usage, it was hence conceived to circumvent the situation by putting in place a continuous recording system, on a day-to-day basis, to create standardized recording tools across the levels and the states, and “follow the money” toward the primary health care users. While the CPETS approach of instituting day-to-day data collection system has several potential benefits, it has also various potential limitations. In this section, we review some of the potential advantages and limitations of the CPETS in comparison to standard PETS, before examining in the following section the actual implementation of the CPETS in Nigeria and analysing data quality arising from the tools, to help identify lessons learned for future endeavor using CPETS. 7.2.2. Comparison with traditional PETS: pros and cons of the approaches We now discuss various potential advantages and limitations of the CPETS in comparison to standard PETS, examining in particular records requirement, team composition, training, monitoring and supervision, as well as survey duration and costs. Access and availability of records The PETS being a data collection tool at various levels of the administrative and service delivery system, it makes use of various questionnaires and instruments applied at various levels. The tracking survey relies on various types of records on budget and expenditures, resources and activities kept at the facility and other administrative levels, as well as direct observation by enumerators of resources, stocks, infrastructure and equipment available at the various level. In the absence of records, recall of information based on interviews with officials sometimes substitute as a second best for paper records, but recall could certainly 46 For instance, the first PETS implemented in the health sector in 1996 in Uganda, contrary to its education sector counterpart where after a pilot survey, the focus was specifically restricted to a particular program (capitation grants), in the health sector the focus remained on all public resources and administrative levels. However, lack of almost any financial information at the facility level and the heavy reliance on in-kind measures were not anticipated at design and pilot stage of the survey. Consistent quantitative data could not be collected. Ultimately, the survey did not produce any reliable quantitative measures of expenditure flows or leakage assessment (Ablo and Reinikka, 1998; Gauthier and Ahmed 2012, p. 158) 47 See for instance Couttolenc 2013 and Khemani 2006. 120 not provide accurate information for numerous or detailed resources management information required to establish financial or in-kind resource tracking, affecting data quality and survey success. The rapid data assessment (RDA) conducted as a preliminary step in the standard PETS protocol (see for instance Reinikka and Smith 2004, Gauthier and Ahmed 2012) serves at identifying data availability and reliability for tracking and triangulation purposes across levels and sources. Limitations in data availability lead to redefinition of the scope of the study and potentially also its objectives and research questions, in order to focus on the resources, programs or/and time periods for which reliable data records are available to develop a sound resource tracking exercise. Successful PETS have properly aligned the scope and objectives with available and verifiable data.48 PETS that have not properly performed the RDA or redefine its scope and objectives according to available verifiable data based on records have faced various data quality problems.49 In addition, in order to facilitate access to required records at the various levels, various communications are to be undertaken prior to the implementation of the PETS and enumerators visit to the various administrative and service delivery units, to inform officials regarding records and data requirements, in order to facilitate their availability during survey implementation. In the Nigerian health system, prior PETS and a RDA study indicated the very poor record keeping practices and availability at the district and facility levels in particular (Khemani, 2006; Couttolenc 2013). This could have led to a scope narrowing and the identification of specific programs or resources for which records and observability could have been more feasible. However, in addition, one of the key project objective was to assess the level of overall resources available for PHC services. This complementary objective has motivated the wide scope of the survey and data availability requirement, and have led to the design of a continuous data collection approach, using day-to day data recording tool. In weak governance setting such as the Nigerian health system, various factors could explain the absence or poor quality of records: in particular, the lack of system in place, lack of capacity and resources to perform the record keeping tasks, the lack of incentives to accomplish such administrative tasks, but also the unwillingness to keep official records on activities and resources given that the information could be used for accountability purposes. The information could indeed be used to monitor performance of staff productivity, to monitor the valuable resources generated or received by the centers, in particular there are resources generated through services rendered (e.g. user fees for consultation, diagnostic tests) and sales of drugs. Given the potential incentive to misreport, a key issue with continuous data recording is that if incentives are affected and improved by the continuous recoding tools. It is certainly a key issue that was discussed in chapter 6’s experiments and that should become the focus of a RIE. 48 This redefinition of survey scope following a RDA was for instance observed in the first PETS in education in Uganda where the focus was placed on a small capitation grant program for which data was available at the school level and a five-year time period was chosen (Ablo and Reinikka, 1998; Reinikka and Svensson 2001; Gauthier 2006). The existence of a fixed rule of allocation simplified the data requirement which only required number of students and capitation budget over the years. 49 This was for instance the case of the 2002 Mozambique health PETS/QSDS which tried to track all non-wage recurrent expenditures, drugs and other supplies, and human resources, at three levels: provinces, districts and facilities. However, data quality turned out to be a serious concern especially at the provincial and district levels and the data collected did not permit reliable assessments of leakage. (Gauthier and Reinikka, 2013). 121 Team composition and capacity The management structure of in PETS studies requires a balance between technical expertise and independence brought by a team of external experts and a survey firm, and the policy knowledge, relevance and field coordination brought by government agencies and policy makers. In traditional PETS, the survey firm needs to assemble a team of high-quality enumerators and supervisors with adequate experience in facility surveys. Supervisors are generally responsible for supervising teams of enumerators who administer the various questionnaires at the different administrative and service delivery levels. However, contrary to standard PETS survey where enumerators (and supervisors) are in charge of data collection -- conducting interviews and collecting data at various administrative and service delivery levels, generally over one day per unit or facility--50 in addition in CPETS, the continuous (prospective) questionnaires are to be filled by government official and facility staff on a day to day basis. In Nigeria, this continuous recoding process by a team of government officials involved PHC staff and LGA officials who were responsible for data recording on a day-to-day continuous basis during the 6 months of the survey, with verification by enumerators on a weekly basis, as well as by supervisors. While in any PETS, survey design and implementation process requires close cooperation and coordination with policy makers and officials for ensuring adequate design of the study, implementation and policy relevance. In the continuous setting, this cooperation with officials and service delivery staff is magnified. The delegation of data recording responsibility to the health facility staff and district officials introduces additional problems of control over the agents involved in the process. The multiplicity of data recording agents, some being employed by the survey firm under contractual obligation with the principal, but others, the PHC staff and district officials, being agents of the government clearly introduces greater problems of incentivizing and supervising agents. While, in traditional PETS several goals could be pursued, including building internal monitoring capacity within government agencies -- and in that regard, closer sensitization and involvement of staff and program operators in the project could be chosen—in a CPETS, this staff involvement, cooperation and sensitization is at the core of the approach. In this context, incentives and capacity of the additional governmental agents involved in the survey process are a key to its success, including through information dissemination and training. Training Training plays a key role in capacity building and incentive development. In standard PETS, a training workshop is generally held in the weeks before the planned survey fielding to enable enumerators and supervisors to familiarize themselves with the survey instruments and field work. The training workshop generally last two weeks of training, and is completed by a field pilot of the instruments 51 (Gauthier and Ahmed, 2010, p. 80). In the context of a CPETS, based on self-reporting by officials and staff which make them at the center of data collection, the training is complexified. Indeed, instead of having to simply having to train enumerators and supervisors, officials and PHC staff to be involved in data recording need to be identified and trained. 50 In combined PETS-QSDS, a second unannounced visit is generally conducted on a different day among the facilities to measure staff absenteeism. In some surveys, a second visit by enumerators to the units or facilities was required to completing the collection of all information required. 51 The training workshop consists in extensive classroom and on-site training of all survey enumerators and supervisors to ensure that all questions and procedures are well understood. (Gauthier and Ahmed 2012, p80) 122 A potential problem comes from the identification and participation of the facility staff and district officials that will become responsible at each data recording levels of using the instruments. In addition, the detailed enumerator and supervisor manual used in PETS, including a survey protocol describing the main procedures for survey implementation, which is utilized in the training sessions and also support the teams’ field work, is prepared prior to the training of the survey firm for PETS surveys. In the context of CPETS, the manual needs to be completed with specific manuals and detailed protocols for government officials and staff involved in data recording. Monitoring and Supervision Quality of monitoring and supervision of data collection and data entry are key factors in PETS survey quality. In traditional PETS, the survey firm must ensure close supervision and oversight in collecting the data, as this is a significant factor in ensuring the collection of high-quality data. In involves in particular that the field supervisors devote sufficient time and attention to supervising the work of the enumerator teams. In addition, PETS guidelines recommended random visits to enumerators in the field which are essential to ensure quality control and coherence in the interpretation of questionnaires. Furthermore, random checks of questionnaires and data quality should be done throughout the survey implementation. In that regard, PETS guidelines recommend real-time quality control procedures in which through which completed questionnaires are carefully reviewed by supervisors and the main management team for incompleteness, incoherence, etc. Feedbacks and questions are then to be relayed immediately to enumerators for clarifications, corrections, and potential return visits to collect missing data (Gauthier and Ahmed 2012: p. 83). In that regard, a test of these verifications and quality control procedures are recommended and an account of the results normally required at the pilot exercise stage. In addition, it is recommended that survey firm provide a system of ex-post interview verification and/or random spot checks to ensure quality control of enumerators’ work. Furthermore, is recommended that a percentage of sampled facilities and units be revisited for quality control checks of the data (Gauthier and Ahmed 2012: p. 84). In the context of CPETS, all these procedures of monitoring and supervision are rendered much more complex. Indeed, the CPETS approach demultiplies the problem and importance of M&S given that data recoding is done at the basis by facility staff and officials and that collection is not done one day on a facility but potentially every working day for the duration of the continuous period covered (6 months in Nigeria). Indeed, one of the main challenge of the CPETS is its reliability with regard to self-recording by staff and officials instead of relying on enumerators collecting the data through direct record verification and interviews. In the continuous setting, this step of enumerator’s record verification and staff interviews comes as a second step, following the initial staff recording. The initial and main reliance of the approach is the self-usage of the tools. It is somewhat related to household surveys diaries in which household head record their consumption. Hence, a key component of the tool is its potential capacity at actually recording the activities. This requires that the staff have an incentive and capacity to fill the tools (i.e. that the tools are adapted to what is to be received, which is compounded by the numerous medical supplies and drug formats potentially received) and simple enough to use. Another key element relates to incentives. There was a presumption leading to the development of the approach that the staff were not inclined to keep records because of incentive problems. A key issue is then to determine how the new tools and training would help in solving these incentive problems. Furthermore, the continuous nature of the process during several months of survey period would require to have a constant, day to day real time supervision and analysis of the data collected and transmitted. Quality control procedures and indicators are needed to continuously monitor the data collected. 123 Timing, tracking period, survey duration and costs A traditional PETS generally takes between 4-8 weeks of field work to collect retrospective data, according to the size of the sample, survey team size and local conditions.52 With regard to the timing of the field work, it is recommended that PETS surveys be fielded two to three months after the end of a fiscal year (for accounting books to be closed). Quantitative tracking period is expected to cover at least the last completed fiscal year (and a maximum of two financial years to maximize data collection quality (Gauthier and Ahmed 2010, p 67-68). If monthly data are collected, seasonality issues have to be considered and a proper strategy devised so as to reduce potential biases.53 While the budget necessary for conducting a PETS/QSDS depends on a number of factors,54 overall, the costs of a PETS generally range between US$ 75,000 and US$ 250,000 for a full-scale study (Gauthier and Ahmed 2012, p. 39). In the context of CPETS, these various components are quite different. The tracking time period in CPETS being prospective, the continuous tracking period determines the length of the tracking period obtained. While it would certainly be useful to have a whole year of continuous data collected, in order to observe patterns and seasonality, Indeed, attendance intensity and flow of resources are potentially seasonal and at different time period, often may not have capture all flows, but if we have whole year cover all relevant. However, it may not be feasible to cover such a long time-period. There is indeed a cost trade-off between the length of the continuous survey period and additional information to be gained. It is very expensive to do a long-lasting survey given that enumerators and supervisors, as well as the entire research team needs to be active for a much longer period of field work. If we want to do it, want to do it long enough for information to be adequate but too long as to be too costly. In Nigeria, the “screen shot” of 6 months of data collection was chosen, but it should be noted that Nigeria and the 2 states examined were going through a deep economic and budgetary crisis which affected resources levels ad activities tracked. In a more typical year, maybe different picture would have emerged. 7.3. Assessment of survey data Having reviewed some of the potential pros and cons of continuous vs standard retrospective PETS, we now examine the quality of the data collected with the CPETS approach in the health sector in Nigeria. As discussed earlier, to assess the performance of the tools, we analyze the collected survey data along two dimensions: (i) frequency of entry, to measure usage of the tools, (ii) quality of entry, that is was the information collected consistent and potentially accurate. We focus on the sets of questionnaires addressed to PHCs and LGAs, being the two levels for which continuous data tools and protocols were applied (i.e. health staff and officials being the main data recorders). At the other administrative levels, survey tools administered consisted of traditional retrospective PETS instruments with data collection performed by enumerators. We perform the analysis of data quality on the following questionnaires: at the PHC level, we examine financial and non-financial resources, including medicines; at the LGA level, we also examine financial 52 The overall duration of a traditional PETS study is generally between 8-12 months to complete, from planning to data analysis and reporting. 53 See Gauthier and Ahmed 2012, p.126 for details. 54 Including its scope, sample size, complexity of the survey instruments, sector, geography, and labor and survey costs in the country, as well as the management structure chosen, in particular the extent of work done internally or contracted out to local or international consultants or survey firms (Gauthier and Ahmed 2012, p.39) 124 and non-financial resources, as well reporting on budget and expenditures. We also look at potential data triangulation to further assess consistency, focusing on reported transfers by LGA toward PHC facilities vs reported reception of resources at the facility level by PHC personnel. The data quality assessment faces several challenges, especially linked to the absence of benchmarks. For the first dimension, we wish to assess, the frequency of reporting, we confront the problems that we do not have information toward which we could compare usage of the tools and hence establish the validity of entry frequency, (either on quantity, value or sources). Several of the resources received and activities rendered at the PHC or LGA levels are irregular and hence difficult to benchmark. Similarly regarding entry levels, there are also no real benchmarks that could be used to calibrate the registered data. With regard to frequency, a way around this benchmarking challenge is to focus on components of the tools that are closer to a continuous recording, that is where we would expect to have relatively frequent activities recorded. These higher-frequency characteristics in PHCs are particularly to be found for the revenues accruing from sale of services (e.g. consultation, diagnostic tests) and of goods (e.g. drugs, supplies), as well as for services rendered (output) at the PHC level. For that purpose, we make use of the cashbook tools for such higher frequency examination, which consisted of a form for daily recording of revenues and expenditure at the PHC level, with a weekly data verification by enumerators. Cashbook forms were expected to be filled everyday by facility staff, so frequency could be assessed on a daily basis. The consistency and logic of entry could also be assessed to analyse the quality and level of entries through triangulation. We benefit from the fact that the small RIE experiments (see chapter 6) were conducted on the cashbook tools, and a checklists tool was implemented during the experiment to record the quality of cashbook entries. We examine the evolution of these recordings (frequency and levels/quality) over the entire survey period among control (untreated) facilities, that is, PHCs which were not subject to the incentive experiments, but in which only the CPETS intervention took place. PHC level We first examine the quality of data collected at the PHC level (module 1), focussing on some of the main questionnaire components, in particular financial (Form A) and non-financial resources received (Form B), as well as recording of IGR (Form C). Data was recorded on paper questionnaires by PHC officers, with verifications to be realized on a weekly basis by enumerators, followed by electronic data entry by each enumerator. Financial resources (Form A) As part of the tracking exercise and in order to establish the level of resources available at the facility level, PHC officers were asked to record on a day-to-day basis on monthly forms, during the 6 months of the survey, the amount of revenues received from various sources, including transfers from public and private donors, but excluding IGR. Information on the date of reception, sources of these transfers (11 categories), amounts received and earmarked components were required to be completed. To evaluate data quality on financial resources recorded at the PHC level, Table 7.1 presents the frequency of financial resource reception during the survey period and the number and percentage of PHC reporting such support. We observe that overall there are only 33 cases of positive transfers among the PHCs over the survey period. Among the 140 PHC (65 Ekiti and 75 Niger), only 26 report receiving any type of financial resource during the 7 months of recording at the PHC level, or about 19% of the PHCs. Most of these financial transfers were reported in Ekiti, with 23 PHC (about 1/3 of the sample) reporting financial support, while in Niger only 3 PHC (4%) reported any financial support during the survey period. 125 On a monthly basis, in Ekiti one fifth of the PHCs report transfer in February and very seldom transfers the other months, while in Niger less than 4% of the PHC report transfer any other months. While the overall low frequency and especially the difference in reported support between the states are striking, in the absence of benchmarks it is difficult to establish if the low frequency of reported transfers, especially in Niger, is due to underreporting or actual low frequency and level of financial transfers toward PHCs. Table 7.1: Quality of data: PHC Financial support Number of PHC % of PHC Frequency implied in implied in of positive positive positive Form A observations observations observations Financial resources received by PHCs Total 33 26 18.57 Ekiti 26 23 35.38 October 4 4 6.15 November 5 4 6.15 December 3 3 4.62 January 0 0 0.00 February 13 13 20.00 March 1 1 1.54 April 0 0 0.00 Niger 7 3 4.00 October 2 2 2.67 November 1 1 1.33 December 3 3 4.00 January 1 1 1.33 February 0 0 0.00 March 0 0 0.00 April 0 0 0.00 126 Non-Financial Resources (Form B) In addition to financial transfers, PHC could receive support taking the form of various in-kind resources, for instance equipment and medical supplies, or drugs and vaccines from various sources. PHC officers were requested to report these items and equipment received as they happened, on a monthly form, over the 7 months of the survey. The main reporting categories were medical supplies, medical equipment, other material and non-medical equipment. Information was to be reported on quantity or/and value of items received when feasible, and on the source. Table 7.2 presents the frequency of cases (column 1) of in-kind items reported among the sample PHCs over the survey period, as well as the number and percentage of PHCs reporting such receptions (columns 2 and 3). Table 7.2: Quality of data: PHC in-kind support Number of Percentage Frequency PHC of PHC of positive reporting reporting Form B1 observations reception reception Quantity of in-kind support received by PHCs 228 48 34.04 Ekiti 63 20 30,77 October 13 5 7,69 November 2 1 1,54 December 3 1 1,54 January 5 3 4,62 February 26 11 16,92 March 0 0 0,00 April 14 2 3,08 Niger 165 28 36,84 October 38 8 10,53 November 45 16 21,05 December 32 14 18,42 January 11 3 3,95 February 21 4 5,26 March 17 2 2,63 April 1 1 1,32 Value of in-kind support received by PHCs 16 13 9.22 Ekiti 6 6 9,23 October 3 3 4,62 November 0 0 0,00 December 0 0 0,00 January 1 1 1,54 February 1 1 1,54 March 0 0 0,00 April 1 1 1,54 Niger 10 7 9,21 127 October 2 1 1,32 November 5 3 3,95 December 1 1 1,32 January 0 0 0,00 February 1 1 1,32 March 0 0 0,00 April 1 1 1,32 We observe in the top part of the table un-frequent recording of reception of quantity of in-kind items among PHC, with about 1/3 of PHCs overall declaring at least a reception of in-kind resources during the 7-month period. A slight higher reception rate is observed in Niger compared to Ekiti (37% vs 31 % respectively). On a monthly basis in Ekiti, while February shows the most reported arrivals with 17% of PHCs reporting a reception, the average is only about 4% for the other months. In Niger, between 10% and 20% of PHC report an arrival in October-December compare to an average of 3% for the other months. We also observe that values of in-kind shipments (bottom part of the table) are even less frequently reported. For the entire 7-month survey period, only 9% of PHCs provide a value of in-kind support. This rate of reporting on valuation is comparable among PHCs in both states. Cash Book- Form C As mentioned earlier, one of the main challenge at collecting quality data of the continuous PETS survey vs traditional PETS is its reliability on staff’s self-reporting instead of on enumerators data collecting through record verification and interviews. The risk trade-off is between the lack of entries associated with staff’s weak incentives and unwillingness to collaborate in the self-reporting process on day-to-day basis vs the lack of retrospective records to allow enumerator’s verification and recording. To measure the quality of the data obtained with the continuous instruments, we focus on completeness of self-reporting and the consistency of these data, as mentioned, for some of the instruments, the completeness of reporting is difficult to establish without the existence benchmark levels or the availability of supplementary supervision or monitoring tools (we will come back to these issues). For instance, arrival of financial or even in-kind items such as health equipment tend to be un-frequent and irregular. It is hence expected that relatively few items will be recorded. In such context, additional information, such as a benchmark or information on the sending side to allow triangulation would be required. For other tools such as those used to measure for instance IGR from user fees or drug sales, the assessment of quality of data and completeness of recording is somewhat easier. Indeed, for the first dimension we which to assess, the frequency or completeness of entry (i.e. usage of the tools), we could focus on survey components that are close to a continuous recording. For instance, we expect to have relatively frequent or at least daily activities taking place within PHCs with regard to service provision (output) as well as for the revenues accruing from sale of services (consultation, diagnostic tests) and of goods (drugs, supplies). In the continuous survey, these components were recorded through the cash book tools (Form C). It consisted in a daily recording of main service charges (8 itemized categories, including consultation, drug sales test) as well as expenditures (25 itemized categories, including drug purchase). Verification of the cashbook entries was to be realized weekly by the enumerators, followed by electronic data entry. The cashbook form was expected to be filled everyday, so frequency of entries could be assessed on a daily basis. Furthermore, the consistency and levels of entries could be analyzed, including through data triangulation and reconciliation (e.g. facility output). 128 Furthermore, we could use the additional information generated by the incentive experiments discussed in chapter 6 conducted using the cashbook data and a cashbook checklist tool. While the experiment tested the impact of two different interventions in selected PHCs, a sub-sample of facilities in both states were used as a control (untreated) group, in which only the CPETS intervention itself was put in place. We examine usage of the cashbook and data entry behavior within these control PHCs over the course of the experiments, as well as during the overall 29 weeks of the survey.55 Table 7.3 presents the average daily IGR revenues reported by PHCs, as well as the percentage of facilities that report no revenues from user fees during the week in each state. Regarding the completeness of recording, we observe what appears to be substantial gap in IGR reporting, especially in Ekiti as on average 38% of PHC do not report any revenues from IGR on a weekly basis during the course of the survey in that state, compared to 13% in Niger. Furthermore, regarding the quality of the data reported itself, we observe that the level of revenues reported appears very low especially in Ekiti. Daily revenues from IGR reported by PHCs in Ekiti average only 217 Naira during the course of the survey (about 1.09 US$). Reported IGR are 5 times higher in Niger but still very low, with revenues of 1145 Naira (about 5.75 US$)56 per day of user fees declared by PHCs. During the experiments, a checklist was used by enumerators to verify the entries during their weekly visits. For the control PHCs group in both states, these checklist data are available for weeks 17 to 24 (illustrated in grey in the table) out of the 29 weeks of the survey.57 The implementation of a verification checklist of the cashbook form by enumerators does not seem to have led to changes of behavior of the PHC staff as the frequency of zeros and average IGR revenues do not show significant variations before, during and after the experiment period. 55 70 control PHCs, 33 in Ekiti and 37 in Niger. 56 For only positive entries, averages of 349 N (0.97 US$) and 1319 N (3.67US$) in Ekiti and Niger, respectively. Average exchange rate in 2015 of 199N/US$ 57 Week 17 constituted the baseline. The checklist was not administered during week 18-20 while the experiment lasted between week 21 and 28, with control PHCs for the first 4 weeks. 129 Table 7.3: Completeness of reporting: PHC Cashbook receipts Cash Receipts- Form C Ekiti Niger Average Average daily Percentage daily Percentage of Week revenues of 0 revenues 0 1 290 0,33 1375 0,22 2 224 0,33 1322 0,19 3 258 0,36 1291 0,19 4 332 0,30 1334 0,16 5 337 0,27 1684 0,05 6 312 0,27 1418 0,05 7 164 0,30 1193 0,08 8 237 0,36 924 0,24 9 168 0,33 1158 0,08 10 161 0,39 1005 0,08 11 208 0,36 899 0,05 12 139 0,48 659 0,16 13 129 0,55 1253 0,08 14 184 0,42 947 0,08 15 210 0,39 925 0,03 16 168 0,45 916 0,08 17 147 0,36 897 0,16 18 286 0,39 1159 0,08 19 269 0,36 1206 0,11 20 146 0,45 1311 0,03 21 138 0,30 1402 0,08 22 190 0,45 1356 0,08 23 191 0,42 1233 0,05 24 178 0,36 1308 0,05 25 170 0,39 1063 0,16 26 137 0,36 1075 0,22 27 433 0,39 1032 0,27 28 249 0,33 979 0,35 29 251 0,45 882 0,24 Average 217 0,38 1145 0,13 Note: Ekiti control, 33 PHCs; Niger control 37 PHCs; data from sub-total B cash receipts The evolution of average reported IGR (weekly) revenues by control PHCs facilities over the course of the entire survey as well as the experiment period in both states is represented in Figure 7.1. Week 17 corresponds to the beginning of the baseline implementation of the cashbook checklist. We observe a lack of clear trend in revenues before and during the experiment period. If a longer time period had been available, structural breaks in the relationship between revenues and determinants of revenues could have been examined. 130 Figure 7.1: Weekly average cash receipts from IRG over the 29 weeks in Ekiti and Niger (control PHCs) Note: 33 control PHCs in Ekiti and 37 control PHCs in Niger The somewhat higher reported IGR in Niger reflects in part the fact that PHCs in Niger use private drug supplies compared to Ekiti public drug support (see chapter 5). While difference in this context, it is difficult to interpret greater zero reporting as evidence of underreporting. The checklist tool used during the experiment further allows examining the quality of the data collected in the cashbook form. Indeed, as discussed in chapter 6, the cashbook checklist form produced a weekly score for each facility, including the control (untreated) facilities, for 10 vari ables based on enumerators’ verification of the quality of the entries.58 Table 7.4 presents the mean and frequency of responses for the 10 checklist variables for the control PHC facilities over the 5-week period (including the baseline week and 4 experiment weeks), per state and total, to which we added 3 other indicators also derived from the checklist form. 58 Questions 1, 2, and 8 were scored using the checklist enumerators’ responses on a binary basis (“yes,” =1, “no,” =0), questions 3-7 are fractions, from 0 to one, depending on the number of days that various conditions are satisfied as a fraction of the days that the facility is open; questions 9 and 10 are scored zero when there is no documentation ; one when all relevant transactions have documentation ; and 0.5 when only some of the transactions have documentation. Question 8 pertains to “Form B2” (Non-financial resources) which records dates and quantities of in- kind items including for drug shipments that the facility receives, allowing enumerator to check Forms B2 and C for consistency on at least the arrival of drugs in the facility on a given day. See chapter 6 for details. 131 Table 7.4: Mean & frequency of the Checklist of the cash book variables, per state and weeks, for the control PHCs, with additional variables Week 0 - Baseline 1 2 3 4 Total Question State Ekiti Niger Total Ekiti NigerTotal Ekiti Niger Total Ekiti Niger Total Ekiti Niger Total Had facility staff completely filled Form 0,48 0,41 0,44 0,50 0,78 0,65 0,58 0,73 0,66 0,61 0,76 0,69 0,56 0,81 0,70 0,63 1) C before you arrived? 33 37 70 32 37 69 33 37 70 33 37 70 32 37 69 348 Did OIC/Treasurer check Form C accuracy 0,52 0,73 0,64 0,48 0,81 0,67 0,53 0,75 0,65 0,67 0,75 0,71 0,70 0,83 0,78 0,69 2) and completeness? 27 37 64 27 37 64 30 36 66 30 36 66 27 36 63 323 Fraction of days complete: Balance 0,78 0,90 0,85 0,71 0,99 0,87 0,79 0,92 0,86 0,71 0,94 0,84 0,76 0,97 0,88 0,86 3) Forward 27 36 63 27 36 63 30 35 65 30 36 66 27 36 63 320 Fraction of days complete: Cash Receipts 0,70 0,97 0,86 0,70 0,95 0,84 0,70 0,88 0,80 0,65 0,92 0,80 0,76 0,97 0,88 0,83 4) from User Charges 27 36 63 27 36 63 30 35 65 30 36 66 27 36 63 320 Fraction of days complete: Cash Receipts 0,70 1,00 0,87 0,72 0,94 0,85 0,68 0,90 0,80 0,65 0,94 0,81 0,76 0,97 0,88 0,84 5) from Drug Sales 27 36 63 27 36 63 30 35 65 30 36 66 27 36 63 320 0,63 0,97 0,83 0,66 0,86 0,78 0,65 0,83 0,75 0,61 0,87 0,75 0,80 0,94 0,88 0,80 6) Fraction of days complete: Expenditure 27 36 63 27 36 63 30 35 65 30 36 66 27 36 63 320 Fraction of days complete: Expenditure 0,63 0,86 0,76 0,66 0,67 0,67 0,64 0,60 0,62 0,59 0,69 0,65 0,68 0,67 0,67 0,67 7) from Drug Sales 27 36 63 27 36 63 30 35 65 30 36 66 27 36 63 320 Drug purchases and sales: Form C 0,96 0,95 0,95 0,96 0,95 0,95 0,93 0,92 0,92 0,93 0,94 0,94 0,96 0,97 0,97 0,95 8) consistent with Form B2 27 37 64 27 37 64 30 36 66 30 36 66 27 36 63 323 Documentation substantiates cash 0,69 0,00 0,29 0,85 0,01 0,37 0,82 0,00 0,37 0,78 0,00 0,36 0,89 0,00 0,38 0,35 9) receipts from user charges 27 37 64 27 37 64 30 36 66 30 36 66 27 36 63 323 Documentation substantiates cash 0,61 0,19 0,37 0,61 0,15 0,34 0,62 0,22 0,40 0,73 0,22 0,45 0,81 0,24 0,48 0,41 10) receipts from expenditure 27 37 64 27 37 64 30 36 66 30 36 66 27 36 63 323 Q.1a. Does the facility receive drug from 0,63 0,50 0,58 0,63 0,69 0,67 0,63 0,69 0,67 0,75 0,77 0,76 0,75 0,77 0,76 0,70 11)* any source? 8 4 12 8 13 21 8 13 21 8 13 21 8 13 21 96 Q. 1b. Does the facility purchase drug 0,13 1,00 0,42 0,13 1,00 0,67 0,38 1,00 0,76 0,13 1,00 0,67 0,25 1,00 0,71 0,67 12)* from any source? 8 4 12 8 13 21 8 13 21 8 13 21 8 13 21 96 0,75 0,00 0,50 0,63 0,08 0,29 0,38 0,15 0,24 0,38 0,08 0,19 0,63 0,08 0,29 0,28 13)* 4. Are all entries on Form C ‘0’? 8 4 12 8 13 21 8 13 21 8 13 21 8 13 21 96 * Non-Drug revolving fund PHCS only; The frequency is presented under the mean. We observe that control PHCs in Ekiti report much less frequently the requested cashbook information than Niger’s PHCs. About 1/5 of the control facilities in Ekiti do not provide any information regarding the checklist variables compare to a situation of full reporting by control PHCs in Niger. Furthermore, mean values for all questions 1-8 are significant greater in Niger, including for fraction of days with completed forms for cash receipts from consultation and drug sales (questions 4-5). However, availability of documents for enumerators to validate the recording are much less frequent in Niger, especially for user charges (question 9), as well as receipts for expenditure (question 10). For user charges, none of the 33 control PHCs could document receipts from user charges, compared to about 77% in Ekiti over the 5 weeks of the checklist implementation. This pattern could be interpreted with the fact that Niger, not keeping records of IGR receipts, faces a much lower cost of reporting cashbook information. Indeed, it is much easier to fill up the table without receipts compare with Ekiti PHCs who tend to keep records. 132 LGA level We now examine the quality of data collected at the LGA level, for each of the main component of the questionnaire (module 2). Data at district level was collected on paper questionnaires by LGA officers on a continuous basis, with verifications realized on a monthly basis by enumerators, followed by electronic data entry. Financial resources (Section 1A) For the financial tracking exercise, LGA officers were asked to record on a monthly basis, their level of revenues, including wages and salaries, and revenue sources derived from federal government, state transfers and own revenues, during the 6 months of the survey, Table 7.5 evaluates completeness of financial resources data recorded by presenting the percentage of missing information on LGA revenues and number of cases of missing recording (LGA-months) overall and per months. We observe that among the 39 sampled LGA (16 in Ekiti and 23 in Niger), about 13% of the information on total LGA financial revenues was missing over the period of the survey, which correspond to 30 missing LGA-month expected. Culprits are mainly found in Niger, as the lack of monthly reporting is associated with 12 LGAs in Niger (33% of the sample) and only 1 LGA in Ekiti. Table 7.5 : LGA: Completeness of reporting: LGA financial resources Percentage with missing Number of Section 1A information cases Total LGA revenues 12.82 30 Detailed Financial resources received from government sources 87,93 2469 Ekiti 86,63 998 October 85,94 165 November 86,46 166 December 79,69 153 January 91,67 176 February 84,38 162 March 91,67 176 Niger 88,83 1471 October 82,61 228 November 88,41 244 December 88,77 245 January 90,94 251 February 90,94 251 March 91,30 252 133 An even larger problem of missing information is observed with regard to the recording of sources of revenue, as LGAs have for the most part not provided details about their revenue sources (Table 7.5). Adequate tracking of resources toward PHCs requires to identify and follow funding down the supply chain toward ultimate users and hence would require to link received transfers at the LGA level with a potential sending source at the federal or state level for instance. However, as observed, about 88% of observations reported by LGAs on sources of revenues are missing in both states. We also observe over the 6 months of the survey a slight increase in percentage of missing information, especially in Niger, reaching 91% of missing information on amounts of LGA revenues by sources in March. The consistency between total revenues reported and itemized revenues by sources is also very low. As observed in Table 7.6, perfect matching between a total revenue and its sources is observed in only one case out of 244 (that is, total revenue recorded is perfectly explained by the itemized revenues by source among the 39 LGAs during the 6-month period). Partial matching between total revenue and sits itemized sources is only found about 25% of the time (LGA-months) in both states, with a tendency to decrease matching in Ekiti over the survey period. Table 7.6: Consistency of reporting: LGA financial resources Percentage with Section 1A information Number of Matching between the total LGA revenue and revenue sources missing cases Lack of perfect matching 99.57 233 Lack of partial matching 73.93 173 Ekiti 75.00 72 October 37.50 6 November 37.50 6 December 75.00 12 January 100.00 16 February 100.00 16 March 100.00 16 Niger 73.19 131 October 69.57 16 November 73.91 17 December 69.57 16 January 78.26 18 February 73.91 17 March 73.91 17 Non-Financial Resources (Section 1B) In addition to financial transfers, LGA officers were asked to report information on in-kind resources received from various levels and sources, on a monthly basis. The main reporting categories were medical supplies, medical equipment, other material and non-medical equipment. Information on quantity or/and a value of items received when feasible and the source were also to be reported. 134 Table 7.7 presents the percentage of LGA reporting a positive reception of any items over the 6-month period (first column), as well as the information provided on this reception (columns 2-4). We observe a low reporting rate of in-kind reception over the months (column 1) with about 62 % of LGA declaring not receiving any items during the entire survey period (top part of the table). While about 2/3 of LGAs report receiving at least one in-kind item in Ekiti during the survey period, that proportion was four times lower in Niger (18%). Table 7.7 : Completeness of reporting: LGA in-kind items 1 2 3 Total variable variables variables SECTION 1B % % % % % of the LGAs that declared receiving a positive quantity &/or source &/or total value of an item 38.46% 17.95% 17.95% 2.56% Ekiti 68.75% 31.25% 31.25% 6.25% October 37.50% 18.75% 18.75% 0.00% November 18.75% 6.25% 6.25% 6.25% December 18.75% 6.25% 6.25% 6.25% January 12.50% 6.25% 6.25% 0.00% February 12.50% 6.25% 6.25% 0.00% March 0.00% 0.00% 0.00% 0.00% Niger 17.39% 8.70% 8.70% 0.00% October 8.70% 4.35% 4.35% 0.00% November 0.00% 0.00% 0.00% 0.00% December 0.00% 0.00% 0.00% 0.00% January 0.00% 0.00% 0.00% 0.00% February 8.70% 4.35% 4.35% 0.00% March 0.00% 0.00% 0.00% 0.00% 3 SECTION 1 variable 2 variables variables 1B Total % % % % Total % of positive observations reported and % of positive variables (source &/or quantity &/or total value) per observation reported 9.28% 4.13% 3.71% 1.44% Ekiti 18.93% 8.09% 7.44% 3.40% October 32.00% 17.00% 15.00% 0.00% November 6.00% 2.00% 2.00% 2.00% December 63.54% 21.88% 21.88% 19.79% January 8.18% 4.55% 3.64% 0.00% February 9.28% 5.15% 4.12% 0.00% March 0.00% 0.00% 0.00% 0.00% Niger 2.15% 1.20% 0.96% 0.00% October 3.62% 2.17% 1.45% 0.00% November 0.00% 0.00% 0.00% 0.00% December 0.00% 0.00% 0.00% 0.00% January 0.00% 0.00% 0.00% 0.00% February 9.42% 5.07% 4.35% 0.00% 135 March 0.00% 0.00% 0.00% 0.00% On a monthly basis and by state, we observe that reporting of in-kind items in Ekiti dropped during the survey period from about 38% of LGA reporting at least one item received in October, to none in March. In Niger, the pattern over the survey period appears to be of consistent very low reporting (less than 8% of LGA reporting an item). In terms of observations (bottom part of the table), we observe that less than 10% of the various items were reported positively received over the survey period, with less than 2% of expected observations reported as positive in Niger. The table also presents the reporting quality of these in-kind items recording, indicating for these positive reports if the required information on quantity, value and sources were present, with three, two and one information recorded (columns 2-4) . We observe that less than 3% of LGA provide a full report including quantity, value and source, for at least one item over the 6-month period. Niger has a particularly low reporting completeness quality with less than 18% of LGA reporting 2 requested information for at least one item received during the survey period, with no LGA having provided full information on an item received. Drugs (Section 1C) With regard to drugs, tracking concentrated on priority medicines for mother and pregnant women, requesting quantities, however, the format does not appear (clearly a mistake in customization) and source of the items. It focused also on priority medicines for children, again requesting a quantity and a source Table 7.8 presents the frequency and percentage of LGA reporting a positive reception of medicines during the survey period. The frequency of these reports is very low as only 12 reception of drugs were reported in Ekiti and 13 in Niger. We observe that reported drug reception Ekiti is concentrated in December with with more than 2/3 of the LGA reporting a reception and almost none for the other months. In Niger, recorded drug reception is more spread over the period. 136 Table 7.8: LGA Completeness: Drug reception SECTION 1C Number Drugs of cases % % of the LGAs that declared receiving a positive quantity of an item Total 25.00 64.10% Ekiti 12.00 75.00% October 2.00 12.50% November 1.00 6.25% December 11.00 68.75% January 0.00 0.00% February 0.00 0.00% March 0.00 0.00% Niger 13.00 56.52% October 7.00 30.43% November 2.00 8.70% December 0.00 0.00% January 3.00 13.04% February 1.00 4.35% March 7.00 30.43% Budget and Expenditure LGA officers were also asked to report LGA budget and expenditure relating to health on on monthly basis for the period of the survey. Detailed reporting on various categories, including wages and salaries for LGA Health administration as well as for PHC services as well as various other budget and expenditure regarding PHC services (maintenance. good and services, etc.) were requested. Table 7.9 presents on completeness of reporting presenting the percentage of observations with missing information, as well as the number of such occurrences for budgeted and actual expenditure on health administration and PHC services overall and for both states on a monthly basis. 137 Table 7.9: LGA Completeness: Budget and Expenditure Budgeted LGA health Actual LGA health administration and PHC administration and PHC expenditures expenditures Percentage Percentage with missing Number of with missing Number of information cases information cases Total 79.38 2972 77.99 2920 Ekiti 80.73 1240 81.77 1256 October 41.41 106 43.75 112 November 42.97 110 46.88 120 December 100.00 256 100.00 256 January 100.00 256 100.00 256 February 100.00 256 100.00 256 March 100.00 256 100.00 256 Niger 78.44 1732 75.36 1664 October 31.52 116 23.91 88 November 39.13 144 28.26 104 December 100.00 368 100.00 368 January 100.00 368 100.00 368 February 100.00 368 100.00 368 March 100.00 368 100.00 368 While the survey was expected to gather data monthly on the various budgeted and actual expenditure categories, we observe a high percentage of budget and expenditure observations with missing information. Only about 20% of the expected budget information was reported overall over the 6-month survey period and about 22% for expenditure data. In both states, budget and expenditure data were collected essentially during the first two months of the survey (October and November) for about 1/3 to 2/3 of the expected data in Ekiti and in about 2/3 to ¾ of observations in Niger. For the following four months in both states, budget an expenditure data at the LGA level were not recorded. Factors explaining the incompleteness and patterns in data recording relate to potential absence of record keeping on a monthly basis at the LGA level, in particular regarding health administration and PHC services. This situation remained undetected during most of the continuous survey despite a pre-pilot and pilot phase of the instruments as well as expected verification and supervision by enumerators and supervisors. Additional data collection using retrospective yearly data from state sources was used to complete part of the missing LGA continuous information. Transfers from LGA to PHC (sections 4a 4b) LGA officers were requested in section 4a and 4b of the LGA module to report items shipped to specific PHC under their jurisdiction over the period of the survey. Under section 4A, officers were expected to record on a monthly form the total monthly value of financial and in-kind transfers to specific PHCs (9 categories of transfers were itemized, including medical supplies, equipment, drugs, etc.). Section 4B required officials to list the specific items sent to PHC under the various categories, including quantity and value. The information was to be verified by enumerators and supervisors on a monthly basis. 138 On the receiving side, the continuous survey tool requested PHCs to record all reception, financial and in- kind, received during the period and by source. Using this information, we triangulate the reported data on both sides to examine the consistency of information about resources sent and received. The matching analysis involves linking the data sets using the LGA and PHC codes, as well as the item codes that were transferred and received. Table 7.10 presents the number of cases of financial and in-kind transfers, as well as number of LGAs and PHC involved in the transfers overall, as well as by state and on a monthly basis. We observe at the LGA level that the instruments were populated with very little observations of reported transfers toward PHC under the various LGAs’ jurisdiction. Indeed, among the sampled 39 LGAs and 140 PHCs under their jurisdiction, and over the 6-month survey period, only 10 LGA-PHC transfers were recorded59 which corresponds to 0.13 % of positive value. Only four LGAs reported these 10 transfers to 7 different PHCs. All of these transfers were reported in Niger during the first 2 months of the survey while no transfer is recorded for the four other months and none in Ekiti. Using the itemized LGA form 4B, transfer information is reported only in Ekiti with 25 different types of items reported transferred between a single LGA and 3 PHCs during one month over the survey period. For all the other months as well as in Niger, no itemized transaction is recorded. Furthermore, it should be noted that the information in both states in the itemized form is inconsistent with the aggregated records reported in form 4A. At the PHC level, health officials were requested to record financial and in-kind transfers as they were received on a monthly form, indicating the date, type of items, quantity and value when available. We observe that on the receiving side, while PHC report very few reception of resources from LGA sources over the 6 months’ survey period, still they report much more frequently an arrival than the LGA report sending. Indeed, at that level, 112 types of reception were recorded among 13 PHC, reporting arriving from 9 LGAs. We triangulate the data reported on the sending and receiving sides in order to verify the consistency of information collected by both instruments and assess the quality of the tracking exercise. As observed in Table 7.10, the data reported by LGA and facility staff allows to match only 4 of the 112 transactions reported by PHCs with the 35 transfers reported by LGAs during the period. This corresponds to a matching rate of only 3.57% of the transactions which could be triangulated on the sending and receiving side. Among states, we observe that the matching rate is higher in Ekiti due especially to relatively more consistent reporting in February. During that month, the matching rate was almost 29% between the 14 transactions reported by PHCs and the 25 transactions recorded by LGAs. These matched transactions were actually the only one matched one for all the months and states in the data set. 59 Out of the 140 PHCs x 6 months x 9 categories of transfers = 7560 possible observations. 139 Table 7.10: Matching of Reported Transfer and Reception of Resources from LGA to PHC, by state and month Matchin LGA LGA g of financia in- PHC LGA l LGA kind: receptio PHC sending Numbe financial: Numbe LGA in kind n Reception with r of Number of r of Number of Number Number of PHC cases LGA and cases PHC & LGA of cases PHC & LGA receptio Section (4a) PHC (4a) (4b) (4b) (MB1) (MB1) n Matching reported value of resources sent by LGA to PHC Facilities (module 2 sections 4a and 4b) and value declared by PHC (module1 MB1) Tota 4 LGA- 1 LGA-3 9 LGA-13 l 10 7PHC 25 PHC 112 PHC 3,57% 1 LGA-3 Ekiti 0 - 25 PHC 26 3 LGA-5 PHC 15,38% October 0 - 0 - 9 1LGA-2 PHC 0,00% Novembe r 0 - 0 - 0 - 0,00% Decembe r 0 - 0 - 3 1 LGA-1 PHC 0,00% January 0 - 0 - 0 - 0,00% 1 LGA-3 February 0 - 25 PHC 14 1 LGA-2 PHC 28,57% March 0 - 0 - 0 - 0,00% Nige 4 LGA- r 10 7PHC 0 - 86 6 LGA-8 PHC 0,00% 3 LGA-4 October 4 PHC 0 - 19 1 LGA-3 PHC 0,00% Novembe 3LGA-6 r 6 PHC 0 - 20 4 LGA-4 PHC 0,00% Decembe r 0 - 0 - 10 2 LGA-2 PHC 0,00% January 0 - 0 - 9 1 LGA-1 PHC 0,00% February 0 - 0 - 13 2 LGA-2 PHC 0,00% March 0 - 0 - 15 1 LGA-1 PHC 0,00% These patterns of over reporting of PHCs relative to LGAs, while not inconsistent with potential leakage, tends mainly to indicate the under reporting of activities and transfers at the LGA level. Furthermore, this very poor matching and tracking performance of the instruments could be explained especially by the difficulty of coding consistently between levels and modules with multitudes of in-kind items, sources and 140 destinations. More fundamentally, at least three potential factors could explain missing information and weak quality and consistency in self-reported information, i) Weak customization of the tools, rendering them difficult and costly to complete; ii) weak verification by enumerators and supervisors of self-reported data; iii) lack of incentive and willingness to provide the required information. Regarding the first potential factor, various indications tend to support this source of problems. Indeed, despite that customization of instruments took place during the institutional analysis phase as well as during the Rapid data assessment, followed by the 3 week pilot of the questionnaires, it appears that unresolved issues of customization and protocols were not corrected before the fielding of the survey. It was only after 6 months of data collection, potential verification by the enumerators and supervisors, and data analyst at the survey firm and principal, that still unresolved categorization of revenues and misunderstanding of category title were detected. This led to request of additional retrospective data collection at the state level to obtain part of the desired information. The second potential factor, the weak verification exercised by enumerators and supervisors, is also associated with the first factor of lack of customization of instruments. However, this implementation problem could not be ascertained due especially to lack of expected information on enumerators and supervisors activities during the survey period (see section 4). Information in that regard could have been used to examine potential links between frequency of visits by enumerators and supervisors with frequency and quality of reporting. Identity of the source reporting the data (among officials and enumerators) would also have been useful for such quality analysis. We have little information on the third potential factor, staff weak incentives, for poor records and a RIE on such fundamental problem would be in order. 7.4. Lessons learned and way forward In this section, we examine lessons learned from this first implementation of a new CPETS approach. We analyse in particular some of the elements that worked as anticipated, but also unanticipated factors or methodological choices that may have reduced the capacity of our new approach at reaching its objectives of (i) devising useful data collection instruments able to capture relevant information on various dimensions of resource availability and usage (ii) improve the incentive to collect the relevant information (iii) generate useful data and information on resource availability for PHC services. Various of the challenges and ultimate shortcoming faced by the CPETS are similar to the ones facing traditional PETS. However, the extended length of the survey period and the introduction of a new layer of agents (government officials) external to the survey firm with potentially weak incentives to self-recording, adds to the challenge of the approach. We review some of these challenges and shortcomings, from the design and implementation phases, and propose some recommendations for future CPETS. Objectives and context One of the main objectives of the CPETS project in Nigeria was to increase the quality of records in the health sector on dimensions of resource levels and use for PHC, as well as budgeting, output and services. This was achieved by introducing instruments that allow recording in context where such instruments were mostly unused and multiple ones existed without coordination -- and in doing so, it sought at providing hard information for a resource tracking of PHC services. Another key objective of the CPETS was to build capacity and incentives to accomplish data recording activities and in particular to promote the sustainability of these practices in particular through the institutionalization of the CPETS recording tools following the project completion. The analysis of the CPETS implementation in Nigeria presented in section 3 is inconclusive on both of these dimensions of data quality and sustainability. While greater record do exists for the period examined, 141 thanks to the continuous data collection exercise at the PHC and LGA, it is unclear how to evaluate quality, consistency and sustainability of these data. The analysis of frequency of reporting and consistency clearly show highly un-frequent transactions recorded and low levels of valued transfers and activities across PHCs. We could suspect underreporting, especially for IGR and clear mismatch between PHC and LGAs, hinting toward underreporting of sending by LGA. However, the absence of adequate benchmarking information precludes such conclusions. Furthermore, in the absence of information in that regard, it is unclear, in both states ‘facilities and LGAs, if the instruments were adopted following the completion of the project and if data availability and quality has improved. A review a few months after the end of the survey to measure tools’ utilisation and presence of records would be necessary to answer these fundamental questions. Nigeria public health system, like several other African countries, is characterized by poor governance and low accountability of expenditure and services. Efficiency and equity analysis are especially hard to perform in a context of reliance on discretionary financing and transfer toward various budget centers and services, including PHC and health facilities. Indeed, most resources, other than personnel, are not earmarked, and are allocated at the discretion of officials at various levels. PHC facilities in particular, do not have earmarked budgetary envelopes or information about expected public resources. Furthermore, health facilities are supported through in-kind transfers instead of financial and most of the non-wage resources at PHC level are generated through user fees (see chapter 5). In addition, the federal nature of the country makes it much harder to track and monitor flows toward service delivery units. The three levels of government being actives and sharing responsibilities in PHC services and none having a requirement to provide information to the others or keeping such aggregate information, the measurement of resources and results is rendered even harder given the multiplicity of uncoordinated actors and flows (see chapter 2). In such context, the new CPETS approach was facing important challenges. A key problem is the multitude of potential flows from large number of sources and for a large part non-financial. Scope and sampling of in-kind items As discussed in section 2, in presence of complex and multiple sources and non-financial transfers, PETS guidelines recommend reducing the scope of the tracking exercise to focus on a limited number of more easily trackable flows, but significant in terms of leakage risk factors (ex. Uganda education 1996). Surveys that sought a too large scope generally encountered important data quality problems (Gauthier 2006; Gauthier and Ahmed, 2010, p… See for instance Mozambique 2001, Uganda 1996). Furthermore, in presence of a multitude of in-kind items, it is recommended to track a selected sample of items (high frequency items) instead of tracking a census of non-financial transfers (ex. Chad health 2004).60 60 As recommended in Gauthier and Ahmed (2012) p. 67, “the focus of the tracking exercise should be on high - frequency items instead of high value ones, and the choice of materials (or medications) should be based on their shipment frequency in the Ministry or other level invoice list (if available). With such an approach based on frequency instead of total value, the risk inherent in choosing a rare but high-value material (e.g., X-ray machine) is not finding that material in the visited service provider simply because not all of them were able to receive it. In contrast, by choosing frequently shipped materials of small value (e.g., syringes), it is likely that a maximum number of facilities would report receiving them (Gauthier and Wane, 2008). This would give an upwardly biased percentage of facilities receiving materials from the authorities. Valuation of in-kind items should be done using a standardized price list (such as the line ministry list, if available). Resources received should be estimated relative to the share of the sample items in the global budget allocation.” 142 However, in Nigeria the most wider scope was chosen, encompassing the entire universe of items and flows from all sources, reflecting an additional objective, of measuring overall public and private resources available at the facility level, in addition of tracking resources flows. Regarding drugs for instance, instead of selecting a sample of essential drugs and medical supplies, it was chosen to track all drugs and supplies in any formats, with the rationale that the variety of formats and drugs allocated across states and districts would have required prior information on drugs and medical supplies distribution and value across states and districts. Customization of instruments These methodological choices added layers of complexity to the design and customization of the instruments. The sheer number of items, especially medical supplies and drugs in particular, with wide variety of formats, dosage and packaging, made it especially complex to codify and exhaust possibilities of potential item reception in LGAs and PHCs in each state. For the health staff and officials, it represented an important task of recording resources across this universe of items. The coding of this wide variety of items and formats were to be realized and verified during the pilot phase of the survey for PHC and LGA tools in both states. However, several coding and questionnaire design challenges were not resolved during the pilot phase of the tools and spilled over the full continuous survey period.61 It should be noted that pilot data could not be analyzed nor as customization and verification of the tools and protocols completed during and following the pilot due to capacity and time constraints. In addition, the various recording tools at the PHC level were prepared in Excel format during the questionnaire design and pilot phase, and were expected to be recorded using tablets in electronic format directly by the staff. However, when the full survey was launched, instead of electronic entry on tablets, paper versions of the tools were provided to PHC staff. This meant that for instance the various format of drugs which were customized with excel sheets and cells were not available in the single paper version provided to facilities. The rationale for the change was that it would allow preserving a paper trail. However, paper entry did not allow to account for the different formats (given that the customization was not fully present in the paper version). Furthermore, when electronic data entry was later done at the central level, coding of specific drug format and packaging was not performed adequately, which required several rounds of ex-post recoding. Furthermore, the linkages of items and drug codes between the various questionnaires, which were essential for potentially triangulating invoices, for instance from LGA level to reception declared by PHCs, was not realized during tools customization and piloting. This greatly complexified data analysis of resource tracking. In future CPETS endeavors, verified and tested customization of instruments and linkages between electronic instruments should be put forward with verification of matching realized during instrument testing and piloting of survey tools, and further verification and corrections on a continuous basis during the completed data collection and reporting exercise. Monitoring and supervision As mentioned earlier, one of the main challenge of the CPETS is its reliability on voluntary self-recording by the health personnel and government officials instead of relying, as in standard PETS, on enumerators’ data collection through records verification and interviews. In this context of additional layer of delegation 61 The survey pilot lasted 3 weeks and was realized in September 2015. 143 of data recording responsibility to government officials, M&S is rendered more complex but also much more essential to ensure data quality. In any PETS, one of the most important factor affecting data quality in standard PETS is proper supervision, monitoring and control over the survey tools and implementing teams. Monitoring and supervision of enumerators, as well as the firm responsible for data collection is an essential component of data quality, ensuring quality data and respect of protocol. CPETS demultiplies the problem and importance of M&S given that data collection for continuous instruments is done by facility staff and government officials adding a layer of required M&S in addition to the standard monitoring of enumerators themselves. Enumerators plays the role of monitors of the self recorded data and mechanisms need to be in place to ensure the efficient M&S of these two layers of agents. Furthermore, data collection being not only done once, through enumerators visits, as in standard surveys, but potentially every working day for several months or duration of the continuous survey, CPETS require continuous mechanisms of M&S that ensure adequate flow of verified information at various levels. The continuous nature of the data collection process would require to have a constant, day to day, real time supervision and analysis of the data collected and transmitted toward a central system of data analysis and verification. This system requires quality control steps and indicators to continuously monitor recorded data. Mechanisms to verify and correct missing, inaccurate or inconsistent data need to be in place and to correct practices through correction of protocols or refreshment of procedures. Capacity building and incentive mechanisms need to be built in these systems of M&S and controls. In the context of CPETS, a fully electronic data recording, monitoring and verification system would need to be in place. Indeed, a system of real time verification and detection of data entry or quality problems need to be detected as the continuous recording takes place and corrections in procedures, capacity or verification be introduced in real time. In Nigeria, paper questionnaires were used for continuous recording at the facility level as discussed instead of the electronic instruments using tablets as initially planned. Non-electronic recording during the continuous self-entry stage, in addition to introducing problems of customization of items which were to be completed electronically on a continuous basis discussed above, forbids a possibility of real time M&S, which would have been possible in the presence of a central control unit with real time observation and monitoring of activities and data entries with continuous instruments. Furthermore, as mentioned earlier, another key problem of CPETS relates to evaluating the validity of entries given the difficulty of benchmarking. In Nigeria, it was to be addressed by benchmarking self-entries in the daily cashbook, which constitutes one of the main sources of PHC revenues but which is also the most likely area prone to incentive problems of underreporting. To provide a benchmark by which voluntary recording of health staff could be compared, the survey instruments included a “Monitoring Module - Monthly Observation and Recording of Drug Sales and User fees”. Initially, the module was devised to be collected by enumerators consisting in a weekly observation and recording of drug sales and user fees. Protocols prescribed that once a week, in each PHC facility, an enumerator should conduct a direct observation and recording of cash received from drug sales and user fees during one hour for each activity. For the monitoring of drug sales, the enumerator was to complete the form as and when patients come and are served at the drug counter, recording of each patient and transaction the cash received by the facility for the sale, for each drug purchased. It was required that list of drugs and drug packaging on the form should correspond to the ones used in the facility, and was to be harmonized with the weekly cash book sheet (Module 1, section C.2). For user fees monitoring, the enumerator was to complete the form as patients pay the respective fees, recording for each patient/transaction, the enumerator the cash received for each service by the facility. To ensure consistency across modules, again the list of services on the module form was to 144 correspond to the ones provided in the facility, and was to be harmonized with the weekly cashbook sheet (Module 1, Section C.3). At the beginning of survey implementation, the two-hour weekly observation was substituted for a one-day monthly observation and recording of user fees and drug sales to be realized by enumerator in each PHC over the course of the survey. These monitoring observation modules would have allowed benchmarking the frequency and levels of self-entry cashbook forms of user fees and drug sales compiled on a voluntary basis by staff given the anticipated underreporting of revenues. Unfortunately, the expected monitoring and benchmarking instruments were ultimately not implemented. Another factor affecting M&S as well as the analysis of its potential efficiency relates to absence of observation on survey firm agents during the course of the survey fielding. Indeed, survey firm’s enumerators and supervisors’ role in M&V and other data collection activities were expected to be measured and verified throughout the survey. This was to be accomplished through the information on their various activities, including timing and duration of visits at different administrative and service delivery levels, that were to be recorded on a continuous basis in the various instruments. These included activities performed, in particular verification and potential modifications of self-reported staff entries (or completion of missing entries based on observation or primary source records). However, these verification entries and recordings were not performed by enumerators and supervisors. Similarly, for supervisors, their supervision activities were not recorded Furthermore, the survey firm itself did not keep track of its agents’ activities. Hence, no assessment of enumerators and supervisors’ role and impact, as well as simply the frequency of observation and quality of their monitoring and supervisory activities could not be performed, with potential effects on survey quality. In future CPETS, Continuous real-time quality control procedures need to be implemented. These quality control systems would take advantage of being put in place in the context of electronic data entry at each level, allowing continuous monitoring of data captured, with criterions to inform on slacks and under reporting. On a day-to-day basis, in real time, registered entries should be carefully reviewed by supervisors and the main management team for sources and primary record incompleteness or incoherence, etc. Various criterions and tests of these verifications and quality control procedures should be put in place and real time account of the results be available. Furthermore, the survey firm should be required to provide a system of continuous data verification to ensure quality control of continuous data recording as well as of health staff and enumerators’ work. In addition, it could be useful to have a percentage of facilities revisited and controlled through random spot checks. Stress test Given the added complexity of un-frequent transfers, from many various sources and levels, and benchmarking difficulties in the presence of self-reporting by health agents, an additional verification and benchmarking mechanism could take the form of a controlled shipment or “stress test”. Such mechanism could seek to test the proper recording of some invoices of goods or finances through the supply chain system, for instance a control shipment or an already planned shipment in coordination with a specific donor or administrative level. It would consist at verifying if the transfer has been received and properly recorded using the survey instruments through the supply chain and at the service delivery level. This controlled invoice analysis could be perceived as somewhat equivalent of a stress test for the facility or other levels, allowing to verify the accuracy and quality of recording. According to the test results, adjustment in tool customization, protocols, capacity building or verification mechanism wold be expected to be made. In future CPETS, it would be recommended that stress tests be performed, through some control shipments or program allocation, to verify the accuracy and quality of recording at the various levels in the 145 supply chain toward service delivery, with adjustment made in tools, protocols, capacity building and verification mechanism according to the test results. Country supervisor It should be noted that PETS guidelines highlight the essential role of a core team (principal) survey supervisor to be present in the country for the whole survey implementation and data entry period. Its role is to closely supervise survey implementation, enumerators and supervisor’s activities, including of the firm responsible for data collection itself and provide real time information and control by the core team during survey fielding. In the case of Nigeria, due to budget constraint, a core team supervisor was unfortunately not present for the duration of the survey. In the future, given the fundamental importance of continuous M&S including of the survey firm and additional layer of government officials, CPETS should be implemented with a principal’s project supervisor in the field for the duration of the survey, to ensure survey and data quality control, respect of protocol and especially quality of M& S procedures on a continuous basis. Training Health staff and government officials being actively involved in continuous data recording, training and capacity building of these first line agents need to be recognized and supported. Proper identification and participation of the facility staff and district officials that will become responsible at each data recording levels of using the instruments and inclusion in the training and capacity building activities are an essential component of successful CPETS. Indeed, there is a need to properly identify the staff that will perform the continuous data recording. Once identification is made, the choice could be made to train them together (standardisation and scale) or to decentralized the training the PHC level. In Nigeria, the first option was chosen. However, it is not clear who actually participated in the training (roles in data entry) and if it was them who performed the continuous data entry task at the LGA and PHC levels. The adequate identification and training of health staff and government officials being actively involved in continuous data recording, with continuous day-to-day support, constitute a key element of survey success. Incentives and capacity building At the core of the new approach is the potential problem of incentive driving the poor data quality observed in the health system. There was a presumption to start with that the staff were not inclined to keep records because of incentive problems (capacity, complexity of tools, self-interest, etc). To promote potential impact of CPETS along this line involves determining how the new recording tools, protocols and training helps in solving these incentive problems. It should be noted that by introducing CPETS instruments at the district and facility levels, it was expected that it would potentially create an incentive for new types of capture and leakage in the system, that is through collusion between district and facility level staff given the creation of information systems encompassing these two levels. This potentially fundamental question of new biases of incentives was not examined in this investigation but would be worth to be examined in future endeavor with such instruments. CPETS should include a review of capacity building, development and tool adoption to promote sustainability. This would involve analyzing if the PHC personnel and LGA officials have continued daily tracking and information collection following the survey period institutionalization of the tools introduction of a tool he information to the effect and to assess the quality of information generated and available. 146 Randomized Impact Evaluation It would have been useful to perform a RIE of the CPETS intervention to measure the effect on various quantifiable measure of quantity and quality of records that these new data recording instruments were allowing. While RIE experiments were put forward (see chapter 6) they did not allow measuring the base intervention but focused instead on additional external incentives (monetary and non-monetary) that could affect the quality of data recording. Furthermore, these experiments, by making use of some of enumerators and supervisors limited resources, time and energy, and with their focus on additional data collection tools, may have affected the survey quality with regard to other instruments and other dimensions. In the case of Nigeria, it would still be possible to conduct a rigorous evaluation ex-post using for instance the replacement facilities drawn at sample selection during the CPETS. Indeed, when the original CPETS sample was drawn, each of the 140 surveyed facilities were associated, in case of needs, with a replacement facility. This replacement sample, selected with a vector of similar characteristics as the facilities surveyed, could serve as the non-intervention group in the RIE. In that context, a retrospective PETS on both surveyed and this new control group (replacement) would seek to assess the impact of the continuous PETS intervention on the treated facilities vs the non-treated (replacement). The retrospective PETS would need to be realized a certain period after the end of the CPETS, for instance one year, while the tracking period would encompass the previous fiscal year as well as the period of the continuous survey. Alternatively, a future CPETS could be performed and evaluated rigorously in a country by first establishing a baseline survey on key service delivery facility characteristics in order to stratify the sample. Following the implementation of a CPETS on the randomized sub-sample of treated facilities, the intervention would then be evaluated using a retrospective PETS after some period (6 month or a year for instance) on both treated and untreated facilities. This would allow, through a diff-diff, to measure the difference in trends between treated and untreated facilities, before and after the CPETS intervention. Future CPETS should include, at the time of the survey design and implementation, a rigorous RIE of the continuous data recording intervention to evaluate the effects of the on the availability and quality of the health data and sustainability of the instruments. 7.5. Conclusion This chapter, while not benefiting from the rigorous RIE methodology, sought to be a first step trying to analyze ex-post the successes and drawbacks of this first CPETS approach, in order to highlight lessons learned and propose recommendations for future tracking survey. This analysis of the CPETS implementation in Nigeria presented is inconclusive on both dimensions of data quality and sustainability. While greater record does exist for the period examined especially at the LGA and PHC levels, thanks to the continuous data collection exercise, it is unclear how to evaluate quality, consistency and sustainability of these data. The analysis of frequency of reporting and consistency clearly show highly un-frequent transactions recorded and low levels of valued transfers and activities across PHCs. We could suspect underreporting, especially for IGR and clear mismatch between PHC and LGAs, hinting toward underreporting of sending by LGA. However, the absence of adequate benchmarking information precludes such conclusions. Furthermore, it is unclear, in both states’ facilities and LGAs, if the instruments were adopted following the completion of the project and if data availability and quality has improved. A review a few months after 147 the end of the survey to measure tools’ utilisation and presence of records would be necessary to answer these fundamental questions. Because of capacity and budget constraints, the evaluation of the new continuous data recording intervention could not be performed using a rigorous RIE at the time of the survey implementation. Still, it would still be possible in the case of Nigeria to conduct a rigorous evaluation ex-post using the replacement facilities drawn at sample selection when the original CPETS sample was selected. This replacement sample could serve as a control group to assess the impact of the CPETS intervention on the treated facilities through for instance retrospective PETS to be realized a certain period after the end of the CPETS. In the future, CPETS approach should include, at the time of the survey design and implementation, a rigorous RIE of the continuous data recording intervention to evaluate the effects of the on the availability and quality of the health data and sustainability of the instruments. 148 References for Chapter 7 Ablo, Emmanuel and Ritva Reinikka (1998) “Do Budgets Really Matter? Evidence from Public Spending on Education and Health in Uganda” Policy Research Working Paper 1926, The World Bank, Washington, D.C. Couttolenc, Bernard F. (2013) Nigeria Service delivery and resource tracking study: component 2 resource tracking, Report 1: situation analysis, October Gauthier, Bernard (2006) “PETS and QSDS in Sub-Saharan Africa: A Stocktaking Study”, Working paper #45091, The World Bank, September Gauthier, Bernard and Zafar Ahmed (2012) “Public Expenditure Tracking Surveys (PETS) and Quantitative Service Delivery Survey (QSDS) Guidebook”, PREM, The World Bank, Washington DC., January. Gauthier, Bernard and Ritva Reinikka (2013) “Methodological Approaches to the Study of Institutions and Service Delivery: PETS, QSDS and CRCS”, (with), in M. Kimenyi and O. Ajakaiye (eds) Institutions and Service delivery in Africa, African Economic Research Consortium. Gauthier, Bernard and Waly Wane (2008) “Leakage of Public Resources in the Health Sector: An Empirical Investigation of Chad”, Journal of African Economies (18): 52-83 Khemani, Stuti (2006) “Local Government Accountability for Health Service Delivery in Nigeria”, Journal of African Economies, 15: 285-312. Reinikka, Ritva and Nathanael Smith (2004) Public Expenditure Tracking Surveys in Education, International Institute for Educational Planning, UNESCO, Paris. Reinikka, Ritva and Jakob Svensson (2001) “Explaining Leakage of Public Funds” Policy Research Working Paper 2709. The World Bank, Washington, DC. 149 REFERENCES Allcott, Hunt, and Sendhil Mullainathan (2012), External Validity and Partner Selection Bias, NBER Working Paper 18373, Cambridge, MA. Arrow, Kenneth J. (1972), Gifts and Exchanges, Philosophy and Public Affairs, 1(2), 343–362. Ashraf, Nava (2013), Rx: Human Nature: How Behavioral Economics Is Promoting Better Health Around the World, Harvard Business Review, 91(4), 119–125. Ashraf, Nava, Oriana Bandiera, and Kelsey Jack (2014a), No Margin, No Mission? A Field Experiment for Public Services Delivery, Journal of Public Economics, 120, 1–17. Ashraf, Nava, Oriana Bandiera, and Scott S. Lee (2014b), Awards Unbundled: Evidence from a Natural Field Experiment, Journal of Economic Behavior & Organization, 100, 44–63. Ashraf, Nava, Oriana Bandiera, and Scott S. Lee (2016), Do-gooders and Go-getters: Career Incentives, Selection, and Performance in Public Service Delivery, Unpublished paper, London School of Economics. Banerjee, Abhijit, Dean Karlan, and Jonathan Zinman (2015), Six Randomized Evaluations of Microcredit: Introduction and Further Steps, American Economic Journal: Applied, 7:1, 1-21. Besley, Timothy J., and Maitreesh Ghatak (2008), Status Incentives, American Economic Review, 98(2), 206–211. Bold, Tessa, Mwangi Kimenyi, Germano Mwabu, Alice Ng’ang’a, and Justin Sandefur (2013), Scaling Up What Works: Experimental Evidence on External Validity in Kenyan Education, CGD Working Paper 321, Washington, DC. Chetty, Raj, Mushfiq Mobarak and Monica Singhal (2014), Increasing Tax Compliance through Social Recognition, Policy Brief. Gneezy, Uri, Stephan Meier, and Pedro Rey-Biel (2011), When and Why Incentives (Don’t) Work to Modify Behavior, Journal of Economic Perspectives, 25(4), 191–210. Kahneman, Daniel and Amos Tversky (1979), Prospect Theory: An Analysis of Decision under Risk, Econometrica, 47 (2), 263–291. Kimmel et al. (2012), Randomized trial of lottery-based incentives to improve warfarin adherence, American Heart Journal, 164(2), 268–274. Krizan, Zlatan, and Paul D. Windschitl (2007), The influence of outcome desirability on optimism. Psychological Bulletin, 133(1), 95–121. Lacetera, Nicola, Mario Macis, and Robert Slonim (2013), Economic rewards to motivate blood donations, Science, 340(6135), 927–928. Larkin, Ian (2011), Paying $30,000 for a gold star: An empirical investigation into the value of peer recognition to software salespeople, Mimeo. 150 Mathauer, Inke and Ingo Imhoff (2006), Health worker motivation in Africa: the role of non-financial incentives and human resource management tools, Human Resources for Health, 4(1), 24. Moldovanu, Benny, Aner Sela, and Xianwen Shi (2007), Contests for Status, Journal of Political Economy, 115(2), 338–63 Naritomi, Joana (2015), Consumers as Tax Auditors, Working Paper, Harvard University. Neckermann, Susanne and Bruno S. Frey (2008), Awards as incentives, SSRN Working Paper 1021208. Nelson, Robert B. (2001), Factors that encourage or inhibit the use of non-monetary recognition by U.S. managers, Dissertation, Claremont Graduate University, available at http://novascotia.ca/psc/pdf/employeeCentre/recognition/toolkit/step1/BobsPhDDissertation.pdf. Accessed July 12, 2016. Peterson, Suzanne J. and Fred Luthans (2006), The impact of financial and nonfinancial incentives on business-unit outcomes over time, Journal of applied Psychology, 91(1), 156. Ryan, Richard M., and Edward L. Deci (2000), Intrinsic and Extrinsic Motivations : Classic Definitions and New Directions, Contemporary Educational Psychology, 25(2000), 54–67. Stajkovic, Alexander D. and Fred Luthans (1997), A meta-analysis of the effects of organizational behavior modification on task performance, 1975–95, Academy of Management Journal, 40(5), 1122–1149. Stajkovic, Alexander D. and Fred Luthans (2001), Differential effects of incentive motivators on work performance, Academy of Management Journal, 44(3), 580–590. Stilwell, Barbara (2001), Health worker motivation in Zimbabwe, Unpublished paper/internal report for the Department of Organization of Health Care Delivery. Thaler, Richard and Cass R. Sunstein (2009), Nudge: Improving Decisions About Health, Wealth, and Happiness, Penguin Books, New York. Volpp Kevin G., George Loewenstein, Andrea B. Troxel, et al. (2008a), A test of financial incentives to improve warfarin adherence, BMC Health Services Research, 8(1), 272–277. Volpp Kevin G., George Loewenstein, Andrea B. Troxel, et al. (2008b), Financial incentive-based approaches for weight loss: a randomized trial, JAMA, 300, 2631–2637. World Bank (2004). Making Services Work for Poor People, World Development Report, Washington, D.C. World Bank (2015). Mind, Society and Behavior, World Development Report, Washington, D.C. 151 APPENDICES 5.10.1. Figure A1. Recognition Certificate (here showing poor performance: 1 star out of a possible 5) Question guide for interviews with facility staff, community members and project enumerators Analysing the impacts of the RT incentive interventions in Ekiti and Niger states This analysis will be conducted through a series of semi-structured interviews with various facility staff, clients, and the enumerators involved in the implementation of the intervention in both states. The focus of the interviews will be on depth rather than breadth; consequently, it is proposed that, after the manner of the diagnostic study conducted prior to the intervention, the interviews be conducted in a maximum of three LGs per state. The participating LGs will be selected in conjunction with the implementation team at Hanovia. An overarching concern of the analysis is the need to make a distinction between the idea of the intervention and the implementation of it. This will enable reflection on the sustainability of impacts beyond the 8-week intervention period and help to draw out lessons for application in other contexts. The proposed areas of questioning for facility staff are as follows: 152 1. How well did they understand the implementation of each intervention (i.e., the certificate and lottery schemes)? a. What is your level of education? b. Can you describe the idea of the incentive scheme in your own words? c. Can you describe the implementation of the incentive scheme in your own words? d. How did you introduce/explain the scheme to your staff? How well do you think they understood it? e. Did you feel the need to take any steps to sustain the interest of your staff in the scheme over the period of the intervention? How well did this work ? 2. What value do they attach to the filling of cash books in the first place? a. What was the standard practice around the filling of cash books in your facility before the incentive scheme was introduced? b. Why do you think this practice prevailed? c. What were the upsides and downsides to the prevailing practice? d. What do you think could have made the prevailing practice better? 3. Did the intervention have any effects on their interest in filling cash books and in RT in general? a. How would you compare the current level of commitment in the facility to maintaining accurate financial records to the level that existed before the lottery/certificate scheme? (Probe for OIC and other relevant staff members) b. What do you think is responsible for this difference (or the lack of it) in commitment levels? 4. Did the intervention have any effect on the behaviour of individuals around cash book keeping and RT in general? a. How would you compare current practices around bookkeeping in the facility to the practices that obtained before the lottery/certificate scheme? (Probe for OIC and other relevant staff members) b. What do you think is responsible for this difference (or the lack of it) in practice? 5. Did the intervention spur any changes in group dynamics and behaviours around cash book keeping and RT in general? a. Are the differences reported in 3 and 4 above mainly attributed to certain individuals or a team of workers? b. In the case of the latter, please give examples of how team members worked together to improve record-keeping practices in the facility. Can you link these examples to specific components of the lottery/certificate scheme? 6. Did leadership play any role in influencing group behaviour around cash book keeping, regardless of intervention? a. If the examples given above cannot be linked to components of the lottery/certificate scheme, what do you think was responsible for the team’s adherence to positive record- keeping practices? b. Do you think your team might have picked up some of these practices from you? Please explain, and give examples. 7. Was the length of the intervention sufficient for leadership to influence group cultures? a. How long does it usually take you to embed new ideas/cultures within your team? 153 b. How does the lottery/certificate scheme compare to your past experiences of trying to introduce change within the facility? Please explain using at least one concrete example of a past programme/intervention. c. Given the duration of the lottery/certificate scheme, how sustainable do you expect any improvements in RT to be? 8. How valuable/worthwhile do they think the grand prize is, and how does it compare to other potential rewards? a. How motivated were you and your team members to stay with the programme for the chance to win a smart phone? b. Which do you think mattered most to you and your team members – the journey (i.e. keeping better records on a sustained basis over time), or the prize to be obtained at the end? c. Would you have been better motivated by another type of reward? What would that be, and why ? 9. To what extent might the prospect of a material reward have doused whatever degree of intrinsic motivation they may have initially possessed? a. Would you or your team members have done anything differently if there was no prospect of a reward at the end of the intervention? b. Do you think your team is generally more or less focused on the prospect of rewards in the wake of the intervention than they used to be? 10. Are there any facilities in which the certificate scheme has clearly harnessed and honed the intrinsic motivation of staff? What conditions are present in those facilities ? a. Are there members of your staff who visibly worked harder at getting stars on the certificate than others? What do you think best explains the high levels of commitment among the former? b. How committed were these hard workers to bookkeeping prior to the introduction of the certificate scheme, relative to their peers? Do you think the scheme made a significant impact on their willingness to work? c. Please give examples of workers in whom the certificate scheme has produced the greatest change. Why do you think these workers responded to the incentive to a greater degree than others? 11. Is there a certain level of material/financial reward alongside which the certificate scheme would work even better? a. What did you treasure most about the weekly certificates? b. Is there anything that would have made the certificates even more meaningful for you? (Probe for cash and in-kind add-ons) 12. Do they find the alternate intervention (not the one applied to the facility) an attractive proposition? Why, or why not ? a. If you’d had a chance to choose between the certificate and lottery schemes at the beginning, which would you have chosen, and why? b. Have you changed your mind following the intervention? Why ? 13. What is the nature and strength of their existing ties with clients and the community at large, and has the intervention influenced the integrity of these ties in any way? a. What do you see as your responsibility to the community? 154 b. Do you think the certificate scheme has strengthened or weakened this sense of responsibility in any way? c. Has there been any shift in the community’s attitudes towards facility staff and operations since the certificate scheme was introduced? The above enquiries will be followed up with observation of the cash book in each facility, with the aim of conducting a ‘before’ and ‘after’ analysis of changes in RT behaviour on the basis of the intervention. The observation data will be triangulated with the interview data gathered from the facilities. The proposed areas of questioning for community members are as follows: 1. Did they know about the intervention, and how well did they understand it? a. What is your level of education? b. Do you hold any position in the facility or community? c. Can you describe the certificate scheme in your own words? d. How did you come to learn about it? e. What aspects of it do you not understand? f. Do you perceive any drawbacks to the scheme? 2. What was their conception of their role in successful implementation of the interventions, and how does this compare with the designers’ intentions? a. Did you follow the progress of the facility for the duration of the lottery scheme? Why, or why not? b. Did you discuss the ratings of the facility with any member of staff at any time? Did any such discussions prove to be worthwhile? c. Did you discuss the ratings of the facility with other community members? Were the discussions fed back to facility staff? d. Were you able to compare the ratings of your facility with those of other facilities? Did this spur you to take any action in your own facility? 3. In what ways, did they engage with the interventions, and what role did their engagement play in incentivising facility staff (particularly in the certificate scheme)? a. Did you take any interest in the facility’s cash book prior to the introduction of the certificate scheme? b. Did the scheme make you engage more or less with the record keeping practices in the facility? c. Did your engagement have any impact on the seriousness and precision with which facility staff attended to the records? d. Are there any factors outside of the intervention that have contributed to changes in staff attitudes towards record keeping? 4. How did the placement of the certificates within the facilities affect their ability and willingness to engage with the intervention? a. Where were the certificates displayed in the facility? b. Did you think this was a good place to display them? Why, or why not? c. Would the certificates have seemed more prestigious to you if they had been displayed elsewhere in the facility? 155 5. How do they perceive their relationship with facility staff, and what impact does that have on the way the intervention played out? Conversely, what effects have the intervention had on this relationship? a. What do you see as your responsibility to the community? b. Do you think the certificate scheme has strengthened or weakened this sense of responsibility in any way? c. Has there been any shift in the community’s attitudes towards facility staff and operations since the certificate scheme was introduced? 6. Do they make a link between the ratings accorded to the facility (whether in the form of certificates or lottery tokens) and improved service delivery to the community? How important do they think this is, compared with other aspects of their relationship with facility staff? a. Do you think the facility’s ratings have been a true reflection of its record-keeping performance over the past couple of months? b. Are you concerned about the facility’s performance in this regard? Why, or why not? c. What are the most important expectations you have of the facility and its staff? Do you see any links between these expectations and the facility’s record-keeping practices? The proposed areas of questioning for the enumerators are as follows: 1. How much did you feel you had to depart from the ‘script’ and improvise during implementation of the intervention? (This may help to tease out some of the assumptions that informed the design of the intervention.) 2. How does the reaction of facility staff in the certificate scheme compare with the reaction of those in the lottery scheme, both in the initial learning phase and over the period of the intervention? Which of the interventions do you think had more traction among facility staff? 3. Did you observe other issues (e.g., non-payment of salaries) that may have influenced the motivation/behaviour of facility staff with respect to cash book filling and RT in general? How relevant are these issues across the facilities, and to what extent do they promote or hamper the objectives of the intervention? 4. If there were more time and resources to run the intervention, what additional components would you propose, and why? 156