GPR 67281 v1 THE GLOBAL FUND TO FIGHT AIDS, TUBERCULOSIS AND MALARIA, AND THE WORLD BANK’S ENGAGEMENT WITH THE GLOBAL FUND —VOLUME 1: MAIN REPORT— GLOBAL PROGRAM REVIEW Volume 6 Issue 1 THE WORLD BANK GROUP WORKING FOR A WORLD FREE OF POVERTY The World Bank Group consists of five institutions—the International Bank for Reconstruction and Development (IBRD), the International Finance Corporation (IFC), the International Development Association (IDA), the Multilateral Investment Guarantee Agency (MIGA), and the International Centre for the Settlement of Investment Disputes (ICSID). Its mission is to fight poverty for lasting results and to help people help themselves and their envi- ronment by providing resources, sharing knowledge, building capacity, and forging partnerships in the public and private sectors. THE INDEPENDENT EVALUATION GROUP IMPROVING DEVELOPMENT RESULTS THROUGH EXCELLENCE IN EVALUATION The Independent Evaluation Group (IEG) is an independent, three-part unit within the World Bank Group. IEG-World Bank is charged with evaluating the activities of the IBRD (The World Bank) and IDA, IEG-IFC focuses on assessment of IFC’s work toward private sector development, and IEG-MIGA evaluates the contributions of MIGA guarantee projects and services. IEG reports directly to the Bank’s Board of Directors through the Director-General, Evaluation. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank Group’s work, and to provide accountability in the achievement of its objectives. It also improves Bank Group work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. Global Program Review The Global Fund to Fight AIDS, Tuberculosis and Malaria, and the World Bank’s Engagement with the Global Fund Volume 1: Main Report February 8, 2011 Public Sector Evaluation http://www.globalevaluations.org ©2012 Independent Evaluation Group, The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-458-4497 Internet: http://www.globalevaluations.org E-mail: ieg@worldbank.org All rights reserved This volume is a product of the staff of the Independent Evaluation Group (IEG) of the World Bank Group. It is part of an ongoing series that reviews global and regional partnership programs in which the World Bank is engaged as one of the partners. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. IEG does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of IEG concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. IEG encourages the dissemination of its work and permits these reviews to be copied or otherwise transmitted, with appropriate credit given to IEG as the authoring body. How to cite this report: IEG (Independent Evaluation Group). 2012. “The Global Fund to Fight AIDS, Tuberculosis and Malaria, and the World Bank’s Engagement with the Global Fund: Electronic Survey Administered to World Bank Task Team Leaders and Global Fund Secretariat in March 2011– Volume 1: Main Report.� Cover photo: Children stand in a circle at a day school facility in Richards Bay, South Africa. The school is for children who have lost their parents to AIDS or have been affected in some way by HIV. Photo by Brent Stirton, courtesy of Getty Images. ISBN-13:978-1-60244-201-6 ISBN-10: 1-60244-201-0 Independent Evaluation Group Strategy, Communication, and Learning (IEGCS) E-mail: ieg@worldbank.org Telephone: 202-458-4497 Printed on Recycled Paper IEG Mission: Improving Development Results Through Excellence in Evaluation The Independent Evaluation Group (IEG) of the World Bank annually reviews a number of global and regional partnership programs (GRPPs) in which the Bank is a partner, in accordance with a mandate from the Bank’s Executive Board in September 2004. The three main purposes are (a) to help improve the relevance and effectiveness of the programs being reviewed, (b) to identify and disseminate lessons of broader application to other programs, and (c) to contribute to the development of standards, guidelines, and good practices for evaluating GRPPs. IEG does not, as a matter a policy, recommend the continuation or discontinuation of any programs being reviewed. A global or regional program review (GPR) is a review and not a full-fledged evaluation. The preparation of a GPR is contingent on a recently completed evaluation of the program, typically commissioned by the governing body of the program. Each GPR assesses the independence and quality of that evaluation; provides a second opinion on the effectiveness of the program, based on the evaluation; assesses the performance of the World Bank as a partner in the program; and draws lessons for the Bank’s engagement in GRPPs more generally. The GPR does not formally rate the various attributes of the program. Assessing the independence and quality of GRPP evaluations is an important aspect of GPRs in order to foster high-quality evaluation methodology and practice more uniformly across Bank-supported GRPPs. Providing a “second opinion� on the effectiveness of the program includes validating the major findings of the GRPP evaluation. Assessing the performance of the World Bank as a partner in the program provides accountability to the Bank’s Executive Board. In selecting programs for review, preference is given to (a) those that are innovative, large or complex; (b) those in which the Bank is sufficiently engaged to warrant a GPR, (c) those that are relevant to upcoming IEG sector studies; (d) those for which the Executive Directors or Bank management have requested reviews; and (e) those that are likely to generate important lessons. IEG also aims for a representative distribution of GPRs across sectors in each fiscal year. A GPR seeks to add value to the program and to the World Bank beyond what is contained in the external evaluation, while also drawing upon IEG’s experience in reviewing a growing number of programs. It reports on key program developments since the evaluation was completed, including the progress in implementing the recommendations of the evaluation. A GPR involves a desk review of key documents, consultations with key stakeholders, and a mission to the program management unit (secretariat) of the program if this is located outside the World Bank or Washington, DC. Key stakeholders include the Bank’s representative on the governing body of the program, the Bank’s task team leader (if separate from the Bank’s representative), the program chair, the head of the secretariat, other program partners (at the governance and implementing levels), and other Bank operational staff involved with the program. The writer of a GPR may also consult with the person(s) who conducted the evaluation of the GRPP. Each GPR is subject to internal and external peer review and IEG management approval. Once cleared internally, the GPR is reviewed by the responsible Bank department and the secretariat of the program being reviewed. Comments received are taken into account in finalizing the document, and the formal management response from the program is attached to the final report. After the document has been distributed to the Bank’s Board of Executive Directors, it is disclosed to the public on IEG’s external Web site. v Contents Abbreviations and Acronyms .................................................................................................. ix  Acknowledgments.................................................................................................................... xi  Foreword ................................................................................................................................ xiii  Program at a Glance: The Global Fund to Fight AIDS, Tuberculosis and Malaria................ xv  Key Bank Staff Responsible during Period under Review ................................................... xvii  Program Manager.................................................................................................................. xvii  Glossary ................................................................................................................................. xix  Summary .............................................................................................................................. xxiii  Secretariat Management Comments: The Global Fund .......................................................... xli  Management Comments: The World Bank Group ................................................................ xlv  Chairperson’s Summary: Committee on Development Effectiveness................................. xlvii  1.  Introduction, Purpose, and Methodology .......................................................................... 1  Organization of the Review ............................................................................................... 1  Methodology ..................................................................................................................... 2  2.  Overview of the Global Fund ............................................................................................ 5  Origin of the Global Fund ................................................................................................. 5  Objectives and Design ....................................................................................................... 7  Governance and Management ......................................................................................... 11  Financing ......................................................................................................................... 14  World Bank Engagement with the Global Fund ............................................................. 20  The Five-Year Evaluation of the Global Fund ................................................................ 21  3.  Validating the Major Findings of the Five-Year Evaluation ........................................... 27  Additionality, Predictability, and Sustainability of Global Fund Support ...................... 30  Performance of Country Coordinating Mechanisms ....................................................... 36  Effectiveness of Country-level Partnerships ................................................................... 42  Application of Performance-Based Funding ................................................................... 46  Access and Coverage of Service Delivery ...................................................................... 51  Equity in Country-Level Governance and Grant Objectives .......................................... 53  Impact of Donor Support for the Three Diseases on Domestic Health Systems............. 54  Institutional Risk Management by the Global Fund ....................................................... 57  4.  The World Bank’s Engagement with the Global Fund at the Global and Country Levels ...................................................................................................................................... 62  The Bank’s Involvement in Global Health Partnerships Prior to the Establishment of the Global Fund ........................................................................................................... 62  The Bank’s Roles in the Global Fund at the Corporate Level ........................................ 64  vi The Global HIV/AIDS Program, the International Health Partnership, and Related Initiatives ......................................................................................................................... 67  The Bank’s Country Programs in the Health Sector ....................................................... 74  The Extent and Nature of Bank’s Engagement with the Global Fund at the Country Level ................................................................................................................................ 77  Factors Facilitating and Hindering Effective Engagement ............................................. 89  Prospects for Future Engagement at the Country Level.................................................. 93  5.  Lessons from the Five-Year Evaluation for the Evaluation of Global Partnership Programs ................................................................................................................................. 98  Oversight and Management of the Evaluation ................................................................ 98  Participation, Transparency, and Dissemination ........................................................... 102  Study Areas 1, 2, and 3: Their Evaluation Approaches, Methodology, and Instruments .................................................................................................................... 104  Evaluation Capacity Building in Study Area 3 ............................................................. 113  6.  Conclusions and Lessons............................................................................................... 115  Lessons for the Global Fund ......................................................................................... 115  Lessons for the World Bank .......................................................................................... 117  Lessons for the Evaluation of Global and Regional Partnership Programs .................. 118  Bibliography ......................................................................................................................... 121  Boxes Box 1. Global Fund Guiding Principles .................................................................................... 7  Box 2. Changes to the Global Fund Grant Architecture ......................................................... 25  Box 3. Commercial Private Sector Participation in the Four Low-Income Countries ........... 45  Box 4. Performance-Based Funding in the Global Fund ........................................................ 47  Box 5. Findings from the Second Independent Evaluation of UNAIDS ................................ 71  Box 6. Key Recommendations and World Bank Commitments Relating to Engagement with the Global Fund ............................................................................. 77  Figures Figure 1. Global Fund: Grant Preparation and Approval Processes ......................................... 8 Figure 2. The Global Fund: Grant Implementation Processes ................................................ 10 Figure 3. The Global Fund: Organizational Chart, June 2011 ................................................ 12 Figure 4. Global Fund: Top 15 Donors (as of December 2010) ............................................. 14 Figure 5. Global Fund Grants and World Bank Health Projects, Fiscal Years 2000–11 ....... 17 Figure 6. To what extent have the findings and recommendations of the Five-Year Evaluation had the following impacts on the Global Fund? ....................................... 26 Figure 7. Official Development Assistance and Other Official Flows from OECD/DAC Member Countries and Multilateral Agencies to Developing Countries.................... 32 Figure 8. U.S. Contributions to the Global Fund and the Larger Global Effort ..................... 35 Figure 9. Overall, how would you best characterize the relationship between the World Bank vii and the Global Fund during the years that you were working on this country? ......... 78 Figure 10. In which of the following country-level processes of the Global Fund did World Bank staff or consultants participate during the years that you were working on this country? ............................................................................................ 84 Figure 11. In what other ways were World Bank staff or consultants involved in country- level activities that directly or indirectly contributed to the work of the Global Fund during the years that you were working on this country? .................................. 85 Figure 12. To what extent do you consider the World Bank to be a partner of the Global Fund (a) at the global level and (b) at the country level? ........................................... 86 Figure 13. World Bank Project Managers: In your opinion, do the following factors make it easier or more difficult for World Bank staff or consultants to engage with Global Fund-supported activities at the country level? ...................................... 90 Figure 14. Global Fund: In your opinion, do the following factors make it easier or more difficult for Global Fund managers, staff or agents to engage with the World Bank at the country level? .................................................................................................... 91 Figure 15. What changes would you like to see in the Global Fund and the World Bank to facilitate greater engagement between the two organizations to achieve positive results at the country level, while also respecting each organization’s fundamental purposes and principles? ............................................................................................. 95 Figure 16. Evaluation Framework for Study Area 3............................................................. 111 Tables Table 1. Success Rate of Proposals Approved for Grant Funding ........................................... 9  Table 2. Global Fund: Annual Income and Expenditures, Calendar Years 2002–10 ............. 16  Table 3. Global Fund and World Bank Health Commitments and Disbursements, by Disease/Theme, Fiscal Years 2003–11 ..................................................................... 18  Table 4. Global Fund and World Bank Health Sector Commitments and Disbursements, by Region, Fiscal Years 2003–11.............................................................................. 19  Table 5. Epidemiological Profile of the Six Countries Visited by IEG .................................. 28  Table 6. Global Fund Grants by Disease in the Six Countries Visited by IEG, 2002–2011 ..... 30  Table 7. Disbursement Pattern of Approved Global Grants, by Region and by Disease, 2003–10 ..................................................................................................................... 34  Table 8. Country Coordinating Mechanisms in the Six Countries Visited by IEG, 2010 ...... 37  Table 9. Six Countries: Grant Proposals Submitted and Approved, Rounds 1–10 ................ 39  Table 10. Multicountry AIDS Program Projects, by Region and Approval Year .................. 64  Table 11. Contributions to and Disbursements from UNAIDS and GAVI Trust Funds at the World Bank (US$ millions) ............................................................................. 68  Table 12. World Bank Communicable Disease Projects and Commitments, Fiscal Years 1997–2011.................................................................................................................. 75  Table 13. References to the Global Fund and the Fast Track Initiative in Country Assistance Strategies and Project Appraisal Documents, Fiscal Years 2003–10 ....................... 79  Table 14. Comparative Advantages of the World Bank and the Global Fund: Each Organization’s Perspectives of the Other Organization ............................................ 94  Table 15. Five-Year Evaluation Timeline .............................................................................. 99  Table 16. Toward a Common Conceptual Framework for Assessing Country-Level Partnerships ............................................................................................................. 109  viii Volume 2: Appendixes Appendix A. Review Framework for This GPR of the Global Fund Appendix B. Timeline of the Global Fund and Related Events in the World Bank and Elsewhere Appendix C. Global Fund: Purpose, Principles, and Results Chain Appendix D. Global Fund: Core Structures Appendix E. Members of the Global Fund Board Appendix F. Global Fund: Sources and Uses of Funds Appendix G. Global Fund Five-Year Evaluation: Major Findings, Recommendations, and Program Response Appendix H. Global Fund and World Bank Assistance to the Six Countries Visited Appendix I. Major Findings from the Six Country Visits Appendix J. World Bank Participation at Global Fund Board Meetings, January 2002 to December 2010 Appendix K. World Bank Involvement in Global Health Partnerships and Financial Intermediary Trust Funds Appendix L. Overview of the Global Environment Facility and the World Bank’s Roles Appendix M. The World Bank’s Country Programs in the Health Sector Appendix N. IEG Assessment of the Independence and Quality of the Five-Year Evaluation Appendix O. Toward a Common Conceptual Framework for Assessing Country-Level Partnerships Appendix P. Quality Review of Study Area 3 of the Five-Year Evaluation Appendix Q. Results of the Electronic Survey of World Bank Task Team Leaders and Global Fund Secretariat Staff Appendix R. Persons Consulted ix Abbreviations and Acronyms ACT Artemisinin combination therapy AIDS Acquired immunodeficiency syndrome AMFm Affordable Medicines Facility for Malaria ART Antiretroviral therapy or treatment ARV Antiretroviral drug ASAP AIDS Strategy and Action Plan Service (UNAIDS and World Bank) CAS Country Assistance Strategy (World Bank) CCM Country Coordinating Mechanism (Global Fund) CFP Concessional Finance and Global Partnerships Vice Presidency (World Bank) CHAT Country Harmonization and Alignment Tool (UNAIDS) CPA Country Partnership Assessment CSO Civil society organization DAC Development Assistance Committee (OECD) DFID Department for International Development (United Kingdom) DGF Development Grant Facility (World Bank) DOTS Directly Observed Treatment Short-Course (for tuberculosis) FIF Financial Intermediary Trust Fund (World Bank) FPM Fund Portfolio Manager (Global Fund) FRM Financial Resource Mobilization Department of CFP (now called IDA Resource Mobilization Department: CFPIR) FYE Five-Year Evaluation of the Global Fund GAMET Global HIV/AIDS Monitoring and Evaluation Support Team GAVI Global Alliance for Vaccines and Immunization (a global partnership program) GEF Global Environment Facility (a global partnership program) GHAP Global HIV/AIDS Program (World Bank and UNAIDS) GIST Global Implementation Support Team GMP Good Manufacturing Practice GPR Global or Regional Program Review (IEG) GRPP Global and/or regional partnership program HIV Human immunodeficiency virus HNP Health, nutrition and population HSS Health systems strengthening IBRD International Bank for Reconstruction and Development ICR Implementation Completion Report (World Bank) IDA International Development Association IDU Injecting drug user IEG Independent Evaluation Group, formerly OED (World Bank) IETF Impact Evaluation Task Forces IHP International Health Partnership IHP+ International Health Partnership and Related Activities JANS Joint Assessment of National Strategies (a component of IHP+) LFA Local Fund Agent (Global Fund) Logframe Logical framework M&E Monitoring and evaluation MAP Multi-country AIDS Program (World Bank) MDGs Millennium Development Goals MOU Memorandum of understanding NGO Nongovernmental organization NSA National Strategy Application (Global Fund) OECD Organisation for Economic Co-operation and Development OIG Office of the Inspector General (Global Fund) PBF Performance-Based Funding (Global Fund) PEPFAR President’s Emergency Plan for AIDS Relief (United States) x PMTCT Prevention of Mother to Child Transmission of HIV PRSP Poverty Reduction Strategy Paper PUDR Progress Update and Disbursement Request (Global Fund) RBM Roll Back Malaria (a global partnership program) SSF Single Stream of Funding (Global Fund) Stop TB Stop Tuberculosis Partnership (a global partnership program) SWAp Sector-Wide Approach TB Tuberculosis TERG Technical Evaluation Reference Group (Global Fund) TRP Technical Review Panel (Global Fund) UNAIDS Joint United Nations Program on HIV/AIDS UNFPA United Nations Population Fund UNICEF United Nations Children’s Fund UNDP United Nations Development Programme USAID United States Agency for International Development WHO World Health Organization Fiscal Year of the Global Fund January 1 – December 31 xi Acknowledgments This Review has been prepared by a team led by Chris Gerrard under the direction of Mark Sundberg, Manager, IEG Public Sector Evaluations, following the outline and methodology in the Approach Paper that was issued in January 2010. Other members of the core team were Anna Aghumian, Elaine Wee-Ling Ooi, Rogerio Pinto, and Bowen Uhlenkamp. Chris Gerrard and Elaine Wee-Ling Ooi were the principal authors of the main report. The six country reports were prepared by Ed Burger (on the Russian Federation), Julia Dayton (on Burkina Faso), the late Phil Musgrove (on Nepal), Elaine Wee-Ling Ooi (on Cambodia), and Rogerio Pinto (on Brazil and Tanzania). Judy Twigg prepared an in-depth assessment of a World Bank-supported health project in Lesotho that was specifically designed to increase the capacity of the country to effectively use Global Fund grants for HIV/AIDS. Cheryl Cashin prepared the detailed comparison of the monitoring and evaluation systems of the Global Fund and the World Bank, and Andaleeb Alam prepared an analysis of Study Area 3 of the Five- Year Evaluation on the impact of the collective efforts to reduce the burden of the three diseases. Martha Ainsworth, Ed Elmendorf, Bernhard Liese, and Pia Schneider have provided invaluable guidance to the team throughout. William Hurlbut, Caroline McEuen, and Margarita Sanchez have provided editorial and other support. None of the people who worked on this Review have previously done any work for the Global Fund. IEG gratefully acknowledges all those who made their time available for interviews and provided useful information and insights that made this Review possible. IEG especially wishes to acknowledge the cooperation of Global Fund staff in providing all requested information and documents, and World Bank staff in the Bank’s country offices for facilitating the visits to Brazil, Burkina Faso, Cambodia, Lesotho, Nepal, the Russian Federation, and Tanzania. The list of persons consulted is contained in Appendix R. IEG would especially like to acknowledge Emily Bosch in the Development Co-operation Directorate of the Organisation for Economic Co-operation and Development for her extensive assistance in gathering the information necessary to undertake the additionality analysis in Chapter 3. The draft report was peer-reviewed by Ruth Levine, Director, Global Development and Population Program in the Hewlett Foundation, and Charles Griffin, Senior Adviser in the Europe and Central Asia Vice Presidency of the World Bank. The Norwegian Agency for Development Cooperation contributed financial support to part of this Review. Caroline Heider, Director-General, Evaluation Emmanuel Jimenez, Director, Public Sector Evaluation Mark Sundberg, Manager, Public Sector Evaluation Chris Gerrard, Task Team Leader xii xiii Foreword The Global Fund to Fight AIDS, Tuberculosis and Malaria was founded in January 2002 to mobilize large-scale donor resources for the specific purpose of reducing infections, illness and death caused by the three diseases. Since then, the Global Fund has become the largest of the 120 global and regional partnership programs in which the World Bank is involved. It disbursed more than $3 billion in grants to developing and transition countries in 2010 to finance investments at the country level, and is supported by the largest financial intermediary trust fund administered by the World Bank. The Independent Evaluation Group annually reviews a number of global and regional partnership programs in which the World Bank is involved. This Global Program Review is based on the Five-Year Evaluation of the Global Fund that was completed in May 2009, and it focuses on the World Bank’s engagement with the Global Fund at the global and country levels. Its principal purpose has been to learn lessons from the experience of the Global Fund about (a) the design and operation of large global partnership programs that are financing country-level investments, (b) the engagement of the World Bank with such programs, and (c) the evaluation of these programs. This Review found that the Five-Year Evaluation — consisting of three Study Areas and a Synthesis Report — was an independent and quality evaluation. Study Area 1, on the organizational efficiency and effectiveness of the Global Fund, and Study Area 2, on its partner environment at the global and country levels, were formative evaluations that have had major impacts on the Global Fund’s organizational and institutional arrangements. Study Area 3 was a summative evaluation of the collective efforts to reduce the burden of the three diseases; it could not, by design, assess the independent contribution of the Global Fund to country-level results. Official donor commitments to combat the three diseases have increased more than sixfold, from $1.7 billion in 2002 to $11.4 billion in 2009, of which almost 40 percent now flows through the Global Fund. Country Coordinating Mechanisms have successfully brought country-level stakeholders together to submit grant proposals to the Global Fund, but have lacked the authority and the resources to exercise effective oversight of grant implementation. Collective donor efforts have contributed to increased availability and use of disease-control services, particularly for HIV/AIDS, and increased coverage of affected communities, which should ultimately reduce the disease burden. However, reliance on external funds and inadequate investments in long-term capacity raise concerns about the sustainability of recipient countries’ disease-control programs. If external support is not sustained, this will put pressure on governments in recipient countries to reallocate their own budgetary resources to costly treatment activities, and away from other health and non-health priorities. To the extent that resources spent on prevention of new infections decline, the long-term sustainability of treatment programs will be further undermined. xiv The World Bank has had extensive engagement with the Global Fund at the global level through the Global HIV/AIDS Program, the International Health Partnership, and related initiatives. The Bank has also had some degree of engagement with the Global Fund — from information sharing to active collaboration — in about three-quarters of the 90 countries in which both organizations have been active since 2002. This Review found that the situation has improved since the Five-Year Evaluation in terms of the World Bank and other partners’ providing technical assistance in support of Global Fund activities. There is a need to define these technical support functions with greater clarity and formality within the context of improved donor harmonization. This Review found that country-level stakeholders still tend to regard the Global Fund as another, largely separate, development partner agency with its own distinct modalities that have not been well integrated into existing donor coordination mechanisms. The Global Fund, the World Bank, and other multilateral organizations have expressed good intentions to coordinate and streamline monitoring and evaluation (M&E) processes at the country level, but this has been difficult to achieve in practice. The organizations have different requirements for project-level M&E — in the case of the Global Fund, to facilitate its performance-based funding approach to grant disbursements. This Review found tensions between these two imperatives in the Global Fund, deficiencies in the application of performance-based funding in three of the six countries visited, and no contribution of the program’s grant-level M&E to the summative assessment in the Five-Year Evaluation. Both the Global Fund and the World Bank could contribute to improved M&E at the project and country levels by making a stronger commitment to the “Three Ones� principles of a common action framework, a single coordinating authority, and one M&E framework to monitor collective efforts in each disease area. Project-level M&E could focus on accountability for achieving the specific outputs of each project, and country-level M&E on tracking the higher-level outcomes and impacts collectively. To build the knowledge base about which approaches most successfully contribute to achieving collective outcomes, the Global Fund could also consider undertaking evaluations of a random sample of the single streams of funding now taking place under its “new grant architecture� and institutionalizing regular country-level evaluations, both of which could feed into subsequent evaluations of the overall program. Caroline Heider Director-General, Evaluation xv Program at a Glance: The Global Fund to Fight AIDS, Tuberculosis and Malaria Start date January 2002. The Board of the Global Fund met in Geneva for the first time. The Global Fund was registered with Swiss legal authorities, and its by-laws were adopted. Purpose The purpose of the Fund is to attract, manage, and disburse additional resources through a new public-private partnership that will make a sustainable and significant contribution to the reduction of infections, illness, and death, thereby mitigating the impact caused by HIV/AIDS, tuberculosis, and malaria in countries in need, and contributing to poverty reduction as part of the Millennium Development Goals. Principles A. The Fund is a financial instrument, not an implementing entity. B. The Fund will make available and leverage additional financial resources to combat HIV/AIDS, tuberculosis, and malaria. C. The Fund will base its work on programs that reflect national ownership and respect country-led formulation and implementation processes. D. The Fund will seek to operate in a balanced manner in terms of different regions, diseases, and interventions. E. The Fund will pursue an integrated and balanced approach covering prevention, treatment, and care and support in dealing with the three diseases. F. The Fund will evaluate proposals through independent review processes based on the most appropriate scientific and technical standards that take into account local realities and priorities. G. The Fund will seek to establish a simplified, rapid, innovative process with efficient and effective disbursement mechanisms, minimizing transaction costs and operating in a transparent and accountable manner, based on clearly defined responsibilities. The Fund should make use of existing international mechanisms and health plans. Major activities Global Fund grants are used to support a range of a activities, including:  Pharmaceuticals, medical commodities and diagnostics, and insecticide- treated bed nets  Surveillance studies and surveys  Technical assistance to build capacity  Actual service delivery provision  Salaries. Grants target the three diseases, plus strengthening of underlying cross- cutting health systems, such as procurement, supply management, human resources, and health information systems. World Bank Group The World Bank is the limited trustee of the Global Fund trust fund, a contributions nonvoting ex-officio member of the Board, and a development partner at the global and country levels. The Bank does not contribute financial resources to the trust fund, but has engaged with Global Fund-supported activities in about 65 of the 90 countries in which both organizations have been active in the health sector since 2002. The nature of this engagement has ranged from sharing information about each organization’s activities to active collaboration, including serving on the Country Coordinating Mechanism in about 20 countries and joint supervision missions. xvi Other donor More than 50 public and private sector donors contributed US$18.8 billion to contributions the Global Fund trust fund through December 31, 2010. The six largest donors (the United States, France, Japan, Germany, United Kingdom, and the European Commission) contributed two-thirds of these resources. Location The Global Fund Secretariat is located in Geneva, Switzerland. Web site www.theglobalfund.org Governance and The Global Fund is an independent legal entity incorporated as a foundation management under Swiss law. The Global Fund is governed by a constituency-based Board comprising eight representatives of donor governments (including the European Commission), seven representatives of recipient governments, and one representative each from private foundations, affected communities, developed country nongovernmental organizations (NGOs), developing country NGOs, and the commercial private sector. The Board also has six nonvoting ex officio members: the Global Fund Executive Director, UNAIDS, the World Health Organization (WHO), the World Bank, other development partners (currently represented by the Stop Tuberculosis Partnership), and Switzerland. The Global Fund had an administrative services agreement with WHO from 2002 to 2008 under which WHO provided a range of administrative and financial services, including human resources, finance, administration, procurement, and information technology services. The Global Fund became an administratively autonomous organization, effective January 1, 2009. WHO continues to act as a technical partner in many Global Fund recipient countries. Latest program-level Macro International, The Five-Year Evaluation of the Global Fund to Fight evaluation AIDS, Tuberculosis, and Malaria: Synthesis of Study Areas 1, 2 and 3, March 2009. Macro International was also the lead contractor for each of the three study areas, as follows:  Study Area 1: Organizational Effectiveness and Efficiency of the Global Fund, October 2007  Study Area 2: The Global Fund Partner Environment, at Global and Country Levels, in Relation to Grant Performance and Health System Effects, Including 16 Country Studies, June 2008  Study Area 3: The Impact of Collective Efforts on the Reduction of the Disease Burden of AIDS, Tuberculosis and Malaria, May 2009 xvii Key Bank Staff Responsible during Period under Review Position Person Period Global Program Task Team Ivar Andersen, Sr. Operations 2002–2003 Leader Officer, FRM Keith Jay, Lead Policy Analyst, 2004–2006 FRM Alice Miller, CFPMI 2007–2009 David Crush, Sr. Financial Sept 2009 – Dec 2010 Officer, CFPMI Alexandru Cebotari, Financial Jan 2011 – Mar 2011 Officer, CFPMI Veronique Bishop, Sr. Financial April 2011 – present Officer, CFPMI Bank’s Representative on the Geoffrey Lamb, Director, RMC Jan 2002 – April 2003 Governing Body Geoffrey Lamb, Vice President, Apr 2003 – May 2006 CFP Philippe Le Hourerou, Vice May 2006 – August 2009 President, CFP Axel van Trotsenburg, Vice August 2009 – present President, CFP Vice President, Human Eduardo Doryan 1999–2001 Development Network (HDN) Jozef Ritzen 2001–2003 Jean-Louis Sarbib 2003–2006 Joy Phumaphi 2007–2010 Tamar Atinc June 2010 – present Sector Director, Health, Christopher Lovelace 1999–2002 Nutrition, and Population Jacques Baudouy 2003–2007 Department (HDNHE) Cristian Baeza (Acting) 2007 Julian Schweitzer 2007–2010 Cristian Baeza 2010 – present Director, CFPMI Susan McAdams July 2007 – present Director, Global Programs and Margret Thalwitz May 2004 – September 2008 Partnerships Group (GPP) Director, Global Partnerships and Junhui Wu March 2009 – present Trust Fund Operations (CFPTO) Note: CFP = Concessional Finance and Global Partnerships Vice Presidency; CFPMI = Multilateral Trusteeship and Innovative Financing Department; FRM = Resource Mobilization Department; RMC = Resource Mobilization and Concessional Financing Department. Program Manager Position Person Period Executive Director Richard Feachem July 2002 – April 2007 Executive Director Michel Kazatchkine April 2007 – present xviii xix Glossary Antiretroviral therapy The use of at least three antiretroviral (ARV) drugs to maximally (ART) suppress the HIV virus and stop the progression of HIV disease. Artemisinin combination An approach to malaria treatment that combines several drugs, therapy (ACT) including drugs based on an ancient Chinese medicinal plant known as artemisinin. ACT treatment is gradually becoming the treatment of choice under many African countries’ drug and treatment protocols. ACTs are much more expensive than current standard treatments that have lost their potency. Concentrated epidemic In the case of HIV/AIDS, the epidemic is concentrated when infection levels have risen substantially among those who practice high-risk behavior, but have yet to rise in the general and much larger low-risk population. DOTS Directly Observed Treatment Short-Course — the basic treatment package for tuberculosis that is recommended by WHO and underpins the Global Plan to Stop Tuberculosis. DOTS-Plus The adaptation of DOTS to respond to multidrug-resistant tuberculosis by adding second-line drugs. Drug A substance intended for use in the diagnosis, cure, mitigation, treatment, or prevention of a disease. Extrapulmonary TB Tuberculosis affecting a part of the body other than the lungs. Generic drugs Non-proprietary pharmaceutical products. Genome All of the genetic information, the entire genetic complement, and all of the hereditary material possessed by an organism. Global Drug Facility A mechanism (facility) established as an initiative of the Stop Tuberculosis Partnership to expand access to, and availability of, high- quality tuberculosis drugs to facilitate global DOTS expansion. Green Light Committee A committee established by the Stop Tuberculosis Partnership that provides technical policy and procedural support for drug-resistant tuberculosis to WHO and its members. It facilitates procurement of quality-controlled, affordable second-line anti-tuberculosis drugs. Generalized epidemic In the case of HIV/AIDS, the epidemic is generalized when HIV has moved out of populations with high-risk behavior and has substantially infected the low-risk population. Heath systems Strengthening the overall performance of health systems (including strengthening financing, regulatory framework for private-public collaboration, governance, insurance, logistics, provider payment and incentive mechanisms, information, well-trained personnel, basic infrastructure, and supplies) to ensure equitable access to effective health, nutrition, and population interventions and a continuum of care to save and improve people’s lives (World Bank 2007c, p.14). High Burden Countries The 22 countries accounting for approximately 80 percent of all new tuberculosis cases arising each year. HIV status The state of being HIV-positive or HIV-negative. HIV-related TB Tuberculosis occurring in somebody infected with HIV. xx Identification In the research and development of malaria drugs, the identification of a biological system or target, the inhibition of which will result in parasite death. Incidence The number of new cases of a disease arising in a given period in a specified population. Independent evaluation An evaluation that is carried out by entities and persons free from the control of those involved in policy making, management, or implementation of program activities. This entails organizational and behavioral independence, protection from outside interference, and avoidance of conflicts of interest. Indication A symptom or circumstance indicating the advisability or necessity of a specific medical treatment or procedure. Indicator A quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor. Latent TB infection The presence in the body of tuberculosis bacilli that are dormant (usually in the lung) and not causing harm, but that may become active and cause disease. Logical framework or A management technique that is used to develop the overall design of a logframe program or project, to improve implementation monitoring, and to strengthen evaluation, by presenting the essential elements of the program or project clearly and succinctly throughout its cycle. It is a “cause and effect� model that aims to establish clear objectives and strategies based on a results chain, to build commitment and ownership among the stakeholders during the preparation of the program or project, and to relate the interventions of the program or project to their intended outcomes and impacts for beneficiaries. Malaria endemic country A country in which malaria prevails constantly. Monitoring The continuous assessment of progress achieved during program implementation in order to track compliance with a plan, to identify reasons for noncompliance, and to take necessary actions to improve performance. Monitoring is usually the responsibility of program management and operational staff. An effective monitoring system provides the information required for scheduled reporting to the governing body on the use of resources and the progress of activities as well as information on outputs and outcomes that contributes to future evaluations. Multidrug-resistant TB Tuberculosis infection that is resistant to treatment by isoniazid and rifampicin (the two most effective anti-tuberculosis drugs). Oversight One of the core functions of the governing body of a program: Monitoring the performance of the program management unit, appointing key personnel, approving annual budgets and business plans, and overseeing major capital expenditures. Pharmacovigilance The detection, assessment, understanding, and prevention of adverse reactions of patients to drugs — a response to a drug that is noxious and unintended, and which occurs at doses normally used. xxi Prequalification of Prior approval by a competent authority such as WHO of prospective manufacturers or suppliers bidders before the initiation of a procurement process. Prequalification of drugs is based upon the capability and resources of prospective bidders to perform the particular contract satisfactorily. Prequalification includes certification following a Good Manufacturing Practice (GMP) inspection. Prevalence The number of cases of a disease in a defined population at a specified point of time. Pulmonary TB Tuberculosis affecting the lungs. Resistance Ability of an organism to develop strains that are impervious to specific threats to their existence. For example, the malaria parasite has developed strains that are resistant to drugs such as chloroquine, and the Anopheles mosquito, which transmits the malaria parasite to human beings, has developed strains that are resistant to DDT and other insecticides. The ability to avoid or delay development of resistance is important in research and development for new drugs. Shareholders In the case of GRPPs, the subset of donors that are involved in the governance of the program. Therefore, this does not include individual (particularly anonymous) donors who choose not to be so involved, or who are not entitled to be involved if their contribution does not meet the minimum requirement, say, for membership on the governing body. Stakeholders Parties who are interested in or affected, either positively or negatively, by a development intervention. Stakeholders are often referred to as “principal� and “other,� or “direct� and “indirect.� While other or indirect stakeholders — such as taxpayers in both donor and beneficiary countries, visitors to a beneficiary country, and other indirect beneficiaries — may have interests as well, these are not ordinarily considered in evaluations unless a principal stakeholder acts as their proxy. Toxicity A measure of the degree to which something is poisonous. Vector An invertebrate animal, such as a mosquito, capable of transmitting an infectious agent without itself becoming infected. Source: For evaluation terms, IEG and OECD/DAC, Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards (World Bank, 2007). xxii xxiii Summary Purpose, Scope, and Methodology 1. The principal purpose of this Global Program Review (GPR) is to learn lessons from the experience of the Global Fund and its interaction with the Bank in three areas: (a) the design and operation of large global partnership programs like the Global Fund that are financing country-level investments, (b) the engagement of the World Bank with these partnership programs, and (c) the evaluation of these programs. The Review has an intensive focus on the Bank’s engagement with the Global Fund at the country level because of the potential for competition or collaboration between Global Fund-supported activities and the Bank’s lending operations at the country level. Therefore, it also focuses on the design and operation of the Global Fund-supported activities at the country level. 2. The Review has been prepared, first and foremost, for the Bank’s Executive Board to facilitate an informed discussion about the Bank’s past, current, and future engagement with the Global Fund. Since the Millennium Declaration in 2000, the World Bank has become involved in a growing number of partnership programs like the Global Fund that pool donor resources to finance country-level investments to help countries achieve specific Millennium Development Goals, that have inclusive governance structures, and that subscribe to the 2005 Paris Declaration on Aid Effectiveness. Other programs include the Global Alliance for Vaccines and Immunization (established 2000), the Global Partnership for Education (2002), the Climate Investment Funds (2008), and the Global Agriculture and Food Security Program (2010). The World Bank generally plays three roles in these programs — (a) as a trustee of donor funds supporting the program; (b) in the corporate governance of the program, and (c) as a development partner at the global and country levels. 3. This GPR is a review and not a full-fledged evaluation. Like other GPRs the Independent Evaluation Group (IEG) has conducted, this is based on an external evaluation that was commissioned by the governing body of the program — in this case, the Five-Year Evaluation (FYE) of the Global Fund, launched by the Global Fund Board in November 2006 and completed in May 2009. The Review (a) assesses the independence and quality of that evaluation; (b) validates the findings of the evaluation; and (c) assesses the extent and nature of the Bank’s engagement with the Global Fund at the global and country levels since the Global Fund was founded in 2002. 4. The findings and lessons of this Review are also informed by (a) structured interviews with Global Fund and World Bank staff as well as with other stakeholders; (b) visits to a sample of six recipient countries in which both organizations have been active in the health sector (Brazil, Burkina Faso, Cambodia, Nepal, the Russian Federation, and Tanzania); (c) an in-depth assessment of a World Bank-supported health project in Lesotho that was specifically designed to increase the capacity of the country to effectively use Global Fund grants for HIV/AIDS; (d) an electronic survey of Global Fund staff and World Bank project managers of health projects on the engagement between the Global Fund and the World Bank at the country level; and (e) a detailed comparison of the monitoring and evaluation (M&E) systems of the Global Fund and the World Bank. xxiv 5. Following IEG’s normal procedures, copies of the draft GPR were sent for review and comment to the Global Fund Secretariat in Geneva, to the two Bank units responsible for the World Bank’s engagement with the Global Fund — the Multilateral Trustee and Innovative Financing Department and the Health, Nutrition, and Population Department — and to other Bank units that have responsibility for the Bank’s involvement with global partnership programs. Their comments have been taken into account in finalizing the GPR. The formal responses received from the Global Fund and World Bank management are included in this document immediately after this Summary. 6. This Review was initiated before the High-Level Independent Review Panel on Fiduciary Controls and Oversight Mechanisms of the Global Fund was commissioned in February 2011, and it was drafted before their final report, Turning the Page from Emergency to Sustainability, was issued on September 19, 2011. While the two studies are complementary and overlap to some extent, they were conducted independently of each other, for different audiences, and for different purposes. Background on the Global Fund 7. The Global Fund was officially established in January 2002 “to attract, manage and disburse additional resources through a new public-private partnership that will make a sustainable and significant contribution to the reduction of infections, illness and death, thereby mitigating the impact caused by HIV/AIDS, tuberculosis and malaria in countries in need, and contributing to poverty reduction as part of the Millennium Development Goals� (Global Fund 2002). The Global Fund has fostered new approaches to development assistance to complement the existing aid architecture. It mobilizes donor resources on a large scale that are earmarked for a specific purpose and that are provided to recipient countries based on principles such as country-owned and aligned programs (Box S-1). Many of these principles were later adopted by signatories to the 2005 Paris Declaration. Box S-1. Global Fund Guiding Principles A. The Fund is a financial instrument, not an implementing entity. B. The Fund will make available and leverage additional financial resources to combat HIV/AIDS, tuberculosis, and malaria. C. The Fund will base its work on programs that reflect national ownership and respect country-led formulation and implementation processes. D. The Fund will seek to operate in a balanced manner in terms of different regions, diseases, and interventions. E. The Fund will pursue an integrated and balanced approach covering prevention, treatment, and care and support in dealing with the three diseases. F. The Fund will evaluate proposals through independent review processes based on the most appropriate scientific and technical standards that take into account local realities and priorities. G. The Fund will seek to establish a simplified, rapid, innovative process with efficient and effective disbursement mechanisms, minimizing transaction costs and operating in a transparent and accountable manner, based on clearly defined responsibilities. The Fund should make use of existing international mechanisms and health plans. Source: Global Fund (2002), “Framework Document of the Global Fund to Fight AIDS, Tuberculosis and Malaria,� pp. 1–2. xxv 8. The Global Fund committed $18.3 billion in grants to developing countries through June 2011, and disbursed $14.0 billion. Nearly three-quarters of these grant resources were awarded to low-income countries, and nearly two-thirds to Sub-Saharan Africa — the region most seriously affected by the three diseases. Almost half were awarded for HIV/AIDS programs, 35 percent for malaria, and 16 percent for tuberculosis. The largest share of the grants was awarded for medicines and pharmaceutical products (18 percent) and health products and equipment (17 percent). More than half of the grants were awarded to government agencies, about one-quarter to civil society organizations (CSOs), and about one-sixth to multilateral organizations such as United Nations Development Program. The Independence and Quality of the Five-Year Evaluation of the Global Fund 9. The Five-Year Evaluation comprised three Study Areas and a Synthesis Report undertaken over a two-and-a-half year period. Study Area 1, on the organizational efficiency and effectiveness of the Global Fund, was issued in October 2007; Study Area 2, on the Global Fund partner environment at the global and country levels, was issued in June 2008; Study Area 3, on the impact of collective efforts on reducing the burden of the three diseases, was issued in May 2009; and the final Synthesis Report was issued in March 2009. 10. Overall, this Review found that the FYE was an independent and quality evaluation, assessed against the indicative principles and standards of the Sourcebook for Evaluating Global and Regional Partnership Programs (IEG and OECD/DAC 2007). The evaluation has helped the Global Fund Board and management make significant strategic adjustments to its organizational and institutional arrangements. The three study areas reinforced each other, and the Synthesis Report effectively pulled together key messages in a coherent and integrated manner. Charged with a complex evaluation and an ambitious scope of work within a tight timeframe, the evaluation teams fulfilled the majority of their terms of references. 11. The conduct of the FYE was organizationally and behaviorally independent. An external body of experts appointed by and reporting to the Global Fund Board — the Technical Evaluation Reference Group (TERG) — oversaw all aspects of the evaluation, including contracting the evaluation to an independent consortium of evaluators. The evaluation teams were able to report candidly about how slowly and less strategically the Global Fund governance processes had developed to guide this new approach to development assistance; about the need for a robust risk management strategy to alert the Global Fund about likely suspension of ongoing treatment activities; and about the risk of increased drug resistance, among other things. Notwithstanding the TERG’s very “involved� oversight style, the FYE was protected from outside interference and the potential conflicts of interest that arose were appropriately identified and managed. 12. Although the FYE did not achieve two objectives — developing the “determinants� of good grant performance in Study Area 2 and building evaluation capacity in Study Area 3 countries — it was an innovative and participatory evaluation experience. It sought the active participation of a range of country-level stakeholders throughout the evaluation process. The xxvi formulation of evaluation questions and issues reflected the views and concerns not only of the public health and development community, but also of program beneficiaries and people affected by the diseases. The level of inclusiveness, participation, and transparency helped engender ownership from a broad stakeholder base throughout the world. This generally provided for a quality evaluation and learning experience, but the degree of participation declined toward the end of the evaluation process. 13. The FYE was objectives-based and evidence-based against the stated purpose and principles of the Global Fund. The overall assessment was fair and balanced, portraying both the strengths and weaknesses of the Global Fund. The FYE met three of the four standard IEG criteria for assessing quality — evaluation scope, instruments, and feedback. It did not meet the M&E criterion that the program’s activity-level M&E system should contribute to the evaluation’s assessment of the overall outcomes of the program because the Global Fund’s grant-level M&E system was not initially designed to do so. Therefore, the FYE used other methods, notably the impact assessment in Study Area 3. 14. The FYE was one of the first evaluations of a global partnership program to undertake an extensive assessment of its operational modalities at the country level, based on the 16 country case studies in Study Area 2. This covered all salient Global Fund processes at the country level, such as Country Coordinating Mechanisms (CCMs) and Local Fund Agents (LFAs), their interactions with development partner agencies, the availability of technical assistance, performance-based funding, and grant oversight. Recommendations were directed toward improving the CCMs, LFAs, performance-based funding, and grant oversight functions. This part of the evaluation provided support for the continuation of the Global Fund model, noted how it represented a new approach to development assistance, and underlined the need for strengthening the mostly informal nature of its partnerships. It found that partner agreements to high-level principles of collaboration needed to be translated into operational realities. 15. In spite of the initial ambition, Study Area 3 was not a rigorous impact evaluation. The evaluation teams did not attempt to show attribution or causality between program inputs and the intended development outcomes because Global Fund-supported interventions had not been designed to facilitate impact evaluations, and country-level data were inadequate. Many countries had also not yet completed one five-year grant cycle. Rather, the evaluation approach can best be compared against the analytical framework of a contribution analysis, since it attempted to assess the collective contribution of all donors and countries, based on the 18 country case studies in Study Area 3. 16. On balance, Study Area 3 did an adequate job in this regard, but with some shortcomings. It demonstrated that the collective efforts have resulted in increased access to services, better coverage, and some overall reduction in the burden of the three diseases. The Step-Wise Evaluation Framework that was used emphasized contextual factors, but this Review found that few contextual factors were actually considered, based on an in-depth review of two of the country case studies (Burkina Faso and Cambodia). Assumptions and risks, also important in contribution analysis, were not delineated in the logframe. Instead, they were described in different parts of the document and were not clearly defended. The xxvii evaluation could not, by design, assess the independent contribution of the Global Fund to country-level results. Validating the Major Findings of the Five-Year Evaluation 17. The Global Fund requires each recipient country, with limited exceptions, to establish a Country Coordinating Mechanism to review and endorse funding proposals for submission to the Global Fund, based on a national strategy for combating the disease in question. Eligible proposals for each round of grants are reviewed by an independent Technical Review Panel. The CCM nominates a Principal Recipient, or lead implementing agency, for each grant. Once a grant is approved and the grant agreement signed, the Global Fund Secretariat instructs the World Bank, as the trustee of the Global Fund, to release funds to the Principal Recipient. 18. Each grant agreement contains a disease-specific performance framework outlining the performance expected over the lifetime of the grant and key indicators that are to be used to measure outputs and coverage on a regular basis. Grants are initially approved for two years, and renewed for up to three additional years in accordance with these principles of performance-based funding. The Global Fund also contracts with an LFA to oversee, verify, and report to the Global Fund on grant performance at every stage of the implementation process, starting with an assessment of the financial, administrative, and implementation capacity of the nominated Principal Recipient to implement the approved grant. 19. The FYE conducted 16 country case studies as part of Study Area 2 in 2007. IEG consultants revisited four of these countries (Burkina Faso, Cambodia, Nepal, and Tanzania), as well as two middle-income countries (Brazil and the Russian Federation) in 2010 to confirm the FYE findings and to assess changes (either improvements or deteriorations) in the intervening three years, using the FYE and the four Study Area 2 country reports as a baseline. 20. The remainder of this section summarizes what IEG found, organized according to eight of the nine major findings of the FYE, as presented in the Synthesis Report. (IEG did not address the ninth major finding because this related to the global governance of the Global Fund, not its country-level activities.) 21. Additionality of Global Fund Resources. This Review confirmed the FYE finding that the Global Fund has provided substantial resources for HIV/AIDS, tuberculosis, and malaria control programs. Global Fund commitments of $4.3 billion in 2009 accounted for almost 40 percent of total official commitments (both concessional and nonconcessional) to combat the three diseases and 19 percent of commitments to the overall health sector, according to data from the Organisation for Economic Co-operation and Development. At the same time, other donor commitments to the three diseases outside of the Global Fund have not decreased, but have also increased, from $1.7 billion in 2002 to $7.1 billion in 2009 (in constant 2008 prices), and commitments to the overall health sector have grown from $9.2 to $18.1 billion. Whether total donor commitments to the three diseases have been higher or lower than they would otherwise have been in the absence of the Global Fund is not known. However, this Review found that other donor commitments for health have been essentially constant since 2002 in three of the four low-income countries visited (Burkina xxviii Faso, Cambodia, and Nepal). Similar to the FYE, this Review did not find evidence that governments have reduced their own expenditures on the three diseases in response to Global Fund grants, except in one country (Tanzania). 22. Sustainability of External Financial Support for the Three Diseases. The FYE found that reliance on external funds (from the Global Fund and other international donors) and inadequate investments in long-term domestic capacity raised concerns about the long- term sustainability of recipient countries’ disease-control programs. This Review found that the low-income countries visited were becoming increasingly dependent on Global Fund support for antiretroviral treatment of people living with AIDS. The Review also found increasing concerns at the global level that other donors’ support for treatment may be less forthcoming in the future. The sustainability of resources to support people living with AIDS who are already receiving antiretroviral treatment is of particular concern, since interrupted treatment increases not only the risk of death among those already being treated, but also the risks of new infections and of drug-resistant strains of the virus. Any retreat of bilateral donors from financing treatment is likely to result in increased demand on the Global Fund to finance the shortfall. If overall external support for treatment is not sustained, governments in recipient countries will face pressures to reallocate their own budgetary resources to costly treatment activities, and away from other health and non-health priorities. To the extent that the amounts of Global Fund and government resources spent on the prevention of new infections decline as a result and are not taken up by other donors, the long-run affordability and sustainability of treatment programs will be further undermined. 23. Predictability of Global Fund Support. This Review also found short-term gaps in the timing of Global Fund financing in several countries due to the uneven pattern of grant proposals and the unpredictability of grant approvals (only half the proposals are approved, on average). Very much aware of this issue, the Global Fund is currently transitioning its entire grant portfolio into single streams of funding (SSFs), which are intended to make it easier for the Global Fund to support a national program approach for each disease that is better aligned with national systems and budget cycles. The Secretariat has so far signed over 80 SSFs and plans to have completed most of the transition to SSFs by the end of 2013. 24. Performance of CCMs. The FYE found that the CCMs were successful in mobilizing domestic and international partners for submission of grant proposals to the Global Fund and in enabling CSOs and affected communities to participate in the proposal preparation process, but that CCMs were ill-equipped to provide adequate oversight of grant implementation. This Review found that the CCMs were functioning better than the 2007 FYE findings indicated in two countries (Burkina Faso and Cambodia), about the same in two countries (Tanzania and Brazil), and worse in two countries (Nepal and the Russian Federation). The two countries (Cambodia and Tanzania) with their own national-level technical review panels also had the highest grant approval rates. This Review found little improvement since 2007 in the capacity of CCMs to oversee the implementation of Global Fund grants from the country perspective, because they generally lacked the authority and the resources to do so effectively. Inadequate management of the inevitable conflicts of interest that arise in bodies such as the CCMs also hindered effective oversight in some countries. The Global Fund has taken steps over the last two years to strengthen CCMs’ capacity to oversee grant implementation and to manage conflicts of interest. xxix 25. Effectiveness of Country-Level Partnerships. The FYE found that country-level partnerships were based mostly on good will and voluntary collaboration rather than on negotiated commitments with clearly articulated roles and responsibilities. They did not yet comprise a fully functioning system — representing more of a “friendship model� than a genuine “partnership model.� This Review found that partnerships with other development agencies such as the World Bank and bilateral donors have generally improved since 2007 in terms of other partners’ providing technical assistance in support of Global Fund activities. However, country-level stakeholders still see the Global Fund as a largely separate development agency with its own distinct modalities that are not well integrated into the existing donor coordination mechanisms in the countries. This was also true of other large donors such as USAID, the President’s Emergency Plan for AIDS Relief (PEPFAR), and the World Bank in particular countries. This Review found that civil society representation in decision making was effective in three of the six countries (Brazil, Burkina Faso, and Cambodia). IEG found little evidence of effective partnerships with the commercial private sector at the time of its country visits (April–June 2010). 26. Application of Performance-based Funding. The FYE found that the Global Fund had attempted to implement performance-based funding on a scale unprecedented in the international health arena. However, this “focus on results� remained a work in progress and had evolved into a complex and burdensome system that focused more on project inputs and outputs than on development outcomes and impacts. This Review found that performance- based funding was working reasonably well in three countries (Burkina Faso, Cambodia, and the Russian Federation) in terms of monitoring outputs and coverage in relation to the key performance indicators in the grant agreements. It was not working well in the other three countries — it was hindered by low-quality data in Tanzania, by political instability in Nepal, and by its unsuitability for the types of Global Fund grants in Brazil (focusing on intermediate products in the health system). 27. Adequacy of Grant-Level Monitoring and Evaluation. The Global Fund has very detailed and well-documented requirements for grant-level monitoring, which are tied to its performance-based funding approach. However, the Global Fund does not have a system for end-of-grant evaluations. Its grant-level M&E system is designed more to facilitate grant disbursements than to contribute to an overall assessment of the program. While the FYE was an independent and quality evaluation, it was constrained by the absence of an M&E framework for the cumulative assessment of grant performance; it had to rely on other approaches, such as the in-depth country studies. The lack of such a framework made it unclear what criteria the FYE used to draw conclusions — both positive and negative — about the overall efficacy of Global Fund grants. 28. Access and Coverage of Service Delivery. The FYE found that collective donor efforts had contributed to increased access to disease-control services, particularly for HIV/AIDS, and increased coverage of affected communities, which should ultimately reduce the disease burden. The survival rate of people on antiretroviral therapy had increased, and the incidence of HIV among young people had probably declined in some countries. This Review found that Burkina Faso and Cambodia have used Global Fund grants to expand services for all three diseases, and that Brazil has used the grants to improve the quality of services for tuberculosis and malaria (the only two diseases for which the country has received grants). Tanzania has xxx had a weaker record of grant implementation, and Nepal and the Russian Federation have yet to put in place an effort of sufficient scale to reach high-risk and marginalized groups of HIV- vulnerable individuals, and thereby thwart the spread of HIV into the general population. 29. Equity in Country-Level Governance and Grant Objectives. The FYE found that the Global Fund had modeled equity in its guiding principles and organizational structure — for example, in ensuring representation of women and marginalized populations on the CCMs. However, few systems had been put in place to monitor gender, sexual orientation minorities, urban-rural, wealth, education, and other types of equity as part of grant performance. This Review also found significant attention to equity issues in most of the six countries visited in terms of membership of affected communities on the CCMs and the objectives of the grants themselves. Expanding access to diagnostic and treatment services in rural areas has been a key focus of Global Fund grants in all four low-income countries. Reaching high-risk groups in the case of HIV/AIDS has been more difficult, and has been more successful in some countries visited, such as Cambodia. 30. Impact of Donor Support for the Three Diseases on Domestic Health Systems. The FYE found that the large increases in external funding for the three diseases had stretched existing, generally weak, health systems to their limit. Health systems needed to be strengthened if countries were to scale up the delivery of services financed by the Global Fund. This Review found mixed results, risks, and opportunities associated with the effects of Global Fund grants for the six countries’ health systems. The large inflow of Global Fund resources into small low-income countries with high disease burdens has tended to create dependency on the Global Fund for treatment of the three diseases, and to weaken domestic health systems by drawing talent away from the public sector. However, Global Fund grants have directly expanded the service delivery capability of local health systems in Burkina Faso and Cambodia, where the participation of CSOs, community-based organizations, and faith- based groups has enhanced access to health services in rural areas. The Global Fund model also encourages establishing relationships beyond the conventional ministries of health — for example, with drug enforcement agencies, to help strengthen country systems in the fight against counterfeit drugs and drug resistance. 31. Institutional Risk Management by the Global Fund. The FYE found that weak management of risks — including financial, organizational, operational, and political risks — was a particular vulnerability of the Global Fund. The main risk-mitigation instruments had comprised LFA assessments, financial disbursement red flags, and the Early Alert and Response System that was intended to provide early identification of underperforming projects and to facilitate timely corrective actions. This Review found that the Global Fund Secretariat has given priority attention to improving risk management at the corporate and country levels following a Board directive in 2007 and in response to the FYE findings and recommendations. IEG found that the LFA’s verification and reporting on grant performance was better in four countries (Brazil, Burkina Faso, Cambodia, and the Russian Federation) than indicated in the 2007 FYE findings. However, as already mentioned, the CCMs’ programmatic oversight of Global Fund grant implementation was still weak. Communications between the LFAs and the CCMs have proven to be a sensitive matter, since the LFA is an agent of the Global Fund Secretariat, not of the CCM. xxxi The World Bank’s Engagement with the Global Fund THE WORLD BANK’S ROLES IN THE GLOBAL FUND 32. The World Bank plays three major roles in the Global Fund: (a) as the trustee of donor contributions to the Global Fund; (b) in the corporate governance of the program, and (c) as a development partner at the global and country levels. This Review has focused on the third role, and on the first two roles mainly as they contribute to the third. 33. First, as trustee, the Bank receives and invests funds from Global Fund donors, disburses the funds to grant recipients on the instruction of the Secretariat, and provides regular financial reports to the Board. The Global Fund trust fund is the largest of the 15 financial intermediary funds (FIFs) administered by the Bank that are supporting global and regional partnership programs. The income from investing undisbursed funds represented 5.4 percent of the total resources available to the Global Fund during 2002–10, and has more than covered the cumulative administrative costs of the Global Fund since it was established. 34. Second, the Bank is a permanent nonvoting “institutional� member of the Board, along with the World Health Organization (WHO), UNAIDS, and the Stop Tuberculosis Partnership (Stop TB), and a member of two Board committees — the Finance and Audit Committee, and the Policy and Strategy Committee. The Global Fund employs a constituency-based stakeholder model of governance in which voting membership on the Board includes not only donors but also nonfinancial contributors such as recipient countries, affected communities, nongovernmental organizations (NGOs), and the commercial private sector. 35. The FYE found that the Global Fund governance structure and processes had achieved both broad participation and genuine power-sharing among key constituencies in the fight against the three diseases. The participation of civil society and private sector constituencies has been broadly viewed as effective, while that of some other constituencies (such as affected communities) has been less so, due to the size of the constituencies and the absence of easy mechanisms to communicate effectively within them. This Review also found that the Global Fund represented a significant shift in the roles and responsibilities of different stakeholder groups compared with more exclusive shareholder models of governance in which voting membership on the governing body is limited to financial contributors. However, it is doubtful that this diminished status of the Bank and other nonvoting members significantly reduces the Bank’s reputational risks of involvement with the Global Fund, given the role that the Bank plays in global health, and its extensive engagement with the Global Fund at the global and country levels, as documented in this Review. 36. Third, the Bank’s role as a development partner has been less clearly defined than the first two roles. The Bank plays an operational role, as one of the implementing agencies, in most other global partnership programs supported by FIFs. That it might also play such a role in the Global Fund was never seriously considered by the Transitional Working Group (the precursor to the Global Fund Board) in 2001. However, there were considerable pressures at the outset for the Bank to take on an “enhanced fiduciary role� in addition to being the trustee, to ensure that the Global Fund grants were used for the intended purposes. When the Bank xxxii declined, the Global Fund Board decided in April 2002 to establish the LFA system of contracting out in-country fiduciary functions to LFAs. 37. This Review found that there was a strong expectation among members of the Global Fund Board at the outset that development partner agencies — including WHO, UNAIDS, RBM, Stop TB, and the Bank — would provide technical support to Global Fund-supported activities at the country level. The extent to which the Bank accepted or acknowledged this role appears to have been left deliberately vague due to the tensions surrounding the establishment of the Global Fund in 2002. There was — and remains to this day — no formal agreement or memorandum of understanding (MOU) between the World Bank and the Global Fund in terms of working together at the country level, and there have been no written directives or guidelines issued to staff in either organization for engaging with the other at the country level. Although the Bank was involved in 11 other global and regional health partnerships in 2002, the Global Fund was the first one that was financing country-level investments in which the program expected the Bank to provide such technical support. THE WORLD BANK’S ENGAGEMENT WITH THE GLOBAL FUND AT THE COUNTRY LEVEL 38. This Review examined more closely an area not covered by the FYE: the engagement between the World Bank and the Global Fund at the country level. There is no systematic record of this engagement. Therefore, this Review has pieced together this record from Bank databases, word searches and reviews of World Bank County Assistance Strategies and Project Appraisal Documents, key informant interviews, and the electronic survey of health sector project managers at the World Bank and Global Fund staff in Geneva, administered in March 2011. Collectively, these results suggest that the Global Fund and the Bank have had some degree of engagement — from sharing information about each other’s activities to active collaboration in the pursuit of commonly agreed objectives — in about three-quarters of the 90 countries in which both organizations have been active in the health sector since 2002 (Figure S-1). This amounts to about 65 countries overall, of which 25–30 countries have been in Africa, the region most seriously affected by the three diseases. 39. Engagement has generally started with a request from the government of the country. The government — as the chair or an influential member of the CCM — has often requested the Bank’s technical support for preparing grant proposals to the Global Fund, particularly during the earlier Global Fund rounds and for HIV/AIDS proposals in countries in which the Bank was supporting a Multi-country AIDS Program project. Recognizing that the Bank’s overarching mission is to contribute to the development of its client countries and their institutions, Bank staff have generally responded positively, to the extent that their time and resources permitted. Bank staff have also become involved in Global Fund-supported activities through their participation in health sector donor-coordination processes in the country, through participation in joint World Bank-Global Fund workshops, and through the direct request of Global Fund Regional Team Leaders and Fund Portfolio Managers. World Bank Sector Managers have also encouraged engagement in some cases. 40. Bank staff and consultants have generally not been involved in specific Global Fund processes at the country level. They have been members of the CCM in at most one-third of the 65 countries, according to survey results, helped to prepare grant proposals in 30 percent, xxxiii Figure S-1. Global Fund and World Bank Country-Level Staff: Overall, how would you best characterize the relationship between the World Bank and the Global Fund in the country you were working on? Collaborative: The two organizations’ staff, consul-tants and agents worked together on common activ-ities in the pursuit of commonly agreed objectives. Complementary: The two organizations’ staff, consultants, and agents worked alongside each other in the pursuit of common objectives. Consultative: The two organizations’ staff, consultants, and agents consulted each other regularly in the course of their own activities. Sharing information only: The two organizations’ staff, consultants, and agents only shared information about each other’s activities. Unrelated and independent: The two organizations worked independently of each other supporting different health initiatives in the country. Competitive: The two organizations competed for business among the same potential clients. 0% 10% 20% 30% 40% GF � Country Programs Staff WB � Task Team Leaders Source: IEG Survey of World Bank health sector project managers and Global Fund Secretariat staff, administered in March 2011. Note: Each respondent was limited to only one choice; therefore, the responses from each organization add up to 100 percent. The survey response rates were 62 percent (36 out of 58) for Global Fund Country Programs staff and 33 percent (42 of 128) for World Bank task team leaders (project managers). and provided formal technical assistance to the Principal Recipients in 25–30 percent of countries (some of which have also been implementing agencies for Bank-supported projects). Bank staff and consultants have more frequently contributed to other country-level activities, such as strategic and analytical work, that directly or indirectly contributed to the work of the Global Fund. 41. The engagement between the Bank and the Global Fund has also been dynamic in many countries, such as Burkina Faso and Tanzania. Bank-supported Multi-Country AIDS Program projects helped to institutionalize the CCMs in these countries and to prepare the initial grant proposals. Then, as the Global Fund expanded its support, the Bank moved toward providing complementary support to the countries’ health sectors more generally. Key factors contributing to positive engagement have been a proactive government and a strong donor coordination mechanism at the country level. The personal commitment of the World Bank’s project managers and Global Fund’s Portfolio Managers has also played a role in sustaining successful cooperation, as in Lesotho during the implementation of the HIV and AIDS Capacity Building and Technical Assistance Project from 2004 to 2008 and in the xxxiv Russian Federation during the implementation of the Tuberculosis and AIDS Control Project from 2003 to 2009. 42. There have been numerous avenues for World Bank-Global Fund engagement at the country level, including corporate-level contacts, the Bank’s own support for communicable disease control in client countries, and the various initiatives associated with the Global HIV/AIDS Program (GHAP) and the International Health Partnership (IHP). GHAP, which was established in the Bank in June 2002 in partnership with UNAIDS, led to the establishment of the Global HIV/AIDS Monitoring and Evaluation Support Team (GAMET) in 2002, the Global Implementation Support Team (GIST) in July 2005, the AIDS Strategy and Action Plan Service (ASAP) in July 2006, as well as an unsuccessful attempt to formulate an MOU between the Bank and the Global Fund in 2007. The IHP, which was launched in September 2007, led to the Health-8 group in 2007, the Joint Assessment of National Strategies in July 2009, and the Health Systems Funding Platform in early 2010. (UNAIDS has contributed $57.1 million to a Bank-administered trust fund over 2003–10 to support the various activities of GHAP, and WHO has recently established a trust fund at the Bank to support IHP activities.) But none of these avenues has so far led to a formal agreement between the World Bank and the Global Fund on country-level engagement. PROSPECTS FOR FUTURE ENGAGEMENT AT THE COUNTRY LEVEL 43. There are growing pressures from donors for the Global Alliance for Vaccines and Immunization (GAVI), the Global Fund, and the World Bank — as the three largest multilateral financiers of country-level investments in health — to improve collaboration at the country level. A consultation on Donor Harmonization of AIDS Funding in April 2004 endorsed the “Three Ones� principles to be applied in each recipient country: (a) one agreed HIV/AIDS action framework for coordinating the work of all partners, (b) one national AIDS coordinating authority, and (c) one agreed country-level M&E system. A 2006 study on the comparative advantages of the World Bank and the Global Fund found the Bank’s comparative advantage to be systematic strengthening of health systems to support communicable disease control, among other things. The Health Systems Funding Platform has since incorporated these ideas into its efforts to accelerate progress toward achieving all the health-related Millennium Development Goals in addition to combating communicable diseases. While the Three Ones principles were first developed for HIV/AIDS, they are also relevant for other disease areas, and for donor-supported health sector activities in general. 44. The different business models of the World Bank and the Global Fund provide both opportunities and hindrances. The survey of World Bank project managers and Global Fund staff found that both groups tend to have a positive view of the opportunities for engagement associated with the GHAP and the IHP. Global Fund staff generally appreciate the relatively strong country presence of the World Bank and the Bank’s support for strengthening country-level health sector M&E systems. Bank project managers generally appreciate the presence of CSOs on the CCMs and the fact that Principal Recipients of Global Fund grants need not be government agencies. 45. Yet both organizations also view engagement as difficult in some respects. Bank project managers regard unprogrammed technical support as an unfunded mandate. Global Fund staff xxxv regard it as problematic that World Bank funding for the health sector has to compete with other sectors for its place in the Bank’s Country Assistance Strategy. Both regard their own organizations as more flexible in responding to country needs, based on interviews. But the survey results also suggest that some other factors raised in interviews are not significant impediments to collaboration: the different professional backgrounds of World Bank project managers and Fund Portfolio Managers, the different types of financial support (loans versus grants), the success of the Global Fund in mobilizing donor resources to combat the three diseases, and the role of the LFA in the fiduciary oversight of Global Fund grants. 46. Global Fund staff view the World Bank as a partner of the Global Fund at both the global and country levels to a greater extent than do Bank staff (Figure S-2). Both Global Fund and Bank operational staff would prefer engagement in the context of their own organization’s business model. They generally viewed the comparative advantages of the other organization in terms of what the other could contribute to its own method of operation. Figure S-2. Global Fund and World Bank Staff: To what extent do you consider the World Bank to be a partner of the Global Fund (a) at the global level and (b) at the country level? Global Fund – All Clusters: (a) At the global level: (b) At the country level: World Bank – Project Managers: (a) At the global level: (b) At the country level: 0% 20% 40% 60% 80% 100% High Substantial Modest Negligible Source: IEG Survey of World Bank health sector project managers and Global Fund Secretariat staff, administered in March 2011. Note: The survey response rates were 49 percent (52 out of 106) for Global Fund staff and 33 percent (42 of 128) for World Bank project managers (task team leaders). 47. Global Fund staff would like the Bank to make a greater effort to include them in high- level government discussions, as has happened in some countries, such as Cambodia, and for the Bank to contribute its health sector expertise to Global Fund processes, such as the CCM at the country level. World Bank project managers would like the Global Fund to contribute to multidonor Sector-Wide Approaches (SWAps) or cofinance World Bank projects in the health sector, and for the Global Fund’s donors to establish a trust fund for financing Bank-supervised technical assistance in support of Global Fund-supported activities. 48. Neither World Bank project managers nor Global Fund Portfolio Managers are satisfied with “business as usual.� Both groups viewed the absence of an MOU on country-level xxxvi collaboration between the two organizations as a significant impediment to such collaboration. Both found the absence of guidelines within their own organizations for engaging with the other organization to be problematic. 49. If World Bank engagement with Global Fund-supported activities remains at current levels, or increases, there needs to be a clearer institutional mandate for Bank staff to work with the Global Fund for the benefit of client countries — particularly low-income countries with high disease burdens — with resources allocated for the purpose and with appropriate institutional recognition of contributions and achievements. Whether or not the Bank reaches a formal or informal agreement with the Global Fund for working together at the country level, the ways in which the Bank’s country teams and staff are permitted, encouraged, or required to engage with Global Fund-supported activities at the country level simply need to be defined and resourced. And for sustainability, the relationships need to move beyond the personal to the institutional level. Such directives and guidelines are not contrary to country-driven development; they can allow for case-by-case judgment, taking into account country differences. Lessons 50. Since it was founded in 2002, the Global Fund has become a prominent example of large global partnership programs that pool donor resources to finance country-level investments to help countries achieve specific Millennium Development Goals in accordance with the Paris Declaration principles of country ownership, alignment, harmonization, managing for results, and mutual accountability. This Review provides a number of lessons for the Global Fund and other similar programs, for the World Bank in engaging with these programs, and for evaluating global partnership programs more generally. LESSONS FOR THE GLOBAL FUND 51. Harmonization. The Global Fund is facilitating donor coordination at the point at which donors contribute to the trust fund and serve on the Global Fund Board, but this has not yet translated into a similar degree of coordination at the country level. Country-level stakeholders tend to regard the Global Fund as another, largely separate development partner agency with its own distinct modalities that have not been well integrated into existing donor coordination mechanisms in the countries, or with national budget cycles, contrary to the harmonization principle of the Paris Declaration. While this situation may improve as the Health Systems Funding Platform matures and as the Global Fund transitions its grant portfolio to single streams of funding under its new grant architecture, the Global Fund has not generally contributed to harmonization through existing mechanisms for pooling funds at the country level, such as SWAps. 52. Technical Support to Enhance Country Ownership. Development partners need to provide greater technical support to strengthen the ability of governments to effectively coordinate donor efforts around agreed national strategies. This Review found that the situation has generally improved since the FYE in terms of other partners’ providing technical assistance in support of Global Fund activities. The Global Fund has also developed a new partnership strategy, signed MOUs with Stop TB and RBM in 2009 and 2010, respectively, and is reaching out to other development partner agencies more generally. xxxvii However, the Global Fund needs to find ways to finance such technical assistance, provide it directly, or work effectively with other development partner agencies to do so. 53. Sustaining the Benefits of Global Fund Support. The long-term sustainability of the benefits of Global Fund-supported activities depends on the complementary activities of donor partners and strengthening the capacity of recipient countries. This will require a substantially more coordinated approach to external financial support at both the global and country levels than has occurred to date. It will be difficult for the Global Fund “to adjust its demand-driven model� to support “the most cost-effective interventions tailored to the type and local context of specific epidemics,� as recommended by the FYE, if it ends up becoming the residual financier financing others’ shortfalls. The scarce resources available to fight the three diseases — including those raised by the country from its own resources and those provided by its external partners, including the World Bank — need to be allocated collectively and proactively in each country in accordance with an agreed long-term strategy for fighting each disease in the country. 54. Managing for Results. The M&E requirements of different development partners have so far thwarted their good intentions to coordinate and streamline M&E for the three diseases at the country level. The Global Fund, the World Bank, and other agencies have endorsed the Three Ones principles of a common action framework, a single coordinating authority, and one M&E framework to monitor collective efforts in each disease area. They jointly prepared an M&E Toolkit in 2004 (revised in 2006, 2009, and 2011) to establish norms and identify indicators to be used by all the agencies, but it has been difficult to achieve their use in practice because each agency has its own project-level M&E requirements. Both the Global Fund and the World Bank could contribute to improved M&E at the project and country levels by making a stronger commitment to the Three Ones principles. Project-level M&E could focus on accountability for achieving the specific outputs of each project, and country-level M&E on tracking the higher-level outcomes and impacts collectively. 55. Managing Conflicts of Interest. Real and perceived conflicts of interest are an inherent and essentially unavoidable feature of all partnership programs, deriving in the first instance from the multiple roles that the key partners play in a given program. The Global Fund has brought recipient countries, CSOs, and affected communities into its governance arrangements at both the global and country levels. It has also established independent review processes at key stages in its operations such as the reviewing of grant proposals (by the Technical Review Panel), verification and reporting on grant performance (by the LFAs), and overseeing evaluations (by the TERG). It has also established, and recently expanded, its conflict of interest guidelines for the operation of CCMs. The key is to identify and manage potential conflicts of interest in a way that does not impede the effectiveness of the program. Reconciling these two imperatives will remain a continuing challenge for the Global Fund and for other global and regional partnership programs. 56. Global Public Policy. Neither the Global Fund nor the World Bank can address by itself “global communicable disease governance issues� such as the risk of drug resistance for current treatments of the three diseases. This Review found that drug resistance is a live issue in the countries visited, amplified by incomplete treatments and the presence of counterfeit drugs. Global Fund grants could help strengthen the capacity of drug regulatory and enforcement agencies in assuring quality compliance by the pharmaceutical industry, and xxxviii CCMs could invite drug regulatory agencies to participate in specialized committees of the CCMs. The Global Fund and the World Bank also need to support ongoing efforts by organizations with relevant competence, such as WHO and the United Nations Office on Drugs and Crime, to ensure that the sizable investments that the world has made in combating the three diseases are not diminished by inaction in this area. LESSONS FOR THE WORLD BANK 57. Financial Intermediary Trust Funds. This Review provides evidence to support IEG’s recent recommendation that “the Bank should strengthen its framework for guiding its acceptance and management of FIFs going forward� (IEG 2011a, p. 85). Like other FIFs, the Global Fund trust fund was established in an ad hoc way in 2001–02 to accommodate the particular requirements of the Global Fund and its donors. This has resulted in some ambiguities in the relationship between the Bank and the Global Fund. For example, the trust fund management agreement was crafted to limit the Bank’s responsibility for the development outcomes of the use of trust fund resources, yet Global Fund donors expected that the Bank would contribute technical assistance to Global Fund-supported activities at the country level. Also, the Bank’s accountability for the effective governance of the Global Fund as a permanent nonvoting institutional member of the Board has not been clarified. The Bank is currently in the process of preparing a stronger framework for the acceptance and management of FIFs, along the lines recommended by IEG. 58. Engagement Strategy. This Review also provides evidence to support IEG’s recent recommendation that “the Bank should have an explicit engagement strategy for each GRPP in which it is involved, including . . . the expected roles of the Bank in the program at both the global and country levels, . . . how the program’s activities are expected to be linked with the Bank’s country operations, and how the risks to the Bank’s participation will be identified and managed� (IEG 2011b, p. 101). This Review has found that the Bank has been actively engaged in the corporate governance of the Global Fund and with Global Fund- supported activities in about 65 countries, in addition to being the trustee of the Global Fund trust fund. Yet the trustee role has been the only one of the Bank’s roles in which the Bank’s contributions to and expectations of the relationship have been expressed, so that the trustee relationship is bearing the burden of the Bank’s entire engagement with the Global Fund, which it was not designed to do. It would be better for the Bank to have a more complete engagement strategy with the Global Fund that encompasses all the roles that the Bank plays in the partnership. This would include guidance to country-level Bank staff for engaging with Global Fund-supported activities at the country level. 59. The Bank is in the process of preparing a new partnership framework for the Bank’s engagement with GRPPs more generally. The Bank’s 2007 Health Strategy also provides general statements about its engagement with the Global Fund. However, something more than these general statements is also needed to provide guidance to country teams and Bank staff. The Global Fund will likely continue to disburse for communicable disease control more than what the Bank disburses for the entire health sector (Figure S-3). Nine years of experience have shown that the Bank can contribute meaningfully to the work of the Global Fund at the country level without taking on supervisory or operational roles. Undertaking such roles — as the Bank currently performs for the Global Environment Facility — might also be considered on a pilot xxxix basis under certain circumstances, such as a SWAp operation or a common implementing agency (Principal Recipient). The Global Fund or its donors could also establish a trust fund at the World Bank for financing Bank-supervised technical assistance in support of Global Fund- supported activities, following the precedents of UNAIDS for the Global HIV/AIDS Program and WHO for the International Health Partnership. Figure S-3. Global Fund Grants and World Bank Health Projects, Fiscal Years 2000–11 Commitments (year of approval) Disbursements US$ Billions US$ Billions 5.0 3.5 4.5 3.0 4.0 Global Fund World Bank 3.5 2.5 Global Fund 3.0 2.0 2.5 World Bank 1.5 2.0 1.5 1.0 1.0 0.5 0.5 0.0 0.0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Source: Global Fund and World Bank data. Note: Global Fund commitments and disbursements are totals. World Bank commitments and disbursements represent the proportions of total project commitments and disbursements to the health sector. 60. Community of Practice. The Bank could establish a community of practice among its project managers who are working with the Global Fund to learn cross-cutting lessons of experience. This would be similar to the regionally coordinated community of practice that currently exists for the Bank’s engagement with the Global Environment Facility. Such a community of practice could lead, among other things, to standard terms of reference for Bank staff serving on CCMs, and could be supported by a central database to keep track of the Bank’s engagement with the Global Fund over time. As many have observed, “what gets measured, gets done.� LESSONS FOR THE EVALUATION OF GLOBAL AND REGIONAL PARTNERSHIP PROGRAMS 61. Early Stage Evaluations. Formative evaluations, like Study Areas 1 and 2 of the FYE, are more useful in the early stages of a global program in helping the program make strategic adjustments to its organizational and institutional arrangements than the contribution analysis that was undertaken in Study Area 3. Furthermore, the diversity of components in a global or regional program and the resulting complex causality and aggregation issues by their nature make impact evaluation difficult if not infeasible. Nonetheless, impact evaluations may be valuable in helping to identify the impacts of interventions and key causal linkages for subsets of activities where impacts are more measurable than for the program as a whole. xl 62. Project-Level Monitoring. Good monitoring systems should not only assess progress in implementing activities but also contribute to periodic summative evaluations and to effective policy dialogue. The Global Fund has established different objectives for M&E at the grant, country, and corporate levels, yet the three levels are not well connected with each other. Its grant-level M&E system is designed more to facilitate its performance-based funding approach to grant disbursements than to contribute to an overall assessment of the outcomes of the program or to policy dialogue. The only country-level evaluations that it has so far undertaken are the 18 country assessments for Study Area 3 of the FYE. The Global Fund could consider undertaking evaluations of a random sample of the single streams of funding for each disease now taking place under its new grant architecture. The Global Fund might also institutionalize regular country-level evaluations, the results of which could feed into, rather than be part of, subsequent evaluations of the overall program. This would also help build the knowledge base about which approaches most successfully contribute to achieving collective outcomes. 63. Objectives and Scope of Global Program Evaluations. These are best kept to a manageable size consistent with the most immediate evaluation needs of the program — allowing for realistic schedules and avoiding evaluation fatigue and conflicts with other evaluation efforts in countries. Large numbers of upstream processes built into the evaluation design can distract instead of facilitate the evaluation process. Sufficient time should also be allowed to adequately pretest new evaluation instruments. 64. Participatory Evaluation. Participatory evaluations that engage country partners need to manage expectations, since unmet expectations dampen country ownership of the evaluation process and of the end product. Evaluation schedules should be realistic and allow for productive exchanges and consultation between evaluation teams and country partners. Otherwise country partners may perceive their roles as largely collecting critical data, with little involvement in the analysis and deliberations about their significance. 65. Evaluation Capacity Building. Development activities such as building country-level evaluation capacity within the context of a global program evaluation are commendable but difficult to implement and sustain in the context of a one-off evaluation. Building M&E capacity is a long-term endeavor that is better undertaken through more conventional approaches given the condensed schedule in a global program evaluation. The tension between the two objectives can be very pronounced: an external evaluation emphasizes independence and objectivity, while capacity building emphasizes learning and strong engagement with the implementing bodies. xli Secretariat Management Comments: The Global Fund “The Global Fund values very highly the partnership with the World Bank. It welcomes many of these recommendations. It is implementing many actions as part of its comprehensive transformation plan which is the corporate priority in 2012.� The Global Fund Secretariat values very highly the partnership with the World Bank, and the opportunities to further improve it at the global and country levels as outlined in this Program Review. We are pleased that the Review recognized the independent and quality evaluations the Global Fund has undertaken so far and the improvements in partnership it has accomplished over time. Overall, the Secretariat supports most of the recommendations aimed at strengthening the partnership with the World Bank at the global and country levels, and ensuring benefits of joint approaches to project- and program-level M&E. We are pleased to see that the findings of the IEG Program Review are very much in line with the findings of the Five-Year Evaluation of the Global Fund and that the recommendations of the Review have been quite widely covered by recent review processes undertaken by the Global Fund in the course of 2011, namely by the Comprehensive Reform Working Group and the High-Level Panel. Many of the proposed new approaches to M&E, including an increased focus on outcomes and impact, are important components of the Global Fund Consolidated Transformation Plan, recently approved by the Global Fund Board, whose implementation represents the Secretariat’s major corporate priority in 2012. Through the implementation of the Consolidated Transformation Plan, we will also be able to improve the way we work with our partners, including the World Bank, at the country level. The Secretariat would like to highlight that there are some limitations in drawing general conclusions from the six country studies covered by this review. The Global Fund will take many of the lessons of this program review into account in the framework of the Consolidated Transformation Plan implementation, the implementation of the new evaluation strategy and Global Fund strategy in 2012, and in continued efforts to strengthen the formal partnership with the World Bank. Overall Comments on the Program Review Findings and Recommendations The Global Program Review is not a full-fledged evaluation and it is based on a recently completed evaluation of the Global Fund—the Five Year Evaluation—that was considered as a good quality and influential independent evaluation exercise. xlii The Program Review aimed to: 1. Assess the independence and quality of that evaluation, which it found to be “an independent and quality evaluation.� 2. Provide a second opinion on the effectiveness of the Program. 3. Assess the performance of the World Bank as a partner in the Program. 4. Draw lessons for the Bank’s engagement in the global and regional partnership programs more generally. The recommendations of the Program Review build on the Five-Year Evaluation, and the Global Fund has responded to many of its major findings, including:  The implementation of country teams combining functional and Fund Portfolio Manager expertise to manage high-impact and high-risk grants  The implementation of the new grant architecture, including single streams of funding and periodic reviews built on country evaluations  The approval of the new evaluation strategy  The development of the new Global Fund strategy, including modifications to the proposal process and promotion of reprogramming and learning in grants. We welcome the methodology used for the Program Review based on: (a) Desk review of key documents and academic literature (b) Structured interviews with Global Fund and World Bank staff as well as with other stakeholders (c) An analysis of Global Fund–supported activities (d) Visits to a sample of six countries in which both organizations have been active in the health sector (Brazil, Burkina Faso, Cambodia, Nepal, the Russian Federation, and Tanzania) (e) Electronic survey of Global Fund staff and World Bank project managers focused on the partnership, and, in addition, a detailed comparison of the M&E systems of the Global Fund and the World Bank. However, we would like to stress that there are some limitations in drawing lessons based on such a small sample of countries as they emerge from some of the findings of the Program Review:  The Global Fund–World Bank partnership is stronger at the global than at the country level. Despite recent improvements in coordination with partners on technical assistance, there is a need to define technical assistance functions and funding modalities.  The Global Fund is facilitating donor coordination at the global level through its Board, which has not yet translated into a similar degree of coordination at the country level.  There are some sustainability issues with decreasing donor funds in a number of countries, and short-term funding gaps are affecting the Global Fund.  Disease-control programs are increasingly available, but there are concerns over “inadequate investments in long-term capacity.� xliii  Country Coordination Mechanisms (CCMs), have successfully brought country-level stakeholders together to submit grant proposals, but have failed in exercising effective oversight of grant implementation.  There is little or no evidence of effective partnerships with the commercial private sector in the six countries.  The principle of “Performance Based Funding� is working well in three countries, and less well in three other countries.  The Global Fund is giving priority to improving risk management at the corporate and country level, with more progress in dealing with financial, operations, and organizational risks  The LFA system is working well in the majority of the countries visited. In this response, we are focusing primarily on the following recommendations:  Stronger institutional agreements between the World Bank and Global Fund through: o A Memorandum of Understanding on country-level collaboration between the two organizations o Global Fund contribution to World Bank Sectorwide Approaches or other co- financing opportunities in the health sector o A trust fund for financing Bank-supervised technical assistance activities o Mechanisms such as the International Health Partnership, the Health-8 Group, Joint Assessment of National Strategies, and Health Systems Platform. We very much welcome the recommendation to improve the partnership with the World Bank by working more closely at the country level, and we believe that we will be able to do so as a result of the transformation process we have recently undertaken. We note with interest the proposal to sign a Memorandum of Understanding on country-level collaboration, but would be interested in discussing further with the World Bank any possible alternative arrangement allowing for increased collaboration in specific country contexts. As to the funding of technical assistance activities to countries, the Global Fund is committed to explore appropriate modalities and mechanisms with its technical partners.  Further collaboration on M&E to define program- and country-level activities, including harmonizing M&E requirements and building evaluation capacity into programs. The Global Fund welcomes this recommendation and has collaborated strongly with the World Bank in developing its M&E approaches and toolkits. The Global Fund will pursue opportunities to further strengthen this collaboration in both monitoring and evaluation.  Build evaluations more routinely in Global Fund programs and initiatives. IEG suggests considerable scope for improved evaluation and learning, both separately and jointly between the two organizations. In particular, IEG recommends including early-stage and impact evaluations in grants and building evaluation capacity in the programs the Global Fund supports. xliv The Global Fund shares this concern and has included systematic strengthening of program evaluation capacity in its newly approved evaluation strategy.  Managing Conflicts of Interest – of partner involvement in CCMs and at all levels. The Global Fund has introduced increasingly detailed conflict of interest policies and monitoring for CCMs, and will continue to strengthen work in this area.  Sustaining the benefits – IEG recommends introducing more coordinated approaches to external financial support at both the global and country levels to promote the sustainability of country programs and avoid dependence on Global Fund financing. This is a major priority of the Global Fund, and the Consolidated Transformation Plan provides a great opportunity to implement this recommendation.  Global Public Policy – IEG suggests working with partners to address global governance issues and involving drug regulatory and enforcement agencies to ensure quality of programs. The Global Fund recognizes this as an important issue, but many of these recommendations go beyond the current mandate of the Global Fund and would be best implemented by our technical partners. Conclusions The Global Fund is committed to learn and change as necessary in relation to the findings and recommendations of the IEG Program Review. The Global Fund is committed to implement most of the recommendations of the Program Review in the context of the implementation of the Comprehensive Transformation Plan, which will represent the corporate priority for 2012. xlv Management Comments: The World Bank Group World Bank management welcomes the opportunity to comment on IEG’s Global Program Review (GPR) on the Global Fund to Fight AIDS, Tuberculosis, and Malaria. Bank management strongly supports IEG’s function as the independent evaluator of the Bank’s performance, including its performance in partnerships. Bank management highly values its partnership with the Global Fund, which plays a critical role in helping developing countries address the three deadly diseases of AIDS, tuberculosis, and malaria. The work of the Global Fund and the Bank is complementary. The Global Fund focuses on supporting treatment and prevention efforts to achieve Millennium Development Goal 6. As a development institution, the Bank takes a holistic approach to health systems – what is preventing people from being healthy; how countries can promote improved health outcomes, especially for the poor, in a sustainable way; and what impact this will have on development. As set out in our Health, Nutrition and Population Strategy endorsed by the Board in 2007, the Bank helps countries deliver better health for their people by strengthening their health systems and supporting investments in all of the sectors that impact health. Bank Management notes that the findings of the IEG review are consistent with the findings of the Global Fund’s own Five-Year Evaluation, which IEG considered to be an independent and quality evaluation, and that IEG’s lessons have been quite widely addressed by the Global Fund’s recent review processes, namely the Comprehensive Reform Working Group and the High-Level Panel. Bank management concurs with IEG on a number of points, including (1) the lessons derived from the Global Fund’s own evaluations; ( 2) the importance of paying attention to health systems strengthening while scaling up response to priority diseases; (3) the importance of having strong M&E systems in place to ensure aid effectiveness; and (4) the value of civil society engagement and participation in development assistance for health. We note, however, that given the fundamentally different operational and financing models of the Global Fund and the Bank, and the relatively recent establishment of the Global Fund, it is difficult to compare the two institutions. Bank management concurs with the Global Fund Secretariat’s response to the IEG Review, and notes that the Global Fund is already implementing most of IEG’s lessons in the context of its Consolidated Transformation Plan recently approved by the Global Fund Board. In addition, we agree with the Global Fund that there are limitations in drawing lessons based on a small sample of countries. The lessons of the IEG Report aim to further strengthen the strategic partnership between the Bank and the Global Fund. The Bank remains strongly committed to collaboration with the Global Fund and to doing our part to halt and reverse the spread of these three diseases. As part of this approach, the Bank pioneered the early scale-up of the global AIDS response, before the creation of the Global Fund, and also has provided substantial support to countries to fight malaria and tuberculosis. In fiscal year 2011, the Bank committed $3 billion for new health investments across multiple sectors to help countries strengthen their health systems, boost disease prevention and treatment, and improve maternal and child health and nutrition. There is effective collaboration and coordination between the Bank and the Global Fund at xlvi both the global and country levels. Through our joint participation with the Global Fund in the International Health Partnership platform (with over 50 development partners) and the Health Systems Funding Platform (with Global Fund, the Global Alliance for Vaccines and Immunization, and WHO), we use joint assessments and fiduciary systems in support of the country’s health sector plan, reducing burdens on countries and in the spirit of the Paris- Accra-Busan principles. Like the Global Fund, we will look at how we can step up our collaboration and strategic partnership in ways that allow us to respond flexibly to the needs of developing countries within a country-owned framework and harmonized with all partners working at country level. In addition, Bank management is in the process of developing a Partnership Program Management Framework and Financial Intermediary Fund Framework to provide an institutional foundation for how the Bank engages with partners (e.g., alignment, selectivity, engagement, review, exit, etc.). These framework papers will go to the Board for consideration in the first half of 2012. In conclusion, while we concur with IEG’s lessons with respect to strengthening the Bank’s partnership with the Global Fund, we believe that the assessment of the Global Fund’s performance is its own responsibility, and we agree with the Global Fund that IEG’s lessons have been widely addressed through the Global Fund’s recent review processes, including the Global Fund’s High-Level Panel and the Consolidated Transformation Plan. As trustee and partner, the Bank will continue to work closely with the Global Fund to maximize our collective impact on global health. xlvii Chairperson’s Summary: Committee on Development Effectiveness The Committee on Development Effectiveness (CODE) considered the documents entitled IEG Global Program Review: The Global Fund to Fight AIDS, Tuberculosis, and Malaria (Global Fund), and the World Bank’s Engagement with the Global Fund together with the Global Fund Secretariat Management Response (CODE2011-0067) and Draft World Bank Management Comments. CODE had endorsed the Approach Paper: Global Program Review of the Global Fund to Fight AIDS, Tuberculosis and Malaria in February 2010. Summary The Committee welcomed the discussion and agreed with the main findings of the Independent Evaluation Group (IEG) Global Program Review (GPR), including the findings on country-level engagement. It noted the important lessons that Review provides with respect to engagement and the considerable convergence of IEG’s findings with the Global Fund’s Five-Year Evaluation. The CODE Chair highlighted that the main messages of the Global Fund’s response supported most of the lessons in the IEG Review, especially those aimed at strengthening the partnership at the global and country levels, underlining the Global Fund’s wish to have more of a strategic partnership with the World Bank. The Chair further noted that such a partnership should avoid rigidities and include a flexible framework for country-based approaches involving the Bank’s country teams in working to fulfill the commitments made in Paris, Accra and Busan. Acknowledging the Global Fund’s constructive response, members agreed on the need to reinforce collaboration in country- level programs and the World Bank’s important role in bringing players together, which could sustain the Global Fund’s momentum. With respect to enhanced engagement, members noted that, while the overall coordination between the World Bank and the Global Fund is working well, there is room for improvement in country-level engagement and for greater collaboration on project preparation, supervision, monitoring and evaluation. Members urged Management to take stock of the findings to determine what can be improved on the ground, particularly with respect to strengthening health systems capacity, country-level support, monitoring, and evaluation collaboration. IEG highlighted the main findings of the review, noting that the global scarcity of resources to fight AIDS, tuberculosis, and malaria will require collective and proactive resource allocation, which creates a need for a clearer institutional mandate for Bank staff to work in closer partnership with the Global Fund. IEG also stressed the need for a strengthened framework for managing the Bank’s financial intermediary funds and explicit engagement strategies for each global partnership program. Management underlined the complementary work of the Global Fund and the Bank, with the latter focusing on the whole health sector, and the different operational and financial models of the two institutions. Management concurred with the finding regarding health systems strengthening, while scaling up the response to priority diseases, the importance of strong monitoring and evaluation systems, and the value of civil society engagement and participation in development assistance in health. With respect to the IEG finding on the need to strengthen the Bank-Global Fund engagement at the country level, management concurred with the Global Fund’s response xlviii that collaboration arrangements should allow partners to respond flexibly to country needs within a country-owned framework. Management added that papers on the Bank’s Partnership Framework and Framework for Financial Intermediary Funds were under preparation, which would cover such issues as engagement, alignment, selectivity, evaluation, and potential conflicts of interest, among others. 1 1. Introduction, Purpose, and Methodology 1.1 This Review has been prepared, first and foremost, for the World Bank’s Executive Board to facilitate an informed discussion about the Bank’s past, current, and future engagement with the Global Fund. Since the Millennium Declaration in 2000, the World Bank has become involved in a growing number of partnership programs like the Global Fund that pool donor resources to finance country-level investments to help countries achieve specific Millennium Development Goals (MDGs), that have inclusive governance structures, and that subscribe to the 2005 Paris Declaration on Aid Effectiveness. Other such programs include the Global Alliance for Vaccines and Immunization (GAVI, established 2000), the Global Partnership for Education (2002), the Climate Investment Funds (2008), and the Global Agriculture and Food Security Program (2010). The World Bank generally plays three major roles in these programs: (a) as a trustee of donor funds supporting the program; (b) in the corporate governance of the program, and (c) as a development partner at the global and country levels. 1.2 The principal purpose of this Global Program Review (GPR) is to learn lessons from the experience of the Global Fund about (a) the design and operation of these large global partnership programs that are financing country-level investments, (b) the engagement of the World Bank with these programs, and (c) the evaluation of these programs. The Review has an intensive focus on the Bank’s engagement with the Global Fund at the country level because of the potential for competition or collaboration between Global Fund-supported activities and the Bank’s lending operations at the country level. Therefore, it also focuses on the design and operation of the Global Fund-supported activities at the country level. The review framework in Appendix A provides the specific issues and questions addressed. 1.3 Like other GPRs, this Review is based on an external evaluation that was commissioned by the governing body of the program — in this case, the Five-Year Evaluation (FYE) of the Global Fund, launched by the Global Fund Board in November 2006. The final Synthesis of Study Areas 1, 2, and 3 was issued in March 2009. The Review (a) assesses the independence and quality of that evaluation; (b) validates the findings of the evaluation; and (c) assesses the extent and nature of the Bank’s engagement with the Global Fund at the global and country levels since the Global Fund was founded in 2002. 1.4 By design, this GPR does not compare the effectiveness of the World Bank’s health sector operations with those of the Global Fund. Nor does it compare the effectiveness of the Global Fund model with that of other financing entities such as the Global Environment Facility. Both comparisons are explicitly beyond the scope of the Review. Nor does the Review assess the effectiveness of Global Fund structures at the corporate level, such as the Global Fund Board and Secretariat, with the exception of the Technical Evaluation Reference Group (TERG) insofar as this body was responsible for overseeing the FYE. Organization of the Review 1.5 The Review has one primarily descriptive chapter, three substantive chapters, and a conclusion. Chapter 2 describes the origin and evolution of the Global Fund; its objectives 2 and design; and its governance, management, and financing to provide context for the subsequent chapters of the Review. It also describes the roles of the World Bank in the program, the conduct of the FYE that was completed in 2009, and the principal impacts of the evaluation on the Global Fund to date. 1.6 Chapter 3 presents IEG’s findings in relation to the operation of the Global Fund at the country level, based primarily on visits to a sample of six countries in which both the Global Fund and the World Bank have been active in the health sector. The chapter is organized in accordance with eight of the nine major findings of the FYE, as presented in the FYE Synthesis Report. (IEG did not address the ninth major finding because this related to the global governance of the Global Fund, not its country-level activities.) IEG’s country visits, which took place from April to June 2010, sought to confirm the findings of the FYE and assess changes (either improvements or deterioration) in the intervening three years since the FYE country visits were conducted in 2007 as part of Study Area 2, using the FYE and the Study Area 2 country reports as a baseline. 1.7 Chapter 4 presents IEG’s findings with respect to the World Bank’s engagement with the Global Fund at the global and country levels. The first part of the chapter addresses the Bank’s engagement at the global level, including the roles that the Bank plays at the corporate level of the Global Fund as well as the initiatives associated with the Global HIV/AIDS Program (GHAP) and the International Health Partnership (IHP) that have provided additional avenues for World Bank-Global Fund engagement at the country level. Therefore, the first part of this chapter also provides context for the second part on the Bank’s engagement with the Global Fund at the country level. 1.8 Chapter 5 assesses the independence and quality of the FYE and draws lessons from this experience for the evaluation of other global partnership programs. It assesses independence and quality based on the standard framework that IEG uses for this purpose (Appendix Table A-3), which is based on the Indicative Principles and Standards in the Sourcebook for Evaluating Global and Regional Partnership Programs (IEG and OECD/DAC 2007). 1.9 Chapter 6 is a concluding chapter that presents the major lessons of this Review (a) for the Global Fund, (b) for the World Bank, and (c) for the evaluation of global and regional partnership programs. Methodology 1.10 The findings and lessons of this Review are based on the following:  Desk reviews of key documents, including the final FYE reports; Global Fund and World Bank strategies; and evaluations of World Bank activities in health, nutrition, and population (HNP).  Review of the academic literature on the Global Fund.  Portfolio analysis of Global Fund-supported activities and World Bank HNP lending operations. 3  Structured interviews with Global Fund staff, with World Bank staff who have been involved with the Global Fund and its activities, and with other stakeholders.  Visits to a sample of six recipient countries (Brazil, Burkina Faso, Cambodia, Nepal, the Russian Federation, and Tanzania) during April–June 2010 to consult with country-level stakeholders about Global Fund-supported activities in each country, and the World Bank’s engagement with these activities.  An in-depth assessment of a World Bank-supported health project in Lesotho specifically designed to increase the capacity of the country to effectively use Global Fund grants for HIV/AIDS.  An electronic survey in March 2011 of Global Fund staff and World Bank project managers of HNP projects on the engagement between the Global Fund and the World Bank at the country level, followed by a focus group of World Bank project managers to discuss the survey results.  A detailed comparison of the monitoring and evaluation (M&E) systems of the Global Fund and the World Bank. 1.11 The six countries visited were a stratified random sample (two in Africa and one in each of the other four regions) of countries in which both the World Bank and the Global Fund have been active in combating communicable diseases and/or strengthening health systems since 2002, after eliminating six countries that IEG was also visiting at the same time for an evaluation study on trust funds. When it was not possible to arrange visits to two of the countries initially selected (Nigeria and India), these were replaced by Burkina Faso and Nepal (also randomly selected). The draft reports on each country were shared with country-level stakeholders, World Bank project managers, and Global Fund Portfolio Managers (FPMs), and revised in the light of comments received. 1.12 The purpose of these country visits was threefold: (a) To validate the findings of the FYE and assess changes between 2007 and 2010, using the FYE and the four Study Area 2 country reports as a baseline — presented in Chapter 3. (b) To learn about the nature and scope of the World Bank’s engagement with Global Fund-supported activities in the six countries — presented in Chapter 4. (c) To assess the familiarity of country-level stakeholders with the findings of the FYE and its impacts on the Global Fund — presented in Chapter 5. 1.13 The findings from the six country visits are intended to be representative of experiences in other, similarly situated countries. They are not intended to single out the performance of individual stakeholders in the individual countries visited. 1.14 The electronic survey was administered to project managers of Bank-supported health projects that were disbursing when, or approved after, the Global Fund became active in the same country (the date of its first grant commitment to the country). A parallel survey was also administered to Global Fund Secretariat staff in the Country Programs Cluster, the External Relations and Partnerships Cluster, and the Strategy, Performance and Evaluation Cluster. The survey results are contained in Appendix Q. 4 1.15 This Review was initiated before the High-Level Independent Review Panel on Fiduciary Controls and Oversight Mechanisms of the Global Fund was commissioned in February 2011, and it was drafted before their final report, Turning the Page from Emergency to Sustainability, was issued on September 19, 2011. While the two studies are complementary and overlap to some extent, they were conducted independently of each other, for different audiences, and for different purposes. 5 2. Overview of the Global Fund 2.1 Since its founding in 2002, the Global Fund has become by far the largest of the 15 global health partnerships in which the World Bank is involved, and supported by the largest financial intermediary trust fund that the Bank currently administers. The present chapter describes the origin and evolution of the Global Fund, its objectives and design, and its governance, management and financing to provide context for the subsequent chapters of this Review. It also describes the roles of the World Bank in the program, the conduct of the FYE that was completed in 2009, and the principal impacts of the evaluation on the Global Fund to date. Origin of the Global Fund 2.2 A confluence of world events in the international health arena led to the creation of the Global Fund. HIV/AIDS was spreading across the developing world in the 1990s, exacting a toll on lives and reversing gains in development at an unprecedented pace. At the same time, there was a resurgence in tuberculosis and malaria in large parts of the globe due to weak control efforts and growing drug resistance. A global consensus was emerging that too little was being done, and done too slowly, to effectively address the three scourges. HIV/AIDS, in particular, had become the defining epidemic of our time, and while there were drugs and a growing body of knowledge to mitigate its impact, these were simply not available or affordable in the developing world. 2.3 Governments in developed countries began responding to strong advocacy movements to marshal large increases in financing to combat HIV/AIDS. The G8 meeting in Denver in 1997 was among the earliest of such occasions, and led to strong donor commitments to combat AIDS. By the time of the G8 meeting in Okinawa in 2000, heads of state and government had broadened their commitments to include tuberculosis and malaria, noting that bilateral and multilateral efforts were woefully insufficient. 2.4 Similar events were also taking place in Africa, the continent most affected by the three diseases. At a special Summit of the African Union in Abuja in 2001, heads of state and government lent weight to the fight, and the UN Secretary General Kofi Annan called for the creation of a special fund for this cause. That same year, the World Health Organization (WHO) Commission on Macroeconomics and Health provided economic arguments in support of these endeavors. Chaired by Jeffrey Sachs, the Commission demonstrated the detrimental effects of the three pandemics on growth and poverty alleviation, and called for urgent reforms and massive new financial resources to combat the diseases. 2.5 A Special Session of the UN General Assembly on AIDS in June 2001 endorsed the creation of the fund. The next month, the donor community pledged $1.3 billion at the G8 Summit in Genoa. The U.S., French and U.K. governments led the way. (See Appendix B for the complete Global Fund timeline through 2011.) 2.6 Starting in the mid-1990s, a number of new global health partnerships had been established, including the Joint United Nations Program on HIV/AIDS (UNAIDS) in 1994, the International AIDS Vaccine Initiative in 1996, Roll Back Malaria (RBM) in 1998, the Medicines for Malaria Venture in 1999, GAVI in 2000, and the Stop Tuberculosis Partnership 6 (Stop TB) in 2001. Researchers were also producing new drugs and therapies to combat the three diseases: antiretroviral drugs (ARVs) for HIV; new anti-tubercular drugs and the Directly Observed Treatment Short-Course (DOTS) for tuberculosis; and new anti-malarial drugs, artemisinin combination therapy (ACT), and long-lasting insecticide-treated bed nets for malaria. But these innovations required new modalities and financing to deliver them and to facilitate their use. The sheer magnitude and global nature of the AIDS, tuberculosis, and malaria pandemics required concerted and well-coordinated responses on a global scale — far beyond the capacity of individual donors, or that of UNAIDS, RBM, and Stop TB (which were largely technical assistance programs) to address. 2.7 The new partnership programs reflected not only the need for collective action to address global challenges but also the involvement of new actors and constituencies in development and dissatisfaction with the ability of existing aid mechanisms to address emerging global challenges. New philanthropies (such as the Gates Foundation) and international nongovernmental organizations (NGOs) advocated new approaches to development assistance emphasizing country-led development, greater participation of beneficiaries and civil society groups, and stronger ties with the private sector to tap its finances, innovation, and the power of the market. 2.8 Large partnership programs such as the Global Fund and GAVI that are financing country-level investments on a large scale, usually on a grant basis to help countries achieve specific MDGs, have several common features. First, they pool donor resources to finance country-level investments, which distinguishes them from the large majority of much smaller global and regional partnership programs (GRPPs) that are primarily financing technical assistance, or generating knowledge about development. Second, they employ inclusive governance structures in which membership on the governing body is not limited to financial contributors but is also extended to other stakeholders, including recipient countries, civil society organizations (CSOs), and the commercial private sector. Third, they generally subscribe to the 2005 Paris Declaration principles of country ownership, alignment, harmonization, managing for results, and mutual accountability.1 The programs also raise funds from nontraditional sources outside the public sector, including private foundations and the business community. 2.9 After the G8 Summit in Genoa, a Transitional Working Group was formed in August 2001 to develop general organizational guidelines. The new Fund would need to be a visible entity to mobilize the needed additional resources; to use quick and efficient modalities to operationalize and disburse the funds; and to forge strong ties with country partners, CSOs, and the private sector. The Fund should complement and not duplicate the existing multilateral and bilateral assistance agencies. Indeed, as a financing entity and not an implementing agency, it would need to rely strongly on its development partners to expand and accelerate the response to the pandemics. 2.10 The Global Fund for AIDS, Tuberculosis and Malaria was officially established in January 2002 when the Transitional Working Group was converted into the founding Global Fund Board and held its first meeting. The chair and vice chair were elected, operating 1. Existing multilateral organizations such as the International Development Association also pool donor resources to finance country-level investments and have subscribed to the Paris Declaration principles. 7 procedures adopted, and a staff member of the Swedish International Development Agency was selected to be the interim head of the Secretariat. Richard Feachem, former World Bank Director for HNP, was appointed the first Executive Director and head of the Secretariat at the Fund’s second Board meeting in April 2002. Objectives and Design 2.11 The stated purpose of the Global Fund is “to attract, manage and disburse additional resources through a new public-private partnership that will make a sustainable and significant contribution to the reduction of infections, illness and death, thereby mitigating the impact caused by HIV/AIDS, tuberculosis and malaria in countries in need, and contributing to poverty reduction as part of the Millennium Development Goals� (Global Fund 2002). Its Framework Document also establishes principles, scope, financing, country processes, eligibility criteria, grant application processes, and monitoring and fiduciary responsibilities (Appendix C). Seven Guiding Principles form the core values of the Global Fund (Box 1). National ownership of disease-control programs and country-led formulation and implementation processes reflect a strongly held principle and a firm belief that these approaches offer greater promise of fairness and sustainability. The meaning of “country� is not limited to the government but encompasses all other country-level stakeholders, including CSOs, the private sector, and affected communities. Box 1. Global Fund Guiding Principles A. The Fund is a financial instrument, not an implementing entity. B. The Fund will make available and leverage additional financial resources to combat HIV/AIDS, tuberculosis and malaria. C. The Fund will base its work on programs that reflect national ownership and respect country-led formulation and implementation processes. D. The Fund will seek to operate in a balanced manner in terms of different regions, diseases, and interventions. E. The Fund will pursue an integrated and balanced approach covering prevention, treatment, and care and support in dealing with the three diseases. F. The Fund will evaluate proposals through independent review processes based on the most appropriate scientific and technical standards that take into account local realities and priorities. G. The Fund will seek to establish a simplified, rapid, innovative process with efficient and effective disbursement mechanisms, minimizing transaction costs and operating in a transparent and accountable manner based on clearly defined responsibilities. The Fund should make use of existing international mechanisms and health plans. Source: Global Fund, 2002, “Framework Document of the Global Fund to Fight AIDS, Tuberculosis and Malaria,� pp. 1-2. 2.12 Global Fund resources are intended to supplement existing efforts to deal with the three diseases — over and above the resources that multilateral and bilateral agencies as well as the governments of recipient countries were already spending. It seeks to strengthen country-level coalitions among public and private actors to reduce the burden of the three diseases. Its founding principles call for it to be efficient, effective, and inclusive, and to act in a transparent and accountable manner. The Fund has a broad mission statement and goals, but has not set physical 8 targets for disease reduction. It measures itself against its Figure 1. Global Fund: Grant purpose and the guiding principles on which it was founded. Preparation and Approval Processes 2.13 IEG has found the Global Fund to be the most transparent of the 21 GRPPs that IEG has reviewed in the THE BOARD last five years. The Global Fund Board has mandated a Issues Calls for Proposals high degree of transparency since its founding and the Global Fund Secretariat has effectively implemented this mandate to the extent that it has become an integral part of COUNTRY COORDINATING its organizational culture. In the interests of its developing MECHANISM (CCM) country clients who have less access to broadband, its Devises National Strategy, Submits Proposals Web site is clean with a minimum of graphics. This site is also organized around its support to individual countries, because the Fund is receiving grant proposals and SECRETARIAT financing them on a country basis. Screens for Eligibility GRANT PREPARATION AND APPROVAL PROCESS TECHNICAL 2.14 Each round of Global Fund grants starts with a call REVIEW PANEL (TRP) for proposals by the Global Fund Secretariat on behalf of the Reviews and Recommends Board (Figure 1). The first call for proposals (Round 1) was for Funding issued in February 2002, and the most recent (Round 11) in August 2011. Each round has contained specific policies and THE BOARD guidelines, including eligibility and minimum requirements, Approves Funding published on the Global Fund Web site. Each call for for First Two Years proposals may also prioritize specific themes for that round.2 2.15 The Global Fund requires each country, with CCM limited exceptions,3 to establish a Country Coordinating Designates Organization Mechanism (CCM) to review and endorse funding to Serve as Principal Recipient proposals for submission to the Secretariat, based on a national strategy for combating the disease in question. PRINCIPAL RECIPIENT CCM members are drawn from the government, CSOs, Implements Grant the private sector, academia, affected communities, and external development partner agencies such as bilateral Source: Global Fund Web site. agencies, WHO, UNAIDS, and the World Bank. 2.16 Conceptually, the collective knowledge and ability of CCM members provides the capacity to prioritize country needs for each disease, develop grant proposals in accordance with a national strategy, and identify gaps in financing. There was a strong expectation among members of the Global Fund Board, although this was not formalized in writing, that 2. For example, the minimum eligibility requirements for the composition of the CCM were revised in Round 10. These revisions aimed to enhance (a) inclusiveness; (b) partnerships between government, private sector, and NGOs; (c) participation of affected communities; and (d) alignment with national policies and processes. 3. In the case of the Russian Federation, for example, a consortium of five NGOs, already active there and led by the Open Health Institute, submitted their own proposals for addressing HIV/AIDS and tuberculosis to the Global Fund in Round 3, in the absence of the government’s willingness to establish a CCM. 9 development partner agencies would contribute formal and informal technical assistance to this process as needed. Proposed activities were expected to be part of the overall national program for HIV/AIDS, tuberculosis, or malaria, and linked to other domestic and donor- funded programs. 2.17 Both CCM and non-CCM applicants who submit proposals are first screened by the Screening Review Panel of the Global Fund Secretariat for eligibility and the completeness of their proposals, according to established criteria relating to membership and representation, transparency, and management of conflicts of interest. Proposals from eligible applicants are then reviewed by the Technical Review Panel (TRP), which is made up of technical, scientific, and programmatic experts. The TRP makes its funding recommendations to the Board based on the technical merit of each proposal in terms of effective and proven interventions, cost- effectiveness, potential for scaling up and impact, strengthening of communities, alignment with government/national systems, and a measurable results framework. Over the last five rounds, about 80 percent of the applicants have been found eligible, and about 50 percent of eligible proposals have been recommended to the Board for approval (Table 1). The Board has so far approved all TRP-recommended proposals, but not always at the requested funding levels. Table 1. Success Rate of Proposals Approved for Grant Funding Screening Process Technical Review Process Number Total Eligible Success Number of recommended Success Round Applicants Applicants Rate proposals Rate for funding 6 144 108 75% 196 84 43% 7 110 88 80% 150 74 49% 8 125 98 78% 94 174 54% a 9 121 101 83% 159 85 53% 10 117 105 90% 150 79 53% Total 617 500 81% 792 396 50% Source: Global Fund Secretariat a. In Round 9, there were two cases (Kyrgyz Republic and Mali) in which one proposal submitted by the applicant was screened out by the Screening Review Panel, while the other proposal was deemed eligible and reviewed by the TRP. 2.18 Once a grant is approved, the CCM nominates one or more organizations to be the Principal Recipients, or lead implementing agencies for the grant. These may be a government department or agency, a CSO, an academic institution, or even an international organization such as the United Nations Development Programme (UNDP). Then, before a grant agreement is negotiated and signed, the Global Fund Secretariat contracts with a Local Fund Agent (LFA) to assess the financial, administrative, and implementation capacity of the nominated Principal Recipients to implement the approved grant.4 4. The assessment determines whether the nominated Principal Recipient possesses the minimum required capacities in five functional areas: (a) financial management and systems, (b) program management, (c) Sub- Recipient management, (d) pharmaceutical and health product management, and (e) M&E. 10 2.19 As the fiduciary agent of the Global Fund Secretariat in the country, the LFA plays an important financial oversight and risk management role during the entire grant implementation process, starting with this assessment. If the nominated Principal Recipient fails the LFA assessment, then the CCM nominates a replacement. If the nominated Principal Recipient passes the LFA assessment, then the Global Fund Secretariat starts to negotiate a grant agreement with the Principal Recipient. This specifies both the conditions to be met preceding the first grant disbursement and the programmatic indicators and milestones to be used by the Principal Recipient to track and report on progress.5 GRANT IMPLEMENTATION PROCESS 2.20 Once the grant agreement is signed, the Global Fund Secretariat instructs the World Bank, as the trustee of the Global Fund, to release funds to the Principal Recipient to implement prevention, treatment, and care and support activities (Figure 2). It typically takes 12–15 months from grant approval by the Board to the first release of funds by the trustee. Typically, Principal Recipients also enlist other organizations such as service-delivery NGOs — known as Sub- Recipients — to help implement the planned activities. The CCM has the overall responsibility for the oversight of grant implementation from the country perspective in accordance with the Global Fund “Guidance Paper on CCM Oversight.� Figure 2. The Global Fund: Grant Implementation Processes CCM Oversight PRINCIPAL RECIPIENT 1. Request for disbursement 4. Disbursement with documented of grant funds 5. Disbursement results 6. Reporting GLOBAL FUND TRUSTEE SUB� LOCAL FUND THE WORLD BANK RECIPIENTS AGENT 3. Instructions 2. Data verification to disburse and advice of release of funds SECRETARIAT Source: Global Fund Web site 5. UNDP is a special case. The Global Fund and UNDP reached an umbrella agreement in 2003 under which the UNDP could implement Global Fund grants using its own regulations, policies, and procedures when it acts as a Principal Recipient of Global Fund grants. 11 2.21 The Global Fund follows the principles of performance-based funding (PBF) in making disbursement decisions. Grants are initially approved for two years (Phase 1) and renewed for up to three additional years (Phase 2), based on the performance of the grant-funded activities. Tied to PBF are detailed and documented requirements and outputs for grant-level monitoring. Each grant agreement contains a disease-specific performance framework outlining the performance expected over the lifetime of the grant and containing key indicators and targets that are used to measure outputs and coverage on a routine basis. Funding is disbursed incrementally every three to six months throughout the life of the grant. The Principal Recipient prepares Progress Update and Disbursement Requests (PUDRs), which link the historical and expected program performance with the level of financing to be provided to the Principal Recipient. The LFA reviews these periodic requests for funding, undertakes site visits to verify results, reviews the Principal Recipient’s audit reports, and then makes a confidential recommendation to the Global Fund Secretariat to disburse (or not to disburse) the funds. When the initial two-year grant commitment period is completed, the CCM requests further funding for the remaining three years of the approved grant. The LFA again reviews these requests before the Global Fund Secretariat instructs the trustee to release additional funds.6 2.22 IEG’s findings in relation to the design and operation of the Global Fund at the country level are presented in Chapter 3 of this Review. Governance and Management BOARD 2.23 Like most of the GRPPs in which the World Bank is involved, the Global Fund employs a constituency-based stakeholder model of governance in which membership on the governing body includes stakeholders in addition to financial contributors. The Board comprises eight representatives of donor governments, seven representatives of recipient governments, and one representative each from private foundations, affected communities, developed country NGOs, developing country NGOs, and the commercial private sector (Appendix E). The Board also has six nonvoting ex officio members: the Global Fund Executive Director, UNAIDS, WHO, the World Bank, one representative from other development partners (RBM, Stop TB, and UNITAID), and Switzerland.7 The Board meets at least semi-annually and is responsible for the overall governance of the organization, including the final approval of grants vetted by the TRP. 6. Because the Global Fund provides grants for an initial two-year period, its approach to PBF is not, strictly speaking, “output-based aid� as this term is used in the development literature in relation to delivering basic infrastructure services such as water, sanitation, or electrical connections. Under a typical output-based aid scheme, the contracted service provider (usually a private firm) is responsible for pre-financing the project until the services or outputs have been delivered. Only after these have been delivered and verified by an independent agent does the service provider receive the public subsidy to deliver the services or outputs. See the Global Partnership on Output-Based Aid, “Output-Based Aid – Fact Sheet,� August 2010. 7. The Deputy Director of the United Kingdom Department for Economic Development is the current Board Chair and the Lesotho Minister of Health and Social Welfare is currently the Vice Chair. 12 2.24 An even broader group of stakeholders8 meet at the Partnership Forum that is held every other year. Considered a formal ancillary body of the Board, the Forum provides feedback to the Board on the Fund’s strategic direction and implementation framework. The Board is not formally accountable to the Forum, but the Forum plays an important role in mobilizing and sustaining political commitment from a very broad constituency.9 Four Forums have been held so far (Bangkok, Durban, Dakar, and São Paulo), with approximately 400 stakeholders in attendance each time. An e-forum facilitates an ongoing online debate/ dialogue among members of the Forum in between the meetings. 2.25 The Board is supported by six committees (Figure 3), a Technical Evaluation Reference Group (TERG), and the Office of Inspector General (OIG). The mandates of the respective committees are reviewed during Board meetings, and more responsibilities have been delegated to them as a result of the FYE. Figure 3. The Global Fund: Organizational Chart, June 2011 Board Ethics Finance & Policy & Portfolio Affordable Market Committee Audit Strategy Committee Medicines Dynamics & Committee Committee Facility Commodities for Malaria Committee Committee Office of the Technical Evaluation Inspector-General Reference Group Office of the Executive Director Global Fund Secretariat • Executive Director • Deputy Executive Director • Chief of Staff Corporate Country Finance External Strategy, Technical Service Programs Cluster Relations & Performance & Review Cluster Cluster Partnerships Evaluation Panel Cluster Cluster Source. Constructed by IEG from information on the Global Fund Web site. 8. Stakeholders include CSOs, service providers, technical experts, people affected with the disease, etc., who are aligned with the Global Fund mission, but are not necessarily actively engaged in Global Fund processes. 9. About a third of the GRPPs in which the World Bank is involved have similar such forums to involve a broader group of stakeholders in the governance of the program. Where the governing body is formally accountable to the forums, they are usually called annual general meetings. 13 2.26 The Ethics Committee guides the overall value system and code of conduct of the organization, assisting in overseeing the management of reputational risks. The Finance and Audit Committee assists the Board on fiscal management policies and processes and leads the Fund’s replenishment process. The OIG, which was established in July 2005, reports to the Board through the Finance and Audit Committee. The Policy and Strategy Committee assists the Board on core governance issues of the Global Fund, including processes and structures of the Board, the Partnership Forum, and CCMs. The largest of the committees, it assists the Board on overall strategic planning and resource mobilization policies. The TERG reports to the Board through this Committee. The TERG oversees independent evaluations (such as the FYE) on behalf of the Board and its Committees, and advises the Global Fund Secretariat on evaluation approaches and practices, independence, reporting procedures and other technical and managerial aspects of M&E at all levels of the program. 2.27 The Portfolio Committee assists the Board on all policy and strategic matters concerning the grant portfolio, including operational partnerships with development partner agencies to facilitate expanded technical assistance support at the country level. It leads on issues pertaining to guidelines for grant proposals, TRP membership and review criteria, and appeal processes. The last two committees are ad hoc committees constituted in 2008. The Affordable Medicines Facility for Malaria (AMFm) Committee oversees the Fund’s new business line in the affordable provision of ACT combination drug therapy to the sick who would otherwise be paying 10 times more, or resort to using old antimalarials that are no longer effective due to drug resistance. The Committee on Market Dynamics and Commodities reviews and develops options for the Global Fund to better utilize its buying power in relation to purchasing pharmaceuticals and medical commodities, which account for roughly half its expenditures of grant funds. The Committee also oversees measures to improve aligning and harmonizing Global Fund procurement systems with those of other major donors for the three diseases. SECRETARIAT 2.28 The Global Fund Secretariat in Geneva is responsible for day-to-day operations, including mobilizing resources; administering grants; providing financial, legal, and administrative support; and reporting information on the Global Fund’s activities to the Board and the public. The Executive Director and about 560 employees representing about 100 nationalities work at the Secretariat’s headquarters. 2.29 The office of the Executive Director oversees Board relations, Secretariat support for the TERG, and the rolling out of the Fund’s new grant architecture (see below). Among the five functional clusters in the Secretariat, the Corporate Service Cluster is responsible for personnel issues, administration, information systems, and legal affairs. The Finance Cluster, headed by the Chief Financial Officer, is responsible for all program accounting and financial reporting. 2.30 The Country Programs Cluster supports country-level activities, including dedicated teams to support CCMs, LFAs, and the grant renewal process. In this cluster, the Unit Directors (for each region) are the closest equivalent to regional HNP Sector Managers in the World Bank, Regional Team Leaders (for each subregion) are the closest equivalent to Lead Specialists or Coordinators, and FPMs are the closest equivalent to Task Team Leaders (the Bank’s term for project managers). 14 2.31 The External Relations and Partnership Cluster is responsible for consolidating and building partnerships with constituencies in client countries, and with multilaterals, bilateral donors, CSOs, and the private sector at the global level. It oversees all media, communications, and branding of the Global Fund and leads the Secretariat’s efforts in resource and demand mobilization at the global level, including recent innovations such as the “Debt2 Health Initiative� (see below). The Strategy Performance and Evaluation Cluster is responsible for overall strategy, policies, M&E, aid effectiveness, and the TRP proposal review process. It is also responsible for facilitating synergies between the various new initiatives of the Fund, including the AMFm and the Voluntary Pooled Procurement.10 Financing RESOURCE MOBILIZATION 2.32 Donor contributions to the Global Fund have increased from about $800 million in 2002 to about $3 billion annually during the years 2007 to 2010 (Appendix Table F-2). The top 15 donors have accounted for 94 percent of all contributions to date (Figure 4). The United States has been by far the largest contributor, with more than $5 billion in contributions. France has been a strong second with close to $2.5 billion, followed by Japan, Figure 4. Global Fund: Top 15 Donors (as of December 2010) United States France Japan Germany United Kingdom European Commission Italy Canada Spain Gates Foundation Netherlands Sweden Norway Russian  Federation Denmark 0 1,000 2,000 3,000 4,000 5,000 6,000 Millions  of U.S. Dollars Source. World Bank as Trustee for the Global Fund. See Appendix Table F-2 for more details. 10. Voluntary Pooled Procurement allows countries to receive price reductions by purchasing pharmaceuticals and other commodities in bulk. 15 Germany, the United Kingdom, and the European Commission with $1.2–1.3 billion each. The Gates Foundation has been largest foundation contributor with $670 million, and the Russian Federation the largest non-OECD country with $257 million. 2.33 To mobilize resources, the Global Fund follows a periodic replenishment model on a voluntary basis for all public donors, complemented by ad hoc contributions from other donors. There have been three replenishments so far. The third replenishment, which concluded in October 2010, raised $11.7 billion for the 2011–13 period. 2.34 The Fund has undertaken a strong effort to identify and mobilize new resources, including private and foundation sources, for the Global Fund, both at the global level and within grant-recipient countries. One example is the Product Red Initiative launched by Bono of U2 and Bobby Shriver of ONE/DATA at the World Economic Forum in Davos in January 2006. Partner companies create a product with the Product Red logo and donate a percentage of their profits to the Global Fund in return for the opportunity to increase their own revenues through the Product Red products that they sell.11 “Debt2Health� is another innovation launched in 2007, in which donors forgo debt repayment and recipient countries invest 50 percent of the debt forgiven to support Global Fund activities in their respective countries. Thus far, Côte d’Ivoire, Indonesia, and Pakistan have participated in this initiative through debt cancelled by Australia and Germany. Chevron became a Corporate Champion of the Global Fund in 2008, committing $5 million to Global Fund grant recipients in each of six countries (Angola, Indonesia, Nigeria, the Philippines, South Africa, and Thailand). These focus on improving the reach and performance of Global Fund grants through capacity development initiatives, joint advocacy, awareness campaigns, and workplace wellness initiatives. EXPENDITURES 2.35 Grant disbursements to Principal Recipients in beneficiary countries have represented about 92.7 percent of its expenditures since 2002 (Table 2). The investment income derived from donor funds received but not yet disbursed has more than covered the Fund’s cumulative administrative costs of 7.3 percent for disbursing these grants. The largest categories of administrative expenses have been staff salaries and benefits (2.8 percent), other Secretariat expenses (2.6 percent), and fees to the LFAs (1.7 percent). The Fund has reimbursed the World Bank about $2.0–3.0 million a year (0.15 percent of total expenditures) for administering the Global Fund trust fund.12 11. Participating companies include Nike, American Express (U.K.), Apple Inc., Starbucks, Converse, Bugaboo, Penguin Classics (U.K. & International), Gap, Emporio Armani, Hallmark (U.S.), and Dell. 12. By way of comparison, the World Bank’s administrative costs as a percentage of total expenditures and disbursements (loans, credits, grants, and recipient-executed trust funds) were 9.0 percent over the same time period (FY02–10). See Appendix Table F-2. Thus, the administrative costs of the two organizations are comparable when one takes into account the following factors: the Bank has a resident Board, it has large research and training departments that are generating and disseminating knowledge about development, the Bank has become a significant administrator of trust funds, and it spends more resources on self-evaluation and independent evaluation of completed projects, both of which are recorded as part of administrative expenditures. 16 Table 2. Global Fund: Annual Income and Expenditures, Calendar Years 2002–10 2002 2003 2004 2005 2006 2007 2008 2009 2010 Share Income   Contributions 880.8 1,416.7 1,254.7 1,430.3 2,429.6 2,963.8 3,714.2 2,590.4 2,329.0 95.1% Bank and Trust 10.1 28.2 33.8 58.9 126.5 240.5 289.7 150.4 149.7 5.4% Fund income Foreign currency 0.0 0.0 0.0 0.0 0.0 -50.9 -83.7 124.8 -97.1 -0.5% gain/(loss) Total Income 890.9 1,444.9 1,288.5 1,489.3 2,556.1 3,153.4 3,920.2 2,865.7 2,381.5 100.0% Expenditures   Grants disbursed 0.9 231.2 627.5 1,054.3 1,307.0 1,710.8 2,259.3 2,749.5 3,060.7 92.7% Employment 2.8 9.8 16.9 25.1 30.6 41.1 71.7 91.7 107.1 2.8% expenses Other Secretariat 7.0 10.8 19.6 27.3 28.9 41.1 63.1 74.8 90.3 2.6% expenses LFA fees 0.7 10.1 12.2 19.2 23.9 32.9 27.1 57.1 57.9 1.7% CCM funding 0.0 0.0 0.0 0.0 0.0 0.0 1.4 2.2 4.1 0.1% Board constituency 0.6 0.0% funding Trustee fee 2.3 1.9 2.2 2.3 2.4 2.3 2.4 2.6 2.7 0.15% Foreign currency 0.0 0.0 0.0 0.0 0.0 13.6 -4.9 -7.5 -35.8 -0.2% (gain)/loss Uncollectible 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.1 26.7 0.2% contributions Total 13.7 263.8 678.3 1,128.2 1,392.8 1,841.6 2,420.0 2,971.4 3,314.3 100.0% Expenditures Income - 877.2 1,181.1 610.3 361.1 1,163.3 1,311.8 1,500.3 -105.7 -932.9 Expenditures Movement in un- 51.1 832.1 226.9 454.9 510.5 871.7 110.5 1,248.8 160.5 disbursed grants a Source: Global Fund Annual Reports. See Appendix Table F-1 for more details. a. The annual change in the value of grant commitments that have not yet been disbursed. 2.36 Global Fund grants can be used to support investments (pharmaceuticals, medical commodities and diagnostics, bed nets), surveillance studies and surveys, technical assistance to build capacity, actual service delivery, and salaries. Grants target the three diseases, plus strengthening of underlying cross-cutting health systems, such as procurement, supply management, human resources, and health information systems. A snapshot of the types of activities, their scale, and geographical distribution is as follows, based on the U.S. dollar amount of grants recommended by the TRP after 10 rounds of proposals:  By disease: HIV/AIDS (43 percent), malaria (35 percent), tuberculosis (16 percent), health systems strengthening (6 percent)  By region: Sub-Saharan Africa (62 percent), East Asia and the Pacific (13 percent), Europe and Central Asia (8 percent), Latin American and the Caribbean (7 percent), South Asia (8 percent), Middle East and North Africa (2 percent)  By income level of recipient country: low income (72 percent), lower middle-income (24 percent), upper middle-income (4 percent) 17  By expenditure category: medicines and pharmaceutical products (18 percent), health products and equipment (17 percent), human resources (14 percent), training (11 percent), infrastructure and equipment (10 percent), planning and administration (6 percent), M&E (4 percent), living support to clients/target populations (5 percent), communications materials (5 percent), other (9 percent)  By type of Principal Recipient: government agency (55 percent), CSOs (24 percent), multilateral organizations (16 percent), private sector (2 percent), other (2 percent). 2.37 The Global Fund committed $18.3 billion and disbursed $14.0 billion in grants to recipient countries between July 2002 and June 2011 — corresponding to the World Bank’s fiscal years 2003–11. By way of comparison, the Bank committed $19.2 billion and disbursed $15.8 billion in loans, credits, and grants during the same time period to the health sector.13 Although the orders of magnitude have been the same, the trends have been somewhat different (Figure 5). World Bank commitments and disbursements were relatively constant at about $1.5 billion a year during 2000–08, the disbursements reflecting commitments made both before and after the Global Fund was founded in 2002. Global Fund disbursements have been rising steadily since 2002, reflecting the rapidly growing commitments during its first three years of operation, before declining significantly in 2006. The World Bank significantly increased its commitments during the last three years, 2009–2011, as part of the Bank’s response to the global financial crisis, which then resulted in higher disbursements in 2010–11. The Global Fund also increased its commitments and disbursements significantly in 2010–11. Figure 5. Global Fund Grants and World Bank Health Projects, Fiscal Years 2000–11 Commitments (year of approval) Disbursements US$ Billions US$ Billions 5.0 3.5 4.5 3.0 4.0 Global Fund World Bank 3.5 2.5 Global Fund 3.0 2.0 2.5 World Bank 1.5 2.0 1.5 1.0 1.0 0.5 0.5 0.0 0.0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 Source: Global Fund and World Bank data. Note: Global Fund commitments and disbursements are totals. World Bank commitments and disbursements represent the proportions of the Bank’s total project commitments and disbursements to the health sector. 13. These are the share of total project commitments and disbursements that are assigned to the four health sector codes in the Bank’s coding system: (a) health, (b) public administration–health, (c) compulsory health finance, and (d) non-compulsory health finance. 18 2.38 More than half of Global Fund commitments and disbursements have been for HIV/AIDS, followed by malaria and tuberculosis (Table 3). Only about 13 percent of World Bank commitments have been for these three diseases, and another 4 percent for other communicable diseases such as avian flu and leprosy. The Bank has a broader mandate; the largest portion of its support (43 percent) is for health systems strengthening (HSS) — an Table 3. Global Fund and World Bank Health Commitments and Disbursements, by Disease/Theme, Fiscal Years 2003–11 Global Fund World Bank Disease/Theme Commitments Disbursements Commitments Disbursements US$ Millions HIV/AIDS 9,913.3 7,470.3 1,728.2 1,564.5 Malaria 5,164.9 4,051.6 729.6 343.8 Tuberculosis 2,872.9 2,127.1 414.3 661.6 HIV/Tuberculosis 202.3 195.2 Other communicable diseases 837.9 426.1 Health systems strengthening 159.1 170.9 9,359.5 7,035.1 Integrated 3.1 3.1 Child health 2,644.9 1,871.6 Population & reproductive health 1,728.8 1,541.6 Nutrition & food security 1,118.0 1,081.5 Injuries & non-communicable diseases 1,745.8 1,257.0 Other human development 1,330.0 903.9 Total 18,315.6 14,018.3 21,636.8 16,686.7 World Bank subtotal mapped to 12,863.7 9,144.1 the HNP Sector Board Share of Total HIV/AIDS 54.1% 53.3% 8.0% 9.4% Malaria 28.2% 28.9% 3.4% 2.1% Tuberculosis 15.7% 15.2% 1.9% 4.0% HIV/Tuberculosis 1.1% 1.4% Other communicable diseases 3.9% 2.6% Health systems strengthening 0.9% 1.2% 43.3% 42.2% Integrated 0.0% 0.0% Child health 12.2% 11.2% Population & reproductive health 8.0% 9.2% Nutrition & food security 5.2% 6.5% Injuries & non-communicable diseases 8.1% 7.5% Other human development 6.1% 5.4% Source: Global Fund and World Bank data. See Appendix Tables F-9 to F-12. Note: Each World Bank project can identify up to five themes promoted by the project. World Bank commitments and disbursements represent the proportions of total project commitments and disbursements to each theme. The subtotal “mapped to the HNP Sector Board� represents the share of these commitments and disbursements under the control of the HNP Sector Board. That is, each Bank-supported project is supervised by a project manager who reports to a regional manager, who is represented on a Bank-wide sector board. Each project is thereby “mapped� — or becomes the responsibility of — that sector board, in this case the HNP Sector Board. 19 expansive category that encompasses virtually all activities that aim to bring about improvements in the management, financing, and overall performance of health systems (World Bank 2007c, p. 14). Other important priorities for the Bank are child health, and population and reproductive health. 2.39 The different mandates of the two organizations are also manifested in the geographical distribution of their commitments and disbursements (Table 4). The distribution for the Global Fund reflects, first of all, different countries’ income levels, and then other factors such as disease burden, population size, vulnerability, local institutions and policies, and the quality of proposals received. The Global Fund focuses its support on low-income countries as classified by the World Bank — equivalent to International Development Association (IDA)-eligible countries. Only these countries are eligible for all forms of support offered by the Global Fund. Lower middle-income applicants must focus their grant proposals on their countries’ poor or vulnerable populations, and upper middle-income applicants on their countries’ poor and vulnerable populations. Lower middle-income countries must also contribute at least 35 percent of the costs of the proposed interventions, and upper middle-income countries at least 65 percent. Therefore, fully 60 percent of Global Fund support has gone to Africa, which is also the epicenter of the HIV/AIDS epidemic and suffers from rampant malaria. East Asia has large populations and high rates of multi-drug resistant tuberculosis. South Asia and Europe Table 4. Global Fund and World Bank Health Sector Commitments and Disbursements, by Region, Fiscal Years 2003–11 Global Fund World Bank Region Commitments Disbursements Commitments Disbursements Africa 11,131.2 8,371.9 3,934.8 3,595.0 East Asia & the Pacific 2,611.9 1,984.2 1,277.3 1,159.7 Europe & Central Asia 1,438.7 1,264.7 2,592.5 2,223.5 Latin America & the Caribbean 1,364.9 1,114.5 7,692.2 5,484.2 South Asia 1,505.9 1,068.8 3,359.1 2,777.2 Middle East & North Africa 263.0 214.2 301.8 513.9 World - - 11.9 1.9 Total 18,315.6 14,018.3 19,169.6 15,755.3 World Bank subtotal mapped to 12,498.3 9,967.8 the HNP Sector Board Share of Total Africa 60.8% 59.7% 20.5% 22.8% East Asia & the Pacific 14.3% 14.2% 6.7% 7.4% Europe & Central Asia 7.9% 9.0% 13.5% 14.1% Latin America & the Caribbean 7.5% 8.0% 40.1% 34.8% South Asia 8.2% 7.6% 17.5% 17.6% Middle East & North Africa 1.4% 1.5% 1.6% 3.3% World 0.0% 0.0% 0.1% 0.0% Source: Global Fund and World Bank data. See Appendix Table F-13. Note: World Bank commitments and disbursements represent the proportions of total project commitments and disbursements to the health sector. Bank disbursements to the Middle East and North Africa are slightly higher than commitments because the data also reflects disbursements arising from Bank commitments prior to 2003. 20 and Central Asia have a high tuberculosis burden and increasingly high risks of HIV/AIDS. Africa and South Asia have the poorest and most vulnerable risk groups. The Middle East and North Africa has a relatively smaller population and a smaller disease burden and risks overall. 2.40 The Bank, in contrast, provides only 40 percent of its commitments in the form of concessional loans and grants to low-income countries, and the remaining 60 percent in the form of nonconcessional loans to middle-income countries. Therefore, while Africa is the Bank’s priority region, since most of the countries in Africa are low-income, the Bank also has sizeable health sector portfolios in all regions except the Middle East and North Africa. 2.41 A Global Task Team on Improving AIDS Coordination among Multilateral Institutions and International Donors, which met in 2005, found that the Global Fund and the World Bank “increasingly seem to finance the same types of goods and activities in the same countries without any clear sense of their respective comparative advantages or complementarity with each other� (UNAIDS 2005). The above comparisons show that the greatest potential for constructive engagement between the two organizations at the country level occurs in the low-income, high-burden countries where the Global Fund is most active. 2.42 It should also be emphasized that the Bank is a multisectoral organization that takes a multisectoral approach to improving health outcomes, involving contributions from the education, sanitation, nutrition, public administration, and finance sectors, among others, in addition to the health sector. This having been said, health sector project managers — those who are managing projects under the control of the HNP Sector Board — have a greater potential to engage with Global Fund staff and agents at the country level. Such projects account for about 60 percent of the total commitments and disbursements to the health sector (Tables 3 and 4). World Bank Engagement with the Global Fund 2.43 The World Bank has played three major roles in the Global Fund — as the trustee of the Global Fund trust fund, as a member of the Board and two of its committees, and as a development partner at the global and country levels. TRUSTEE 2.44 First and foremost, the Bank is the administrator of Global Fund trust fund. Under the trustee agreement (signed in May 2002), the Bank receives and invests funds from Global Fund donors, disburses the funds to grant recipients on the instruction of the Global Fund Secretariat, and provides regular financial reports to the Global Fund Board. The Multilateral Trusteeship and Innovative Financing Department (CFPMI) is responsible for managing the trustee operations of the Global Fund trust fund, the largest trust fund that the Bank administers. As indicated earlier, the Global Fund has reimbursed the Bank about $2.0–3.0 million annually for the costs incurred in administering the trust fund (Table 2). 2.45 In World Bank parlance, the Global Fund trust fund is a financial intermediary fund (FIF) in which the Bank does not have an operational role. That the Bank might play an implementing role in the Global Fund, like the Bank supervises projects financed by the Global Environment Facility, was never seriously considered by the Transitional Working Group. However, there 21 were considerable pressures in the Working Group for the Bank to take on an “enhanced fiduciary role,� in addition to being the trustee, to ensure that Global Fund grants were used for the intended purposes. The Bank was unenthusiastic about exercising fiduciary oversight for projects for which it did not also have programmatic oversight in accordance with its own operational policies, which would have required a substantial scaling up of country-level HNP staff. When the Bank declined, the Global Fund Board decided at its second meeting in April 2002 to establish the LFA system of contracting out in-country fiduciary functions to LFAs.14 GOVERNANCE 2.46 The Bank is a permanent (as opposed to rotating) nonvoting “institutional� member of the Global Fund Board, along with WHO, UNAIDS, and one representative from partners (RBM, Stop TB, and UNITAID), and a member of two Board committees — the Finance and Audit Committee by virtue of its trusteeship role, and the Policy and Strategy Committee by virtue of its experience in the health sector. DEVELOPMENT PARTNER 2.47 As already indicated, the Bank has been a significant lender for strengthening health systems and controlling communicable diseases, as well as for other health priorities such as child health and population and reproductive health. The potential for Bank staff to be engaged in the country-level processes of the Global Fund, and in other ways that contribute directly or indirectly to the work of the Global Fund, is obviously greater where the Bank is financing technical assistance or investment projects related to the three diseases. 2.48 In recent years, the Bank and Global Fund have taken a number of steps to improve coordination and collaboration on country work through various initiatives associated with GHAP, IHP, RBM, Stop TB, and AMFm. But the full extent of the Bank’s engagement with the Global Fund at the country level has not been systematically tracked. There have been no Bank-wide directives or guidelines to staff for engaging with the Global Fund at the country level, or a memorandum of understanding (MOU) between the Global Fund and the World Bank for collaborating at the global or country level. 2.49 IEG’s findings and lessons in relation to the Bank’s role as a development partner of the Global Fund are presented in Chapter 4 of this Review. The Five-Year Evaluation of the Global Fund 2.50 The Global Fund completed its first five-year evaluation in 2009 — a comprehensive three-part evaluation covering the first years of its existence. Approved by the Board in November 2006, the FYE was launched in April 2007, and the final synthesis report was submitted in March 2009 for discussion at the 19th Board meeting in May 2009. 14. Macro International 2009b, The Five-Year Evaluation of the Global Fund to Fight AIDS, Tuberculosis and Malaria: Synthesis of Study Areas 1, 2, and 3, pp. 12, 36, and 54; Minutes of the Transitional Working Group, October 11–12, 2001; and Report of the Second Meeting of the Global Fund Board, April 22-24, 2002. 22 2.51 Overall, this Review found that the FYE was an independent and quality evaluation. Assisted by a team of staff provided by the Secretariat, the TERG oversaw all aspects of the evaluation, including contracting the evaluation to an independent consortium of evaluators. The evaluation teams were able to report candidly about how slowly and less strategically the Global Fund governance processes had developed to guide this new approach to development assistance; about the need for a robust risk management strategy to alert the Global Fund about likely suspension of ongoing treatment activities; and about the risk of increased drug resistance, among other things. Notwithstanding the TERG’s very “involved� oversight style, the FYE was protected from outside interference, and the potential conflicts of interest that arose were appropriately identified and managed. 2.52 The evaluation design was organized around three study areas, each of which resulted in one evaluation report:  Study Area 1: The Organizational Efficiency and Effectiveness of the Global Fund — issued in October 2007.  Study Area 2: The Global Fund Partner Environment, at Global and Country Levels, in Relation to Grant Performance and Health System Effects, Including 16 Country Studies — issued in June 2008.  Study Area 3: Impact of Collective Efforts on the Reduction of the Disease Burden of AIDS, Tuberculosis and Malaria — issued in May 2009. 2.53 The FYE was objectives-based and evidence-based against the stated purpose and principles of the Global Fund. The overall assessment was fair and balanced, portraying both the strengths and weaknesses of the Global Fund. The three study areas reinforced each other, and the Synthesis Report effectively pulled together key messages in a coherent and integrated manner, although the evaluation did not deliver on two objectives — developing the “determinants� of good grant performance in Study Area 2 and building evaluation capacity in Study Area 3 countries. The FYE met three of the four standard IEG criteria for assessing quality — evaluation scope, instruments, and feedback (Appendix Table A-3). It did not meet the M&E criterion that the program’s activity-level M&E system should contribute to the evaluation’s assessment of the overall outcomes of the program because the Global Fund’s grant-level M&E system was not initially designed to do so. Therefore, the FYE used other methods, notably the impact assessment in Study Area 3. 2.54 The total cost of the FYE was $16.2 million, of which $11.7 million was spent on Study Area 3. The high cost of Study Area 3 was largely due to its extensive country-level activities. Eighteen countries were studied to obtain a broad view of progress in different country contexts. Primary data collection and analysis were conducted in eight countries (Burkina Faso, Cambodia, Ethiopia, Haiti, Malawi, Peru, Tanzania, and Zambia) and secondary data analysis was done in ten countries (Benin, Burundi, Democratic Republic of Congo, Ghana, Kyrgyzstan, Lesotho, Moldova, Mozambique, Rwanda, and Vietnam).15 15. A large part of the Study Area 3 work also aimed at the participation and evaluation capacity building of country institutions. As planned, 70 percent of the $11.7 million evaluation budget for Study Area 3 was spent on country institutions — 40 percent on data collection/analysis and 30 percent on technical assistance and training. The total cost of the evaluation represented 1 percent of the average annual expenditures of the Global Fund in 2007 and 2008 (not including the movement in undisbursed grants). IEG has observed that independent 23 2.55 The major findings of the FYE are presented in Chapter 3 in conjunction with IEG’s findings from six country visits in 2010 (to Burkina Faso, Tanzania, Cambodia, Nepal, Brazil, and the Russian Federation). IEG’s detailed assessment of the independence and quality of the FYE, and its lessons for the evaluation of GRPPs more generally, are presented in Chapter 5 of this Review. IMPACTS OF THE FIVE-YEAR EVALUATION ON THE GLOBAL FUND 2.56 IEG has found that the FYE was a landmark and influential evaluation exercise, which has had a major impact on the Global Fund. Even the preparatory events leading up to the FYE had impacts, because these generated support and visibility for the organization. The evaluation has helped the Global Fund Board and management make significant strategic adjustments to its organizational and institutional arrangements. The first formal Management Response was presented to the Board in November 2009, and an Update (with time lines) was presented at the Third Replenishment Meeting in The Hague in March 2010.16 There have also been a number of external and internal review studies — for example, of the CCM mechanism and the LFA system — that were conducted either as inputs into the FYE or to supplement it. Collectively, these have helped to forge new and strategic directions for the Global Fund. 2.57 As a young and rapidly evolving program, the Global Fund was already acting on some evaluation findings before the final findings and recommendations were formally issued, since the TERG regularly updated the Board and the Policy and Strategy Committee on key findings of the interim reports. By March 2009, when the final Synthesis Report was submitted, the Global Fund Secretariat, and in some instances, the Board and its Committees, had already initiated steps in some 20 activity areas, in response to the Study Area 1 and 2 reports, and related TERG recommendations. 2.58 The Global Fund has taken the following actions, among others, in response to the FYE. (Appendix G provides the formal and more detailed response of the Secretariat to the FYE.) Global Fund Principles (a) The Global Fund Board has reaffirmed that the Global Fund is a financing entity. The Secretariat is taking steps to communicate the Global Fund’s business model more clearly to countries and partners alike. The Board has also reaffirmed its commitment to the country-owned model and to the importance of inclusion and engagement of CSOs at all levels. (b) As a signatory to the Paris and Accra Accords, the Global Fund will abide by the guiding principles of alignment and harmonization. At the country level, the Global evaluations of GRPPs typically cost between 1 and 3 percent of annual expenditures, closer to 1 percent for larger programs such as the Global Fund and closer to 3 percent for smaller programs. (See IEG 2011b, The World Bank’s Involvement in Global and Regional Partnership Programs: An Independent Assessment, p. 26.) 16. Global Fund, 2010b, “The Five-Year Evaluation: An Update,� The Global Fund Third Replenishment (2011–2013). 24 Fund will emphasize the alignment of its grant cycle with country planning and budgeting cycles, and harmonization of salary support and compensation. The Global Fund will encourage CCMs to be more in line with other national coordinating bodies. Governance and Management (c) The Board has delegated more decision-making authority, especially on operational matters, to its Committees and to the Secretariat in order to focus more on core strategic issues, consistent with its governance role. (d) The Global Fund is now an autonomous international financing agency, having terminated its administrative agreement with WHO in December 2008. The Secretariat is reorganizing itself in order to become more efficient, and is implementing human resource measures to strengthen performance. The Secretariat has proposed that its administrative budget will not exceed 10 percent of total expenditures. Partnership Strategy (e) The Secretariat has developed a new Partnership Strategy that has been approved by the Board, which provides a framework for a strategic division of labor, clarity of roles, and coordination. The Global Fund is strengthening existing relationships with RBM, Stop TB, UNAIDS, UNICEF, and WHO, and engaging more with GAVI and the World Bank both directly and through the IHP+ and the Health Systems Funding Platform. In addition, the Global Fund will give more emphasis to HSS, maternal and child health, and the prevention of mother to child transmission of HIV (PMTCT). (f) Global Fund donors have not agreed to provide funding to development partner agencies to provide complementary technical assistance at the country level. Therefore, the Secretariat is seeking innovative options for financing or providing country-level technical assistance based on studies carried out by the Gates Foundation, GTZ, and UNAIDS, and on additional targeted studies that address key questions with regard to technical assistance planning, access, and financing. Operational Modalities (g) The Secretariat is simplifying the grant mechanism and implementing a new grant architecture to move from a project-based approach to a single stream of funding mode (Box 2). This is intended to ensure greater cohesiveness and coherence among grant activities, to foster greater alignment with national strategies for the three diseases, to avoid service disruptions, and to reduce transactions costs. Each Principal Recipient will have a single grant agreement for any one disease (single stream of funding), and may win a subsequent grant to scale up or extend the duration of activities from the first grant, based on periodic reviews of satisfactory performance. Grants will no longer be fragmented and piecemeal, but instead will be approved on the basis of adherence to a sustained national programmatic approach. The Global Fund also plans to shift its funding to support National Strategy Applications — that is, to support a national strategy instead of multiple grants for each disease in a country, and to group all grants under this strategy. 25 (h) The Global Fund still views PBF as the cornerstone of the management of its grant portfolio. The Global Fund will make greater investments in M&E (including data quality audits) in light of the tremendous data quality issues in recipient countries. It is placing greater emphasis on strengthening country information systems and on aligning Global Fund M&E requirements with the National Health Management Information Systems of countries to reduce the burden of reporting. (i) The Secretariat has launched a Risk Management Framework17 to mitigate fraud and corruption with Global Fund grants in countries. It has developed an accountability framework encompassing all of the Global Fund structures, systems, and controls for managing risks at all levels. Board Committees have direct oversight responsibility over risks that have been identified. The Secretariat is providing clearer policy and guidelines to countries, and the OIG now has a stronger role in providing independent and objective assessments of high-risk topics and in establishing Global Fund controls. Box 2. Changes to the Global Fund Grant Architecture The Global Fund is currently going through the process of transitioning its entire grant portfolio to single streams of funding (SSF), following Board approval of its “new grant architecture� in November 2009. The first two SSFs were for tuberculosis in Fiji and for HIV/AIDS in Moldova, both effective April 1, 2010, and both with their respective Ministries of Health as Principal Recipients. The Secretariat has signed over 80 SSFs as of October 2011, and expects to sign a total of 145–150 SSFs by the end of 2011, most of these from the remaining Round 10 grant agreements. The Secretariat plans to have completed the better part of the SSF transition by the end of 2013. Thus, SSFs are becoming the Global Fund’s new modality for structuring its funding. The former grants were one grant per Principal Recipient per approved Round, with little or no alignment to any in-country cycles, and with Phase 2 reviews taking place at different times for grants in different Rounds. SSFs represent one grant per Principal Recipient per disease or HSS program (consolidated if a single Principal Recipient was previously implementing multiple grants). The timings of Periodic Reviews (which are replacing Phase 2 Reviews) are now aligned to each country’s fiscal and programmatic reporting cycles. The first 25 or so Periodic Reviews will take place during the first two quarters of 2012. These will cover all Global Fund grants in a disease area (or cross-cutting HSS support), if there are multiple Principal Recipients in that disease area in the country. These will also take into account the available national program review information and impact studies, with the intention of relying more and more on country-driven information over time. The Global Fund Board approved the First Learning Wave of National Strategy Applications (NSAs) in November 2009 — for China (malaria), Madagascar (malaria), Nepal (tuberculosis), and Rwanda (one each for HIV and tuberculosis). Ten countries are participating in the Second Wave of NSAs that was launched in January 2011. The countries have recently completed a joint assessment of their national disease strategies, which is a prerequisite for submitting an NSA request. The final outcome of these Second Wave NSAs will be determined in 2012. Source: Global Fund Secretariat. See also “Changes to the Global Fund Grant Architecture: a Fact Sheet for Implementers.� www.theglobalfund.org/documents/grantarchitecture/Fact_Sheet_for_Implementers_en.pdf 17. Endorsed at the 20th Board Meeting in November 2009. 26 2.59 One of the questions on the electronic survey administered to staff of the Global Fund Secretariat related to the impacts of the FYE on the Global Fund. According to the survey results, staff perceive that the Global Fund is making significant progress in some areas in implementing these new directions, such as (a) sharpening its practices in relation to procurement, audit, and anti-corruption; (b) aligning its grants with each country’s planning and budgeting cycles; and (c) becoming more program-based in line with its new grant architecture (Figure 6). Other areas are still works in progress, such as strengthening national health management systems and enhancing the capacity and effectiveness of CCMs. Global Fund grants are still focusing more on treatment, care, and support activities than on disease- prevention activities, and Secretariat staff still find the personnel reward system to be lacking, according to these survey results. Figure 6. To what extent have the findings and recommendations of the Five-Year Evaluation had the following impacts on the Global Fund? The Global Fund has sharpened its practices in the areas of procurement, audit, and anti- corruption. The Global Fund is improving the alignment of its grants with each country’s planning and budgeting cycles. The Global Fund has become more program- based, as opposed to individual grant-based, in its funding decisions. Global Fund grants are providing more support to strengthening national health management information systems. The Global Fund is devoting more resources to enhancing the capacity and effectiveness of CCMs in their full range of functions. Global Fund grants are putting more focus on disease-prevention activities, as opposed to treatment, care, and support activities, taking into account the local context of each epidemic. The Global Fund has improved its ability to adequately reward and retain its staff. 0% 20% 40% 60% 80% 100% High Substantial Modest Negligible Source: IEG Survey of Global Fund staff, administered in March 2011. See Appendix Q. Note: There were 52 usable responses to the survey for a response rate of 49 percent (52 out of 106): 36 of these respondents were from the Country Programs Cluster, 7 from the External Relations and Partnerships Cluster, and 9 from the Strategy, Performance, and Evaluation Cluster. There was no significant difference in responses to these questions across the three clusters. 27 3. Validating the Major Findings of the Five-Year Evaluation 3.1 The Global Fund represents an ambitious attempt by the international community to use a global partnership program to deliver the global public good of controlling HIV/AIDS, tuberculosis, and malaria in high-burden countries, with a particular focus on low-income countries. Guided from the beginning by principles later adopted by signatories to the 2005 Paris Declaration on Aid Effectiveness, it has become a basis of comparison for other global partnership programs that are financing investments at the country level.18 Thus, the experience of the Global Fund provides lessons not only for the Fund itself, but also for other global partnership programs in relation to issues such as additionality, sustainability, country ownership, alignment, harmonization, and managing for results. 3.2 The FYE presented findings in all these areas. Those on the effectiveness of the Global Fund approach at the country level were drawn primarily from the 16 country case studies carried out in 2007 as part of Study Area 2 — Evaluation of the Global Fund Partner Environment at the Global and Country Levels, in Relation to Grant Performance and Health System Effects. IEG consultants revisited four of these countries (Burkina Faso, Cambodia, Nepal, and Tanzania) as well as two middle-income countries (Brazil and the Russian Federation) in 2010 to confirm the findings of the FYE and to assess changes (either improvements or deteriorations) in the intervening three years, using the FYE and the four Study Area 2 country reports as a baseline. 3.3 The current chapter presents what IEG found, organized according to eight of the nine major findings of the FYE, as presented in the Synthesis Report. (IEG did not address the ninth major finding because this related to the global governance of the Global Fund, not its country- level activities.) This introductory section concludes with a summary of the epidemiology of the three diseases in the six countries, and the activities of the Global Fund in those countries. Then each of the sections that follow starts with the major findings of the FYE, followed by the findings from IEG’s six country visits, supplemented by other material as appropriate. 3.4 HIV/AIDS. HIV/AIDS is a significant public health problem in all six countries visited. Tanzania is the most heavily affected, with an estimated 5.7 percent of the adult population living with AIDS, followed by Burkina Faso, with 1.2 percent of adults infected (Table 5). Unprotected heterosexual sex is the primary mode of transmission in Sub-Saharan Africa. In both Tanzania and Burkina Faso, infection rates are particularly high among people with high-risk sexual behavior, but HIV has moved out of these groups to infect many of their partners who exhibit lower-risk behavior. Women and girls are disproportionately affected (with prevalence rates 2–4 times those of males in some surveys); the high infection rate among women of childbearing age has resulted in significant mother-to-child transmission of HIV.19 18. This would include the Education for All–Fast Track Initiative (established 2002), the Climate Investment Funds (established 2008), and the Global Agriculture and Food Security Program (established 2010). The Global Environment Facility (established 1991) and the Global Alliance for Vaccines and Immunization (established 2000) have also started to compare themselves to certain aspects of the Global Fund. For example, the first evaluation of GAVI specifically compared the organizational efficiency and effectiveness of GAVI with that of the Global Fund, based on the findings of Study Area 1 of the FYE. See Chee and others 2008, pp. 124-127. 19. Information is taken from UNAIDS 2011, UNAIDS Report on the Global AIDS Epidemic 2010. 28 Table 5. Epidemiological Profile of the Six Countries Visited by IEG Russian Burkina Faso Tanzania Cambodia Nepal Brazil Federation Population 16,287,000 45,040,000 15,053,000 29,853,000 195,423,000 140,367,000 Upper middle- Upper middle- Income level a Low income Low income Low income Low income income income GNI per capita $510 $500 $610 $440 $8,040 $9,340 (Atlas method) a Total health expenditure per $29 $22 $36 $20 $606 $493 capita b People living with 110,000 1,400,000 63,000 64,000 730,000 980,000 HIV c Adult HIV prevalence rate (%), ages 15-49, 1.2 5.7 0.5 0.4 0.3-0.6 1.0 est. 2009 c Estimated number of people receiving ART, 26,448 199,413 37,315 3,226 195,984 d 75,900 2009 c Estimated ART coverage (%), 37-58 27-34 68-95 9-13 50-89 16-24 2009 e TB incidence, (incl. HIV) (rate per 100,000 215 183 442 163 45 132 population) f Probable and confirmed 4,399,837 3,812,350 83,777 132,012 308,498 107 malaria cases g Sources: a. World Bank, World Development Indicators database 2010. b. WHO, 2010a, World Health Statistics. c. UNAIDS, 2010, Report on the Global AIDS Epidemic 2010. d. 2008 data. e. The estimated antiretroviral therapy coverage is based on the 2010 WHO guidelines, as presented in UNAIDS 2010, Annex 2. Coverage rates based on the 2006 WHO guidelines are higher. f. WHO, 2010b, Global Tuberculosis Control Report. g. WHO, 2010c, World Malaria Report. 3.5 In most of South and East Asia, including Cambodia and Nepal, the epidemic is concentrated among commercial sex workers, injecting drug users, men who have sex with men, and migrant labor. About one in two hundred adults is infected in those two countries, but this rate is much higher in the severely affected groups. In Cambodia, a rigorous prevention program targeting the riskiest behavior has reduced the incidence of HIV (number of new infections). The epidemic is no longer considered to be “generalized� in Cambodia, 29 but threatens to spread more widely in Nepal due to cross-border migration along the India- Nepal border among sexual and drug-using networks. Control of the epidemic is limited by the shortage of voluntary counseling and testing services in Nepal and the limited capacities of the authorities and health workers to address existing cases. Migrants also have less access to health services in the settings to which they have migrated. 3.6 Like Cambodia, Brazil has launched major prevention campaigns among the groups most at risk (commercial sex workers, intravenous drug users, and men who have sex with men), involving federal/state partnerships with broad participation of NGOs. Compared with the other five countries, the Russian Federation is currently experiencing a growing national epidemic that has infected 1 percent of the population. This has been driven by injecting drug users and commercial sex workers, which is now leading to increasing heterosexual transmission and prevalence among women. 3.7 HIV/AIDS is not a curable disease at present; combination antiretroviral therapy (ART) can suppress the infection but must be taken for a lifetime. The six countries differ greatly in terms of the coverage of persons in need of such treatment, from less than a quarter in Nepal and the Russian Federation to more than two-thirds in Cambodia and Brazil (Table 6). Thus, there are a number of low-income countries with relatively high treatment coverage rates. 3.8 Tuberculosis. Four of the 6 countries are listed in WHO’s list of 22 high-burden tuberculosis countries that account for 80 percent of all new tuberculosis cases arising each year: the Russian Federation (no. 12), Tanzania (no. 14), Brazil (no. 15), and Cambodia (no. 21). Incidence ranges from 150,000 new cases in the Russian Federation in 2009 to 80,000 in Tanzania, 85,000 in Brazil, and 65,000 in Cambodia. Incidence has been declining in Brazil, and is stable in the other three countries. Although incidence is lower in Nepal and Burkina Faso, at 48,000 and 34,000 new cases, respectively, the numbers have been increasing in both countries, and incidence rates per 100,000 population are high. Once a “forgotten disease,� tuberculosis has reemerged on a global scale. This is partially due to insecurity in the drug supply, gradually emerging resistance to first-line drugs due to inadequate or interrupted treatment, and HIV/AIDS as an amplifier of tuberculosis incidence and spread. WHO estimates that HIV is prevalent in 47 percent of tuberculosis cases in Tanzania, 12 percent in Brazil, 8 percent in the Russian Federation, and 6.4 percent in Cambodia. Thus, tuberculosis has become a leading killer disease among AIDS patients. 3.9 Malaria. Malaria is concentrated in Africa and other tropical regions where climatic conditions are favorable to mosquito breeding. Eighty-five percent of the 250 million annual cases of malaria occur in Africa, and that region accounts for 90 percent of the annual deaths from malaria. Malaria has been the single most significant disease in Tanzania affecting the health and welfare of its 45 million inhabitants. However, the number of reported cases has been declining, from more than 9 million cases in 2003 to 3.8 million in 2008. Malaria is also a leading cause of morbidity and mortality in Burkina Faso, Cambodia, and Nepal — it is most intense in the southern third of Burkina Faso, in the forested areas of Cambodia, and the lowland areas of Nepal, along the Indian border. There were 4.4 million reported cases in Burkina Faso in 2009, 84,000 cases in Cambodia, and 132,000 cases in Nepal. The number of reported cases in Brazil, which are concentrated in the country’s Amazon region, has been 30 declining, from 606,000 cases in 2005 to 308,000 cases in 2009. Malaria is essentially nonexistent in the Russian Federation, with only 100 or so cases annually. 3.10 Each of the four low-income countries has received between 11 and 15 grants from the Global Fund since 2002, about 40 percent for HIV/AIDS, one-third for malaria, and one-quarter for tuberculosis. Brazil has received four grants, two each for tuberculosis and malaria, and the Russian Federation has received six grants, three each for HIV/AIDS and tuberculosis (Table 6). a Table 6. Global Fund Grants by Disease in the Six Countries Visited by IEG, 2002–11 Russian Burkina Faso Tanzania Cambodia Nepal Brazil Federation Number of Global Fund Grants Approved, 2002–11 HIV/AIDS 4 7 6 7 - 3 Tuberculosis b 4 2 3 3 2 4 Malaria 4 5 5 4 2 - HSS - 1 2 - - - Total 12 15 16 14 4 7 Global Fund Commitments, 2002–11 ($ millions) c HIV/AIDS 123.3 391.4 175.1 47.0 - 261.9 Tuberculosis b 30.9 91.0 23.5 33.0 23.0 169.3 Malaria 86.3 330.6 109.6 29.6 24.1 - HSS - 74.6 15.2 - - - Total 240.5 887.5 323.4 109.6 47.1 431.2 Global Fund Grant Disbursements, 2002–11 ($ millions) HIV/AIDS 59.5 313.1 136.0 23.2 - 258.7 Tuberculosis b 24.2 82.0 21.8 18.7 20.0 102.7 Malaria 86.3 273.1 75.5 21.2 18.0 - HSS - 15.6 9.0 - - - Total 161.3 683.6 242.4 63.1 38.1 361.4 Source: Global Fund Web site. See Appendix H for more details. a. Through June 30, 2011. The totals also include three Round 10 proposals that have been approved by the Global Fund Board, whose grant agreements have not yet been negotiated and signed: $53.8 million for HIV/AIDS in Burkina Faso; $16.2 million for HIV/AIDS in Nepal; and $63.5 million for tuberculosis in the Russian Federation. b. The totals for Tanzania include one grant classified as HIV/TB. c. These represent commitments in relation to signed grant agreements plus the amounts approved by the Board for the three pending grant agreements in Burkina Faso, Nepal, and the Russian Federation. Additionality, Predictability, and Sustainability of Global Fund Support 3.11 FYE Findings. The FYE found that the Global Fund has provided substantial resources for disease control programs, and has increased the potential pipeline for resources by magnifying the focus on the three diseases. The assessment of country-level additionality in four countries where National Health Accounts data were available did not show a strong 31 evidence of decline in domestic funding. However, the reliance on external funds raised concerns with respect to (a) external resources replacing national ones; (b) the long-term sustainability of recipient countries’ disease control programs, and (c) the cost-effectiveness and maintenance of the programs. The FYE also found that the longer-term capacity investments — which were critical for sustainability of prevention, treatment, and care — had been hindered by the lack of alignment between Global Fund and country systems, and by the shift of staff and resources from the public sector to the NGO sector, serving as implementers of Global Fund grants (Macro International 2009b, pp. 15–17). ADDITIONALITY OF GLOBAL FUND RESOURCES 3.12 Additionality has two dimensions — from the point of view of donors and from that of recipient governments. To what extent are donors and recipient governments increasing or decreasing their own commitments to combating the three diseases in response to the Global Fund grants to the countries? 3.13 At the global level, the Global Fund has become a significant contributor to official donor commitments, both to the three diseases and to health overall, since it was founded in 2002. Global Fund commitments of $4.3 billion in 2009 accounted for 37 percent of official commitments to the three diseases and 19 percent of donor commitments to the overall HNP sector, according to Organisation for Economic Co-operation and Development (OECD) data (Figure 7). At the same time, other donor commitments to the three diseases outside of the Global Fund have not decreased, but also increased, from $1.7 billion in 2002 to $7.1 billion in 2009 (in constant 2008 prices), and commitments to the overall health sector have grown from $9.2 billion to $18.1 billion during the same period. Thus, donors have increased their commitments to the three diseases through both the Global Fund and their own bilateral programs. Whether total donor commitments to the three diseases have been higher or lower than they would otherwise have been in the absence of the Global Fund is not known. This general global picture does not change even if one removes the largest donor, the United States, which supports its own large programs for HIV/AIDS and malaria — the President’s Emergency Plan for AIDS Relief (PEPFAR) and the President’s Malaria Initiative. 3.14 Notwithstanding this overall picture, IEG found — based on interviews and confirmed by OECD data — that other donor commitments to the health sector have been essentially constant since 2002, although fluctuating from year to year, in three of the four low-income countries visited (Burkina Faso, Cambodia, and Nepal). Other donors have decreased their funding for HIV/AIDS in Burkina Faso in response to Global Fund grants. In Cambodia, government-donor attempts to reduce aid fragmentation in 2006 led to a “division of labor� and the disengagement of the Asian Development Bank — the largest donor in Cambodia — from the health sector in order to focus its efforts on the agricultural sector. The U.K.’s Department for International Development (DFID) was planning to exit the health sector in both Cambodia and Nepal at the time of IEG’s country visits. Tanzania was the only one of the four low-income countries visited by IEG in which donor commitments for the three diseases and for health overall have increased steadily since 2002. 3.15 Similar to the FYE, IEG did not find evidence that governments are reducing their own expenditures on the three diseases in response to the Global Fund grants, except in one 32 Figure 7. Official Development Assistance and Other Official Flows from OECD/DAC Member Countries and Multilateral Agencies to Developing Countries a. Commitments to HIV/AIDS, Tuberculosis, and Malaria US$ Billions  (constant  2008 prices) 12  10  Global Fund 8  IBRD/IDA 6  Other Donors 4  2  � 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 b. Commitments to Health, Nutrition, and Population US$ Billions  (constant  2008 prices) 25  20  Global Fund IBRD/IDA 15  Other Donors 10  5  � 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 Source: OECD. Note: Official Development Assistance represents concessional flows including IDA. Other Official Flows are non- concessional flows such as lending by IBRD and regional development banks. See Appendix Table F-4. 33 country, Tanzania. Systematic National Health Accounts were not available to answer this question definitively in the countries visited, but the available data indicated that government expenditures on the health sector have been increasing in Burkina Faso, Cambodia, and Nepal.20 The data for Tanzania indicated that government expenditures on the health sector and on HIV/AIDS had decreased as external assistance had increased. The Tanzania Ministry of Finance and Economic Affairs has been the Principal Recipient for most Global Fund grants, and the Ministry of Health has been the lead Sub-Recipient for grants implemented by the government. Given Tanzania’s high dependence on external assistance, the Government of Tanzania appears to have shifted its own expenditures to other priority areas not benefiting from the abundance of resources provided by the Global Fund. 3.16 In Brazil, Global Fund grants have been small relative to the magnitude of national resources dedicated to fighting the three diseases. Government budgets have been set, regardless of the size of Global Fund grants. In the Russian Federation, the Government increased its national budget for HIV/AIDS from $20 to $100 million in 2004 upon conclusion of the Round 3 grant agreement between the Open Health Institute (the Principal Recipient) and the Global Fund, in line with understandings reached during the negotiation stage. PREDICTABILITY OF GLOBAL FUND SUPPORT 3.17 IEG found short-term gaps in the timing of Global Fund financing in several countries due to the unpredictability of the awarding of Global Fund grants. In Burkina Faso, for example, the long-term sustainability of Global Fund financing for HIV/AIDS was threatened by a funding gap until the country’s Round 10 proposal was approved by the Global Fund Board in December 2010. At the time of IEG’s visit in May 2010, Round 6 financing was slated to terminate at the end of 2011, and Burkina Faso had failed to secure additional Global Fund financing in Rounds 8 and 9. The failure to achieve Round 9 financing had come as a complete surprise to all stakeholders, since they had viewed the quality of their proposal as very high. As a result, the President of Burkina Faso had publicly called on his Ministry of Finance and Economy to find funds to continue the drug treatment and prevention programs beyond 2011. Other donors had also said that they would look for emergency funds to keep the programs going.21 3.18 Country-level stakeholders in Tanzania and Nepal also complained about short-term gaps in Global Fund financing. For the Global Fund as a whole, this relates mostly to the uneven pattern of grant proposals and the unpredictability of grant approvals, as opposed to delays in disbursements flowing from signed grant agreements. However, an analysis of the pattern of Global Fund grant disbursements since the first grants were awarded in 2002 shows a gap of seven months or more between grant disbursements about 29 percent of the time. Such delays in grant disbursements appear to be a bigger issue in Eastern and Southern Africa, South Asia, and 20. National AIDS Spending Assessment (NASA) data in Cambodia show that the government has only increased spending for preventive activities. The government deliberately does not finance treatment activities, relying largely on the United States, the Global Fund, and international NGOs to finance treatment activities. 21. The overall objective of Burkina Faso’s Round 10 proposal for HIV/AIDS is to promote universal access through securing ARV treatments, strengthening of PMTCT, and strengthening of HIV prevention for most at-risk populations. The grant agreement is still pending as of November 2011. 34 the Middle East and North Africa compared to other regions (Table 7). A similar analysis of grant disbursements by disease finds a more uniform pattern among the three diseases. 3.19 The Global Fund has been very aware that country-level grant management has become increasingly complex as countries receive multiple grants for the same disease, each grant with different reporting deadlines in accordance with its own performance framework. As a result, the Global Fund is currently going through the process of transitioning its entire grant portfolio to single streams of funding (SSFs), which are intended to make it easier for the Global Fund to support a national program approach for each disease that is better aligned with national systems and budget cycles. The Secretariat has signed over 80 SSFs as of October 2011, and expects to sign a total of 145–150 SSFs by the end of 2011, most of these from the remaining Round 10 grant agreements. The Secretariat plans to have completed the better part of the transition to SSFs by the end of 2013 (Box 2 in Chapter 2). Table 7. Disbursement Pattern of Approved Global Grants, by Region and by Disease, 2003–10 More than 6 Total number of months between sequential Share of total disbursements disbursements By Region or Subregion Sub-Saharan Africa: East Africa 210 439 48% Sub-Saharan Africa: Southern Africa 169 441 38% South Asia 144 398 36% North Africa & the Middle East 144 428 34% Eastern Europe & Central Asia 158 585 27% Latin America & the Caribbean 172 759 23% Sub-Saharan Africa: West & Central Africa 199 884 23% East Asia & the Pacific 181 851 21% By Disease HIV/AIDS / Tuberculosis 23 58 40% Malaria 369 1,200 31% Tuberculosis 392 1,338 29% HIV/AIDS 584 2,159 27% Health systems strengthening 7 27 26% Total 1,377 4,785 29% Source: Calculated by IEG from Global Fund disbursement database. SUSTAINABILITY OF EXTERNAL FINANCIAL SUPPORT 3.20 IEG found that some countries have become heavily dependent on the Global Fund support for antiretroviral treatment of people living with AIDS. In Burkina Faso, the Global Fund is now the only external financier of ARV therapy and drugs to prevent mother-to-child transmission of HIV/AIDS. The Global Fund has become the exclusive supplier of ARVs and related health products into Nepal (except for USAID’s providing PMTCT drugs on a small scale to its own projects). In Cambodia, with very high coverage for AIDS treatment, the United States has historically supported ARV treatment through programs implemented by NGOs. However, recent National AIDS Spending Assessment data show an overall decline in bilateral and NGO 35 financing for AIDS treatment, while that from the Global Fund has increased. Some country- level stakeholders in these countries now view the Global Fund as responsible for sustaining HIV/AIDS treatment. If the Global Fund is unable to sustain its financial support for treating people living with AIDS, then this will put pressure on governments to reallocate their own budgetary resources. The allocation of resources for prevention measures would likely be the first to be adversely affected, since it is morally problematic to terminate ARV treatments for people already receiving treatment, followed by reallocating resources from other (non-health) priorities. Global Fund staff have also exhibited some frustration, based on interviews and the survey results (Figure 6), that the Global Fund has been unable to allocate more to prevention measures due to the political demand for treatment, since the long-run affordability of treatment also depends on financing effective prevention programs to prevent new HIV/AIDS cases. 3.21 There are also increasing concerns at the global level that other donors’ support for treatment may be less forthcoming in the future. The sustainability of resources to support people living with AIDS who are already receiving antiretroviral treatment is of particular concern since interrupted treatment increases not only the risk of death among those already being treated but also the risks of new infections and of the development of drug-resistant strains of the virus. A lot depends on the United States, which has been the largest donor to the Global Fund (accounting for 27 percent of donor contributions through 2010), and whose own bilateral program (PEPFAR) is roughly twice the size of the Global Fund (Figure 8). The U.S. Congress had earmarked no less than 55 percent of PEPFAR funds for treatment until 2008. Figure 8. U.S. Contributions to the Global Fund and the Larger Global Effort US$ Millions 8,000 PEPFAR Budget 7,000 U.S. Contributions  to the Global Fund Total Global Contributions  to the Global Fund 6,000 5,000 4,000 3,000 2,000 1,000 0 2004 2005 2006 2007 2008 2009 2010 2011 Source: Appendix Table F-3 and the Henry J. Kaiser Family Foundation, “PEPFAR Fact Sheet,� March 2011. Note: U.S. contributions to the Global Fund come from PEPFAR’s overall budget. The United States contributed $5.1 billion to the Global Fund through December 2010 and has pledged a further $4 billion for the next three years, subject to Congressional appropriation. 36 Then U.S. policy shifted in 2009, allocating 50 percent to treatment and care services, and allowing more for prevention activities. It also moved from an “emergency� response mode to greater engagement with countries and to increased use of multilateral platforms such as the Global Fund. Nonetheless, PEPFAR still aims to provide ongoing treatment to 4 million people living with AIDS by 2014. Performance of Country Coordinating Mechanisms 3.22 FYE Findings. The FYE found that the CCMs were successful in mobilizing domestic and international partners for submission of grant proposals to the Global Fund, and in enabling CSOs and affected communities to participate, thereby reducing stigma and raising the visibility of the three diseases. However, the CCMs were largely perceived as Global Fund entities rather than as mechanisms for promoting country ownership and representing the country to the Global Fund. Despite the important gains in institutional development in many countries, CCMs fell short of expectations — as the governing body of the Global Fund Partnership in each country — in achieving greater country ownership, coordination, accountability, and partnership. The FYE found that the CCMs were ill-equipped—in terms of resources, capacity, and political will—to provide adequate grant oversight and management. The FYE also found that government members of CCMs were often reluctant to share “policy space� with other members, and that the involvement of the commercial private sector in CCMs has been weak at best (Macro International 2009b, pp. 39–43). PARTNERSHIP, LEADERSHIP, AND PARTICIPATION 3.23 IEG found that the CCMs were functioning better than the 2007 FYE findings indicated in two countries (Burkina Faso and Cambodia), about the same in two countries (Tanzania and Brazil), and worse in two countries (Nepal and the Russian Federation). 3.24 In Burkina Faso, the CCM now has broad participation in decision making compared with the situation in 2007, at the time of the Study Area 2 Country Program Assessment. Now established as an independent legal entity with its own office space, the CCM is more independent of government than in the past (Table 8). Nongovernment actors such as NGOs, community-based organizations, affected communities, and academia comprise almost half of CCM membership — higher than the Global Fund’s 40 percent requirement. The chair is now an academic (rather than automatically the Minister of Health), and the two vice chairs are the WHO representative and an association of people living with the diseases. Members of the CCM are integrally involved in the national strategic planning and program implementation for the three diseases. The CCM’s Proposal Development Committee has strong national leadership and broad representation of stakeholders. The process of selecting Principal Recipients and Sub-Recipients is transparent and fair: applications are solicited in the newspapers; then the CCM reviews the applications and selects the winner by voting. 3.25 In Cambodia, strong donor coordination mechanisms, in health and other sectors, preceded the arrival of the Global Fund in the country — a legacy of the large donor commitments to Cambodia after the Paris Peace Agreements in 1992. When the CCM was formed in 2002, it drew its membership from the joint government-donor Committee for Coordination of Health Activities that had been established in 1994, to avoid creating a 37 Table 8. Country Coordinating Mechanisms in the Six Countries Visited by IEG, 2010 Russian Burkina Faso Tanzania Cambodia Nepal Brazil Federation CCM Chair Academia Government Government Government Government Academia (University of (Prime (National AIDS (Ministry of (Secretaria de (Central Ouagadougou) Minister’s Authority) Health and Vigilância em Research Office) Population) Saúde) Institute for Epidemiology) CCM Vice WHO Faith-based WHO People living NGO Government Chair(s) representative, organization representative with disease (Movimento (Central and (Christian and (National Social da Institute for NGO (AED) Social NGO network Association of Tuberculose) Organization represents Services (HAAC) People Living and people living Commission, with AIDS) Informatization with disease CSSC) of Health Care, Ministry of Finance) CCM Legal Independently Not Not Not Not Not Status incorporated independently independently independently independently independently legal entity incorporated incorporated incorporated incorporated incorporated CCM Located in Embedded in Located in the Embedded in Embedded in Embedded in Secretariat rented office the Tanzania premises of the the Ministry of the Ministry of the Central Location space in the Commission Ministry of Health and Health Research center of for AIDS – Health Population Institute for Ouagadougou a government Epidemiology, agency Ministry of Health and Social Development International Organizations and Bilateral Donors Represented on the CCM WHO Yes Yes Yes Yes Yes Yes UNAIDS Yes Yes Yes No No Yes World Bank Yes No No No No Yes Other UN, UNICEF, UN, UNDP, Australia, International USAID USAID, UNDP, World USAID France, Japan, Labour EC delegation Food Program, USAID Organization France, Italy, (as a chair of Netherlands, UN theme USAID group), USAID Source: Constructed by IEG. Note: UNICEF = United Nations Children’s Fund. parallel structure. 22 Restructured in 2010, CCM membership is now more inclusive than before. Representation of nongovernment actors has increased. The chair is now from the National AIDS Authority for Cambodia rather than the Ministry of Health, and the Vice Chairs are the WHO representative and an AIDS NGO network. The CCM Secretariat has been 22. The Cambodia CCM is actually called the Country Coordinating Committee, and that in Tanzania is called the Tanzania National Coordinating Mechanism. However, the present Review uses the term CCM for all these committees for ease of exposition. 38 professionally staffed, initially with funding from GTZ, then with an annual $44,000 grant from the Global Fund (first introduced in 2008), and subsequently with an expanded grant from the Global Fund, which is providing $218,000 for two years starting June 1, 2010. CSOs feel that the Global Fund approach and engagement provides greater opportunities for them to share “policy space� with the government and donors in the country’s development agenda. 3.26 By contrast, both the Tanzanian and Brazilian CCMs have a dominant government presence. The Permanent Secretary in the Prime Minister’s Office chairs the Tanzanian CCM and the Secretary for Health Surveillance in the Ministry of Health chairs the Brazilian CCM (permanently so, according to the current by-laws). Both secretariats are currently embedded in government agencies. Both CCMs have extensive representation from CSOs, but less effective representation than in Burkina Faso or Cambodia, based on IEG interviews. The Brazilian CCM also does not have effective representation from multilateral or bilateral development partners in the country.23 3.27 IEG found considerable tension between the Ministry of Health and CSOs in Nepal and the Russian Federation. In Nepal, this arose from the lack of capacity of the Ministry of Health to function as the Principal Recipient for the Round 2 grant for HIV/AIDS (approved December 2005). When the Global Fund found that the Ministry lacked capacity, the Ministry sought help from UNDP for management support. The Global Fund formally designated the UNDP as a co-Principal Recipient in 2007, after which UNDP essentially took over the project rather than helping to build up the capacity of the Ministry of Health to implement it. When the Global Fund approved three HIV/AIDS grants in Round 7, it assigned one to UNDP and two others to NGOs, thus bypassing the government entirely. While the Ministry of Health considers itself to be the natural agency to be the Principal Recipient, the NGOs depend on the grant funds to function and want to see results, whoever serves as the Principal Recipient. UNDP, on the other hand, while pleased to help in the fight against HIV/AIDS, does not relish its role as Principal Recipient and would like to discontinue playing this role as soon as another arrangement can be found. 3.28 In the Russian Federation, the early interactions between the Global Fund and the Ministry of Health were difficult. When approached by the Global Fund for what would be Round 3, the government refused to develop a specific proposal and establish a CCM. Instead, a consortium of five NGOs, already active in the Russian Federation and led by the Open Health Institute, submitted their own proposal to the Global Fund (approved in June 2004) in the absence of an established CCM. This action, combined with the publication of a study on the economic and political impact of an unchecked AIDS epidemic in the Russian Federation, led the Russian Federation Government to embrace the AIDS issue more seriously and accept a cooperative association with the Global Fund for the duration of the grant. However, the current climate is not very conducive to Global Fund activities in the country. The CCM lacks substantial representation from the federal Ministry of Health and Social Development. This has led to a 23. Commenting on an earlier draft of this report, the Global Fund Secretariat indicated that the Tanzanian CCM has provided an excellent forum to enhance partnership arrangements among the various country stakeholders and development partner agencies that have contributed to the effective scale up of the country’s HIV, tuberculosis, and malaria responses over the last three years. Partners have provided critical support to capacity building and technical assistance, including proposal development. 39 considerable degree of cynicism concerning the usefulness of the CCM in practice, particularly among recipient NGOs, who see a pronounced adversarial relationship with the Ministry. PROPOSAL PREPARATION 3.29 Cambodia and Tanzania have had the highest grant approval rates among the six countries (Table 9). They are also the two countries in our sample with their own national- level technical review panels, which review all proposals before submission to the Global Fund.24 Cambodia actually has one panel each for HIV/AIDS, tuberculosis, malaria, and HSS, which report to the CCM Oversight Committee. WHO, UNAIDS, UNICEF, and to a lesser extent the World Bank have provided technical support. Tanzania has one Technical Working Group, chaired by the CCM Secretariat and reporting to the CCM Executive Committee. The working group initiates discussions on new proposals, contracts with consultants to prepare concept notes, and submits these to the Executive Committee. If cleared, consultants then develop these concept notes into full proposals that are returned to the Executive Committee for final clearance. All members of the Executive Committee must sign off before submitting a proposal to the Global Fund. Table 9. Six Countries: Grant Proposals Submitted and Approved, Rounds 1–10 Proposals Proposals Recommended by Proportion Submitted to Recommended Technical Review Panel a Global Fund Cambodia 14 25 56% Tanzania b 11 20 55% Burkina Faso 8 20 40% Russian Federation c 3 8 38% Nepal 7 19 37% Brazil 2 8 25% Source: Global Fund Secretariat. a. Totals are less than in Table 6 because some recommended proposals have been converted into more than one grant. b. For Tanzania in Round 9, only the cross-cutting health-systems strengthening part of the HIV proposal was recommended, not the disease part. c. Results for the national-level CCM in Moscow only, not the subnational CCM, which operates in the Tomsk region. OVERSIGHT OF GRANT IMPLEMENTATION 3.30 FYE Findings. While CCMs had been successful in mobilizing country-level stakeholders to submit grant proposals to the Global Fund, the FYE found that CCMs were ill-equipped — in terms of resources, capacity, and political will — to provide adequate oversight of grant implementation. Country-level stakeholders perceived CCMs as political bodies and questioned how such political structures could have an appreciable role in grant oversight. Both gaps and overlaps had emerged in the oversight and implementation 24. The Study Area 2 Country Program Assessment also noted that its own TRP was a factor in Cambodia’s success rate. 40 responsibilities of CCMs, LFAs, Principal Recipients, and Sub-Recipients, as the Global Fund partnership had changed and developed during its first six years. 3.31 IEG found little improvement during its country visits in April-June 2010 in the capacity or effectiveness of CCMs to exercise programmatic oversight of the implementation of Global Fund grants from the country perspective. Generally speaking, the CCMs had neither the authority nor the resources to exercise effective oversight of grant implementation, as envisaged in the Global Fund “Guidance Paper on CCM Oversight.� 3.32 The CCMs generally lacked both the authority and the resources to exercise effective oversight, since they were not a conventional governing body of a partnership program. The CCMs are representative of the clear trend toward stakeholder models of governance in which diverse stakeholder groups are represented, which is changing the power dynamics in many countries. Each CCM also has a secretariat, and therefore an institutional separation of governance and management functions. But these secretariats are small and only responsible for administration and supporting the execution of decisions made by the CCM, such as submitting grant proposals to the Global Fund. Unlike in a conventional governance and management structure, the secretariats are not responsible for implementing the program of Global Fund grants to the country. Rather, the Global Fund Secretariat in Geneva contracts directly with the Principal Recipients to implement the grants. The CCM must endorse the grant agreement for it to be binding. The grant agreement also includes a number of articles that give the CCM the legal authority to carry out its oversight responsibilities and that mandate the Principal Recipient to cooperate with the CCM in performing its role. However, their authority, or their capacity to exercise this authority, was weak at the time of IEG’s country visits in April-June 2010.25 3.33 The CCMs also had few resources to exercise oversight, since the resources to implement the grants flowed directly from the World Bank (as trustee) to the Principal Recipients on the instructions of the Global Fund Secretariat.26 If the CCM were the conventional governing body of a partnership program, the resources to implement the program would flow through the CCM and its secretariat to the Principal Recipients, and the Principal Recipients would be directly accountable to the CCM for implementing the grants. The CCM would also use a share of those resources for its own secretariat to effectively supervise, or contract out the supervision of the implementation of the grants on a day-to-day basis from both a financial and a programmatic 25. Commenting on an earlier draft of this report, the Global Fund Secretariat agreed that the CCMs were ill- equipped to conduct their oversight role in 2010, but that this has improved during the last two years due to significant investments by the Global Fund and its partners in strengthening CCM abilities to provide oversight and in improving the funding streams that support their efforts to conduct proper oversight. Under the new grant architecture, CCMs also have enhanced opportunities to make strategic program-level decisions, including reprogramming and reallocation of funding or responsibilities across Principal Recipients, the addition of new Principal Recipients, and/or the discontinuation of existing Principal Recipients. 26. The Global Fund started providing $44,000 a year to CCM Secretariats in 2008. This was expanded in February 2010. (See www.theglobalfund.org/en/ccm/support/funding.) While basic single-year funding requests are limited to $50,000 a year, expanded two-year funding requests may exceed $50,000 a year. For requests exceeding $100,000 a year, the CCM must mobilize at least 20 percent of the amount above $100,000 from other sources. While this direct financial support was welcome, IEG found that this generally only covered core administrative expenses, including office space and salaries for a small number of staff. It did not cover all the costs incurred in grant proposal preparation, including meetings and seminars, preparation of background reports, and technical assistance, let alone programmatic oversight of grant implementation. 41 perspective. Currently, the FPM is responsible for managing Global Fund grants from both perspectives with the assistance of the LFA, who verifies and reports on grant performance. CONFLICTS OF INTEREST 3.34 Inadequate management of conflicts of interest has also hindered effective oversight at the country level. IEG has found, based on evaluating and reviewing many partnership programs, that real and perceived conflicts of interest are an inherent and essentially unavoidable feature of partnership programs, deriving from the multiple roles that the principal partners play in a program.27 For example, when the Minister of Health chairs the CCM, when the CCM Secretariat is embedded in the Ministry, and when the Ministry is the Principal Recipient of the Global Fund grant (all of which are common situations), then the Ministry has at least three potentially conflicting interests in the program. Other potential conflicts of interest that IEG observed on its country visits include: (a) the CCM Secretariat being located in the National AIDS Commission, (b) voting members of the CCM who are Principal Recipients and Sub-Recipients (which actually violates the Global Fund guidelines for CCM membership), and (c) selecting Principal Recipients and Sub-Recipients from among those CSOs that played a role in originating the grant proposals. 3.35 Some CCMs seem to be identifying and managing these conflicts of interest better than others. In Cambodia, the CCM has had a conflict of interest policy since 2003, soon after it was established. The policy was revised in 2010 along with the structural changes that occurred in the CCM that year. Their new policy requires each member to sign a conflict of interest declaration on an annual basis, and for all contracts and agreements involving Global Fund resources to incorporate a conflict of interest clause. Their 2010 proposal development manual further stipulates that any members of the CCM whose organization or department proposes to be the Principal Recipient or Sub-Recipient in the round at issue may not sit on the Proposal Development Committee, and it also lays down requirements for publication of the names of individuals on the selection committees and the technical review panels. 3.36 In Tanzania, the CCM also has rules for identifying and managing conflicts of interest, but these appear to be less effectively enforced, according to local observers interviewed by IEG. The involvement of many interested CSOs in the grant preparation process also appears to have led to a cascading system of Sub-Recipients. A conservative estimate of the overhead costs incurred by each layer in the five-layer deep implementation structure for the Round 4 and Round 8 HIV/AIDS grants left less than 50 cents of every dollar for the ultimate beneficiary. The indirect cost of management and communication through this complex layered system should also be added to these direct overhead costs. 3.37 It will not be possible to completely avoid conflicts of interests in CCMs, any more than in the 100 or so GRPPs in which the World Bank is involved, particularly those located in the World Bank or in other partner organizations. When a conflict of interest situation arises, one is not automatically in the wrong, just facing a problem.28 Given the 27. IEG, 2011b, The World Bank’s Involvement in Global and Regional Partnership Programs, pp. 59-60. 28. World Bank, 2007, Global Programs and Partnerships, “Identifying and Addressing Partnership Conflict of Interest in Global Programs and Partnerships,� Guidance Note for Bank Staff. 42 pervasiveness of conflicts of interest in partnership programs, the key is to identify and manage them transparently. 3.38 The Global Fund recognizes the challenges that potential conflicts of interest pose to CCMs. It has recently strengthened the CCM requirement for conflict of interest management. The new CCM guidelines, approved in May 2011, require CCMs to develop, publish, and apply a conflict of interest policy to all CCM members, across all CCM functions, and throughout the life of all Global Fund grants. CCM members must periodically declare conflicts of interest affecting themselves or other CCM members, and not take part in decisions where there is an obvious conflict of interest, including decisions relating to oversight and selection or financing of Principal and Sub-Recipients.29 Effectiveness of Country-level Partnerships 3.39 FYE Findings. The FYE found that partnerships at the country level depended mostly on good will and voluntary collaboration to achieve shared impact-level objectives, rather than on negotiated commitments with clearly articulated roles and responsibilities, and did not yet comprise a fully functioning system. As such, they represented more of a “friendship model� than a genuine “partnership model.� Effective operational relationships between the Global Fund and other international organizations in the international health system were largely absent, particularly in providing essential technical assistance in support of Global Fund grants. 3.40 The FYE found that CSOs were now represented in decision-making processes and involved in scaling up disease prevention and treatment efforts, but that tensions remained concerning how closely CSOs could collaborate with government without undermining their commitment to their membership to counter-balance government perspectives. 3.41 The FYE also found consistent weaknesses, problems, and barriers to establishing effective partnerships with the commercial private sector. One of the reasons was the lack of trust of the private sector toward activities led by the government or CSOs. There also remained a perception within the private sector that the Global Fund’s assessment of the private sector’s capacity and resources to support the Global Fund’s agenda was limited to their cash contributions, without sufficient recognition of in-kind support or capacity to leverage resources through co-investment (Macro International 2009b, pp. 33–38). PARTNERING WITH INTERNATIONAL ORGANIZATIONS AND BILATERAL DONORS 3.42 Generally speaking, IEG found the Global Fund was finding its way in existing partner environments characterized by different degrees of (a) the ability of the government to effectively coordinate donor efforts around agreed national strategies and (b) the willingness of donors to collaborate among themselves. Three years after the Study Area 2 Country Program Assessments, IEG found that the situation had generally improved in terms of other partners’ providing technical assistance in support of Global Fund activities. However, country-level partners (both international and domestic) still saw the Global Fund 29. The full CCM Guidelines are available at www.theglobalfund.org/en/ccm/guidelines. 43 as a largely separate development partner agency, represented in the country primarily by the CCM and the FPM, and with its own distinct modalities that were not well integrated into the existing donor coordination mechanisms in the countries. Persons interviewed pointed out that the same was also true of some other large donors (USAID, PEPFAR, and the World Bank) in particular countries. 3.43 IEG found that WHO and UNAIDS were the principal technical partners in the four low-income countries visited, providing in-kind technical assistance in the preparation of background papers, grant proposals, and other technical work. (WHO was a voting member of all four CCMs, and UNAIDS of three.) Technical assistance has also been provided by short-term consultants financed by bilateral donors or provided by embedded resident advisors who serve as counterparts to key managers in the health sector. (USAID and the Centers for Disease Control were using the latter modality extensively in Tanzania.) 3.44 Nevertheless, providing in-kind technical support has put a lot of pressure on partner agencies’ staff time, since this has represented an unfunded mandate. WHO, which has provided the lion’s share of partner agency involvement (followed by UNAIDS) in the CCM and in its technical committees in Cambodia, has only been able to manage this because its office in Phnom Penh (with 23 resident experts) is among the largest in the world. UNAIDS has drawn on its regional office (in Kuala Lumpur) to support the Cambodian program. France and USAID also have dedicated personnel in Cambodia working on Global Fund and CCM activities. 3.45 In Burkina Faso, Tanzania, and Cambodia, multilateral and bilateral donors have negotiated and formalized country-level partnerships with the government in the health sector. These have taken the form of (a) a common funding basket for the general health sector in Burkina Faso and annual plans for HIV/AIDS to coordinate all partners’ financial support, (b) the Health Sector Basket and the SWAp for the health sector in Tanzania, and (c) a sector- wide implementation and management approach and various parallel projects in support of the implementation of the Health Sector Strategic Plans I and II in Cambodia. (The World Bank is contributing to these funding pools in Tanzania and Cambodia.) 3.46 Country-level stakeholders expressed some frustration that the Global Fund was not contributing to these common funding pools. Many interviewees said that they would like to see the Global Fund coordinate its support more closely with that of other donors — for example, by contributing to these common funding pools. In Burkina Faso, however, the Global Fund is contributing to agreed national strategies and programs, even though its funds are not pooled. In Cambodia, IEG found that the Global Fund was willingly being drawn into existing government- donor coordination mechanisms, and that it was forging clear connections with national strategies and action plans. The FPM was consistently participating in the annual joint country-donor planning and review processes in the health sector, but not in the CCM technical working group and development partner agency meetings due to the lack of an on-the-ground presence. PARTNERING WITH CIVIL SOCIETY ORGANIZATIONS 3.47 Some CSOs are primarily engaged in advocacy on behalf of vulnerable groups affected by the three diseases, trying to influence government policies and donor allocations. Other CSOs are providing services to affected persons with grant funds and from their own 44 resources. Both types of CSOs are now represented in CCM decision making, due to the membership requirements of the Global Fund. 3.48 IEG found that this representation has been effective in Burkina Faso, Cambodia, and Brazil, where CSOs have brought their perspectives to bear on Global Fund-supported activities. CSO representatives interviewed by IEG said they felt that they were able to provide genuine input and to influence the collective decisions regarding grant proposals to the Global Fund. Indeed, they found the process refreshing compared to working with other donors who had more specific preferences regarding programming. 3.49 As already indicated, the relationship between the government and CSOs has been strained in Nepal and the Russian Federation. In Tanzania, the government has chaired and played a dominant role in the CCM. The chair appears to have mitigated tensions by arranging for those CSOs to serve on the CCM that have less tendency to challenge the government on Global Fund business, according to IEG interviews. Service-provision CSOs have also been less inclined to challenge the government to avoid damaging their chances of becoming Sub-Recipients of Global Fund grants. 3.50 The capacity of CSOs to provide health services to affected persons is not a significant issue in Brazil or the Russian Federation. Brazil has a robust CSO sector that has been heavily involved in the country’s effective response to the HIV/AIDS epidemic. In the Russian Federation, it was largely the initiative and energies of CSOs that led to the first Global Fund grants to the country, in the absence of a national-level CCM. However, IEG found that the capacity of domestic CSOs to deliver health services to the standards expected of the Global Fund was an issue in the other four countries. Donor preferences for using well-established international NGOs rather than local organizations has hindered opportunities to strengthen the latter’s capacity. Their weak technical, programmatic, and management skills have prevented them from being selected as Principal Recipients and Sub- Recipients, although they are generally better connected to local communities, which will be relevant in sustaining services and benefits in the future. PARTNERING WITH THE COMMERCIAL PRIVATE SECTOR 3.51 IEG found little evidence of effective partnerships with the commercial private sector at the time of its country visits (April–June 2010). While representatives of the commercial private sector have been members of the CCM in five countries (all but the Russian Federation), they have generally been less vocal or influential in decision making, according to IEG interviewees. Other members of the CCM have tended to see the commercial private sector as a potential source of funds for the wider community. Private sector representatives such as the Cambodia Business Coalition on AIDS and the Tanzania AIDS Business Coalition would like to have seen more Global Fund support for their own disease-control programs for private sector workers, such as the Cambodia HIV/AIDS program for garment industry workers (typically poor village girls unfamiliar with urban lifestyles and at higher risk of infection). 3.52 Private sector representatives in Brazil saw, as a possible shift toward greater private sector participation, the recent initiative of the Global Fund to familiarize the Brazilian corporate sector with its operations in Brazil and to promote a sharing of experiences in 45 fighting the three diseases. This took the form of a seminar organized in São Paulo in March 2010, entitled “Public Private Partnerships to Fight HIV/AIDS, TB and Malaria,� with CCM members, the World Bank, and several Brazilian corporations and multinationals based in Brazil in attendance. This initiative could encourage the corporate sector to bring forth some of its own social responsibility initiatives for possible joint funding by the Global Fund and the corporate sector. For this to materialize, however, the Global Fund would need to approach such grant proposals for joint funding with the private sector with some flexibility. 3.53 Commenting on an earlier draft of this report, FPMs cited growing private sector involvement in the countries’ responses to the three diseases in the four low-income countries IEG visited (Box 3). Box 3. Commercial Private Sector Participation in the Four Low-Income Countries In Burkina Faso, the National Coalition of the Private Sector and Enterprises coordinates a range of responses to the HIV/AIDS epidemic for employers and workers, and their families and communities, primarily through a system of committees at the enterprise level. Private sector contributions have included (a) financing the health care of workers and their families (salaries of health workers, medical visits, etc.); (b) setting up contribution funds from enterprises and workers to help workplaces fund their own initiatives; and (c) funds for coordinating activities such as information, a education, and communication activities, training peer educators, and condom distribution. In Tanzania, the Medical Stores Department (MSD) is collaborating with Coca-Cola to improve the supply-chain management and distribution of drugs and commodities from the centralized MSD to rural pharmacies. Coca-Cola is transferring distribution expertise from its bottling companies as well as logistical and supply-chain-management skills via Accenture Development Partners. The Tanzanian Ministry of Health is working with Unilever Tea’s employee clinic in the Mufindi area to be one of 91 medical centers to provide ARVs free of charge to the communities, with financial support from the Global Fund and PEPFAR. The Ministry of Health and the MSD are providing ARVs; Deloitte is providing financial management services; and Unilever is providing the hospital building, staff, and equipment to improve treatment for the surrounding community. In Cambodia, the representative of the Business Coalition of Cambodia is an active member of the CCM, and is coordinating private sector contributions to the national response to HIV/AIDS, tuberculosis and malaria. These include (a) HIV prevention activities for factory workers and other businesses, (b) tuberculosis prevention and treatment through public-private DOTS programs, and (c) establishing public-private committees to help address the problem of counterfeit antimalarial drugs including ACTs. In Nepal, the Federation of the Nepalese Chambers of Commerce and Industry launched the Business Coalition on AIDS in Nepal in May 2011 to help reduce HIV infections among the country’s workforce. The Coalition aims to put in place HIV prevention, treatment, and care programs for b employees and their families living with and affected by HIV. Source: Global Fund Secretariat. a. See also IOE and PEC, 2009, HIV/AIDS Challenges in the Workplace: Responses by Employers’ Organizations and Their Members in Africa, Case Studies and Good Practices, pp. 42–44. b. UNAIDS, “Business Boost for Nepal’s AIDS Response, May 24, 2011. www.unaids.org/en/resources/presscentre/featurestories/2011/may/20110524businessnepal/ 46 Application of Performance-Based Funding 3.54 FYE Findings. The FYE found that the scale at which the Global Fund had attempted to implement PBF was unprecedented in the international health arena. However, this “focus on results� remained a work in progress and had evolved into a complex and burdensome system that had thus far focused more on project inputs and outputs than on development outcomes and impacts. The FYE found important gaps in the quality of PBF data. Inadequate M&E capacities at the country level also limited the feasibility of the PBF approach espoused by the Global Fund. While the system was generating extensive data, it often failed to provide the key elements of information required to inform judgments on effectiveness. The Global Fund’s efforts to improve the PBF system had made it more confusing at the implementation level, contributing to inconsistent application of the model (Macro International 2009b, pp. 30–32). FINDINGS FROM THE SIX COUNTRY VISITS 3.55 On its country visits, IEG found that the Global Fund’s approach to PBF (Box 4) was working reasonably well in three countries (Burkina Faso, Cambodia, and the Russian Federation) in terms of monitoring outputs and coverage in relation to the key performance indicators in the grant agreements, and not well in the other three countries (Tanzania, Nepal, and Brazil). 3.56 In Burkina Faso, IEG found a significant change in perception among Principal Recipients and Sub-Recipients since the Study Area 2 Country Program Assessment in 2007. While the Principal Recipients had found it difficult to adapt to the PBF system at first, they now found it to be a useful system. Several grant recipients had now integrated the Global Fund performance-based indicators into their own planning processes and relied on them for their own decision making and planning. 3.57 PBF was working reasonably well in Cambodia because the country has had considerable experience with it. The Asian Development Bank had first introduced results- based financing in Cambodia in 1999 for contracting of Preferred Health Care and Maternal- Child Health service delivery to district health authorities and NGOs, based on compensation for results. Subsequently other development partner agencies, including the World Bank, had followed with results based financing-type schemes. The experience with applying the Global Fund’s PBF approach has been imperfect, but improving as more Principal Recipients understand the standards against which they are being measured, and the Principal Recipients and the LFA develop a better working relationship. The Principal Recipients viewed PBF as a means to upgrade administrative, procurement, and performance standards to the international level. As in Burkina Faso, this represented a significant improvement from the Study Area 2 Country Program Assessment in 2007. However, the requirement for PBF favors the selection of “established� groups as Principal Recipients such as the Ministry of Health, international NGOs, and the large local NGO networks compared to smaller, local NGOs. 3.58 Both the concept and the details of PBF appear to be well received and well established in Russia. The Local Fund Agent (KPMG) was very satisfied with the way in which the PBF process was working. An important element in its successful implementation 47 Box 4. Performance-Based Funding in the Global Fund As described on its Web site, the Global Fund has very detailed and well documented requirements for grant-level monitoring, which are tied to its PBF approach. The performance framework for each grant, which forms part of the grant agreement, contains a summary of key indicators and targets, which are used to measure output and coverage on a routine basis. Information is collected and used at three main stages of performance evaluation: (a) Regular disbursements (every six months is the default). A few indicators of progress are used for regular financial release every three-to-six months. (b) Annual reviews (every 12 months). These collect the results for all indicators for the year and include a self-assessment of progress, barriers, successes, and failures. The Global Fund uses these updates to report on progress in grant implementation across its portfolio. (c) Phase 2 evaluation (from 18 to 20 months). Funding is committed for an initial period of two years. After 18 months the Principal Recipient makes a submission for Phase 2 funding to cover an additional three years. This overall review of performance includes a comprehensive report on results against targets and against the goals of the grant, and is used as a basis for the Global Fund Secretariat to recommend further funding in Phase 2. For each reporting period, the Principal Recipient prepares a Progress Update and Disbursement Request (PUDR), which consists of a progress report on the implementation of the grant, and a request for funds for the next reporting period. The progress report includes information on the results of the grant against targets, and information on expenditures. The PUDR is reviewed by the LFA and submitted to the Global Fund Secretariat. The Secretariat reviews the PUDR and assesses:  Programmatic achievements: Have programmatic targets been reached?  Financial performance: Are expenditures in line with budgets?  Grant management: Are there issues related to M&E, procurement, and/or financial management? Based on the assessment of the PUDR, the Secretariat assigns a performance rating to the grant on the following scale: A1 – exceeded expectations; A2 – met expectations; B1 – adequate; B2 – inadequate but potential demonstrated; and C – unacceptable. The Secretariat then decides whether to allow the requested disbursement of funds, to allow partial disbursement of funds, or to deny the disbursement request. An outright denial of the request is rare and only happens if a grant is in serious trouble. The Grant Performance Report is prepared by the Global Fund Secretariat when the grant agreement is signed, and it is updated with every PUDR received throughout the life of the grant. Before the end of Phase 1 of the grant, the Global Fund decides whether to continue funding for Phase 2. A Grant Scorecard is prepared with a structured assessment of the grant performance, the decision about whether to continue funding Phase 2 of the grant, and justification for the decision. The PUDRs, Grant Performance Reports, and Grant Scorecards are completed consistently and made public. The Global Fund will only approve Phase 2 funding if the grant is performing adequately. In practice, it is rare to award a grant a “no go� and completely discontinue the grant in Phase 2, and more common for a portion of the funding to be reallocated to better-performing grants. Over the period 2005–09, only 1.9 percent of grants were discontinued after Phase 1, and 13.7 percent of total funding was reallocated from poorer-performing grants (including “no-go�) to better-performing grants. Source: Global Fund Web site. Note: This box describes the Global Fund’s approach to performance-based funding in its own language. The use of concepts and terms is not necessarily the same as for the World Bank or IEG. 48 in Russia appears to be the contribution of information from the Central Research Institute for Health — the research and epidemiology institute for health within the Ministry of Health and Social Development, which is responsible for monitoring and measurement. The work of this institute has provided some of the basis for establishing appropriate monitorable indicators and their measurement. 3.59 The low quality of data and the lax discipline in its collection have compromised the application of PBF in Tanzania. Credibility and timely availability of data have also been issues.30 The recent OIG audit found that PUDRs were not being prepared and submitted on time by the Principal Recipient (the Ministry of Finance and Economic Affairs), and that their accuracy and completeness were not verifiable.31 The LFA has had to contextualize the use of performance information for the purposes of recommending disbursements to the Global Fund Secretariat. Some interviewees suggested that the absence of major disruptions in disbursements to Tanzania has also reduced the effort to ensure that funding is driven by demonstrable performance of results against targets. 3.60 Given the chaotic political situation that prevailed in Nepal until 2008 (covering most of the period of the Rounds 2 and 4 grants), the successful application of PBF is a remote goal. Attention has been focused on the more basic issues of obtaining grants and selecting appropriate Principal Recipients to implement them. The extensive OIG report did not even address the application of PBF in the country.32 It is hard to see how PBF could be instituted rapidly in new grants to Nepal without risking disruption, particularly for HIV/AIDS.33 Applying PBF may be more feasible for tuberculosis and malaria, where local capacity for implementation is greater, but it would still require careful specification of what “performance� means. Unlike the situation with respect to HIV/AIDS, the Ministry of Health and Population, backed by WHO expertise and with a well-defined protocol (DOTS), has established a reasonably well-functioning tuberculosis control program, and also has reasonable capacity to deliver malaria control services with financial support from donors. 30. Commenting on an earlier draft of this report, the Global Fund Secretariat said that two major challenges have been late reporting by the Government Principal Recipient (the Ministry of Finance and Economic Affairs) and the absence of a well-functioning Health Management Information System. The Round 8 HIV grant has plans for strengthening the reporting mechanisms and tracking of funds and health products at all levels; improving overall data quality; and integrating the parallel systems for Global Fund reporting into the mainstream M&E system. The Round 8 grant is also providing funding for satellite installation at the district level to enhance the quality of data collection and the flow of information. 31. Global Fund, Office of the Inspector General, Audit Report on Global Fund Grants to Tanzania, Report No.: TGF-OIG-09-001, June 2009. 32. Global Fund, Office of the Inspector General, Audit Report on Global Fund Grants to Nepal, Audit Report No: TGF-OIG-09-006, February 2010. 33. Commenting on an earlier draft of this report, the Global Fund Secretariat did not agree that implementing PBF in new grants might lead to disruption of services. The application of PBF is challenging in Nepal, but PBF needs to work in situations where M&E is weak and also provides important incentives for improving M&E. The World Bank, the Global Fund, and other external development partners have contributed to institutional capacity building during the last two years, particularly in the National Centre for AIDS and STD Control, which is now the Principal Recipient for the Round 7 and 10 grants. The external development partners, together with the Ministry of Health and Population, recently agreed to make M&E a core element in the country’s HSS grant application for Round 11. Nepal is no different from other countries where support for HIV control is particularly sensitive, and needs constant support and supervision. 49 3.61 The Brazil LFA has found that PBF is not well-suited for the types of Global Fund grants provided to Brazil. The Principal Recipients have been parastatals and foundations that are providing intermediate products in the health system rather than products at the end of the service delivery chain. For example, the tuberculosis grant seeks “to enhance timely TB detection and quality treatment by improving the current information system and by training health workers from HIV and tuberculosis programs in ten metropolitan and Manaus areas in treatment for co-infections.�34 The multiple data systems associated with the multilayered government health systems in Brazil are also inconsistent and do not lend themselves to an assessment of the performance of grants that are small links in a long service chain. The Principal Recipients can only assume that their intermediate inputs contribute to improved final outcomes. Nevertheless, the LFA has taken upon itself to systematically instruct the Principal Recipients on creating recorded trails that allow it to carry out its verification function. COMPARING THE GRANT/PROJECT-LEVEL M&E SYSTEMS OF THE GLOBAL FUND AND THE WORLD BANK 3.62 IEG has undertaken a detailed comparison of the project-level M&E systems of the Global Fund and the World Bank (a) to identify whether and how the findings and conclusions that emerge from the two organizations’ M&E systems can be compared, and (b) to contribute to the ongoing process of identifying good practices for project-level M&E. The comparison is based on actual experience in three countries: Burkina Faso, Lesotho, and the Russian Federation.35 Burkina Faso and the Russian Federation were chosen from the six countries visited because of the existence of World Bank-supported projects with similar objectives to those of the Global Fund grants that were being implemented during roughly the same time period, thereby enabling a comparison with the project-level M&E in the World Bank projects. Lesotho was chosen for the same reason and because IEG has recently completed a Project Performance Assessment Report of the World Bank project that was specifically intended to increase the capacity of Lesotho “to use effectively the resources provided through the Global Fund grant to support the implementation of HIV and AIDS programs� in Lesotho.36 3.63 Grant-level M&E in the Global Fund is specifically tied to its PBF system (Box 4). Project-level M&E in the World Bank aims to create a traceable pathway from a project’s intent and objectives to inputs and activities, to performance against indicators, and ultimately to conclusions about effectiveness — both by the project team and by independent evaluators. This includes an assessment of the Bank’s own performance and that of the borrower, in addition to the outcome of the project as a whole. A results framework, which describes the pathway from project activities to intermediate outcomes and ultimately to the project development objective, is a required annex in the Bank’s project appraisal documents. 34. Grant Number BRA-506-G02-T: “Strengthening of the DOTS Strategy in Large Urban Centers with High Tuberculosis Burden in Brazil.� 35. Cheryl Cashin, forthcoming, “Comparison of the Monitoring and Evaluation Systems of the World Bank and Global Fund for the Global Program Review of the Global Fund to Fight AIDS, Tuberculosis and Malaria.� 36. IEG, Project Performance Assessment Report, Lesotho Health Sector Reforms Project and HIV and AIDS Capacity Building and Technical Assistance Project, June 2010. 50 3.64 IEG found that both World Bank projects and Global Fund grants in the three countries suffered from weak M&E design at the beginning of the projects/grants. There was a particular problem regarding performance indicators. Typically there were too many indicators, they lacked validity, and they often did not fit into a logical framework of inputs, outputs, outcomes, and impacts. The Global Fund has attempted to address the inadequacy of performance indicators by developing a set of “Top Ten� indicators that it recommends to its grantees, but these indicators often were not routinely available in the countries. 3.65 Neither the World Bank nor the Global Fund was successful at identifying data sources up front. The indicators relating to outcomes and impacts were difficult to report due to inadequate data sources in the countries. In general, the performance indicators provided little added value for assessing project/grant performance, for contributing to periodic summative evaluations, or for enhancing policy dialogue. Good monitoring systems do all three — assess progress in implementing activities, facilitate a cumulative assessment of project performance, and identify issues that require policy responses and other solutions beyond the scope of the projects. 3.66 Both the World Bank and Global Fund M&E products were more useful when they were supplemented with other analysis and when results were synthesized and interpreted more broadly. In the projects and grants reviewed, this was done more frequently in World Bank projects. There were also examples of more analytical M&E in Global Fund grants (for example, the Russian Central Public Health Research Institute database used for M&E of the HIV/AIDS grant). 3.67 The World Bank aims to overcome some of these deficiencies in project monitoring with a standardized evaluation process that combines internal self-evaluation and independent review of individual projects. Each project team undertakes a self-evaluation at the completion of every project using a standardized Implementation Completion and Results Report (ICR) submitted to the Bank’s Board within six months of the project closing date. Project M&E data, performance-related reports, and other relevant operations documentation provide input into the ICR. The performance of the project is assessed against standard criteria. Then IEG undertakes an independent review of all completed projects and their ICRs using a standardized desk review that assesses both the project experience, based on information in the ICR, and the quality of the self-evaluation. 3.68 An emphasis on learning from implementation has led to a World Bank culture of acceptance of critical evaluations. The overall outcomes of 38 percent of Bank-financed HNP projects approved since 1997 have been rated moderately unsatisfactory or worse.37 The traceable pathway in the World Bank’s M&E system from project inputs/activities to outcomes made it possible for IEG to complete its 2009 evaluation of the World Bank 37. Cheryl Cashin, forthcoming, Appendix G. This compares to 21 percent of all World Bank-financed projects, rated moderately unsatisfactory or worse, during the same time period. IEG has also rated the quality of 92 percent of the ICRs in the HNP sector as satisfactory or better, compared to 90 percent for all Bank-financed projects. However, IEG has rated the overall quality of project-level M&E as modest or negligible (as opposed to substantial or high) in 76 percent of the HNP projects closing since 2006, compared to 67 percent for all Bank-financed projects (consistent with the findings of the three projects IEG examined in depth in Burkina Faso, Lesotho, and the Russian Federation). 51 Group’s support to HNP based on cumulative self-evaluations and independent reviews of individual project outcomes.38 The conclusions of the evaluation reflected the aggregate performance of projects, which did not lend itself to reinterpretation and subjective conclusions. Given the real challenges that have been faced by the complex nature of World Bank HNP projects in challenging environments, and the willingness to rate projects as unsatisfactory, the evaluation, based on cumulative project performance, was unable to paint an overly positive picture. 3.69 By contrast, evaluation at the Global Fund has a conspicuous gap — the lack of an evaluation at the completion of individual grants. There has been no policy or process until recently within the Global Fund M&E system to determine the overall effectiveness of individual grants, or to generate lessons for future Global Fund activities in the country or in other programs.39 There was also no contribution of the grant-level M&E of Global Fund grants to the summative assessment in Study Area 3 of the FYE. As discussed below in Chapter 5, the FYE was an independent and quality evaluation, but it was constrained by the absence of assessments of the outcomes of individual grants, both because there was no framework in place to do so and because few grants had been completely implemented at the time of the FYE. Therefore, the FYE was based on other information, studies, and analysis, including the 16 country studies for Study Area 2 and the 18 country studies for Study Area 3. The lack of a framework and cumulative assessment of grant performance made it possible to draw conclusions — both positive and negative — about the overall efficacy of Global Fund grants that were not necessarily supported by objective criteria.40 Access and Coverage of Service Delivery 3.70 FYE Findings. The FYE found that the additional funds provided by the Global Fund had clearly resulted in greater availability and utilization of disease-control services and better coverage of affected communities, which should ultimately reduce the disease burden. In the majority of high-burden countries, however, it was not possible to directly measure the impact of the advent of the Global Fund on their disease burdens. Death registration systems and expensive population prevalence surveys were absent. The FYE made projections about impacts, based on measured increases in access and coverage (such as HIV tests and counseling, DOTS treatment, and insecticide-treated bed nets). To address weaknesses in health data systems, the FYE recommended strengthening and integrating country — not just disease-specific — health information systems to fully capture important nationwide events in health. The FYE also recommended that prevention and treatment approaches be country- specific due to the wide variation in disease epidemiology in countries, and the different levels of country capacity to respond. Further, Global Fund grants should be supporting the most 38. IEG, 2009, Improving Effectiveness for the Poor in Health, Nutrition and Population: An Evaluation of World Bank Group Support Since 1997. 39. The Global Fund now expects this gap to be filled with the use of periodic reviews under the single streams of grant funding, as well as the national program reviews and program evaluations planned under its new Evaluation Strategy that the Board approved in November 2011. 40. IEG has found that the weak M&E frameworks have adversely affected the evaluations of most GRPPs. As a result, few evaluations have found much systematic evidence relating to the achievement of programs’ objectives at the outcome level (IEG 2011b, pp. 27 and 34). 52 cost-effective measures, which would require “adjustment� of the Global Fund’s “demand- driven model� (Macro International 2009b, pp. 18–20). 3.71 IEG found that Burkina Faso and Cambodia have used Global Fund grants to expand services for all three diseases, and that Brazil has used the grants to improve the quality of services for tuberculosis and malaria (the only two diseases for which the country has received grants). Burkina Faso and Cambodia have relatively good donor coordination, strong participation of CSOs, appropriate disease-control strategies for their epidemiological conditions, and expanding delivery systems that involve partnerships between central government agencies, local governments, and CSOs. In Brazil, with far less dependence on external funds, Global Fund grants are financing small infrastructure, equipment, and training inputs to improve the quality of diagnosis, treatment, and care within the existing Integrated (federal, state, and local) Health Service. 3.72 Tanzania has a weaker record in grant implementation than in getting grant proposals approved. That the Ministry of Finance and Economic Affairs has been the Principal Recipient for most Global Fund grants, as the financial gatekeeper for all official flows to Tanzania, has led to delays in the flow of funds, such as losses of grant funds in Round 3 and critical delays in the release of funds in Round 8. The Ministry of Finance has continued to put pressure on the CCM to clear new grant proposals, in spite of backlogs in disbursements of existing grants, according to IEG interviewees. Grant performance has been moderate, with some challenges experienced. Two hundred thousand people are currently on ARVs (compared to 20,000 in 2002), over 70,000 pregnant women have received PMTCT, and over 8.5 million people have been treated for malaria using ACT. The Round 8 grant for malaria has financed the distribution of over 18 million insecticide-treated bed nets under the Universal Coverage Campaign. 3.73 Nepal is effectively a post-conflict country. Fortunately, the Ministry of Health and Population had established reasonably well-functioning tuberculosis and malaria programs before the conflict started and was able to sustain these with donor support during the conflict. HIV/AIDS is a newer disease in Nepal that is concentrated in high-risk groups and is threatening to spread more widely due to cross-border migration along the India-Nepal border among sexual and drug-using networks. As indicated above, the Ministry of Health lacked the capacity to function as the Principal Recipient for the Round 2 grant for HIV/AIDS, which led the Global Fund to transfer responsibility to the UNDP. When the Global Fund approved three HIV/AIDS grants in Round 7, it assigned one to UNDP and two others to NGOs, thus bypassing the government entirely. The strained relationship between the government, UNDP, and NGOs has made it challenging to put together an effective response to the disease.41 41. Commenting on an earlier draft of this report, the Global Fund Secretariat agreed that the performance of HIV grants in Nepal is vulnerable. Grants have been rated poorly mainly due to dysfunctional governance. But the situation has improved since 2010. The Global Fund has actively supported the CCM in transferring more and more responsibility to the National Centre for AIDS and STD Control. The Global Fund supported the Family Planning Association of Nepal, an important NGO working with most at-risk people, through a difficult phase and despite severe malfunctions, in order to strengthen national capacity. External development partners have joined hands in building capacity in the Procurement Department of the Ministry of Health and Population to take over ARV procurement fully in 2012. 53 3.74 The Russian Federation has effectively used Global Fund grants to address tuberculosis. Both Global Fund and World Bank resources effectively catalyzed and leveraged substantial additional spending on tuberculosis by the Federation Government. This combined financial support increased the availability of diagnostic laboratory equipment and pharmacologic agents for treating the disease in both civilian and prison settings (preventing discharged and amnestied prisoners from infecting the wider population is a significant tuberculosis-control issue in the Russian Federation). Strong leadership, an effective strategy, and two government orders dealing with treatment have led to successful outcomes. However, the same cannot be said for HIV/AIDS because of the cultural and social forces surrounding the disease and the principal risk groups. Reaching the high-risk and marginalized groups of HIV-vulnerable individuals such as injecting drug users, and preventing the spread of HIV into the general population remains a serious challenge. The relationship between the government and NGOs remains strained. The Principal Recipient NGOs that have been engaged in preventive endeavors remain frustrated over a job only partially accomplished. Equity in Country-Level Governance and Grant Objectives 3.75 FYE Findings. The FYE found that the Global Fund had modeled equity in its guiding principles and organizational structure — for example, in ensuring representation of women and marginalized populations at the level of the Board, Secretariat, and CCMs. However, few systems had been put in place at the country level or in the Global Fund’s own systems to monitor gender, sexual orientation minorities, urban-rural, wealth, education, and other types of equity as part of grant performance or impact assessment (Macro International 2009b, pp. 25–29). 3.76 IEG found significant attention to equity issues in most countries, as evidenced by the membership of affected communities on the CCMs and the objectives of the grants themselves. Many of these efforts aimed to address existing inequities in the delivery of health services between urban and rural areas, between males and females, and to high-risk groups for HIV/AIDS (commercial sex workers, injecting drug users, and men having sex with men). Grants for tuberculosis, in particular, are evidence of attention to equity, since the disease mostly affects poor and marginalized populations. 3.77 IEG found that expanding access to diagnostic and treatment services in rural areas has been a key focus of Global Fund grants in all four low-income countries, all of which are predominantly rural societies. Efforts to decentralize service provision have resulted in noticeable improvements in access to services in rural areas. The Nepal tuberculosis program, for example, now offers DOTS throughout the country, although rural populations still have farther to travel to a health post or clinic. Burkina Faso has recently removed user fees for ARVs, lifting what was perceived to be a high financial burden for many. 3.78 IEG found that reaching high-risk groups in the case of HIV/AIDS has been more difficult and had more variable success. Brazil (with its own HIV/AIDS program) and Cambodia (with the support of Global Fund grants) appear to have had the greatest success in targeting and reaching high-risk groups. Global Fund grants in Nepal have targeted high-risk groups, but with less success. The prevention and treatment programs in the Round 10 grant for HIV/AIDS in Burkina Faso will target high-risk groups for the first time. The Russian 54 Federation Government has yet to face the imbalance in the provision of HIV/AIDS services, which are not reaching marginalized risk groups such as injecting drug users. 3.79 IEG observed some improvements in monitoring the provision of services to previously unserved or high-risk groups. However, further improvements in this area are intimately connected with strengthening the overall country-level health sector M&E systems. Impact of Donor Support for the Three Diseases on Domestic Health Systems 3.80 FYE Findings. The FYE found that the health systems were weak in most developing countries and that large increases in external funding for the three diseases had stretched existing health systems to their limit. The weakness of existing health systems had limited the potential positive impacts of Global Fund-supported activities. While the Global Fund’s reporting requirements had contributed to capacity building in the areas of financial management and M&E skills, they had created additional burdens on limited health systems capacity, in part because these requirements were poorly harmonized and aligned. 3.81 The FYE also found a strong relationship between the existing health system capacity and the quality of grant management. Health systems needed to be strengthened in order to scale up the services financed by the Global Fund. The increasing focus on HSS by the Global Fund and its global partners (GAVI, UNAIDS, and the World Bank) presented an opportunity to collectively address this issue (Macro International 2009b, pp. 21–24). 3.82 During the six country visits, IEG found consequences, risks, and opportunities associated with the effects of Global Fund grants on country health systems. CONSEQUENCES AND RISKS 3.83 IEG found that the large inflow of Global Fund resources into small low-income countries with high disease burdens has tended to create dependency on the Global Fund in the fight against the three diseases. This may be exacerbated because the United States has reduced its earmarked support for ARV treatment, leaving the Global Fund vulnerable to becoming the primary external financier of ART, as has already happened in Burkina Faso and Cambodia. Some Global Fund-supported programs have also become separate and distinct from the broader health sector, as appears to be the case in the Russian Federation, where the Ministry of Health and Social Development is neither represented on the Principal CCM,42 nor involved in the development and implementation of Global Fund grants. 3.84 IEG found that the Global Fund has also drawn away talent from the public sector, due to disproportionately higher financial compensation allowed in the implementation of 42. The Principal CCM operates at the national (strategic) level in Moscow. There is a subnational CCM which operates at the regional level in the Tomsk region for the Round 3 tuberculosis grant. 55 Global Fund grants.43 In Cambodia, the average civil service wage was less than $100 per month, below subsistence level, but salary top-ups approaching $1,800 per month were allowed by Global Fund grants for some very senior positions. This issue has recently been addressed with the adoption of a uniform compensation scheme called the Priority Operating Costs, put forth by the government and signed onto by the Global Fund and other donors.44 OPPORTUNITIES 3.85 Going forward, new initiatives such as the Health Systems Funding Platform45 within the context of the IHP+ should provide opportunities for the Global Fund to better align with country processes. Since the FYE, Platform members (GAVI, the Global Fund, WHO, and the World Bank) have been in joint negotiations with the Cambodian government, and have agreed on (a) joint health reviews, (b) strengthening of the Ministry of Health Management Information Systems and alignment with existing indictors, and (c) harmonization of financial management procedures. In Nepal, large donors (DFID, GAVI, the United Nations Population Fund, UNICEF, USAID, and the World Bank) have recently reached agreement, under the auspices of the Platform, on a common financial management framework — with one report and one audit replacing multiple, agency-specific reports and audits. However, IEG found no evidence during its visit to the country in May 2009 that that the Global Fund was involved in this joint endeavor at the country level. 3.86 It is unlikely, however, that the Global Fund will be able to take the lead in major HSS initiatives, such as that required for the Ministry of Health in Nepal in relation to HIV/AIDS. In practical terms, this means making the Ministry of Health sufficiently competent to receive Global Fund grants for HIV/AIDS and adequately operate as a Principal Recipient. The need is urgent; the country may be losing the fight against the disease. IEG found a consensus among interviewees during its country visit that the World Bank would be best suited to leading such an initiative. The Global Fund could not do so because it lacked a country presence beyond that of the LFA, UNDP appeared eager to withdraw from its unaccustomed position as a Principal Recipient, and none of the NGOs involved in the fight against HIV/AIDS had the capacity to play this role.46 43. Many external agencies have contributed to the loss of institutional capacity in the public sector, as the more talented move to better paying NGOs and project implementation units. 44. An Aid Effectiveness Team in the Strategy, Performance and Evaluation Cluster of the Secretariat is now assisting country teams, including the team in Cambodia, in negotiating and aligning country salaries to local frameworks during grant negotiations. This is pursuant to a coordinated approach to salaries and compensation in Global Fund grants, endorsed by the Policy and Strategy Committee in September 2008. Rather than getting into a detailed analysis of proposed compensation structures, this approach relies on evidence presented by countries of how their proposal is harmonized nationally or based on an interagency framework (if one exists), such as the Priority Operating Costs Framework in Cambodia. 45. The Platform seeks to support health systems and improve health outcomes through improving the harmonization and alignment of member support to countries’ health systems. 46. The World Bank effectively played this role in the case of the Lesotho HIV and AIDS Capacity Building and Technical Assistance Project (approved July 2004). The project was explicitly designed to enhance the country’s capacity to absorb the large amount of resources offered by the Global Fund. The Bank stepped in to provide such 56 FLEXIBILITY OF GLOBAL FUND BUSINESS MODEL 3.87 IEG found that the Global Fund grants have facilitated the expansion of the service- delivery capability of local health systems. In Burkina Faso and Cambodia, strong support for the participation of CSOs, community-based organizations, and faith-based groups in grant implementation has led to greater access to health services in the rural areas. These groups have become a bona fide extension of the countries’ health service. In Burkina Faso, successive grant support for capacity building of community-based organizations has resulted in one of them achieving Principal Recipient certification to implement a tuberculosis grant. The participation of CSOs in the CCMs of both countries has led to the government’s “sharing of policy space� with nongovernmental groups in a constructive way in the country’s health agenda. In Brazil, which has a strong health system, Global Fund grants have supported outreach to vulnerable and marginalized groups and facilitated the participation of people affected by diseases in decision-making committees.47 3.88 IEG found that Global Fund grants have been sufficiently flexible to support non- conventional or innovative measures, as long as these initiatives have the potential to lead to good health outcomes. Global Fund grants have supported state-of-the-art mobile clinics in Burkina Faso, which are now providing counseling, diagnostic, and treatment services for HIV/AIDS and malaria in isolated parts of the country, and malaria grants are providing increasing support for pharmacovigilance in Burkina Faso, Cambodia, and Tanzania. The Global Fund’s AMFm is now providing grant funding to Cambodia, where ACT-resistant malaria has recently been detected, to advance the fight against drug resistance. STRENGTHENING THE RESPONSE OF COUNTRY SYSTEMS BEYOND THE MINISTRY OF HEALTH 3.89 IEG found that the Global Fund business model encourages establishing relationships that go beyond the conventional ministries of health. The Global Fund could help strengthen country systems in the fight against counterfeit drugs and drug resistance by establishing linkages with drug enforcement agencies, and by strengthening their competencies in ensuring quality compliance by the pharmaceutical industry.48 Since one-third of the grant amounts go to drugs and medical commodities, drug regulatory agencies could be invited to participate in specialized committees of the CCMs. In Tanzania, there is already some indication of resistance to ARVs.49 Here, the United Nations Industrial Development Organization — in capacity building support when the Global Fund Secretariat was about to issue a “No Go� recommendation for Phase 2 of its first (Round 2) grant to Lesotho, which would have effectively canceled the grant. 47. An example is the Metropolitan TB Committees, which plan, monitor, and provide social accountability for tuberculosis services. 48. A special Session of African Ministers of Health at the Roll Back Malaria Board Meeting in Geneva, May 2011, “called for strengthening of drug regulatory authorities by building capacity of personnel to enforce licensing and marketing bans, and also to conduct surveillance to ensure the removal of counterfeit and substandard products. Ministers also called for strengthening procurement and supply chain management for ACTs to ensure constant availability within both public and private sectors.� 49. Mosha and others, 2011, “Prevalence of Genotypic Resistance to Antiretroviral Drugs in Treatment-naïve Youths Infected with Diverse HIV Type I Subtypes and Recombinant Forms in Dar es Salaam, Tanzania.� AIDS Research and Human Retroviruses, Apr 27(4): 377–82; Epub 2010, Oct 18. 57 partnership with GTZ, U.S. universities, and faith-based NGOs — has supported quality assurance training, regulatory compliance, and overall quality procedures in the workplace for pharmacists from regulatory agencies, drug manufacturers, and technical training schools.50 3.90 There are opportunities here for the Global Fund to scale up such activities in the context of strengthening country systems for good health outcomes. Drug manufacturers who graduate from such courses can now produce selected drugs that meet Good Manufacturing Practice (GMP) standards, and two companies in Tanzania have applied for WHO prequalification for producing ACTs. Ability to manufacture locally (meeting GMP standards and WHO certification) can help reduce domestic stock outages of essential drugs, including pediatric ARVs. Regulatory agents who have graduated can better detect counterfeit and substandard medicines, and contribute to reducing the risks of drug resistance, a global public good. Thus, this relatively small investment can reap significant national and global health gains. Institutional Risk Management by the Global Fund 3.91 FYE Findings. The FYE found that weak management of risks — including financial, organizational, operational, and political risks — has been one of the vulnerabilities of the Global Fund. The main risk-mitigation instruments had comprised LFA assessments, financial disbursement “red flags,� and the Early Alert and Response System, which was intended to provide early identification of underperforming projects and to facilitate timely corrective actions.  Financial risks stemmed from poor procurement practices at the Principal and Sub- Recipient levels, and from high reliance on the CCMs (which had no legally binding relationship with the Global Fund) to protect the Global Fund from misuse of funds.  Organizational risks arose from the difficulty in demonstrating the right kind of results to its investors and partners (such as outcomes and impacts as opposed to inputs and outputs), from the weak absorptive capacity of domestic health systems to receive Global Fund grants, and from the absence of a comprehensive partnership strategy that clearly delineated responsibilities among partners.  Operational risks arose from the tensions between the Global Fund Secretariat, CCMs, Principal Recipients, and LFAs around the application of country ownership and PBF principles, weak institutional capacities, and insufficient investment by the Global Fund and its partners in country-level health information systems to report on the outcomes and impacts needed for PBF.  Political risks arose from the Global Fund being misunderstood and being seen to have exclusive responsibility for financing life-saving treatments in poor countries and from unclear responsibility (among the Global Fund and it partners) for addressing “global communicable disease governance issues� such as the risk of drug resistance for the current treatments for the three diseases (Macro International 2009b, pp. 44–49). 50. http://www.unido.org/fileadmin/user_media/Services/PSD/BEP/Flyer%2018%20Nov2010%20TEGLO- 0515-08030%20Generics_fin.pdf 58 3.92 IEG found that the Global Fund Secretariat is giving priority attention to improving risk management at the corporate and country levels following a Board directive in 2007 and in response to the FYE recommendations. An accountability framework has been developed with a new grant-rating system, and a cross-Secretariat Risk Management Working Group has been established to address fraud and corruption in countries. A risk register has been created at the Secretariat with focal persons dedicated to managing each risk area. Clearer policies and guidelines have been provided to countries. The authority and resources of the Office of the Inspector-General have been strengthened to provide independent and objective assessments of topics that pose risks to the Global Fund, and on fiduciary risks and controls. 3.93 The Secretariat started to deploy Country Teams in September 2010 to manage grants in 13 high-impact countries with large volumes of funding, multiple grants, complex operations, or other major challenges. These Country Teams replaced the previous system in which the FPM had to obtain “sign offs� sequentially from other staff responsible for technical compliance, particularly finance, M&E, and procurement. The teams aimed to foster a sense of joint ownership and responsibility among all team members (including the LFA, who is a part of the Country Team), shifting their roles from compliance-checking to a more proactive and supportive stance. The teams are bringing together the full grant- management expertise of the team members, and, based on the initial experience, deepening the involvement of technical experts in grant-related processes, enabling them to develop better relations with in-country stakeholders, and freeing the FPMs to focus more attention on in-country interactions, partnership building, and risk mitigation, which had previously received insufficient attention. The presence of a Partnership Officer on each Country Team is also nurturing links with civil society, the commercial private sector, and other country- level stakeholders. Although the country team approach is significantly increasing the demands on some staff, the Secretariat deployed teams for an additional 29 countries in April 2011, and plans to deploy teams for a further 5 countries by December 2011. 3.94 Financial risks. The Global Fund contracts with LFAs to verify and report on grant performance. They make recommendations to the Global Fund on grant disbursements and identify risks relating to grant implementation. As LFAs, PricewaterhouseCoopers and KPMG have been responsible for the largest number of countries and the largest amount of approved funds.51 3.95 IEG found that this system was working well in Burkina Faso, Cambodia, Brazil, and the Russian Federation. LFAs have often been criticized for not having enough public health expertise. However, the LFAs in Burkina Faso and Cambodia (from the Swiss Tropical and Public Health Institute) had expertise in both public health and finance, enabling them to “speak the same language� of public health when working with the Principal Recipients and Sub-Recipients. The LFA in Russia (KPMG) was assisted by a Central Coordination Team in San Francisco, which included health professionals. 51. Global Fund, 2007, Evaluation of the Local Fund Agent System, p. 3. Other LFA service providers have been CARDNO Emerging Markets (formerly Emerging Markets Group), Crown Agents, Deloite Touche Tohmatsu, Finconsult, Grant Thornton, Swiss Tropical and Public Health Institute, and the United Nations Office of Project Services (UNOPS). The World Bank was initially the LFA for a Round 1 grant for tuberculosis control in India. 59 3.96 The Global Fund has undertaken a number of steps to strengthen the performance of LFAs pursuant to a Board decision in April 2007 that “LFAs must be able to monitor financial management performance and program performance and link the two components together.� The Secretariat retendered all LFA contracts in 2008, requiring applicants to be able to monitor not just the financial management of the grants but also programmatic health aspects, procurement, supply-chain management, and M&E. The Secretariat updated the LFA Manual in August 2008, providing more explicit guidance on identifying risks to grant performance, and introduced a performance evaluation and feedback system for LFAs in 2009. It is likely that these actions have contributed to the improving situation since the FYE conducted country case studies in 2007. For example, the previous LFA in Brazil was found to be underperforming and was retendered. IEG found that the current LFA in Brazil (Deloite Touche Tohmatsu) was diligent and strict about the use of grant funds. The LFA had recommended rejection of one disbursement application because funds had been shifted from one line item in the grant to another, thereby sending the message that Principal Recipients had to respect the planned use of grant funds. 3.97 IEG was not able to form a judgment on the current situation in Nepal — to what extent things had improved since the chaotic situation that prevailed during the civil war in 2005–06. The LFA in Tanzania (PricewaterhouseCoopers) has identified misuse of funds and fraud, but has faced a government reluctance to prosecute such acts. The LFA welcomed the recent OIG audit which shed light on many irregularities in procurement, a common locus for fraud.52 Correcting this vulnerability would require rigorous implementation of the many recommendations in the OIG report.53 3.98 Organizational risks. The principal organizational risk that IEG identified on its six country visits was the failure to implement an effort of sufficient scale in Nepal and the Russian Federation to reach high-risk and marginalized groups of HIV-vulnerable individuals. 3.99 The NGOs whom IEG interviewed in Nepal were justifiably proud of the efforts that had been made to educate people about the disease, to provide voluntary counseling and testing, and to deliver treatment to some HIV-positive patients, but prevalence seems to be rising and expanding treatment increases the financial burden. The Global Fund is not able to take the lead in building up the capacity of the Ministry of Health to effectively deal with the AIDS epidemic; it has to rely on other partners, including the World Bank, to step up to the plate. 3.100 In the Russian Federation, the Government has yet to face up to the challenge of reaching marginalized risk groups such as injecting drug users, and thereby prevent the 52. Global Fund, 2009d, Office of the Inspector General, Audit Report on Global Fund Grants to Tanzania, Report No. TGF-OIG-09-001, June. 53. Commenting on an earlier draft of this report, the Global Fund Secretariat said that the LFA in Tanzania has put in place a risk management framework as mandated by the Global Fund. The Global Fund is also working with the CCM and Principal Recipients to ensure that each Principal Recipient has a risk management framework in place. The CCM, Principal Recipients, and development partners are also involved in a graft-theft mitigation initiative to proactively find joint solutions. 60 disease from spreading into the general population. Nepal and the Russian Federation had the lowest coverage, among the six countries visited, of persons in need of treatment (Table 5). 3.101 Operational risks. As already indicated, IEG found that there has been little improvement since the Country Program Assessments in 2007 in the capacity or effectiveness of CCMs to oversee the implementation of Global Fund grants from the country perspective. Communications between the CCM, which is responsible for programmatic oversight, and the LFA, which is responsible for fiduciary oversight, have proven to be a sensitive matter, since the LFA is an agent of the Global Fund Secretariat, not the CCM. While the LFA seeks to preserve his or her independence and obligations to the Global Fund Secretariat, the CCM and the Principal Recipients seek better feedback from the LFA about grant performance.54 The chair of the Tanzanian CCM, for example, expected complete openness on the part of the LFA, but the LFA viewed its own communications with the Global Fund Secretariat as a confidential matter.55 3.102 Both the FYE and the Global Fund Report on the CCM Model found a need for better communication between these two entities.56 In Burkina Faso, only the chair and the CCM secretary meet regularly with the LFA; neither the CCM nor a CCM committee does so. In Cambodia, the LFA attends CCM meetings as an observer and is well informed about CCM matters. The Cambodia CCM has also stepped up oversight procedures, improved conflict of interest management by disallowing potential Principal Recipient membership on the CCM, and established a separate Oversight Committee. Given the emergence of ACT-resistant strains of malaria in the country, the Cambodian Oversight Committee could potentially invite representation from national drug regulatory authorities in order to improve national oversight and quality assurance of pharmaceuticals procured by Global Fund grants, and help eliminate poor quality and counterfeit drugs (a significant issue in Cambodia). Representation of drug regulatory authorities could also be an important consideration for the Tanzanian CCM or its committees because of the large drug portfolio in its Global Fund- supported activities. 3.103 Political risks. The Global Fund is now perceived as the largest external financier of ARV for people living with AIDS and for PMTCT, and the primary financier of first- and second-line tuberculosis drugs, and malaria ACTs. This was evident in all four of the low- income countries visited. Among these countries, only Cambodia was taking immediate steps to more stringently manage all ART programs, introduce cost controls, and strike a better balance between prevention and treatment. This did not appear to be the case in Tanzania, which has a large Global Fund portfolio in all three diseases. In Nepal, the political risks 54. The Global Fund Secretariat is well aware of this issue. See Global Fund 2010g. “Recommendations to Enhance In-Country Communications between the Secretariat, LFA, PR, CCM and Other Partners.� 55. Commenting on an earlier draft of this report, the Global Fund Secretariat said that the Tanzanian CCM has now given the LFA a platform during every CCM meeting to highlight key issues in grant implementation/ management and to provide a second opinion on the Principal Recipient’s progress reports. The LFA has also made regular presentations to the Development Partners’ Group. 56. Global Fund, 2008, Lessons Learned in the Field: Health Financing and Governance: A Report on the Country Coordinating Mechanism Model, p. 52. 61 were specifically associated with the HIV/AIDS program and with securing new Global Fund grants to continue ARV treatment begun with past grants.57 3.104 There are also higher-level political risks, such as those relating to the global governance of communicable diseases and drug resistance. The increased risks of drug resistance (and ethical issues) arise from the unprecedented scale of treatment supported by Global Fund grants, should treatment regimens be disrupted for any reason, such as the inability to meet performance standards. One response to date has been the establishment of AMFm — the Fund’s new business line in the affordable provision of ACT combination drug therapy, which started pilot activities in 2010. The Global Fund has also improved its operational procedures (single streams of funding and the National Strategy Applications) to reduce disruption of grant activities, but these may not be sufficient to manage the political and reputational risks associated with becoming the world’s primary external financier of treatment for AIDS, tuberculosis, and malaria. 3.105 The increasing quantity of counterfeit drugs is accelerating drug resistance. WHO estimates that as much as 25 percent of the drugs sold in the developing world are counterfeit — a lucrative trade that will reach $75 billion a year in 2010 according to the Center for Medicine in the Public Interest in New York City. More than 50 percent of the antimalarial artesunate in South East Asia is counterfeit; some of it has toxic ingredients, while other portions have small amounts of genuine artesunate, which increases the risk of drug resistance. Following IEG’s country visit to Cambodia, the Global Fund, through AMFm, planned to flood Cambodia and 10 African countries with cheap, high-quality malaria medications to reduce the use of substandard medications by patients and to make the market less profitable for counterfeiters. By negotiating with legitimate ACT producers and subsidizing the costs of the medicine, AMFm is aiming to reduce retail treatment costs from more than $6.00 to less than 50 cents per patient. 57. Commenting on an earlier draft of this report, the Global Fund Secretariat agreed that the effectiveness of the HIV program in Nepal remains a big concern, but that the situation has improved since IEG’s country visit in May 2010. The Global Fund Board approved the country’s Round 10 proposal for HIV/AIDS in December 2010, thus securing external financial support for HIV/AIDS for the next five years. The National Centre for AIDS and STD Control is now the Principal Recipient for the Round 7 and 10 grants. Still, strategic and day-to- day management are weak, and forecasting ARV needs remains challenging due to poor stock management and consumption data surveillance. 62 4. The World Bank’s Engagement with the Global Fund at the Global and Country Levels 4.1 The purpose of this chapter is to draw lessons for the future from the past engagement between the World Bank and the Global Fund at the country level. The context for this engagement includes: (a) the Bank’s engagement with global health partnerships more generally, (b) the roles that the Bank plays in the Global Fund at the corporate level, (c) the various initiatives associated with GHAP and IHP that have provided additional avenues for World Bank-Global Fund engagement at the country level, and (d) the Bank’s own country programs in the health sector. The first part of this chapter describes the World Bank-Global Fund engagement at the global level to provide context for the findings on country-level engagement in the second part. The Bank’s Involvement in Global Health Partnerships Prior to the Establishment of the Global Fund 4.2 The Bank has been involved in global and regional health partnerships for more than 30 years, starting with the Special Programme of Research, Development and Research Training in Human Reproduction in 1972, the Onchocerciasis Control Program in West Africa in 1974, and the Special Programme for Research and Training in Tropical Diseases in 1975, all of which have been financed by the Development Grant Facility (DGF) and its predecessor, the Special Grants Program (Appendix K). Then the Bank became involved in eight more global and regional health partnerships between 1994 and 2001, as follows, all of the them supported by the DGF:  UNAIDS (the Joint United Nations Program on HIV/AIDS), 1994  International AIDS Vaccine Initiative, 1996  European Observatory on Health Systems and Policies, 1997  Global Forum for Health Research, 1998  Roll Back Malaria (RBM), 1998  Medicines for Malaria Venture, 1999  Global Alliance for Vaccines and Immunization (GAVI), 2000  Stop Tuberculosis Partnership (Stop TB), 2001. 4.3 Along with the pressures of globalization, the Bank has played important but quite varied roles in contributing to the growth of GRPPs for better health outcomes. Some have suggested that the World Bank’s 1993 World Development Report, Investing in Health, played a major role in putting health on the global agenda. Bill Gates has explicitly stated that reading the 1993 World Development Report prompted him to become involved in global health, initially by donating more than $1 billion to support vaccinations in the developing world.58 The 1993 World Development Report, along with the parallel study, Disease Control Priorities in Developing Countries, also raised awareness of some important global public goods dimensions of health such as health research and communicable diseases – two of the new 58. Michael Specter, “What Money Can Buy,� The New Yorker, October 24, 2005. 63 programs fit into both these categories (the International AIDS Vaccine Initiative and the Medicines for Malaria Venture). Other influences, such as the growing AIDS epidemic, are clearly important. World Bank President James Wolfensohn also explicitly promoted the establishment of such partnerships during his tenure (1995–2005) in order to open up the Bank and improve the efficiency of international development assistance.59 4.4 When the Global Fund was established in 2002, the Bank was involved in six global health research programs, three technical assistance programs (UNAIDS, RBM, and Stop TB), and two country-level investment programs (the African Programme for Onchocerciasis Control and GAVI), but the Bank did not have a country-level operational role in any of these programs. The Global Fund was the first global or regional health partnership program that would finance country-level investments in which the program expected the Bank to provide technical support along with other development partner agencies (WHO, UNAIDS, RBM, and Stop TB).60 However, the extent to which the Bank accepted or acknowledged this role appears to have been deliberately left vague due to the tensions surrounding the establishment of the Global Fund at the time. There was no formal agreement or MOU between the World Bank and the Global Fund to this effect, and there were no written directives or guidelines issued to staff in either organization for engaging with the other at the operational level in the country.61 The only formal agreement between the two organizations was the trusteeship agreement relating to the Bank’s management of the Global Fund trust fund.62 4.5 The World Bank’s principal prior experience with global programs that financed investments at the country level were the GEF and the Multilateral Fund for the Implementation of the Montreal Protocol, both established in 1991. However, these programs had a different operational model from that of the Global Fund. The Bank was explicitly designated as one of the implementing agencies for both programs, and as such was explicitly responsible for preparing project proposals and supervising their implementation, as for regular Bank projects. The two programs also reimbursed the Bank for services rendered in assisting eligible governments in the development, implementation, and management of their projects. (See Appendix L.) 4.6 When the Global Fund was established, the Bank had also recently reviewed and expanded its own response to the AIDS pandemic to help fight the disease in countries where 59. James D. Wolfensohn, 2010, A Global Life, pp. 305-306. 60. While GAVI was established in 2000, before the Global Fund, the program was initially located in UNICEF in Geneva, and UNICEF was the principal implementing agency for GAVI. 61. World Bank Management made four presentations to the Bank’s Board between January 2002 and March 2005 on the Bank’s evolving relationship with the Global Fund, but these did not constitute directives or guidelines to country-level Bank staff for engaging with Global Fund-supported activities at the country level. These discuss possible roles for the Bank in the Global Fund in addition to trustee, such as Principal Recipient for some Global Fund grants, and cite examples of existing country-level engagements such as common implementing agencies and coordinated supervision of parallel World Bank projects and Global Fund grants, but do not provide specific guidance in terms of what is permissible, encouraged, or required for country-level Bank staff. 62. More recently, the World Bank’s Integrity Vice President and the Global Fund Inspector-General also signed an information-sharing MOU in October 2010 to share information relating to fraud and corruption in the use of each organization’s resources. 64 AIDS was most threatening. The Bank issued an expanded Africa HIV/AIDS Strategy in June 1999, Intensifying Action against HIV/AIDS in Africa: Responding to a Development Crisis,63 and the Bank’s Board approved the first Multi-Country AIDS Program (MAP) in September 2000, earmarking $500 million in IDA credits for financing AIDS projects in Africa, and $155 million in Caribbean countries. The Board approved a second $500 million envelope in February 2002. The second set of MAP projects allowed financing of antiretroviral treatment and, for the first time in the history of IDA, support to client countries in the form of IDA grants.64 The Bank ended up committing almost $2 billion to MAP projects in Africa and the Caribbean over the subsequent 10 years (Table 10). Table 10. Multicountry AIDS Program Projects, by Region and Approval Year 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 Total Number of Projects Africa 7 9 5 9 5 4 6 4 2 3 54 Caribbean 2 1 3 2 3 1 1 13 Total 9 10 8 11 8 4 6 5 3 3 67 Commitments (US$ millions) Africa 287.2 262.3 172.8 355.9 80.0 247.7 185.4 65.8 55.0 55.0 1,767.1 Caribbean 40.2 15.0 30.1 19.0 21.4 10.0 35.0 170.6 Total 327.4 277.3 202.9 374.9 101.4 247.7 185.4 75.8 90.0 55.0 1,937.7 Source: World Bank data. Note; All projects except one are mapped to the HNP Sector Board. (One Mali project, approved in 2004, was mapped to the Finance and Private Sector Development Sector Board.) The Bank’s Roles in the Global Fund at the Corporate Level TRUSTEE 4.7 First and foremost, the Bank is the administrator of Global Fund trust fund. Under the trusteeship agreement, the Bank receives and invests funds from Global Fund donors, commits and disburses the funds to grant recipients on the instruction of the Global Fund Secretariat, and provides regular reports to the Global Fund. The Bank is not responsible for mobilizing donor resources or for fiduciary oversight to ensure that grant disbursements are used for the intended purposes, only that fund recipients are legitimate entities. 4.8 In World Bank parlance, the Global Fund trust fund is a financial intermediary fund (FIF) in which the Bank provides “a specified set of administrative, financial, or operational 63. Previous Bank strategies to address AIDS in Africa included AIDS: The Bank’s Agenda for Action in 1988; Combating AIDS and Other Sexually Transmitted Diseases in Africa: A Review of the World Bank’s Agenda for Action in 1992; the Regional AIDS Strategy for the Sahel in 1995; AIDS Prevention and Mitigation in Sub- Saharan Africa: An Updated World Bank Strategy in 1996. See IEG 2005, Box 2.1 on page 14. 64. Donors agreed that 18–21 percent of IDA 13 resources (2003–05) should be provided on a grant basis. All AIDS projects or components in low-income countries have been eligible for IDA grants since April 2003, as have 25 percent of AIDS projects or components in blend countries (those eligible for both IDA credits and IBRD loans). 65 services.� It is currently 1 of 16 such funds administered by the World Bank that are providing financing for 13 GRPPs (Appendix K), and that collectively account for more than 50 percent of the trust funds administered by the Bank.65 In 2002, the Global Fund trust fund was the sixth FIF to be established at the Bank and the second in the health sector (after the Onchocerciasis trust fund that supports the African Programme for Onchocerciasis Control). Subsequently, three more FIFs have been established to support GAVI, four more to support four agriculture and environment programs, and one for the Global Partnership for Education.66 4.9 The World Bank plays an operational role, as one of the implementing agencies, in all seven of the agriculture and environment programs that are supported by FIFs, and the Global Partnership for Education. As indicated in Chapter 2, that the Bank might play such an operational role in the Global Fund was never seriously considered by the Transitional Working Group in 2001. However, there were considerable pressures in the Working Group for the Bank to take on an “enhanced fiduciary role,� in addition to being the trustee, to help ensure that grant disbursements were used for the intended purposes. The Bank was unenthusiastic about exercising fiduciary oversight for projects for which it did not also have programmatic oversight in accordance with its own operational policies, which would have required a substantial scaling up of country-level HNP staff. When the Bank declined to do so, the Global Fund Board decided in April 2002 to establish the LFA system of contracting out in-country fiduciary functions to LFAs. 4.10 According to the trusteeship agreement signed in May 2002, the World Bank invests undisbursed funds “in such manner, and such form, as it may decide, consistent with its established practice of managing other trust funds held by it.� The income from these investments, which is credited to the trust fund, represented 5.4 percent of the total resources available to the Global Fund from 2002 to 2010, and has more than covered the cumulative administrative costs of the Global Fund, including staff salaries, other Secretariat costs, LFA fees, funding for CCMs, and the trustee fee paid to the Bank for administering the trust fund (Table 2 in Chapter 2). GOVERNANCE 4.11 The Bank is a permanent nonvoting member of the Global Fund Board, along with UNAIDS, WHO, and one representative of partners (RBM, Stop TB, and UNITAID). For the Bank to be a nonvoting member is the usual situation for FIF-supported programs, even those for which the Bank is an implementing agency. The Bank is an official observer on the GEF Council, a nonvoting member of the Trust Fund Committees for the two Climate Investment Funds, and a nonvoting member of the Global Agriculture and Food Security Program (Appendix K). The Bank is only a voting member of the governing bodies of those FIF- supported programs (the African Programme for Onchocersiasis Control, GAVI, and the Consultative Group on International Agricultural Research) in which it has also been a financial contributor, by means of annual grants from the DGF. The Bank is also a voting member of all 65. The World Bank also administers the Debt Relief Trust Fund (formerly the Heavily Indebted Poor Countries Initiative) and two country-level FIFs (for Guyana and Haiti), which are not GRPPs. 66. See IEG, 2011a, Trust Fund Support for Development: An Evaluation of the World Bank’s Trust Fund Portfolio, Appendix F, for a brief description of all the FIFs managed by the Bank. 66 the other global health partnerships to which it is contributing financially (also through the DGF), except for the International AIDS Vaccine Initiative and the Medicines for Malaria Venture.67 4.12 The Bank is officially represented on the Global Fund Board by the Vice President for Concessional Finance and Global Partnerships, by virtue of this vice presidency being responsible for managing the Global Fund trust fund. However, the Director of the Multilateral Trusteeship and Innovative Financing Department usually attends the Board meetings on behalf of the vice president along with representatives of the HNP Department (the Bank’s alternate representative). Each Board member is entitled to send up to 10 representatives to each Board meeting, but the Bank has never sent more than 5 (Appendix J). The Bank is also a member of two Board committees — the Finance and Audit Committee, by virtue of its trusteeship role, and the Policy and Strategy Committee, by virtue of its experience in the health sector. 4.13 The Global Fund is representative of the clear trend toward stakeholder models of governance of GRPPs in which membership on the governing body is not limited to financial contributors, but is also extended to noncontributors such as beneficiary countries and CSOs (and to a lesser extent the commercial private sector).68 Most of these GRPPs, like the Global Fund, also have constituency-based boards in which various stakeholder constituencies have a certain number of seats. 4.14 However, IEG has not been able to observe that one governance model is more effective than the other. Direct representation does not necessarily translate into effective voice; noncontributing stakeholders may be able to express their interests more effectively in other ways (IEG 2011b, p. 50). For instance, the Bank has a robust civil society engagement around health issues including HIV/AIDS, malaria, tuberculosis, reproductive health, and nutrition. The Bank also launched a Civil Society Consultative Group for HNP in early 2011 to facilitate and expand this engagement.69 4.15 In IEG’s experience, whether nonvoting members have as much influence over Board decisions as voting members depends on the history and culture of each organization and the extent to which decisions are made by consensus rather than by voting. What is clear, however, is that stakeholder models of governance represent a significant shift from shareholder models in which membership on the governing body is limited to financial contributors and with which the Bank has had more experience in other sectors. 4.16 When the Global Fund was established, the influence of the Bank on the Global Fund Board would not be determined by what the Bank was — that is, the largest external financier of health sector investments in developing countries in 2002 — but by its ability to 67. Unlike the other global health partnerships that have constituency-based boards in which each constituency is assigned a certain number of seats, the International AIDS Vaccine Initiative and the Medicines for Malaria Venture are product development public-private partnerships that are governed by self-perpetuating boards in which board members appoint their successors. Their boards consist of distinguished individuals from industry, academia, and technical agencies. The Gates Foundation is the major donor represented on both these boards. 68. IEG, 2011b, The World Bank’s Involvement in Global and Regional Partnership Programs, pp. 49–50. 69. Direct representation of CSOs on the Bank’s Executive Board would not, of course, be possible without a major change in its Articles of Agreement since the Bank is an intergovernmental organization in which only member countries can be represented on its Board. 67 make a positive contribution to the governance of the program. It is doubtful that the diminished status of the Bank and other nonvoting members significantly reduces the Bank’s reputational risks of being involved with the Global Fund, given the role that the Bank plays in global health and its extensive engagement with the Global Fund at the global and country levels, as documented in this chapter. 4.17 The FYE found that the Global Fund governance structure and processes had achieved both broad participation and genuine power-sharing between key constituencies in the fight against the three diseases. The participation of CSO and private sector constituencies has been broadly viewed as effective, while that of some other constituencies (such as affected communities) has been less effective due to the size of the constituencies and the absence of easy mechanisms to communicate effectively within the constituencies. 4.18 The FYE also found that the Board had tended to focus its attention on near-term and micro issues such as the operational functions of country mechanisms (CCMs, PBF, and the LFAs) to the relative neglect of longer-term and larger issues such as organizational vision and strategy. The FYE suggested that the Board’s focus on operational issues was “an unavoidable consequence of a previous decision to establish the LFA system rather than rely on the in-country capacities of the World Bank or other partners� (Macro International 2009b, pp. 50–55). As indicated earlier, the Board has since chosen to delegate more decision-making authority on operational matters to its Committees and the Secretariat and to focus on core strategic issues more consistent with its governance role. The Global HIV/AIDS Program, the International Health Partnership, and Related Initiatives 4.19 The various initiatives associated with GHAP and IHP have provided additional avenues for the World Bank to engage with the Global Fund at the country level. THE GLOBAL HIV/AIDS PROGRAM AND RELATED INITIATIVES 4.20 The Bank established GHAP, in partnership with UNAIDS, in June 2002 to support the Bank’s efforts to address the HIV/AIDS pandemic from a cross-sectoral perspective, and to lead the M&E efforts of UNAIDS partners through the Global HIV/AIDS Monitoring and Evaluation Support Team (GAMET). Supported by a UNAIDS trust fund established for the purpose, GHAP was established to strengthen institutional capacity across the Bank to respond to the AIDS epidemic, provide specialized technical expertise and knowledge, and support cross-cutting and multisector engagement. It has become the central coordination unit that supports the management of the Bank’s institutional capacity on AIDS. 4.21 UNAIDS contributed $57.1 million from 2003 through June 2010 to the Bank- administered trust fund to support the various activities of GHAP discussed immediately below (Table 11).70 By way of comparison, GAVI has also contributed $11.3 million to a 70. It could be argued that 60 percent of the funds to the UNAIDS trust fund have effectively come from the World Bank, since the DGF contributed $36 million to UNAIDS over the same time period. This potential conflict of interest — receiving with the left hand what was given by the right hand, when DGF funds are supposed to leave the Bank — has not been transparently acknowledged in DGF Annual Reports. 68 trust fund at the World Bank to finance Bank-executed activities in support of GAVI’s goals and objectives. Table 11. Contributions to and Disbursements from UNAIDS and GAVI Trust Funds at the World Bank (US$ millions) 2002 2003 2004 2005 2006 2007 2008 2009 2010 Total a UNAIDS Trust Fund for GHAP Donor contributions - 3.1 8.4 1.9 6.4 6.9 11.1 9.6 11.2 58.6 b Disbursements - 2.4 3.8 3.8 4.0 6.0 8.4 6.7 10.4 45.6 GAVI Trust Fund Donor contributions 0.5 0.3 – 0.6 – – 5.1 2.5 2.3 11.3 Disbursements 0.4 0.3 0.0 0.4 0.1 0.2 1.3 2.0 2.7 7.4 c DGF Grants To UNAIDS 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 4.0 36.0 To GAVI 0.5 1.5 1.5 1.5 1.0 0.5 – – – 6.5 Source: World Bank data. a. DFID contributed 0.5 million and 1.0 million to GHAP in 2009 and 2010, respectively, for an evaluation of community response to HIV/AIDS. b. All but $200,000 has been Bank-executed, indicating that Bank staff have been directly responsible for supervising the GHAP activities financed by the trust fund. c. Contributions from the Bank’s DGF to UNAIDS and GAVI. 4.22 GAMET. This team aims to improve the quality of HIV/AIDS M&E and to build national capacity for one country-owned M&E system in each country — what has come to be known as the third of the “Three Ones.� It helps to strengthen national M&E capacity through an international team of M&E specialists, based primarily in developing countries, who aim to provide rapid, flexible, and practical M&E support to beneficiary countries. 4.23 The Three Ones. A consultation on Donor Harmonization of AIDS Funding held in Washington, DC, in April 2004 endorsed the application of the Three Ones principles, to be applied in each recipient country based on consultations among internal and external partners in each country:  One agreed HIV/AIDS action framework that provides the basis for coordinating the work of all partners  One national AIDS coordinating authority, with a broad-based multisectoral mandate  One agreed country-level M&E system. 4.24 The Global Task Team. UNAIDS, the United Kingdom, and the United States co- hosted a high-level meeting in March 2005 — involving leaders from donor and developing- country governments, CSOs, UN agencies, and other multilateral and international institutions — to review the global response to AIDS. Key donors reaffirmed their commitment to the Three Ones and established a Global Task Team on Improving AIDS Coordination among Multilateral Institutions and International Donors to make recommendations to this effect. Composed of representatives of 24 countries and institutions, the Global Task Team reported in June 2005 that the major actors needed to find more 69 effective ways of working together at the country level in line with their respective comparative advantages. Three of its recommendations were of relevance for Global Fund- World Bank engagement at the country level: (a) to create a Global Implementation Support Team, (b) to undertake a study on the comparative advantages for the Global Fund and the World Bank, and (c) to assist countries in preparing AIDS strategies and action plans. 4.25 Global Implementation Support Team (GIST). This team was formed in July 2005, with a secretariat in UNAIDS, to support country partners in making effective use of the increasingly large funds being made available to fight AIDS. High-level officials from multilateral organizations, national AIDS authorities, and others met regularly (initially monthly) to help countries address urgent implementation issues, to stimulate early diagnosis of technical support needs, and to ensure that the deployment of multilateral support was well- coordinated. Following a 2007 review, GIST revised its mandate to focus on strengthening coordination and mutual accountability with respect to technical support, addressing systemic problems at the global level, and identifying good practices and disseminating lessons learned. Its key initiatives under the revised mandate were (a) the development of a set of Principles of Technical Support for users and providers of technical support; and (b) development of CoATS (Coordinating AIDS Technical Support), which is a real-time global-level database to assist the countries in monitoring technical support to facilitate greater accountability and country ownership. In line with these objectives, GIST has commissioned case studies to assess the effectiveness of technical support for Global Fund-related activities. 4.26 Comparative Advantage Study. The Global Task Team had found that the Global Fund and the Bank “increasingly seem to finance the same types of goods and activities in the same countries without any clear sense of their respective comparative advantages or complementarity with each other� (UNAIDS 2005, p. 17). Therefore, the Global Fund and GHAP commissioned a study on the comparative advantages of the Global Fund and the World Bank at the country level. Completed in January 2006, the report recommended, first, that both institutions should make stronger efforts to adhere to the Three Ones principles (along with some concrete suggestions in this regard).71 Second, the report recommended that the Global Fund should give “much greater strategic and operational precision� to its role as a financing entity, and not an implementing agency. This would require enhanced specificity on “what it will not do as well as what it will do.� Third, the report recommended that the World Bank’s strategic and programmatic focus should emphasize — to a much greater extent and with enhanced clarity — that its main comparative advantage lay in systemic health sector capacity building. Strengthening health systems was a difficult and complex area, but it was fundamentally important to achieve progress not just on AIDS, but also on other diseases and, more generally, on the sustainability of all efforts to improve human health in poorer countries. The report pointed out that no other agency had the reach, the expertise, and the experience to provide such support. 4.27 AIDS Strategy and Action Plan Service (ASAP). UNAIDS and the World Bank launched ASAP in July 2006 to assist countries in preparing country-owned strategies and action plans. Billed on its Web site as a service of UNAIDS, the coordinating unit is located 71. Alexander Shakow, 2006, “Global Fund–World Bank HIV/AIDS programs: Comparative Advantage Study,� Report Prepared for the Global Fund and The World Bank HIV/AIDS Program. 70 in GHAP in the World Bank. This provides peer reviews of draft national strategies, offers technical and financial support to assist countries in strengthening their strategic response to HIV/AIDS, develops tools to assist countries in preparing strategies and action plans, and organizes capacity-building activities for policymakers and practitioners. The unit receives the most requests from National AIDS Councils, UNAIDS Country Coordinators, and Regional Support Teams. 4.28 The UNAIDS Second Independent Evaluation, 2002–2008 (Poate, Balogun, and Attawell 2009), generally found that GAMET, GIST, and ASAP were effective initiatives, with some shortcomings (Box 5). There was a high-visibility meeting in 2006, in response to the Comparative Advantage Study, between World Bank President Paul Wolfowitz and Global Fund Executive Director Richard Feachem, as well as discussion of the study at the Bank’s Executive Board. There was also an effort to develop an MOU between the Bank and the Global Fund to lay out a division of labor and ways of collaborating at the country level, but the high-level changes in the leadership of both the World Bank and the Global Fund in 2007 — Robert Zoellick replaced Paul Wolfowitz and Michel Kazatchkine replaced Richard Feachem — hindered attempts to finalize the MOU. The Global Fund was also evolving and expanding rapidly, and the HNP Department was focused on preparing its new HNP Strategy, Healthy Development (World Bank 2007c). 4.29 IEG found that attitudes toward renewing this effort were lukewarm among senior managers of both organizations. Global Fund managers felt that it would not help much unless the Bank actively encouraged its staff to effectively and systematically collaborate with the Global Fund at the country level. There would still need to be an operational framework to execute the MOU, or at least a clear and specific operational understanding of the MOU by staff in both organizations. THE INTERNATIONAL HEALTH PARTNERSHIP AND RELATED INITIATIVES (IHP+) 4.30 The various initiatives associated with the International Health Partnership (referred to as IHP+) have also provided avenues for World Bank-Global Fund engagement at the country level. But none of these initiatives has so far led to a formal agreement between the two organizations on country-level engagement either. 4.31 Compared to GHAP and its related initiatives, IHP+ represents a broader coalition of partners and efforts to accelerate progress in achieving all the health-related MDGs in accordance with the principles of the Paris Declaration and the Accra Agenda for Action. Launched in September 2007, IHP+ is intended to achieve better health results by mobilizing donor countries and other development partners around a single country-led national health strategy, by improving coordination among actors, by strengthening health systems, and by building momentum at the national level for improving existing country-led health plans. IHP+ is open to all developing and developed country governments, and agencies and CSOs involved in improving health who are willing to sign up to the commitments of the IHP+ Global Compact. IHP+ currently counts 47 members. 71 Box 5. Findings from the Second Independent Evaluation of UNAIDS The evaluation commended the work of Global HIV/AIDS Monitoring and Evaluation Support Team (GAMET) in some regions and countries. For example, the UNAIDS Regional Support Team for East and Southern Africa and GAMET have jointly led the development of the 12-component framework on M&E and have developed a regional generic training curriculum on M&E together with other partners. The UNAIDS Secretariat and GAMET have provided pivotal support for strengthening M&E in Swaziland, including building capacity in M&E skills and revising the health sector HIV M&E framework. Overall, however, the evaluation found duplication of M&E work at the country level and weak coordination of M&E roles in HIV/AIDS. “It is not clear how the work of GAMET complements that of the UNAIDS Secretariat, which is also supporting expenditure tracking and M&E capacity building, and there appears to be less collaboration in other regions� (p.118). The evaluation recommended a rationalization of support for M&E between GAMET, the UNAIDS Secretariat and WHO. The evaluation found that the Global Implementation Support Team (GIST) and the UNAIDS global coordinators have improved the coordination of technical support in AIDS. “There is a consensus that the GIST has played an important role in addressing management and implementation bottlenecks at global and country levels relating to Global Fund and World Bank procedures and in providing a link between the UN system and the Global Fund� (p. 112). However, the evaluation questioned the value and sustainability of CoATS (Coordinating AIDS Technical Support) since, like all such databases, this depends on users keeping it up to date (p.112). CoATS had been rolled out in ten countries, as of 2009, initially through UNAIDS Country Coordinators, but with the intention that activities would ultimately be managed by National AIDS Commissions. The evaluation found that the AIDS Strategy and Action Plan Service (ASAP) has been active in over 75 countries and has supported 2 regional initiatives and 3 civil society networks. Along with the regional Technical Support Facilities and WHO Knowledge Hubs, ASAP has helped increase the capacity of UNAIDS to expand technical support to national AIDS responses. ASAP has effectively engaged the UNAIDS Secretariat and five UNAIDS’ cosponsors (UNESCO, UNDP, UNICEF, International Labour Organization, and WHO) in peer review processes and country missions. However, the evaluation found that National Strategic Plans could benefit from stronger analysis of the evidence base, better links between evidence and strategy, a focus on achieving results, more attention to gender and marginalized groups, and improved operational and human resource planning. The evaluation conducted a review of the joint UNDP, World Bank, and UNAIDS Secretariat program to strengthen capacity to integrate HIV into Poverty Reduction Strategy Papers (PRSPs) in 7 of the 14 countries that had so far participated in this program. The review found that the program had enhanced the participation of stakeholders in PRSP formulation, enhanced integration of HIV in PRSPs, increased understanding of the links between poverty and AIDS, and improved alignment of PRSPs and national AIDS strategic plans. The evaluation found that the UNAIDS Secretariat and the program’s seven cosponsors have provided significant technical support for CCMs and Global Fund processes and proposal development both directly and through mechanisms such as the ASAP and the Technical Support Facilities (p.111). The evaluation also found that the Bank had been less actively engaged in joint teams at the country level, even though it was the only cosponsor that contributed financially to UNAIDS as well as being an active cosponsor at the global level. Source: Derek Poate, Paul Balogun and Kathy Attawell for ITAD and HLSP, UNAIDS Second Independent Evaluation, 2002–2008, Final Report, September 2009. 72 4.32 Global and country-level compacts set out a process of mutual responsibility and accountability for the development and implementation of national health plans. Development partners agree to better coordinate external support to help develop and implement comprehensive national health plans; provide aid in ways that strengthen health systems; and, where possible, provide more long-term, flexible support through national systems. Partner countries agree to further invest in their own health systems, address policy constraints to progress, strengthen planning and accountability mechanisms to make them more inclusive and transparent, and better link external support to improvements in health outcomes. CSOs and other stakeholders play an important role in the design, implementation, and review of the IHP at the global and country levels and in holding all parties to account. The performance of all parties is subject to a joint high-level review at the country and global levels. 4.33 IHP+ is not a formal partnership program with a governing body or legally binding agreement between the partners in relation to governance. Its activities are coordinated by an interagency Core Team based in WHO, the World Bank, and WHO-AFRO. The World Bank’s contributions to Phase I of IHP+ were supplemented by a grant from WHO,72 in addition to allocations from the Bank’s administrative budget. The Bank’s contributions to Phase II are being supplemented by a WHO trust fund at the World Bank, called the IHP+ Trust Fund, established in February 2010 to support country-level coordination work. 4.34 Health-8. This is an informal subgroup of IHP+ comprising eight health-related organizations — the Bill & Melinda Gates Foundation, GAVI, the Global Fund, UNICEF, the United Nations Population Fund, UNAIDS, WHO, and the World Bank. Established in July 2007, it meets semi-annually to stimulate a global sense of urgency about reaching the health-related MDGs, to strengthen their own cooperation on global health, and to discuss coordination and aid effectiveness issues in global health. 4.35 High-Level Taskforce on Innovative International Financing for Health Systems. This Taskforce was launched in September 2008 to help strengthen health systems in the 49 poorest countries in the world. Chaired by U.K. Prime Minister Gordon Brown and World Bank President Robert Zoellick, the Taskforce released its recommendations in May 2009 and completed its work in September 2009. The Taskforce identified a menu of innovative financing mechanisms to complement traditional aid flows in health. It launched new initiatives to raise more money, and to use money more effectively, to achieve the health-related MDGs. 4.36 Health Systems Funding Platform. This idea originated with GAVI and the Global Fund, when their Executive Directors addressed the Taskforce to announce a new initiative of joint programming of GAVI and Global Fund resources for HSS, as a way to enhance the capacity of grant-recipient countries to more effectively absorb the significant donor resources being made available. However, the launching of this initiative was not without issues. Some saw this as being done hastily, shortly after the Global Fund had become administratively autonomous from WHO, and without consultation with GAVI’s own Health Systems Strengthening Task Team. The latter pointed out a number of weaknesses, including governance issues and the technical capacity of the two entities to manage joint 72. In World Bank parlance, this was an externally-financed output (EFO) because it was smaller than the minimum amount ($1 million) to establish a Bank-administered trust fund. 73 programming. As a result, WHO and the World Bank agreed to lend their expertise to raise the technical profile of the new initiative, thereby constituting the new Platform Team of GAVI, the Global Fund, WHO, and the World Bank, under the leadership of the Bank.73 4.37 Launched in earnest in early 2010, the Platform is intended, like other IHP+ initiatives, as a mechanism to accelerate progress toward the health-related MDGs, and specifically to “coordinate, mobilize, streamline and channel the flow of existing and new international resources to support national health strategies.�74 It is being developed initially by GAVI, the Global Fund, and the World Bank, and facilitated by WHO in consultation with recipient countries and other key stakeholders, including CSOs. The current partners are coordinating efforts to start harmonizing their activities and aligning them to country priorities and budget cycles. While this work varies with each country context, it seeks to develop one common financial management framework, one M&E framework, and one joint review process in support of one national health strategy.75 The intent is to prepare participating pilot countries to have access to additional funding for HSS. The initial participating countries include those that are already receiving some funding assistance for HSS. 4.38 Following the launching of the new Platform Team, some members of the Global Fund Board expressed concerns that the initiative might shift donor funds away from the three epidemics and toward HSS, where the World Bank had a comparative advantage. The Global Fund had already started funding HSS explicitly by including a specific funding window for HSS during Round 5 grant applications in 2005.76 However, some donors resisted the continuation of this funding window based on the proposed division of labor in the Comparative Advantage Study, which had found the World Bank to have a comparative advantage in providing such support. As a result, the Global Fund removed the HHS funding window in Round 6. Subsequently, however, the Global Fund’s Technical Review Panel recommended to the Global Fund Board that $356 million be allocated for HSS in Round 7 and $594 for Round 8 (Kress and Shaw 2009, p. 9). 4.39 Starting in Round 11 (launched in August 2011), applicants could submit cross-cutting HSS proposals as separate, stand-alone proposals, as for HIV, tuberculosis, and malaria proposals, rather than attaching them to a disease proposal. The Global Fund and GAVI also developed a common HSS proposal form for this purpose. A sub-set of eligible countries could now request support from both the Global Fund and GAVI using the same form (Global Fund 2011a). 73. Dan Kress and R. Paul Shaw, September 2009, “GAVI and Global Fund Joint Programming for Health Strengthening: Turf Wars or an Opportunity to Do Better?� 74. Recommendation 9 of the Taskforce on Innovative Financing for Health Systems, 2009, More Money for Health and More Health for the Money, p. 7. 75. Unlike the Three Ones principle for HIV/AIDS, this alignment effort does not call for one focal point to oversee the national health strategy, potentially resulting in efforts by multiple entities within the Ministry of Health, the Ministry of Planning, and other government entities undertaking policy work to claim ownership over national strategies. 76. Some observers found fault with the Global Fund’s initial approach to HSS because it focused on the provision of human resources by way of salary payments or supplements, procurement of equipment, and other logistical inputs. These items did not necessarily translate into strengthened health systems, although they may have mitigated common shortages in the fight against the three diseases. 74 4.40 Joint Assessment of National Strategies (JANS). This is a shared approach to assessing the strengths and weaknesses of national health strategies, developed by an IHP+ interagency group, and endorsed by IHP+ partners in July 2009. The idea is not new. Renewed interest has arisen from the increased number of international health actors in recent years, and renewed efforts to get more partners to support a single national health strategy/plan. JANS is also becoming a principal precursor to funding national health plans under the Health Systems Funding Platform. A joint assessment helps to strengthen national health strategies and increase partner confidence in those strategies, thereby securing more predictable and better aligned funding. It may also reduce transaction costs arising from multiple separate agency assessments. Five countries — Ethiopia, Ghana, Nepal, Uganda, and Vietnam — had completed formal joint assessments of their new national health sector strategies or plans by early 2011. Other countries are also using the JANS tools more informally at different stages of plan development and implementation. The Bank’s Country Programs in the Health Sector 4.41 The Bank has issued two sector strategies for HNP — in 1997 and 2007. The 1997 Strategy was clear about the Bank’s role in health, citing its comparative advantage as its ability to work across multiple sectors and to conduct country-specific research and analysis in support of programs to which it could bring significant financing. The Strategy did not view the Bank as having a comparative expertise in communicable disease control, epidemiology, and the like in comparison with WHO, UNAIDS, and UNICEF. The Bank would focus on the broader aspects of health such as systems stewardship and oversight, systems performance, and health financing. 4.42 The Bank had become the largest single source of donor financing for HNP by 1997, with a portfolio of 154 active and 94 completed HNP projects, for a total cumulative value of $13.5 billion. The Strategy identified three priority areas: (a) to improve health outcomes for the poor; (b) to enhance performance of HNP services; and (c) to improve health care financing. It viewed investing in communicable disease control in the context of poverty alleviation, since communicable diseases disproportionately affected the poor, and the poorest 20 percent of the population experienced about 60 percent of all deaths from communicable diseases. Many who fell ill and recovered still had lowered productivity, spent high out-of-pocket costs for treatment, and became impoverished. Thus, while HSS was the Bank’s comparative strength, improving health outcomes for the poor also justified support for communicable disease control. 4.43 The 1997 Strategy did not anticipate the amount of lending that the Bank would provide for communicable disease control over the next 10 years. The Bank responded flexibly to the demand for such lending, among other things, with (a) a strategy for intensifying action against HIV/AIDS in Africa in July 1999, (b) the $1 billion Multi-country AIDS Program (MAP) in September 2000, and (c) the Malaria Booster Program in June 2005. (See Appendix M for more details.) In the event, Bank lending for communicable disease control accounted for 36 percent of HNP projects and 32 percent of HNP commitments between 1997 and 2011 inclusive (Table 12). This has provided more opportunities than might otherwise have been available for engaging with the Global Fund at the country level. 75 Table 12. World Bank Communicable Disease Projects and Commitments, Fiscal Years 1997–2011 Approved Projects Commitments Project Type Number Share US$ millions Share Freestanding communicable disease projects 112 74% 6,580 90% Single disease projects 97 64% 4,989 69% HIV/AIDS 70 46% 2,735 38% Tuberculosis 3 2% 374 5% Malaria 5 3% 547 8% Avian influenza 7 5% 65 1% (H1N1) Influenza 5 3% 723 10% Cholera 1 1% 15 0% Leprosy 1 1% 32 0% Polio 4 3% 474 7% Schistosomiasis 1 1% 25 0% Multiple disease projects 15 10% 1,591 22% Projects with a communicable disease component 40 26% 696 10% Total number of communicable disease projects 152 100% 7,277 100% Total number of HNP projects 423 22,729 Share of HNP projects 36% 32% Source: For FY1997–2006, Gayle H. Martin, 2010, “Portfolio Review of World Bank Lending for Communicable Disease Control,� IEG Working Paper 2010/3. Updated by IEG through FY2011 from World Bank databases. Note: The full project commitments are included for freestanding communicable disease projects, and only the commitments to the communicable disease component for projects with components. Therefore, these commitments are somewhat larger than those in Table 3 in Chapter 2. 4.44 The 1997 Strategy also did not anticipate the growth of SWAp operations in the Bank’s portfolio. Introduced by the World Bank and other donors as a means to overcome inefficiencies, reduce transactions costs to the country, and bring better development results, SWAps embraced the principles of alignment and harmonization that were subsequently endorsed by the Paris Declaration in 2005. Health SWAps represented (a) higher and more committed levels of donor support and coordination to a country’s overall development program in the health sector; and (b) a shift in the relationship between donors and governments, with all parties jointly supporting nationally defined health programs through parallel or pooled financing of general budget support, or a combination of the two. The World Bank approved 45 HNP projects supporting health SWAps in 32 countries between 1997 and 2011 — representing about 11 percent of all (423) approved HNP projects during this period.77 Almost 60 percent (26) of the projects that supported health SWAps were in Sub-Saharan Africa. 77. Denise Vaillancourt, “Do Health Sector-Wide Approaches Achieve Results? Emerging Evidence and Lessons from Six Countries,� IEG Working Paper 2009/4, and Appendix M. 76 4.45 The Bank’s 2007 HNP Strategy acknowledged that the global HNP aid architecture had changed significantly since 1997, with many new players entering the field, such as GAVI, the Global Fund, and several foundations, bringing with them innovative financing mechanisms, mostly earmarked for specific diseases or issues. The Bank was no longer the largest financier of investments in the HNP sector, as it had been 10 years earlier. 4.46 The 2007 Strategy reaffirmed the Bank’s comparative advantages in the following areas: (a) its capacity in HSS (including health financing, insurance, demand-side interventions, regulation, and systemic arrangements for fiduciary and financial management); (b) its intersectoral approach to country assistance; (c) its advice to governments on regulatory frameworks for private-public collaboration in the health sector; (d) its capacity for large-scale implementation of projects and programs; (e) its convening capacity and global nature; and (f) its pervasive country focus and presence (World Bank 2007c, pp. 17–18). 4.47 The 2007 Strategy underscored a focus on results: that is, in health outcomes in addition to operational modalities. It reiterated the contribution of multisectoral approaches and interventions to improve health outcomes, such as safe drinking water and household sanitation, among other health infrastructure investments. It did not see a contradiction between Bank support for health systems and support for the control of priority diseases. Bank investments were seen as necessary to ensure synergies between health system and single-disease approaches, especially in low-income countries where fighting communicable diseases was still a priority. The Strategy also recognized the growing need to support interventions against non-communicable diseases. 4.48 The 2007 Strategy found that the HNP partnership portfolio had become fragmented with a multiplicity of GRPPs and needed “stronger strategic direction.� The Strategy stated that the HNP sector would practice greater selectivity when deciding to participate in partnership programs: (a) to complement Bank work in areas in which it has no comparative advantages or to complement other partners needing Bank expertise, all in direct benefit of client countries; and (b) to contribute to the international community’s support for global public goods and prevention of global public “bads.� The Strategy also proposed the establishment of a Global Health Coordination and Partnership Team in the HNP Department to coordinate partnerships, and to facilitate selective fund-raising and trust fund management, DGF management support, selective joint ventures around comparative advantages, and harmonization. This team has not been established, but a position of Partnerships Adviser has been created in the HNP Department. 4.49 In summary, Box 6 highlights key recommendations and World Bank commitments relating to country-level engagement with the Global Fund since 2006. The 2007 Strategy repeatedly stated that the World Bank would strengthen its engagement with the Global Fund, particularly in low-income countries. However, it did not articulate how this engagement would take place, except that it would reach “specific agreements with WHO and the Global Fund on a collaborative division of labor at the country level (next 12 months).� 77 Box 6. Key Recommendations and World Bank Commitments Relating to Engagement with the Global Fund “Global Fund–World Bank HIV/AIDS Programs: Comparative Advantage Study� (Shakow, January 2006)  Both institutions should make stronger efforts to adhere to the “Three Ones� principles.  The strategic and programmatic focus of the World Bank should emphasize to a much greater extent and with enhanced clarity that its main comparative advantage lies in systemic health sector capacity building. Healthy Development: The World Bank Strategy for Health, Nutrition, and Population Results (World Bank, April 2007)  The Bank has played a crucial role in advocacy, awareness, and development of new international initiatives and organizations such as the Global Fund and GAVI. The Bank will reach “specific agreements with WHO and the Global Fund on a collaborative division of labor at the country level (next 12 months).� Management Response to Improving Effectiveness and Outcomes for the Poor in Health, Nutrition, and Population: An Evaluation of World Bank Group Support since 1997 (IEG, January 2009)  The World Bank Group uses a range of engagement instruments, such as . . . working through international networks and partnerships, such as the GAVI, the Global Fund, and the European Union Observatory. Working with partners through pooled funding, country systems and joint strategies and supervision (as opposed to ring-fenced Bank operations) is also anchored in international commitments and agreements such as the Paris and Accra Declarations. The success of UNAIDS, the Global Fund, GAVI, Roll Back Malaria, EU Observatory and other major international partnerships is also the shared success of the Bank Group’s HNP work, as we exercise substantial technical and financial influence in these networks and partnerships. More Money for Health, and More Health for the Money (Taskforce on Innovative International Financing for Health Systems, September 2009)  Make the allocation of existing and additional funds in countries more efficient, by filling gaps in costed and agreed national health strategies.  The Taskforce requests OECD/DAC with partners should undertake a review of all current technical assistance, with a view to focusing it on strengthening national and local institutional capacity in priority areas such as public administration and accountability, financing, service delivery arrangements and the non-state sectors.  Establish a health systems funding platform for the Global Fund, GAVI Alliance, the World Bank and others to coordinate, mobilize, streamline and channel the flow of existing and new international resources to support national health strategies. The Extent and Nature of Bank’s Engagement with the Global Fund at the Country Level 4.50 There is no systematic record of the Bank’s engagement with the Global Fund at the country level. Therefore, IEG has pieced together this record from Bank databases, word searches 78 and reviews of World Bank Country Assistance Strategies and Project Appraisal Documents, key informant interviews, and the electronic survey of health sector project managers at the World Bank and Global Fund staff in Geneva, administered in March 2011 (Appendix Q).78 4.51 The findings of the survey are indicative rather than determinative. They are more representative of the experience in Africa, East Asia, and South Asia, where survey coverage was better than for other Bank Regions. About one-quarter of the survey respondents from both the World Bank and the Global Fund indicated that they viewed the relationship between the two organizations in the countries in which they were working as “unrelated and independent� (Figure 9). About three-quarters indicated some degree of engagement, ranging Figure 9. Overall, how would you best characterize the relationship between the World Bank and the Global Fund during the years that you were working on this country? Collaborative: The two organizations’ staff, consul-tants and agents worked together on common activ-ities in the pursuit of commonly agreed objectives. Complementary: The two organizations’ staff, consultants, and agents worked alongside each other in the pursuit of common objectives. Consultative: The two organizations’ staff, consultants, and agents consulted each other regularly in the course of their own activities. Sharing information only: The two organizations’ staff, consultants, and agents only shared information about each other’s activities. Unrelated and independent: The two organizations worked independently of each other supporting different health initiatives in the country. Competitive: The two organizations competed for business among the same potential clients. 0% 10% 20% 30% 40% GF � Country Programs Staff WB � Task Team Leaders Source: IEG Survey of World Bank HNP project managers and Global Fund Secretariat staff, administered in March 2011. Note: Each respondent was limited to only one choice; therefore, the responses from each organization add up to 100 percent. The survey response rates were 62 percent (36 out of 58) for Global Fund Country Programs staff and 33 percent (42 of 128) for World Bank task team leaders (project managers). 78. The 42 project managers who responded to the survey covered 37 separate countries (since five countries had two responses), which represented 47 percent of World Bank HNP commitments during FY03–11 inclusive, and 54 percent of disbursements during the same period. The 36 responses from the Global Fund Country Programs Cluster covered 42 countries (since some staff covered more than one country), which represented 48 percent of Global Fund commitments during FY03–10 inclusive, and 47 percent of Global Fund disbursements during the same period. Overall, the responses from the 42 project managers and from the 36 Country Programs staff covered 64 different countries. 79 from “information sharing only� to “collaborative.� Global Fund staff tended to view the relationship as more collaborative and consultative, and World Bank project managers as more complementary and sharing of information. If one were to extrapolate these findings to the 90 countries in which both the Bank and the Global Fund have been active in the health sector since 2002, this would translate into some degree of engagement in about 65 countries overall, of which 25–30 have been in Africa, the Region most seriously affected by the three diseases. 4.52 This extent of engagement is consistent with the number of countries (63) in which the Bank’s Country Assistance Strategies and Project Appraisal Documents over fiscal years 2003–10 inclusive make reference to the Global Fund (Table 13), not including additional countries involved in regional projects in Africa, Central Asia, and Latin America. By way of comparison, this represents about 60 percent more than such references to the Education For All–Fast Track Initiative, a global partnership program started in the same year as the Global Fund (2002) that is also financing investments at the country level, but which has been located in the World Bank and for which the World Bank is an implementing agency. 4.53 The variation in the degree of engagement from “independent and unrelated� to “collaborative� is also consistent with the findings from the seven countries that IEG visited in 2010 for this Review, including an IEG visit to Lesotho to prepare a Project Performance Assessment Report of two health projects. IEG found that the two organizations worked independently of each other in supporting different health initiatives in two countries (Brazil and Nepal), that they collaborated on disease-control projects in three countries (Cambodia, Lesotho, and the Russian Federation), and that they cooperated to a lesser extent into two countries (Burkina Faso and Tanzania). Table 13. References to the Global Fund and the Fast Track Initiative in Country Assistance Strategies and Project Appraisal Documents, Fiscal Years 2003–10 Latin Middle Sub- East Europe America East and South Saharan and the and Central Total and the North Asia Africa Pacific Asia Caribbean Africa Global Fund Country Assistance Strategies 23 2 13 6 – 1 45 Project Appraisal Documents 62 8 11 14 – 6 101 Number of Different Countries a 31 6 13 8 – 5 63 Education for All – Fast Track Initiative b Country Assistance Strategies 18 2 7 3 2 – 32 Project Appraisal Documents 18 3 3 6 4 4 38 Number of Different Countries 22 4 5 3 2 4 40 Source: IEG data a. These do not include the countries involved in six regional projects in Africa, one regional project in Central Asia, and two regional projects in Latin America (in the Andes and Central America). b. The Fast Track Initiative changed its name to the Global Partnership for Education in 2011. 80 INDEPENDENT AND UNRELATED ACTIVITIES: BRAZIL AND NEPAL 4.54 The World Bank has been active in the health sector in Brazil since 1976, and has supported the government’s fight against HIV/AIDS with five projects since 1988 (approved in 1988, 1993, 1998, 2003, and 2010), as well as one malaria project in the Amazon basin (approved in 1989). The Global Fund has approved two grants for tuberculosis (in 2007) and two for malaria (in 2009). Thus, the two organizations have been supporting different disease-control efforts in recent years. While the CCM has submitted grant proposals for HIV/AIDS prepared by the Bank’s counterpart in the Ministry of Health (the HIV/AIDS Department), the Global Fund’s Technical Review Panel has not yet recommended funding any of these. Neither the government nor the CCM has requested support from the World Bank in relation to Global Fund-supported activities in Brazil, but this could change if an HIV/AIDS proposal were successful. The Bank has not been a member of the CCM, because the Brazil CCM is dominated by government and CSOs, with very little representation from any multilateral or bilateral development partners in the country. 4.55 The World Bank has been active in the health sector in Nepal since 1994. It approved a first Health Sector Program project (a SWAp) in 2004 in which IDA, DFID, and later the Australian Agency for International Development pooled their financial support for the Government’s health program. The project contained a component to strengthen health service delivery, which included a subcomponent on communicable disease control (HIV/AIDS, tuberculosis, malaria and leprosy). However, this did not result in any significant engagement with the Global Fund-supported activities in the country that were being implemented under the oversight of the CCM. 4.56 As the first Health Sector Program project was closing, the Bank started to identify a specific HIV/AIDS project in 2009, which would have been the first single-disease Bank- supported project in the country. Subsequently, the Bank and the Government of Nepal decided not to pursue this, opting instead for a second HNP and HIV/AIDS project, which was approved in 2010. This second SWAp operation, as its name implies, includes a significant AIDS component in addition to a range of HNP activities. So the Bank is now more significantly involved in one of the three Global Fund diseases, thereby opening the door for greater collaboration with the Global Fund in the future. 4.57 Nepal is currently a pilot country for both JANS and the Health Systems Funding Platform. A joint assessment of the national health strategy was carried out in January 2010, and a Joint Financing Agreement supporting the National Health Support Program, 2011–15, was signed by the government and the major donors in August 2010 (DFID, GAVI, UNFPA, UNICEF, USAID, and the World Bank). Funding for NGOs that cater to most at-risk groups is now transitioning from DFID/UNDP funding to pooled funding, managed by the World Bank. While the Global Fund is not a party to this pooling arrangement, it has become the exclusive supplier of ARVs and related health products into Nepal (except for USAID’s providing PMTCT drugs on a small scale for its own projects). The Global Fund is also supporting NGOs and Principal Recipients that are delivering prevention, treatment, and care to people living with AIDS and to people in high-risk groups. 81 COLLABORATIVE ENGAGEMENT: CAMBODIA, LESOTHO, AND THE RUSSIAN FEDERATION 4.58 The World Bank has supported three health projects in Cambodia. The first (approved in 1997) had one component for combating HIV/AIDS, tuberculosis, and malaria ($13 million) and a second component for strengthening health systems ($18 million). The second and third projects (approved in 2004 and 2008) have supported the operationalization of the government’s Health Strategy Plans I and II, respectively. Although these projects have not been specific to the Global Fund diseases, the Bank has developed a collaborative relationship with the Global Fund in Cambodia as a member of the CCM from inception in 2002 until it was restructured in 2010. The Bank has contributed to improving the quality of grant proposals and served on the Cambodian technical review panel on occasion. The Bank’s analytical work on the role of health in the country’s overall development has helped to anchor health issues in policy dialogue at the macroeconomic level, and to facilitate access of the FPM to key government officials in the Ministry of Finance. 4.59 Although the Global Fund does not generally pool funds, the FPM has been participating in joint annual performance reviews and annual operating plans of the Health Sector Support Program. The Global Fund has also endeavored to align its work in Cambodia with other donors under the IHP+. A first joint country mission (including GAVI, the Global Fund, WHO, and the World Bank) took place in early June 2010. As a result of these discussions, all three funding agencies agreed to align their performance indicators with those of the government, and work with the Department of Planning and Health Information in the Ministry of Health to strengthen the M&E system. 4.60 The Bank has also been pursuing reforms in financial management and administration for the overall Cambodian civil service, including a merit-based performance initiative (MBPI).79 This would have aligned the different ad hoc payment practices of donors and developed a performance culture within the civil service. As a result of these initiatives, and the subsequent emergence of the Priority Operating Costs scheme,80 the FPM also interacted frequently with Bank’s Country Manger and project managers in public and financial administration. World Bank-Global Fund collaboration in Cambodia led to the Global Fund’s agreement to significantly reduce its salary top-ups in alignment with these schemes. 4.61 The World Bank has been active in the health sector in Lesotho since 1985. The Bank approved a Health Sector Reform Project with an HIV/AIDS component (for $2 million) in June 2000, and started to work with the authorities to prepare a follow-on MAP project in 2003. In the interim, the Lesotho CCM submitted a successful Round 2 proposal to the Global Fund in 2002 for $34 million ($29 million for HIV/AIDS and $5 79. Donors have been paying incentive money to civil servants because the average wage is below the subsistence level ($100 /month). Global Fund grants had allowed very high salary top-ups for some senior posts. The MBPI initiative attempted to align the different ad hoc payment practices of donors, and to apply only to mission-critical categories of staff. The MBPI would only be paid if performance standards were met, and was therefore intended to inculcate a performance culture. 80. The government replaced MBPI with a new incentive scheme, the Priority Operating Costs (POC), which are applied to public functions that are considered critical, and which are donor-financed only. The POC is not performance-based, but will harmonize rates across ministries and categories of staff. 82 million for tuberculosis). The Principal Recipient was the Ministry of Finance and Development Planning, and the two Sub-Recipients were the Ministry of Health and Social Welfare and the Lesotho AIDS Program Coordinating Authority. 4.62 Given the size of the Global Fund grant, the Bank’s long-standing concerns about the country’s existing implementation capacity, and the emerging risk that the Global Fund might cancel the grant after two years due to implementation problems, the Bank changed course. Bank and government personnel quickly prepared an HIV and AIDS Capacity Building and Technical Assistance project with the specific objective of increasing Lesotho’s capacity “to use effectively the resources provided through the Global Fund grant.� This resulted in a close collaboration between the Global Fund and the World Bank during the implementation of the Global Fund grant and the World Bank project, in which the Bank’s project manager was essentially supervising both projects through completion. This collaboration has continued with the approval of a follow-on Bank-financed HIV and AIDS Technical Assistance Project approved in August 2009. 4.63 IEG has rated the outcome of the Bank-supported HIV and AIDS Capacity Building and Technical Assistance project as moderately satisfactory in terms of increasing Lesotho’s capacity to effectively use Global Fund resources, based on an in-depth review of the project.81 The project greatly improved the capacities of the Ministries of Health and Finance and the National AIDS Commission to manage and disburse Global Fund resources, and of the National Drug Supply Organization to procure and distribute drugs, thereby promising to improve the efficacy of treatment programs. However, few project resources were used to strengthen the technical capacity of CSOs to provide interventions, beyond the largest umbrella NGOs. Key positions in the Ministry of Health and Social Work for improving the technical capacity and effective use of funds were not filled due to high staff turnover. The vacant staff positions, particularly in the behavior change communications unit, hindered the formulation and implementation of an effective HIV prevention campaign. In the absence of the Bank’s intervention, the Global Fund’s Round 2 grant would likely have been cancelled. The Bank-supported project has also increased Lesotho’s capacity to mobilize additional resources for the national HIV/AIDS program, as exemplified by additional Global Fund grants (in Rounds 5, 6, 7, and 8), but its capacity to use these funds effectively to prevent HIV and mitigate its impact remains weak. 4.64 The World Bank has been active in the health sector in the Russian Federation since 1996. Following a request from the government in early 1999, the Bank initiated work on the development of a tuberculosis project with the Ministry of Health and the Ministry of the Interior, to which HIV/AIDS was later added. Finally approved in April 2003, the Bank provided an IBRD loan of $150 million toward the total project cost of $286 million. Around the same time, a consortium of five NGOs already active in the Russian Federation and led by the Open Health Institute submitted two Round 3 proposals to the Global Fund for HIV/AIDS and tuberculosis that were approved. Both the World Bank project and the Global Fund grants were implemented by the same agency, the Russian Health Care Foundation. By agreement among all parties, the two projects settled on an effective division of labor. The 81. IEG 2010, Project Performance Assessment Report: Lesotho Health Sector Reform Project and HIV and AIDS Capacity Building and Technical Assistance Project, Report No. 55417. 83 Bank project became responsible for the support of physical facilities, including laboratories, second-line tuberculosis drugs, and professional training. The Global Fund grants financed activities not financed under the Bank project, including second-line tuberculosis drugs and ARVs, together with, to a small extent, support for some equipment. The investment in laboratory infrastructure for HIV and other sexually transmitted diseases throughout the Russian Federation contributed to treatment with ARVs funded by the Global Fund and the Russian Federation national program. The World Bank project manager and the Global Fund Portfolio Manager developed a close collaboration, and the Bank had a seat on the CCM during the life of the projects. 4.65 IEG has rated the outcome of the Bank-supported Tuberculosis and AIDS Control project as satisfactory, but the risks to sustaining the benefits achieved by the project as significant. The project substantially achieved all four of its development objectives, but political, financial, and institutional risks remain high as the Russian Federation transitions away from international support for HIV/AIDS and tuberculosis programs. It is unclear that there is government commitment to addressing high-risk groups or to applying international best practices on harm reduction. INTERMEDIATE DEGREES OF ENGAGEMENT: BURKINA FASO AND TANZANIA 4.66 The World Bank has been active in the health sector in Burkina Faso since 1985. The Bank approved a first-generation MAP project in July 2001 and a second MAP component in a health sector support project in April 2006. A Regional HIV/AIDS Treatment Acceleration Project was also implemented in 2003–08 in Burkina Faso, Ghana, and Mozambique to test different approaches for scaling up existing treatment initiatives. This was the first project to finance ARV therapy in Burkina Faso. Its success helped to secure the large-scale Global Fund support for ARV therapy that came later by giving the Global Fund the confidence to support this endeavor. As the Global Fund has expanded its support to Burkina Faso — it was financing all ARV therapy in the country at the time of IEG’s visit — the Bank has moved toward providing complementary support to the Burkinabe health sector, mainly in the form of HSS. The respective roles of the two organizations have been self-selected, and not the result of an explicit agreement or understanding between the two organizations. As a member of the CCM, the Bank has provided technical assistance during the preparation of grant proposals in the form of staff time and the hiring of consultants. The Bank’s project manager has been more active in the CCM when resident in the country. 4.67 The World Bank has been active in the health sector in Tanzania since 1990 and approved a MAP operation in 2003. This project helped to institutionalize the National AIDS Commission, which hosts the CCM Secretariat, and to build up the capacity of the Tanzanian CCM. As the volume of Global Fund grants for all three diseases grew and the Tanzanian CCM and National AIDS Commission Secretariat took hold, the Bank became less engaged with Global Fund-supported activities. The Bank’s attendance at the CCM meetings became less frequent. Subsequent Bank operations (notably the Second Health Sector Development Scale-Up, approved 2007) deliberately addressed areas not covered by Global Fund grants. The Bank is well informed on both health systems and communicable diseases in Tanzania through its small team of health specialists in the country, supported by HNP specialists in Washington. However, the Bank has essentially drifted away from Global Fund-supported 84 activities; its remaining involvement has taken the form of active participation in the health sector and HIV/AIDS donor groups. What started as active collaboration with Global Fund- supported activities evolved into consultation and information-sharing in relation to what are now essentially independent activities. BROADER PATTERNS BEYOND THE SEVEN COUNTRIES VISITED 4.68 Interviews with Bank staff and a focus group discussion with project managers to discuss IEG’s survey results indicated that World Bank-Global Fund engagement has generally started with a formal or informal request from the government of the country. The government — as the chair or an influential member of the CCM — has often requested the Bank’s technical support for preparing grant proposals to the Global Fund. This was particularly the case during the earlier Global Fund rounds and for HIV/AIDS proposals in countries in which the Bank was supporting a MAP project. Recognizing that the Bank’s Figure 10. In which of the following country-level processes of the Global Fund did World Bank staff or consultants participate during the years that you were working on this country? (Percent “Yes�) Member of the Country Coordinating Mechanism Providing formal technical assistance to the Principal Recipients of Global Fund grants Helping to prepare grant proposals for submission to the Global Fund Helping with the oversight/supervision of Global Fund-financed activities Helping to select Principal Recipients to implement approved Global Fund grants Helping with financial management/procurement of Global Fund- financed activities Participating in the selection of grant proposals for submission from the CCM to the Global Fund 0% 10% 20% 30% 40% GF � Country Programs Staff WB � Task  Team Leaders Source: IEG Survey of World Bank HNP project managers and Global Fund Secretariat staff, administered in March 2011. 85 overarching mission is to contribute to the development of its client countries and their institutions, Bank staff have generally responded positively to the extent of their available time and resources. Bank staff have also become involved in Global Fund-supported activities through their participation in health sector donor coordination processes in the country, through participation in joint World Bank-Global Fund workshops, and through the direct request of Global Fund Regional Team Leaders and FPMs. World Bank Sector Managers have also encouraged engagement in some cases. 4.69 Bank staff and consultants have not generally been involved in specific Global Fund processes at the country level (Figure 10). They have been members of the CCM in at most one-third of the 64 countries in which survey respondents worked, helped to prepare grant proposals in 30 percent, and provided formal technical assistance to the Principal Recipients in 25–30 percent of countries (60 percent of which have been government agencies).82 4.70 Bank staff and consultants have more frequently contributed to other country-level activities, such as strategic and analytical work, that directly or indirectly contributed to the work of the Global Fund (Figure 11). Contacts between World Bank and Global Fund staff have occurred primarily at the country level. World Bank project managers have had their most regular contacts with the Principal Recipients of Global Fund grants and with the CCMs, and more occasional contact with FPMs and Regional Team Leaders based in Geneva (Appendix Q). Figure 11. In what other ways were World Bank staff or consultants involved in country-level activities that directly or indirectly contributed to the work of the Global Fund during the years that you were working on this country? (Percent “Yes�) Supporting analytical work in relation to strengthening health systems Helping to prepare country strategies such as an AIDS Strategy and Action Plan (ASAP) for combating HIV/AIDS, TB or malaria Supporting analytical work in relation to combating HIV/AIDS, TB, or malaria Helping to build human resource capacity to prepare and implement Global Fund grants in the country 0% 20% 40% 60% 80% 100% GF � Country Programs Staff WB � Task  Team Leaders Source: IEG Survey of World Bank HNP project managers and Global Fund Secretariat staff, administered in March 2011. 82. The World Bank is currently (October 2011) serving on 16 CCMs — in 8 countries in Africa, 5 countries in Europe and Central Asia, and 3 countries in South Asia. 86 4.71 Uniformly, in response to every question in Figures 10 and 11, Global Fund respondents felt that World Bank staff and consultants participated more in Global Fund processes and contributed more to the work of the Global Fund than Bank project managers felt. Global Fund staff also viewed the World Bank as a Global Fund partner to a much greater extent, at both the global and the country levels: 76 percent of Global Fund staff viewed the Bank as a high or substantial Global Fund partner at the global level, and 46 percent at the country level, compared with 31 percent and 26 percent, respectively, of World Bank project managers (Figure 12). Figure 12. To what extent do you consider the World Bank to be a partner of the Global Fund (a) at the global level and (b) at the country level? Global Fund – All Clusters: (a) At the Global Level: (b) At the Country Level: World Bank – Project Managers: (a) At the Global Level: (b) At the Country Level 0% 20% 40% 60% 80% 100% High Substantial Modest Negligible Source: IEG Survey of World Bank HNP project managers and Global Fund Secretariat staff, administered in March 2011. Note: The survey response rates were 49 percent (52 out of 106) for Global Fund staff and 33 percent (42 of 128) for World Bank project managers (task team leaders). 4.72 World Bank project managers in the focus group suggested two reasons for this pattern. First, participating in Global Fund processes and contributing to the work of the Global Fund generally represents a small part of a project manager’s work; his or her primary relationship is with the government and the implementing agency of the Bank project. Second, project managers may be less aware of the contributions that the Bank has made to the work of the Global Fund in the country if these contributions have been mediated by third parties such as the government. 4.73 The relationship between the Bank and the Global Fund has also been dynamic in many countries, such as Burkina Faso and Tanzania, due to number of factors such as a change in the Bank’s work program or a change of staff on either side. Whether survey respondents characterized their engagement with the other organization as collaboration, consultation, or information sharing only, successful engagement has had similar characteristics. The engagement often takes place in a broader setting or context where there are other interested partners and stakeholders involved as well. Key factors contributing to positive engagement have been a proactive government and a strong donor coordination mechanism at the country level. The personal commitment of the World Bank’s project managers and Global Fund’s Portfolio Managers have also played a role in sustaining successful cooperation, as in Lesotho 87 during the implementation of the HIV and AIDS Capacity Building and Technical Assistance Project from 2004 to 2008, and in the Russian Federation during the implementation of the Tuberculosis and AIDS Control Project from 2003 to 2009. 4.74 Based on interviews, the survey results, and document reviews, World Bank staff and consultants appear to have been most engaged with Global Fund processes at the grant preparation stage rather than at the grant implementation stage, as the following examples illustrate. The two organizations have been least engaged at the strategic level, apparently because Global Fund staff and agents have been less involved with the government in formulating health sector strategies. 4.75 Grant Preparation Stage. The engagement of the World Bank with Global Fund- supported activities has often taken place through Bank staff assisting in the preparation of the Global Fund grants; establishing joint funding arrangements at the country level (as in Benin, Ethiopia, and Honduras); and working with the same Project Implementation Units (as in Djibouti and Uganda).  In Benin, the Bank has been represented in the CCM, which has facilitated regular sharing of information and avoided duplication of activities. The two organizations’ HIV/AIDS and malaria projects have complemented each other by supporting different activities in different areas and during different time periods. More recently, GAVI, the Global Fund, and the World Bank have established a joint funding platform for HSS.  In Djibouti (as in the Russian Federation), the Global Fund selected the implementing agency of the World Bank’s HIV/AIDS project to be the recipient of the Global Fund grants, due to the existing capacity of the agency. This created the opportunity for harmonization of procurement procedures.  In Ethiopia, the Global Fund has built upon the achievements of two Bank-supported MAP projects (approved in 2000 and 2007) in the areas of HIV/AIDS and HSS. More recently, the Bank and the Global Fund are coordinating their support through the establishment of a joint funding platform for HSS.  In Uganda (as in Lesotho), the Bank decided not to proceed with a second MAP project after the first one closed in 2006 because the country was receiving large grants from the Global Fund. The Bank collaborated closely with the Global Fund on its exit strategy. Together, World Bank and Global Fund staff supported a review of the complementarity and sustainability of HIV interventions to ensure that the Global Fund would continue to fund some project components after the Bank project closed. That the same Principal Recipient was implementing both the Bank project and the Global Fund grant facilitated this collaboration.  In India, complementary activities supported by the organizations have helped to build NGO capacity in the country. The Bank helped to build the managerial and fiduciary capacity of a local NGO network, which enabled them to qualify as the Sub- Recipient of a Global Fund grant.  In Central Asia, the Bank-supported Central Asia AIDS Control regional project 88 (approved in 2005) has helped the Central Asian countries to prepare Global Fund grant proposals at the request of the countries’ CCMs.  In Central America, the Bank-supported Central American Integration System for the Regional HIV/AIDS project (approved in 2005) provided the avenue for cooperation. This project aimed at supporting key HIV/AIDS activities that were best addressed regionally and not covered by Global Fund grants, such as regional efforts to develop a regional HIV/AIDS laboratory, to support coordinated surveillance, to systematically share best practices in prevention, and to help prevent HIV in mobile populations. The regional project helped the regional Central American CCM to prepare Global Fund grant proposals, provided technical assistance to CSOs to become eligible as Principal Recipients or Sub-Recipients, and helped design a comprehensive HIV/AIDS program in Honduras to be financed by the Global Fund. However, the Bank and the Global Fund were not able to reach an agreement to ensure the sustainability of the Regional Laboratory by means of an endowment fund to finance the laboratory. 4.76 Grant Implementation Stage. World Bank engagement with the implementation of Global Fund grants has often taken place through joint monitoring missions to supervise the projects and harmonize approaches on the ground (as in Bangladesh, Benin, Malawi, and the Maldives). In other cases, the World Bank has provided technical assistance to Principal Recipients of the Global Fund to build local capacity for implementing Global Fund grants.  In Côte d’Ivoire, the Bank is an observer on the CCM and there has been little communication between the two organizations. However, the ministry in charge of HIV/AIDS requested that a Bank-financed project provide continuing support for NGOs that had previously been supported by a Global Fund grant.  In Guinea-Bissau, the Bank has been sharing information at the end of each mission. A World Bank assessment of the National AIDS Secretariat (NAS) in 2008 served as the basis for the government and the Global Fund to restructure the Secretariat, resulting in a leaner and more operational institution, able to implement the Global Fund grant in a satisfactory manner.  In Bangladesh, neither the Bank nor any other donor is a member of the CCM and the Global Fund is not contributing to the Health SWAp. Nonetheless, joint monitoring missions have made efforts to coordinate approaches and to avoid duplicating interventions to reach high-risk groups for AIDS. The Bank and the Global Fund have also carried out a joint review of the portfolio of HIV projects in collaboration with other partners, focusing on government performance and aiming at shared learning.  In St. Vincent and the Grenadines, the government has played a critical role in achieving complementarity and avoiding duplication between Bank and Global Fund- supported activities. The Global Fund grant has provided resources to finance drugs, and the World Bank project has provided other complementary inputs and activities. 4.77 Strategic Stage. World Bank-Global Fund engagement at the strategic stage has often taken place around the ASAP, JANS, or other national frameworks. 89  In Burundi, the Bank was involved in the preparation of the National HIV/AIDS strategy with support from ASAP, and in the design of the national HIV/AIDS M&E system with support from GAMET. There has been a constant policy dialogue since 2005 on how best to use HIV/AIDS resources and on how to set up a steering body for ARV procurement and monitoring. The Bank and the Global Fund have also carried out joint missions. The Bank has provided comments on Project Update and Disbursement Requests. The Bank agreed that funds from the second MAP project (approved 2008) could be used to cover some of the financing gap in HIV/AIDS.  In the Maldives, the Bank collaborated with the CCM, the Principal Recipient (National AIDS Program), UNAIDS, and the United Nations Office on Drugs and Crime in conducting a comprehensive review of the National Strategy Paper (2009). The Bank also helped with the first mapping of high risk groups in HIV — “The Research Proposal on Mapping High Risk Groups for HIV Prevention in the Maldives� — in close collaboration with the University of Manitoba.  In Guyana, both the Bank and the Global Fund supported the preparation of the new National HIV/AIDS Strategic Plan (2007–11), the implementation of which has been supported by the IDA-financed Guyana AIDS Prevention and Control Project, the Global Fund, the Canadian International Development Agency, PEPFAR, and other bilateral and UN agencies. At the request of the Ministry of Health, the Global Fund is continuing to fund some of the activities previously supported by the World Bank project. Acting flexibly, the Global Fund agreed that some of its support could be reprogrammed to continue funding NGOs previously supported by the World Bank project. Factors Facilitating and Hindering Effective Engagement 4.78 The World Bank follows a different business model from the Global Fund (Appendix M). The similarities and the differences provide both opportunities and hindrances to effective country-level engagement in support of their clients. 4.79 The Bank’s operational involvement in each country is based on periodic Country Assistance Strategies (CASs), negotiated between the Bank and the government. Each sector has to compete for its place in the CAS in accordance with the agreements reached on the priority sectors for Bank support to the country. The CAS lays out a set of activities that the Bank will support over the next 3–4 years, comprising both analytical and advisory work and lending operations. Lending operations are almost always implemented by a government department or agency, although governments may enlist NGOs and CSOs to help implement the project — and generally do so in the case of HIV/AIDS projects. Each lending product has a project manager who is responsible for preparing the project from the point of view of the Bank, and for supervising the subsequent implementation of the project with the support of his/her task team. The majority of Bank project managers are now based in the field, particularly in East Asia and South Asia (Appendix Table M-1). Where the project manager is not based in the country, project supervision involves multiple missions over the 5–7 year life of a project, with the assistance of a range of specialized consultants. 90 Figure 13. World Bank Project Managers: In your opinion, do the following factors make it easier or more difficult for World Bank staff or consultants to engage with Global Fund- supported activities at the country level? The presence of other mechanisms through which the World Bank and the Global Fund may interact, such ASAPs, JANS, and the Joint Funding Platform. The focus of the Global Fund on low-income countries (similar to IDA-eligible countries). The presence of civil society organizations on the Country Coordinating Mechanism. The fact that the Principal Recipient for Global Fund grants is not restricted to government agencies. The fact that the Local Fund Agent is responsible for overseeing the integrity of the implementation of Global Fund grants from the Global Fund perspective. The success of Global Fund in mobilizing substantial donor resources to combat the three diseases. The fact that the Global Fund provides financial assistance in the form of grants. The absence of written Bank-wide guidelines or directives for engaging with the Global Fund. The fact that Fund Portfolio Managers generally have a different professional background from the Bank’s health sector project managers. The fact that Global Fund uses a disease-specific monitoring system to support its performance- based funding approach to development assistance. The different project cycle of the Global Fund compared to the World Bank. The lack of financial compensation for providing technical support. The absence of a Memorandum of Understanding between the Global Fund and the World Bank for collaborating at the country level. The limited country presence of the Global Fund. (Their FPMs based in Geneva.) 0% 20% 40% 60% 80% 100% Much easier Somewhat easier Neither easier nor more difficult  Somewhat more difficult Much more difficult Source: IEG Survey of World Bank HNP project managers and Global Fund Secretariat staff, administered in March 2011. 91 Figure 14. Global Fund: In your opinion, do the following factors make it easier or more difficult for Global Fund managers, staff or agents to engage with the World Bank at the country level? The relatively strong country presence of the World Bank. (Their project managers are often based in the country.) The fact that the World Bank provides technical and/or financial support to strengthen country- level health sector monitoring and evaluation systems. The fact that a project manager is responsible for overseeing the implementation of World Bank- supported projects and technical assistance activities. The success of the Global Fund in mobilizing sub-stantial donor resources to combat the three diseases. The presence of other mechanisms through which the World Bank and the Global Fund may interact, such as ASAPs, JANS, and the Joint Funding Platform. The focus of the Global Fund on low-income countries. The fact that Bank health sector project managers have a different professional background from Fund Portfolio Managers. The World Bank requirement of Bank budgetary or trust fund resources for everything done by staff, including provision of technical support. The fact that World Bank-supported projects are implemented by government agencies. The fact that the World Bank provides financial assistance primarily in the form of loans. The absence of written Global Fund guidelines for engaging with the World Bank at the country level. The fact that World Bank investment projects and technical assistance activities are based on a Country Assistance Strategy. The different project cycle of the World Bank compared to the Global Fund. The fact that the World Bank is less engaged with civil society organizations compared to the Global Fund. The absence of a Memorandum of Understanding between the Global Fund and the World Bank for collaborating at the country level. 0% 20% 40% 60% 80% 100% Much easier Somewhat easier Neither easier nor more difficult Somewhat more difficult Much more difficult Source: IEG Survey of World Bank HNP Project Managers and Global Fund Secretariat staff, administered in March 2011. 92 4.80 IEG’s interviews with World Bank and Global Fund staff identified more than a dozen key factors that made it either easier or more difficult for the two organizations to engage effectively at the country level. Then IEG asked survey respondents’ views in relation to each these factors, as presented in Figures 13 and 14, for World Bank and Global Fund respondents, respectively. The results are presented in descending order in both figures from those factors that make it easier at the top, to those which make it more difficult at the bottom. Horizontal lines have been inserted in the figures to distinguish the factors that make engagement easier, from those that are neutral, and from those that make engagement more difficult, in the overall view of the survey respondents. 4.81 Both Global Fund staff and Bank project managers generally have a positive view of other mechanisms through which the two organizations may interact, such as ASAPs, the JANS, and the Health Systems Funding Platform. They also have a positive view of the focus of the Global Fund on low-income countries. Global Fund staff view positively the fact that a project manager is responsible for overseeing the implementation of World Bank-supported projects and technical assistance activities. 4.82 Global Fund staff generally appreciate the relatively strong country presence of the World Bank, while Bank project managers find the limited country presence of the Global Fund to be problematic. Global Fund staff appreciate the Bank’s support for strengthening country-level health sector M&E systems, while Bank project managers find the Global Fund’s disease-specific monitoring system that supports its PBF funding approach to be problematic. 4.83 Bank project managers appreciate the presence of CSOs on the CCMs and the fact that Principal Recipients of Global Fund grants are not restricted to government agencies. Global Fund staff found it problematic that the World Bank is less engaged with CSOs and that World Bank-supported projects are generally only implemented by government agencies. Bank project managers who participated in the focus group asserted that the Bank has been more engaged with CSOs, particularly in HIV/AIDS projects, than may be readily apparent. As the Bank’s MAP projects have wound down, the CSOs engaged in MAP operations have appreciated the continuing opportunity to be involved in disease-control efforts through participation in the CCM and as Sub-Recipients. 4.84 Yet both Global Fund staff and Bank project managers also view engagement as difficult in some respects. Bank project managers regard the lack of financial compensation for providing technical support to be an unfunded mandate. Global Fund staff regard as problematic that World Bank funding for the health sector, and associated budget support for project supervision, has to compete with other sectors for its place in the Bank’s CAS and associated work program. Both regard their own organizations as more flexible in responding to country needs and priorities, based on interviews. But the survey results suggest that some other factors raised in interviews are not significant impediments to collaboration: the different professional backgrounds of project managers and FPMs, the different types of financial support (loans versus grants), the success of the Global Fund is mobilizing donor resources to combat the three diseases, and the role of the LFA in fiduciary oversight of Global Fund grants. 93 4.85 Neither World Bank project managers nor Global Fund Portfolio Managers are satisfied with “business as usual.� Both groups viewed the absence of an MOU on country-level collaboration between the two organizations as a significant impediment to collaborating at the country level — the most significant factor for Global Fund staff and the second-most significant factor for Bank project managers. Both found the absence of guidelines within their own organizations for engaging with the other organization to be problematic. Prospects for Future Engagement at the Country Level 4.86 There has been growing engagement between the Bank and the Global Fund at the corporate level through the Bank’s involvement in Global Fund governance, through secondments of Bank staff to the Global Fund, and through contacts such as that between the Global Fund’s Inspector-General and the World Bank’s Integrity Vice President. Both organizations are already working together at the global level in the context of the IHP and related initiatives, including the Health-8, the Health Systems Funding Platform, and the JANS. And now there are growing pressures, particularly from donors, for GAVI, the Global Fund, and the Bank — as the three largest multilateral financiers of country-level investments in health — to improve collaboration at the country level. 4.87 Both the Global Fund and the Bank staff recognize that each organization has certain comparative advantages in financing health sector investments at the country level. The current climate also seems more propitious than during the last attempt in 2006–07 to work out a division of labor and ways of collaborating at the country level in the form of an MOU — that is, before the IHP was launched in September 2007 and before the FYE was issued in March 2009. That evaluation found a need to define with greater clarity and formality operational partnerships among the Global Fund, World Bank, and other major multilateral organizations involved in global health and, “as a first priority, resolving the issues that impede the provision of essential technical assistance on a reliable and timely basis� (Macro International 2009b, p. 33). 4.88 However, based on interviews and the survey results, staff in both organizations would clearly prefer to engage on their own terms: that is, in terms of their own organization’s business model. They generally viewed the comparative advantages of the other organization in terms of what the other could contribute to their own method of operation. Global Fund staff viewed the principal comparative advantages of the World Bank as (a) facilitating dialogue with Ministries of Finance, Planning, and other central ministries; (b) helping to improve financial management and procurement; and (c) providing finance for long-term investments in health infrastructure (Table 14). They would like the Bank to make a greater effort to include them in high-level government discussions, as has happened in some countries, such as Cambodia, and for the Bank to contribute its health sector expertise to Global Fund processes such as the CCM at the country level. 4.89 World Bank project managers viewed the principal comparative advantages of the Global Fund as (a) mobilizing donor resources to combat the three diseases in the short term, (b) promoting country-owned strategies and other responses to combat the three diseases, and (c) sustaining financial resources to combat the three diseases over the long term. They would like the Global Fund to contribute to multidonor SWAps or cofinance World Bank projects in 94 Table 14. Comparative Advantages of the World Bank and the Global Fund: Each Organization’s Perspectives of the Other Organization (in descending order from “most important� to “least important�) Global Fund Staff of the World Bank World Bank Project Managers of the Global Fund Facilitating dialogue with Ministries of Finance, Mobilizing donor resources to combat the three 1 Planning and other central ministries. diseases in the short term. Helping to improve financial management and Promoting country-owned strategies and other 2 procurement. responses to combat the three diseases. Providing finance for long-term investments in Sustaining financial resources to combat the three 3 health infrastructure. diseases over the long term. Helping to design and prepare investment projects Facilitating an effective rapid response to the three 4 in the health sector. diseases in the short term. Developing specialized expertise in the Helping to formulate appropriate strategies and 5 prevention, treatment, and care and support in policies in the health sector. dealing with the three diseases. Helping to reform health care finance systems Lowering the transactions costs of development 6 over the long term. assistance from the point of view of donors. Helping to strengthen health delivery systems over Promoting a results focus to development 7 the long term. assistance. Organizing and facilitating policy dialogue at the Lowering the transactions costs of development 8 national, sectoral, and project levels. assistance from the point of view of beneficiaries. Building institutional and human resource 9 Managing country-specific donor trust funds. capacity to combat the three diseases over the long term. Supervising investment projects and field Ensuring that aid resources are used efficiently 10 operations. and effectively. Source: IEG Survey of World Bank HNP Project Managers and Global Fund Secretariat staff, administered in March 2011. the health sector, and for the Global Fund’s donors to establish a trust fund for financing Bank-supervised technical assistance in support of Global Fund-supported activities (Figure 15). 4.90 The two areas of greatest agreement between Global Fund staff and World Bank project managers, in terms of changes each would like to see in the future, were (a) the Bank’s being an ex officio member of the CCM whenever the Bank is an active player in the health sector in the country, and (b) the two organizations’ establishing an active staff exchange program. 4.91 The Global Fund signed an MOU with UNAIDS in June 2002, in its first year of operation. It has more recently signed MOUs with Stop TB (in February 2009) and with RBM (in April 2010). The Global Fund formed a Partnership Group in the Global Fund Secretariat, and the Board approved a Partnership Strategy in November 2009 in direct response to the findings and recommendations of the FYE (Appendix G). The Global Fund is seeking a strategic division of labor with other development partner agencies, greater clarity of roles, and mechanisms for coordinating and funding technical assistance at the country level. 95 Figure 15. What changes would you like to see in the Global Fund and the World Bank to facilitate greater engagement between the two organizations to achieve positive results at the country level, while also respecting each organization’s fundamental purposes and principles? (Percent “Yes�) The Global Fund’s participating in multi- donor Sector-Wide Approaches in support of nationally-defined programs to combat the three diseases. The Global Fund’s donors establishing a trust fund at the World Bank for financing Bank- supervised TA in support of Global Fund- supported activities. The Global Fund’s co-financing World Bank projects n the health sector, like bilateral donors currently co-finance Bank projects. The World Bank’s being an ex officio member of the CCM wherever the Bank is an active player in the health sector in the country. The Global Fund’s providing direct financing for World Bank-supervised TA in support of Global Fund-supported activities. The two organizations’ establishing an active staff exchange program. The Global Fund’s using the World Bank’s Project Implementation Unit as the Principal Recipient for selected Global Fund grants, and World Bank staff overseeing these grants like for Bank projects. 0% 20% 40% 60% 80% 100% GF � All Clusters WB �Task  Team Leaders Source: IEG Survey of World Bank HNP project managers and Global Fund Secretariat staff, administered in March 2011. Note: TA = technical assistance. 4.92 On the other hand, senior managers at the World Bank have expressed reservations about the appropriateness of an MOU. MOUs can provide guidelines to staff in both organizations, but they are not usually considered legally binding even if they are signed documents. However, they can raise more expectations than intended on one side or the other, thereby having a practically binding effect. If there is a willingness on the part of senior managers in both organizations to forge greater collaboration at the country level, there are other ways of doing this more clearly and effectively, such as the following:  Establishing a trust fund at the World Bank, like those that have been established by UNAIDS, GAVI, and WHO (for IHP+), specifying how the resources will be used to support Global Fund activities at the country level.  Signing a “service agreement� that spells out in detail what the Bank will do and how the Bank will be compensated.  Strengthening collaboration (without a formal agreement or flow of funds) by means of an exchange of letters, agreed terms of reference, summarized and confirmed 96 minutes of meetings, workshops, and other events where both sides agree to work together on certain activities. 4.93 Whether or not the Bank reaches a formal or informal agreement with the Global Fund along any of these lines, there needs to be a clearer institutional mandate for Bank staff to work with the Global Fund at the country level, if World Bank engagement with Global Fund- supported activities remains at current levels, or increases. For the benefit of client countries — particularly low-income countries with high disease burdens — the ways in which the Bank’s country teams and staff are permitted, encouraged, or required to engage with Global Fund- supported activities at the country level simply need to be defined. Resources need to be allocated for the purpose with appropriate institutional recognition of contributions made and achievements accomplished. Expected contributions of Bank staff should be part of work program agreements, and achievements recognized in performance reviews. If staff are directed to serve on CCMs, they should have a terms of reference specifying the timeframe, their responsibilities, and their reporting requirements.83 And for sustainability, relationships need to move beyond the personal level (the current situation) to the institutional level. Such directives and guidelines are not contrary to country-driven development; they can allow for case-by-case judgment, taking into account country differences. 4.94 Experience has shown that the Bank can contribute meaningfully to the work of the Global Fund at the country level in ways that also benefit its own programs, but without undertaking supervisory or operational roles for the Global Fund in client countries. Undertaking such roles — as the Bank currently performs for the GEF and as essentially happened for Global Fund grants in Lesotho and the Russian Federation — might also be considered on a pilot basis under certain circumstances, such as a SWAp operation or a common implementing agency implementing related activities supported by each organization. However, the Bank has its own rules of engagement that would have to apply when it takes on such roles. The Bank still needs to be able to carry out its own work program in each country. Thus, agreeing to supervise the implementation of specific Global Fund grants, even on a pilot basis, would need to be viewed as part the Bank’s own operations in the country, subject to the Bank’s operational policies and procedures, as is currently the case for GEF-financed projects. 4.95 The World Bank, the Global Fund, and other multilateral organizations have expressed good intentions to coordinate and streamline M&E processes at the country level. They have endorsed the Three Ones principles and they have jointly prepared an M&E Toolkit in 2004 (revised in 2006, 2009, and 2011) to establish norms and identify indicators to be used by all the agencies. In terms of developing frameworks and identifying indicators, there has been some progress. The approach and the indicators in the M&E Toolkit make a lot of sense, but these have been difficult to achieve in practice because each agency has its own project-level M&E requirements, which often provide very little value for program assessment, program management, or policy dialogue. Achieving the third “One� — one county-level M&E system 83. This mirrors an IEG recommendation at the global level that Bank staff serving on partnership boards should have standard terms of references. In its formal response, Bank management agreed with this recommendation, only disagreeing that the terms of reference should be standard. See IEG, 2011b, The World Bank’s Involvement in Global and Regional Partnership Programs, p. xxxii. 97 in each disease area — is also dependent on achieving the first two “Ones� — a common action framework with a single coordinating authority. 4.96 Both the Global Fund and the World Bank could contribute to improved M&E at the project and country levels by making a stronger commitment to the Three Ones principles. Then project-level M&E — including the Global Fund’s PBF approach to disbursements — could focus on accountability for achieving the specific outputs of each project, and country-level M&E could focus on tracking the higher-level outcomes and impacts collectively. The World Bank should also continue to provide technical assistance to strengthen national M&E capacity through components of health projects and through GAMET, as it has done in the past. 98 5. Lessons from the Five-Year Evaluation for the Evaluation of Global Partnership Programs 5.1 This chapter assesses the independence and quality of the FYE of the Global Fund and draws lessons for the evaluation of other global partnership programs.84 The independence and quality of the evaluation is assessed against the standard framework that IEG uses for this purpose (Appendix Table A-3), which is based on the Indicative Principles and Standards in the Sourcebook for Evaluating Global and Regional Partnership Programs (IEG and OECD/DAC 2007). 5.2 The FYE comprised three Study Areas and a Synthesis Report undertaken over a two- and-a-half year period. Overall, IEG found that the FYE was an independent and quality evaluation that has helped the Global Fund Board and management make significant strategic adjustments to its organizational and institutional arrangements. The three study areas reinforced one another, and the Synthesis Report effectively pulled together key messages in a coherent and integrated manner. Charged with a complex evaluation and an ambitious scope of work that had to be completed within a tight timeframe, the evaluation teams fulfilled the majority of their terms of references. 5.3 The chapter is organized into four major sections: (a) the oversight and management of the FYE; (b) the evaluation’s participation, transparency, and dissemination activities and practices; (c) the quality of Study Areas 1, 2, and 3 in terms of their evaluation approaches, methodology, and instruments; and (d) the evaluation capacity building initiative in Study Area 3. (Appendixes N, O, and P provide additional evidence for the conclusions reached in this chapter.) Oversight and Management of the Evaluation BACKGROUND 5.4 The Global Fund did not have an evaluation policy at the outset, even though evaluation was a clear corporate priority. Initial plans for the FYE were conceived as early as October 2003, a year after the establishment of the Global Fund (Table 15). At its first meeting in January 2002, the Board established a Working Group to develop an M&E strategy and program of work. This Strategy, adopted in 2003, called for a review of the Fund’s overall performance against its goals and principles after one full grant cycle had been completed. The M&E Strategy also called for the development of an M&E Operations Plan, and the creation of an independent body to provide advice, assessment, and oversight for the Fund’s work on M&E. The same body would oversee the execution of the evaluation. 84. This assessment is based on interviews with Global Fund staff, TERG members, country-level stakeholders (government and civil society counterparts, UNAIDS, WHO, and other development partner agencies), members of the Impact Evaluation Task Force in Cambodia, Macro International team members, Social and Scientific Systems, Inc., and extensive reviews of Global Fund Board, TERG, and Secretariat documents. Other parties and supporting research materials were also consulted as necessary. 99 Table 15. Five-Year Evaluation Timeline Date Event 2003 Monitoring and Evaluation Strategy adopted by Global Fund Board in 2003 called for establishment of an independent expert group — the TERG — (a) to advise Global Fund Board and (b) to support the Global Fund Secretariat’s M&E work. Nine members appointed by Board and four ex officio members. (October) Board approves undertaking a five-year evaluation of overall performance of the Global Fund against its goals and principles, after at least one full grant cycle has been completed. The FYE to be planned and implemented under TERG oversight. General areas for study: organizational efficiency and effectiveness, effectiveness of the partner environment, and impact of the Global Fund on the burden of HIV/AIDS, tuberculosis, and malaria. 2004 (September) TERG established; evaluation discussion paper issued on FYE. 2006 (March–May) Stakeholder consultation on overarching questions for FYE (March–June) Stakeholder Assessment conducted online, with 900 respondents (July) Global Fund Partnership Forum in Durban endorses FYE’s overarching questions. (July–October) Design of the evaluation by Social & Scientific Systems contractor, with Secretariat and TERG (November) Board approves launch of the FYE, based on TERG proposal/evaluation plan. 2007 (January) Requests for Proposals issued for (a) Study Areas 1, 2, and Synthesis Report, and (b) Study Area 3. (April) Contract issued to evaluation consortium, Macro International (May-September) Country work plan development and approval; workshops (June) Inception Report for Study Area 1 and Study Area 2 (October) Study Area 1 Report issued (Nov-Dec) Disbursements to countries for country impact studies — 47 subcontracts as part of FYE. 2008 (June) Study Area 2 Report issued 2009 (March) Synthesis Report issued (May) Study Area 3 Report issued (May) Board and Policy and Strategy Committee (PSC) discuss FYE. TECHNICAL EVALUATION REFERENCE GROUP 5.5 The independent body formed in 2004 was called the Technical Evaluation Reference Group (TERG). It is a external body of experts, with a range of skills that include public health, evaluation, social science, organizational management, and development. Directly appointed by the Board, the nine members serve three-year terms with an honorarium. 5.6 TERG has two mandates that potentially conflict. On the one hand, it oversees and manages independent assessments, provides advice, and reports directly to the Board. On the other hand, it also advises the Secretariat on evaluation approaches and practices, reporting procedures, and other technical and managerial aspects of M&E. This includes reviewing the 100 Global Fund’s progress toward the implementation of its M&E Strategy and providing guidance to the Secretariat in refining the M&E Strategy as the Global Fund evolves. 5.7 The Global Fund Board and Secretariat are aware of this potential conflict, and are managing it transparently. They believe that the existing set-up allows TERG to be objective, and still able to foster a culture of learning and self-correction in the Secretariat.85 The Global Fund’s internal M&E function is managed directly by the Strategy, Performance and Evaluation Cluster. TERG’S ROLE DURING THE FIVE-YEAR EVALUATION 5.8 TERG is to act independently of the Secretariat, while reporting to the Board. Both can heed or ignore TERG recommendations. Nonetheless, documentary evidence and interviews by IEG show that TERG has wielded considerable influence and particularly so during the execution of the FYE. TERG was involved with each stage of the FYE, was the ultimate signatory for all FYE evaluation products, and approved the payments to the evaluators. 5.9 According to its terms of reference, TERG can provide independent assessments to the Board, interpret the findings of evaluation reports, and make its own recommendations.86 Consequently, TERG submitted its own recommendations for the consideration of the Global Fund Board, alongside the full reports and recommendations of the evaluation consortium. TERG also reported on how effectively the consortium had fulfilled its terms of reference. IEG found that these practices contributed to perceptions of micromanagement,87 and even interference by TERG during the FYE, even though these practices were consistent with TERG’s practices during previous evaluations of the CCM model and the LFA system. 5.10 During the later part of the FYE, TERG’s relations with the contractors and the Secretariat grew increasingly tense. Changes in senior management at the Global Fund also led to different expectations of TERG’s oversight role. These differences and the large TERG demands on Secretariat resources contributed to this tension.88 The Board, however, reiterated its strong support for TERG, and appreciated its oversight of the FYE.89 5.11 At the end of the day, IEG found that the conduct of the FYE was organizationally and behaviorally independent. The evaluation teams were able to report candidly about how slowly and less strategically the Global Fund governance processes had developed to guide this new approach to development assistance, about the need for a robust risk management strategy to 85. This debate continues, as the terms of reference for the TERG are being revised, in the follow up after the FYE. For instance, IEG found that the Global Fund’s M&E system does not include a standard end-of-grant evaluation process, the absence of which hinders learning lessons from completed grants for future Global Fund activities (Cheryl Cashin, forthcoming, pp. 40-41). 86. Terms of Reference for TERG, 2007. 87. TERG took seriously its role of quality assurance, and intervened on occasion because it sought to ensure appropriate evaluation methods were used. 88. Staff dedicated to support TERG were under tremendous strain due to the frequent meetings. 89. Board documents, post-FYE, at the 19th and 20th Board Meetings. 101 alert the Global Fund about likely suspension of ongoing treatment activities, and about the risk of increased drug resistance, among other things. The fact that the TERG reported to the Board did not prevent TERG from submitting findings that were critical of the Board. Notwithstanding TERG’s very “involved� oversight style, the FYE was also protected from outside interference, and the potential conflicts of interest that arose were appropriately identified and managed. TERG members themselves signed full disclosure statements, since many had previously been associated in one way or another with the broader Global Fund partnership.90 FORMULATING THE EVALUATION PRIORITIES AND QUESTIONS 5.12 Preparation for the evaluation passed through three phases that helped define and frame the evaluation, before it was formally launched in November 2006:  An initial face-to-face consultation with experts was conducted during March–May 2006 to formulate the overarching questions and priority issues for the evaluation.  This consultation was expanded to a broader audience through targeted e-mails and a Web survey during March–June 2006 about the Global Fund’s reputation, performance, strengths, and weaknesses.  The cumulative results were presented to the Global Fund’s biennial Partnership Forum in Durban, South Africa, in July 2006 for further validation of the evaluation priorities and issues. 5.13 IEG found that these steps allowed for transparency, and strategic and quality input, and engendered ownership and participation from a broad stakeholder base. EARLY DESIGN AND BUDGETING 5.14 IEG found that the planning for the FYE was carefully done and well resourced. Once the overarching questions and topics were finalized, TERG contracted with Social & Scientific Systems, Inc., to develop a comprehensive work program for the evaluation. A senior evaluation staff member of the Global Fund, who was assigned to work full time with the team, was a key asset. Her in-depth knowledge and familiarity with the Global Fund were invaluable. Not only was she knowledgeable about the Global Fund business model and resources (including relevant research and evaluations that would serve as inputs to the FYE), but she was also able to informally advise on the character of the organization and on its expectations for the FYE. 5.15 Social & Scientific Systems produced the Technical Background Paper (Global Fund 2006b) that outlined the full scope of work, the number of evaluation studies to be produced (Study Areas 1, 2, 3 and Synthesis), and how these would relate to each other. The Paper also described the purpose, methodologies, options for implementation, timelines, and budgets for each study. It recommended sources of data, other studies on which to draw, staffing needs, costs, countries to be visited, and the required skills of evaluation teams to be formed. 90. IEG has observed that the pool of candidates with the required technical skills, knowledge, and experience to evaluate large GRPPs like the Global Fund and the Global Environment Facility, who have never done any work for the program, is often limited due to the program’s overwhelming presence in the sector (IEG and OECD/DAC 2907, p. 41). In such cases, the key is to identify and manage conflicts of interest transparently. 102 5.16 Social & Scientific Systems consulted TERG closely in the development of the Technical Background Paper, and together they determined the final budget for the FYE.91 The Global Fund Board approved special budgetary allocations for the FYE. In the interest of collaboration, TERG reached out to development partners such as the U.S. Office of Global AIDS, PEPFAR, and UNAIDS, who participated in and cofinanced selected parts of the evaluation. The Technical Background Paper became the basis for the Evaluation Plan and Framework Document, which the Board approved in November 2006. IEG found that the broad-based consultation also generated high expectations about the product and its anticipated value as a global public good. REVIEW, REPORTING, FEEDBACK AND PROGRAM RESPONSE 5.17 TERG regularly updated the Board and the Policy and Strategy Committee as the evaluation teams submitted FYE reports to TERG for review, so that the Global Fund often started to make changes before the final evaluation products were publicly disclosed. TERG also invited the Global Fund Secretariat to provide comments on the findings. Then TERG deliberated on the findings and submitted the recommendations to the Board for review and consideration.92 Some of TERG’s recommendations differed from those of the contractors. In all cases, the Board welcomed both the Macro International and TERG reports, and directed the Secretariat to respond and act on them. 5.18 The Secretariat has issued a Formal Management Response and a Management Update (Global Fund 2010b), and has already initiated reforms. The Board delegated the preparation of a formal Board response to an Ad Hoc Committee composed of members from the Board’s Finance & Audit, Policy & Strategy, and Portfolio & Implementation Committees.93 5.19 Conclusions. Planning for the FYE was deliberative, systematic, and innovative. The extensive consultations, at the outset, in the identification and formulation of evaluation topics and questions, engendered ownership and support from a broad stakeholder base of donors, governments, and civil society. Significant effort and resources were also devoted to designing and developing the evaluation work program through Social & Scientific Systems. Greater flexibility and discretion could have been accorded to Macro International, however, in the execution of this workplan. It might have been more efficient if the evaluation implementation team had been involved in the original study design and methodology. Participation, Transparency, and Dissemination 5.20 Judged against the indicative principles and standards of the IEG and OECD/DAC Sourcebook for Evaluating Global and Regional Partnership Programs, IEG found that the design of the FYE approached the standard of good practice with respect to participation, transparency, and dissemination. The Board mandated five guiding principles for the evaluation: 91. Details of Study Area 3 and its budget requirements were handled separately by another contractor, and then combined with the main Technical Paper. 92. TERG deliberations about the FYE products, and the review and management response processes are available on the Global Fund Web site. 93. Board documents GF/B19/11 and GF/B19/DP29. 103 (a) to be inclusive, (b) to be country focused/led, (c) to build country evaluation capacity, (d) to collaborate with local institutions, and (e) to share and disseminate the knowledge developed as a national and global public good. Overall, the FYE adhered to these principles. 5.21 Upstream participation. To begin with, the TERG chair and a consultant conferred with 23 experts — including Global Fund Board members, government ministers, and directors of donor and civil society groups — in the formulation of the overarching evaluation questions. Then TERG opened up the whole process through a Web survey and targeted e-mails to 5,700 contacts.94 TERG received questionnaires from 900 respondents on issues related to the evaluation and its intended use. Then TERG presented these results for discussion at the Global Fund’s biennial Partnership Forum in Durban, South Africa, in 2007, attended by some 400 participants (many from CSOs) from 118 countries. 5.22 Each phase of the consultation was documented, and a detailed analysis of the issues was made available on the Global Fund Web site. This report, called the “360 Stakeholder Assessment,� provided the aggregate profiles of respondent groups, and their respective positions on different issues for the FYE. 5.23 One of the goals of participatory evaluations is to gain greater stakeholder ownership of the evaluation product, process, and intended use. IEG found that this was largely achieved at the global level. The consultation on core issues of the evaluation, carried out at the upstream design stage of the FYE, helped win many supporters for the evaluation. There was also strong support for incorporating the learning and capacity-building functions in the FYE as a global public good. 5.24 Participation during planning and execution. The FYE sought the active participation of development partners and country clients in its implementation. At the global level, UNAIDS, PEPFAR, and USAID contributed to selected evaluation activities of the FYE. PEPFAR provided $3.5 million to cofinance the data quality management training and dissemination workshops in Study Area 3, while UNAIDS led and coordinated the Impact Evaluation Task Forces (IETFs) at the country level. 5.25 The IETFs were formed to tap country knowledge and expertise in planning, implementing, and coordinating the work for Study Area 3. Comprised of representatives from government, CSOs, development partners, CCMs, and research, academic, and statistical institutions, the IETFs were to strengthen country ownership, and act as sounding boards and reviewers of the evaluation as it progressed. TERG also envisaged that the evaluation processes, techniques, and tools developed collaboratively with the IETFs would continue to be used in the countries after completion of the FYE. 5.26 Many stakeholders were involved in the implementation of the FYE. In Study Area 3 alone, Macro International subcontracted 50 local institutions in the data collection and analysis, where almost $5 million of the evaluation budget was expended. Driven by the desire for a high-quality evaluation product, TERG members participated in some country missions and visited with the IETFs. 94. E-mail questionnaires were distributed in English, French, Spanish, and Russian. 104 5.27 The country-led concept was good and had tremendous support at the global level. However, IEG found that its execution was problematic and that the in-country mechanisms and structures were not fully engaged. The tight implementation schedule did not allow for the evaluation teams to fully engage with the IETFs in Study Area 3, some of which needed more time to achieve consensus on issues, while others needed capacity building to do their jobs. 5.28 Learning and dissemination. Learning workshops called Partners in Impact Forums allowed the IETFs to exchange ideas with one another and with global experts on technical issues about impact evaluation, data quality, and their management. The IETFs discussed how they would “integrate� studies planned for Study Area 3 into their existing evaluation work programs for AIDS, tuberculosis, and malaria. Countries without such a work program received technical assistance to develop them. At the end of the FYE, the Partners in Impact Forums were reconvened to discuss the results of Study Area 3. 5.29 Transparency. All processes and results of the FYE, from the conceptual to the execution stages, have been made available on the Global Fund Web site. This includes the deliberations by TERG, the Board, and its Committees on the Macro International reports, findings and recommendations. IEG found the overall conduct of the FYE to be highly transparent, as mandated by the Board. However, FYE products have not been translated into other languages. 5.30 Conclusions. The consultative and participatory nature of the evaluation led to the many preparatory steps, approval/vetting mechanisms, and country-level evaluation task forces that characterized the FYE. Such participatory processes were intended to engender ownership by the stakeholders and shared decision making for the use of evaluation results. Stakeholder ownership was achieved at the global level, but not at the country level among Study Area 3 participants. Based on IEG interviews, documentary review, and direct feedback from two countries, the lack of engagement with IETFs as full partners was a key factor. 5.31 The overall conduct of the FYE has been highly transparent. The learning workshops for Study Area 3 and the Global Fund’s Web site were good dissemination mechanisms. However, even though the Web site supports English, French, Spanish, Russian, and Chinese, FYE reports are only available in English, which limits their potential as a public good. The development approach of the FYE was extensive, with $5 million spent in participating countries. Subcontracting institutions in Study Area 3 countries to participate in the FYE was an innovative attempt to build evaluation capacity and to sustain the use of these techniques and tools in these countries after the FYE. However, stronger ownership by Study Area 3 countries would have been necessary to realize the intended development benefits. Study Areas 1, 2, and 3: Their Evaluation Approaches, Methodology, and Instruments 5.32 Overall, IEG finds that the FYE was a quality evaluation. The evaluation was objectives-based and evidence-based against the stated purpose and principles of the Global Fund (Appendix C). The assessment was fair and balanced, portraying both the strengths and weaknesses of the Global Fund. Although the FYE did not deliver on two objectives — developing the determinants of good grant performance and building institutional evaluation 105 capacity in the Study Area 3 countries — it was an innovative evaluation experience, from which to draw procedural and methodological evaluation lessons. 5.33 The FYE met three of the four standard IEG criteria for assessing quality (Appendix Table A-3) — evaluation scope, instruments, and feedback. It did not meet the M&E criterion that the program’s activity-level M&E system should contribute to the evaluation’s assessment of the overall outcomes of the program because the Global Fund’s grant-level M&E system was not initially designed to do so. Even if it had been so designed, it would have been too early in the life of the program to make such a contribution. Therefore, other methods, notably the impact assessment in Study Area 3, had to be used. STUDY AREA 1: ORGANIZATIONAL EFFICIENCY AND EFFECTIVENESS OF THE GLOBAL FUND 5.34 Study Area 1 was charged with evaluating the degree to which the Global Fund (a) had established a business model that adhered to its Guiding Principles, and (b) had built an organizational architecture and governance structure to support that business model.95 This included reviewing and benchmarking the resource mobilization strategy and efforts against those of comparable institutions. Accordingly, the evaluation consortium led by Macro International set out to assess if the Global Fund model was “fit for the purpose� from the outset and whether it could or should endure as the Global Fund evolved and matured. 5.35 Macro conducted Study Area 1 guided by the Evaluation Plan for the FYE and in accordance with the plans, questions, methods, and tools outlined in the Inception Report. The terms of reference did not require Macro to examine the “relevance� of the Global Fund business model in the global health architecture, or the validity of the assumptions behind the model. Instead, Macro was to assess if the organization had been set up and operated in adherence with the values embedded in the Guiding Principles.96 5.36 Like many formative evaluations of GRPPs, there was strong emphasis in the terms of reference on assessing the appropriateness of the program’s organizational setup and institutional arrangements — that is, its governance and management arrangements. Consistent with the IEG and OECD/DAC Evaluation Sourcebook, there was much greater focus on governance, and a more limited scope of work for examining the Global Fund’s management.97 The Secretariat had also commissioned a separate Management Review in parallel with the Study Area 1.98 95. See Macro International (2007b), Inception Report Summary for Study Areas 1 and 2. 96. IEG has found that the Guiding Principles have been central to the Global Fund mission and have been used extensively to guide the design, makeup, and operation of the organization. 97. “It is neither practical nor appropriate for evaluations to assess all aspects of management. Thus the terms of reference should specify clearly which aspects of management have been selected for assessment. The assessment should focus on those aspects that most directly affect program performance, and avoid the type of ‘micromanagement’ or ‘microevaluation’ that is outside the purview of both a program’s governing body and an evaluation team� (IEG and OECD/DAC 2007, pp. 74-75). 98. Booz, Allen and Hamilton, Organizational & Management Review, Draft Executive Summary, November 2007. 106 5.37 The Management Review could have been an important input to Study Area 1, but IEG found little reference to it in the Study Area 1 report. It would have been important to apprise the reader of the findings of the Management Review, and the extent to which the findings of the two reviews were consistent. It may be that the Management Review was not completed in time to be shared with the Study Area 1 evaluation team. If so, this fact should have been recorded. 5.38 Study Area 1 used an “organizational development� approach to assess the efficiency and effectiveness of the Global Fund structures, in accordance with plans outlined in the Technical Background Paper. Normally, such an evaluation report would have discussed the underlying theoretical concept and basis for this approach. The Study Area 1 report did not do this. 5.39 Study Area 1 tackled challenging topics. It concluded that the Board had failed to provide adequate strategic direction; that the Board had tended to overly manage the Secretariat; that conflicts and tensions between the Guiding Principles had affected program performance; that the Global Fund had relatively neglected partnership issues despite its high dependence on its partners to achieve objectives; and that the Global Fund lacked a risk management framework. 5.40 Study Area 1 identified risks in four key areas that were mission-critical in nature and needed better management:  Corporate reputational risks (the Global Fund was dependent on the CCM and Principal Recipient for grant oversight and appropriate use of funds)  Loss of donor confidence (not meeting expectations of results regarding disease outcomes, especially given the demand-driven nature of Global Fund grants)  Risks to beneficiaries and control of the three pandemics (PBF increases the risk of stopping already ongoing treatment services and accelerating drug resistance)  Human resources and institutional intelligence risks (portfolio management was too dependent on individuals and not adequately systematized, coupled with high staff turnover). 5.41 IEG found Study Area 1 to be influential. TERG exercised very close oversight of Study Area 1, to the point of requiring several drafts, which delayed the submission of the final report. Ultimately, TERG expressed overall satisfaction with the quality of the product, noting a number of limitations: the Global Fund was benchmarked against fewer organizations than planned; some interview methods were lacking in clarity; and certain analyses were anecdotal in nature, such as the role of Executive Director on the Board and workplace issues.99 99. The Study Area 1 report concluded with some shortcomings of its own, such as: (a) not fully covering the Global Fund’s organizational structures; (b) the difficulty of benchmarking the Global Fund due to its unique nature and mission; (c) the limitations in the qualitative assessment of the grant negotiations and rating practices; (d) the poor timing for the assessment of the Board governance review; (e) the limited number of interviews; and (f) the limited review of the TERG, the Partnership Forum, and the Inspector General’s Office. 107 5.42 The Study Area 1 report contributed to six of the nine major findings in the Synthesis Report. The Board and Secretariat have accepted these findings and initiated organizational reforms such as changing the CCM requirements, setting up a Partnership Unit within the Global Fund, and establishing stronger partnership agreements. 5.43 Conclusions. Formative evaluations like Study Area 1 are very important in the early stages of a global program to help a program make strategic adjustments to its organizational and institutional arrangements. Study Area 1 took a longer time to complete than planned, with considerable involvement of TERG in finalizing the draft report. Part of this may have been due to the longer learning curve needed by the evaluation team who had not been involved in determining the design, methodology, and timeframe for carrying out the evaluation study. Both the commissioner and executor of evaluations need to be prepared for such delays when the design and executing teams are not the same. 5.44 The findings of Study Area 1 are particularly relevant for those GRPPs that have adopted inclusive stakeholder models of governance, with broad representation from beneficiary countries and CSOs in addition to financial contributors, since the report covers in some depth the difficulties of managing an inclusive board like that of the Global Fund. STUDY AREA 2: EFFECTIVENESS OF THE GLOBAL FUND PARTNERSHIP ENVIRONMENT 5.45 Study Area 2 was tasked with assessing the Global Fund’s fit in the overall development architecture at the global and country levels. The study team examined all salient areas such as the CCMs and LFAs, their interactions with development partner agencies and country processes, the availability of technical assistance, PBF (central to the Global Fund business model), and grant oversight. Given the broad scope of the work and the methodological challenges (few benchmarks, tight timelines, and sequencing with other Global Fund studies that were occurring in the same countries), the evaluation team focused largely and most importantly on the partnering arrangements at the country level. In-depth qualitative and quantitative assessments were carried out in 16 countries to examine how the model had played out, and the effects on grant performance and on the countries’ health systems. 5.46 Study Area 2 was unable to develop “determinants of good grant performance� by statistical analyses, because the countries selected (through purposive sampling) had insufficient outliers of good and poor performers to allow for generalization of findings. But its impact on the Global Fund has been substantial. Study Area 2 covered topics critical to Global Fund’s mission, provided grounds for the continuation of the Global Fund model, and underlined the need for strengthening the mostly informal nature of its partnerships. It directed recommendations toward improving the CCM, LFA, PBF, and grant oversight functions. It found that for many partners, negative perceptions and expectations of the Global Fund had been “filtered through 60 years’ experience of the conventional development assistance model,� and that partner agreements to high-level principles of collaboration needed to be translated into operational realities. 5.47 Lessons that emerge for other GRPP evaluations call for prudence, to keep the scope of evaluations to manageable size, allow reasonable schedules, and avoid conflicts/competition with other evaluation efforts going on in the same countries. Large 108 GRPP evaluations normally take more time than expected, despite detailed plans, because of the large number of parties involved. Sufficient time should be allowed to pre-test new evaluation instruments. A common conceptual framework and approach to assess country partnerships would be helpful to save time, avoid confusion, and is feasible. The following section presents such a framework. Toward a Common Evaluation Instrument for Assessing Country-Level Partnerships100 5.48 The Country Partnership Assessment (CPA) instrument used in Study Area 2 was a structured questionnaire with seven modules.101 It is a good building block toward developing a conceptual evaluation framework for GRPPs that emphasize country-led processes and alignment with country systems and mechanisms. Based on the CPA instrument and comparable instruments used by UNAIDS and the OECD/DAC, IEG has developed a draft generic Partnership Assessment Tool that could be refined, validated, and then used for other GRPPs financing country-level investments and/or technical assistance. (See Appendix O.) 5.49 IEG has compared and contrasted the CPA tool against the UNAIDS Country Harmonization and Alignment Tool (CHAT), and against the analytical framework used in the Phase 1 Evaluation of the Paris Declaration. Each of these instruments assesses the effectiveness of the collective action of members102 by examining the partnering arrangements on the ground and how they played out. All three frameworks converged on country ownership/commitment, alignment, and harmonization as key elements of their analysis. 5.50 The Study Area 2/CPA had seven modules: private sector resource mobilization, harmonization, in-country partnerships, technical assistance, country ownership and alignment, PBF, and procurement. The UNAIDS/CHAT had four criteria: national AIDS coordinating authority and national strategic framework; M&E; finances; and administration, support, coordination, and communications.103 The Paris Declaration had five criteria: country ownership, alignment, harmonization, management for results, and mutual accountability. 5.51 All three frameworks were developed and utilized in 2007 and 2008, and reflect a growing trend in development. They have the unifying trait of a strong focus on use, support, and alignment with country systems and mechanisms. The evaluation of the Paris Declaration was cross-cutting, looked at all sectors, and had a country-level perspective, while the Study Area 2/CPA and UNAIDS/CHAT focused on AIDS and the health sector. All three assessments used stakeholder mapping, mixed methods, qualitative and quantitative data analysis, case studies, and a variety of survey instruments, including focus groups and face-to-face interviews. 100. GRPPs are programmatic partnerships among multiple entities (donors, developing country clients, international organizations, nongovernmental groups). There is joint decision making and accountability at the governance level. The “Assessment Framework� described here applies to these types of partnerships, and not to partnerships in which one organization “collaborates� with another party to achieve a subset of its own goals. 101. The Study Area 2/CPA also had an introductory module, making it eight in all. 102. Members were broadly interpreted to include not only country governments, but also civil society groups, the private sector, and foreign development entities resident in the countries. 103. The elements of country ownership, alignment, and harmonization were embedded in the “National AIDS Authority,� “National Strategic Framework,� and “Coordination� parts of the CHAT framework. 109 5.52 Combining the three assessment instruments yields a generic framework with nine criteria, as shown in Table 16. The CPA was the most comprehensive of the three because the Global Fund finances country-level investments and has a complex grant performance component. For the purpose of this generic tool, IEG has extracted only the evaluation criteria and topics common to the Paris Declaration Evaluation and the UNAIDS/CHAT, and more broadly applicable to other GRPPs. The CPA, however, lacked the “mutual accountability� element of the Paris Declaration Evaluation Framework, and the “reporting� requirements of the CHAT that make for a more level playing field between donors and recipient countries. These two criteria have been combined into one in Table 16 and called “mutual accountability (reporting and transparency).� Table 16. Toward a Common Conceptual Framework for Assessing Country-Level Partnerships Criteria Possible Topics for Analysis A. Country Ownership Existence of a policy framework and operational work program; existence and performance of country governance and management bodies to direct program activities, e.g., the CCM and National Aids Council B. Alignment By donor partners with country policies and strategies, priorities, M&E systems, payment and reward structures and procedures C. Harmonization Use by donor partners of existing aid coordinating systems of aid, sharing analytical and diagnostic work, joint or collaborative planning and reporting requirements; joint missions and assessments D. Finance and Extent of pooled funding for the same development objectives, moving toward multiyear Resource Mobilization funding for greater aid predictability, inclusion of external aid in national budgets; quality of financial management E. Managing for Results Use of PBF, linkage between diagnostic results and planning, move toward supporting and (M&E) using country management information systems, having transparent and monitorable assessment frameworks that allow for tracking progress against national development strategies, goals, and targets F. Procurement and A key element for partnerships that finance investments. For example, in the health sector, as Supply Management a much as 40–60 percent of a low-income country’s total health expenditures may be spent on drugs, medical supplies, and vaccines. G. Capacity Building Evidence of adequate assessment of external technical assistance needed for key national and Technical processes and its funding and execution. Capacity building and technical assistance should be Assistance demand-driven and consistent with relevant national strategies. This includes capacity building of country processes and institutions to allow for alignment activities described above. H. Mutual Accountability Extent of transparent, timely, and accurate communications among different partnership (Reporting, members; processes that advance mutual accountability (for countries and donors alike) for Transparency) development effectiveness I. Other Criteria Depending on the contextual needs of the GRPP in question Source: Developed by IEG. See detailed Assessment Tool in Appendix O. a. Coordinated logistics by the different partners in a health GRPP to prevent drug outages, and streamlining or use of the same procurement guidelines, organization, approach, or suppliers brings economy of scale and tremendous savings in reduced drug prices. 110 5.53 The Framework also includes a ninth criterion to take into account the contextual needs of a given GRPP. The resulting framework could be used to assess the “functionality� of the majority of GRPPs, regardless of sectoral focus. The proposed evaluation criteria and topics for analysis are presented in greater depth in Appendix O. STUDY AREA 3: IMPACT ON HIV, TUBERCULOSIS, AND MALARIA 5.54 Study Area 3 was called an impact evaluation of the collective efforts of all donors and beneficiary countries on the burden of the three diseases in 18 case study countries. This section assesses the quality and applicability of this approach (impact evaluation of collective efforts) for other GRPPs that are financing investments at the country level. 5.55 The defining characteristic of an impact evaluation is to show attribution or causality between program inputs and the intended development outcomes. In spite of the initial ambition of the Global Fund and TERG to do an impact evaluation, the evaluation teams did not attempt a rigorous impact evaluation to attribute the reduction in the overall disease burden in case study countries to Global Fund-supported interventions, because the interventions had not been designed to facilitate impact evaluations and country-level data were inadequate. In addition, many countries had not yet completed one five-year grant cycle. (Appendix P provides a more detailed analysis of Study Area 3.) 5.56 Given that Study Area 3 attempted to assess the collective efforts of all donors and countries, its evaluation approach may be best compared against the analytical framework of a contribution analysis. In contribution analysis, the program’s contributions are not quantified, but plausible association has to be demonstrated. Contextual factors are important considerations in such an analysis. 5.57 On balance, IEG has found that Study Area 3 did an adequate job of conducting a contribution analysis, but with some shortcomings. It was able to demonstrate that the collective contributions have resulted in increased access to services, better coverage, and some overall reduction of disease burden, as presented in more detail below. The Step-Wise Evaluation Framework adopted by Study Area 3 (Figure 16) placed importance on contextual factors, but IEG found that few contextual factors were actually considered, based on an in- depth review of two country case studies in Study Area 3 (Burkina Faso and Cambodia). 5.58 Assumptions and risks, important in contribution analysis, were not clearly delineated in the evaluation framework for Study Area 3. Instead they were described in different parts of the document, and were not clearly defended. Two such assumptions were: (a) in the absence of scaling-up efforts, mortality and morbidity from the three diseases and intervention coverage would have at best remained the same or worsened; and (b) expected expenditure is flat from 2003 to 2006. These assumptions were not adequately defended in the case studies. 5.59 Another important contextual factor that was not discussed in the Study Area 3 studies was the quality of the services provided by the different donors and country institutions. Study Area 3 implicitly assumed that all donor spending was equal in quality and 111 Figure 16. Evaluation Framework for Study Area 3 INPUTS ACTIVITIES OUTPUTS OUTCOMES IMPACTS Other International Training & resources capacity building Health services Morbidity Intervention Procurement & Delivery & Global Fund coverage supply quality Mortality resources Behavioral Guidelines Behavioral Disease coverage IEC interventions & consequences Domestic Community knowledge resources mobilization CONTEXTUAL FACTORS Has funding Has access and Has coverage Have health increased? quality of improved and outcomes How much? services risk behavior improved? What sources? improved? changed? Sources: Constructed/adapted by IEG from Technical Background Paper, Synthesis Report, and Study Area 3. Note: IEC = information, education, and communication Results, yet the different donor and country programs interact in various ways, and determine final outcomes at the country level. They are implemented by different agents with a variety of strengths and weaknesses. These contextual factors about the different partner inputs were not addressed in Study Area 3. 5.60 The main Study Area 3 findings were that collective efforts had contributed to: (a) Increased access to services, particularly for HIVAIDS — for instance, in most countries, the number of facilities that provide HIV testing and counseling or ART more than doubled between 2004 and 2007. (b) Increased coverage of HIV/AIDS and malaria interventions — for instance, major progress has been made in ART coverage for HIV/AIDS; and for malaria, progress in coverage of insecticide-treated bed nets, appropriate treatment for pregnant women, and indoor residual spraying. Although there were data limitations, there was preliminary evidence to suggest that: (c) Some countries had experienced a possible decline in HIV incidence rates among young people — for instance, mathematical modeling with HIV prevalence and sexual behavior trend data showed that three countries (Malawi, Tanzania, Zambia) offered evidence suggestive of a decline since 2003. 112 (d) There had been an increase in the survival rate among people on ART, with the number of adult life years added due to ART estimated to have increased from just 6,607 in 2003 to 576,438 in 2007 in the 18 countries. (e) A few countries (such as Rwanda and Zambia) provided evidence of reductions in parasite prevalence and a potential decline in malaria-attributed child mortality. 5.61 Using modeling and using the coverage of the interventions as the main input for 11 of the evaluation study countries, Study Area 3 estimated that 110,000 lives had been saved by insecticide-treated bed nets and 24,000 by intermittent treatment of pregnant women.104 Study Area 3 also emphasized areas of slow progress (for example, ACT treatment for malaria), as well as intervention areas requiring greater attention — for instance, gaps in basic requirements such as trained personnel, guidelines, medicines, and equipment (HIV/AIDS), and scope for improving the quality of diagnostic and treatment services (tuberculosis). 5.62 Study Area 2 was expected to generate determinants of grant success (good outcomes), including country context and the strength of the Global Fund partnership on the ground. These would have indicated the conditions required for the most successful outcomes, which could then have been corroborated by the results of Study Area 3. But Study Area 2 was unable to deliver on this score, because the countries selected were all in the moderate-performers range. 5.63 Conclusions. Study Area 3 was not an impact evaluation in spite of the title, nor did it set out to be one. This was clear, because there was no attempt at attribution. If the contribution analysis framework is applied, the Study Area 3 reports did not sufficiently take into account contextual factors about the collective action of the different donor and country programs. If these factors were covered in other evaluation studies accompanying the FYE (of which there were many), they were not referred to. 5.64 Regardless of the approach (impact evaluation or contribution analysis), it was too early for the Global Fund to conduct an assessment of the scaled-up efforts to change behavior and reduce disease burden. The usefulness of such a resource-intensive exercise for a young program105 needs to be seriously considered. It clearly takes time to realize and document the full health impact of such interventions, especially considering the lag between funding and implementation, and the necessary data collection and reporting. The contribution of collective efforts to changing behaviors and reducing the disease burden needs to be interpreted with this in mind. Conducting a multi-level program-wide evaluation like the FYE is an enormous enterprise, especially for GRPPs, given the diversity of components and the resulting complex causality and aggregation issues (IEG and OECD/DAC 2007, p. 95). Nonetheless, impact evaluations may be valuable in helping to 104. The model also indicated that a significant part of this positive effect may have been offset by children with malaria getting less treatment in the Democratic Republic of Congo, where there were an additional 90,000 deaths. 105. See also the life-cycle approach to determining the scope of a GRPP evaluation in Tables 5 and 6 of the IEG and OECD/DAC 2007, pp. 34–35. 113 identify the impacts of interventions and key causal linkages for subsets of activities where impacts are more measurable than for the program as a whole. 5.65 TERG has proposed a different approach going forward. Evaluations of scale-up and impact will be conducted each year in a selected number of countries, building on the Study Area 3 experience, and with sufficient preparation time to involve development partner agencies and to integrate with country health information systems. Evaluation Capacity Building in Study Area 3 5.66 The TERG wanted to foster different perspectives and approaches toward organizational learning throughout the Global Fund system. Therefore, an important objective of the FYE was to strengthen country evaluation systems and capacity during Study Area 3, so that countries would continue carrying out impact measurements using harmonized tools and approaches, developed for their use, after the conclusion of the FYE. IETFs comprising country stakeholders106 were set up in 20 countries. These devised an evaluation work plan for Study Area 3 and oversaw its implementation in their respective countries.107 The evaluation consortium provided technical assistance and on-the-job training in data collection and analysis, surveillance, study protocols, and tools to 50 local institutions and individuals, and then subcontracted with them to collect and analyze the data. Subsequently, countries were expected to have the capacity and collective experience to replicate (in whole or in part) the same tools and procedures to measure trends, after the FYE. 5.67 Study Area 3 operated on a tight schedule that generally did not allow for adequate stakeholder involvement, consensus building among the different IETF members, nor the use of evaluation findings in country planning exercises.108 Some of the IETFs had high expectations of being full partners in the evaluation process. When this did not happen, it adversely affected the relationships between the IETFs and the FYE evaluation teams. Thus, in spite of the developmental focus of Study Area 3 (30 percent of the Study Area 3 budget was spent on capacity building), it was largely viewed as a Global Fund product with low ownership by country-level stakeholders. Stronger ownership by Study Area 3 countries would have been necessary to sustain the use of FYE evaluation tools and techniques. 5.68 The total cost of the FYE of $16.2 million represented 1 percent of the average annual expenditures (including grant disbursements) of the Global Fund in 2007 and 2008. This ratio is consistent with program-level evaluations of other GRPPs.109 As planned, 70 percent of the $11.7 million evaluation budget for Study Area 3 was spent supporting country institutions: 40 percent on data collection/analysis, and 30 percent on technical assistance and training.110 106. Assembled by UNAIDS, they were derived from groups normally engaged in health measurements: relevant development partners, government agencies (Ministry of Health, Bureau of Statistics), and civil society. IETFs also included members of CCMs. 107. Work plans included the use of Study Area 3 results in the countries’ health sector reviews and sector planning exercises. 108. USAID has continued to finance some of the Partners in Learning Forums since the FYE. 109. IEG 2011b, The World Bank’s Involvement in Global and Regional Partnership Programs, p. 26. 110. Board Documents, TERG presentation to 18th Board Meeting. 114 Whether this represents value-for-money clearly depends on the value placed on learning, capacity building, and country ownership. The Global Fund Board clearly did place value on these when it commissioned the evaluation, judging by the five guiding principles for the evaluation, two of which focused on learning and participation.111 5.69 The evaluation has contributed to the availability of approaches and tools for improving the quality of routine and survey data systems. Evaluation by-products also include a data depository of raw data from the country studies on the Global Fund Web site and a “Model Evaluation Platform� — a package of evaluation tools and lessons drawn from the Study Area 3 experience. TERG reports have talked about attempts to standardize and harmonize with other evaluation tools, such as the Health Metrics Network, WHO’s Country Health Systems Surveillance, and the IHP+ Evaluation Platform, and to make available the Model Evaluation Platform as an open source resource available for all to use, copy, and modify. It is not known to what extent the specialized training, country data, and knowledge generated by Study Area 3 have been tapped by countries, researchers and academics as intended in the FYE design. There is little indication that the Model Evaluation Platform has moved significantly beyond the conceptual stage, or that the large datasets amassed by the FYE have been tapped by researchers and academics as intended. 5.70 Conclusion. It is extremely difficult to implement and sustain systematic capacity building in the context of a one-off evaluation like the FYE. Other GRPPs should only attempt to do so with caution. Building M&E capacity is a long-term endeavor that is better undertaken through more conventional approaches, given the condensed schedule in a global program evaluation. External evaluations emphasize independence and objectivity, while capacity building emphasizes learning and strong engagement with the implementing bodies. Managing the inherent tensions between these principles was a challenge for the FYE evaluation team. Greater value-for-money could potentially have been achieved if the same resources had been used to build capacity prior to the FYE, rather than as an integral part of it. 111. Learning and capacity building: The evaluation is designed not only as an external audit of performance, but also to support learning and capacity building in close partnership with countries. Capacity building efforts must focus on improving countries’ existing data collection and analysis mechanisms or building these mechanisms where they do not exist. Country-driven processes: The evaluation supports the principles of coordinated program M&E processes and all efforts are to be made to avoid duplication and fragmentation in order to promote national M&E goals. Further, the evaluation must balance the principle of country ownership with the need for independence and maximize the use of existing data and information systems. 115 6. Conclusions and Lessons 6.1 The Global Fund was officially established in January 2002 “to attract, manage and disburse additional resources through a new public-private partnership that will make a sustainable and significant contribution to the reduction of infections, illness and death, thereby mitigating the impact caused by HIV/AIDS, tuberculosis and malaria in countries in need, and contributing to poverty reduction as part of the Millennium Development Goals.� Since then, the Global Fund has become the largest of the 120 GRPPs in which the World Bank is involved. It disbursed more than $3 billion in grants to developing and transition countries in 2010, and is supported by the largest FIF currently administered by the World Bank. Other such programs include the GAVI (established 1999), the Global Partnership for Education (2002), the Climate Investment Funds (2008), and the Global Agriculture and Food Security Program (2010). 6.2 Large partnership programs such as the Global Fund that are financing country-level investments on a large scale have several common features. First, they pool donor resources to finance country-level investments, which distinguishes them from the large majority of much smaller GRPPs that are primarily financing technical assistance, or generating knowledge about development. Second, they employ inclusive governance structures in which membership on the governing body is not limited to financial contributors but also extended to other stakeholders, including recipient countries, CSOs, and the commercial private sector. Third, they generally subscribe to the 2005 Paris Declaration principles of country ownership, alignment, harmonization, managing for results, and mutual accountability. The programs also raise funds from nontraditional sources outside the public sector, including private foundations and the business community. 6.3 As the largest of these programs, the Global Fund has become a basis for comparison not only for the programs listed above, but also for other global funds that were established a decade earlier, such as the GEF. Thus, the experience of the Global Fund provides lessons not only for the Fund itself, but also for these other programs, for the engagement of the World Bank with these programs, and for evaluating GRPPs more generally. Lessons for the Global Fund 6.4 Harmonization. The Global Fund is facilitating donor coordination at the point at which donors contribute to the trust fund and serve on the Global Fund Board, but this has not yet translated into a similar degree of coordination at the country level. Country-level stakeholders tend to regard the Global Fund as another, largely separate development partner agency with its own distinct modalities that have not been well integrated into existing donor coordination mechanisms in the countries, or with national budget cycles, contrary to the harmonization principle of the Paris Declaration. While this situation may improve as the Health Systems Funding Platform matures and as the Global Fund transitions its grant portfolio to single streams of funding under its new grant architecture, the Global Fund has not generally contributed to harmonization through existing mechanisms for pooling funds at the country level, such as SWAps, first introduced in the 1990s as a means to overcome inefficiencies and reduce transactions costs to the country. 116 6.5 Technical Support to Enhance Country Ownership. Development partners need to provide greater technical support to strengthen the ability of governments to effectively coordinate donor efforts around agreed national strategies. This Review found that the situation has generally improved since the FYE in terms of other partners’ providing technical assistance in support of Global Fund activities. The Global Fund has also developed a new partnership strategy, signed MOUs with the Stop TB Partnership and Roll Back Malaria in 2009 and 2010, respectively, and is reaching out to other development partner agencies more generally. However, the Global Fund needs to find ways to finance such technical assistance, provide it directly, or work effectively with other development partner agencies to do so. 6.6 Sustaining the Benefits of Global Fund Support. The long-term sustainability of the benefits of Global Fund-supported activities depends on the complementary activities of donor partners and strengthening the capacity of recipient countries. This will require a substantially more coordinated approach to external financial support at both the global and country levels than has occurred to date. It will be difficult for the Global Fund “to adjust its demand-driven model� to support “the most cost-effective interventions tailored to the type and local context of specific epidemics,� as recommended by the FYE (Macro International 2009b, p. 18), if it ends up becoming the residual financier financing others’ shortfalls. The scarce resources available to fight the three diseases — including those raised by the country from its own resources and those provided by its external partners, including the World Bank — need to be allocated collectively and proactively in each country in accordance with an agreed long-term strategy for fighting each disease. The sustainability of resources to support people living with AIDS who are already receiving antiretroviral treatment is of particular concern, since interrupted treatment also increases the risks of new infections and drug resistance. The long-run affordability and sustainability of treatment also depends on financing effective prevention programs to prevent new HIV/AIDS cases. 6.7 Managing for Results. The M&E requirements of different development partners have so far thwarted their good intentions to coordinate and streamline M&E for the three diseases at the country level. The Global Fund, the World Bank, and other agencies have endorsed the Three Ones principles of a common action framework, a single coordinating authority, and one M&E framework to monitor collective efforts in each disease area. They jointly prepared an M&E Toolkit in 2004 (revised in 2006, 2009, and 2011) to establish norms and identify indicators to be used by all the agencies, but it has been difficult to achieve their use in practice because each agency has its own project-level M&E requirements. Both the Global Fund and the World Bank could contribute to improved M&E at the project and country levels by making a stronger commitment to the Three Ones principles. Project-level M&E could focus on accountability for achieving the specific outputs of each project, and country-level M&E on tracking the higher-level outcomes and impacts collectively. 6.8 Managing Conflicts of Interest. Real and perceived conflicts of interest are an inherent and essentially unavoidable feature of all partnership programs, deriving in the first instance from the multiple roles that the key partners play in a given program. The Global Fund has brought recipient countries, CSOs, and affected communities into its governance arrangements at both the global and country levels. It has also established independent 117 review processes at key stages in its operations such as the reviewing of grant proposals (by the TRP), verification and reporting on grant performance (by the LFAs), and overseeing evaluations (by the TERG). It has also established conflict of interest guidelines for the operation of CCMs. The key is to identify and manage potential conflicts of interest in a way that does not impede the effectiveness of the program. Reconciling these two imperatives will remain a continuing challenge for the Global Fund and for other GRPPs. 6.9 Global Public Policy. Neither the Global Fund nor the World Bank can address by itself “global communicable disease governance issues� such as the risk of drug resistance for current treatments of the three diseases. This Review found that drug resistance is a live issue in the countries visited, amplified by incomplete treatments and the presence of counterfeit drugs. Global Fund grants could help strengthen the capacity of drug regulatory and enforcement agencies in assuring quality compliance by the pharmaceutical industry, and CCMs could invite drug regulatory agencies to participate in specialized committees of the CCMs. The Global Fund and the World Bank also need to support ongoing efforts by organizations with relevant competence, such as WHO and the United Nations Office on Drugs and Crime, to ensure that the sizable investments that the world has made in combating the three diseases are not diminished by inaction in this area. Lessons for the World Bank 6.10 This Global Program Review has confirmed findings of previous IEG reviews on global partnership programs and trust funds in the following three areas. 6.11 Financial Intermediary Trust Funds. This Review provides evidence to support IEG’s recent recommendation that “the Bank should strengthen its framework for guiding its acceptance and management of FIFs going forward� (IEG 2011a, p. 85). Like other FIFs, the Global Fund trust fund was established in an ad hoc way in 2001–02 to accommodate the particular requirements of the Global Fund and its donors. This has resulted in some ambiguities in the relationship between the Bank and the Global Fund. For example, the trust fund management agreement was crafted to limit the Bank’s responsibility for the development outcomes of the use of trust fund resources, yet Global Fund donors expected that the Bank would contribute technical assistance to Global Fund-supported activities at the country level. Also, the Bank’s accountability for the effective governance of the Global Fund as a permanent nonvoting institutional member of the Board has not been clarified. The Bank is currently in the process of preparing a stronger framework for the acceptance and management of FIFs, along the lines recommended by IEG. 6.12 Engagement Strategy. This Review also provides evidence to support IEG’s recent recommendation that “the Bank should have an explicit engagement strategy for each GRPP in which it is involved, including . . . the expected roles of the Bank in the program at both the global and country levels, . . . how the program’s activities are expected to be linked with the Bank’s country operations, and how the risks to the Bank’s participation will be identified and managed� (IEG 2011b, p 101). This Review has found that the Bank has been actively engaged in the corporate governance of the Global Fund and with Global Fund- supported activities in about 65 countries, in addition to being the trustee of the Global Fund trust fund. Yet the trustee role has been the only one of the Bank’s roles in which the Bank’s 118 contributions to and expectations of the relationship have been expressed, so that the trustee relationship is bearing the burden of the Bank’s entire engagement with the Global Fund, which it was not designed to do. It would be better for the Bank to have a more complete engagement strategy with the Global Fund that encompasses all the roles that the Bank plays in the partnership. This would include guidance to country-level Bank staff for engaging with Global Fund-supported activities at the country level. 6.13 The Bank is in the process of preparing a new partnership framework for the Bank’s engagement with GRPPs more generally. The Bank’s 2007 Health Strategy also provides general statements about its engagement with the Global Fund. However, something more than these general statements is also needed to provide guidance to country teams and Bank staff. The Global Fund will likely continue to disburse for communicable disease control more than what the Bank disburses for the entire health sector. Nine years of experience have shown that the Bank can contribute meaningfully to the work of the Global Fund at the country level without taking on supervisory or operational roles. Undertaking such roles — as the Bank currently performs for the Global Environment Facility — might also be considered on a pilot basis under certain circumstances, such as a SWAp operation or a common implementing agency (Principal Recipient). The Global Fund or its donors could also establish a trust fund at the World Bank for financing Bank-supervised technical assistance in support of Global Fund-supported activities, following the precedents of UNAIDS for the Global HIV/AIDS Program and WHO for the International Health Partnership. 6.14 Community of Practice. The Bank could establish a community of practice among its project managers who are working with the Global Fund to learn cross-cutting lessons of experience. This would be similar to the regionally coordinated community of practice that currently exists for the Bank’s engagement with the Global Environment Facility. Such a community of practice could lead, among other things, to standard terms of reference for Bank staff serving on CCMs, and could be supported by a central database to keep track of the Bank’s engagement with the Global Fund over time. As many have observed, “what gets measured, gets done.� Lessons for the Evaluation of Global and Regional Partnership Programs 6.15 Early Stage Evaluations. Formative evaluations, like Study Areas 1 and 2 of the FYE, are more useful in the early stages of a global program in helping the program make strategic adjustments to its organizational and institutional arrangements than the contribution analysis that was undertaken in Study Area 3. Furthermore, the diversity of components in a global or regional program and the resulting complex causality and aggregation issues by their nature make impact evaluation difficult, if not infeasible. Nonetheless, impact evaluations may be valuable in helping to identify the impacts of interventions and key causal linkages for subsets of activities where impacts are more measurable than for the program as a whole. 6.16 Project-Level Monitoring. Good monitoring systems should not only assess progress in implementing activities but also contribute to periodic summative evaluations and to effective policy dialogue. The Global Fund has established different objectives for M&E at 119 the grant, country, and corporate levels, yet the three levels are not well connected with each other. Its grant-level M&E system is designed more to facilitate its PBF approach to grant disbursements than to contribute to an overall assessment of the outcomes of the program or to policy dialogue. The only country-level evaluations that it has so far undertaken are the 18 country assessments for Study Area 3 of the FYE. The Global Fund could consider undertaking evaluations of a random sample of the single streams of funding for each disease now taking place under its new grant architecture. The Global Fund might also institutionalize regular country-level evaluations, the results of which could feed into, rather than be part of, subsequent evaluations of the overall program. This would also help build the knowledge base about which approaches most successfully contribute to achieving collective outcomes. 6.17 Objectives and Scope of Global Program Evaluations. These are best kept to a manageable size, consistent with the most immediate evaluation needs of the program — allowing for realistic schedules and avoiding evaluation fatigue and conflicts with other evaluation efforts in countries. Large numbers of upstream processes built into the evaluation design can distract instead of facilitate the evaluation process. Sufficient time should also be allowed to adequately pretest new evaluation instruments. 6.18 Participatory Evaluation. Participatory evaluations that engage country partners need to manage expectations, since unmet expectations dampen country ownership of the evaluation process and of the end product. Evaluation schedules should be realistic and allow for productive exchanges and consultation between evaluation teams and country partners. Otherwise, country partners may perceive their roles as largely collecting critical data, with little involvement in the analysis and deliberations about their significance. 6.19 Evaluation Capacity Building. Development activities such as building country-level evaluation capacity within the context of a global program evaluation are commendable but difficult to implement and sustain in the context of a one-off evaluation. Building M&E capacity is a long-term endeavor that is better undertaken through more conventional approaches, given the condensed schedule in a global program evaluation. The tension between the two objectives can be very pronounced: an external evaluation emphasizes independence and objectivity, while capacity building emphasizes learning and strong engagement with the implementing bodies. 121 Bibliography ASAP (AIDS Strategy and Action Plan Service) and UNAIDS. 2008. “The HIV Epidemic in Tanzania Mainland: Where Have We Come from, Where Are We Going, and How Are We Responding? Final Report.� World Bank, Washington, DC. Baker, J. L. 2000. Evaluating the Impact of Development Projects on Poverty: A Handbook for Practitioners. Washington, DC: The World Bank. Booz, Allen and Hamilton. 2007. “Organizational & Management Review.� Geneva: The Global Fund. www.theglobalfund.org/documents/board/16/BM16_03EDReport_Appendix_en/ Cashin, Cheryl. Forthcoming. “Comparison of the Monitoring and Evaluation Systems of the World Bank and Global Fund for the Global Program Review of the Global Fund to Fight AIDS, Tuberculosis and Malaria.� Prepared for this Review and forthcoming as an IEG Working Paper. IEG, Washington, DC. Center for Global Development. 2006. Will We Ever Learn? Improving the Lives through Impact Evaluations. Report of the Evaluation Gap Working Group. Washington, DC: Center for Global Development. Center for Strategic and International Studies. 2010. “Global Health Policies: The US Should Slow the Growth of its Bilateral AIDS Treatment Commitments.� Debate Series, CSIS, Washington, DC. Chee, Grace, Vivikka Molldrem, Natasha Hsi, and Slavea Chankova. October 2008. “Evaluation of GAVI Phase 1 Performance.� Abt Associates Inc., Bethesda, MD. Dubinsky, Joan Elise. 2008. “Ethics and Reputational Risk Assessment: The Global Fund.� The Rosentreter Group, Kensington, MD. Duflo, E. 2001. “Schooling and Labor Market Consequences of School Construction in Indonesia: Evidence from an Unusual Policy Experiment.� American Economic Review 91(4): 795-813. Galasso, E., and M. Ravallion. 2004. “Social Protection in a Crisis: Argentina’s Plan Jefes y Jefas.� World Bank Economic Review 18 (3): 367–400. GAO (United States Government Accountability Office). 2005. “Global Health: The Global Fund to Fight AIDS, TB and Malaria Is Responding to Challenges but Needs Better Information and Documentation for Performance-Based Funding.� Report to Congressional Committee. GOA-05-639. Government Printing Office, Washington, DC. www.theglobalfund.org/Documents/library/Library_IE12_Report_en/ ———. 2007. “Global Health: Global Fund to Fight AIDS, TB and Malaria Has Improved Its Documentation of Funding Decisions but Needs Standardized Oversight Expectations and Assessments.� Report to Congressional Committee. GOA-07-627. Government Printing Office, Washington, DC. www.theglobalfund.org/Documents/library/Library_SEd07627_Report_en/ GEF (Global Environment Facility). 2007. “Operational Guidelines for the Application of the Incremental Cost Principle.� GEF Council Document GEF/C.31/12, Washington, DC. ———. 2011. “Broadening the GEF Partnership Under Paragraph 28 of the GEF Instrument.� GEF/C.40/09, Washington, DC. 122 GEF (Global Environment Facility) Evaluation Office. 2006. “Evaluation of Incremental Cost Assessment.� GEF Council Document GEF/ME/C.30/2, Washington, DC. ———. 2009. Mid-Term Review of the Resource Allocation Framework. GEF Evaluation Report No. 47. Washington, DC: GEF Evaluation Office. ———. 2010. The GEF Monitoring and Evaluation Policy 2010. Washington, DC: GEF Evaluation Office. ———. 2010. OPS4: Progress Toward Impact. Fourth Overall Performance Study of the GEF. Washington, DC: GEF Evaluation Office. Global Fund to Fight AIDS, Tuberculosis and Malaria. 2002/2003 to 2009. Annual Reports. The Global Fund, Geneva. http://www.theglobalfund.org/en/library/publications/annualreports/ ———.2002/2003 to 2010. Board Meeting Documents. The Global Fund, Geneva. www.theglobalfund.org/en/board/meetings/ ———.2004 to 2010. TERG Meeting Reports. The Global Fund, Geneva. www.theglobalfund.org/en/terg/reports/ ———. n.d. “Guidance Paper on CCM Oversight.� The Global Fund, Geneva. www.theglobalfund.org/documents/ccm/CCM_CCMOversightGuidance_Paper_en/ ———. 2002. “The Framework Document of the Global Fund to Fight AIDS, Tuberculosis and Malaria.� The Global Fund, Geneva. www.theglobalfund.org/Documents/core/framework/Core_GlobalFund_Framework_en/ ———. 2004. “Summary Report of the Technical Evaluation Reference Group (TERG) of the Global Fund: First Meeting.� The Global Fund, Geneva. ———. 2005. “Technical Evaluation Reference Group Report on the Assessment of Country Coordination Mechanisms: Performance Baseline.� Geneva: The Global Fund. www.theglobalfund.org/en/ccm/guidelines/ ———. 2006a. 360° Stakeholder Assessment: Perceptions and Opinions of Stakeholders of the Global Fund. Geneva: The Global Fund. www.theglobalfund.org/en/terg/evaluations/360/ ———. 2006b. Technical Background Document on the Scale and Scope of the Five-Year Evaluation of the Global Fund to Fight AIDS, Tuberculosis and Malaria. Geneva: The Global Fund. www.theglobalfund.org/en/terg/evaluations/5year/ ———. 2007. “Evaluation of the Local Fund Agent System.� Euro Health Group for the Global Fund, Geneva. www.theglobalfund.org/en/library/evaluationlibrary/specificevaluations/ ———. 2008. Lessons Learned in the Field: Health Financing and Governance. A Report on the Country Coordination Mechanism Model. Geneva: The Global Fund. ———. 2009a. “Global Fund Aid Effectiveness Scorecard.� http://www.theglobalfund.org/en/performance/effectiveness/aideffectiveness/measuring/ ———. 2009c. Synthesis Report of the Five-year Evaluation of the Global Fund. Technical Evaluation Reference Group Summary Paper. Geneva: The Global Fund. www.theglobalfund.org/en/terg/evaluations/5year/sr/ ———. 2009d. Office of the Inspector-General, Report on Lessons Learnt from the Country Audits and Reviews Undertaken. Report No: TGF-OIG-09-002. Geneva: The Global Fund. 123 ———. 2009e. Office of the Inspector-General, “Audit Report on Global Fund Grants to Tanzania.� Report No.: TGF-OIG-09-001. The Global Fund, Geneva. www.theglobalfund.org/.../oig/OIG_TanzaniaCountryAudit_Report_en/ - ———. 2010a. “Changes to the Global Fund Grant Architecture: a Fact Sheet for Implementers.� www.theglobalfund.org/documents/grantarchitecture/Fact_Sheet_for_Implementers_en.p ———. 2010b. “The Five-Year Evaluation: An Update.� The Global Fund Third Replenishment (2011–2013). The Global Fund, Geneva. ———. 2010c. Office of the Inspector-General, “Audit Report on Global Fund Grants to Nepal.� Audit Report No: TGF-OIG-09-006. The Global Fund, Geneva. ———. 2010d. Office of the Inspector-General, “Country Audit of the Global Fund Grants to Cambodia.� Audit Report No: TGF-OIG-09-014.� The Global Fund, Geneva. ———. 2010e. Office of the Inspector-General. “The OIG Review of the Global Fund Grant Application Process.� Audit Report No: TGF-OIG-10-001. The Global Fund, Geneva. ———. 2010f. Office of the Inspector-General. “Review of Oversight of Grant Procurement and Supply Chain Management Arrangements.� Audit Report No: TGF-10-002. The Global Fund, Geneva. ———. 2010g. “Recommendations to Enhance In-Country Communications between the Secretariat, LFA, PR, CCM and Other Partners.� The Global Fund, Geneva, May 11. ———. 2011a. “The Global Fund’s Approach to Health Systems Strengthening (HSS): Information Note.� The Global Fund, Geneva, July. ———. 2011b. Turning the Page from Emergency to Sustainability: The Final Report of the High- Level Independent Review Panel on Fiduciary Controls and Oversight Mechanisms of the Global Fund to Fight AIDS, Tuberculosis and Malaria. Geneva: The Global Fund. The Global Fund, Health Metrics Network, PEPFAR, Roll Back Malaria Partnership, Stop TB Partnership, UNAIDS, USAID, U.S. Department of Health and Human Services/Centers for Disease Control and Prevention, WHO, and the World Bank. 2004. Monitoring and Evaluation Toolkit: HIV, Tuberculosis, and Malaria. Geneva: The Global Fund. ———. 2006. Monitoring and Evaluation Toolkit: HIV, Tuberculosis, and Malaria. Geneva: The Global Fund. ———. 2009. Monitoring and Evaluation Toolkit: HIV, Tuberculosis, and Malaria and Health Systems Strengthening. Geneva: The Global Fund. Global Partnership on Output-Based Aid. 2010. “Output-Based Aid—A Fact Sheet.� GOPA, Washington, DC. Goosby, Eric. 2010. “PEPFAR’s Partnership with the Global Fund Improves the Response to HIV/AIDS.� http://www.pepfar.gov/countries/cambodia/index.htm Habicht, J.P., C.G. Victora, and J.P. Vaughan. 1999. “Evaluation Designs for Adequacy, Plausibility and Probability for Public Health Program Performance and Impact.� International Journal of Epidemiology 28: 10–18. Haverkamp, C. and K. Janovsky. 2007. Medium-Term Review: Health Sector Strategic Plan 2003– 2007. An Assessment of Progress under Sector Wide Management (SWIM) and Recommendations to the MOH and Health Partners for Improving Harmonization and Alignment in the Health Sector. Consolidated Report Phase 1 and 2. Phnom Penh, Cambodia, 124 and Washington, DC: Ministry of Health of Cambodia. http://www.pepfar.gov/countries/cambodia/index.htm Hendricks, M. 1996. “Performance Monitoring: How to Measure Effectively the Results of Our Efforts.� Presented at the American Evaluation Association Annual Conference, Atlanta, November 6. IEG (Independent Evaluation Group) and OECD/DAC (Organisation for Economic Co-operation and Development/Development Assistance Committee), Network on Development Evaluation. 2007. Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards. Washington, DC: World Bank. IEG. 2001. Investing in Health: Development Effectiveness in the Health, Nutrition, and Population Sector. Washington, DC: World Bank. ———. 2005. Committing to Results: Improving the Effectiveness of HIV/AIDS Assistance: An OED Evaluation of the World Bank’s Assistance for HIV/AIDS Control. Washington, DC: World Bank. ———. 2006a. “Impact Evaluation: The Experience of the Independent Evaluation Group of the World Bank.� IEG ECD Series, World Bank, Washington, DC. ———. 2006b. Conducting Quality Impact Evaluations under Budget, Time and Data Constraints. Washington, DC: The World Bank. ———. 2007, The Development Potential of Regional Programs: An Evaluation of World Bank Support of Multicountry Operations. Washington, DC: The World Bank. ———. 2009. Improving Effectiveness and Outcomes for the Poor in Health, Nutrition, and Population: An Evaluation of World Bank Group Support Since 1997. Washington, DC: World Bank. ———. 2010. “Lesotho Health Sector Reforms Project and HIV and AIDS Capacity Building and Technical Assistance Project.� Project Performance Assessment Report. IEG, Washington, DC. ———. 2011a. Trust Fund Support for Development: An Evaluation of the World Bank’s Trust Fund Portfolio. Washington, DC: World Bank. ———. 2011b. The World Bank’s Involvement in Global and Regional Partnership Programs: An Independent Assessment. Washington, DC: World Bank. Institute for Health Sector Development. 2003. “Independent External Evaluation of the Global Stop TB Partnership: Report.� Prepared for the Stop TB Partnership, Geneva. http://www.stoptb.org/resources/publications/achievement_evaluations.asp IOE (International Organization of Employers) and PEC (Pan-African Employers’ Confederation). 2009. HIV/AIDS Challenges in the Workplace: Responses by Employers’ Organizations and Their Members in Africa. Case Studies and Good Practice. Geneva: IOE. Kabell Consulting ApS. 2008. Evaluation of the Implementation of the Paris Declaration. Synthesis Report of Phase 1. Copenhagen: Denmark Ministry of Foreign Affairs. www.evaluation.dk Kaiser Family Foundation. Various years. U.S. Global Health Policy Fact Sheets. www.kff.org ———. 2009-2011. “U.S. Global Health Initiative.� www.kff.org/globalhealth/factsheets.cfm ———. 2009-2011. “U.S. President’s Emergency Plan for AIDS Relief.� www.kff.org/globalhealth/factsheets.cfm 125 ———. 2009-2010. “The U.S. and the Global Fund.� www.kff.org/globalhealth/factsheets.cfm ———. 2011. “U.S. Funding for the Global Health Initiative: The President’s FY 2012 Budget Request.� www.kff.org/globalhealth/factsheets.cfm Kress, Dan, and R. Paul Shaw. 2009. “GAVI and Global Fund Joint Programming for Health Strengthening: Turf Wars or an Opportunity to Do Better?� The Lancet. 2009. “Maintaining Momentum for Malaria Elimination.� The Lancet 374 (9686): 266. Macro International. 2007a. Evaluation of the Organizational Effectiveness and Efficiency of the Global Fund to Fight AIDS, Tuberculosis, and Malaria: Results from Study Area 1 of the Five-Year Evaluation. Report submitted to the Global Fund to Fight AIDS, Tuberculosis, and Malaria. Geneva: The Global Fund. ———. 2007b. Global Fund Five-year Evaluation Study Areas 1 and 2: Inception Report Summary. Geneva: The Global Fund. ———. 2008. Evaluation of the Global Fund Partner Environment, at Global and Country Levels, in Relation to Grant Performance and Health Systems Effect, Including 16 Country Studies: Final Report. Report submitted to the Global Fund to Fight AIDS, Tuberculosis, and Malaria. Geneva: The Global Fund. ———. 2009a. Global Fund Five-year Evaluation: Study Area 3: the Impact of Collective Efforts on the Reduction of the Disease Burden of AIDS, Tuberculosis, and Malaria. Report submitted to the Global Fund to Fight AIDS, Tuberculosis, and Malaria. Geneva: The Global Fund. ———. 2009b. The Five-Year Evaluation of the Global Fund to Fight AIDS, Tuberculosis, and Malaria: Synthesis of Study Areas 1, 2 and 3. Geneva: The Global Fund. Martin, Gayle H. 2010. “Portfolio Review of World Bank Lending for Communicable Disease Control.� IEG Working Paper No. 3, Washington, DC. Mayne, J. 1999. “Addressing Attribution through Contribution Analysis: Using Performance Measures Sensibly: Discussion Paper.� Discussion Paper, Office of the Auditor General of Canada, Ottawa. ———. 2001. “Addressing Attribution through Contribution Analysis: Using Performance Measures Sensibly.� The Canadian Journal of Program Evaluation 16(1): 1–24. Mosha, F., W. Urassa, S. Aboud, E. Lyamuya, E. Sandstrom, H. Bredell, and C. Williamson. 2011. “Prevalence of Genotypic Resistance to Antiretroviral Drugs in Treatment-naïve Youths Infected with Diverse HIV Type I Subtypes and Recombinant Forms in Dar es Salaam, Tanzania.� AIDS Research and Human Retroviruses Apr 27(4): 377–82; Epub, 2010, Oct 18. Osili, U.O., and B.T. Long. 2007. “Does Female Schooling Reduce Fertility? Evidence from Nigeria.� NBER Working Paper 13070. NBER, Cambridge, MA. PBS Online Newshour. 2009. “Fake and Substandard Drugs Threaten Malaria Treatment in Cambodia.� July 24, 2009. www.pbs.org/newshour/updates/health/july-dec09/malaria_0724.html Pharma Industry/Biotech Industry. 2009. “Africa Should Manufacture Generic HIV/AIDS Drugs, UNAIDS Head Sidibe Says.� Pharma Industry/Biotech Industry 24 April. www.medicalnewstoday.com/releases/147437.php Piehl, A., S. Cooper, A. Braga, and D. Kennedy. 2003. “Testing for Structural Breaks in the Evaluation of Programs.� Review of Economics and Statistics 85(3): 550-58. 126 Poate, Derek, Paul Balogun, and Kathy Attawell. 2009. UNAIDS Second Independent Evaluation 2002–2008: Final Report. Document prepared for the 25th Meeting of the UNAIDS Programme Coordinating Board, Geneva, Switzerland, 8-10 December. Purdue University, College of Pharmacy. 2011. “Sustainable Medicine Program in Tanzania.� http://www.ipph.purdue.edu/sustain/ Radelet, Steven. 2004. The Global Fund to Fight AIDS, Tuberculosis and Malaria: Progress, Potential, and Challenges for the Future. Washington, DC: Center for Global Development. Ravallion, M. 2008a. “Evaluating Anti-Poverty Programs.� In Handbook of Development Economics, Vol. 4. Amsterdam: North-Holland. ———. 2008b. “Evaluating Anti-poverty Programs.� In Handbook of Development Economics, Vol. 4. Amsterdam: North-Holland. Ritchie, Daniel, and Celina Schocken. 2007. “Incentives for Collaboration and Harmonization among the Global Fund, PEPFAR and the World Bank on HIV/AIDS: Final Report.� Global Fund, PEPFAR, and World Bank, Geneva and Washington, DC. Roll Bank Malaria. 2011. “Strengthening of Drug Regulatory Authorities, Conducting Surveillance to Remove Substandard and Counterfeit Drugs, and Strengthen Logistics and Supply Chain Management for ACTs.� Special Session of African Ministers of Health at Board Meeting of RBM Partnership, May 2011, Geneva. www.rbm.who.int/calendarofevents.html Shakow, Alexander. 2006. “Global Fund–World Bank HIV/AIDS Programs: Comparative Advantage Study.� Report prepared for the Global Fund and The World Bank HIV/AIDS Program.� Washington. DC, and Geneva. http://siteresources.worldbank.org/INTHIVAIDS/Resources/375798- 1103037153392/GFWBReportFinalVersion.pdf Smithsonian Magazine. October 2009. “The Fatal Consequences of Counterfeit Drugs.� Washington, DC. www.smithsonianmag.com/people-places/Prescription-for-Murder.html Specter, Michael. 2005. “What Money Can Buy.� The New Yorker, October 24. Taskforce on Innovative International Financing for Health Systems. September 2009. More Money for Health, and More Health for the Money. http://www.internationalhealthpartnership.net/pdf/IHP%20Update%2013/Taskforce/Johansbourg/Final %20Taskforce%20Report.pdf UNAIDS. 2005. “Global Task Team on Improving Coordination Among Multilateral Institutions and International Donors: Final Report.� UNAIDS, Geneva. ———. 2007. Country Harmonization and Alignment Tool (CHAT): A Tool to Address Harmonization and Alignment Challenges by Assessing Strengths and Effectiveness of Partnerships in the National AIDS Response. Geneva: UNAIDS. ———. 2010. Report on the Global AIDS Epidemic 2010. Geneva: UNAIDS. UNAIDS and WHO (World Health Organization). 2009. AIDS Epidemic Update. Geneva: UNAIDS. UNIDO (UN Industrial Development Organization) Global Project. 2010. Strengthening the Local Production of Essential Medicines in Developing Countries. Vienna: UNIDO. Vaillancourt, Denise. “Do Health Sector-Wide Approaches Achieve Results? Emerging Evidence and Lessons from Six Countries.� IEG Working Paper No. 4, IEG, Washington, DC. 127 Victora, C.G., R.E. Black, J.T. Boerma, and J. Bryce. 2010. “Measuring Impact in the Millennium Development Goal Era and Beyond: A New Approach to Large-scale Effectiveness Evaluations.� Lancet, 377: 85-95. Wolfensohn, James, D. 2010. A Global Life: My Journey among Rich and Poor, from Sydney to Wall Street to the World Bank. New York, NY: Public Affairs, Perseus Books Group. World Bank. 1988. AIDS: The Bank’s Agenda for Action. Washington, DC: World Bank. ———. 1992. Combating AIDS and Other Sexually Transmitted Diseases in Africa: A Review of the World Bank’s Agenda for Action. Washington, DC: World Bank. ———. 1993a. Disease Control Priorities in Developing Countries. Washington, DC: World Bank. ———. 1993b. World Development Report 1993: Investing in Health. Washington, DC: Oxford University Press for the World Bank. ———. 1994a. Better Health in Africa: Experience and Lessons Learned. Washington, DC: World Bank. ———. 1994b. World Population Projections: Estimates and Projections with Related Demographic Statistics (World Bank) 1994-95. Washington, DC: Johns Hopkins University Press for the World Bank. ———. 1995. Regional AIDS Strategy for the Sahel. Washington, DC: World Bank. ———. 1996. AIDS Prevention and Mitigation in Sub-Saharan Africa: An Updated World Bank Strategy. Washington, DC: World Bank. ———. 1997a. Confronting AIDS: Public Priorities in a Global Epidemic. World Bank Policy Research Study. Washington, DC: Oxford University Press for the World Bank. ———. 1997b. Health, Nutrition, and Population Sector Strategy. Washington, DC: World Bank. ———. 1997c. “Intensifying Action against HIV/AIDS in Africa: Responding to a Development Crisis.� Africa Region, the World Bank, Washington, DC. ———. 2003. Education and AIDS: A Sourcebook of HIV/AIDS Prevention Programs. Washington, DC: World Bank. ———. 2004. HIV/AIDS: A Decision-Maker’s Guide to Procurement of Medicines and Related Supplies. Washington, DC: World Bank. ———. 2005a. Reaching the Poor with Health, Nutrition, and Population Services: What Works, What Doesn’t, and Why. Washington, DC: World Bank. ———. 2005b. Rolling Bank Malaria: The World Bank Strategy and Booster Program. Washington, DC: World Bank. ———. 2006a. Disease Control Priorities in Developing Countries, 2d edition, ed. Dean T. Jamison, Joel G. Breman, Anthony R. Measham, George Alleyne, Mariam Claeson, David B. Evans, Prabhat Jha, Anne Mills, and Philip Musgrove. New York and Washington, DC: Oxford University Press for the World Bank. ———. 2006b. Health Financing Revisited: A Practitioner’s Guide. Washington, DC: World Bank. ———. 2006c. Priorities in Health, ed. Dean T Jamison, Joel G Breman, Anthony R Measham, George Alleyne, Mariam Claeson, David B Evans, Prabhat Jha, Anne Mills, and Philip Musgrove. Washington, DC: World Bank. 128 ———. 2007a. The Africa Multi-Country AIDS Program, 2000-2006: Results of the World Bank’s Response to a Development Crisis. Washington, DC: World Bank. ———. 2007b. “Global Programs and Partnerships. Identifying and Addressing Partnership Conflict of Interest in Global Programs and Partnerships.� Guidance Note for Bank Staff. GPP Group, World Bank, Washington, DC. ———. 2007c. Healthy Development: The World Bank Strategy for Health, Nutrition, and Population Results. Washington, DC: World Bank. ———. 2007d. Our Commitment: The World Bank’s Africa Region HIV/AIDS Agenda for Action 2007-2011. Washington, DC: World Bank. ———. 2008. Education and AIDS: A Sourcebook of HIV/AIDS Prevention Programs. Washington, DC: World Bank. ———. 2009. Averting a Human Crisis during the Global Downturn: Policy Options from the World Bank’s Human Development Network. Washington, DC: World Bank. ———. 2010. “Unfinished Business: Mobilizing New Efforts to Achieve the 2015 Millennium Development Goals.� World Bank, Washington, DC. ———. 2010. World Development Indicators database. Available at http://data.worldbank.org/indicator White, H., 2009. “Theory-Based Impact Evaluation: Principles and Practice.� 3ie Working Paper No. 3, International Initiative for Impact Evaluation, New Delhi. WHO (World Health Organization). 2001. Macroeconomics and Health: Investing in Health for Economic Development. Geneva: WHO. ———. 2010a. World Health Statistics. Geneva: WHO. ———. 2010b. Global Tuberculosis Control Report. Geneva: WHO. ———. 2010c. World Malaria Report. Geneva: WHO. ———. 2011. “National Health Accounts.� www.who.int/nha/developments/en/ The Global Program Review Series The following reviews are available from IEG. Volume #1, Issue #1: ProVention Consortium Issue #2: Medicines for Malaria Venture Issue #3: Development Gateway Foundation Issue #4: Cities Alliance Volume #2, Issue #1: Critical Ecosystem Partnership Fund Issue #2: Association for the Development of Education in Africa Issue #3: Population and Reproductive Health Capacity Building Program Issue #4: International Land Coalition Volume #3, Issue #1: Consultative Group to Assist the Poor Issue #2: Global Development Network Issue #3: Global Forum for Health Research Issue #4: Global Invasive Species Program Volume #4, Issue #1: Stop Tuberculosis Partnership Issue #2: International Assessment of Agricultural Knowledge, Science, and Technology for Development Issue #3: The Global Water Partnership Volume #5, Issue #1: Multi-Donor Trust Fund for the Extractive Industries Transparency Initiative Issue #2: The Mesoamerican Biological Corridor Issue #3: Marrakech Action Plan for Statistics, Partnership in Statistics for Development in the 21st Century, and Trust Fund for Statistical Capacity Building Volume #6, Issue #1: The Global Fund to Fight AIDS, Tuberculosis and Malaria, and the World Bank’s Engagement with the Global Fund GPR The Global Fund to Fight AIDS, Tuberculosis and Malaria was founded in 2002 to mobi- lize large-scale donor resources for the specific purpose of reducing infections, illness, and death caused by the three diseases. The Global Fund has since become the largest of the 120 global and regional partnership programs in which the World Bank is currently involved, disbursing more than $3 billion in grants to developing and transition countries in 2010. The World Bank plays three major roles in the Global Fund: (a) as the trustee of donor contributions to the Global Fund, (b) in the corporate governance of the program, and (c) as a development partner at the global and country levels. This Review found that the Bank has had extensive engagement with the Global Fund at the global level through the Global HIV/AIDS Program, the International Health Partnership, and related initiatives, but has been less engaged at the country level. The Global Fund has fostered new approaches to development assistance. This Review found that its Country Coordinating Mechanisms have successfully brought country-level stakeholders together to submit grant proposals to the Global Fund, but have lacked the authority and the resources to exercise effective oversight of grant implementation. The situ- ation has improved in recent years in terms of the World Bank and other partnersʼ providing technical assistance in support of Global Fund activities, but these technical support func- tions need to be defined with greater clarity and formality within the context of improved donor harmonization. Collective donor efforts have contributed to increased availability and use of disease-con- trol services, particularly for HIV/AIDS, and increased coverage of affected communities. However, sustaining client countriesʼ disease-control programs in the face of decelerating external support will require a substantially more coordinated approach than has occurred to date. The scarce resources available to fight the three diseases — including those raised by each country and those provided by external partners — need to be allocated collectively and proactively in each country in accordance with a long-term strategy for fighting each dis- ease that is agreed among all the principal stakeholders.