GPR 57642 THE GLOBAL FORUM FOR HEALTH RESEARCH GLOBAL PROGRAM REVIEW Volume 3 Issue 3 THE WORLD BANK GROUP WORKING FOR A WORLD FREE OF POVERTY The World Bank Group consists of five institutions--the International Bank for Reconstruction and Development (IBRD), the International Finance Corporation (IFC), the International Development Association (IDA), the Multilateral Investment Guarantee Agency (MIGA), and the International Centre for the Settlement of Investment Disputes (ICSID). Its mission is to fight poverty for lasting results and to help people help themselves and their envi- ronment by providing resources, sharing knowledge, building capacity, and forging partnerships in the public and private sectors. THE INDEPENDENT EVALUATION GROUP IMPROVING DEVELOPMENT RESULTS THROUGH EXCELLENCE IN EVALUATION The Independent Evaluation Group (IEG) is an independent, three-part unit within the World Bank Group. IEG-World Bank is charged with evaluating the activities of the IBRD (The World Bank) and IDA, IEG-IFC focuses on assessment of IFC's work toward private sector development, and IEG-MIGA evaluates the contributions of MIGA guarantee projects and services. IEG reports directly to the Bank's Board of Directors through the Director-General, Evaluation. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank Group's work, and to provide accountability in the achievement of its objectives. It also improves Bank Group work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. The Global Forum for Health Research June 23, 2009 Corporate and Global Evaluations and Methods http://www.globalevaluations.org ©2009 Independent Evaluation Group, The World Bank Group 1818 H Street NW Washington DC 20433 Telephone: 202-458-4497 Internet: http://www.globalevaluations.org E-mail: grpp@worldbank.org All rights reserved This volume is a product of the staff of the Independent Evaluation Group (IEG) of the World Bank Group. It is part of an ongoing series that reviews global and regional partnership programs in which the World Bank is engaged as one of the partners. The findings, interpretations, and conclusions expressed in this volume do not necessarily reflect the views of the Executive Directors of The World Bank or the governments they represent. IEG does not guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply any judgment on the part of IEG concerning the legal status of any territory or the endorsement or acceptance of such boundaries. Rights and Permissions The material in this publication is copyrighted. IEG encourages the dissemination of its work and permits these reviews to be copied or otherwise transmitted, with appropriate credit given to IEG as the authoring body. Cover image: Clare McLean, University of Washington Medicine ISBN-13: 978-1-60244-119-4 ISBN-10: 1-60244-119-7 Printed on Recycled Paper Independent Evaluation Group Communication, Learning, and Strategy (IEGCS) E-mail: grpp@worldbank.org Telephone: 202-458-4497 IEG Mission: Improving Development Results Through Excellence in Evaluation The Independent Evaluation Group (IEG) of the World Bank reviews global and regional partnership programs (GRPPs) in which the Bank is engaged as one partner among many for two main purposes: (a) to provide accountability in the achievement of the program's objectives by providing an independent opinion of the program's effectiveness, and (b) to identify and disseminate lessons learned from the experience of individual GRPPs. The preparation of a global or regional program review (GPR) is contingent on a recently completed evaluation of the program, typically commissioned by the governing body of the program. The first purpose above includes validating the findings of the GRPP evaluation with respect to the effectiveness of the program, and assessing the Bank's performance as a partner in the program. The second purpose includes assessing the independence and quality of the GRPP evaluation itself and drawing implications for the Bank's continued involvement in the program. Assessing the quality of GRPP evaluations is an important aspect of GPRs, since encouraging high quality evaluation methodology and practice more uniformly across Bank-supported GRPPs is one of the reasons why IEG embarked on this new product in 2005. IEG annually reviews a number of GRPPs in which the Bank is a partner. In selecting programs for review, preference is given to those that are innovative, large, or complex; those that are relevant to upcoming sector studies; those for which the Executive Directors or Bank management have requested reviews; and those that are likely to generate important lessons. IEG also aims for a representative distribution of GPRs across sectors in each fiscal year. A GPR is a "review" and not a full-fledged "evaluation." It assesses the independence and quality of the relevant evaluation; provides a second opinion on the effectiveness of the program; assesses the performance of the Bank as a partner in the program; and draws lessons for the Bank's engagement in global and regional programs. The GPR does not formally rate the various attributes of the program. A GPR involves a desk review of key documents, consultations with key stakeholders, and a mission to the program management unit (secretariat) of the program if this is located outside the World Bank or Washington, DC. Key stakeholders include the Bank's representative on the governing body of the program, the Bank's task team leader (if separate from the Bank's representative), the program chair, the head of the secretariat, other program partners (at the governance and implementing levels), and other Bank operational staff involved with the program. The writer of a GPR may also consult with the person(s) who conducted the evaluation of the GRPP. Each GPR is subject to internal IEG peer review, Panel review, and management approval. Once cleared internally, the GPR is reviewed by the responsible Bank department and the secretariat of the program. Comments received are taken into account in finalizing the document, and the formal management response from the program is attached as an annex to the final report. After the document has been distributed to the Bank's Board of Executive Directors, it is disclosed to the public on IEG's external Web site. Abbreviations and Acronyms AFM Annual Forum Meeting (Global Forum) AFRISTAT Observatoire économique et statistique d'Afrique Subsaharienne (Economic and Statistical Observatory of Sub-Saharan Africa) AHPSR Alliance for Health Policy and Systems Research (formerly of Global Forum, now WHO) APOC African Program of Onchocerciasis Control ARD Agricultural research for development (Global Forum on Agricultural Research) AU African Union, successor to OAU BB Bank's administrative budget (World Bank) CAM Combined Approach Matrix (for research priority setting, of Global Forum) CD-ROM Compact disc with read-only memory CEO Chief executive officer CGIAR Consultative Group on International Agricultural Research (World Bank) CMH Commission on Macroeconomics and Health (WHO) CODE Committee on Development Effectiveness (of World Bank Board of Directors) COHRED Council on Health Research for Development -- a Geneva-based NGO CSDH Commission on Social Determinants of Health (WHO) CSO Civil society organization DANIDA Danish International Development Agency DALY Disability adjusted life year DFID Department for International Development (UK) DGF Development Grant Facility (World Bank) DNDi Drugs for Neglected Diseases Initiative (MSF) DPT3 Diptheria, pertusis, tetanus (vaccine) -- DPT3 coverage frequently used as a measure of availability of health services ECA Europe and Central Asia Region (World Bank) ED Executive Director ET Evaluation team FAO Food and Agricultural Organization FC Foundation Council (GFHR Board of Directors) FGM Female genital mutilation FM Financial management G8 The Group of 8 leading industrial countries (G7 without Russia) GAVI Global Alliance for Vaccines and Immunizations GFAR Global Forum on Agricultural Research GFATM Global Fund to Fight AIDS, TB, and Malaria GFHR Global Forum for Health Research GNP Gross national product GNRNMH Global Network for Research in Mental and Neurological Health (of Global Forum) GPG(s) Global public good(s) GPP Global Partnership Program (World Bank) GPR Global Program Review (IEG) GRPP Global or regional Partnership Program (World Bank) HDNGA Human Development Vice-Presidency, Global AIDS Program (World Bank) Health 8 WHO, World Bank, GAVI, GFATM, UNICEF, UNFPA, UNAIDS, Bill and Melinda Gates Foundation HepB3 3 doses of hepatitis B vaccine HD Human development Hib3 3 doses of Haemophilus influenzae type b (Hib) vaccine to prevent severe bacterial pneumonia, a leading cause of death in young children in the developing world HIROs Heads of International Health Research Organizations HMN Health Metrics Network iii HNP Health, Nutrition and Population (World Bank) HRP Special Program of Research, Development and Research Training in Human Reproduction (WHO) IANPHI International Association of National Public Health Institutes IAVI International AIDS Vaccine Initiative -- a PDPPP IDA International Development Association (of World Bank) ICH Initiative for Cardiovascular Health -- IC Health (Global Forum) IC Health Initiative for Cardiovascular Health (Global Forum) IDDR,B International Disease Research Institute, Bangladesh IDRC International Development Research Center (Canada) IEG Independent Evaluation Group (World Bank) IFFIm International Financial Facility for Immunization IFPMA International Federation of Pharmaceutical Manufacturers and Associations IGWG Inter-Governmental Working Group on Public Health, Innovation and Intellectual Property (WHO) IHP+ International Health Partnership plus ­ a grouping of health agencies and institutions promoting more effective development cooperation in the field of health IPPF International Planned Parenthood Federation -- an NGO IPPPH Initiative for Public-Private Partnerships (in health, of Global Forum) LDC Less developed country LIC Low-income countries LMICs Low and middle income countries MDGs Millennium Development Goals (UN) M&E Monitoring and evaluation MFA Ministry of Foreign Affairs MIM Multilateral Initiative against Malaria MMV Medicines for Malaria Venture -- a PDPPP MOH Ministry of Health MOU Memorandum of Understanding MRC(s) Medical Research Council (UK and other countries) MSF Médicins sans frontières (Doctors without Borders) NCDs Non-communicable disease(s) nd No date NEPAD New Partnership for Africa's Development (AU) NGO Non-governmental organization NIH(s) National Institutes of Health (United States and others) NORAD Norwegian Agency for Development Cooperation OAU Organization of African Unity, now called the African Union ODA Official development assistance (OECD) PAHO Pan American Health Organization (regional office of WHO) PCU Project coordination unit (for World Bank-financed projects) PEPFAR President's Emergency Plan for AIDS Relief (United States) OECD Organization for Economic Cooperation and Development PDPPP Product development public-private partnership PHN Population, health and nutrition (World Bank) PhRMA Pharmaceuticals Research and Manufacturers Association of America PPP Public-private partnership PRPP Pharmaceutical R&D Policy Project (George Institute for International Health, Australia) R&D Research and development R&P Research and programs (Global Forum) RBM Roll Back Malaria (WHO partnership) RCS Research capacity strengthening RH Reproductive health RTIRN Road Traffic and Injuries Research Network (Global Forum) SAREC Swedish Agency for Research Cooperation and Development SDC Swiss Agency for Development and Cooperation SIDA Swedish International Development Agency SRH Sexual and reproductive health S-S South-south SSA Sub-Saharan Africa STRATEC Strategic and Technical Advisory Group (of Global Forum Foundation Council) SVRI Sexual Violence Research Initiative (Global Forum) TDR Joint UNICEF, WHO, UNDP, World Bank Special Program for Research and Training in Tropical Diseases TOR Terms of reference UK United Kingdom UN United Nations UNDP United Nations Development Program UNFPA United Nations Population Fund UNICEF United Nations Children's Fund USAID United States Agency for International Development VP Vice-President (World Bank) WB World Bank WDR93 World Development Report 1993 Investing in Health (World Bank) WHA World Health Assembly (WHO) WHO World Health Organization WIPO World Intellectual Property Organization WTO World Trade Organization Fiscal Year of Program January 1 ­ December 31 Director-General, Evaluation Mr. Vinod Thomas Director, Independent Evaluation Group (World Bank) Ms. Cheryl Gray Manager, Corporate and Global Evaluations and Methods Unit Mr. Mark Sundberg Global Programs Coordinator Mr. Chris Gerrard Consultant Mr. A. Edward Elmendorf v Contents PROGRAM AT A GLANCE: GLOBAL FORUM FOR HEALTH RESEARCH .....................................IX KEY BANK STAFF RESPONSIBLE DURING PERIOD UNDER REVIEW ..........................................X GLOSSARY...........................................................................................................................................XI PREFACE............................................................................................................................................ XV SUMMARY ........................................................................................................................................ XVII 1. PROGRAM OBJECTIVES, ACTIVITIES, FINANCIAL RESOURCES AND GOVERNANCE . 1 Objectives, Vision, Mission, and Activities........................................................................... 1 Financial Resources................................................................................................................ 5 Governance, Management, and Organization ...................................................................... 6 Key Global Forum Partnerships: WHO and COHRED ......................................................... 9 The Global Forum at 10 Years .............................................................................................. 10 2. THE SECOND EXTERNAL EVALUATION OF THE GLOBAL FORUM ............................... 10 Scope, Process and Approach of the Second Evaluation ................................................ 11 Independence and Quality .................................................................................................... 12 Findings and Recommendations and Global Forum Response....................................... 13 Impact of the Second Evaluation: A Revised and Evolving Strategy .............................. 14 3. THE EFFECTIVENESS OF THE GLOBAL FORUM .............................................................. 17 Relevance of the Global Forum............................................................................................ 17 Efficacy ................................................................................................................................... 23 Efficiency ................................................................................................................................ 28 Governance ............................................................................................................................ 31 The Global Forum and the Future of Research on Developing Country Health ............. 35 4. WORLD BANK PERFORMANCE IN THE GLOBAL FORUM PARTNERSHIP .................... 41 The Multiple and Occasionally Conflicting Roles Played by the Bank in the Global Forum...................................................................................................................................... 41 The Global Forum and Bank Corporate and Sectoral Strategies and Priorities ............. 45 World Bank DGF Support for the Global Forum and DGF Policies.................................. 46 vi DGF Sub-Grants through the Global Forum and World Bank Policies ........................... 47 The Global Forum and World Bank HNP Grant Strategy .................................................. 49 Exit Strategy .......................................................................................................................... 49 Oversight of the Global Forum Partnership ....................................................................... 50 Risks and Benefits to the Bank of Participating in the Global Forum Partnership........ 51 Conclusion ............................................................................................................................. 52 5. LESSONS ............................................................................................................................... 52 REFERENCES ..................................................................................................................................... 55 ANNEX A. EVALUATION FRAMEWORK FOR GLOBAL PROGRAM REVIEWS............................ 58 ANNEX B. PROGRAM GOALS, OBJECTIVES, ACTIVITIES, AND ACHIEVEMENTS .................... 67 ANNEX C: GLOBAL FORUM ANNUAL FORUM MEETINGS ........................................................... 84 ANNEX D: GLOBAL FORUM INFORMATION AND COMMUNICATIONS ....................................... 88 ANNEX E. PROGRAM TIMELINE ...................................................................................................... 95 ANNEX F. EXTERNAL EVALUATION: MAJOR FINDINGS AND RECOMMENDATIONS OF THE GLOBAL FORUM'S SECOND EXTERNAL EVALUATION AND THE GLOBAL FORUM RESPONSE ....................................................................................................................................... 106 ANNEX G. MEMBERS OF THE GLOBAL FORUM GOVERNING BODIES AND THEIR GLOBAL FORUM CONSTITUENCIES ............................................................................................................. 116 ANNEX H. GLOBAL FORUM FINANCIAL INFORMATION ............................................................ 121 ANNEX I. PERSONS CONSULTED ................................................................................................. 125 ANNEX J. RESPONSE OF THE PROGRAM TO IEG'S GLOBAL PROGRAM REVIEW ............... 128 Boxes Box 1. The Global Forum and the "10/90 Gap" ................................................................................. 1 Box 2. The Vision and Mission of the Global Forum in 2008........................................................... 3 Box 3. Initiatives Created by the Global Forum ................................................................................ 3 Box 4. Constituencies of the Global Forum ...................................................................................... 8 Box 5. A Surge of International Interest in Global Health .............................................................. 19 vii Box 6. The Emerging WHO Research Strategy ............................................................................... 23 Box 7. Evidence for Efficacy of the Global Forum: Global Attention to the 10/90 Gap............... 27 Box 8. Innovative Health Research in a Middle-Income Country -- Brazil ................................... 35 Box 9. The Global Forum "Report Card" on R&D for Health ......................................................... 36 Box 10. The Global Forum for Agricultural Research..................................................................... 40 Figures Figure 1. The Growth in Funding for Health Research, 1986­2005 ................................................. 2 Figure 2. Participants' Evaluation of the Global Forum Annual Forum Meetings ......................... 5 Figure 3. Governance and Management of the Global Forum ......................................................... 8 Figure 4. Global Forum for Health Research: Donor Financial Contributions ............................. 29 Tables Table 1. Global Forum for Health Research: Average Annual Expenditures on Core Functions, by Category, 2001­07 .............................................................................................................. 6 Table 2. Donor Support for the Global Forum for Health Research, 1999­2007 ............................ 7 Table 3. The Second External Evaluation: Major Recommendations and Global Forum Response ................................................................................................................................ 15 Table 4. Advocacy and Knowledge Networking Activities of the Global Forum ......................... 21 Table 5. Evolution of GFHR Strategy Statements and Objectives: From the 10/90 Gap to Health Equity and the Poor ............................................................................................................... 25 Table 6. The Comparative Advantages of the Global Forum ......................................................... 36 Table 7. The World Bank's Performance as a Partner in the Global Forum for Health Research ................................................................................................................................................. 42 Table 8. The Global Forum and World Bank Corporate and Sectoral HNP Strategic Priorities . 46 Table 9. HDNHE Administrative Budget Expenditures on Oversight and Liaison Activities in Relation to the Global Forum for Health Research, FY05­09 ........................................... 51 Annex Table 1. Assessing the Independence and Quality of the Evaluation .............................. 58 Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program ......... 59 Annex Table 3. Assessing the Bank's Performance as a Partner in the Program ...................... 65 Annex Table 4. Common GRPP Activities ....................................................................................... 66 Annex Table 5. Estimated Global Health R&D Funding, 1998­2005 ($, billion) ........................... 68 Annex Table 6. Initiatives of the Global Forum for Health Research ............................................ 75 Annex Table 7. Global Forum Sub-Grantees Benefiting from World Bank DGF Funds: An Overview ................................................................................................................................. 78 Annex Table 8. The Global Forum Objectives and Strategies, 2008­14: Relationship of Objectives and Strategic Priorities ...................................................................................... 83 Annex Table 9. Number of Participants, Countries Represented, and Share of Participants by Gender, and by High- and Low-Middle-Income Countries, by Forum.............................. 84 Annex Table 10. Share of Participants by Region and Forum (Percent) ...................................... 85 Annex Table 11. Share of Participants by Institutional Affiliation and by Forum (Percent) ....... 86 Annex Table 12. Share of Presenters by Gender and High- (HIC) and Low-Middle-Income Countries (LMIC), by Forum (Percent) ................................................................................ 87 Annex Table 13. Evaluation of Annual Forum Meetings by Participants ..................................... 87 Annex Table 14. Number of GFHR Publication Orders, 2005­08 .................................................. 91 viii Annex Table 15. Global Forum Publications Ordered by Type ..................................................... 92 Annex Table 16. Global Forum Web Site Unique Visitors, Visits, and Numbers of Pages Viewed: September 2005 ­ September 2008 ...................................................................... 92 Annex Table 17. Global Forum Web Site Downloads, September 2005­08 ................................. 93 Annex Table 18. Organizations in the GFHR Database by GFHR Constituency ......................... 94 Annex Table 19. Contacts in the GFHR database by Country Income Level............................... 94 Annex Table 20. Global Forum Constituencies of Foundation Council and STRATEC Members, as of April 1, 2008................................................................................................................ 120 Annex Table 21. Global Forum for Health Research: Expenditures on Core Functions, by Category and Year, 2001­07 .............................................................................................. 121 Annex Table 22. Global Forum for Health Research: Donor Financial Contributions, by Donor and Year, 1999­2007 (US$ thousands) ............................................................................. 122 Annex Table 23. World Bank DGF Financial Support for Global Forum Initiatives and Projects, 2000­07 (US$ thousands) .................................................................................................. 123 Annex Table 24. World Bank DGF Budget Allocations for the Global Forum for Health Research, FY1998­2009, in Overall DGF Context ............................................................ 124 ix Program at a Glance: Global Forum for Health Research Start Date January 1998 Vision/Mission Vision: A world in which the potential of research and innovation is fully (as of 2008) utilized to address the health problems of the poor. Mission: To play a leadership role in catalyzing global research applied to the health problems of the poor. Objectives Improved priorities in health research and innovation (as of 2008) Increased coherence and partnerships among global players in health research Strengthened research and innovation on health and health equity Expanded use of evidence in policy and decision making relating to health research Major Activities Commissioning, executing, and publishing analytical work on the flow of financial resources for health research, health research priorities, and related matters, oriented towards health in developing countries. Bringing together key actors, including health researchers, health research policy makers, and other stakeholders, in Annual Forum Meetings and other events, for mutual understanding and consensus on health research and research priorities. Disseminating information, evidence and arguments, to influence stakeholders in research and innovation for the health of the poor. WBG DGF /a $25.3 million (1999­2007) contributions Other Donor $30.7 (1999­2007) Contributions Location Geneva, Switzerland Governance and An NGO consisting of: Management Board of Directors (Foundation Council -- FC) of about 20, representative of but not formally representing 9 Global Forum constituencies. About half of the FC members come from developing countries. Donor members come from the Bank, IDRC, NORAD, SDC and the Consultative Science Council of Mexico. Ex officio FC members are appointed by WHO (currently the TDR Director) and by the Council on Health Research for Development (COHRED). Strategic and Technical Advisory Group (STRATEC) of a maximum of six FC members Secretariat of approximately 20 persons led by an Executive Director Latest Program- Vis Navaratnam, Piroska Ostlin, and Victor Penchaszadeh, Second External Level Evaluation Evaluation of the Global Forum for Health Research, Final Report, August 2007 /a This figure excludes DGF sub-grants of $24.5 million channeled through the GFHR at the direction of the Bank to other entities, such as the Medicines for Malaria Venture (MMV), pursuing goals and activities compatible with those of the Global Forum. x Key Bank Staff Responsible during Period under Review Position Person Period DGF/Health Partnership Janet Nassim 1998­2005 Coordinator, HDNHE Nicole Klingen 2005­2008 Miriam Mirasol 2008 (July)­present HNP Partnership Adviser, Armin Fidler 2008 (April)­present HDNHE Bank's Representative on Richard Feachem 1998­1999 the GFHR Governing Body Director, HNP Maureen Law 1999­2002 Health Sector Manager, East Asia and Pacific Region Charles Griffin 2002­2003 Human Development Sector Director, South Asia Region Robert Hecht 2003­2004 Sector Manager, HNP Department Ok Pannenborg 2004­present Senior Advisor, HNP, Africa Region Director, HDNHE Richard Feachem 1998­999 Christopher Lovelace 1999­2002 Jacques Baudouy 2003­2007 Cristian Baeza (Acting) 2007 Julian Schweitzer October 2007­present Vice President/Sr. Vice- David de Ferranti 1998­1999 President, Human Eduardo Doryan 1999­2001 Development Network Jozef Ritzen 2001­2003 Jean-Louis Sarbib 2003­2006 Joy Phumaphi February 2007­present HDNVP Trust Fund Nancy Pinto August 2006­present Operations and Global Program Partnerships Coordinator Global Programs and Margret Thalwitz, Director May 2004­November 2008 Partnerships Program Chairs and Managers Position Person Period Chair, Foundation Council Prof. Adetokunbo Lucas 1998­2003 Prof. Richard Feachem 2003­2004 Dr. Pramilla Senanayake 2004­2007 Dr. Gill Samuels 2007­present Executive Secretary Dr. Louis Currat 1998­2003 Executive Director Prof. Stephen Matlin 2004­present xi Glossary Constituencies In the Global Forum the term refers to institutional actors in the field of health research; 9 constituencies are recognized in the bylaws of the Global Forum. Devolution or exit strategy A proactive strategy to change the design of a program, to devolve some of its implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the grounds that it has achieved its objectives or that its current design is no longer the best way to sustain the results which the program has achieved. Disability-adjusted life year A measure of life lost to disease or injury which permits comparison across health conditions, countries and years. Donor Any organization or entity that makes a financial contribution to the program that is reflected in its audited financial statements. Drug A substance intended for use in the diagnosis, cure, mitigation, treatment or prevention of a disease. Efficacy The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their relative importance. The term is also used as a broader, aggregate measure -- encompassing relevance and efficiency as well -- of the overall outcome of a development intervention such as a GRPP. Efficiency The extent to which the program has converted or is expected to convert its resources/inputs (such as funds, expertise, time, etc.) economically into results in order to achieve the maximum possible outputs, outcomes, and impacts with the minimum possible inputs. European Observatory The European Observatory on Health Systems (WHO), which supports and promotes evidence-based health policy-making through rigorous analysis of the dynamics of health systems in Europe. Evaluation The systematic and objective assessment of an ongoing to completed policy, program, or project, its design, implementation, and results. The aim is to determine the relevance and achievement of its objectives, and its developmental effectiveness, efficiency, impact, and sustainability. Foundation Council Board of Directors of a Swiss body, under the laws of Switzerland. G8 The Group of 8 leading industrial countries (Canada, France, Germany, Italy, Japan, Russia, the United Kingdom, and the United States) meet in an annual summit of political leaders on global issues, including health. Governance The structures, functions, processes, and organizational traditions that have been put in place within the context of a program's authorizing environment to ensure that the program is run in such a way that it achieves its objectives in an effective and transparent manner. It is the framework of accountability and responsibility to users, stakeholders and the wider community, within which organizations take decisions, and lead and control their functions, to achieve their objectives. Health 8 The Health 8 (WHO, World Bank, GAVI, GFATM, UNICEF, UNFPA, UNAIDS, Bill and Melinda Gates Foundation) is a group of agencies deeply engaged in global health, whose senior officials informally meet occasionally to strengthen collaboration in global health for better health outcomes in developing countries. Health Metrics Network The Health Metrics Network (HMN) is a global partnership sponsored by WHO that facilitates better health information at country, regional and global levels. Partners include developing countries, multilateral and bilateral agencies, foundations, other global health partnerships and technical experts. HMN seeks to bring together health and statistical constituencies in order to build capacity and expertise and enhance the availability, quality, dissemination and use of data for decision making. xii Impacts Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended. Independent evaluation An evaluation that is carried out by entities and persons free from the control of those involved in policy-making, management, or implementation of program activities. This entails organizational and behavioral independence, protection from interference, and avoidance of conflicts of interest. Indicator A quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor. Innovation In the Global Forum innovation refers to the applications of research to deliver solutions leading to better health for the poor; it encompasses social and economic as well as technological innovations. Legitimacy A criterion for assessing governance and management, the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program -- including shareholders, other stakeholders, implementers, beneficiaries, and the community at large. Logical framework or logframe A management technique that is used to develop the overall design of a program or project, to improve implementation monitoring, and to strengthen evaluation, by presenting the essential elements of the program or project clearly and succinctly throughout its cycle. It is a "cause and effect" model which aims to establish clear objectives and strategies based on a results chain, to build commitment and ownership among the stakeholders during the preparation of the program or project, and to relate the program's or project's interventions to their intended outcomes and impacts for beneficiaries. Management The day-to-day operation of a program within the context of the strategies, policies, processes, and procedures that have been established by the governing body. Merit good According to Palgrave's Dictionary of Economics, the term "merit good" has no generally agreed application but is best applied where individual choice is restrained by community values. Thus, for purposes of this review a merit good is a good or service deemed in an expression of community values by a public decision maker to be insufficiently supplied by markets at an acceptable price and therefore deserving subsidization. Monitoring The continuous assessment of progress achieved during program implementation in order to track compliance with a plan, to identify reasons for noncompliance, and to take necessary actions to improve performance. Monitoring is usually the responsibility of program management and operational staff. Neglected diseases Diseases that have received relatively little attention from researchers and policy makers in the industrial world but have significant effects in the tropics. Malaria, TB, and a number of less well known tropical diseases are in this category. The commercial profit motive does not provide sufficient incentive for levels of R&D that could significantly reduce the burden of these diseases. By way of comparison, R&D on so-called "orphan drugs" for rare diseases in the industrial world receives incentives under legislation in the United States, Japan, Australia, the European Union, Singapore and Korea. Outcomes The achieved or likely short-term and medium-term effects of the outputs of a development intervention. Oversight One of the core functions of the governing body of a program: Monitoring the performance of the program management unit, appointing key personnel, approving annual budgets and business plans, and overseeing major capital expenditures. Paris Declaration A March 2005 OECD statement adopted by representatives of over 100 developing and industrial countries, along with international organizations, aimed at increasing donor harmonization and alignment for results at the country level. xiii Partners Stakeholders who are involved in the governance or financing of the program (including the members of the governing, executive, and advisory bodies). Public goods Goods which produce benefits that are non-rival (many people can consume, use, or enjoy the good at the same time) and non-excludable (it is difficult to prevent people who do not pay for the good from consuming it). If the benefits of a particular public good accrue across all or many countries, then the good is deemed a global or international public good. Relevance The extent to which the objectives and design of the program are consistent with (a) the current global/regional challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and groups. Research for health The Global Forum defines "research for health" as research undertaken in any discipline or combination of disciplines that seeks to (a) understand the impact on health of policies, programs, processes, actions or events originating in any sector; (b) assist in developing interventions that will help prevent or mitigate that impact; and/or (c) contribute to the achievement of health equity and better health for all (Burke and Matlin, 2008). Shareholders The subset of donors that are involved in the governance of the program. Therefore, this does not include individual (particularly anonymous) donors who choose not to be so involved, or who are not entitled to be involved if their contribution does not meet the minimum requirement, say, for membership on the governing body. Stakeholders The parties who are interested in or affected, either positively or negatively, by the program. Stakeholders are often referred to as "principal" and "other", or "direct" and "indirect". While other or indirect stakeholders -- such as taxpayers in both donor and beneficiary countries, visitors to a beneficiary country, and other indirect beneficiaries -- may have interests as well, these are not ordinarily considered in evaluations unless a principal stakeholder acts as their proxy. Sustainability When the term is applied to the activities of a program, the extent to which the benefits arising from these activities are likely to continue after the activities have been completed. When the term is applied to organizations or programs themselves, the extent to which the organization or program is likely to continue its operational activities over time. Transparency As a criterion for assessing governance and management, the extent to which a program's decision-making, reporting, and evaluation processes are open and freely available to the general public. This is a metaphorical extension of the meaning used in physical sciences -- a "transparent" objective being one that can be seen through. Value for money The extent to which a program has obtained the maximum benefit from the outputs and outcomes it has produced with the resources available to it. Source: For evaluation terms, the Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards, Independent Evaluation Group -- World Bank, 2007; for other terms, GFHR documents and relevant Web sites. xv Preface The Global Forum for Health Research was established in 1998 as an independent Swiss foundation, to promote health research on the problems of poor countries and people. The creation of the Global Forum responded to the growing awareness among policy makers in industrial and in developing countries that research related to the health problems affecting developing country populations was receiving inadequate attention on the global agenda. The Global Forum has become known as the principal advocate of bridging the so-called "10/90" gap -- a metaphor for the global imbalance in health research spending suggesting that less than 10 percent of global health research expenditures were devoted to developing countries, where more than 90 percent of preventable mortality was to be found. The Global Forum has devoted its energies increasingly to health equity as a way to focus the attention of researchers and policy makers on the problems of the poor. Annual expenditures on the core activities of the Global Forum have been about $3.5 million. Bilateral donors and the World Bank finance virtually all of the Global Forum's activities. Following an initial independent evaluation in 2001, the Foundation Council commissioned a second evaluation in 2005, which covered the period 2002­05. This Global Program Review (GPR) assesses the quality and independence of the second evaluation of the Global Forum; provides a second opinion on the effectiveness of the Forum; assesses the performance of the Bank as a partner of the Forum; and draws lessons for the future. It contains data from the beginning of the Forum to the present, including key developments during the last two years since the second external evaluation was completed. The Global Forum was chosen for a GPR because it provides lessons for the design and operation of other global programs, especially for advocacy programs, and for international support of health research more generally. The Review follows IEG's Guidelines for Global Program Reviews (Annex A). It is based on a desk review of relevant documents including, in addition to the 2001 and 2007 evaluation reports, Global Forum documents, consultant studies, journal articles, and Web sites, and discussions in Geneva and beyond with 34 key informants (Global Forum Foundation Council members, Forum managers, Forum staff, knowledgeable observers, the members of the 2007 evaluation team, and World Bank staff). A mission to the Global Forum took place in October 2008. IEG gratefully acknowledges all those who made time available for interviews and provided information for this GPR, in particular Global Forum Foundation Council members, management, and staff. A list of people consulted can be found in Annex I. Copies of the draft GPR were sent to the Global Forum, to the Bank unit which is responsible for the Bank's involvement with the Global Forum (the Health, Nutrition and Population Department), and to other Bank units that have responsibility for the Bank's engagement with global programs more generally. Their comments were taken into account in finalizing this GPR. The formal response of the Global Forum can be found in Annex J. xvii Summary Objectives, Activities, Financial Resources, and Governance 1. In the 1990s relatively little was known about the investments made in research for health in low- and middle-income countries. In 1990, the Commission on Health Research for Development estimated that only about 5 percent of the total of $30 billion spent on health research in 1986 was applied to the health problems of developing countries, where 93 percent of the world's burden of preventable mortality occurred. Global expenditures on health research have increased more than four times in the last 20 years, to more than US$ 160 billion per year. However, imbalances and inequities in health research spending still persist, and the picture has grown increasingly complicated due to epidemiological changes. Developing countries now bear a multiple burden of old, new, and re-emerging communicable diseases; steep increases in levels of non-communicable diseases; and rising rates of injury. 2. To provide advocacy support to address these issues, the Global Forum for Health Research was established in Switzerland in 1998, in response to the growing awareness among policy makers in industrial and in developing countries that research related to the health problems affecting developing country populations was then receiving inadequate attention on the global agenda. The Forum is a small independent international non- governmental organization, called a foundation in Swiss law. 3. As updated in 2008, the Global Forum's vision is a world in which the potential of research and innovation is fully utilized to address the health problems of the poor. Its mission is to play a leadership role in catalyzing global research applied to the health problems of the poor. The objectives of the forum are (a) improved priorities in health research and innovation, (b) increased coherence and partnerships among global players in health research, (c) strengthened research and innovation on health and health equity, and (d) expanded use of evidence in policy and decision making relating to health research. While the Forum writes globally about the poor and health equity, its orientation is clearly towards developing countries as a whole. 4. In pursuit of its objectives the Global Forum commissions, executes, and publishes analytical work on the flow of financial resources for health research, health research priorities, and related matters, oriented towards health in developing countries. It brings together key actors, including health researchers, health research policy makers, and other stakeholders, in Annual Forum Meetings and other events, for mutual exchange and consensus on health research and research priorities. It disseminates information, evidence and arguments to influence stakeholders in research and innovation for the health of the poor. 5. Communication is central to the work of the Global Forum. One of its principal communication activities is the holding of an Annual Forum Meeting (AFM) and marketplace, where health research problems and priorities are presented and discussed by a wide range of researchers, policy makers and decision makers. Other principal communication activities include publication and dissemination of advocacy documents and xviii engagement with different stakeholders through diverse meetings. The AFMs afford opportunities for presentation of new research on developing country health problems and priorities, and particularly for contact and communication among developing country health researchers and health research sponsors and financiers. The AFMs are appreciated by participants for the networking opportunities among researchers from developing and developed countries. Global Forum communications and publications include a Web site from which its publications can be downloaded or ordered in hard copy without cost to the recipient. A database of some 14,000 contacts (at the end of 2008) is maintained and used for communication with Global Forum constituencies and others. 6. The Global Forum devotes approximately $3.5 million each year to its core activities of research and programs, annual forum meetings, and other information and communications. For its financial resources the Global Forum depends almost entirely on funding from bilateral donors and the World Bank. The Global Forum statutes provide for the representation of 9 constituencies in the Forum. Special efforts are consistently made to ensure engagement of developing country researchers and policy makers in Annual Forum Meetings, as presenters and as regular participants. 7. The Global Forum is managed by a self-perpetuating Board of Directors -- the Foundation Council (FC) -- of up to 25 people from its various constituencies. A Strategic and Technical Advisory Group (STRATEC) of up to six FC members helps to provide technical guidance and prepare decisions for the FC. A small secretariat of about 20 people led by an Executive Director completes the organizational structure of the Global Forum. The Second External Evaluation of the Global Forum 8. Following an initial evaluation in 2001, the Foundation Council commissioned the second external evaluation of the Global Forum in early 2005. The evaluation team's final report was submitted in February 2007, and issued in August of that year. The purpose of the evaluation was to assess the overall relevance, appropriateness, adequacy, efficiency and effectiveness of the Forum in relation to its objectives, strategies and values. The Terms of Reference did not cover governance. A three-person evaluation team of health researchers was chosen. The evaluation included global and topic-specific questionnaires and over 60 interviews. In the collection and analysis of data, the evaluation team was independent of the Global Forum. The team was extremely thorough in its work. Its final report was the product of extended discussions, over many months, with the Secretariat and the FC. There were important differences of view and difficult personal exchanges between members of the evaluation team and the Global Forum. The final evaluation report contains large numbers of pertinent individual observations and recommendations, but it is occasionally difficult to grasp the overall messages, partly as a result of an unclear results framework for Global Forum activities on which to base the evaluation. 9. The final evaluation is deficient in important respects. It contains some political content and personal judgment, especially on the team's strongly held view that the Global Forum should be substantially engaged at the country level and eventually merged with the Council on Health Research for Development. The principal sources of weakness in the evaluation concern absence of an inception report that might have led to further guidance, xix excessive reliance on interview data, and biases in sources of information. A skills mix encompassing expertise in economics or finance might have strengthened the evaluation team's report, and improved its relatively weak analysis of efficiency. 10. The evaluation called upon the Global Forum to develop a new strategic plan. It encouraged the Forum to revisit the concept of the 10/90 gap. It proposed that the Global Forum should focus its attention on current and controversial issues in health research. Finally, it encouraged the Forum to review current procedures regarding management practices and decision making, to facilitate greater discussion, transparency and involvement of Foundation Council members in the strategic functioning of the Forum. 11. The principal positive result of the second evaluation has been the adoption of a new Global Forum strategy. The revised strategy calls upon the Forum to play a leadership role in catalyzing global research on the health problems of the poor, and posits the core values of the Global Forum as (a) health as a right, (b) equity as a principle, and (c) research as an indispensable tool. In pursuit of its reformulated objectives (paragraph 3 above) under the new strategy, Forum's work is to concentrate on three strategic priority areas: (a) linking resources with priorities for research for health; (b) increasing the role of research in supporting health systems development; and (c) strengthening innovation for health in low and middle income countries. The Effectiveness of the Global Forum 12. This review finds that the Global Forum has been somewhat effective but needs to focus its activities further and seek broader and deeper engagement with the largest funders of health research and the commercial private sector. RELEVANCE 13. The Global Forum makes implicit assumptions which are worth articulating and examining with regard to relevance, including (a) that lack of appropriate research is a critical factor in the unsatisfactory health outcomes prevailing among poor countries and peoples and (b) that the need for advocacy on health research for the poor is not met adequately by other organizations. The Global Forum's mechanisms for action further assume (c) that measuring expenditures on research is a valid proxy for measuring how much relevant and high-quality research is carried out, and (d) that research can be encouraged by drawing attention to its absence, highlighting the nature of the gaps and stimulating efforts to bridge the gaps. It has not been possible to examine each of these assumptions in detail but they are certainly plausible. 14. Beyond these basic assumptions, the vision and mission of the Global Forum are responsive to current global challenges and policies, to the growing availability of development assistance for health, to the increasing world-wide expenditures on health research, and to the particular needs of developing countries. The surge of interest in global health extends well beyond the United Nations and the multilateral development banks. The growth in global spending on health research, to $160 billion annually, increases the relevance of an advocacy effort to promote spending on the health problems of low and middle income countries. xx 15. The relevance of the design of the Global Forum activities is less clear than its objectives. The sweeping breadth of the Global Forum for Health Research strategy and the great need for promotion of health research on the problems of poor countries and poor peoples would appear to call for greater focus, and a higher degree of selectivity in Global Forum activities. 16. The nearly complete absence from the Global Forum partnership, particularly from its governance and financing, of the largest funders of health research on the problems of poor countries and peoples, and its minimal engagement with commercial private sector actors who are so important as health research financiers, limit the relevance of the Forum. Its activities are complementary to the health, nutrition, and population (HNP) project operations of the Bank at the country level, but its available human and financial resources are dwarfed by the resources available to major commercial, philanthropic and public financiers and promoters of health research. EFFICACY 17. Measuring the results of Global Forum efforts to close the 10/90 gap is extremely difficult because of the problem of attribution, difficulties in the concept of the gap, the rapidly changing external environment for Global Forum activities, the evolving objectives of the Forum, and the failure of the Forum to establish an overall results framework. External observers have markedly varied views on the efficacy of the Global Forum. The 2007 evaluation found much room to improve the standing and influence of the Forum. However, public and political figures refer widely to the importance of health research on developing countries, and, overall, it is thought that the Forum has been successful in creating awareness of the need for increased expenditures on relevant research.. It is much less clear that the Forum has had an impact on global as distinct from national health research priorities. Its core advocacy expenditures of $3.5 million a year could hardly be expected to have a substantial impact on the level and allocation of the current world total of $160 billion in annual spending on health research. Health research spending from developing countries has, however, grown substantially, from an estimated $3.6 billion in 1998 to $5.1 billion in 2005. EFFICIENCY 18. While the evaluation team produced only limited quantitative data, observers generally consider that the Global Forum has been reasonably efficient in its use of funds. Overhead and governance represent about 20 percent of total expenditures. Some concern was expressed by the evaluation team about appearances of excessive staff travel and generous meeting environments for Global Forum activities in developing counties. The evaluation team was unable to correlate increased spending and activity with increased advocacy outcomes. 19. The Forum's financial and budget management are sound, and expenditures have followed budgets reasonably closely. The evaluation team expressed some concern that professional skills requirements for Global Forum staff were not precisely delineated. While the funding of Global Forum core activities has been stable, its total support from donors has fallen. Fund-raising approaches have been reviewed in camera by the Foundation Council, but the absence of a published Global Forum business plan and fund-raising strategy is xxi noteworthy. It has not been possible to analyze the cost-effectiveness of the Global Forum's work programs and activities. GOVERNANCE 20. The Global Forum has a stakeholder-based model of governance -- its principal constituencies are recognized in its statutes. It is sensitive to the importance of ensuring that developing countries have significant voice in the activities and decision making of the Forum. Relative to others such as policy makers and the commercial private sector, health researchers appear to be somewhat over-represented on the Foundation Council. The constituencies have unexploited potential to increase Global Forum legitimacy. Moving the Global Forum's Annual Forum Meetings to developing countries and promoting developing country participation and speakers have helped increase legitimacy. Among donors, the World Bank has long been the predominant influence in the Foundation Council. The Bank's representative currently serves as interim Chair of the Strategic and Technical Advisory Group (STRATEC) of the Council. Overall, aside from the Chair, Foundation Council members appear not to have assumed significant responsibility, either individually or collectively, for the Global Forum's fund-raising needs. 21. Although the Global Forum, as a relatively small NGO, cannot be expected to operate at the governance standards applicable to public international organizations, this GPR finds certain deficiencies in its governance. These include lack of arrangements -- due in part to the statutory requirement that they serve in an individual capacity -- for accountability of Foundation Council members to their constituencies, need for greater external transparency of Foundation Council managerial decisions and financial information, and failure to distinguish adequately between Foundation Council members' performance in their oversight role and in their role of providing technical advice and support to the Forum. 22. Because of the importance of fund-raising and of the very large engagement of the commercial private sector in health research, the Global Forum may need to consider new ways to bring additional partners, especially from the private sector, into its governance. There has been some resistance to engagement with the commercial private sector in the Global Forum, especially among Foundation Council members and particularly in the evaluation team. This appears to be decreasing, with the addition of two members to the Council from private firms. THE GLOBAL FORUM AND THE FUTURE OF RESEARCH ON DEVELOPING COUNTRY HEALTH 23. The principal challenges for the Global Forum as it examines its future role in research advocacy on developing country health are to rethink its goals and roles and its position in the evolving institutional architecture for developing country health research. The environment for the work of the Forum is changing quickly: It faces rapidly shifting epidemiology on the diseases of poor countries and peoples and new awareness of non- communicable diseases without overall declines in infectious diseases; financial resources for health research are growing rapidly; innovative developing countries such as Brazil are emerging on the health research scene; and institutional funders with which the Forum has not had significant prior engagement are playing larger and larger roles. The enormous gap between the human and financial resources available to the Global Forum and those available xxii to other major players in health research (such as the U.S. National Institutes of Health, the commercial private sector, and philanthropic bodies including the Wellcome Trust and the Bill and Melinda Gates Foundation) makes a highly concentrated focus desirable, along with the definition of criteria for selecting among approaches and activities. Hard choices are likely to be required, and some stakeholders are likely to be opposed. 24. In late 2008, WHO initiated consultations on cooperation among eight Geneva-based health research partnerships. In early 2009 the dialogue was broadened and deepened, towards possibilities for some form of merger under the wide umbrella of a partnerships board. More recently the focus has been more on strengthening collaborative links where the partnerships share complementary interests. During these discussions the Global Forum has emphasized its role of watchdog and advocate. While the Consultative Group on International Agricultural Research (CGIAR) sponsors research on a much larger scale than the Global Forum and the other Geneva-based health research partnerships, the CGIAR model of coordinating donor funding for designated research centers and work programs has inspired the responses of some key participants in the new dialogue. The small financial size of the Global Forum in relation to total publicly funded health research, and the almost complete absence of the most important health research funders from the financing and governance of the Global Forum, suggest that it would be virtually impossible for the Global Forum to move alone in the direction of the CGIAR model. 25. Financial and institutional sustainability are likely to represent continuing challenges for the Global Forum, regardless of its position in the future institutional architecture for health research. While the financing of Global Forum core activities has been stable, total donor support for core activities and Global Forum initiatives and networks has declined from a high point in 2004. The Bank's DGF Council has been dissatisfied with HNP DGF submissions, and reduced the HNP DGF funding level and shifted two programs, including the Global Forum, from long-term support under Window 1 to limited grants under Window 2. The Bank currently plans to terminate DGF support to the Global Forum in FY11. As WHO begins implementation of a new research strategy, as TDR expands beyond a limited number of diseases, and as the Forum's CEO retires at the end of 2009, one possibility would be for the Global Forum to declare success and to offer to merge with another organization while ensuring that the independent watchdog role remains intact. WORLD BANK PERFORMANCE IN THE GLOBAL FORUM PARTNERSHIP 26. The World Bank has played many roles in the Global Forum, starting with active engagement and use of its convening power in the dialogue among stakeholders that led to establishment of the Forum in 1998. In the early years the Bank's HNP leaders expected that the Forum would increasingly assume the role of raising money, allocating funding to high priority activities and coordinating health research generally. The HNP Sector Board expected that the Bank would exit from separate international health R&D grants and channel its health R&D funding through the Global Forum. Key Bank HNP leaders expected the Forum to develop along the lines of the CGIAR. The Bank's role, and the Forum more generally, have not evolved in this way, at least in part because of the relative power, both within the Bank and outside, of disease-specific activists compared to those who have an overall view of health and health research. Nonetheless, the Bank has been by far the most xxiii important financier of the Global Forum. Its DGF financing, at 45 percent of total donor support, has significantly exceeded the DGF guideline of 15 percent. The HNP Hub and the DGF staff appear to share responsibility for this violation of Bank DGF guidelines. 27. The Bank's representative to the Global Forum, currently the Africa Region Senior Health Adviser, has been a member of the Foundation Council since inception of the Global Forum, and its representative is currently interim Chair of the Global Forum's Strategic and Technical Advisory Group (STRATEC). Within the Forum, the Bank's representative has consistently provided technical and managerial oversight of Global Forum activities, along with a valuable bridge to broader development concerns. The Bank has played initiating, financing and oversight roles in many specific Forum activities and initiatives, and has provided general and activity-specific advice and guidance, assistance on financial management, and overall organizational support well beyond the DGF. The Bank's engagement with Global Forum-related activities becomes even more complex when the sub- grantees of DGF resources channeled through the Global Forum are taken into account. Conflict of interest issues have arisen for the Bank, both in its direct role(s) of technical support and independent oversight of the Global Forum as well as in consequence of providing DGF support to a Global Forum sub-grantee while a senior Bank staff member was serving on the sub-grantee board. 28. The Global Forum is relevant to the Bank's corporate and HNP sector strategies and priorities. However, the Forum was much more relevant to the Bank's 1997 HNP strategy than to its 2007 HNP strategy. The Bank's performance in the Global Forum partnership was well regarded throughout the interviews carried out for this GPR. However, its oversight budget and expenditures have been substantially less than what is reflected in the very substantial staff time devoted to work on the Global Forum. Other Bank work program tasks appear to have cross-subsidized work on the Forum. 29. The Bank's participation in the Global Forum partnership poses reputational risks, but brings benefits to the Bank. The risks arise from the multiplicity and complexity of the health research partnerships in which the Bank is engaged in addition to the Global Forum, and the fact that the Bank has become the primus inter pares among the partners in the Global Forum and is reducing its support. Benefits include the global increase in awareness of the importance of health research to developing countries, some influence on health research priorities as a consequence of earmarking resources provided to the Forum, facilitating funding by others, and providing a forum for disseminating Bank research and for responding to criticism of the Bank among external HNP activists and observers. 30. Overall, this review concludes that -- despite the many weaknesses in the performance of the Global Forum and of the Bank in it -- the benefits to the Bank and its clients of its participation in the Global Forum partnership have exceeded the costs and risks. LESSONS 31. The following lessons can be distilled from the Bank's experience with the Global Forum partnership: xxiv The multiplicity and complexity of health research partnerships in which the Bank is engaged pose reputational risks to the Bank and readily give rise to conflicts of interest in the roles played by Bank staff. This is not because the partnerships lack value but because their very multiplicity and complexity require multiple roles and demand a level of oversight and liaison that exceeds the small oversight budgets made available by HNP sector management. Since the oversight work has little internal audience, the staff incentives to carry it out are weak. By their very nature, advocacy and knowledge networks such as the Global Forum tend to have problems of focus and selectivity, and difficulty in establishing a clear results framework. This makes the effort to do so all the more important. In the Global Forum the absence of an agreed results framework allowed an environment of relatively low overall institutional accountability for results. A global program that does not actively engage the most important actors, and bring them into its governance and, where possible, its financing, is likely to have great difficulty in being effective. In the Global Forum the absence of major private commercial, philanthropic and public health research financiers from its Foundation Council and from its donors reduced its effectiveness. Transparency and specificity in the relationships and responsibilities among partners and grant recipients is critical to effective partnerships. In the Global Forum the relationships and responsibilities lacked clarity in respect of sub-grants and sub- grantees. This confusion -- reflected in a vision but an unclear umbrella role for the Global Forum -- contributed to the lack of respect for the Bank's requirement that the DGF financing represent a maximum of 15 percent of donor funding. Governance issues in a small NGO such as the Global Forum can be as challenging in their own way as in a public international organization. The several roles played by board members call for special attention to preserving their independent oversight role. The Bank's deep engagement in the Forum led it to lose its independent, arms- length oversight role. Basic changes in course are extremely difficult to effect in established institutions, especially where the evidence base for change is limited. Mature partnerships like the Global Forum and COHRED develop a life of their own. Making fundamental institutional change has proven to be extraordinarily difficult despite what appear to be the obvious benefits to be derived by the partners from such changes. This underscores the difficulty, the importance, and the need for assertive leadership in the dialogue now under way on rationalization of the many global health research partnerships. Evaluation Terms of Reference and teams should routinely be expected to include financial, economic, and/or business management perspectives and expertise, and those responsible for commissioning evaluations should guard against the risk that the evaluation team will not include such perspectives and expertise. 1 1. Program Objectives, Activities, Financial Resources, and Governance Objectives, Vision, Mission, and Activities1 1.1 The Global Forum for Health Research was established in Switzerland in 1998 in response to the growing awareness among policy makers in industrial and in developing countries that research related to the health problems affecting developing country populations was receiving inadequate attention on the global agenda. The Forum is an independent international non-governmental organization, called a foundation in Swiss law, with a mission to help focus research efforts on the problems of the poor. The early strategy of the Forum concentrated on gathering and disseminating information and evidence on expenditures for health research, on identifying major gaps and priorities in health research for developing countries, and on developing communication channels to bring this information to the attention of those responsible for policy formulation and resource allocation for health research. The Global Forum became known as the principal advocate for bridging the "10/90 gap" (Box 1). Box 1. The Global Forum and the "10/90 Gap" Until recently, relatively little was known about the investments made in research for health in low- and middle-income countries. In 1990, the Commission on Health Research for Development/a estimated that only about 5 percent of the total of $30 billion spent on health research in 1986 was applied to the health problems of developing countries, where 93 percent of the world's burden of preventable mortality occurred. Of the $1.6 billion spent on health problems of developing countries, 42 percent originated in the developing countries themselves, with 7 countries accounting for three quarters of this sum. The discrepancy between spending on health research relating to the problems of the poor and on other health research later became symbolized in the expression the "10/90 gap," and the Global Forum summarized its mission as "helping to correct the 10/90 gap." Over time, however, it became apparent that it was more appropriate to talk of "imbalance" in resources for health research or, as the evaluation team for the second external evaluation of the Global Forum stated, that the "10/90 gap" is more a "metaphor of global inequity than an accurate, measurable figure." The Global Forum continues to use the expression "10/90 gap" in quotation marks, as a symbol rather than quantitative measure of the imbalance, and the latest Global Forum documentation starts with the new tagline "... because health equity is a priority." /a Commission on Health Research for Development. Health Research: Essential Link to Equity in Development. Oxford University Press, New York, 1990. 1.2 Global expenditure on health research has increased more than four times in the last 20 years to above US$160 billion per year (Figure 1). The available expenditure data, disaggregated in Annex Table 5, show only the sources of funding, and not their purpose. A superficial reading of the data might suggest that only 3­4 percent of world health research expenditures is devoted to the health problems of low and middle income countries. 1. This section of the report draws on the Global Forum's 2007 Operations Report and Audited Financial Statements. 2 Figure 1. The Growth in Funding for Health Research, 1986­2005 US$ Billions 180 160 Not-for-profit Public 140 Private 120 100 80 60 40 20 0 1986 1992 1998 2001 2003 2005 Source. Burke and Matlin 2008. Additional relevant resources come from special initiatives of private industry, the not-for profit sector, and the public sector. However, imbalances and inequities in health research spending persist, and the picture has grown increasingly complicated due to epidemiological changes. Developing countries now bear a multiple burden of old, new and re-emerging communicable diseases; steep increases in levels of non-communicable diseases; and rising rates of injuries. This multiple burden challenges the world to develop new and improved solutions and presents a research agenda that spans the biomedical sciences (creation of new drugs, vaccines, diagnostics and medical appliances), health policy and systems, the social sciences, and operational research. 1.3 Under its most recent vision and mission statement (Box 2) the Global Forum seeks to focus more attention and resources on health research that will benefit poor populations. The Global Forum's overall objective is to achieve improvement in the allocation of research funds, support of better priority setting processes and methodologies, promotion of relevant research, support for concerted efforts in health research and dissemination of research findings. In pursuing this, the Global Forum has engaged in gathering information and evidence concerning expenditures on health research, the identification of major gaps and needs in health research for developing countries, the elaboration of tools, and the development of channels of communication to bring this information to the attention of those making policy and controlling resources for health research. The Global Forum works to close the gaps in health research and focus research efforts on the health problems of the poor by bringing together key actors and creating a movement for analysis and debate on health research priorities. It aims to change the priorities and allocation of resources for health research, to encourage new resources to be directed to research in neglected areas and to foster research in these areas. 3 Box 2. The Vision and Mission of the Global Forum in 2008 Vision: A world in which the potential of research and innovation is fully utilized to address the health problems of the poor. Mission: To play a leadership role in catalyzing global research applied to the health problems of the poor through 1. Engaging current and future high-level decision makers from high-, middle-, and low-income countries 2. Brokering coherence and partnerships between global players in research and innovation 3. Promoting relevant research on health and health equity 4. Advocating increased resources for relevant research and innovation by all sectors 5. Encouraging the use of evidence in policy and decision making 6. Stimulating the dissemination of research findings in ways that will enable their utilization. Source: Downloaded from GFHR Web site February 9, 2009. 1.4 The Global Forum commissions and executes analytical work on the flow of financial resources for health research, health research priorities, and related matters. The wide scope of the Global Forum studies program is indicated by the list of its publications in Annex D.2 The Secretariat, the Foundation Council, and the Global Forum Strategic and Technical Advisory Group (STRATEC) work closely together on the Forum program of research and analysis. Proposals for studies are vetted by STRATEC and approved by the Foundation Council. 1.5 Since the creation of the Global Forum in 1998, it has stimulated the creation of seven "Initiatives" on specific health issues or conditions (Box 3), largely under the legal umbrella of the Forum and often with the encouragement of the World Bank. The initiatives had their origin in the need for identification of specific research gaps. The initiatives (Annex Table 6) have generally had separate governance, management and operational organs to which authority has been delegated by the Foundation Council. The operations and activities of the initiatives have been performed by staff employed by the organizations that host them. In Box 3. Initiatives Created by the Global Forum Initiative for Public-Private Partnerships in Health Alliance for Health Policy and Systems Research Initiative for Cardiovascular Health in Developing Countries Sexual Violence Research Initiative Child Health and Nutrition Research Initiative Global Network for Research on Mental and Neurological Health Road Traffic and Injuries Research Network Source: Annex Table 6. 2. It should, however, be noted that as an advocacy organization the Global Forum is not significantly engaged in commissioning or executing health research in any but the broadest sense. 4 consequence of the exit strategy being pursued by the Foundation Council in recent years, the Global Forum has gradually decreased its association with the initiatives. Their expenditures are no longer accounted for in the financial records of the Global Forum nor reported in the audited financial statements and operations reports. From 1999 through 2007 the World Bank provided nearly $17.8 million in earmarked Development Grant Facility (DGF) funding to support Global Forum initiatives, networks and special projects (Annex Table 22). 1.6 Communication is central to the work of the Global Forum. One of its principal communication activities has been the holding of an Annual Forum Meeting (AFM) and marketplace (Annex C), where health research results, problems and priorities are presented and discussed by a variety of researchers, policy makers and decision makers. Other principal communication activities include publication and dissemination of advocacy documents and engagement with different stakeholders through diverse meetings. The AFMs afford opportunities for presentation of new research on developing country health problems, and particularly for contact and communication among developing country health researchers and health research sponsors and financiers. 1.7 The AFMs are especially appreciated by participants for the networking opportunities among researchers, donors, and policy planners from developing and developed countries.3 In the early years there were several hundred participants, more recently a little under 1,000. AFMs are increasingly held in developing countries, and participation from low- and middle- income countries (LMICs) has generally been greater than 50 percent. Researchers, NGO/CSO staff, and government officials have dominated the participant lists, with but small numbers (1­2 percent) from private firms. Evaluations of the AFMs by participants consistently show high ratings for interest in the topics and relevance of the sessions to participants' work (Figure 2). 1.8 The latest conference in the AFM series was the Bamako Ministerial Forum on Research for Health in November 2008 co-sponsored by the Global Forum and other agencies, including the Bank. The Bamako conference, and its most recent prior comparable conference, the 2004 Mexico Health Research Summit/AFM8, contributed to redirecting health research towards health systems research, partly as a result of the emphasis on health systems in the World Bank's 2007 Health, Nutrition and Population (HNP) Strategy. These conferences also encouraged movement towards multi-sectoral "research for health" and away from the single sector orientation of most health and medical research. Beyond its large conferences, the Global Forum (Annex J) works to establish new global and national coalitions to influence research policy and prioritize or strengthen capacities for research on priority health issues in developing countries. The Forum has a large network of contacts in LMICs and OECD countries, creating channels to influence decision making on research for LMIC health priorities. 1.9 As an advocacy organization, the Global Forum devotes major emphasis to communications and publications, including a Web site from which its publications can be downloaded or ordered in hard copy without cost to the recipient. A database of some 3. Almost half of the participants in Forum 11 reported that the purpose of their participation was to meet fellow researchers, donors and policy planners from different parts of the world (GFHR FC-08-1-0.1). 5 Figure 2. Participants' Evaluation of the Global Forum Annual Forum Meetings: How Would You Rate the Overall Content of the Meeting? (1 = low, 5 = high) Interest Relevance/Importance to Your Work Score 1 Score 2 Score 3 Score 4 Score 5 Score 1 Score 2 Score 3 Score 4 Score 5 Forum 8 Forum 8 Forum 9 Forum 9 Forum 10 Forum 10 Forum 11 Forum 11 Bamako 2008 Bamako 2008 0% 20% 40% 60% 80% 100% 0% 20% 40% 60% 80% 100% Source: Global Forum evaluation data in Annex C. 14,000 contacts is maintained and used for communication with Global Forum constituencies and others. During recent years, approximately 2 million pages on the Forum's Web site have been viewed per year, with over 400,000 visits to the Web site by over 200,000 visitors. Over the past three years there have been 1.7 million downloads from the Global forum's Web site (Annex Tables 16 and 17). Financial Resources 1.10 Funds flowing to and through the Global Forum fall into three categories: (a) core grant resources for financing the routine operations of the Forum; (b) earmarked funds managed by the Forum for special studies, projects or initiatives sponsored by the Global Forum; and (c) sub- grant resources from the World Bank Development Grant Facility (DGF) flowing through the Forum for use by third parties whose programs are compatible with those of the Global Forum. The sub-grants are governed by letters of agreement between the sub-grant recipient and the World Bank. The activities of sub-grantees are not part of the Global Forum's work program, and they are reviewed in this study only in Chapter 4 on the performance of the World Bank in the Global Forum partnership. 1.11 The Global Forum each year devotes approximately $3.5 million to its core activities of research and programs, annual forum meetings, and other information and communications (Table 1).4 Overhead expenditures on administrative support, governance and executive management amount to approximately one-fifth of core function spending. 1.12 For its financial resources the Global Forum depends almost entirely on funding from bilateral donors and the World Bank. The only non-governmental source of core funds for 4. Donor support for initiatives and special projects increase the annual expenditure levels on activities for which the Global Forum retains some level of direct responsibility to a little over $6 million. 6 Table 1. Global Forum for Health Research: Average Annual Expenditures on Core Functions, by Category, 2001­07 Annual Average 2001­07 Share (US dollars) (%) Research and Programs 1,248,945 36% Annual Forum Meeting 785,975 23% Information and Communication 733,046 21% Sub-total: Substantive Activity 2,767,966 79% Administrative Support Services 426,823 12% Governance and Executive Functions 321,114 9% Sub-total: Overhead 722,730 21% Grand Total 3,490,686 100% Source: Annex Table 21. the Forum in the years from 1999­2007 was the Rockefeller Foundation.5 Total donor funding over this period managed by the Global Forum amounted to $56 million (Table 2). This figure includes special studies and initiatives but does not include the sub-grants from the World Bank DGF for specific recipients, including -- to mention but one example -- the Medicines for Malaria Venture (MMV), a product development public-private partnership that was the subject of a separate IEG global program review.6 Excluding the sub-grants of the Development Grant Facility (DGF), Bank DGF resources represented 45 percent of Global Forum resources for core activities and initiatives over the 1999­2007 period. Governance, Management, and Organization 1.13 The Global Forum has a typical governance structure for an NGO, with a Board of up to 25 people known as the Foundation Council. The Foundation Council is self-perpetuating. Term limits7 require constant consideration of new candidates, and lead to turnover in FC membership. The Foundation Council normally meets twice a year, including one session immediately following the Annual Forum Meeting. Aside from its board, the Global Forum has no membership structure of individuals and organizations. The Foundation Council Chair reportedly devotes 20­30 days per year to the work of the Global Forum without compensation, and other FC members much less. There have been substantial variations among individual chairpersons in the style and approach that they have brought to their role. The immediate past Chair, from Sri Lanka, was widely perceived to take a somewhat distanced and passive position, and the present Chair, from the United Kingdom, is bringing a more directly engaged style. Former Chairs include a former World Bank HNP Director, and a former TDR 5. The Bill and Melinda Gates Foundation contributed a little over $1 million for the Global Forum initiative on product development public-private partnerships. 6. IEG ­ World Bank, July 6, 2007. 7. The normal term is three years, renewable once. For institutions making substantial financial contributions to the Global Forum, FC membership may be extended. Similarly, the Chairs of the Foundation Council and of STRATEC may serve on the Council longer than the normal six year maximum. The heads of TDR and of COHRED are ex officio members. 7 Table 2. Donor Support for the Global Forum for Health Research, 1999­2007 Donor 1999­2007 ($ thousands) Share (%) Canada ­ IDRC 760.229 1.4 Denmark 786.267 1.4 Ireland ­ Irish Aid 883.445 1.6 /a Mexico ­ MOH 400.000 0.7 Netherlands 368.750 0.7 Norway ­ MFA 5,007.279 8.9 Rockefeller Foundation 2,425.000 4.3 Sweden ­ SIDA 4,102.866 7.3 Switzerland ­ SDC 4,815.251 8.6 World Bank DGF 25,270.000 45.1 Designated Contributions 1,201.137 2.1 Other Income ­ Core 1,617.506 2.9 Other Income ­ Initiatives 8,337.383 14.9 Grand Total 55,975.113 100.0 Number of donors 10 Source: Annex Table 22; includes core support, Global Forum initiatives, projects and networks; excludes DGF sub-grants. /a For Annual Forum Meeting held in Mexico City. Director. Two of the four persons who have served as FC Chair came from developing countries, and two have been women. A senior World Bank HNP staff member has been a member of the Foundation Council ever since the Global Forum was established. 1.14 The Global Forum's Strategic and Technical Advisory Group, STRATEC (Figure 3) represents a unique element in the Global Forum's governance. The STRATEC is composed of a limited number of Foundation Council members, usually about 6. It is advisory to both the FC and the Secretariat, and reviews research proposals of the staff before they are considered by the FC. More generally, STRATEC appears to serve as a preparatory body for the larger FC. At the initiative of the Bank, a Finance Committee was established several years ago. This will become a Finance and Audit Committee in 2009, with co-opting of an external member with audit experience. Other ad hoc committees of the FC have been established from time to time, for example, for nomination of new FC members and to oversee the Global Forum's second external evaluation. 1.15 The Forum statutes provide for the representation of 9 constituencies in the Forum's work (Box 4). Special efforts are consistently made to ensure engagement of developing country researchers and policy makers in Annual Forum Meetings, as presenters and as regular participants. The Global Forum has paid increasing attention in recent years to potential conflicts of interest affecting its work. Prior to each FC and each STRATEC meeting each member is required to sign a conflict of interest statement.8 8. The statement requires each participant to confirm that she or he is not aware of any conflict of interest or has disclosed such a conflict. 8 Figure 3. Governance and Management of the Global Forum Foundation Council (FC) Strategic and Technical Secretariat Advisory Group (STRATEC) Accountability Advice Source: Global Forum for Health Research. 1.16 In its 10 years the Global Forum has had only two chief executives, one from Switzerland and the other from the UK. Its staff is organized into three units dealing respectively with research and programs, communications and external relations, and finance Box 4. Constituencies of the Global Forum The founding documents of the Global Forum establish 9 distinct constituencies, which the Forum endeavors to respect in its communications and in identification of potential members of the Foundation Council: Government policy makers UN and other multilateral aid agencies Bilateral development cooperation agencies Foundations International and national NGOs Women's organizations Research institutions Private firms, especially pharmaceutical enterprises Media As of April 2008 the FC was dominated by representatives of research institutions and research policy makers, with 10 of the then serving 20 members coming from these constituencies. One FC member, the Editor of the prestigious British health journal, The Lancet, came from the media. One FC member came from the private sector. There were 8 women FC members, and 11 of 20 members came from low- and middle-income countries. The Global Forum database of contacts is also disaggregated by constituency, with a heavy emphasis on researchers. Source: GFHR documents. 9 and administration. The Forum staff, initially led by an Executive Secretary, now by an Executive Director, has grown slowly over the years. As this report is written, the staff consists of about 20 people. The Executive Director frequently consults the chairs of the FC and of STRATEC on issues where their knowledge and contacts might be valuable, where FC views might be important, where strategic or policy questions arise between meetings or where confidential advice is needed on sensitive matters. A number of the staff are often in touch with the FC or STRATEC chairs or members on particular matters, such as -- currently -- work on the development of indicators where one FC member and her institution have particular expertise. Generally speaking, the work of the Secretariat is carried out through activity teams that cut across the organizational units in a matrix approach to task execution. Key Global Forum Partnerships: WHO and COHRED 1.17 The Global Forum has long had a close partnership and multiple working relationships with the World Health Organization (WHO). The failure of WHO leadership to engage successfully on health research issues during the 1990s following the report of the 1990 Independent International Commission on Health Research and the 1996 report of the WHO ad hoc Committee on health research, as well as the Bank's World Development Report 1993 on health, contributed significantly to the willingness of international stakeholders to establish the Global Forum. Despite some tensions, the Forum today collaborates very closely with WHO at many levels, and WHO provides administrative support services such as office technology as in-kind contributions to the Global Forum. In late 2008 WHO was completing the elaboration of a first WHO research strategy for review and approval by its World Health Assembly in 2009. The Global Forum Executive Director participated in advisory meetings on the strategy. 1.18 Because of the overlap in mandates -- complementary in some respects, competitive in others9 -- the Global Forum gives continuing special attention to its collaboration with the joint WHO, UNICEF, World Bank and UNDP program of research and training on tropical diseases (TDR). Since TDR has expanded its focus beyond a limited number of specific diseases with emphasis now on "the infectious diseases of needy populations" and growing attention to capacity building and the social contextualization of research, the TDR-Global Forum relationship has become particularly important. The TDR Director is a member of the Global Forum Foundation Council, appointed by the Director-General of WHO to fill the ex officio position allocated to WHO. 1.19 The Council on Health Research for Development (COHRED), a Geneva-based NGO, was established shortly before the Global Forum, in 1993, to promote and carry out practical operational health research and health research capacity strengthening at the country level. Like the Global Forum, COHRED is dependent on donor financial support. In recent years, the Global Forum and COHRED have been working increasingly closely. A memorandum of understanding was signed by the Board Chairs and Executive Directors of 9. Complementary because of the TDR emphasis on financing execution of health research and the GFHR on advocacy for health research, competitive because the much larger size of TDR, at about $50 million a year, inevitably leads it to give increasing attention to advocacy activities in its own work program. 10 the two organizations in March 2005. COHRED offices were relocated to be in the same building as the Global Forum, and joint approaches for donor support have been made. The COHRED Director is an ex officio member of the Forum FC. The Global Forum at 10 Years 1.20 In 2008 the Global Forum celebrated its tenth anniversary with a number of special products, events, and actions demonstrating its analytical, communications and advocacy capacity. A symbol of the ten years of activity was developed and used consistently on all publications, as well as on the Web site banner. The symbol enabled the Global Forum to make reference to its ten years of existence at any meeting attended or in any presentation made during the year. Reactions were reported by the Global Forum to be very positive. A searchable CD-ROM of Global Forum publications from 1999­2008 was widely distributed. The Global Forum published a tenth anniversary public relations pamphlet with photos and charts, which summarized a key event during each of its 10 years.10 An anniversary reception in May 2008 led to the opening of a "book of birthday wishes." An electronic version was subsequently opened on the Web site and Global Forum contacts were emailed to invite their contributions. 2. The Second External Evaluation of the Global Forum 2.1 The Foundation Council commissioned an initial external evaluation of the Global Forum in 2001. The evaluation team (ET) found that the Global Forum "seemed to have succeeded in creating awareness about the 10/90 gap by repeated use of its message" but that its strategy of reaching researchers directly rather than through their governments might have resulted in insufficient reach to policy makers. The evaluation stressed the continuing importance of the Forum's mission to promote greater attention to health research for the benefit of the poorest, most disadvantaged and marginalized people of the world. The ET found the Global Forum to be a neutral entity with credibility and mandate to bring partners from its different constituencies around the table, and stressed the importance of encouraging critical debate and reassessment of methodologies and policies. It underscored that the Global Forum's role should be to promote health research more than to undertake it, and suggested that the Forum should declare success on a research initiative as soon as it is established with partners and initial funding and cap the total number of initiatives. The ET concluded that the Annual Forum Meetings were extremely useful but needed more attention to the market place concept of discussions and interactions among participants from diverse constituencies. The ET recommended the establishment of closer relations with WHO and COHRED, and the creation of a Scientific Advisory Board. While summarizing the main recommendations of the initial evaluation, the team for the second evaluation made no assessment of their impact. 10. Because health equity is a priority ­ Global Forum for Health Research 1998 ­ 2008, 10th Anniversary. 11 Scope, Process, and Approach of the Second Evaluation 2.2 The second external evaluation of the Global Forum was launched in March 2005 by the Foundation Council with the adoption of terms of reference and creation of an FC Sub- Committee to oversee the evaluation. The evaluation covered the period 2002­05. Although the initial schedule called for presentation of the evaluation study to the FC in March 2006, the final report was not submitted by the ET until February 2007 and only issued by the Forum with a foreword by the Foundation Council Chair in August 2007. Looking back, one observer commented that the timing of the evaluation may have been poor, without indicating when might have been a more opportune time. 2.3 As stated in the terms of reference, the purpose of the evaluation was to "assess the overall relevance, appropriateness, adequacy, efficiency and effectiveness of the Global Forum in relation to its current objectives, strategic approaches and stated values." The terms of reference (TOR) did not call on the team to evaluate Global Forum governance. The evaluation was to comment on the "indicative impact" of the Global Forum but to devote "the main focus of its attention to questions relative to the future of the Global Forum." The TOR called for the evaluation to take into account findings from the IEG study of World Bank approaches to global programs.11 2.4 As with the initial evaluation, a three-person evaluation team (ET) of health researchers12 was agreed by the Foundation Council Sub-Committee and the Executive Director, following extensive consultations but under some time pressure to permit participation of at least some ET members in the September 2005 Mumbai Annual Forum Meeting. There was no competitive invitation of proposals for the external evaluation, and no member of the ET had a background on economics or finance. 2.5 The approach of the Evaluation Team included a desk review of GFHR studies and FC documents, analysis of over 500 questionnaires on the general impact of the Forum sent to 4,000 people in the Forum database, analysis of over 100 questionnaires sent to other researchers, institutions, and health officials, and analysis of the results of four specific questionnaires on selected topics with a low response rate. The ET conducted 64 confidential, structured interviews with key informants, and interviewed 15 Forum staff. The evaluation was budgeted for $90,000 and ultimately cost $105,000, largely financed by the World Bank from DGF resources. 2.6 The final text of the evaluation report can readily be downloaded from the Global Forum Web site. The foreword to the report by the Foundation Council Chair welcomed its overall positive findings, summarized certain points in the evaluation report, and stated that the ET expressed its own strong views, beyond its mandate, on the question of global versus 11. Addressing the Challenges of Globalization: An Independent Evaluation of the World Bank's Approach to Global Programs, Phase 2 Report. Operations Evaluation Department, 2004. 12. The members of the team are reported in a DGF progress report to be have good knowledge and understanding of health and health research, good knowledge and understanding of global players in health research, understanding of the GFHR's underlying values, expertise in evaluation methodologies, good writing skills, international stature, gender balance, and fluency in English, with at least two of the three members from a developing country. 12 country-level action. The foreword concluded on a positive note with emphasis on development of a new Global Forum strategy. Independence and Quality 2.7 The evaluation team was independent of the Global Forum FC and Secretariat. No ET member had previously been a beneficiary of any Global Forum support. Indeed, the Team was so independent that it administered its evaluation questionnaires entirely separately from the Forum Secretariat and -- beyond the draft report itself -- was unwilling to provide disaggregated quantitative results of its analysis, such as, for example, of individual Forum initiatives, on the basis of questionnaires. 2.8 The draft evaluation was reviewed in detail with the Secretariat and Foundation Council. Consistent with the status of the Global Forum as a mature partnership with over five years of operations, the evaluation paid attention not only to outputs but also to outcomes and strategic issues. It did not address sustainability of the Forum itself, though it did address sustainability of Forum initiatives. However, the extremely long period taken to produce and release the final report, combined with its inevitably backward-looking character at a time when the landscape of global health research was changing rapidly, meant that much of the report was overtaken even before it was issued. 2.9 The ET was thorough in its work and addressed all the many issues raised in its lengthy, detailed, and highly specific but perhaps insufficiently focused TOR. The final evaluation report shows, indirectly, that it was the product of extended discussions, over many months, with the Secretariat and the FC. The report contains large numbers of pertinent individual observations and recommendations, but its main thrust is occasionally difficult to grasp due to the lack of an over-arching, limited number of findings and recommendations. This appears to be partly the result of an unclear results framework for Global Forum activities (as discussed below), partly the consequence of an ET mandate that called for detailed review of a large number of issues, and partly the result of the ET dialogue with the Global Forum on the draft report. Interviews carried out for this review make clear that there were important disagreements, difficult personal exchanges, and unpleasant meetings between the ET and the Global Forum, including observations that the ET went beyond its TOR in commenting on some personnel matters.13 The impetus for revisions to the draft report appears to have come from the Forum Secretariat more than the Foundation Council. 2.10 The final evaluation report is deficient in several respects. The report contains little financial analysis or review of the efficiency of the Global Forum. The work and management of the Forum are not benchmarked against comparable advocacy organizations, such as the Global Forum on Agricultural Research, or compared with the advocacy activities of other health research bodies. 13. In this connection it should be noted that the UN Evaluation Group norms summarized in the IEG Sourcebook for Evaluating Global and Regional Partnership Programs state that evaluators are not expected to evaluate personal performance of individuals (IEG ­World Bank 2007). The GFHR reports that the initial draft of the evaluation contained references to the performance of identifiable members of the GFHR research and programs staff, and that this was a major reason for the considerable delay in finalization of the ET report. 13 The evaluation relies excessively on interviews and surveys. While the report discussed the AFMs and certain initiatives, the ET did not assess the quality of specific outputs. The evaluation team failed to produce an inception report. Such a report might have provided an opportunity for strategic guidance by the FC's Evaluation Sub- Committee. The ET needed to acknowledge possible biases in its sources of information. Most of its interviewees were close to the Forum, but this source of bias was not acknowledged. The ET was unwilling to reveal disaggregated survey data. Yet, the credibility of such evaluation studies depends greatly on the ability of others to validate evaluation findings, subject naturally to protection of the confidentiality of individual sources. Despite its independence in executing the evaluation, the ET was not fully independent at the review stage.14 Interviews for this study and the extended delays in completing the report suggest that the ET went beyond the standard of submitting a draft report, receiving comments, and then immediately completing the final text. While the ET discussed the Global Forum management, the ET did not explicitly evaluate the Forum's governance, and its evaluation of Forum efficiency was weak. 2.11 According to several sources, at least some people sought by the Global Forum as evaluators were not available. Time pressure in ET selection and for report completion may have contributed to the weaknesses in the ultimate product. Interviews conducted for this GPR give a more positive impression of the Global Forum than does the evaluation. The ET raised issues and expressed views with considerable political content and personal judgment.15 One observer remarked that the evaluation report is marked by the possible bias of an a priori view that the Global Forum should be engaged at the country level and that this made it hard to recognize the Global Forum in the report. The team's TORs were insufficiently focused. A clear results framework in the TOR, or agreed indicators against which the evaluation team could objectively measure progress, might have helped to obviate such problems. A skills mix not limited to health research and including expertise in finance or economics might have increased the quality of the ET's report. Findings and Recommendations and Global Forum Response 2.12 The principal findings and recommendations of the evaluation and the program response, as summarized by the GFHR (Table 3), leave out one notable controversy in the ET's work, namely its view of the Forum-COHRED partnership in the larger context of its view that the Global Forum should increasingly engage at the country level. The evaluation team was briefed by the FC Evaluation Sub-Committee on the development of an MOU 14. The GFHR maintains that the ET remained fully independent even at the review stage, to the point of ignoring points made by the GFHR, including clear cases of factual error. 15. For example, the ET report identifies research gaps including: (a) intellectual property and the role of the pharmaceutical industry in the exploitation of vulnerable populations in LMICs for clinical trials of dubious ethics; (b) barriers for the implementation of well-known research results into actions for better health; (c) implementation of a human rights approach to health research and its applications for better health; and (d) barriers to the access by the poor to products resulting from research. 14 between the GFHR and COHRED, and plans for joint ventures. Nonetheless, its final report concluded, without a specific recommendation, that "the most likely scenario in the near future would be the existence of only one strong organization dealing with the needs of health research in LMICs." Prior to the evaluation, SIDA had already informed the Global Forum and COHRED that the continuation of its funding after 2006 would be contingent on a merger of the two organizations. In the absence of such a merger, SIDA's financial support for the GFHR and COHRED ceased in 2007. Since the time of the evaluation the Global Forum and COHRED have worked increasingly closely together. An MOU was signed in 2005 and a consultancy study16 on enhancing collaboration was carried out in 2006­07. SIDA's financial support has not been renewed. 2.13 The Evaluation Team also saw a "gap between the language of Global Forum documents and the content of the interviews conducted with key players and partners and the information obtained from the field." For the ET, it was "as if there were two discourses: the one expressed in the documents and language of high-level meetings conducted with global players, and the other one expressing the reality and expectations of health researchers, health research policy-makers and populations of developing countries." To bridge the gap, the ET found a need for the Global Forum to "immerse itself in the concrete realities and needs of the populations of LMICs to catalyze change." This perspective was not accepted by the Foundation Council (Table 3). 2.14 The GFHR evaluation team also considered that the Annual Forum Meetings should be held every two years rather than annually, with regional meetings in the intervals. The Foundation Council has carefully considered this, and decided that at least through 2011 the GFHR will continue to have Annual Forum Meetings. However, the Global Forum is giving increasing attention to planning other types of meetings, including particularly high-level informal sessions with decision makers. Impact of the Second Evaluation: A Revised and Evolving Strategy 2.15 As the principal positive result of the second external evaluation, the GFHR Secretariat and Foundation Council have had an intensive dialogue, and many iterations of documents, on the Global Forum Strategy. Key questions considered included whether the Forum should focus on health research for neglected diseases (where it was noted that the Forum's concern has been wider than infectious diseases); on the health needs of populations in developing countries; or on the health of the poor, disadvantaged and marginalized in all countries.17 It is not known whether this dialogue would have taken place, in the same way, without the evaluation.18 Probably a new strategy would have been prepared but its content and the preparation process were certainly influenced by the ET. 16. The external consultancy was engaged to identify opportunities for collaboration between the Global Forum and COHRED and especially for "closing the loop" at the interface between the global and country-based perspectives that the two organizations represent. Representation on each other's governing bodies and meetings of the senior management teams are among the institutional mechanisms that have been adopted to ensure continuing dialogue. Collaborative activities include working together on global and regional meetings and joint publications (Annex E). 17. FC document 7.1, "Focusing the Global Forum for Health Research," nd. 18. One interviewee stated that the report had not been used by the GFHR. 15 Table 3. The Second External Evaluation: Major Recommendations and Global Forum Response Evaluation Recommendations Program Response The FC should develop a five year strategic plan of A new Strategy 2008­14 has been developed and is action, including increased action at the country level. now being implemented. The FC did not agree with the It may not be cost-effective to hold AFM meetings on ET's view that the Global Forum should become an annual basis. directly active at country level, since this is the sphere of activity of COHRED. The FC concluded that AFMs should be held at least annually through 2011. The FC should revisit the appropriateness of "10/90 In the new Strategy: gap reduction" as a relevant activity and "redefine its "Helping correct the 10/90 gap" is no longer the Global /a goals in ways that are measurable and attainable." Forum strapline. The research needs of LMICs should be addressed as Measurable indicators and milestones of progress are defined by the countries and not based on perceptions under development. or prescriptions from outside. This locally-based Stakeholder inclusion and self-determination of country bottom-up approach is essential for all the Global priorities are emphasized, within a framework of Forum program development. national health research systems. The FC should review current procedures regarding The FC continually reviews and updates practices and management practices and decision-making processes, procedures in its own work and in its engagement with to facilitate greater discussion, transparency and the Secretariat. Several major changes have been involvement by FC members in the strategic functioning adopted by the new Chairs of FC and STRATEC to of the Global Forum. Periodic routine "closed door improve and streamline decision making and enhance sessions" could be designated in the agenda of the FC, the interactions with the Secretariat. In camera where members can raise issues and concerns in the sessions have been introduced as a standing agenda absence of Secretariat staff. Before new staff positions item by both FC and STRATEC. Several staff are established, a thorough Human Resources replacements have been/are being made in connection assessment should be carried out against a clearly with the new Strategy, ensuring that newly recruited defined strategic work plan, using international work staff have the skills required for its implementation. For productivity norms in benchmarking these needs. An senior/professional appointments, an external external assessor system could be introduced in order assessor is used as part of the interview team. to ensure that the best available candidates are selected. Greater budgetary stringency is recommended in view of the decreasing level of funding. The Global Forum should focus its attention on current Within the new GFHR Strategy: and controversial issues in health research, including Human rights is seen as a cross-cutting issue rather (a) a human rights approach to health research; (b) /b than having a separate portfolio. ways in which the different actors benefit (or not) from Those who benefit and those who do not benefit from the results of health research; (c) barriers to access to health research is covered by the cross-cutting new preventive methods and therapeutics developed attention to equity; by research; (d) the inequities of the current system of intellectual property for drugs and diagnostics and Barriers were highlighted in Forum 11 in Beijing, which possible changes based on the notion of health as a had the overall theme of Equitable access: research public good; (e) obstacles that have prevented the challenges for health in developing counties; development of universally accessible and sustainable Intellectual property has been given major international health systems that can apply the results of research attention through the work of the WHO Inter- for health in an equitable way; (f) transformation of Governmental Working Group on Public Health, health systems in incubators for innovation in health Innovation and intellectual Property (IGWG), to which technologies, products and processes that improve the Global Forum has contributed. efficiency and equitable access to discoveries; and Obstacles to health systems development, (g) different approaches to health financing, transformation of health systems, and health financing particularly the political and economic factors behind are represented by two of the three main Strategic specific health systems financing policies. Priorities in the 2008­2014 GFHR Strategy. Source: Summarized by the author from the GFHR presentation in Annex F. /a As early as 2004 it was evident that the 10/90 gap was not measurable, though some FC members continued to hold to the concept. /b Human rights does not appear to be a significant issue in the revised strategy. 16 2.16 Under its new strategy (Annex Table 8), the Global Forum is pursuing four objectives: Improved priorities in research and innovation Increased coherence and partnerships among global players Strengthened research and innovation on health and health equity Expanded use of evidence in policy and decision making. The new strategy has three strategic priorities: (a) Linking resources with priorities for health research; (b) increasing the role of research in supporting development of effective and equitable health systems; and (c) strengthening research and innovation for health in LMICs. The key functions that the Global Forum plans to carry out as it moves to implementation of the new strategy include serving as a watchdog, a platform for dialogue, and an advocate for change. 2.17 The Global Forum considers (Annex J) that its revised strategy represents a major departure from its work in the first decade in at least three respects: (a) the shift from "health R&D" to "research for health," with a broader area of concern; (b) a shift from nine portfolios to three strategic priorities; and (c) a very strong emphasis on direct engagement with the top R&D decision makers in the world. IEG finds that the new strategy represents an important evolution but not a sharp break with the past. 2.18 While the term is still used with quotation marks, the 10/90 gap figures less prominently than in the past in the work of the Global Forum. Instead there is greater emphasis on improved health research priorities, health equity, and coherence and partnerships among global players. The new strategy sees the GFHR mission as "to play a leadership role in catalyzing global research on health problems of the poor." It posits the core values of "health as a right, equity as a principle and research as an indispensable tool." It is less explicitly focused than in the past on developing countries as such, but they implicitly remain the principal concern of the GFHR. The strategy observes that, especially for long-term impact, good measures do not now exist and a variety of indicators will need to be developed, tested and applied. Draft indicators for the three strategic priorities were under discussion within the Global Forum at the time this report was written. 2.19 Communications are assuming an increasingly prominent role in the GFHR as it implements its new strategy. In 2008 the Global Forum commissioned a consultant study on re-positioning and branding the GFHR. The study was expected to map the research landscape, analyze the GFHR's current identity, identify target audiences, review partnerships, define the GFHR's personality, draft aspirational messages, validate the new identify through stakeholder perception analysis, and redefine the GFHR visual identity. In response to the study, 19 the Global Forum approved its first-ever logo and adopted a new strapline: "...because health equity is a priority." 19. While the full text of the consultant study was not available for this GPR, this states that the Global Forum will: (a) Assist: The Global Forum intends to provide sound, reliable and impartial evidence and practical tools, which focus on proven methodologies, solutions, case studies, lessons learnt and good practices. It proposes to help identify and stimulate attention to priority research agendas. Furthermore, it expects to operate as a watchdog on research for health and health equity, gathering intelligence on global issues and trends; 17 3. The Effectiveness of the Global Forum 3.1 This chapter discusses the relevance of the Global Forum, its efficacy, its efficiency, its governance, and its relation to future of research on developing country health. The chapter finds that the Global Forum has been somewhat effective but needs to focus its activities further and seek broader and deeper engagement with the largest funders of health research and the commercial private sector. Relevance of the Global Forum 3.2 The relevance of the Global Forum depends significantly on several key implicit assumptions. Two assumptions underlie the rationale for its existence: Lack of appropriate research, including both technical innovation and its diffusion and application, is a critical factor explaining the weak health outcomes of poor countries and peoples; and Advocacy for health research on the problems of poor countries and peoples is not adequately carried out by other organizations. Two further assumptions underlie the mechanisms chosen by the Forum for its work: Measuring expenditures on research is a valid proxy for assessing the quantity of high quality research carried out;20 and Research can be encouraged and elicited by drawing attention to its absence, highlighting gaps, and engaging in dialogue with people responsible for research policy and the allocation of financial resources to research. 3.3 It has not been possible in this review to examine each of these assumptions in detail, but they are certainly plausible. The discussion in this chapter sheds light on them. The importance of research for health improvement -- the first assumption above -- is dramatically illustrated by the finding that technical progress21 explains 66 percent of inter- country variation in the decline in infant mortality from 1962­1987, whereas change in income explains 9 percent (Jamison, Sandbu and Wang 2004). The problem of measuring expenditures as a proxy for measuring outcomes -- a particularly important issue in the early years -- is hardly unique to the Global Forum, but it underscores the critical importance of (b) Link: The Global Forum intends to engage a critical mass of actors in research for health, including current and future decision makers in a cross-boundary dialogue leading to comprehensive rather than piecemeal solutions. It proposes to catalyze and convene coalitions within and across sectors, disciplines and geographical borders; and (c) Influence: The Global Forum intends to use the voice of its constituencies to influence high- level individuals and groups that shape, fund and implement global research agendas. It expects to do so through its annual forums, face-to-face meetings and a variety of media -- in an evidence-based way to bring about changes in health priorities, resources and policies for the benefit of poor populations. 20. Other, complementary approaches have been explored by the GFHR, including bibliometric analyses of the literature, surveys of research capacity and expert consultations to define research agendas. 21. Presumably, technical progress is understood to include diffusion as well as technical innovation. Health research must be understood to include technical innovation as well as adaptation, dissemination and implementation research. 18 high quality evaluation of GFHR studies and initiatives.22 3.4 The vision of the Global Forum of a world in which the potential of research and innovation is fully utilized to address the health problems of the poor, and its mission of playing a leadership role in catalyzing global research applied to the health problems of the poor (Box 2) are relevant by being responsive to current global challenges and policies, to the growing availability of development assistance for health, to the increasing world-wide expenditures on health research, and to the particular needs of developing countries.23 3.5 The world's current global challenges and policies focus heavily on health and the environment. Beyond the sectoral mandate of WHO, the UN and other world-wide bodies are giving ever greater attention to health. The Millennium Development Goals adopted by the United Nations at the turn of the millennium give a central place to health. The G8 has discussed health issues in its annual summit meetings since 1996. Meeting in Toyako, Japan in July 2008, the G8 Summit welcomed the report of the G8 Health Experts Group and adopted the Toyako Framework for Action on Global Health. There is growing awareness, worldwide, that infectious diseases are not a problem of the past, and that non-communicable diseases affect developing countries as much as industrial ones. The surge of interest in health, world-wide, includes but is not limited to major global institutions (Box 5). 3.6 Development assistance for health has risen markedly in real terms in recent years. According to OECD data, HNP ODA more than doubled in real terms from 1993 to 2003, from $3.1 billion to $6.7 billion. The HNP share in total ODA grew from 5.5 to 6.7 percent over the same period (MacKellar 2005). Studies under way by others suggest that the increase and the total amounts of development assistance for health in developing countries may be even greater. The growing importance of the Bill and Melinda Gates Foundation as a financier of health programs in the interest of developing countries is further testimony. Yet it is self-evident that the additional resources increasingly available for health programs and projects in developing countries cannot be well used in the absence of a sound foundation in appropriate health research. 3.7 World-wide expenditures on health research have risen even more rapidly than ODA for health. As shown in Figure 1 above, they more than quadrupled over the past twenty years. However, detailed disaggregated information is not available to show the extent to which these financial resources are devoted to research on the health problems and conditions of poor countries and peoples, and whether this funding has grown or fallen in proportion to the total.24 This increases the relevance of the Global Forum's objectives. 22. As noted above, the second external evaluation was deficient in several key respects; it did not assess individual GFHR studies. 23. The finding in the recent IEG HNP evaluation (IEG ­ World Bank 2009) that accountability of Bank-financed HNP projects for results to the poor has been weak and its recommendation to renew the Bank's commitment to HNP outcomes among the poor in both project and analytic work increase the relevance of the Global Forum. 24. Annex Table 5 suggests that the growth in funding relevant to the problems of poor countries and peoples has been much less than that on the problems of others. The share of health research funding in low and middle income countries has remained fairly stable. The absolute amounts funded from industrial countries have grown enormously. As discussed in the Annex, some caution is, however, merited in interpreting the data in Annex Table 5, since some of the health research spending in industrial countries, such as on NCDs, is relevant to the problems of at least some poor populations in the developing world. 19 Box 5. A Surge of International Interest in Global Health The International Health Partnership (IHP) was launched in September 2007, with all signatories signing a Global Compact to achieve the health MDGs. The Partnership, initially between a number of international agencies involved in global health,/a has included developing countries and has been expanded to an IHP+. The European Foundations Centre has produced a European Glossary on Global Health, which is helping to shape the global health policy of the EC. The UK's presidency of the EU in the second half of 2005 included "health equity" as a theme; Finland's presidency of the EU in the second half of 2006 was marked by a focus on "health in all policies". A European Council on Global Health (ECGH) is becoming operational in 2009. It will serve as a think tank and engage with Brussels institutions and beyond, to advocate for greater policy coherence in Europe on global health issues. The Netherlands government, in collaboration with OECD, has organized a consultation which led to the Noordwijk Medicines Agenda. As a follow-up, it is collaborating with COHRED to establish an African Medicines Agenda. Norway's Foreign Minister has led an international initiative on "health and diplomacy." Switzerland has developed a cross-government approach to global health -- generating a coherent approach to issues that will affect its dealings with international institutions, including WHO, WTO, and WIPO. Spain is generating a set of coherent, cross-government policy papers on global health. COHRED has initiated studies of the implementation of the Paris Declaration with regard to donor alignment and harmonization in relation to health research support for low income countries. Sweden's SIDA-SAREC has organized consultations and a follow-up process on the application of the Paris Declaration to health research in developing countries. The UK Department of Health has published a global health strategy. DANIDA is developing a first position paper on global health. The United States has renewed its commitment to PEPFAR and allocated a further US$ 50 billion to global health programs, with a focus on Africa. Discussions are under way on the possibility of creating a Global Health Consortium, bringing together newly established Global Health centers, institutes and programs that work to develop an interdisciplinary approach to global health and show a particular concern for the interface of global public health and foreign policy/global affairs./b With active donor leadership, the World Bank is moving towards adoption of an advance market commitment program aimed to accelerate development and use of new vaccines in developing countries. /a African Development Bank, Bill and Melinda Gates Foundation, European Commission, Global Fund to fight AIDS, Tuberculosis and Malaria, GAVI Alliance OECD/DAC, UNAIDS, UNICEF, UNFPA, WHO, World Bank. /b Conveners of the initial meeting, which was planned for January 2009, were expected to be the Graduate Institute of International and Development Studies, Geneva; the Institute for Global Health, Beijing, China; the Centre for Global Health at FIOCRUZ, Rio de Janeiro, Brazil; and the Global Forum for Health Research. Source: The changing external environment/landscape/architecture -- implications for the Global Forum, document prepared for November 2008 STRATEC meeting. 20 3.8 There is evidence of growing beneficiary responsiveness to the issues raised by the Global Forum. Developing country interest in health and health research has sharply risen over the past decade. Targets initially largely proposed outside governments -- of 5 percent of country-level health ODA applied to research, of 2 percent of developing country health budgets devoted to health research, and of 15 percent of developing country budgets devoted to health -- are gaining increasing standing in government-sponsored resolutions and declarations, in WHO, the AU and beyond (Annex E). This evolution strengthens the relevance of GFHR country studies on health research spending. Larger developing countries, including Brazil, India, China and South Africa, are gaining prominence as health innovators and as incubators of health research networks (Morel 2005). 3.9 The Global Forum is an advocacy and knowledge network engaged in providing global public goods. The goods and services produced by the Forum are non-rival, non- excludable, and of global reach. No other institution of comparable legitimacy is producing such goods as its core mission, but -- as discussed below -- some other institutions engaged in financing and commissioning health research are engaged in research advocacy as a complement to their main activity. 3.10 The activities of the Global Forum facilitate communication among practitioners, generate and disseminate information and knowledge, and engage in direct advocacy (Table 4). Among other types of activities frequently performed by advocacy and knowledge networks (Annex Table 4), "improving donor coordination" is a realm of potential GFHR interest where it has participated in meetings organized by others, but it has not undertaken substantial initiatives of its own. "Implementing conventions, rules or formal and informal standards and norms" is also an area which the GFHR could enter if its advocacy activity, most recently reflected in a "scorecard," were to gain sufficient standing to establish meaningful, if still informal global norms. 3.11 In recent years and especially under its most recent strategy, the Global Forum has been reaching out increasingly to high level policy makers -- a difficult activity for a relatively small organization with limited financial resources. As noted above, the contacts with policy makers are based on the assumption that such dialogue can have impact on the actions of policy makers. Naturally, a more forceful and effective way to have an impact on research decision makers would be to allocate specific human or financial resources to research tasks. The Global Forum does not have access to such resources, but its initiatives have had the strength of producing consensual research agendas in areas where they did not exist previously. 3.12 The relevance of the design of GFHR activities is less clear than the relevance of its objectives.25 In discussing the design of the GFHR we see risks that the many health research partnerships may lead to competitive behavior rather than complementary activities. In the early years of the Forum, its activity was unique. More recently, GFHR partners such as WHO and TDR, and external entities such as the Bill and Melinda Gates Foundation have 25. The crowded landscape of partnerships engaged in research for health is an important factor in this. In Geneva alone, there are at least 8: The Global Forum, TDR, HRP, Initiative on Vaccine Research, COHRED, Alliance for Health Policy and Systems Research, the IGWG, and the WHO Research Strategy. 21 Table 4. Advocacy and Knowledge Networking Activities of the Global Forum Generic Activities of Advocacy Global Forum Activities and Knowledge Networks Facilitating communication Brokering contacts, organizing meetings of Global Forum among practitioners in the sector constituents, such as the Annual Forum Meetings, and catalyzing and convening coalitions and networks such as its Initiatives Generating and disseminating Providing evidence and serving as a watchdog on research for information and knowledge health and health equity: preparing reports monitoring financial flows for health research, carrying out studies on methodology for allocation of resources in health research, writing other research reports; executing communication activities including an interactive Web site Advocacy Organizing informal high-level meetings on the occasion of Annual Forum Meetings; participating in consultations with Heads of International Research Organizations (HIROs); attending and presenting at large numbers of conferences and meetings begun increasingly to engage in advocacy for health research on the problems of the poor. However, these advocacy activities are more derivative of other parts of their work than central to the organization. The fact that in Geneva alone there are at least 8 health research partnerships inevitably leads to elements of competition. Globally, however, while there are some alternative sources of supply for GFHR activities, they have not achieved the legitimacy in limited areas of health research advocacy for the poor that has been achieved by the GFHR.26 The GFHR compilations of data on funding of health research are a case in point. 3.13 There have been differences of opinion concerning whether the GFHR was the appropriate body to undertake certain activities in its work program. This was especially pronounced in the case of the GFHR initiatives. These are being phased out as Global Forum activities. The Bank and the Global Forum had determined that the initiatives should be gradually weaned from Bank/GFHR funding and encouraged to develop as independent entities. Yet, they continue to represent practical efforts to generate research agendas in discrete areas and to mobilize support for them. 3.14 The GFHR strategy (Annex Table 8) also raises issues of design. Its sweeping breadth and the great need for promotion of health research on the problems of poor countries and poor peoples would appear to call for greater focus, and a higher degree of selectivity, in GFHR activities. The "report card" set out in the most recent GFHR report on monitoring financial flows suffers from the same problem of apparently excessive breadth. The finding of the GFHR's second external evaluation that the world is at a turning point in health research policies that affect the poor27 also suggests a possible need for further reflection by the Global Forum, particularly as it considers the results of a "re-positioning" consultant study carried out in 2008. 26. GFHR legitimacy would certainly be increased with a broader donor base; the GFHR base seems to be somewhat broader than that of the GFAR. 27. Report, page 16. 22 3.15 WHO, the principal alternative source of supply of the type of services provided by the Global Forum, is nearing completion of a first-ever research strategy for the organization (Box 6).28 The draft strategy was endorsed by the WHO Executive Board early in 2009, but deferred for approval by the World Health Assembly (WHA) of Ministers of Health in May 2009 as a result of the priority given by the WHA to the H1N1 or "swine flu" virus. The Global Forum, the World Bank and others were consulted during preparation of the strategy, and the Forum contributed to the emphasis on "research for health" and on focusing initial attention to research within WHO. As WHO increases its engagement in health research, the GFHR-WHO partnership is likely to face strains and require reconsideration and renewal. The plans of WHO to prepare reports on global health research priorities and the allocation of resources to them could fundamentally challenge the relevance of much GFHR activity.29 3.16 WHO has estimated the ten-year life-cycle cost of its new strategy at $39 million, of which $3 million would be required in the first two years, including incremental funds amounting to $1.5 million. For strategy implementation WHO plans a focused resource mobilization effort. For the preparatory phases of strategy development, the Bill and Melinda Gates Foundation and the Wellcome Trust provided financial resources.30 If WHO allocates significant financial, human and political resources to the implementation of its new research strategy, and particularly to its advocacy dimensions, the Global Forum will face significant challenges. However, it should be noted that WHO is perceived by some to have a strong public sector bias. Because of the number and variety of actors in global health research in both the public and the private sectors, the independence of the Global Forum will remain an important aspect of its relevance, particularly if WHO becomes a major actor in health research. Despite the active engagement of the Global Forum during preparation of the WHO research strategy, the nearly complete absence from GFHR governance and financing of major funders of research on the problems of poor countries and peoples, especially the Bill and Melinda Gates Foundation and the U.S. National Institutes of Health, as well as the GFHR's limited engagement with the commercial private sector,31 limit the relevance of the Forum.32 28. The Global Forum considers that viewing the WHO as an alternative source of supply for services which the Global Forum provides misunderstands the different roles played by the Forum and WHO. In commenting on the draft of this report the Global Forum stated that the Forum serves as an independent watchdog and as a platform, advocate, and catalyst across the whole spectrum of sectors and actors involved in research for health. It considers that this is not practical for an intergovernmental agency whose membership comprises ministries of health. 29. The Global Forum comments that this challenge is more apparent than real. Responding to the draft of this report, the GFHR wrote that during the evolution of the WHO research strategy, WHO recognized that it could and should not do all the tasks in the strategy alone and would, for example, depend on its partnership with the Global Forum as the source of resource tracking information. 30. Source: WHO document EB124/12 Add.1, December 18 2008. 31. Of the 20 members of the FC, aside from its chair only one can be considered a private sector representative. Yet, approximately half of global health research spending (Figure 1) is by the private sector. 32. Commenting on the draft of this report, the Global Forum observed that these major players are frequent attendees at Forum meetings and participants in other Forum activities. 23 Box 6. The Emerging WHO Research Strategy Now more than ever, global health needs the global research community -- this is the point of departure of the emerging WHO research strategy. Adopting the concept used by the Global Forum, the WHO strategy starts from the concept of "research for health," rather than "health research," to connote the idea that its interest is any research, regardless of the sector, that relates to health services, objectives and outcomes. The WHO strategy seeks to strengthen the research culture across the entire organization, and to stimulate changes in behavior in this sense, at WHO Headquarters, in the regional offices, and at its country offices. The strategy proposes to champion research that addresses priority health needs in relation to health equity and the MDGs, notes the historical inequity in the distribution of global research funding (often symbolized, it states, by the "10/90 gap"), and the lack of compelling evidence to make the case for research in competition with other priorities. The strategy proposes that every four years WHO prepare a report on global health research priorities and the allocation of resources to them -- both areas of immediate and direct concern to the Global Forum. The WHO strategy states that WHO needs to foster global and regional networks among researchers and research institutions -- again a concern of the Global Forum. The strategy concludes that, for strategy implementation, the WHO Secretariat will need to collaborate effectively with partners with independent governance and to work more effectively with key research partners, including industry, civil society, foundations and academia. An evaluation framework set forth in the strategy posits 8 indicators of input, output, outcome, and impact. The wide overlap between the WHO concerns and those of the Global Forum puts a premium on effective collaboration between the two organizations. Source: WHO 2008. Efficacy 3.17 The principal outputs of the GFHR are (a) meetings, (b) publications and (c) dissemination of evidence. The growing numbers of participants in the AFMs, the positive evaluations by participants, the large shares of developing country participants (Annex Tables 9 through 12), and the lists of publications and publications orders, along with the GFHR data on Web site use (Annex D) testify to Global Forum success in producing these expected outputs, many from initiatives specific to the Forum.33 A thorough evaluation of the Global Forum might be expected to assess the quality of individual outputs, and to compare use of the GFHR Web site with the use of the Web sites of similar organizations.34 While the Global Forum's AFMs have moved in the direction of providing opportunities for the Forum to convene high level sessions with decision makers, it is too early to assess outputs in this area. 3.18 Measuring the results of GFHR efforts to help correct the 10/90 gap is extremely difficult, for four reasons: The first overwhelming obstacle is attribution. The nature of the GFHR activities makes it all but impossible to attribute specific results to the Forum's work. 33. Ideally, these outputs should be compared with outputs of comparable organizations, such as the Global Forum for Agricultural Research. 34. This type of analysis was not feasible for this GPR with the available resources. 24 The second reason lies in the concept of the 10/90 gap itself. By the time of its first evaluation in 2001, the GFHR had recognized this problem by establishing knowledge of the 10/90 gap, number of initiatives, and improvement in resource flows as indicators, rather than change in the gap itself.35 By the time of the second external evaluation, the GFHR had revisited the concept. The evaluation recommended that the Global Forum redefine the GFHR goals so that they are measurable and attainable.36 The third obstacle lies in the external environment: The landscape of global health, including especially global health research, is changing so rapidly that a results framework with highly specified indicators would risk becoming irrelevant within a short period of time. The fourth reason lies in the evolving GFHR objectives (Table 5) and absence of an agreed results framework. The marked shifts in GFHR objectives preclude determining the efficacy of its work in the sense of the extent to which it has achieved or could be expected to achieve its objectives. While the Global Forum has consistently formulated indicators in relation to specific subjects, it has not established an overall results framework with associated indicators permitting objective evaluation of its overall accomplishments against a limited number of specific overall objectives. The latest strategy is remarkable for stating, after many rounds of discussion, that good indicators to measure outcomes do not now exist and that a variety of indicators will need to be developed, tested and applied. As this report was being completed, indicators were reported to be in active preparation. 3.19 The Global Forum has performed competently on outputs such as AFMs and publications. But the absence of a clear overall results framework has contributed to an environment allowing low levels of institutional accountability for outcomes, as distinct from accountability for specific outputs, despite all intentions to the contrary and the establishment of specific indicators for specific activities. To some extent this problem is inherent in any advocacy organization. Yet, it underscores the importance of the effort to create a results framework, at least to provide the basis for subsequent evaluation and accountability. The weakness of the GFHR results framework recalls the finding of IEG's 2009 evaluation of the Bank's HNP activities at the country level that monitoring remains weak and evaluation almost non-existent. The HNP evaluation found that strengthening M&E is one of the key elements in achieving the Bank's HNP strategy's objective of better governance in the sector (Independent Evaluation Group ­ World Bank 2009). 35. It is clear that awareness of the 10/90 gap concept has become widespread. Indeed, the term remains in use, and is occasionally criticized (see, for example, Stevens 2008), even though its originator -- the Global Forum -- is using it less. 36. The ET also observed that due to the lack of measurable indicators it is extremely difficult to quantify the impact of the GFHR in helping to close the global gap in health research and in focusing research efforts on health problems of the poor. 10/90 gap critics (such as Stevens 2008) observe that proponents of the 10/90 gap are "inaccurate" when they claim that low-income countries suffer from completely different diseases than high income countries. Yet the GFHR itself was an early observer of the convergence of LDC and industrial country epidemiology on NCDs. 25 Table 5. Evolution of GFHR Strategy Statements and Objectives: From the 10/90 Gap to Health Equity and the Poor Year and Source Statement of Global Forum Indicators used by GFHR: Approach to Objectives M&E 2001 Initial Central objective: To help correct (a) Knowledge of the 10/90 gap among Evaluation of the the 10/90 gap by focusing researchers, donors and policy makers; Global Forum research efforts on the health (b) Number of initiatives promoted in key problems of the world's poor, research areas; and improving the allocation of (c) Improvement in resource flows to fill the research funds and facilitating gap. collaboration among research partners from public and private sectors. Global Forum Central objective: Help correct the Indicators are set out to measure the Strategic 10/90 gap in health research and contribution of Annual Forum meetings to Orientations 2003­ focus research efforts on the correction of the 10/90 gap, along with a 05 health problems of the poor variety of other details on the research and communication programs. Under M&E the strategy expects to measure the results of the work of the Global Forum through monitoring of progress indicators specific to particular programs and activities and through periodic external evaluations. No global indicators are set out. Global Forum The Global Forum for Health The Global Forum states that it needs to Strategy 2004­ 08 Research will work to be: monitor and evaluate both the results of its (a) A source of information, own efforts to close the 10/90 gap and the arguments and expertise; progress that the world in general is (b) An acknowledged generator of making towards this goal. It indicates that ideas and networks; there will inevitably be some difficulties with attribution. The Forum finds it especially (c) A respected and trusted important to develop clear indicators that partner in major initiatives; reflect its own activities and systematically (d) A recognized leader in build them into the design of its work in all changing attitudes and practice. areas. No global indicators are given. 2008­14 Strategy (a) Improved priorities in research The strategy states that the Global Forum and innovation; will apply or, where needed, develop a (b) Increased coherence and variety of indicators to measure and partnerships among global monitor the short, medium and long-term players; impact of its work. Long-term impact (c) Strengthened research and indicators are expected to include: (a) innovation on health and health evidence that priorities have changed, equity; and resources targeted to agendas and priorities have been identified; and (d) Expanded use of evidence in research and innovation are contributing to policy and decision making greater health equity; (b) monitoring the behavior of actors at national and global levels; (c) policy implementation, monitoring and evaluation for effects on disparity reduction, equity, and health of the poor; and (d) policy makers commissioning research to identify policy examples using evidence. 26 3.20 External observers have markedly varied views on the efficacy of the Global Forum. The evaluation team for the second evaluation expressed concerns about what it described as "relatively low global impact of the Global Forum," but found it "very likely that the GFHR has contributed much to making the research community -- researchers and research policy makers -- aware of the research imbalances and to helping them" correct the imbalances. About half of the respondents to a survey of 400 key researchers undertaken by the evaluation team indicated that they were unaware of the work of the Global Forum. For an organization concerned to raise awareness, this finding in itself raises questions about its effectiveness. The ET also found much room to improve the standing and influence of the Global Forum in LMICs. 3.21 While the scope and number of interviews for this GPR were significantly less than for the second external evaluation, interviews for this study and other research (Bloom 2006) suggest that the Global Forum has been fairly successful, at least in creating awareness of the need for increases in research related to the health problems of the poor. The recently completed second edition of the Bank-supported Disease Control Priorities Project report (Jamison et al. 2006) states that the Global forum "took the most effective advocacy position" on the importance of research on developing country health problems, and finds that the arguments of the Global Forum and its predecessors have "galvanized global recognition that more research funding should be devoted to improving the health of the 85 percent of the world's population who live in developing countries." 3.22 It is fairly clear that the Global Forum has had an impact on awareness of the importance of health research on the problems of poor populations (Box 7), but it is not clear that the GFHR has substantially influenced the level and allocation of total global health research expenditure. Its core advocacy expenditures of $3.5 million a year could hardly be expected to have a substantial impact on the level and allocation of the current world total of $160 billion in annual spending on health research. Health research funding from developing countries has, however, grown substantially, from an estimated $3.6 billion in 1998 to$5.1 billion in 2005. The Forum does not appear to have had a significant impact on research priority setting within given allocations. This is especially the case at the global level which is the core of its mission. The research agendas prepared by the GFHR initiatives represent important contributions, but they do not appear to have been fully funded (Annex Table 6). The freely available GFHR Combined Approach Matrix (CAM) tool for setting health research priorities has been a useful intellectual contribution, and been widely disseminated. But, it has not had the weight of direct links to funding for the priorities identified.37 Its principal impact appears to have been at the country level rather than at the global level. 3.23 The availability of funding is the most critical variable in determining research priorities, whether on developing country health issues or other matters. As late as 2005 (Annex Table 5) only 3 percent of estimated global health R&D spending took place in low and middle income countries. Even though some of the 97 percent spent in the industrial world was probably relevant to health conditions of poor countries and peoples,38 the 37. The Global Forum reports (Annex J) that the CAM has been widely disseminated in English and Portuguese and used by a range of global, national and local organizations. 38. Research on NCDs in the industrial countries, for example, may be relevant to NCDs in low- and middle- income countries. 27 Box 7. Evidence for Efficacy of the Global Forum: Global Attention to the 10/90 Gap Despite the ambiguities in its meaning, the 10/90 gap has been widely cited by public and political leaders, and health advocates. While occasionally the language used has been unclear, global attention to the case for more investment in health research on the problems of developing countries is clear: The UK government called for a new commitment to tackling the diseases of poverty in a 2001 paper issued by Gordon Brown as Chancellor of the Exchequer, mentioning that only 10 percent of all international research on health goes towards diseases which make up 90 percent of the world's disease burden. /a Newsweek magazine wrote in a 2002 cover story on Bill and Melinda Gates that poor countries were carrying 90 percent of the world's disease burden but only receiving 10 percent of its health resources. /b The G8 2002 Kananaskis Summit adopted an Africa Action Plan committing members to "supporting health research on the diseases prevalent in Africa with a view to narrowing the health research gap." /c Former US President Bill Clinton and philanthropist Bill Gates talked of the gap at the Time Magazine Global Health Summit, November 1­3, 2005. /c In a 2009 report, the Board on Global Health of the U.S. Institute of Medicine stated that the US commitment to health research cannot be overemphasized, and reported that one-half of the world's health research can be attributed to investments by the American taxpayer. /e /a HM Treasury, February 26, 2001. /b Newsweek, February 4, 2002. /c As cited in Ronald Labonte et al. Fatal Indifference: The G8, Africa and Global Health, IDRC, Ottawa 2004. /d Response to questions at the 12th Canadian Conference on International Health, November 6­9, 2005. /e Institute of Medicine 2009. The US Commitment to Global Health: Recommendations for the New Administration, Washington DC, The National Academies Press. discrepancy remained enormous. The initiatives of the Bill and Melinda Gates Foundation have been central in guiding research priority setting on the problems of poor countries and peoples. Its dedication of $200 million alone to the Grand Challenges in Global Health is indicative of the scope of its research funding and priority-setting effort. 3.24 Despite their success in elaborating research agendas, the seven initiatives of the Global Forum (Box 3 and Annex Table 6) have a somewhat mixed record. The Initiative for Public-Private Partnerships in Health produced a data base but was wound down in 2005 for lack of donor support. The Alliance for Health Policy and Systems Research is well regarded and has been absorbed into WHO. The Initiative for Cardiovascular Health in Developing Countries has sponsored economic studies but has had difficulty attracting financial resources beyond those of the Bank. The Sexual Violence Research Initiative has done good work but has an uncertain future despite strong leadership. The Child Health and Nutrition Research Initiative has had little success in gaining support beyond the Bank. 28 The Global Network for Research on Mental and Neurological Health has had low expenditure levels and -- at least in 2006 -- was still dependent on the Global Forum and the Bank. The Road Traffic and Injuries Research Network has become increasingly strong and independent and has enjoyed Bank transport sector support outside of the DGF. 3.25 A 2007 evaluation of the initiatives completed by Global Forum staff at the request of the Foundation Council39 found that there might have been a more focused and cost-effective approach to raising the profile of each of the areas supported, but at the risk of losing the distinctive "voice" of dedicated groups of supporters for each individual neglected area. While nearly all the initiatives and networks have already moved towards legal independence, the staff evaluation concluded that in many cases the financial viability of the initiative was questionable. The evaluation concluded that, for the future, "if an initiative is needed to boost attention to a neglected area of research, there should be well defined entry and exit criteria, goals and milestones set at the outset; clear sources of funding should be identified and plans and timetables agreed in advance for either sunsetting the activity or ensuring a sustainable future for it." Despite the Bank's leading role in this aspect of the Global Forum's work, the initiatives are gradually being phased out as an element of GFHR work. Overall, the performance of GFHR initiatives, where the Global Forum has had direct accountability, at least at the outset, has been less successful than that of the sub-grantees, where oversight for DGF resources was the responsibility of the Bank (Annex Tables 6 and 7) and the issue of the efficacy of the Global Forum does not arise. Efficiency 3.26 Quantitative data shedding light on the efficiency of Global Forum resource use, particularly in relation to comparable organizations, are not available. Observers generally consider that the Forum has been reasonably efficient in its use of funds. As shown above (Table 1), Global Forum overhead and governance expenditures represent about 20 percent of total spending. Aside from a large overhead in the start-up period, this share has been stable. The competitive selection process under which the Medical Research Council in South Africa was chosen as the host for the GFHR Sexual Violence Research Initiative is an example of GFHR efficiency in resource management.40 However, there have been a few expressions of concern about appearances of excessive staff travel and generous meeting and other environments for Global Forum events in developing countries as being inconsistent with the values of the Forum. The evaluation team observed an increase in spending and activity from 2002 to 2005. While it provided no further detail, the ET stated that it was unable to correlate this with increases in advocacy outcomes. 3.27 The Global Forum's financial and budget management are sound, and its operational and financial reports are thorough and well-presented. Expenditures have reasonably closely followed budgets. The Forum's financial statements are routinely audited by external 39. Foundation Council Document No. FC-07-1-8. 40. In its comments on the draft of this report (Annex J), the GFHR states that considerable effort is being invested in improving the efficiency of GFHR operations. This includes streamlining the FC from up to 25 to 12­14 members, replacement of some staff, and consolidating three units into one. 29 auditors, and the World Bank's Internal Auditing Department has carried out one audit of the Global Forum. A 2008 Bank financial management assessment was positive, and confirmed that the Forum is devoting the appropriate resources to financial management. 3.28 The evaluation team for the second evaluation expressed some concern that while the research and programs unit seemed to run efficiently, the professional skills requirements were not precisely delineated. More importantly, the loss of key professional staff with standing in developing countries has hurt the Global Forum. The ET saw outsourcing as an alternative to increasing staff. This review finds an absolute minimum number of highly qualified and credible technical personnel in the GFHR Secretariat essential, and understands that some qualified candidates have rejected GFHR offers. 3.29 The evaluation team was concerned about what it saw as continued annual deficits in GFHR activities and recommended greater budgetary stringency, especially in view of what it perceived as a decreasing level of funding. IEG finds that donor funding for GFHR core activities has in fact grown slowly and been relatively stable. However, total donor financial contributions to the Global Forum, including initiatives and networks, decreased from a maximum of $8.0 million in 2004 to $7.1 million in 2005, $6.9 million in 2006 and $5.6 million in 2007 (Figure 4). The importance of strengthening GFHR fund-raising has been repeatedly stressed. Figure 4. Global Forum for Health Research: Donor Financial Contributions US$ Thousands 9,000 8,000 7,000 Total Donor Contributions, 6,000 including initiatives and networks 5,000 4,000 3,000 Core Funding 2,000 1,000 0 1999 2000 2001 2002 2003 2004 2005 2006 2007 Source: Annex Table 22. 3.30 The Global Forum points out (Annex J) that its budget position in respect of core activities has been sound. It maintains reserves equal to roughly one year of spending on core activities. This suggests a strong underlying financial position for its core. Nonetheless, despite the advent of financial support from Mexico and Brazil, the decrease in overall donor support of the Global Forum, the limited number of ten donors, the expected termination of 30 World Bank DGF funding, and the nearly complete absence of private sector financial support augur poorly for the future in the absence of major new initiatives. 3.31 The absence of a published business plan and a fund-raising strategy in the GFHR work program and budget documents is noteworthy.41 Options and risks seem generally not to be part of the GFHR approach in the documents for the FC. The mental self-image of the organization appears to be somewhat more that of a public sector bureaucracy than of an entrepreneurial non-profit "business" seeking and exploiting opportunities and managing risks.42 3.32 It has not been possible in this review to analyze the cost-effectiveness of frequently changing and evolving GFHR work programs and activities.43 Nor -- beyond what external evaluators find largely at the level of perceptions -- can one determine whether the benefits outweigh the costs. Interviews carried out for this study suggest that outside opinion on this matter is fairly though not universally positive and generally more favorable to the Global Forum than was the second ET. It should also be noted that the ET encouraged the Secretariat to "start gathering data to enable it to assess the efficiency and effectiveness" of the Global Forum -- a task that should have been carried out by the ET itself. One astute observer commented for this review that the Global Forum has done reasonably well with limited resources. 3.33 All in all, from the perspective of developing country health officials and especially health research policy makers, the costs of the Global Forum would appear to be worthwhile, though as usual these people would almost certainly prefer to see greater concentration of GFHR activities at the country level and in research capacity strengthening. From the perspective of developed country donors concerned to increase health research in developing countries and health research spending on developing country problems, the Global Forum advocacy activities represent a "good buy," if not a "best buy." Unfortunately, unambiguous "best buys" do not appear to exist in the realm of GFHR advocacy and knowledge networking activities. The limited information available from interviews and the ET leads IEG to conclude that the GFHR's outputs appear to have been produced at reasonable cost and that its outputs represent reasonably good value for the money of its donors. In any case, in assessing value for money in the Global Forum, it is important to bear in mind that the 41. Commenting on the draft of this report, the GFHR states that there is a separate confidential fund-raising strategy discussed in camera by the FC and not intended for publication. Further information on the GFHR fund-raising strategy is contained in Annex J: (a) maintaining/increasing support from existing donors; (b) seeking additional sources of direct support from HICs and LMICs; (c) seeking in-kind support from LMICs for elements of GFHR work; (d) identifying ways in which foundations can provide support; and (e) developing the potential for private sector contributions. 42. The GFHR states that the reference to the absence of a business plan gives a false impression, and that the latest strategy and its amplification in three FC-approved strategic priority frameworks set out a 7-year perspective on the GFHR business, which is then translated into biennial workplans and budgets. Judging the IEG characterization in this paragraph as erroneous, the GFHR sees itself as a lean, lightweight, and opportunistic organization responding rapidly to the changing environment. It reports adjusting annual expenditures in line with actual income, even in the face of donor departures. 43. The evaluation team did not estimate the cost effectiveness of GFHR activities. 31 GFHR annual spending of around $3.5 million on core activities represents a miniscule share of much, much less than 1 percent of global health research spending of $160 billion.44 Governance 3.34 Global Forum governance exhibits weaknesses against standards expected of large public organizations, particularly in respect of Board accountability, external transparency, conflicts of interest, and independence of oversight. As shown below, these standards need to be applied with some flexibility in the case of much smaller NGOs, but the GFHR governance performance could be improved in a number of respects. 3.35 The Global Forum has a stakeholder-based model of governance. Its principal constituencies are recognized in its statutes, and the Forum is sensitive at both the Secretariat and Foundation Council levels to the importance of ensuring that developing countries have significant voice in the activities and decision making of the organization. Yet, despite their formal recognition in the founding documents of the Global Forum, its nine constituencies have no organized role in the governance of the organization. Indeed, under the GFHR statutes, FC members serve in an individual capacity, without a mandate to represent their constituencies of origin. Furthermore, since length of service tends to increase power in such bodies, donors have much greater power than the structure of the FC suggests: The two longest-serving FC members come from two GFHR donors: the Canadian International Development Research Center and from Norway (whose Ministry of Foreign Affairs has been an important donor). Thus it seems that the constituencies are much more important on paper than in GFHR governance practice, and the accountability of the FC to stakeholders other than its donor partners is an issue that might merit further consideration by the Forum. 3.36 FC members are "selected": from the constituencies but serve in an individual capacity. While the GFHR bylaws provide that each constituency will "normally be represented by at least one member," the FC members do not appear to consider themselves accountable to their constituencies. Thus the Global Forum appears to have no oversight and accountability mechanism, such as an active broad base of members, beyond the FC itself and periodic external evaluations. The GFHR constituencies seem more to represent targets of GFHR influence than stakeholders to be involved in governance. The constituencies have unexploited but demanding potential to increase GFHR legitimacy. However, as one interviewee observed, they face the risk of appearing to have been constructed as if not to leave anybody out. Finally, it is not clear how accountability to constituencies could be organized in the GFHR as presently constituted, since the constituencies are very loosely defined, as -- to mention several examples -- "women's groups," "policy makers," and "private sector." 3.37 In the initial years the GFHR Annual Forum Meetings (AFM) were held largely in Geneva, but this has changed, with AFMs in Bangkok, Mexico City, Mumbai, Cairo, Beijing, and Bamako. Furthermore, the GFHR monitors the number and the share of developing 44. It may be objected that juxtaposition of the GFHR core spending on health research advocacy of about $3.5 million annually with total annual expenditures on health research of $160 billion represents an invalid comparison; it does at least suggest that a much greater level of spending on advocacy than $3.5 million would be needed to have significant impact on the $160 billion spent on health research. 32 country participants and presenters at AFMs (Annex C). The substantial portion from low and middle income countries, and the endeavors of the GFHR to increase them, contribute to its legitimacy.45 It is striking that over half of the members of the Foundation Council come from developing countries (Annex Table 20). Also, the GFHR has strived to present itself as a neutral forum, where both public and private sector actors and researchers are welcome. Finally, the GFHR has started sponsoring country-level studies of resource flows for health research. Taken together, while there remain some perceptions that the Global Forum is a creature of WHO and especially the World Bank,46 actions taken by the Forum have enhanced its legitimacy as an independent organization. 3.38 Members of the Foundation Council and STRATEC are deeply involved in the work of the Global Forum. The extent of this involvement would compromise the members' exercise of their oversight responsibilities in a public international organization. In a much smaller NGO, however, the engagement of FC members in the work of the Secretariat is an important complement to the work of the staff, as long as the roles are distinguished. It does not appear, however, that FC and Secretariat personnel have adequately separated FC members' work in oversight and as complements to the staff. 3.39 As the FC and its Nominating Committee47 consider potential new members of the Council, they give considerable weight to participation of people from developing countries, to gender balance, and to the major GFHR donors. Among the Global Forum's 9 constituencies, health researchers appear to be somewhat over-represented on the Foundation Council (relative to policy makers, for example), with 10 of some 20 current members (Annex Table 20). Among donors, the World Bank is a very powerful voice because it has been the Global Forum's principal source of funds and because the Bank has consistently been represented in the Global Forum by senior and highly regarded HNP staff.48 Despite the Foundation Council's overall responsibility for the work of the Global Forum, aside from the Chair and the Bank's representative the FC members appear not to have assumed significant responsibility, either individually or collectively, for the Global Forum's fund-raising needs. This reduces a potential source of GFHR legitimacy. Also, aside from conflict of interest statements, the Global Forum does not have any even informal "contract" between FC members and the Chair or the Forum as a whole.49 Such a "contract" might help to manage the conflicts in FC members' roles between oversight and support to ongoing activities. 3.40 Because of the critical importance of fund-raising and because of the very large engagement of the private sector in health research (Figure 1 and Annex Table 5), the Global 45. The ET was nonetheless concerned that insufficient efforts were being made in this respect. 46. Some observers find that the Bank's presence increases the legitimacy of the GFHR. 47. The FC agreed in November 2008 that STRATEC would assume the role of a Nominating Committee. 48. See list of key Bank staff responsible for its work on the Global Forum in the front material for this GPR. 49. The by-laws of the GFHR set out the overall responsibilities and authorities of the FC and of the FC Chair in some detail, but only provide in respect of individual FC members that they are "expected" to participate in two FC meetings each year, that they are "encouraged" to take part in AFM sessions and to contribute in various ways as session chairs, discussants or presenters, and that they are expected to dedicate about 10 days a year to the GFHR. There is no expectation in the by-laws for the FC or for FC members aside from the Chair to play any role in fund raising. 33 Forum may need to consider new ways of bringing donor partners, whether from the public or the private sector, into its governance, as members of the Foundation Council or in some other way.50 This would have to be done while respecting also the importance for legitimacy of substantial engagement of developing country stakeholders -- a point repeatedly underscored by the Evaluation Team. The private sector, foundations, and policy makers appear to be under-represented in an institution that presents itself as a neutral forum for dialogue and which aims to serve a convening function.51 3.41 While this is decreasing, there seems to be some resistance in the Global Forum to increased engagement with the for-profit private sector.52 Private firms are not, for example, included in the marketplace of exhibitors at the Annual Forum Meetings.53 There are some indications that the relatively new FC Chair, who has a private sector background, will help to make it possible for the Forum to increase its openings to the private sector. The GFHR comments on the draft of this report (Annex J) indicate that private sector engagement is increasing and point out that the FC has recently expanded the number of FC members with private sector background from one to three. There was a plea at the 2008 Bamako Ministerial Forum on Health Research for the private sector to engage in detailed dialogue with a broad range of stakeholders about the roles that industry can play in research and innovation for health, and the Global Forum responded with a commitment to create a forum for such a dialogue.54 3.42 All major decisions, including on launching and allocating resources for studies above a fairly low level, are taken or endorsed by the FC. As part of it critique of what it saw as excessive autonomy for the GFHR Executive Director, the evaluation team for the second evaluation expressed the view that it would be desirable for the FC to play an even more central role in the work of the Global Forum, and to establish more guidelines for GFHR activity.55 This may be partly attributable to a passive posture by the FC Chair at the time of the second evaluation. She left very large latitude to the Executive Director and Secretariat -- a situation regretted by the ET. Several interviewees for this study underscored the same point, and emphasized a need for greater accountability within the Global Forum, for a reduction of what one interviewee described as "inbreeding" among FC members, and for 50. Some consideration has been given to creation of a high level Scientific Advisory Group, which could have substantial private sector representation, but this idea has not found sufficient favor among FC members. 51. The critical importance of the private sector is indicated by the fact that, as shown in Annex Table 5, the private commercial sector represents around 50 percent of global health research expenditures. 52. The ET, for example, supported participation of private sector research organizations in the AFMs but expressed the view that their involvement needs to be monitored to ensure that it does not lead to the "commercialization" of the AFM marketplace for exchange of ideas; responding to the ET recommendations (Annex F), the GFHR observes that there has never been any participation by private sector research organizations in the AFM marketplace nor any intention to "commercialize" it. 53. Some organizations with public policies strongly critical of private firms, such as the American Public Health Association, manage to engage with the private sector and earn substantial revenue from private sector exhibitors at conferences. 54. Source: Communiqué, Global Ministerial Forum on Research for Health, 17­109 November 2008, Bamako, Mali. 55. One suggestion made during interviews for this study was that the FC could establish guidelines for GFHR partnerships for use by the staff in discussions with others. 34 greater listening and responsiveness to FC members' views by the Executive Director. The current Chair is taking a much more forceful, active leadership position in the Council than her predecessor. Some concern has, however, occasionally been expressed about alleged Anglo-Saxon or industrial country domination of the Global Forum, as a result of its having an Executive Director and FC Chair who both hail from the UK.56 The risk of misperceptions led the Bank's member of the FC to decline election as STRATEC Chair and only to accept this post on an interim basis. The decision of the Executive Director to retire at the end of 2009 provides an opportunity to address these perceptions. 3.43 STRATEC and the FC hold lengthy discussions on issues, and executive sessions without the presence of staff when that might be helpful. Retreats have also aided in building consensus. Extensive minutes on the debates and decisions are prepared and made available on the private portion of the Global Forum Web site accessible only to FC members. The second External Evaluation and the Forum's Annual Reports are readily available on its Web site. For 2007 the Annual Report contained only one page of financial highlights, with little detail and no comparative data from year to year. The annual operations reports and audited financial statements are thorough and professional but not available on the Global Forum Web site. This suggests that, while the Global Forum makes major efforts at communications, its own external transparency could be regarded as inadequate. 3.44 While the Foundation Council has frequently discussed governance issues, it is not known whether the FC has considered possibilities for increasing external transparency. It has not adopted a disclosure policy. Possibilities to increase transparency include distribution of decision summaries of FC actions after meetings and increasing the management and especially financial information provided in the Global Forum Annual Reports. 3.45 The sound conflict of interest policy of the Global Forum contributes to its legitimacy. No case of conflict of interest between an FC member's external activities and the member's engagement in the Global Forum appears to have arisen, perhaps in part because of the existence of the policy. The internal conflicts of interest between FC members' oversight roles and their activities in direct support of the staff appear, however, not to have been consciously managed. A few candidates nominated for appointment to the Foundation Council have been excluded from consideration because of the conflict of interest that would be created as a result of their membership on the board of initiatives supported by the Global Forum. Overall, despite some conflicts, the Global Forum's internal record on conflict of interest in relation to its activities has been stronger than that of the more complex links of World Bank staff to various Global Forum-related activities (see below). 3.46 Collaboration with COHRED poses numerous governance challenges to the Global Forum. While the Global Forum and COHRED have endeavored to define the limits of each other's activities in a way that will avoid overlap, overlaps remain. COHRED states that it provides a "voice for the south" in determining the international health research agenda. The GFHR emphasis on health equity and the poor is merely different phraseology for the same thing. The Evaluation Team for the GFHR's second evaluation all but formally 56. One commentator also observed that neither the FC Chair nor the Executive Director is a public health professional. 35 recommended merger of the two organizations. This GPR finds that re-examination of the institutional architecture for health research could envisage a merger. A rational institutional NGO architecture for the promotion and support of health research in the interest of developing countries would create only one organization, especially since -- as one interviewee for this study observed -- it is structurally inefficient to have two boards.57 The Global Forum and the Future of Research on Developing Country Health 3.47 The principal challenges for the Global Forum as it examines its future role in research on developing country health are to rethink its goals and roles and its position in the evolving institutional architecture for developing country health research. The external environment profoundly affects this examination by the changing epidemiology of developing country health conditions, by the rapidly growing financial resources for health research, and by the emergence of innovative developing countries on the health research scene (Box 8) and of major institutional financiers of health research (particularly but not exclusively the Bill and Melinda Gates Foundation and the US National Institutes of Health) with which the Global Forum has not had significant prior engagement. Changing external circumstances call not only for flexibility -- frequently emphasized by the Global Forum in its self-definition -- but also for focus and selectivity. A very wide focus has characterized the Global Forum in recent years, as seen, for example, in its new strategy discussed at the conclusion of Chapter 2 and in its "Report Card" on R&D for Health (Box 9).58 Box 8. Innovative Health Research in a Middle-Income Country -- Brazil Brazil is unique in Latin America in having a well-structured health research policy coordinated by a Vice-Ministry for Science and Technology of the Ministry of Health. Since 2003 this policy has involved a wide range of public, academic and private institutions at the national and state levels, as well as private laboratories and universities in the Northern Hemisphere. Brazil's policy places special attention on state of the art biotechnology and research to address neglected diseases. Over $30 million in grants and loans from the Ministry of Education and national development banks support this work. This activity has established a critical mass of researchers, who have published over 400 international journal papers annually. Brazil's research policy also includes a joint Ministry of Health-Ministry of Science and Technology $10 million program to address six neglected diseases -- dengue, malaria, Chagas, leishmaniasis, TB, and leprosy. A special effort has been made to involve academic institutions in poor, disease-affected regions of Brazil. In sum, Brazil has become an innovative developing country (IDC) in health. Sources: Royal Tropical Institute 2007 and Gardner et al. 2007. 57. The practical question now is whether the benefits of efforts to merge COHRED and the Global Forum would exceed the costs. The views of knowledgeable, disinterested interviewees on the matter for this GPR varied: Some observers considered that merger would be highly appropriate, with benefits substantially exceeding costs, but others thought otherwise and were not inclined to encourage stakeholders to devote the energy to the issue which would be required to effect a merger. 58. The GFHR disagrees. In its comments on the draft IEG GPR, it observed that its new strategy is sharply focused around three clear strategic priorities and biennial workplans and budgets and, within the three strategic areas there is said to be further strong focusing in terms of the number and types of activities that will be pursued in each biennium. 36 3.48 The enormous gap between the human and financial resources available to the Global Forum and those available to other major players in health research makes a highly concentrated focus desirable in order to achieve impact. The Global Forum "Report Card" (Box 9) established with the 2008 GFHR report on monitoring financial flows illustrates the problem of focus and selectivity for the Global Forum. If the Foundation Council decides to move towards a more concentrated focus, hard choices will be required, and members of some of its constituencies are likely to be opposed. The differences between the Global Forum's comparative advantages as seen by the Forum and as they emerge from this GPR are significant. Box 9. The Global Forum "Report Card" on R&D for Health Starting with its 2008 report on Monitoring Financial Flows for Health Research, the Global Forum sets forth a "report card" of areas that it proposes to examine in its annual analyses of financial flows. The "report card" is revealing, almost breathtaking,59 in the scope of the 10 areas in which the Global Forum proposes to monitor commitments: A. All Countries 1. National R&D total expenditures as a percentage of GDP 2. National R&D for health as a percentage of GDP 3. National R&D for health as a percentage of national health expenditures 4. National R&D for health as a percentage of total R&D B. High-income Countries 1. Gap between actual ODA and commitment to invest 0.7 percent of GNI in ODA 2. Gap between actual annual increase in ODA and commitment to double aid between 2005 and 2010 3. Gap between actual ODA expenditures on R&D for health and target to spend 5 percent of health ODA on R&D for health C. Low- and Middle-Income Countries 1. Gap between actual expenditures on health and target to spend 15 percent of domestic public expenditures on health 2. Gap between actual investments in R&D for health and target to spend 2 percent of national health budgets on health research D. Global Health Initiatives and development agencies 1. Gap between actual investments and commitment to invest 5 percent of overall health investment portfolios of Global Health Initiatives and development agencies to support research capacity of countries. Source: Burke and Matlin 2008. 3.49 Identifying and defining the comparative advantages of a networking and advocacy organization such as the Global Forum poses special challenges because of the necessity to rely greatly on perceptions. Table 6 and especially Box 8 illustrate the problem. They suggest also 59. Commenting on the draft IEG GPR, the Global Forum states that it has always included in its analysis funding flowing from the three domains of development, research and health, and that all that is being added now is to compare the amounts flowing with the targets that various actors have set themselves. A greater focus by the Global Forum could, for example, reduce the number of variables to be monitored by the Forum by eliminating those covered in the reports of other more prominent actors such as the OECD and the United Nations. 37 Table 6. The Comparative Advantages of the Global Forum Global Forum Assessment IEG Assessment 10 year expertise in research for health: authoritative Capacity for collaborative identification of health source of reliable data on resource flows for health research gaps through GFHR initiatives and overall research; proven developer of practically applicable resource misallocations in annual resource flows priority setting tools; track record of well-reasoned and studies; resource flows analysis strategy merits effective arguments rethinking for narrower focus Neutrality and independence of political and Confidence and trust; greater independence than other commercial influence major actors in the global health arena, especially WHO Nimble and flexible organization, able to respond Light administrative structure rapidly and flexibly to emerging opportunities; Ability to work on the wider determinants of health and Not cited in second evaluation report or in IEG across sectors interviews Trusted broker, convener, and partner Visibility and respect from some funding agencies; brokerage capacity; leadership status with a portion of LMIC health researchers and health research policy makers; incubating networks; despite evidence of some bias against private sector, relatively greater (than WHO) access to private sector and NGOs; success with AFM and other meetings in bringing many stakeholders together; helping innovative developing countries to become known; a voice for developing countries in global discussions on health research. Lack of effective insertion at country level a comparative disadvantage.60 Source: Global Forum Strategy, 2008­14. Sources: Second External evaluation, GFHR documents, and IEG interviews. that networking and advocacy bodies such as the Global Forum face a demanding problem of focus and selectivity. The more the Global Forum and its partners and stakeholders can achieve consensus on focus and criteria for choice among alternatives, the greater the Global Forum's likelihood of success.61 3.50 Financial and institutional sustainability is likely to represent a continuing challenge for the Global Forum. The likely emergence of WHO as a significant player in global healthresearch (Box 6) could be a boon for the Global Forum by giving new attention to the issue, but it could also make others wonder about the appropriateness and role of the Global Forum. Despite the Forum's ability to sustain funding of core activities, the decline in total donor support for the Global Forum (Figure 4) from a high of $4.8 million in 2004 to $2.9 million in 2007, and the shift of World Bank DGF funding from Window 1 to Window 2 (see below) are likely to put particular stress on the Global Forum at a time when international economic conditions already pose significant threats to research on developing country health. Global Forum leaders may need to initiate a round of high-level exploratory consultations with the Forum's principal partners and current and likely major funders of 60. The GFHR points out that its COHRED partner has the country-level research-capacity building mandate and considers that COHRED, TDR and the GFHR have complementary roles. 61. The GFHR considers (Annex J) that its new strategy answers the need for the Forum to focus its activities further, to give greater attention to selectivity, and to seek broader engagement with the largest funders of health research and the private sector. 38 health research world-wide (such as the Bill and Melinda Gates Foundation, the US National Institutes of Health, the Wellcome Trust in the UK, and major private commercial enterprises engaged in health research) before the Foundation Council reaches conclusions. The Global Forum may also be required to draw upon its financial reserves (nearly sufficient for a year of operations without further support) during this period, in order to sustain on-going operations while discussions on the future of the Forum take place. 3.51 In late 2008 WHO commissioned a consultant study to assess opportunities for alternative institutional architecture in the health research partnerships based in Geneva. The purpose of the study was to inform a collective decision among concerned parties whethersuch opportunities were worth pursuing. The context for this exercise was the awareness of seemingly converging priorities and activities among the 8 organizations concerned, including the Global Forum.62 The consultant's discussions covered improving the mechanisms to set health research priorities, increasing donor effectiveness, improving focus on the core competencies of the organizations and preventing overlapping, and sharing of knowledge to exploit synergies. Options were presented, and the participants in a follow- up meeting agreed on the desirability of change. The CGIAR model was frequently cited in the discussions. The views expressed in the consultant's work echoed observations at the November 2008 Bamako Ministerial Forum on Research for Health, where developing country delegations called for increased harmonization of global health research governance in accordance with the Paris Declaration. 3.52 At the WHO Executive Board meeting in January 2009 a resolution was proposed to the World Health Assembly requesting WHO to facilitate a further consultative process to assess specific options for greater collaboration among the research partnerships, including governance, common areas of research and of research support, and operational collaboration. While the Bank as such was not part of the initial set of consultations, its representative participated in his capacity as Chair of the GFHR STRATEC. He expressed support for consolidation of health research partnerships, and is expected to participate in further discussions as the consultations are extended to wider numbers of stakeholders. One idea particularly being mooted is the creation of an over-arching global health researchpartnerships board. Such a board could oversee a number of research partnerships, including partnerships where the Bank is currently engaged and some where it is not. The discussions around these ideas have started to gain momentum, and began to appear in the press early in 2009 (Roettingen et al. 2009). The Global Forum reports that the discussions have shifted towards developing active collaboration in areas of shared, complementary interests and that the Forum will provide a platform for a wider dialogue about the global health research and innovation system at its 2009 AFM in Cuba. 3.53 The Global Forum faces both threats and opportunities as it considers how to position itself in the accelerating dialogue on the creation of an obviously needed more rational institutional architecture for research on the health problems of poor countries and poor peoples. One option for the Global Forum to consider is whether it wishes to position itself to 62. The other parties were TDR, HRP, Initiative for Vaccine Research (IVR), COHRED, the Alliance for Health Policy and Systems Research, the WHO research strategy, and the Intergovernmental Working Group on Public Health, Innovation and Intellectual Property. 39 form part of the core of a CGIAR model of coordinated but largely separate funding of a generally agreed set of research centers and work programs under the kind of umbrella partnership board mentioned above. The small financial size of the Global Forum in relation to total publicly funded health research, and the almost complete absence of the most important health research funders from the Global Forum, suggest that it would be virtually impossible for the Global Forum to evolve alone in the direction of the CGIAR. Nor, it should also be pointed out, was it the mandate of the Global Forum to be or become a research financier. Most stakeholders interviewed for this study expressed a hope that the Global Forum would develop in the direction of the CGIAR, but a significant minority thought it would not be politically, financially or organizationally feasible or even appropriate to try. As a small non-profit advocacy organization with reserves limited to a few million dollars, the Global Forum seems extraordinarily unlikely to be able to become a significant health research financier.63 3.54 Another option for the Global Forum, in the face of major players with much larger human, financial, and political resources, would be to take the occasion of the retirement of its chief executive at the end of 2009 and the new discussions simply to declare success as a result of the enormous increases in financial flows for health research, and to offer to be taken over by a larger entity, such as TDR.64 Beyond the World Health Assembly, the IHP+ could be a forum for discussions on options.65 3.55 As the Global Forum and its stakeholders consider its options, they may find it worthwhile to consider the experience of the Global Forum on Agricultural Research (GFAR, Box 10). The GFHR's legal and organizational independence from WHO appears to represent a comparative advantage, relative to the GFAR and FAO. This also suggests the importance of autonomy of decision making for any new entity that might emerge from the broader discussions on global health research partnerships currently under way. 3.56 The picture is not all somber for health research on developing country problems or for health research advocacy on the problems of poor countries and peoples, whether in the Global Forum or elsewhere. The massive growth in financial resources for health research in the past twenty years and the increasing recognition of the importance of health research among decision makers represent an enormous opportunity for change. The private sector is a partner whose interests and resources merit closer examination as part of any rethinking of 63. In 2001, the WHO Commission on Macroeconomics and Health led by Prof. Jeffrey Sachs envisaged creation of a global health research fund with annual disbursements on LMIC health problems of $1.5 billion. 64. The GFHR comments that such an offer would bear no relation to the strategic goals of the Global Forum, especially if it involved merging with an intergovernmental organization. 65. The IHP+ -- the International Health Partnership and related initiatives -- builds on the global framework of development assistance that has emerged over the past decade. This framework commonly calls for rationalizing the health architecture and changing the way donors work together with partner countries through coordinated efforts of national governments, international agencies, bilateral donors, regional governing bodies, civil society, foundations, and the private sector, as well as the need to strengthen in-country health system services. In line with the Paris Declaration, the IHP+ aims to scale-up coverage and use of health services in order to deliver improved health outcomes against the health-related MDGs and universal access commitments. As of January 2009, the IHP+, which emerged from the original IHP, consisted of 14 developing countries, 11 international agencies (including the World Bank) and 12 donors. 40 Global Forum roles and the larger health research architecture. The growth of large markets in the major developing countries has given major enterprises, especially pharmaceutical firms, reason to engage increasingly in these countries (Jack 2009). Work under way at the University of Washington suggests the financial importance of the private sector in developing country health may be under-estimated, so increasing attention to this dimension may be warranted. Thus it will be extremely important to extend the range of participants in the discussions well beyond the eight Geneva health research partnerships and the ministry of health representatives who dominate the World Health Assembly. Box 10. The Global Forum for Agricultural Research An entity similar to the Global Forum on Health Research, the Global Forum on Agricultural Research (GFAR) is a multi-stakeholder-led initiative that serves as a neutral forum for dialogue and action on strategic issues in agricultural research for development (ARD). Created in 1996 and operational in 1998 -- the same year as the GFHR -- the GFAR has its origins in an initiative to link national agricultural research systems and to strengthen their voice. The Global Forum for Health Research has encouraged the development of national health research systems and financing. In the field of agricultural research GFAR is a facilitator of processes, a broker, and an advocate. The key strategic objectives of GFAR are: advocacy for change through agricultural research to meet the future needs of humanity; reshaping institutions for the future to link agricultural science and society; increasing ARD effectiveness by fostering inter-regional partnership and learning; and bridging the knowledge gaps and enabling the poor to access critical knowledge to empower their own innovation and development. GFAR facilitates and promotes cost-effective partnerships and strategic alliances among ARD stakeholders in their efforts to alleviate poverty, increase food security and promote the sustainable use of natural resources. GFAR activities concentrate on consensus-building and advocacy for action on agricultural research and innovation priorities; promotion of global and regional partnerships for collaborative research and innovation; knowledge and communication for agricultural research and innovation; and strengthening the institutional capacities of GFAR and its stakeholders. Major conferences are held about every three years. GFAR has a particularly strong communications program, with an interactive Web site supported by FAO. Following an initial two years located at the World Bank, GFAR is now housed at FAO in Rome. It is managed by a Steering Committee of 16 stakeholders, in cooperation with a donor support group. It has no independent legal status, and a very small staff sometimes strengthened by secondments. Annual expenditures are about $1 million, financed by a range of donors, including particularly Canada and France, but not the World Bank. The CGIAR housed at the World Bank is one GFAR partner. A second external review of GFAR was completed in 2007. It reaffirmed the original vision of GFAR. It found a number of achievements but also a need for greater visibility for the Forum and for closer links with the CGIAR and the World Bank. It called for a sharper focus on advocacy, strategic thinking and partnerships, and stressed a need for greater donor support to address cash flow problems. Source: GFAR Web site. 41 4. World Bank Performance in the Global Forum Partnership 4.1 IEG's 2004 evaluation of the Bank's approach to global programs, including global health programs, recommended that the Bank engage more selectively in global programs, favoring those that exploit the Bank's comparative advantages and provide global public goods. It also recommended that the links between global programs and the Bank's regional and country operations be strengthened (Independent Evaluation Group ­ World Bank 2004). 4.2 Against the above background, this chapter reviews in turn (a) the multiple and occasionally conflicting roles played by the Bank in the Global Forum partnership; (b) the Global Forum and Bank corporate and sectoral strategies and priorities; (c) World Bank DGF support for the Global Forum and DGF policies; (d) DGF Sub-Grants through the Global Forum and World Bank policies, (e) the Global Forum and World Bank HNP grant strategy, (f) the Bank's exit strategy from support of the Global Forum, (g) oversight of the Global Forum partnership, and (h) the risks and benefits to the Bank of participating in the Global Forum partnership. Table 7 contains an overview of the Bank's performance in the partnership at the time of its founding, at present and its potential future roles. The Multiple and Occasionally Conflicting Roles Played by the Bank in the Global Forum 4.3 The World Bank has played many roles in the Global Forum -- initiator at the time of creation, leader in the Foundation Council and STRATEC for many years, technical advisor on specific health issues, promoter of sound management, and more, as discussed in this chapter. 4.4 The Bank has been prominently and continuously engaged as a Global Forum partner since the discussions leading to the GFHR's creation more than a decade ago. At the time of the establishment of the Global Forum, the Bank's convening power and credibility were effectively used. Indeed, the Bank must be considered a "prime mover" in creation of the Forum. In 2000 Bank staff thought that the Global Forum would increasingly assume the role of raising money, channeling funding to high priority activities, and coordinating health research generally. While this may not have been a very realistic vision, the HNP Sector Board expected that the Bank would exit from separate international health R&D grants over time and channel its health R&D funding through an arrangement analogous to the CGIAR.66 The Bank's HNP grant funding proposals for FY01 for the GFHR state that the Forum is greatly assisting the Bank in rationalizing its grant program, first by setting priorities and second by providing a channel for contributions to new initiatives. As recently as FY06, the DGF budget submission to the Bank's Board of Directors observed that, "As the GFHR becomes established, the Bank expects it to play an ever-larger role in channeling funding to high priority health R&D programs." This portrayal was inconsistent with the underlying reality, though perhaps not with the hopes of the staff. 66. DGF summary note by GFHR Task Manager Maureen Law, nd. 42 Table 7. The World Bank's Performance as a Partner in the Global Forum for Health Research Global Forum Founding The Global Forum Today Dimension Potential Future Roles (1997­98) (2008) Using its The Bank was deeply The Bank remains deeply The Bank needs to comparative engaged in the dialogue that engaged in the GFHR, consider how it wishes to advantage at the led to GFHR establishment. financially, through its position itself in the on- global level: (1) WDR93 found that an membership on the going discussions on global mandate international mechanism with Foundation Council, as possible rationalization of and reach, (2) stable funding over the interim Chair of STRATEC, the institutional architecture convening power, medium to long term could and through active for health research on and (3) catalyzing effectively build research participation in the developing country other resources capacity in developing preparation and execution concerns. The Bank could and partners for countries. This finding led of the 2008 Bamako help to move the Global the program. more to COHRED than to the Ministerial Conference on Forum further in the GFHR, but the DALY concept Research for Health and directions initially of the WDR was picked up by other AFMs. Bank financial conceived, of substantial the GFHR. The Bank's 1997 support has been important research coordination, but HNP Sector Strategy to at least some other this would require a major anticipated strengthening GFHR donors. investment of Bank staff collaboration with other time, convening power and agencies on health R&D, credibility to bring together including continued financial partners whose financial support to the new GFHR. The engagement in health matrix annexed to the strategy research is many, many report foresaw Bank times that of the Bank, at participation in major collective both the global and country initiatives proposed by the levels. GFHR. In the early years some Bank HNP staff expected the GFHR to become analogous to the CGIAR. Contributing its No linkages at the country The only current direct Possibilities abound for comparative level. linkages are through cooperation on analytic advantage at the encouragement and work and capacity building country level: (1) loan/credit financing of AFM as the Forum pays greater multi-sectoral participants; however, the attention to financial flows capacity, (2) Bank has independent links at the country level and analytical with initiatives of the Global works increasingly closely expertise, and (3) Forum such as the Road with COHRED. If the GFHR country-level Traffic Injuries and concept of research for knowledge. Research Network, and health, as distinct from with GFHR DGF sub- sector-specific health grantees, including GAVI, research, gains favor and the European Observatory, country-level content, the and IAVI. Bank could engage increasingly through its country teams. Here, too, significant and currently unfunded allocations of human resources would be required. Exercising Minimal oversight but deep Bank oversight budgets With sufficient human effective and engagement through HNP have not permitted resources, the Bank could independent leadership. sufficient engagement, but contribute to greater oversight of its HNP leaders have devoted balance among the Global involvement in the substantial time to GFHR Forum constituencies, program. activities and management, including particularly including sound financial greater roles for the media 43 Global Forum Founding The Global Forum Today Dimension Potential Future Roles (1997­98) (2008) management, frequently in and private sector. The connection with other tasks. Bank could assist the The GFHR upgraded its Forum to address the financial management and problem of focus and appointed a Finance and become more selective. Administration Head as a result of a Bank audit, and the FC established a Finance Committee at Bank initiative. MOUs are being developed with the remaining initiatives, and the Bank is monitoring this work. An Audit Committee is to be established in 2009 pursuant to a Bank financial management assessment. The HD Vice President has attended Global Forum meetings. Identification and Risks seem not to have been Risk management appears The Bank's 2007 HNP management of discussed. not to be on the current strategy seems to suggest risks. agenda, but reputational greater DGF support and risks are growing because other attention to country of the very high level of focus and movement away financial dependency of the from GPGs and other GFHR on the Bank and the GFHR-type activities. shift in DGF support from However, the Bank could Window 1 to Window 2. still deploy its convening power to support GFHR resource mobilization. Facilitating an Not reviewed. No disengagement strategy The Bank's corporate effective, flexible as such has been priorities suggest that the and transparent designed; instead,what was Bank should not disengage disengagement perceived as an inadequate from the global research strategy, as HNP DGF strategy led the partnerships on the health appropriate. DGF Council in FY08 to problems of the poor but reduce HNP DGF funding, only reconsider the reduce the GFHR institutions through which allocation, and shift the these research issues and GFHR from Window 1 to research priorities should Window 2 and thus to be addressed. The 2007 termination of GFHR HNP strategy appears to support in FY11. allow keeping 50 percent of DGF support for activities beyond a rigorous definition of Bank HNP comparative advantages. 4.5 The Bank's representative has been a member of the Foundation Council ever since its inception. A former HNP Director, Professor Richard Feachem, served as Chair of the Foundation Council after leaving the Bank and being elected Executive Director of the Global Fund to Fight AIDS, TB and Malaria. The Bank's member on the Council currently serves also as interim Chair of STRATEC. Bank staff at various levels up to and including Vice Presidents have made presentations at GFHR Annual Forum meetings. The AFMs have served as a convenient forum for dissemination of Bank-sponsored research, including the 44 Bank's studies of health financing, and for efforts to correct misperceptions of the Bank among key stakeholders in Bank HNP work.67 The Bank joined the Global Forum and other partners in sponsoring and planning the 2008 Bamako Ministerial Summit on Research for Health. Within the Global Forum the Bank's FC member has consistently provided technical and policy oversight of GFHR work programs and budgets, and encouraged strengthening of GFHR financial management capacity. The GFHR Executive Director has occasionally presented the Global Forum's work to the HNP Sector Board and Bank HNP staff. 4.6 Beyond its overall roles in the Global Forum, the Bank has played initiating, financing, and oversight roles in many specific GFHR initiatives (Annex Tables 6 and 23). The Bank's engagement with these activities has extended beyond the Global Forum itself, as for example in Bank Transport Sector trust fund support of the Forum-initiated Road Traffic and Injuries Research Network. As mentioned above, despite the Bank's leading role in this aspect of the Global Forum's work, the Forum's involvement with the initiatives is gradually being phased out. 4.7 The linkages between the Global Forum and the Bank become even more multiple and complicated, and occasionally give rise to conflicts, when the sub-grantees of DGF funding through the Global Forum are included (Annex Table 7). Without more detailed research and interviews, extending well beyond the scope of this GPR, it is not possible to prepare an exhaustive catalogue of the many roles played by the Bank in relation to the GFHR, including sub-grantees to the extent that they may be considered to be linked to the GFHR. One apparently accidental consequence of the opacity of the network of relationships in the various HNP partnerships in which the Bank participates is that several years ago the Bank's then Vice President for Concessional Finance, who had oversight responsibility for the DGF, became a Board member of a DGF grant recipient receiving funds channeled through the Global Forum.68 4.8 The Bank's representative to the Global Forum, the Africa Region Senior Advisor for HNP, serves as interim Chair of STRATEC, and as such has assumed the role of an advocate for the Global Forum. But, at the same time he also has responsibility for independent oversight of the Global Forum as the member of the HNP Sector Board with special responsibility for HNP research. When he was elected STRATEC Chair he planned to submit the role to review by the Bank's Committee on Outside Interests. He subsequently chose to serve only as interim Chair, and it was then thought that this would make a review by the Outside Interests Committee unnecessary. The central position of the Bank in GFHR governance, and the conflicts to which the Bank is exposed, were recently underscored by the FC decision that STRATEC should serve as Nominating Committee for future FC members. As a result the Bank has been unable to exercise an independent arms-length oversight role in the Global Forum partnership. 4.9 The Bank's experience in the GFHR partnership reveals sharply the challenges for the Bank in managing multiple and complex staff engagements in such partnerships and avoiding conflicts of interest. Clear leadership by the HNP hub left no ambiguity regarding the Bank's 67. A case in point is the presentation on structural adjustment and health in Africa at the 2002 Arusha AFM meeting. 68. The Vice President retired shortly after the conflict of interest was discovered. He recused himself from participation in review of an IAD audit of the GFHR due to his role as the Board Chair of IAVI. 45 engagement at the time of the GFHR's founding. Over time the Bank's engagement has deepened as the GFHR work program has grown, and the sub-grant system has given rise to conflicts of interest. The Bank is deeply involved with one sub-grantee, GAVI, at the HNP Hub and country level. ECA HNP is deeply engaged with another DGF GFHR sub-grantee, the European Observatory of Health Systems in what appears to be a complementary relationship. As noted above, a Bank Vice-President responsible for concessional finance, including the DGF, served as a Board member for a GFHR DGF sub-grantee, but retired from the Bank shortly before the conflict was identified and before the issue was addressed.69 In the future, identifying and monitoring of Bank linkages with all sub-grantees may be appropriate. 4.10 Despite the occasionally conflicting roles of the Bank in the Global Forum partnership, the Bank's performance in the GFHR partnership (Table 7) was well regarded throughout the interviews carried out for this GPR. While some interviewees would have wished the Bank's representative on the FC to play a more assertive role on issues of GFHR strategy, others expressed appreciation for the modesty brought by the current member and awareness of the risk of being perceived to be dominant. The deep engagement of Bank HNP leaders over many years was widely noted, with great appreciation. There was recognition that the Bank brings not only financial resources and institutional credibility but also technical expertise to bear on the work of the Global Forum, along with a perspective linking health with wider development concerns including macroeconomic policies, strategies and poverty reduction. One interviewee commented, simply but probably correctly, that without the Bank the Global Forum would not exist. The Global Forum and Bank Corporate and Sectoral Strategies and Priorities 4.11 The Global Forum is relevant to the Bank's corporate and HNP sector strategies and priorities (Table 8). The 2007 decision of the Bank's HNP Sector Board to focus on health system strengthening and to concentrate HNP DGF resources in areas of Bank HNP comparative advantage for country-level results on the ground tends to move the Bank's HNP DGF concerns -- at least relative to the 1997 HNP Strategy -- somewhat away from the realm of research for health, despite the remaining significant points of alignment, such as around health equity and intersectoral action for better health outcomes.70 By FY10, according to the Bank's 2007 HNP strategy, the HNP Sector Board would allocate 50 percent of HNP DGF grants approved by the DGF Council and the Bank's Board of Directors in partnerships directly related to Bank HNP priorities. 69. An exhaustive list of these engagements would require further research and consultations beyond the resources available for this GPR. 70. By way of contrast, it should be noted that the Bank's 1997 HNP strategy -- formulated in an expansive mode at a time when the Bank was the largest single source of ODA for HNP -- anticipated strengthening the Bank's collaboration with other agencies on international HNP R&D. It expected continued support to the GFHR, observing that the Global Forum provides a mechanism for focusing R&D resources more tightly on priority subjects, including health policy research, low-cost management of NCDs, and slowing the spread of drug-resistant malaria. The 1997 HNP strategy stated that the continuation of grant financing for priority international initiatives that improve and share knowledge in the fields of nutrition and RH would be encouraged. It said that, in partnership with the Global Forum, the Bank would collaborate with industry to strengthen the R&D pipeline for HNP products needed by poor people in low-income countries. 46 Table 8. The Global Forum and World Bank Corporate and Sectoral HNP Strategic Priorities Bank Strategic Priority Relevance of the Global Forum Corporate Priorities Foster regional and global public goods (GPGs). GPGs are central to the identity, purposes and products of the Global Forum. Promote knowledge and learning in development The Global Forum promotes knowledge and experience. learning in health research. HNP Sector Priorities Improve health conditions for the poor and Health equity is the central theme of the Global prevent poverty due to illness by improving Forum. financial protection. Strengthen health systems and synergy between The Global Forum promotes increasing the role health system strengthening and disease-specific of research in supporting the development of interventions. effective and equitable health systems; the WHO Alliance for Health Policy and Systems Research, a Global Forum initiative, facilitates health systems research. Concentrate grant funding from DGF on areas of In its focus on research for health, the Global Bank HNP comparative advantage, identified as Forum has a conscious inter-sectoral focus; it health system strengthening; intersectoral underscores its convening capacity and approach; regulatory framework for public-private organizes Annual Forum Meetings; and it is cooperation; large-scale implementation of beginning to undertake country-specific analytic projects and programs; convening capacity and work on financial flows for health research. But, global nature; and country focus and presence. the overlap of currently perceived Bank HNP comparative advantage and its participation in the GFHR partnership is only partial. Sources: For corporate strategy -- Speech by R. Zoellick to the 2007 Annual Meetings, October 22, 2007, "Catalyzing the Future: An Inclusive & Sustainable Globalization." This table only includes two of the six themes of corporate priority and strategy in the President's speech. For HNP strategy -- World Bank 2007. 4.12 Partnership programs and the DGF are expected to support activities which the Bank does not undertake through regular network and regional and country operations. However, the Bank also wishes that its partnership programs be relevant to network and regional priorities and activities. Reconciling these two imperatives poses challenges for the HNP Sector Board and the Bank's management. Since the Global Forum Foundation Council has decided consciously not to engage deeply at the country level, the scope for the Bank to bring its comparative advantage of country-level focus to the Global Forum partnership has necessarily been limited.71 World Bank DGF Support for the Global Forum and DGF Policies 4.13 The Bank has long been the Global Forum's most important financial supporter (Annex Table 22). DGF funding for the Global Forum started from a low level, rose substantially, and 71. In its observations on the draft GPR (Annex J), the GFHR suggests that the Bank needs to acknowledge the global public goods necessity of research for long-term new approaches and solutions in operational health programs. The GFHR observes that current malaria programs, in which the Bank has invested heavily, would enjoy little of their effectiveness and success today without the new malaria products and approaches that resulted from health and medical research over several decades. 47 has then declined in the past several years (Annex Table 24). The total approved HNP DGF budget fell from a height of $25 million in FY01 and FY02 to $21 and $18 million in FY08 and FY09. The Global Forum's share of the HNP DGF total reached its maximum of 39 percent in FY02 and has since fallen to 21 percent in FY09. This evolution stands in marked contrast to the expectation of Bank HNP leaders at the time of GFHR establishment that the Global Forum would become the principal channel for Bank grant financing of health research under an umbrella arrangement resembling a mini-CGIAR. That vision continues to animate some HNP leaders but it has not been converted into practice. 4.14 Bank DGF funding of the Global Forum has exceeded the DGF criterion of a maximum DGF support ratio of 15 percent.72 From 1999­2007 DGF funding was 45 percent of total donor financial support for Global Forum activities, excluding sub-grants not managed by the Global Forum. DGF support constituted 20 percent of GFHR resources devoted to GFHR core activities (Annex Table 22). Beyond its support of GFHR core activities, the Bank has supported many GFHR initiatives, studies, projects and networks with earmarked resources, including as many as 7 earmarks in any given year (Annex Table 23). Indeed, Bank earmarked resources for GFHR initiatives, studies, projects and networks represented 32 percent of total donor resources managed by the GFHR over 1999­ 2007 (Annex Table 22).73 Bank DGF core support was 13 percent of total donor resources directly managed by the GFHR, including initiatives, studies, networks, and projects. Among the variety of different ways of looking at the DGF share in GFHR funding, this is the only one that respected the DGF criterion of limiting DGF support to a maximum of 15 percent of donor resources for any given beneficiary. 74 Financially at least, and in the perception of many observers, the Global Forum is extremely closely linked to the Bank, despite its independent legal status.75 DGF Sub-Grants through the Global Forum and World Bank Policies 4.15 While the sub-grantees have been fairly successful (Annex Table 7), the system of DGF sub-grants channeled through the GFHR raises many questions. In the DGF budget submission to the Bank's Board of Directors for FY08,76 the Global Forum is described as an 72. This finding is unaffected by the GFHR analysis in Annex J. The figure and observations there relate only to the core GFHR funding. 73. Since the GFHR is not accountable to the Bank for the DGF sub-grants channeled through the GFHR to other beneficiaries, these figures exclude sub-grants. 74. In its comments on the draft GPR (Annex J), the GFHR observes that the Bank has had difficulty effectively appreciating, managing and overseeing its support for the GFHR and the many research partnerships that have had their origins in it. This is thought to include the Bank's inability properly to distinguish between the global research partnerships still directly under or involved with the GFHR and those no longer directly involved but where the Bank had intended the GFHR to serve as an umbrella. The notion of umbrella suggests, once again, the thinking in the GFHR and at least among some key Bank staff, that the Bank's relations with the GFHR were conceived to evolve towards a CGIAR-type relationship, in which the responsibility for allocation of funds to various research issues and institutions would lie with the GFHR rather than the Bank. 75. The extremely close links between the GFHR and the Bank are also reflected in the Forum comments on the draft of this GPR (Annex J). 76. R2007-0092, May 25, 2007, and Annex Table 23. 48 "umbrella program" including sub-grants, even though the Forum bears no responsibility for oversight of sub-grantee activities and serves only as a channel used by the Bank to direct funds to activities and organizations chosen and overseen by the Bank. The sub-grantees are directly accountable to the Bank. As was noted in IEG's GPR on MMV (IEG 2007), the sub- grant system has created ambiguities regarding responsibilities among the GFHR, the Bank, and the sub-grantees. Oversight and transparency have been diminished compared to what would have been possible with direct grants. 4.16 Information in the Bank's Operations Portal indicates that, in their HNP DGF submissions, the Bank staff have included sub-grants and external funds provided by other financiers to sub-grant recipients in estimating future DGF support ratios.77 The large sums provided to GAVI, IAVI and other sub-grant recipients by non-Bank donors have the effect of reducing the DGF support ratio in these statements of expected sources of funds (all sources) to 3 and 4 percent by inflating the denominator for the calculation. On this basis the GFHR was inaccurately presented as being in compliance with DGF guidelines. The DGF staff routinely cleared letters of agreement with sub-grantees, thus being aware of the fact that they fell outside the scope of GFHR supervision, yet it accepted a budget presentation that integrated the sub-grantees into the HNP support of the Global Forum. Responsibility for the failure to respect Bank DGF guidelines appears to be shared between the HNP Hub and the DGF staff. 4.17 In order to test the DGF support ratio with the inclusion of actual data on funds provided to sub-grantees, for this GPR an effort was made to collect actual data on funding from other sources to six beneficiaries of DGF sub-grants channeled through the Global Forum. With the available human and time resources it proved impossible to collect, reconcile, and add to total amounts actual data comparable to the staff estimates in the DGF submissions. However, for the four sub-grantees on which relevant information was available, the support ratio ranged from less than 1 to 14 percent (Annex Table 7). 4.18 According to the information assembled for this GPR (Annex Table 7), all 6 of the sub- grantees were active, on-going organizations, mostly with independent legal status. Five of the 6 grantees had undertaken evaluations. Bank linkages with the activities of the sub-grantees vary greatly. At one extreme, in the case of the European Observatory of Health Systems, cooperation at the country level has been close. At the other extreme, in the cases of IAVI and INDEPTH, there appears to have been virtually no engagement at the country level. 4.19 The issues raised on the DGF support ratio, concerning the DGF sub-grants and the general lack of transparency concerning different types and levels of financing by the Bank to and through the Global Forum all raise questions concerning DGF governance and the future of the Bank in the Global Forum partnership. How and why were grants made which appear to be conflated into a larger whole by some (Annex J and HNP DGF submissions), 77. Prior year leverage ratios could not be found on the Operations Portal. The audited financial statements of the Global Forum state that the "sole responsibility of the Global Forum is to transfer these grants in accordance with the written instructions of the World Bank." According to the financial statements, the sub-grants are "deducted from total income in the Global Forum's income statements to arrive at the amount of income available to the Global Forum to be used for its core activities and for the projects, initiatives, and networks that it supports." 49 but were in fact processed and signed as separate activities? Either the Bank intended to assign allocation responsibility for DGF funds to the Global Forum, or it did not. It appears that even up to the present there have been significant staff sentiments that the Global Forum should play this role as a step in the direction of a CGIAR model, but the grant agreements were never made in this way. The DGF staff report that Senior Management and the Board prefer to see coherent a grouping of DGF grants for different programs within the same objective of health research, but this is not the way the DGF grants have been operationalized. The Global Forum and World Bank HNP Grant Strategy 4.20 The DGF support for the Global Forum served as a vehicle for targeted support to a variety of organizations while the DGF presented it within the Bank as a single program. 4.21 In 2006 the DGF Council was critical of the HNP submission for the Global Forum and other beneficiaries of HNP DGF grants, for four reasons: (a) lack of a strategic framework and expected results; (b) evaluations due that were not completed on time; (c) poor completion of DGF documents, especially progress reports; and (d) coherence of the portfolio with appearance of duplication across programs. The HNP DGF total was reduced from $24.2 in FY06 to $21.6 million in FY07 (Annex Table 24). Some serious reflection by the Sector Board ensued. Questions were asked as to whether DGF partnerships complement operational work, or whether they should fill in gaps. It was recognized that, while there was little short-term connection with operational work, health research might have direct operational impact in the long-term. In 2007, following a review of HNP DGF strategy, the Sector Board agreed to group the HNP submission into two clusters: (a) overall priority disease partnerships and (b) health systems strengthening. This was a move in the direction of more global funding of health research and health research advocacy. At least some of the allocation responsibility, within an overall envelope, was to be moved from the Bank to a partner such as the Global Forum. However, in practice, the political pressures from disease- specific activists and the particular concerns of senior stakeholders in the Bank led to retaining earmarks and highly disaggregated DGF grants in the health sector, including sub- grants.78 The DGF umbrella is a convenient presentational device that conceals the underlying reality. 4.22 As IEG observed in its 2009 HNP evaluation, in 2007 the Bank was participating financially in 19 global HNP partnerships receiving financial support from the Bank. The number 19 suggests a need for some type of consolidation of Bank support, whether as part of discussions on the Bank's future role in the Global Forum, on the future architecture for health research partnerships, or beyond. Exit Strategy 4.23 During the preparation of the FY09 HNP DGF program the DGF Council expressed continuing concern about the pace of realignment of the DGF support with the Bank's 2007 78. The publicity given to AIDS and malaria suffices to illustrate the power of single disease activists to influence policy makers, to the detriment of people in the Global Forum, the Bank, and beyond who would take a broad overview. 50 HNP strategy. The Council reduced the overall HNP allocation and moved two programs, including the GFHR, from Window 1 to Window 2.79 The effect of this action was to move a program that had enjoyed long-term Bank support and was expected to be a continuing program, towards termination of Bank support in FY11. This action appears to have been based in part on continuing dissatisfaction with performance by the HNP Hub staff and Sector Board and reportedly a lack of Bank follow-through on actions requested by the DGF Council. 4.24 Put simply, GFHR DGF support is expected to end in FY11. Initial exploratory discussions were under way as this report was being completed on possibilities for softening the very sharp distinction between DGF Windows 1 and 2 which might permit continuation in some form. In commenting on the draft GPR (Annex J), the Global Forum has suggested that the Bank and the DGF Council reconsider and fund the GFHR either anew under Window 1 or phase out its funding more gradually through FY13 or FY14. 4.25 The Bank has not prepared an explicit exit strategy from the GFHR partnership. Yet, the termination of Bank funding appears to pose existential risks for the Global Forum. For FY09 the Bank specified that its core support for the Global Forum would help to link resources with priorities in research for health and increase the role of research to support the development of effective and equitable health systems. But, sub-grants for particular issues and institutions were retained, and the overall strategy appeared to represent no change in respect of the Global Forum despite the move towards termination. 4.26 The HNP Sector Board seems, at least implicitly, to have decided that the somewhat declining relevance of the Global Forum to the Bank's HNP strategy and priorities trumps the continuing relevance of the Forum to two major corporate priorities (Table 8). With little conscious management action, the Bank appears to be on the way to exit from broad-based health research and research priority setting (as distinct from disease-specific research) as strategic institutional priorities, despite their obvious importance to the effective achievement of both its corporate and its long term HNP goals. Oversight of the Global Forum Partnership 4.27 The Bank's representative to the Global Forum appears to have carried the oversight responsibility for the Bank's participation in the GFHR partnership almost alone. The HNP Sector Board gave it little attention, and focused on HNP grant strategy only at the time of DGF budget preparation. The Bank's oversight budget and expenditures to support its active engagement in the Global Forum bear little relationship to the extent of the Bank's very deep engagement in the partnership. Expenditures (Table 9) have been heavily concentrated on travel (77 percent of the total, FY05­09) for AFM, FC and STRATEC meetings. This implies that staff time on GFHR oversight and liaison has been cross-subsidized by other tasks in the Bank's work program, and that the Global Forum has been highly constrained in the support that it most needs from the Bank, namely staff time, if the GFHR and the Bank 79. Under Bank DGF policies, DGF Window 1 supports programs that are expected to be of indefinite duration, with grants renewed according to performance. DGF Window 2 supports programs for a maximum of three years. 51 Table 9. HDNHE Administrative Budget Expenditures on Oversight and Liaison Activities in Relation to the Global Forum for Health Research, FY05­09 Fiscal Year Labor Travel Other Total Cost 2005 4,280 9,965 122 14,367 2006 0 20,326 466 20,792 2007 0 12,927 679 13,606 2008 0 22,682 10,459 33,141 2009 9,260 19,987 694 29,942 Total 13,541 85,887 12,420 111,847 Source: Downloaded from SAP, the Bank's Management Information System, on June 10, 2009. are to achieve maximum impact from the Bank's participation in the partnership. Limited Bank budget availability led the Bank's key staff member principally responsible for GFHR oversight not to participate in a GFHR AFM and associated Foundation Council meeting. The funds spent on GFHR oversight compare unfavorably with BB resources devoted to supervision of technical assistance projects of similar magnitudes. Risks and Benefits to the Bank of Participating in the Global Forum Partnership 4.28 The Global Forum partnership presents both risks and benefits for the Bank. 4.29 The Bank faces a reputational risk as a result of having effectively established itself as primus inter pares in the Global Forum partnership along with a widely accepted expectation of continuing financial support for the Global Forum and then terminating that support over a relatively short period.80 The Bank faces unknown risks as a result of the apparent lack of clear oversight of the totality of health and health research partnerships.81 The Bank also faces a reputational risk from establishing certain DGF policies and then failing to adhere to them in the Global Forum partnership. 4.30 The benefits to the Bank of its participation in the Global Forum partnership are at least four. First, by participating (and especially by earmarking more than half of its financial support for particular activities) the Bank has influenced health research priorities towards issues of concern to the Bank and its clients, though it could not be argued that the Bank has had a significant influence on global health research priorities reflected in the allocation of the $160 billion in annual spending. Second, by funding the Global Forum the Bank has contributed to the credibility of the Global Forum and has facilitated mobilization of GFHR financial resources from others. Third, the external criticism of the Bank for its support of economic policy reform and fiscal retrenchment has been softened among GFHR stakeholders, and understanding of Bank perspectives on these issues has increased. Fourth, the Global Forum has served as an effective channel for dissemination of Bank policy 80. It must, however, be admitted that the planned termination of Bank support, while seeming to be sudden and arbitrary from the outside, is slower than in the case of other GFHR partners, such as SIDA. 81. The recent appointment of an Advisor for HNP Partnerships in the Hub may help to manage these risks. 52 studies, such as its 2006 study of Health Financing Revisited: A Practitioner's Guide, which was launched at the Cairo AFM. 4.31 Overall, this review concludes that -- despite the many weaknesses in the performance of the Global Forum and of the Bank in it -- the benefits to the Bank and its clients of its participation in the GFHR partnership have exceeded the costs and risks. Conclusion 4.32 In one important respect the perspectives and paradigms that drive the work of the Bank and of the Global Forum differ. The Bank tends to divide the world into groups of countries -- developing and developed, high, middle and low-income, or developing, in transition, and industrial, or IDA Part I and Part II. While remaining sensitive to developing countries, the Global Forum, in contrast to the Bank, defines itself and analyzes data from a perspective of concern with health research and the poor without a strong conceptual break between or among groups of countries. Naturally, of course, the detailed texts of the Global Forum make frequent reference to improvement of health and health equity in low and middle-income countries. The commonality of health problems among countries, seen especially in the increasingly common concern with non-communicable diseases and the re- emergence of infectious diseases as a major issue in industrial countries, represents a paradigm challenge to the Bank as the Bank turns its attention increasingly to global public goods and issues of common concern to all humanity, such as climate change.82 5. Lessons 5.1 The following lessons can be distilled from the World Bank's experience with the Global Forum partnership: The multiplicity and complexity of health research partnerships in which the Bank is engaged83 pose reputational risks to the Bank and readily give rise to conflicts of interest in the roles played by Bank staff. This is not because the partnerships lack value but because their very multiplicity and complexity require multiple roles and demand a level of oversight and liaison that exceeds the small oversight budgets made available by HNP sector management. Since the oversight work has little internal audience, the staff incentives to carry it out are weak. By their very nature, advocacy and knowledge networks such as the Global Forum tend to have problems of focus and selectivity and difficulty in establishing a clear results framework. This makes the effort to do so all the more important. In the Global Forum the absence of an agreed results framework allowed an environment of relatively low overall institutional accountability for results, despite the efforts made by the program 82. Bloom (2006) considers that the challenge of shifting paradigms towards a common perspective on disease across the entire world lies at the core of priority setting for global health research. In a visionary view, he argues that, while health care is national, health research is global. 83. The health research partnerships in which the Bank is engaged encompass the six GFHR sub-grantees (Annex Table 7). 53 management and in instruments such as the Bank's DGF commitment letters to specify deliverables and indicators. A global program that does not actively engage the most important actors, and bring them into its governance and, where possible, its financing, is likely to have great difficulty in being effective. In the Global Forum the absence of major private commercial, philanthropic and public health research financiers from its Foundation Council and from its donors reduced its effectiveness. Transparency and specificity in the relationships and responsibilities among partners and grant recipients is critical to effective partnerships. In the Global Forum the relationships and responsibilities lacked clarity in respect of sub-grants and sub- grantees. This confusion -- reflected in a vision but an unclear umbrella role for the Global Forum84 -- contributed to the lack of respect for the Bank's requirement that the DGF contribute finance a maximum of 15 percent of donor funding. Governance issues in a small NGO such as the Global Forum can be as challenging in their own way as in a public international organization. The several roles played by board members call for special attention to preserving their independent oversight role. The Bank's deep engagement in the GFHR led it to lose its independent, arms-length oversight role. Basic changes in course are extremely difficult to effect in established institutions, especially where the evidence base for change is limited. Mature partnerships like the Global Forum and COHRED develop a life of their own. Making fundamental institutional change has proven to be extraordinarily difficult despite what appear to be the obvious benefits to be derived by the partners from such changes. This underscores the difficulty, the importance, and the need for assertive leadership in the dialogue now under way on rationalization of the many global health research partnerships. The central responsibility of the governing body for financial and organizational sustainability as well as programmatic sustainability merits underscoring. This should not be a Secretariat task. In the case of the Global Forum, the Foundation Council has yet to seriously face the issues of financial resource mobilization and sustainability. Evaluation TORs and teams should routinely be expected to include financial, economic, and/or business management perspectives and expertise. Those responsible for commissioning evaluations should guard against the risk that the evaluation team will not include such perspectives and expertise. 84. The lack of clarity is also noteworthy in the challenges to the Bank reviewed in the concluding section of the Global Forum's comments on the draft GPR (Annex J). 55 References Bloom, Barry R. et al., 2006. "Priorities for Global Research and Development of Interventions," Chapter 4 in Disease Control Priorities in Developing Countries, 2nd Edition, World Bank, WHO, Fogarty International Center (NIH), Population Reference Bureau, National Library of Medicine, and Bill and Melinda Gates Foundation Burke, Mary Anne and Stephen A. Matlin 2006, eds. Monitoring Financial Flows for Health Research 2006: The changing landscape of health research for development, Global Forum for Health Research, Geneva Burke, Mary Anne and Stephen A. Matlin 2008, eds. Monitoring Financial Flows for Health Research 2008: Prioritizing Research for Health Equity, Global Forum for Health Research, Geneva, November Chee, Grace, Vivikka Molldrem, Natasha Hsi, and Slavea Chankova, 2008, Evaluation of GAVI Phase I Performance, Abt Associates, Bethesda MD, October Commission on Health Research for Development 1990 Health Research: Essential Link to Equity in Development, Oxford University Press, Cambridge Commission on Macroeconomics and Health 2001, Macroeconomics and Health: Investing in Health for Development, World Health Organization, Geneva Cooper, Andrew F. and John J. Kirton, eds. 2009, Innovation in Global Health Governance ­ Critical Cases. Center for International Governance Innovation. Ashgate Publishing Group, Surrey, England Das Gupta, Monica and Lawrence Gostin 2009, How can donors help build Global Public Goods in Health? Policy Research Working Paper 4907, Development Research Group, The World Bank, April Gardner, Charles A., et al 2007. Technological and Social Innovation: a Unifying new Paradigm for Global Health, Health Affairs 26, No. 4, p. 1052­1061, July/Aug. Gardner, Charles A., Tikki Pang, Ok Pannenborg et al. 2008. From Mexico to Mali: a new course for Global Health, The Lancet, Vol. 371, pp 91­93, January 12 Gittinger, Price and Carol Bradford 1992 World Bank Project-Financed Research on Population, Health, and Nutrition, Working Paper 1046, Population and Human Resources Department, The World Bank, November Global Forum for Health Research, Work plan and Budget, various years. Global Forum for Health Research 2001, Findings from the External Evaluation, a Report to the Foundation Council by Fred Binka, Jan Holmgren, and Nirmala Murthy, December Global Forum for Health Research 2002, Strategic Orientations 2003­2005, Global Forum for Health Research, Geneva, October Global Forum for Health Research, nd. Strategy 2008­2014: Making the Case for Research for the Health of the Poor, Global Forum for Health Research, Geneva Independent Evaluation Group ­ World Bank 2004. Addressing the Challenges of Globalization. An Independent Evaluation of the World Bank's Approach to Global Programs Independent Evaluation Group of the World Bank 2007. The Medicines for Malaria Venture, Global Program Review Vol. 1 Issue 2, June 26 Independent Evaluation Group ­ World Bank 2007. Sourcebook for Evaluating Global and Regional Partnership Programs: Indicative Principles and Standards 56 Independent Evaluation Group - World Bank, 2009, Improving Effectiveness and Outcomes for the Poor in Health, Nutrition and Population Jack, Andrew 2009, Big Pharma bets on Emerging Economies? Global Health, Issue 01, Winter Jamison, Dean, Martin E. Sandbu and Jia Wang 2004. Why has infant mortality decreased at such different rates in different countries? Disease Control Priorities Project, Working Paper No. 21, February 6 Jamison, Dean et al, eds. 2006. Disease Control Priorities in Developing Countries, 2nd Edition, World Bank, WHO, Fogarty International Center (NIH), Population Reference Bureau, National Library of Medicine, and Bill and Melinda Gates Foundation Kirton, John 2007, Improving Implementation of G8 Global Commitments: How Scholarly and Civil Society Involvement Helps, Paper presented at the annual meeting of the International Studies Association 48th Annual Convention, Hilton Chicago, Feb. 28 Lee, Kelley, ed. 2003. Health Impacts of Globalization: Towards Global Governance, Palgrave Macmillan, New York MacKellar, Landis 2005. Priorities in Global Assistance for Health, AIDS, and Population (HAP), OECD Development Center Working Paper No. 244, June Moran, Mary et al. 2009, Neglected Disease Research and Development: How much are we really spending? Health Policy Division, The George Institute for International Health, Sydney, February Morel, Carlos et al. 2005, "Health Innovation Networks to Help Developing Countries address Neglected Diseases," Science Vol. 309, pp. 401­403, July 15 Morgan, Lindsay 2009. What does the Financial Crisis mean for Global Health? Global Health, Issue 01, Winter Neufeld, Victor et al. 2001. The Rich-Poor Gap in Health Research: Challenges for Canada, Canadian Medical Association Journal, April 17: 1159­1160 Operations Evaluation Department of the World Bank 2004, "Global Health Programs, Millennium Development Goals, and the World Bank's Role," Case study Roettingen, John-Arne et al. 2009, Global health research architecture -- time for mergers? The Lancet, pp. 193­195, Vol. 373, Jan. 17 Royal Tropical Institute 2007. Strategic Options for Increasing Collaboration between the Council on Health Research for Development (COHRED) and the Global Forum for Health Research, April Rugemalila, Joas B., Olumide A. T. Ogundahunsi, Timothy T. Stedman and Wen L. Kilama, 2007, "Multilateral Initiative on Malaria: Justification, Evolution, Achievements, Challenges, Opportunities, and Future Plans," American Journal of Tropical Hygiene 77: 296­302 Sridhar, Devi and Rajaie Batnaji 2008, Misfinancing Global Health: A case for transparency in disbursements and decision-making, The Lancet, Vol. 372, September 27: 1185­1191 Special Program of Research and Training in Tropical Diseases (TDR) 2008, TDR News, No. 81, November Stevens, Philip 2008. Diseases of Poverty and the 10/90 Gap, in Stevens, Philip, ed., Fighting the Diseases of Poverty, Transaction Publishers, New Brunswick, NJ WHO 2008, WHO's Role and Responsibilities in health research: Draft WHO strategy on research for health, WHO document EB124/12, December 18 World Bank 1993. World Development Report 1993, Investing in Health 57 ------ 1997. Health, Nutrition and Population Sector Strategy, Human Development Network ------ 2007. Healthy Development: The World Bank Strategy for Health, Nutrition and Population Results 59 Annex A Annex A. Evaluation Framework for Global Program Reviews Note: This evaluation framework is a general framework that has been designed to cover the wide range of such programs in which the World Bank is involved, encompassing policy and knowledge networks, technical assistance programs, and investment programs. It is not expected that every global program review will cover every question in this table in detail. Annex Table 1. Assessing the Independence and Quality of the Evaluation Evaluation Questions 1. Evaluation process To what extent was the GRPP evaluation independent of the management of the program, according to the following criteria: Organizational independence? Behavioral independence and protection from interference? Avoidance of conflicts of interest? Factors to take into account in answering these questions include: Who commissioned and managed the evaluation? Who approved the terms of reference and selected the evaluation team? To whom the evaluation team reported, and how the evaluation was reviewed? Any other factors that hindered the independence of the evaluation such as an inadequate budget, or restrictions on access to information, travel, sampling, etc.? 2. Monitoring and evaluation framework of the program To what extent was the evaluation based on an effective M&E framework of the program with: Clear and coherent objectives and strategies that give focus and direction to the program? An expected results chain or logical framework? Measurable indicators that meet the monitoring and reporting needs of the governing body and management of the program? Systematic and regular processes for collecting and managing data? 3. Evaluation approach and scope To what extent was the evaluation objectives-based and evidence-based? To what extent did the evaluation use a results-based framework -- constructed either by the program or by the evaluators? To what extent did the evaluation address: Relevance Governance and management Efficacy Resource mobilization and financial management Efficiency or cost-effectiveness Sustainability, risk, and strategy for devolution or exit 4. Evaluation instruments To what extent did the evaluation utilize the following instruments: Desk and document review Consultations/interviews and with whom Literature review Structured surveys and of whom Site visits and for what purpose: for interviewing implementers/beneficiaries, or for observing activities being implemented or completed Case studies Other Annex A 60 Evaluation Questions 5. Evaluation feedback To what extent have the findings of the evaluation been reflected in: The objectives, strategies, design, or scale of the program? The governance, management, and financing of the program? The monitoring and evaluation framework of the program? Annex Table 2. Providing an Independent Opinion on the Effectiveness of the Program Every review is expected to cover the first four criteria in the following table: (a) relevance, (b) efficacy, (c) efficiency, and (d) governance and management. A review may also cover (e) resource mobilization and financial management and (f) sustainability, risk, and strategies for devolution or exit if the latter are important issues for the program at the time of GPR, and if there is sufficient information available on which to base an independent opinion. Evaluation Criteria and Questions Relevance: The extent to which the objectives and design of the program are consistent with (a) current global/regional challenges and concerns in a particular development sector and (b) the needs and priorities of beneficiary countries and groups. 1. Supply-side relevance -- the existence of an international consensus that global/regional collective action is required. To what extent does the program reflect an international consensus on the need for action, on the definition of the problem being addressed, on priorities, and on strategies for action? Is the original consensus that led to the creation of the program still present? Is the program still needed to address specific global/regional public concerns? Take into account the origin of the program in answering these questions: Is the program formally responsible for implementing an international convention? Did the program arise out of an international conference? Is the program facilitating the implementation of formal standards and approaches? Is the program primarily donor-driven? Did donors establish the program with little consultation with developing countries? Is the program primarily Bank-driven? Did the World Bank found the program and then seek other partners? 2. Demand-side relevance -- alignment with beneficiary needs, priorities, and strategies. To what extent are the objectives consistent with the needs, priorities, and strategies of beneficiary countries as articulated in the countries' own PRSPs, and in donors' strategies such as the World Bank CASs, and the UN Development Assistance Frameworks? To what extent has the voice of developing and transition countries been expressed in the international consensus underlying the program? 3. Vertical relevance -- consistency with the subsidiarity principle. To what extent are the activities of the program being carried out at the most appropriate level -- global, regional, national, or local -- in terms of efficiency and responsiveness to the needs of beneficiaries? To what extent are the activities of the program competing with or substituting for activities that individual donors or countries could do more efficiently by themselves? Pay particular attention to those programs that, on the face of it, are primarily supporting the provision of national or local public goods. 61 Annex A Evaluation Criteria and Questions 4. Horizontal relevance -- the absence of alternative sources of supply. What is the comparative advantage, value added, or core competency of the program relative to other GRPPs with similar or complementary objectives? To what extent is the program providing additional funding, advocacy, or technical capacity that is otherwise unavailable to meet the program's objectives? To what extent are the good and services being provided by the program in the nature of public goods? Are there alternative ways of providing these goods and services, such as by the private sector under regular market conditions? 5. Relevance of the design of the program To what extent are the strategies and priority activities of the program appropriate for achieving its objectives? What are the major activities of the program: Policy and knowledge networking? Financing country and local-level technical assistance? Financing investments to deliver national, regional, or global public goods? (See Annex Table 7.) Has the program articulated an expected results chain or logical framework, along with assumptions that relate the progress of activities with the achievement of the objectives? Does the results chain identify the extent to which the achievement of the objectives depends on the effective functioning of bureaucracies, markets, or collectivities? If so, to what extent are these assumptions valid? For programs providing global or regional public goods, is the design of the program consistent with the way in which the individual efforts of the partners contribute to the collective outcome for the program as a whole -- whether "best shot", "summation", or "weakest link?" Efficacy: The extent to which the program has achieved, or is expected to achieve, its objectives, taking into account their relative importance. 6. Achievement of objectives To what extent have the stated objectives of the program been achieved, or has satisfactory progress been made towards achieving these objectives? To what extent are there implicit objectives that are well understood and agreed upon by the partners and to which the program should also be held accountable? To what extent are there any positive, unintended outcomes of the program that have been convincingly document? To what extent have these assessments by the program or the evaluation been evidence-based? 7. Progress of activities, outputs, and outcomes. To what extent has the program or the evaluation measured the progress of activities, outputs, and outcomes? How did the program or the evaluation aggregate its outputs and outcomes at all levels -- global, regional, national, and local -- to provide an overall summary of its results? To what extent have factors such as changes in the location of the program, its legal structure, or governance processes affected the outputs and outcomes of the program? To what extent have there been outcomes that can be uniquely attributed to the partnership itself -- such as the scale of or joint activities made possible by its organizational setup as a GRPP, or its institutional linkages to a host organization? 8. Linkages to country or local-level activities. To what extent has the program established effective operational linkages with country-level activities, taking into account that: The desired nature of these linkages will vary according to the objectives, design, and implementation of each program? Positive outcomes at the country or local level are generally a joint product of both global/regional and county- level activities? Annex A 62 Evaluation Criteria and Questions Efficiency or cost-effectiveness: Efficiency -- the extent to which the program has converted or is expected to convert its resources/inputs (such as funds, expertise, time, etc.) economically into results. Cost-effectiveness -- the extent to which the program has achieved or is expected to achieve its results at a lower cost compared with alternatives. 9. Efficiency To what extent is it possible to place a monetary value on the benefits arising from the activities of the program? To what extent has the program or the evaluation conducted impact evaluations of representative program activities? To what extent has the program or the evaluation analyzed the program's costs in broad categories (such as overhead vs. activity costs), and categorized the program's activities and associated benefits, even if these cannot be valued in monetary terms? 10. Cost-effectiveness To what extent is the program measuring up against its own business plans: Has the program cost more or less than planned? How did it measure up against its own costing schedule? Have there been any obvious cases of inefficiency or wasted resources? To what extent is the program delivering its activities cost-effectively in comparison with alternatives: How do actual costs compare with benchmarks from similar programs or activities? Are the overhead costs of governing and managing the program reasonable and appropriate in relation to the objectives and activities of the program? How does the program compare with traditional development assistance programs: For beneficiary countries, has receiving the development assistance through the GRPP increased the transactions costs compared with traditional development assistance programs? For donors, has delivering the development assistance through the GRPP reduced donor costs by harmonizing efforts among donors or by reducing overlapping work (such as through joint supervision, monitoring and evaluation)? Governance and management: Governance -- the structures, functions, processes, and organizational traditions that have been put in place within the context of a program's authorizing environment to ensure that the program is run in such a way that it achieves its objectives in an effective and transparent manner. Management -- the day-to-day operation of the program within the context of the strategies, policies, processes, and procedures that have been established by the governing body. Whereas governance is concerned with "doing the right thing," management is concerned with "doing things right." 11. Compliance with generally accepted principles of good governance. To what extent are the governance and management structures and processes well articulated and working well to bring about legitimate and effective governance and management? To what extent do governance and management practices comply with the following seven principles: Legitimacy -- the way in which governmental and managerial authority is exercised in relation to those with a legitimate interest in the program -- including shareholders, other stakeholders, implementers, beneficiaries, and the community at large? Accountability -- the extent to which accountability is defined, accepted, and exercised along the chain of command and control within a program, starting with the annual general meeting of the members or parties at the top and going down to the executive board, the chief executive officer, task team leaders, implementers, and in some cases, to the beneficiaries of the program? Responsibility -- the extent to which the program accepts and exercises responsibility to stakeholders who are not directly involved in the governance of the program and who are not part of the direct chain of accountability in the implementation of the program? 63 Annex A Evaluation Criteria and Questions Fairness -- the extent to which partners and participants, similarly situated, have equal opportunity to influence the program and to receive benefits from the program? Transparency -- the extent to which a program's decision-making, reporting, and evaluation processes are open and freely available to the general public? Efficiency -- the extent to which the governance and management structures enhance efficiency or cost- effectiveness in the allocation and use of the program's resources? Probity -- the adherence by all persons in leadership positions to high standards of ethics and professional conduct over and above compliance with the rules and regulations governing the operation of the program? 12. Partnerships and participation To what extent has the program identified a complete list of stakeholders, or "stakeholder map", including the agreed- upon or perceived roles and responsibilities of the categories of stakeholders identified? To what extent is this a routine programmatic function, updated regularly, and transparently available? Has the program adopted primarily a shareholder model of governance (in which membership on the governing body is limited to financial and other contributors), or a stakeholder model (in which membership also includes non- contributors)? To what extent, if any, is the program's legitimacy being sacrificed in order to achieve greater efficiency, or vice-versa? 13. Programs located in host organizations To what extent is the location of the program in the Bank or other partner organization adversely affecting the governance, management, or other aspects of the program, such as compliance with the principles of transparency and fairness? For which functions is the program manager accountable to the host organization and the governing body of the program, respectively? Are conflicts of interest being managed appropriately? To what extent does the host organization play such a dominant role in the program, thereby reducing the incentives of other partners to participate effectively, or reducing the ability of the host organization to look at the weaknesses of the program objectively? Resource mobilization and financial management: Resource mobilization -- the processes by which resources are solicited by a program and provided by donors and partners. Financial management -- the processes that govern the recording and use of funds, including allocation processes, crediting and debiting of accounts, controls that restrict use, accounting, and periodic financial reporting systems. In cases where funds accumulate over time, this would also include the management of the cash and investment portfolio. 14. Resource mobilization To what extent has the program succeeded in raising financial resources commensurate with its objectives? And from what sources -- the Bank, bilateral donors, foundations, etc.? To what extent has the program succeeded in diversifying its funding beyond a small number of donors? To what extent are the sources of funding for the program (including donor restrictions on the use of resources) affecting, positively or negatively: The strategic focus of the program? The outputs and outcomes of the program? The governance and management of the program? The sustainability of the program? Annex A 64 Evaluation Criteria and Questions 15. Financial management Are there any issues that have emerged during the course of the review in relation to: The quality of financial management and accounting? The methods, criteria, and processes for allocating funds among different activities of the program? Financial management during the early stages of the program? Sustainability, risk, and strategy for devolution or exit: Sustainability -- When applied to the activities of a program, the extent to which the benefits arising from these activities are likely to continue after the activities have been completed. When applied to a program itself, the extent to which the organization or program is likely to continue its operational activities over time. Devolution or exit strategy -- a proactive strategy to change the design of a program, to devolve some of its implementation responsibilities, to reduce dependency on external funding, or to phase out the program on the grounds that it has achieved its objectives or that its current design is no longer the best way to sustain the results which the program has achieved. 16. Sustainability of the benefits of the program's activities What is the risk, at the time of evaluation, that the development outcomes (or expected outcomes) of the program will not be maintained (or realized)? This depends on (a) the likelihood that some changes may occur that are detrimental to maintaining or realizing the expected outcomes, and (b) the affect on the expected outcomes if some or all of these changes actually materialize? 17. Sustainability of the program This will depend on a number of factors, such as the continued legitimacy of the program, its financial stability, its continuity of effective management, and its ability to withstand changing market or other conditions. To what extent is there still a sufficient convergence or accommodation of interests among the major partners to sustain the program financially? To what extent has the program developed institutional capacity such as performance- based management, personnel policies, learning programs, and knowledge management that help to sustain a program? In what areas could the program improve in order to enhance its sustainability, such as better marketing of the program's achievements in order to sustain its reputation? 18. Prospects for continuation and strategies for devolution or exit To what extent should the program be sustained? Is the continuation of the program the best way of sustaining the results achieved? Should the design of the program be modified as a result of changed circumstances, either positive or negative? What other alternatives should be considered to sustain the program's results more cost-effectively, in the light of the previous evaluation findings with respect to relevance, efficacy, efficiency, and sustainability: Reinventing the program with the same governance? Phasing out the program? Continuing country or local-level activities with or without devolution of implementation? Seeking alternative financing arrangements, such as revenue-generation, or self-financing to reduce dependency on external sources? "Spinning off" from the host organization? 65 Annex A Annex Table 3. Assessing the Bank's Performance as a Partner in the Program Evaluation Questions 1. Comparative advantage at the global/regional level. To what extent is the Bank playing up to its comparative advantages at the global/regional level -- its global mandate and reach and convening power? To what extent is the Bank's presence as a partner in the program catalyzing other resources and partners for the program? 2. Comparative advantage at the country level. To what extent is the Bank contributing multi-sector capacity, analytical expertise, and country-level knowledge to the program? To what extent has the Bank's country operations established linkages to the GRPP, where appropriate, to enhance the effectiveness of both? 3. Oversight. To what extent is the Bank exercising effective and independent oversight of its involvement in the program, as appropriate, whether the program is housed in the Bank or externally managed? To what extent is the Bank's oversight independent of the management of the program? To what extent does the Bank's representative on the governing body have a clear terms of reference? 4. Risks and risk management. To what extent have the risks associated with the program been identified and are being effectively managed? For example, IEG identified the following risks in its global review: Bank bears a disproportionate share of responsibility for governing and managing in-house programs? Confusion at the country level between global program activities, Bank activities, and Borrower activities? Representation of NGOs and the commercial private sector on program governing bodies? Unclear role and application of Bank's safeguards? Trust-funded consultants and seconded staff representing the Bank on some program governing bodies? 5. Disengagement strategy. To what extent is the Bank engaged at the appropriate level in relation to the Bank's new strategic framework: Watching brief? Research and knowledge exchange? Policy or advocacy network? Operational platform? To what extent is the Bank facilitating an effective, flexible, and transparent disengagement strategy for the program, in relation to the Bank's objectives for its involvement in the program: The program declares "mission accomplished" and closes? The program continues and the Bank withdraws from all aspects of its participation? The program continues and the Bank remains engaged, but the degree of the Bank's engagement in some or all aspects (such as financing) declines over time? Annex A 66 Annex Table 4. Common GRPP Activities Policy and knowledge networking 1. Facilitating This includes providing a central point of contact and communication among practitioners communication among who are working the sector or area of development to facilitate the sharing of analytical practitioners in the results. It might also include the financing of case studies and comparative studies. sector 2. Generating and This comprises two related activities. The first is gathering, analyzing and disseminating disseminating information, for example, on the evolving HIV/AIDS epidemic and responses to it, information and including epidemiological data collection and analysis, needs assessment, resource knowledge flows, and country readiness. The second is the systematic assembling and dissemination of knowledge (not merely information) with respect to best practices in a sector on a global/regional basis. 3. Improving donor This should be an active process, not just the side effect of other program activities. This coordination may involve resolving difficult interagency issues in order to improve alignment and efficiency in delivering development assistance. 4. Advocacy This comprises proactive interaction with policymakers and decision makers concerning approaches to development in a sector, commonly in the context of global, regional, or country-level forums. This is intended to create reform conditions in developing countries, as distinct from physical and institutional investments in public goods, and is more proactive than generating and disseminating information and knowledge. 5. Implementing Rules are generally formal. Standards can be formal or informal, and binding or conventions, rules, or nonbinding, but implementing standards involves more than simply advocating an formal and informal approach to development in a sector. In general, there should be some costs associated standards and norms with noncompliance. Costs can come in many forms, including exposure to financial contagion, bad financial ratings by the IMF and other rating agencies, with consequent impacts on access to private finance; lack of access to OECD markets for failing to meet food safety standards, or even the consequences of failing to be seen as progressive in international circles. Financing technical assistance 6. Supporting national- This is more directed to specific tasks than advocacy. This represents concrete level policy, institutional, involvement in specific and ongoing policy, institutional, and technical reform processes and technical reforms in a sector, from deciding on a reform strategy to implementation of new policies and regulations in a sector. It is more than just conducting studies unless the studies are strategic in nature and specific to the reform issue in question. 7. Capacity strengthening This refers to strengthening the capacity of human resources through proactive training and training (in courses or on-the-job), as well as collaborative work with the active involvement of developing country partners. 8. Catalyzing public or This includes improving regulatory frameworks for private investment and implementing private investments in pilot investments projects. the sector Financing investments 9. Financing country-level This refers primarily to physical and institutional investments of the type found in Bank investments to deliver loans and credits (more than the financing of studies), the benefits of which accrue national public goods primarily at the national level. 10. Financing country-level This refers primarily to physical and institutional investments of the type found in Bank investments to deliver loans and credits (more than the financing of studies) to deliver public goods such as global/regional public conserving biodiversity of global significance and reducing emissions of ozone-depleting goods substances and carbon dioxide, the benefits of which accrue globally. 11. Financing global/ This refers to financing research and development for new products and technologies. regional investments to These are generally physical products or processes -- the hardware as opposed to the deliver global/regional software of development. public goods 67 Annex B Annex B. Program Goals, Objectives, Activities, and Achievements85 The Global Forum gathers, analyses and synthesizes information that is used to influence the global agenda of research to improve the health of the poor, disadvantaged and marginalized people living in developing countries. To provide evidence-based arguments, information is gathered on a range of critical areas, including: world spending on health research, both as an overall total and in relation to specific geographical areas and health problems and challenges faced by low- and middle-income countries (LMICs); research capacity strengthening and ways to remove the barriers to building and using capacities to improve policy-making and practice; and major areas where research gaps exist and where issues of reducing poverty and enhancing health equity are paramount. The Global Forum develops, tests and disseminates tools to assist in research priority setting and in eliminating biases from research in the health field. It collaborates with expert groups and organizations to map major gaps and to help set the health research agenda for neglected diseases and conditions and for neglected peoples. In addition, the Global Forum promotes attention to the wider horizons of "research for health," recognizing the complex and multi-sectoral origins of factors that determine the health of people and the need for better understanding of social, economic, political and environmental determinants of health and for improved mechanisms to set policies that favor enhanced health. The Global Forum also brings groups together, facilitating dialogue, arranging collaborations, initiatives and networks. This Annex contains an overview of the objectives, activities, and overall achievements of the Global Forum. It includes with a table on the initiatives and networks supported by the Global Forum and an overview of the DGF sub-grants for which the Global Forum has acted as a conduit for World Bank funds. The Annex concentrates on the work of the Research and Programs (R&P) staff, with particular reference to 2007. The R&P work to be supported by the Global Forum is identified in the Annual Work plan and Budget approved by the Foundation Council. When work is outsourced, individuals or institutions who will undertake activities are identified through a "call for proposals" or as part of a procedure for a "commissioned study" organized by the Secretariat. The objective of this procedure is to ensure that the Global Forum has access to the best sources of knowledge and expertise. The Global Forum actively promotes the participation of partners from the South in the work it supports. Tracking resources for research for health Tracking resource flows for research for health has been a central concern since the establishment of the Global Forum. The Global Forum is the only organization known systematically and regularly to track these resources. Annual publications alternate between providing an aggregate view of overall global spending and focusing on specific geographical or health-related areas, as well as on methodologies for improving and systematizing data collection. Within each publication, efforts are made to highlight the relationship (and often the imbalance) between expenditures and needs. 85. This annex draws heavily on material in the GFHR 2007 Annual Operations Report and Financial Statements; it also draws upon the 2006 Annual Operations Report and Financial Statements. Annex B 68 Global expenditure on health research has increased more than four times in the last 20 years to above US$160 billion per year, as shown in Annex Table 5. The available expenditure data show only the sources of funding, and not their purpose. A superficial reading of the data would suggest that only 3­4 percent of world health research expenditures is devoted to the health problems of low and middle income countries. Additional resources that fit into analysis of the total picture of financial flows for health research relevant to LMIC needs come from special initiatives of private industry, the not-for profit sector, and the public sector. According to the GFHR, there is no global figure for private commercial sector financing of clinical trials in developing countries. In some countries the external funding of clinical trials can be comparable with or even exceed the national funding for health research. As concerns special industry initiatives, a recent IFPMA report documents about 200 ongoing health programs in developing countries, many with a research component. The financial value of these initiatives was independently assessed to be $1.5 billion. The bulk of global health R&D in the private commercial sector -- about $81 billion or 51 percent of the total -- is devoted to NCDs, and a significant portion of this is relevant to LDC needs because of shifts in developing country epidemiology. In the private philanthropic sector, most of the Wellcome Trust's $0.7 billion for 2007­08 was on health problems of LMICs, and the Bill and Melinda Gates Foundation paid $1.8 billion in grants in 2008. In the public sector, relevant areas include direct funding of medical research in LMICs, such as the UK Medical Research Council center in The Gambia. There are also disease-specific research institutes and initiatives of the US NIH and comparable institutions in other countries, largely on NCDs, with a significant fraction relevant to the needs of LMICs.86 The Global Forum has worked with a number of countries and regional and global initiatives to promote capacity building and the regular and systematic collection, analysis and use of health research funding data as an essential aspect of priority setting for the use of research funds. A study by Mexican investigators assessed expenditures by the National Institutes of Health in Mexico and is reported in the 2007 edition of Monitoring Financial Flows for Health Research: Behind the global numbers. This publication also includes a new assessment of trends in expenditures on research for health in Shanghai. A technical workshop on tracking expenditures on research for health was conducted as part of the Annual African Statistical Symposium in Kigali, Rwanda. The Workshop resulted in the development of a proposal by AFRISTAT for a project aimed at building capacity for the collection of resource flows data in the African region. Collaboration was initiated with the Health Metrics Network to incorporate information on expenditures on research for health in country-specific collections of a range of health-related parameters. 86. Source: Global Forum. 69 Annex B Annex Table 5. Estimated Global Health R&D Funding, 1998­2005 ($, billion) 1998 2001 2003 2005 $ % $ % $ % $ % World-Wide Total Public Sector 38.5 45% 46.6 44% 56.1 45% 66.3 41% Total Private Sector 46.4 55% 59.3 56% 69.6 55% 94.0 59% /a Total Private for Profit 40.6 48% 51.2 48% 60.6 48% 81.2 51% Total Private Not for Profit 5.9 7% 8.1 8% 9.0 7% 12.8 8% Total 84.9 100% 105.9 100% 125.8 100% 160.3 100% /b High-Income Countries (HICs) Public Sector 36.2 43% 44.1 42% 53.8 43% 63.3 39% Private for Profit Sector 40.0 47% 49.9 47% 59.3 47% 79.7 50% /c Domestic Pharmaceuticals 35.0 42% 44.1 42% 53.2 41% 71.0 44% /c Foreign Pharmaceuticals 5.0 5% 5.8 5% 6.1 6% 8.7 5% /d Private Not-for-Profit 5.6 7% 7.7 7% 8.6 7% 12.2 8% Total HIC 81.8 97% 101.6 96% 121.7 96% 155.2 97% Low and Middle-Income /e Countries (LMICs) Public Sector 2.3 1.9% 2.5 2.4% 2.4 2.7% 3.0 1.9% Public Sector Domestic 1.8 1.5% 2.0 1.9% 1.9 2.1% 2.3 1.4% Public Funding from foreign /f 0.4 0.3% 0.4 0.4% 0.4 0.5% 0.6 0.4% ODA Public Funding for International /f 0.07 0.06% 0.07 0.07% 0.08% 0.10 0.06% Research Private for Profit Sector Foreign and Domestic 1.0 1.1% 1.3 1.3% 1.4 1.2% 1.6 1.0% Pharmaceuticals Domestic Private Not-for-Profit 0.08 0.07% 0.08 0.08% 0.08 0.10% 0.12 0.07% /f Foreign Private Not-for-Profit 0.2 0.2% 0.3 0.3% 0.3 0.3% 0.4 0.3% Total LMIC 3.6 3.3% 4.3 4.0% 4.1 4.2% 5.1 3.2% Source: Burke and Matlin 2008. /a The effect of a change in methods and sources of data for the pharmaceutical industry results in an increase of $10.1 billions in 1998, by comparison with previous estimates. /b HIC: Israel 2001, Singapore 2001. /c Foreign Pharmaceutical R&D stands for R&D expenditure outside the United States by US-owned PhRMA member companies and R&D conducted abroad by the US divisions of foreign-owned PhRMA member companies. Domestic pharmaceutical R&D corresponds to the global estimates for the pharmaceutical R&D in HICs reduced from foreign pharmaceuticals R&D. /d Private not-for-profit includes $3.1 billion estimated for private general university funding in 2001, and $2.5 billion in 1998. /e LMIC China (including Taiwan) 2001, Brazil 2001 / 2003, Chile 2001, Cuba 2001, Philippines 2001, Romania 2001, Russia 2001, Slovenia 2001, South Africa 2001/2003, Venezuela 2001. /f International research, foreign private non-profit and foreign ODA are very rough estimates. Annex B 70 Directing attention to major gaps and priorities in research for health Aiding priority setting. The Combined Approach Matrix (CAM) was developed by the Global Forum as a tool to assist priority setting. Based on experiences on the application of the CAM in different settings and feedback from users around the world, a revised CAM monograph was prepared in 2007. Work also advanced on the production of a report in Spanish presenting the experiences of using the CAM in Latin American countries (Argentina, Brazil, Colombia and Mexico) for wide dissemination and use within the region. During 2007 presentations on the CAM and its use were made at the Second National Forum on Health Research in Argentina. Highlighting gaps in equity and health. The BIAS FREE Framework is an analytical tool whose final development, publication and dissemination were supported by the Global Forum. It has rapidly gained popularity in a number of settings in both developed and developing countries as a methodology for uncovering biases in research due to a wide range of social factors, including ability, caste, class, ethnicity, gender and religion. Assisted by co-funding from the Swiss Tropical Institute, in 2007 work progressed on the BIAS FREE Framework Database Initiative, based at the Ifakara Health Research and Development Centre in Dar es Salaam, Tanzania. The project builds an on-line system which will collect global experiences in the application of the BIAS FREE Framework to uncovering examples of bias in health research that arise from social factors. During 2007 the reach of the Framework was extended through presentations in meetings, conferences and workshops in Canada, the USA, and in global meetings. An Executive Summary and the 3-page BIAS FREE Framework were translated into Vietnamese, Chinese and Italian; translations into Russian and German commenced. These translated documents are to be made available on the Global Forum Web site, along with the English, French, Spanish, Portuguese and Arabic versions that were finalized in 2006. Gender. The Global Forum places strong emphasis on the incorporation of gender perspectives in all its work. During 2007 it engaged with and contributed to the work of the CSDH Knowledge Network on Gender and Women's Equality; the Global Consultation on the Eradication of Female Genital Mutilation (FGM) in Addis Abba, Ethiopia, organized by UNDP; a survey being conducted by the Wallace Global Fund on funders of work on FGM, including the drafting of a question for a survey on funding of research on FGM; a chapter for a book, (Walter, U. and Neumann, B. (eds.) Gender in Prevention and Health Promotion: Policy, Research and Practice. Vienna: Springer Verlag, 2007); and discussions on proposals for a Global Women's Health Research Conference that would focus on gender inequality and other key determinants of women's health. Disability. The Global Forum pays attention to disability issues. During 2007 the GFHR engaged with and contributed to a regular inter-agency meeting on disability, at the invitation of the Human Rights and Disability Office of the UN High Commissioner for Human Rights, to provide inputs for the development of the research agenda related to the health of disabled persons and the links between human rights and health. In addition, the GFHR participated in a World Bank workshop on the Convention on the Rights of Persons with Disabilities: Opportunities for Development Agencies in the Health Sector and in the fight against HIV/AIDS. 71 Annex B Poverty and health Identifying approaches to financing health systems in ways that provide social protection for the poor has emerged as a key area of concern of the Global Forum. Its work has included the collection of examples of good practice in the equitable financing of health systems. The GFHR 2007 report on "Learning from Experience: Health care financing in low- and middle-income countries" offers a framework to assess the performance of a health care financing system and to make it more equitable, efficient and sustainable by optimizing the three main functions of health care financing: revenue collection, risk pooling and health care purchasing. The report was launched during a special session at the 6th World Congress on Health Economics in Copenhagen. Research capacity strengthening (RCS) Five studies, ranging from systematic reviews of RCS to the role and contribution of media in strengthening research capacities, were commissioned and written during 2007, in collaboration with COHRED and WHO-TDR. A monograph comprising these 5 studies along with two-page "messages for decision makers" on RCS was planned for subsequent publication. Public-private partnerships (PPPs) Since the winding down of the Initiative on Public-Private Partnerships for Health (IPPPH) in 2005, the Global Forum has invested in updating the PPP database to make it attractive to a potential partner. INNOGEN (a research group jointly run by the Open University and Edinburgh University in the UK) was identified as a collaborator willing to take over the database, keep it up to date, develop it further and ensure its availability on the web. A new database platform was validated by INNOGEN and work proceeded with completing the data input and transfer of the database to the UK. Mental and neurological health Mental and neurological disorders are responsible for 13 percent of the global burden of disease. Despite this evidence, mental health remains one of the most neglected areas of both treatment and research, with high levels of inequity, marginalization and abuses of human rights being seen in some countries. In collaboration with WHO-MER (Mental Health: Evidence and Research), the Global Forum has conducted a project on mapping of research capacities in Mental Health in LMICs. The global report, Research capacity for mental health in low- and middle-income countries: Results of a mapping project, was finalized and launched on World Mental Health Day, 10 October 2007, through list-servs, a media release and emailing. The report provides an account of the current status of mental health research in 114 LMICs of Africa, Asia, and Latin America and the Caribbean. The study is the first systematic attempt to confirm the pressing needs of improving research capacity in mental health. Sexual and reproductive health (SRH) In collaboration with WHO-HRP (Special Program of Research, Development and Research Training in Human Reproduction), in 2007 a survey was conducted of research Annex B 72 priorities in the field of SRH. A questionnaire requesting views on gaps and priorities in the field led over 500 respondents to produce about 1,500 answers to questions. This information was analyzed and presented at the Global Forum's Annual Forum Meeting 11 in Beijing. An analysis of relevant data was presented at the 11th Summit meeting on Male Contraception in Seattle, a high-level policy-maker group of private and public sector actors involved in male contraception research. The presentation reported that about 1 in 10 final questions emerging from the survey relate to male involvement in reproductive health programs. Malaria The Pharmaceutical R&D Policy Project (PRPP), based at the George Institute for International Health in Australia conducted a World Bank-supported project with US$ 1 million funding channeled through the Global Forum. This project was aimed at determining the likely funding needed for clinical development of new malaria drugs and vaccines, and how and where this funding might best be delivered. The Global Forum acted as co-convener with the PRPP for stakeholder meetings held in London to consult on the preliminary findings. The final report87 was released in September 2007. Promoting and supporting research in neglected areas through networks and initiatives The Global Forum has helped to generate and foster a number of networks and initiatives addressing neglected diseases, conditions and determinants of health outcomes, and maintains close working relationships with them through processes such as membership of Boards, provision or channeling of funds or collaboration on activities. A summary of each initiative is in Annex Table 6 below. Global Forum Achievements As summarized in the Global Forum 2008­2014 Strategy, after a decade of operations, the Global Forum points to its significant achievements in the field of health research for development: The Annual Forum Meetings have become established as premier events in the international calendar, with the capacity to attract presidential and ministerial-level speakers to the opening ceremonies and to secure the participation of senior national and international policy-makers and organization executives in the sessions. The several hundred participants each year represent a cross-section of the constituencies with which the Global Forum seeks to engage -- including policy-makers, representatives of international NGOs, women's organizations, civil society, researchers and research leaders and funders, development agencies, donors, the private sector and the media. The Forums are moving increasingly in their focus: away from only highlighting messages about the financial gaps in health research for development and presentation of initiatives being created to address them; and towards engaging in discussions of 87. M. Moran, J. Guzman, A.L. Ropars, M. Jogensen, A. McDonald, S. Potter, H.H. Selassie, The Malaria Product Pipeline: Planning For The Future. George Institute for International Health, Sydney, 2007. http://www.thegeorgeinstitute.org 73 Annex B the significance and policy implications of successes and failures in research for health and debates about the evolving knowledge of solutions and how to apply them on a large scale. Analytical and research work that the Global Forum has undertaken in several areas is well known and increasingly widely quoted and applied. Reports are now produced annually on financial flows for health research, which alternate between overviews of aggregate global spending and in-depth, disaggregated studies of resources for specific geographies, health conditions or segments of the research spectrum. The Global Forum is the only organization regularly collecting, analyzing and disseminating global spending data on the whole spectrum of research for health and development. The results are cited by many groups, from researchers to leading politicians and donors, to argue for increasing efforts to focus health research on the needs of the developing world. A number of countries are now taking up the tracking of their resources for health research and making correlations with national (and, increasingly, global) health priorities. The analytical tool that the Global Forum has developed to assist in research priority setting -- the Combined Approach Matrix -- has proved valuable across a spectrum of health issues and in a range of settings including international organizations, research institutions and ministries. The BIAS FREE Framework is an analytical tool whose final development, publication and dissemination were supported by the Global Forum. It has rapidly gained popularity in a number of settings in both developed and developing countries as a powerful methodology for uncovering biases in research due to a wide range of discriminatory social factors. There has been widening recognition of the Global Forum as an important and authoritative voice in the field of health research for development, with growing requests for the Forum to play a role as the convener, co-convener or facilitator of meetings and for its staff to participate as chairs, speakers and discussants in a variety of international gatherings. There has been considerable success in generating, incubating and nurturing a range of networks and initiatives to address specific neglected areas of research. For example, the Initiative on Cardiovascular Health in Developing Countries (IC-Health) and the Child Health and Nutrition Research Initiative (CHNRI) have matured into independent foundations; the Alliance for Health Policy and Systems Research (Alliance HPSR) has become permanently embedded in WHO as a key research resource that is attracting substantial donor support. Some of the regular publications of the Global Forum are now well known and widely cited as authoritative sources of information on health research for development. In particular, the four successive editions of the "10/90 Report" played a major role in securing international recognition for the Global Forum's message promoting health research for the needs of the poor, while the newer annual compilation of authoritative and provocative articles in the "Global Forum Update on Research for Health" is extending the message and has already had significant impact in popularizing the broader idea of the spectrum of "research for health." Annex B 74 An important partnering relationship has developed with the Council on Health Research for Development (COHRED). This is already bearing fruit in a range of areas, including joint publications and fund-raising and collaboration in the annual Forums and as members of the core group (with WHO, World Bank, UNESCO and Ministry of Health, Mali) organizing the Global Ministerial Forum on Research for Health in Bamako in 2008 and the core group (with Ministry of Health, Brazil and PAHO) organizing a regional conference for Latin America in Rio in 2008. A variety of international groups now turn to the Global Forum as a potential partner, seeking to draw on characteristics such as its technical expertise, its established convening power, its reach into the international community and its status as an independent and neutral party. Recent examples of such partnering approaches include: setting the agenda for research on sexual and reproductive health (with the Special Program of Research, Development and Research Training in Human Reproduction located at WHO); mapping the mental health research capacities of developing countries (with WHO-Mental Health: Evidence and Research); co-organizing a consultation on public health, innovation and intellectual property (with Knowledge Ecology International, MSF and DNDi), and working with the HIROs group (Heads of International Biomedical Research Organizations) and the European Foundations Center on meetings promoting health research for development. Sub-Grants of World Bank DGF resources At the request of the Bank, the Global Forum has served as a channel for certain DGF sub-grants, to beneficiaries chosen by the Bank and for purposes and activities defined by the Bank in agreement with the sub-grantee. A summary of the activities of each sub-grantee is presented in Annex Table 7. 75 Annex B Annex Table 6. Initiatives of the Global Forum for Health Research Name Mission, Goals, Year Status Global Forum Expenditure World Bank Assessment and Principal Established (December Contributions and Financing Involvement Activities 2008) Initiative for Study and analyze 2000 Closed by Global Initiation of the Rockefeller Promotion of the Wound down in Public-Private the full range of Forum in 2004, IPPPH; database Foundation, Initiative 2005 for lack of Partnerships in product following maintenance World Bank, donor support to Health (IPPPH) development publication of Global Forum continue public-private various research partnerships in studies. health, from early Database of 90 research through PD-PPPs is product being migrated to development to a new host in the product introduction UK and distribution; foster development of new partnerships Alliance for Health Raise the 1999 Legally and Initial legal Annual Member of the Well regarded, Policy and international profile administratively responsibility and expenditure Partnership well embedded in Systems Research of health policy and part of WHO provision of core around $2.4 Board; about WHO; (AHPSR) systems research, since 2006, with funding for the million. Alliance $400,000 per independent and encourage an independent secretariat and receives major year provided evaluation carried knowledge Partnership projects support from from 2001 out in 2004 generation and Board and several donors, through 2005 use; makes small Strategic and including SIDA- grants for LDC Advisory SAREC, health researchers Committee of Norway, IDRC, experts; Global and DFID. Forum a Board member Initiative for Provide a research 1998 Independent Initial legal About $680,000 Initiated and Has had difficulty Cardiovascular response to the Swiss foundation responsibility and a year. World through Global attracting Health in rising burden of since April 2006; processing of core Bank principal Forum-funded financial Developing cardiovascular Global Forum a funds and projects financier studies on micro resources Countries (IC diseases in Board member from World Bank; and macro beyond World Health) developing but gradually limited economics of IC Bank countries disengaging administrative and health, including financial support, country case including accounts studies preparation Annex B 76 Name Mission, Goals, Year Status Global Forum Expenditure World Bank Assessment and Principal Established (December Contributions and Financing Involvement Activities 2008) Sexual Violence Promote research 2000 Became Actively engaged Around Bank has Good work Research Initiative on sexual violence operational, Board member; $240,000 per strongly program and (SVRI) and generate initially housed in facilitating funding year; secured promoted SVRI progress but empirical data that WHO, only in from the World grant of uncertain future ensures sexual 2004, after 4 Bank; funded three $300,000 from despite strong violence is years of year project from Ford leadership recognized as a consensus- own resources; Foundation legitimate public building; starting SVRI remains a health problem in 2006, housed Global Forum in the MRC in project. South Africa following an international competition. The SVRI will organize a Forum on coordinated evidence-based responses to end sexual violence in South Africa in July 2009 Child Health and Promote and 2001 Physically Initially provided Annual budget CHNRI is Has had little Nutrition Research support expansion located at secretariat; legal around essentially a success in Initiative (CHNRI) of research on IDDR,B in responsibility and $650,000. World Bank attracting support priority child health Bangladesh; processing of core initiative, carried beyond World and nutrition legally an funds from World out through the Bank issues, including independent Bank; extensive Global Forum identification of new Swiss foundation administrative and research priorities since June 2006; financial support, Global Forum a including Board member administrative and but gradually financial support for disengaging compliance with Swiss law. 77 Annex B Name Mission, Goals, Year Status Global Forum Expenditure World Bank Assessment and Principal Established (December Contributions and Financing Involvement Activities 2008) Global Network for Support key 2001 Registered as a Core funds for Annual World Bank As of 2006 was Research on research and US non-profit secretariat from the expenditure initiative still dependent on Mental and implementation of organization in World Bank; main around Global Forum Neurological needed services for 2003 funder for research $30,000; and needed to Health (GNRMNH) mental and studies funding sought work on fund- neurological from US NIH raising strategy disorders in poor and Center for for sustainability populations Mental Health and impact Services in USA. As of 2006, Global Forum was sole funder. Road Traffic and A partnership of 2002 Since January Board member; Annual budget Large grant Increasingly Injuries Research 175 stakeholders 2006 housed at main funder of $100,000 in ($500,000) strong and Network (RTIRN) interested in the Aga Khan activities, with 2006; 2007 provided by independent collaboration on University in World Bank funding Global Road road traffic injury Pakistan; Global resources; Forum approximately Safety Facility of research in LMIC Forum still legally administers less 18% Global the Bank's responsible but and less of World Forum, 79% transport sector RTIRN effectively Bank grant, with World Bank, 3% independent signature authority WHO in RTIRN Secretariat Sources: Web sites of networks; Global Forum documents; World Bank documents; author's assessments. Annex B 78 Annex Table 7. Global Forum Sub-Grantees Benefiting from World Bank DGF Funds: An Overview Global Forum DGF Sub-Grant Status Contributions Sub-Grantee Mission, Goals, and Year (FY­FY) Other World Bank Name (December Expenditure and Evaluation Principal Activities of Origin Involvement 2008) World Bank Share Financing (DGF Support Ratio) /a International IAVI aims to ensure 1996 IAVI is a thriving From 2004 through IAVI expenditure Not fully known; a IAVI's first AIDS Vaccine that safe and effective independent 2008, the DGF was $40 million in former World Bank external Initiative preventive HIV organization, provided $8.6 million 2003, and rising staff member led evaluation in (IAVI) vaccines are established at out of a total of $649.9. rapidly. IAVI had the first and second 2003 was developed that are the initiative of over 30 donors external positive, appropriate for use the Rockefeller World Bank share: by 2003, evaluations. A outlining key throughout the world, Foundation. Its 1.3%. including bilateral World Bank Vice areas of very in particularly in those donor relations donors, major President was a important IAVI regions most affected have been foundations, and member of the IAVI contributions by HIV/AIDS. IAVI particularly private firms. By Board. This created and some areas sponsors research and strong. 2007 IAVI a conflict of interest, where its work advocacy activities at Multilateral spending had since the VP was could be the country and global organizations risen to nearly also responsible for enhanced. levels and $90 million. concessional IAVI's second development finance in the Bank, external banks initially including the DGF. evaluation, were expected By the time the strongly to provide 15% issue was promoted by of funds but remarked, the Vice the Bank, was provided only President was nearing 2% through retiring. completion at 2002. the end of 2008. 79 Annex B Global Forum DGF Sub-Grant Status Contributions Sub-Grantee Mission, Goals, and Year (FY­FY) Other World Bank Name (December Expenditure and Evaluation Principal Activities of Origin Involvement 2008) World Bank Share Financing (DGF Support Ratio) /a Medicines for Promote, finance, and 1999 Independent DGF contributed $6.21 Average of $22 Through advocacy An external Malaria supervise a Swiss million from 2000­08. million per year and financial evaluation was Venture competitive research foundation; the through 2006; commitment, the completed in /b portfolio of activities GFHR played a rising rapidly Bank contributed 2005, with (MMV) World Bank share of executed by facilitating role MMV income: 2.1% credibility at the positive independent third in the founding time of MMV's findings. An parties, such as of MMV. founding; there has IEG GPR universities and been no linkage at completed in private firms, for the country level, 2007 found development, and limited MMV a registration, access, programmatic successful and delivery of new oversight. product low-cost malaria drugs development public-private partnership -- PDPPP. European The Observatory 1998 Very active DGF $2.35 million Partners The ECA Region An independent Observatory supports and (2004­08) contribute an HNP staff closely evaluation of on Health promotes evidence- average of collaborate with the the Systems based health policy- World Bank share: 200,000 per Observatory; 3 Observatory's (WHO) making through year, leading to a Bank staff (including dissemination rigorous analysis of not available budget of about a Sector Manager) activities the dynamics of health 5 million are shown on its undertaken in systems in Europe, Web site as 2004 found the and by engaging members of the 30 Observatory directly with policy- member highly regarded makers. The Observatory among ECA Observatory is at the Steering Region HNP forefront of Committee. The policy makers. international health Bank has held a The policy and systems number of Observatory research. It has a collaborative events was found to Secretariat based in with the have strong Brussels but Observatory, such brand strength, administratively is part as its 2004 ECA associated with of the WHO European HNP PCU meeting. its WHO and Annex B 80 Global Forum DGF Sub-Grant Status Contributions Sub-Grantee Mission, Goals, and Year (FY­FY) Other World Bank Name (December Expenditure and Evaluation Principal Activities of Origin Involvement 2008) World Bank Share Financing (DGF Support Ratio) /a Office based in Staff consider the World Bank Copenhagen. Observatory a good partnerships. example of a research partnership that directly benefits Bank operational work. Global The GAVI Alliance and 1999 Active, dynamic, $4.5 million Donor Along with UNICEF, An independent Alliance for GAVI Fund are linked evolving (2004­08) contributions WHO, and the evaluation Vaccines and PPPs dedicated to organization, through 2005 Gates Foundation, completed in Immunization increasing child with Secretariat amounted to $1.6 the Bank is one of 4 2008 found (GAVI) vaccine use, created of about 50 World Bank share: billion; donors key partners in that, overall, in response to an offer people less than 1% included 10 GAVI. The Bank coverage rates of the Bill and Melinda bilaterals, has a permanent increased in Gates Foundation of European seat on the 16 GAVI $750 million. The two Commission. member GAVI beneficiary PPPs have gradually Board; its countries merged in the past representative has through the end several years into one been the HD of 2005 for Swiss non-profit. GAVI VP/SVP. The Bank DPT3 from 64% funds vaccine is the Treasury to 71%, for programs throughout Manager for the HepB3 from the developing world, IFFIm, with GAVI as 16% to 46% in cooperation with the implementing and Hib3 from governments and with agency. As of 2008, 1% to 7%. leadership from the IFFIm had UNICEF and WHO. raised $1.23 billion through two bond issues. Staff support for GAVI is provided through the HNP Hub; extent of cooperation at country level unknown. 81 Annex B Global Forum DGF Sub-Grant Status Contributions Sub-Grantee Mission, Goals, and Year (FY­FY) Other World Bank Name (December Expenditure and Evaluation Principal Activities of Origin Involvement 2008) World Bank Share Financing (DGF Support Ratio) /a Multilateral MIM is an alliance of 1997 MIM is an DGF provided nearly Total MIM None, apparently, An independent Initiative organizations and active, ongoing $1.5 million from 1999 funding for 1999­ aside from the review by a against individuals working partnership. to 2004. 2004, managed Bank's close links panel of 7 Malaria (MIM) together to maximize TDR provides through TDR, with TDR. scientists led by the impact of scientific an umbrella for World Bank share of amounted to the Burroughs research on malaria. funding of total contributions: $10.2 million. Wellcome Trust MIM grew out of an research 14% Other donor was carried out Africa malaria capacity partners: USA in 2002. conference in Senegal strengthening. (nearly 50% of in 1996; its An MIM-TDR the total through coordinating task force 2008), Sweden, secretariat -- one of 4 awards research Japan, MIM arms -- has been grants to African Rockefeller successively housed researchers. Foundation, at the Wellcome Trust MIM appears to RBM. In recent (United Kingdom), the operate under years Exxon Fogarty International the legal Mobile has Center (US NIH), the umbrella of TDR become an Karolinska Institute and not to have important donor (Sweden) and the status of a to MIM. African Malaria separate Network Trust organization. (Tanzania). Annex B 82 Global Forum DGF Sub-Grant Status Contributions Sub-Grantee Mission, Goals, and Year (FY­FY) Other World Bank Name (December Expenditure and Evaluation Principal Activities of Origin Involvement 2008) World Bank Share Financing (DGF Support Ratio) /a INDEPTH INDEPTH aims to 1998 Active, $0.850 million annually Recent annual Not known, None Network ­ improve availability autonomous in 2002, 2003, and expenditure data generally. The initial apparently International and flow of reliable organization but 2004 are not available, Executive Director undertaken. Network for information about legal status but the DGF of INDEPTH led the INDEPTH the health and disease unknown; Web World Bank share: funding was 1st external personnel continuous and to provide a site shows apparently used evaluation of the provided a Demographic facility for testing in considerable not available in its entirety. GFHR, while chapter for a Evaluation of poor areas of new activity, Financial partners INDEPTH was World Book Populations approaches and including No annual report or include receiving DGF book on and Their interventions. It brings Secretariat, other financial data are Rockefeller funding channeled disease and Health together demographic Board, Scientific available on the Foundation, through the GFHR; mortality in SSA surveillance sites, Advisory INDEPTH Web site; a SIDA-SAREC, the conflict of INDEPTH work analyses and Committee, financial statement for Wellcome Trust, interest seems only appears to be disseminates participation in 2002 to mid-2005, Bill and Melinda to have been well regarded. information, and builds GFHR AFMs provided to the Bank Gates Foundation discovered after the capacity. INDEPTH Other partners as part of a completion and WHO. Large work was aims to provide "Better include report, only shows Gates Foundation completed. Health Information for Population receipts from the grant announced Better Health Policy." Council, Health Bank. late in 2008. Work started in Africa Metrics but has spread to Network, Asia; it includes Volkswagen collaboration with Stiftung, DFID. developed country institutions. Sources: Web sites of sub-grantees, Global Forum documents, World Bank documents. /a The DGF support ratio is the share of DGF funds in total financing for the partnership; DGF policy provides that the support ratio should not exceed 15%. Note that this table contains no data (except for MMV) on financing prior to 2004). /b See also IEG Report of June 26, 2007, a Global Program Review of MMV. 83 Annex B Annex Table 8. The Global Forum Objectives and Strategies, 2008­14: Relationship of Objectives and Strategic Priorities Objectives Objective 1 Objective 2 Objective 3 Objective 4 Influencing priorities of Increasing coherence Strengthening research Expanding the use of evidence in policy- and research and innovation among global players and innovation in health decision making, through: for health by engaging and the contribution of and health equity to Encouraging systematic attention to current and future high- partnerships in address the health research on the health problems of the level decision makers research and innovation problems of the poor by poor by researchers and policy-makers; for health through promoting focused brokerage and catalytic efforts and increased Widening the dissemination of research roles resources for relevant findings in ways that will enable their research for health utilization, by stimulating improved directed to all sectors communication between the producers and users of research for health Strategies Strategic priority area 1: Linking resources with priorities for research for health 1a Tracking resources for research ++ ++ for health 1b Helping shape the global agenda and priorities for research for ++ + ++ + health Strategic priority area 2: Increasing the role of research in supporting the + ++ ++ + development of effective and equitable health systems Strategic priority area 3: Strengthening + ++ + + innovation for health in LMICs Cross-cutting issue 1: Enhancing health + + ++ + equity in and through research Cross-cutting issue 2: Stimulating research into the wider determinants of + + ++ + health Source: Global Forum for Health Research, nd. Notes: This table shows the relationship between the Global Forum's objectives, strategic priorities and cross-cutting issues. Each strategic priority is to be central (++) to the delivery of at least one of the objectives and may also contribute (+) to the delivery of others. While the cross-cutting issues are to be delivered mainly through their integration into work programs in the three strategic priority areas, the Global Forum are shows them separately in the table to demonstrate how they relate to the objectives and to ensure that they do not lose their visibility or lack attention while being mainstreamed. Annex C 84 Annex C: Global Forum Annual Forum Meetings Annex Table 9. Number of Participants, Countries Represented, and Share of Participants by Gender and by High- and Low-Middle- Income Countries, by Forum Share of Participants from High Share of Participants, by Gender (HIC) and Low-Middle-Income (Percent) (LMIC) Countries (Percent) Number of Number of Female Male Unknown HIC LMIC Participants Countries Forum 0, Geneva, 1997 72 34 22 74 4 69 31 Forum 1, Geneva, 1997 100 34 33 63 4 69 31 Forum 2, Geneva, 1998 180 48 30 70 0 60 40 Forum 3, Geneva 1999 329 58 34 65 1 61 39 Forum 4, Bangkok, 2000 599 97 36 60 4 38 62 Forum 5, Geneva, 2001 763 109 36 63 1 38 62 Forum 6, Arusha, 2002 577 83 31 67 2 31 69 Forum 7, Geneva 2003 550 78 41 57 2 66 34 Forum 8, Mexico City, 2004 690 86 37 52 11 47 53 Forum 9, Mumbai, 2005 601 71 42 57 1 29 71 Forum 10, Cairo, 2006 542 75 44 55 1 36 64 Forum 11, Beijing, 2007 619 77 59 42 63 39 61 Bamako 2008 935 96 28 63 9 28 72 85 Annex C Annex Table 10. Share of Participants by Region and Forum (Percent) Europe- North South Forum/Region Africa Asia Europe Transition America America Forum 0, Geneva, 1997 9 19 46 1 21 4 Forum 1, Geneva, 1997 12 15 51 0 20 2 Forum 2, Geneva, 1998 16 19 44 0 18 3 Forum 3, Geneva 1999 15 18 45 1 16 5 Forum 4, Bangkok, 2000 22 35 20 1 16 6 Forum 5, Geneva, 2001 23 31 30 3 9 4 Forum 6, Arusha, 2002 44 19 17 3 14 3 Forum 7, Geneva 2003 13 16 46 2 17 6 Forum 8, Mexico City, 2004 17 17 22 2 35 7 Forum 9, Mumbai, 2005 11 55 17 1 11 5 Forum 10, Cairo, 2006 31 28 20 1 11 9 Forum 11, Beijing, 2007 14 42 23 0 13 8 Bamako 2008 62 7 18 0 10 3 Annex C 86 Annex Table 11. Share of Participants by Institutional Affiliation and by Forum (Percent) Research Private Development Not Institution type oriented NGO/CSO Government Media commercial Individual Agency determined body enterprise Forum 0, Geneva, 1997 24 18 22 18 0 1 3 14 Forum 1, Geneva, 1997 26 12 19 32 0 0 3 8 Forum 2, Geneva, 1998 38 14 23 20 0 0 1 4 Forum 3, Geneva 1999 38 14 18 23 0 1 2 4 Forum 4, Bangkok, 2000 43 17 24 10 0 1 2 3 Forum 5, Geneva, 2001 43 18 19 13 0 1 3 3 Forum 6, Arusha, 2002 41 15 28 11 0 1 2 2 Forum 7, Geneva 2003 34 21 16 22 0 2 1 4 Forum 8, Mexico City, 2004 35 19 28 12 0 1 5 0 Forum 9, Mumbai, 2005 61 21 6 7 0 1 3 1 Forum 10, Cairo, 2006 36 19 25 7 7 2 2 2 Forum 11, Beijing, 2007 42 19 19 7 6 3 2 2 87 Annex C Annex Table 12. Share of Presenters by Gender and High (HIC) and Low-Middle Income Countries (LMIC), by Forum (Percent) Female Male Unknown HIC LMIC Forum 0, Geneva, 1997 .. .. .. .. .. Forum 1, Geneva, 1997 .. .. .. .. .. Forum 2, Geneva, 1998 .. .. .. .. .. Forum 3, Geneva 1999 .. .. .. .. .. Forum 4, Bangkok, 2000 .. .. .. .. .. Forum 5, Geneva, 2001 41 59 0 59 41 Forum 6, Arusha, 2002 38 62 0 42 58 Forum 7, Geneva 2003 38 62 0 63 37 Forum 8, Mexico City, 2004 40 59 1 55 45 Forum 9, Mumbai, 2005 45 55 0 37 63 Forum 10, Cairo, 2006 47 53 0 34 66 Forum 11, Beijing, 2007 47 53 0 41 59 Bamako 2008 36 64 0 54 46 Annex Table 13. Evaluation of Annual Forum Meetings by Participants: How Would You Rate the Overall Content of the Meeting? (1 = low, 5 = high) Forum 8 Forum 9 Forum 10 Forum 11 Interest (%) (%) (%) (%) Bamako2008 (%) Score 1 6 1 0 1 2 Score 2 4 5 2 3 6 Score 3 16 21 15 17 17 Score 4 40 50 53 45 29 Score 5 34 23 30 34 46 Relevance/importance (relevance to your work) Score 1 6 2 0 0 4 Score 2 4 4 1 5 0 Score 3 11 20 14 19 17 Score 4 50 44 47 42 34 Score 5 29 30 38 34 45 Dealing with topical/controversial subjects Score 1 7 1 2 2 .. Score 2 18 14 8 11 .. Score 3 30 37 29 31 .. Score 4 33 35 42 37 .. Score 5 13 13 19 19 .. Annex C 88 How would you rate the presentations and discussions? Breadth of relevant topics Forum 8 Forum 9 Forum 10 Forum 11 covered (%) (%) (%) (%) Bamako2008 (%) Score 1 6 1 0 1 4 Score 2 3 6 4 5 4 Score 3 14 26 26 24 21 Score 4 56 47 50 46 53 Score 5 21 20 19 24 19 Balance between topics Score 1 1 1 0 0 2 Score 2 17 7 6 10 4 Score 3 26 38 29 28 36 Score 4 43 41 45 44 38 Score 5 13 13 19 18 21 Coverage of health research for theme of meeting Score 1 5 2 0 0 .. Score 2 4 6 8 4 .. Score 3 31 24 29 17 .. Score 4 41 41 42 38 .. Score 5 20 27 21 41 .. Coverage of health research and the "10/90" gap/research for health Score 1 4 1 0 1 .. Score 2 11 13 9 4 .. Score 3 29 32 31 19 .. Score 4 36 39 41 37 .. Score 5 20 15 18 39 .. How would you rate the accomplishments of the Forum in providing an environment of debate and a marketplace for networking? Forum 8 Forum 9 Forum 10 Forum 11 Environment to debate (%) (%) (%) (%) Bamako2008 (%) Score 1 8 2 0 .. .. Score 2 10 12 3 .. .. Score 3 24 32 34 .. .. Score 4 44 38 42 .. .. Score 5 13 17 20 .. .. Marketplace Score 1 2 0 0 1 .. Score 2 6 6 3 3 .. Score 3 18 21 18 15 .. Score 4 44 39 48 45 .. Score 5 30 34 31 36 .. 89 Annex D Annex D: Global Forum Information and Communications This annex provides background on (1) Key GFHR publications and their distribution; (2) the use of the GFHR Web site; and (3) the GFHR data base used for systematic contacts with people and institutions with a stake in health research. Most important GFHR publications, by year88 1999 1. The 10/90 Report on Health Research 1999 2000 1. Economic Analysis of Malaria Control in Sub-Saharan Africa 2. 10/90 Report on Health Research 2000 2001 1. Monitoring Financial Flows for Health Research 2001 2. Interventions against Antimicrobial Resistance 3. Public-Private Partnerships for Health and Guaranteeing Drug Delivery through Health Systems: Issues Needing Further Analysis 2002 1. 10/90 Report on Health Research 2001­2002 2. Child Health Research: A Foundation for Improving Child Health Research makes a difference 3. Sex, gender and the 10/90 gap in health research 4. Public-Private Partnerships for Improving Access to Pharmaceuticals: Lessons from Field Implementation in Selected Countries 2003 1. Donation Programmes for HIV/AIDS-Related Drugs: Documenting the Early Experience of the Diflucan® Partnership Programme and Viramune® Donation Programme 2. Impact of Public-Private Partnerships Addressing Access to Pharmaceuticals in Low Income Countries: Uganda Pilot Study 3. Valuing Industry Contributions to Public-Private Partnerships for Health Product Development 4. Mainstreaming Gender at Forum 6 2004 1. Global Forum Update on Research for Health 2005: Health research to achieve the Millennium Development Goals 2. 10/90 Report 2003­2004 3. Gender and Health Sector Reform 4. The Economics of Malaria Control Interventions 5. Monitoring Financial Flows for Health Research 2004 6. The Combined Approach Matrix: A priority-setting tool for health research 7. Workshop on Clinical Trials Capacity in Low- and Middle-Income Countries: Experiences, Lessons Learned and Priorities for Strengthening 88. Source: Global Forum for Health Research, January 2009 Annex D 90 8. Liability and other Legal Issues for Organizations Engaged in Product Development through Public-Private Collaboration 9. Partnerships for Developing World Health: Decision and Management Issues for Pharmaceutical Companies 10. Impact of Public-Private Partnerships Addressing Access to Pharmaceuticals in Low and Middle Income Countries: Zambia 11. Impact of Public-Private Partnership Addressing to Pharmaceuticals in Selected Low and Middle Income Countries: A Synthesis Report from Studies in Botswana, Sri Lanka, Uganda and Zambia 12. Impact of Public-Private Partnerships Addressing Access to Pharmaceuticals in Low and Middle Income Countries: Sri Lanka 13. Public-Private Management of Intellectual Property for Public Health Outcomes in the Developing World: The Lessons of Access Conditions in Research and Development Agreements 14. Impact of Public-Private Partnerships Addressing Access to Pharmaceuticals in Low- and Middle-Income Countries: Botswana 2005 1. Combating Diseases Associated with Poverty: Financing product development and the potential role of public-private partnerships 2. Gender, Health and the Millennium Development Goals 3. High-income Country Investors: Financial flows for international health research 4. Report on Forum 8: Health research for the Millennium Development Goals 5. No Development Without Research: A challenge for research capacity strengthening 6. Report on Forum 9: Poverty, Equity & Health Research 7. Global Forum Update on Research for Health Volume 2 2006 1. Flows of Financial Resources for Health Research and Development in Brazil 2000­2002 2. Fluxos de Recursos Financeiros para a Pesquisa e Desenvolvimento em Saúde no Brasil 2000­ 2002 3. Flujo de los Recursos Financieros para la Investigación y Desarrollo en Salud en el Brasil 2000­ 2002 4. Young Voices in Research for Health: Winners of the Forum 10 essay competition for the under- 30s 5. 2005 Review: Focusing research to improve global health 6. Application of Burden of Disease Analyses in Developing Countries: Implication for policy, planning and management of health systems 7. The BIAS FREE Framework: A practical tool for identifying and eliminating social biases in health research 8. Global Forum Update on Research for Health Volume 3: Combating disease and promoting health 9. Why research for health? Research for Health: Policy briefings (series) vol. 1 10. Monitoring Financial Flows for Health Research 2006: The changing landscape of health research for development 2007 1. Report on Forum 10: Combating disease and promoting health 2. Forum 11: Book of abstracts 3. Research issues in sexual and reproductive health in low- and middle-income countries 4. Aspectos de la investigación sobre la salud sexual y reproductiva en países con ingresos bajos e intermedios 5. Por que pesquisa em Saúde? Textos para tomada de decisão 91 Annex D 6. Global Forum Update on Research for Health Volume 4: Equitable access: research challenges for health in developing countries 7. Young Voices in Research for Health 2007: Winners of the 2007 essay competition for the under- 30s 8. Research capacity for mental health in low- and middle-income countries: Results of a mapping project 9. Learning from Experience: Health care financing in low- and middle-income countries 10. 2006 Review: Innovating for better health 2008 1. A Report on Forum 11: Equitable access: Research challenges for health in developing countries 2. 2007 Review: Catalysing innovative solutions for the health of the poor 3. Monitoring Financial Flows 2007: Behind the global numbers 4. Health Partnerships Review 5. Enseignements tirés de l'expérience : Le financement des soins de santé dans les pays à faibles et moyens revenus 6. Monitoring Financial Flows 2008: Prioritizing Research for Health Equity 7. Global Forum Update on Research for Health Volume 5: Fostering innovation for global health 8. Climate Change Report 9. Young Voices in Research for Health 2008: Climate change and health: research challenges for vulnerable populations 10. Policy brief: The use of evidence in policy-making: Six options to improve national policy- making for health 11. Policy brief: Priority research to improve workers' health and safety 12. Searchable Publications CD-ROM 1999­2008 13. Changing mindsets: research capacity strengthening in low- and middle-income countries (COHRED, Global Forum, TDR) 14. 10th Anniversary publication 15. Institutional leaflet in business card holder for Bamako 2008 Distribution of publications GFHR publications are distributed free upon request, and may be downloaded from the GFHR Web site. The following tables describe publications ordered from September 2005 through September 2008. Annex Table 14. Number of GFHR Publication Orders, 2005­08 Orders Period (numbers) From 1/9/2005 to 1/9/2006 4,827 From 1/9/2006 to 1/9/2007 5,005 From 1/9/2007 to 1/9/2008 9,611 Total 19,443 Source: GFHR. Annex D 92 Annex Table 15. Global Forum Publications Ordered by Type 2005­06 2006­07 2007­08 Neglected priorities 1,515 998 1,784 Tools 891 1,176 1,398 Update 786 541 1,380 Financial data 780 1,134 2,970 Other 444 187 96 Forum Report 411 244 287 Institutional .. 418 592 Young Voices .. 307 794 Abstracts 310 Total 4,827 5,005 9,611 Source: GFHR. Web site Use Annex Table 16. Global Forum Web Site Unique Visitors, Visits, and Numbers of Pages Viewed: September 2005 ­ September 2008 Number of Pages viewed Visits Visitors visits per visitor Sept. 2005 ­ Sept. 2006 2,196,585 432,051 153,180 2.8 Sept. 2006 ­ Sept. 2007 2,167,041 623,532 205,850 3.0 Sept. 2007 ­ Sept. 2008 2,099,800 472,406 224,919 2.1 Source: GFHR. 93 Annex D Annex Table 17. Global Forum Web Site Downloads, September 2005­08 Downloads Downloads Visits Rank Downloaded Files (number) (%) 1 Health Partnerships Review Full. PDF 39,256 2.3 2,114 2 Financial Flows 2006.pdf 35,001 2.0 2,532 3 ippph_cd/06.PDF 28,287 1.6 1,926 4 Young Voices in Research for Health 2007.pdf 26,957 1.6 1,759 5 RCS/RCS_Nuyens.pdf 25,150 1.5 3,489 6 Mental Health RC/MHRC_FullText.pdf 24,697 1.4 1,231 7 Global Forum Monitoring Financial Flows2007.pdf 17,030 1.0 1,628 8 global_update2/1_poverty.pdf 15,214 0.9 2,262 9 global_update2/7_addressing.pdf 14,600 0.8 2,100 10 Forum 10 Book of Abstracts.pdf 14,421 0.8 966 Subtotal for rows: 1­10 240,613 13.8 20,007 Other 1,497,748 86.2 898,710 Total 1,738,361 100.0 918,717 Numbers of downloaded files, 2005­08 Period Downloads From 1 Sept. 2005 to 1 Sept. 2006 475,328 From 1 Sept. 2006 to 1 Sept. 2007 563,596 From 1 Sept. 2007 to 1 Sept. 2008 701,309 Total 1,740,233 The GFHR database of contacts The GFHR database of contacts is intended to facilitate the efforts of the Global Forum to promote utilization of existing institutional expertise in the area of research for health through collaboration and information-sharing. In addition, it is a tool to gather information on key players in health research in order to integrate and/or to inform them on Global Forum activities. The database gathers and stores, in a structured way, all data related to the Global Forum's activity, including personal and institutional profiles on partners and contacts; Global Forum events data, including participant data from the AFM and other meetings. The database is also used to communicate with partners, through mass e-mailing and mass surface mailing. Annex D 94 Annex Table 18. Organizations in the GFHR Database by GFHR Constituency Percent Constituency Number (%) Bilateral agency 71 0.8 Foundation 337 3.9 Government 1,610 18.4 Individual 1,360 15.6 International NGO 1,111 12.7 Media 363 4.2 Multilateral agencies 208 2.4 Private commercial enterprise 366 4.2 Research oriented body 3,241 37.1 Service providers 5 0.1 Women's organization 65 0.7 Total 8,737 100.0 Source: GFHR. Annex Table 19. Contacts in the GFHR Database by Country Income Level Country Income Group Number Percent (%) High income 4,824 36 Low income 2,181 16 Lower middle income 4,875 36 Upper middle income 1,632 12 Total 13,512 100 Source: GFHR. 95 Annex E Annex E. Program Timeline Year / month Events Within the Global Forum Events Outside the Global Forum 1990 The independent international Commission on Health Research for Development finds an estimated 93 percent of the world's burden of preventable mortality (measured as years of potential life lost) occurs in the developing world, and that, of the $30 billion global investment in health research in 1986, only 5 percent or $1.6 billion was devoted specifically to health problems of developing countries. The Commission recommended that all countries should vigorously undertake essential national health research. It called upon developing countries to invest at least 2 percent of their national health expenditures on research. It proposed that donors should commit at least 5 percent of health project assistance for health research and capacity building. Finally, it recommended establishment of an international mechanism to monitor progress and promote financial and technical support for research on the health problems of developing countries. 1992 November A World Bank PHN Working Paper (Gittinger and Bradford 1992) finds, from examination of 109 project appraisal reports, that around 90 percent of Bank-financed PHN operations over the previous decade anticipated financing some research, and that, overall, research spending was expected to be between one and two percent of project costs. Only 1 of the 109 projects was a free-standing research project, in Brazil. 1993 World Bank publishes WDR 1993 "Investing in Health." WDR underscores importance of health research and proposes some priorities for research and product development, ranked by the top six contributors to the global burden of disease. Report states international financing for research is needed when the benefits transcend national borders and the research will not be undertaken by the private sector at socially optimal levels, and observes that the total investment in health technology research is woefully inadequate. It finds that an international mechanism with stable funding over the medium to long term could effectively build research capacity in developing countries. Annex E 96 Year / month Events Within the Global Forum Events Outside the Global Forum Establishment of the Council on Health Research for Development (COHRED), in response to the report of the 1990 Commission and the Bank's WDR93. COHRED is a Swiss NGO concentrating on health research capacity strengthening in developing countries. 1996 Report of the WHO Ad hoc Committee on Health Research relating to Future Interventions (funded by the Bank and others), Investing in Health Research and Development, underscores need for increasing resources for biomedical and health policy research, predicting that NCDs would become the leading cause of disability and premature mortality within 25 years. The Committee recommended that a forum for investors in health R&D should be formed to review the needs and opportunities for global health R&D, to help focus resources more sharply on the highest priorities. Visionary proposals by Brazil and Kenya for a global health research treaty are discussed at WHO but very substantially diluted. G8 begins what becomes annual discussion of global health issues, signifying arrival of health issues onto world political agenda at the highest level. 1997 June Stakeholders (including World Bank and others) meet in Geneva in World Bank publishes Health, Nutrition and Population Sector Strategy. what becomes First Annual Forum Meeting, launching the Global Strategy anticipates strengthening collaboration with other agencies on Forum for Health Research. Forum brings together a wide variety of health R&D, including continued financial support to the new GFHR; stakeholders to discuss health research -- government policy makers, continuation of grant financing for priority international initiatives that multilateral organizations, bilateral aid donors, international improve and share knowledge in nutrition and RH; and collaboration foundations, national and international NGOs, women's organizations, with pharmaceutical and other industries on the R&D pipeline for research-oriented bodies and universities, private sector companies, products needed by poor people in low-income countries. The strategy and the media. Former TDR Director Prof. Adetokunbo Lucas (Nigeria) matrix annexed to the report foresees Bank participation in major is elected first Chair. collective initiatives proposed by GFHR. 1998 January Global Forum starts operations at WHO headquarters; Dr. Louis Currat (Switzerland), a former World Bank Young Professional, provided as an in-kind SDC contribution, is first Executive Secretary; Forum initially housed in offices at TDR. June GFHR established as an independent non-profit organization under Swiss law (a Swiss "foundation"). 97 Annex E Year / month Events Within the Global Forum Events Outside the Global Forum June Annual Forum Meeting (AFM) 2 held, in Geneva. The idea of the "10/90 Gap" is introduced, to reflect: (a) Relation between dollars invested in health research and DALYs; (b) imbalance in priorities for research; (c) need to promote research on conditions which affect populations in developing countries. First Forum networks and initiatives are established, to focus on health research related to specific diseases and conditions (see Annex Table 6). 1999 Annual Forum Meeting 3 held in Geneva GFHR initiates publications on financial flows for health research with first "10/90 Report on Health Research", published subsequently in 2000 and further years; this initial 10/90 report introduces the strapline: "Promoting research to improve the health of poor people." GFHR establishes Advisory Group on Monitoring Financial Flows for Health Research International community arrives at the turn of the millennium with an increasingly densely populated set of actors on the health research stage, including new advocates such as COHRED and the Global Forum, new major non-governmental donors such as the Bill and Melinda Gates Foundation, and major growth in public-private partnerships for product development, such as IAVI and MMV (Burke and Matlin 2006). 2000 Strapline "Promoting research to improve the health of the poor" used by Global Forum, in minor adjustment from prior formulation Bangkok Conference on Health Research and Development -- GFHR, COHRED, World Bank, and WHO hold first high-level advocacy meeting for health research; action plan adopted at conference emphasizes social and gender equity in health research, stakeholder inclusiveness, and fostering effective health research systems. AFM 4 held in Bangkok as part of Conference on Health Research for Development GFHR establishes additional networks and initiatives, including IC Health, CHNRI. October FC decides to launch an initial external evaluation of the Global Forum 2001 Annex E 98 Year / month Events Within the Global Forum Events Outside the Global Forum October AFM 5 held in Geneva WHO Commission on Macroeconomics and Health (CMH) proposes establishment of a new Global Health Research Fund, with annual disbursements of around $1.5 billion, to support research on the health problems affecting the world's poor and on the health systems and policies needed to address them. The CMH further calls for an additional $1.5 billion to year of R&D support through existing channels such as TDR and recently established public-private partnerships. The CMH envisages that the Global Forum would play an important role in the effective allocation of this assistance, and foresees the eventuality of a body for health research along the lines of the Consultative Group on International Agricultural Research (CGIAR). GFHR launches study on financial flows for health research updating 1992 estimates which served for WHO Ad hoc Committee. GFHR establishes additional networks and initiatives, on road traffic UN General Assembly adopts Millennium Development Goals for injuries, mental health, and public-private partnerships achievement by 2015, with strong emphasis on health, flowing from Millennium Declaration adopted by Heads of State in 2000. April OAU heads of state and government adopt Abuja Declaration pledging to allocate at least 15% of annual budgets to the health sector. December Findings and recommendations from the initial evaluation of the Forum are reported to the FC. 2002 UN Secretary-General Kofi Annan launches Global Health Initiative (GHI) of the World Economic Forum at its Annual Meeting 2002 in Davos. The Initiative's mission is to engage businesses in public- private partnerships to tackle HIV/AIDS, TB, malaria, and health systems. New GFHR strapline introduced: "Helping correct the 10/90 gap". October GFHR completes "Strategic Orientations 2003­05" -- a document revisiting the global strategic orientations of the Forum after five years and defines them for 2003­2005. The central objective of the Global Forum is reaffirmed as helping to correct the 10/90 gap in health research and focussing research efforts on the health problems of the poor. GFHR proposes 4 domains for health research: (a) on diseases and conditions; (b) on proximate determinants and risk factors; (c) on priority setting methodologies; and (d) on policies and cross-cutting issues affecting health and health research. 99 Annex E Year / month Events Within the Global Forum Events Outside the Global Forum November AFM 6 held in Arusha, Tanzania 2003 Former World Bank HNP Director and then current GFATM Executive Director Prof. Richard Feachem (UK) succeeds Prof. Adetokunbo Lucas (Nigeria) as Foundation Council Chair AFM 7 held in Geneva November NEPAD Ministerial Conference on Science and Technology Declaration by Ministers responsible for science and technology reaffirms African governments' commitment to increase public spending on R&D to at least 1% of GDP within 5 years. 2004 January Foundation Council appoints Professor Stephen A. Matlin (UK) as second GFHR Executive Director, to succeed Dr. Louis Currat (Switzerland) GFHR creates new publication "Global Forum Update on Research for Health"; publishes "Combined Approach Matrix" as priority-setting tool for health research. Former IPPF Assistant Secretary-General Pramilla Senanayake (Sri Lanka) becomes Foundation Council Chair Mexico Summit on Health Research organized by WHO and the Government of Mexico; GFHR and COHRED participate in the Program Committee. Forum 8 is organized as a separate event but there is an interface between the two meetings. AFM 8 held in parallel with the Mexico Summit; GFHR launches RealHealthNews, at Summit, as an independent print and web magazine about the connections between health research and policy- making worldwide, in action for the world's poorest. November FC approves Memorandum of Agreement with COHRED. 2005 January The G7 Finance Ministers launch a consultation process on technical aspects of a pilot advance market commitment to accelerate development and availability of priority new vaccines against diseases that kill millions of people in developing countries Consultations subsequently accelerate toward a final decision. Annex E 100 Year / month Events Within the Global Forum Events Outside the Global Forum WHO World Health Assembly urges Member States to consider implementing the recommendations of the 1990 Commission on Health Research for Development calling for developing countries to devote at least 2% of national health expenditures to research and research capacity and at least 5% of donor health support to research and related capacity building. March Memorandum of Agreement with COHRED signed by Board Chairs and Executive Directors of GFHR and COHRED. Initiatives and networks supported by the GFHR include Alliance for Health Policy and Systems Research (AHPSR), Child Health and Nutrition Research Initiative (CHNRI), Global Network for Research on Mental and Neurological Health (GNRMNH), Initiative for Cardiovascular Health in Developing Countries (IC Health), Sexual Violence Research Initiative (SVRI), and Road Traffic Injuries Research Network (RTIRN) (see Annex Table 6) Brazil organizes national seminar on health research, showing how one innovative developing country is meeting challenges of expanding health R&D March Commission for Africa sponsored by UK Prime Minister Tony Blair releases report calling upon donors to develop incentives for health R&D that meets Africa's needs, set up advance purchase agreements for medicines, and increase funding for African-led research. March Meeting in Rio de Janeiro, the GFHR Foundation Council decides to launch the second external evaluation of the Forum. It establishes an Evaluation Sub-Committee including the Bank's member of the Council. Evaluation is primarily funded by accumulated interest on the /a World Bank funds, plus $25,000 from WB core grant. Secretariat conducts induction seminar for FC members. FC endorses work on BIAS FREE framework for health research and poverty and health research. June GFHR and UK MRC co-host the Heads of International Biomedical Research Organizations (HIROs) meeting in London on "Global Health Research-Africa Priorities: Research funders' contribution to Global Health Research." HIROs decide to (a) develop a collaboration to strengthen health research capacity (RCS) in Africa and (b) develop systems for better tracking of their own allocations for LDC needs. FC members subsequently express concern that MRCs may risk taking a "top down" approach to RCS. 101 Annex E Year / month Events Within the Global Forum Events Outside the Global Forum WHO establishes a Commission on Social Determinants of Health (CSDH), to marshal the evidence on what can be done to promote health equity, worldwide, and foster a movement to promote it July GFHR ED participates in consultation at Ellison Institute of World Health at Harvard University on National Health Accounts, including possibility that health research sub-accounts may be established. A /b newspaper account suggests that the Ellison Institute subsequently became the more modestly named Health Metrics Institute at the University of Washington. Report of UN Millennium Project commissioned by UN Secretary- General calls for massive increase in scientific research for development. By 2015 the Report envisages at least $7 billion in public funding will be required, of which $4 billion would be directed at public health. September AFM 9 held in Mumbai, India, under the theme "Poverty, Equity and Health Research"; two of three Evaluation Team members attend. September FC decides to establish a Finance Committee to assist Secretariat with strategic advice about financial management, fund-raising, strategic budgeting, risk management and financial best practices; FC discusses draft conflict of interest statement for FC and STRATEC members to sign; revisions requested but statement signed; future STRATEC and FC meetings begin with signature of conflict of interest statement. Canada decides to allocate 5% of all new health research funds to /c research relevant to LDCs. GFHR ccontributes to the creation of the Argentine Health Research Forum as a direct interface among researchers, funders and health policy makers in the country 2006 Gates Foundation completes formal launch of a five year $20 million grant for the International Association of National Public Health Institutes (IANPHI) -- a global initiative dedicated to creating, strengthening and linking national public health institutes to improve the scientific basis for public health policies and programs. Young voices essay competition launched in partnership with The Lancet. September COHRED moves its offices to be co-located in the same building as GFHR. Annex E 102 Year / month Events Within the Global Forum Events Outside the Global Forum GFHR flagship publication Monitoring Financial Flows for Health Research 2006: The Changing Landscape of Health Research for Development gives particular attention to the changing scene AFM 10 held in Cairo, Egypt. FC discusses draft external evaluation and initiates elaboration of updated GFHR strategy. FC Finance Committee holds initial meeting. December GFHR Executive Director briefs World Bank HNP Sector Board on shifting patterns of health research around the world. Positive exchange of views ensues with recognition by the Sector Board of the importance of the Global Forum as the only organization worldwide that engages with all the top players in health research to pursue changes in priorities and financial flows towards those most urgent for people in LMICs. The primary role of the Bank in the Global Forum was recognized and strongly supported. 2007 February Final report of the second external evaluation of the Global Forum completed. April Royal Tropical Institute (Netherlands) issues final consultancy report on strategic options for increasing collaboration between GFHR and COHRED GFHR flagship publication Monitoring Financial Flows for Health Research 2007: Behind the Global Numbers focuses on funding of health research in Argentina, Mexico, China, United States, and donor funding overall on 20 communicable diseases Updated World Bank HNP strategy finds that the Bank's partnership portfolio has grown dramatically and that sharper strategic direction is required. The Bank proposes to concentrate its HNP advocacy on sound intersectoral and health systems strengthening policies. Bank global partnership (DGF) grants are to be gradually reoriented to areas of Bank comparative advantage, identified as health system strengthening; intersectoral approach; regulatory framework for public- private cooperation; large-scale implementation of projects and programs; convening capacity and global nature; and country focus and presence. By the end of FY10, 50% of HNP DGF funding is to be allocated in partnerships related to Bank HNP comparative advantages 103 Annex E Year / month Events Within the Global Forum Events Outside the Global Forum Launch of the International Health Partnership (IHP, or IHP+, as it has become known with additional partners) bringing together developing countries, international agencies and donors in support of development cooperation for health according to the principles of the Paris Declaration. July Informal inaugural meeting of the Health-8 (or H8, as it has become known) -- WHO, World Bank, GAVI, GFATM, UNICEF, UNFPA, Bill and Melinda Gates Foundation, UNAIDS -- aimed at strengthening cooperation on global health; WHO and World Bank provide secretariat. This self-appointed group provides by its very existence a vivid demonstration of the complexity of the global health landscape. July Second external evaluation of the Global Forum issued with foreword by the Chair of the Foundation Council. AFM 11 held in Beijing, China, under the theme "Equitable Access -- Research Challenges for Health in Developing Countries"; FC decides AFM 3­4 day meetings to continue through 2009, with one day focussed on a "high level" segment. FC also decides to explore possibility of a private "Davos-type" event along the lines of the World Economic Forum. Former pharmaceutical industry researcher and UK government scientific advisor Gill Samuels (UK) succeeds Pramilla Senanayake (Sri Lanka) as 4th Chair of GFHR Foundation Council. 2008 February President Bush announces a five-year, $350 million initiative for combating neglected tropical diseases in high priority countries across Africa and Asia. WHO WHA adopts global strategy and plan of action on public health, innovation, and intellectual property aiming to promote new thinking and provide a medium term framework for securing enhanced and sustainable needs-driven essential health research relevant to diseases which disproportionately affect developing countries. The Global Forum engages with the Secretariat on how to contribute its expertise, especially with regard to the elements of the strategy dealing with resources and incentives for pharmaceuticals needed for diseases of most importance to LMICs. Annex E 104 Year / month Events Within the Global Forum Events Outside the Global Forum Adoption of new Global Forum Strategy, 2008­14. "Helping correct the 10/90 gap" is no longer the Global Forum strapline. Instead, the Global Forum documents have a new strapline "because health equity is a priority" implying but not stating that health research benefits everyone. New logo introduced. STRATEC begins to discuss the possibility of creating a Health Research 12 (HR12) analogous to the H8, to bring together those with the most influence on health research policies and funding worldwide. April SIDA organizes meeting on donor alignment and harmonization in cooperation on research for health, oriented to application of OECD Paris Declaration principles to health research. TDR has major role in follow-up. Gates Foundation presentation underscores need to increase amount and effectiveness of R&D and health aid from donors. World Bank financial management assessment report on the Global Forum finds the current financial management capacity of the GFHR adequate and its overall FM risk to be low. An action plan agreed by the GFHR provides for establishment of a GFHR Audit Committee with adequate independent members by the end of March 2009. WHO advisory panel discusses draft WHO research strategy, expected to be approved by WHA in 2009. Draft emphasizes national research systems, strategies, and capacity in LDCs, development of a research culture in WHO, and involvement with other research institutions and leaders. GFHR and World Bank participate in panel. Strategy mentions prior consultations requesting WHO to make better use of its convening power to draw attention to, build consensus around and catalyze actions to pursue new directions in research. Like GFHR, WHO proposes "research for health" as central concept. August WHO publishes report of its Commission on Social Determinants of Health, Closing the Gap in a Generation: Health Equity through Action on the Social Determinants of Health. Commission underscores importance of knowledge and research and their dissemination. It foresees a global health observatory or clearing house for evidence on interventions for health equity. It recommends that research funding bodies create a dedicated budget for generating and sharing research on social determinants of health and equity. TDR launches new African Network for Drugs and Diagnostics Innovation; the Network addresses funding and advocacy. 105 Annex E Year / month Events Within the Global Forum Events Outside the Global Forum November GFHR flagship publication Monitoring Financial Flows for Health Research 2008 sets forth an initial "Report Card" on financing R&D for health. Targets monitored include resources for R&D for health in relation to national research budgets; national health budgets (follow- up on 15% OAU Abuja target of 2001); resources for ODA in relation to GNP (0.7% UN target set in the 1960s); 5% share of ODA for health and 2% of health ODA for health research (1990 Commission target). GFHR publishes Health Partnerships Review on public-private partnerships for health, with contributed chapters by PPP leaders, including some whose establishment was brokered by GFHR November Bamako Global Ministerial Forum (GMF) on Research for Health held by Global Forum, WHO, World Bank, UNESCO, COHRED, and Government of Mali. GMF calls upon funders and development agencies to better align, coordinate and harmonize the global health research architecture and its governance. AFM 12 held as an integral part of Bamako Global Ministerial Forum on Research for Health /a Source: Final Report of the GFHR on activities financed by the FY06 DGF Grant, extended by Amendment No. 1 to Dec. 31, 2007. /b Seattle Times, Februrary 15, 2007. /c According to Neufeld 2001, this was already the case in CIDA practice in 2000. Annex F 106 Annex F. External Evaluation: Major Findings and Recommendations of the Global Forum's Second External Evaluation and the Global Forum Response Responses by the Global Forum for Health Research Recommendations by the Evaluation Team (ET) Secretariat and Foundation Council (FC) 10.0. Impact of the Global Forum and future directions The ET recommends that the FC develop a five year strategic plan of action, based in well thought A new Strategy 2008­14 has been developed and is now priorities and harnessing the comparative advantages of the Global Forum, namely a light being implemented. This focuses on three main strategic administrative structure, high visibility vis-à-vis and respect from funding agencies and its leadership priority areas, building on the recognized strengths and status with at least a portion of health researchers and health research policy-makers. While the comparative advantages of the Global Forum; has the lack of effective insertion at country level is probably the result of this not being an explicit promotion of research for the health of the poor as its component of the Global Forum's mission, the impact would certainly improve by increasing mission; incorporates equity and research on a broad contacts and actions at the country level for specific projects, where the local players are in the range of determinants of health as cross-cutting elements; driver's seat and the Global Forum plays a supportive and catalytic role. and seeks collaboration with other global and regional In order to exploit its comparative advantages to the maximum, the ET recommends that the Global bodies to ensure country-level impact. Forum continue developing its current focus on cross-cutting issues of poverty and health, gender and equity, but that it also explore other controversial or novel priority areas, such as the following: 10.1. (a) Research capacity strengthening (RCS), with a strategy to identify key institutions and The FC did not agree with the ET's view that the Global actors in selected countries, that are associated with research for health (research institutes, Forum should become directly active at country level, since universities, research councils) and to conduct local meetings between researchers and health this is the sphere of activity of COHRED (which the ET research policy-makers to outline specific RCS activities that address the concrete needs of that inappropriately dismissed as a significant actor). The country. The Global Forum would thus position itself as a more relevant broker for the funding of the Global Forum strategy involves: specified RCS activities determined by the researchers and policy makers in those countries. 1. promoting the strengthening of national health research systems (which incorporates "RCS" but treats it in a much more systemic way) and influencing this at country level through collaborations (e.g., with COHRED and TDR -- results of recent work are currently in the press). 2. fostering S-S collaboration between key research institutions including universities, MRCs, NIH, etc. 10.2. Identification of research gaps and development of portfolios on "orphan" controversial issues, Taking account of all these features, the Strategy 2008­14: such as: 1. makes provision for flexibly taking up opportunities to (a) intellectual property and the role of the pharmaceutical industry in the exploitation of vulnerable highlight hot topics and current controversies; populations in LMICs for clinical trials of dubious ethics; 2. incorporates "innovation for health and health equity" (b) barriers for the implementation of well-known research results into actions for better health; as one of its three main priority areas (c) implementation of a human rights approach to health research and its applications for better 3. is built around the concept of "research for health" health; taking a rights-based and cross-sectoral approach to (d) barriers to the access of products resulting from research by the poor. research on all the determinants of health (including economic, environmental, political and social determinants). 107 Annex F Responses by the Global Forum for Health Research Recommendations by the Evaluation Team (ET) Secretariat and Foundation Council (FC) 10.3. Research on how to transform the health systems in incubators for innovation in health The priority area of innovation in the new Strategy technologies, products and processes that improve efficiency and equitable access to discoveries. specifically incorporates the issue of how to create conditions to stimulate both technological and social innovation to improve health and health equity. 10.4. Research on health financing. Much has been said about the fact that the actions prescribed The Global Forum has commissioned and published a by the World Bank in the 90's (emphasis in privatization and cost efficiency) have not resulted in study on experiences in health care financing in low- and better health and have worsened inequities. The forces behind specific financing policies of health middle-income countries (LMICs) and the new Strategy will systems are a very legitimate and relevant subject of research, particularly in current circumstances continue to pursue this area. in which equity is of wide concern and new sustainable ways are sought to finance health systems. Finally, the ET firmly believes that the Global Forum could considerably increase its impact if priority The FC has determined that the Global Forum will have its is given to working at country level with local health researchers and health research policy makers main focus on engaging with decision makers globally and in public and private institutions, incorporating their own views and approaches. In fostering will extend its country-level impact through collaborations interactions among local stakeholders, the Global Forum should exercise its brokerage capacity to with other organizations including COHRED. find support for their work with funds and technical advice and making sure that actors in LMICs are the real protagonists of these activities. The announced new agreement to work jointly with COHRED will definitely strengthen the perspectives of success. 11.0. Recommendations In addition to the recommendations formulated in the precedent section, the ET recommends that the Global Forum undertake the following actions: 11.1. General In the new Strategy: 11.1.1. It is recommended that the FC revisit the current appropriateness of the "10/90 gap 1. "Helping correct the 10/90 gap" is no longer the Global reduction" as a relevant activity and redefine its goals in a way that are measurable and attainable. It Forum strapline. is recommended also that the Global Forum review its strategies with greater emphasis on working 2. Measurable indicators and milestones of progress are at the country level in conjunction with COHRED in order to have more impact and ensure utilization under development of its research tools appropriately. 3. The Global Forum seeks to enhance its country-level impact through collaborations with a range of partners, including COHRED. 11.1.2. There is a need to establish clear rationales for research portfolios and advocacy program The new Global Forum Strategy emphasizes the principles development in which the research needs of LMICs are addressed as defined by the countries and of stakeholder inclusion and self-determination of country not based on perceptions or prescriptions from outside. This locally-based bottom-up approach is research priorities, within the framework of strengthening essential for all the Global Forum program development. national health research systems. Annex F 108 Responses by the Global Forum for Health Research Recommendations by the Evaluation Team (ET) Secretariat and Foundation Council (FC) 11.1.3. LMICs often face conflicting and competing demands, in general, and specifically within the The Global Forum is working, directly and through partners: health care sector. There is a need to provide technical support to assist these countries in the 1. To promote evidence-based, inclusive priority-setting definition of their health research policy needs and in decision-making processes to implement processes. The Combined Approach Matrix (CAM) is one them. A first step would be to help countries identify the availability of data/evidence, to identify and tool to assist this, now being published in revised edition use existing research capacity including research infrastructure, and to promote research capacity with poverty and equity dimensions added; country strengthening towards innovation in health. These are prerequisites to define and develop/adapt experiences using CAM in Latin America are also being research methodologies and implement them in a useful and a successful manner. published. 2. To encourage development of country capacities to track and utilize information on research resource flows in priority-setting; 3. To identify and promote the conditions for innovation in health in LMICs. 11.2. Annual Forum Meetings (AFMs) The FC carefully reviewed the purpose and value of AFMs and concluded that, at least for the period up to 2011: The ET acknowledges the important contribution made by the AFMs to date and the fact that these 1. AFMs will continue to be run annually, moving to a meetings are without doubt the visible face of the organization. However, taking into consideration different region and with a different theme each year to the extensive financial and human resources needed to organize such large meetings, the ET maximize access, ensure freshness and relevance and considers that it may not be cost effective to host these meetings on an annual basis. The Global sustain the impact and visibility of the Global Forum; Forum may want to consider organizing AFMs once every two years to achieve a bigger and more focused meeting that will have greater impact in moving the health research agenda forward. In designing these AFMs, there is a need to review the organizational structure and content. The ET 2. Within the overall theme, AFMs have a set of clear and welcomes the proposed changes to the organization of the AFMs being proposed by the Secretariat. specific streams that examine key issues in depth; are Whilst it is stated that "there will be an overall theme for each meeting," the ET considers that there focused on discussions (panel discussions and round- is a need to carefully design a "core" for each Forum meeting to address specific questions and tables) and endeavor to move the debate away from identify outcomes. The non-core topics can be organized into parallel and poster sessions. In the problem definition and towards solutions. organization of all oral sessions, it is essential that adequate time be allowed for discussions. Absolute pre-eminence should be given to formats privileging interactions (parallel panel discussions and workshops) over large plenary sessions. To assist in this process, the ET recommends that the FC establish an international Scientific 3. Forum 2009 has an international group of advisors Advisory Board for each meeting to help STRATEC in determining the "core topics" for presentation assisting the Secretariat in developing all aspects of the and for selecting abstracts. The ET welcomes the new practice of soliciting abstracts from the wider program, including the selection of abstracts for inclusion. health research community and strongly suggests that the selection of these abstracts for oral or poster presentations be determined by the proposed Scientific Advisory Board. The ET supports the continuing organization of the "marketplace" as part of the AFMs. The 4. The marketplace continues to be an important and participation of private sector health research organizations together with the public sector players is valued element of the AFM. There has never been any welcome. However, it needs to be monitored to ensure that this does not lead to the participation by private sector health research organizations in the marketplace, nor any intention to 109 Annex F Responses by the Global Forum for Health Research Recommendations by the Evaluation Team (ET) Secretariat and Foundation Council (FC) "commercialization" of the marketplace, which would dilute its value. "commercialize" it. The Poster sessions are a significant platform for interaction between health researchers, both 5. The poster sessions have continued to be a feature of amongst themselves as well as with health policy makers. It also is a platform for networking. The the AFM and poster presenters have continued to be ET recommends that the amount of space and time be expanded for such poster sessions. Further, allocated prominent space for display. The institution of a it must be organized in such a manner to give greater visibility and certainly separated from the prize for the best under-40 poster has helped promote the Marketplace location. posters. The ET recommends that the AFMs should not be organized in luxurious settings, but in settings 6. AFMs are organized mainly in LMICs; in modest that are affordable to LMIC participants. venues that nevertheless meet the requirements for space, services and hygiene; very cheap accommodation options are always provided for those attending on low budgets. The ET recommends that the Global Forum develop a system that will ensure that the 7. This recommendation has been considered of limited recommendations from the AFMs are followed up and translated into action and that such follow-up relevance, since AFMs have been designed as a platform actions taken by the Secretariat be reported at the following Forum meeting. for a wide range of stakeholders to bring their views together and to make recommendations that are often not aimed at the Global Forum but at governments, agencies, researchers, etc, for which the Global Forum cannot take primary responsibility. However, within the AFM there are usually some sessions that specifically focus on Global Forum core business (e.g., resource tracking, priority setting, defining research agendas) and the relevant staff members ensure that the outcomes of these discussions are incorporated into their ongoing work. In line with the Global Forum strategy to build health research capacity in LMICs and to encourage 8. The Global Forum has collaborated with COHRED to greater involvement of young health researchers, the ET strongly encourages the FC to give due organize regional meetings in Africa (Nairobi, 2007 -- consideration to the organization of regional AFMs. Such regional meetings can be organized in Human Resources for Health Research) and Latin America alternate years and should focus on health research issues that confront the concerned region. This (Rio, 2008 -- Research and Innovation for Health), bringing will also give greater opportunity for health researchers to present their research data and interact together researchers and policy-makers. The Global Forum amongst them and regional health policy makers. is encouraging participation by young professionals through the annual essay competition it has developed in partnership with The Lancet -- Young Voices in Research for Health. Now in its 4th year, it is proving very successful, has a regional basis and the regional winners are brought to the AFM and included in the program. 11.3. Resource Flows The Global Forum has a twin-track approach, in which: Work on resource flows tracking should be continued with the new emphasis already started of a. Every two years, the Global Forum collects, analyses paying closer attention to additional levels of analysis. These should include: and publishes data on the global level of aggregate Annex F 110 Responses by the Global Forum for Health Research Recommendations by the Evaluation Team (ET) Secretariat and Foundation Council (FC) Analysis of resource flows should be conducted at country level. While it is acknowledged that spending in health R&D. This analysis also looks at sub- Global Forum has already started to disaggregate levels of analysis, the difficulties experienced in sectors such as the private for-profit and not-for-profit project 13/2004 suggest that better forms of interaction that give more room for actors in LMICs to actors. devise their own objectives and strategies may improve effectiveness and impact. . Analysis of country-level resource flows should be followed up with lobbying to persuade policy b. In the alternate years, the Global Forum collects and makers to increase allocation to prioritized health research, and with studies assessing how the publishes individual studies behind the global numbers. knowledge generated is translated into action, including outputs, achievements, hurdles and These include country studies (e.g., resource flows studies setbacks. have been published from Brazil, China, Mexico, USA). The different ultimate goals for, and types of, investments in health research should be analyzed Studies are currently being supported in several Latin and exposed according to types of funding sources in both HICs and LMICs (public, private not-for- American countries, adapting Global Forum methodology profit, private-for-profit). to suit different country circumstances, the final element of which will involve engagement with policy-makers on the results. The Global Forum should promote the study of links between investments in health research and The Global Forum has now instituted a Report Card which measurable improvements in health, particularly among the poor. A human rights approach to the analyses performance of countries and organizations value of health research and application of knowledge into action should be incorporated into the against targets that have been set internationally in the analysis of resource flows, looking at equity and access to the benefits of research. three interlocking domains of development, health and In order to adequately evaluate relevance and impact of this strategy, it is essential to develop research. indicators of outcomes of this work. 111 Annex F Responses by the Global Forum for Health Research Recommendations by the Evaluation Team (ET) Secretariat and Foundation Council (FC) 11.4. Priority Setting Methods/Tool (CAM) 1. The Global Forum is working, directly and through partners: The work on priority setting is of great relevance for the allocation of health research funds and the a. To promote evidence-based, inclusive priority-setting Global Forum has duly established its legitimacy and reputation in this field. This work should processes. The CAM is one tool to assist this: now being continue, incorporating the following suggestions: published in revised edition with poverty and equity dimensions added; country experiences using CAM in Latin America are also being published. Foster more interaction between health research policy makers and researchers at the country level b. To encourage development of country capacities to in LMICs, facilitating a larger role for them in defining their own criteria for the relevance of particular track and utilize information on research resource flows in dimensions in establishing priorities. priority-setting; The Global Forum should recognize the limitations of DALYs as a measure of burden of disease 2. The Global Forum has long recognized the limitations and the search for alternative indicators should become a major initiative. The ET supports the of DALYs and has highlighted this, for example, in its 2006 current interactions with the disability movement to reach a more equitable approach to health publication on Burden of Disease measurements as well as research priority setting. in its work on the BIAS FREE Framework. More attention should be given to identifying and exposing political, social, cultural, economic and 3. The BIAS FREE Framework has been developed and professional obstacles to the implementation of established health research priorities. strongly promoted as part of the Global Forum's efforts to expose and reduce barriers in the conduct and use of research. Current emphases on gender perspectives in setting priorities for health research are commended 4. The new Strategy 2008­14 explicitly starts with "health and should be pursued. A wider comprehensive human rights framework to health research priority as a right" as its first core principle and health equity as its setting would position the Global Forum even better at the forefront of current efforts to make health over-arching goal. research more relevant to inform policies, in order to make the right to health a reality, particularly among the poor. 11.5. Poverty and Health Portfolio The study of the relationship of poverty and health is a cross cutting issue at the heart of the mission Work on poverty and equity continues to be an important of the Global Forum, and should be pursued vigorously by identifying research priorities on this topic element of the Strategy 2008­14. and by supporting health researchers in LMICs to produce actionable knowledge on the subject that can guide policy makers in concrete actions to ameliorate poverty and its ill effects on health. Annex F 112 Responses by the Global Forum for Health Research Recommendations by the Evaluation Team (ET) Secretariat and Foundation Council (FC) 11.6. Equity Portfolio 1. The Global Forum continues to disseminate and 11.6.1. The ET finds the BIAS FREE Framework project very promising and encourages the Global promote the uptake of the BIAS FREE Framework. Short Forum to further develop the tool and make it applicable in different regional and country contexts. summaries have now been translated into many languages and a detailed case study of experiences in Costa Rica is being edited for publication. 11.6.2. The ET considers the work on social determinants of health as highly important and 2. The Strategy 2008­14 highlights social determinants encourages the Global Forum to highlight social determinants of health as a priority issue for of health as a priority issue for research in LMICs. research in low-income countries and to contribute to RCS within this field in different settings. 11.6.3. The Global Forum's work in this area should complement the ongoing work of the 3. The Global Forum has engaged with the CSDH and is Commission on Social Determinants of Health (CSDH), particularly the work of its Knowledge currently studying the Commission's final report to identify Networks, and play an important role in taking forward its future report, building on its work to research issues and agendas that require further attention. identify knowledge gaps and research needs related to the root causes of disease and ill-health in different settings. 11.6.4. Acknowledging the criticism of DALYs as a measure of burden of disease (BoD) for its 4. The Global Forum has promoted attention to the issue inherent limitations from an equity perspective, the ET encourages the Global Forum to foster the of DALYs through its publications and discussions in development of alternative new indicators to measure health and its determinants, and to further Forums. examine the limitations of DALYs as a tool for priority setting, and stimulate research on more balanced and comprehensive alternative indicators of BoD. 11.6.5. The ET recommends a close collaboration with the WHO Equity Team, in order to jointly 5. The Global Forum has engaged in dialogue with the take forward the recommendations of the WHO Taskforce on Research Priorities for Equity and WHO Equity Team, including in relation to the work of the Health. CSDH and this has helped inform the development of the equity focus in the Strategy 2008­14. The Global Forum will continue to seek opportunities for collaboration with the team as it implements the Strategy's priority programs. 11.7. Research Initiatives The Global Forum should in the future give more careful thought to launch new "research initiatives", The Global Forum has not created any new initiatives or establishing well-grounded criteria to select such initiatives, as well as outlining objectives and networks since 2002 and, following a review conducted in expected outcomes with precision. 2007, has no plans to create any new ones for the time being. 113 Annex F Responses by the Global Forum for Health Research Recommendations by the Evaluation Team (ET) Secretariat and Foundation Council (FC) 11.8. Research Capacity Building (RCS) 11.8.1. The Global Forum should emphasize and expand its role in fostering RCS at different levels These points have already been answered in 10.1 and in selected LMICs in partnership with TDR, COHRED and other relevant players. The approach 10.2. should be hands-on in selected countries, harnessing already existing capacities towards a comprehensive policy of training and capacity building in research for health according to the priorities set by the countries. 11.8.2. The Global Forum should catalyze building links between researchers and policy-makers in selected countries towards the development of national health research systems (NHRS) that provide valuable evidence and knowledge to guide policies for the improvement of health with equity. These activities should be implemented in partnership with TDR, COHRED and other international and multilateral players. 11.8.3. The Global Forum should utilize its perceived role as broker and lobbyist for health research funds, to help channel international resources to finance RCS activities in selected countries. These countries would be selected according to criteria agreed with the partners. 11.9. Gender The Global Forum should continue to give high priority to gender issues in all its activities. 1. The Global Forum continues to give high priority to gender and equity issues in all its activities. The Global Forum should ensure that recommendations from participants at different workshops 2. See answer (7) under (11.2). GFHR has conducted a and AFMs concerning the integration of gender perspectives in the Global Forum's analytical work range of workshops (in and out of AFM settings) and tools are followed up as well as include the gender component in RCS activities. specifically designed to expose gender issues and has incorporated gender perspectives in every area of its work. The ET finds the BIAS FREE project very promising and encourages the Global Forum to further 3. See answer (1) under (11.6). develop the tool and to continue the work on making the tool applicable in different regional and country contexts. The Global Forum should put more efforts into pushing its partners to take important sex differences 4. The Global Forum always raises gender issues when and gender dimensions in health research into consideration in their work. these are absent -- e.g., most recently highlighting in a WHO workshop on setting the research agenda for climate change and health that the approach under development was gender blind. The Global Forum should undertake a review of the magnitude of gender imbalances in research 5. This is a massive undertaking and has not been ethics and other research review committees, research funding agencies and advisory bodies, and selected as a priority in the current Strategy of the Global of the different treatment of women scientists. Based on the results of the review, the Global Forum Forum, as it falls within the mandates of many other should develop recommendations leading to the correction of these imbalances. organizations concerned with women in science. The Global Forum should advocate for the inclusion of gender experts in research ethics 6. The Global Forum advocates for gender perspectives committees and other research review committees and editorial boards. to be incorporated in ALL areas of the health research system. Annex F 114 Responses by the Global Forum for Health Research Recommendations by the Evaluation Team (ET) Secretariat and Foundation Council (FC) 11.10. Operating Environment and Management Practices 1. The FC continually reviews and updates practices and The ET recommends that the FC review the current procedures regarding management practices procedures in its own work and in its engagement with the and decision-making processes, to facilitate greater discussion, transparency and involvement by Secretariat. Several major changes have been adopted by FC members in the strategic functioning of the Global Forum. the new Chairs of FC and STRATEC to improve and streamline decision making and enhance the interactions with the Secretariat. Periodic routine "closed-door sessions" could be designated in the agenda of the FC, where 2. In camera sessions have been introduced as a members can raise issues and concerns in the absence of Secretariat staff. standing agenda item by both FC and STRATEC. Before new staff positions are established, a thorough Human Resources assessment should be 3. Several staff replacements have been/are being made carried out against a clearly defined strategic work plan, using international work productivity norms in connection with the new Strategy, ensuring that newly in benchmarking these needs. If, in future, the Global Forum conducts more intensive and in-depth recruited staff have the skills required for its work at the country level, relevant staff experience will be an important criterion for recruitment of implementation. For senior/professional appointments, an program personnel. An external assessor system could be introduced in order to ensure that the external assessor is used as part of the interview team. best available candidates are selected. 11.11. The Global Forum and WHO 1. The Global Forum collaborates closely with WHO The Global Forum should continue to develop strong collaborations with the WHO, catalyzing a across many themes, topics and departments and has more proactive stance in health research to meet the health research needs of LMICs. been closely involved in the consultations leading to the development of the first ever WHO Research Strategy. The Global Forum should engage the WHO particularly on cross cutting research areas such as 2. The Global Forum has engaged with WHO on a poverty, equity, health systems and policies, health financing, RCS, and in the development of number of these themes and will continue to seek every human resources for health research. opportunity to collaborate where appropriate and to influence WHO's research agenda. 11.12. The Global Forum and COHRED An external consultancy was engaged to identify COHRED and the Global Forum should increasingly work jointly in coordinated fashion combining opportunities for collaboration between the Global Forum global and local approaches and actions to increase the impact of research that addresses the and COHRED and especially for "closing the loop" at the needs of LMICs. interface between the global and country-based perspectives that the two organizations represent. Representation on each other's governing bodies and meetings of the senior management teams are among the institutional mechanisms that have been adopted to ensure continuing dialogue. Collaborative activities include working together on global and regional meetings and joint publications. 115 Annex F Responses by the Global Forum for Health Research Recommendations by the Evaluation Team (ET) Secretariat and Foundation Council (FC) 11.13. New Areas of Research In developing the Strategy 2008­14, the FC considered a large menu of options and decided in which areas the The ET recommends that the Global Forum focus its attention on a number of current and Global Forum should concentrate most of its effort and controversial issues in health research, for example: attention in order to ensure a critical mass of resources and optimum impact. Within the new Strategy: Develop a portfolio on a human rights approach to health research. 1. Human rights is seen as a cross-cutting issue rather Study in more details the ways in which the different actors benefit (or not) from the results of health than having a separate portfolio; research. 2 . Is covered by the cross-cutting attention to equity; 3. Was highlighted in Forum 11 in Beijing (which had the Study the barriers to access to new preventive methods and therapeutics developed by research. overall theme of Equitable access: research challenges for health in developing counties) Study the inequities of the current system of intellectual property for drugs and diagnostics and propose changes based on the notion of health as a public good. 4. Has been given major international attention through the work of the Inter-Governmental Working Group on Public Health, Innovation and intellectual Property (IGWG), Research obstacles that have prevented the development of universally accessible and sustainable to which the Global Forum has contributed. The Global health systems that can apply the results of research for health in an equitable way. Forum is now collaborating with the IGWG Secretariat in Research the transformation of health systems in incubators for innovation in health technologies, WHO on follow-up in the Expert Group. products and processes that improve efficiency and equitable access to discoveries. 5, 6,7 are represented by two of the three main Strategic Research different approaches to health financing, particularly the study of the political and Priorities (Research to support the strengthening of equitable economic factors behind specific financing policies of health systems. health systems; Innovation for health and health equity). Annex G 116 Annex G. Members of the Global Forum Governing Bodies and their Global Forum Constituencies Members of the Global Forum Board of Directors (Foundation Council), as of November 2008 Dr. Gill M.R. Samuels Executive Director (retired), Science Policy and Scientific Affairs, Europe Sandwich Laboratories, Pfizer Global Research and Development Advisor to UK Government, OECD and WHO Chair, Foundation Council Dr. Zulfiqar Ahmed Bhutta Husein Lalji Dewraj Professor Chairman, Department of Pediatrics and Child Health Aga Khan University Hospital Pakistan Dr. Paulo Marchiori Buss President Oswaldo Cruz Foundation Brazil Professor Jie Chen Director Ministry of Health Key Lab of Health Technology Assessment Fudan University School of Public Health China Dr. Mushtaque Chowdhury Deputy Executive Director Bangladesh Rural Advancement Cooperative (BRAC) Essential Health Care Program Bangladesh Dr. Alejandra Lopez Gomez Coordinator Mujer y Salud en Uruguay Uruguay Professor Maria Guzman Head, Virology Department Director, PAHO/WHO Collaborating Center for the Study of Dengue and its Vector Pedro Kouri Tropical Medicine Institute Cuba 117 Annex G Dr. Richard Horton Editor-in-Chief The Lancet United Kingdom Professor Carel IJsselmuiden Director Council on Health Research for Development (COHRED) Ex officio member Dr. Rose Gana Fomban Leke Professor of Parasitology and Immunology Faculty of Medicine and Biomedical Sciences University of Yaoundé I Cameroon Prof. Adel A.F. Mahmoud Department of Molecular Biology Princeton University USA Dr. Adolfo Martinez-Palomo Coordinador General Consejo Consultivo de Ciencias Mexico Dr. Daniel Mäusezahl Senior Adviser for Health Swiss Agency for Development and Cooperation Switzerland Professor Anthony David Mbewu President Medical Research Council South Africa Dr. Ravi Narayan Community Health Adviser Bangalore India Dr. C.O. (Ok) Pannenborg Senior Adviser for Health World Bank Annex G 118 Dr. TK Sundari Ravindran Honorary Professor of Health Science Studies Achutha Menon Centre for Health Science Studies Sree Chitra Tirunal Institute for Medical Sciences and Technology Kerala India Dr. Robert George Ridley Director Joint UNICEF, UNDP, WHO and World Bank Special Program for Research and Training in Tropical Diseases Ex officio member Professor Nelson K. Sewankambo Principal College of Health Sciences Kampala Uganda Dr. Ragna Valen Director Faculty of Psychology Bergen Norway Professor Judith Whitworth Director John Curtin School of Medical Research The Australian National University Canberra City Australia Dr. Christina Zarowsky Program Manager Program and Partnership International Development Research Centre Ottawa Canada Members of the Global Forum Strategic and Technical Advisory Group (STRATEC), as of April 2008 Dr. C.O. (Ok) Pannenborg Senior Adviser for Health World Bank 119 Annex G STRATEC Interim Chair Professor Jie Chen Director Ministry of Health Key Lab of Health Technology Assessment Fudan University School of Public Health China Dr. Alejandra Lopez Gomez Coordinator Mujer y Salud en Uruguay Uruguay Dr. Ravi Narayan Community Health Adviser Bangalore India Professor Nelson K. Sewankambo Principal College of Health Sciences Kampala Uganda Dr. Ragna Valen Director Faculty of Psychology Bergen Norway Annex G 120 Annex Table 20. Global Forum Constituencies of Foundation Council and STRATEC Members, as of April 1, 2008 Research Multi- STRATEC Policy Bilateral Founda- Women/ Institutions Private Start of End of Member lateral NGOs Media Makers Agencies tions Gender Managers Sector Term Term members Agencies Researchers Current Past Gill Samuels, FC Chair 2004 2010 X Jie Chen X X 2004 2010 X Mushtaque Chowdhury X 2006 2009 Maria Guzman X 2004 2010 Richard Horton X 2003 2009 Carel IJsselmuiden X 2004 Ex Off Rose Leke X 2006 2009 Alejandra Lopez Gomez X X 2004 2010 X Adel Mahmoud X 2007 2010 Adolfo Martinez-Palomo X X 2006 ­ X Daniel Mäusezahl X 2004 ­ Ravi Narayan X 2004 2010 X Ok Pannenborg X 2004 ­ X Robert Ridley X 2006 Ex Off Nelson Sewankambo X 2004 2010 X Ragna Valen X X 2000 ­ X Judith Whitworth X X 2004 2010 Christina Zarowsky X X 2000 ­ X /a Zulfiqar Bhutta X 2007 2010 /a Anthony.Mbewu X 2007 2010 Total 2 3 4 0 1 3 10 1 1 /a New members normally begin their term of appointment following the conclusion of the November Foundation Council meeting, when the outgoing member retires Note: At 1 April 2008: F:M = 8:12; LMIC:HIC: = 11: 9. 121 Annex H Annex H. Global Forum Financial Information Annex Table 21. Global Forum for Health Research: Expenditures on Core Functions, by Category and Year, 2001­07 Annual Average 2001 2002 2003 2004 2005 2006 2007 2001­07 U.S. Dollars Research and Programs 824,550 935,356 1,311,734 1,311,734 1,545,185 1,429,861 1,384,193 1,248,945 Annual Forum Meeting 321,528 580,486 963,367 963,367 699,151 789,973 1,183,953 785,975 Information and Communication 302,010 472,846 852,638 852,638 751,459 769,678 1,130,052 733,046 Sub-total: Substantive Activity 1,448,088 1,988,688 3,127,739 3,127,739 2,995,795 2,989,512 3,698,198 2,767,966 Administrative Support Services 419,974 368,086 464,694 242,379 444,125 473,486 575,020 426,823 Governance and Executive Functions 153,320 116,776 242,694 464,694 414,490 392,751 286,624 321,114 Sub-total: Overhead 573,294 484,862 707,388 707,073 858,615 866,237 861,644 722,730 Grand Total 2,021,382 2,473,550 3,835,127 3,834,812 3,854,410 3,855,749 4,559,842 3,490,696 Percent of Total Research and Programs 41 38 34 34 41 37 30 36 Annual Forum Meeting 16 23 25 25 19 20 26 23 Information and Communication 15 19 22 22 20 20 25 21 Sub-total: Substantive Activity 72 80 82 82 78 78 81 79 Administrative Support Services 21 15 12 6 13 12 13 12 Governance and Executive Functions 8 5 6 12 7 10 6 9 Sub-total: Overhead 28 20 18 18 22 22 19 21 Grand Total 100 100 100 100 100 100 100 100 Overhead as Share (%) of Grand Total 28.4 19.6 0.2 18.4 22.3 22.5 18.9 20.7 Source: Global Forum documents. Annex H 122 Annex Table 22. Global Forum for Health Research: Donor Financial Contributions, by Donor and Year, 1999­2007 (US$ thousands) Donor 1999 2000 2001 2002 2003 2004 2005 2006 2007 Total Share Canada ­ IDRC 67.2 47.7 19.4 47.9 65.9 101.2 96.7 155.8 158.4 760.2 1.4% Denmark 0.0 0.0 0.0 131.5 315.5 339.3 0.0 0.0 0.0 786.7 1.4% Ireland ­ Irish Aid 0.0 0.0 0.0 0.0 0.0 0.0 146.0 322.7 414.8 883.4 1.6% Mexico ­ MOH 0.0 0.0 0.0 0.0 0.0 0.0 200.0 100.0 100.0 400.0 0.7% Netherlands 182.6 65.4 98.6 22.1 0.0 0.0 0.0 0.0 0.0 368.8 0.7% Norway ­ MFA 398.8 440.2 454.2 558.3 538.8 646.5 597.9 642.1 730.6 5,007.3 9.0% Rockefeller 500.0 500.0 150.0 275.0 200.0 150.0 275.0 250.0 125.0 2,425.0 4.3% Foundation Sweden ­ SIDA 547.9 519.3 423.7 462.6 556.6 591.0 575.0 426.7 0.0 4,102.9 7.3% Switzerland ­ SDC 369.8 121.2 541.8 501.3 551.0 652.2 697.7 730.1 650.1 4,815.3 8.6% Designated 0.0 113.0 10.0 42.6 304.8 157.5 174.0 154.4 245.0 1,201.1 2.2% Contributions Other income ­ 183.4 317.7 304.3 106.8 49.9 58.1 73.5 231.2 292.7 1,617.5 2.9% Core Other income ­ 0.0 0.0 1,448.1 1,722.9 1,655.9 2,120.2 1,058.8 179.8 151.7 8,337.4 14.9% Initiatives Subtotal excluding 2,249.6 2,124.5 3,450.3 3,871.0 4,238.3 4,816.0 3,894.6 3,192.7 2,868.3 30,705.1 54.9% World Bank World Bank DGF 1,400.0 1,400.0 2,420.0 4,125.0 3,020.0 3,225.0 3,225.0 3,725.0 2,730.0 25,270.0 45.2% Of which: core 550.0 550.0 600.0 700.0 850.0 850.0 850.0 1,250.0 1,295.0 7,495.0 13.4% support Of which: GFHR 0.0 0.0 670.0 2,150.0 1,900.0 2,325.0 1,982.0 1,250.0 518.0 10,795.0 19.3% initiatives Of which: projects 850.0 850.0 1,150.0 1,275.0 270.0 50.0 393.0 1,225.0 917.0 6,980.0 12.5% and networks Grand Total 3,649.6 3,524.5 5,870.3 7,996.0 7,258.3 8,041.0 7,119.6 6,917.7 5,598.4 55,975.1 100.0% Number of donors 7 7 7 8 7 7 8 8 7 10 World Bank 38.4% 39.7% 58.8% 51.6% 41.6% 40.1% 45.3% 53.8% 48.8% 45.1% % of Total Source: Global Forum. Note 1. This table excludes in-kind contributions; while it endeavors to encompass all cash contributions, some designated contributions and other resources for initiatives, such as from the Bill and Melinda Gates Foundation, may be excluded. Note 2: In some of the early years, World Bank support was not disaggregated; sub-grants are excluded from this table. 123 Annex H Annex Table 23. World Bank DGF Financial Support for Global Forum Initiatives and Projects, 2000­07 (US$ thousands) Initiative/Project 2000 2001 2002 2003 2004 2005 2006 2007 Total Alliance for Health Policy and Systems 400.0 400.0 400.0 400.0 400.0 500.0 0.0 0.0 2,500.0 Research Initiative for Cardiovascular Health (IC 0.1 70.0 200.0 400.0 500.0 811.0 650.0 0.0 2,631.1 Health, core activities) Child Health and Nutrition Research Initiative 0.1 760.6 389.0 500.0 500.0 500.0 500.0 150.0 3,299.7 Initiative on Public-Private Partnerships for 0.2 200.0 400.0 500.0 500.0 0.0 0.0 0.0 1,600.2 Health Sexual Violence Research Initiative 0.0 0.0 0.0 100.0 200.0 80.0 100.0 218.0 698.0 Global Network for Research in Mental and 0.1 0.0 0.0 0.0 0.0 91.0 0.0 0.0 91.1 Neurological Health Road Traffic Injury Research Network 0.0 0.0 0.0 110.0 225.0 0.0 0.0 0.0 335.0 Pharmaceutical R&D Policy Project 0.0 0.0 0.0 0.0 0.0 225.0 250.0 507.0 982.0 IC Health study of economic impact of NCDs 0.0 0.0 0.0 0.0 0.0 100.0 400.0 0.0 500.0 at country level Bamako Ministerial Forum 0.0 0.0 0.0 0.0 0.0 0.0 320.0 325.0 645.0 Demand for Health Research Project 0.0 0.0 0.0 0.0 0.0 0.0 180.0 0.0 180.0 Other 0.1 80.0 50.0 60.0 50.0 40.0 75.0 85.0 440.1 Total 400.5 1,510.6 1,439.0 2,070.0 2,375.0 2,347.0 2,475.0 1,285.0 13,902.1 Total GFHR Initiatives and Projects 5 4 4 6 6 7 7 4 Supported, excluding "other" Source: Global Forum for Health Research, yearly operations reports and audited financial statements; for 2000, an approximation based on data in DGF application form. Annex H 124 Annex Table 24. World Bank DGF Budget Allocations for the Global Forum for Health Research, FY1998­2009, in Overall DGF Context FY98 FY99 FY00 FY01 FY02 FY03 FY04 FY05 FY06 FY07 FY08 FY09 Total Global Forum DGF ($, million) 2.50 2.84 3.13 6.00 9.93 7.15 7.45 6.78 6.78 5.28 5.55 3.74 67.13 HNP DGF Total ($, million) 22.1 22.8 20.2 25.4 25.3 24.9 24.5 24.0 24.2 21.6 21.1 17.8 274.0 DGF Total ($, million) 122.1 125.0 155.9 176.9 176.9 157.0 178.2 174.2 171.9 171.8 175.8 167.8 1,953.5 HNP % of DGF 18 18 13 14 14 16 14 14 14 13 12 11 14 Global Forum % of HNP DGF 11 12 15 24 39 29 30 28 28 24 26 21 25 Global Forum % of DGF Total 2.0 2.3 2.0 3.4 5.6 4.6 4.2 3.9 3.9 3.1 3.2 2.2 3.4 Source: World Bank Document R2007-0092, FY08 Development Grant Facility Budget and Review of Global Programs, May 25, 2007; corresponding document for FY09. Note: This table includes sub-grants, since the GFHR DGF proposal is presented with sub-grants as part of the World Bank DGF budget presentation. 125 Annex I Annex I. Persons Consulted Organization and Individual Position Date Council on Health Research for Development (COHRED) Ms. Sylvia De Haan Head, Projects and Programs October 1, 20008 Global Forum for Health Research Dr. Louis Currat Former Executive Secretary, Global October 3, 2008 Forum Prof. Richard Feachem Former Chair, Foundation Council; November 4, currently Director, Institute for Global 2008 Health, University of San Francisco, (by phone) California (USA) Dr Andres de Francisco Former Deputy Executive Director, October 28 and Global Forum; currently Special November 4, Adviser, Strategy and Scientific Policy, 2008 Partnership for Maternal Newborn and (by phone) Child Health (PMNCH), WHO Dr. Abdul Ghaffar Former Research Advisor, Global December 12, Forum; currently Regional Advisor, 2008 Research Policy and Cooperation, (by phone) Regional Office for the Eastern Mediterranean Region (EMRO), WHO Ms. Monika Gehner Head, Publications and Information, September 29, Global Forum 2008 Mr. David Hayward Head, Finance and Administration, September 29, Global Forum 2008 Dr. Richard Horton Member, Foundation Council; Editor-in- October 14, 2009 Chief, The Lancet (by phone) Ms. Susan Jupp Head, External Relations, Global September 29, Forum 2008 Prof. Adetunkobo Lucas Former Chair, Foundation Council; January 22, 2009 Professor (retired), Harvard School of (by phone) Public Health Prof. Stephen A. Matlin Executive Director, Global Forum September 29, 2008 Prof. Anthony MBewu Member, Foundation Council; October 23, 2008 President, South African Medical (by phone) Research Council Ms. Alexandra Petersen Ndow Head, Meetings Organization, Global September 29, Forum 2008 Dr. Gillian M. R. Samuels Chair, Foundation Council (Board of October 1, 2008 Directors), Global Forum Annex I 126 Organization and Individual Position Date Dr. Pramilla Senanayake Former Chair, Global Forum December 17, Foundation Council; retired former 2008 (by phone) Assistant Director-General, International Planned Parenthood Federation (IPPF) Mr. John Warriner IT and Administration Manager, Global September 29, Forum 2008 Global Forum for Health Research Evaluation Team Prof. Visweswaran Navaratnam Leader, Global Forum Evaluation December 10, Team, and Biomedical Research 2008 (by phone) Professor, Malaysia; Chairperson, Advisory Board, Global Alliance for TB Drug Development Dr. Piroska Östlin Member, Evaluation Team and November 3, Associate Professor, Karolinska 2008 Institute (Sweden), Department of (by phone) Public Health Sciences, Division of International Health (IHCAR) Prof. Victor Penchaszadeh Member, Evaluation Team and December 15, Professor of Clinical Epidemiology, 2008 Mailman School of Public Health, (by phone) Columbia University; Advisor to WHO and PAHO Global Fund to Fight AIDS, TB and Malaria Mr. Serge Xueref Manager, Evaluation and Scientific October 3, 2008 Policy UNAIDS Ms. Catherine Hankins Chief Scientific Advisor to UNAIDS and October 1, 2008 Associate Director, Evidence, Monitoring and Policy Department World Bank Dr. Armin Fidler HNP Partnership Adviser, HNP Hub March 3, 2009 Mr. Ok Pannenborg Senior Advisor, Office of the Vice October 2 and President for Human Development; November 4, Member, Global Forum Foundation 2008, June 11, Council and interim Chair, STRATEC 2009 World Health Organization Dr. Catherine d'Arcangues Coordinator, Department of October 3, 2008 Reproductive Health and Research Ms. Sara Bennett Manager, Alliance for Health Policy and October 2, 2008 Systems Research Dr. Andrew Cassels Director, Department of Health Policy, October 2, 2008 Development and Services Dr. Timothy Evans Assistant Director-General, Evidence October 3, 2008 and Information for Policy 127 Annex I Organization and Individual Position Date Dr. Michael Mbizvo Coordinator, Department of October 2, 2008 Reproductive Health and Research Dr. Tikki Pang Director, Research Policy & December 9, Cooperation 2008 (by phone) Mr. Alex Ross Director, Partnerships September 20, 2008 Dr. Robert Ridley Director, Joint UNICEF, UNDP, World September 30, Bank Program for Research and 2008 Training in Tropical Diseases Dr. Shehkar Saxena Program Manager, Department of October 2, 2008 Mental Health and Substance Abuse Others Prof. Dean T. Jamison Research Professor, Institute for Health January 27, 2009 Metrics and Evaluation, University of (by phone) Washington; Staff Director, WDR93 Prof. Victor Neufeld Prof. Emeritus, McMaster University, October 16, 2008 Hamilton, Ontario, Canada (by phone) Annex J 128 Annex J. Response of the Program to IEG's Global Program Review The Global Forum for Health Research welcomes the GPR's detailed and largely balanced and constructive assessments and its many solid conclusions on the uniqueness, relevance, and successes of the Global Forum. On the whole, it is critical where warranted and provides valuable lessons and guidance and insights into features of the Forum's work. Facing the challenge of being grounded in evidence of outcomes while striving to recognize how the Forum is currently changing, the GPR is more successful than the 2nd External Evaluation (2EE). But, since the GPR does not always fully recognize the extent and nature of the most recent changes, the Global Forum offers the following comments. Vision, mission, relevance, effectiveness The Forum's mission is to focus attention and resources on research for health of poor populations in LMICs. In the 1990s there was no knowledge of the health and medical research funds being deployed for health in poor countries; few data on the health status of poor populations; and little insight in how to influence decision makers regarding research funding for health priorities in poor countries. While noting that measurement of research outcomes and attribution of R&D impacts are major challenges, the GPR acknowledges that the vision and mission of the Global Forum are relevant by being responsive to current global challenges and policies, to the growing availability of development assistance for health, to the increasing world-wide expenditures on health research, and to the particular needs of developing countries. The GPR notes: no other institution of comparable legitimacy is producing such [global public goods] as its core mission; and research resources, agendas and priorities are surrogates for the intensity and quality of efforts to bring research to bear on addressing disparities in health and health equity affecting poor populations. Resources: The Forum has made the tracking of health research resources worldwide (both poor and rich countries) fashionable, but remains the only organization that regularly tracks and reports on the comprehensive global picture. It continues to enhance the quality and comprehensiveness of the global overview of global health research budget allocations (public and private sector); to provide a more detailed, disaggregated picture; and to serve as a convenor (e.g., at the Forum in November 2009) of the groups undertaking detailed resource tracking studies in specific areas. Agendas: The GPR saw significant Global Forum effectiveness in building networks of health researches and research advocates on the health problems of poor countries and poor people, which have the strength of producing consensual research agendas in areas where they did not exist previously. The Forum also plays a direct role in collaborative research agenda-setting for health research priorities for poor people (e.g., on climate change, NCDs, sexual and reproductive health). Mindful of the need for focus, the Forum continues to do this only in one or two selected areas in each biennial period, balancing urgency and strategic needs with exercising a flexible and rapid response to opportunities arising. 129 Annex J Priority setting: The GPR seems ambivalent about the Global Forum's role in influencing research priorities, stating in different places that the Forum does not appear to have had a significant impact on research priority setting, including resource allocation; but that benefits include influence on health research priorities. This insufficiently recognizes the fact that the Forum's CAM tool has been widely disseminated in English and Portuguese (at the request of the MoH Brazil) and used at global, national and local levels to set health research priorities for and in poor countries -- including TDR, the WHO "Priority Medicines for Europe and the World" (commissioned by the Netherlands in its EU Presidency), Ministries responsible for Health and Research/ Science & Technology in e.g., Brazil, Malaysia, Pakistan, and Colombia, as well as research institutes in Argentina, Bolivia and Mexico. Influencing the global health research and innovation system The GPR highlights the importance of the Bank-supported creation of developing country health research networks and initiatives to drive the agenda in a number of highly neglected research areas of priority diseases and health risk factors in poor countries; it also recognizes the important roles of the Global Forum Annual Forum Meetings (AFMs). Through experimenting with locations and formats, great effort has been made to increase the utility of AFMs and further substantial changes for future AFMs are planned. Less attention was given by the GPR to other mechanisms for influencing resources and priorities for research on the health problems of the poor, which we would like to highlight here: The Bank identified the Global Forum as its potential umbrella health research oversight partner to channel support to global research partnerships such as IAVI, MMV, GAVI, the European Observatory, etc. In some cases, the Forum has played a strong role beyond allocating and overseeing sub-grants -- e.g., it was an active agent in helping to create MMV. Similarly, again at the Bank's specific request, the Forum had an assessment done of the malaria product pipeline and allocated US$1 million of Bank funding and oversaw the production of the report.89 There has been an ongoing dialogue about expanding this umbrella role and the Forum stands ready to collaborate in this. The Global Forum also works to help establish new global (e.g., Consortium for Global Health Diplomacy), regional (e.g., Global Health Europe) and national (e.g., National Forums for Health Research in Argentina, Norway) coalitions, to influence policy and prioritize or strengthen capacities for research for priority health issues in developing countries. The Forum (staff and FC) has a large "network" of contacts with decision makers (in LMICs and in OECD countries where the largest R&D funding decisions are made, e.g., United States, Japan, the U.K., Norway, the Netherlands), creating channels to influence decision making on research for LMIC health priorities. 89. Moran M, et al, The malaria product pipeline: planning for the future. The George Institute for International Health, September 2007. http://www.thegeorgeinstitute.org Annex J 130 The Forum comparative advantage (especially vis-à-vis WHO) to engage with the private sector is mentioned and the GPR considers that the private sector is a partner whose interests and resources merit closer examination. This is already well under way: the FC recently expanded the number of members with private sector background from one to three; the Forum gave an undertaking at Bamako to create a platform for dialogue between the stakeholders in research for health on the role of the private sector (which will feature at Forum 2009); and engagement with the private sector on resource tracking commenced in 2009. Efficiency Overall, the GPR finds the Global Forum has been reasonably efficient in its use of funds; financial management is sound; and operational and financial reports are thorough and well presented. Expenditures have reasonably closely followed budgets. The references to "budget deficits" seem to reflect an error and are incorrect (see Appendix). In implementing the Strategy 2008­14, considerable effort is being invested in further improving the efficiency of operations at all levels. This includes streamlining the FC to 12­ 14 members (most from developing countries and a high proportion of women); restructuring the GFHR to align with the new focus of the Strategy; replacement of many staff of the Research and Program Unit (emphasis on technical and communication skills); condensing three units into one new Communications Unit, with communications/advocacy objectives now being taken as the driver for much Forum work; and developing biennial rather than annual work plans and budgets, delivered through project teams with a strong focus on results, quality and impact. Moving forward: The new strategy 2008­14 and responding to the changing landscape The GPR notes that the principal positive result of the second evaluation has been the adoption of an updated Global Forum strategy and that the FC and Secretariat of the Global Forum invested a great deal of time and effort in an intensive dialogue, and many iterations of documents in the development of the new strategy. We would stress that this strategy goes much further than being an "update" and answers the need for the Forum to focus its activities further and seek broader engagement with the largest funders of health research and the private sector going forward; and for greater focus, and a higher degree of selectivity in Global Forum activities. It represents a major departure from the strategic approaches during the first decade of the Forum in at least three key respects: 1. The new strategy makes a quantum shift from "health R&D" to "research for health." This concept, which the Forum clarified and promoted from 2004, was accepted by the Forum's partners as the overall theme for the Bamako Global Forum Ministerial Summit 2008; became the subject of the WHO's and PAHO's first ever research strategy; and has been taken up within UNESCO. The concept is directly aligned with the Bank's multi-sectoral approach to development across government and public and private sectors. 131 Annex J 2. "Research for health" defines a larger domain in which the Global Forum plans to influence research resource allocation for health priorities of developing countries; in practical terms the new Forum strategy actually gives a substantially greater focus to the work of the Forum, going from nine portfolios of activities to three strategic priorities. 3. To influence research policy and R&D budget allocations, the Global Forum places a very strong emphasis on direct engagement with the top 100 R&D decision makers in the world in order to have them pay greater attention to research allocations for developing country health priorities, in addition to the broader dissemination of evidence and arguments about the "10/90 gap" and health equity. Governance The GPR notes that: All major decisions, including on launching and allocating resources for studies above a fairly low level, are taken or endorsed by the FC. The governance arrangements of the Global Forum are currently undergoing major evolutionary changes, congruent with the needs of the new strategy and the changing external environment. Future challenges Financing challenges: With a reserve currently in excess of US$3.5 million the Global Forum is reasonably placed to address the potential ending of Bank support in 2011; as such the Forum is extremely mindful of the need to secure its funding base and diversify its funding sources. The Forum's fund-raising strategy includes (i) maintaining/increasing support from existing donors; (ii) seeking additional sources of direct support from high- income countries and from LMICs (Mexico is already supporting); (iii) seeking in-kind support from LMICs for elements of the Forum's work (Brazil began such support in 2009; discussions with China are in progress); (iv) identifying ways in which foundations can provide support; (v) developing the potential for private sector contributions (actively in discussion with IFPMA and likely to lead to some funding in 2009). Competitive challenges: The GPR refers to current consultations on the consolidating the many Geneva-based health research partnerships. The Global Forum's engagement led to recognition of the need for a parallel process looking, beyond Geneva and "architecture," at the entire global health innovation system, with the Forum playing the leading convening and platform roles in November 2009 to take this forward. This illustrates a key feature of the Global Forum as a facilitator, neutral convenor and platform and a watchdog. As the GPR observes: Because of the number and variety of actors in global health research in both the public and the private sectors, the independence of the Global Forum will remain an importance aspect of its relevance, particularly if WHO becomes a major actor in health research. The GPR notes the confidence and trust placed in the Global Forum and its greater independence than other major actors in the global health arena, especially WHO; and relatively greater (than WHO) access to private sector and NGOs; and success with AFMs and other meetings in bringing many stakeholders together. Annex J 132 The GPR records the 2EE view that lack of effective insertion at country level [is] a comparative disadvantage. While this would possibly be true if the Global Forum were the only actor in this field, in practice it is COHRED which has the country-level research capacity-building mandate, with the Forum explicitly focusing to engage with the major global health research decision makers and institutes and industry to influence and bring about their reallocation of research budgets towards the developing country health priorities. Together with the majority of members from developing countries in its Foundation Council, its very large AFMs in a developing country and its collaboration with COHRED and TDR, the complementarity of having both this "insertion" and the influence with the world's main government, industry and academic health research decision makers is practically yielding an effective combined approach. The World Bank and the Global Forum The GPR Preface observes that the Global Forum was chosen because it provides lessons for the design and operation of other global programs -- in particular, for advocacy programs, and for international support of health research more generally. Overall, the experience of the Forum has evidently been positive and successful, with the GPR concluding that actual benefits to the Bank of its support for the Forum have outweighed the costs and potential downside risks and that benefits include influence on health research priorities, facilitating funding by others, and providing a forum for disseminating Bank research and for responding to criticism of the Bank among external HNP activists and observers. The GPR also highlights several challenges, including Bank weaknesses in its dealings with the Forum, including: the Bank's internal inconsistency between its "corporate strategies" (consistent with the Global Forum Strategy) and the Bank's "operational health strategy" (characterized by the GPR as less consistent with the new Global Forum strategy -- but in error, since the new Forum Strategy is also focused on research to strengthen health systems, which complements and supports the Bank's new operational health strategy). It is suggested that the Bank needs to reconcile this aspect by having its operational health strategy better acknowledge the global public goods necessity of "research" for long-term new approaches and solutions in operational health programs (e.g., current malaria programs -- in which the Bank has invested heavily -- would enjoy little of their effectiveness and success today without the new malaria products and approaches that resulted from health & medical research over several decades). weak Bank budgets and staff time allocations for proper DGF grant task management and oversight, linked with the sad (but true) diagnosis that there is little, if any, "audience" in the Bank for the value and importance of health and medical research funding and support (as distinct from "economic" research). the difficulty for the Bank to effectively appreciate, manage and oversee its support for the Global Forum and the many research partnerships that originate from -- and were incubated by -- the Forum and then are spun off over time; this includes the 133 Annex J Bank's inability to properly distinguish between the global research partnerships still directly under or involved with the Global Forum (both in terms of substance and in terms of financial involvement and support) and those no longer directly involved with the Forum (but originally intended by the Bank to be closer linked with the Forum, using the Forum as the umbrella health research organization for the Bank to manage all its DGF-funded and otherwise supported health research partnerships). The GPR insufficiently identifies these categories of original Global Forum- supported and Bank-instigated global research partnerships (created to prioritize the sub-field more prominently worldwide) and the other ones funded through the Forum for greater cohesion and consolidation among them (but where the Bank managerially and administratively so far was unable to follow through). The GPR interestingly highlights the global public good argument of a "global" research necessity for "national" or local health priorities. Together with the report's observations on DGF responsibilities to more carefully look at the financial and budget implications and future effects of Bank DGF support to research organisations and the Bank's history in them (in the case of the Global Forum a long-standing and preeminent relationship) and possible reductions in DGF support or phasing out of such support over time, the report indicates that the Bank and its DGF Program should revisit its somewhat opaque decision last year to (too abruptly) move DGF Global Forum support to the DGF Window 2 and end DGF funding in FY11. It is suggested that the Bank and the DGF Council reconsider and fund the Global Forum either anew under Window 1, reinstate support to the Global Forum under Window 1 (not an unprecedented action for DGF) or phase out its funding more gradually throughout FY13 or FY14, to allow the Forum to realistically explore alternative funding in time to prevent major disruptions of its mandate and work program and among its staff. The Forum looks forward to continuing the very fruitful and positive relationship with the Bank and to further the common objectives of ensuring that the potential benefits of research for health reach the poor -- especially populations in LMICs. Annex J 134 Appendix Finances of the Global Forum for Health Research 1998-2007 The core income, program expenditure and capital reserve history of the Global Forum is illustrated in the chart below. Established in 1998 with a capital of US$1 million, the core income, which finances the annual work programs of the Global Forum, increased every year from 1998 to reach US$4 million in 2006 and US$4.01 million in 2007. The capital reserve had grown to US$3.55 million by the end of 2003 and at this point the FC took the decision that the desirable target for the level of reserve was US$3.0 million. Subsequently, each year from 2004, the approved budget level has been set slightly larger than the anticipated income for that year, so as to reduce the capital reserve to this target of US$3.0 million. As the chart shows, the level of reserves has continued to remain above US$3.0 million (generally due to a combination of greater than expected actual income and/or smaller than expected expenses, coupled with prudent and flexible financial management by the Secretariat), while decreasing to $3.13 million by the end of 2007. 5 Income, Expenditure and Capital Reserve 4.5 Income US$ millions 4 Expense US$ millions Reserve US$ millions 3.5 3 US$ millions 2.5 2 1.5 1 0.5 0 1 1998 2 3 1999 4 5 2000 6 7 2001 8 9 2002 10 11 2003 12 2004 14 13 15 2005 16 2006 18 17 19 2007 Year It was pointed out to the Evaluation Team that they had incorrectly read the Forum's financial statements and confused core funding with designated funding for initiatives and networks; that the actual core income rose continuously from 1998 onwards; and spending was never greater than had been budgeted and approved by the FC in any year. Budgetary control was therefore exactly as required and there was never a "budget deficit." The amount of designated funding provided for new priority research networks and initiatives (mainly by the Bank but also by the Rockefeller Foundation, the Gates Foundation and DFID) has varied according to the decisions of their donors. The Bank has pursued a policy of gradually reducing funding to the initiatives it has encouraged the Global Forum to 135 Annex J incubate. Other donors prefer to use the Global Forum as an active partner in supporting Forum initiatives -- an important case being the funding of GB£ 5 million for the Alliance for Health Policy and Systems Research, which the Forum has been handling on behalf of DFID (with active engagement in contracts and disbursement of funds to projects of the Alliance). Also separately, the amount of money the Bank has chosen to provide through the Forum in sub-grants to various independent bodies like MMV and the European Observatory has varied with time and, while these designated sub-grants decreased over time, again this has no bearing on the core finances of the Forum itself. If anything, it only strengthens the case for designating the GFHR as the Bank's DGF umbrella organization through which to channel all DGF funding for global health research partnerships. The Global Program Review Series The following reviews are available from IEG. Volume #1, Issue #1: ProVention Consortium Issue #2: Medicines for Malaria Venture Issue #3: Development Gateway Foundation Issue #4: Cities Alliance Volume #2, Issue #1: Critical Ecosystem Partnership Fund Issue #2: Association for the Development of Education in Africa Issue #3: Population and Reproductive Health Capacity Building Program Issue #4: International Land Coalition Volume #3, Issue #1: Consultative Group to Assist the Poor Issue #2: Global Development Network Issue #3: Global Forum for Health Research GPR The Global Forum for Health Research is an advocacy program established in 1998 to promote health research on the problems of developing countries. The Global Forum has become known as the principal advocate of bridging the "10/90" gap--a metaphor for the global imbalance in spending on health research that suggests that less than 10 percent of global health research expenditures are being devoted to developing countries where more than 90 percent of preventable mortality is to be found. The Global Forum seeks improved pri- orities in health research and innovation, with particular attention to equity. This review found that the Global Forum has been somewhat effective. Although the funding of Global Forum core activities has been stable, its total support from donors has fallen, and the World Bank-- a key partner from the beginning--currently plans to phase out its financial support. The growth in global spending on health research, to $160 billion annually, increases the rele- vance of an advocacy effort to promote spending on the health problems of low- and middle- income countries, but the resources available to the Global Forum have been dwarfed by those available to major commercial, philanthropic, and public financiers and promoters of health research. The Global Forum needs to focus its activities and seek broader and deeper engagement with the largest funders of health research and the commercial private sector.