43015 PLANNING AND MANAGING FOR HIV/AIDS RESULTS A HANDBOOK Global AIDS Monitoring and Evaluation Team - GAMET World Bank Global HIV/AIDS Program September 2007 World Bank Global HIV/AIDS Program This series is produced by the Global HIV/AIDS Program of the World Bank's Human Development Network, to publish interesting new work on HIV/AIDS quickly, make it widely available, and encourage discussion and debate. The findings, interpretations, and conclusions expressed in this paper are entirely those of the author(s) and should not be attributed in any manner to the World Bank, to its affiliated organizations or to members of its Board of Executive Directors or the countries they represent. Citation and the use of material presented in this series should take into account that it may be provisional. Papers are posted at www.worldbank.org/AIDS (go to "publications"). For free print copies of papers in this series please contact the corresponding author whose name appears the bottom of page iii of the paper. Enquiries about the series and submissions should be made directly to Joy de Beyer (jdebeyer@worldbank). Submissions should have been previously reviewed and cleared by the sponsoring department, which will bear the cost of publication. The sponsoring department and author(s) bear full responsibility for the quality of the technical contents and presentation of material in the series. Global AIDS Monitoring and Evaluation Team The World Bank hosts the Global Monitoring and Evaluation Team (GAMET) on behalf of the UNAIDS family. GAMET's task is to improve the quality of HIV/AIDS monitoring and evaluation (M&E), build national capacity and functioning M&E systems within countries, and promote the use of M&E data to improve country programs and results. GAMET also helps project teams build a results-focus and M&E into projects. Cover photographs by Curt Carnemark, Yosef Hadar, Janet Leno and Shehzad Noorani © 2007 The International Bank for Reconstruction and Development / The World Bank 1818 H Street, NW Washington, DC 20433 All rights reserved. ii PLANNING AND MANAGING FOR HIV/AIDS RESULTS A HANDBOOK Rosalía Rodriguez-García and Jody Zall Kusek The Global AIDS Monitoring and Evaluation Team Abstract: This Handbook aims to support countries as they develop or revise their national strategy, or formulate new HIV/AIDS policies. It introduces an approach to thinking and planning that is guided by evidence and oriented towards results. The Results Cycle is an organizing framework that guides and supports both the planning process and the production of the strategy document that will guide future implementation. It is a logical approach to improving policy development and strategy planning, and thus to improving program performance. Keywords: Management for Results, Results-Based Management, Results Cycle, HIV/AIDS, Monitoring, Evaluation, World Bank, GAMET, Strategy, Planning. Disclaimer: The findings, interpretations and conclusions expressed in the paper are entirely those of the authors, and do not represent the views of the World Bank, its Executive Directors, or the countries they represent. Corresponding Author: Rosalia Rodriguez-Garcia, World Bank, 1818 H Street, NW, MSN G8-802, Washington DC, 20433. Email: rrodriguezgarcia@worldbank.org Vice President Joy Phumaphi Human Development Network Director Debrework Zewdie Global HIV/AIDS Program iii Acronyms and Abbreviations AIDS Acquired immunodeficiency syndrome ARV Antiretroviral ASAP AIDS Strategy and Action Plan service BCC Behavior-Change Communication CRIS Country Response Information System DHS Demographic and Health Survey GAMET Global AIDS Monitoring and Evaluation Team GFATM Global Fund to Fight AIDS, Tuberculosis, and Malaria HIV Human immunodeficiency virus IEC Information, education, and communication MDG Millennium Development Goal M&E Monitoring and evaluation MSM Men who have sex with men NAC National AIDS Committee or Council NGO Nongovernmental organization PEPFAR President's Emergency Program for HIV/AIDS Relief PLHIV People living with HIV PRS Poverty Reduction Strategy UNAIDS Joint United Nations Programme on HIV/AIDS UNDP United Nations Development Program UNFPA United National Population Fund UNICEF United Nations Children's Fund WHO World Health Organization iv Table of Contents PREFACE....................................................................................................................................VII ACKNOWLEDGEMENTS.......................................................................................................... IX PART ONE................................................................................................................................. 1 1. Introduction to "Managing for Results"........................................................................ 1 2. Global HIV / AIDS Context............................................................................................. 3 3. Using Evidence to Ground the Results Cycle................................................................. 4 3.1. The policy development process............................................................................... 4 3.2. Strategic Planning..................................................................................................... 4 3.3 Managing for Results................................................................................................ 6 3.4. Results-based Monitoring and Evaluation................................................................ 7 4. Applying the Results Cycle to Strategy Formulation.................................................... 9 Phase 1: Analyze HIV/AIDS Data and the National Response......................................... 11 Phase 2: Identify Results ­Outcomes and Indicators......................................................... 12 Phase 3: Select Strategic Programs.................................................................................... 15 Phase 4: Select Critical Interventions for each Program and Cost Them.......................... 16 Phase 5: Specify how Results will be Monitored and Resources Tracked........................ 18 Phase 6: Specify how Changes in the Epidemic will be Evaluated, Including the Results of the National Response ................................................................................................... 22 Phase 7: Use the evidence for the next strategy................................................................. 23 5. The Strategy Document ................................................................................................. 24 5a. Main Sections of a Strategy Document .................................................................. 24 5b. Indications that a Strategy Document has a Results Focus .................................... 26 5c. Quality Enhancement.............................................................................................. 26 6. Conclusion....................................................................................................................... 27 PART TWO ­ NOTES FOR TRAINERS.............................................................................. 29 PART THREE ­ POWERPOINT SLIDES........................................................................... 31 References and Resources ..................................................................................................... 86 Internet Sources Consulted (in alphabetical order) ............................................................... 90 APPENDIX 1: Definitions ................................................................................................... 91 APPENDIX 2: Results Framework and Indicators Menu.................................................... 94 APPENDIX 3: Illustrative Checklist to Manage the Process of Planning the Formulation and/or Review of a National Strategy ................................................................................... 97 v List of Figures Figure 1: The Results Cycle............................................................................................................ 2 Figure 2: Skills Mix of a National Strategy Planning Team........................................................... 5 Figure 3: Principles of Strategic Planning...................................................................................... 5 Figure 4: Evolution of Managing for Results................................................................................. 7 Figure 5: The Logical Approach of the Results Chain................................................................... 8 Figure 6: Characteristics of a Results-Based Strategy.................................................................... 9 Figure 7: Definitions of Key Terms Used to Describe Epidemics ............................................... 12 Figure 8: Definitions Used in Outcomes-Results and Results Frameworks................................ 12 Figure 9: Examples of Outcomes, Indicators, Baselines and Targets (Prevention) ..................... 14 Figure 10: Examples of Outcomes, Indicators, Baselines and Targets (Treatment) .................... 14 Figure 11: Applying the Logic of the Results Chain to HIV/AIDS ............................................ 15 Figure 12: Essential Programmatic Actions for HIV Prevention ................................................. 17 Figure 13: Template for Summarizing Indicators and Targets..................................................... 19 Figure 14: Results Framework Template with an Illustrative Example....................................... 20 Figure 15: Components of a Functional HIV M&E System......................................................... 21 Figure 16: Availability of Information on Key Groups, Central American Countries................. 22 Figure 17: Analysis of Progress: ZANARA Logical Framework ................................................ 24 vi PREFACE This Handbook evolved from GAMET's experience advising policymakers and program planners about effective planning and programming, in which expectations of results are clear and explicit. It reflects years of coaching and teaching policy analysis, strategic planning, and program monitoring and evaluation, and the lessons learned from the collective experience of the Global HIV/AIDS Program advising on operations and working with colleagues to improve results-based planning, measurement, monitoring and evaluation in many higher and lower income countries. The Handbook's three parts support each other. Part I, the text, provides the grounding. It describes the results approach to planning strategically, which requires strategic information to be generated and used, which in turn necessitates sound monitoring and evaluation. An important use of this Handbook is for coaching and/or training. Part II therefore includes some very brief guiding thoughts for trainers. Part III is a set of slides that further elaborates the approach, and provides examples and templates to help planners formulate national strategies and elaborate their strategy documents. The slides are organized in a logical manner following the seven phases of the Results Cycle. These slides and selected sections of the Handbook have been presented in several fora and used successfully to support operations, programming, strategy formulation and planning strategic information for managing for results. This publication is designed as a generic handbook that countries, implementing agencies and NGOs can adapt for their own use and as appropriate to the situation. It is not intended to be guidelines; rather it synthesizes key information to underscore the results approach to strategic planning and to support its application. The Handbook aims to be user-friendly. To facilitate its use, the Handbook for Results Planning is presented in printed and electronic form ­ a CD-ROM is included. This publication is meant to be an evolving and living document. Future editions will incorporate lessons learned from its use in different settings. This Handbook for Results Planning can also be accessed from the website of the World Bank Global HIV/AIDS Program at: www.worldbank.org/aids (in the M&E part of the site, or the publications page) The Handbook provides links to other agencies and makes references to other relevant documents the reader may like to consult to read more about specific topics or a particular agency's approach. The intention is to encourage further study of complementary resources. vii viii ACKNOWLEDGEMENTS This Handbook was conceived and developed by two principal authors, Rosalia Rodriguez- García and Jody Zall Kusek. However, the final publication is the result of a team effort. The authors appreciate the valuable contributions and insights of colleagues inside and outside the World Bank, including UNAIDS, the Pan American Health Organization/WHO, The National Public Health Institute in Cuernavaca, Mexico, and the Futures Institute. As a token of our recognition, their names ­ in alphabetical order ­ are presented below: Juliana Ahuchogu-Victor Stefano M. Bertozzi Rene Bonnel Steven Forsythe Eric Gaillard Marelize Gorgens-Albino Janet Leno N'della N'Jie Robert Oelrichs Patrick Lumumba Osewe Greet Peersman Hernán Rosenberg Deborah Rugg George Tempo Bob Verbruggen David Wilson Annick Wouters The authors appreciate the good editorial advice of Joy de Beyer and the help of Uma Balasubramanian. ix ix PLANNING AND MANAGING FOR HIV/AIDS RESULTS A HANDBOOK PART ONE 1. Introduction to "Managing for Results" The global development community has recognized the need to "manage better for results" ­ that is, to improve the way government entities work by ensuring that resources are used more effectively in implementing HIV policies and programs to achieve results on the ground. This gives performance information at the operational level great strategic value. The challenges are to provide high-quality and timely information for decision making at critical points, and to help development practitioners incorporate the use of outcome information in their work.1 In early 2003, the World Bank embraced "managing for results" as a guiding approach for improving the effectiveness of its assistance.2 Managing for results deepens dialogue with clients and governments. It sharpens the focus of business models on policy changes and demonstrable results on the ground.3 As countries' experience with HIV/AIDS matures, national AIDS authorities and policymakers are increasingly pressed to show the results that national policies and programs are achieving. Governments and their partners are committed to ensuring that resources are used effectively to support national responses to the HIV epidemic. The effectiveness of national strategies will ultimately be judged by their impact and results ­ that is, by how well they contain the HIV epidemic through prevention, treatment and care, and improve the lives of people with HIV. Countries want to shape policies and strategies and implement successful programs and projects to scale up and sustain HIV prevention, treatment and care. They also strive to strengthen partnerships at the country level to improve the effectiveness of HIV support and its results. It is a difficult challenge to formulate HIV/AIDS policies and programs, plan and execute a strategy, mobilize resources and use them effectively. The reality of planning national policies and strategies is complex; so models and tools that simplify the process into core areas can help planners know where they are in the process, and to focus systematically on the tasks that need to be completed. This is not a "how to" paper. Rather, this Handbook for Planning and Managing for HIV/AIDS Results aims to support countries as they formulate new HIV/AIDS policies and/or develop or revise their national strategy, by introducing an approach to thinking 1Rodriguez-Garcia and White, 2005. 2http://www.mfdr.org. Managing for Results Website of the World Bank. 3Rodriguez-Garcia and White, 2005. 1 and planning that is guided by evidence and oriented towards results. The Results Cycle is an organizing framework that guides and supports both the strategic planning process and the production of the strategy document that will guide future implementation. Phases 1 to 7 in figure 1 reflect a logical approach to improving policy development and strategy planning, and thus to improving program performance. Figure 1: The Results Cycle Introducing the Results Cycle RESULTS CYCLE Applied to HIV Use the Strategy Planning Evidence for the NextStrategy 7 1 Formulate/Revise the HIV Strategy Analyze HIV Specify Evaluation Epidemic and National Of Changes Response Data in the Epidemic 2 6 Identify Results Select Critical Specify -Outcomes & Indicators- Interventions ­ Cost Monitoring Them ­ Identify of Results Resources 3 5 4 Select Strategic Programs Source: R. Rodriguez-García, GHAP, World Bank, 2007. The Results Cycle is based on the principle that strategy formulation is guided by national policy, but the application of the Results Cycle brings to the table both policy decision makers and technical staff as strategic planning is done. For instance, an important aspect of planning strategically ­based on evidence­ is presenting to the political leadership (often the Minister of Health or the Council of Ministries) the evidence data gathered about changes and trends in the epidemic and the performance of the national response in affecting those changes. This is important because policy makers need to understand the meaning and implications of the data for society, systems, and resources.4 4Inter-American Development Bank, 2006. 2 2. Global HIV / AIDS Context One of the Millennium Development Goals is to halt and reverse the AIDS epidemic by 2015. To reach this goal, advocacy and funding are paramount. And it is essential to "make the money work" effectively. At the 13th International Conference on AIDS and Sexually Transmitted Infections in Kenya in 2003, country and international representatives developed the "Three Ones" -- a set of guiding principles for improving AIDS responses in countries: · One agreed AIDS action framework that provides the basis for coordinating the work of all partners · One national AIDS coordinating authority, with a broad-based multisectoral mandate · One agreed country-level monitoring and evaluation system. To achieve the First One, UNAIDS and partners support countries to formulate and/or revise their national HIV/AIDS strategies and action plans5 and to integrate AIDS into national development plans such as poverty reduction strategies (PRS) and public expenditure frameworks.6 To achieve the Third One, UNAIDS and partners provide technical advice and support to national AIDS authorities to improve surveillance and strengthen monitoring and evaluation systems.7 National HIV/AIDS Strategies are needed to attract and sustain funding (from the Ministry of Finance, donors, private sector, others); assure an important role for civil society and communities; align financial resources to national goals and programs; respond to the heterogeneity of epidemics; and implement the Three Ones principles. National Policies help define overall goals and principles, and are part of the legislative and regulatory framework that supports the implementation of the strategy. Action Plans identify the programs and projects that implement the strategy and ensure timely responses to changes in the epidemic and environment, and link these programs to government planning cycles and budgets. Policies, strategies and action plans are supported by national monitoring and evaluation activities that measure changes in the indicators identified in the strategy and action plans to assess progress towards performance targets. M&E plans describe how data will be collected, compiled, processed and reported within the country at all levels, with a focus on using information for decision-making, accountability and learning. 5A global technical assistance service, the AIDS Strategy & Action Plan (ASAP), housed in the World Bank's Global HIV/AIDS Program since January 2006, responds to country requests for support in developing well-prioritized, evidence-based, results-focused, costed AIDS strategies and action plans. www.worldbank.org/ASAP. ASAP was created by the Global Task Team. The final report of the Global Task Team (in which numerous organizations participated) can be found at www.UNAIDS.com. 6UNAIDS 2006, pp. 254-258. 7 Set up in 2002, the Global HIV/AIDS Monitoring and Evaluation Team (GAMET), housed in the World Bank's Global HIV/AIDS Program, supports evidence building and utilization, and works closely with UNAIDS and other partners to support improvements in country M&E systems and capacity. 3 3. Using Evidence to Ground the Results Cycle The national HIV/AIDS response involves three key processes: (i) policy development; (ii) strategic planning and implementation, and (iii) managing for results. These are described briefly below. 3.1. The policy development process Policy formulation is inherently a political process ­ it reflects the priorities of the government currently in power and focuses on how that government communicates to citizens and other stakeholders its position on various issues. Although political in how it is communicated, policy development in most democracies is typically informed by evidence acquired through a rigorous analytical process to guide and justify government action. For example, prior to setting national HIV/AIDS policy, a government would want to know the key drivers of the epidemic, in addition to the views of its citizens on acceptable measures to contain the epidemic. 3.2. Strategic Planning The formulation of a national HIV/AIDS strategy is a process through which a country defines fundamental principles, priority programs, expected achievements and the institutional framework to guide the national response to the epidemic. What makes an HIV/AIDS strategy "strategic" is that it takes into account the underlying determinants of the epidemic and how they affect different social groups, and then carefully selects approaches and actions that will address each group to achieve specified results.8 The social situation and the HIV epidemic change over time, so a strategy has to be flexible enough to adapt in a significant way to these changes and to new information from monitoring and evaluation. Hence the process is shown as a cycle (figure 1) to reflect an iterative process. The strategy formulation process contributes to shaping policy outcomes and brings together political and technical elements. Interventions and population groups must be selected to be prioritized to receive services, based on evidence of what works and what does not work, and stakeholder support and resources need to be mobilized to implement the strategy. Therefore, the planning team would include individuals skilled at policy- making and social analysis as well as service delivery and cost estimation (see figure 2). 8Readers may like to consult UNAIDS, 2000, Guide to the Strategic Planning Process for a National Response to HIV/AIDS: (1) Situation Analysis, (2) Response Analysis, (3) Strategic Plan Formulation, and (4) Resource Mobilization; and also ASAP, 2006 Self-Assessment Tool and Guidelines for National HIV/AIDS Strategies, at: www.worldbank.org/asap 4 Figure 2: Skills Mix of a National Strategy Planning Team · Capacity and authority to make decisions and manage the planning process. · Knowledge of how government functions. · Ability to manage stakeholders consultations. · Ability to coordinate multi-sectoral and private sector inputs. · Ability to coordinate community and civil society inputs. · Knowledge of the HIV/AIDS situation (epidemiological, social behavioral, legal elements). · Program development and management experience. · Results-based planning, monitoring and evaluation expertise. · Financial management skills (expenditures, budgeting, costing). Source: R. Rodriguez-García, J. Reno and R. Bonnel, GHAP, World Bank, 2007. Traditional planning approaches focus on tracking inputs (financial and human resources) to provide goods and services and to achieve corporate or program goals. Results-based planning applies a logical approach to the use of inputs for implementing projects and programs in order to achieve longer term results with greater societal benefits. The Results Cycle provides added value to the traditional approach to strategic planning in that it helps decision makers look back ­ at evidence of past achievements and things that did not achieve results ­ and forward to select priority interventions that would help reach the strategy's goals. This process is mindful of the country contextual environment for policy formulation and implementation and the role of stakeholders. Figure 3: Principles of Strategic Planning 1 Strategic Planning uses evidence and focuses on results that reflect the realities of the epidemic and the achievements of the national response. 2 Strategic Planning is not a lineal process --is an iterative process. It requires several phases ­as shown in the Results Cycle- which reinforce each-other. 3 Monitoring and evaluation is an essential mechanism of the national response ­ its planning, management, monitoring and accountability. - If results are not measured, managers cannot differentiate success from failure. 4. The strategy document is a "living document" It can and should be adjusted as needed during implementation -based on data from program monitoring, studies, evaluation and others. Source: GAMET, GHAP, World Bank, 2007. 5 Strategic planning applies a logical approach to define how: · underlying determinants and biological/social factors affect HIV outcomes and over time demographic outcomes (i.e. health impacts) = conceptual framework; · programs should operate (i.e., from inputs to impacts) = logical framework; · policies, strategies and programs achieve goals = results framework. 3.3 Managing for Results Managing for results refers to a comprehensive and integrated management system that focuses on achieving national objectives for the population while assuring accountability for public funds. Earlier defined as performance management,9 managing for results started in the private sector, where it is embedded in all levels of the management structures of many high performing corporations. Results management enables managers and staff to value open and honest performance assessment and reporting, which in turn improves productivity, accountability and learning. In the 1990s, many governments and the international development community adopted performance approaches and tools in support of more effective development. As results-based management was gradually introduced in the public sector in the 1990s, a more systematic approach to program planning and management became the basis for numerous public sector reforms.10 Since 2003, the World Bank has adopted a results agenda to improve Bank operations and achieve stronger results on the ground; i.e., outcomes, defined as specific improvements in the quality of life and economic productivity. A focus on results recognizes that counting inputs, outputs and disbursements is not enough; there needs to be attention to outcomes and impact. A focus on results includes: (i) promoting better financial management, (ii) supporting capacity enhancement and statistical institutions for monitoring and evaluation within countries, (iii) assisting national efforts to disclose and disseminate results information, especially to policy makers, (iv) preparing results- based country assistance strategies, (v) improving monitoring and evaluation components of loans and grants, (vi) promoting pursuit of the MDGs, and (vii) strengthening attention to measuring the success of analytical and advisory work and evaluation efforts.11 A 2006 assessment of the Bank's results focus approach to development identified several important lessons from country experiences and assistance programs that have delivered the strongest results. Effective programs: (i) emphasize growth and the measures that help the poor share in the growth process, (ii) build on realistic and informed assessments of political commitment, and capacity to deliver results, (iii) combine sustained engagement with clear intermediate outcomes, and (iv) emphasize transparency and local control of public institutions. Effective articulation and utilization 9The basis for these three concepts "results-based management", "managing for development results" and "managing for results" are the same and have the same aim: to improve performance. 10OECD-DAC, 2007. 11World Bank, 2004b 6 of the results-chain is an essential tool for effective aid, together with country capacity to collect and use performance information.12 In the international development community, the introduction of management for development results was driven by the recognition that international aid needed to be more effective in improving development outcomes if countries were to achieve the Millennium Development Goals (MDGs). Figure 4 summarizes the evolution of managing for results to increase aid effectiveness. Figure 4: Evolution of Managing for Results 1970s 1990s 2000s 2010s Results-based International Results-based Development M&E is being development management partners adopt mainstream in agencies and incorporates a logic Management for governments pubic sector adopt chain approach to Development and agencies as Logic Models to help systematically Results to an essential tool plan the efficient identify objectives, improve aid to measure utilization of impacts, outcomes, efficiency and progress in project resources outputs and inputs outcomes towards to track the results project/program reaching the of public sector outcomes MDGs reforms Source: R. Rodriguez-García, GHAP, World Bank, 2007. Managing for Results uses a variety of tools and processes to generate information that can be used to improve policies, planning, management, monitoring, evaluation and delivery of good and services. The Strategy Results Cycle is such a tool. It responds to a key goal: to provide timely, reliable, and useful feedback about ongoing and completed actions, and to derive relevant strategic implications that are immediately useful in strategy design and program management.13 3.4. Results-based Monitoring and Evaluation Increased reliance on information for decision making demands continuous refining of planning and monitoring and evaluation methods. The more so when countries are willing to be held accountable for providing goods and services to their populations through effective use of resources, and for showing partners the outcomes of their investments. It follows that M&E becomes an essential tool for decision makers ­ a mechanism to collect and provide evidence of outcomes that decision makers need. A results-based monitoring and evaluation system adds the "fourth leg" to the governance 12World Bank, 2006 13Rodriguez-Garcia and White, 2005 7 chair traditionally built around systems for budgeting, human resources and auditing, providing an essential feed-back mechanism on the outcomes and consequences of government policies and actions.14 A Results Chain helps identify the logic (cause-and-effect relationships) behind policy and strategy design. It posits that inputs are needed for activities and products and these lead logically to the achievements of outcomes and impacts (see figure 5). It depicts an "if-then" theory of clearly articulated goals coupled with specific expectations of achievement at each stage of the logical results chain.15 (See selected definitions in Appendix 1.) So, for example, if pregnant women who are HIV-positive are offered counseling, testing, appropriate treatment and follow-up, then parent-to-child transmission can be reduced. Monitoring produces the data that managers need to make routine programmatic adjustments towards achieving the desired outputs and outcomes. Evaluation and analytical reviews go a step further to provide the strategic information policymakers need to steer policy formulation and strategy planning towards sustainable outcomes. Together, monitoring and evaluation are critical tools for management, learning and accountability. Decision-makers manage for results by using information to steer policies and programs to achieve results on the ground, and making decisions that improve learning and knowledge geared towards improving performance and accountability. Figure 5: The Logical Approach of the Results Chain Goal · Long-term, widespread improvement (Long-term in society Impacts) y "Big picture" (country longer term strategy) NGINNALP · Effects or behavior changes resulting Strateg Outcomes from a strategic program S Outputs · Products and services that LT ng need to be developed to mi achieve the expected outcomes ESUR ma Activities · What actually was done with the available resources to produce the ogrrP intended outputs Inputs · Critical resources (expertise, equipment and supplies) needed to implement the planned activities Source: Binnendijk, 2000; Kusek and Rist, 2004. 14Kusek, Rist and White 2004. 15"If-then" refers to the logical progression that if X is done or happens, then Y will follow. 8 4. Applying the Results Cycle to Strategy Formulation The Results Cycle helps make HIV/AIDS strategies results-oriented by identifying key phases in the strategic planning process in a logical approach where inputs are linked logically to expected impacts through activities, outputs and outcomes. The application of the Results Cycle does not assume that decision makers are starting from nothing; rather, it assumes that there are experiences and information available, including a previous strategy, epidemiological and behavioral reports, an M&E plan and other information. This national experience is essential. The Results Cycle provides an organizing framework to make valuable use of this national experience, basing decision- making on existing evidence and desired results (see figure 6). Figure 6: Characteristics of a Results-Based Strategy How do you Develop a Strategy that is Focused on Results? · Base it on evidence on the epidemic's evolution and effects on specific population groups, · Understand and address the root causes and main modes of infection in the country · Identify specific results to be achieved · Indicate how the strategy will be implemented (who will do what, and when) · Explain how the national response will be funded, monitored, and evaluated. Source: GAMET, GHAP, World Bank, 2007. So a focus on results starts with using existing evidence on the national HIV/AIDS situation to answer two broad questions: (a) What is the status of the epidemic? And (b) What results have been achieved by the national response strategy ­ how well has the country contained and managed the epidemic? The answers to these questions provide the underpinnings for (1) identifying results/outcomes, (2) selecting the programs and population groups, and (3) shaping the results-based monitoring, evaluation and research elements of the strategy, including resource tracking and analysis. The phases of the Results Cycle are iterative, not strictly sequential. Applying the Results Cycle supports the planning process and production of the actual strategy document. Each phase of the Results Cycle relates to a section of the national HIV/AIDS strategy document. Implicit in the Results Cycle is the need for an enabling environment for policy and strategy formulation. This would include: compliance with the "Three Ones" principles, stakeholder consultation, a multisectoral approach, donor coordination, and 9 systems strengthening.16 During the planning process, specific actions would be identified around these themes to be integrated in the national response. The planning process takes place within a policy context, as does implementation of the strategy. The policy actions that are essential to promote HIV prevention include:17 1. Ensure that human rights are promoted, protected and respected and that measures are taken to eliminate discrimination and combat stigma. 2. Build and maintain leadership from all sections of society, including governments, affected communities, non-government organizations, faith-based organizations, the education sector, media, the private sector and trade unions. 3. Involve people living with HIV in the design, implementation and evaluation of prevention strategies, addressing their distinct prevention needs. 4. Address cultural norms and beliefs, recognizing both the key role they may play in supporting prevention efforts and the potential they have to fuel HIV transmission. 5. Promote gender equality and address gender norms and relations to reduce the vulnerability of women and girls, involving men and boys in this effort. 6. Promote widespread knowledge and awareness of how HIV is transmitted and how infection can be avoided. 7. Promote the links between HIV prevention and sexual and reproductive health. 8. Support mobilization of community-based responses throughout the continuum of prevention, care and treatment. 9. Promote programmes targeted at HIV prevention needs of key affected groups and populations. 10. Mobilize and strengthen financial, and human and institutional capacity across all sectors, particularly in health and education. 11. Review and reform legal frameworks to remove barriers to effective, evidence- based HIV prevention, combat stigma and discrimination and protect the rights of people living with HIV and people who are vulnerable or at risk of HIV. 12. Ensure that sufficient investments are made in research and development of, and advocacy for, new prevention technologies. 16See UNAIDS Country Harmonization Assessment Tool (CHAT) developed to assess the level of partnerships in HIV/AIDS at the country level. Some countries conduct a CHAT assessment as part of national response reviews. 17Practical Guidelines for Intensifying HIV Prevention: Towards Universal Access, 2007, p.23. 10 Appendix 2 suggests key actions in the strategy planning process. These actions can be expanded and adapted to real situations. The list can be used as a check-list to help the planning process and to specify the products expected at each phase of the planning cycle, as well as specifying the responsible entities and deadlines.18 Phase 1: Analyze HIV/AIDS Data and the National Response A national HIV/AIDS strategy that is focused on achieving results starts with an analysis of the trends, drivers and risks factors of the HIV epidemic in the country. If a recent synthesis report or epidemic review does not exist, the strategy planning team would need to access all reports and available data from national and international sources, such as behavioral, epidemiological or other studies, clinical trials, impact evaluations and policy or operations research, in order to identify how the epidemic has progressed during the last strategy period, and include this in the strategy document. Equally important is to determine the achievements of the national response, the goods and services that were provided and the extent to which access to these services benefited at- risk groups and the general population. Both primary and secondary sources are useful. Primary data are new data collected specifically for the task at hand (for example through surveys, direct observation or interviews). Secondary data have been collected previously by someone else, perhaps for a different purpose. Examples include survey data collected by another agency, a Demographic Health Survey, or data from a marketing company. The strategy planning team can consider out-sourcing preparation of a synthesis report that uses available data. If evidence is scant, computer models can be used to estimate and forecast the pattern and trends in the national epidemic. In analyzing the epidemic, both quantitative and qualitative data are important in understanding HIV infection drivers and trends, as well as prevention, treatment and mitigation program coverage and costs, and adherence to ARV treatment. The results of this analysis and review would shed light on changes in the epidemic, how specific population groups are affected, and which groups are most at-risk. It will also underscore the achievements of the national response and point to where resources could be most effective in the next strategy period. This will help define clearly the goals and outcomes for the new strategy (see figure 7). In practical terms, this stock-taking exercise may take the form of a comprehensive situational assessment that analyzes the epidemiological changes and trends, the effects of the national response and the available and needed resources. If the current epidemiological situation is known, the evidence building exercise may also take the form of a self-evaluation of the national response that systematically reviews the outcomes of the strategy and identifies strengthens and gaps in achieving results, which encourages problem-solving and improved planning.19 18See also the ASAP Strategy Planning Road Map , on line at www.worldbank.org/ASAP 19Rodriguez-Garcia and White, 2005, p.29. 11 Figure 7: Definitions of Key Terms Used to Describe Epidemics Incidence refers to the number of new infections in a population during a year. Prevalence is the total number of infected people at a point in time expressed as a percentage of the population. Drivers refer to the structural and social factors, such as poverty, gender and human rights that are not easily measured that increase people's vulnerability to HIV infection. Now, the term drivers is used to describe the risk factors which primarily account for the increase and maintenance of an HIV epidemic. Risk is the probability that a person may acquire HIV infection. Certain behaviors create, enhance and perpetuate risk. Examples include unprotected sex, multiple partners, injecting with contaminated needles. Vulnerability results from a range of factors that reduce the ability of individuals and communities to avoid HIV infection. Source: Adapted from UNAIDS (1998). Expanding the Global Response to HIV/AIDS through Focused Action: Reducing Risk and Vulnerability: Definitions, Rationale and Pathways, Geneva. Phase 2: Identify Results ­Outcomes and Indicators Based on the evidence gathered about the epidemic, the HIV/AIDS strategy planning team will want to state clear goals and specific results (outcomes and impacts) to be attained. It may be helpful to explain the difference between goals and results: A goal is a statement of vision and direction describing what the strategy aims to achieve (see figure 5 for an example). A result is a representation of what success would look like (e.g. The percentage of people aged 15-19 reached with HIV prevention programs increases from 48% to 75% by 2010; utilization of testing services by pregnant women increases by 50% by 2012). Most HIV/AIDS strategies include broad goals, usually related to prevention, care and treatment, and specific results that would help achieve the goals, expressed in terms of coverage, utilization and behavioral change (see figure 8). Figure 8: Definitions Used in Outcomes-Results and Results Frameworks Coverage is the percentage of the population needing a service that has access to the service. Access may depend on many things such as the proximity of the nearest service point, the times when the service is available, cost to users, and eligibility criteria that may be established by national guidelines or service providers. As a practical matter, it is often better to measure coverage in terms of utilization. Utilization is the percentage of the population in need that actually uses the service. Source: Practical Guidelines for Intensifying HIV Prevention: Towards Universal Access, 2007, p.5. 12 For the strategy, the focus would be on identifying a small set of core impacts and outcomes relating to each goal ­ keeping in mind the feasibility of measuring them. Intermediate outcomes and outputs can be included in the Action Plan for implementing the strategy (sometimes called Operational or Work Plan). All results need to be measurable (quantifiable) and linked to the monitoring and evaluation system. To ensure this, the relevant questions are: Are data available for measuring results? Are there mechanisms in place to collect, compile, process and report data?20 Each outcome should have at least one corresponding outcome indicator. Each indicator would show the baseline value (and the year), and the expected performance target. Countries find it useful to consider a hierarchy of indicators starting with national and global indicators for higher-level goals using standardized indicators developed by UNAIDS and partner countries and agencies (see Appendix 2).21 These include: · Indicators for Implementation of the Declaration of Commitment on HIV/AIDS · Universal Access indicators · National impact and outcome indicators · Country-relevant and national program indicators. Many strategies include a Results Table (sometimes referred to as a results framework) to summarize information on outcome indicators, baselines and targets. Figure 9 shows an example for prevention and figure 10 for treatment. Baselines are essential for knowing what progress has been made towards targets; yet baselines are often lacking or are based on old estimates. The first step should always be to mine existing data, searching for values that can be used as legitimate baselines. An analysis of existing data is also important to identify programmatic findings and trends that can help elucidate the characteristics of the epidemic in the country and its progression and variation over time. When baseline values are not available, gathering baseline data on key indicators (using primary or secondary data) should be included as an output of the new strategy, and baselines studies conducted, as appropriate, early in the strategy period. 20For an in-depth discussion on this topic, please refer to Kusek and Rist, 2004. 21For more discussion on indicators consult: Guidelines on Construction of Core Indicators-2008 Reporting, UNAIDS, 2007. Note: An inter-agency "Indicator Harmonization Working Group", a technical working group of the Monitoring and Evaluation Reference Group (MERG) meets annually to review and agree on M&E issues, and especially to harmonize HIV/AIDS indicators. 13 Figure 9: Examples of Outcomes, Indicators, Baselines and Targets (Prevention) Building Blocks for Measuring the Results of the HIV/AIDS Strategy Example: Prevention - some commonly used indicators and indicative targets Outcomes Indicators Baselines Targets 1. IDUs will have 1. Percent of IDUs with access to 20% in 50% better access to needle needle exchange programs 2005 by 2010 exchange programs 2. Use of condoms by 2. Percent of SWs who report 35% in 70% SWs will increase using a condom with their most 2004 by 2010 recent client 3. Use of condoms by 3.1 Percent of men aged 15-24 48% in 60% young men and who report using a condom 2006 by 2010 women will increase during last sex in the last 6 months. 23% in 50% 3.2 Percent of women aged 15- 2006 by 2010 24 who report using a condom during last sex in the last 6 months. Source: R. Rodriguez-García, GHAP, World Bank, 2007. Figure 10: Examples of Outcomes, Indicators, Baselines and Targets (Treatment) Building Blocks for Measuring the Results of the HIV/AIDS Strategy Example: Treatment and Care ­ commonly used indicators and indicative targets Outcomes Indicators Baselines Targets 1. Better access to 1. Percent of Adults and children with 10% in 80% by ART for PLHIV advanced HIV infection receiving ART 2006 2010 2. Improved 2. Percent of health care facilities that 35% in 95% by hospital and clinic have the capacity and conditions to 2006 2010 capacity to treat provide basic-level HIV testing and HIV disease HIV/AIDS clinical management 3. Opportunistic 3. Percent of PLHIV receiving 20% in 90% by infections diagnosis and treatment for 2006 2010 managed better opportunistic infections 4. Reduced AIDS 4. % of adults and children with HIV 50% in 70% in mortality alive 12 months after initiating ART 2006 2010 (extend by 12 months ea year) 5. First line ART 5. Percent adults and children with HIV 70% in 85% in regimen at remaining on first line ART 24 months 2006 2010 optimum duration after initiation Source: R. Oelrichs, GHAP, World Bank, 2007. 14 Phase 3: Select Strategic Programs A strategy selects programs aimed at achieving the expected results or identified outcomes of the strategy. Therefore, the starting point in programming is not to identify the activities to be undertaken, but rather to use the expected results to be achieved as a destination, and identify the key policy actions and programs that will get the country there. This involves creating a hypothetical cause-effect link from inputs, to processes, outputs, outcomes and impacts, applying the logical results chain (figure 11 applies this logical approach to HIV/AIDS programming). Figure 11: Applying the Logic of the Results Chain to HIV/AIDS 22 Applying the Logic of the Results Chain to HIV/AIDS IMPACTS Reduced HIV Incidence and Prevalence. HIV prevalence is the bedrock of surveillance, Strategy monitoring, and evaluation (longer-term goal) Increased Coverage. Access and use OUTCOMES of services and behavior changes (strategy objectives and results) Programming Products and Services. First OUTPUTS level results needed to achieve the outcomes (program outputs) Critical resources. Includes money, people, equipment, supplies INPUTS and know how (program inputs) Source: R. Rodriguez-García, GHAP, World Bank, 2007. The evidence on the epidemic should help focus the strategy towards the appropriate programs. Strategies are likely to include the following: Programs HIV Prevention aimed at key drivers (knowledge, behavior) AIDS Treatment Care and Support (coverage and utilization = access) Impact Mitigation (coverage and utilization = access) 22Incidence ­ new infections in a population during a year ­ is difficult and costly to measure. Prevalence ­ the total number of infected people (no matter when they became infected) at a point in time expressed as a percentage of the population ­ is not a good measure of change in the epidemic because (a) expanded access to ARV treatment that prolongs healthy life of infected people can cause prevalence to rise even if new infections are falling, and (b) different techniques for estimating prevalence (antenatal clinic surveillance and population-based testing) do not give comparable results. Prevalence in young people (say aged 15-19) is a reasonable proxy for incidence since most infections are recent. 15 Enabling Mechanisms Institutional Frameworks (legal, policy and programmatic) Monitoring and Evidence Building (surveillance, research, information systems) Critical Resources (funds, human resources, infrastructure, know-how) Partnerships and multi-sector engagement The selection of programmatic areas and interventions can be facilitated by applying the following questions: · What is the evidence saying? Is the epidemic generalized, concentrated or mixed? · What are the underlying social factors such as poverty, gender, discrimination? (sometimes also referred to as the drivers of the epidemic). · What are the key risk factors driving the epidemic? ­ behaviors such as injecting drugs or unprotected sex. · How effective has the current strategy been at responding to the epidemic? · What sectors need to support a national response? · Where are the current gaps in programming, implementation, evidence building or capacity? Phase 4: Select Critical Interventions for each Program and Cost Them The strategy should include the programmatic interventions that provide a logical "line of sight" to desired outcomes. This means that programs should be chosen to be funded and implemented if the program objectives and interventions would logically contribute to achieving the desired outcomes. There are limits to what can be done each year, and being selective and prioritizing is perhaps one of the hardest tasks. In addition to ensuring that target groups are clearly defined, the strategy planning team would want to select the priority program actions and interventions most likely to achieve the results. These are likely to include a combination of prevention (see figure 12), treatment, care and support actions. While prevention remains the thrust of a national program, treatment is an essential element. In addition to offering therapeutic benefits to HIV-positive individuals, enhanced access to ART offers important new opportunities to strengthen and expand HIV prevention efforts.23 A comprehensive treatment, care and support program would include actions such as: · Attending to the medical, psychological, socioeconomic, and legal needs of people affected by HIV/AIDS. · Supporting development of necessary laws and policies to enable ART. · Developing national HIV clinical management guidelines. 23WHO. National AIDS Programmes-A Guide to Indicators for Monitoring and Evaluating National Antiretroviral Programmes. WHO, 2005, p. 7. 16 · Developing, administering and ensuring adherence to therapy protocols. · Monitoring and managing drug tolerance and side effects. · Reliable laboratory up-grades and support to monitor treatment. · Ensuring a continuous stock of antiretroviral drugs for AIDS treatment and preventing mother-to-child transmission · Managing drug selection, supplies, storage and distribution. · Training medical staff on clinical ART management and treatment of opportunistic infections, patient counseling and treatment adherence. · Promoting public-private partnerships to extend treatment to employees of corporations and businesses and their dependents in the communities.24 Figure 12: Essential Programmatic Actions for HIV Prevention 1. Prevent the sexual transmission of HIV 2. Prevent mother-to-child transmission of HIV 3. Prevent the transmission of HIV through injecting drug use, including hard reduction measures 4. Ensure the safety of the blood supply 5. Prevent HIV transmission in health-care settings 6. Promote greater access to voluntary HIV counselling and testing while promoting principles of confidentiality and consent. 7. Integrate HIV prevention into AIDS treatment services 8. Focus on HIV prevention among young people 9. Provide HIV-related information and education to enable individuals to protect themselves from infection. 10. Confront and mitigate HIV-related stigma and discrimination 11. Prepare for access and use of vaccines and microbicides Source: UNAIDS, Policy Position Paper, Intensifying HIV Prevention, 2005, p.32. Specific interventions and activities can be described in more detail in the strategy's action or implementation plan. Action plans describe the interventions/activities likely to produce the expected results of the program, the inputs (financial and human) needed to implement services and activities, the timing, and who is responsible for each activity. Estimating the cost of planned interventions is an essential activity in the strategic planning process. Analysis of past expenditures and unit costs can help provide reasonable approximations of actual costs.25 Cost estimates allow an analysis of the gaps 24Compiled from USAID, HIV/AIDS Technical Areas Fact-Sheets: Treatment and USAID Programs Antiretroviral therapy. www.usaid.gov. 25Consult UNAIDS National AIDS Spending Assessment approach. 17 between available resources and resources needed for the programs. This is essential for effective resource mobilization and utilization.26 When interventions are costed and reviewed against available resources, it may be necessary to go back and re-prioritize key programs and interventions. Thus, Phases 2, 3 and 4 are iterative. It is advisable at this point to review the results table showing the outcomes and indicators identified earlier, to check whether the Results Framework needs any adjustment or refinement after the iterative process of programming, selecting/prioritizing, and costing. It is also critical to compare the costing of the strategic plan with the priorities identified in the text of the strategic plan. Interventions which are identified as having the highest priority should also receive priority for funding. If the costing shows that the allocation of resources is not consistent with the emphasis and prioritization described in the strategy and plan, the costing should be reviewed and made consistent with the plan. Analyzing resources needed against those available is an important part of the strategy formulation. Most likely, available financial resources will not be sufficient to meet the needs of the new strategy and programming period and there would be need to seek additional funding. Some countries start early on to mobilize resources by identifying possible sources of funding for which the country is eligible and plan an advocacy strategy accordingly. Sometimes it may be necessary to develop project proposals, together with agencies or institutions that are familiar with this process and/or will benefit from it. The country will have to demonstrate the absorptive capacity of the national HIV/AIDS program and indicate how national authorities will manage possible multiple sources of funding for particular components of the program; for example, by mapping how and where funding will be used to avoid duplication. It may also be necessary to indicate how funding from the different sources will be channeled to implementation partners and what tools already exist or will be developed to ensure accountability and transparency at national and decentralized levels. It is also important to describe on-going or planned efforts for strengthening donor coordination such as a contributing to a "common basket" for funding HIV/AIDS or harmonizing fiduciary mechanisms or procedures. A thorough resource mobilization and management plan is likely to be needed. Phase 5: Specify how Results will be Monitored and Resources Tracked At this stage the strategy planning team will discuss what data would need to be collected (i) routinely to follow progress, and (ii) periodically, to measure the indicators of progress made by the national response towards the performance targets previously set. Then, the team would discuss how the data will be collected and reported using existing surveillance systems, program data, periodic surveys or other means. Countries often find it useful to present this information in a table, like the template in figure 13. Both financial and programmatic data are necessary to monitor the national response. 26 UNAIDS, 2000a, Resource Mobilization Module 4. 18 Figure 13: Template for Summarizing Indicators and Targets Impact Baseline Performance Targets Data Compilation and and Value reporting Outcome (year) Indicators Yr1 Yr2 Yr3 Yr4 Disaggregated Data Entity Reporting by Sources Responsible Frequency for compilation Source: GAMET, GHAP, World Bank, 2007 This section of the strategy (or a related Monitoring and Evaluation Plan) would also describe how data flows among central, sub-national and community levels and among all sectors, and may specify the electronic data management system that is being used or would be installed to process and report data. Surveillance systems are an essential element of monitoring. Public heath first-generation surveillance helps define the nature and extent of HIV infection, and assess the impact of programs and services on HIV. This usually includes: (1) HIV patient case reports from clinical settings, and (2) sentinel sero-surveillance which uses blood samples. First generation HIV surveillance is necessary but not sufficient to assess the evolution of the epidemic because it records infections that have already taken place, but does not give early warning of the potential for infection. Second generation HIV surveillance expands the scope to include: risky behaviors (unprotected sex, injecting drug use), biological markers (STDs), knowledge (or lack) of how HIV is transmitted, and behavioral and other studies (i.e., DHS).27 The output of the strategic discussion about monitoring the national HIV/AIDS strategy would be an M&E section in the strategy document that may include tables such as the one shown in figure 14. 27UNAIDS and WHO, 2002 & 2006 19 Figure 14: Results Framework Template with an Illustrative Example -A Central Asian Country- Program area: Prevention Goal: Contribute to the control of the spread of HIV by strengthening cooperation between public and private sectors and NGOs. Intervention: Achieve high coverage of initiatives that will help contain the epidemic. Outcome-Results Baseline Performance Target Type and Frequency & Source Indicators value mid-point end-point Reference Responsi- (year) bility % female sex workers 69% 80% 90% UNGASS Annually Sentinel reached with HIV (2006) (2008) (2010) Surveillance prevention programs NAC/CDC % of men reporting 52% 60% 70% UNGASS Annually DHS or use of a condom the (2006) (2008) (2010) MDG secondary last time they had anal NAC/others surveillance sex with a male (behavioral partner survey) Source: GAMET, GHAP, World Bank, 2007. The thrust of phases 5 and 6 of the results Cycle is not to develop an M&E Plan, but to provide the parameters within which an M&E Plan can be developed and to agree on key principles. For instance: · To use information from the M&E system to inform the review of the national strategy. This would help to identify gaps in service delivery and learn which interventions work and which do not achieve the desired results. · To review the status and functioning of the M&E system itself periodically to maintain the relevancy and appropriateness of the system and maximize the use of the data it produces. · To ensure that national HIV/AIDS authorities coordinate partners' consultations and contributions to the national HIV/AIDS database. · To align the M&E plan to the national strategy priorities and expected outcomes. At this point in the strategy planning process it is appropriate to identify system issues and capacity weaknesses that need to be addressed to strengthen HIV/AIDS monitoring. Figure 11 shows the key components of a functional national M&E system, useful for identifying gaps in the national M&E system, or in a sub-system. How to fill those gaps in order to strengthen the system should be included in the strategy or implementation plan, listing key actions to be undertaken during the time period of the strategy, and indicators to measure progress in strengthening the system. 20 Figure 15: Components of a Functional HIV M&E System Source: Gorgens-Albino & Nzima, 2006. Historically, limited human and financial resources, multiple reporting demands from donors, and poorly developed national HIV information systems have restricted countries' capacity to use data. UNAIDS developed an information system that can be used to support monitoring and evaluation of national goals (there are also others). This system ­ the Country Response Information System (CRIS) ­ provides countries with a platform for storing data for indicators, projects and research. Monitoring of actual results against national targets needs to be strengthened with evaluation data to learn what works and what does not work, and why. Therefore the strategy would include an evaluation and research plan outlining the key epidemiological and behavioral surveys, operations research, impact evaluations, or programs reviews to be conducted during the strategy time period, to understand changes in the epidemic and the effects of the national response. This evidence-building plan would provide information on vulnerable groups and populations most at-risk to inform policy and programming. Especially for countries that do not have data on some of the key populations that drive their epidemics, it is of paramount importance to include evidence- building activities in the strategy. Figure 16 shows that even middle-income countries in Central America lack data on most-at-risk and most vulnerable populations. For instance, Honduras has no information on six groups (), very little information on three others () and reasonable information for only one sub-population (). 21 Figure 16: Availability of Information on Key Groups, Central American Countries Reasonable information available: Not enough information available: No information available: Studies being carried out**. Source : Bortman, M et al. 2006, p.10. Phase 6: Specify how Changes in the Epidemic will be Evaluated, Including the Results of the National Response Over the last decade, more emphasis has been placed on developing and using monitoring systems, particularly with requirements to measure UNGASS indicators every two years. However, evaluation of epidemic changes and the results achieved by national HIV/AIDS programs is less frequently undertaken even though this is essential for understanding what works and what does not work and for informing new or revised strategies. Many HIV and AIDS indicators depend on gathering information from individuals or households. Yet some countries do not address in their HIV/AIDS strategy whether surveillance data are available or what surveys (i.e., Demographic Health Survey) will be conducted during the strategy time period. The scaling up of treatment programs adds an urgent need for evidence to guide the delivery of ART on a large scale.28 In applying the Results Cycle, the planning team will have started with a review of existing information on the national HIV epidemic and the results and difference made (or not) by the national response. This will have identified gaps in available information. In phase 6 of the planning process, it is important to plan for key studies and evaluations 28Zewdie D, Piot P, and De Cock, K. (Eds.) op. cit. 2007. 22 that would provide evidence of results, help obtain baseline values or help understand what interventions are effective for different population groups. Epidemiologists and researchers can help develop the evaluation plan. If capacity needs to be strengthened in these areas, the strategy implementation and/or the M&E plan should address how the country will improve the situation. For instance, some countries have outsourced major evaluations in the short-term while staff increases their own skills in critical M&E areas. Thus, this section of the strategy should start by identifying major gaps in knowledge and then consider the appropriate studies (behavioral, epidemiological, operations research) and evaluations that would be conducted, including impact evaluations. Major barriers to research and potential solutions should also be addressed. (Appendix 3 elaborates). Often, strategies fail to indicate the evaluations that are needed or required. Two are essential: self-assessments though program reviews, and program evaluations. It is also important to identify whether independent evaluations are required by the government or by donors. In either case, independence ­not being involved in planning or implementing in any way­ and impartiality ­free of bias­ are pre-requisites for credible evaluations. Of course, these criteria do not guarantee the quality of the evaluation.29 The objective of both Phase 5 (Monitoring) and Phase 6 (Evaluation) is to generate strategic information to support policy making, strategic planning and programming to help achieve national goals. Strategic information is information on the epidemic and its drivers. Sources may include: · Surveillance and Surveys · Research (i.e., quality of care) · Policy and program documents · Analysis of existing research and programmatic data · Stakeholder consultations · Monitoring and evaluation · Desk reviews · Community response assessments Monitoring and evaluation should be able to provide policy-makers with responses to three broad questions: · Are we doing the right things? · Are we doing them right? · Are we doing them on a large enough scale? Phase 7: Use the evidence for the next strategy It was said early on in this Handbook that strategic planning is an iterative process. The Results Cycle is built on this fundamental concept. As strategic information about the 29 World Bank, 2007 23 epidemic and programs is generated, compiled and analyzed, policy makers and managers can use this information to drive programmatic improvements to better reach the population in need with HIV prevention, treatment, care and mitigation services. Data on the results of these efforts and their effects on the epidemic will be used to improve implementation and to inform the next strategy, in this iterative process (see figure 17). Figure 17: Analysis of Progress: ZANARA Logical Framework Narrative Key Performance (Measurable) Baseline Targets Results and Comments Summary Indicators (2002/03) (Sept 06) Strategic 1.Median age at first sex increased by 17.5% 18.5% For males the increase was Program 1 year for both male and female by (SBS 2003) (SBS 2006) from 17.5 to 19.5; for females Objective: 2008 (age group 20-49) it was from 17.5 to 18.5 (2003 and 2006 Sexual Behavioral Increased Survey) use of 2.Median age at first sex increased by 16.5% 18.5% For males the increase was HIV/AIDS 1 year for both male and female by (SBS 2003) (SBS 2006) from 16.5 to 18.5; for females prevention, 2008 (age group 15-24) it was from 16.5 to 18.5 (2003 care and and 2006 Sexual Behavioral impact Survey) mitigation 3.Percentage of teenagers aged 15-19 59.40% 31.60% This might indicate that programs years who are mothers or pregnant (SBS 2003) (SBS 2006) younger girls are not by the with their first child reduced from becoming sexually active at targeted 59.4% to 45% by 2008 an early age or are turning to vulnerable safer sex practices groups 4.Reported condom use at last sex Males:30%, males 41%, Performance targets were not with non-regular partner increased females females fully met. from 30% to 45% for males, and 17% 29% from 17% to 30% for females by (SBS 2003) (SBS 2006) 2008 (Age Group 15- 24) Source: Zambia Ministry of Finance and National Planning, 2006. 5. The Strategy Document The Results Cycle guides the formulation of a strategy that is informed by evidence and focused on results. It also guides the preparation of the strategy document, the basis for operational plans for implementing and monitoring the national response. The strategy document needs to present the information related to all phases of the Results Cycle in a specific and concise manner. The way information is presented to decision-makers and implementers affects its use, and there is little point in developing a strategy unless it is used. 5a. Main Sections of a Strategy Document The main sections of a results-based Strategy Document would include: · Situation analysis: A discussion of the characteristics and drivers of the epidemic, trends and how specific groups are affected. It would also include an overview of the national response to the epidemic, achievements and missed opportunities. This section is based on all possible available data. This discussion 24 needs to inform the selection of outcomes and the targeting of programs and interventions. · A results framework with outcomes, indicators, baselines and performance targets that are based on an understanding of the drivers, patterns and trends of the epidemic and evidence of programmatic achievements. · Description of priority programs and major interventions, including critical system strengthening activities. · Detailed but succinct information on how the progress and achievements of the national response at the central and decentralized levels will be monitored and evaluated. · Data: Clear information on data flows, including surveillance and community data, and on data information systems. Some strategies also indicate the information system used to manage routine data and track resources. · Research agenda: Specifics about the surveys, studies and research ­ including impact evaluations, annual program reviews and independent evaluations ­ to be conducted during the strategy period; and how capacity gaps will be addressed. · Description of how results will be reported and disseminated. · The costing of the strategy and information on available budgets with sources, if available. Experience in Madagascar, Mozambique, Tanzania and Uganda shows that results reporting and utilization is facilitated by: · Strategies and programs that are measured against results-based objectives and outcomes · Well-defined indicators · Reliable data · Context-appropriate monitoring systems, and · Analytical capacity to turn routine data and surveillance and research data into evidence to support decisions, including showing costs and benefits.30 Given this experience, it is important that the strategy document be a tool that guides reporting and the utilization of data on outcomes to facilitate and encourage managing for results. The strategy document should be reviewed against the benchmarks identified at the beginning; especially that the strategy is (i) based on evidence, and (ii) focused on results. 30White, Rodriguez-Garca and Balasundaran, 2005, pp. 33-43. 25 5b. Indications that a Strategy Document has a Results Focus Possible indicators that a Strategy Document meets the benchmarks mentioned above include: · The extent to which the strategy document describes the current situation of the epidemic and the effects of the national response. · The extent to which the strategy document describes expected outcomes for each programmatic objective. · The extent to which there is a results chain, linking activities and outputs to a sequence of logical outcomes. · The extent to which evidence on the epidemic, its drivers and at-risk-populations is carried through and informs programming, targeting and prioritization. · The extent to which programs and priority interventions are logically linked to the evidence and to key outcomes, as per the results chain. · The extent to which each outcome has at least one indicator, and each indicator shows a baseline value and performance target(s). · The extent to which the strategy document identifies evaluation activities and studies too be conducted. · The extent to which the strategy document describes how surveillance and routine data collection, management, reporting and utilization will be done. Other relevant indicators include: · The extent to which the strategy document recognizes the need for partnering with the private sector, NGOs, civil society, international partners and other stakeholders. · The extent to which the strategy document describes spending, costing and/or includes a budget. Once the strategy document is completed, it is likely to undergo final technical validation by government agencies in charge of implementing specific HIV/AIDS activities and other key players, especially the Ministry of Finance. The strategy would be utilized by the government for mobilizing resources from donors who support the national HIV/AIDS program. At the end of this process the Government approves and then publishes and disseminates the national strategy. 5c. Quality Enhancement Once the strategy document is completed, many countries choose to check the quality of the Strategy Document before finalizing it. This may entail: · Conducting a self-evaluation of the strategy31 31SAT= HIV/AIDS Strategy Self-Assessment Tool, www.worldbank.org/asap. 26 · Submitting the strategy to independent peer review32 Regardless of the action(s) taken, the strategy planning team will want to make sure the final document is reviewed by an editor before it goes to the printers; and that the final printed copy is delivered to the appropriate national authorities on time for formal dissemination. 6. Conclusion The Results Cycle is a tool that helps formulate a policy and/or plan an HIV/AIDS strategy, and prepare a strategy document. This planning should be supported by structured multisectoral involvement and a consultation process that involves stakeholders ­ elected officials, community leaders, public and private sector actors, donors, civil society, people with HIV and other stakeholders as appropriate. Consultation and participatory processes provide important information about realities and needs. They also add transparency and credibility to the planning and help mobilize resources and the commitment and "buy in" of implementers. The strategy planning team may need to be supported by people with specific expertise at critical phases, for instance to help compile existing data, prepare an evaluation plan, or estimate the costs of activities and the strategy. The advantage of using a planning model such as the Strategy Results Cycle is that sequencing, expectations and links to results can be made explicit. This improves management and productivity and fosters accountability (backward looking) and learning (forward looking). ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 32See footnote 6 and www.worldbank.org/asap 27 28 PLANNING AND MANAGING FOR HIV/AIDS RESULTS A HANDBOOK PART TWO ­ NOTES FOR TRAINERS The Results Cycle described in this Handbook is being used for training people responsible for formulating national HIV strategies in countries around the world. The set of slides in Part Three is also used, adapted and updated as appropriate to the profile of the participants and to the characteristics of the different countries and regions. For instance, slides are edited to add country- or region-specific examples. When using the slides for training/learning purposes, the following notes may be useful to trainers: · This Handbook for Planning and Results is a new product based on GAMET's Managing for Results approach. It fills a gap in technical support for the 3- Ones by introducing a results-focus approach to planning, monitoring and evaluation. · The Results Cycle is an organizing framework that guides and supports both the planning process and the production of the strategy document that will guide future implementation. It is a logical approach to improving policy development and strategic planning that bridges planning, monitoring and evaluation, and thus is more likely than less integrated approaches to help improve performance. · The Handbook will be more valuable to professionals who already have a working knowledge of key planning, monitoring and evaluation concepts and some working experience. · The Handbook can be used for coaching as well as a teaching module that focuses on planning, monitoring and evaluation that is results-focused and evidence- based. The audience for this module is the national team responsible for formulating the strategy and producing the strategy document. However, it can be useful for other audiences such as NGOs or implementing agencies who also develop strategies. · The Handbook ­ with the slide set ­ is not a trainer's guide. It provides the logic, core content and key messages for learning about: results-based strategies, planning strategically, monitoring and evaluation, and utilizing the information to improve policies and programs. · Trainers will want to adjust the delivery of the training to the different regions, given that each region and country is likely to exhibit an epidemic that has a 29 different epidemiological, behavioral, legal, economic and cultural profile. For instance, by adding more region-specific data slides or country-specific examples. · This Module does not repeat definitions and descriptions that can be found in the wealth of materials on the subject of planning published by UNAIDS, PEPFAR, WHO and other specialized agencies. · Each section concludes with a Break for questions and discussions, and exercises for group work. More can be added as needed. · As a basic overview, this module will take a minimum of 1 - 2 days, depending on how much time is devoted to exercises and group work. However, as a more in- depth workshop, it could take 3-5 days. For instance, it would take 5 days if participants are expected to draft an actual strategy. · If used for training or coaching purposes, a successful outcome of the training and learning would be demonstrated by the ability of participants to apply the Strategy Results Cycle in: 1. Completing the results framework for the strategy, and 2. Drafting the monitoring, evaluation and research sections of the strategy (but not in developing a full monitoring and evaluation plan), and/or 3. Drafting the strategy document. 30 PLANNING AND MANAGING FOR HIV/AIDS RESULTS A HANDBOOK PART THREE ­ POWERPOINT SLIDES This Part of the Handbook for Results Planning consists of a set of slides that further elaborates the results approach to planning strategically. The slides are organized in a logical manner following the seven phases of the Results Cycle. The Results Cycle is used as the organizing framework. The presentation moves through each of the stages of the Cycle with an opportunity for discussion and an exercise at the end of each phase which is linked to the preparation of the corresponding section for the Strategy Document. The slides in their entirety as well as selected sections have been presented several times and used successfully to support operations, program planning, and strategy formulation or plan how to build strategic information for managing for results. 31 Planning and Managing for Results Planning, Monitoring and Evaluation in HIV Policies and Programs R. Rodriguez-Garca and J. Z. Kusek. Planning and Managing for HIV/AIDS Results. A Handbook. Global HIV/AIDS Program, The World Bank, 2007. 1 CONTENTS Part 1: Part 3: · Overview and · Applying the Results Outcomes Cycle to Strategic · About Results Planning · Managing for Results Part 4: in Agencies and · Applying the Results Country Programs Cycle to the Strategy · About Results-based Document Planning Part 5: Part 2: · Key Messages · The Results Cycle · Reference web sites · Launching the Planning Process 2 32 Underpinnings This presentation Is grounded in planning, monitoring, measurement and evaluation principles Incorporates the "Management for Results" approach Applies the Results Cycle Enables participants to incorporate results- based M&E in HIV policy, strategy and program planning and design 3 Introducing the Results Cycle RESULTS CYCLE Applied to HIV Strategic Planning Use the Evidence for the NextStrategy Formulate/Revise 7 the HIV Strategy 1 Analyze HIV Specify Evaluation Epidemic and National Of Changes Response Data in the Epidemic 2 6 Identify Results Specify -Outcomes & Indicators- Monitoring of Results Select Critical 3 Interventions ­ Cost 5 Them ­ Identify Select Resources 4 Strategic Programs 4 33 Learning Outcomes Participants will be able to apply the Results Cycle to guide the formulation of a national policy, strategy or program. This will be demonstrated by the ability of participants to prepare: 1. a results framework, and 2. monitoring, evaluation and research sections of the strategic document. Note: Participants could use the strategy from their own countries. 5 Results: Key Questions What do we mean by results? · Sustained improvement in HIV outcomes at country level, in specific geographic areas and/or among at-risk groups (such as farm workers in border towns). It may include outputs. How do we get better results? · By increasing attention to results in the strategy planning process and in work plans · By measuring prevention, treatment, care and mitigation inputs, outputs and outcomes · By ensuring that results-based data are used for management, learning and decision making ­ not just reporting and accountability. 6 34 Managing for Results: Key Questions What do we mean by Managing for Results? Strategic planning and management that focuses on outcomes ­ rather than only on inputs and outputs. Management that is focused on improving performance. Using information to improve policy, programming and implementation towards achieving results. Why is Managing for Results Important? to improve strategy and programmatic decisions (management) to measure and report on performance (accountability) to identify and apply best practices (learning) Source: Self-Assessment in Managing for Results by R. Rodriguez-Garca and E.M. White. World Bank Working Paper # 41, 2005. 7 Box 2.1: Elements of Results Based Management 1. Identify clear and measurable objectives. St 2. Select indicators that will be used to measure Planning M Pe progress towards each objective. Pe 3. Set explicit targets for each indicator, used to rategic easur rform judge performance rform ement 4. Develop performance monitoring systems to ance regularly collect data on actual results. ance 5. Review, analyze and report actual results vis-à-vis M the targets. anagemen 6. Integrate evaluations to complement performance information not readily available from performance monitoring systems. t 7. Use performance information for internal management accountability, learning and decision making processes, and also for external performance reporting to stakeholders and partners. Source: A. Binnendijk, Results Based Management in the Development Co-Operation Agencies, OECD/DAC, 2001 Agencies use the Managing for Results approach ­ also called Results-based Management ­ to improve performance. 8 35 Results Framework in a World Bank Country Program POLICY Country Development OUTCOMES Goals MANAGEMENT Medium and Longer--term Outcomes PROGRAM Shorter-term Outcomes RESULTS MANAGEMENT Process Monitoring: Inputs, Activities and Outputs FINANCIAL MANAGEMENT Financial Monitoring The World Bank and partners support Country X towards achieving graspable policy and program results. Source: Managing for Results in a Fragile Post-Conflict Setting by R. Rodriguez-Garca, E.M. White and 9 R. Alonso I Terme, in Managing for Development Results Sourcebook, OECD-DAC, 2006. BREAK Use questions to: · Encourage participation, · Clarify any issue that may remain unclear for participants, and · Check participants' grasp of key concepts and messages. 10 36 RESULTS CYCLE Usethe Evidence for the Formulate/Revise NextStrategy 7 the HIV Strategy 1 Analyze HIV Specify Evaluation Epidemic and National Of Changes Response Data in the Epidemic 2 6 Identify Results Specify -Outcomes & Indicators- Monitoring of Results Select Critical 3 Interventions ­ Cost 5 Them ­ Identify Select Resources 4 Strategic Programs 11 The Strategy Results Cycle Helps Develop an HIV Policy or Strategy by focusing on: 1. Evidence on the evolution of the epidemic and its effects on specific population groups, and Evidence of the effects of the national response 2. Specific results to be achieved ­ outcomes with indicators, baselines and performance targets 3. Selection of programs and interventions 4. Clarity on how the national response will be monitored, measured, and evaluated. 5. Stakeholder participation and resource mobilization. 6. The multi-sectoriality of the HIV response. 12 37 Principles of Strategic Planning (SP) 1 Strategic Planning uses evidence and focuses on results that reflect the realities of the epidemic and the achievements of the national response. 2 Strategic Planning is not a linear process, it is an iterative process. It requires several phases ­ as shown in the Results Cycle ­ which reinforce each other. 3 Monitoring and evaluation is an essential mechanism of the national response ­ its planning, management, assessment and accountability. If results are not measured, managers cannot differentiate success from failure. 4. The strategy document is a "living document". It can and should be adjusted as needed during implementation, based on new data from monitoring, studies, evaluation and other sources. 13 Where Do We Start to Formulate or Revise the HIV Policy or Strategy? A Results- based approach drives accountability and performance Performance over time: Why is there a decrease? Strategic planning is an iterative process of reflection and analysis. Being "strategic" means being flexible and able to respond to change; and remaining relevant to the changing epidemic and its underlying causes. 14 Source: Guide to the Strategic Planning Process for a National Response, UNAIDS, August, 2000. 38 Planning Process based on the Strategy Results Cycle · Purpose · Methodology/process · Key persons & stakeholders · Links with other strategies · Finalize · Roadmap/timetable · Disseminate · Use plans · Monitor results · HIV/AIDS situation · Evaluate impacts · Current response · Capacity ­ available & needed · Data sources · Data collection · Analyze & use · Expected results · Strategic programs · Workplan · Responsibilities · Resources 15 Source: Adapted from P. Osewe, WBI, World Bank. Presentation in St. Lucia, 2006 Launching the Planning Process · Be systematic in applying the Strategy Results Cycle when planning for the strategy. · Be a catalyst for ideas and participation of stakeholders. · Be efficient by setting a time-table and agreeing on roles and responsibilities for completing the strategy. A new strategy may take between 4 and 8 months to complete ­ based on how much data are already available. · Be focused by ensuring that each phase of the Cycle concludes with a product ­ a section of the Strategy document. · DON'T be rigid. The phases of the Results Cycle build on each other ­ do not see them in isolation. Source: Adapted from Self-Assessment in Managing for Results by R. Rodriguez-Garca and 16 E.M. White. World Bank Working Paper 41, 2005. 39 Skills Mix of a National Strategy Planning Team - Illustrative - · Capacity and authority to make decisions and manage the planning process. · Knowledge of how government functions. · Ability to manage stakeholders consultations. · Ability to coordinate multi-sectoral and private sector inputs. · Ability to coordinate community and civil society inputs. · Knowledge of the HIV/AIDS situation (epidemiological, social behavioral, legal elements). · Program development and management experience. · Results-based planning, monitoring and evaluation expertise. · Financial management skills (expenditures, budgeting, costing). 17 Analyze HIV/AIDS Data and 11 National Response Data Start with a clear articulation of the present status of the epidemic: 1. What type of epidemic is evident in the country? 2. Where did the last 1000 or 100 infections come from? 3. Which are the most at-risk groups? Followed with: 4. Did the national program respond to the epidemic? Note: If data are not current or accurate, make sure that filling this gap is included as a output of the next strategy. 18 40 Evidence Building Uses Primary and Secondary Data Sources of Primary Data include: ­ Quantitative program data from services coverage. ­ Qualitative data from program staff, key informants and direct observation. ­ Surveys: demographic health surveys, epidemiological, behavioral and other studies (i.e., size estimation). ­ Public Health surveillance data. ­ Research and impact evaluations. Sources of Secondary Data include: ­ National response documentation, expenditures reports and program review reports. ­ Comparative national and global reports, such as: UNGASS, MDGs, UNAIDS, Global Fund, PEPFAR, World Bank and other studies (i.e., estimations, projections, trends). ­ Research articles published in peer-review, reputable journals. 19 What Type of Epidemic Exists in the Country? 1 of 2 ­ Epidemics are concentrated if transmission is mostly within specific groups, and if preventing new infections in those groups would slow or stop the epidemic spreading. ­ Conversely, epidemics are generalized if transmission is mainly in the general population, and would continue despite effective interventions to prevent infections in specific groups. ­ Most countries experience a mixed epidemic. Source: David Wilson Presentation, GHAP, World Bank, 2006. Also consult UNAIDS Practical Guidelines for Intensifying HIV Prevention, 2007. 20 41 What Type of Epidemic Exists in the Country? 2 of 2 · Countries may experience both type of epidemics ­ a mixed epidemic ­ at once in specific provinces or among different groups. · For instance, a country may have an epidemic concentrated around sex workers, but in a particular province it may be generalized to the entire population (e.g., a province in a transport corridor) 21 HIV INFECTION IN RUSSIA Maternal Hetero 2% MSM 5% 1% IDU 92% 22 Source: Russian Federal AIDS Center, 2002 42 HIV Infection in Zambia SEX WORKERS­6% SOLDIERS­2% GENERAL POPULATION­ 92% 23 Sources: Shields et al, 2004 HIV INFECTION IN MEXICO Blood Hemophilia/transfusion Perinatal coag. 6% 2% Men who have 4% sex with men 55% IDU 1% Homo Hetero- bisexu sexual 55% 32% 24 Sources: Padian and Bertozzi, 2005 43 Treatment is an Essential Part of National Response People on ARV therapy in developing countries 2002--2005 Thousands 1 500 North Africa and the Middle East Europe and Central Asia East, South and South -East Asia 1 000 Latin America and the Caribbean Sub-Saharan Africa 500 0 end mid- end mid- end mid- end 2002 2003 2003 2004 2004 2005 2005 Source: B. Schwartlander, Global Fund to Flight Aids, Tb and Malaria, 2006. 25 Summary Messages about Concentrated and Generalized Epidemics Concentrated epidemics Generalized epidemics Driven by sexual and injecting Driven primarily by sexual behavior in the practices, especially among HIV- general population vulnerable groups, including sex workers, men-having-sex-with-men and injecting drug users Require large-scale condom and Require large-scale, fundamental harm reduction programs (needle changes in community norms, sexual exchange and drug substitution) to values and practices and sexual protect HIV-vulnerable groups environments Prevention is the top priority Treatment and prevention are priorities What to do? What to do? Expanding coverage of proven Social and community change and interventions is vital safer sexual environments critical Source: David Wilson Presentation, GHAP, World Bank, 2006 26 44 Summary Messages About Evidence-Based Strategic Planning · A results-based HIV strategy is driven by the evidence on specific groups and behaviors that are spreading the epidemic, and on the geographic areas most affected. · Evidence -- epidemiological, behavioral, context and systems data -- also provides baselines on the situation. These baselines are used to set performance targets for the next strategy and to measure progress. · If current data are limited and there are few baseline values, the strategy should include a brief discussion of how baselines and performance data will be collected during the strategy period. Data sources are discussed under section 7: Evaluate changes in the epidemic. 27 Exercise Examine the next two data slides from Honduras, and: Explain what each figure shows about the epidemic and the national response. Explain the relationship between the data in the two slides, and explain the programmatic implications. Write up your answer in one paragraph. 28 45 Coverage of HIV Testing at Antenatal Services (Honduras 2007) 120,000 100,000 80,000 60,000 40,000 20,000 0 2001 2002 2003 2004 2005 2006 MECP MECP con Prueba Green= PMTCT Red= PMTCT with testing 29 National Antenatal Coverage of HIV Positive Women and HIV Prevalence in Newborns (Honduras 2007) 25% 20% 15% 10% 5% 0% 2001 2002 2003 2004 2005 2006 Prevalencia RN Cobertura VIH+ Red= Newborns 30 Blue= Women 46 BREAK Use questions to: · Encourage participation, · Clarify any issue that may remain unclear for participants, and · Check participants' grasp of key concepts and messages. · Go over definitions of key concept as needed 31 RESULTS CYCLE Usethe Evidence for the Formulate/Revise NextStrategy 7 the HIV Strategy 1 Analyze HIV Specify Evaluation Epidemic and National Of Changes Response Data in the Epidemic 2 6 Identify Results Specify -Outcomes & Indicators- Monitoring of Results Select Critical 3 Interventions ­ Cost 5 Them ­ Identify Select Resources 4 Strategic Programs 32 47 2 Identify Results ­ Outcomes and Indicators Follow the logical approach of the Results Chain -shown next- connecting inputs to outcomes, by: · Letting the expected outcomes drive the strategy, and · Using the evidence -- epidemic and national response data (biological, behavioral, epidemiological and programmatic) -- to select outcomes and performance targets. 33 The Logical Approach of the Results Chain Goal · Long-term, widespread improvement (Long-term in society Impacts) y "Big picture" (country longer term strategy) · Effects or behavior changes resulting Strateg Outcomes from a strategic program PLANNING Outputs · Products and services that LTSU need to be developed to achieve the expected outcomes RES Activities · What actually was done with the available resources to produce the intended outputs Programming Inputs · Critical resources (expertise, equipment, supplies) needed to 34 Source: Binnenen and Kusek&Rist, 2004. implement the planned activities 48 Apply the Results Chain to HIV Strategic Planning For a Results-based Strategy ask: 1. What longer term improvement in HIV is aimed at? (national goal) = impact 2. What improvements are aimed at by the end of the strategy period? = outcomes 3. How will one know success - which targets need to be met? = outcomes (i.e., coverage and utilization) 4. What will be achieved by implementing the interventions? = outputs 5. What financial, human, material, and technical resources are needed? = inputs Note: Refer to definitions in previous pp. and on pps 66 and 73. Also see terms on pp # 103. 35 Applying the Logic of the Results Chain to HIV IMPACTS Reduced HIV Incidence and Prevalence. HIV prevalence is the bedrock of surveillance, Strategy monitoring, and evaluation (longer-term goal) Increased Coverage. Access and use OUTCOMES of services and behavior changes (strategy objectives and results) ngOUTPUTS Products and Services. First level results needed to achieve the Programmi outcomes (program outputs) Critical resources. Includes money, people, equipment, supplies INPUTS and know how (program inputs) 36 49 Assessing the Success of the National HIV Policy or Strategy Once the results (outcomes) of the strategy are identified, how should they be measured? · HIV strategy results are measured by tracking outcome indicators. · For each indicator identify: - the baseline (the value now), and - targets (value to aim to achieve) over the time period of the strategy. For in-depth discussion about indicators consult UNAIDS Guidelines on Constructing Core Indicators, 2007. 37 Different Types of Indicators to be included in the HIV Strategy: Impact and Final Outcomes National Indicators are used for national and global reporting (UNGASS, MDGs, UNAIDS - including Universal Access) Final and Intermediate Outcomes Program Indicators are used for strategy and programmatic reporting to national authorities and donors. Intermediate Outcomes and Selected Final Outputs Selected Critical Interventions Indicators (e.g. approval of a policy, completion of a major study, or set up of a data system) are used for programmatic decision making. However, most of these types of indicators are more appropriate in the workplan. Inputs Financial resource allocation indicators may also be included. Note: Refer to Slide 37 and 38 for example of indicators at each level. 38 50 After Selecting Indicators for the Strategy, Establish Performance Targets Baseline Desired Target Indicator Level of Perform- Value + Improve- = ance ment Desired level Assumes a of finite and performance Current level expected to be reached of level of within a performance inputs, specific time activities, and outputs Condom use by SWs Current condom use Increase use of will reach 60% by the among sex workers is condom use by SWs end of the strategy 40% by 50% in five years time period 39 Source: Adapted from Ten Steps to a Results-based M&E System by J.Kusek and R. Rist, 2004, p. 91. Building Blocks for Measuring the Results of the HIV Strategy Example: Prevention - some commonly used indicators and indicative targets Outcomes Indicators Baselines Targets 1. IDUs will have better 1. Percent of IDUs with 20% in 50% access to needle access to needle 2005 by 2010 exchange programs exchange programs 2. Use of condoms by 2. Percent of SWs who 35% in 70% SWs will increase report using a condom 2004 by 2010 with their most recent client 3. Use of condoms by 3.1 Percent of men aged 48% in 60% young men and women 15-24 who report using a 2006 by 2010 will increase condom during last sex in the last 6 months. 3.2 Percent of women 23% in 50% aged 15-24 who report 2006 by 2010 using a condom during last (Source: R. Rodriguez- sex in the last 6 months. Garcia, GHAP, World Bank, 40 2007.) 51 Building Blocks for Measuring the Results of the HIV Strategy Example: Treatment and Care ­ some commonly used indicators and indicative targets Outcomes Indicators Baselines Targets 1. PLHIV will have better access 1. Percent of Adults and 10% in 2006 80% by 2010 to ART children with advanced HIV infection receiving ART 2. Improved hospital and clinic 2. Percent of health care 35% in 2006 95% by 2010 capacity to treat HIV disease facilities that have the capacity and conditions to provide basic- level HIV testing and HIV clinical management 3. Better management of 3. PLHIV receiving diagnosis -20% in 2006 -90% by 2010 opportunistic infections and treatment for opportunistic infections (percentage) 4. Reduced AIDS mortality 4. Adults and children with HIV 50% in 2006 10% in 2010 still alive 12 months after initiation of ART (extend to 2, 3, 5 years as program matures) (percentage) 5. First line ART regimen at 5. Percent Adults and children 70% in 2006 85% in 2010 optimum durability with HIV remaining on First line ART 24 months after initiation 41 Source: R. Oelrichs, GHAP, World Bank, 2007. When regular results measurement suggests actual performance diverges sharply from planned performance ....... ......... it is time for Evaluation and Research Planned Actual Source: J.Kusek and R. Rist, 2004, p. 118. 42 52 Summary Messages about Outcomes and Indicators 1of 2 1. M&E informs the Strategy Results Cycle by helping to identify strategic results in terms of expected outcomes. 2. Outcome indicators are used to measure the results of the national HIV strategy. 3. Quantitative and qualitative targets (expected situation) measure the performance of the strategy against baselines (current situation values) for each indicator. 4. At the strategy level, the focus is on outcomes and selected milestones or final output indicators. 43 Summary Messages about Outcomes and Indicators 2 of 2 5. The strategy may also include longer-term impact and outcome indicators used for national and global reporting (such as MDGs or universal access). 6. Measuring indicators will require a combination of data, including: programmatic, tracking, surveys, surveillance, program reviews, impact evaluation and/or research studies. 44 53 Exercise Based on the characteristics of the HIV epidemic in your country, use the template shown on the next graph to prepare the Results Framework: Identify impact and outcome indicators to be monitored at the strategy level. Indicate baselines and performance targets for each indicator. Disaggregate by gender, age and location. Note: Data sources and responsibilities can be completed later as another exercise using the template shown in the monitoring section 45 Results Framework Template Applying the Results Chain Expected Outcome IMPACTS Outcomes Indicators Performance (Results) Baseline (year) targets Example: Increase Improve life utilization of expectancy by.. testing ...lowering the services by (a) incidence sex workers. and (b) prevalence of HIV. 46 54 BREAK Use questions to: · Encourage participation, · Clarify any issue that may remain unclear for participants, and · Check participants' grasp of key concepts and messages. 47 RESULTS CYCLE Usethe Evidence for the Formulate/Revise NextStrategy 7 the HIV Strategy 1 Analyze HIV Specify Evaluation Epidemic and National Of Changes Response Data in the Epidemic 2 6 Identify Results Specify -Outcomes & Indicators- Monitoring of Results Select Critical 3 Interventions ­ Cost 5 Them ­ Identify Select Resources 4 Strategic Programs 48 55 Select Strategic Programs 3 In selecting broad programmatic areas, ask the following questions: · What is the evidence saying? Is the epidemic generalized, concentrated or mixed? · What are the key drivers of the epidemic? ­ social factors such as poverty, gender, discrimination. · What are the key risk factors? ­ behaviors such as injecting drugs or unprotected sex. · How effective has the current strategy been at responding to the epidemic? · What sectors need to support the national response? · Where are the current gaps in programming, implementation, evidence building or capacity? 49 The evidence on the epidemic should help focus the strategy towards the appropriate programs Programs HIV Prevention aimed at key drivers (knowledge, behavior) AIDS Treatment Care and Support (coverage and utilization = access) Impact Mitigation (coverage and utilization = access) Enabling Mechanisms Institutional Frameworks (Legal, policy and programmatic) Monitoring and Evidence Building (surveillance, research, information systems) Critical Resources (funds, human resources, infrastructure, know-how) 50 56 HIV INFECTION IN GHANA GENERAL POPULATION 15% SEX WORKERS 85% 51 Sources: Cote et al, 2004 Select Strategic Programs by being Clear about Desired Expected Results Example: Prevent HIV Infections RE If not infected, prevent people GNINNALP from getting infected SULTS If infected, prevent people from getting sick PLANNI If sick, prevent deaths STLUSER NG through treatment If on treatment, prevent drop- outs and drug resistance If there are deaths, mitigate the social effects 52 57 Knowing the Trends and Drivers of the Epidemic Supports Rational Allocation of Resources For instance, In the 1990s, Ethiopia believed their HIV epidemic to be generalized and thus invested heavily in treatment. New evidence later showed that Ethiopia's epidemic is concentrated around sex workers. The strategy needed to be revised to ensure prevention activities were funded to achieve high coverage of sex workers and their clients. Note: The HIV prevalence rate in adults aged 15-49 in Ethiopia is 0.9-3.5%. Source: UNAIDS 2006 Report 53 Lessons Learned from Ethiopia · Evidence helped adjust the HIV national strategy and achieve better results · Data were collected on HIV trends and patterns · Ethiopia was proactive and consulted the evidence to make policy and program choices 54 58 Exercise Considerations for selecting the broad strategic programs of the strategy · If the epidemic in your country looks like the ones in the next graphs, what would be the most appropriate programs? Which programs and interventions would you emphasize for (a) each risk group and (b) for the population? Which programs/interventions would you NOT use and why? 55 Secondary HIV Prevention: The Difference Between USA and CHINA Know HIV Status Don't Know HIV Status 1,200,000 1,000,000 800,000 600,000 400,000 200,000 0 56 Source: Yip, 2005 USA China 59 Make Sure Resources Support Priority Programs and Interventions Example: In this West Africa country, there was a misalignment between the cause of infections and the resources directed to at interventions for high-risk groups. Resources Infections Infections from high risk partnerships 57 Sources: MAP review, 2004; GARFUND, 2004. BREAK Use questions to: · Encourage participation, · Clarify any issue that may remain unclear for participants, and · Check participants' grasp of key concepts and messages. 58 60 RESULTS CYCLE Use the Evidence for Formulate/Revise 1 the Next 7 HIV Policy or Strategy Strategy Analyze HIV and National Response Specify Evaluation Data Of Changes in the Epidemic 2 6 Identify Results Select Critical Specify -Outcomes & Indicators- Interventions ­ Cost Monitoring Them ­ Identify of Results Resources 3 5 4 Select Strategic Programs 59 Select Multisectoral Program Interventions 4 1 of 2 In response to the type and pattern of HIV epidemic experienced by the country, the strategic programs are selected by deciding: the appropriate combination of and emphasis on: ·Prevention, ·Treatment, and ·Impact Mitigation Programs For instance, in a concentrated epidemic, prevention would be the major focus. Generalized epidemics also need strong prevention, but are likely to need substantial treatment and mitigation programs as well. For more in-depth discussion of this topic consult UNAIDS Practical Guidelines for Intensifying HIV 60 Prevention, 2007. 61 Select Multisectoral Program Interventions - Identify Activities and Outputs - Cost Them - Identify Resources 2 of 2 1. For each supporting sector, select the program interventions ­ matched to the country's epidemic For instance: · Prevention Condom distribution & use; IDU harm reduction · Treatment Coverage of ART · Impact Mitigation Orphans and PLHIV programs 2. Cost each intervention based on actual unit costs and/or average costs as proposed by UNAIDS. 3. Identify the resources available and needed. The strategy is a key tool for mobilizing resources. Note: The interventions would be described in detail in the implementation workplan to be developed after the strategy is defined. 61 Deciding on Program Interventions Example: Treatment Program If you want better outcomes of the Treatment Program, improve: GNINNALP · infrastructure & capacity to diagnose RESULTS · access to diagnostics · treatment coverage & adherence to treatment STLUSER PLANNING · management of drug resistance · mitigation programs 62 62 Summary Messages on Elements of Success in Programming · Commitment of all the major stakeholders · Capacity to implement · Clarity of objectives and results · Roles and responsibilities are clearly articulated · Stakeholder involvement · Readiness for implementation · Identification & mobilization of funds and other resources Source: Adapted from J.C. Brown, D. Ayvalikli and N. Mohammad, 2004, Transforming Bureaucrats into Warriors, p.5. 63 Exercise If the epidemic in your country is concentrated among SWs and IDUs - as shown on the next graph - one expected outcome of the strategy is to stop the link of sexual work and drug use. What would be the focus of strategic programming and why? What would be the key interventions? What will the costs be? Are the interventions feasible with the resources available? 64 63 IDUs Trigger HIV Infection among 70 Sex Workers in East Asia 60 50 40 30 20 10 0 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 Guangxi, IDU Guangxi, sex workers Hanoi, IDU Hanoi, sex workers Jakarta, IDU Jakarta Sex workers 65 Sources: Pisani et al, 2005 BREAK Use questions to: · Encourage participation, · Clarify any issue that may remain unclear for participants, and · Check participants' grasp of key concepts and messages. 66 64 RESULTS CYCLE Use the Evidence for Formulate/Revise 1 the Next 7 HIV Policy or Strategy Strategy Analyze HIV and National Response Specify Evaluation Data Of Changes in the Epidemic 2 6 Identify Results Select Critical Specify -Outcomes & Indicators- Interventions ­ Cost Monitoring Them ­ Identify of Results Resources 3 5 4 Select Strategic Programs 67 5 Specify how Results will be Monitored After the expected results, outcomes and performance targets have been identified - followed by key strategic programs and critical interventions - the HIV Strategy planning process continues by considering how data will be collected, processed, analyzed and reported, including surveillance data (See table on next slide) Note: See Phase 6: Evaluating the Changes in the Epidemic 68 65 Data Collection Methods and Instruments Consider cost, time, and expertise requirements Key informant Routine Panel interviews facilities Surveys Field Conversation Focus data experiments with Group relevant Interviews One-Time individuals Survey Impact Participant Evaluation Observation Direct Community Census observation Interviews Reviews of official records (MIS and Field admin data) visits Questionnaires Informal/Less Structured Methods More Structured/Formal Methods Source: J. Kusek and R. Rist, 2004. 69 Global HIV/AIDS M&E Framework & Illustrative Data Assessment Inputs Activities Outputs Outcomes Impact & Planning (Resources) (Interventions, (Immediate (Intermediate (Long-term Services) Effects) Effects) Effects) Situation Analysis Staff Trainings # Staff Trained Provider Behavior Social Norms Response Analysis Funds Services # Condoms Distributed Risk Behavior HIV prevalence Stakeholder Needs Materials Education # Clients Served Service Use STI Incidence Resource Analysis Facilities Treatment # Tests Conducted Clinical Outcomes AIDS Morbidity Collaboration plans Supplies Interventions Quality of Life AIDS Mortality Econ. Impact Program Program-based Data Population-based Biological, Development Behavioral & Social Data Data In addition to monitoring programs, conduct process and outcome evaluations Source: D. Rugg, G. Peersman, G. & M. Carael in Advances in Global HIV/AIDS Monitoring and Evaluation: New Directions in Evaluation, UNAIDS, 2004. 66 First Generation HIV Surveillance is Necessary to Assess the Evolution of the Epidemic Public Heath surveillance helps: Types of Surveillance: - define the nature and - HIV patient case extent of HIV reports from clinical infection, and settings - assess the impact - Sentinel sero- that programs and surveillance which services have on the uses blood samples HIV problem. Surveillance is the bedrock of public health monitoring 71 Source: Compiled from UNAIDS 2006 Report on the Global AIDS Epidemic, pp 278-279. First Generation HIV Surveillance is Necessary but not Sufficient to Assess the Evolution of the Epidemic · It records infections that have already taken place, but does not give early warning of the potential for infection · Second Generation HIV Surveillance expands the scope to include: · At risk behaviors (unprotected sex) - Biological markers (STIs) - Knowledge (or lack) of how HIV is transmitted - Behavioral and other studies (DHS) 72 Source: Compiled from UNAIDS 2006 Report on the Global AIDS Epidemic, pp 278-279. 67 Exercise Based on the characteristics of the HIV epidemic in your country, complete the template shown on the next graph to: Reflect the indicators of the Results Framework Disaggregate by gender, age, location, or other criteria. Indicate data compilation responsibilities and reporting frequency. 73 Indicators Data Collection Template Impact & Baseline Outcome Value Performance Targets Data Compilation Indicators (year) and Reporting YR1 YR2 YR3 YR 4 Disaggr Data egated Source/ by s Entity Reporting responsible Frequency for Compiling data Source: Adapted from M. Görgens-Albino, GAMET, GHAP, World Bank, 2007 68 STRATEGY RESULTS CYCLE Usethe Evidence for Formulate/Revise 1 theNext 7 HIV Policy or Strategy Strategy Analyze HIV and National Response Specify Evaluation Data Of Changes in the Epidemic 2 6 Identify Results Select Critical Specify -Outcomes & Indicators- Interventions ­ Cost Monitoring Them ­ Identify of Results Resources 3 5 4 Select Strategic Programs Evaluate Changes in the Epidemic by Asking Fundamental Questions 6 "Why" Questions ­ Why has the epidemic evolved in a particular way? Causality is difficult to determine "How" Questions ­ How did the sequence or processes lead to successful (or not) outcomes of the national HIV response? "Compliance/ ­ Did the promised activities actually take Accountability Questions" place and as they were planned? Process/ Was the implementation process followed Implementation Questions as anticipated, with what consequences? At what cost? Source: Adapted from J. Kusek and R. Rist, 2004. 76 69 A Public Health Questions Approach to HIV/AIDS M&E Are we doing Are collective efforts being implemented on a large Determining Collective OUTCOMES them on a large enough scale to impact the epidemic? (coverage; Effectiveness & IMPACTS enough scale? impact) Surveys & Surveillance OUTCOMES Are interventions working/making a difference? Outcome Evaluation Studies Are we Are we implementing the program as planned? doing them Monitoring & Evaluating OUTPUTS Outputs Monitoring right? National Programs ACTIVITIES What are we doing? Process Monitoring & Evaluation, Quality Assessments What interventions and resources are needed? Are we INPUTS Needs, Resource, Response Analysis & Input Monitoring doing the Understanding What interventions can work (efficacy & effectiveness)? right Potential Responses Special studies, Operations research, Formative research & Research synthesis things? What are the contributing factors? Determinants Research Problem What is the problem? Identification Situation Analysis and Surveillance Source: Rugg, Peersman, & Carael, 2004. Some Examples of Results Monitoring Infant Health Girls Education Policy Decreasing Infant Increasing girls Monitoring Mortality Rates education attainment Program Clinic-based pre-natal # of girls in secondary Monitoring care is being used by schools completing math pregnant women and science courses # of girls in four urban Project Information on good neighborhoods Monitoring pre-natal care provided in 6 targeted villages completing primary education The results approach can be applied to all sectors 70 Definition of Results-Based Evaluation Results-Based Evaluation is an assessment of a planned, ongoing, or completed intervention to determine its relevance, efficiency, effectiveness, impact and sustainability. The intent is to incorporate lessons learned into the decision- making process. Designing Good Evaluations · Getting the questions right is critical · Answering the questions is critical · Supporting public sector decision- making with credible and useful information is critical 71 Designing Good Evaluations "Better to be approximately correct than precisely wrong." Paraphrased from Bertrand Russell Evaluating Changes in the HIV Epidemic Requires at least Two Data Points Two data points allow comparisons of present (performance) to past data (baseline) to look for trends and other changes. One data point is not enough cesscA ? cesscA Time Time Improving access to Improving access to HIV testing HIV testing Source: Adapted from J. Kusek and R. Rist, 2004, p. 111 82 72 Evaluation and Research Types of Evaluation: After identifying · Program performance the fundamental questions to be · Implementation Process studied, choose · Impact of interventions the type of evaluation most · Case analysis appropriate to · Synthesis study answering the questions · Epidemiological and behavioral surveys · Cost effectiveness analysis · Client satisfaction survey 83 Agencies often Identify Types of Evaluations that are more Relevant to their Mission and Work Programs Theory-based evaluation Formal surveys Rapid appraisal methods Participatory methods Public expenditure tracking surveys Cost-benefit and cost-effectiveness analysis Impact evaluation Source: Independent Evaluation Group, World Bank, 2000. 84 73 Some Examples of Evaluation Privatizing Water Systems Resettlement Comparing model Comparing strategies Policy approaches to used for resettlement of Evaluations privatizing public water rural villages to new supplies areas Assessing the degree to Program Assessing fiscal which resettled village Evaluations management of government systems farmers maintain previous livelihood Assessing the Project improvement in water Assessing the farming Evaluations fee collection rates in 2 practices of resettled provinces farmers in one province Complementary Roles of Results-Based Monitoring and Evaluation Monitoring Evaluation Clarifies program objectives Analyzes why intended results were or were not achieved Links activities and their Assesses specific causal resources to objectives contributions of activities to results Translates objectives into Examines implementation process performance indicators and set targets Routinely collects data on Explores unintended results these indicators, compares actual results with targets Reports progress to Provides lessons, highlights managers and alerts them to significant accomplishment or problems program potential, and offers recommendations for improvement 74 Summary Messages: When to use Program Monitoring and When to use Evaluation Monitoring Evaluation Routinely collects data on Analyzes why intended results indicators, compares actual were or were not achieved results with targets Links activities and Assesses specific causal resources to their objectives contributions of activities to results Translates objectives into Examines implementation process performance indicators and set targets Clarifies Program Explores unintended results Objectives Reports progress to Provides lessons, highlights managers and alerts them significant accomplishment or to problems program potential, and offers recommendations for improvement Source. Kusek and R. Rist, 2004. DECLINING HIV PREVALENCE IN THE CARIBBEAN 7 6.2 6 5 4 3.1 3 3 2 2 2 1.4 1 0.7 0.3 0 Haiti DR Bahamas Barbados 1993-1999 2004 88 Source: UNAIDS AIDS Epidemic Update, 2004 and 2005. 75 Two Data Points Are Necessary But Not Sufficient For Programmatic Decision-making The more data points are available, the easier it is to make appropriate decisions and determine trends. So on-going surveillance and programmatic monitoring is essential. cesscA ? cesscA Time Time Improving access to Improving access to HIV testing HIV testing 89 EXERCISE Review this graph with two Declining HIV Prevalence in The Caribbean data points per country and explain: 7 6.2 6 · Possible reason/s for the 5 decrease in HIV 4 prevalence. 3.1 3 3 2 2 2 · Key messages that matter 1.4 1 0.7 at the national level. 0.3 0 Haiti DR Bahamas Barbados · Implications for national and international partners. 1993-1999 2004 90 76 SUMMARY MESSAGES: A Results Based HIV Policy or Strategy that is Driven by Evidence: 1. Supports national HIV prevention and treatment efforts 2. Helps formulate and justify budget requests 3. Helps provide services more efficiently 4. Helps allocate resources 5. Monitors the performance of contractors and grantees 6. Motivates personnel to continue making program improvements 7. Facilitates continuous monitoring and in-depth program evaluations 8. Triggers in-depth examinations of performance issues 9. Responds to demands for accountability 10. Communicates better with the public and stakeholders to build public trust. 91 BREAK Use questions to: · Encourage participation, · Clarify any issue that may remain unclear for participants, and · Check participants' grasp of key concepts and messages. 92 77 THE STRATEGY DOCUMENT How does one know by reading the Strategy document that the Strategy is Results-based? Because the Strategy Paper: 1. Uses the evidence to select goals, targets, populations and programs. 2. Includes a Results Framework, with outcomes indicators, baselines and performance targets. 3. Addresses strategic Information ­collecting, compiling, finding, analyzing, processing, reporting, disseminating, using. 93 THE STRATEGY DOCUMENT · The Strategy Results Cycle guides the formulation of the strategy and also the preparation of the Strategy Paper. · The Strategy Paper needs to present information related to all phases of the Results Cycle in a specific and concise manner. · The evidence, results framework, programs, populations and plans for monitoring, evaluation and research are the main elements of the Strategy Paper. 94 78 THE STRATEGY DOCUMENT How Information is presented to Decision-Makers Makes a Difference to its Utilization When preparing the Strategy document, sort out key findings and priorities, including: Epidemiological and behavioral changes and trends since the last strategy and reflect them in programming. Identify issues that are time-sensitive or require decisions. Focus on key messages that matter at the national and sub- national levels. Consider implications of program priorities for partners and stakeholders. Identify resources ­ available and needed ­ identify financing gaps and look for ways to fill them or ensure that the most pressing activities will be funded first. 95 THE STRATEGY DOCUMENT Lessons Learned from Reporting Results to Decision-makers Results Reporting Experience in Madagascar, Mozambique, Tanzania and Uganda shows that robust reporting and utilization is facilitated by: · Strategies and programs that are measured against results-based objectives and outcomes · Well-defined indicators · Reliable data · Context-appropriate monitoring systems · Analytical capacity to turn routine data, surveillance and research data into evidence to support decisions, including cost benefits Source: Harmonization around Results reporting by E.M. White, R. Rodriguez-Garca and R. 96 Balasundaran, in Sustainable Development Law and Policy, Vol. VI, Issue 1, Fall 2005, pp. 33-43. 79 A Results-based STRATEGY DOCUMENT includes: · A results framework with the outcomes, indicators, baselines and performance targets, that is based on evidence of epidemiological changes and programmatic achievements. · Detailed but succinct information on how the achievements of the national response at the central and province levels will be monitored and evaluated. · Clear information of data flows, including surveillance and community data. Addressing data information systems. · Specifics about the surveys, studies and research ­ including impact evaluations ­ to be conducted during the strategy time period. · How results will be reported and disseminated. · How capacity gaps will be addressed. 97 Summary Messages... RESULTS CYCLE Use the Evidence for Formulate/Revise 1 the Next 7 HIV Policy or Strategy Strategy Analyze HIV and National Response Specify Evaluation Data Of Changes in the Epidemic 2 6 Identify Results Select Critical Specify -Outcomes & Indicators- Interventions ­ Cost Monitoring Them ­ Identify of Results Resources 3 5 4 Select Strategic Programs 98 ...The Cycle helps countries to use evidence for policymaking and programming 80 Summary Messages: A Results-based Strategy document Includes: · A discussion of the evidence · Surveillance systems linked to the selection of · Epi-behavior surveys programs and interventions · Evaluation (+ impact · A Results Framework evaluation) with outcomes, indicators, baselines and targets · Research (operations, treatment protocols) · Monitoring (epidemic, program, services, funds) · Reporting and dissemination · Data flows & levels · Governance and · Data collection and participation Compilation by outcome indicator 99 Finishing Exercise · Participants divide into several small groups. · There will be at least two scenarios: One country with a generalized epidemic and another country with a concentrated epidemic. · The focus of the exercise is to: 1. Prepare the results framework, and 2. Draft the M&E/Research section of the 100 strategy ­ indicating how capacity gaps fill 81 BREAK Use questions to: · Encourage participation, · Clarify any issue that may remain unclear for participants, and · Check participants' grasp of key concepts and messages. 101 SUMMARY: Take Home Messages 1 of 2 The strategy document is a living document. It can and should be revised as needed during implementation, in response to new monitoring data and research findings. Strategic planning is not a linear process. It is iterative; it involves a series of phases - as shown in the Results Cycle - which overlap. Information is more likely to be used when results are analyzed and findings are synthesized and reported. By using evidence, the strategy planning process creates demand for quality information. 102 82 SUMMARY: Take Home Messages 2 of 2 Strategic planning is about using evidence and aiming at results that address the realities of the epidemic. M&E is a tool for strategic planning, management and learning --If you do not measure results, you can not tell success from failure. Monitoring and evaluation are two separate, but interrelated strategies to collect data and report the findings on how well (or not) the national HIV response is performing. Evaluations (programmatic, impact), epidemiological and behavioral studies, and research are essential components of strategic planning, monitoring and evaluation. 103 BREAK Use questions to: · Clarify any issue that may remain unclear for participants. · Ensure participants' grasp of key concepts and messages. · Encourage participants to use and apply what they have learned. 104 83 As the HIV National Response Program is Implemented ..... · Make adjustments based on evidence of epidemiological changes, programmatic results, and research findings. 105 Core Results Terminology · Input Financial, human and material resources used for an intervention · Output Products, goods and services which result from an intervention · Outcome Likely or achieved effects or behavior changes resulting from an intervention · Performance Degree to which an intervention or partner operates according to specific standards or criteria · Result The outcome or impact of an intervention. It may include outputs. · Results The causal sequence for an intervention to achieve Chain impacts, moving from inputs and activities to outputs, outcomes and impacts. Source: OECD, IEG. For more definitions refer to UNAIDS. 106 84 For more information, see · UNAIDS www.unaids.org · United Nations Development Programs www.undp.org · Word Bank Global AIDS Program www.worldbank.org/aids · World Health Organization AIDS Program www.who.org 107 85 References and Resources ASAP. Self-Assessment Tool and Guidelines for National HIV/AIDS Strategies, 2007 (updated) at: www.worldbank.org/asap Bertozzi S, Padian NS, Wegbreit J, et al. HIV/AIDS Prevention and Treatment. In: Disease Control Priorities in Developing Countries. April 2006. Bertrand J. T. and M. Solis. Evaluando Proyectos de Prevencion de VIH/SIDA. Measure Evaluation, 2000. Binnendijk, A. Results-Based Management in the Development Cooperation Agencies: A Review of Experience. Background Report, DAC OECD Working Party on Aid Evaluation, Paris, 2000. Retrieved (May 2, 2005) at http://www.oecd.org/dataoecd/17/1/1886527.pdf Bortman, M. et al. Reducing HIV/AIDS Vulnerability in Central America. World Bank, Washington DC, 2007. www.worldbank.org/AIDS > publications Brown, J.C., D. Ayvalikli and N. Mohammad. Transforming Bureaucrats into Warriors. World Bank, 2004. www.worldbank.org/AIDS > publications Global Fund to Fight AIDS, TB and Malaria. Investing in Impact, Mid-Year Results Report. The Global Fund, 2006. www.tgfatb.org Global Fund to Fight AIDS, TB and Malaria. Performance Measurement ­ What the Global Fund Measures and How. Accessed on line, 2006. Global Task Team on Improving Aid Coordination Among Multilateral Institutions and International Donors. Final Report, UNAIDS, Geneva, 14 June 2005. Gorgens-Albino, M. & M. Nzima. Eleven components of a fully functional HIV M&E system. Getting Results, GHAP, World Bank September 2006. www.worldbank.org/AIDS > Getting Results Inter-American Development Bank. The Politics of Policies, Economic and Social Progress in Latin America. Inter-American Development Bank, 2006. Kusek, J. Z. and R. Rist. Ten Steps to a Results-based M&E System, World Bank, 2004. Retrieved (June 1, 2007) English (also available in French and Spanish) http://imagebank.worldbank.org/servlet/WDSContentServer/IW3P/IB/2004/08/27/00016 0016_20040827154900/Rendered/PDF/296720PAPER0100steps.pdf 86 Kusek, J.Z., R.C. Rist and E. M. White. How Will we Know Millennium Development Results when we See Them? Africa Region Working Paper # 66, The World Bank, Washington DC, May 2004. Retrieved (June 1, 2007) http://imagebank.worldbank.org/servlet/WDSContentServer/IW3P/IB/2005/01/12/00009 0341_20050112153351/Rendered/PDF/312680Millennium0Dev0results0AFR0wp66.pdf Measure Evaluation/USAID. Data Demand and Information Use in the Health Sector, Strategies and Tools. (Not dated.) OECD-DAC and World Bank. Managing for Development Results Principles in Action - Sourcebook on Emerging Good Practice. OECD, Second edition, Paris, 2007. Retrieved (June 1, 2007) http://www.mfdr.org/Sourcebook.html OECD (Organization for Economic Cooperation and Development). Glossary of Key Terms in Evaluation and Results Based Management. Paris, OECD/DAC, 2002. Oelrichs, Robert. Personal communication. Global HIV/AIDS Program, World Bank, 2007. Osewe, Patrick Lumumba. Presentation in St. Lucia, 2006, World Bank Institute, World Bank. Rehle, T., T. Saidel, S. Mills and R. Magnani (eds). Evaluating Programs for HIV/AIDS Prevention and Care in Developing Countries. Family Health International/USAID, 2001. Rodriguez-Garca, R. and E.M. White. Self-Assessment in Managing for Results.. World Bank Working Paper # 41, Washington DC, 2005. Rodriguez-Garca, R., E.M. White and R.Alonso I Terme. Managing for Results in a Fragile Post-Conflict Setting in Managing for Development Results Sourcebook, OECD- DAC, 2006. Rugg, D., G. Peersman and G. & M. Carael (eds). "Global Advances in Global HIV/AIDS Monitoring and Evaluation" New Directions in Evaluation, 103, UNAIDS, Fall 2004. Schwarzlander, B. (The Global Fund) presentation in "Sustaining Treatment Costs - Who will pay?" World Bank event, Washington DC, November 2006. UNAIDS et al. National AIDS Programmes: A Guide to Monitoring and Evaluation. UNAIDS and partners, 2000. http://www.who.int/hiv/pub/me/pubnap/en/index.html UNAIDS. Guide to the Strategic Planning Process for a National Response, UNAIDS, Geneva, 2000. 87 UNAIDS and the World Bank. National AIDS Councils - Monitoring and Evaluation Operations Manual. UNAIDS and the World Bank, Geneva and Washington DC, 2002. UNAIDS. AIDS Epidemic Update 2004. UNAIDS Geneva, 2004. UNAIDS. AIDS Epidemic Update 2005. UNAIDS Geneva, 2005. UNAIDS. National AIDS Spending Assessment (NASA): A notebook on methods, definitions and procedures for the measurement of HIV and AIDS financing flows and expenditures at country level. UNAIDS, Geneva, 2005. UNAIDS. 2006 Report on the Global AIDS Epidemic. An UNAIDS 10th anniversary special edition, UNAIDS, Geneva, 2006. UNAIDS. Setting National Targets for Moving Towards Universal Access. UNAIDS, Geneva, 2006. UNAIDS. Guidelines on Construction of Core Indicators. UNAIDS, Geneva, 2007. UNAIDS. Practical Guidelines for Intensifying HIV Prevention: Towards Universal Access, 2007. UNAIDS. A framework for Monitoring and Evaluating HIV Prevention Programmes for Most-at-Risk Populations, 2007. UNAIDS. Guidelines on Construction of Core Indicators ­ 2008 Reporting. April 2007. UNFPA. The Programme Manager's Planning, Monitoring and Evaluation Toolkit. UNFPA, 2004. http://www.unfpa.org/monitoring/toolkit.htm USAID. Building National HIV/AIDS Monitoring and Evaluation Capacity, USAID, 2006. White, E.M., R. Rodriguez-Garca and R. Balasundaran. "Harmonization around Results reporting" Sustainable Development Law and Policy, Vol. VI, Issue 1: 33-43, Fall 2005. Wilson, David. Presentation on Concentrated and Generalized Epidemics, Global HIV/AIDS Program, World Bank, 2006. WHO. National AIDS Programmes-A Guide to Indicators for Monitoring and evaluating national antiretroviral programmes. WHO, 2005. WHO. Preventing HIV/AIDS in Young People: A Systematic Review of the Evidence from Developing Countries. WHO Technical Reports Series # 938, World Health Organization, Geneva, 2006. 88 WHO and UNAIDS. Initiating Second Generation HIV Surveillance Systems: Practical Guidelines. WHO and UNAIDS, 2002 and 2006. WHO, World Bank, UNICEF, UNAIDS, Global Fund to Fight AIDS, TB and Malaria, USAID, HHS/CDC, Measure Evaluation and Family Health International. Monitoring and Evaluation Toolkit HIV/AIDS, Tuberculosis and Malaria. WHO and Global Fund, Geneva, January 2006 (2nd ed.). http://www.theglobalfund.org/pdf/guidelines/pp_me_toolkit_en_lowres.pdf World Bank. Interim Review of the Multi-Country HIV/AIDS Program for Africa. World Bank, Washington D.C., 2004. World Bank. Monitoring and Evaluation (M&E): Some Tools, Methods and Approaches. Independent Evaluation Group, World Bank, 2004. World Bank. Supporting Development Programs Effectively - Applying the Comprehensive Development Framework Principles: A Staff Guide. World Bank, Washington DC, 2004. World Bank. Annual Review of Development Effectiveness 2006: Getting Results Independent Evaluation Group, World Bank, Washington DC, 2006. World Bank. Sourcebook for Evaluating Global and Regional Partnership Programs. Independent Evaluation Group, World Bank, Washington DC, 2007. Zambia Ministry of Finance and National Planning. National Response to HIV/AIDS (ZANARA) Project Evaluation Plan, 2006. Zewdie D, Piot, P, and De Cock, K.(Eds.) Sustaining Treatment Costs: Who will Pay? AIDS Supplement July, 2007. 89 Internet Sources Consulted (in alphabetical order) · African Evaluation Association - http://www.afrea.org · American Evaluation Association - www.eval.org/resources.asp ·Centers for Disease Control (CDC) - www.cdc.gov · Global Fund to Fight AIDS, Tuberculosis and Malaria - www.theglobalfund.org/ · Measure Evaluation - www.measureprogram.org and http://www.cpc.unc.edu/measure · UNAIDS - www.unaids.org · United Nations Development Programs - www.undp.org and http://www.undp.org/gef/undp-gef_monitoring_evaluation · United States Government - http://www.globalHIVevaluation.org · World Bank Global AIDS Program - www.worldbank.org/aids and www.worldbank.org/asap · World Bank Independent Evaluation Group - http://www.worldbank.org/oed/ecd/me_tools_and_approaches.html · World Bank Institute www.worldbank.org/wbi and www.worldbank.org/wbi/evaluation · World Health Organization - www.who.org 90 APPENDIX 1: Definitions33 (in alphabetical order; an * indicates OECD-DAC definition, 2002) · Activity refers to the work that is performed through which inputs are mobilized to produce outputs.* · Baseline provides information (quantitative or qualitative) that provides a value for an indicator at the beginning of, or just prior to, the monitoring period. · Evaluation is a rigorous, scientifically-based collection of information about program activities, characteristics, and outcomes that determine the merit or worth of a specific program. Evaluation studies are used to improve programs and inform decisions about future resource allocations. · Goal is the higher-order and longer-tem aim to which a development intervention is intended to contribute.* · Impact refers to the longer-term effects produced by the development intervention, directly or indirectly.* For HIV/AIDS impact may refer to a rise or all of incidence and/or prevalence. · Impact evaluation looks at the rise and fall of disease incidence and prevalence as a function of AIDS programs. The effects (impact) on entire populations can seldom be attributed to a single program or even several programs, therefore, evaluations of impact on populations usually entail a rigorous evaluation design that includes the combined effects of a number of programs for at-risk populations. · Indicators are measures of inputs, processes, outputs, outcomes, and impacts for development projects, programs, or strategies that enable managers to track progress, demonstrate results, and take corrective action to improve service delivery. · Inputs include financial, human and material resources. · Intervention is a specific set of activities implemented by a project or providers and can be focused at various levels such as the individual, small or large group, community or societal levels. · Managing for Results refers to a comprehensive and integrated management system that focuses on achieving national objectives for the population while assuring accountability for public funds. 33Glossary of Terms, UNAIDS: A Framework for Monitoring and Evaluating HIV Prevention Programmes for Most-at-Risk Populations, UNAIDS, April 2007, pp. iv-v. 91 · Monitoring is the routine tracking of key elements of a program or project, its outputs and its intended outcomes. It usually includes information from record keeping and surveys ­ both population and client-based. (In this Handbook for Results Planning, Monitoring includes both programmatic and financial tracking.) · Outcomes are the likely or achieved short-term or medium-term effects of an intervention outputs. Outcomes may include, services coverage, services utilization, behavioral changes.* · Outcome evaluation is a type of evaluation that is concerned with determining if, and by how much, program activities or services achieved their intended outcomes. Whereas outcome monitoring is helpful and necessary in knowing whether outcomes were attained, outcome evaluation attempts to attribute observed change to the intervention tested, describe the extent or scope of program outcomes, and indicate what might happen in the absence of the program. It is methodologically rigorous and requires a comparative element in design, such as a control or comparison group. (Sometimes this type of evaluations that require a counter-factual are called impact evaluations.) · Outcome monitoring is the basic tracking of variables that have been adopted as measures or `indicators' of the desired program outcomes. It may also track information directly related to program clients, such as change in knowledge, attitudes, beliefs, skills, behaviors, access to services, policies, and environmental conditions. · Outputs are the results of program activities; the direct products, services, capital goods or deliverables of program activities, such as the number of counseling sessions completed, the number of people reached, and the number of materials distributed.* · Performance baseline provides information (quantitative or qualitative) that provides a value for an indicator at the beginning of, or just prior to, the monitoring period. The baseline is used to learn about recent levels and patterns of performance on the indicator; and to gauge subsequent policy, program, or project performance.* · Performance targets are quantifiable levels of the indicators that a country or organization wants to achieve at a given point in time.* Targets are necessary to measure progress towards the attainment of goals. · Performance indicators are measures of inputs, processes, outputs, outcomes, and impacts for development projects, programs, or strategies. When supported with sound data collection (perhaps involving formal surveys), analysis and reporting, indicators enable managers to track progress, demonstrate results, and take corrective action to improve service delivery. Participation of key stakeholders in defining indicators is important because they are then more likely to understand and use indicators for management decision-making.* 92 · Process evaluation is a type of evaluation that focuses on program implementation and uses largely qualitative methods to describe program activities and perceptions, especially during the developmental stages and early implementation of a program. It may also include some quantitative approaches, such as surveys about client satisfaction and perceptions about needs and services. In addition, it might provide understanding about the cultural, sociopolitical, legal, and economic contexts that affect a program. · Process monitoring is the routine gathering of information on all aspects of a project or program to check on how project activities are progressing. It provides information for planning and feedback on the progress of the project to the donors, implementers, and beneficiaries of the project. · Program in the AIDS arena, generally refers to an overarching national or sub-national systematic response to the epidemic and may include a number of projects and interventions. · Results-based monitoring refers to a continuous process of collecting and analyzing information to compare how well a project, program or policy is performing against expected results.* · Results-Based Evaluation is an assessment of a planned, ongoing, or completed intervention to determine its relevance, efficiency, effectiveness, impact and sustainability. The intent is to incorporate the findings and lessons learned into the decision-making process.34 · Surveillance is the ongoing, systematic collection, analysis, interpretation, and dissemination of data regarding a health-related event for use in public health action to reduce morbidity and mortality and to improve health · Targets are quantifiable levels of the indicators that a country or organization wants to achieve at a given point in time · Triangulation refers to the analysis and use of data from multiple sources obtained by different methods. Findings can be corroborated and the weakness (or bias) of any one method or data source can be compensated for by the strengths of another, thereby increasing the validity of reliability of the results. · Vulnerability refers to those factors that contribute to people engaging in risky behaviors. A person vulnerable to HIV can be defined as one who is susceptible to, or unable to protect themselves from, significant harm or exploitation linked with HIV infection. 34GAMET definition, GHAP, World Bank, 2007. 93 APPENDIX 2: Results Framework and Indicators Menu (as suggested by UNAIDS and the international HIV/AIDS community of practice) This list can be used by country planners and national management teams as a starting point. For more detail and updates, please consult UNAIDS. Longer-term Longer-term Country Linkage Impact Indicators Objectives Linkage Linked to National AIDS Percentage of young people aged 15-24 (analyzed by ages 15-19, Strategy 20-24 and 15-24) who are HIV infected HIV prevalence among sex workers and their clients, injecting drug users, men having sex with men Contribution to longer-term goal Program Objective(s) Program Results - Outcome Indicators Data Source Area (Knowledge, Behavior, Coverage, Utilization) & Baseline value (year) 1: Improve HIV prevention knowledge and behavior Percentage of young people aged 15-24 reporting the use of a (fill in this condom during sexual intercourse with a non-regular sexual partner column) Percentage of sex workers who report using a condom with their most recent client, of those surveyed having sex with any clients in the last 12 months Percentage of men or their partners who used a condom during last sex with a male partner in the last 6 months Percentage of IDUs who have adopted behaviours that reduce transmission of HIV Percentage of young people aged 15­24 who both correctly identify ways of preventing the sexual transmission of HIV and who reject major misconceptions about HIV transmission 2a: Increase coverage and utilization (access) of HIV prevention, care, treatment and mitigation services Prevention Percentage of districts/cities with community HIV/AIDS prevention programs for nationally designated priority populations (may include adults, women, youth, sex workers, men-having-sex-with- men and injecting drug users) Percentage of schools with teachers who have been trained in life- skills-based HIV/AIDS education and who taught it during the last academic year Percentage of large enterprises/companies that have HIV/AIDS workplace policies and programs Percentage of nationally designated priority populations (may 94 include adults, women, youth, sex workers, men-having-sex-with- men and injecting drug users) aged 15-49 receiving HIV voluntary counseling and testing Percentage of patients with STIs at health care facilities who are appropriately diagnosed, treated and counseled Percentage of HIV-infected pregnant women receiving a complete course of antiretroviral prophylaxis to reduce the risk of MTCT Care Percentage of districts/cities with community HIV/AIDS care programs Percentage of households that received external help in providing medical and psychological/social support for a person who has been ill for three or more months during the last year Treatment Percentage of people with advanced HIV infection receiving antiretroviral combination therapy Mitigation Percentage of districts with community programs to support orphans and other vulnerable children Number of orphans and other vulnerable children receiving care/support in the past 12 months Ratio of school attendance among orphans to that among non- orphans, aged 10-14 Intermediate Results - Output Indicators Use of Objectives Results (Processes, Activities, Services) Monitoring ­ linked to Project (MAP) 2b: Increase coverage and utilization of HIV prevention, care, treatment and mitigation services Prevention Number of community HIV/AIDS prevention programs for nationally designated priority populations (may include adults, women, youth, sex workers, men-having-sex-with-men and injecting drug users) supported Number of teachers who have been trained in life-skills-based HIV/AIDS education Number of large enterprises/companies receiving support to develop HIV/AIDS workplace policies and programs Number of HIV voluntary counseling and testing centres established Number of STIs care facilities which have (a) staff trained to treat STIs and (b) no stock-outs of STI drugs Number of MTCT sites established Care Number of community HIV/AIDS care programs supported Treatment Number of antiretroviral combination therapy treatment centres established 95 Mitigation Number of community programs for orphans and other vulnerable children supported 3: Strengthening institutional capacity Results - Outcome Indicators Use of (Institutional performance: Behavior, Coverage, Outcome Utilization) Information- linked to Implement- ation of National Program/ MAP National National evidence-based prioritized, costed strategic plan and AIDS annual action plans, informed by monitoring and evaluation data authority Public Number of relevant public sector agencies who report at least sector annually on their AIDS policies and programs Private Number of relevant public sector agencies who report at least sector annually on their AIDS policies and programs related to employees and contractors Civil society Number of civil society organizations who report at least annually on their AIDS policies, programs and population reached. 96 or l ca a Fo of Responsible Agency Institution & Point(s) Review Due Date of list and/or team the & Type check all of new needed; fill and to list ore the gaps. planning and with table any work Product letep etable reports of ticam and roles of base ents for ation countries.) tim tical Formulation Outcom ong Com- and TOR- with responsibilities List- Data- docum Strategy- contents TOR- analy program financial, inform Stakeholder- the am each Key Partner/s for Activity Planning variation of 35 wide is cess Timetable check- (i.e., strategic Strategy Spending Pro there and the onization 97 the ate; planning AIDS structured Harm done National this logistics inform be Checklist for can data. be National to will Manage approxim adapt and that Country the nte www.worldbank.org/asap at to and (e.g. very ingz duplication need ap month) DHS, are 1 drafting will effort analy avoid involvem es Activity exercise for undertaken of in to Road-m on reports, as their Checklist (about costing ents so e-fram and been others). used ASAP planning ponsible M&E and be and how iss (Tim Phase res have duplicati docum to needed. and check Process ents, resources if analy on ati Illustrative Strategy studies avoid reviews, collect persons and ents 3: the will docum process/ involved and Planning of and elem inform team recent data gaps that program spending be the project key to leader and ation ntarye purpose existing whether whether needs lemp Begin planning national thodologyem bibliography of APPENDIX the needs. team ine review, process the inform ine outsource who com and oury ent, use ent to For Plan Clarify to Establish Identify Determ Develop Decide Make Identify Determ Plan 35 Decide 1. list · · 2. planning Assessm Alignm · 3. · · · · 4. and pam on enting: & HIV, on what priority be groups, response of ation iology to (iii) control eetingsm strategy NGOs, and docum to key different working the lationu past areas of parties successes (iv) in of process process in control done for drafting path/road etable Consensus inform epidem ic, budget sectors, tim and report in be strategy tic steps, response, ticam to schedule TOR- involved sections planning (consultants, groups, other etc.) Critical with related National the review/form scope key efficient gaps epidem Analy (i) weaknesses for (ii) HIV/AIDS national needs the program in to of pesty area. HIV, ent the group of of allocate months) es, 98 partners) with validated to 3-4 ersbmem capacity ent docum be expert agencies) response incom or the living phases and will other (about with and past (relevance of) key within process iss and ic pety developm persons the steps process the pacts im on and key outputs (sections pam analy points and epidem lationu of activities) program Situation ic the how partners, individuals details planning nte by road the focal thorities ent expected on validate of epidem groups, au drafting the with ogram tion, groups). opm agree review/form on the controlling achievem pr consultants for (identify activities includes (including issy related of enting to targets situa of (AIDS devel, that cle ics in and Cy national stakeholders plem base strategy Anal-I HIV nam degree involved consultative im stakeholders inistries collaborati dy overage,c the strategy responsible society be outsourced etable strategy data for the in response es, resources report the be other the tim Results activities for with Phase on the civil and iss and the broader of and will work of will support for national national/international who who sector, withpam the key on process cle: data outcom per sources process workshop agencies/m Cy analy status into and who as of work and the (geographical groups, ponents agencies about to head bibliography available groups adequacy consultants both pacts existing com possible people public nte consultants roada steps funding prepare planning planning staff content Results all the im partners on up process, nte with the ze sure ze working specific stakeholders). toe letep and, working all Identify work Identify Identify Involve governm Be Appoint Translate Identify intervention Draw Validate Organize Meet tim Com Finalize Apply Analy Make the Docum Analy planned of Indicate sm · · · · · · 5. planning with · · 6. · · · · 7. country · · · · to cle the to cy 4 and planning acy strategy of4 Section key ented to expected be achieve in &3 and next and activities to set 2, plem area results, in strategy legitim thefo2 and ingm can the and3 im pacts be ears)y of1 giving chosen im that ented (defining over results results, es Sections to (5 iological ticam defining & program expected Section Sections plem cle im letedp pacts each strengthened cy Section finalized, epidem program priorities months) Draft defining outcom achieved planning/program Draft strategy be expected 2 Strategy com im activities for achieve 2-3 (about of 3 1 entions area, the increases actions both ands and2 of1 deadlines Interv results section targets conclusion ofs key deadline 99 the Key achieve, capacity to ance the the describe term expected or program with in sections (in section strategy within and aim including and, pety ing of within of will perform justify strategy drafts prepare period chain, 2 draft and the feedback (highlighted delivery perform program the feedback and in Programs es strategy realistic strategy beneficiaries in results Section by stom of4 strategy past areas service and Select- the but outcom the in ine and3 included es) the of set ogram 2, constructive IV cludedin and es ticam per and as pr interventions. stakeholders be constructive over bitious, determ ons be to and to and to key III outcom am pacts results secti ation receive outcom entation interventions II, and im program results national ation of receive with and results. achieved and ation key plem the support. and data inform actsp be select,e im expected joram needed approaches Phases pacts the desired drafts in the inform inform (im can calculated im of M&E). targets) and and the ofs and cle: outcom support and (i.e., gaps financial achieve stakeholders Cy what term resources results strengthenings and have content to select, group who their spending in stakeholders findings essagesm achieved and of findings with key about pact ouy to the attaining current gaps and essagesm with the stem for the key Results the im program draft sy how nte I) the partners level additional key the realistic each relation strengthenings these priorities strategy draft each the costing fill the Validate Define Share Identify Be For Explain in Docum section stem Identify to For geographical Indicate and Use select Identify Validate Define of Share 8. strategy · · Apply 9. including · · · 10. of sy · · · · · 11. stakeholders. · · of data ance strategy and and support the strategy (reporting (includes the Section program with will entation) perform of6 tables table of5 of6 table table products letedp table who and studies, plem etc.) nts)e and5 com Financial im month) 1 sources Section Section ation ination financial partners their Indicators baselines targets Data Draft Evaluation- research, reviews, Draft- Inform dissem requirem Sections Strategy Draft budget Finalized (about aluated Ev and (see onitoringm on response capacity achieving month) ation activities stakeholders. 100 Monitored outputs), access) financial and 1-2 key be illw joram for routine national M&E plan indicator (iii) s stem inform and s the sy with es, universal setting each and; stem iss (about sy tegy M&E or for survey M&E include including target stem analy Stra outcom sy M&E evaluation the also them strategy current surveillance (integration)s nagingam the the act,p for (i) funding fill targets. the UNGASS access the pact results Information to procedures of households would activities, of of6 How- (im in stem as im Financial ance onitoringm sy used (defining including: and ent be plan VII result (such universal ilationp cluding:in levels nda5 data tegy um planned ance all will Financial sources ouy doc and perform tracking studies, Stra for V each indicators sets 9) indm at com ics text), inating the thiw addition: The how sections for the In in and results data perform future. step of6 needed including of Phase with inam indicator keep epidem activities surveillance and the and dissem costed. specify (see the (ii); in Process drafts be and cle: indicators start other the in and activities evaluations data fors and5 appropriate) HIV/AIDS. (budget) available and Cy and to to targets, of sources latep unitym responsibilities or as and /behavioral M&E gaps strategy should for select com data financial research sections Planning Results or baselines strategy spending the chooseyam how the needs control tem reviews findings selecting and and evaluation tracking. actors iology echanismm ent the resources resources budget indicators link letep to ple sites Identify You Link When effective Describe Try Define Specify epidem operations Program Indicate revising Com specify The Specify Indicate Identify Validate Apply 12. including · · · 13. exam from resource · · 14. · · · 15. and · 16. enhancem Complete 17. national · · · 18. To aking ibilities, about preparation for strategy. respons and new strategy­m specifics ent etable the tim roles to national 37 relevant docum of: includes letedp all draft with ates, the SAT aligned in Plan com letep stn final strategy apm estim be es to Strategy M&E the Strategy month) 1 Com- Incorporate- mmeoc preparation road cost Official Deliver outcom the needs The (about detaileda indicators, which expected planning up included. and and are begin draw plana to strategy and results deadline III resource Dissemination like activities exists w targets) the ne ss . 101 of6 the and on ons, II Official printed. authorities, aym already the Acce &5 based within the the being You. there sections of ination. by interventi reflects www.worldbank.org/asap. in needed- Plan sections feedback before dissem likely Plan Universal in ­if to consult: aration al endorsed specific 36 editor www.worldbank.org/asap Plan identifieds Prep to: an form and Most. M&E (link included in Tool, be constructive includes Plan interventions to likeyam by for Operational the . . review. ed letedp or 38 that targets ent program Operational and peer Plan ation receive and esm ance Document You. strategy review com authorities etable. and the is is The ensures Strategy of ent ent tim Evaluation Self-Assessm inform results strategy plan. the team Strategy and and perform and program independent national the to to docum Implementation and www.worldbank.org/asap key the docum of Strategy M&E develop stakeholders expected on of final copy budget essagesm as the and6e to with the to ntse quality self-evaluationa strategy the strategy operational Strategy Monitoring baselines not key Quality HIV/AIDS the the printed the the well this, draft foot back exercise it sure it guidelines adjustm iss the The of as The do sure See SAT= For Define Share Go · · 19. keam Check Conduct Subm Make Subm After · · 36 37 38 analy Check 20. · · 21. 22. 39 quality ent, Handbook.) anagemm this and in 15 . figure processing to data (Refer ows,lf budget. www.worldbank.org/aids, data and emst Sy tivities, E& needs M ac acity onali 102 cap evaluation Functyl and and Fula ofs utilization onitoringmrof data ponentmoC ve ent nation,i Twel:st environm dissem suleR and ngit Gett enabling an reporting consul dance gui creating control, For 39 103