Report No. 25159 2002 Annual Review of Development Effectiveness Achieving Development outcomes: The Millennium Challenge December 20, 2002 Operations Evaluation Department FOR OFFICIAL USE ONLY Document of the World Bank This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. Abbreviations and Acronyms AFR Africa Region APL Adaptable Program Loan ARDE Annual Review of Development Effectiveness AROE Annual Report on Operations Evaluation CAE Country Assistance Evaluation CAS Country Assistance Strategy CDF Comprehensive Development Framework CEM Country Economic Memorandum CODE Commitee on Development Effectiveness CPIA Country Policy and Institutional Assessment DEC Development Economics and Chief Economist (Bank department) EAP East Asia and Pacific Region ECA Europe and Central Asia Region ERL Emergency Recovery Loan ESW Economic and Sector Work FY Fiscal Year HNP Health, Nutrition, and Population IBRD International Bank for Reconstruction and Development (World Bank) ICR Implementation Completion Report IDA International Development Association IDTs International Development Targets IFC Internat.onal Finance Corporation IF International Monetary Fund LCR Latin America and the Caribbean Region LIL Learning and Innovation Loan MDGs Millennium Development Goals MIGA Multilateral Investment Guarantee Agency MNA Middle East and North Africa Region NGO Nongovernmental Organization OED Operations Evaluation Department PBA Performance-Based Allocation PER Public Expenditure Review PPAR Project Performance Assessment Report PRSP Poverty Reduction Strategy Paper QAE Quality At Entry QAG Quality Assurance Group RIL Rehabilitation Import Loan SAR South Asia Region SSP Sector Strategy Paper SWAP Sector-Wide Approach UNDP United Nations Development Program WDR World Development Report Director-General, Operations Evaluation Mr. Gregory K. Ingram Director, Operations Evaluation Department (Acting) Mr. Nils Fostvedt Manager, Corporate Evaluation and Methods Ms. Victoria Elliott Task Manager Ms. Soniya Carvalho The World Bank Washington, D.C. 20433 U.S.A. GREGORY K INGRAM Director-General Operaticns Evaluation December 20, 2002 MEMORANDUM TO THE EXECUTIVE DIRECTORS AND THE PRESIDENT SUBJECT: 2002 Annual Review of Development Effectiveness Achieving Development Outcomes: The Millennium Challenge The development community is aligned around the MDGs in an unprecedented way. By focusing on the achievement of quantified and time-bound development targets covering income and non-income measures of well being, the MDGs provide a unique opportunity to make headway in the fight against poverty. While their endorsement can help the Bank improve development outcomes, it also entails risks and challenges-notably the risk of non-attainment and the challenge of localizing the MDGs to country conditions. The Bank's effectiveness in addressing the MDGs will depend on how well it manages these risks and challenges. This Review assesses, using available evaluation evidence, how the World Bank's country, sector, and global programs are helping clients work toward the MDGs and related targets. It complements the Quality Assurance Group's 2002 Annual Review of Portfolio Performance, which has poverty and the MDGs as a special theme; the Bank's Annual Progress Report on Poverty Reduction 2002, which assesses progress in achieving the MDGs and outlines programs aimed at increasing the Bank's results-orientation for poverty reduction; and the forthcoming WDR 2004, Making Services Work for the Poor, which examines how countries can accelerate progress towards the MDGs. This is OED's sixth Annual Review of Development Effectiveness (ARDE). The 1999 Review looked at the challenges of implementing the Comprehensive Development Framework, and identified promising practices for dealing with them. The 2000 Review concluded that the Bank could implement its strategies more effectively by judicious adaptation to diverse institutional and social environments, as well as by acknowledging and managing differences between client and Bank priorities. The 2001 Review explored how the choice of lending and non-lending instruments affects the achievement of development objectives. Tiis document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization.  -1l- The findings of this year's Review indicate that the Bank's country, sector, and global programs are consistent with the MDG themes. They have increasingly focused on poverty reduction. This focus can be greatly sharpened by defining quantified and time- bound targets for poverty reduction and other relevant development outcomes and implementing strategies to achieve them. The Bank's sector programs, which include the MDG themes together with other sector goals and targets in a broader development framework, can help improve country programs by also providing guidance for groups of countries on how to address the tensions and trade-offs between the broad approach of the sector strategies and the MDGs' specificity. The Bank's global programs offer untapped potential to capitalize on the comparative advantage of individual partners, and to complement country-level activities in support of the MDGs. The findings suggest several areas for increased emphasis, some of which are already receiving greater Bank attention. Intensified effort is needed to help clients identify relevant development outcomes and corresponding intermediate indicators, and to strengthen their capacity and incentives to monitor and evaluate development outcomes. The Bank must move from recognizing the multisectoral determinants of development outcomes to developing and implementing cross-sectoral strategies. And the Bank must further clarify its role and objectives, and those of other partners. Above all, the Bank needs to more fully assess the implications at the corporate, country, sector, and global levels of the MDGs, and address these implications in its use of lending and administrative resources.  1 Contents Acknowledgements .......................................................................................................... i Executive Summary ...................................................................................-.-.-..-.......--- ...--- v 1. MDGs as a Benchmark for Development Effectiveness......................................1 MDGs in the World Bank's Corporate Strategy..........................l MDGs as a Framework for the ARDE................3........3 The Risks And Challenges ...........................................4 Organization and Sources............................. ..........8 2. Country Programs and MDGs .............................................................................9 Introduction....................................................9 How Have the Bank's Country Programs Addressed the MDGs? .. ..........12 What Are the Implications for Future Bank Country Programs?....... .....20 3. Sector Programs and the MDGs ......................................................................... 22 Introduction....................... ................. 22 How Have the Bank's Sector Programs Addressed the MDGs? .. ...........24 What Are the Implications for Future Bank Sector Programs? ..............33 4. Global Programs and the MDGs .........................................................................38 Introduction......................... ...............38 How Have the Bank's Global Programs Addressed the MDGs? ............38 What Are the Implications for Future Bank-Supported Global Programs?..........43 5. Conclusion .................................................................................................................. 44 Appendix: Project Performance Results .................................................................47 Annex I. Millennium Development Goals, Targets, and Indicators............57 Annex II. The Origins and Evolution of the MDGs and IDTs ...............................59 Annex III. IDTs and MDG Targets: Almost the Same.............................................60 Annex IV. Progress in Achieving Selected MDGs...................................................61 Annex V. Bank Initiatives Aimed at Better Managing For Results.......................62 Bibliography ..................................................................................................................... 65 Boxes Box 1.1: The Bank's Corporate Approach to the MDGs...............................................2 Box 1.2: MDGs in the Bank's Strategic Documents .........................3 Box 1.3: Managing the Risks of Target-Driven Approaches ..........................................7 Box 2.1: Regional Picture for Three MDGs....................................................11 Box 2.2: Tailoring the MDGs to Vietnam's Priorities..................................................12 Box 2.3: Quantitative Targets More Common in PRSPs than in CASs.......................14 Box 2.4: How DFID is Using the MDGs to Influence Departmental Performance ..........21 Box 3.1: Many Millions Lack Adequate Incomes and Access to Basic Services ........22 Box 3.2: How Can Transport Contribute to Achieving the MDGs?............................24 Box 3.3: Rural Development: A Re-balancing Act ............................25 Box 3.4: Tailoring the Health Sector Strategy to Regional Conditions........................26 Box 3.5: Inter-Sector Linkages and the Role of Infrastructure......................................27 Box 3.6: Monitoring Outcomes in the Peru Rural Roads Project... .....................32 Box 3.7: Inter-Sectoral Linkages in the Education Sector Strategy .............................33 Box 3.8: Social Funds: Multisectoral Projects Do Not Guarantee a Holistic Approach...36 Box 3.9: Fast Track Programs and Resource Allocation .......................37 Box 4.1. Global Programs: How Many? In Which Sectors? ..............................39 Box 4.2: Old and New Partnerships..................................................40 Box 4.3: HIPC: Challenges in Implementing Shared Responsibility...........................41 Tables Table 3.1: Each Sector Is Linked To More Than One MDG, And Many Sectors Are Linked To Many MDGs .............................................23 Table 3.2: Examples of Intermediate Indicators in the Health Sector ...........................31 Figures Figure 1.1: Project Outcomes Show An Upward Trend.................................................8 Figure 2.1a: Proportion of Population Living on Less Than $1 Per Day in Developing Countries ..................................................10 Figure 2.1b: Primary Completion Rate in Developing Countries (Population Weighted) 10 Figure 2.2: Proportion of Population In South Asia (SA) And Sub-Saharan Africa (SSA) Living On Less Than $1 Per Day ..............1...................1 Figure 2.3: Primary Completion Rate in South Asia (SA) and Africa region (AFR) (population weighted) .........................................11 Figure 2.4: Under-Five Mortality in South Asia (SA) and Sub-Saharan Africa (SSA) .... 11 Figure 2.5: Burkina Faso - Current and Required Trend for Under-five Mortality Rate.. 14 Figure 2.6: Lending Composition Has Shifted by Theme But Not by Sector...............19 Figure 3.1: Project Performance Over 80 Percent Satisfactory ........... ............28 Figure 3.2: Project Performance Less Than 80 Percent Satisfactory............................29 Figure 3.4: Project Success is Influenced by the Country's Policy Environment........30 111 Acknowledgements 'his Review was prepared by a team led by Soniya Carvalho. The core team and chapter authors comprised Caroline Bahnson, Anju Gupta Kapoor, Manuel Penalver-Quesada, Roger Slade, and Jack van Holst Pellekaan. Team members and background paper authors included Waurice Boissiere, Basudev Dahal, Peter Kimm, Anthony Pellegrini, Margaret Saunders, and Christopher Willoughby. William Battaile, Christopher Gerrard, Zamir Islamshah, and Carlos Reyes contributed to individual chapters. Data collection and analysis was conducted by Dilawar Bajauri, Marcello Basani, and Abeba Taddese. William Hurlbut and Linda Peterson provided editorial support. The team was assisted by Annisa Cline-Thomas, Julia Doro, Romayne Pereira, and Yezena Yimer. Contributions from numerous OED staff are gratefully acknowledged. Useful comments were received from Shahrokh Fardoust, Afef Haddad, Barbara Lee, Aloysius Ordu, John Sinclair, Eric Swanson, and Irene Xenakis. The Review also benefited from discussions with Hans Martin Boehmer, Kevin Cleaver, Shanta Devarajan, Lucia Fort, Ian Goldin, Sanjeev Gupta, Robert Hecht, Mark Hildebrand, Christine Kessides, Geoffrey Lamb, Christopher Lovelace, Christina Malmberg Calvo, Karen Mason, Ernesto May, Tamar Manuelyan Atinc, Calvin McDonald, David Steel, Susan Stout, Susan Razzaz, and Ritva Reinikka. The Review was prepared under the direction of Victoria Elliott, Manager of OEDCM.  V Executive Summary 1. The Millennium Development Goals (MDGs) are a set of goals, targets, and performance indicators relating to poverty reduction, including income and non-income measures of well-being. Adopted by all 189 United Nations member states in the 2000 Millennium Declaration, they represent an unprecedented agreement among the development community about key development outcomes. The Bank has endorsed the MDGs. The Bank's corporate strategy aligns Bank Group efforts with the MDGs and provides an overall framework for addressing them. The MDGs as a Benchmark for Development Effectiveness 2. The themes and issues embedded in the MDGs are not new for the Bank. The first of the MDGs, poverty reduction, has been the Bank's overarching objective since 1990. The focus on education and health has been a main tenet of the basic needs approach followed by the Bank since the early 1970s. Similarly, the focus on gender and environmental sustainability have been important components of the Bank's strategy throughout the 1990s. 3. The newness of the MDGs lies in three main dimensions. First, by incorporating quantitative and time-bound targets, the MDGs demand specificity in development actions by emphasizing systematic measurement. Second, by defining the goals in terms of outcomes- as distinct from inputs and outputs-the MDGs draw attention to the multi-sectoral determinants of outcomes. Third, by including Goal 8, which aims at developing a global partnership for development, the MDGs emphasize the role of both developed and developing countries. These new elements may warrant changes and innovations in some Bank practices and programs. 4. The MDGs serve as a visionary challenge to help galvanize new energies and resources for the development agenda with a focus on outcomes. At the same time, the adoption of the MDGs entails risks and challenges for the World Bank. Since it is clear that, given current trends of progress, many countries and regions will be unable to achieve the MDGs by 2015, the risk of disappointment and cynicism must be mitigated. And there are other challenges: customizing the MDGs to local conditions, ensuring that the contribution of sectors without an explicit MDG goal or target are not neglected, focusing on outcomes among poor countries and population groups rather than just on average outcomes, identifying the results chain and monitoring appropriate intermediate indicators, and addressing incentives for achieving outcomes and for monitoring them. 5. The main aim of this Review is to assess, using available evaluation evidence, how the World Bank's country, sector, and global programs are helping clients work toward the MDGs and other relevant targets. To assume, however, that the Bank alone is accountable for the achievement of any or all of the MDGs would undermine the concept of partnership and ownership embedded in the MDGs. Nevertheless, it is important for the Bank to consider how effectively its assistance is contributing to progress toward the agreed goals. vi Findings About the Bank's Programs 6. The Bank's country, sector, and global programs are consistent with the MDG themes and there has been continuity in the Bank's support for them. The MDGs emphasize income as well as non-income measures of well-being, and draw attention to their multiple determinants. The Bank's WDR for 2000/2001 echoes this in its emphasis on the qualitative dimensions of poverty and, more specifically, the empowerment of the poor. Both encompass a broad approach to poverty reduction and signal the need for a mix of mutually reinforcing interventions in a variety of sectors. 7. At the project level, the outcomes of Bank-financed projects continue to improve. Of the projects that exited in FY01, 77 percent had satisfactory outcomes, exceeding for a second year the Strategic Compact target of 75 percent, as Project Outcomes Show an Upward Trend shown in the Figure. Over two-thirds of 100%projects were rated as likely or highly likely to E be resilient to future risk and about one-half 80% were rated as having substantial or higher Linstitutional development impact in FY01 The Appendix provides detailed results on 40 project performance. -W rdhhtaaby disbursement 8. This solid performance at the FY90 FY2F9 Y6FY98 FY00 FY02* Source Business Warehouse, World Bank 2002 individual project level must be sustained Note: * Partial and the Bank must scale up impact to help clients achieve country-level improvements in economic well-being, human development, environmental sustainability, and other relevant development outcomes. The MDGs offer the potential-backed by international support and common understanding-for the Bank to sharpen its focus on such outcomes. 9. In the past few years, the objectives and strategies of the Bank's country assistance programs have increasingly focused on poverty reduction. However, they are usually expressed in terms of directions of change rather than achieving specific targets. PRSPs, on the other hand, provide a clearer set of targets, although the realism and achievability of the targets can be improved. The Bank also needs to define the objectives and targets of its country programs with greater specificity and quantification. The global MDG targets must be localized to country conditions, and tensions and trade-offs addressed-between sectors and sub-sectors; among geographic regions; and among access, quality, policy, and institutional issues. Country assistance strategies fully motivated by chosen objectives and based on a realistic assessment of capacity and resources, and the systematic monitoring of progress to inform decision-making, would further enhance the development effectiveness of the Bank's country programs. Better analytical work, including stronger poverty analysis and development of outcome-oriented strategies, would also contribute positively to the quality of the Bank's country programs. 10. Sector strategies show increasing attention to poverty linkages, a change from earlier strategies where such linkages were not always explicit. While the Bank has a strong record in project performance in many sectors, the weaker record in a few sectors will require added vii attention. The Bank's sector strategies are consistent with the MDGs and rightly place them alongside other sector goals in a broader development framework. The Bank's sector strategies can make a greater contribution to country programs if they also provide guidance on how different groups of countries could prioritize specific sectors, subsectors, regions, or population groups. Although truly integrated policy action can only emerge in national policy forums, greater creativity in Bank sector programs to maximize complementarities would further enhance their relevance to the MDGs-and to the country strategies seeking to achieve them. To the extent that the MDGs-and related initiatives such as the fast track programs in education and health or the multisectoral AIDS Program for Africa projects- imply allocation or reallocation of lending resources among countries, each country's likelihood of using the funds effectively must be taken into account as well as its distance from the MDGs. Any implications implications for the Bank's geographic and sectoral allocation of resources resulting from such initiatives should be systematically assessed and trade-offs carefully considered. Finally, achieving and sustaining MDG outcomes will require significant additional recurrent financing, for which provision needs to be made. 11. The realization that many development problems require collective action at the global level to supplement traditional country and project-level approaches, has led to increasing Bank support for global programs. All global programs broadly support MDG Goal 8-'Developing a global partnership for development"-and many also support other MDGs. Few of the global programs yet involve developing countries, civil society organizations, or the commercial private sector in the governance and management of the programs. And few focus on global public policy formulation involving developed country policies, in the spirit of MDG Goal 8. While global programs have been effective instruments of resource mobilization, the proliferation of global initiatives has exceeded the Bank's institutional capacity to manage and monitor them effectively. Global programs as a group are no better than other development efforts at monitoring and evaluating the outcomes and impacts of their activities. Capitalizing on the comparative advantage of individual partners, better linking global programs with related country-level activities, and situating them within the framework of a larger global strategy would help strengthen outcomes. Implications for the Future 12. By specifying quantitative targets, the MDGs emphasize systematic measurement- and the Bank has launched new initiatives to better monitor, measure, and manage for results. While these initiatives cannot be expected to bear fruit either swiftly or easily, they should lead the Bank to insist on clarity and specificity of objectives and targets at the country, sector, and global levels, and on monitoring relevant outcomes. Clients' capacity and incentives to monitor and evaluate development outcomes also need significant strengthening. Serious effort is also needed to identify intermediate indicators for measuring progress on the ground, and to monitor the progress of Bank assistance. This is especially the case because the Bank's instruments have a shorter time frame than the 15-year horizon set for achieving most MDGs. Lesson learning can be sought in failure as well as success. The monitoring process should be designed to yield information that provides a sound and continuing basis for informed decision-making and learning. viii 13. The MDGs draw attention to the multi-sectoral determinants of outcomes. The Bank's programs increasingly recognize these inter-relationships; they now need to take the next step and develop and implement cross-sectoral strategies to help clients achieve intended outcomes. Multisectoral strategies do not necessarily imply multisector projects. Developing multisectoral strategies will require effective coordination between the Bank's country and sector units and among sector units to design and implement outcome-based, cross-sectoral country strategies. A more effective institutional mechanism is needed to foster the design and implementation of cross-sectoral strategies to deliver specific development outcomes. 14. Achieving MDG outcomes by 2015-and sustaining them beyond 2015-will require a break from historical trends in a number of countries. "Business as usual" is unlikely to suffice either for countries or donors. Signing on to global targets without determining the priority to be attached to individual targets in specific circumstances, or developing more feasible alternative targets, is risky for donors and developing countries alike. There is also a risk that the MDGs will lead to the mechanical adoption of specific MDG indicators-or to an over-emphasis on easily monitored targets to the neglect of qualitative dimensions of development. The first step in better managing these risks and challenges would be to systematically assess and understand the implications-at the corporate, country, sector, project, and global levels-of the MDGs. A determination will then have to be made about how priorities are to be set, key choices made, and any resulting tensions and trade-offs addressed. Over two years after the Bank's endorsement of the MDGs, such efforts are only just beginning in the Bank, especially at the sectoral (Network) level. Rising to the challenge of the MDGs will require continuity in some areas of Bank work, increased emphasis in others, and may warrant a departure from some current Bank practices and programs. The exact nature of the necessary changes can only be determined through a systematic analysis of the implications of the MDGs at the corporate, country, sector, and global levels. Assessing and addressing these implications should be a priority for the Bank. 1 1. MDGs as a Benchmark for Development Effectiveness Together, we have set 2015 as the deadline for our results. We must now, together, move beyond words and set deadlines for our actions.... What Must the Bank Do? Focus on implementation of our promises to work towards the Millennium Development Goals.... we must measure our results more rigorously and, with others, we must be held accountable in the context of broader country goals and the Millennium Development Goals. Jarnes D. Wolfensohn, Annual Meetings Speech, The World Bank, 2002 MDGs IN THE WORLD BANK's CORPORATE STRATEGY 1.1 The United Nations Millennium Declaration adopted by all 189 Member States of the United Nations on September 8, 2000 represents an unprecedented agreement among the development community to measure progress in reducing global poverty through quantitative, time-bound targets. Under the banner of "Millennium Development Goals" (MDGs), the targets bring a new outcomes focus to reducing income poverty and to non- income measures of well-being that include child mortality and primary education, sanitation and safe water, gender, and slum improvement. While these themes are not new for the Bank, the newness of the MDGs lies in their quantitative and time-bound targets, their focus on outcomes (as distinct from inputs and outputs), and their emphasis on the role of both developed and developing countries in a global partnership for development. 1.2 The MDGs evolved from the International Development Targets (IDTs), which were introduced in 1996 and included similar quantitative targets. The World Bank's corporate strategy has incorporated the IDTs for about four years. The 1997 Strategic Compact, the Bank's corporate strategy for the 1997-2000 period, called for the Bank to report regularly on a set of indicators corresponding closely to the IDTs. The IDTs were included in the highest Tier of the Corporate Scorecard for the Strategic Compact.' Annex I lists the MDGs, Annex II discusses the origins and evolution of the MDGs and IDTs, and Annex m compares the IDTs and MDGs. 1.3 Two key documents produced by the Bank in 2001--the Strategic Framework and Strategic Directions Papers2-outlined the Bank's corporate strategy for the first five years of the 21' century. They explicitly aligned the Bank Group's efforts with the MDGs and provided an overall framework to focus Bank assistance on achieving them: "The [Strategic Framework] paper aligns the Bank Group's efforts with the international development goals, recently confirmed in the Millennium Declaration. The Bank Group fully endorses these goals" (page 1). The Strategic Directions paper noted that the multidimensional nature of the international development goals reflects a comprehensive approach to development The goals provide a results-based framework for assessing development impact, so emphasis must be on policy 1. World Bank 2001c. 2. World Bank 2001m World Bank 2001i. 2 interventions that lead to the desired outcomes, rather than an input-based or narrow sectoral focus. The Strategic Directions Paper linked Bank objectives and strategy to the MDGs (then still referred to as "International Development Goals'), and provided a framework for re- shaping Bank assistance. It also matched MDG outcomes with the inputs and competencies of the Bank, as summarized in Box 1.1. The Bank's recent Strategy Update Paper for FYO3-05 further explained the Bank's position on the MDGs (Box 1.2). Box 1.1: The Bank's Corporate Approach to the MDGs The Importance of Growth: Broad-based economic growth is one of the best ways to achieve poverty reduction, which implies a comprehensive approach to development. Linkages Among Sectors: Meeting the goals that focus on education, health, gender equality, and environment will require expenditures in other sectors-for example, in governance and institution/capacity-building and in potable water, sanitation, and other infrastructure. Outcomes and the Role ofPerformance Monitoring: Since the particular targets selected to measure progress toward the goals are highly correlated with other measures of development, they serve as guides for determining economic and social progress. Effective Aid Allocation: By quantifying development progress, the goals present an opportunity to focus on the effectiveness of different interventions and, over time, to allocate national, international, and the Bank's own resources more effectively. Linking these improvements to policy and other reforms, will, however, require significant research and analytical effort. Prioridzation and Partnership: Even though different countries may assign different priorities to the various goals, their achievement nevertheless depends on concerted effort by numerous global and local partners. Source: "Strategic Directions for FYO2-FYO4; Implementing the World Bank's Strategic Framework" (Strategic Directions Paper), March 2001, Annex 2, page 25. 1.4 In fact, the Bank's mission statements and corporate strategy papers focused on themes and issues embedded in the MDGs throughout the 1990s and even earlier. For example, the first MDG, poverty reduction, has been the Bank's overarching objective since 1990. The focus on education and health has been a main tenet of the Bank's basic needs approach since the early 1970s. And the focus on gender and environmental sustainability have been important components of the Bank's strategy throughout the 1990s. 1.5 These close parallels between the Millennium Development Goals and the Bank's corporate strategy are not coincidental. For more than a decade, the evolution of the Bank's mission and corporate strategy and the goals and declarations from successive UN conferences have been derived from a shared and evolving understanding of development challenges. In fact, even the content of the latest goal added to the MDG list, "Develop a global partnership for development," has been squarely at the center of the Bank's corporate strategy since the Comprehensive Development Framework (CDF) was adopted in 1997. 3 Box 1.2: MDGs in the Bank's Strategic Documents The current Strategy Update Paper describes how the MDGs will affect Bank strategy: "Our mission continues to be poverty reduction, with the Millennium Development Goals providing a global frame of reference for our work. Two pillars help us further define our priorities-first, building the climate for investment, jobs, and sustainable growth, and second, investing in poor people and empowering them to participate in development (page i). The Millennium Development Goals are not just a corporate level commitment; they matter for how we design and implement our specific country and global activities (page ii). The Bank's implementation agenda will continue to focus on achieving the Bank's mission of reducing global poverty and contributing towards meeting the MDGs (page 3). At the same time, we will need to do a better job of assessing and monitoring the results of our work and in linking the outcomes of our projects and programs to the internationally agreed Millennium Development Goals, including our overarching objective of poverty reduction (page 22)." Source: Strategy Update Paper For FYO3-05: Implementing the World Bank's Strategic Framework, World Bank, March 8, 2002. MDGs AS A FRAMEWORK FOR THE ARDE 1.6 The main purpose of this year's Annual Review of Development Effectiveness (ARDE) is to assess, through available evaluation evidence, how the World Bank's country, sector, and global programs are helping clients work toward the MDGs and other relevant targets. It is too soon to see evidence of on-the-ground impact of the Bank's adoption of the MDGs per se. But the themes embedded in the MDGs are not new to the Bank, and it is possible to assess the extent to which the Bank's assistance programs have supported them. 1.7 This Review interprets the MDGs broadly, as representing quantified development goals and targets rather than the precise percentages and timeframes specified. Since the MDGs are global targets, they must be localized to fit country circumstances. For example, not every country can reduce under-five mortality by two-thirds by 2015. 1.8 The concepts of partnership and country-ownership are at the heart of the MDGs. Therefore, the Review stresses mutual accountability; to claim that the Bank alone is accountable for achieving any or all of the goals would undermine these core concepts. At the same time, it is important for the Bank to consider how effectively its assistance is contributing to progress toward the agreed-on goals, in keeping with the concepts of distinct responsibility and mutual accountability. This is unlikely to be easy and will require that the mechanisms and processes through which Bank actions get converted into outcomes are spelt out by a clear strategy (with an explicit logical framework). The links between inputs, outputs, and outcomes should be further analyzed via existing and continuing research. 3. For a discussion of this issue, see H. White "Using Targets to Measure Development Performance" in "Targeting Development: Critical Perspectives on the Millennium Development Goals," R. Black and H. White (eds.), forthcoming. Theory-Based Evaluation can provide useful guidance as discussed in C. Weiss 1998. 4 1.9 The MDGs imply continued attention to a number of themes and sectors that are already high priorities in Bank operations-for example, economic growth, poverty reduction, human development, and environmental sustainability. They also imply an increased emphasis on outcomes and their measurement, the multisectoral determinants of outcomes, and partnerships. In other areas, the MDGs may entail changes or innovations in current Bank practices and programs, for example, with respect to the allocation of resources across countries and sectors. THE RIsims AND CHALLENGES 1.10 The MDGs serve as a visionary challenge to help galvanize new energies and resources for the development agenda with a focus on outcomes. The 2002 United Nations Conference on Financing for Development (held in Monterrey, Mexico) helped focus the effort by laying out basic elements of a new global partnership. The adoption of the MDGs, however, also presents risks to the Bank and to the larger development community-risks posed by the cynicism that failure (or only partial success) could engender. Such cynicism is a danger, given that many health and social sector goals the development community had set for itself over the past quarter century, as shown in Chart 1.1, remain either unattained or only partially attained. It is already clear, as Chapter 2 illustrates, that given current trends of progress, many countries and some regions will be not be able to achieve the Millennium Development Goals by 2015.4 1.11 Development challenges posed by the MDGs include: * Incorporating country priorities and conditions. Even though most MDGs are defined in relative rather than absolute terms, individual countries will still have to customize goals and targets according to their national priorities, institutional capacity, and level of progress. Careful attention must be paid to assigning relative weights across the MDGs and the sectors and subsectors they cover, as well as across regions within a country. Choices will also have to be made about the relative emphasis on "access" versus "quality," and any resulting tensions and trade-offs resolved striking the right balance. Implementing the MDGs and adapting the targets appropriately to conditions within the countries also requires that many of the necessary strategies and actions be adopted and "owned" at the level of sub-national (especially municipal) governments, working with other local stakeholders. Assisting countries to improve the coherence and effectiveness of fiscal, administrative, and political decentralization will be critical to achieving MDG outcomes. * Harnessing the contributions of sectors without an aplicit MDG goal or target. The MDGs focus explicitly on some sectors and thematic areas, and only implicitly on others. A danger is that the sectors and themes not explicitly mentioned-for example, rural development or transportation-will be neglected. Yet progress in these areas is vital to achieving MDG outcomes. Economic growth, while not mentioned specifically among 4. Measuring country progress against global MDG targets is an interim step, pending the establishment of localized, realistic targets for each country. Chart 1.1: Targets are Often Set, But Seldom Met... WHO World Summit for Education For All International International World Summit for Fourth World Second World IDTs Children (EFA) Conference on Conference on Social Conference on Conference on UN Nutrition by World Population and Development Women Human Jomtien Food Summit Development Development Settlements Rome Cairo Copenhagen Beijing (Habitat II) 1977 1990 1990 1992 1994 1995 1995 1996 1996 I I I I I I I I i Health for All by Reduce child Ensure Education Reach consensus Improve population Reduce Infant Reduce maternal Promote the 7 targets related to 2000. Spend at mortality by 1/3 of for All by 2000 on roots of policies, mortality, maternal mortality, Increase importance of poverty, human least 5% of GNP 1990 rates, halve malnutrition reproductive health mortality, etc. school enrollment access to adequate development, & on health. Increase 1990 maternal through food, (matemal mortality, & literacy for girls & safe drinking water environment (See vt access to safe mortality rates & disease, & family planning women, etc. & proper sanitation Annex Ill) water to 85% malnutrition In behavior, bring services, etc). for all, along with coverage; children, achieve about major availability of sanitation facilities 90% Immunization reforms in five adequate shelter. In the home to 75% rates among Intersectoral areas coverage; children under one, to address Immunization to achieve 80% malnutrition, etc. 90% coverage, etc. primary completion rates; provide dean water & sanitation for all, etc. 6 the MDGs, underlies the achievement of each of the MDGs and its primacy cannot be overemphasized. The challenge is to ensure that quick, short-term gains are not supported to the neglect of more difficult but durable reforms, including in country-level governance structures, which would require action over the medium to longer term. * Focusing on outcomes among the poor, not just average outcomes. It would be possible to achieve the MDG global targets simply by focusing on a few large countries-for example, India, China, and Brazil. Within countries, MDG targets could be achieved by improvements in the MDG indicators among the non-poor. For example, a recent study points out that since the health MDGs are stated in terms of improvement in societal averages rather than in terms of gains among poor population groups within societies, improvements in any population group, including the better-off would produce progress toward the health targets.' It emphasizes that special efforts will be required to ensure that health interventions reach people below the poverty line if those people are to gain significantly from progress toward the MDG targets for health. Efforts to reach global targets should not lead to neglect of countries deemed insignificant to achieving the targets at the global level, or of difficult-to-reach population groups. The way that the Bank and countries interpret the MDGs needs to be clarified and examined. Do they treat the goals as a stimulus to enhanced efforts at reducing poverty in all member countries, albeit with an emphasis on the poorest countries? Or do they interpret them to mean a call to concentrate specifically on the poorest countries and within them on those earning less than $1 per day? The appropriateness of the interpretation will depend on country circumstances. Depending on the interpretation, there can be significant implications for the deployment of Bank resources across sectors and countries. * Identifting the results chain and monitoring appropriate intermediate indicators. Currently, MDG indicators lack measures of well-being that are important for capturing more qualitative elements of development effectiveness6 -for example, learning achievement. Furthermore, MDG indicators are a mix of output and outcome indicators. They need to be complemented by explicit intermediate indicators that can monitor the steps leading up to desired outcomes. Such indicators can be used in performance management and also to help reduce the attribution problems associated with outcome indicators.! In addition, the need for intermediate indicators is underlined by the fact that data on some MDG indicators are collected infrequently or with long lags, reducing their usefulness for routine performance management." Even where data are available, they may be of poor quality-for example, maternal mortality data are notoriously unreliable.9 The challenge is to build in-country data collection, analysis, and utilization capacity. 5. D. Gwatkin, forthcoming. 6. World Bank 2002w. 7. H. White "Using Targets to Measure Development Performance," in "Targeting Development: Critical Perspectives on the Millennium Development Goals," R. Black and H. White (eds.), forthcoming. 8. R. Black and H. White (eds.), forthcoming. 9. Ibid. 7 Addressing incentives for outcomes and for monitoring them. Governments and donors often lack incentives to measure how they are performing. Without attention to incentives for outcomes and for monitoring them, the MDGs will not fulfill their promise. 1.12 There is an emerging literature about managing the risks of target-driven programs; ee Box 1.3. 13ox 1.3: Managing the Risks of Target-Driven Approaches A major risk of target-driven approaches such as those embodied in the Millennium Development Goals is that they can distort priorities. For example, when the U.K. government set Health Service targets for hospitals to reduce the number of people on waiting lists, hospitals gave priority to cases that were easy to treat. Thus, while waiting lists were quickly reduced, attention to acute conditions was delayed. Target-driven approaches can avoid such priority distortion by: Localizing targets. Global targets should be supplemented by, or adapted to, local targets with local definitions and local indicators. The desirability of including key stakeholders in efforts to locally define and monitor their implementation cannot be overemphasized. In order to avoid manipulation or distortion of actions driven by quantitative targets, a process that gives the ultimate stakeholders a prominent role in defining targets and tracking their implementation will be helpful as illustrated by ihe Bank's interventions targeted at slum-dwellers. ,Combining quantitative and qualitative targets. Targets should incorporate qualitative as well as quantitative elements. Client satisfaction with health services is equally important, if not more important, than number of hospitals built-even if not as easily measured. Creating the right incentives. Targets can be used to structure incentives. For example, many Drganizations use performance-related pay to do so. It is important, however, to design incentives that reward genuine value-added or difference-made characteristics. For example, exam results can be a poor guide to the quality of teaching: it may be easier to teach prosperous, well-motivated children who easily obtain good results than poor and poorly motivated children for whom average results may represent an excellent outcome. Reciprocal accountability. In the spirit of learning organizations, the principle of reciprocal accountability implies that managers and staff, central planners and line ministries, headquarters and field units all share responsibility for success and failure. Achievement or failure must be looked at in the context of a system as a whole. Adopting a process approach that allows quick changes. Plans should be made, but it should be possible to make revisions quickly as conditions warrant. This is particularly important where the operating environment is evolving. Source: Based on "Lost in Translation? Implementation Constraints to Results-Based Management," S. Maxwell, 2002; "New Thinking on Poverty in the UK: Any Lessons for the South?" Maxwell and Kenway, ODI Poverty Briefing 9, 2000; and World Bank staff comments. 8 ORGANIZATION AND SOURCES 1.13 The next three chapters discuss how the Bank's country, sector, and global programs have addressed the MDGs, and the implications for future work. The final chapter concludes, and the Appendix presents project performance results. Figure 1.1 summarizes the results of project outcomes, which have Figure 1.1: Project Outcomes Show An continued to improve -in FY01 they Upward Trend exceeded for a second year the 100% Strategic Compact target of 75 percent. E Achieving the MDGs will require that 80%_-* 0 'X this solid performance is sustained and the Bank scales up impact to help 60% -- 40% clients achieve country-level 40% - By project improvements in economic well-being, - Weighted by disbursement human development, environmental M sustainability, and other relevant FY90 FY92 FY94 FY96 FY98 FY00 FY02* o. Source: Business Warehouse, World Bank 2002. Note: * Partial 1.14 In addition to sources outside OED and the World Bank, this Review draws upon ED evaluation findings from: S Project Performance Assessment Reports and Evaluation Summaries for 331 projects evaluated since the last Review, and more than 5,000 previously evaluated projects in OED's database. t Country Assistance Evaluations prepared during FY99-02. Studies of sectors and thematic areas. 9 2. Country Programs and MDGs Our primary focus remains at the country level within the framework of the Comprehensive Development Framework/Poverty Reduction Strategy Paper/Country Assistance Strategy and informed by the five corporate advocacy priorities. Strategy Update Paper For FYO3-05 "Implementing the World Bank's Strategic Framework," The World Bank, 2002" INTRODUCTION 2.1 The MDGs represent global goals and targets. They are expressed in terms of common standards, such as the number of people living below $1 per day, in order to enable cross-country comparisons. Global targets also play an important role in helping mobilize additional resources for development assistance-important because Bank estimates suggest that the additional cost of achieving MDGs for poverty reduction, education, health, and the environment would require doubling the real value of aid flows." 2.2 At the global level, the MDG for eradicating extreme poverty is likely to be met on current trends (Figure 2.1 a). The universal primary education MDG could be achieved if a shift from current trends occurs (Figure 2.1b). This global picture obscures important regional differences. Achieving the MDG for poverty poses particularly daunting challenges for Sub- Saharan Africa (Box 2.1). Broadly, many of the countries of Eastern Europe, Latin America and the Caribbean, and East Asia are on track to achieve many of the MDG targets, while few Sub-Saharan African countries are likely to meet them in light of current trajectories." Wide disparities are evident within South Asia where several countries are still far from achieving many of the MDGs.3 Annex IV presents the prospects of achieving the global MDG targets for low-income, low-middle income, and upper-middle income countries. While significant efforts have been made to compile the data, uneven quality and availability of the data make tracking progress toward the attainment of the MDGs a difficult task. For some goals, such as improving maternal health and reducing the spread of tuberculosis, insufficient data makes tracking progress particularly difficult. Moreover, the data are derived from available country averages which themselves mask important variations that occur within each country.4 10. The five corporate advocacy priorities are: investment climate; public sector governance; empowerment, security, social inclusion; education; and health. The first two priorities fall under the pillar "building the climate for investment, jobs, and sustainable growth," and the latter three under "investing in poor people and empowering them to participate in development" 11. S. Devarajan et al., 2002 estimate that the incremental resources of achieving the MDGs for poverty reduction in low-income countries that are not projected to reach their poverty goal would cost $54462 billion depending on the extent of policy reform. The estimated additional cost of reaching MDGs for education, health, and the environment is $35476 billion, although it is acknowledged that there is double counting. 12. World Bank 2002w. 13. Ibid. 14. Ibid. 10 Figure 2.1a: Proportion of Population Figure 2.1b: Primary Completion Rate In Living on Less Than $1 Per Day in Developing Countries (Population Developing Countries" Weighted) 30 0 Required Trend , 25 Eo001 290 ~ - p20 CurntTr * RequiredTrend , 5.80 Cunent Trend . . . . . .70 990 994 998 2002 2006 201 2014 1990 993 996 999 2002 2005 2008 2011 2014 Source: Data for 1990, 1999 and projection until 2015 are Source: Data for 1990-2000 and the projection until 2015 from GEP 2003. The required trend is based on are from HDNED. The required trend is based on exponential extrapolation using 1999 data and one-half of exponential extrapolation using 2000 data and 100 1990 data for the year 2015. percent primary completion rate for the year 2015. 2.3 Measuring country progress against global MDG targets is an interim step, pending the establishment of localized, realistic targets for each country. If the MDGs are to have an impact on development programs, they need to be tailored to each country's current level of progress and its institutional capacity for improvement. The goals also need to reflect local priorities, as established by sovereign governments and their citizens. Vietnam's PRSP offers a good illustration of how the MDGs can be tailored to country circumstances (Box 2.2). The Bank's CAS for Vietnam is also grounded in the same goals. 2.4 This chapter assesses how CASs and other instruments of Bank assistance have addressed the MDGs. It also examines the treatment of quantitative targets in PRSPs. It draws upon OED's Country Assistance Evaluations (CAEs)6 as well as a review of recent CASs7 and PRSPs.1 The chapter focuses primarily on MDG Goals 1 through 6, for which 15. The $1 a day is in 1993 consumption purchasing power parity terms. The numbers are estimated from those countries in each region for which at least one survey was available during the period 1985-2001. Survey dates often do not coincide with the dates in the table. To line up with the dates, the survey estimates were adjusted using the closest available survey for each country and applying the consumption growth rate from national accounts. Using the assumption that the sample of countries covered by surveys is representative of the region as a whole, the numbers of poor are then estimated by region. This assumption is obviously less robust in the regions with the lowest survey coverage. The headcount index is the percentage of the population below the poverty line. Further details on data and methodology can be found in M. Ravallion and S. Chen (2000). The historical series to 1999 was updated in October 2002 for the 2003 edition of Global Economic Prospects. 16. OED has prepared a total of 53 Country Assistance Evaluations (CAEs) since 1995 when CAEs were initiated. All CAEs completed in FY99-02 were reviewed, amounting to 38 CAEs. 17. Eighteen CASs were reviewed. The main criterion for the choice of the CASs was that they were prepared in FYO1-02 (until May) and had a completed OED Country Assistance Evaluation (CAE) in FY99-02. This criterion resulted in less than the desirable number of large and medium countries and hence Bangladesh and Turkey were also added. For all these countries, there were CASs in FY01-02 and at an earlier point in time so that changes in the focus of CASs on MDGs could be compared. The 18 countries were Argentina, Bangladesh, Bulgaria, Burkina Faso, Chile, Egypt, El Salvador, India, Indonesia, Jamaica, Kazakhstan, Maldives, Mexico, Morocco, Russia, Turkey, Uganda, and Ukraine. These CASs were distributed as follows: Sub-Saharan Africa 2, East Asia Pacific 1, South Asia 3, Eastern Europe and Central Asia 4, Middle East and North Africa 3, and Latin America and the Caribbean 5. Other countries for which a full CAS was completed in FYO1-02 (but for which no CAE was available) were Armenia, Belarus, Belize, Chad, Djibouti, Latvia, Mauritania, Romania, Slovak Republic, Turkmenistan, Uzbekistan. 11 there were corresponding IDTs. Therefore, these goals have had development targets for a longer period than have the goals in sectors such as water or urban development. Hence there is a better chance of Goals 1 through 6 being reflected in Bank-supported country programs. Box 2.1: Regional Picture for Three MDGs South Asia and Sub-Saharan Africa are home Figure 2.2: Proportion of Population In South Asia (SA) to hlf f th wold'spoo. Asdepcted~ And Sub-Saharan Africa (SSA) Living On Less Than $1 Per to half of the world's poor. As depicted in Dy1 Figure 2.2, South Asia will be able to achieve the poverty MDG, but Sub-Saharan Africa will 50 need to more than triple its currently slow pace SSA Current of poverty reduction if the poverty MDG is to - 40 Trend . R be achieved in the region. Pro Cutrent Tof P ut Trend 30 SA: Required Tre 20 1990 1993 1996 1999 2002 2005 2008 2011 2014 Source: Data for 1990,1999 and projection for 2015 are from GEP 2003. The required trend is based on exponential extrapolation using 1999 data and one half of 1990 data for the year 2015. Figure 2.3: Primary Completion Rate in South Asia Figure 2.4: Under-Five Mortality In South Asia (SA) and (SA) and Africa region (AFR) (population weighted) Sub-Saharan Africa (SSA) 100- 170 SA: Required Trend 10 000000 SSA. Current Trend ' eTrend go 0AFR Required i00o S.Current Trae 60 ag AFR Current Trend SA Required Trend 40 1 1 1 2 2 2 2 2014 1990 1993 1996 1999 2002 2005 2008 2011 20141 Source: 1990-2000 figures provided by HDNED. The trend lines are based on exponential extrapolation. Source: Data for 1990 and 2000 are from World Development Indicators 2002. The required trend line is based on exponential extrapolation using 2000 data and on-third of 1990 data for the year 2015. In the case of primary school completion rates, As for under-five mortality (Figure 2.4), both Sub- progress in South Asia is slow, as shown in Saharan Africa and South Asia will require Figure 2.3. In Africa, the completion rate has significant improvements from current trends to meet stagnated around 51 percent over the past the goals. Sub-Saharan Africa will need a decline of decade. The region will have to have an annual 8 percent per annum, and South Asia 6 percent per rate of 4.5 percent in order to achieve 100 annum. percent primary school completion rate by 2015. 18. All 12 PRSPs completed to the end of FY02 were reviewed. They were PRSPs for Burkina Faso, Mauritania, Mozambique, Niger, Tanzania, Uganda, Zambia, Bolivia, Honduras, Nicaragua, Albania, and Vietnnm. 19. See footnote attached to Figure 2. R a. 12 Box 2.2: Tailoring the MDGs to Vietnam's Priorities Vietnam's 2001 Comprehensive Poverty Reduction and Growth Strategy chose eight themes to guide the country's fight against poverty. Several themes were linked to the MDGs, while four goals were added to reflect local conditions and interests. These included reducing vulnerability, improving governance, reducing ethnic inequality and ensuring pro-poor infrastructure development. The targets were customized to suit the Vietnamese context. For example, Vietnam's child mortality rates have already dropped significantly in the last decades, so it was deemed unrealistic to adopt the MDG target of reducing child mortality by two-thirds. Instead, a target of reducing child mortality from 58 to 32 per 1000 births by 2010 was adopted which would mean a 45 percent reduction instead of the 67 percent (two-thirds) reduction indicated in the child mortality MDG. Considering that the child mortality rate in Vietnam was already 34 per 1000 births in 2000, the target adopted for 2010 appears modest, though. On the other hand, Vietnam's target for reducing malnutrition is more ambitious than the MDG-it has been set at a 60 percent reduction by 2010, rather than the MDG target of a 50 percent reduction by 2015. Source: Bank staff interviews; "Poverty Reduction Strategy Papers (PRSP}-Progress in Implementation." Development Committee Paper, DC2002-0016. Washington D.C.: World Bank, 2002. How HAVE THE BANK'S COUNTRY PROGRAMS ADDRESSED THE MDGs? The Bank's country programs have increasingly focused on poverty reduction and human development. 2.5 Poverty reduction and human development have been important institutional emphases in the Bank for three decades-in the early 1970s, former Bank President Robert McNamara announced a renewed focus on poverty reduction and human development. Since then, World Development Reports (WDRs) of 1980, 1990, and 2000/01 have further developed these themes, with the latest WDR adding empowerment as a key factor for poverty reduction. The Bank's country programs have become increasingly aligned with this institutional focus and thus with the MDGs in these areas. 2.6 Evaluation results confirm that the Bank's country programs pay significant attention to poverty reduction, primary education, and primary health services. The Bank's forthcoming CAS Retrospective, which reviewed CASs completed over FYOO-1(first half), rated 61 percent of the CASs satisfactory or better with respect to their poverty focus based on revised criteria (71 percent based on original criteria)." The CASs showed significant improvement in terms of poverty diagnosis. About 90 percent of the CASs gave fair or good treatment to the social sectors. The Retrospective also noted that virtually all CASs identified macroeconomic issues as priority areas, while 71 percent identified infrastructure and 29 percent energy as priorities, and most of the CASs addressed rural issues. Issues identified as needing greater attention included increasing the coverage of environmental issues (including natural resource management), linking identified gender priorities with follow-up actions, 20. World Bank, forthcoming(c), CODE discussion scheduled for January-June 2003. The Poverty Board's revised criteria to evaluate CASs place more emphasis on monitoring and evaluation, adequate analysis of past interventions on the poor, and the expected impact of planned interventions. 13 and integrating analytical findings from the work on governance into the design of the Bank's country programs. 2.7 A number of OED CAEs have noted improved treatment of poverty in the Bank's country programs. For example, the FYO1 India CAE noted that the relevance of the Bank's country assistance strategy had improved substantially over the past two years, with a sharper focus on poverty reduction, a more selective approach to state assistance, and greater attention to governance and institutions. Similarly, the FY00 Burkina Faso CAE noted that the country's adjustment and debt relief operations had focused on poverty benchmarks, which had been a critical factor in gaining government attention to these issues. 2.8 OED CAEs have also observed that even in some countries with low levels of poverty, poverty reduction has been a major focus of the Bank's country programs. In Chile, for instance, even though extreme poverty had dropped to about six percent of the population, poverty reduction was still part of the Bank's country assistance strategy. Given the absence of a common strategic framework for poverty reduction in middle-income countries of the kind provided by PRSPs in low income countries, the Bank will have to take special measures to ensure that poverty reduction is not neglected in these countries especially in specific regions or among particular population groups, and to involve governments and other stakeholders in designing appropriate strategies. 2.9 The Bank's country programs in transition countries have been less successful at incorporating poverty goals. In Kazakhstan, for instance, the OED CAE noted that in hindsight, the Bank along with other donors, was overly optimistic in its expectation that the transition from a planned to a market economy in the former Soviet Union countries could be accomplished in a short time and at low social costs. The strategy did not focus forcefully enough on institutions, protection of the poor, or gender issues. According to the FYO1 Kazakhstan CAS, the findings of the Public Expenditure Review will be used to shape a Social Protection Adjustment Reform Loan that will help reorient public expenditures toward investment in infrastructure and social assistance. And a household survey for Kazakhstan will provide an improved basis for poverty analysis. PRSPs use targets more frequently than CASs--but PRSP targets are ambitious. 2.10 PRSPs have incorporated targets into development programs more commonly than have CASs. As Box 2.3 shows, the majority of sampled CASs lacked quantitative targets for key MDGs such as poverty reduction, under-five mortality, and universal primary enrollment. By contrast, the 12 PRSPs prepared during a similar period as the CASs but for a different set of countries, all established poverty reduction targets based on national poverty lines?.2 All PRSPs (for countries with less than universal primary education) also set targets for universal primary enrollment and for reductions in maternal mortality. As more PRSPs are completed, they will provide an improved basis to introduce targets into CASs for low- income countries. 21. Two countries -Uganda and Burkina Faso-had both a CAS and a PRSP. In both cases, the PRSP pre-dated the CAS, and yet the PRSP included many more quantitative targets than the corresponding CAS. 14 Zou 2.3: Quantitative Targets More Common in PRSPs than in CASs" Is there a quantitative target? CAS PRSP Targes for redwclng ational poverY rates No Yes Targeesfor redocing i0fant mortabiy C Targetsfor reducing za3ersal morliy I Targetsfor achieving universal primary education' Targets for achieving gender eqmality in educati(o 2.11 The PRSPs' use of targets needs to be improved by a more realistic assessment of their achievability. While all PRSPs reviewed stated the per capita growth rate the country will need to reach the poverty target, only a quarter of the countries appear likely to reach that rate based on past performance. This type of disconnect also affects the universal primary Figure 2.5: Burkdna Faso - Current and enrollment target. Based on past trends, only Required Trend for Under-five Mortality Bolivia, Uganda, and Vietnam are on track to achieve universal primary enrollment by 2015. As for under-five mortality, only five countries 1 200 d stand a good chance of meeting their goals, 150- based on past performance. In the remaining seven countries, the target is unlikely to be R............ achieved-as illustrated by Burkina Faso So Requi ..d.T... (Figure 2.5), where the rate of decline would I 1990 1993 1996 1999 2002 2005 2008 2011 2014 have to increase from the current 1 percent to Source: World Development Indicators 2002. more than 6 percent. While government and Note: Required trend is based on exponential extrapolation using 2000 data and one-third of 1990 donor actions can influence trends, the expected data for the year 2015. changes must still be realistic. The 2002 Joint Staff Review of the PRSP Approach highlighted setting more realistic PRSP goals and targets as a challenge.2 22. Based on 18 sampled CASs and 12 PRSPs (see footnotes 17 and 18). 23. The pie chart is based on 15 CASs since three countries had already achieved universal primary education. For PRSPs, the pie chart is based on 11 PRSPs since one country had already achieved universal primary education. 15 2.12 The CASs that do include targets do not always incorporate a Bank strategy to help he country achieve them. In the case of maternal mortality, for instance, five CASs set the goal of a three-quarters reduction by 2015-but none discuss the current rates and causes of mortality or ways to accomplish reductions. As for under-five mortality, three CASs include ;ipecific health projects, but none of these CASs relate the projects to the objective of reducing child mortality. The Bank's Health, Nutrition, Population Anchor is currently in the process of developing an Action Plan for FYO3-05 based on an individual country approach for all countries. Targets for health outcomes in terms of the MDGs are being set in regional and country-level discussions as an integral part of Action Plan development. Performance monitoring is increasing, but systematic monitoring and evaluation of ,utcomes is not yet a core component of country programs. 2.13 Many OED reports evaluating country programs in the 1990s concluded that attention to monitoring and evaluation was weak.25 The Bank's forthcoming CAS Retrospective oonfirms that monitoring remains one of the weakest areas of CASs.26 It notes that while there have been improvements, only about 40 percent of recent CASs are satisfactory or better; about 50 percent of CASs do not contain core targets; and about 60 percent do not distinguish between country and Bank performance targets. The Bank has recently launched major initiatives to better manage for results as summarized in Annex V. These are likely to improve attention to monitoring and evaluation but cannot be expected to bear fruit swiftly or easily. 2.14 PRSPs, despite their greater attention to quantitative targets, often do not yet follow through with monitoring and evaluation programs. Only two PRSPs-those for Uganda and Vietnam-indicate that a program will be in place to monitor and collect information on the three poverty MDG indicators." Another four PRSPs (Mozambique, Mauritania, Niger, and Zambia) state that the country will monitor two of the three indicators, and the remaining six PRSPs will monitor and collect information on poverty headcount alone. With regard to education, only Niger and Vietnam will monitor all three key MDG indicators (net enrollment; proportion reaching grade 5; and illiteracy of 15-24 year olds). The choice of indicators in the PRSPs could differ from the precise MDG indicators depending on country conditions and program objectives. A recent PRSP Progress Report2 recommended that PRSPs should be more explicit in identifying short-term and intermediate indicators based on policies and programs and linking them to longer-term development objectives. It also recommended that in preparing PRSPs, early attention should be paid to the monitoring and evaluation framework, the definition of intermediate indicators, the collection of baseline data, and the assessment of institutional capacity for monitoring and evaluation. The 2002 24. World Bank 2002y. 25. World Bank 1994a; World Bank 2002c. 26. World Bank, forthcoming(c). 27. Namely, proportion of population below $1 (PPP) per day, poverty gap ratio (incidence x depth of poverty), and share of poorest quintile in national consumption. 28. World Bank 2002y. 16 PRSP for Albania contains a detailed discussion of how progress on PRSP objectives will be monitored. Poverty-related analytical work underlying country programs needs further strengthening. 2.15 The analysis, data, and projections that underpin the Bank's programs could be further improved. QAG reviews of ESW confirm that there is room to enhance the quality of the Bank's ESW relating to poverty-reduction strategies and poverty-related analysis, to poor policy environments, and to fiduciary issues.2 2.16 Poverty Assessments, done in the Bank since 1991, are an important source of the analytical work underpinning the Bank's poverty-reduction efforts. Al999 OED review found that there had been a modest improvement in the proportion of poverty assessments rated satisfactory on "economic quality."30 But it also found that over 60 percent of the poverty assessments did not state their objectives with the necessary specificity, and more than half of the reviewed assessments conducted limited policy analysis that was not usually based on rigorous methodological approaches---the assessments rarely incorporated a clear conceptualization of the link between macro, structural, and sectoral reforms and poverty, or of distributional outcomes. The 2000 OED evaluation of the Bank's poverty-reduction strategy found that nearly half of the poverty assessments did not adequately address the individual elements of broad-based growth, social service provision, and safety nets, nor did they justify the balance among these elements in strategy recommendations.' It also noted that few CASs clearly explained the meaning of broad-based growth or established the links between growth-oriented policies and poverty reduction within the context of specific country conditions. Moreover, few of the Bank's Public Expenditure Reviews helped clients sort out trade-offs in public expenditure allocation or suggested how sector or poverty outcomes could be monitored.12 2.17 There has been continued and substantial progress in the availability of household survey data in many regions, including in Sub-Saharan Africa. In Egypt-where lack of access to poverty data had been flagged as a major problem in the FY01 CAS-the Bank now has access to primary household survey data. This has enabled the Bank and the Government to undertake joint analytical work on the incidence and causes of poverty, which is expected to lead to the preparation of a comprehensive poverty reduction strategy. Overall, however, the lack of access to adequate poverty data remains an issue and has been flagged in many CASs. Given sample size issues, many surveys do not yet allow enough 29. World Bank 2002a. 30. World Bank 1999d. The review made a distinction between "economic quality" (which included, for example, and quality of the poverty profile, the policy analysis, and the policy recommendations, and the quality of the assessments coverage with prioritization and selectivity adequately justified), and "social, political, and institutional quality" (which included, for example, a multidimensional approach to poverty profiles, soundness of methodology for the qualitative/participatory work, and triangulation of quantitative and qualitative analyses). 31. World Bank 2000e. 32. Ibid. 17 representative breakdown of data, thus limiting the ability to conduct analysis across or within cities, sub-regions, and population groups in a country. 2.18 There are a number of efforts underway to improve the analysis of poverty data. For example, as part of the Russia CAS, the Russian government, DFID, and the Bank will support a collaborative program to measure and analyze poverty, to improve methodology, and to develop policy recommendations on how to reduce poverty and improve social protection of vulnerable groups. Implementation of this program is one of the triggers for the Bank's base case lending scenario for FYO3-05. In some countries, the focus on poverty is diluted by the limited capacity to analyze available data. The capacity to analyze the factors explaining the current levels in MDG-related indicators and other relevant development outcomes is a requirement for understanding trends, formulating strategies to change those trends, and making projections for the future. 2.19 The focus on MDGs will increasingly require that the Bank and countries make projections for various indicators to provide the basis for designing strategies for achieving MDGs or similar targets. Of the 18 sampled CASs, less than half discussed projections of future poverty levels. Although this was an improvement over previous CASs for the same countries, in which none had discussed projections of poverty, only the FY00 CAS for Burkina Faso based poverty projections on a formal model. For some countries, CASs used the government's targets or poverty projections such as those prepared for Five-Year Plans or "aligned" the Bank with the MDGs adopted by the government, but did not explain the reason or basis for the alignment. A CAS should assess the analytic underpinnings and realism of such government projections and targets. 2.20 Different analytic methods used to make projections of poverty within the Bank often result in discrepancies between poverty projections prepared by country teams and projections prepared by the Bank's development economics research department (DEC). Depending on which projections are used, the implications for the design of poverty reduction strategies can be vastly different. It will be important for the Bank to ensure the soundness of the projections on which it bases its strategies. The East Asia and Pacific Region has for some years published short term projections of poverty which are used in the country dialogue on poverty reduction. The Latin America and the Caribbean Region has also recently prepared preliminary projections for selected MDGs to assess whether they will be achieved. Differences in projections resulting from different methodologies and data limitations should be explained to avoid confusion and to highlight the strengths and limitations of the different projection methods. Alignment of lending programs with country assistance strategies and MDGs needs greater attention. 2.21 Strategies for poverty reduction proposed in CASs are not always reflected in the actual assistance program. A 1997 Bank report on poverty in Sub-Saharan Africa concluded that although poverty assessments in Sub-Saharan African countries had done a reasonably good job of identifying the policy and strategic options that will assist the poor to become more active participants in growth, these options typically were not reflected in the Bank's 18 assistance strategies and operations." In its review of the poverty focus of projects under implementation in rural areas over the FY99-01 period, QAG found that projects were weakest in linking their interventions to the poverty diagnosis." Other sources have made criticisms of donor programs by referring to the "missing middle;" that is, donors lay out lofty development goals that are at best weakly linked to the activities supported on the ground." The differences between strategy and lending programs could result from natural disasters or the need to adjust to deliberate changes in government policies or donor programs. At other times, the main reason is likely to be a change in the government's budget or its priorities and the related lack of country ownership. 2.22 While each project proposed in the CAS may individually be consistent with the CAS objective, it is not always clear that the summation of Bank lending in the CAS is the best way to address the CAS objectives. The Bank's forthcoming CAS Retrospective finds that instrument choice and selectivity in lending and non-lending activities to support the CAS remains weak. Less than 60 percent of the CASs are rated satisfactory or better, on the basis of whether they used selectivity criteria to discuss the rationale for instrument choice.'6 2.23 The overall sectoral composition of Bank lending has remained almost the same since the advent of the MDGs while there has been a shift in lending among some themes between FY97-00 and FYO1-02 as shown in Figure 2.6 (the correlation coefficient is 0.83 for themes and 0.99 for sectors). Increased public expenditures in sectors that are explicitly mentioned in the MDGs are not necessarily the way for countries to achieve the goals. For example, background work for the 2004 WDR "Making Services Work for the Poor" indicates that public funds may be spent on the wrong services and people, they may be sucked away by corruption, they may reach service providers who lack the incentives to provide quality services, or there may be no demand for the services provided. The Bank's Public Expenditure Reviews have also noted rapidly decreasing returns from increasing expenditure allocations to education in the absence of sector reform and increased institutional capacity. Nevertheless, lending allocations across sectors need to be assessed-in the context of each and every CAS-for their consistency with the objectives of poverty reduction and other MDGs. Closer scrutiny is needed of the links between the analytical components of each CAS that asses the country's development challenges, the Bank strategy proposed to address the issues identified, and the lending program and other instruments proposed to implement the strategy (including the role of the private sector and development partners). 33. World Bank 1997c. 34. World Bank 2002w. 35. H. White and D. Booth "Using Targets to Design Country Strategies," in "Targeting Development: Critical Perspectives on the Millennium Development Goals," R. Black and H. White (eds.), forthcoming. 36. World Bank, forthcoming(c). 19 Figure 2.6: Lending Composition Has Shifted by Theme But Not by Sector 100% Theme 100% Sector Other Other 80% ------- --------------- ---... 80% ------ Others -------------- Others ----- RD UD 60% 70 - --7----- - P+SD---- -------- --- 60% ----- Rf ---------60% ' S F SPTSE . SP+SD _7_WF "gp- WSFP 40% - ------ --------------- PS ------ 40% ------- TIS ------------ -- ------ PSDEdu 20% - ------ --------------- ------ 20% ------ E&M -------------- E&M ------ PSG PSG Fin Fin EM EM 0% - - - - - 0% - 1997-99 2000-02 1997-99 2000-02 Source: Business Warehouse, World Bank 2002. Sector: Fin/Finance; E&M/Energy & Mining; Edu/Education; HSS/Health and Other Social Services; AFF/Agriculture, Fishing and Forestry; WSFP/Water, Sanitation and Flood Protection; Trans/Transportation; Others/Law and Public Administration, Information and Communications, Industry & Trade; Theme: EM/Economic Management; PSG/Public Sector Governance; PSD/Financial and Private Sector Development; SP+SD/Social Protection+Social Development; RD/Rural Development; UD/Urban Development; Env/Environment; Others/Rule of Law, Trade & Integration. There are usually no trade-offs between adjustment lending and government expenditures for human development 2.24 What role does adjustment lending play in the Bank's support for the MDGs? Adjustment lending is designed to address obstacles to growth and poverty reduction consistent with the poverty reduction MDG. There has been debate about whether such lending "crowds out" lending to the social sectors within a country assistance program. OED's 1995 review of the question concluded that in most countries adjustment lending did not have a negative impact on the social sectors.37 An OPCS report" found that social expenditures in adjusting countries rose-as a share both of total expenditure and GDP- relative to countries with no adjustment program. Social expenditures per capita were also found to rise substantially more often than they fell one to three years after an adjustment operation. On the other hand, a 1999 OED review of the "higher impact adjustment lending" (HIAL) program in Sub-Saharan African countries in 1999 did find some evidence that, based on data for a limited sample of countries, adjustment programs have had a negative impact on the share of health and education expenditures in GDP.39 It is conceivable that even where the volume of post-adjustment social expenditure declines, its poverty reduction impact may increase due to increased efficiencies or greater access to services. While Sub- Saharan African countries may be a special case, the issue is of relevance for Bank adjustment and social sector lending and for its actions to address the MDGs, and will need continuing attention. 37. World Bank 1995. 38. World Bank 200lb. 39. World Bank 1999b. 20 WHAT ARE THE IMPLICATIONS FOR FUTURE BANK COUNTRY PROGRAMS? CASs must include quantified and time-bound objectives and targets. 2.25 The global MDGs must be translated into country-specific goals with quantified and time-bound targets and effective strategies for achieving them. This will require a realistic assessment of capacity and resources at the country level and a credible strategy to reach the targets that makes the necessary trade-offs. The Bank's CAS, as the instrument for arraying Bank interventions in support of a results-focused strategy, must specify quantified and time- bound targets, relate them to the MDGs and other relevant development objectives, and ensure Bank interventions that will support them. 2.26 In low-income countries, the government's PRSP provides a key anchor for the CAS, especially if its targets are realistic and its strategy well-reasoned. In middle income countries, for which there is no common strategic framework for setting out the poverty reduction objectives and strategy, the CAS nonetheless should define quantitative and time- bound targets, and the Bank must engage with the government and other stakeholders in this process. Improved monitoring and evaluation measures are required to track the performance of Bank actions as well as country level progress in meeting agreed development goals and targets. 2.25 More attention to monitoring and evaluation in CASs and PRSPs is required, with respect both to Bank and to country-level performance measurement. Bringing about such improvements is likely to be expensive. In recent years, the Bank's direct costs on country monitoring, formulating and monitoring CASs, and providing global monitoring services, have fluctuated between 0.42 percent of total direct costs in FY99, 2.05 percent in FY00, 1.83 percent in FY01, and 1.97 percent in FY02.4 The adequacy of these resource levels should be reviewed and their use monitored. In addition, appropriate incentives must be ensured-in countries and within the Bank- to monitor and evaluate performance against development outcomes (as opposed to simply measuring performance in terms of resources disbursed, as has traditionally been the case). 2.26 The timeframe of the CASs and PRSPs is much shorter than the 15-year timeframe of the MDGs. Hence, it is important to identify and track appropriate intermediate indicators in the CASs and PRSPs that can provide interim measures of progress towards the final outcomes. 2.27 In 2002, the Bank launched an initiative, emerging from the Monterrey agreement, to give increased attention to "Better Measurement, Monitoring, and Managing for Development Results."' The two pillars of this approach are helping countries to adopt 40. Direct costs include: client services (viz., country, sector and global, and other services), support services (viz., receive training, financial, administrative, corporate, and sustaining costs), and other services. World Bank website http://act.worldbank.org/Glossaries. 41. World Bank 2002d. 21 results-based strategies and increasing the Bank's own results orientation. Annex V describes the initiatives supported by the Bank under each of these pillars; the Bank is currently preparing a plan for implementing them. In addition, the Bank has put in place a system which tracks the experience of more than 50 countries in implementing the Comprehensive Development Framework principles. This includes a focus on development results and the extent of links to the MDGs." The experience of the UK Department for International Development in addressing the MDGs, summarized in Box 2.4, should be reviewed for lessons it may have for the Bank. Box 2.4: How DFID is Using the MDGs to Influence Departmental Performance The U.K. Department for International Development (DFID) has placed the MDGs at the center stage of its corporate strategy, embracing them more strongly than any other donor. Target Strategy Papers have been prepared to indicate how DFID will address a single development target (e.g., halving world poverty) or a group of targets (e.g., health). MDGs are used as the basis of DFID's Public Service Agreements-written commitments of a department's objectives, related performance measures, and the activities to be undertaken in support of those objectives. Source: "Using Targets to Measure Development Performance," H. White, in "Targeting Development: Critical Perspectives on the Millennium Development Goals," Eds. R. Black and H. White, forthcoming, Routledge, London. 2.28 The Bank is also helping to build government monitoring and evaluation systems through Evaluation Capacity Development (ECD). Since 1999, ECD activities have started in 21 countries, including eight in the Africa region, five in Latin America and Caribbean, and four in the Europe and Central Asia region. In addition, related capacity-building is provided by the World Bank Institute (staff/client training and on-demand distance learning courses), and the Bank's Research and Africa Quality and Knowledge groups (statistical capacity building), among others. It is too early to know the results of these initiatives. 42. World Bank 2001 d.  22 3. Sector Programs and the MDGs We will use the Sector Strategy Papers and Corporate Scorecard more to track our performance with a longer-term comprehensive view as they relate to development effectiveness-including progress towards the International Development Goals (DGs). In particular, our focus will be on enhancing measurement of the Bank's impact on the IDGs. "Strategic Directions for FYO2-FYO4," World Bank Group, page 13, March 28, 2001 1 NTRODUCTION .1 By emphasizing a multidimensional concept of poverty that includes income and non- income measures of well-being, the MDGs highlight the importance of Bank action in a number of sectors. The urgency of such action is underscored by the large number of people iwho lack access to adequate income and basic services, shown in Box 3.1. Box 3.1: Many Millions Lack Adequate Incomes and Access to Basic Services Income: Nearly 1.2 billion people live on less than $1 a day, and 2.8 billion on less than $2 a day. Basic Healthcare: Two million African infants die each year, and as many again before they reach five. This figure has not changed in over two decades and represents 10,000 avoidable deaths every single day for over 20 years. Frimary Education: Just over half of all school-aged children complete primary school in Sub- Saharan Africa (55 percent) and South Asia (56 percent). In the developing world as a whole, the average primary school completion rate is 72 percent. Roads: About 900 million rural people in developing countries lack access to an all-weather road. Easic Urban Services: About 840 million people worldwide live in slums without basic services. Energy: About 1.6 billion people in Africa, Asia, and Latin America have no access to electricity, including 90 percent of the population of most Sub-Saharan African countries. About 2 billion people lck access to clean, safe cooking fuels and remain dependent on traditional biomass sources. Water: About 1 billion people lack access to improved water services; 2 billion people live without improved sanitation; 4 billion people live without sound wastewater disposal; and 3 million children die from waterborne diseases every year. Source: "Poverty Reduction and The World Bank: Progress in Operationalizing the WDR 2000/2001," World Bank, 2002; "Reducing Infant and Child Mortality," H. White, in "Targeting Development: Critical Perspectives on the Millennium D velopment Goals," Eds. R. Black and H. White, forthcommng, Routledge, London; "Educating the World's Children by 2015: A New Policy and Financing Framework", Human Development Network, World Bank, 2002; World Bank Sector Staff; and Secure Tenure Index, UN-Habitat. 3.2 This chapter examines how the Bank's sector programs (including sector strategies, analytical work, and lending) have addressed the MDGs and other relevant development outcomes, and the implications for future work. Sector strategies assessed are those for education, health, rural development, transportation, urban, energy, environment, and water, 23 as well as strategies for the private sector and the financial sector. Much of the evidence for this chapter comes from OED's extensive number and range of sector evaluations undertaken since 1998." 3.3 The fact that MDGs are multidimensional also means that they link to various sectors. Each sector is linked to more than one MDG, and many sectors are linked to many MDGs- highlighting the need for complementary solutions, integrated across several sectors (Table 3.1). Box 3.2 provides a focused look at the transportation sector. Table 3.1: Each Sector Is Linked To More Than One MDG, And Many Sectors Are Linked To Many MDGs Millennium Development Goal 5 o VlU 0UIl, .~ INCOME POVERTY AND HUNGER 1. Eradicate extreme Poverty & Hunger x X X X X X X x x X HUMAN DEVELOPMENT 2. Achieve Universal Primary Education X x x x X x x x x x 3. Promote Gender Equality & Empower Women X x x x X X x x x x 4. Reduce Child Mortality X x x x X x X X x X 5. Improve Matemal Health X x x x X x x x x X 6. Combat HIV/AIDS, Malaria & Other Diseases X x x x X x x x X x ENVIRONMENT AND SOCIAL DEVELOPMENT 7. Ensure Environmental Sustainability x X X x x x X X X X GLOBAL PARTNERSHIP 8. Develop a Global Partnership for Development x X X X X X x x x X Note: Bold X=strong links; x=less strong links. Source: Review of sectoral literature for ARDE. 43. World Bank. 1999a; 1997a; 1997b; 2002bb; 1996b; 2002f; 1999e; 2001k; 2000d; 2002u; Forthcoming(a); 1993b, 200211; 2002z; 2001j. 44. World Bank. 1994b; 1998a; 1999c; 1999d; 1999f; 2000e; 2000f; 2000g; 2001f; 2001h; 20011; 2002e; 2002t; 2002aa; 2002dd; 2002ii; 2002jj; World Bank, forthcoming(d). 24 Box 3.2: How Can Transport Contribute to Achieving the MDGs? Incvrme Poverty: Efficient transport systems are needed to support access to markets and economic growth, and recent studies show that transport has had powerful effects on reducing rural poverty. In urban areas, most slums have markedly inadequate roads and paths, which raises the costs of access to srvices and participation in the broader economy. The transportation sector contributes directly to poverty reduction by providing efficient operation, expansion, and maintenance of transport systems linked to places where poor people live. Huian Development: Primary and secondary education and mother and child health require safe local infrastructure (paths; sidewalks) and reliable and affordable local public transport over a wide area. The former is essential for access to primary schools and health centers; the latter for timely availability of teaciers and medical supplies and for access to secondary schools (especially for girls) and to hospitals. Environmental Sustainability: It is important to minimize the negative environmental side effects of tram port, including the possible depletion of forest cover and reduction of biodiversity as a result of road cons truction, as well as the effects of carbon dioxide and atmospheric pollutants from vehicle emissions. Soure: Fan, Hazell & Thorat 1999; Fan, Zhang & Zhang 2000; Kwon 2001; Drize & Kingdon 1999; "Morocco Impact Evahation Report: Socioeconomic Influence of Rural Roads," Report 15808-MOR, OED. Washington, D.C.: World Bank 1996: Sustainable Transport: Priorities for Policy Reform. Washington, D.C.: World Bank 1996; Cities in Transition: World Bank Urban and Local Government Strategy Washington, D.C.: World Bank 2000; "Educating the World's Children by 2( 15: A New Policy and Financing Framework," HD Network, Washington, D.C.: World Bank 2002. How HAVE THE BANK'S SECTOR PROGRAMS ADDRESSED THE MDGs? The Bank's sector strategies are consistent with the MDGs and rightly place them alongside other sector goals in a broader development framework. 3.4 The MDG themes of poverty reduction, human development, and environmental sustainability are not new to the Bank, and the Bank's sector strategies are consistent with therm. The Bank's sector strategies are not narrowly focused only on the themes of the MDGs. This is appropriate given that achieving MDG outcomes requires action on several sect oral and subsectoral fronts and that the Bank's sector strategies must be applicable over a range of differing countries. The Bank's education sector strategy-while consistent with the Education For All (EFA) initiative and therefore with the universal primary education MDG-also covers post-primary education. If countries are successful in achieving universal primary education, there is likely to be increased demand for secondary education. Similarly, the Bank's urban strategy goes beyond the MDG target of improving the lives of 100 million slum-dwellers by incorporating broader measures aimed at poverty reduction and improved health outcomes. OED's Urban Review noted that, in addition to supporting the slum MDG target, the Bank's urban strategy supports the MDGs specifically in regard to reducing poverty and lowering infant mortality rates. The Bank's energy strategy is consistent with the MDGs through its push for increased system efficiency and by seeking to improve access- for ihe poor as well as others-to convenient and efficient forms of energy. The Bank's environment strategy addresses the environmental sustainability goal and takes a broad approach in recognizing the importance of economic growth for poverty reduction and the inevitability of environmental change, and highlights the need for supportive policy, regulatory, and institutional frameworks for sustainable environmental management. 25 Sector strategies show increasing attention to poverty linkages. 3.5 Sector strategies-whether or not they have a specific MDG associated with them- show increasing attention to poverty linkages, a change from earlier strategies where such linkages were not always explicit. Although there is no specific MDG for rural development, the Bank's forthcoming strategy for rural development emphasizes that agricultural growth is the primary source of increased food production and the cornerstone of rural development, which is itself central to reducing poverty. The Bank's approach to rural development has evolved toward greater attention to food production as a critical ingredient for reducing poverty and hunger (Box 3.3). Box 3.3: Rural Development: A Re-balancing Act The MDG emphasis on poverty reduction and the elimination of hunger lends weight to current rethinking of rural development by the Bank and other partners. The four decades between 1960 and 2000 witnessed marked swings in the scope of rural development. The first shift was from heavy state involvement in agricultural production in the 1960s (green revolution technology; irrigation) to the all-encompassing but still state-led approach of integrated rural development programs in the 1970s. The second shift moved from the structural adjustment and privatization programs of the 1980s-with more emphasis on the efficacy and efficiency of service delivery than direct welfare impact-to, in the 1990s, supporting reforms focused also on poverty reduction through sustainable livelihoods (human development, environmental sustainability). The relative neglect throughout the 1980s and 1990s of the productive core of the rural economy-agriculture-persists, with research and extension investment down. More stress on agriculture production, especially food production, is still needed, without losing sight of complementary human development and environmental goals. The Bank's strategy is now shifting in this direction. 3.6 The Bank's energy strategy emphasizes access to clean household energy because it improves health and reduces poverty, for example, by reducing unhealthy fumes from open fires in closed spaces and by increasing productivity of home-based economic and educational activities. Compared to earlier private sector strategies, the Bank's 2002 private sector development strategy pays substantially more attention to poverty reduction as the overarching objective of the Bank Group's private sector activities. The strategy suggests two basic ways to support poverty reduction: improve the investment climate in order to extend the reach of markets, thereby providing greater job and income opportunities for poor people, and use markets or market-type mechanisms to provide better basic services to the poor. The Bank's financial sector strategy advocates greater economic efficiency and competitiveness in order to create more opportunities for the poor and more financial resources for social and other programs that benefit the poor. The strategy stresses reduction of financial sector vulnerabilities at both the global and national level (through, for example, the Financial Sector Assessments Program, a joint undertaking with the IMF), to reduce the risks of financial sector crisis and economic volatility that hit the poor the hardest. The strategy also seeks to help the poor more easily and cheaply obtain the financial services they most need (such as deposit taking, payments systems, credit, and insurance). 3.7 Some Bank strategies could benefit from stronger links to poverty reduction. For example, OED's 2002 review of the Bank's environmental policies and activities, covering 26 the period before the Bank's 2002 environmental strategy, found that links between poverty reduction and environmental sustainability could be strengthened by being made more explicit. Similarly, OED's 2002 gender evaluation, also covering the period before the Bank's 2002 gender strategy, concluded that the Bank needed to clarify how its gender policy is linked with the Bank's poverty-reduction mandate, and to explain the operational implications of Bank processes and practices. For the most part, the Bank's sector strategies have provided limited guidance on how coi ntry programs can address unresolved tensions and trade-off& 3.8 While the Bank's sector strategies rightly place the MDGs alongside other sector goals in a broad development framework, the strategies do not usually demonstrate how trace-offs should be addressed in situations of limited financial resources and institutional capacity. Greater guidance on the relative emphasis to be given in different groups of countries to specific subsectors and on the appropriate balance between access and poli cy/institutional strengthening would be helpful. For example, the education strategy's dual emphasis on universal primary education as well as on all levels of education, creates tension over the allocation of resources. A key question in the water sector is whether the MDGs will lead to a focus on expanding coverage in the short term at the expense of a focus on long-term sustainability, which would require institutional and policy strengthening. There is currently no Bank strategy on water supply and sanitation. This lack will be rectified by a Water and Sanitation Business Strategy now under development. 3.9 Regional strategy papers have been used as building blocks for overall sector strategy. For example, the health sector strategy has been adapted to countries with different ___________________________income levels and types of health Boxproblems, as shown in Box 3.4. The Strategy to Regional Conditions importance of varying the strategy to suit Straegy o Rgionl Cnditonsparticular conditions is also illustrated by The Bank's health strategy recognizes that needs the gender strategy. The Bank has focused vary by region and by country group-low or on addressing gender issues in countries middle income-and shapes priorities where there is noticeable gender disparity accordingly. In low-income countries, reducing in education enrollments or where infant and child mortality, maternal mortality, women's health status is poor. While these ar d the transmission of communicable diseases are appropriate entry points for Africa and are stressed. For middle-income countries, South Asia, they are less useful in regions accurate targeting of the poor and addressing w broader issues of health reform to improve here gend er poc in economic overall health systems operation are emphasized. oppo Aid regional strategy papers adapt the strategy to A more systematic analysis of choices and regional conditions. The East Asia Regional trade-offs in different groups of countries Health Strategy, for instance, defines an could be helpful in such situations. The assistance program for three country groupings MDG indicator "share of women in wage at d highlights those countries that differ from the employment in the non-agricultural mean with respect to mortality and morbidity sector" may be expected to bring increased arpiong vulnerable groups. attention to women's economic Source: "Health, Nutrition, and Population SecoTStrategy opportunities. Paper," World Bank, 1997. 27 3.10 In the absence of adequate guidance on how priorities should be set or trade-offs made, it is not surprising that sector strategies are not fully reflected in country programs. For example, OED's 2002 review of the water resource strategy found that while some country- specific water strategies have been developed, they are seldom linked to CASs or country priorities. To address this gap, the 2002 water resource strategy highlighted the need for strong country ownership and the development of country water resources management strategies. Similarly, OED's 2002 environmental review found that environmental sustainability had not been integrated into the Bank's core objectives and country assistance strategies. In particular, no clear linkages were forged between macroeconomic policy, poverty reduction, and environmental sustainability. However, in the wake of the 2002 Environment Strategy Paper, progress has been made in covering environment concerns in CAS and PRSP processes. Cross-sectoral linkages are increasingly recognized in country programs, but developing cross-sectoral strategies is still lacking. 3.11 The MDG focus on outcomes highlights the need for cross-sectoral strategies that address the multiple determinants of these outcomes. CASs are intended to be a key tool in organizing the different objectives and instruments into a coherent strategy and program. Although they are doing better than in the past, the CAEs still find that linking instruments to objectives remains a weak area, particularly when cross-sector links are involved. This Review finds that even the CASs that do include specific targets for infant or maternal mortality lack a clear strategy for reaching these goals-thereby missing a significant opportunity offered by the CAS for tapping cross-sectoral synergies. The Bank is increasingly recognizing cross-sectoral linkages in its sector programs (see, for example, Box 3.5). This recognition has not yet translated into cross-sectoral country strategies, however. Although truly integrated policy action can only emerge in national policy forums, greater creativity in Bank sector programs to maximize complementarities would further enhance their relevance to the MDGs-and to the country strategies seeking to achieve them. Box 3.5: Inter-Sector Linkages and the Role of Infrastructure Achieving targets in one sector requires contributions from many other sectors. For example, infant mortality is affected by access to safe water as well as the mother's education and access to transport. Recent work by the Bank's PSI Network has summarized the evidence from the literature and has also provided some econometric estimates of inter-sector linkages for achieving MDG targets. Infrastructure interventions (water, sanitation, energy, transport, and housing) are key inputs into the production functions for the poverty, education, health, and gender equality MDGs. The CAS should address these inter-sector linkages. The study recommends that: (1) countries develop intermediate targets and goals; (2) develop customized production functions for key objectives-region or country- specific; and (3) package multisectoral interventions around key objectives. Source: D. Leipziger, "Millennium Development Goals: The Infrastructure Contribution," PSI Retreat, September 2002. M. Kerf and A. Brenneman: "A Literature Review of the Impact of Infrastructure on Poverty," Mimeo, The World Bank, 2002. 28 Better mechanisms to support sector strategy implementation are needed 3. A2 Implementation has been the weakest dimension of the sector strategies and policies."' OED's 2001 gender evaluation found that the Bank had not established processes for invtitutionalizing and operationalizing its gender policy, and had not organized systematic trE ining for its staff on the gender policy or provided adequate support or tools for undertaking gender work for most of the 1990s. Measures to implement gender equality especially in secondary and tertiary education will require particular attention. OED's forestry review assessed Bank performance in the light of the Bank's 1991 Forest Strategy and found that the Bank had implemented the strategy only partially. Its ambitious goals were not matched by commensurate means to implement the strategy. OED's environment review found that the Bank's environment work lacked consistent management commitment and clear assignment of responsibilities and accountabilities. Some recent sector strategy papers have used these lessons to improve the mechanisms to support sector strategy implementation. The Bank has a strong record on project performance in several sectors, but a weaker record in others. 3.) 3 As shown in Figure 3.1, 80 percent or more of the projects in several sectors had salisfactory outcomes (weighted by disbursements). This performance provides a strong platform to support the MDGs. Project performance was weaker in other sectors (Figure 3.2). Figure 3.3 shows performance of projects that have priority themes (as identified by the Bank's new thematic codes) that match specific MDGs. In the education and health sectors, lending with an MDG-related priority theme does slightly better on outcomes than other lending in those sectors. Figure 3.1: Project Performance Over 80 Percent Satisfactory ' J3 40% --- ---- -- -- - C 0% Edication Finrcial Private Putbic Sector Rural Sector Trarsport Lktn Barkonide Sector Sector Goverance Demelopnt Avg Development Source BusinessmWrhouse,fridBank2002. a FY9799 exits Based on"SectorBoarF codes. g FY00-02* (artial) exits aBi nk-wide average for FY0OO-02* (Partial). 45. World Bank 2001a. 29 Figure 3.2: Project Performance Less Than 80 Percent Satisfactory' 4 - - ---- - ----- - ---- ---- ------ ---- ---- --- --------- -------- -0%-- & Energyand Environment EconomicPolicy Health,Nutrition Social WaterSupplyand BankwideAvg. Mining and Population Protection Sanitation Source: Business Warehouse, World Bank 2002. U FY97-99 exits Based on"SectorBoard codes. E FYO-02*(Partial)exits 'Bank-wide average for FY00-02* (Partial). Analydcal work needs to be revitalized in several sectors and should more fully address MDG-related issues. 3.14 OED's 1999 health sector review called attention to the limited amount of economic and sector work (ESW) in the sector, recently confirmed by the 2002 HNP Topical Briefing to the Board. The low levels of funding for ESW in other sectors, especially the water supply and sanitation and transportation sectors, have also been previously noted in Bank and OED reports. Recently, ESW is receiving more Figure 3.3: Education and Health Lending with attention in some sectors. An example is MDG-Related Priority Themes Has Slightly health, where analytical work has provided Better Outcomes than Other Lending in those timely technical information on key topics Sectors for MDGs. A recent piece of country-level M ---------------- ------------------- ESW relevant to the health MDGs is "Better Health Systems for India's Poor." Education 1a% ESW in Africa, which includes work on 170% sector-wide approaches and community - participation, is relevant to key aspects of the Education MDGs. But cross-sectoral ESW remains inadequate. More ESW is E- Basic necessary to address the challenges Heath presented by MDGs including to better Source: Business Warehouse, World Bank 2002. Note: EFA and Basic Health are matched with thematic understand the multisectoral determinants of codes. EFA Education for all; Basic Health = Child specific outcomes; intermediate indicators to Health, Population & reproductive health, Fighting track performance; and poverty impacts of communicable diseases. Number of EFA projects = 120; sectoral interventions, including policy Basic Health = 69. Outcomes are for FY97,02* (Partial). reforms. Future ESW should also address distributional issues within countries-by region, income group, and ethnicity-since there can be serious inequities in access and quality across parts of the same country. The Bank's education sector work in China, for instance, 30 an-knowledges vast differences in providing primary education in poor provinces relative to w ealthier provinces." 3.15 OED's forthcoming review of water supply and sanitation finds that, in order to reach the water target, Bank member countries will need systematic sector development plans or sirategies. Most countries lack plans that describe the present situation and analyze and identify policy reforms, pinpoint priorities, and identify appropriate projects. The Bank is well placed to help prepare these sector development strategies, given its knowledge of worldwide best practices. In the past decade, about 90 pieces of economic and sector work were prepared by the Bank for the water supply and sanitation sector, but only 10 percent focused on sector development. The appropriate amount of resources for sector development sirategies and economic and sector work will need to be determined and made available. 3.16 A country's policy environment has a strong influence on the success of all interventions, including those focused on the MDGs. Projects in countries with weak policies and institutions (as measured by low Country Policy and Institutional Assessment, CPIA, siores) have consistently worse outcomes than those in other countries. This is true even for M [DG-related projects, as shown in Figure 3.4. Education-for-all (relevant to the education M[DGs) and environment projects are particularly sensitive to the policy setting. These findings confirm that economic reform and policy dialogue must continue to play an important role in the Bank's programs. The Bank's proposed investment climate surveys aimed at better understanding the linkages between the investment climate and productivity aid income growth especially for the poor" may be expected to help identify improvements in country policies. Figure 3.4: Project Success is Influenced by the Country's Policy Environment e 8 j 100%- - N Low CFA N MediumCPIA N HighCF - O E 75% ----------- -------------- ---------------- --------------- ---- U0%, W_ p P Blucation For AN Basic Health Blvironment AI Sectors Source: Business Wamhouse, Wodd Bank 2002; CPIA figures are from the OPCS database. Note: Lending outcomes are for FY97-021(partlal) exits. Categories are defined bythematic codes. Environment (projectsc 268) includes Biodiversity Climate change; Environmental policies and Institutions; Land management; Pollution management and environmental health, Water resource management; Other environment and natural resources management. See noteto Fig3.3forEducatlonForAlland Basic Health. 45. World Bank 1999g. 47. World Bank 2002w. 31 Monitoring and evaluation, including the use of intermediate indicators, has not received adequate attention. 3.17 Analytical work at the sector level can help identify the appropriate sector, project, and country-level indicators that should be monitored. Intermediate indicators provide realistic medium-term measures for monitoring progress toward MDG outcomes. OED's 1999 health sector review pointed out severe deficiencies in monitoring and evaluation in the sector. Since then, there have been increased efforts to support the MDGs by identifying appropriate intermediate indicators. A technical consultation organized by the Bank led to a compilation of a recommended group of core intermediate indicators for health (Table 3.2). The precise indicators chosen in a particular case may need adaptation to be adapted depending on the precise objectives and nature of the intervention and the country conditions. An examination of FYOO-02-approved health projects for this Review showed that while recommended intermediate indicators for achieving health MDGs in projects have been applied in some fashion, they have not yet been fully applied. Table 3.2: Examples of Intermediate Indicators in the Health Sector Milennium Development Health and Nutrition Examples of intermediate or "proxy" indicators Targets Target: Halve, between 1990 and 2015 the * Prevalence of underweight children under-five proportion of people who suffer from hunger * Proportion of infants under six months who are exclusively breastfed * Percent of children 6-59 months who received one dose of vitamin A in the past six months Target: Reduce by two-thirds, between 1990 and * Proportion of 1 year old children immunized 2015, the under-five mortality rate against measles * Proportion of children with diarrhea in the past two weeks who received ORT * Proportion of children with fast or difficult breathing in the part two weeks who received an appropriate antibiotic Target Reduce by three-quarters, betweenml990 * Proportion of women with any antenatal care and 2015, the maternal mortality ratio * Percentage of births with skilled birth attendant and or institutional delivery * Contraceptive prevalence rate Source: "Health, Nutrition and Population Development Goals: Measuring Progress Using the Poverty Reduction Strategy Framework." Report of a World Bank Consultation, Nov 28-29. Washington, D.C.: World Bank 2001. It provides a complete list 3.18 Greater attention is needed to developing or collecting data on appropriate performance indicators in some sectors. For example, the Bank's financial sector strategy does not include any explicit performance measures to assess the impact of the Bank's interventions on the economic performance of countries or on progress in improving the global financial and trade system. OED's 1998 financial sector review presented a set of relevant indicators that should be among the options considered by the sector Board. Similarly, performance indicators for monitoring poverty reduction and social impacts 32 nquire greater attention in the transportation sector." The Second Peru Rural Roads Project i. a good example of design and implementation of monitoring (Box 3.6). Box 3.6: Monitoring Outcomes in the Peru Rural Roads Project The goal of the Peru Rural Road Rehabilitation and Maintenance project was to provide a well- integrated and reliable rural road system to support the government's poverty-reduction strategy and to raise living standards in rural areas. OED's performance assessment found that the project was highly satisfactory, noting three poverty-reducing outcomes: a small increase in farmed land; a moderate increase in livestock ownership and access to markets; and a large increase in farm prices in the beneficiary areas. Project monitoring was especially effective. The project team used a logical E-amework, developed with community participation, to describe the project components and to monitor and report on their performance. The outcome indicators were relevant to key MDG objectives and included linking at least 80 percent of the communities in program areas through r:1iable and affordable public transportation; generating 4,000 non-skilled permanent jobs through r>ad maintenance work contracted out to microenterprises; and assuring that 17,000 kilometers of rhabilitated road were available in the 12 poorest areas. All of the project's key objectives were met or exceeded. Source: Second Peru Rural Roads Project PPAR, Report No. 22474, July 2001. 2.19 Some of the risks and challenges that accompany MDG performance monitoring are illustrated by the education, water, and urban sectors. In the education sector, for example, the MDG indicators focus on enrollment and completion rates, which are likely to be meaningless without improved learning achievement. In the water sector, reaching a common definition of "safe water" has always been difficult where much water comes from wells and rivers, and where many urban neighborhoods have access to piped water for only a few hours each day. While the MDGs include the concept of "sustainable access," there is no accepted definition of the term or how to measure it. The environmental MDG also poses measurement issues since it is not specifically defined and refers to "integrating the principles of sustainable development into country policies and programs" leaving the level of integration and its intensity undefined. Furthermore, the slum MDG target is expressed in absolute rather than proportional terms. This means that in a country with a fast-growing population, steps must be taken to stop new slums forming faster than existing slums are upgraded. Data are also scarce or unavailable on sanitation coverage (the first MDG indicator for the slum target) and on secure tenure (the second MDG indicator for the slum target). The Bank has been involved with UN-Habitat on urban indicators and is currently working on a major multilateral effort to define and operationalize the secure tenure and slum welfare concepts. 3.20 ARPP 2002 notes that Adaptable Program Loans (APLs) account for 18 percent of FY02 new investment lending and are "rapidly gaining in popularity." Strong monitoring and evaluation would be a prerequisite for APLs given their rationale of continual learning and adaptation. OED has noted a number of instances where monitoring and evaluation has been weak in APLs. 48. Walker, W. and C. Sagna, forthcoming. 33 Sector strategy papers recognize the multisectoral determinants of outcomes, but provide little guidance on how to develop multisectoral strategies. 3.21 All sector strategy papers recognize the multisectoral determinants of outcomes. For example, the education strategy describes some of the inter-sector relationships in the social sectors (Box 3.7). And recognizing the thematic and cross-cutting nature of its mandate, the environment strategy discusses environmental health risks arising from unsafe water, air pollution, transport fuels, and traditional household biomass energy sources. But OED's 1999 review of the health sector found that little attention has been paid to the determinants of health, such as behavioral change and cross-sectoral interventions outside the public health system. Moreover, the lack of adequate coordination between the Bank's country and sector units and among sector units discourages cross-sectoral country strategies." Ensuring such coordination has also been a challenge for other donors agencies. Box 3.7: Inter-Sectoral Linkages in the Education Sector Strategy The relationships between societal conditions and the outcomes of education investments are complex: education outcomes are heavily influenced by learners' income, health status, and perceptions of labor market opportunities. The presence of HIV/AIDS affects the supply of teachers and often determines parents' ability and willingness to pay for schooling. Education status also has an impact on the individual's future income, fertility, and health. At the societal level, it affects institutions, the economy as a whole and, in the long run, on values, traditions, and culture. The Bank's comparative advantage in this area is in its work on school de-worming and micronutrient programs and reproductive health education, including information on STDs and HIV prevention. Further links can be found in early child education and health services, parent education and shelter and nutrition for the poorest families; teacher service reform and broader civil service reform; universal basic education and child labor, vocational training and labor market regulation; and in language of instruction and decentralization. Source: The World Bank's Education Sector Strategy, World Bank, 1999. WHAT ARE THE IMPLICATIONS FOR FUTURE BANK SECTOR PROGRAMS? Monitoring and evaluation of sectoral outcomes needs more attention. 3.22 In the wake of the Wapenhans report and subsequent Bank reports on implementation and development performance, there has been a steady increase in attention paid to development outputs, outcomes, and impacts.' The MDGs' ambitious targets challenge existing practices in performance management and measurement of development assistance at the corporate level. They also highlight the importance of country-level monitoring. At 49. The Latin American and the Caribbean Region supports a model where Sector Leaders are members of both the sector and country management teams. 50. The explicit attention to development effectiveness in the Bank has its origin in the 1992 Wapenhans Report, Effective Implementation Key to Development Effectiveness, and a subsequent report Getting Results: the World Bank's Agenda for Improving Development Effectiveness, which concentrated on country portfolio management and improving the quality of operations. 34 both the corporate and country levels, monitoring and evaluation have a role in helping to inform decision-making and to manage performance as well as to establish progress. 3.23 At the corporate level, while it is true that the Bank alone cannot be held accountable for achieving intended outcomes, managing performance requires that the Bank track the e.tent to which its sector programs are contributing in the direction of the agreed goals. The Bank is taking steps to improve performance measuring, monitoring, and management (Annex V lists these initiatives). OPCS notes that while much of FY02 was devoted to planning the architecture for the Bank's migration to a results focus, the focus of FY03 is on dtailed engineering and piloting, and FY04 on operational mainstreaming. The Bank's st,ategic directions for FYO3-05" do not yet indicate how resources will be deployed, re- deployed, or enhanced52 in pursuit of these objectives. The resource implications and the technical complexities of monitoring outcomes should not be underestimated. Working with purtners (e.g., UNESCO in the education sector) would be an option to consider. Where partners support differing methods and approaches to monitoring and evaluation, consistency s ould be enhanced through harmonization of policies and practices. 3.24 At the country level, the involvement of relevant stakeholders in the definition, collection, and analysis of data will be critical to ensure its subsequent use in local decision- making. As noted earlier, such participation can also prevent distortion and manipulation of actions resulting from quantitative targets as illustrated by the Bank's interventions aimed at slam-dwellers. MDG indicators lack qualitative measures of well-being, which are important for capturing key dimensions of development effectiveness." Therefore, the existing MDG indicators will have to be supplemented by qualitative measures-for example, levels of client satisfaction relating to economic and social services or to learning achievement. 3.25 Governments and donors, including the Bank, often lack incentives to measure how they are performing.' The Bank and other donors must better understand the decision- making processes in client governments and the existing demand for information on outcomes, and help create incentives where they are lacking. Similar work will also be needed in donor agencies themselves. The Annual Report on Portfolio Performance (ARPP) 2002 notes that while the corporate commitment to MDGs is very explicit, frontline staff still n(d to be engaged. It also notes that new incentives are not yet in place, and that frontline stiff feel some uncertainty as to how to integrate the MDGs into country dialogues and operations: are they an add-on or a basic rethinking of priorities?" 51. World Bank 2002ee. 52. The planning assumptions include in that part of the administrative budget devoted to the Regions and Ni-tworks a real increase of about 3.5% in FY03, 1% in FY04 and less than 1% in FY05 despite a shortfall in spnding in FY02 of about 10%. This shortfall was incorporated into the planning base on the grounds that thore have been efficiency gains. 53. World Bank 2002w. 54. L. Pritchett, forthcoming. 55. World Bank, forthcoming(b). 35 3.26 The types of indicators that are monitored at the various levels-project, sector, countiy, global, Bank, and other donor-will depend, inter alia, on program objectives, nature of activities, and data collection and analysis capacity. In developing the monitoring system, the full picture must be kept in mind with monitoring at each level consistent with, and complementary to, monitoring at other levels. Careful attention will be needed to dealing with issues of aggregation and attribution.- Multisectoral strategies need to be developed to better reflect the multisectoral determinants ofMDG outcomes. 3.27 By focusing on outcomes-as distinct from inputs and outputs-the MDGs draw attention to the multisectoral determinants that contribute to outcomes. The close relationship between actions in one sector on outcomes in another has already been well-documented, and further work is ongoing (for example, in the context of the 2004 WDR "Making Services Work for the Poor"). Although each sector strategy is relevant to achieving one or more MDGs, an important question is how they fit together. 3.28 Multisectoral strategies do not necessarily imply multisector projects. Some multisectoral strategies may best be implemented through single-sector operations, depending on the particular sectors involved and the institutional set-up that governs the coordination of inputs from the various sectors. The focus must therefore be on thinking multisectorally and acting either sectorally or multisectorally, as conditions warrant. The experience of Bank-supported social fund projects provides some relevant lessons (Box 3.8). 3.29 Two recent initiatives involve joint work by a number of sectoral groups. The health, energy, environment, and gender sectors have jointly developed interventions to alleviate indoor air pollution which contributes to the health burden of many poor rural communities, including respiratory illness and associated deaths in children. Similarly, the water and health sectors, and the Business Partners Outreach Group in the Bank are collaborating on the hand- washing initiative which aims at promoting hand-washing with soap in developing countries for diarrhea prevention. Initially launched by USAID and now supported by the Bank in a number of countries, the approach is to create partnerships in which the private sector achieves a better market penetration in poorer households, and the public sector achieves improvements in hygiene programs. 56. Aggregation refers to judging the overall program based on an assessment of the individual building blocks that make up the program. Attribution refers to identifying the impact of an agency's inputs. 36 Box 3.8: Social Funds: Multisectoral Projects Do Not Guarantee a Holistic Approach Social fund projects are innovative projects supported by the Bank since 1987. They offer communities a multisectoral menu of investments from which to choose (mainly comprising small- scale school buildings, health clinics, water supply and sanitation investments, and roads). A recent OED evaluation found that social fund projects have been highly effective in delivering small-scale infastructure, and that this additional infrastructure has increased facility access and utilization. It aho found that social fund projects have had neutral or very mildly progressive household targeting, and their outcomes and welfare impacts-for example, academic achievement or incidence of diarrhea and wasting-have not always been better than the control group. Achieving outcomes and wlfare impacts requires, inter alia, ensuring an adequate and sustained supply of complementary inputs (e.g., teachers in schools, drugs in health clinics), and related investments (e.g., water supply in health clinics). Source: "Social Funds: Assessing Effectiveness," OED, World Bank, 2002; "Letting Communities Take the Lead: A Cross- Country Evaluation of Social Fund Performance" (Social Funds 2000 Impact Evaluation), forthcoming. World Bank. 7te MDGs may have implications for the geographic and sectoral composition of Bank leading 3.30 The Bank has a well-established system for allocating lending resources across IDA countries commensurate with the quality of countries' policies and performance. This system helps to ensure that scarce aid resources are delivered to the poor countries that can use them m)st effectively, as required by the IDA donors. To the extent that the MDGs-and related initiatives such as the fast track programs in education and health (Box 3.9) and the mltisectoral AIDS Program for Africa (MAP)-entail allocation or reallocation of lending re;ources among countries, each country's likelihood of using the funds effectively must be taken into account as well as its distance from the MDGs. Any implications for the Bank's geographic and sectoral allocation of resources resulting from such initiatives should be systematically assessed and trade-offs carefully considered. 3.31 Achieving and sustaining the MDGs will require significant additional recurrent fimancing. In the education sector, for example, recurrent costs account for about two-thirds of the financing gap associated with Education For All." The recurrent costs of newly created services will need to be financed, which could necessitate a review of donor policies in this ar.-a. Sector-wide approaches with multiple external financial partners may be one option. A re-ent Development Committee paper" noted that in countries with a PRSP, a sound public expenditure framework and appropriate transparency and governance standards in budgetary processes and execution (as well as a credible sector strategy that delivers results), donors have agreed to move beyond projects to directly support governments' overall approach and strategy. They do so by channeling more of their resources through national budgets. This is, however, only beginning to happen. One example is Uganda, where World Bank support to th- education sector no longer comes through traditional education projects but instead through intensive dialogue on sector issues and overall budgetary support (through a sector- 57. B. Brims, A. Mingat, R Rakotomalala, forthcoming. 58. World Bank 2002h. 37 wide approach). And in Brazil and Ghana, donor support for the health sector comes through a sector-wide approach in which the Bank participates with other donors and funding is pooled with the government's own budget. Box 3.9: Fast Track Programs and Resource Allocation At the World Education Forum on Education for All (EFA) in Dakar, Senegal (April 2000), the Bank's president proposed a fast track action plan in which a select group of countries would forge the way in achieving EFA even sooner than 2015. Eighteen countries are now involved, with an additional five countries part of the analytical fast track. The eighteen countries are to prepare their own country plans to be assessed by the Bank and other partners in terms of technical and financial feasibility, as well as the countries' institutional capacity to carry them out. The fast track initiative aims at supporting countries with good policies thereby giving incentives to countries that do not yet have good policies in place to do so. Its indicative framework provides the focus for domestic resource mobilization (up to 80 percent of resources would have to come from domestic expenditure by 2015) and donor support, as well as fiscal discipline. The health sector is now contemplating a similar fast track approach. Only 10 of the 78 least developed countries are on track to reach the MDG of cutting child deaths by two-thirds; a more ambitious fast track approach would aim to raise the number on track to 39. As proposed, fast tracking would identify two groups of countries: one where modest policy shifts, accompanied by major increases in domestic and donor financing can lead to accelerated progress on the HNP MDGs; and a second with weak policy frameworks and limited capacity, where large infusions of additional funding will not result in rapid progress on the MDGs. In the latter countries, non-financial barriers hinder large-scale implementation and alternative options for overcoming them would be considered. There are still questions to be answered about the Bank's fast track initiatives including what will happen to lending and other support in countries not selected for fast tracking, whether the fast track initiative will ensure sustainable outcomes, and which intermediate outcome indicators will be monitored to assess whether countries are on track. Given finite resources, if the fast track countries and sectors receive more resources, other countries and sectors may receive less. This will have implications for the Bank's strategy and lending pipeline over the short and long run. Source: World Bank Press Release (November 27, 2002) "Education for All the World's Children: Donors Have Agreed to Help First Group of Countries on Education Fast Track Initiative," www.worldbank.org, "Education for Dynamic Economies: Action Plan to Accelerate Progress Towards Education for All (EFA)." April. Prepared for the Spring 2002 Development Committee Meeting. Washington, D.C.: World Bank 2002; "Concept Note for a Development Committee Paper Proposing A Fast Track Initiative on Health, Nutrition and Population Millennium Development Goals," World Bank, 2002; and World Bank staff interviews. 38 4. Global Programs and the MDGs It is increasingly important for the Bank to be active in the provision of global public goods vital to developing countries' interests. But it is also important for the Bank to be carefully selective in its criteria and priorities, collaborative in its approach to public goods problems, and measured in its deployment of resources for these purposes. These are not easy challenges. "Poverty Reduction and Global Public Goods: A Progress Report," Development Committee Paper, DC2001-0007, 2001 IN'IRODUCTION 4.1 It is increasingly realized that many development problems require collective action at the global level to supplement traditional country and project-level approaches. This realization is rflected in the Bank's rising level of support for global programs.6o Box 4.1 summarizes thene programs which cover all eight MDG goals. This chapter examines how the Bank's glo ,al programs have been addressing the MDGs and implications for future work. The main sources of evidence for the chapter include recent OED reviews related to global programs.6' How HAVE THE BANK'S GLOBAL PROGRAMS ADDRESSED THE MDGs? All global programs broadly support MDG Goal 8, "developing a global partnership for development," and many also support other MDGs, ranging from poverty, education, and heialth to environmental sustainability. 4.2 The Bank's support for global programs is crystallized in Goal 8 of the MDGs, which call s for developing a global partnership for development. Consistent with Goal 8, almost all global programs involve global partnerships with other international public sector organizations such as UN agencies, the OECD, bilateral donors, and regional development banks., The most cormnon reasons for these partnerships include to pool financial and other resources in order to achieve common objectives more effectively and efficiently, to realize economies of scale and 59. The progress report was prepared by Bank Management at the specific request of the Development Coxamittee in its September 2000 Communiqu6, which established criteria for the Bank's involvement in the pro, rision of global public goods. 60. The Development Committee Communiqu6 of September 2000 endorsed four criteria to guide the Bank's involvement in global programs: a clear value added to the Bank's development objectives; the need for Bank acti Dn to catalyze other resources and partnerships; a significant comparative advantage for the Bank; and an emTging international consensus that global action is required. The Bank's Strategic Framework Paper of 2001 identified, and its Strategy Update Paper for FYO3-05 of 2002 reaffirmed, five global public good priorities for the Bank: communicable diseases, environmental commons, information and knowledge, trade and integration, and international financial architecture. 61. World Bank 1998b, World Bank 2001f, World Bank 2002jj; World Bank, forthcoming(f); World Bank, forthcoming(g). 62. Developing countries are only involved in the governance and management of a few global programs. Mo itly they are viewed as "participants" rather than full "partners" with voting rights. 39 scope in the production and delivery of goods and services; and to capitalize on the comparative advantages of the different partners. For the World Bank, leveraging its financial contributions is a key aspect of these partnerships, since the Bank's ability to give grants out of its net income is limited. Other partners value the World Bank's global reach, its convening power, its multi- sectoral capacity, and for providing a "seal of approval." Box 4.1. Global Programs: How Many? In Which Sectors? The Bank is currently involved in 70 global programs (an increase from fewer than 20 programs in 1991) 30 of which are managed inside the Bank, and 40 elsewhere. In FY01, the Bank spent about $30 million of its net administrative budget on global programs; contributed $120 million from the Devel- E 111*M , arid opment Grant Facility; and disbursed about $500 Global Expencrtures million from Bank-administered trust funds. The latter 4% represented about 10 percent of the Bank's gross 20 "P administrative budget and 40 percent of trust fund Educati disbursements. 3% ICT Environment, rural development, and health were the Devef sadaI Protcom top three sectors in overall program expenditures, 1% , 0 4 including co-financing from other partners. Sa0= OE dt Washington, D.C., 2002. PEM = Nroty Rwucalion and eanorric Mannat 4.3 Over a third of Bank-supported global programs address health and environmental issues relating to MIDG Goals 4 through 7."1 In addition, some global programs address primary education,6"and many programs have an explicit focus on poverty reduction. The Consultative Group on International Agricultural Research (CGIAR), the oldest and largest global program supported by the Bank, has had important impacts on directly reducing poverty by increasing production of subsistence foods, and furthering employment, and income generation. And CGIAR has had an indirect effect through reducing prices for food- deficit households. Another example of a global program is the Consultative Group to Assist the Poorest (CGAP), a microfinance program started in 1995 that is promoting sustainable financial services for the poor, to help them build assets, increase income, and reduce vulnerability to economic stress.61 63. For example, Health: Medicines for Malaria Venture, Special Program for Research and Training in Tropical Diseases, Global Alliance for Vaccines and Immunnization, International AIDS Vaccine initiative, Roll Back Malaria, Global Forum for Health Research. Environment: Global Environmental Facility Water and Sanitation Program, Energy Sector Management Assistance Program. 64. For example, The Partnership for Child Development Program for Education Statistics, Program for the Assessment of Student Achievement, World Links for Development Focusing Resources on Effective School Health. Recently, there has been an increase in fimding for education through the Development Grant Facility budget 65. The Bank is also supporting important regional programs such as the Nile Basin Initiative and the African Program for Onchocerciasis Control. 40 Lack of clarity about the institutional roles of various partners. 4.4 The Bank has played an important role in several global programs and related activities. A recent OED review of aid coordination concluded that the development Qommunity had been well served by the leadership of the Bank in managing aid coordination processes." A promising partnership, recently created with Bank support in the course of preparing its urban sector strategy paper, is the Cities Alliance (Box 4.2). .Box 4.2: Old and New Partnerships Water and Sanitation Program The Water and Sanitation Program (WSP) is an international field-based partnership dedicated to .ielping the poor gain sustained access to improved water supply and sanitation services. One of the ,oldest external partnerships in the Bank, with staff present in 19 countries, the Program collaborates with many partners. Through its engagement with government clients and partners and through wide international efforts at knowledge management, advocacy and promotion of new thinking, WSP has been able to help build capacity, support sector reform and demonstrate new approaches in both water supply and sanitation. By working closely with its partners, WSP helped promote the sanitation agenda at the World Summit on Sustainable Development at Johannesburg in 2002. WSP's work helps to create the conditions needed for meeting MDG goals for water supply and sanitation. Cities Alliance The Cities Alliance, one of the newest external partnerships in the Bank, is a global coalition of cities and their development partners initiated by the Bank and UN Habitat, committed to the development of citywide, sustainable programs of improving slums and to the creation of city strategies to cope with future development. In part through the efforts of the Cities Alliance, the IDTs were expanded by the Millennium Declaration to adopt the Cities Alliance's "Cities Without Slums" target of improving the lives of at least 100 million slum-dwellers by 2020. Source: Water and Sanitation Program http://www.wsp.org; Cities Alliance http://www.citiesaliance.org 4.5 Sometimes, however, the Bank's institutional role has been unclear and the Bank has played multiple-and sometimes conflicting-roles. For example, in the CGIAR program, the Bank plays three major roles-as convener, donor, and lender for complementary activities. While the Bank's unique contribution to the CGIAR has been widely acknowledged, its multiple roles have led to excessive dependence of the system on the Bank, a disproportionate share of CGIAR management responsibility assigned to a Bank vice president who chairs the CGLAR, and reporting arrangements fraught with real or perceived conflict of interest. It has been difficult for the Bank to acknowledge the need for, and to press for, major reforms while also making the case for continued funding. Consequently, it not surprising that the Bank has been more successful in using its convening power to raise additional resources for the system than in providing the necessary strategic leadership needed to help the CGIAR respond to a rapidly changing environment. 66. World Bank 2001 f. 41 4.6 While global programs provide avenues for donor coordination, the roles and responsibilities of partners are often insufficiently spelled out or not understood equally by all the partners. The HIPC initiative faces particular challenges in implementing shared responsibility (Box 4.3). The Integrated Framework for Trade-Related Technical Assistance (IF) program has similar problems. IF is a partnership among six international agencies, some 15 bilateral donors, and the 49 officially classified least-developed countries. Its purpose is to help these countries better integrate into the world economy (including the multilateral trading system), as a basis for sustainable growth and poverty reduction. Lacking a secretariat, it is run by a large interagency working group. Despite the group's frequent meetings, coordination problems persist because of differences in agencies' mandates, in their views of the objectives of the program, and their working procedures. Box 4.3: HIPC: Challenges in Implementing Shared Responsibility When the HIPC Initiative was launched in 1996, it was an unprecedented partnership that united bilateral and multilateral creditors in an ambitious program of debt relief for some of the world's poorest countries. The costs of providing debt relief to 34 countries is $37.2 billion in 2001 net present value terms, divided about equally between bilateral and multilateral creditors. These shared costs, as well as the connection between debt relief and the availability of other development assistance, have raised several challenges related to the enforcement of the partnership. Although the shareholders of the Bank and the IMF endorsed the initiative, it remains a voluntary agreement lacking a legal basis for enforcement. The broad consensus by donor countries on the principle of additionality (that is, debt relief should be in addition to other development aid rather than a substitution) cannot be guaranteed. Creditor non-participation, especially by some commercial and non-Paris Club bilaterals, also remains an unresolved issue. Source: "The Heavily Indebted Poor Countries (HIPC) Debt Initiative: An OED Review." OED. Washington, D.C.: World Bank, forthcoming. Complementary country-level investments and coordination of activities on the ground are key to maximizing benefits from global programs. 4.7 Externalities, spillovers, and scale economies justify financial and other support for global programs. But this support will not yield full benefits unless there is in-country capacity to access the new products and services being generated. The level of Bank lending to environment, agriculture, and health sectors (about 20 percent of Bank lending) contrasts with expenditures on these three sectors at the global level (about 85 percent of global program expenditures). Complementary country-level activities are also inadequate. Some global programs are moving into this vacuum at the country-level by taking on the delivery of country-level services-an approach that is leading to the proliferation of country-level programs each associated with a different global program, rather than one coordinated effort in each sector. 42 4.8 OED's forthcoming review of seven global programs relating to health research and disease control6 (part of its phase I global programs evaluation), finds that the growing number of global programs is imposing large transactions costs on countries with weak health systems. Government officials in developing countries have questioned the proliferation of global health programs, each with its own parallel institutional structures at the country level. This proliferation is straining public health systems by adding long-term recurrent costs, by diverting resources away from existing programs, and by increasing demands on health ministry staff to respond to calls for proposals from the many global programs-particularly fiom the financing entities like the Global Alliance for Vaccines and Immunization (GAVI) aad the newly created Global Fund to Fights AIDS, TB, and Malaria (GFATM). Country-level siakeholders have also asserted that there seems to be little effort at the global level to seek out aad exploit synergies among the different health programs. There is substantial scope for realizing the achievement of the MDGs and increasing impacts on poverty reduction through better coordination of global programs and country-level activities. T he voice of developing countries needs to be amphfied in the management ofglobal programs as well as in global policy-making. 4.9 Developing countries have so far had little voice in the design, governance, and management of most global programs. Developing countries are for the most part "participants," rather than full "partners" who help to implement each program at the country lovel. While donors are largely driving the agenda of global programs, they often seek World Eank involvement as a proxy for developing country involvement. Relatively few global programs as yet involve partnerships at the governing level with developing countries, civil society organizations, or the commercial private sector. 4.10 Introduced by the developing countries at the Millennium Summit Goal 8 seeks a broader commitment of all partners-developing and developed--to the MDGs. The addition of Goal 8 marked a major change from the IDTs to the MDGs. Few global programs focus on global public policy formation. So far, Bank-supported global programs have not had much influence on developed country policies, either through advocacy or other means. The Bank's advocacy role, especially on trade issues, has been hampered in the past by inappropriate iternal organization. Recently, new arrangements have been put in place through the creation of a Trade Department. The Department's stated purpose is to improve the Bank's capacity to r-spond to the growing demand for Bank services on trade, and to scale up the work (including global advocacy) on trade-related development issues. MDG Goal 8 has created a strategic opportunity for the Bank to amplify the voice of the poor in global policy debates. I l7e proliferation of initiatives has strained the Bank's institutional capacity. 4.11 The Bank's involvement in multiple global programs has over-taxed its ability to keep t-ack of operational, budgetary, and fiduciary aspects of global programs, resulting in a lag in E ecessary monitoring and evaluation of new activities. In addition, growth of global programs 6 7. The seven programs are the Global Forum for Health Research, the Special Program for Research and Training in Tropical Diseases (TDR), Global Alliance for Vaccines and Immunization (GAVI), Global Micronutrient Initiative (GMI), Roll Back Malaria (RBM), Stop TB, and UNAIDS. 43 and partnerships has added to the tension arising from the Bank's '"matrix approach," since most of the global initiatives are located in the Network and Sector Anchors. Regional vice presidencies have sometimes seen the new initiatives as competitors (in terms of resources and managerial attention), and as a threat to the "country focus" of the Bank's work. Although the oversight arrangements, resource allocation practices, and reporting processes have been strengthened during the past three years, more changes are needed. OED's evaluation of global programs recommended that the Bank sharpen the criteria for selectivity-, clarify responsibilities and accountabilities for global programs; and improve monitoring and evaluation. Global programs as a group are no better than other development efforts at monitoring and evaluating the outcomes and impacts of their activities. 4.12 Global programs have typically focused on monitoring grant amounts rather than on the outcomes resulting from the funded activities. Even the CGIAR, which has an impressive tradition of external assessments of its individual research programs, has conducted little independent evaluation of the system as a whole. Such evaluation is needed to permit the Bank or the other donors to assess the CGIAR's system-level strategies, priorities, allocation of resources, impacts, and client's views. The HIPC initiative provides for close and detailed monitoring and reporting of recipient countries' expenditures on social services. This attention to expenditures is not, however, complemented by monitoring the program's impact on poverty reduction. The 1998 OED review of the Bank's grant programs noted that there were gaps in evaluation reporting and dissemination and that no systematic evaluation regime existed for the grants program as a whole." WHAT ARE THE IMPLICATIONS FOR FUTURE BANK-SUPPORTED GLOBAL PROGRAMS? Linkages between global and country programs need to be tightened through improved selectivity and greater involvement of borrowers. 4.13 Phase I of OED's evaluation of global programs noted that there is no consensus among Bank member countries on the appropriate blend between Bank-supported global programs and country-level activity by the Bank. Some argue that increased global activity will be required either to enhance development effectiveness or to position the Bank to meet future challenges in a dynamic external environment. In any event, tightening the linkages between global and country programs is needed in order to maximize the benefits from both global and country programs. Monitoring and evaluation ofglobal programs needs to be strengthened 4.14 The main implication for evaluation is to provide resources for and create the necessary incentives to more systematically monitor and evaluate the impacts of global programs than has been done in the past. More attention is needed to developing improved methodologies for evaluating global programs and partnerships. Phase II of OED's evaluation of global programs is pioneering new techniques and approaches in this regard and could point the way forward. 68. World Bank 1998b. 44 5. Conclusion 5.1 The MDGs build on the Bank's corporate priorities and capabilities. The main td emes embedded in the MDGs-poverty reduction, human development, environmental sustainability-had become priority areas for the Bank long before it officially endorsed the MDGs. Poverty reduction has been the Bank's overarching objective since 1990, and the focus on education and health has been a main tenet of the "basic needs" approach followed by the Bank since the early 1970s. Similarly, the focus on gender and environmental sustainability have been important components of the Bank's strategy through the 1990s. The Bank's country, sector, and global programs are broadly consistent with these themes. 5.2 The multidimensional concept ofpoverty endorsed by the MDGs and by the Bank rEpresents an advance over more traditional conceptions ofpoverty. The MDGs emphasize income as well as non-income measures of well-being and draw attention to their multiple determinants. Mutually reinforcing interventions in a variety of sectors-including those not explicitly addressed by the MDGs-are vital to achieving MDG outcomes. Most Bank sector strategies have established links to poverty reduction and the MDGs, but the means to implement sector strategies and monitor their results remain weakly developed. 5.3 The MDGs offer the potential-backed by international support and common unvderstanding-for the Bank to sharpen its focus on outcomes. Seizing this opportunity will require significantly increased action in three areas in which the Bank has already begun work. * Monitoring and evaluation. By specifying quantitative targets, the MDGs require systematic measurement of the outputs and outcomes of development activities. The Bank's new initiatives to better monitor, measure, and manage for results will be particularly helpful if they result in clearer, and more specific objectives and targets at the country, sector, and global levels. Clients would benefit from Bank advice that helps them identify development outcomes relevant to their country circumstances; select corresponding intermediate indicators to monitor progress on those outcomes; and strengthen both the capacity to monitor and evaluate development outcomes as well as the incentives to encourage such measurement. Learning can be sought in success, but also in failure. The monitoring process should be designed to yield information that provides a sound and continuing basis for informed decision-making and learning. * Addressing intersectoral linkages. By focusing on outcomes rather than inputs and outputs, the MDGs draw attention to the multi-sectoral determinants of outcomes. The Bank's country and sector programs are increasingly recognizing the multisectoral determinants of development outcomes. These Bank programs now need to take the next step of developing and implementing cross-sectoral strategies. Multisectoral strategies do not necessarily imply multisector projects. Developing multisectoral strategies will require that the Bank's country and sector units cooperate to design and implement outcome-based, cross-sectoral country strategies. A more effective institutional mechanism is needed to foster the design and implementation of cross-sectoral strategies to deliver specific development outcomes. 45 Strengthening partnerships. The Bank has entered into a large number of partnerships, notably for global programs relevant for achieving the MDGs. For these partnerships to deliver better outcomes on the ground, these global programs should capitalize on the comparative advantage of individual partners, link global programs with related country-level activities, and situate them within the framework of a larger global strategy. Periodic re-evaluation of the relevance and impact of global programs is needed, in order to avoid inertia and the unintended accretion of programs. 5.4 The adoption of the MDGs entails risks and challenges. The development community signed on to global targets before establishing clear methods for prioritizing them within and among countries, and before adapting them to each country's circumstances. There is a danger the sectoral and geographic composition of assistance programs could be distorted if the appropriate goals and targets or the priority ranking among them in different country circumstances is not clearly established. The challenge is to ensure that targets that are easily monitored do not get over-emphasized to the neglect of the qualitative dimensions of development, and that quick, short-term gains are not favored over more difficult and time-consuming reforms, including in country-level governance structures. The first step in better managing these risks and challenges would be to systematically assess and understand the implications--at the corporate, country, sector, project, and global levels--of the MDGs. A determination can then be made about how priorities are to be set, key choices made, and any resulting tensions and trade-offs addressed. Over two years after the Bank's endorsement of the MDGs, such efforts are only just beginning in the Bank, especially at the sectoral (Network) level. The Bank needs to identify the implications of the MDGs and address them, including questions about the allocation of lending and administrative resources. 5.5 A better definition of objectives and targets-quantifled and time bound-is needed, especially in the Bank's country programs. In the past few years, the objectives and strategies of the Bank's country programs have increasingly focused on poverty reduction. This focus can be greatly enhanced by more specificity in defining poverty reduction and other objectives. The lack of quantified and time-bound objectives and targets makes prioritization difficult, and results in strategies that are broad and general. The global MDG targets must be localized. The tensions and trade-offs-between sectors and sub-sectors, between geographic regions, and among access, quality, and policy and institutional issues-- that will inevitably arise given finite financial and institutional resources can most effectively be addressed at the country level. The Bank needs to develop more coherent country assistance strategies grounded in specific objectives and targets and based on a realistic assessment of capacity and resources, and monitor progress systematically. The Bank's sector strategies-which rightly place the MDGs alongside other sector goals and targets in a broader development framework-could greatly assist in this process. They could do so by also providing guidance for groups of countries on how to address the tensions and trade-offs between the broad approach of the sector strategies and the specificity of the MDGs. Better analytical work can strengthen both the Bank's country and sector programs. 5.6 More attention is needed to achieving and sustaining MDG outcomes. The Bank has endorsed the MDGs. Achieving MDG outcomes by 2015-and sustaining them beyond 46 that date-will require a break from historical trends in a number of countries. 'Business as usual' is unlikely to be a sufficient approach for either countries or donors. The recurrent costs of newly created services will need to be financed, which could necessitate a review of donor policies in this area. Over the past few years, the Bank has demonstrated steadily improving project performance. Now, the Bank must scale up impact to help clients achieve country-level improvements in economic well-being, human development, environmental sustainability, and other relevant development outcomes. Rising to the challenge of the MI)Gs will require continuity in some areas of Bank work, increased emphasis in others, and may warrant changes and innovations in some Bank practices and programs. The exact nature of such changes can only be determined through a systematic analysis of the implications of the MDGs. Assessing and addressing these implications, including in the use of lending and administrative resources, should be a priority for the Bank. 5.7 The Bank's self-evaluation and independent evaluation processes measure the success of development activities in meeting their stated objectives. The MDGs offer an opportunity to sharpen the definition of program objectives, and thereby the focus and utility of evaluation. It is not too soon to begin adapting evaluation criteria and methods to address the ext,-nt to which Bank programs help countries work toward poverty reduction, the first Mi:lennium Development Goal, and the other seven goals as well. The MDGs also place new demands on client capacities for monitoring and evaluation including data collection, analysis, and utilization. The Bank should demonstrate tangible improvements in this area.  47 Appendix Appendix: Project Performance Results 1. This Appendix presents long-term trends in project performance based on evaluations conducted by OED. In line with the focus of this year's ARDE, the Appendix also assesses the performance of projects related to specific MDGs using the Bank's new sector and thematic codes. CoMPosMON OF THE ARDE'02 EXITING COHORT 2. OED has evaluated 331 closed projects since last year's ARDE, 81 percent of which exited the Bank's portfolio during FY01 and FYO2* (* indicates partial)'. These evaluations cover US$23 billion in nominal disbursements, and consist of 276 Evaluation Summaries (ES) and 68 Project Performance Assessment Reports (PPARs) of completed projects". This newly evaluated cohort includes 41 adjustment operations, and 289 investment operations, the vintage of which is illustrated in Figure 1. Figure 1. Approval Years of 331 Evaluated Projects 60 * 50 - Adjustment 9 40 M Investment > 30 W ID .0 -10 E 20 Re- 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 1990 Approval Fiscal Year Source: Business Warehouse, World Bank 2002. OED database 2002. 69. The FYO1-02* cohort includes all OED project evaluations through November 12, 2002. This includes partial coverage of FY01 and FY02 exits. For FYO1 exits, Implementation Completion Reports (ICRs) for 16 projects have not yet been received or evaluated by OED, representing 22 percent of total disbursement For FY02 exits, 155 projects remain to be evaluated representing 55 percent of total disbursements. 70. For 13 closed projects, OED had both an ES and a PPAR. Appendix 48 PERFORMANCE TRENDS Outcome7 3. Exit year FY01 marks the second year of project performance exceeding the Strategic Compact Target of 75 percent satisfactory outcomes. As Figure 2 illustrates, 77 percent of projects in the FY01 cohort had satisfactory outcomes. Combining the partial FY02* result with that for FYO1 exits results in a 79% satisfactory rating for the FY01-02* exiting cohort. Overall performance weighted by disbursement decreased slightly from the previous period dropping from 80 to 78 percent satisfactory between FY00 and FY01 and 79 percent for the FY01-02* exiting cohort. Figure 2. Project Performance Exceeds Strategic Compact Target E 100% - 0 80% - 60% - 49 40% - By project -Weighted by disburserment c20%- FY90 FY92 FY94 FY96 FY98 FY00 FY02* 0. Source: Business Warehouse, World Bank 2002. Note: * Partial 4. The improvement in project performance can be attributed to a number of factors. Quality at entry and quality of supervision indicators continued the trend of improvement beginning in the 1990s. Borrower performance2 has improved over the last two years. Improved project performance may also be influenced by the reduced complexity and riskiness" of the FY02* exiting portfolio. The level of complexity decreased for the first time in five years, while project riskiness dropped from 74 percent to 67 percent over the last two exit years. The final results for both FY01 and FY02 will depend on the performance of the remainder of the exiting portfolio. For the FYO1-02 exit period, this amounts to 35 percent of total disbursements. The Quality Assurance Group's (QAG) "at risk" analysis rates 10 percent of this unevaluated portfolio "at risk." 71. OED's measure of outcome considers three factors; relevance, efficacy and efficiency. Relevance measures the expected development impact of a project design by weighing the continuing relevance a project's objectives. Efficacy refers the extent to which each objective was achieved, or expected to be achieved. Efficiency measures the cost-effectiveness of a project, based mainly on sector-wide best practices and indicators where available. 72. OED rates borrower performance on preparation, implementation and compliance. 73. OED rates each project on its complexity and riskiness. Complexity refers to such factors as the range of policy and institutional improvements contemplated, the number of institutions involved, the number of project components and their geographic dispersion, the number of cofinanciers, etc. Riskiness refers to the likelihood that the project, as designed, would be expected to fail to meet relevant project objectives efficiently. 49 Appendix Sustainability and Institutional Development Impact 5. OED ratings of sustainability continue the upward trend beginning in 1996, as demonstrated by Figure 3. Eighty-two percent of the FYO1 exiting disbursements are rated "likely" or "highly likely" to be resilient to future risks'4. For the FY01-02* exiting cohort 84 percent of disbursements are rated as sustainable. 6. The performance of institutional development impact improved marginally over last year's levels with half of the lending in the FY01-02* exiting cohort evaluated as.having "substantial" or "high" institutional development impact. The improvement in sustainability and marginal increase in institutional development may be explained by the improvement in Bank and borrower performance. Figure 3. Improvement in Sustainability' and Institutional Development 100%_ 100%- Sustainablilty Institutional Development 80% 0% - By project a a- Weighted by disbursement 60% - 60% 40% y 40% Weighted by disbursement 20% . . . . . 20% -, FY90 FY92 FY94 FY96 FY98 FYO0 FYO2 Et FY90 FY92 FY94 FY96 FY98 FY00 FY02* ~ ~ I'~ FY ~ ~ Exit Fiscal Year Source: Business Warehouse, World Bank 2002. Note: (i) In July 2000, the rating scale for the sustainability criterion was changed from a 3-point scale (Likely, Uncertain, Unlikely) to a 4-point scale (Highly likely, Likely, Unlikely, Highly unlikely), with the new scale available for projects exiting in FY00-02* (Partial). Regional Performance 7. The Africa (AFR) region continues to narrow the outcome performance gap with other regions in the Bank, with 74 percent satisfactory outcomes for the FY00-FY02* exiting cohort compared with 65 percent for the FY97-99 exiting cohort (Table A.2). As Figure 4 illustrates, the percentage of satisfactory outcomes by disbursements in the Middle East and North Africa (MNA) and South Asia (SAR) regions also improved for the FY00-02* cohort compared with the FY97-99 cohort. The other three regions showed a small deterioration, although Latin America and Caribbean (LAC), and East Asia and Pacific (EAP) regions remain the top performers for both cohorts. 74. In July 2000, the rating scale for the sustainability criterion was changed from a 3-point scale (Likely, Uncertain, Unlikely) to a 4-point scale (Highly likely, Likely, Unlikely, Highly unlikely), with the new scale available for projects exiting in FYOO-02*. Appendix 50 Figure 4. The AFR, SAR and MNA Regions Show Improvement Bankwide Distibution of disbursements SAR by Region (FY97-02*) SAR MNA EAP 22% CECA LCR AFR 18% 13% 40% 50% 60% 70% 80% 90% 100% Suc:Bsns Percent Satisfactory Outcome (Weighted byWaeos,ordBn Disbursements)20. 8. ompredwit FY7-9 exts th oucom peforanc ihtedo b disbursement EEAP 22% AAFR 18% for he Y 2Pexiet Saimpcrove inusiof tWihfteen Seto Boar os-rra, eodcain gD c usenictist 2 nFY97-99 exits pFYoO -02*exits Active Portfolio "Not at Risk" N terioration Sectoral Performance 8. Compared with FY97-99 exits, the outcome performance weighted by disbursement for the FYO0-02* exits improved in six of the fourteen Sector Boards-rural, education, global information and communications technology, financial, private sector development, and transportation (Table A.2, Figure 5). The reasons for improvement and deterioration differ from sector to sector. For example, in the rural sector, increased attention to community participation and knowledge sharing, and the improved performance of irrigation projects contributed to the improved proportion of satisfactory outcomes overall in FY00- 02*. In the HNP sector, the overall performance of the sector in FY00-02* was adversely affected by the poor performance of a few large projects in two regions. 51 Appendix Figure 5. Pattern of Project Outcomes by Sector Board" Bankwide Avg. Distribution of disbursements by WSS Sector Board (FY97-02*) UD Tran RS SP SP 4% %_1%_7% Tran RS HNP 12% UD 5% 4% 2 PSG GCl FS WSS __________ 1% O PSD 9% EP 4% E&M Eu8% HNP En 17% 2% 8% GC1I2 FS FS Sector Boards: EPI Economic Policy EdufEducation; E&MIEnergy and Mining; Env/ Env Environment; FS/Financial Sector, GCII Global Communications/Information, HNP/Heafth, E&M Nutrition and Population; PSDIPtdvate Sector ------------ __ _ JDevelopment; PSGlPublic Sector Governance; Edu RS/Rural Sector SP/Social Protection, TranfTransport; UDIUrban Development; WSSI EP Water Supply and Sanitation 40% 50% 60% 70% 80% 90% 100% Source: Business Warehouse. World Bank Percentage Satisfactory Outcome (Weighted by Disbursements) 2002 Note. * Partial C FY97-99 exits N FYO0-02* (partial) exits EActive Portfolio "Not at Risk" Lending Instrument Performance 9. The percentage of satisfactory outcomes by disbursement for adjustment lending operations was a record high for FY00 exits at 95 percent, but dipped to 71 percent for FY01 exits (Figure 6). Performance for FY01 was highly influenced by two jumbo loans to Russia (each over $400 million), accounting for 70 percent of unsatisfactory disbursements. 10. The percentage of satisfactory outcomes by disbursement for investment lending operations have improved consistently since FY00, after nearly stagnant outcome performance during the 1990s. The performance of investment projects improved from 77 to 80 percent satisfactory between FY97-99 and FY00-02* (Table A.2). 75. The "Sector Board" classification is used rather than the "sector" or "theme" classifications because OED provides a single rating for the entire project rather than rating individual project components that cover particular sectors or themes. The "Sector Board" classification applies to the whole project and enables outcomes to be matched to it. Appendix 52 Figure 6. Long Term Trends in Adjustment and Investment Lending 100% 100% E 9 Investment 980%- --Z 980%-- Adjustment 160%. 1 60% - W 0 0 - Bypect - Byproject - Weighted by disbursements - Weighted by disbursements 20% -20%. 0. 1997 1998 1999 2000 2001 2002* 1997 1998 1999 2000 2001 2002* Ed Ffctl Yw Exit F1scal Year Source Business Warehouse, World Bank 2002. 11. There were ten Adaptable Program Loan (APL) projects and Learning and Innovation Loan (LIL) projects evaluated as of November 2002. Both instruments together had a 72 percent satisfactory outcome rating weighted by disbursement. ASSESSING OUTCOME PERFORMANCE WITHIN THE MILLENNIUM DEVELOPMENT GOALS (MDG) FRAMEWORK 12. The new sector and thematic coding system adopted recently by the Bank identifies themes that can be directly linked to five of the eight MDGs, and enables the assessment of sectoral performance as it relates to those MDGs. Under the new system, projects can be associated with up to five themes identified as either "priority" or "secondary." The analysis in this section describes the performance of priority themes in education, health, and environment. The performance of these themes can be used as a proxy for progress towards the corresponding MDG goals, as indicated in Table 1 below. Table 1. Matching MDGs to the Bank's New Thematic Coding MDGs Bank's New Thematic Coding Achieve Universal Primary Education Education for all Reduce child mortality Child Health Improve matemal health Population and Reproductive Health Combat HIVIAIDS, Malaria and other Fighting communicable diseases diseases Ensure Environmental Sustainability Biodiversity Climate change Environmental policies and institutions Land management Pollution management and environmental health Water resource management Other environment and natural resources management 53 Appendix Achieving Universal Primary Education 13. Education For All (EFA) was a priority theme in 120 projects that exited the Bank's portfolio in FY97-02*. The Education Sector Board was responsible for 84 percent of all EFA projects, which also Figure 7. Lending performance for MDG-related accounted for 89 percent of themes in education, health, and environment disbursements by the Sector ~100% -Board. As illustrated in Figure 7, I 'n _ 1-- -0EFA lending was more 10% - ------------ ------------------------------------------- successful than other types of 80% -------- n=69------------------------- l n=268 euainlnig 70% -------- -------- I~O%14. Of the EFA projects, the e60% --- ----- ------- - 50%-,lending performance in the AFR EFA Education Basic ith, Enoe and MNA region were worse a. except Health except Bank wide, than the Bank-wide average for EFA Basic except Health Environment FY97-02* exits. The NA region has shown improvement Source: Business Warehouse, World Bank 2002. Note: EFA, Basic Health, Environment categories are matched with thematic codes. EFA = Education for all; Basic Health includes Child percentage of satisfactory health, Population & reproductive health and Fighting communicable outcomes increasing from 79 diseases. Outcomes are for FY97-02* (partial) cohort pretfrF9-9eist 0 pecente for 89 exiets o10 percent for the two projects in FYOOd02* exiting cohort; but the performance of the AFR region has stagnated in EFA projects at 66 percent. Basic Health 15. Basic Health, defined in this Review as comprising Child Health, Population and Reproductive Health, and Fighting Communicable Diseases, was a priority theme in more than half of the 96 health projects exiting the Bank's portfolio in FY97-02*. The HNP Sector Board was responsible for 81 percent of Basic Health projects. A higher percentage of Basic Health projects exiting in FYOO-02* had satisfactory outcomes by disbursements relative to other health projects. 16. Of the Basic Health projects, the lending performance in the AFR and NA region was worse than the Bank-wide average for FY97-02* exits. The MNA region has shown no improvement in lending performance, but the performance of the AFR region improved between FY97-99 and FYeis02* exits. Ensuring environmental sustainability 17. Environment was a priority theme in 19 percent of all projects and 16 percent of total disbursements for FY97-02* exits. For environment, the percentage of satisfactory outcomes by disbursements was 7 percentage points below the Bank-wide average, as illustrated in Figure 7. Of the projects with theme of environment, the lending performance in the AFR region were worse than Bank-wide average for FY97p02* exits. Appendix 54 Conclusion 18. Results at the project level continue to meet corporate targets. The impact of Bank assistance must now be scaled up to a higher plane-directly linked to country and global objectives. The MDGs offer the potential for the Bank to sharpen its focus on these goals and on clear indicators by which progress can be demonstrated. 19. The introduction of the new sector and thematic coding system has been a useful step and should be fine-tuned over time to incorporate lessons emerging from its application, and to relate Bank performance to the MDGs and other relevant development priorities more directly and fully. Table A. 1. Outcome, sustainability, institutional development (ID) impact and aggregate by various dimensions, weighted by projects, FY97-99 and FYOO-02* exits; Projects at Risk, similarly disaggregated, for the active portfolio. Exit FY97-99 Exit FY0-02 Active Portfolio Projects Shr Outcome Projects Shar Outcome Sustainability D p rojects S e # Sat lkely or better or better # Sat likely or better or better Risk Economic Policy 64 8 67 52 25 22 3 77 85 41 45 3 80 Education 68 9 78 50 28 72 11 86 80 51 154 11 82 Energy and Mining 88 11 64 53 40 71 11 69 68 48 98 7 79 Environment 22 3 68 62 45 26 4 77 84 54 83 6 80 Financial Sector 57 7 65 59 43 31 5 77 83 55 62 4 90 Global Informatlion/Communications Technology 10 1 90 100 70 8 1 100 100 63 13 1 85 Sector Health, Nutrition and Population 44 6 70 56 30 53 8 65 63 35 135 9 81 Board Private Sector Development 36 5 72 62 31 35 5 65 70 41 72 5 81 PubicSectorGovemance 43 6 88 70 50 51 8 82 85 61 ill 8 77 Rural Sector 148 19 68 47 35 107 16 73 60 44 215 15 87 Social Protection 44 6 84 40 43 39 6 87 72 41 105 8 78 Transport 67 9 88 64 64 76 12 92 84 75 163 11 84 Urban Development 46 6 73 44 31 34 5 74 61 35 76 5 80 Water Supply and Sanitation 44 6 58 33 23 24 4 67 64 38 100 7 80 Lending Adjustment 126 16 79 62 39 70 11 84 88 53 62 4 74 Type Investment 655 84 70 51 38 580 89 77 71 49 1372 96 82 ESSO 170 22 68 49 37 134 21 74 65 46 309 22 85 FSE 57 7 65 59 43 31 5 77 83 55 62 4 90 Network HDN 156 20 78 49 33 163 25 80 73 43 383 27 80 PREM 107 14 74 59 35 73 11 81 85 55 158 11 78 PSI 291 37 72 54 42 246 38 77 73 53 522 38 81 Afica 230 29 58 35 31 171 26 68 58 40 367 26 78 East Asia and Pacific 121 15 83 58 44 100 15 79 70 53 242 17 87 Europe and Central Asia 106 14 82 68 49 144 22 80 83 50 288 20 83 Reg Lan America and Caribbean 168 22 79 62 43 125 19 82 81 58 294 21 83 Middle East and North Afica 58 7 66 53 28 45 7 77 76 50 114 8 73 South Asia 98 13 71 54 33 65 10 86 81 53 129 9 87 Income Low 388 49 65 40 33 319 49 74 62 41 697 49 82 Inome Lower middle 237 30 75 60 40 212 33 82 83 57 495 34 88 Group Uper Middle 148 19 84 72 49 113 17 81 84 80 231 16 81 Grand Total 781 100 72 53 38 650 100 77 73 49 1434 100 82 Notes: Exit FY denotes the year In which the prolect leaves the World Bank's active portfolio. normallv at the end of disbursements Percentanes exclude prolects not rated. Sector Board desianations are taken The data for FY02 exits represents a partial IBRD/IDA lending sample (132 out of 285) and reflects all OED project evaluations through November 12, 2002. The processing of the remainder of the FY02 exits Table A. 2.Outcome, sustainability, institutional development (ID) impact and aggregate by various dimensions, weighted by disbursement, FY97-99 and FYOO-02* exits; Projects at Risk, similarly disaggregated, for the active portfolio. Exit FY97-99 Exit FYOO-02' Active Portfolio Disburse S Share Outcome Sustainability ID Impact Disburse Share Outcome Sustainability ID Impact Disburse Share Disburse millions (Sat.) (likely or better) (better or $ millions (Sat) (likely or (better or $ millions nM at risk substantial) better) substantial) Economic Policy 6300 11 60 69 18 2173 5 76 98 36 4380 4 74 Education 4414 8 86 62 34 4059 9 94 89 55 9945 10 84 Energy and Mining 8938 16 73 68 51 7946 18 66 63 52 9736 10 85 Environment 920 2 71 67 45 1030 2 70 90 35 3789 4 84 Financial Sector 6123 11 81 70 70 3043 7 83 94 62 5333 5 96 Global Information/Communications 630 1 94 100 86 738 2 100 100 69 299 0 75 Sector Technology Board Health, Nutrition and Population 2854 5 85 72 36 1858 4 68 85 39 8109 8 85 Private Sector Development 2087 4 82 71 41 2080 5 84 84 63 2360 2 78 Public Sector Govemance 2814 5 93 72 49 3467 8 90 94 59 4692 5 58 Rural Sector 7634 13 74 55 43 5615 12 84 76 53 15130 15 87 Social Protection 3810 7 97 68 46 2907 8 70 79 38 4994 5 80 Transport 5205 9 87 61 63 7050 16 93 93 78 19166 19 85 Urban Development 2843 5 89 65 31 1608 4 83 72 29 5484 5 84 Water Supply and Sanitation 2984 5 62 22 18 1615 4 57 56 26 8579 7 83 Lending Adjustment 17900 31 83 75 44 11898 26 83 92 53 11070 11 74 Type Investment 39654 69 77 59 44 33295 74 80 77 54 88943 89 85 Ut ESSD 8584 15 73 56 43 8850 15 82 78 51 19194 19 87 FSE 6123 11 81 70 70 3043 7 83 94 62 5333 5 96 Network HDN 11078 19 89 67 39 8763 19 81 81 46 22774 23 83 PREM 9114 16 70 70 27 5639 12 85 96 50 9089 9 66 PSI 22886 39 78 61 47 20579 46 79 76 59 43624 44 84 Africa 8096 14 65 41 32 5255 12 74 62 40 16053 18 80 East Asia and Pacific 12109 21 87 68 49 10284 23 85 80 66 24868 25 91 Europe and Central Asia 8841 15 70 74 47 9941 22 89 83 48 16221 16 81 Region Latin America and Caribbean 14743 28 91 71 55 10851 24 87. 87 58 20685 21 80 Middle East and North Africa 4097 7 69 54 36 2441 5 85 79 57 5229 5 70 South Asia 9668 17 73 61 31 6422 14 84 85 47 16957 17 88 Lncome LOw 22441 39 72 48 31 16535 37 80 71 45 42537 43 85 I ower middle 18632 32 75 71 49 17121 38 75 85 56 32890 33 88 Group Upper Middle 16280 28 91 76 56 11293 25 90 90 65 24199 24 80 Grand Total 57555 100 79 64 44 45193 100 80 81 54 100014 100 84 Notes: Exit FY denotes the year In which the project leaves the World Bank's active portfollo, normally at the end of disbursements Percentages exclude projects not rated. Sector Board designations are taken from Business Warehouse. Income groups designations are taken from the World Development Indicators 2001. Active Portfolio data reflects projects active as of November 12, 2002, and is provided by the Quality Assurance Group (QAG). * The date for FY02 exits represents a partial IBRD/IDA lending sample (132 out of 285) and reflects all OED pmject evaluations through November 12, 2002. The processing of the remainder of the FY02 exits is ongoing, expected to be completed by end FY03. VD 57 Annex I Annex I. Millennium Development Goals, Targets, and Indicators 3oal 1: Eradicate extreme poverty and hunger rarget 1: Halve, between 1990 and 2015, the proportion of 1. Proportion of population below $1 per day eople whose income is less than one $1 per day 2. Poverty gap ratio [Incidence x depth of poverty] 3. Share of poorest quintile in national consumption arget 2: Halve, between 1990 and 2015, the proportion of 4. Prevalence of underweight children (under-five years of age) ole who suffer from hunger 5. Proportion of population below minimum level of dietary energy rnsumption oal 2: Achieve universal primary education irarget 3: Ensure that, by 2015, children everywhere, boys 3. Net enrolment ratio In primary education and girls alike, will be able to complete a full course of prmary. Proportion of pupils starting grade 1 who reach grade 5 ooling Uteracy rate of 15-24 year olds oal 3: Promote gender equality and empower women argot 4: ElIminate gender disparity In primary and secondary 9. Ratio of girls to boys in primary, secondary and tertiary education ucation preferably by 2005 and to all levels of education no 10. Ratio of literate females to males of 15-24 year olds ater than 2015 11. Share of women In wage employment In the non-agricultural sector 12. Proportion of seats held by women In national parliament 3oal 4: Reduce child mortality Target 5: Reduce by two-thirds, between 1990 and 2015, the 13. Under-five mortality rate under-five mortality rate 14. Infant mortality rate 15. Proportion of 1 year old children immunized against measles 3oal 5: Improve maternal health arget 6: Reduce by three-quarters, between 1990 and 2015, 16. Matemnal mortality ratio e matemnal mortality ratio 17. Proportion of births attended by silled health personnel Goal 6: Combat HIVIAIDS, malaria and other diseases Target 7: Have halted by 2015, and begun to reverse, the 18. HIV prevalence among 15-24 year old pregnant women spread of HIVAIDS 19. Contraceptive prevalence rate 20. Number of children orphaned by HIVIAIDS Target 8: Have halted by 2015, and begun to reverse, the 21. Prevalence and death rates associated with malaria nidence of malaria and other major diseases . Proportion of population in malaria risk areas using effective malaria vention and treatment measures 3. Prevalence and death rates associated with tuberculosis 4. Proportion of TB cases detected and cured under DOTS (Directly bserved Treatment Short Course) Annex I 58 3oal 7: Ensure environmental sustainability rarget 9: Integrate the principles of sustainable development 5. Proportion of land area covered by forest nto country policies and programs and reverse the loss of 6. Land area protected to maintain biological diversity invironmental resources 7. GDP per unit of energy use (as proxy for energy efficiency) 8. Carbon dioxide emissions (per capita) lus two figures of global atmospheric pollution: ozone depletion and the ccumulation of global warming gases] farget 10: Halve, by 2015, the proportion of people without 9. Proportion of population with sustainable access to an Improved water ;ustainable access to safe drinking water urce rarget 11: By 2020, to have achieved a significant 0. Proportion of people with access to improved sanitation mprovement In the lives of at least 100 million slum-dwellers 1. Proportion of people with access to secure tenure Urban/rural disaggregation of several of the above indicators may be levant for monitoring improvement in the lives of slum-dwellers] Goal 8: Develop a Global Partnership for Development arget 12: Develop further an open, rule-based, predictable, Some of the Indicators listed below will be monitored separately for the on-discriminatory trading and financial system Least Developed Countries, Africa, landlocked countries and small island ncludes a commitment to good govemance, development, Jeveloping states. nd poverty reduction-both nationally and intemationally Ofcial Development Assistance arget 13: Address the Special Needs of the Least Developed 32. Net ODA as percentage of DAC donors' GNI [targets of 0.7% in total untries nd 0.15% for Least Developed Countries] ncludes: tariff and quota free access for Least Developed 3. Proportion of ODA to basic social services (basic education, primary ntry exports; enhanced program of debt relief for HIPC ealth care, nutrition, safe water and sanitation) nd cancellation of official bilateral debt and more generous . Prot of ODA that is untied PDA for countries committed to poverty reduction DAfrotu4:driess comitthe op es of ed ou s 35. Proportion of ODA for environment in small island developing states nd small island deveioping states 6. Proportion of ODA for transport sector in land-locked countries 'through Barbados Program and 22nd General Assembly arket A 3rovisions) 7. Proportion of exports (by value and excluding arms) admitted free of rarget 15: Deal comprehensively with the debt problems of uties and quotas leveloping countries through national and Intemational 8. Average tariffs and quotas on agricultural products and textiles and neasures in order to make debt sustainable in the long term othing 9. Domestic and export agricultural subsidies In OECD countries . Proportion of ODA provided to help build trade capacity bt Sustainability 1. Proportion of official bilateral HIPC debt cancelled 2. Debt service as a percentage of exports of goods and services . Proportion of ODA provided as debt relief 44. Number of countries reaching HIPC decision and completion points arget 16: In cooperation with developing countries, develop 45. Unemployment rate of 15-24 year olds nd implement strategies for decent and productive work for arget 17: In cooperation with pharmaceutical companies, 46. Proportion of population with access to affordable essential drugs on a rovide access to affordable, essential drugs in developing sustainable basis untries arget 18: In cooperation with the private sector, make 47. Telephone lines per 1000 people valiable the benefits of new technologies, especially 48. Personal computers per 1000 people nformation and communications Source: Wodd Bank Website http/sima/mdg (December 08, 2002). 59 Annex H Annex II. The Origins and Evolution of the MDGs and IDTs The MDGs are the result of the evolution of specific development objectives and targets on poverty, hunger, education, health, gender, and sustainable development adopted by different UN resolutions and conferences throughout the 1990s and, in some cases, much earlier. Throughout the 1990s, the Bank influenced the outcome of the various resolutions and conferences and was also influenced by them." The first impulse for a consolidation and systematization of the different objectives came from the Development Assistance Committee (DAC) of the OECD. In 1995, the OECD countries agreed to review past experience with development assistance and to prepare a blueprint for a more effective program of development assistance. The DAC report, "Shaping the 21' Century: The Contribution of Development Cooperation," was adopted in May 1996, and included seven "International Development Targets" (IDTs)." The IDTs were perceived as a developed-country initiative. Consequently, the UN pushed for a fully participatory set of development objectives that could be endorsed both by developing and developed countries alike. The result was the 2000 Millennium Declaration, and the MDGs. The transition from the IDTs to the MDGs, and the compromises required to achieve broader consensus led to some changes in the goals. Goal 8 "Develop a Global Partnership for Development" was added, while Goal 6 changed from "General access to reproductive health services.. .including safe and reliable family planning methods..." in the IDTs, to a performance indicator under the goal "Combat HIV/AIDS, malaria, and other diseases" in the MDGs. Otherwise, close similarities remained between the IDTs and the MDG Targets. Annex m compares them. The MDGs were further refined in an inter-agency meeting-with World Bank participation-in June 2001, in order to establish more specific targets as well as monitorable performance indicators. The preliminary formulation comprising the 8 "Goals" as well as 18 "Targets," and 48 "Performance Indicators," was published in the UN's "Road Map Towards the Implementation of the United Nations Millennium Declaration," of September 2001 (Annex I provides a listing). These were subsequently reviewed and revised by an interagency expert group, which met between November 2001 and May 2002, and further modifications continue to be considered. In its implementation plan, the 2002 Johannesburg Summit on Sustainable Development, for example, agreed to halve by 2015 the proportion of people who do not have access to basic sanitation. Each of the seven goals addresses an aspect of poverty. The eighth goal is about a global partnership for development designed to help achieve the first seven goals. As conceived by the Millennium Declaration, the eight goals are mutually reinforcing. 76. Worthy of special note is the Bank's 1990 World Development Report that had a profound effect on the return of poverty reduction to the top of the development agenda. Equally, the Bank was much influenced by the UN conferences on gender, habitat and the environment as well as the emphasis on human development that inter alia came with the UNDP's Human Development Report. 77. The IDTs are also called the International Development Goals or IDGs. The term IDTs is used in this Review.  An aex III 60 Annex III. IDTs and MDG Targets: Almost the Same- Intcrnational Development Targets (IDTs) MDG Targets Reduce the proportion of people living in extreme Halve, between 1990 and 2015, the poverty by half between 1990 and 2015 proportion of people whose income is less than $1 per day NEW Target Halve, between 1990 and 2015, the proportion of people who suffer from hunger Em oil all children in primary school by 2015 More Stringent Target Ensure that, by 2015, children everywhere, boys and girls alike, will be able to complete a full course of primary schooling Make progress toward gender equality and Eliminate gender disparity in primary and secondary empowering women by eliminating gender education, preferably by 2005, and to all levels of disparities in primary and secondary education by education no later than 2015 20(5 Reduce infant and child mortality rates by two- Reduce by two-thirds, between 1990 and 2015, the thiids between 1990 and 2015 under-five mortality rate Reduce maternal mortality ratios by three-quarters Reduce by three quarters, between 1990 and 2015, the between 1990 and 2015 maternal mortality ratio Prcvide access for all who need reproductive health Officially dropped from the MDGs; UNFPA, World services by 2015 Bank and other partners include access for all to reproductive health services goal as part of the maternal health MDG NEW Goal and Targets New Goal: Combat HIV/AIDS, malaria, and other diseases New Target: Have halted by 2015 and begun to reverse the spread of HIV/AIDS New Target: Have halted by 2015 and begun to reverse the incidence of malaria and other major diseases implement national strategies for sustainable Integrate the principles of sustainable development into dc relopment by 2005 so as to reverse the loss of country policies and programs and reverse the loss of eirrironmental resources by 2015 environmental resources New Target: Halve by 2015 the proportion of people without sustainable access to safe drinking water New Target: By 2020 to have achieved a significant improvement in the lives of at least 100 million slum-dwellers New Goal and Targets New Goal: Develop a Global Partnership for Development New Targets: Several (See Annex I for a listing) 78. The IDTs and MDGs are being continually updated. This Annex is based on the original targets.  61 Annex IV Annex IV. Progress in Achieving Selected MDGs This Annex presents progress on achieving the global the MDG targets by income grouping-low income, low-middle, and upper-middle income countries. The source is the WDI database of October 2002. The calculations for all but two indicators (HIV/AIDS and maternal mortality) are based on past trends between two points, the earliest taken from the period 1990-1994 and the other from 1995- most recent year. Then it was determined how long it would take a country to achieve the MDG based on the percent change between the two points. Adjustments were made for the indicators based on levels. (Those close to the target were considered more likely to reach it, regardless of past trends.) For the two indicators-prevalence of IffV/AIDS (among young women 15-24) and maternal deaths per 100,000-data were available for only one year. In order to determine progress, thresholds based on industry standards and research were used. Low Income Countries (65 Countries) MDGs Likely Possible Unlikely Very unlikely No data Child malnutrition 8% 11% 5% 22% 55% Primary school completion 12% 23% 18% 22% 25% Gender equality in school 25% 11% 12% 12% 40% Child mortality 5% 23% 29% 23% 20% Matemal mortality 6% 9% 20% 46% 18% HIVIAIDS prevalence 15% 11% 15% 29% 29% Access to water 15% 14% 20% 0% 51% Low-Middle Countries (52 Countries) MDGs Likely Possible Unlikely Very unlikely No data Child malnutrition 29% 6% 2% 8% 56% Primary school completion 42% 17% 10% 4% 27% Gender equality In school 60% 12% 4% 2% 23% Child mortality 31% 29% 15% 10% 15% Matemal mortality 19% 31% 10% 0% 40% HIV/AIDS prevalence 27% 4% 4% 4% 62% Access to water 19% 10% 6% 0% 65% Upper-Middle Countries (36 Countries) MDGs Likely Possible Unlikely Very unlikely No data Child malnutrition 11% 3% 3% 8% 75% Primary school completion 44% 11% 6% 0% 39% Gender equality in school 78% 3% 0% 0% 19% Child mortality 31% 36% 8% 3% 22% Matemal mortality 47% 14% 6% 0% 33% HIVIAIDS prevalence 39% 6% 3% 3% 50% Access to water 6% 3% 8% 0% 83% Note: Child malnutrition refers to prevalence of malnutrition among children under age five, measured by weight for age. Primary school completion refers to percentage of children of appropriate age completing last grade of official primary school. Gender equality in school refers to ratio of girls to boys enrolled in primary and secondary school. Child mortality refers to under-five child mortality. Maternal mortality refers to maternal deaths per 100,000 live births. HIV/AIDS prevalence refers to prevalence of HIV/AIDS among young women (ages 15-24). Access to water refers to percentage of population with access to an improved water source.  Aniaex V 62 Annex V. Bank Initiatives Aimed at Better Managing For Results This Annex lists recent initiatives taken by the Bank to improve its results-orientation. Two sources of information are used, (I) "Poverty Reduction and The World Bank: Progress in Fiscal Year 2002," 2002, Poverty Reduction and Economic Management (PREM), World Bank and (II) "Better Meisuring, Monitoring, And Managing For Development Results," Development Committee Paper, DC2002-0019, 2002. I. Source: "Poverty Reduction and The World Bank: Progress in Fiscal Year 2002." Tho Bank is helping countries to adopt results-based strategies. Specifically, it is: * Providing help to prepare, design and implement PRSP more effectively using Bank- managed Multi-donor Poverty Reduction Strategies Trust Fund. This includes program of learning events that cover a wide range of topics, for example monitoring and evaluation systems, poverty and social impact analysis, and analysis of growth strategies. * Preparing and disseminating good practices with respect to PRSP approach, includes a PRSP Sourcebook. * Conducting research and analytical work to understand "pro-poor" growth, and to evaluate the role of sectoral and structural policy measures in achieving growth and poverty reduction. * Planning to provide knowledge services and analytical support to develop domestic capacity and put in place national monitoring systems. * Deepening country knowledge and understanding of the linkages between public actions and poverty outcomes through (i) poverty and social impact analysis of policy reforms, and (ii) public expenditure management. * Providing support to upgrade the collection, processing, and storage of data by national statistical offices and sector ministries. * Through PARIS21 Consortium, conducting regional meetings to initiate dialogue between information producers, decision-makers, the media and civil society and to prepare national action development plans; conducting follow-up activities at national level to implement the plans; and providing guidance on specific technical works. * Through the Trust Fund for Statistical Capacity Building, supporting 29 projects at country level for improved collection, processing, analysis, storage, dissemination and use of statistics. * Through Evaluation Capacity Development which entails strengthening government monitoring and evaluation (M&E) systems, supporting 21 country level activities to strengthen in-country monitoring and evaluations capacity. Through World Bank Institute (WBI), building client capacity for program and poverty monitoring and evaluation. Since 1998, WBI has provided on-demand distance learning courses on Program Evaluation to clients and joint client/staff groups in approximately 40 countries. In FY03, courses covering basic monitoring and evaluation methods have expanded to include issues of gender and health. This program will expand to strengthen M&E capacity in 12 focus countries. In addition, WBI builds client capacity for poverty monitoring and evaluation, with particular attention to PRSPs: in FY03, courses were offered in 4 countries. * Conducting risk and vulnerability assessment (RVAs), which includes a conceptual framework to analyze the sources of vulnerability (20 completed RVAs, and 11 under preparation). * Conducting gender assessments which lay out a framework for the Bank and borrower country to analyze gender dimensions of development and identify gender responsive policies and actions. 63 Annex V * Through the empowerment and poverty reduction sourcebook, helping to develop empowerment strategies. The Bank is also increasing its own results orientation. Specifically, it is: * Helping to align CAS to national poverty reduction strategies (particularly PRSPs) * Monitoring the ex-post impact of CASs by Bank's Poverty Reduction Group of CASs. * Revising operational guidelines to strengthen linkages between CASs and national poverty reduction strategies; ground bank's strategy in country's poverty situation, and strengthen M&E approaches for CAS. * Strengthening arrangements to allocate IDA resources to favor countries with good performance records, measured by CPIA. * Helping sector interventions to better address poverty reduction in low-income countries. * Refining a new pilot methodology (developed by QAG with support from the Poverty Reduction Board) to help capture ways in which interventions affect poverty. * Conducting selective ex-post analysis of poverty impact of specific intervention and policies. * Rating a sample of analytical work (example poverty assessments, public expenditure reviews, financial accountability assessments, development policy reviews and country economic memoranda), providing an assessment of the poverty focus. * Revising operational guidance on poverty analysis, to ensure that satisfactory poverty analysis is in place for all borrowers and that, when it is not available, arrangements are developed to carry out the analytical work. * Developing criteria for satisfactory poverty analysis, in terms of timing, content, and methods. H. Source: "Better Measuring, Monitoring, And Managing For Development Results." Planned Follow-up In Taking the Results Agenda Forward Client capacity building: Especially PREM's work to support results-focused PRSPs and Joint Staff Assessments; DEC Development Data Group (DECDG)/ Operations Policy and Country Services' (OPCS) work to support statistical capacity building; and OPCS/ PREM, Public Sector Management Division's (PRMPS) work to help countries strengthen their monitoring and evaluation capacity. * Knowledge accumulation and dissemination: Development Economics (DEC), the Networks, and WBI are working to ensure that the Bank's sectoral and thematic advice is up- to-date, practical, and relevant, and readily accessible to clients in a convenient and consistent manner. They also have been charged to develop a work program that reflects the increasingly multisectoral challenges that our clients face in pursuing the MDGs. The World Development Indicators will update MDG data and a United Nations-DEC Committee under the chairmanship of Nick Stem is responsible for tracking progress on MDGs. DEC, Human Development Network, and Regional staff are undertaking MDG country studies in six countries. * Bank strategy and instruments: The focus is on the results-based CAS-which is being piloted by the country teams for Brazil, Cambodia, Cameroon, Sri Lanka, and Ukraine. OPCS staff are also working to see how best to align the M&E framework for investment and adjustment lending with that of the CAS. * Staff learning and incentives: As the above work is completed, it will need to be reflected in staff training programs, along with the results of the ongoing OPCS/WBI review of current Ainex V 64 offerings. Meanwhile, Human Resources and the Networks will be looking into promotion and panel clearance criteria for their alignment with the results agenda. Corporate reporting: (1) Strategy Resource Management's (SRM) work to see how best to reflect results in corporate strategy and budget documents; (2) QAG's work to enhance the results focus of its assessments and to transform its Annual Report on Portfolio Performance into the Annual Report on Portfolio Performance and Results; and (3) OPCS's work with Resource Mobilization unit and other units to see how to strengthen the results measurement system under IDA13, in a way that is fully consistent with the emerging PRSP and CAS M&E frameworks and the ongoing measurement work of DECDG, QAG, and SRM.  65 Bibliography Bibliography Ashley, C. and S. Maxwell (eds.). 2001. "Rethinking Rural Development: Special Issue." Development Policy Review 19:4 December. Baldacci, E., M. T. Guin-Siu, and L. de Mello. 2002. "More on the Effectiveness of Public Spending on Health Care and Education: A Covariance Structure Model." IMF Working Paper. Washington D.C.: IMF. Black. R. and H. White (eds). Forthcoming. "Targeting Development: Critical Perspectives on the Millennium Development Goals." Routledge, London. Brook, P. J. and S. M. Smith. 2001. Contracting for Public Services: Output-Based Aid and its Application. IFC. Washington D.C.: World Bank. Bruns, B., A. Mingat, and R. Rakotomalala. Forthcoming. "Achieving Universal Primary Education by 2015: A Chance for Every Child." Washington D.C.: World Bank. Carvalho, S. and H. White. 1994. "Indicators for Monitoring Poverty Reduction." World Bank Discussion Paper 254. Washington, D.C.: World Bank. Carvalho, S. and H. White. 1997. "Combining the Quantitative and Qualitative Approaches to Poverty Measurement and Analysis: The Practice and the Potential." World Bank Technical Paper 366. Washington, D.C.: World Bank. Claeson, M., C.C. Griffin, T. Johnston, M. McLachlan, A.L.B. Soucat, A. Wagstaff, and A. Yazbaeck. 2001. "Health, Nutrition, and Population" in Poverty Reduction Strategy Sourcebook. June 2001. (September 18, 2002). Collier, P. and D. Dollar. 2001. "Can the World Cut Poverty in Half? How Policy Reform and Effective Aid Can Meet the International Development Goals." World Development 29, No. 11:1787-802, November 2001. Demery, L. and M. Walton. 1998. "Are Poverty Reduction and Other 21st Century Social Goals Attainable?" Poverty Reduction and Economic Management Network. Washington, D.C.: World Bank. Devarajan, S. 2002. "The Costs of Attaining the Millennium Development Goals." HDNVP. World Bank. (September 18, 2002). Devarajan, S., M. J. Miller, and E. V. Swanson. 2002. "Goals for Development: History, Prospects and Costs." HDNVP and DECRG. Washington, D.C.: World Bank. Dillinger, W. 1995. "Better Urban Services: Finding the Right Incentives." Development in Practice. Washington, D.C.: World Bank. Dr6ze, J. and G.G. Kingdon. 2001. "School Participation in Rural India." Review ofDevelopment Economics 5, Issue 1:1-24, February. Easterly, W. 2001. "The Effect of IMF and World Bank Programs on Poverty." World Institute for Development Economics Research, United Nations University. Wider discussion paper (international); No. WDP 2001/102:1-18, October 2001. Easterly, W. 2002. "The Cartel of Good Intentions." Foreign Policy. July/August 2002. (September 18, 2002). Bibliography 66 Ezemenari K., J. Owens, and G. R. Soto. 2002. "Progress on Impact Evaluation Plans in Bank Projects: Comparison of fiscal years 1998-2000." PRMPO, 2000. Fan, S. and P. Hazell. 1999. Are Returns to Public Investment Lower in Less-favored Rural Areas? An Empirical Analysis ofIndia. EPTD Discussion Paper No. 43. Washington, D.C.: International Food Policy Research Institute. May 1999. (September 18,2002). Fan, S., L. Zhang, and X. Zhang. 2000. Growth, Inequality, and Poverty in Rural China: The Role of Public Investments. EPTD Discussion Paper No. 66. Washington, D.C.: International Food Policy Research Institute. (September 18, 2002). Fan, S., P. Hazell, and S. Thorat. 1999. Linkages between Government Spending, Growth and Poverty in Rural India. Research Report 110. Washington, D.C.: International Food Policy Research Institute. (September 18, 2002). Gannon, C. and Z. Liu. 1997. "Poverty and Transport." Discussion Paper TWU-30, Transport Water and Urban Development Department Washington, D.C.: World Bank. Goldin, I., H. Rogers, and N. Stern. 2002. The Role and Effectiveness ofDevelopment Assistance: Lessons from World Bank Experience. Development Economics Vice Presidency. Washington, D.C.: World Bank. (September 18, 2002). Gupta, S., M. Verhoeven, and E. R. Tiongson. 2001. Public Spending on Health Care and the Poor. Washington, D.C.: International Monetary Fund. September 2001. (September 18, 2002). Gwatkin, Davidson R. Forthcoming. "How Much Would the Poor Gain from Progress Toward the Millennium Development Goal Targets for Health?" Health Department Washington, D.C.: World Bank. Hammer, L. and F. Naschold. 2001. "Attaining the International Development Targets: Will Growth be Enough?" Development Policy Review 18, No. 1:11-36, March 2000. Heller, P. S. and S. Gupta. 2002. Challenges in Expanding Development Assistance. IMF Policy Discussion Paper. Washington D.C.: IMF. IMF/World Bank. 2002. "Actions to Strengthen the Tracking of Poverty-Reducing Public Spending in Heavily Indebted Poor Countries (HIPC)." March 2002. (September 18, 2002). International Monetary Fund/OECD/United Nations/World Bank. 2000. 2000: A Better World for All. (September 18, 2002). Johansen, R., et al. 2001 Education and Health in Sub-Saharan Afica: A Review ofSector-Wide Approaches. Report No. 21905. Washington, D.C.: World Bank. Kerf M. and Brenneman, A. 2002. "A Literature Review of the Impact of Infrastructure on Poverty." Mimeo. Washington, D.C.: World Bank. Kessides, C. 1999. World Bank Experience with the Provision of Infrastructure Services for the Urban Poor: Preliminary Identification and Review ofBest Practices. Working Paper Report No. 19634. Washington, D.C.: World Bank. 67 Bibliography Khandker, S.R., V. Lavy, and D. Filmer. 1994. "Schooling and Cognitive Achievements of Children in Morocco." Discussion Paper No. 264. Washington, D.C.: World Bank. Knight, J. and R. Sabot 1990. Education, Productivity and Inequality: the East Afican Natural Experiment. New York: Published for the World Bank, Oxford University Press. Kwon, E. 2001. "Infrastructure, Growth, and Poverty Reduction in Indonesia: A Cross-sectional Analysis." Mimeo, Asian Development Bank, Manila. Lavy, V., J. Strauss, D. Thomas, and P. de Vreyer. 1995. "Quality of Health Care, Survival and Health Outcomes in Ghana." Journal ofHealth Economics 15 (1996) 333-357. Maxwell, S. 2002. "Lost in Translation? Implementation Constraints to Results-Based Management." Maxwell and Kenway. 2000. "New Thinking on Poverty in the U.K: Any Lessons for the South?" ODI Poverty Briefing 9. McMahon, W. 1999. Education and Development: Measuring the Social Benefits. New York: Oxford University Press. OECD. 1996. Shaping the 21st Century: The Contribution of Development Cooperation. Development Assistance Committee, Paris. Washington, D.C.: OECD Washington Center. Picciotto, R. 1999. "Global Public Policy and International Organization (Presentation)." Washington, D.C.: World Bank. Picciotto, R. 2002. "Development Cooperation and Performance Evaluation: The Monterrey Challenge." Operations Evaluation Department. Washington, D.C.: World Bank. Pritchett. L. Forthcoming. "It Pays to be Ignorant: A Simple Political Economy of Rigorous Program Evaluation." Cambridge, MA: Kennedy School of Government. Psacharopoulos. G. 1994. "Returns to Investment in Education: A Global Update." World Development 22. No: 1325-1343 September 1994. Sahn, D. E., and D.C. StifeL 2001. "Progress Toward the Millennium Development Goals in Africa." Cornell University. (September 20, 2002). Sommers, Marc. 2002. "Children, Education and War: Reaching Education for All (EFA) Objectives in Countries Affected by Conflict" Working Papers, Paper No. 1. Conflict Prevention and Reconstruction Unit and Education Team, HDN. Washington, D.C.: World Bank UN Statistics Division/DESA. 2002. "United Nations Millennium Development Goals Data and Trends 2002." Inter-agency Expert Group on MDG Indicators. (September 18, 2002). UNDP. 2001. Development Effectiveness, Review ofEvaluative Evidence. New York: UNDP. (September 20, 2002). UNDP. 2002. Human Development Report, New York: UNDP. United Nations. 200la. "Road Map Towards the Implementation of the United Nations Millennium Declaration." 56th session of the General Assembly, United Nations. (September 20, 2002). Bibliography 68 United Nations. 2001b. Millennium Declaration, A/RES/55/2, 31". Fifty-fifth Session, United Nations, New York. (September 20, 2002). Vandemoortele, J. 2002. "Are the MDGs Feasible?" Bureau for Development Policy, New York: United Nations Development Programme. (September 20, 2002). Walker, W. and C. Sagna. Forthcoming. Social and Poverty Issues/Impacts in the Africa Transport Sector (AFITR): A Technical Audit of the Portfolio. World Bank. Weiss, C. 1998a. Evaluation, 2d ed. Upper Saddle River, NJ: Prentice Hall. World Bank. 199 1a. Forestry Strategy. Washington, D.C.: World Bank. World Bank. 1991b. Urban Policy and Economic Development: An Agenda for the 1990s. Report No. 9576. Washington, D.C.: World Bank. World Bank. 1992. Effective Implementation: Key to Development Impact. Report of the World Bank's Portfolio Management Task Force. Washington, D.C.: World Bank. World Bank. 1993a. "Getting Results: The World Bank's Agenda for Improving Development Effectiveness." Washington, D.C.: World Bank. World Bank. 1993b. "Water Resource Strategy." Washington, D.C.: World Bank. World Bank. 1994a. "An Overview of Monitoring and Evaluation in the World Bank." Report No. 13247. OED. Washington, D.C.: World Bank. World Bank. 1994b. "Twenty Years of Lending for Urban Development 1972-1992: An OED Review." OED. Washington, D.C.: World Bank. World Bank. 1994c. World Development Report 1994: Infrastructure for Development. New York: Oxford University Press. World Bank. 1995. The Social Impact ofAdjustment Lending: An Overview. Report No. 14776. OED. Washington, D.C.: World Bank. World Bank. 1996a. "Morocco Impact Evaluation Report: Socioeconomic Influence of Rural Roads." Report 15808-MOR, OED. Washington, D.C.: World Bank. World Bank. 1996b. Sustainable Transport: Priorities for Policy Reform. Washington, D.C.: World Bank. World Bank. 1997a. "Health, Nutrition, and Population Sector Strategy Paper." Washington, D.C.: World Bank. World Bank. 1997b. "Rural Development From Vision to Action." Washington, D.C.: World Bank. World Bank. 1997c. Taking Action to Reduce Poverty in Sub-Saharan Africa. Development in Practice. Washington, D.C.: World Bank. World Bank. 1998a. "Financial Sector Reform- A Review of World Bank Assistance." Report 17454. OED. Washington, D.C.: World Bank. World Bank. 1998b. "Process Review of World Bank Grant Programs." Report 18317. OED. Washington, D.C.: World Bank. 69 Bibliography World Bank. 1999a. "Education Sector Strategy." Washington, D.C.: World Bank. World Bank. 1999b. Higher Impact Adjustment Lending (HIAL): Initial Evaluation. Report No. 19797, OED. Washington, D.C.: World Bank. World Bank. 1999c. Investing in Health, Development Effectiveness in the Health, Nutrition, and Population Sector. OED. Washington, D.C.: World Bank. World Bank. 1999d. Poverty Assessments: A Follow-up Review. Report No. 19630, OED. Washington, D.C.: World Bank. World Bank. 1999e. "Revisiting Cities in Transition: The Urban and Local Government Strategy Two Years Later." Washington, D.C.: World Bank. World Bank. 1999f. "Rural Development: From Vision to Action," Part I. OED. Washington, D.C.: World Bank. World Bank. 1999g. "Strategic Goals for Chinese Education in the 21st Century." Washington, D.C.: World Bank. World Bank. 2000a. 1999 Annual Review ofDevelopment Effectiveness. OED. Washington, D.C.: World Bank. World Bank. 2000b. Cities in Transition: World Bank Urban and Local Government Strategy Washington, D.C.: World Bank. World Bank. 2000c. "Country Assistance Strategies: Retrospective and Outlook." Operations Policy and Country Services. Washington, D.C.: World Bank. (September 20, 2002). World Bank. 2000d. "Fuel For Thought: An Environment Strategy for the Energy Sector." Washington, D.C.: World Bank. World Bank. 2000e. Poverty Reduction in the 1990s: An Evaluation of Strategies and Performance. OED. Washington, D.C.: World Bank. World Bank. 2000f. "Rural Development. From Vision to Action," Part H. OED. Washington, D.C.: World Bank. World Bank. 2000g. The World Bank Forest Strategy: Striking the Right Balance. OED. Washington, D.C.: World Bank. World Bank. 200 1a. 2000 Annual Review ofDevelopment Effectiveness: From Strategy to Results. OED. Washington, D.C.: World Bank. World Bank. 200 lb. Adjustment Lending Retrospective Final Report OPCS. Washington, D.C.: World Bank. World Bank. 2001c. "Assessment of the Strategic Compact." Washington, D.C.: World Bank. World Bank. 2001 d. "Comprehensive Development Framework, Meeting the Promise? Early Experience and Emerging Issues." SecM2001-0529/1. Washington, D.C.: World Bank. World Bank. 2001e. "Cultural Properties in Policy and Practice." OED. Washington, D.C.: World Bank. World Bank. 200 1f. Drive to Partnership: Aid Coordination and the World Bank. OED. Washington, D.C.: World Bank. Bibliography 70 World Bank. 2001g. "Health, Nutrition and Population Development Goals: Measuring Progress Using the Poverty Reduction Strategy Framework." Report of a World Bank Consultation, Nov 28-29. Washington, D.C.: World Bank. World Bank. 2001h. "Integrating Gender in World Bank Assistance." OED. Washington, D.C.: World Bank. World Bank. 2001i. "Strategic Directions for FYO2-FYO4; Implementing the World Bank's Strategic Framework" (Strategic Directions Paper)." Washington, D.C.: World Bank. World Bank. 2001j. "Strategy for the Financial Sector." Washington, D.C.: World Bank. World Bank. 2001k. "The World Bank's Energy Program: Poverty Reduction, Sustainability and Selectivity." Washington, D.C.: World Bank. World Bank. 20011. "Towards Sharpening the Focus on Rural Poverty. A Review of World Bank Experience." OED. Washington, D.C.: World Bank. World Bank. 2001m. "World Bank Group Strategic Framework paper (Strategic Framework Paper)." Washington, D.C.: World Bank. World Bank. 2002a. 2001 Annual Review ofDevelopment Effectiveness: Making Choices. OED. Washington, D.C.: World Bank. World Bank. 2002b. 2002 Annual Report on Evaluation Capacity Development. OED. Washington, D.C.: World Bank. World Bank. 2002c. 2002 Annual Report on Operations Evaluation, Report No. 24322, OED. Washington, D.C.: World Bank. World Bank. 2002d. "Better Measuring, Monitoring, And Managing For Development Results." Development Committee Paper, DC2002-0019, 2002. Washington, D.C.: World Bank. World Bank. 2002e. Bridging Troubled Waters, Assessing the World Bank Water Resources Strategy. OED. Washington, D.C.: World Bank. World Bank. 2002f. "Cities on the Move" (Urban Transport Strategy), Washington, D.C.: World Bank. World Bank. 2002g. "Concept Note for a Development Committee Paper Proposing A Fast Track Initiative on Health, Nutrition and Population Millennium Development Goals." Washington, D.C.: World Bank. World Bank. 2002. "Development Effectiveness and Scaling Up: Lessons and Challenges from Case Studies." Prepared for the Development Committee. Washington, D.C.: World Bank. World Bank. 2002i. "Development Effectiveness, Partnership, and Challenges for the Future." An issue paper prepared for the Development Committee, March 21. Washington, D.C.: World Bank. World Bank. 2002j. "Educating the World's Children by 2015: A New Policy and Financing Framework." HD Network, Washington, D.C.: World Bank. World Bank. 2002k. "Education for All the World's Children: Donors Have Agreed to Help First Group of Countries on Education Fast-Track Initiative." November 27, Press Release. World Bank Intranet. World Bank. 20021. "Education for Dynamic Economies: Action Plan to Accelerate Progress Towards Education for All (EFA)." April. Prepared for the Spring 2002 Development Committee Meeting. Washington, D.C.: World Bank. 71 Bibliography World Bank. 2002m. "Gender in the PRSPs: A Stocktaking." PREM and GDG. Washington D.C.: World Bank. World Bank. 2002n. "Global Development Finance-Financing the Poorest Countries." World Bank, Washington, D.C. World Bank. 2002o. "HNP Topical Briefing (Presentation)." Washington, D.C.: World Bank. World Bank. 2002p. IDA 's Partnership for Poverty Reduction. OED. Washington, D.C.: World Bank. World Bank. 2002q. "Implementing the World Bank's Strategic Framework." Washington, D.C.: World Bank. World Bank. 2002r. India's Transport Sector- The Challenges Ahead. Washington, D.C.: World Bank. World Bank. 2002s. "Integrating Gender into the World Bank's Work: A Strategy for Action." Washington, D.C.: World Bank. World Bank. 2002t. "Investing in Cities for Better Lives for the Poor." OED. Washington, D.C.: World Bank. World Bank. 2002u. "Making Sustainable Commitments: An Environmental Strategy for the World Bank." Washington, D.C.: World Bank. World Bank. 2002v. "Morocco: The Difference A Road Made." Our Dream, A World Free of Poverty. (September 18, 2002). World Bank. 2002w. Poverty Reduction and the World Bank: Progress in Fiscal Year 2002, PREM, Washington D.C.: World Bank. World Bank. 2002x. "Poverty Reduction and the World Bank. Progress in Operationalizing the WDR 2000/01." (September 18, 2002). World Bank. 2002y. "Poverty Reduction Strategy Papers (PRSP}-Progress in Implementation." Development Committee Paper, DC2002-0016. Washington D.C.: World Bank. World Bank. 2002z. "Private Sector Development Strategy." Washington, D.C.: World Bank. World Bank. 2002aa. Promoting Environmental Stability in Development: An Evaluation of the World Bank's Performance. OED. Washington, D.C.: World Bank. World Bank. 2002bb. "Reaching the Rural Poor-A Renewed Strategy for Rural Development." Washington, D.C.: World Bank. World Bank. 2002cc. "Scaling up: Lessons and Challenges from Case Studies." Development Committee Meeting, SecM2002-0445, Washington D.C.: World Bank. World Bank. 2002dd. "Social Funds: Assessing Effectiveness." OED, Washington, D.C.: World Bank. World Bank. 2002ee. "Strategy Update Paper for FYO3-05: Implementing the World Bank's Strategic Framework." SRM. Washington, D.C.: World Bank. World Bank. 2002ff. "Successes and Challenges in Tertiary Education Lending, 1990-2000." OED Reach. Washington, D.C.: World Bank. World Bank. 2002gg. "The Bank Case for Public Health in Reaching the MDGs and Beyond." Bank's World Today, May 21, 2002. (September 18, 2002). Bibliography 72 World Bank. 2002hh. "The Environment and the Millennium Development Goals." World Bank, Washington D.C. (September 23, 2002). World Bank. 2002ii. "The Gender Dimension of Bank Assistance." OED. Washington, D.C.: World Bank. World Bank. 2002jj. "The World Bank's Approach to Global Programs: An Independent Evaluation-Phase I." OED. Washington, D.C.: World Bank. World Bank. 2002kk. Toward Sharpening the Focus on Rural Poverty: A Review of World Bank Experience. Report No. 24072. OED. Washington, D.C.: World Bank. World Bank. 200211. "Water Resources: Strategic Directions for World Bank Engagement." Washington,. World Bank. Forthcoming(a). "A Revised Forest Strategy for the World Bank Group." Washington, D.C.: World Bank. World Bank. Forthcoming(b). Annual Report on Portfolio Performance (ARPP). QAG. Washington, D.C.: World Bank. World Bank. Forthcoming(c), CODE discussion scheduled for January-June 2003 "Country Assistance Strategies: Retrospective and Future Directions" Washington, D.C.: World Bank. World Bank. Forthcoming(d). "Efficient, Sustainable Service For All? Evaluating the World Bank Assistance to the Water Supply and Sanitation Sector." OED. Washington, D.C.: World Bank. World Bank. Forthcoming(e). "Letting Communities Take the Lead; A Cross-Country Evaluation of Social Fund Performance (Social Funds 2000 Impact Evaluation). Washington, D.C.: World Bank. World Bank. Forthcoming(f). "The Heavily Indebted Poor Countries (HIPC) Debt Initiative: An OED Review." OED. Washington, D.C.: World Bank. World Bank. Forthcoming(g). "The World Bank's Approach to Global Programs: An Independent Evaluation, Phase II Report." OED. Washington, D.C.: World Bank.    OER 25159 Type: