39288 THE WORLD BANK INDEPENDENT EVALUATION GROUP An Independent Evaluation of the World Bank's Support of Regional Programs Case Study of the Lake Victoria Environmental Management Project Shawki Barghouti Director-General: Vinod Thomas Director: Ajay Chhibber Manager: Victoria Elliott Task Manager: Catherine Gwin 2006 This paper is available upon request from IEG. The World Bank Washington, D.C. ENHANCING DEVELOPMENT EFFECTIVENESS THROUGH EXCELLENCE AND INDEPENDENCE IN EVALUATION The Independent Evaluation Group (IEG) is an independent unit within the World Bank; it reports directly to the Bank's Board of Executive Directors. IEG assesses what works, and what does not; how a borrower plans to run and maintain a project; and the lasting contribution of the Bank to a country's overall development. The goals of evaluation are to learn from experience, to provide an objective basis for assessing the results of the Bank's work, and to provide accountability in the achievement of its objectives. It also improves Bank work by identifying and disseminating the lessons learned from experience and by framing recommendations drawn from evaluation findings. IEG Working Papers are an informal series to disseminate the findings of work in progress to encourage the exchange of ideas about development effectiveness through evaluation. The findings, interpretations, and conclusions expressed here are those of the author(s) and do not necessarily reflect the views of the Board of Executive Directors of the World Bank or the governments they represent. The World Bank cannot guarantee the accuracy of the data included in this work. The boundaries, colors, denominations, and other information shown on any map in this work do not imply on the part of the World Bank any judgment of the legal status of any territory or the endorsement or acceptance of such boundaries. Contact: Independent Evaluation Group Knowledge Programs and Evaluation Capacity Development (IEGKE) e-mail: eline@worldbank.org Telephone: 202-458-4497 Facsimile: 202-522-3125 http:/www.worldbank.org/ieg 1 Contents Acronyms..........................................................................................................................iii Preface............................................................................................................................... iv Evaluation Objectives and Methodology................................................................iv Evaluation Criteria.................................................................................................iv Executive Summary......................................................................................................... vi Background.............................................................................................................vi 1. Introduction..................................................................................................................1 Challenges Facing the Sectors.................................................................................1 Regional Program Summary Description................................................................2 2. Relevance: Rationale, Alignment, and Design ..........................................................4 Subsidiarity Principle ..............................................................................................4 Alignment with Country, Regional, and Bank Goals and Strategies.......................5 Regional Consensus.................................................................................................5 Design of the Regional Program .............................................................................6 Clarity and Monitorability of Objectives.................................................................8 3. Efficacy: Outcomes, Impacts, and Sustainability .....................................................9 Achievement of Objectives.......................................................................................9 Capacity Building ..................................................................................................11 Realized Distribution of Costs and Benefits ..........................................................12 Risk to Outcomes and Impacts...............................................................................12 4. Efficiency: Governance, Management, and Financing ..........................................14 Efficient Use of Resources .....................................................................................14 Governance, Management, and Legitimacy ..........................................................14 Financing...............................................................................................................17 Donor Performance ...............................................................................................18 5. Monitoring and Evaluation.......................................................................................19 6. World Bank Performance.........................................................................................20 Comparative Advantage.........................................................................................20 i The Bank's Coordinating Role with Other Donors ...............................................20 Quality of Support and Oversight..........................................................................21 Structures and Incentives.......................................................................................22 Linkages to Other Bank Country Operations ........................................................22 Disengagement Strategy ........................................................................................23 7. Country Participation................................................................................................24 8. Conclusions.................................................................................................................26 Summary of Findings.............................................................................................26 Annex A: Summary of Changes in Lake Victoria Addressed by LVEMP.................29 Annex B: Persons Consulted...........................................................................................31 Annex C: Documents Reviewed......................................................................................32 Tables Table 2.1: Summary of Project Design................................................................................7 Table 4.1: Budget and Expenditures..................................................................................18 Figure Figure 1.1: Maps of the Lake Victoria Region ....................................................................2 ii Acronyms CAS Country Assistance Strategy CIFA Committee for Inland Fisheries of Africa CLVDP Committee for the Lake Victoria Development Program EAC East African Community EAFFRO Establishment of the African Freshwater Fisheries Research Organization ESSD Environmentally Sustainable and Social Development EU European Union FIRI Fisheries Research Institute (Uganda) GEF Global Environmental Facility ICR Implementation Completion Report IDA International Development Association IEG Independent Evaluation Group ISR Implementation Status Reports IUCN The World Conservation Union KARI Kenya Agricultural Research Institute KEMFRI Kenya Marine and Fisheries Research Institute LVEMP Lake Victoria Environment Management Project MIS Management Information System NEAP National Environment Action Plan PMA Program for the Modernization of Agriculture PRSP Poverty Reduction Strategy Papers QAG Quality Assurance Group RPSC Regional Policy and Steering Committee SAR Staff Appraisal Report TAFIRI Tanzania Fisheries Research Institute iii Preface EVALUATION OBJECTIVES AND METHODOLOGY This review of the Lake Victoria Environmental Management Project is one of 19 reviews undertaken as part of an independent evaluation by the Independent Evaluation Group (IEG) of the effectiveness of World Bank support for multicountry regional programs over the past 10 years (1995-2004). Twelve of the reviews are desk reviews; the other seven reviews, including this assessment, are in-depth field studies. This desk review has considered the following relevant documents: the Country Assistance Strategy (CAS) documents for Tanzania (1994), Uganda (1995), and Kenya (1996);. the poverty reduction strategy papers (PRSP) for the three countries; the Lake Victoria Environmental Management Staff Appraisal Reports (SAR) for the three countries, the stock-taking mission report of 2003; the inspection panel report of 2001; and the Quality Assurance Group's (QAG) quality of supervision review of the project in Uganda in 2004. Interviews with Bank staff--including past and present task team leaders and sector mangers--were held to deepen understanding of some issues addressed. See Annex B and C for a list of individuals interviewed and all documentation reviewed. EVALUATION CRITERIA The 19 reviews use the IEG evaluation criteria of relevance, efficacy, and efficiency. In addition, they assess the Bank's performance and examine the performance of the regional program's participating countries. The key evaluative questions addressed under these criteria--designed to deal with the special characters of multicountry programs--are as follows. Relevance · Subsidiarity: To what extent is the program being organized and carried out at the lowest level effective, and how does it complement, substitute for, or compete with Bank country or global programs? · Alignment: To what extent does the program arise out of a regional consensus, formal or informal, concerning the main regional challenges in the sector and the need for collective action? To what extent is it consistent with the strategies and priorities of the region/sub- region, countries, and the Bank? · Design of the regional program: To what extent is program design technically sound, and to what extent does it take into account the different levels of development and interests of participating countries, foster the confidence and trust among participants necessary for program implementation, and have clear and monitorable objectives? iv Efficacy · Achievement of objectives: To what extent has the program achieved, or is it likely to achieve, its stated objectives, including its intended distribution of benefits and costs among participating countries? · Capacity building: To what extent has the program contributed to building capacities at the regional and/or participating country levels? · Risk to outcomes and impact: To what extent are the outcomes and impacts of the program likely to be resilient to risk over time? To what extent have the risks to project outcomes been identified and have measures to integrate them been undertaken? · Monitoring and Evaluation: Has the program incorporated adequate monitoring and evaluation processes and addressed available findings? Efficiency · Efficient use of resources: To what extent has the program realized, or is it expected to realize, benefits by using reasonable levels of time and money? · Governance, management, and legitimacy: To what extent have the governance and management arrangements clearly defined key roles and responsibilities; fostered effective exercise of voice by program participants and coordination among donors; contributed to or impeded the implementation of the program and achievement of its objectives; and entailed adequate monitoring of program performance and evaluation of results? · Financing: To what extent have financing arrangements affected positively or negatively the strategic direction, outcomes, and sustainability of the program? World Bank's Performance · Comparative advantage and coordination: To what extent has the Bank exercised its comparative advantage in relation to other parties in the project and worked to harmonize its support with other donors? · Quality of support and oversight: To what extent has the Bank provided adequate strategic and technical support to the program, established relevant linkages between the program and other Bank country operations, exercised sufficient oversight, and developed an appropriate disengagement strategy? · Structures and Incentives: To what extent have Bank policies, processes, and procedures contributed to, or impeded, the success of the program? Participating Countries' Performance · Commitments and/or capacities of participating countries: How have the commitments and/or capacities of participating countries contributed to or impeded the success of the program? Have one or more countries exercised a primary leadership role? · Program coordination within countries: To what extent have there been adequate linkages between the regional program's country-level activities and related national activities? v Executive Summary BACKGROUND 1. Lake Victoria, the second largest body of fresh water in the world, has experienced sustained environmental deterioration over the past several decades, compounding development challenges for the lake's three riparian countries--Kenya, Tanzania, and Uganda--which remain among the poorest countries in the world. For these countries, the lake basin is a source of food, energy, drinking and irrigation water, shelter, and transport. But the lake also serves as a repository for human, agricultural, and industrial waste. With rapid population growth in the area, these multiple activities have led to an alarming degradation of soil, water, wetlands, and fragile ecosystem conditions; loss of fish populations; and rising resource management costs. The regional Lake Victoria Environmental Project (LVEMP) was designed to address the main underlying features of these problems: declining biodiversity, oxygen depletion in the lake, and reduced water quality. Program Summary Description 2. LVEMP, launched in 1997 as the first phase of a longer-term program, intended to build scientific and institutional capacities needed to stem the environmental deterioration of the lake and its surrounding ecosystems. The effort comprised separate projects, implemented by national secretariats in Kenya, Tanzania, and Uganda and coordinated by a small regional secretariat, established in Arusha, Tanzania. 3. Objectives: LVEMP's three main objectives were to: · Maximize sustainable benefits to riparian communities using basin resources to generate food, income, and safe water, and to reduce disease. · Conserve biodiversity and genetic resources for the benefit of the riparian countries and global community. · Harmonize national management programs, to reverse the increasing environmental degradation. 4. Activities: To achieve these objectives, LVEMP supported three sets of activities. · Activities to provide information on the status of the lake's ecosystem-- particularly the conditions of fisheries, water quality, water hyacinth control, land use and wetlands--and building monitoring capacity for effective lake management. · Activities analyzing the information gathered and building analytical capacity at a regional and country level. · Activities involving trials to address afforestation, watershed management, and treatment of industrial waste in selected pilot zones. 5. National and regional roles: The bulk of these activities were carried out at the national level. Regional activities were limited to technical support to harmonize rules, vi regulations, and standards; training to facilitate the exchange of information and experience; and coordination of donors with international research and training centers. 6. Funding: Funding for LVEMP totaled some $75 million over a seven-year period (1997-2005). Almost all of this amount was external financing, with half from the Global Environmental Facility (GEF) in the form of grants and half in the form of IDA credits, allocated in roughly equal amounts to each of the three countries. The three national governments were initially expected to contribute the remaining 10 percent of the project resources, but their shares were later reduced. Rationale for the Regional Program 7. The three riparian countries have need for efficient and sustainable management of Lake Victoria in order to improve the surrounding communities' livelihoods, transport, commercial fishing, and energy supply.1 8. A regional approach was warranted by the fact that Lake Victoria is a regionally shared natural resource, or regional public good that requires coordinated action by the riparian countries to ensure its environmental sustainability and its continued use. The regional approach also facilitated the mobilization and coordination of external resources, and the achievement of economies of scale in the data gathering, data management, information and knowledge sharing, and capacity building needed to support national lake management efforts. Quality of Design and Implementation 9. Lack of practical application for scientific findings: The project design was based on commendably extensive consultations with stakeholders at the national and local level, and focused on appropriate scientific questions. But it was too complex, and the multiple components were poorly linked to the development objectives. Specifically, the design did not support efforts to translate findings of scientific studies into practical policy and development plans. 10. Effective institutional framework: Overall, the institutional framework provided a sound platform for the implementation of LVEMP's first phase of activities, and would likely work well for a second phase, though with some adjustments. The regional secretariat effectively managed the project in support of the activities largely carried out by the national secretariats--which worked well in Tanzania and Uganda, but not in Kenya. The lines of authority in this institutional structure were clear, the functions of the regional secretariat were kept modest and complementary to local-level actions, and accountability resided appropriately at the country level. But more needed to be done by the regional secretariat to serve as a scientific databank, a clearing center for technical assistance, and a think tank for future policy analysis and capacity building. 1The project was also indirectly relevant to the conditions of the White Nile--for which Lake Victoria is the source--and to the nine countries along the Nile River into which the White Nile flows. vii 11. Inadequate governance mandate: Three permanent secretaries from each country representing the three most concerned ministries--environment, finance, and agriculture/fisheries--served on the Regional Policy and Steering Committee (RPSC). While the RPSC fulfilled its mandate to oversee the project's work program, it lacked a mandate to address quality performance issues such as Kenya's nonperformance, or to address strategic questions such as trade-offs among multiple water uses, the establishment of baseline data, and the linkage of scientific work to policy planning. Program Performance 12. Areas of improvement: Although the pace of implementation was slow, the project made significant impact in three areas: fisheries research, fisheries management, and hyacinth control. Limited achievements were realized in research on improving water quality and pollution control. Pilot schemes in afforestation and watershed management were completed. But the project did not support the development of plans for scaling up these pilots, nor did it support efforts to translate findings of scientific studies into practical policy and development plans that could guide country activities and investments in a follow-up phase. 13. Risks to benefits: LVEMP was not seen at the outset as a high-risk project because all three riparian countries were expected to benefit from investments in mitigation and preservation measures. Yet when the project closed in 2005, its outcomes faced three main risks to their impact on the ground. · The scientific capacity building would not generate sufficient commitment and capacity in the countries for a next phase because the work did not provide a comprehensive picture of the lake's conditions and related policy responses. Nor did it include a cost/benefit analysis to ensure continued national- and community-level buy-in. · Conflict among countries over use of the lake, which emerged under drought conditions, would deepen if the drought was prolonged. This situation would require a response to encourage Uganda, which has hydropower needs, to cooperate with the other countries. · Components at the national level would remain fragmented, in part due to poor donor coordination, and this would make it difficult for the regional and national secretariats to develop a comprehensive vision and plan for future investments. Effectiveness of World Bank Performance 14. The Bank managed its support of LVEMP satisfactorily, maintaining close supervision of the project's several components during implementation and making timely operational adjustments as needed. But it underestimated the time and resources required, and did not foster coordination among components or the needed synthesis of scientific findings. Although it was engaged in complementary analytical work and lending in the individual countries, there was little interaction among the country teams on the regional dynamics of this work. viii 1. Introduction CHALLENGES FACING THE SECTORS 1.1 Lake Victoria, the second largest body of fresh water in the world, has experienced sustained environmental deterioration over the past several decades. This deterioration is compounding development challenges for the lake's three riparian countries--Kenya, Tanzania, and Uganda--which remain among the poorest countries in the world. 1.2 Lake is shared resource: The three countries share the lake's water surface, which covers an area of approximately 68,800 km2, as follows: Tanzania, 49 percent; Uganda, 45 percent; and Kenya, 6 percent. They also have the largest shares of the surface area of the watershed surrounding the lake, together with Rwanda and Burundi. Two rivers flowing out of the lake form the main source of the White Nile; therefore, the decline in the quality of the lake water is also negatively affecting the nine riparian countries of the Nile Basin. (See Figure 1.1 for the maps of the Lake Victoria region.) 1.3 Lake has multiple uses: For Kenya, Tanzania, and Uganda the resources of the lake basin are a source of food, energy, drinking and irrigation water, shelter, and transport. The lake also serves as a repository for human, agricultural, and transport waste. With rapid population growth in the area, the multiple activities in the lake basin have led to an alarming degradation of soil, water, wetlands, and fragile ecosystem conditions; loss of fish populations; and rising resource management costs. 1.4 Specific problem areas: There are three main underlying features of the lake's deteriorating condition: declining biodiversity, oxygen depletion, and reduced water quality. The sources of these problems are interrelated. · Too many nutrients: The inflow of nutrients has increased from rural areas, untreated municipal sewage, and industrial discharges. This increase in nutrients in the water has led to a large increase in the growth of toxic algae, which is causing, in turn, deoxygenating of the lake, increased sickness of humans and animals, clogging of water intake filters, and increased chemical treatment costs. · Too many Nile perch: The proportion of Nile perch in the total fish population has expanded. Nile perch was introduced in the 1950s for commercial fishing. Its increased numbers have had two unintended consequences: fewer endemic fish and less biodiversity. · Too much water hyacinth: The spread of the water hyacinth has further reduced fish populations, hindered fishing operations, begun to choke off important waterways and landings, and provided a preferred breeding habitat for carriers of disease. 1 1.5 Annex A provides a more detailed description of these sources of environmental degradation of the lake and lake basin, which were addressed in the regional Lake Victoria Environmental Project (LVEMP) from 1996 to 2005. Figure 1.1: Maps of the Lake Victoria Region REGIONAL PROGRAM SUMMARY DESCRIPTION 1.6 LVEMP, which was conceived as the first phase of a longer-term program, aimed at building the scientific and institutional capacities needed to stem the environmental deterioration of Lake Victoria and its surrounding ecosystems. The effort entailed separate projects in Kenya, Tanzania, and Uganda that were implemented by national secretariats. A small regional secretariat, established in Arusha, Tanzania, coordinated the national efforts. 1.7 The project derived from the National Environmental Action Plans (Neaps) of the three riparian countries, developed in 1995 and 1996. Each plan noted that Lake Victoria required urgent attention through enhanced regional cooperation; and the emphasis led in 1994 to agreement among the countries to jointly prepare and implement a first five-year phase of a broad Lake Victoria Environmental Management Program. 1.8 Objectives: The main objectives of LVEMP were both environmental and economic. Specifically, they were to: · Maximize the sustainable benefits to riparian communities from using resources within the basin to generate food, employment and income; supply safe water; and reduce disease. · Conserve biodiversity and genetic resources for the benefit of the riparian communities and the global community. · Harmonize national management programs in order to reverse, as much as possible, the increasing environmental degradation. 1.9 Activities: To achieve these objectives, LVEMP supported three sets of activities. The first set was lakewide in scope and focused on providing information on the status of 2 the lake's ecosystem and building monitoring capacity for effective management of the lake. This information gathering focused on the conditions of fisheries, water quality, water hyacinth control, land use and wetlands. The second set of activities was designed to support analysis of the information gathered and analytical capacity building at a regional and country level. A third and final set of activities focused on such areas as afforestation, watershed management, and treatment of industrial waste in selected pilot zones that faced specific environmental threats. 1.10 National and regional roles: The bulk of these activities were carried out at the national level, with some regional facilitation. At the national level, the project supported research and data gathering, piloting of technology to deal with specific problems, and capacity building of national staff in relevant agencies. Regionally, the project was meant to support harmonization of rules, regulations, and standards; strengthen regional units by facilitating exchange of information and experience among the countries through workshops and training events; and maintaining communications with donors and international centers of excellence in research and training. 1.11 Components: The project was originally designed with eight components, and was reduced to seven components after the midterm review. For a list of components, see para 2.12 and Table 2.1. 1.12 Funding: Funding for LVEMP totaled some $75 million over the seven-year period (1997-2005). Virtually this entire amount was external financing, of which half came from the Global Environmental Facility (GEF) in the form of grants and half was in the form of IDA credits, allocated in roughly equal amounts to each of the three countries. The three national governments were expected to contribute the remaining 10 percent of the project resources, but their share was later reduced. 1.13 Evaluations: The project closed in December 2005. During implementation, it was reviewed by Quality Assurance Group (QAG) and the Bank commissioned an independent stocktaking to assess implementation progress and results on the ground, and to identify options for performance improvements. Implementation Completion Reports (ICRs) for the three country operations were prepared in 2006 and provided lessons for the design of a second project phase to be launched in 2007. The second phase will focus on lake management and coordination of action plans to scale up development activities around the lake. 3 2. Relevance: Rationale, Alignment, and Design 2.1 Summary: LVEMP was relevant to the need of the three riparian countries to achieve efficient and sustainable management of Lake Victoria for the livelihoods, transport, commercial fishing, and energy needs of its surrounding communities. The project was also indirectly relevant to the conditions of the White Nile--for which Lake Victoria is the source--and to the nine countries along the Nile River into which the White Nile flows. Based on extensive consultations with stakeholders in the three riparian countries, the project was designed to address local as well as national and regional issues, and was well-focused on appropriate scientific questions. But the project's design was complex, and its multiple components were poorly linked to its development objectives. Also, the design of its several pilot activities failed to mainstream their results into the countries' national agendas. SUBSIDIARITY PRINCIPLE 2.2 The subsidiarity principle states that a program should be organized and carried out at the lowest level effective. In the case of LVEMP, there were two main reasons for establishing a lake management program at a regional level. 2.3 Shared resource: The primary reason was that the program dealt with a regionally shared natural resource, or regional public good, which required the coordinated action of all countries using the resource to ensure its environmental sustainability and its continued beneficial use. No one country acting alone could manage the resource, nor would any one country have sufficient interest to undertake its share of the required actions unless the others also did their part. 2.4 Three levels of operations: The coordinated actions necessary include scientific research about, and ongoing monitoring of, the conditions of the lake; remedial measures in response to overuse, pollution, and other causes of the lake's degradation; and harmonization of rules and regulations to prevent future deterioration. While for the most part these are actions that have to take place on a national or local level, to be most effective they needed to be embedded in a framework of regional cooperation. In other words, the efficient and sustainable management of the lake depended on operations at three levels. · At the community level, the millions of people who lived around the lake shores had harmed the lake, and needed training and extension services to change their behavior. · At the country level, national rules, regulations, standards, and associated policies governing the use and preservation of the lake were needed to facilitate, support and complement community actions. · At the regional level, cross-country interaction was needed to harmonize of policies and regulations of the individual states, and to build and maintain a comprehensive picture of the conditions of the lake. 4 2.5 Economies of scale: The second reason for a regional program was to mobilize and coordinate external resources for the lake management program, and to achieve economies of scale in the design and implementation of data gathering, data management, information and knowledge sharing, and capacity building. The design of LVEMP, as discussed below (paras 2.10-2.13), reflects both of these rationales for regional cooperation. ALIGNMENT WITH COUNTRY, REGIONAL, AND BANK GOALS AND STRATEGIES 2.6 LVEMP, as noted above, was established in direct response to the countries' national environmental plans. It was also consistent with the Bank's country assistance strategies (CAS) for each of the three countries. The Bank supported the three riparian countries in their efforts to enhance economic growth, improve social services to the poor, and protect the environment. As part of this support, the Bank helped each country prepare its national environment plan, and invested in operations to improve the environment within a sustainable development strategy. In the mid-1990s, the CAS for each country reflected this attention to environmental issues. A key element of the 1996 CAS for Kenya was to improve the environmental management of the country, and to help Kenya respond to its commitments to enhance protection of the global environment. The 1994 CAS for Tanzania called for improved public sector management to create a climate for an environmentally sustainable development program; and the 1995 CAS for Uganda supported the objective of building domestic environmental management capacity and, in particular, addressing issues related to the degradation of Lake Victoria. 2.7 In subsequent years, the PRSPs of the three riparian countries and related Bank support addressed rural development and environmental management as a matter of priority. For example, in Uganda the Bank financed several investment operations that dealt with water supply and urban sanitation in the Lake Victoria basin, natural resources management such as soil conservation, and afforestation of catchments around the lake. REGIONAL CONSENSUS 2.8 Regional consensus on the need for cooperation in the management of the lake has existed for several decades, though the riparian countries did not agree to a comprehensive effort until the 1990s. As early as 1928, studies recommended the establishment of a unified lakewide authority for regulation and collection of fishery statistics. Although this authority did not survive the countries' independence, the three riparian countries became members of an African Freshwater Fisheries Research Organization (EAFFRO) established in 1947, and, in the 1970s, they also joined the FAO Committee for Inland Fisheries of Africa (CIFA). 2.9 In the late 1980s and the 1990s, in the context of discussions within the predecessor to the East African Community (EAC), the three countries began to consider a wider range of issues threatening the environmental sustainability of the lake and the lake basin resources. Because the EAC was largely dormant when LVEMP was launched in 1997, the project's management unit was located within the Tanzania national 5 secretariat. But during implementation, and in preparation for a second phase of financing, the project came under the aegis of a revitalized EAC. In 2003, the EAC Council of Ministers approved a Protocol for Sustainable Development of the Lake Victoria Basin, which established a Lake Victoria Basin Commission. In addition, the EAC established a Committee for the Lake Victoria Development Program (CLVDP) composed of permanent secretaries of ministries. The CLVDP is a multisectoral policy- making body which monitors all regional activities related to the lake including LVEMP, which is treated as an integral part of the Lake Victoria Development Program. DESIGN OF THE REGIONAL PROGRAM 2.10 Planning process: The project was prepared over a three-year period (1994-97). There was a commendably high degree of stakeholder consultation during the planning process and ownership of the resulting project design. The process produced three project documents--one for each riparian country--that were subsequently combined into one project document for funding by the International Development Association (IDA) and the Global Environmental Facility (GEF). 2.11 Overly complex and weak framework: The resulting project design was complex, encompassing 8 components and more than 30 subcomponents focused on research of and responses to specific problems. This original design was simplified after a 1999 midterm review prompted a reduction to 7 components and 17 subcomponents, summarized below in Table 2.1. But even with this restructuring, the project remained based on a weak logical framework that poorly linked the project components and that lacked well-defined development objectives. For example, design of the fisheries management components and the wetlands management component did not reflect the importance of wetlands in fish production. Nor did the fisheries management component directly link to the objective of conserving biodiversity, which lacked benchmarks to connect the activity to the goal. Also, several activities were designed to pilot-test technologies to address such issues as afforestation and watershed management, in the hopes that the results would be replicable and scaled up around the lake. But there was no well-defined strategy for how to replicate the pilot activities. 2.12 Components: The project's seven components are: · Fisheries research · Fisheries management · Water hyacinth control · Water quality and ecosystem management · Land and wetlands management · Catchments afforestation · Capacity building 6 Subcomponents and pilot projects are listed in Table 2.1. Table 2.1: Summary of Project Design Component Subcomponents and Pilot Projects Cost Fisheries Research Fish Biology and Biodiversity $13.33m Aquaculture Socioeconomics Information and Database Fisheries Management Establishment of a Fish Levy Trust $18.40m Co-Management (Fisheries Law Enforcement and Closed Fishing Areas) Strengthening the Delivery of Extension Services Micro-Projects and Community Participation in Fisheries Management Quality Assurance Training and Information Dissemination Strengthening Data Collection and Frame Surveys Water Hyacinth Biocontrol Program $8.31m Control Mechanical/Manual/Chemical Control Program Supervision, Legislation and Public Awareness Water Hyacinth Research Water Quality and Management of Industrial and Municipal Effluent Treatment $19.47m Eco-System Industrial and Municipal Waste Management Management Integrated Tertiary Municipal Effluent Treatment Integrated Tertiary Industrial Effluent Treatment Priority Waste Management Investments Land Use Management Management of Pollution Loading Assessment of the Role of Agrochemicals in Pollution of Lake Victoria In-Lake Water Quality Monitoring Sedimentation Studies Hydraulic Conditions in Lake Victoria Lake Victoria Water Quality Model Land and Wetlands Buffering Capacity of Wetlands $14.05m Management Sustainable Utilization and Management of Wetlands and Wetland Products Catchments Afforestation Capacity Building 2.13 National and regional secretariats: These scientific capacity-building activities were coordinated through national secretariats in each country, and were based in the appropriate government institutions concerned with the environment. They were implemented by the relevant government agencies and related institutions (as discussed in more detail in Section 4). In addition, a regional secretariat was established in the office of the Tanzania National Secretariat, using the same staff and facilities, to support the national units and monitor progress on the components. The regional secretariat reported to a Regional Policy and Steering Committee (RPSC)--consisting of three permanent secretaries from each country from the ministries of environment, finance, and agriculture/fisheries--which was responsible for overseeing the overall project. (See Section 4 for more detailed discussion of the main functions and the performance of these institutional structures.) 7 2.14 Kenya's project oversight moved from national secretariat: In Kenya, the Bank moved the project from the national secretariat to the Kenya Agricultural Research Institute (KARI) in the hope of more efficient performance. Instead of speeding up implementation, this move delayed the project for six months and subsequently led to closure of the project before its completion. The effect of the project's complex design on the pace of its implementation and achievements is discussed in more detail in Section 3 on efficacy. CLARITY AND MONITORABILITY OF OBJECTIVES 2.15 Unclear outcome objectives: The major aim of the project was to gather data, carry out research, and pilot responses to specific problems or conditions most harmful to the sustainable management of the lake. But the clarity and monitorability of the project's outcome objectives were not well-specified. There are two distinct aspects of this shortcoming. 2.16 Failure to prioritize objectives: First, project objectives were neither adequately prioritized nor well-connected to one another. The project aimed to address numerous problems, but it established little or no hierarchy among these issues to guide project implementation and the allocation of resources. Instead, all aspects of the lake were seen as needing attention, all issues were treated of equal importance, and although the main objective was scientific capacity building, several nonresearch activities were included, and their objectives did not indicate clearly how they were to be sequenced and implemented in relation to the scientific research. 2.17 Objectives lacked targets: Second, the project did not clearly define a set of monitorable objectives and related performance indicators for its research and data gathering activities. There were numerable objectives stated at the component level and these increased overtime. For example, a report from Uganda in 2002 shows different objectives for every activity, resulting in 145 pages of tables to describe the project. The project performance and impact indicators are without quantified targets. There was no clearly defined plan to develop a state-of-the-lake baseline. The lack of such a baseline undermined the policy relevance of the proposed research. While the scientific emphasis of the project was appropriate, the objectives of the research work did not specify who was expected to use the research and data obtained, or how the findings could be put to use and for what purposes. As discussed in the following chapters, these shortcomings in the clarity and monitorability of the project's stated objectives were subsequently reflected in weaknesses in project monitoring during implementation, and in ensuring the usefulness for policy makers of the research undertaken. 8 3. Efficacy: Outcomes, Impacts, and Sustainability 3.1 Summary: Although the pace of implementation was slow, the project made significant impact in three areas: fisheries research, fisheries management, and hyacinth control. Limited achievements were realized in research on improving water quality and pollution control. Pilot schemes in afforestation and watershed management were completed, but no plan for expanding them was supported by the program. The weak links between scientific activities and policy planning reduced the efficacy of the project; the scientific findings could be useful only if adopted by the stakeholders at all levels, including policy makers, communities surrounding the lake shores, local institutions, civil societies, and others. The adoption process was not well­supported, and the weak monitoring indicators did not allow for effective measurements of how the new discoveries were put into practice. The sustainability of the project's research gains and development results depend on the funding of a second phase project, which the World Bank plans to support with IDA and GEF funding, at a cost almost double that of the initial phase. ACHIEVEMENT OF OBJECTIVES 3.2 Too early to assess achievement of main objectives: It is not possible at the end of LVEMP's first phase (which focused largely on scientific capacity building) to assess the extent to which the project's overarching objectives--to maximize benefits to riparian communities, conserve biodiversity, and harmonize national management programs-- have been achieved. Results will depend on the success of the development investments made in a second phase.2 Therefore, what follows is a summary of progress on implementing the projects components. 3.3 Slow progress on project components: Project implementation was slow. The first two years of implementation produced few research outputs. Much time was taken to establish the one regional and three national secretariats and related management, financial, and procurement procedures. The project closing date was extended by two years, which allowed for a few activities to be completed. But the lack of adequate measurable development objectives and associated performance indicators for each component makes it difficult to assess achievement in terms of results on the ground related the improvement of the lake ecosystem. The following summary of the extent of project achievements draws heavily on the stocktaking commissioned by the Bank in 2003, the Quality of Supervision assessment of 2004, and the Implementation Status Reports (ISRs). 3.4 Components with substantial achievements: In three of the eight project components, achievements were substantial: 2Some initial progress on harmonization has occurred: fisheries legislation was reviewed and updated in Tanzania and Uganda; new legislation to manage water hyacinth was drawn up in those two countries; and gazetting and regulating fish landing sites, and reducing illegal fishing, has been achieved. But this is just the start of a longer-term process. 9 · Fisheries research. Several research projects were completed. Numerous publications and scientific reports on fish species and diversity, and population dynamics were published. · Fisheries management. Considerable efforts were made to reduce illegal fishing through registration and other measures to reduce exploitation of immature fish. Co-management with beneficiaries, including beach management by the communities, has been established to support enforcement and improve post-harvest quality. · Water hyacinth control. Progress was made in both Tanzania and Uganda to assess this problem and initiate mitigation measures, but in Kenya the achievements were far less significant. This component has led to a reduction of water hyacinth coverage by 80 percent to 90 percent in most of the formerly infested areas. The design of the component was weakened by not considering the Kagera River as a significant source of water hyacinth in the lake. Rwanda should be included in a second phase of the regional program to maximize this component's benefits. 3.5 Components with limited achievements: In four other components, achievements were limited: · Water quality and ecosystem management. The water quality monitoring component was satisfactory in both Tanzania and Uganda, but unsatisfactory in Kenya. Yet the industrial and municipal waste management component had poorly formulated objectives, was not well-designed in Kenya and Tanzania, and performance proved unsatisfactory in these two countries. · Land and wetlands management, soil and water conservation, and catchments afforestation. These three components were impeded by inconsistent goals and delays in establishing monitoring networks, especially in Kenya. Work on management of pollution loading was also delayed in Tanzania and Uganda, with limited focus on research and data gathering. 3.6 Weak use of data: The results of the scientific work, if properly presented and synthesized, could provide essential information to guide future management. Such efforts also could establish an environmental and resource baseline so the management of the lake and its basin could be evaluated. But despite the quality and quantity of the information gathered, shortcomings in communication, data storage and dissemination services undermined the value of all these scientific components. LVEMP has not established a specialized data-sharing agreement among the countries. The end-users of data need to be identified and their data needs specified. This would include establishing the data parameters needed for lakewide decision-making management tools. An annual state-of-the-lake report--intended to communicate simply and clearly to stakeholders-- should include a major section containing final data output and analysis. Disseminating information, exchanging ideas, and sharing best practices were part of the project strategy. Several dissemination activities took place--such as stakeholder workshops and 10 seminars, newsletters, radio and television programs, leaflets, newspaper articles, and participation in public events such as agricultural shows. But the large number of research activities did not lead to any overall strategic conclusions about where the focus should be for future investments in remedial and preventive measures. For example, national and regional scientific conferences were held in 2001, but there is no record of how the results of these conferences were disseminated, or what the impact was of the disseminated information. In addition, the project failed to establish a baseline of data on the environmental state of the lake, even though some components made progress in this regard. 3.7 Capacity-building component: Finally, the project did much to build scientific capacity on a regional and national level. It also involved communities in several activities, but there has not been a strong capacity-building program to underpin the community participation. (These capacity-building efforts are discussed in more detail below). CAPACITY BUILDING 3.8 The regional project was designed to build capacity in two ways that complemented its knowledge generation and date gathering: by strengthening the scientific capacity of national institutions to monitor conditions of the lake over time, and by engaging the participation of local communities. Progress was made on both fronts, though with some significant shortcomings. 3.9 Strengthening national institutions: The scientific capacity increased in the national institutions responsible for providing information on the state of Lake Victoria. The project financed post-graduate studies, diplomas, and short courses, leading to 28 Ph.D.s, 88 master's degrees, 10 diplomas, and attendance at 1,487 short courses. This support has generated a significant volume of information on the lake conditions by national scientists, thereby greatly reducing dependence on expatriate expertise. Yet few of the reports produced had been peer reviewed either regionally or internationally by the end of the project, and many reports, books, chapters, and papers remained in draft. 3.10 Failure to address practical uses for research: The capacity-building effort was largely research oriented and made little attempt to translate findings into concrete advice for policy planning or formulating action plans. The information gathered should be interpreted for policy planning, updated regularly to determine positive or negative trends in the lake's resources and environment, and properly used to formulate adaptive strategies to changing conditions. The key, immediate, challenge is to conduct regionwide analysis of the data collected by the national institutions, and to establish a vision for the lake that can provide a framework for setting objectives, defining monitorable indicators, and putting programs in place for managing the lake. The project supported the information gathering required to perform such a strategic analysis and form an action plan, but actual regional analysis and planning had yet to begin by the time the project closed. 11 3.11 Engaging local communities: The project succeeded in engaging the communities in activities such as the management of fish landing sites, afforestation schemes, water hyacinth control efforts, and the promotion of micro-projects. But there was not a strong capacity-building program to underpin the community participation. There has been training in basic skills for the local project management committees. But more resources should have been allocated to orienting district administrative and technical staff, as well as politicians, on the rationale and strategies for community participation. There was poor regional-level coordination of the community participation initiatives. The community participation officers in each of the three countries were not appointed until two years into the project. Moreover, the micro-projects initiated by the communities were weakly linked to the project's overall objectives. REALIZED DISTRIBUTION OF COSTS AND BENEFITS 3.12 Inadequate cost-benefit analysis: Overall, the costs and benefits of the project were not adequately analyzed at the national or regional level. The benefits to the local communities were viewed in limited terms as improved fishing stocks and related diversification of fish species. The communities around the lake can increase their income to improve their livelihood through improved management of the lake, but the costs and benefits to other elements of the national population, such as the upstream farmers, were not considered. 3.13 Conflicting lake uses: At the regional level, the gross economic product of the lake catchments is in the order of $3 billion to $4 billion (U.S. dollars) annually, and supports about 25 million to 30 million people with incomes in the range of $100 to $300 per year. No assessment was done to identify trade-offs among various uses of the lake and possible conflicts between national and regional benefits. Several components were designed to sustain the lake as the largest fresh water lake in Africa, which provides benefits for fishing and tourism. Uganda needs more water to generate power than the other two countries, and the trade-off between achieving environmental standards and increasing water use for energy, especially during drought seasons, was not well- addressed during project design or implementation. RISK TO OUTCOMES AND IMPACTS 3.14 LVEMP was not seen at the outset as a high-risk project because the riparian countries shared interests in a cooperative effort to manage the lake. All three countries stood to benefit--assuming continued external funding for a second phase that would invest in mitigation and preservation measures based on the knowledge gained in phase one. Yet, when the project closed in 2005, it faced several risks to the sustainability of the scientific capacity building it had achieved. · Nonperformance on the part of one or more countries. One risk was anticipated at the design of the project and materialized--nonperformance on the part of one of the countries (Kenya). This reduced Kenya's benefits from the project because its national project was cancelled before completion and supplementary funding was not 12 provided (as it was for Tanzania and Uganda). But at the regional level, Kenya's nonperformance was not a significant problem because scientific capacity-building activities continued in the other two countries. · Other risks. Three other risks, which were not anticipated, have materialized and they pose significant risks to the sustainability of LVEMP's outcomes. · Scientific capacity building was not a sufficient basis for action. The scientific phase has not sufficiently motivated and enabled the countries to move to the next phase. First, the scientific work did not provide a comprehensive picture of the lake's conditions, and there has been no plan for translating the scientific work into policy measures. Second, a cost/benefit analysis was not conducted at the national or community level to ensure continued stakeholder buy-in beyond the initial phase of research work. · Conflict among countries over using the shared resource. This was not seen as a problem at the outset, but has emerged under conditions of continuing drought. If drought continues, the project will need to develop a response that keeps the countries willing to cooperate. · Lack of coordinated donor support. Components at the national level remain fragmented, in part due to poor coordination, and this undermines the ability of the regional and national secretariats to work together in developing a comprehensive vision and plan for future investment actions. 3.15 These risks will have to be addressed in the planning for Phase Two. Measures need to be put in place to respond to possible nonperformance by any individual country, and mechanisms for conflict resolution need to be established to resolve problems arising from emergency conditions, such as continued drought, that create disputes over water use. 13 4. Efficiency: Governance, Management, and Financing 4.1 Summary: On the whole, the project was well-managed by a regional secretariat supported by national secretariats--which worked well in Tanzania and Uganda, but not in Kenya. There were some inefficiencies in the use of resources, though they were not major. The line of authority in the management structure was clear, accountability was well-defined at the country level, and the functions of the regional institutions were kept modest and complemented to where the action needed at the local level. The project was managed by professionally competent units. Members of regional and national units were selected from a pool of experts from specialized national agencies. High-level government officials, who reported regularly to a regionally established policy steering committee, managed the units. The roles of these units were well­defined, especially in terms of implementation, monitoring, reporting, financial management, procurement, and disbursement. Although the Bank closed the project in Kenya before work was completed, due to poor government performance at the country level, it continued support to Tanzania and Uganda and provided supplementary credit to enable those two countries to complete their project implementation. EFFICIENT USE OF RESOURCES 4.2 It is impossible to analyze the cost­effectiveness of the project as a whole, given the lack of overall indicators for each component, and the absence of measurable outputs and activities that have been individually budgeted. Based on the data that is available, several inefficiencies are evident. Regional activities appear to have been underfunded. Notably, there was too little support for regional research activities to maintain an updated body of information on the lake, which would have required building up a regional data bank and collecting information from the three countries on a continuing basis. Flow of funds and procurement procedures were slow in all three countries and were a major factor in lack of progress in the first three years. Improvements took place in Tanzania and Uganda in the out years, but not in Kenya, and completion dates were extended by two years in the former two countries. In addition, the project was more expensive than planned. These resource issues are discussed in more detail below. GOVERNANCE, MANAGEMENT, AND LEGITIMACY 4.3 The project was prepared through a collaborative process under a Tripartite Agreement signed in 1994. The three countries established a governing body, a small regional secretariat, and three national secretariats. Each of these entities had well- defined roles and responsibilities that were moderately well-carried out--except in the case of the Kenya national secretariat, which was closed down and its functions transferred to another national entity (as discussed above). Overall, the institutional framework provided a sound platform from which to implement LVEMP's first phase of activities, and would likely work well in the preparation and implementation of a second phase, with some adjustments. 14 4.4 Governance: The governing body for the project was the Regional Policy and Steering Committee (RPSC), which comprised three permanent secretaries from each country representing the three most relevant ministries--environment, finance, and agriculture/fisheries. The RPSC functioned well in carrying out its stated mandate of overseeing the project's work program. But it lacked the mandate to address--and therefore, did not address--such strategic issues as Kenya's nonperformance, the trade- offs among uses of the lake, and the need for a baseline and link between the scientific capacity building and policy planning. 4.5 Management at the regional level: The regional secretariat was assigned a modest set of functions, and it was allocated an accordingly small budget. It was located in the office of the Tanzania national secretariat, and shared the Tanzanian secretariat's staff and facilities. Its principal functions were: a) servicing the RPSC and arranging the semi-annual meetings of the RPSC members; b) maintaining links to other international organizations; c) supporting the National Secretariats by creating processes to exchange information on policy and legal frameworks, and facilitating capacity building in the countries' public, private and non-governmental organizations ; and d) coordinating, monitoring, assisting in the supervision of, and reporting on the various project components. On the whole, the regional secretariat's support to the national secretariats and its active interaction with donors and Bank missions was satisfactory. 4.6 Regional secretariat's role should expand: But more needs to be done on a regional level, and the regional secretariat is now pressing for increased resources and expanded authority over national units. Such an expansion of its authority should not undermine national ownership and the availability of resources for national-level activities. But the regional secretariat's role does need to be enhanced, in order to serve as a depository of all scientific data and reports and an information clearing center for technical assistance. It should also serve as a think tank for future capacity building, policy analysis and economic and strategic planning for the management of the lake region. 4.7 Management at the national level: LVEMP, which was similarly structured in all three countries, involved three levels of national institutions. Project activities were coordinated by semi-autonomous national secretariats in each country, based in the government's principal environmental ministry or ministry department. They were headed by a full-time official of the rank of deputy or permanent secretary, who was selected by the government, and supported by a small, highly paid staff. While this arrangement gave the national secretariats stature to coordinate activities across sectoral departments, they have shown some of the weaknesses generally characteristic of project management units (PMUs): since they operated outside of regular government structures and staff has received salaries considerably higher than other government workers in positions of equal responsibility, problems of resentment were created. Also, the costs of running the secretariats far exceeded the 4 percent of project expenditures in the original 15 budget, though that was unrealistically below the international norm of 10 percent to 12 percent.3 4.8 Kenya's national secretariat closed: While in Tanzania and Uganda the national secretariats worked well, in Kenya the national secretariat was closed down at the end of 2002, at the insistence of the World Bank after an unsatisfactory rating on project progress. The responsibility for project coordination was moved to the KARI,4 which was chosen by the Bank because of its experience in managing the resources of another Bank-funded project. KARI agreed to take over responsibilities for LVEMP in Kenya on the condition that management of the project was integrated into the normal KARI structures and procedures. But following the shift to KARI, disbursement remained slow and overall project performance did not improve.5 4.9 National agencies implementing components: Major components of the projects in each country were implemented by various national agencies. Notably: · The countries' three Fisheries Research Institutes (KEMFRI, TAFIRI, and FIRI) played lead roles in all subcomponents of fisheries research, and collaborated with the fisheries departments of their respective governments in fisheries extension and with the ministries of water in the water quality components. · The ministries of water were the lead agencies for the water quality components and have collaborated closely with the ministries of environment, natural resources and agriculture in their implementation of the components on land use and wetland management. · National Wetlands Committees in all three countries were involved centrally in the work in this area, assisted by the World Conservation Union (IUCN). 3In Tanzania, the secretariat, including the cost of the regional secretariat functions, accounted for some 21 percent of total project expenditures, while in Kenya and Uganda, the secretariats accounted for about 13 percent of the total. 4KARI did not use the financing provided to it because it did not have the capacity to implement the project's activities. As a result, the World Bank did not extend IDA financing past the closing date of December 31, 2002. The unspent GEF grant funding was allowed to continue to a closing date of December 31, 2004, on the condition that project management was moved to KARI. 5This was the finding of a stocktaking mission to Kenya in 2003, which concluded that the decision to move to KARI was taken hastily and without an analysis of the consequences of such a move for project implementation. Although improvements in management and flow of funds had begun to occur after the restructuring of the national secretariat in July 2002, the adjustments were not given a chance to take effect. According to the mission report, the Bank should have maintained pressure on the participating government agencies, especially the environment department, which will have to play a major role in a second phase of LVEMP. 16 · Researchers in several universities including Moi, Makarere, and Sokoine Universities, and the Universities of Nairobi and Dar es Salaam, carried out many of the studies. · Water-testing laboratories of the Kisumu and Mwanza Municipal Councils, the Uganda Water and Sewage Corporation, and the Lake Basin Development Authority extended the reach of laboratories already operating or planned by the respective Ministries of Water. · National steering committees and related task forces were set up for the water hyacinth control program, while the respective national agricultural research institutes established and operated the rearing units for biological control agents. 4.10 In sum, LVEMP's approach of assigning responsibility and accountability for most of its activities to the countries was practical. But the role assigned and resources provided for regional information gathering and monitoring was too limited. FINANCING 4.11 The total financing for the project amounted to some $75 million. Virtually all of this financing was from external sources: half in the form of GEF grants and half IDA credits to each country in roughly equal amounts. The three governments were expected to contribute 10 percent of the budget, but this requirement was subsequently waived for Tanzania and reduced to 5 percent in the case of Uganda and Kenya. 4.12 After the value of the original IDA and GEF grants and IDA credits decreased due to a decline in the SDR exchange rate, supplementary IDA credits were made to Tanzania and Uganda, but not to Kenya (reflecting its unsatisfactory project performance). Table 4.1 provides a detailed breakdown of this financing. Both Tanzania and Uganda used the bulk of their initial and supplementary funds, though in Uganda it took the Parliament a year to approve the new credit, during which time implementation activities were virtually suspended. In Kenya, because of problems with the flow of funds and the shift to KARI, delays were more extensive. 17 Table 4.1: Budget and Expenditures (million US $) Country Original IDA Supplementary Total Budget and GEF Budget IDA Credit Kenya 24.3 0 24.3 Tanzania 18.8 5.0 23.8 Uganda 23.1 4.5 27.6 Total 66.1 9.5 75.6 DONOR PERFORMANCE 4.13 Although LVEMP was funded solely by IDA and the GEF, other donors have supported a range of initiatives in and around the lake basin for many years. These efforts were taken into account in the financing and implementation of LVEMP, but have not generally been well-coordinated. Some of the donor-supported activities have addressed priority environmental concerns, but they have not been coordinated through any regional mechanisms. Rather, they have, for the most part, been small, fragmented, and uncoordinated undertakings that have put heavy burdens on national implementing agencies--in particular, the countries' agriculture, fisheries, and watershed departments. 4.14 In the absence of a coordinated management and information system for the entire lake and its ecosystem, the donor interventions have not facilitated development of a holistic strategy for the lake. The one success story is the multidonors' support to the work of LVEMP fisheries sector. The European Union financed the fish stock assessment subcomponent under a separate agreement. This partnership with the European Union (EU) has been well-managed throughout the project's implementation. The proposed second phase of LVEMP would benefit from building on lessons from this partnership experience. 18 5. Monitoring and Evaluation 5.1 Inadequate framework: The inadequate logical framework and associated performance indicators used in project design and planning led to monitoring and reporting procedures that do not give a clear picture of the project's progress. The midterm review in 1999 tightened up the structure of the project by reducing the number of components, and subsequent supervision missions focused on dealing with specific issues. However, monitoring and reporting remained inadequate for assessing project progress. The stocktaking mission in 2003 reported that the project had achieved significant progress, but that it did not document such progress adequately. 5.2 Inconsistent reporting formats: Reporting formats were different from country to country and among the components within each country. Much of the reporting consisted of general, un-quantified statements about project achievements. It is clear that the national secretariats do not have an adequate understanding of log framework design and monitoring methods, and the Bank's supervision missions have not done much to rectify the situation. Monitoring and evaluation training was conducted in each country with the aim of creating a commonality of structure and design in each of the three countries. But the training did not result in improved performance indicators. 5.3 Research and capacity building hard to measure: The project was designed to be a mixture of information gathering, capacity building, and institutional support, with emphasis on fisheries management, water hyacinth control, improving water quality, and land use management, including wetlands. The design acknowledged upfront the difficulties in measuring the project impact on these issues, because the focus was more on studying, assessing, and documenting the current situation than on affecting that situation. This issue was compounded by the lack of historical data, which would have helped in measuring change as a result of the project intervention in the above areas. 5.4 Failure to address use of research: The project also failed to plan and track the usefulness of the research. A Management Information Systems (MIS) strategy was not developed, even though it was identified that each national secretariat should have a MIS officer. The MIS officers have been used mainly for producing information about the activities of the project, rather than to elaborate on strategic applications for the new findings at regional, national, and local levels. A relevant MIS strategy would have identified who the key decision-makers were concerning the lake; what sort of information they needed to make informed decisions; and how that information was to be collected, presented, distributed, and followed-up. In the absence of the MIS, the impact of the research was not monitored for its usefulness in policymaking and planning. As a result, the scientific orientation of the project fell short of the expectations of policy makers and planners, who had expected that it would provide input on what actions should be taken to deal with immediate environmental problems (for example, treating waste water, and improving infrastructure for efficient management of industrial and municipal effluents). After six years of supporting research on the conditions of the lake, the task of determining what to do about the prevailing situation remains unaddressed. 19 6. World Bank Performance 6.1 Summary: The Bank managed its support of LVEMP satisfactorily, though it underestimated the time and resources required, and did not foster coordination among components or the synthesis of scientific findings necessary for guiding follow-up planning. The Bank recognized the need for substantial scientific work and designed the operation accordingly. It managed funds from the GEF and added equivalent IDA financing. It maintained close supervision of the project's several components during the implementation, and devoted adequate budget resources to ensuring timely adjustments as needed to implementation plans. In an effort to assess the project's achievements, the Bank also commissioned a stocktaking exercise that reviewed progress by country and by component, and gave the Bank's performance a satisfactory rating. COMPARATIVE ADVANTAGE 6.2 IDA has a large portfolio of projects and adjustment operations in the three East African countries. Building on its wide-ranging relationships with all three governments, IDA is uniquely capable of helping them improve the management of the shared lake. Its experience in designing and implementing projects, especially those which require scientific cooperation among global centers of excellence in research, made it possible for the project to support scientific partnerships through which specialized institutes and agencies shares their expertise with specialized agencies in the region. IDA's experience in supporting research and adaptive application also enabled it to build a convincing partnership with GEF and to implement the project on behalf of both partners. 6.3 Improving the management of Lake Victoria as an international body of water fits into the GEF Operational Strategy for International Water--which includes degradation of water due to land activities; introduction of nonindigenous species; and excessive exploitation of living resources. With poverty endemic to the region and many competing claims for the scarce development financing, the case for joint partnership between IDA and GEF support for this regional program was well-justified. While other donors have supported a range of modestly funded initiatives in and around the lake, the sustainable management of Lake Victoria as a regional resource affecting water and the environment in a large region requires significant and long-term investment, which can only be provided by IDA in partnership with the GEF and other donors. THE BANK'S COORDINATING ROLE WITH OTHER DONORS 6.4 The Bank was the implementing agency for the GEF grant, which was divided into three country operations, each with its legal framework and implementation plan. These jointly funded IDA/GEF operations were the first substantial investments by IDA in two of the three countries following their preparation of National Environment Action Plans. IDA, in partnership with GEF, supported a regional intervention program based on country-level implementation. IDA stated upfront that this intervention is the first phase in a long-term plan to improve the management of Lake Victoria. 20 6.5 The Bank worked with other bilateral donors to ensure that the GEF/IDA- funded activities complemented the work already funded by other donors, especially the EU, Sweden, and the Netherlands, though these donor activities were not well- coordinated. The Bank also mobilized funds from Norway and the EU to bridge the funding gap until phase two, which will be supported by the IDA and the GEF, becomes effective. 6.6 At the strategic level, the Bank has been instrumental in mobilizing support from the GEF for the regional program. Donors have also contributed to this effort through special bilateral arrangements with the Regional Unit in the East Africa Community. This unit coordinated donors' input and monitored related activities, managed the partners' funds, and prepared regional coordination meetings. The Bank also mobilized financial support from several donors, including the EU and Norway, to secure bridging funds needed to maintain activities supported by the first phase and to support the proposed second phase currently under preparation and expected to be approved by GEF/IDA in FY 2007. QUALITY OF SUPPORT AND OVERSIGHT 6.7 Reviews of project: The project was subjected to several comprehensive reviews including a midterm mission, the stocktaking mission, and the quality of supervision review. All these reviews gave satisfactory ratings to the Bank's performance in lending and supervision.6 6.8 Flow of funds: The Bank followed a participatory process in preparing the operation, consulting with donors, and matching the GEF funds to support the implementation of the first phase of the project. The flow of funds from IDA/GEF to the borrowers was efficiently managed. Many attempts were made to assist the governments in speeding up their own flow-of-funds procedures, which succeeded in Tanzania and Uganda. The audit requirements of the Bank were enforced with a degree of flexibility that gave some leeway in timing. Still, in the case of Kenya, the Auditor-General was consistently incapable of fulfilling the requirements. 6.9 Kenya's mismanagement of funds: The supervision reports clearly criticized the inadequate performance of the Kenya project regarding flow of funds, procurement, audits, and management. However, the criticism did not improve Kenya's performance, resulting in the Bank losing patience and making a hasty decision at the end of 2002 to close the IDA funding and move the project to another institution. As noted above, the stocktaking mission argued that this was a mistake, that the new institution was 6QAG reviewed the quality of supervision and concluded that implementation continued to be satisfactory, despite an extensive delay in obtaining effectiveness of the supplementary credit, and despite the Bank's decision to limit supervision activities to one major mission in FY03, when effectiveness of the supplementary credit was still pending. However, during this period the field staff continued informal "interaction" with the national governments, and commissioned relevant studies to assess project impact and to enhance community participation. The QAG panel recognized that the Bank supervision missions have effectively addressed issues related to slow implementation and disbursement. 21 inappropriate for a broad environmental management project for Lake Victoria, and that the consequences of the decision were inadequately analyzed. Two years after the stocktaking mission, QAG reviewed the quality of supervision of this project in Uganda and rated the supervision of that project satisfactory.7 STRUCTURES AND INCENTIVES 6.10 Funding management: As the implementing agency for the GEF-funded LVEMP, the Bank was responsible for both credit and grant quality implementation in the three countries, and the Bank's portfolios were managed under the different country units. Initially managed by a staff member of the ESSD unit in Washington, D.C., this responsibility was transferred to the LVEMP task team leader who was subsequently selected from the field office of the ESSD team in Nairobi. 6.11 Country-level supervision missions: The Bank sustained its support for the regional program through a series of country-level supervision missions. This support also benefited from several lessons learned from the Bank's increasing involvement in the riparian countries--through in-depth sector work and investment lending in each country's rural development, agriculture, environment, and water sectors--which contributed to a productive relationship between the Bank and each country. 6.12 Lack of coordination among country teams: Missing from this positive relationship were well-coordinated relationships among country teams. There is no record in the three countries' supervision reports to indicate that the Bank's country directors met regularly to discuss this regional program. Coordination work was carried out by the project task team, supported by the sector manager, with little or no involvement of the country directors in shaping the strategic framework of this regional operation. The project files give the impression that this regional program did not induce regional cooperation among the Bank's country units. 6.13 Responsibility delegated to country-level team leaders: The decentralization of the project decision-making process allowed for the appointment of a project team leader in each country office, who provided timely response to the clients' needs, and kept the Bank closely engaged with activities on the ground. The delegation of responsibility to the field office has resulted in improved communication with the clients and in more efficient implementation of the project, especially as related to disbursement procurement, and the overall project performance. LINKAGES TO OTHER BANK COUNTRY OPERATIONS 6.14 Bank's extensive involvement in region: The regional program benefited from the relatively large rural and environment portfolio supported by the Bank in the three riparian countries. The Bank's long-term engagement in Kenya, Tanzania, and Uganda--in financing agricultural research and extension, watershed management, dryland development, and institution building in agriculture and rural development-- 7Quality of Supervision Assessment, 2004. 22 provided the strong knowledge base needed to guide the design and implementation of the Lake Victoria regional program. Because of this long-term experience, there was a sense of overconfidence among Bank staff, which resulted in designing a complex project. This complexity was criticized by both the stocktaking mission of 2003 and QAG review in late 2004. 6.15 No coordination with Nile Basin Initiative: The implementation reports do not refer to any coordination with the ongoing work of the regional Nile Basin Initiative. The draft proposal of the next phase refers to the need to coordinate with this initiative. But no operational or functional relationship has been established between these two closely related regional efforts. DISENGAGEMENT STRATEGY 6.16 The Bank has recently supported the three riparian countries--Kenya, Tanzania, and Uganda--in preparing national visions for Lake Victoria basin development. These processes, which involved the participation of local communities, district authorities, and other stakeholders, were followed by the development of a harmonized regional vision for Lake Victoria basin development--the Vision and Strategy Framework for Management and Development of Lake Victoria Basin. Adopted by the EAC in February 2004, this regional vision statement proposes "a prosperous population living in a healthy and sustainably managed environment providing equitable opportunities and benefits." The Bank intends to support the implementation of this vision through a second phase of LVEMP. The new phase is currently under preparation for possible funding by IDA/GEF.8 It is anticipated that the proposed new phase would entail blended project financing by IDA ($90 million), GEF ($60 million), and the borrowers ($15 million).9 8The natural resources management and environmental issues highlighted in the ongoing preparation will be prioritized through a participatory process, and the outputs used to develop a Strategic Action Plan (SAP) for implementing future interventions in the basin. The SAP will be a key input to the design process for the second phase of the regional project. 9Two upstream countries, Rwanda and Burundi, have applied for expedited consideration of their applications to join the EAC, and through memoranda of understanding signed with the EAC, they are currently part of the broader consultations regarding the Lake Victoria basin and the second phase of the LVEMP. 23 7. Country Participation 7.1 Summary: The riparian countries were convinced that achieving the long- term objectives of the project would maximize benefits to the surrounding communities, conserve biodiversity, and strengthen national management programs designed to sustain the lake as an important water resource for the region and for countries of the East African Community. Since the lake water is replenished from the annual seasonal rains, competition among the countries for water for irrigation, hydropower, and consumption was not seen as an issue prior to the severe drought, when Uganda's utilization of lake water to generate hydropower significantly lowered the water level in the lake. While the countries have not addressed this dimension of potential conflict adequately, they have increased their regional cooperation and strengthened coordination of national management plans. Though this improved cooperation and coordination has been less effective in the case of Kenya, since its national project was cancelled due to its underperformance, the national secretariats were reasonably effective in the case of Uganda and Tanzania. But they were not able to ensure that project components were undertaken in-country in a coordinated way, or to ensure that the scientific results were fed into the policy process. Nor did the countries' high-level participation on the PRCS meet these needs. 7.2 Equal representation and participation at the regional level: No one country exercised leadership in launching or coordinating the regional program. All three were equally represented on the regional policy steering committee and the chair of that committee has rotated among the countries. Also, the national secretariats in each of the countries have worked with the executive secretary of the regional secretariat in preparing reports and organizing meetings. 7.3 Major implementation by national secretariats: While the regional secretariat coordinated the program activities among the participating countries, the national secretariats managed the implementation of most program activities and coordinated the work within their respective countries. The national secretariats were semi-independent institutions within the government structures, reporting to the appropriate environmental ministries. Both the national secretariats in Tanzania and Uganda performed well and managed project implementation efficiently, though this was not the case in Kenya.10 7.4 National-level implementation problems: Still there were difficulties in implementation in all three countries. The relationships between all of the national secretariats and their respective implementing agencies were not equal. The title of "national executive secretary" gave the impression that the national secretariat has executive power over the line agencies responsible for the implementation of their respective components. A more modest title and a clear emphasis on the secretariat's role as the facilitating, rather than controlling, unit responsible for assisting the 10It is difficult to ascertain the reasons behind the Kenya government's failure in this regard. It may have been a reflection of the degenerating political/administrative climate, rather than anything particular to the project. There has been a positive change in atmosphere since Kenya's new government came to power at the end of 2002, which is promising for future arrangements. 24 implementation agencies might have been more appropriate for effective utilization of the normal government structures that are necessary for project implementation. Secretariat costs were also an issue. The secretariats were headed by full-time officers of the rank of deputy or permanent secretary, recruited by their governments. All staff was on contract terms of service, outside the government structures. This meant that their salaries were from 3 to 10 times greater that the equivalent posts in government. The stocktaking mission concluded that this arrangement was appropriate in order to attract well-qualified staff on nonpermanent terms. But some of the allowances, especially the daily allowance for traveling regionally, were unreasonably high and bear no relation to actual costs. Significant savings could have been made on these.11 7.5 This gave rise to resentment and probably contributed to a slow pace of implementation. There were long delays in getting the project launched and starting activities in the field during the first two years. A slow start-up phase is normal in most projects. A one-year mobilization phase would have been more appropriate. Two out of the five years planned for implementation were virtually lost in mobilizing the specialized national agencies for implementation. This delay necessitated a two-year extension of the project. Stakeholder ownership was clearly evident, with the governments relying on their own resources and procedures. It is possible that an external facilitator on an advisory basis would have been helpful in shortening the mobilization phase. 7.6 Weak coordination among the specialized agencies implementing the assigned components or subcomponents was an even more significant shortcoming. Each component was handled as a separate project . Their implementation lacked the strategic coordination required to consolidate findings into a holistic project report, which was necessary to guide a cohesive strategy at the national and regional level. This problem was compounded by limited donor coordination of support to country-level activities. Future implementation plans should emphasize interagency cooperation, and ensure that each component complement the overall project strategy by strengthening each component as an integral part of a well-designed strategy for sustained management of the lake. 11The actual cost of running the national secretariats far exceeded the 4 percent of project expenditures in the original project budget. Including the costs of the regional secretariat functions, Tanzania used 21 percent, while Kenya and Uganda used about 13 percent of total expenditures. Costs should have been kept within an international norm of 10 percent-12 percent. 25 8. Conclusions SUMMARY OF FINDINGS 8.1 Lake Victoria can be sustained through better environmental management by the riparian countries. The region faces major conflict from competition for water consumption by the riparians only under emergency weather conditions which should be able to be handled cooperatively. The main scientific capacity-building objectives of the regional program, which aimed to gather information on the lake's conditions and test new practices and technologies for responding to specific problems, were realized. There was strong ownership by the participating countries for the project's several components because they focused on activities that could help alleviate poverty in the basin and improve the natural resources needed to sustain economic growth are under stress. 8.2 The project was evaluated by several teams and was rated satisfactory in Tanzania and Uganda, and unsatisfactory in Kenya. But the project design did not provide for an articulated logframe that would enable adequate monitoring and evaluation and information management. Nor did the project support efforts to translate findings of 8.3 The impact of the scientific capacity building depends on the financing of a second phase to support follow-up development investments. To this end, the Bank has recently supported the three riparian countries in preparing their national visions for Lake Victoria basin development. The second phase would help these countries translate this vision into action, with the support of the scientific findings already available to guide such plans. The challenge that lies ahead is to translate the knowledge acquired in phase one into reliable and sustainable policy frameworks to be adopted by all riparian countries to guide strategic planning, policy development and associated actions plans. Rwanda and Burundi have applied for expedited consideration of their application to join the East African Community, and have become partners in the consultation and funding process to better manage the environment of Lake Victoria. 8.4 There are seven main implications from these findings for future program support: · Well-defined indicators are needed to ensure country ownership: Ownership by participating countries can be strengthened by presenting the program benefits within carefully identified performance indicators. These indicators should be clear and realistic, valid and reliable. They should monitor global and regional development objectives, which usually justify regional intervention, and also assess local development objectives related to the impact of the intervention on the immediate communities. Logical framework approach should be used that conforms to international standards and that is understood and used by all implementing agencies in the borrowing countries. 26 · Regional and national responsibilities need to be clearly delineated: National support to regional efforts can also be strengthened through well- defined regional and national responsibilities and activities. LVEMP was designed as separate operations in each of the riparian countries, which allowed for well-defined allocation of financial resources to each partners. Separate projects with relevant legal documents at the country level helped create flexible supervision plans and related actions for each borrower. When Kenya did not perform satisfactorily, the Bank acted accordingly without affecting the implementation schedule in the other two riparian countries. Such design requires careful monitoring procedures to measure the contribution of each partner to the overall progress in program implementation. · Regional units should play modest roles: The role of the regional units should be well-defined and modest, with an emphasis on supporting national activities. The regional secretariat of LVEMP played an active, yet modest, role in the implementation of the regional program, including disseminating of best practices, guiding the national secretariats on common standards and formats for planning and reporting, and coordinating the production of periodical and scientific reports and other common outputs. Allocating responsibilities to a regional unit is needed to coordinate technical cooperation, build data bank and information management, and support regional communication activities. Such regional communication activities include organizing regional meetings and seminars; conducting policy workshops; and establishing strategic think tanks to help riparian countries design and coordinate suitable national plans needed to better manage the social, economic, and environmental aspects of the lake. · Local institutions should lead the research process: National experts should take the lead in designing and conducting field studies and related technical research. Technically sound designs can be credible only when translated into a simple and easy-to-monitor implementation plan. Recruitment of international firms to conduct studies should be coordinated with the local capacity to interpret these studies and translate them into easy- to-implement operations. Elaborate studies conducted by expatriate consultants may be helpful in designing a technically sound regional program, but may suffer from weak ownership and poor practical application to local conditions. The Lake Victoria regional program supported active engagement of local specialized institutions in these studies. Such support is also essential for strengthening capacity building and increasing the sustainable and cost- effective implementation of the program. · Project impact can be improved though detailed risk and economic analysis: Objective presentation of costs and benefits of regional intervention is essential to assess the impact on the participating communities. The Lake Victoria program addressed poverty alleviation and improvement of livelihoods as a primary focus. Because poverty is widespread throughout the 27 riparian countries, an analysis should be carried out to assess the amount of public expenditure, and its effect on the national economy's priorities. Designing a program to address global and regional public goods is necessary, but not sufficient, for sustaining local support for regional programs. The impact of regional intervention and its associated risks may vary among the riparian countries. Kenya may not assign the same level of support for improving hydropower as Uganda, where hydropower is very important. Such variation should be carefully analyzed and incorporated in the design and implementation of the project. Regional programs should include detailed analysis of risks of failure, because there are likely to be winners and losers from regional interventions. A comprehensive development plan that focuses on incentives needed to alleviate such risks would provide remedial measures. · Project components, scientific objectives, and related studies should be closely interconnected and linked to the development objectives: Regional programs that invest in studies and data gathering should establish functional links among the components, present components within a holistic design, and invest in integrating and operationalizing major findings and data collected into a practical information management system. This effort should target the decision-makers concerned with the environmental management of Lake Victoria. Research and data collection should remain an element of the project, with the aim of creating sustainable input into national policy planning and development strategies. · Community participation should be carefully targeted: Community-driven activities should be financed in order to achieve project objectives. Strong community participation and contribution (both in kind and cash) are key strategic elements in the success of the regional program. The Lake Victoria program mobilized community participation by financing micro-projects needed by the communities. This approach was found to be ineffective because the communities used the funds for these local projects, which did not always contribute to the objectives of the regional program. Mobilizing community participation should be based on well-targeted operations closely related to the program objectives, and funds should be allocated only to those local micro-projects that serve such goals. 28 Annex A: Summary of Changes in Lake Victoria Addressed by LVEMP Declining Biodiversity: Scientists advance two main hypotheses for these extensive changes. First, the introduction of Nile perch as an exotic species some 30 years ago has altered the food web structure. Until 1978, Nile perch remained a very small proportion (about 5 percent) of the commercial catch. The rapid expansion of the proportion of Nile perch took place in the following years, with the result that by 1990 the commercial catch had a totally different composition, dominated by Nile perch (almost 60 percent). Second, nutrient inputs from adjoining catchments are causing eutrophication. Thus although the lake and its fisheries show the evidence of the dramatic changes in the lake basin over the past century, the lake is not the source of the problem. The problems have arisen in the surrounding basins from human activity. Spreading water hyacinth: The first recorded appearance of water hyacinth (Eichhornia crassipes) in Lake Victoria was in Ugandan waters in 1988. Since then, it has been reported in many locations all around the lake--in the waters of all three riparian countries. It is especially concentrated in Ugandan waters, possibly because the prevailing southerly winds blow mats of the weed from lakes far up in the catchments in Rwanda and Burundi down to the Kagera River to the lake. The hyacinth also flourishes in nutrient-rich waters, as those along the Uganda shoreline. The main detrimental effects of the spreading mats of water hyacinth are as follows:(a) reduction in fish through deoxygenation of water and reduction of nutrients; (b) physical interference with fishing operations, especially in the bays where fish are brought ashore; (c) physical interference with commercial lake transportation for people and goods; (d) physical interference with access to water supply from the lake, for both urban and rural communities, and increased costs of purifying water with higher concentrations of suspended, decaying organic matter; (e) threats to the intakes at the Owen Falls hydroelectric power station in Uganda; and (f) provision of a preferred breeding habitat for the alternative host for Schistosomiasis (bilharzia), namely the Biomphalaria snail, a home for the vector mosquito for malaria, and a haven for snakes. Deteriorating water quality: Water quality in Lake Victoria has declined greatly in the past few decades, chiefly because of eutrophication arising from increased inflow of nutrients into the lake. Nutrient inputs have increased two- to three-fold since the turn of the century, mostly since 1950. Concentrations of phosphorus have risen markedly in the deeper lake waters, and of nitrogen around the edges. Stimulated by these and other nutrients, the five-fold increase in algal growth since 1960 and the shift in its composition toward domination by blue-green algae is causing deoxygenation of the water, increased sickness for humans and animals drawing water from the lake, clogging of water intake filters, and increased chemical treatment costs for urban centers. Water pollution and increased nutrient inflows are coming largely from rural areas. Since many of the farms in the area apply no fertilizers, or use very small quantities, these are not likely to be a major source of the nutrients, nor will they be until fertilizer application rates reach substantially higher levels than currently used. Rather, the nutrients may be 29 released from soil particles washed or blown off the land surface by erosion, from burning wood-fuels, and from human and animal waste from areas surrounding the lake. Untreated sewage from urban areas also provides additional nutrients, as well as increases the disease risk from water-borne pathogens. Thus the water-quality problems of the lake arise in the watershed, not in the lake, and it is in the catchments that the solutions must be found. Some areas of the rivers feeding the lake and the shoreline are polluted by municipal and industrial discharges, particularly from such industries as breweries, tanning, fish processing, agro-processing (sugar, coffee), and abattoirs. Some of these industries have implemented pollution management measures but in general the level of industrial pollution control is low. Small-scale gold mining is increasing in Tanzania, leading to some contamination of the local waterways by mercury, which is used to amalgamate and recover the gold. Also, some traces of other heavy metals, such as chromium and lead, have been found in the lake, although the problem has not yet reached major proportions. 30 Annex B: Persons Consulted 1. Sushma Ganguly, Sector Manger, ARD 2. Karen Brooks, Sector Manager AFR 3. Ernst Luts Task Manger and Senior Economist, AFR 4. Ladisy Komba Chengula, Senior Natural Resources Specialist, AFR 31 Annex C: Documents Reviewed 1. Tanzania Country Assistance Strategy (1994) 2. Uganda Country Assistant strategy (1995) 3. Kenya Country Assistance Strategy (1996) 4. The Lake Victoria Environmental Management Staff Appraisal Reports (SAR) for the three countries 5. The Lake Victoria Sock Taking reports for Kenya, Tanzanian and Uganda.2003 6. The inspection panel report of 2001 7. The Quality Assurance Group (QAG) The Quality of Supervision Review of the project in Uganda in 2004. wb308751 O:\CORP\REGIONAL PROGRAMS EVALUATION\Final Background Papers\Lake Victoria.EDITED.doc 03/07/2007 3:50:00 PM 32