Document of The World Bank FOR OFFICIAL USE ONLY Report No: ICR00004844 IMPLEMENTATION COMPLETION AND RESULTS REPORT IBRD-81970 ON A LOAN IN THE AMOUNT OF US$40 MILLION EQUIVALENT TO THE RUSSIAN FEDERATION FOR THE RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) November 25, 2019 Environment, Natural Resources and Blue Economy Global Practice Europe and Central Asia Region This document has a restricted distribution and may be used by recipients only in the performance of their official duties. Its contents may not otherwise be disclosed without World Bank authorization. CURRENCY EQUIVALENTS (Exchange Rate Effective September 3, 2019) Currency Unit = Russian Ruble (RUB) RUB 1 = US$0.015 US$1 = RUB 67.04 FISCAL YEAR July 1 – June 30 ABBREVIATIONS AND ACRONYMS AM Aide Memoire CPS Country Partnership Strategy EIRR Economic Internal Rate of Return ERR Economic Rate of Return EU European Union EX-ACT Ex-Ante Carbon Balance Tool FAO Food and Agriculture Organization of the UN FFA Federal Forest Agency FLEG Forest Law Enforcement and Governance FM Financial Management FMIS Forest Management Information System GHG Greenhouse Gas GoRF Government of Russian Federation GRM Grievance Redress Mechanism ICR Implementation Completion and Results Report IFFC Interregional Forest Fire Center IIASA International Institute for Applied Systems Analysis ISR Implementation Status and Results Report M&E Monitoring and Evaluation MoF Ministry of Finance MNRE Ministry of Natural Resources and Environment MTR Midterm Review NPV Net Present Value PA Protected Area PAD Project Appraisal Document PDO Project Development Objective PIU Project Implementation Unit RF Results Framework SCD Systematic Country Diagnostic WWF World Wildlife Fund Regional Vice President: Cyril Muller Country Director: Renaud Seligman Regional SD Director: Steven Schonberger Practice Manager: Kseniya Lvovsky Task Team Leader(s): Gayane Minasyan ICR Main Contributor: Andrew Zakharenka TABLE OF CONTENTS DATA SHEET .......................................................................................................................... 1 I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES ....................................................... 6 A. CONTEXT AT APPRAISAL .........................................................................................................6 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION ..............................................................11 II. OUTCOME .................................................................................................................... 14 A. RELEVANCE OF PDOs ............................................................................................................14 B. ACHIEVEMENT OF PDOs (EFFICACY) ......................................................................................14 C. EFFICIENCY ...........................................................................................................................17 D. JUSTIFICATION OF OVERALL OUTCOME RATING ....................................................................20 E. OTHER OUTCOMES AND IMPACTS (IF ANY)............................................................................21 III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME ................................ 21 A. KEY FACTORS DURING PREPARATION ...................................................................................21 B. KEY FACTORS DURING IMPLEMENTATION .............................................................................24 IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME .. 25 A. QUALITY OF MONITORING AND EVALUATION (M&E) ............................................................25 B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE .....................................................27 C. BANK PERFORMANCE ...........................................................................................................28 D. RISK TO DEVELOPMENT OUTCOME .......................................................................................29 V. LESSONS AND RECOMMENDATIONS ............................................................................. 29 ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS ........................................................... 31 ANNEX 2. KEY OUTPUTS BY COMPONENT ............................................................................ 41 ANNEX 3. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION ......................... 44 ANNEX 4. PROJECT COST BY COMPONENT ........................................................................... 46 ANNEX 5. EFFICIENCY ANALYSIS ........................................................................................... 47 ANNEX 6. SUPPORTING DOCUMENTS .................................................................................. 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) DATA SHEET BASIC INFORMATION Product Information Project ID Project Name P123923 RUSSIA FOREST FIRE RESPONSE PROJECT Country Financing Instrument Russian Federation Investment Project Financing Original EA Category Revised EA Category Partial Assessment (B) Partial Assessment (B) Organizations Borrower Implementing Agency Federal Forestry Agency, Ministry of Natural Resources Russian Federation and Environment Project Development Objective (PDO) Original PDO The objectives of the Project are to improve forest fire prevention and suppression in select forest ecosystems, including targetedprotected areas, and to enhance forest management in pilot regions. PDO as stated in the legal agreement The objectives of the Project are to improve forest fire prevention and suppression in select forest ecosystems, including Targeted Protected Areas, and to enhance forest management in pilot regions. Page 1 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) FINANCING Original Amount (US$) Revised Amount (US$) Actual Disbursed (US$) World Bank Financing 40,000,000 16,243,621 14,255,428 IBRD-81970 Total 40,000,000 16,243,621 14,255,428 Non-World Bank Financing 0 0 0 Borrower/Recipient 81,260,000 41,247,606 18,778,374 Total 81,260,000 41,247,606 18,778,374 Total Project Cost 121,260,000 57,491,227 33,033,802 KEY DATES Approval Effectiveness MTR Review Original Closing Actual Closing 20-Sep-2012 05-Dec-2013 07-Dec-2015 31-Jan-2018 31-Mar-2019 RESTRUCTURING AND/OR ADDITIONAL FINANCING Date(s) Amount Disbursed (US$M) Key Revisions 15-Dec-2016 7.18 Change in Results Framework Change in Components and Cost Change in Loan Closing Date(s) Change in Financing Plan Change in Institutional Arrangements Change in Implementation Schedule 25-Mar-2019 16.20 Change in Loan Closing Date(s) KEY RATINGS Outcome Bank Performance M&E Quality Unsatisfactory Moderately Satisfactory Negligible Page 2 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) RATINGS OF PROJECT PERFORMANCE IN ISRs Actual No. Date ISR Archived DO Rating IP Rating Disbursements (US$M) 01 02-Apr-2013 Satisfactory Moderately Satisfactory 0 02 03-Dec-2013 Satisfactory Moderately Unsatisfactory 0 03 31-Dec-2013 Satisfactory Satisfactory 0 04 11-Jun-2014 Satisfactory Satisfactory .61 05 18-Dec-2014 Satisfactory Satisfactory 1.91 06 24-Jun-2015 Moderately Satisfactory Moderately Satisfactory 4.75 Moderately 07 26-Nov-2015 Unsatisfactory 5.00 Unsatisfactory 08 14-Mar-2016 Unsatisfactory Unsatisfactory 5.10 09 24-Oct-2016 Unsatisfactory Unsatisfactory 6.77 10 02-Mar-2017 Unsatisfactory Unsatisfactory 10.38 Moderately 11 03-Aug-2017 Moderately Unsatisfactory 11.64 Unsatisfactory 12 19-Dec-2017 Moderately Satisfactory Moderately Satisfactory 15.87 Moderately 13 28-Jun-2018 Moderately Unsatisfactory 16.20 Unsatisfactory Moderately 14 28-Dec-2018 Moderately Unsatisfactory 16.20 Unsatisfactory SECTORS AND THEMES Sectors Major Sector/Sector (%) Agriculture, Fishing and Forestry 94 Public Administration - Agriculture, Fishing & Forestry 26 Forestry 68 Page 3 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Public Administration 6 Sub-National Government 6 Themes Major Theme/ Theme (Level 2)/ Theme (Level 3) (%) Finance 10 Finance for Development 10 Disaster Risk Finance 10 Public Sector Management 6 Public Administration 6 Municipal Institution Building 6 Social Development and Protection 6 Social Inclusion 6 Participation and Civic Engagement 6 Urban and Rural Development 66 Rural Development 36 Land Administration and Management 36 Disaster Risk Management 30 Disaster Response and Recovery 10 Disaster Risk Reduction 10 Disaster Preparedness 10 Environment and Natural Resource Management 12 Renewable Natural Resources Asset Management 12 Biodiversity 12 ADM STAFF Role At Approval At ICR Regional Vice President: Philippe H. Le Houerou Cyril E Muller Country Director: Michal J. Rutkowski Renaud Seligmann Page 4 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Director: Laszlo Lovei Steven N. Schonberger Practice Manager: Kulsum Ahmed Kseniya Lvovsky Task Team Leader(s): Angela G. Armstrong Gayane Minasyan ICR Contributing Author: Andrew Zakharenka Page 5 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) I. PROJECT CONTEXT AND DEVELOPMENT OBJECTIVES A. CONTEXT AT APPRAISAL Context 1. The Project Appraisal Document (PAD) indicates that the Russian Federation has the largest area of closed canopy forests—some 770 million ha, representing 20 percent of the world’s forests and accounting for 15 percent of the world’s timber stock. Official data on forest fires in Russia showed that on average there were 24,000 forest fires annually covering 1.4 million ha of forest during 2000–2010. 1 These figures varied with analysis undertaken by several international institutions, with the estimates ranging from 4.2 million ha in 1999 to 17.3 million ha in 2003. 2 Preliminary analysis of data collected during project preparation indicated that up to 93 percent of the country’s forest fires are caused by humans. 2. Most fires, in most years, were neither in the far northern boreal regions nor in European Russia but generally concentrated closer to the southern borders in Siberia and the Far East. These fires were near settlements, which for that very reason were the more urgent to prevent, monitor, suppress, and extinguish, given the high risk of loss of life and property. These are matters of global concern because total carbon emissions from fire-disturbed Russian forests were estimated at 50 to 231 million tons of carbon per year. 3 Other human activities which increase carbon release included the large-scale historical drainage and burning of peat swamps in Russia with large smoke releases to the atmosphere, a major factor in the catastrophic summer fire season of 2010 in European Russia. Russia's forests are also highly vulnerable to massive pest infestations, which both increase the fire hazard in affected stands and typically accelerate after they have been fire-weakened or damaged. 3. Before appraisal, during the summer of 2010, some 33,500 fires were recorded (50 percent more than in 2009) covering 2.3 million ha, with a large proportion of high-intensity crown fires. Comparing 2010 with 2009, the Central Okrug experienced 32 times greater burned area, the Volga Okrug 27 times, and the Urals 6 times. Carbon emissions from vegetation fires, estimated at 150 million tons, were 50 percent higher than 2009 estimates. Several plumes of smoke rose to the stratosphere over western Russia and by August coalesced into one massive cloud 3,000 km long. On August 7 in Moscow City, air samples showed 6.6 times the normal levels of carbon monoxide, and illness and mortality due to extreme air pollution reached high levels. International Institute for Applied Systems Analysis (IIASA) estimated economic losses from the 2010 summer fires at US$10 billion. The influence of weather conditions should not be overestimated. However, as some observers note, a neighboring Belarus experienced the same drought and heat as western Russia, and the same objective fire dangers, but due to an organized and disciplined fire service, Belarus extinguished the great majority of its fires on the day of detection. 1 Russia’s Forests 2010, Russian Research Institute for Silviculture and Mechanization of Forestry VNIILM, in press. 2 IIASA (International Institute for Applied Systems Analysis). 2011. The Role of ECA’s Forest Resources in Climate Change Mitigation. Draft Final Report to the World Bank. 3 The country’s total greenhouse gas (GHG) emissions in 2009 equated to approximately 1,915 million tons of carbon. Page 6 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Sector Context and Regulations 4. The Soviet Union had a large and advanced forest fire suppression capability, using 600 aircraft, 8,000 smoke jumpers, 4 and 70,000 full-time forest guards. This capability was decimated by budget cuts after the dissolution of the Soviet Union. Because of shrinking fire management resources, aircraft flight hours for fire control declined by 70 percent between 1991 and 2002, and in initial response time consistently increased from 1991 to 2002. These negative trends were exacerbated as an unintended consequence of a major reform embodied in a new Forest Code, which took effect on January 1, 2007, decentralizing the responsibility for fighting forest fires to the 80+ subjects (oblasts, kray, and republics) of the Russian Federation. 5. With the introduction of the Forest Code (2007), regional administrations were required to formulate their own forest policy (under the overall umbrella of the new Forest Code). This led to a fragmented forestry structure throughout Russia, as different regions developed different approaches. The 2007 Forest Code encouraged the commercial exploitation of forests through the use of long-term leases, which made the selling of standing timber by licenses increasingly complex, with significant procedures and red tape that needed to be followed. Many small and medium enterprises, precluded from the long-term leases because of their size, had difficulty in accessing legitimate sources of timber. This created perverse incentives, leading to the forest being deliberately burned to get access to salvage logging permits or to an increase in illegal harvesting. 6. Institutionally, fires that occur in forest land were the responsibility of regional forestry authorities, but fires in agricultural land and peat areas were the responsibility of the regional Ministries of Agriculture. Similarly, fires in Protected Areas (PAs) were the responsibility of the Ministry of Natural Resources and Environment (MNRE), but once they crossed into villages and towns, fires were managed by the Ministry of Russian Federation for Civil Defense, Emergencies and Elimination of Consequences of Natural Disasters (EMERCOM). Coordination between the different responsible agencies was therefore critical to both monitor and respond to wildfires. 7. The Central Government was providing the regions with approximately US$170 million annually for firefighting equipment during 2011–2013. However, many of the regions did not have sufficient capacity to absorb these funds or provide the necessary co-financing. About 40 regions were benefiting from these funds at the time of project appraisal. A central/federal role in operational forest fire monitoring and control was again required to prevent repetition of the 2010 experience, detect fires as soon as possible, and extinguish these smaller fires quickly and efficiently. Rationale for Bank Assistance 8. The project supported two of the four strategic themes of the Country Partnership Strategy (CPS) for 2012–2016 5. Specifically, improved forest fire management supported sustainable development and effective protection of the environment under the first strategic theme - Increasing Growth and Diversification. In addition, the project would strengthen Russia’s contribution to global public goods, thereby supporting the third CPS strategic theme - Deepening Russia’s Global and Regional Role. By 4 Firefighters who parachute in to fight fires at the early stage of development in remote areas. 5 Report No. 65115-RU Page 7 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) supporting the Government in its forest fire prevention and management efforts, the project would also continue a long-standing and comprehensive engagement of the World Bank with Russia on forest management, where efforts have centered on policy development, institutional strengthening, information and land-use planning systems, and strengthening of regional forest inventory and pest protection organizations. This engagement included analytical and advisory activities, as well as EU- supported Forest Law Enforcement and Governance (FLEG) program, which the World Bank implemented with the World Wildlife Fund and International Union for Conservation of Nature. Through this support, the Government would rely on the World Bank as a source of global best practice on forest fire management and sustainable forest management. Further, the World Bank’s engagement would also provide direct support to the country’s Forest Sector Development Strategy through 2020, a national policy document approved by Executive Order of the Ministry of Industry and Trade and Ministry of Agriculture in October 2008. Both programs created favorable conditions for further World Bank’s engagement with an investment operation in the sector. 9. From the broader development perspective, the project was designed to contribute to the country’s sustainable forest management. Also, Russian forests play an important role in the global carbon cycle. The net annual accumulation of carbon in Russian forests accounts for about one-third of the carbon balance estimated for all forests worldwide, and Russia holds the largest terrestrial carbon pool associated with boreal forests that currently provide significant carbon sequestration service to other countries. As such, the future of global climate will be largely influenced by Russia’s land-use policies and management practices. Theory of Change (Results Chain) 10. The underlying premise of the project design is that effective management of forest fires depends on a system for their prevention, detection, and suppression. Hence, the project inputs, outputs, and outcomes were built around these three aspects. Forest fire prevention required extended public awareness campaigns to explain and avert human-origin fires near the settlements and reverse perverse incentives which increase fire incidence, including forest management practices and pest outbreaks. Enhanced detection of forest fires required building technical capacity, upgrading communication systems, and enhancing coordination between the stakeholders and the regions. Finally, fire suppression required re-establishing local firefighting squads, providing machinery and equipment, and training. Comprehensive forest policy reforms were deemed to support the field and regional operations and to build institutional capacity. The results chain of activities by components, outputs, their link to the project indicators and the Project Development Objective (PDO), as well as longer-term outcomes, are described in table 1. Project Development Objective (PDO) 11. The objectives of the project are to improve forest fire prevention and suppression in select forest ecosystems, including targeted protected areas, and to enhance forest management in pilot regions. Page 8 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Key Expected Outcomes and Outcome Indicators 12. Progress toward achieving the PDO was measured against the Average area of fire at detection, the Percentage of fires suppressed within 24 hours after detection, and New silvicultural systems 6 developed and trialed in project pilot regions 7 and PAs. 8 The project aimed to support the Government of the Russian Federation (GoRF) in reducing the possibilities that a fire season similar to that of 2010 could repeat. As such, it was expected for project results to indicate that fires which occur in the pilot regions, PAs, and those areas monitored by interregional forest fire management coordination centers would be better controlled and reduced in extent and severity as they would be detected sooner, response time would be reduced due to better communications, firefighting capacity in terms of equipment and trained personnel would be increased, and interagency and interregional firefighting coordination and cooperation would be enhanced. Also, other interventions, such as support provided to establishing interregional forest fire centers (IFFCs) and to policy development would play a role in improving forest fire prevention and forest management at a broader level. 6 The intention was to reduce reliance on clear felling and artificial restocking and to encourage selective thinning, regeneration felling, ground preparation, and natural regeneration. 7 Five project regions include Khabarovsk Kray, Komi Republic, Krasnoyarsk Kray, Moscow Oblast, and Voronez Oblast 8 13 project PAs include Meschera, Okskiy, and Meshcherskiy in the Central Federal Okrug; Kerzhensky, Buzuluskiy, Zhigulevskiy, and Samarkskaya Luka in the Volga Federal Okrug; Sayano-Shushenskiy, Stolby, and Shushkenskiy Bor in Krasnoyarsk Kray (Siberian Federal Okrug); and Bureinkiy, Komsomolskiy, and Bastak in the Far East Federal Okrug Page 9 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Table 1. Results Chain Long-Term Component/Activities Outputs Outcomes/PDO Indicators Development Impact 1. Enhancing Forest Fire Prevention, Management,  Ground-based forest fire response and Control improved Sustainable use To improve forest fire prevention and and protection of  Improving detection, response, management,  Early fire detection and response suppression in select forest ecosystems, and coordination of firefighting efforts strengthened forest resources including targeted protected areas  Upgrading the fire danger rating and hazard  Prompt response and cross- index system deployment (regionally) supported PDO #1. Increase in percentage of fires  Restoring forest fire stations  Firefighting systems and machinery contained within 24 hours following  Upgrading communications systems upgraded detection in pilot extensive forest regions Significant carbon-  Supplying suitable firefighting machinery,  Ensured full coverage in 5 regions and project PAs sequestration equipment, protective clothing, gear, and hand  Communications strengthened PDO #2. Average area of fires at tools detection in project PAs  Holding stakeholder consultations  Running public awareness programs and media PDO #3. Area covered by improved campaigns Forest Pest Monitoring System in Pilot Reduced  Preparing related follow-up operations Regions deforestation PDO #5. Reach and acceptance of fire 2. Building Forestry and PA Management Capacity  Incentives for illegal logging and prevention messages among the PA  Providing technical assistance to support the setting of forest fires reversed target population in comparison to institutional framework  Silvicultural practices to reduce fuel baseline Reduced economic  Identifying and addressing key policy and load and area of pest damaged trees loss from fires legislative issues modified  Undertaking economic cost benefit analysis and  Forest regeneration an efficiency of To enhance forest management in pilot improving the recording of the causes of the fires forest management improved regions.  Providing targeted investments to improve forest  Analysis and fire record system regeneration and restoration developed PDO #4. Area of improved forest Reduced pollution  Developing model forests for sustainable forest  Sustainable system and institutions for regeneration based on modern management forest fire management with a clear technology in Pilot Regions  Training forest administration and management distribution of functions and specialists responsibilities established Reduced pest  Building human resource capacity  Technical and human capacity infestations  Preparing related follow-up operations improved Page 10 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Components 13. As presented in the PAD, the project was divided into two primary components. Component 1 would enhance forest fire prevention, management, and control; and Component 2 would build forest and PA management capacity to reduce fire risk and enhance sustainability. Component 1 was designed to improve the effectiveness of forest fire prevention and management by (a) improving the capacity for early detection and quick response to fight forest fires and (b) reducing the number of fires of human origin through awareness raising and environmental education programs. Early fire detection is recognized as a key element in improving protection of forests from fires. Fire prevention and environmental education were expected to be focused on children and youth to encourage long-term behavior change. Component 2 was designed to increase forest and PA management capacity and help address key policy and management issues that either create perverse incentives or exacerbate conditions contributing to the extent and intensity of fires in the extensive forest landscape and PAs. Component 3 (Project Management) financed the operating costs of a Project Implementation Unit (PIU) that implemented both Components 1 and 2. 14. The project was originally designed with two implementing agencies: the Federal Forestry Agency (FFA - a subordinate agency of the MNRE) for Part A of Components 1 and 2 (US$97.75 million total, US$30.30 million loan) and the MNRE for Part B of Components 1 and 2 (US$20.02 million total, US$6.21 million loan). There was a common PIU for both parts of the project (Component 3: Project Management) (US$3.49 million loan). 15. The PAD stage envisioned the following project components and their allocations, as shown in Table 2. Table 2. Original Project Components and Allocations IBRD Project Cost, Financing, Components US$, millions US$, millions Component 1: Enhancing Forest Fire Prevention, Management, and Control 73.375 22.746 1.A Enhancing Forest Fire Prevention, Management, and Control in Russia’s 57.400 Extensive Forest Fund 1.B Enhancing Forest Fire Prevention, Management, and Control in Russia’s 15.975 Protected Areas 1.C Preparation of Follow-up Operation Component 2: Building Forestry and PA Management Capacity 44.393 13.762 2.A Building Forestry Management Capacity in Russia’s Extensive Forest 40.347 Fund 2.B Building Management Capacity in PAs 4.046 2.C Preparation of Follow-up Operation Component 3: Project Management 3.492 3.492 Total 121.260 40.000 B. SIGNIFICANT CHANGES DURING IMPLEMENTATION 16. The midterm review (MTR) held in December 2015 reassessed the implementation status, constraints, and proposed necessary adjustments. A year later, project performance had not improved, Page 11 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) so the project was restructured. Project restructuring (December 15, 2016) aimed at improving the project implementation pace and effectiveness. The restructuring included partial cancellation of project financing (borrower’s side) and realignment of the financing between the components (Financing Plan). The restructured project would reduce the investments in the pilot regions while adding new PAs, adjust the Results Framework (RF), adjust the institutional arrangements and disbursement estimates, and extend the loan closing date. The second restructuring (March 25, 2019) brought the project to closing and loan termination on March 31, 2019. Revised PDOs and Outcome Targets 17. The PDO remained the same. However, the indicators’ targets were revised. Revised PDO Indicators 18. PDO Indicators were revised to reflect the changes in scope and financing of the project components (Table 3). Two indicators were dropped altogether (Area of model forests in pilot regions established and under new silvicultural systems, and Forest area brought under management plans) and three new indicators were added (Reach and acceptance of fire prevention messages among the PA target population in comparison to baseline, Area of improved forest regeneration based on modern technology in pilot regions, and Area covered by improved forest pest monitoring system in pilot regions). Baseline and targets of four other PDO indicators were also revised. Table 3. Revised PDO indicators Original Indicators Revised/New Indicators 1. Increase in percentage of fires contained within Remained 24 hours following detection in pilot extensive forest regions and project PAs 2. Average area of fires at detection in pilot region Revised: Average area of fires at detection in extensive forest and project PAs project Pas 3. Area of model forests in pilot regions Dropped established and under new silvicultural systems 4. Forest area brought under management plans Dropped 5. New: Reach and acceptance of fire prevention messages among the PA target population in comparison to baseline 6. New: Area of improved forest regeneration based on modern technology in pilot regions 7. New: Area covered by improved forest pest monitoring system in pilot regions Revised Components 19. The first restructuring (December 15, 2016) reduced the investments in the pilot regions while it increased the number of participating PAs from 12 to 25. The reduction in investments in Part A were from the reduction in and dropping of planned activities; the geographic coverage remained the same. The total project financing was reduced by 33 percent from US$121.26 million to US$81.25 million as shown in Table 4. The overall change in funding was due to reduction of the borrower's contribution by Page 12 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) US$40.01 million. The loan amount (US$40 million) remained unchanged.. Part A was downsized to contracts already signed or awarded during the time of restructuring and all other activities were cancelled. Part B was scaled up to cover additional 13 PAs, with some additional activities in all areas. Table 4. Restructuring Details Details At Approval Restructured By source(s) Borrower 81.26 41.25 IBRD 40.00 40.00 Total 121.26 81.25 By Component 1. Enhancing Forest Fire Prevention, Management, and Control 73.38 62.04 2. Building Forestry and PA Management Capacity 44.39 14.21 3. Project Management 3.49 5.00 Total 121.26 81.25 Other Changes 20. The first restructuring also shifted the volume of financing between the unchanged implementing agencies from mainly being implemented by the FFA (around 83 percent of expenditure previously to just 34 percent after restructuring) to mainly being implemented by the MNRE (60 percent of expenditure after restructuring). Also, it extended project closing to December 31, 2019 mainly because of the increased scope with additional PAs and initial delays with project effectiveness (December 2013, 14 months after the signing) and implementation. The second restructuring then terminated the loan and project on March 31, 2019. Rationale for Changes and Their Implication on the Original Theory of Change 21. After a reasonable start in 2014, the project was poised to significantly increase in disbursement. However, with the changes in the senior management of the FFA, implementation of Part A (by the FFA) had ceased between February 2015 and March 2016. For example, by December 2015, there were outstanding contracts for approximately US$22 million, of which US$11 million was for contracts which had been awarded but not signed. Following the downgrade of project performance ratings and discussions with the Ministry of Finance (MoF) and MNRE, the MoF requested the World Bank that the project be restructured so that the activities implemented by the FFA are restricted to the already signed and/or awarded contracts and that Part B (MNRE implemented) be increased to expand its scope and coverage from 12 to 25 PAs. All the restructuring changes were made to adapt to the changing administrative environment and to provide better clarity on outputs and outcomes. In terms of the implications, the restructuring would have shifted the impact more toward the PAs and reduced the anticipated project impact on the Extensive Forest Fund. However, the original PDO that envisaged improving forest fire prevention and suppression, as well as enhancing forest management, was maintained. 22. With no further progress in the project implementation and stalled activities, the MoF requested (as of February 22, 2019) the second restructuring to effectively advance the closing date of the loan to Page 13 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) March 31, 2019 and cancel the undisbursed amounts of the loan (US$23.76 million). This restructuring was completed on March 25, 2019. II. OUTCOME A. RELEVANCE OF PDOs Assessment of Relevance of PDOs and Rating 23. Relevance of objectives - High. The relevance of objectives is rated High. The project objectives were supporting two of the four strategic themes of the CPS for 2012–2016 9. Specifically, the project’s improved forest fire management was designed to particularly contribute to sustainable development and effective protection of the environment under the first strategic theme - Increasing Growth and Diversification - by improving sustainable management of forest resources. The project would also strengthen Russia’s contribution to global public goods, thereby supporting the third CPS strategic theme - Deepening Russia’s Global and Regional Role – by reducing the carbon emissions. The objectives are also highly relevant to the current Federal Program on Forests 2013-–2020, which aims – among other priorities – to reduce the damage caused by forest fires and increase effectiveness of forest fires prevention and suppression. 24. Over the project period, the relevance of its objectives has not changed and remains High to the current World Bank Systematic Country Diagnostic (SCD) objectives 10. The operation provided clear evidence of the PDO alignment to the SCD policy priorities, including to improve environment and natural resources (forest and water) management—in the short and medium terms. The SCD identifies Russia’s forest—the second largest only to that of the Amazon—as important to the global climate change agenda in terms of the amount of CO2 it absorbs. It further states that better management of land, forests, and water would help manage climate change and disaster risks and would especially benefit poor and natural- resource-dependent populations that have more limited coping mechanisms. The SCD provides direct reference to Russian forests storing one-third of the forest carbon balance worldwide and to the forest fire threats, pest infestations, and human activities that severely impair forest effectiveness as a carbon sink. B. ACHIEVEMENT OF PDOs (EFFICACY) Assessment of Achievement of Each Objective/Outcome 25. Achievement of PDO (efficacy) - Negligible. The PDO was complex and multipronged and envisaged two major outcomes: (a) improve forest fire prevention and suppression in selected forest ecosystems, including targeted protected areas, and (b) enhance forest management in pilot regions. 9 This is the latest CPF followed by SCD. 10 Report 110765-RU Page 14 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Outcome 1. Improve forest fire prevention and suppression in select forest ecosystems, including targeted protected areas 26. The project has partly achieved the outcome of improving fire prevention and suppression. This was designed to be achieved through a combination of softer activities such as sectoral studies, trainings, planning, and hard investments in the project regions’ firefighting capacity. The results are mixed because the project mostly procured the necessary equipment but did not fully and effectively strengthen the anticipated management systems within the closing date. 27. The project focused on preventive measures such as outreach and communications to the public about the danger of forest fires, as well as on suppression of fires through early fire detection and prompt response, upgrading of forest fire machinery and equipment, upgrading of telecommunication systems, and support to regional cross-deployment of the resources and forest fire suppression capacity. Pests infestation can further exacerbate the forest fire danger by forming large areas of dead trees or stands susceptible to forest fires. Thus, the project also invested in pest monitoring systems in pilot regions that would allow better modelling and preventing the infestations and reducing the fuel load for forest fires. 28. Containing the forest fires within 24 hours following their detection is important for their further spreading and going out of control. At the same time, the faster the detection happens, the smaller the area of the fire and the higher the chances of its containment. The project invested in firefighting, communication, and monitoring equipment. An integrated set of heavy machinery, equipment, tools, and protective gear, scientific equipment and tools were procured and delivered to the regions. Communication systems for increased mobility and early suppression of forest fires were delivered to Komi (188 units), Khabarovsk (515 units), Moscow (1,137 units), and Voronezh (306 units) regions. Integrated information and telecommunication vehicle-mounted mobile systems were delivered to Krasnoyarsk (3 vehicles), Khabarovsk (2 vehicles), Moscow (1 vehicle), and Voronezh (1 vehicle) regions. Early fire detection systems were procured and tested: video camera computer systems installed for monitoring forest fires in Krasnoyarsk (130 units), Khabarovsk (30 units), and Voronezh (20 units) regions. The PAs (13) received firefighting, communication, and monitoring equipment. These machinery and equipment helped to better and faster detect and contain the forest fires, thus reducing their area at detection and increasing the chances of fire containment within 24 hours following detection. 29. The project invested in developing a comprehensive and robust approach to addressing the massive scale of forest fires, defining objectives, tasks, and monitoring and coordination mechanisms in the regions and at the interregional level. Public awareness programs and online information portals were introduced, contributing to higher informational and management transparency in the forest sector. The project also conducted extended stakeholder consultations and developed a model indigenous people engagement plan. However, planned robust sociological surveys of the population and visitors of the PAs to identify the level of fire awareness were not implemented and the indicator’s target was not achieved. 30. As highlighted earlier, the project also invested in preventive measures to contain pest infestation. Equipment procured and delivered for these purposes included integrated mobile labs, equipment for seed quality testing, forest pathology monitoring, and pest and soil analysis provided to Voronezh region. Outcome 2. Enhance forest management in pilot regions Page 15 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) 31. The project planned to increase forest and PA management capacity and helped address key policy and management issues. That was necessary (a) to overcome perverse incentives that drive both illegal logging and the willful and accidental setting of forest fires, (b) to modify silvicultural practices so as to reduce the fuel load and other factors contributing to increasing the extent and intensity of fires once they are ignited, (c) to improve forest regeneration, and (d) to improve the efficiency of forest management. The project intended to invest in: comprehensive policy, institutional, and economic analyses for identifying and addressing key issues; improved forest regeneration and restoration; development of an integrated forest management information system (FMIS) and model forests; and training of forest administration and management staff. 32. The project did not achieve the outcome of enhancing forest management in pilot regions. While providing necessary technical and analytical support at the early stages of implementation, the project neither delivered the planned policy reforms nor the development and adoption of enhanced management plans and silvicultural practices. It also was not effective in strengthening institutions for improved management and control of forest fires. 33. The following service-providing contracts were not fully implemented, which negatively affected the delivery of the critical policy, institutional, and economic analyses: • Development of a National Forest Fire Management Strategy for Federal PAs • Economic Analysis of the Part B Project Investments • Upgrade of the System for Definition of Fire Safety Stages • Implementation of the New System to Measure Risk of Forest Fires in Selected PAs • Development of Projects Firefighting Equipment in the GIS Environment with a Focus on Biodiversity Conservation in Selected PAs 34. Mixed results were achieved in developing the FMIS. A concept of an Integrated Forest Fire Management Information System for PAs was created and tested. However, the MNRE was not satisfied with the final outputs. Some equipment for improved afforestation quality was procured and delivered, including mobile labs for afforestation quality control provided to Leningrad, Volgograd, Voronezh, Krasnoyarsk, Tverskoy, and Tomsk oblasts. Justification of Overall Efficacy Rating 35. Inadequate monitoring and poor reporting of the project results significantly compromised the analysis of the efficacy. The task team reported incomplete results in the last Implementation Status and Results Report (ISR) referring to the missing reports from the FFA, MNRE, and the PIU. Analysis of ISRs and Aide Memoires (AMs) and interviews conducted during the Implementation Completion and Results Report (ICR) Mission (June 3–7, 2019) indicated that the activities’ implementation and the outputs delivery were uneven across the outcomes and implementing agencies. While it may be the case that the outcomes are more consistent and advanced, it was difficult to establish given inadequate and non- systematic monitoring and reporting. Page 16 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) 36. The overall Negligible rating for efficacy was derived based on the officially reported results. The reason for the rating is partially achieved objectives under both outcomes of the project. While the initial analysis and strategies were partially completed, they did not inform the policy and institutional reforms, as envisioned by the project and described in the PAD’s RF. Goods and services were procured and delivered partially, albeit with long delays. While outcome indicators provide evidence of slightly increased success in containing forest fires (from 74 percent to 75 percent in the Extensive Forest Fund and from 72 percent to 84 percent in the project PAs), as well as decreased area of fires at detection (from 14 to 5 ha), there is very little other evidence as to the extent to which forest fire prevention and suppression were achieved, as well as to which extent forest management practices were enhanced by the project. While there may be some improved practices, it is difficult to make a clear attribution of these results to the project inputs. C. EFFICIENCY Assessment of Efficiency and Rating 37. Rating for project efficiency. Efficiency, a measure of how economically project resources and inputs were converted into results in the forest sector, is assessed as Modest. Project efficiency was assessed based on the economic analysis and aspects of project design and implementation. There were methodological difficulties in conducting a comprehensive economic analysis due to weak data on project outputs. 38. Economic and financial analysis. The following economic and financial analysis compares the economic rate of return (ERR) and net present value (NPV) before and after the project for both Parts A and B of the project and considers the results of expanded economic analysis for the Part B investments in the 13 participating PAs. The analysis conducted at project completion updates the assumptions made at appraisal and uses the actual data of the forest fire area, the updated stumpage prices, and the actual exchange rate of Russian rubles/U.S. dollars. It also includes actual investment costs based on the final financial report of the PIU as of March 31, 2019. Additional information on the economic and financial analysis, as well as the assumptions, can be found in Annex 5. 39. Factors limiting the Environmental Assessment were encountered both on the benefits and costs parts. At project appraisal, not all benefits were included in the modeling due to limited data on the monetary values; climate factors (rise of temperatures, precipitation levels, and so on); and the dynamics of reduction of damage from the forest fires during the longer periods as compared with and without project interventions. Therefore, the initial analysis assessed two groups of benefits: reduced timber losses (direct benefit) and reduced amounts of carbon entering the atmosphere as a result of fires (indirect benefit). Further analysis at the MTR included an expanded list of direct and indirect benefits but only for 12 PAs of Part B (not 25 PAs) and did not cover Part A pilot regions of the Extensive Forest Fund (see annex 5 for more details). Limitations to the cost analysis related to the fact that a substantial number of planned investments were not implemented and the operations and maintenance costs for Part A activities were difficult to evaluate. For the analysis, it was assumed that all supplied equipment should be in use by project beneficiaries. 40. The following assumptions were adopted in the analysis: (a) the project generated both local and global benefits, for example, the reduced timber loss for local benefits and reduced carbon emissions for the global ones; (b) project costs were updated based on the actual and final versions of the project’s Page 17 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Procurement Plan; (c) supplied equipment is expected to be in use by project beneficiaries in pilot regions and in 13 PAs for reduction of forest fires; (d) local benefits start only from the year when the equipment was supplied; (e) stumpage price increased in 2019 compared to the appraisal stage and is RUB 98.9 per m3, or US$1.58 per m3; (f) project interventions will reduce the burnt area by 12 percent in the pilot regions and by 2 percent nationally due to reduction of scope of project activities and early project termination. The analyzed benefits and costs of the project helped achieve two PDO indicators: PDO Indicator #1: Increase in percentage of fires contained within 24 hours following detection in pilot extensive forest regions and project PAs and PDO Indicator #2: Average area of fires at detection in project PAs. Due to poor reporting, the fact that fire brigades were equipped with new gear and new firefighting tools was not reflected in the RF. 41. Based on these assumptions and with the limiting factors applied for the project area, Case 1 (both global and local benefits) produces an economic internal rate of return (EIRR) of 42 percent and the NPV of more than US$150 million at project completion, as shown in table 5. Case 2 (local benefits only) produces more modest results: an EIRR of 23 percent and NPV US$21 million. Table 5. Summary of Economic Analysis Results at Appraisal, Midterm, and Completion of the Project Ex Ante Economic Analysis Midterm Economic Analysis Ex Post Economic analysis • Investment costs of • Investment costs of Part B • Investment costs of Part A and B Subcomponent 1A • Discount rate 12% • Discount rate of 12% • Discount rate 12% • Include 10 expandeda types of • Includes both local (reduced • Includes both local local and global benefits (for timber loss) and global benefits (reduced timber loss) and example, reduced carbon (reduced carbon emissions) global benefits (reduced emissions, damage to carbon emissions) biodiversity, and landscape restoration costs of PAs) • EIRR is 23% • EIRR is 36% Case 1 (both global and local • NPV of more than US$50.5 • NPV is US$9.7 million benefits): million • EIRR is 42% • NPV of more than US$150 million Case 2 (local benefits only): • EIRR is 23% NPV is US$21 million Note: a. Reduced timber losses, health and injury related losses due to fires, damage related to the reduced amount of Not-Timber Forest Products (NTFPs), costs of damaged infrastructure in PAs, and costs of firefighting measures. 42. Greenhouse gas (GHG) emissions for the completion stage calculated using the Ex-Ante Carbon Balance Tool (EX-ACT) model demonstrated that the project activities would generate a net sink of approximately 99 million tons of CO2eq over 20 years, with approximately −1 ton of CO2eq per ha over 20 years (with carbon balance of −2,879,504 tons of CO2eq). This amount corresponds to losses of biomass (81.6 Mt CO2) and soil organic matter (16.4 Mt CO2eq) compensated by the avoided N2O and CH4 emissions because of prevented burning (−79 Mt CO2eq and −119 Mt CO2eq). The proposed improvement of reduced burning or fires would compensate for the deforestation activities related to cutting the firebreaks in the forest. At a social carbon price of US$10 per ton of CO2eq, the NPV results in US$127 Page 18 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) million and IRR above 33 percent with a discount rate of 10 percent on 20 years (2014–2033). 11 At appraisal, the EX-ACT modeling demonstrated that the net carbon balance would be a reduction in emissions in excess of 215 million tons of CO2eq (or a 31 percent reduction in emissions over the baseline) over a 25-year period (see annex 5a). 43. Distribution of benefits. Reduction of fire risks helped to better protect local communities and forest users and to reduce future damage and losses. However, planned education and training activities for project beneficiaries were not implemented, which decreased the project’s social benefits. It could have helped to improve outreach of forest authorities to the public, increase public awareness of the ecological and economic values of forests, and strengthen the sense of forest ownership of population living nearby. The policy, legal frameworks, and institutional capacity-building activities were also not implemented as planned, missing the opportunity to strengthen on-the-ground law enforcement, clarifying the rights and responsibilities of forest inspectors and rangers, as well as forest users. 44. Environmental benefits stem from activities and investments in the protection of forests and their future reforestation. The project supplied equipment (mobile pest monitoring laboratory) to the FFA’s regional office of the Department of Forest Protection in Krasnoyarsk region which is critical for helping monitor and model forest pest outbreaks. Specifically, the region’s forests experienced devastating outbreaks of Siberian Moth (Dendrolimus superans sibiricus) affecting more than 300,000 ha of forest since 2017. Project-supplied DNA 12-monitoring equipment strengthened the capacity of Krasnoyarsk Department of Forest Projection to monitor and identify other different pathogens (insects, fungi, and bacteria) affecting the forest health in a longer term. The seed germination testing equipment provided is used for testing the seeds collected for use in reforestation of damaged areas and results in considerable savings and improvement in forest nurseries performance. Equipment supplied to St. Petersburg and Voronezh scientific forestry organizations is in use and is essential for the long-term protection of forest ecosystem services. 45. Aspects of project design and implementation. Aspects that reduced the project efficiency include leadership challenges at the FFA; ineffective process of developing procurement packages, delayed procurement, and delivery of goods and services; and six significant contractual disputes that were not resolved and resulted in court cases. 46. Leadership challenges and resulting delays adversely affected the project, but mostly its Part A, which was implemented by the FFA. At the project preparation stage, two similar projects prepared by the FFA and MNRE were put together into one following the guidance of the MoF, with two parts: Part A implemented by the FFA and Part B implemented by the MNRE. Following massive number of forest fires (some 30,000) and area burnt (more than a million hectares) during the summer of 2010, the GoRF reorganized the FFA and subordinated it directly to the Prime Minister and the Government, with policy and regulation functions over the Extensive Forest Fund. After the project negotiations in 2012, the FFA was subordinated to the MNRE, which further complicated the project implementation. Based on the ICR interviews, AMs, and views of project stakeholders, the perception of insufficient FFA leadership of the 11 With a price range of US$3–US$6 per ton of CO2eq (Hamrick, Kelley, and Melissa Gallant. 2018. Voluntary Carbon Markets. Outlooks and Trends: January to March 2018) the NPV results in US$41.03 million and IRR above 17 percent. 12 Deoxyribonucleic Acid Page 19 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) project is common for the period beginning from October 2014 and until project termination in March 2019. Given FFA subordination to the MNRE, this perception of the need for more proactive resolution of project challenges extends to the ministry as well. At least two interviewees stated about disruption of the procurement processes. Further, the project oversight Interagency Committee 13 (which was chaired by the head of the FFA) did not resolve these constraints. 47. Procurement delays affected the efficiency. The process of putting together procurement packages for required goods (machinery, tools, communication equipment, and their specifications) between the PIU, the implementing agencies, and five regions became very lengthy, cumbersome, and resulted in significant procurement delays. Further, the new FFA leadership significantly delayed authorization of contracts, as stated in the several AMs. By December 2015, 30 contracts were outstanding for a total value of approximately US$22 million, of which US$11 million was for contracts already awarded and ready for signing (some for as long as 11 months). With delays in procurement, authorization of contracts and their payments, six court cases were initiated, which further disrupted delivery of results. 48. Considering the economic and financial analysis, the aspects of project design and implementation stated above, as well as poor results reporting, the efficiency rating is assessed as Modest. D. JUSTIFICATION OF OVERALL OUTCOME RATING 49. The overall outcome rating is Unsatisfactory. The rating was determined based on the High relevance of the project objectives, Negligible efficacy (achievement of the project objectives), and Modest efficiency. The PDO was partially achieved, with delayed and partial procurement of the required goods and services. While the outcome indicators’ targets were partially achieved, the monitoring and reporting remained inadequate and non-systematic throughout the project cycle. It is difficult to fully establish a causal relationship between the project inputs and outputs in many cases. However, the relevance of project objectives remains high in the current conditions when danger of forest fires today is not less but perhaps even bigger in Russia. During summer 2019, Irkutsk and Krasnoyarsk regions experienced high number and intensity of large forest fires in an area of more than 3 million ha. Smoke disrupted public transportation and created highly hazardous conditions in the large part of the region. The project outcomes are consistent with the GoRF development priorities and international commitments on forests, biodiversity, and climate change, as well as World Bank’s SCD 2016. The project restructuring did not result in improved project performance due to staffing changes and administrative delays in implementing agencies; however, it further deteriorated implementation performance leading to the early project termination. The project did not effectively use and disburse 11 percent of the originally allocated amount. Despite the fact that the project was restructured, reducing some of the PDO indicator targets while also increasing the number of the PAs, these factors netted out.A split evaluation is not required as it would have no impact on the overall outcome rating. 13As stated in the PAD “established to support interagency coordination, monitor project progress on a quarterly basis, as well as settle controversies that might arise during project implementation. The Committee will be chaired by the Head of FFA, and will consist of the Deputy Head of FFA, as well as representatives from MNRE, Ministry of Economic Development (MOED), and MINFIN.” Page 20 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) E. OTHER OUTCOMES AND IMPACTS (IF ANY) Gender 50. There were no specific activities envisaged under the project related to gender. Institutional Strengthening 51. The project had a substantial institutional strengthening and development component. It was designed to analyze the policy, institutional, and economic factors underlying underperforming forest fire and forest management system and enable legislative and institutional reforms, including for better interregional collaboration for longer-term impacts. The project successfully held multiple consultations and prepared comprehensive and robust plans and strategies with active participation of Krasnoyarsk, Khabarovsk, Voronezh, and other regions. However, the project was not successful in developing and adopting new legislation and introducing planned institutional changes. Mobilizing Private Sector Financing 52. The project did not envision engagement or mobilizing of private sector financing at the time of approval, nor did such mobilization happen during the implementation. Poverty Reduction and Shared Prosperity 53. The project areas are located predominantly in rural areas with elevated levels of poverty where significant proportion of local people depend on natural resources for supporting their livelihoods. The project may have indirectly contributed to poverty reduction of its stakeholders in project regions and beyond by means of preventing forest fires, better coordinating and managing forest resources, providing technical training, and preparing indigenous people safeguards tools in Khabarovsk region. However, the scale of such contribution was not evaluated. Other Unintended Outcomes and Impacts 54. The project generated several unintended positive outcomes. For example, an increased cooperation and knowledge exchange between PAs and Extensive Forest Fund management in Krasnoyarsk region was recognized by stakeholders, as well as the project’s role in catalyzing and prioritizing the strategic interventions not necessarily supported by the project investments but funded directly by the Government thereafter. III. KEY FACTORS THAT AFFECTED IMPLEMENTATION AND OUTCOME A. KEY FACTORS DURING PREPARATION 55. The project was a continuation of the analytical, policy, and institutional work that the World Bank did with the GoRF in forest sector. Building on the successful relationships in the sector during previous engagements, it was reasonable to expect effective implementation of this investment project. Page 21 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) 56. Overall, the project was prepared with a robust design in a short preparation period. As indicated in early AMs, the project was going to be on emergency track in response to the massive forest fires in the summer of 2010. Brief assessments of the borrower and implementing agencies capacity were conducted and lessons learned from the past World Bank operations were reviewed and incorporated. Project indicators were carefully considered given the complex nature of the issues and project design and allowed capturing the project benefits, outcomes, and achievement of the PDO to the extent possible at the time of project preparation. At a later stage, project indicators required adjustment. Objectives and Design 57. The PDO and project design supported the Government’s priorities and urgent needs to quickly respond to the forest fires. However, the PDO is complex and multipronged. Project design was consistent with the strategic focus areas of the Country Partnership Strategy (CPS) for 2012–2016 and supported two of its four strategic themes: Increasing Russia’s Growth and Diversification, and Deepening Russia’s Global and Regional Role. Project financing (IBRD, US$40 million) was supplemented with the GoRF’s own resources (US$81.3 million) to enhance the catalytic effect of the designed activities and confirming the Government’s support and ownership of the project design. 58. The project was prepared with specific attention to the lessons learned and technical aspects of the proposed activities. An international consultant was hired to advise on forest fire management activities, analyses were conducted, and multiple consultations were held in Moscow and other regions. A comprehensive model of the triple-pronged approach was adopted to conceptualize the necessary interventions: preventive measures, early detection, and effective suppression of forest fires. Outcomes of the stakeholder consultations were incorporated as well. Public consultations were held, the Environmental and Social Management Framework and safeguards tools were prepared, including the Indigenous Peoples Process Framework, and were publicly disclosed on November 16, 2011. The safeguards documents were satisfactorily prepared as verified by the World Bank’s environmental and social specialists, as well as the implementing agency’s safeguards specialist. 59. The project was designed with two main components and project coordination and management activities. Five pilot regions and 12 PAs were selected based on their representation and substantial contribution to the country’s annual dynamic of forest fires. Implementation arrangements included a steering mechanism and an Interagency Committee (FFA, MNRE, Ministry of Economic Development, and MoF) to ensure the effective implementation and resolution of issues that could be beyond the control of line agencies. Part A of the project would be implemented by the FFA in the Extensive Forest Fund, while Part B of the project would be implemented by the MNRE in select PAs, with the similar set of activities under Components 1 and 2. Transactional operations of financial management (FM) and procurement with the regions were delegated to the PIU under the tripartite agreements with the FFA and MoF and with the MNRE and MoF. Results Framework and Monitoring 60. Organizational staffing, monitoring, and reporting processes were designed to reflect the complex nature of planned activities for addressing forest fires. Organizational processes were further strengthened with workshops and trainings, including regional staff. The RF included a systematic and methodological breakdown of the underlying triple-prong approach, with outcome indicators and Page 22 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) intermediate results indicators adequately albeit not realistically measuring the PDO. On one hand, the RF included several indicators disaggregated by project parts (A and B). The project had an appropriate plan for monitoring, with monitoring and evaluation (M&E) staff being established at provincial level, as well as at the national level. The M&E staff conducted their work based on accepted methodologies that included establishing a baseline, obtaining data from local institutions after having provided instructions on data collection standards. On the other hand, the outcome indicators as well as their targets had to be adjusted and supplemented with the new ones during the restructuring and by completion, many remained unreported and targets not achieved. Selection of Beneficiary Groups 61. Borrower and responsible agencies—MoF, FFA, and MNRE—were evaluated as competent institutions with qualified technical and financial staff. The responsible agencies, the project oversight Interagency Committee, and the borrower were defined with specific roles for project steering and implementation. By supporting the GoRF in its forest fire prevention and management efforts, the World Bank continued a long-standing and comprehensive engagement with Russia on forest management. These efforts have centered on policy development, institutional strengthening, information and land-use planning systems, and strengthening of regional forest inventory and pest protection organizations. This engagement included analytical and advisory activities, as well as the EU-supported FLEG program. The project was set to complement work carried out under the U.S. Agency for International Development (USAID) FOREST Project, which supported sustainable forest management in Siberia and the Russian Far East by improving fire prevention, monitoring forest pests, supporting value-added processing of forest products, and introducing biomass energy. Readiness for Implementation 62. While the project has undertaken appropriate measures to ensure readiness for implementation, the project experienced several constraints that negatively affected the readiness. Project effectiveness was delayed by more than a year. Technologies proposed for the project were international best practice and well-proven. Implementation arrangements followed lessons from and practice of ongoing projects, considering the different levels of capacities of project management, from national and provincial level, down to forestry enterprise (leskhoz) and PA level. Further, project design was in line with relevant sector strategies and regional priorities, having had multiple field visits and consultations with the stakeholders. Yet, despite these mitigation efforts designed during the preparation, some of the readiness issues materialized later during the implementation, for example, difficulty in coordination, resulting in the delays and modest outputs. Adequacy of Risk Assessment 63. The identification of risks to the project and its rating was adequate at the PAD stage (Stakeholder: High, Capacity: Moderate, Governance: Moderate, Design: Substantial, Social and Environment: Low, and Monitoring and Sustainability: Substantial). The two main risks proved to be true and the project did not succeed in their full mitigation despite the designed measures. The first risk identified during the appraisal was that the operation involved numerous institutions (the FFA, MNRE, and the pilot regions and PAs) that were not accustomed to working in unison to prevent and respond to forest fires, hindering effective suppression. Mitigation measures of this risk included developing a protocol to support agency Page 23 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) coordination, as well as physical coordination centers (IFFCs) to ensure coordinated forest fire response and provide additional resources to surrounding areas. This measure did not work out. The second risk was in the planned reforms of the policy and institutional framework for forest and forest fire management. Establishing this framework did not work out and it led to continuing perverse incentives in the forest sector leading to unlawful activities, including the willful setting up of fires. One of the decisions taken at the emergency meeting with the Prime Minister of the GoRF discussing the measures to suppress forest fires at the end of July 2019 was measures to address perverse incentives for willful setting of fires. B. KEY FACTORS DURING IMPLEMENTATION 64. Two main factors negatively affected implementation of the project: (a) institutional changes, including FFA subordination to the MNRE after the completion of project negotiations and subsequent change in the FFA leadership that resulted in less direct support to the project implementation, and (b) ineffective procurement process that resulted in six court disputes. Adequacy of Government Commitment 65. Based on the analysis of forest management status, in particular, Government’s limited firefighting capacity, as well as the request of the GoRF in response to the massive forest fires during 2010, it is clear that the project was timely and critically needed to boost the country’s capacity in managing forest fires and their consequences to the economy and public health and safety. Inception missions to Krasnoyarsk, Khabarovsk, and other regions highlighted a dire need in changing the approaches to managing forest fires as well as in boosting technical and human capacity for fire prevention and emergency operations on the ground. The MoF, as the borrower, following the recommendations of the Prime Minister’s office, supplemented the IBRD financing with the GoRF in the proportion 1:2. It indicates the trust to the project design and ownership of the project interventions by the GoRF at the high level in the early stages. Mid-Term Review and Restructuring 66. Nevertheless, by the time of MTR, the task team confirmed that the implementation is delayed and practically stalled, as indicated in the MTR ISR (2015) and conducted interviews with the project staff. Despite several management letters and multiple meetings, it became obvious that the FFA leadership was not supportive of the implementation under Part A. First, the implementation was stalled by requesting to change the project regions, then by seeking changes to the specifications of machinery and equipment, and then by not signing contracts already awarded by the Evaluation Committees, in some cases for more than a year. The MTR AM highlighted that the PDO achievement was at risk due to the slow implementation and limited time left to carry out project activities, most of the activities were behind schedule, and the project was performing poorly mostly due to the implementation delays in Part A. 67. The first restructuring was done in December 2016. At the time, the project disbursed only US$7.2 million or 18 percent of the IBRD total amount of US$40 million. Following downgrade in project performance ratings and discussions with the MoF and MNRE, the MoF requested that the project be restructured so that the activities implemented by the FFA are restricted to the already signed and/or awarded contracts, and that Part B (implemented by the MNRE) is increased to expand its scope and coverage from 12 PAs to 25 PAs. The PDO remained unchanged. Although the loan amount remained Page 24 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) unchanged (US$40.00 million), the counterpart funding was partly reallocated and partly cancelled. The total project funding decreased from US$121.3 million to US$81.3 million. Part A funding dropped from US$97.8 million to US$27.8 million and Part B increased from US$20.0 million to US$48.5 million. For Component 3, project management for Parts A and B increased from US$3.5 million to US$5.0 million. The RF was changed accordingly to reflect the reduced investments in the pilot regions and increased number of participating PAs from 12 to 25. Some indicators were modified, two were dropped altogether, and for others, both the baseline and targets were added. The reductions in Part A were from the reduction in/dropping of planned activities; the geographic coverage remained the same. Factors Subject to the Control of Government and/or Implementing Agencies 68. Majority of the information sources 14 used to prepare this report indicate that factors under the control of the GoRF were largely responsible for the implementation slowdown, its ultimate halt, and the project’s early termination. Coordination and engagement of the multiple agencies and the regions did not work out as it was envisioned. Leadership of the FFA after the change of the agency’s senior management in 2015 did not follow up on the previous commitments which led to stalling of the project implementation. Human resources and organizational capacity were adequate and dedicated, especially at the regional level, coping with initial project delays, lengthy procurement processes, and delays in the delivery of goods and services. Legislation and regulations guiding the project implementation required revisions to fully enable engagement and implementation at the regional level and entities. Despite clear responsibilities and enhanced governance arrangements to mitigate the complex structure of the project and associated risks, and the lack of continued political commitment derailed the procurement process, ultimately halting and terminating the project. While the fiduciary, environmental, and social arrangements were adequate and performed well, the M&E did not sufficiently inform the project implementation and proved to be inadequate. Factors Subject to the Control of the World Bank 69. The task team used US$0.5 million for preparation and US$0.8 million for supervision of this project, total of US$1.25 million, including travel and consultant costs. Overall, the supervision was adequate with the regular and timely (every six months or so) missions, including the field visits in the regions. High-quality ISRs and AMs were regularly prepared, translated into Russian, and shared with the GoRF and World Bank management without delays. Issues, especially those beyond the task team reach, were raised continuously with the Country Director and effectively communicated to the implementing agencies and the MoF. IV. BANK PERFORMANCE, COMPLIANCE ISSUES, AND RISK TO DEVELOPMENT OUTCOME A. QUALITY OF MONITORING AND EVALUATION (M&E) M&E Design 14 ISRs, AMs, and interviews conducted with project staff by the ICR team in June 2019. Page 25 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) 70. The operation’s proposed theory of change is moderately complex but clear. By enhancing forest fire prevention and control and by building forestry and PAs management capacity, the project would contribute to a more sustainable use and protection of forest resources, resulting in significantly reduced forest fires, air pollution, and pest infestations and increased carbon sequestration. Yet, the originally designed RF relied heavily on the operation itself to estimate the baselines, and, as it turned out, had to be adjusted—hence not appropriate to adequately measure the PDO. Part of the baseline data was supposed to be collected through the analysis and studies procured in the initial stages of project implementation, which happened only under Part B in the PAs. Project and subproject-level data were supposed to be aggregated on semiannual and annual bases, and reported by the PIU to the World Bank and Government counterparts, including the borrower. M&E Implementation 71. The project implementation showed constraints of the initially designed M&E framework, mostly due to the organizational weaknesses of data availability and collection. Slippages in preparation and delivery of the procurement packages were later identified during the project supervision missions starting from 2015. Shortcomings of the M&E framework and inadequate quality of data contributed to the decision to restructure the operation. By the MTR, the task team also realized the need to adjust the indicators to capture the changes to the project design, which was subsequently agreed with the borrower and implementing agencies in 2016. The changes were envisioned to invigorate the weak monitoring and reporting and to fully capture the required measurement of the performance of the project and of the achievement of the PDO. 72. Despite the efforts of the task team and continuous commitments on the Government side, the situation with M&E process of data collection and its processing and reporting had not improved after the MTR and the first restructuring. The World Bank team put pressure on the borrower to resolve the challenges and provide the necessary support to bring the procurement and M&E issues in line with the schedule and design. The task team together with the implementing agency teams had prepared necessary safeguards instruments; however, because none of the ground-level project activities were initiated (only procured goods and services), environmental and social impacts were not actually monitored. The World Bank’s safeguard specialists had been closely monitoring the implementation and participated in the missions and the field visits consistently. M&E Utilization 73. Progress indicators were consistently used to identify and keep track of issues in project implementation. The task team also heavily relied on own communications with the PIU and the implementing agencies to better understand the ongoing challenges during the periods when M&E data became unavailable. The M&E framework did not improve the reporting and was not used effectively after the MTR and restructuring. Despite the regular and consistent reminders, bulk of the indicators’ targets remained unreported in the last ISR. Reports from contractors and consultants were used to track progress in the field, but regular, consistent, and fully completed progress reports were never reported. The M&E data was underutilized and failed to inform the proper project performance and several important project activities, including those from policy and institutional activities. M&E has been consistently rated Unsatisfactory or Moderately Unsatisfactory in the ISRs because of poor reporting, limited utilization, and significant gaps in reporting of the RF. Page 26 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Justification of Overall Rating of Quality of M&E 74. Overall rating of quality of M&E is Negligible. The shortcomings of the M&E system were severe and, as implemented, the system was not sufficient to access the achievement of the stated objectives.. The use and impact of the M&E system was limited due to inadequate and non-systematic reporting of the project outputs. B. ENVIRONMENTAL, SOCIAL, AND FIDUCIARY COMPLIANCE Environmental and Social Compliance 75. The project was categorized as B for Environmental Category and has no pending safeguard issues. There have been no changes to the project safeguards category and environmental or social safeguards issues because of two project restructurings. The project triggered the following safeguard policies: Environmental Assessment (OP/BP 4.01), Natural Habitats (OP/BP 4.04), Forests (OP/BP 4.36), Pest Management (OP/BP 4.09), and Indigenous Peoples (OP/BP 4.10). As the project planned to support several small-scale construction activities in five pilot regions that would involve civil works, the PIU prepared an Environmental Management Plan Framework to guide Environmental and Social Impact Assessment of potential subprojects satisfactory to the World Bank. PIU publicly consulted and disclosed safeguard documents before the appraisal in 2011. However, only technical assistance and purchasing of new equipment was financed during the project implementation and no physical works on the ground were conducted. Furthermore, supported technical assistance activities also did not have any impacts on the environment as they have not supported preparation of policy documents or development of forestry business plans that would later generate environmental or social impacts. The social screening was conducted to assess potential impacts, including the indigenous people in 12 PAs and 5 project regions. As a result, anticipated social impacts on Indigenous People in Khabarovsk Krai was prepared to ensure that the project would maximize the benefits and mitigate potential risks in those communities. Project grievance redress mechanism (GRM) was in place, which was accessible to the project-affected people. There was no evidence of receiving or processing complaints. Overall, the project complied with all triggered safeguards policies. Financial Management 76. FM procedures were reviewed periodically as part of the supervision missions. The World Bank considered the FM arrangements satisfactory consistently throughout the implementation period as recorded in the ISRs and AMs. The FM system of the PIU was adequately staffed and managed, including accounting, internal controls, and reporting. The World Bank consistently rated the FM as satisfactory . Quarterly unaudited interim financial reports had been submitted on time and were acceptable to the World Bank. Procurement 77. Procurement under the project was implemented in accordance with the World Bank’s Procurement and Consultant Guidelines and in accordance with the provisions of the Loan Agreement. The borrower’s teams received trainings on procurement at the early stages of project implementation. The MTR recorded significant delays related to the approval of bidding documents and delays in signing Page 27 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) of already awarded contracts (up to 20 percent of project financing cumulatively for more than 12 months). The task team raised several related issues such as mixed commitment to the project goals, insufficient understanding of World Bank procurement rules, limited communications between the technical (implementing agencies) and administrative (PIU) staff, multiple layers of reviews and unclear responsibilities, and inadequate attention to project management. The borrower was informed about the serious violations of the World Bank Procurement Guidelines and possibility of declaring mis- procurement. Later stages of project implementation revealed continuous procurement issues (also see more on procurement delays in the previous paragraphs). 78. ICR consultations with the stakeholders highlighted several issues with procurement process and delays associated with it. Partially, it can be explained by the implementation arrangements, when the PIU operated under the contracts with both implementing agencies and could not act without them timely providing technical specifications and approving bidding documents, as well as confirming the acceptance of delivered goods and services. Also, it seemed that the procurement processes were stalled following the change in FFA leadership in 2015, which was likely the main reason behind the procurement delays. C. BANK PERFORMANCE Quality at Entry 79. The project was prepared within an average time frame, with 18 months from the Concept Review (March 2011) to Board Approval (September 2012) and with adequate budget costs. The project was strategically relevant, addressing significant capacity gaps and designed based on substantial technical, ecological, social, economic, and financial analyses, with adequate beneficiary/stakeholder consultations and their participation. The project invoked a comprehensive approach: a combination of forest fire prevention, detection, and suppression activities in the Extensive Forest Fund and PAs for maximum effectiveness and efficiency in addressing short- and medium-term threats of forest fires in project areas. The World Bank mobilized a team with all the necessary skills, including specialists in forestry, forest fires, social and regional sectors, natural resource economists, FM, procurement, legal, and special advisers. The team designed a mechanism for steering the complex project—Interagency Committee— however, the committee had limited impact in resolving leadership and organizational challenges. An FM action plan was prepared by the World Bank team to increase the FM capacity and to create adequate arrangements for internal and external controls. The World Bank environmental and social specialists identified limited potential impacts and verified adequate arrangements for mitigating and monitoring them. Extended public consultations were held during project preparation. The technical, environmental, and social risk assessments were thorough and appropriate; however, not all identified risk mitigation measures proved to be effective, especially in dealing with institutional and stakeholder risks. There were some shortcomings in the M&E design as discussed earlier. Quality of Supervision 80. World Bank supervision missions with the task team of a qualified skills mix were carried out on a regular basis over the 6.5 years of project implementation. Delays in the implementation schedule and the need to enhance the procurement and reporting processes, as well as the contracts and other project documentation approvals were continuous themes of the supervision mission’s AMs. The MTR mission and the subsequent restructuring unfortunately did not make a desired impact for the achievement of Page 28 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) the PDO. The World Bank strived to maintain a collaborative relationship with the borrower and implementing agencies, including the clients in the regions, with a range of high-level and technical meetings and field visits to Khabarovsk, Krasnoyarsk, Voronezh, and other regions. Adherence to the safeguards instruments was monitored regularly and confirmed by the World Bank safeguards specialists. Procurement issues (lapses in approvals and payments and insufficient coordination between administrative and technical staff and others) remained unaddressed after the restructuring despite consistent, constructive, and rigorous supervision support provided by the World Bank. Candor and quality of performance reporting remained an issue and required constant attention and continuous support from the task team. Justification of Overall Rating of Bank Performance 81. Rating: Moderately Satisfactory. The overall World Bank performance is rated Moderately Satisfactory considering shortcomings at the preparation stage. World Bank supervision was of high quality but due to the moderate shortcomings of quality at entry, the overall rating is Moderately Satisfactory. D. RISK TO DEVELOPMENT OUTCOME 82. The overall risk to development outcome is High for the following reasons. The development outcomes of the project focused on building technical and institutional capacity; increasing resources to improve forest fire prevention, detection, and suppression; and enhancing forest management. The project experienced significant challenges in achieving the development outcomes, mostly delivering the technical capacity in the form of machinery and equipment but not enacting a sustainable system, institutions, and policy for forest fire management with clear distribution of functions and responsibilities. Hence, the likelihood of the outcomes not being sustained is high. This, as stated earlier, is due, in part to policy inconsistences and perverse incentives such as de facto encouraging deliberate burning of the forest to obtain access to salvage logging permits or to conceal illegal harvesting. The new policy, FMIS, silvicultural, and forest regeneration and management system envisioned to support the firefighting operations on the ground, enable regional cross-deployment when necessary, and help reduce forest fire loads that were not enacted. These risks were confirmed during the summer of 2019 with significant spread of forest fires in the project regions and the need for the Government to mobilize and coordinate army and emergency response forces and allocate additional financial resources to contain the spreading forest fires on large areas. V. LESSONS AND RECOMMENDATIONS 83. Lesson 1. Effective forest fire response requires a comprehensive and robust approach that enables policy and institutional support to the investments in ground-based technical capacity. Occurrence and response to human-induced forest fires depend on a combination of complex factors, including weather conditions; presence of fire loads; relationships with local communities; public awareness about the danger of forest fires; and the ability to quickly mobilize the human, technical, and financial resources over the vast, mostly uncontrolled forest areas. Under unfavorable weather conditions, a small uncontrolled fire can quickly grow and transform into life-threatening conditions for many people, businesses, and property. Despite a comprehensive design that addressed such an approach Page 29 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) for the project, its implementation got reduced to mostly investments into the technical capacity and equipment, which led to significantly limited project outcomes and continuation of the threats of catastrophic forest fires in the project regions. Therefore, such projects require a combination of policy incentives, institutional support, and modern technical capacity to be effective in prevention and suppression of forest fires at a large scale. 84. Lesson 2. Design of projects spanning over several regions and involving federal authorities require innovative and effective implementation arrangements to reduce the risks of procurement and implementation delays. Two implementing agencies, working with five regions and 25 PAs using transactional functions of a multi-project PIU, demonstrated mixed results with common issues of multiple approvals and delays. The project experienced substantial fiduciary risks and high stakeholder risks environment throughout the implementation period. The project’s Part A experienced significant mismatch in procured goods in terms of the required items and their costs while Part B smoothly procured proper goods and implemented the activities. The PIU’s resolution of these implementation issues was often delayed; the accountability of the line agencies for results was often unclear; and oversight was not effective in holding implementing agencies accountable for project performance. While reasonable at the design stage, the procurement process proved to be susceptible to external influences, was time consuming, inadequate to properly meet the requests from the regions in substantial number of cases and resulted in six lengthy court disputes. Possible options or their combination can be the following: • Smaller, regional projects with full responsibility and implementation arrangements under the regional authorities to reduce the number of clearance levels and simplify the implementation arrangements. • End-result responsibility. Design implementation arrangements with clear and direct lines of responsibilities for end results of activities, with defined levels and terms of clearances, oversight, and dispute resolution. 85. Lesson 3. Cross-regional cooperation, engagement of the private sector, and disruptive technologies should be an integral part of forest sector’s development. While federal budget allocations for forest management reached RUB 35–RUB 40 billion (US$550–US$630 million) annually, the project area regions benefitted significantly from a systematic planning approach to solving complex issues of forest fire prevention, detection, and suppression and exposure to international expertise, innovative ideas, and technologies that would be otherwise unavailable through normal budgeting processes. Large private companies focusing on timber processing could play an investment role in afforestation, sustainable management, and fire prevention of forest resources given guarantees and a favorable business environment. Small and medium enterprises could also play a role in developing effective firefighting capacity with support or subsidies from the GoRF or regional authorities. Dynamic disruptive technologies, information technology, and space exploration programs in Russia have substantial private sector segment that could play a significant role in supporting remote sensing, monitoring, and potentially informing the law enforcement forest activities such as forest fires, timber harvesting, changes in forest cover, pest infestations, and others. Forest inventory, vocational training, and capacity building are the areas of immense needs and importance for the future Russia’s forest sector requiring support from the private sector as well. Page 30 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) ANNEX 1. RESULTS FRAMEWORK AND KEY OUTPUTS A. RESULTS INDICATORS A.1 PDO Indicators Objective/Outcome: To improve forest fire prevention and suppression in select forest ecosystems Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Increase in percentage of Text N/A N/A N/A N/A fires contained within 24 hours following detection in 15-Dec-2011 15-Dec-2011 15-Dec-2016 31-Mar-2019 pilot region extensive forest and project PAs Increase in percentage of Text 73.9 84.8 76.5 75.00 fires contained within 24 hours following detection in 15-Dec-2011 12-Dec-2014 15-Dec-2016 31-Mar-2019 pilot region extensive forest Increase in percentage of Text 72 90 87 84.44 fires contained within 24 hours following detection in 15-Dec-2011 12-Dec-2014 15-Dec-2016 31-Mar-2019 project PAs Comments (achievements against targets): Page 31 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) This indicator achieved the target partially and was designed to measure a change in responsiveness and preparedness of fire-fighting squads to promptly contain the forest fires, both in the extensive forest and PAs parts of the project and were measured separately. [The language and target of the indicator were revised during the 1st restructuring for clarity and define targets.] The planned activities under this indicator also contributed to PDO Indicator #2 and #5. For Part A, implemented by FFA, the reported increase was from 73.9% to 75%, with the target of 76.5% - in the Extensive Forest Fund. Part A activities included: (i) rehabilitating fire stations; (ii) establishing local forest fire brigades; (iii) providing firefighting & communications equipment; (iv) developing fire preparedness & management plans; (v) upgrading fire danger rating & hazard index system; (vi) piloting early fire detection systems; (vii) establishing IFFCs in 3 Federal Districts; and (viii) supplying these centers with equipment & necessary pesticides for possible large-scale pest infestations. For Part B, implemented by MNRE, the reported increase was from 72% to 84.4%, with the target of 87% - in the project PAs. Part A activities included: (i) improving detection, response, management & coordination of fire-fighting efforts; (ii) upgrading the fire danger rating & hazard index system; (iii) restoration of forest fire stations; (iv) upgrading of communications systems; and (v) suppling suitable firefighting machinery, equipment, protective clothing, gear & hand tools. The data source includes FFA and MNRE reports, the aide memoires, and Evaluation report that was commissioned by the client. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Average area of fires at Text 14 2.2 5 4.92 detection in project PAs 17-Nov-2015 17-Nov-2015 15-Dec-2016 31-Mar-2019 Comments (achievements against targets): This indicator was overachieved. The indicator measures the size of detected forest fires and is related to responsiveness of fire-fighting squads; the faster a fire is detected, the smaller area it consumed, the more responsive the squads are. Based on the reported results, the baseline average size of detected fires 14 hectares, while the achieved result was reported at 4.92 hectares, with the target of 5.0 hectares. The data source includes MNRE reports, the aide memoires, and Evaluation report that was commissioned by the client. Page 32 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Area covered by improved Hectare(Ha) 0.00 190000000.00 10000000.00 0.00 Forest Pest Monitoring System in Pilot Regions 15-Dec-2011 15-Dec-2011 15-Dec-2016 31-Mar-2019 Comments (achievements against targets): This indicator's target was not achieved. The indicator was designed to measure the forest area (ha) that would be covered by improved pest monitoring system, in an effort to reduce the pest outbreaks and thus decrease the fire loads. The target of the indicator was revised during the 1st restructuring: the original target was reduced from 190 million hectares to 10 million hectares. The planned activities under this indicator also would have contributed to PDO Indicator #4. Part A activities included: (i) technical assistance to support the institutional framework; (ii) identifying & addressing key policy & legislative issues; (iii) undertaking economic cost benefit analysis & improving of the recording of the causes of the fires; (iv) providing targeted investments to improve forest regeneration & restoration; (v) developing model forests for sustainable forest management; and (vi) training of forest administration and management specialists. Part B activities included: (i) improving planning & institutional arrangements; (ii) developing guidelines on fire safety, preparedness planning & damage assessment; and (iii) developing an FMIS for PAs. The value for this indicator was consistently not reported and remained unreported at the project closure. The data source includes FFA reports, the aide memoires, and Evaluation report that was commissioned by the client. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Reach and acceptance of fire Text N/A N/A N/A N/A prevention messages among the PA target population in 15-Dec-2011 15-Dec-2011 15-Dec-2016 31-Mar-2019 comparison to baseline Comments (achievements against targets): Page 33 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) This indicator's target was not achieved. This indicator was designed to measure the effectiveness of fire prevention measures and awareness raising among the local population. The language and target of the indicator were revised during the 1st restructuring for clarity and define targets. The baseline for this indicator was not established, and the values were consistently not reported, remaining unreported by the project termination. The data source includes MNRE reports, the aide memoires, and Evaluation report that was commissioned by the client. Objective/Outcome: To enhance forest management in pilot regions Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Area of improved forest Number 0.00 5.00 5.00 0.00 regeneration based on modern technology in Pilot 15-Dec-2011 15-Dec-2011 15-Dec-2016 31-Mar-2019 Regions Comments (achievements against targets): This indicator's target was not achieved. The indicator was designed to measure the improvements in current practices of forest regeneration in the five pilot regions. The value for this indicator was consistently not reported and remained unreported at the time of project termination. The data source includes FFA reports, the aide memoires, and Evaluation report that was commissioned by the client. A.2 Intermediate Results Indicators Component: Enhancing Forest Fire Prevention, Management and Control Page 34 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Development of a National Yes/No N Y Y N Forest Fire Management Strategy for Federal PAs 15-Dec-2011 15-Dec-2011 15-Dec-2016 31-Mar-2019 Comments (achievements against targets): The target of this indicator was not achieved. This indicator was designed to measure development and adoption of a new sectoral policy to enhance the institutional and organizational capacity for managing fires in federal PAs. The contract FFRP/B/CQS-123 was not procured and the policy was not prepared. The issue with the contract was consistently raised by the TT in ISRs. The data source includes MNRE reports, the aide memoires, and Evaluation report that was commissioned by the client. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Proportion of large fires out Percentage 8.00 7.10 7.10 0.00 of the total number of fires in Pilot Regions exentsive 15-Dec-2016 15-Dec-2016 15-Dec-2016 25-Dec-2018 forest Comments (achievements against targets): The target of this indicator was not achieved. This indicator was designed to measure the proportion (percentage) of large forest fires in the five pilot regions of the Extensive Forest Fund. All of the activities mentioned under the PDO Indicator #1 apply to the Intermediate Indicators under this Component. The value of the indicator was consistently not reported, as recorded in the ISRs. The data source includes FFA reports, the aide memoires, and Evaluation report that was commissioned by the client. Page 35 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Decrease in the number of Text N/A N/A N/A N/A fires in Project PAs in comparison to comparable 15-Dec-2016 15-Dec-2016 15-Dec-2016 25-Dec-2018 PAs without project support Comments (achievements against targets): The target of this indicator was not achieved. This indicator was designed to measure a decrease in number of fires in the PAs as a result of the project interventions, in comparison with other PAs. As ISRs indicate, neither baseline nor target values were established for this indicator. The value of the indicator was consistently not reported, as recorded in the ISRs. The data source includes MNRE reports, the aide memoires, and Evaluation report that was commissioned by the client. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Updated fire danger rating Number 0.00 6.00 18.00 0.00 and hazard index system trialed in targeted Protected 15-Dec-2011 15-Dec-2011 15-Dec-2016 25-Dec-2018 Areas Comments (achievements against targets): Page 36 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) The target of this indicator was not achieved. This indicator was designed to measure capacity to monitor risk and hazard of forest fires. The value of the indicator was consistently not reported, as recorded in the ISRs, citing the contract FFRP/B/QCBS-111. The data source includes MNRE reports, the aide memoires, and Evaluation report that was commissioned by the client. Component: Building Forestry and Protected Areas Management Capacity Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Protected Area fire brigades Number 0.00 13.00 25.00 13.00 equipped and trained 15-Dec-2011 15-Dec-2011 15-Dec-2016 25-Dec-2018 Comments (achievements against targets): The target of this indicator was partially achieved. This indicator was designed to measure capacity to fight fires in Protected Areas. The data source includes MNRE reports, the aide memoires, and Evaluation report that was commissioned by the client. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Pilot region fire brigades Number 0.00 5.00 5.00 0.00 equipped 15-Dec-2011 15-Dec-2011 15-Dec-2016 25-Dec-2018 Comments (achievements against targets): Page 37 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) The target of this indicator was not achieved. This indicator was designed to measure the capacity to fight fires in pilot regions. The value of the indicator was consistently not reported, as recorded in the ISRs. The data source includes FFA reports, the aide memoires, and Evaluation report that was commissioned by the client. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Reforms in forest policy, Yes/No N Y Y N legislation or other regulations supported 15-Dec-2011 15-Dec-2011 15-Dec-2016 25-Dec-2018 Comments (achievements against targets): The target of this indicator was not achieved. This indicator was designed to measure support to forest policy reforms. The value of the indicator was consistently not reported, as recorded in the ISRs. The data source includes MNRE and FFA reports, the aide memoires, and Evaluation report that was commissioned by the client. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Forest information Number 0.00 1.00 1.00 0.00 management systems in operation, with data 15-Dec-2011 15-Dec-2011 15-Dec-2016 25-Dec-2018 generated by Protected Areas used to produce reporting on forest fire risks, Page 38 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) hazards, response, and damage Comments (achievements against targets): The target of this indicator was not achieved. This indicator was designed to measure capacity improved capacity for data storage, retrieval, processing and reporting on forest and protected area fire statistics. The value of the indicator was consistently not reported, as recorded in the ISRs. The data source includes MNRE reports, the aide memoires, and Evaluation report that was commissioned by the client. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Landscape restoration Number 0.00 5.00 3.00 0.00 programs developed in disturbed PA landscapes 15-Dec-2011 15-Dec-2011 15-Dec-2016 25-Dec-2018 Comments (achievements against targets): The target of this indicator was not achieved. This is a new indicator introduced at the 1st restructuring was designed to development of landscape restoration programs. The value of the indicator was consistently not reported, as recorded in the ISRs. The data source includes MNRE reports, the aide memoires, and Evaluation report that was commissioned by the client. Formally Revised Actual Achieved at Indicator Name Unit of Measure Baseline Original Target Target Completion Number of PAs covered by Number 0.00 13.00 6.00 2.00 Page 39 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) fire management planning 15-Dec-2011 15-Dec-2011 15-Dec-2016 25-Dec-2018 and implementation to protect biodiversity and habitat Comments (achievements against targets): The target of this indicator was partially achieved. This indicator was designed to measure development and implementation of fire management plans in PAs, including use of maps to manage fire in biodiversity sensitive areas. The data source includes MNRE reports, the aide memoires, and Evaluation report that was commissioned by the client. Page 40 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) ANNEX 2. KEY OUTPUTS BY COMPONENT Outcome 1: To improve forest fire prevention and suppression in select forest ecosystems, including targeted protected areas Outcome Indicators Increase in percentage of fires contained • Firefighting, communication, and monitoring equipment within 24 hours following detection in procured and supplied: an integrated set of heavy machinery, pilot region extensive forest and project equipment, tools, and protective gear, scientific equipment and PAs tools procured and delivered. • Increase in percentage of fires • Communication systems for increased mobility and early contained within 24 hours following suppression of forest fires delivered to Syktyvkar (188 units), detection in pilot region extensive Khabarovsk (515 units), Moscow (1,137 units), and Voronezh forest (306 units) regions; integrated information and • Increase in percentage of fires telecommunication vehicle-mounted mobile systems delivered contained within 24 hours following to Krasnoyarsk (3 vehicles), Khabarovsk (2 vehicles), Moscow (1 detection in project PAs vehicle), and Voronezh (1 vehicle) regions. Target achieved partially - 42 percent • 13 PAs received firefighting, communication, and monitoring and 80 percent, respectively. equipment. Reach and acceptance of fire prevention Planned results of sociological surveys conducted under the messages among the PA target contract FFRP/B/CQS-150: ‘Sociological Surveys of the Population population in comparison to baseline and Visitors to Protected Areas to Identify the Level of Fire Target not achieved. Awareness’ would be generating outputs and helping achieve the indicator, but the contract was not implemented. Average area of fires at detection in • Early fire detection systems procured and tested: video camera project PAs computer systems installed for monitoring forest fires in Target overachieved - 101 percent. Krasnoyarsk (130 units), Khabarovsk (30 units), and Voronezh (20 units) regions. • Six-wheel drive incident command centers provided by the project. The command center is equipped with radio and satellite communications systems, digital displays, dedicated work stations, smoke monitoring equipment, air conditioning, and crew quarters. These centers are used for managing the firefighting of large-scale fire events in remote areas. It was supplied to Khabarovsk, Avia Lesohrana main office, Krasnoyarsk, Voronezh, and Moscow oblast. • 13 PAs received video camera computer systems and early fire detection systems. Area covered by improved Forest Pest Integrated mobile labs, equipment for seed quality testing and for Monitoring System in Pilot Regions forest pathology monitoring, and pest and soil analyses provided to Target not achieved. Roslesozahita and its regional branches; mobile labs for afforestation quality control provided to Leningrad, Volgograd, Voronezh, Krasnoyarsk kray, and Tver and Tomsk oblasts. Page 41 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Outcome 2: To enhance forest management in pilot regions Outcome Indicator Area of improved forest regeneration Equipment for seed quality testing and for forest pathology based on modern technology in Pilot monitoring; pest and soil analysis provided to Roslesozahita and its Regions regional branches; mobile labs for afforestation quality control Target not achieved. provided to Leningrad, Volgograd, Voronezh, Krasnoyarsk, Tverskoy, and Tomsk oblast. Component 1: Enhancing Forest Fire Prevention, Management, and Control Intermediate Results Indicators Development of a National Forest Fire Planned results of the contract №FFRP/B/CQS-123 ’Development of Management Strategy for Federal PAs a National Forest Fire Management Strategy for federal PAs’ would Target not achieved. be helping achieve the indicator’s target but the contract was not implemented as planned. Proportion of large fires out of the total Equipment supplied under the Component 1 also provided inputs to number of fires in Pilot Regions achieve this indicator’s target. extensive forest Target not achieved. Decrease in the number of fires in Planned report procured under the contract №FFRP/B/CQS-104 Project PAs in comparison to comparable ‘Economic Analysis of the Part B Project Investments’ would be PAs without project support generating outputs and helping achieve the indicator’s target but it Target not achieved. was not implemented in full. Updated fire danger rating and hazard Planned results procured under the contract №FFRP/B/QCBS-111 index system trialed in targeted ‘Upgrade of the System for Definition of Fire Safety Stages: Protected Areas Implementation of the New System to Measure Risk of Forest Fires Target not achieved. in Selected PAs» would be helping achieve the indicator’s target but the contract was not implemented as planned. Component 2: Building Forestry and Protected Areas Management Capacity Intermediate Results Indicators Protected Area fire brigades equipped Equipment supplied under Component 1 also provided inputs to and trained this indicator. Only 2 contracts were completed №№FFRP/B/IC-114 Target achieved partially - 52 percent. and 120. Tenders held for №№FFRP/B/IC-115-1, 115-2, 115-3 but did not move further. Technical specifications were prepared for tenders №№FFRP/B/IC-141, 143, 145 but procurement was not completed. Also planned that the results of the execution of these contracts №№ FFRP/B/IC-, 114-1, 120-1, 121-1, 142,144,146-148 help generate outputs and help achieve the indicator’s target but it was not realized as planned. Pilot region fire brigades equipped Equipment supplied under Component 1 also provided inputs to Target not achieved. this indicator. Reforms in forest policy, legislation or other regulations supported Target not achieved. Page 42 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Forest information management systems Because of the execution of the contracts FFRP/B/QCBS-127, in operation, with data generated by MNRE's Needs for Computerization of Activities in the Area of Protected Areas used to produce Forest Fire Protection in Federal PAs was assessed, a concept reporting on forest fire risks, hazards, developed, and a test of Integrated Forest Fire Management response, and damage Information System for PAs created. The MNRE was not satisfied Target not achieved. with the final outputs, went to court, but lost the court case in the end. Landscape restoration programs Planned results procured under the contract FFRP/B/CQS-151 developed in disturbed PA landscapes ‘Development Programs of the Restoration of Disturbed Landscapes Target not achieved. in pilot PAs’ would be helping achieve this indicator’s target, but it was not implemented as planned. Number of PAs covered by fire The final report procured under the contract FFRP/B/CQS-116 management planning and ‘Develop Fire Prevention Plans for the Largest and the Most Fire- implementation to protect biodiversity susceptible PAs’ was accepted by the MNRE. However, planned and habitat contract FFRP/B/QCBS-140 ‘Development of Projects Firefighting Target achieved partially – 33 percent. Equipment in the GIS Environment with a Focus on Biodiversity Conservation in Selected PAs’ was not implemented. Page 43 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) ANNEX 3. BANK LENDING AND IMPLEMENTATION SUPPORT/SUPERVISION A. TASK TEAM MEMBERS Name Role Preparation Angela Armstrong Task Team Leader Andrew Mitchell Sr. Forestry Specialist Marina Smetanina Team Member Galina Kuznetsova Sr. Financial Management Specialist Alexander Rukavishnikov Procurement Specialist Danielle Malek Country Lawyer Anna-Maria Bogdanova Operations Analyst Wolf Pohl Sr. Environmental Specialist Vladislava I. Nemova Team Member Nicolas Perrin Social Development Specialist Anna Eremenko Team Member Linh Van Nguyen Team Member Supervision/ICR Gayane Minasyan Task Team Leader(s) Angela Armstrong Task Team Leader Andrew Mitchell Task Team Leader Alexander Balakov Procurement Specialist(s) Galina S. Kuznetsova Financial Management Specialist Olga A. Gubareva Team Member Gennady Pilch Counsel Irina J. Prusass Team Member Marina Igorevna Smetanina Team Member Linh Van Nguyen Team Member Page 44 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Arcadii Capcelea Environmental Specialist Vladislava I. Nemova Team Member Aimonchok Tashieva Social Specialist Andrew Zakharenka ICR Main Contributor B. STAFF TIME AND COST Staff Time and Cost Stage of Project Cycle No. of staff weeks US$ (including travel and consultant costs) Preparation FY11 32.245 240,629.10 FY12 30.185 203,979.53 FY13 1.920 8,871.87 Total 64.35 453,480.50 Supervision/ICR FY13 5.205 56,319.35 FY14 12.563 97,011.65 FY15 29.029 131,452.77 FY16 22.662 139,781.07 FY17 11.010 114,164.45 FY18 11.080 121,540.34 FY19 25.115 133,706.92 FY20 7.403 37,780.69 Total 124.07 831,757.24 Page 45 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) ANNEX 4. PROJECT COST BY COMPONENT Appraisal Restructuring Actual Percentage of Approval Components Total GoRF IBRD Total GoRF IBRD Total GoRF IBRD Total GoRF IBRD Component 1: Enhancing Forest Fire Prevention, Management, 73.40 50.60 22.80 62.04 39.24 22.80 22.18 13.59 8.58 30 27 38 and Control 1.A Enhancing Forest Fire Prevention, Management, and 57.40 18.9 11.66 7.24 Control in Russia’s Extensive Forest Fund 1.B Enhancing Forest Fire Prevention, Management, and 16.00 3.28 1.93 1.34 Control in Russia’s Protected Areas Component 2: Building Forestry 44.40 30.70 13.70 14.21 0.51 13.70 8.58 5.18 3.39 19 17 25 and PA Management Capacity 2.A Building Forestry Management Capacity in Russia’s Extensive 40.40 6.72 4.18 2.54 Forest Fund 2.B Building Management Capacity 4.10 1.86 1.00 0.85 in PAs Component 3: Project 3.50 0 3.50 5.00 0 3.50 2.32 0.00 2.32 84 Management Total 121.30 81.30 40.0 81.25 39.75 40.00 33.08 18.77 14.29 25 23 37 Note: All the values are in US$, millions. Page 46 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) ANNEX 5. EFFICIENCY ANALYSIS 1. This annex provides an economic analysis for the Forest Fires Response Project. It also includes a brief description of the project context, challenges for carrying out the analyses, the assumptions made, and the results of the economic analyses. Project Context 2. At the time of preparation of this ICR, a critical forest fire situation emerged in Siberia and Ural of Russia. As of July 25, 2019, the official FFA data reported that forest fires territory exceeded 3 million hectares (of which 1,259,000 ha are in Yakutia, 1,017,000 ha in the Krasnoyarsk Territory, and 731,000 ha in the Irkutsk region). According to remote monitoring data, these three subjects of Russia account for about 95 percent of the total area of forest fires operating in Russia. The zone of smoke caused by these fires also included all or part of almost all regions of Siberia and part of the Urals. The largest Siberian cities—Krasnoyarsk, Novosibirsk, Barnaul—and others were covered by thick smoke. Residents of these cities urged for effective actions of the authorities to protect forests from fires. In particular, a petition on the introduction of an emergency regime throughout Siberia in connection with forest fires appeared on the dedicated Change.org portal. This petition was rapidly supported by Russian citizens; on July 25, 2019, more than 100,000 people signed this petition and on November 3, 2019, 1,000,249 people signed it. 3. In the territory of PAs, this year has also been a challenging one in terms of suppression of forest fires. Officially, the MNRE reported that as of mid-August, 11 fires were fixed, covering a total territory of 6,181 ha, of which 3,204 is forest area. 15 4. Forest fires in Russia not only trigger the damage of ecosystems and biodiversity losses and increase of emissions but also cause losses to the economy. As of 2018, the economic damage of forest fires was reported by the FFA as RUB 16.9 billion (in 2017, it was RUB 25.2 billion and in 2016, RUB 23.7 billion). The observed reduction of the economic losses is not likely with the decrease in the number of forest fires; it is due to a change of the methodology applied by the FFA to the forest fire management. Overall, the forestry sector in Russia is characterized as deficit for the federal budget. Total federal revenue in 2018 was RUB 29.5 billion, while the federal expenditures line was RUB 32 billion. 5. The scarce financing of the forest management in Russia complicates the actions to suppress the forest fires or to prevent them. The forest fires measures are neither financed in full and as planned nor targeting the forest firefighting mainly but less or little is financed to enable prevention of forest fire and monitoring. Under the existing Federal Program on Forests 2013–2020, 16 the financing from federal budget of the whole set of activities on forest fires prevention was decreased in 2017; subsidies to the regions were dropped which could be used for the forest fire equipment and machinery. On land patrolling and air and space monitoring activities, budget spending was decreased in 2017. 6. A new national priority project ‘Ecology’, implemented from 2019 to 2024, is targeting the protection of the forests. In particular, it aims to decrease the losses from forest fires from RUB 32.3 billion 15 https://tass.ru/proisshestviya/6759271. 16 Federal Program on Forests 2013-–2020, annual report, February 28, 2018. Pp. 25-26. Page 47 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) to RUB 12.5 billion in 2024. Also, the ratio of logged/damaged forests to reforested is aimed to increase from 62 percent in 2018 to 100 percent in 2024. Challenges in Conducting Economic and Financial Analysis 7. Factors limiting the Environmental Assessment were encountered both on the benefits and costs parts. At the project appraisal, not all benefits were included in the modeling due to limited data on the monetary values, climate factors (rise of temperatures, precipitation levels, and so on), and the dynamics of reduction of damage from the forest fires during the longer periods as compared with and without project interventions. Therefore, the initial analysis assessed two groups of benefits: reduced timber losses (direct benefit) and reduced amounts of carbon entering the atmosphere as a result of fires (indirect benefit). Further analysis at the MTR included expanded list of direct and indirect benefits but only for 12 PAs of Part B (not 25 PAs) and did not cover part A pilot regions of the Extensive Forest Fund. Limitations to the cost analysis related to the fact that the substantial number of the planned investments was not implemented, and the operations and maintenance costs for Part A activities were difficult to evaluate. For the analysis, it was assumed that all supplied equipment should be in use by project beneficiaries. 8. To improve the economic analysis assumptions and overcome the above challenges of the lack of many of the indirect benefits being included and quantified, a separate analytical activity was conducted during 2016–2018. Obtained results of the economic analysis of the 12 Part B PAs 17 were then compared with the PAs of the so-called ‘control group,’ 18 where no project activities were implemented. Other assumptions included the project years from 2015 till 2030 and discount rates of 10 percent and 12 percent. The costs stream included the loan funds and the co-financing costs of the GoRF for Part B, which were then discounted at 10 percent and 12 percent based on the up-to-date Procurement Plan information. The benefits stream included the benefit of reduction of damage (costs) of forest fires due to the improved capacity of PAs to suppress the forest fires and also other 10 parameters qualitatively 17 NP Meschera (including the territory of the former NP Meshchersky), SNBR Okskiy, SNR Kerzhensky, NP Buzuluskiy Bor, SNR Zhigulevskiy, NP Samarkskaya Luka, SNR Sayano-Shushenskiy, SNR Stolby, and NP Shushkenskiy Bor, SNR Bureinskiy, SNR Komsomolskiy, and SNR Bastak. 18 Control group is a group of PAs comparable with the selected PAs and the PAs involved in the project by their area, geographic location, climatic and landscape characteristics, predominant forest formations, and their combinations. The composition of the ‘control group’ as cleared with the MNRE includes NP Nechkinsky, NP Plescheevo Lake, SNR Tungusskiy, SNR Dzhugdzhurskiy, and SNR Khinganskiy. Page 48 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) assessed. 19 As a result, for the Part B economic analysis in the 12 PAs, the NPV resulted in US$7.8 million and US$9.7 million, while IRR ranged from 23 percent to 36 percent. Table 5.1 provides details about the average and the final damage in U.S. dollars for the project PAs and the control group PAs. Table 5.1. Evaluation of Damage and Other Costs Associated with Forest Fires in the PAs Under Study in 2011– 2017 (rubles, thousands) Parameters Characterizing Various Types of Damage and Costs for its No. PAs Control Group Liquidation and Estimated in Value Terms 2017 as 2017 to Percentage Average Average 2011– the of the 2011–2015 2015 Average in Average for Percentage the Years 1 The health and injury-related losses associated with forest fires in terms of 0.0 0.00 0.0 0.00 compensation payments 2 The reduction of opportunities for the development of tourism and recreation 88,510.0 105.81 20,753.0 24.45 due to reduced attractiveness of PAs as a result of forest destruction by fires 3 The damage to biodiversity, including habitats of protected plants, animals 271,871.6 111.08 3,822,133.8 144.93 and fungi (loss of habitat), and rare ecosystems in general 4 The damage related to the reduced amount of non-timber forest products 327.3 234.14 4,550.6 67.84 (berries, mushrooms, medicinal plants, and so on) 5 The damage caused by carbon emissions into the atmosphere in 381.3 41.45 13,059.4 26.16 natural fires 19 (a) The health- and injury-related losses associated with forest fires in terms of compensation payments (to be evaluated if there were injuries reported in connection with fires in PAs); (b) the reduction of opportunities for the development of tourism and recreation due to reduced attractiveness of PAs because of forest destruction by fires (considering data on the number of visitors to PAs, landscape inventory data, and data on the area of lost forests); (c) the damage to biodiversity, including habitats of protected plants, animals and fungi (loss of habitat), and rare ecosystems in general; (d) the damage related to the reduced amount of non-timber forest products (berries, mushrooms, medicinal plants, and so on); (e) the damage caused by carbon emissions into the atmosphere in natural fires; (f) the costs of elimination of consequences of forest fires, including reclamation of disturbed forest landscapes; (g) the costs of damaged infrastructure (buildings, bridges, fences, information boards, equipped recreation sites, and so on), cultural heritage objects in PAs; (h) costs for firefighting measures; (i) other costs for the elimination of fires consequences in PAs not included in the costs of f and g (forest pathological monitoring, other works, if respective activities can be carried out in accordance with management regimes of PAs and their functional zones); and (j) reduction in value of standing timber because of damage to the forest fire and the subsequent development of forest pathology in weakened forest ecosystems (in the order determining payment rates per unit volume of forest resources, under federal ownership, established by the GoRF). Page 49 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Parameters Characterizing Various Types of Damage and Costs for its No. PAs Control Group Liquidation and Estimated in Value Terms 6 The costs of elimination of consequences of forest fires, including 544.4 60.06 1.2 0.00 reclamation of disturbed forest landscapes 7 The costs of damaged infrastructure (buildings, bridges, fences, information 0.0 0.00 0.0 0.00 boards, equipped recreation sites, and so on), cultural heritage objects in Pas 8 Costs for firefighting measures 163,334.3 109,80 — — Expenses for extinguishing fires in Pas 1,765.8 39.66 2,239.4 496.41 9 Other costs for the elimination of fires consequences in PAs not included in the 382.8 186.33 0.0 — costs of l6 and l7 10 Reduction in value of standing timber as a result of damage to the forest fire and the subsequent development of forest 469,564.0 3.59 308,071.6 268.06 pathology in weakened forest ecosystems Total economic damage from fires 4,170,809.0 833,347.2 (rubles) Total damage (US$) 21,830.2 32.62 109,269.3 153.16 9. The analytical activity also demonstrated that the main amount of damage and other costs associated with forest fires in the PAs included in the project was damage to biodiversity, including the habitats of protected plant, animals, and fungi (habitat loss or loss) and rare ecosystems as a whole— 73 percent of the total economic loss (or RUB 301,982,900). The second most important factor determining the extent of damage is the reduction of opportunities for the development of tourism and recreation due to reduced attractiveness of PAs as a result of forest destruction by fires—23 percent (or RUB 93,651,400). Other parameters characterizing the damage and costs associated with forest fires in PAs account for less than 5 percent of the total damage. 10. Estimation of the economic damage from forest fires in the ‘control group’ of PAs was carried out according to the same methodology as for the 12 PAs. A decrease was observed in 2017 in the level of economic damage from forest fires in the PAs of the ‘control group’. Economic losses decreased compared to 2016 and are estimated at RUB 6,388.0 million (in 2016 -RUB 10,558.2 million in current prices), but there was no decline relative to the average.. The economic damage from forest fires in 2017 is 1.5 times higher than the average for 2011–2015 (RUB 4,170.8 million). Results of the Economic and Financial Analysis 11. The ex post economic and financial analysis compared the ERR and NPV before and after the project for both Parts A and B of the project and considers the results of expanded economic analysis for Page 50 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) the Part B investments in the 12 participating PAs. The analysis conducted at project completion updates the assumptions made at appraisal and uses the actual data of the forest fire area, the updated stumpage prices, and the actual exchange rate of Russian rubles/U.S. dollars. It also includes actual investment costs based on the final financial report of the PIU as of March 31, 2019. 12. The following assumptions were adopted in the ex post analysis: (a) the project generated both local and global benefits, for example, the reduced timber loss for the local benefits and reduced carbon emissions for the global ones; (b) project costs were updated based on the actual and final versions of the project’s Procurement Plan; (c) supplied equipment is expected to be in use by project beneficiaries in pilot regions areas and in 13 PAs for reduction of forest fires; (d) local benefits start only from the year when the equipment was supplied; (e) stumpage price increased in 2019 compared to the appraisal stage and is RUB 98.9 per m3, or US$1.58 per m3; and (f) project interventions will reduce the burnt area by 12 percent in the pilot regions and by 2 percent nationally due to reduction of scope of project activities and early project termination. The analyzed benefits and costs of the project helped to achieve two PDO indicators: PDO #1: Increase in percentage of fires contained within 24 hours following detection in pilot extensive forest regions and project PAs and PDO #2: Average area of fires at detection in project PAs. Due to the poor reporting, the fact that fire brigades were equipped with new gear and new firefighting tools was not reflected in the RF. 13. Based on these assumptions and with the limiting factors applied for the project area, Case 1 (both global and local benefits) produces the EIRR of 42 percent and the NPV of more than US$150 million at project completion, as shown in table 5.2. Case 2 (local benefits only) produces more modest results: an EIRR of 23 percent and NPV of US$21 million. Table 5.2. Economic Analysis Ex-Ante Economic Analysis Midterm Economic Analysis Ex Post Economic Analysis • Investment costs of • Investment costs of Part B • Investment costs of Part A and B Subcomponent 1A • Discount rate 12% • Discount rate 12% • Discount rate 12% • Include 10 expandeda types of • Includes both local (reduced • Includes both local local and global benefits (for timber loss) and global benefits (reduced timber loss) example, reduced carbon (reduced carbon emissions) and global benefits emissions, damage to (reduced carbon biodiversity, and landscape emissions) restoration costs of PAs) • EIRR is 23% • EIRR is 36% Case 1 (both global and local • NPV of more than • NPV is US$9.7 millionb benefits):c US$50.5 million • EIRR is 42% • NPV of more than US$ 150 million Case 2 (local benefits only): • EIRR is 23% • NPV is US$21 million Note: a. Reduced timber losses, health and injury-related losses due to fires, damage related to the reduced amount of NTFPs, costs of damaged infrastructure in PAs, and costs of firefighting measures. b. Average nominal exchange rate of the ruble to the U.S. dollar as of 2017 was used from CBR: RUB 58.6/US$ (Source - https://www.cbr.ru › vfs › ex_rate_ind › 03-ex_rate_17). c. Average nominal exchange rate of the ruble to the U.S. dollar as of 2018 was used from CBR: RUB 62.54/US$ (Source - https://www.cbr.ru › vfs › ex_rate_ind › 03-ex_rate_18). Page 51 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) 14. Sensitivity analysis has been undertaken to show the robustness of the assumptions. To reduce the EIRR to 10 percent and NPV of US$10.3 million under Case 2 and EIRR to 37 percent and NPV of US$144 million under Case 1, the assumptions have to be changed individually as follows: • The percentage of forest totally destroyed increases from 50 percent to 90 percent. • Project interventions will reduce the burnt area by only 5 percent in the pilot regions and by 1 percent nationally due to reduction of scope of project activities and earlier closing of the project. 15. Distribution of benefits. Reduction of fires risk helped to better protect local communities and forest users and reduce future damage and losses. However, planned education and training activities for project beneficiaries were not implemented, which decreased the project’s social benefits. It could have helped to improve outreach of forest authorities to the public, increase public awareness of the ecological and economic values of forests, and strengthen the sense of forest ownership of population living nearby. The policy, legal frameworks, and institutional capacity-building activities were also not implemented as planned, missing the opportunity to strengthen on-the-ground law enforcement, clarifying the rights and responsibilities of forest inspectors and rangers, as well as forest users. 16. Environmental benefits stem from activities and investments in the protection of forests and their future reforestation. The project supplied equipment (mobile pest monitoring laboratory) to the FFA’s regional office of the Department of Forest Protection in Krasnoyarsk region is critical for helping monitor and model forest pest outbreaks. Specifically, the region’s forests experienced devastating outbreaks of Siberian Moth (Dendrolimus superans sibiricus) affecting more than 300,000 ha of forest since 2017. Project-supplied DNA monitoring equipment strengthened the capacity of Krasnoyarsk Department of Forest Projection to monitor and identify other different pathogens (insects, fungi, and bacteria) affecting the forest health in a longer term. The seed germination testing equipment provided is used for testing the seeds collected for use in reforestation of damaged areas and results in considerable savings and improvement in forest nurseries performance. Equipment supplied to St. Petersburg and Voronezh scientific forestry organizations is in use and is essential for the long-term protection of forest ecosystem services. 17. Aspects of project design and implementation. Aspects that reduced the project efficiency include leadership challenges at the FFA; ineffective process of forming procurement packages, delayed procurement, and delivery of goods and services; and six significant contractual disputes that were not resolved and resulted in court cases. 18. Leadership challenges and resulted inefficiency affected the project, but mostly its Part A was implemented by the FFA. At the project preparation stage, two similar projects prepared by the FFA and MNRE were put together into one following the guidance of the MoF, with two parts: Part A implemented by the FFA and Part B implemented by the MNRE. Following massive number (some 30,000) and area (more than a million hectares) of forest fires during the summer of 2010, the GoRF reorganized the FFA and subordinated it directly to the Prime Minister and the Government, with policy and regulation functions over the Extensive Forest Fund. After the project negotiations in 2012, the FFA was subordinated to the MNRE, which further complicated the project implementation. Based on the conducted ICR interviews, AMs, and perception of project stakeholders, insufficient institutional oversight arrangements Page 52 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) to drive results is generally highlighted for the period beginning from October 2014 and until project termination in March 2019. In addition, subordination of FFA under MNRE did not result in more effective oversight. Further, the project oversight Interagency Committee 20 (which was chaired by head of the FFA) did not resolve these constraints. 19. Procurement delays affected the efficiency. The process of forming procurement packages for required goods (machinery, tools, communication equipment, and their specifications) between the PIU, the implementing agencies, and five regions became very lengthy, cumbersome, and resulted in significant procurement delays. Further, the new FFA leadership significantly delayed authorization of contracts, as stated in the several AMs. By December 2015, 30 contracts were outstanding for a total value of approximately US$22 million, US$11 million of which was for contracts already awarded and ready for signing (some for as long as 11 months). 20. Considering the economic and financial analysis, the aspects of project design and implementation stated above, as well as poor results reporting, the efficiency rating is assessed as Modest. 20As stated in PAD “established to support interagency coordination, monitor project progress on a quarterly basis, as well as settle controversies that might arise during project implementation. The Committee will be chaired by the Head of FFA, and will consist of the Deputy Head of FFA, as well as representatives from MNRE, Ministry of Economic Development (MOED), and MINFIN.” Page 53 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) Annex 5a. GHG Emissions Modeling EX-ACT 1. Comparing results of carbon balance accounting at appraisal and completion. Based on the EX- ACT model developed by the Food and Agriculture Organization of the UN (FAO), estimates of the impacts of Part A activities of the project were implemented. The EX-ACT tool was presented in Moscow during project preparation by the FAO developers of the tool in Moscow in 2013. Using the project data and assumptions provided by the participants, and based on gains over the whole of Russia mainly due to policy changes and modest changes in capacity, the tool demonstrated that the net balance would be a reduction in emissions over a 25-year period of an excess of 215 million tons of CO2 equivalent (or a 31 percent reduction in emissions over the baseline). If a price of just US$10 per ton of CO2 is assumed to be saved, it would mean that the economic benefit from carbon alone could amount to US$2 billion over 25 years due to the project interventions. 2. GHG emissions modeling EX-ACT for the completion stage calculated using the EX-ACT model demonstrated that the project activities would generate a net sink of approximately 99 million tons of CO2eq over 20 years, with approximately −1 ton of CO2eq per ha over 20 years (carbon balance is - 2,879,504 tons of CO2eq). This amount corresponds to losses of biomass (81.6 Mt CO2) and soil organic matter (16.4 4 Mt CO2eq) compensated by the avoided N2O and CH4 emissions because of prevented burning (−79 Mt CO2eq and −119 Mt CO2eq). The proposed improvement of reduced burning/fires would compensate for the deforestation activities related to cutting the firebreaks in the forest. The NPV calculations in EX-ACT model based on a carbon price of US$10 close to EU market price for the carbon balance per year resulted in positive NPV of US$127 million and IRR above 33 percent with a discount rate of 10 percent on 20 years (2014–2033). 21. Table 5.3. GHG emissions modeling EX-ACT for the completion stage Project Name forest fire response project e Climate Boreal (Moist) Duration of the Project (Years) 20 Continent Asia (Continental) Dominant Regional Soil Type HAC Soils Total area (ha) 72429820 Gross fluxes Share per GHG of the Balance Result per year Components of the project Without With Balance All GHG in tCO2eq Without With Balance All GHG in tCO2eq CO2 N2O CH4 Positive = source / negative = sink Biomass Soil Other Land use changes CO2-BiomassCO2-Soil CO2-Other N2O CH4 Deforestation 0 98,542,664 98,542,664 81,636,357 16,396,773 251,964 257,570 0 4,927,133 4,927,133 Afforestation 0 0 0 0 0 0 0 0 0 0 Other LUC 0 0 0 0 0 0 0 0 0 0 Agriculture Annual 0 0 0 0 0 0 0 0 0 0 Perennial 0 0 0 0 0 0 0 0 0 0 Rice 0 0 0 0 0 0 0 0 0 0 Grassland & Livestocks Grassland 0 0 0 0 0 0 0 0 0 0 Livestocks 0 0 0 0 0 0 0 0 Degradation & Management 1,530,996,640 1,332,065,954 -198,930,685 0 0 -79,049,900 -119,880,785 76,549,832 66,603,298 -9,946,534 Coastal wetlands 0 0 0 0 0 0 0 0 0 0 Inputs & Investments 0 856,835 856,835 856,835 0 0 0 42,842 42,842 Fishery & Aquaculture 0 0 0 0 0 0 0 0 0 Total 1,530,996,640 1,431,465,454 -99,531,186 81,636,357 16,396,773 856,835 -78,797,936 -119,623,216 76,549,832 71,573,273 -4,976,559 Per hectare 21 20 -1 1.1 0.2 0.0 -1.1 -1.7 Per hectare per year 1.1 1.0 -0.1 0.1 0.0 0.0 -0.1 -0.1 1.1 1.0 -0.1 With a price range of US$3–6 per ton of CO2eq. according to Voluntary Carbon Markets. Outlooks and Trends: January to 21 March 2018, the NPV results in US$41.03 million and IRR above 17 percent. Page 54 of 55 The World Bank RUSSIA FOREST FIRE RESPONSE PROJECT (P123923) ANNEX 6. SUPPORTING DOCUMENTS Summary of Borrower’s Comments on Draft ICR Ministry of Finance submitted Borrower’s comments on the draft ICR. A brief summary of the key comments is as following. • Borrower agrees with the Bank that project implementation encountered many problems mostly related to project management and performance of the IAs. These problems were tracked and accurately reflected in the Aide-Memoires. • The correct amount of disbursed IBRD Loan funds is US$14.297,286.61 and the actual co-financing is US$18,778,373.59, with the correct total project cost of US$33,075,660.20, according to the Monthly Disbursement Summary and project audited financial statements. • According to allocation of responsibilities established in the project, activities associated with procurement were split between the IAs and PIU, making the IAs solely responsible for drafting all technical documentation. They were further included by the PIU into the bidding documentation or requests for proposals. Delays in preparation of the technical documentation were formalized in project implementation and procurement plans, as agreed or no-objected by the Bank. • Mismatch in procured goods in terms of the required items is the result of poor interaction between FFA and participating regions which procured drafting technical specifications and schedules of requirements for the goods. PIU had to prepare and agree with the Bank several amendments to signed contracts for reallocation of contracted goods. The cost of procured goods was always established in open competition by conducting international competitive bidding. • IA-based, project specific PIU is an unrealistic option for Russia given complicated fiduciary requirements of the Bank. The best option is to use PIUs with qualified staff to comply with procurement and FM requirements and to hire additional qualified consultants in the core business to assist the ministries for the substance issues. The key to successful project implementation is the commitment of the line ministries and the IAs to the project goals. • The proposed option of project’s Steering / Oversight Committee to be chaired by the MoF or a member of Prime Minister’s Cabinet contradicts the Government Decree #43 and Government Resolution on the signing of the Loan agreement. Project implementing agency bears the whole responsibility of the project implementation, while the responsibility of the MoF is to control project implementation within established limits of responsibilities. Page 55 of 55