Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) Report Number : ICRR0020723 1. Project Data Project ID Project Name P099469 BR (APL2) 2nd National Environmental Country Practice Area(Lead) Brazil Environment & Natural Resources L/C/TF Number(s) Closing Date (Original) Total Project Cost (USD) IBRD-77820 31-Dec-2014 34,720,000.00 Bank Approval Date Closing Date (Actual) 22-Sep-2009 30-Jun-2016 IBRD/IDA (USD) Grants (USD) Original Commitment 24,300,000.00 0.00 Revised Commitment 3,946,649.28 0.00 Actual 3,946,649.28 0.00 Prepared by Reviewed by ICR Review Coordinator Group Ranga Rajan John R. Eriksson Christopher David Nelson IEGSD (Unit 4) Krishnamani 2. Project Objectives and Components a. Objectives The Project Development Objective (PDO) as stated in the Loan Agreement (Schedule 1, page 5) was: " To continue to strengthen environmental capacity in the Borrower's key institutions by enhancing environmental management capacity of institutions at the federal, state, Federal District and municipal levels and by demonstrating effectiveness of targeted partnerships focused on defined priorities." The PDO as stated in the Project Appraisal Document (PAD, page 5) was similar: "To contribute to strengthening environmental capacity in Key Brazilian institutions." The PAD further Page 1 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) states that this objective was be accomplished by enhancing the environmental management capacity of institutions at the federal, state, Federal District and municipal levels and by demonstrating the effectiveness of targeted partnerships focused on defined priorities. b. Were the project objectives/key associated outcome targets revised during implementation? No PHEVALUNDERTAKENLBL c. Will a split evaluation be undertaken? No d. Components This project was the second phase of a three-phase Adaptable Program Loan (APL). The first phase was implemented between June 2000 and June 2006. The envisioned third phase was to be implemented between 2016 and 2019. Actual cost was lower than the appraisal estimate, due to the reduced scope of the project during implementation. States and municipalities, with one exception (Acre state) did not participate in the project, although eleven states applied for eligibility by submitting the necessary documentation and two states became eligible. Consequently, component two activities associated with states, Federal District and municipalities were not implemented and the project scope focused exclusively on activities at the federal level. This project under review was divided into two phases. The first phase (Sub-phase 2A) was expected to last five years and the second phase (Sub-phase 2B) was expected to last another two years. Sub-phase 2B was not initiated and this evaluation is based on assessment of Sub-phase 2A. Institutional Development. (Appraisal estimate US$25.76 million. Actual cost at closure US$4.10 million). This component aimed at providing institutional support to the agencies at the federal, state, Federal District and municipal levels. Activities included: (i) strengthening their environmental licensing policies and procedures: (ii) implementing an environmental monitoring system integrating the existing state, Federal District and municipal networks with the Ministry of Environment's (MMA) environmental monitoring systems: and, (iii) promoting innovative economic instruments for environmental management including inter alia, through environmental audits, payment for environmental services based on the polluter/user pays principle. Integrated Management of Environmental Assets. (Appraisal estimate US$17.16 million. Actual cost at closure US$0.00 million). This component aimed at providing incentives to states for identifying, ranking and addressing some of their most pressing environmental challenges. Eligible states would qualify for receiving matching grants in exchange for fulfillment of eligibility criteria. State level activities were not implemented and project activities focused almost exclusively at the federal level during implementation. Coordination, Dialogue and Communication. (Appraisal estimate US$5.00 million. Actual cost at closure US$2.43 million). This component aimed at strengthening the organizational and operational structure of the Project Management Unit (PMU). Activities included capacity building to the project partners (including federal agencies, states, the Federal District, municipalities and Non-Governmental Organizations (NGOs) on: (i) environmental licensing procedures: (ii) updating/identifying environmental priorities: (iii) partnership proposal development: (iv) environmental monitoring partnerships: and, (v) development of economic instruments. Page 2 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) e. Comments on Project Cost, Financing, Borrower Contribution, and Dates Project cost. The total estimated project cost (including baseline costs and costs associated with physical and price contingencies) was US$34.72 million. The actual cost at closure was US$6.53 million, about 19% of the appraisal estimate. Actual cost was lower than appraised due to the reduced scope of the project during implementation. States and municipalities for the most part did not participate in the project and the project activities focused only on the relevant federal level agencies. Project Financing. The project was financed by an IBRD loan. The appraisal estimate of the loan was US$24.30 million. The amount disbursed at closure was US$3.95 million, about 16% of the loan. US$15.70 million of the loan was cancelled. The low disbursement at closure was partly due to exchange rate fluctuations during implementation (the Brazilian Real (BR) depreciated relative to the US dollar ($) from R$1.78 at the approval date to more than R$3.0 per US$ by 2015 (ICR, page 8). Borrower Contribution. The total appraisal estimate was US$10.42 million. Their actual contribution was US$2.64 million. about 25% of what was planned at appraisal. Dates. There were two Level 2 restructurings. The following changes were made through the first restructuring on October 21, 2014: (i) All activities associated with decentralization of environmental management to states and other subnational entities were dropped: and, (ii) The project closing date was extended by nine months from December 31, 2014 to September 30, 2015, to enable the MMA to submit a thorough restructuring proposal and for providing support for implementing the Brazil's new "Agenda for Environmental Sustainability and Climate Change" outlined at the United Nations (UN) Assembly in September 2015 and presented later at the United Nations Climate Change Conference (COP 21) held in Paris in December 2015. The following changes were made through the second Level 2 restructuring on September 30, 2015. The MMA submitted a reduced restructuring request which was approved by the External Financing Committee of the Ministry of Planning (COFIEX) in December 2015 and the following changes were made formally: (i) The project scope was reduced to activities at the federal level only. (ii) Some activities such as studies to support policy formulation to promote sustainable management of native Amazon forests and monitoring and controlling of energy use and greenhouse gas emissions in large cities were added. and (iii) The project closing date was extended by an additional nine months from September 30, 2015 to June 30, 2016. The project closed about eighteen months behind schedule on June 30, 2016. 3. Relevance of Objectives & Design a. Relevance of Objectives The Brazilian economy in the years before appraisal faced major challenges associated with reversing the trend associated with unsustainable use of natural resources and environmental degradation. Although Brazil had adequate environmental legislation for regulating the natural resource sector activities, it did not have the institutions for effectively applying environmental management policies. Environmental licensing Page 3 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) procedures were not always effective and environmental enforcement relied more on command-and-control methods, rather than through methods such as environmental taxes based on the polluter-pays principle. Strengthening the capacity of environmental institutions at all levels (including the federal, state, Federal District and municipalities) was important in the country context. This was particularly so given that environmental management had been a rising priority to the government in the years before appraisal due to: (I) growing public awareness of environmental concerns: (ii) increasing sensitivity to domestic and external criticism of poor natural resource management: and, (iii) growing recognition in both the public and private sectors that better environmental management would aid in providing a competitive edge to Brazilian business. The PDO was also relevant to the Government's new "Agenda for Environmental Sustainability and Climate Change" outlined at the United Nations Assembly in September 2015, as explained in section 2c above. The PDO was well aligned with two of the World Bank Group's Country Partnership Strategy's (CPS) three pillars: (i) the sustainability pillar which underscored the need for strengthening environmental management, reducing costs associated with environmental degradation and promoting an environmentally sustainable growth pattern in Brazil: and, (ii) the competitiveness pillar which underscored the need for promoting the institutional development, strengthening environmental policies and regulations, streamlining environmental licensing, improving environmental monitoring and supporting the use of economic instruments and targeting decentralized demand-driven partnerships aimed at improving the management and conservation of environmental assets. The Bank's most recent CPS for the 2012-2015 period highlighted the importance of improving efficiency and effectiveness of environmental licensing and monitoring systems. Rating High b. Relevance of Design The causal links between the project activities and their outputs was clear. Component one activities aimed at strengthening institutional strengthening of the various government entities were intended for improving the environmental management capacity of these agencies. The design appropriately entailed sequencing institutional strengthening activities with targeted environmental investments by states to address various categories of environmental challenges (such as in forest management, water resources managements and solid waste). These activities can also be expected to contribute to the higher level objectives of enhancing environmental quality and sustaining economic growth. There were several shortcomings in design: (i) The PDO was vague and general and the intended outcomes were not clear. While the PDO aimed at "contributing to strengthening environmental capacity in key Brazilian institutions", it did not clearly define what would be achieved, even in qualitative terms and the key outcome indicators provided no way of verifying improvements in environmental management. The project was complex, with respect to arrangements pertaining to transferring funds to sub national entities.Transferring matching funds to these entities required a partnership (Convenios - matching grant funding agreements) arrangement between the Ministry of Environment (MMA) and the relevant state or municipality. The modalities of administering such an arrangement, necessitated a large number of decentralized procurement actions that were perceived as cumbersome both by MMA and by the states. The Page 4 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) issue was further exacerbated by the lack of skilled staff in the Project Coordination Unit (PCU). Although the experience with Phase 1 of this project had indicated shortcomings about the cumbersome administrative requirements of the United Nations Development Program (UNDP) in procurement matters and there were legal restrictions in Brazil on using international organizations in projects funded by the federal budget, UNDP technical assistance was retained as part of project design. The UNDP "option" was finally discarded by MMA only in 2011. The PCU capacity problem was compounded by the fact that temporary civil service contracts of PCU staff were to expire in 2009 or early 2010 and there were no arrangements made at appraisal for hiring new MMA staff. The lack of external support in procurement matters compounded by the lack of internal staff, affected project implementation. It is not clear if the project was based on a systematic diagnostic of actual institutional capacities, which would have aided in tailoring institutional strengthening activities to the needs of the various government entities. It is also not clear if there was a concrete plan or proposals for using loan funds for implementing licensing activities, even at the federal level. This is especially so, given that the relevant federal environmental agencies - the Brazilian Institute for the Environment and Renewable Natural Resources (IBAMA) and the Chico Mendes Institute for Biodiversity (ICMBio) - were not involved in project preparation and had not been asked to prepare proposals for funding under the project. The assumption at appraisal that the Ministry of Environment (MMA) would be willing to promote the participation of sub-national entities as partners and stakeholders in the Brazilian National Environment System (SISNAMA) proved to be unrealistic, in view of the MMA's clear shift in preference towards supporting federal agencies only. This contributed to the cancellation of project activities at the state and municipal levels. Rating Modest 4. Achievement of Objectives (Efficacy) PHEFFICACYTBL Objective 1 Objective The PDO "To continue to strengthen environmental capacity in the Borrower's key institutions by enhancing environmental management capacity of institutions at the federal, state, Federal District and municipal levels and by demonstrating effectiveness of targeted partnerships focused on defined partnerships" is evaluated on the basis of the three sub-objectives: Sub-objective One: "To contribute to strengthen capacity in the Borrower's key institutions." Rationale Outputs. •The following studies were completed as targeted: (I) An international comparative study, survey of the Federal Environmental Laws, Environmental Impact Assessment (EIA) and European Commission models Page 5 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) in Brazil and comparative analysis with equivalent systems in the European Union, Germany, the Netherlands, Portugal, Australia, Canada, the United States and Chile was completed as targeted: (ii) Impact matrix for environmental studies: (ICR, page 22). • The licensing staff of the Brazilian Institute for the Environment and Renewable Natural Resources (IBAMA) were trained. No targets were set for this indicator (ICR, page 23). • The Integrated Environmental Management System aimed at systematizing data, mapping and developing technological solutions for the automation of the EIA, environmental licensing, and environmental compensation was completed as targeted (ICR, page 23). • A National Portal on Environmental Licensing was created and was functional at project closure as targeted. This portal was being used by the public for consultation of the federal, state and municipal licenses issued (ICR, page 23). Outcomes. • The project achievements were at the output level. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 2 Objective Sub-Objective Two: To enhance the environmental management capacity of institutions at the federal, state, Federal District and municipal levels. Rationale Outputs. • Eleven states submitted evidence for eligibility criteria in the policy reform matrix. Out of this, two states qualified as eligible. This was as compared to the original target of 18. (ICR, Datasheet, Intermediate outcome Indicator, Number One). • Thirteen or more states and one federal agency were expected to have a fully functioning environmental licensing system. The Federal Agency - the Brazilian Institute for the Environment and Renewable Natural Resources (IBAMA) had a fully functional computerized environmental licensing system at project closure. None of the states had a fully functional licensing system at project closure (ICR, Datasheet, Intermediate Indicator Number Four). • The Brazilian Institute for the Environment and Renewable Natural Resources (IBAMA) produced procedures and technical guidance for two typologies of investments (power lines and roads) as targeted at project closure. None of the states produced such procedures as compared to the target of nine states Page 6 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) (ICR, Datasheet, Intermediate Outcome Indicator, Number Five). • None of the states reduced the time of issuance of preliminary licenses for the three typologies most important in the state as compared to the target of three states (ICR, Datasheet, Intermediate Outcome Indicator Number Six). • Data storage and server capacity of the Ministry of Environment (MMA) was expanded at project closure. This allowed MMA to keep running its many systems and applications for internal and public use such as the Rural Property Cadastre (CAR) database with its user interface for entering rural property data. Outcomes. • Outputs pertaining to the federal agencies were achieved. There is no evidence that capacity for environmental management at the state, federal District and municipal levels was enhanced. Rating Modest PHREVDELTBL PHEFFICACYTBL Objective 3 Objective Sub-Objective Three: To demonstrate effectiveness of targeted partnerships focused on defined priorities. Rationale Outputs. • The outputs described above are also relevant to this objective. Outcomes. • All indicators were output-oriented. Rating Modest PHREVDELTBL PHREVISEDTBL 5. Efficiency Page 7 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) Economic Analysis. Given that the activities entailed only institutional strengthening activities, no cost- benefit was conducted for this project at appraisal. No ex post analysis was undertaken for the activities associated with the economic assessment of the partnerships with the states as no such partnership activities eventually took place. Administrative and Operational Issues. Although project preparation started immediately at the end of the Phase 1 project in June 2006, the project was approved only in October 2009 and became effective only in February 2010 due a combination of factors including change in leadership at the Ministry of Environment (MMA) and lack of adequate commitment on the part of MMA to activities at the sub-national level, which eventually led to cancellation of sub national level activities. The delayed start was further exacerbated by delays during the first two years (2010 and 2011) due to combination of factors including, shortage of skilled staff, departure of temporary staff and delays in replacing them, lack of managerial capacity and authority of the Project Coordination Unit (PCU) and inability to get external support, such as from the United Nations Development Program (UNDP). At closure only about 16% of the loan was disbursed although this was partly on account of the steep depreciation of the Brazilian real relative to the US Dollar during the implementation period. Efficiency Rating Modest a. If available, enter the Economic Rate of Return (ERR) and/or Financial Rate of Return (FRR) at appraisal and the re-estimated value at evaluation: Rate Available? Point value (%) *Coverage/Scope (%) 0 Appraisal 0 Not Applicable 0 ICR Estimate 0 Not Applicable * Refers to percent of total project cost for which ERR/FRR was calculated. 6. Outcome Relevance of the PDO to the government and Bank strategy for Brazil was rated as High. Relevance of design was rated as Modest. Although the causal links between the project activities and their outputs was clear, the PDO was too general and did not clearly define what would be achieved even in qualitative terms and the key outcome indicators provided no way of verifying improvements in environmental management. Efficacy of the two sub-objectives - to contribute to strengthen capacity in the Borrower's key institutions, to enhance the environmental management capacity of institutions at the federal, state, Federal District and municipal levels and to demonstrate effectiveness of targeted of targeted partnerships focused on defined priorities, was rated as Modest as all indicators were output-oriented. Efficiency was rated as Modest in view of the administrative Page 8 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) and operational shortcomings. Taking these factors into account, the overall outcome rating was rated as Unsatisfactory reflecting major shortcomings in the project's achievement of its objective. a. Outcome Rating Unsatisfactory 7. Rationale for Risk to Development Outcome Rating Government Commitment: Although the training provided to the staff of the federal agencies could be expected to improve the transparency and predictability of environmental licensing of the federal agencies, it is not clear if the environmental management capacities were strengthened, given that none of the activities associated with the sub-national entities were implemented, due to the lack of commitment of government leadership towards decentralization of environmental management. Given that improvement and expansion of capacity in information technology require regular maintenance as well as replacement investments, it is not clear if there would be adequate budget allocations for such activities. a. Risk to Development Outcome Rating Substantial 8. Assessment of Bank Performance a. Quality-at-Entry Preparation benefitted from lessons learned from implementing the Phase 1 of this project (Brazil: Second National Environmental Project. Phase 1) which had contributed to significant achievements such as the consolidation of the National Environmental System (SISNAMA) and concrete environmental gains in specific basins. The main lessons incorporated in the design of this project included: (I) Retaining the decentralized approach to environmental management which was successful in Phase 1: (ii) Given that the phase 1 had shown that states were willing to have their performance evaluated objectively, especially when financial incentives were provided for state-defined partnerships, this aspect of design was retained for this project: (iii) a communications component was incorporated: and, (iv). In phase 1, the Ministry of Environment (MMA) had used the United Nations Development Program (UNDP) for procurement related matters. This had resulted in delays during implementation, due to the cumbersome administrative procedures of UNDP. Given this and in view of the weak capacity in MMA, UNDP was still retained, although MMA was required to demand performance improvements from UNDP for this project. Appropriate arrangements were made at appraisal for safeguards compliance (discussed in Section 11). However,there were several shortcomings at Quality-at-Entry. (i) As indicated in Section 3b, the PDO was vague and the intended outcomes were not clear and the key Page 9 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) outcome indicators provided no way of verifying improvements in environmental management. The project was complex, particularly with respect to arrangements of transferring funds to sub-national entities. The assumption that the Ministry of Environment (MMA) would be willing to promote the participation of sub- national entities as partners and stakeholders in the Brazilian National Environment System (SISNAMA) proved to be unrealistic. It is also not clear if the project was based on a systematic diagnostic of actual institutional capacities, given that there was no concrete plan or proposals for using loan funds for implementing licensing activities even at the federal level. (ii) Although the risk associated with the weak management and administrative capacity within MMA was identified, the design underestimated the risk associated with contracting UNDP and the ability of MMA to demand performance improvements from UNDP. The project underestimated the risk associated with non participation by the states. (iii). There were significant drawbacks in M&E design (discussed in Section 10). Quality-at-Entry Rating Unsatisfactory b. Quality of supervision Support provided by the Bank's Financial Management (FM) specialist aided in improving FM (discussed in Section 11). There were three task team leaders during the life of the project. Of this, two were based in Brasilia and this aided in maintaining close contact with the Project Coordination Unit (PCU) and the Ministry of Environment (MMA) during project implementation. The supervision team proposed actions for simplifying implementation. For instance, when it became clear that MMA did not have the capacity or interest to continue working at the state level, the team simplified implementation by focusing only on activities at the level of the federal agencies. There were significant shortcomings. The disconnect between the PDO and outcome indicators and implementation challenges associated with implementing activities at the state level were not addressed during project implementation. The Mid-Term Review was held just about three months before the project closing date. A timely Mid-Term Review would have helped in addressing the implementation challenges. Quality of Supervision Rating Moderately Unsatisfactory Overall Bank Performance Rating Unsatisfactory 9. Assessment of Borrower Performance a. Government Performance The government's commitment to the project towards decentralization in environmental management waned following the change in leadership at the Ministry of Environment (MMA) during the last stages of project preparation, appraisal and negotiation in 2008 and 2009. The lack of strong government support continued during the early stages of implementation in 2010 and 2011. Following this change, the MMA Page 10 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) was only committed to supporting the federal licensing process under the Brazilian Institute for the Environment and Renewable Natural Resources (IBAMA). MMA did not adequately staff the Project Coordination Unit (PCU) and this in combination with failure to establish a Supervisor Committee to provide strategic guidance left the PCU with limited resources to manage a complex project, with multiple federal and state agencies. Although prior experience with the United Nations Development Program (UNDP) with the Phase 1 of this project was not positive, the organization was retained to support procurement support to this project and this decision to retain the UNDP was finally discarded only in 2011 by MMA. The MMA's commitment was greater in the later years of the project during 2014 and 2016 and this helped strengthening - the Brazilian Institute for the Environment and Renewable Natural Resources (IBAMA) - the federal agency responsible for licensing large-scale infrastructure investments. Government Performance Rating Unsatisfactory b. Implementing Agency Performance The Ministry of Environment (MMA) was the main implementing agency. MMA was also helped by IBAMA, the Chico Mendes Institute for Biodiversity (ICMBio), the federal Park Service and agency concerned with conservation of biodiversity. The Project Coordination Unit (PCU) was located in the premises of the Executive Secretariat (SECEX) of the MMA and this aided in coordination with other areas within the ministry and in engagement with the states. The PCU staff consisted of ministry staff selected by competitive examination and many of the PCU staff had participated in preparing and implementing Phase 1 activities associated with managing and supervising 43 partnerships nationwide. Most of the procurement and financial management staff of the Phase 1 project were also retained for this project. There were delays in preparation of concrete proposals for funding under the licensing component of the project. Both IBAMA and ICMBio became proactive only after 2014 and this aided in completion of the reduced scope of project activities. Implementing Agency Performance Rating Moderately Unsatisfactory Overall Borrower Performance Rating Unsatisfactory 10. M&E Design, Implementation, & Utilization a. M&E Design The two key outcome indicators - to measure enhanced environmental management capacity of state institutions and increased capacity for environmental management at federal, state, Federal district and municipal level - were not clear as they did not define the expected outcome of enhanced environmental capacity. It is not clear how strengthened and enhanced environmental management capacity was to be measured. Further, there were no targets. Page 11 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) The intermediate outcome indicators were mainly oriented towards actions to be taken by the states, not by federal agencies. Data for monitoring performance was to be collected through methods including field visits, interviews with beneficiaries, periodic quality testing and monitoring on areas subject to project intervention, measurement of levels of social inclusion and participation and partnership sustainability. The ICR provides no evidence of whether the quality or representativeness of these data were adequate. b. M&E Implementation The PCU did not have systematic monitoring and evaluation arrangements. Since most intermediate indicators were oriented towards state level activities, they were not relevant following the cancellation of all state level activities. The baseline and targets for PDO that were not set at design were not modified during implementation. c. M&E Utilization There was no use made of any monitoring results (ICR, page 9). M&E Quality Rating Negligible 11. Other Issues a. Safeguards The project was classified as a Category B project for environmental purposes. Other than Environmental Assessment (OP/BP 4.01), two safeguard policies were triggered: Involuntary Resettlement (OP/BP 4.12): and, Indigenous Peoples (OP/BP 4.10). The PAD (page 83) notes that the project activities, which aimed at improving environmental quality and conservation and sustainable use of natural resources, were not expected to have adverse environmental impacts. Since specific investments that were to be undertaken under the Environmental Assets Component were not known at appraisal, an Environmental and Social Impact Framework (ESIF) was prepared to assess the social and environmental impact of partnerships (PAD, 25). An indigenous Peoples Framework (IPF) was also prepared as a requirement of appraisal (ICR, page 9). The ICR (page 9) notes that none of the activities approved or executed under the project had social or environmental impacts that would have led to the application of the frameworks, and compliance with safeguards was deemed to be satisfactory throughout the project life. Page 12 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) b. Fiduciary Compliance Financial Management. The implementing agency - the Ministry of Environment (MMA) - had prior experience with Bank's financial management requirements having already implemented the first phase of this project. An assessment of the financial management capacity of the implementing agency conducted at appraisal concluded financial risk was Moderate (PAD, page 22) and a financial management action plan was prepared at appraisal (PAD, page 64). The ICR (page 10) notes that financial management shortcomings were mainly due to instability at the Borrower level, which contributed to delays in finalizing the project reporting system, a need to strengthen internal control arrangements and a slow disbursement rate in the initial years of the project. During supervision missions, the Bank Financial Management (FM) Specialist provided support and this helped in improving FM in the latter years of the project. Audit reports were generally received in a timely fashion and unqualified, with the exception of the first audit report, which was qualified because of payments to services not delivered. This issue of ineligible expenses was properly addressed during implementation and the remaining audit reports were unqualified. Given the reduction in the scope of project activities, the funds that were not used were refunded (ICR, page 10). Procurement. The Project Coordination Unit's (PCU) procurement of goods and services were to be processed by the United Nations Development Program (UNDP) through a technical cooperation agreement. A capacity assessment was conducted of UNDP at appraisal. The ex ante procurement risk was rated as High as the capacities of the other federal agencies, states, Federal District and municipalities were not known and were to be assessed on a case-by-case basis (ICR, page 78). The ICR (page 10) notes that there were major procurement shortcomings, both in the Ministry of Environment (MMA) and the involved federal agencies. These were primarily due to a combination of factors including, the low capacity of the implementing agencies and delays in decision making from the Borrower's side in identifying and giving priority to procurement activities. This was exacerbated by the lack of a suitable monitoring system to support and supervise procurement transactions and contract management of other involved agencies such as the Chico Mendes Institute for Biodiversity (ICMBio) and the Brazilian Institute for the Environment and Renewable Natural Resources (IBAMA during implementation. c. Unintended impacts (Positive or Negative) --- d. Other --- 12. Ratings Reason for Ratings ICR IEG Disagreements/Comment Outcome Unsatisfactory Unsatisfactory --- Risk to Development Modest Substantial There is substantial risk Page 13 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) Outcome associated with government's commitment to decentralization in environmental management. Bank Performance Unsatisfactory Unsatisfactory --- Borrower Performance Unsatisfactory Unsatisfactory --- Quality of ICR Substantial --- Note When insufficient information is provided by the Bank for IEG to arrive at a clear rating, IEG will downgrade the relevant ratings as warranted beginning July 1, 2006. The "Reason for Disagreement/Comments" column could cross-reference other sections of the ICR Review, as appropriate. 13. Lessons The following lessons are taken from the ICR with some adaptation of language. (1) Client ownership needs to be accurately assessed at each stage in project preparation. The case of this project showed that the political and institutional context may change from project preparation through appraisal, and this may entail changes in priorities and interests. (2) Institutional strengthening activities should be based on addressing specified, identified problems with specific measures at project start. In the case of this project, project preparation did not adequately analyze the weaknesses in federal and state agencies. Activities were left to be proposed by the agencies after approval and this resulted in delays associated with preparing sub-projects. (3) An adequate M&E and Results Framework is required. Even if the PDO was broad as in the case of this project, it would have helped if the PDO was complemented with specific and more concrete special activities and verifiable results indicators. 14. Assessment Recommended? No 15. Comments on Quality of ICR The ICR provides a detailed overview of the project and is for the most part, well-written. The narrative supports the ratings and available evidence. It is candid in discussing the issues that arose in the wake of the government's waning commitment to state level activities. The ICR generally follows the guidelines adequately. The quality of evidence is aligned to the messages. The discussion of M&E implementation is very brief and it would have to helped to provide more description of the issues that arose during M&E implementation. Page 14 of 15 Independent Evaluation Group (IEG) Implementation Completion Report (ICR) Review BR (APL2) 2nd National Environmental (P099469) a. Quality of ICR Rating Substantial Page 15 of 15