78127 APRIL 2013 A PROFOR WORKING PAPER WORKING PAPER Approaches to Measuring the Conservation Impact of Forest Management Certification ACKNOWLEDGMENTS This paper by Claudia Romero (Senior Associate, CIFOR) and Tuukka Castrén (Senior Forestry Specialist, World Bank/PROFOR) was produced based on extensive and inspiring discussions in two scientific workshops on forest certification and its impacts. These workshops were held in 2011 in Virginia, U.S., and Montpellier, France, on impact evaluation and SFM certification. The workshops were organized by University of Florida, Center for International Forestry Research (CIFOR) and Agricultural Research Center for Development (CIRAD) with support from PROFOR. Workshop participants included researchers with diverse backgrounds, interests, and experience. The authors are greatly indebted to all participants and contributors in the workshops. Without their views and inspiration, this paper would never have been possible. Special thanks go to Professor Jack Putz (University of Florida), Marko Katila (Indufor Oy), and World Bank colleagues Peter Dewees (Forests Advisor), Nalin Kishor (Senior Natural Resource Economist), Jeff Alumai (Junior Professional Associate), Andrew Mitchell (Senior Forestry Specialist), and Simon Reitbergen (Senior Forestry Specialist) for their valuable comments. Flore de Préneuf assisted with the layout and finalization of the paper. DISCLAIMER Despite the valuable comments received from a large number of people, all remaining omissions and mistakes are those of the authors. The opinions presented in the paper do not necessarily reflect those of PROFOR, the World Bank, the University of Florida, or the workshop sponsors and participants. Suggested citation: Claudia Romero and Castrén Tuukka. 2013. Approaches to Measuring the Conservation Impact of Forest Management Certification. Working Paper. Washington D.C: Program on Forests (PROFOR). Published in April 2013 For a full list of publications please contact: Program on Forests (PROFOR) 1818 H Street, NW Washington, DC 20433, USA profor@worldbank.org www.profor.info/knowledge PROFOR is a multi-donor partnership supported by the European Commission, Finland, Germany, Italy, Japan, the Netherlands, Switzerland, the United Kingdom and the World Bank.: Learn more at www.profor.info 2 Measuring the Conservation Impact of Certification EXECUTIVE SUMMARY Introduction Sustainable forest management (SFM) certification emerged in the 1980s and 1990s as a mechanism to promote responsible forest use and as an alternative to boycotts of forest products amid growing concerns about forest degradation and destruction. Since then, forest certification has evolved into a multifaceted market-based mechanism to promote compliance with sets of ecological, social, and economic criteria to enhance sustainability. Commodity certification has evolved from its origins as a means of verifying organic and environmentally sustainable production; issues like social equity, transparency, participation, and legal compliance have become increasingly relevant. One commonality in all certification schemes is that they are voluntary, market-driven (“willing buyer – willing seller�) schemes aimed at transformational change toward more sustainable production and consumption patterns within existing market structures. Impacts of certification can be assessed through different lenses serving diverse purposes: producers are interested in ensuring their market access and price premia, long-term sustainability of production, and stable operating environments. Consumers, by contrast, are interested in social and environmental outcomes. Consumers also have much less information on individual operations than the producer, and therefore benefit from independent third-party verification such as certification. Additionally, the financing sector and investors (for example, pension funds) as well as investment banks and managers often use certification as an environmental, social, and corporate governance tool. Certification—or lack of it—guides financing and investment flows but to what extent is not fully known. This document presents the state of the current knowledge on how to assess impacts of forest management certification. It also discusses the design, implementation, and use of forest management certification. It focuses on methodologies to provide evidence-based information on the environmental impacts of certification. The concluding chapter briefly discusses the economic and social impacts. The objective is to identify areas where further methodological work is needed to improve understanding on the impacts of certification. Many benefits of certification, like improved information on management practices by outside stakeholders (for example, consumers, governments) are undisputed. At the same time, there is less knowledge on whether or not practices at field level have changed and how much. Although improved information as such is a valuable outcome, more quantitative information on environmental impacts would be welcome. Certification as a Conservation Intervention Forest certification has been justified on several grounds, but often the best known objective of certification is enhancing the environmental sustainability of forest management. This is also a common goal in all certification schemes. If we look at forest certification as a site specific forest management unit (FMU) –level conservation 3 Measuring the Conservation Impact of Certification intervention, impact assessments should look at when, where, how, to what extent, and at what cost certification has changed the forests. Despite the extensive literature discussing the nature and objectives of certification, surprisingly little empirical field-based research clearly identifies the impacts of SFM certification. One specific constraint on evaluation derives from the long-term scope of conservation interventions and their often vague or hard-to-measure objectives (for example, biodiversity conservation, ecosystem service maintenance). This vagueness renders it difficult to identify specific elements to assess. There is also a lack of trained personnel grounded in both conservation sciences and the nascent but rapidly advancing field of environmental impact evaluation. Current Status of Certification The two largest international certification bodies are the Programme for the Endorsement of Forest Certification (PEFC) and Forest Stewardship Council (FSC). PEFC, created in 1999, operates through endorsement of national schemes. FSC, established in 1993, has also endorsed national standards and develops generic indicators. As of December 2012, PEFC covers globally 242 million hectares (ha) and FSC covers 146 million ha. For both schemes, the vast majority of certified forests (more than 80 percent) are found in Europe and North America. The reasons an FMU decides to seek management certification are diverse and vary by region and time. FMUs that opt for certification might expect to secure and enhance their market access. Another expectation is to obtain improved prices (that is, price premium), but this benefit usually has not been achieved except in particular contextual factors. Finally, FMUs may expect that over time the production costs will be reduced as their forest management operations become more efficient. Other benefits from certification are enhanced learning and transparency, increased public confidence and social acceptance, social improvement, and greater environmental responsibility. Research on the real and perceived barriers to certification stresses both its direct and indirect costs. Among the direct costs are the knowledge needs and preparation for certification, costs of improved management practices, equipment, staff training and additional salaries, audit costs, membership fees, monitoring and record keeping, and consultation processes. The indirect costs include forgone or reduced volumes because of the protection of riparian buffer zones and other set asides, as well as increased duration of rotations. Background on Evaluation of Conservation Interventions Conservation interventions are designed to promote long-term impacts and, as such, should be reassessed and revised as prescribed by the framework of adaptive management. The interventions take place in complex realities in which direct linear responses are unlikely and where numerous processes act at different scales in both time and space. Steps that have been proposed to deal with this complexity include (1) detailed disaggregation of components and dynamics in conservation interventions; (2) identification of short and long-term outcomes; (3) scenario building under alternative policy environments to assess changes and impacts; and (4) building of multistakeholder dialogues to involve all key actors in planning, management, credibility-building, and evaluative actions. The indirect effects of conservation interventions are particularly hard to capture, but need to be understood if the overall efficiency and effectiveness of forest management certification are to be assessed. Although there is an extensive literature on evaluation studies in conservation, only a handful of studies have produced evidence-based statements that can be clearly attributable to the intervention. One particular reason is that conservation interventions are seldom implemented in a random manner. The resulting selection bias obscures insights that could be derived from random allocation of individuals to treatments (receiving or not the intervention), as in well-designed experiments. 4 Measuring the Conservation Impact of Certification Approaches other than experimental and quasi-experimental can be qualitative but should still be based in a comparative method to become relevant for evaluating impacts of conservation interventions. These include before- and-after comparisons (between control and treated groups), systematic case studies, and expert judgment. For evaluation of the impacts of forest management certification, it seems most appropriate to employ quantitative methods designed to reveal causal connections between project activities and conservation outcomes along with complementary qualitative approaches that can capture more of the indirect impacts. General Considerations on Forest Management Certification Evaluation An evaluation of the impacts of forest management certification must take into account that, even in the absence of certification, management practices vary widely among FMUs. The certification process involves many steps typically made over several years. Getting and remaining certified requires substantial time and continued investments. Even well-managed FMUs may need to change some practices to become certified. FMUs are located along a continuum, from those with no interest in certification to those that have been certified through several rounds of audits. Given that at different points in time an FMU might be found at different locations along this “certification continuum,� it seems wrong, or at least naïve, to evaluate the impacts of certification solely by comparing certified and uncertified forests. Even among certified FMUs, there is variation in the periods over which they have remained certified, as well as in the number of deficiencies identified by auditors as corrective action requests (CARs), which can be minor (conditions) or major (preconditions). The strategies for advancing understanding of the impacts of forest management certification need to consider that certification is applied in diverse places with differing features. This affects the outcome of the intervention and the way those outcomes change over time (for example, governance, legal frameworks, social issues, market structure and preferences, technology, access to capital, and forest management practices). Proper understanding of these contextual factors will facilitate the design of evaluation of forest management certification. It is important to be explicit about the questions that can be addressed by an evaluation. This determination requires understanding of how the intervention operates and intends to drive change under different dynamic scenarios (theory or model of change), and the identification of financial, knowledge, and technological barriers to its implementation. Forest management certification considers two main mechanisms to produce changes in forest management outcomes. The first is through the adjustment of management practices to comply with predefined standards of each certification scheme (for example, FSC, PEFC), and the second refers to the processes of auditing to verify that compliance has occurred. None of the existing forest certification systems has an explicit theory of change that describes the change process. An evaluation aims to provide a detailed examination of the ecological, social, economic, and political impacts of the intervention compared with situations where there was no intervention, or with alternative options to reach the same goal. For example, in forestry, alternative options could be legislative changes. Besides addressing whether the goals or objectives of the intervention are being achieved, evaluation aims to establish the extent to which the outcomes are due to the intervention (attribution). For forest management certification, the pertinent questions are (1) the extent to which certification is the direct or indirect driver in the forest management (sensu lato) outcomes, and (2) the causal pathways for these changes. Thus, formalizing how the change process occurs (theory of change) becomes relevant, as is the identification of factors that affect decisions related to the change process, and how they do so. 5 Measuring the Conservation Impact of Certification Given that methodology for evaluation of conservation interventions is still in its early development, a creative combination of tools should be encouraged. Different qualitative and quantitative approaches are necessary to understand and integrate different kinds of knowledge on causal complexity, to identify integrated direct and indirect impacts, as well as to attribute correctly the outcomes of certification (for example, through building of comparison groups). Current Knowledge: Types of Assessments of Forest Management Certification With few exceptions, attempts at assessing the environmental impact from forest certification have been based on examination of secondary information and stakeholder perceptions. The most common explanations for the lack of on-the-ground assessments are methodological challenges and funding restrictions. To date, none of the approaches reviewed below can be fully seen as comprehensive evaluations but, nevertheless, they provide some useful information. 1. Analysis of Temporal Changes in Corrective Action Requests One approach that has been used to evaluate the impacts of forest management certification employs a one-time analysis of how reports produced by forest auditors change over time. Lists of initial CARs are used to estimate how far the FMU was from the standards of the certification scheme against which it was audited. Assessments of forest certification based on the evolution of CARs all report that the number of issues initially of concern to auditors decreased, indirectly indicating a positive impact of forest management certification even if a field inspection was not performed to verify whether the changes were made. Results of these approaches should be interpreted with caution and the use of changes in CARs as a surrogate for more direct measures of the impacts of forest management certification has numerous limitations. 2. Literature Reviews Literature reviews on the impacts of forest management certification have mostly centered on particular issues (for example, biodiversity, wildlife, local communities) and regions. Typically, few specific aspects of management are assessed (for example, riparian buffer zones, tree retention). Diverse approaches have been employed to determine changes related to forest management certification, with some studies using research results comparing changes before and after certification, and others looking at certified versus noncertified operations. Other studies have looked at impacts on different aspects of natural resource management (for example, community forestry and enterprises, biodiversity). 3. Interviews with Participants in the Certification Process The impacts of forest management certification as perceived by FMUs, the timber industry, local communities, and buyers have been compiled at the global and regional levels. The impressions of both supporters and critics of forest certification were solicited and most contributors were positive about its impacts. These assessments often have concluded that certification has contributed more to improve tropical forestry than any other initiative, even if it also has stressed the need to generate on-the-ground evidence to demonstrate impacts. 4. Reviewing Single Management Practices One approach is to assess the effects of best-management practices (BMPs) on resource management by reviewing well-designed and executed field studies that evaluated the effects of one specific requirement of certification: for 6 Measuring the Conservation Impact of Certification instance, the establishment of set-asides including riparian buffer zones. Although this approach is based on sound and well-designed research, it does not look at the integral effect of the certification intervention as a whole. This approach will miss any indirect effects of certification. A model of change is not considered as part of this approach and this deficiency precludes an overall understanding of the impacts of certification. The positive side of this method is that unbundling the practices of the complex forest management certification intervention could help explain particularities related to contextual factors, clarify causal mechanisms, and thereby facilitate their assessment. 5. Additional analytical steps in assessing forest certification impact It is clear from the literature review that there are uncertainties on how to evaluate the impact of forest management certification and its contribution to maintaining forest values. At the same time, some issues have not been adequately included in the assessments and should be figured into alternative approaches. The following three possible approaches have been identified: - Promote a better understanding of the structure of the forest sector (for example, types of FMUs and their practices) and how it evolves as a function of the effects of factors external to forest management certification; - Develop a more specific list of the expected outcomes of certification and identify the potential pathways through which these outcomes are achieved; and - Propose an explicit model on how the change in forest management practices associated with certification will come about (“theory of change�), and how to adapt the model of change to the characteristics of each region/country. The Way Forward Several development practitioners have promoted SFM certification in their forest sector operations. These have included multilateral and bilateral donors as well as civil society organizations. Certification has also been promoted in community forestry where it is often seen as a way to both improve market access and ensure the high quality of forest management and extraction. Certification has also been promoted by national forest authorities where forest policies have been reformed, and SFM certification, despite originally being a voluntary market-based instrument, has gained traction also as a law enforcement instrument. This demonstrates that certification is often seen as an essential tool in ensuring that forest management is both environmentally and socially sustainable. At the same time, it is evident that there has not been extensive analysis on the impacts of certification nor evaluation of certification as a policy instrument. The positive impacts of certification are taken for granted and the success of certification is measured by how widely it has been applied. Particularly if certification is supported by donor agencies as part of a larger intervention, the impact of certification alone is hard—or even unnecessary—to establish with scientific precision. Often, the projects have received considerable technical assistance for management, marketing, and even infrastructure development in addition to support for certification, and it is difficult to say how much of the benefits are due to donor support and how much to SFM certification alone. How should forest practitioners view certification? Even if there is not extensive scientific research to confirm the positive, transformational change that has been attributed to forest management certification, it does not mean that investments in forest certification have not provided any benefits. 7 Measuring the Conservation Impact of Certification Despite the methodological uncertainties with the evaluation of forest management certification, this report does not propose that domestic or international support for it should be discontinued. There is not adequate research to state that certification has led to transformational change in forest management at the field level or that forest outcomes would directly change once an FMU seeks or is granted a forest management certification. At the same time, there is no reason to assume that certified forests would not be well managed. Forest certification can be an acceptable proxy for good forest management. Certifying an FMU demonstrates that key principles of good management are adhered to and standards are followed. Certification can be seen as a sufficient but not necessary condition for demonstrated good forest management. Certification is also finding new uses and audiences. For example, there has been an ongoing discussion of whether SFM and chain-of-custody certification could be risk management tools for the financial sector when it makes investments in wood production and processing. As a tool, it could be used to mitigate both reputational and business risks with increased legality and sustainability requirements in global trade and public procurement. Certification, despite all its limitations, is currently the only globally recognized system of understanding and communicating how well forests are managed. Even so, only a small percentage of the world’s forests are currently certified, certification does not measure change over time, and even the concept of SFM is interpreted differently by various stakeholders. National governments and international donors have diverse objectives when it comes to forest policies and operations. Some key recommendations and conclusions can be made: i. It is essential to use systematic analyses when making decisions on forest management and practices. These include better understanding of the complex dynamics and theory of change of forest sector interventions as well as on the ultimate objectives of forest management. ii. Despite not having full understanding of all the dimensions and impacts of forest management certification, it can be assumed that certified forests are, on average, most likely better managed than noncertified forests. Certification also ensures to the public that sustainability of forest management is promoted. iii. This report is an output from of a wider program to research the on-the-ground impacts of forest management certification. Implementation of the research will require extensive support as well as collaboration and sharing of experiences by a wide range of stakeholders and institutions. Forests provide several benefits. Forest management certification mainly deals with the productive functions of the forests and with ensuring that these functions can create economic goods without jeopardizing social benefits and environmental sustainability. National governments and donor agencies should continue their support for well- managed sustainable production forestry, and forest certification is one instrument for achieving it. It needs to be emphasized that methodological challenges in measuring the impacts cannot be interpreted as lack of impact. Forest certification increases the information available in the marketplace for all participants, and this improved information increases confidence in certified producers. Having better and scientifically verified information on the impact would improve the information base even further and could also be used to improve the design of certification schemes. 8 Measuring the Conservation Impact of Certification Abbreviations BMP best-management practice CAR corrective action requests CERFLOR Forest Certification Program, Brazil CERTFOR Chilean System for Sustainable Forest Management Certification CIFOR Center for International Forestry Research CIRAD Agricultural Research Center for Development EU European Union FAO United Nations Food and Agriculture Organization FLEGT Forest Law Enforcement, Governance and Trade FMU forest management unit FSC Forest Stewardship Council MTTC Malaysia Timber Certification Council PEFC Programme for the Endorsement of Forest Certification PES Payments for Environmental Services PROFOR Program on Forests – The World Bank REDD+ Reducing Emissions from Deforestation and Tropical Forest Degradation, including conservation, sustainable management of forests and enhancement of forest carbon stocks SFM sustainable forest management 9 Measuring the Conservation Impact of Certification TABLE OF CONTENTS EXECUTIVE SUMMARY 3 1. Introduction 12 1.1 Background and objectives 12 1.2 Certification as a conservation intervention 14 2. Current Status of Certification 15 2.1 Diversity of Forest Management Certification Schemes 15 2.2 Motivation to Seek Forest Management Certification 16 3. Background on Evaluation of Conservation Interventions 17 4. General Considerations on Forest Management Certification Evaluation 18 4.1 Variations in Forest Management Practices 18 4.2 Certification Continuum 19 4.3 Importance of Context 21 4.4 Evaluation of Impacts of Forest Management Certification 22 Implementation of the Process of Change in Forest Management 22 Detecting and Understanding Impacts 22 5. Current Knowledge: Types of Assessments of Forest Management Certification 25 5.1 Analysis of Temporal Changes in Corrective Action Requests 25 5.2 Literature Reviews 25 5.3 Interviews with Participants in the Certification Process 26 5.4. Reviewing Single Management Practices 26 5.5 Additional Analytical Steps in Assessing Forest Certification Impact 27 5.6 Dynamics along the Forest Management and Certification Continuum 30 6. The Way Forward 32 7. References 36 10 Measuring the Conservation Impact of Certification List of tables, figures and boxes Table 1. Current Coverage of the Main International Certification Schemes by Region, Plantation, and Natural Forests 15 Figure 1. Certified Area under FSC and PEFC Schemes (1995–2012) 16 Figure 2. Forest Management as a Continuum 19 Figure 3. Certification Continuum. 21 Table 2. Potential Approaches to Assess Impacts of Forest Management Certification 24 Table 3. Variables Influencing the Short- and Long-Term Outcomes of Forest Management and the Impacts of Certification 27 Table 4. Outcome Areas and Outcome Statements Expected after Certification 28 Figure 4. Example of Impact Pathways 29 Box 1. Dynamics of Forest Sector Based on Expert Knowledge 31 Box 2. Trade Pattern in Certified Bananas 34 11 Measuring the Conservation Impact of Certification 1. Introduction Sustainable forest management (SFM) certification emerged in the 1980s and 1990s as a mechanism to promote responsible forest use and as an alternative to boycotts of forest products amid growing concerns about forest degradation and destruction. Since then, forest certification has evolved into a multifaceted market-based mechanism to promote compliance with sets of ecological, social, and economic criteria to enhance sustainability. Promoters of certification often view it as a tool for fostering particularly social and environmental improvements in the forest sector through improved management practices and consumer awareness. These supporters include such diverse interest groups as environmental groups, forest industries, forest owners, donors, banks, consumers, and government officials; many of whom must be wondering whether forest management certification has delivered the promised ecological, social, and economic benefits.1 1.1 Background and objectives Forestry is one of many areas where products are being certified. It was not even the first; different types of sustainable production criteria have been issued since the 1940s, when the first guidelines and principles for organic farming were established in Australia. This process led to the establishment of the International Federation of Organic Agriculture Movements (IFOAM) (Paull 2010), which has been setting the standards for organic production. The standards first covered environmental issues and organic production. For example, the first certification for organic coffee production was issued in 1967 to producers in Chiapas (Mexico). Social issues and criteria were introduced only later. This led also to the certification and labeling of fair trade products imported from developing countries (Auld 2010). Commodity certification has evolved from its origins as verification of organic and environmentally sustainable production. Issues like social equity, transparency, participation, and legal compliance have become increasingly relevant. This is demonstrated by certification of extractive minerals which, by definition, cannot be produced in a sustainable way. For example, the Kimberly Process Certification Scheme (KPCS) is a voluntary certification scheme for diamonds. It is not entirely unlike forest certification schemes, but it was established to serve different purposes. Its main function was to verify legal origin and ensure that consumers would not be contributing to civil wars through purchase of “blood diamonds� (Haufler 2010). Another extractive mineral certification scheme is the one for tanzanite, which suffered from allegations that its mining and trade was linked to terrorism financing (Schroeder 2010). These certification examples for such diverse products as timber, coffee, and precious stones demonstrate that certification can have different drivers and ultimate objectives. One commonality in all certification schemes is that they are voluntary and market driven (“willing buyer – willing seller�), aiming at transformational change toward more sustainable production and consumption patterns within existing market structures (Gandenberger et al. 2011). SFM certification requires a process to set the sustainability criteria and indicators. This process has not been the exclusive mission of certification schemes. Intergovernmental platforms have also worked on defining sustainability frameworks. The Montréal process, for instance, defined internationally agreed sustainability criteria that included 12 countries, accounting for 83 percent of the world’s temperate forests and 49 percent of all forests.2 1 For general references on forest management certification, see for example Elliott (2000); Viana et al. (1996); Cashore et al. (2004); Auld et al. (2008); Cashore and Auld (2012); Vogt et al. (2000). 2 Montreal Process (2009). The countries are Argentina, Canada, Chile, China, Japan, Korea, Mexico, New Zealand, the Russian Federation, the United States, and Uruguay. 12 Measuring the Conservation Impact of Certification Impacts of certification can be assessed through different lenses serving diverse purposes: producers are interested in ensuring market access and price premia, long-term sustainability of production, and a stable operating environment (for example, community conflicts are minimized) (Steering Committee Report 2012). Consumers, by contrast, are interested in social and environmental outcomes. Consumers also have much less information on individual operations than the producer and therefore benefit from independent third-party verification such as certification. However, producers could achieve a number of their objectives even if a certificate as such is not issued, as long as management practices are good enough so that they could be certified if needed. Additionally, the financing sector and investors (for example, pension funds) as well as investment banks and managers commonly use certification as an environmental, social, and corporate governance tool. Certification, or lack of it, guides financing and investment flows, though its extent is not fully known. Looking at market structures from the perspective of both producers and consumers, as well as considering the diverse objectives of certification schemes (for example, environmental sustainability, social justice and equity, reforming market structures), complicates the way that the impact of forest certification or any other certification scheme can be assessed. Impact assessment is further complicated by the possibility that certification leads to positive spillover effects; even nonparticipant FMUs improve their performance after being exposed to the feasibility of sustainable practices.3 This document presents the state of the current knowledge on the impacts of forest management certification.4 It discusses the design, implementation, and use of forest management certification, as well as attempts at understanding these impacts in more detail than we currently can implement in practice. It focuses on approaches to provide evidence-based information on the environmental impacts of certification. The economic and social impacts are discussed in the concluding chapter. The objective is to use a literature survey to identify areas where further methodological work is needed to improve understanding of the impacts of certification. The discussion addresses various methodological options to measure impacts but does not aim to provide a final assessment of the impacts. Forest certification has become an established private SFM standard. This paper aims to better understand what various stakeholders—forest managers, industry, consumers and governments5—could expect from forest certification and where additional methodological development is needed. Many benefits of certification, like improved information on management practices by outside stakeholders (for example, consumers, governments), are undisputed. At the same time, there is less knowledge of whether field practices have changed and how much. Although improved information as such is a valuable outcome, more quantitative information on environmental impacts seems necessary given the broad support that certification has received from a range of stakeholders. Chapters 1 and 2 discuss forest certification as an instrument, while chapter 3 discusses general issues in the assessment and evaluation of interventions in environmental conservation. Chapter 4 discusses particular characteristics of forest certification evaluation. Chapter 5 shows various approaches to the issue. Chapter 6 summarizes the discussion, and makes linkages to other social and economic aspects of sustainability, and discusses the overall importance of forest certification. 3 This possibility is discussed, but not fully demonstrated. See, for example, Gandenberger et al. (2011) and Auld (2010). 4 A recent multi-institutional effort led to the publication of an insightful document (Steering Committee of the State-of-Knowledge Assessment of Standards and Certification 2012) covering a range of products where certification has had a prominent role in promoting sustainability. 5 In international development, also host country and donor country governments. 13 Measuring the Conservation Impact of Certification 1.2 Certification as a conservation intervention Forest certification has been justified on several grounds and has some linkages to the fair trade movement and other market-based instruments that aim at socially equitable outcomes. The best-known objective of certification is enhancing the environmental sustainability of forest management. This is also a common goal in all certification schemes. In analyzing the conservation impact of SFM certification, it can be compared to the impact assessment of protected areas and other interventions aimed at biodiversity conservation and maintaining ecosystem services. If we look at forest certification as a site-specific FMU-level conservation intervention, impact assessments should look at when, where, how, to what extent, and at what cost certification has changed forests. Despite the extensive literature discussing the nature and objectives of certification, surprisingly little empirical field-based research clearly identifies the impacts of SFM certification. This scarcity may have been caused by the preexisting assumption that certification is environmentally, economically, and socially beneficial; the cost of such a study; and the methodological challenges in evaluating the direct and indirect impacts of complex intervention like forest certification. Additional constraints on evaluation derive from the long-term scope of conservation interventions and their often vague or hard-to-measure objectives (for example, biodiversity conservation, ecosystem service maintenance). This vagueness makes it difficult to identify specific elements to assess. There is also lack of trained personnel grounded in both conservation sciences and the nascent but rapidly advancing field of environmental impact evaluation. To better understand the impact of certification, one must understand how certification has worked, and why, where, and when it has not. That means that the factors contributing positively and negatively to the impacts of certification also need to be assessed all the way along the forest product market chain. First, the fidelity with which the contents of the certification standards have been applied needs to be appraised. Further down the certification process, the direct and indirect costs and benefits forest managers (for example, private forest owners, communities, the state) experience in becoming certified need to be understood. The positive and negative impacts of certification in the forests themselves, in neighboring areas, and along the forest product market chain also need to be formally assessed. A key factor to consider when attempting to understand the effects of certification is the recognition of other instruments that have emerged that also affect the ways forests are managed. Recent developments in the forest policy and resource management systems have also been introduced since certification came to existence in the 1990s. These can have synergistic roles with certification in improving forest management, and work on their own to achieve maintenance of forest values. First, in many countries (or regions, like the European Union), environmental legislation and its enforcement have become tighter. Second, efforts at decentralizing resource management decisions are an example of changes that occurred within the past decades (Pacheco 2004; Agrawal 2007; Agrawal et al. 2008). Finally, forest legality has been promoted through different mechanisms. The most notable ones are the European Union (EU) Forest Law Enforcement, Governance and Trade (FLEGT) action plan and subsequent EU Timber Regulation and Voluntary Partnership Agreements with producer countries as well as the amended Lacey Act (2008),6 and similar legislation in Australia. These instruments contribute to timber market transparency, and their effects are felt all along the market chain. Incentive mechanisms to enhance forest management can add to the potential benefits of certification. Among these, the emergence of payments for environmental services (PES) for the maintenance of provision of these services (for example, water, biodiversity) addresses some aspects of market failures related to timber. The most ambitious PES 6 Information on FLEGT can be found at www.eufl.egt.int. Information on the Lacey Act can be found at www.forestlegality.org. 14 Measuring the Conservation Impact of Certification scheme that has been proposed, deals specifically with enhancing carbon sequestration in forests (for example, Reducing Emissions from Deforestation and Tropical Forest Degradation [REDD+]).7 2. Current Status of Certification 2.1 Diversity of Forest Management Certification Schemes Numerous forest management certification schemes operate at national levels. The two largest international certification bodies are PEFC and FSC. PEFC, created in 1999, focuses on endorsing national schemes.8 FSC, established in 1993, has recognized 21 national standards.9 Where there are no appropriate national standards, the certifying body develops generic indicators that then are approved by FSC, but the aim is to develop international generic indicators to enhance consistency10 (see Table 1 and Figure 1Error! Reference source not found.Error! Reference source not found.). Table 1. Current Coverage of the Main International Certification Schemes by Region, Plantation, and Natural Forests REGION PEFC (mill. Percent FSC (mill ha) Percent ha) Europe 79.5 32.8 72.9 43.1 North America 144.9 59.7 68.4 40.4 Latin America 3.2 1.4 12.8 7.3 Africa 0 0 7.2 4.4 Asia and Oceania 14.8 6.1 8.0 4.8 TOTAL 242.4 100 145.6 100 Source: FSC (December 2012) and PEFC (October 2012). One of the difficulties encountered when attempting to extract clear lessons from the literature on the impacts of forest management certification is the diversity in certification schemes. FSC explicitly includes in the standards goals related to environmental (for example, environmental impact, maintenance of high conservation value forests, Principles 6 and 9), social (for example, tenure, use rights and responsibilities, and indigenous peoples’ rights, Principles 2 and 3), and economic issues (for example, benefits from the forest, management plan, Principles 5 and 7). Other schemes have previously emphasized more biophysical issues, even if there has been convergence of approaches in the recent years. Differences among forest certification schemes have been the subject of scrutiny. Explanations of this diversity include the scope of application of the systems (for example, national, international) and differences in the biophysical and institutional characteristics of the places where these schemes have emerged. The schemes also 7 REDD+: www.un-redd.org/; www.forestcarbonpartnership.org/. 8 Such as American Tree Farm System and SFI in the United States; Canadian Standards Association; Finnish Forest Certification System; Australian Forestry Standard; CERTFOR (Chilean System for Sustainable Forest Management Certification) in Chile; INMETRO/CERFLOR (Forest Certification Program) in Brazil; and the Malaysia Timber Certification Council (MTCC) in Malaysia. 9 Africa has three, Asia-Pacific has one, Europe and Russia have ten, Latin America has five, and North America has two (January 2013). 10 http://ic.fsc.org/national-standards.247.htm. 15 Measuring the Conservation Impact of Certification vary in the extent of stakeholder participation, in the definition of standards, and in the degree to which they exceed national legal requirements. Unified frameworks to compare the schemes have been proposed.11 Figure 1. Certified Area under FSC and PEFC Schemes (1995–2012) 250 200 Certified Area (M ha) 150 PEFC 100 FSC 50 0 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 Source: PEFC (www.pefc.org) and FSC (www.fsc.org). Note: The sharp increase in area after 2004 for PEFC is due to its endorsement of SFI’s certification system. 2.2 Motivation to Seek Forest Management Certification The reasons an FMU decides to seek management certification are diverse and vary with region and over time. FMUs that opt for certification might expect to secure and enhance their market access, which has been demonstrated in several cases, especially for FMUs with a large share of production for export markets (often linked to the size of the FMU). Another expectation is to obtain improved prices (that is, price premia), but this benefit has not been usually achieved and when it has happened, it has been due to particular contextual factors (for example, community-based operations selling highly valued products in Vermont; higher increments on high timber quality in Malaysia; price increments related to marketing and management strategies in Bolivia).12 Finally, FMUs may expect that over time, the production costs will be reduced as their forest management operations become more efficient, which has been reported in studies, such as in Brazil.13 These benefits of forest certification, both public and private, often are based on individual cases and thus are not necessarily generalizable across time scales, regions, and countries. Other certification benefits are enhanced learning and transparency, increased public confidence and social acceptance, social improvement, and greater environmental responsibility.14 11 The following documents provide frameworks for comparison of certification standards: Holvoet and Muys (2003); Oliver (2004); WWF/World Bank Global Forest Alliance (2006); Tikina and Innes (2008); McDermott et al. (2008); Overdevest (2010); Johansson and Gun (2011). Some specific attempts have been made to compare the two certification frameworks to determine their contribution to forest management sustainability. Because of the diverse methods used in various papers and the lack of true replication, a robust meta-analysis has not been possible in spite of the use of both quantitative and qualitative analytical tools. As a conclusion, comparisons across schemes should be based on field data. See Clark and Kozar (2011). 12 Nebel et al. (2005) for Bolivia; Kollert and Lagan (2007) for Malaysia; Crow and Danks (2010) for Vermont; Chen et al. (2011) for Canada. Increased market access: van Kooten et al. (2005). 13 Pokorny and Steinbrenner (2005). This issue with improved management practices is often higher up-front investment costs in technical capacity and staff training, not overall profitability. 14 Vidal and Kozak (2008), Araujo et al. (2009); Auld et al. (2008); Takahashi et al. (2003); Cubbage et al. (2010). 16 Measuring the Conservation Impact of Certification Research on the real and perceived barriers to certification stresses both its direct and indirect costs. Among the direct costs are the knowledge needs and preparation for certification, costs of improved management practices, equipment, staff training and additional salaries, audit costs, membership fees, monitoring and record keeping, and consultation processes. The indirect costs revealed focus on forgone or reduced volumes because of the protection of riparian buffer zones and other set-asides, as well as longer rotations (Cubbage et al. 2003; Gan 2005; Simula et al. 2004; Chen et al. 2010). 3. Background on Evaluation of Conservation Interventions Program evaluation is a well-developed field, particularly for determining the impacts of public policies (for example, health or education). In contrast, evaluation of conservation interventions lags behind in spite of a fairly recent wave of publications on the topic. However, this field of research remains incipient and quite contested.15 Conservation interventions are designed to promote long-term impacts, and should be reassessed and revised as prescribed by the framework of adaptive management (Holling 1978). They occur in complex realities in which direct linear responses are unlikely and where numerous processes act at different scales in both time and space. In these complex social-ecological systems, multiple causal mechanisms operate simultaneously, the mechanisms are also context dependent, and have unpredictable feedback loops (recursive causality) that give rise to emergent outcomes. Steps that have been proposed to deal with this complexity include (1) detailed disaggregation of components and dynamics in conservation interventions; (2) identification of short- and long-term outcomes; (3) scenariobuilding in alternative policy environments to assess changes and impacts; and (4) building of multistakeholder dialogues to involve all key actors in planning, management, credibility-building, and evaluative actions.16 The indirect effects of conservation interventions are particularly hard to capture, but need to be understood if the overall efficiency and effectiveness of instruments such as forest management certification are to be assessed (Nussbaum and Simula 2004). Although there is an extensive literature on evaluation studies in conservation, only a handful of studies have produced evidence-based statements that are clearly attributable to the intervention. One particular reason is that conservation interventions are seldom implemented in a random manner. Some of these are voluntary schemes and participants self-select. The resulting “selection bias� can obscure insights that could be derived from random allocation of individuals to treatments (for example, receiving or not the intervention), as in well-designed experiments. Selection bias renders it difficult to determine the extent to which the impacts are due to the intervention (that is, attribution), by making it hard to separate the effects of the intervention from the direct and indirect influences of other factors. For example, comparative studies to determine the impacts of protected areas and community- based management have seldom considered and addressed selection bias.17 The direct impacts have been subject to misinterpretation and overstatement while the indirect impacts have been mostly disregarded. These deficiencies 15 Cook et al. (2010). Several systematic efforts are under way to build foundations for understanding the impacts of conservation (for example, Conservation Measures Partnership [2007], with the Results Chain approach and tools –MIRADI); the Cambridge Conservation Forum: Kapòs et al. [2008]; the Collaboration for Environmental Evidence - www.environmentalevidence.org: Pullin et al. [2009]; Wilkie [2009]; Rowe [forthcoming]), and several documents produced by the International Initiative for Impact Evaluation (3ie: www.3ieimpact.org). The nascent field of evaluation of conservation interventions will also benefit from current debates on other fields of evaluation, in particular the insights offered by think-tanks such as Real World Evaluation (www.RealWorldEvaluation.org) that discuss methodological approaches for evaluation with emphasis on development (Leeuw and Vaessen 2009; Pattanayak 2010); the Independent Evaluation Group from the World Bank (Gertler et al. 2011); CIFOR’s REDD+ program (Jagger et al. 2010, especially Appendices A and B) with emphasis on livelihoods analyses; Richards and Panfil (2010) for impact assessment on carbon projects; ONF International for REDD+ 2011. 16 Useful references are Rogers (2008 and 2009); GAO (2009); Bamberger et al. (2009). 17 Ferraro and Pattanayak (2006) and Pattanayak et al. (2010) discuss the limitations of existing studies to determine impacts. 17 Measuring the Conservation Impact of Certification have recently started to be addressed and practical examples of methodologies to address these biases and lack of random assignation to treatments are now available.18 Approaches other than experimental and quasiexperimental19 can be qualitative, but still should be based in comparative methods to help evaluate impacts of conservation interventions. These include before-and-after comparisons, systematic case studies, and expert judgments. To evaluate the impacts of forest management certification, it seems most appropriate to employ quantitative methods designed to reveal causal connections between project activities and conservation outcomes, along with complementary qualitative approaches that can capture more of the indirect impacts.20 Lack of proper evaluations of conservation intervention impacts can be an impediment to designing high-impact interventions and can derail methodological development. The challenges of learning from past experiences affect proper design of new types of interventions (for example, REDD+ and other PES). In forest management certification, the lack of a systematic assessment of certification’s efficacy and full understanding of indirect outcomes makes it more difficult to design most effective practices. 4. General Considerations on Forest Management Certification Evaluation 4.1 Variations in Forest Management Practices An evaluation of the impacts of forest management certification must take into account that, even in the absence of certification, FMU practices vary widely (figure 2). For example, some FMUs employ management practices that closely match the certification requirements, whereas others differ from those standards. 18 Examples of this are found in Andam et al. (2008); Bowler et al. (2010); Ferraro et al. (2011); and Nelson and Chomitz (2011). 19 The term quasi-experimental refers to situations when there has not been random allocation of individuals to treatments and there is need to construct a control group in order to determine the impact of an intervention. Groups can be constructed through the use of sophisticated methods (i.e., instrumental variables and matching methods). For references on quantitative methods see Greenstone and Gayer (2007). 20 Useful references are Garbarino and Holland (2009); White (2009); Rugh et al. (2010). 18 Measuring the Conservation Impact of Certification Figure 2. Forest Management as a Continuum 4.2 Certification Continuum The certification process involves many steps typically made over several years. Becoming and remaining certified requires substantial time and continued investments. In addition to the direct costs of certification (for example, the costs of audits), even well-managed FMUs may need to change some practices to become certified. FMUs are located along a continuum from those with no interest in certification to those that have been certified through several rounds of audits ( 19 Measuring the Conservation Impact of Certification Figure 3). At different points, an FMU might be found at different locations along this “certification continuum,� so it seems wrong or at least naïve to evaluate the impacts of certification solely by comparing certified and uncertified forests. Even among certified (or audited for certification) forests, there is variation in the periods over which they will remain certified, as well as in the number of deficiencies identified by auditors as CARs, which can be minor (conditions) or major (preconditions).21 FMUs vary in their interest and investments in certification. FMUs with revoked certificates work to regain certified status or lose interest in the process. 21 If these are minor non-conformities, the certification body can issue a certificate, on condition that actions will be taken to deal with the non-conformities. If these are major non-conformities, the certification body will not issue a certificate until the non-conformities have been solved (https://ic.fsc.org/forest-management-certification.38.htm). The percentages in Figure 3 refer to the level which these CARs have been met. 20 Measuring the Conservation Impact of Certification Figure 3. Certification Continuum. Recognizing where FMUs fall on this continuum is key to assessing the impacts of certification. Deciding how this continuum is to be set up and populated with data in the different regions will inform decisions about the appropriate design and approaches to evaluation. 4.3 Importance of Context The strategies used to advance understanding of the impact of forest management certification need to consider that certification is applied in diverse places with different features. These differences affect the outcome of the intervention and the way those outcomes change over time (for example, governance, legal frameworks, social issues, market structure and preferences, technology, access to capital, and forest management practices). Proper understanding of these contextual factors will facilitate the design of evaluation of forest management certification. It is also important to consider the impacts of the intervention in the context in which it is applied. Indirect effects of forest management certification on forest policies and practices around the world have been identified in several places. Examples include Bolivia, where certification principles inspired the processes behind national forest policy- making; Cameroon, where forest certification contributed to enhanced forest law compliance; and Russia, where certification was proposed to support local adaptive capacity by increasing local stakeholder knowledge through more participative forest management processes.22 Thus, tools and processes for evaluation need to evolve to incorporate new knowledge generated by those involved in the evaluations and to reflect new benchmarks in the process of collecting evidence of impacts (Rogers 2009). The main questions to be addressed relate to how to account for the ecological, social, economic, and governance 22 Bolivia: Nittler and Nash (1999); Cameroon: Cerutti et al. (2011); Russia: Keskitalo et al. (2009). 21 Measuring the Conservation Impact of Certification contextual conditions and their changes, and how to determine their attribution (directly causing the impacts) or contribution (contributing to factors causing the impacts) roles in forest certification. The selection of evaluation methods is also influenced by the intended uses of the evaluation. 4.4 Evaluation of Impacts of Forest Management Certification Implementation of the Process of Change in Forest Management Before examining the outcomes of previous attempts to assess the effects of forest management certification, it is important to be explicit about the questions that can be addressed by an evaluation. This determination requires understanding how the intervention operates and intends to drive change under different dynamic scenarios (theory or model of change), and identifying financial, knowledge, and technological barriers to its implementation (Annie E. Casey Foundation 2004; Furman 2005; White 2010; Rugh et al. 2010; Gertler et al. 2011). Forest management certification considers two main mechanisms to produce changes in forest management outcomes. The first is the adjustment of management practices to comply with predefined standards to each certification scheme (for example FSC, PEFC), and the second is the processes of auditing to verify that compliance has occurred. None of the existing forest certification systems has an explicit theory of change that describes the change process.23 However, the certification intervention conveys the message that the intervention was designed in such a way that, if implemented correctly, it will maintain forest values, even if implemented differently in different places. It is important that the evaluation process establishes this fact with certainty. Regarding the first mechanism of change—the ways through which forest management certification is put in place— even though certification schemes may vary they all seem to follow a basic general model. This model considers an FMU that contacts one of the several certifying bodies and agrees to have a scoping visit. During the scoping and subsequent audits, the auditors review the forest management operations according to the standards of the certification scheme. Auditors report to the FMU any CARs and their implementation timeframes. If a follow-up audit reveals that the CARs have been addressed, the certificate is granted, and (annual) follow-up audits are performed to assess compliance with certification requirements. For the second mechanism—the processes of auditing to verify that compliance has occurred—it seems necessary to appraise how the achievement of these expectations is assessed by auditing firms and verified by certification bodies. The auditing process includes desk and field assessments, and public consultations with stakeholders. The certification evaluators will need to learn how certification guidelines are interpreted and implemented by the auditors and how their performance is verified by the certifying bodies. Certification schemes vary in the extent to which they keep track of this activity (for example, FSC audits the certifying bodies annually through random checking complemented by field verifications). This topic has not received much attention.24 Detecting and Understanding Impacts An evaluation aims to provide a detailed examination of the ecological, social, economic, and political impacts of the intervention, compared with situations where there was no intervention (Rossi et al. 2004) or with alternative options to reach the same goal. For example, in forestry, alternative options could be legislative changes. Besides addressing whether the goals or objectives of the intervention are being achieved, evaluation aims to establish the extent to which the outcomes are due to the intervention (attribution). 23 Fairtrade recently started discussions towards the formulate a theory of change for that scheme (Nelson and Martin 2011). 24 But see a call to look at this issue in Bartley (2007). 22 Measuring the Conservation Impact of Certification For forest management certification, the pertinent questions are (1) to what extent is certification the direct or indirect driver in the forest management (sensu lato) outcomes, and (2) what are the causal pathways for these changes. Thus, formalizing how the change process occurs (theory of change) becomes relevant, as is the identification of factors that affect decisions related to the change process and how they do so. For instance, forest management decisions at the FMU level are made based on knowledge, technical and financial capacities, information on market behaviors, and the availability of timber volumes, among others. The decisions of the forest manager are constrained by other factors, including lack of knowledge, capacity, and financial and institutional means (for example, participation of social actors relevant to the specific timber operation). At the same time, it has become important to engage the full range of relevant stakeholders, including the implementers of the certification intervention, in the processes to understand certification impacts.25 Given that methodology for evaluating conservation interventions is still being developed, a creative combination of tools should be encouraged. Different qualitative and quantitative approaches are necessary to understand and integrate different kinds of knowledge on causal complexity, to identify integrated direct and indirect impacts, and to attribute correctly the outcomes of certification (for example, through building of comparison groups). Comparisons should be designed to capture the indirect effects of certification, such as when forest managers informed about improved forest management practices adopt these practices out of enlightened self-interest. The lack of a clear dichotomy between the certified and noncertified forests, the heterogeneity, and historical characteristics and dynamics of the context where FMUs operate, as well as the existence of information and expertise-sharing networks, complicates the design and implementation of efforts to detect and understand the impacts of certification.26 Clearly, comparisons between FMUs that have been certified for more than a decade with FMUs that are well on the road to certification will yield different results than if the companies have shown no interest in certification at the onset (Table 2) (GAO 2009; Blackman and Rivera 2010; Jagger et al. 2010). 25 Exploratory approaches (for example, realistic evaluation, general elimination methodology, process tracing, contribution analysis) allow users to explain what has occurred and how, and thus are central to infer causality (Bamberger et al. 2009; White and Phillips 2012). Participatory approaches rely on perceptions to examine behavioral change and how programs can be improved; these are powerful for detecting indirect effects of the intervention (Rogers [2009]; for example, most significant change, the success case method, outcome mapping, method for impact assessment of projects and programs: Rogers [2008]; White and Phillips, [2012][?]; stakeholder analysis, multicriteria analysis, and participatory social mapping: Chambers [2009]). See also Ferraro (2009). 26 Calls for creative approaches that combine qualitative and quantitative methodologies can be found in Bamberger et al. (2009); Rugh et al. (2010); Barrett and Carter (2010). Among the tools are behavioral studies to properly assess risks and perceptions of firms and individuals; process-tracing (Collier 2011); agent-based modeling (Grimm et al. 2005); and policy analysis (Coleman 2009). 23 Measuring the Conservation Impact of Certification Table 2. Potential Approaches to Assess Impacts of Forest Management Certification METHOD DESCRIPTION LIMITATIONS Certified vs. uncertified, Randomly selected FMUs are randomly Forest certification is a voluntary scheme, so experimental allocated to the forest certification there is selection bias. A comparison based intervention. on the experimental approach is not feasible. Quasi-experimental Because FMUs cannot be randomly Processes to construct the comparison allocated to forest certification groups are data-intensive and technically intervention, a control group (e.g., not difficult. These include the use of matching under certification) will need to be techniques (e.g., groups of certified and constructed. Groups are made of FMUs noncertified FMUs will be matched in every that differ only in certification status but factor that also influences the certification are otherwise identical. outcome), and instrumental variables (e.g., correlated and easier-to-assess variables can be used to infer impact of certification intervention). Before–after Baseline information on key outcomes It is not often possible to have data on all related to the certification intervention will the variables before certification was be measured and compared with data granted for both treated and not-treated corresponding to later stages after groups. certification has been granted. Systematic case studies Intensive analyses of certified FMUs, It is time consuming and knowledge drawing on the history of the FMU and demanding, and thus could not address how the particular nature of the general questions to determine the general mechanisms and context are producing impacts of forest management certification. change. Expert judgment Process of generating knowledge on the Because of the complex nature of forest impacts of certification based on the management certification, this approach, synthesis of statements of people with though informative, could fail to capture the profound knowledge of certification and integrated effect of the certification-driven the contexts where forest management changes and interactions with contextual occurs. factors. Source: Authors. 24 Measuring the Conservation Impact of Certification 5. Current Knowledge: Types of Assessments of Forest Management Certification With few exceptions, attempts at assessing the environmental impact of forest certification have been based on examination of secondary information and stakeholder perceptions. The most common explanations given for the lack of on-the-ground assessments are methodological challenges and funding restrictions. To date, none of the approaches reviewed below can be fully seen as comprehensive evaluations, but nevertheless they provide some useful information. 5.1 Analysis of Temporal Changes in Corrective Action Requests One approach that has been used to evaluate the impacts of forest management certification employs a one-time analysis of how reports produced by forest auditors change over time. Lists of initial CARs are used to estimate how far the FMU was from the standards of the particular certification scheme. Typically, a specified timeframe is given to deal with each CAR. Subsequent satisfaction of each CAR, as indicated by its disappearance during following audits, is used as indirect evidence of improved practices. Assessments of forest certification based on the evolution of CARs all report that the number of issues initially of concern to auditors declined, indirectly indicating a positive impact of forest management certification, even though a field inspection to verify that the changes were made was not performed.27 Results of these approaches should be interpreted with caution; as these same researchers admit, the use of changes in CARs as a surrogate for more direct measures of the impacts of forest management certification has numerous limitations. One limitation is the possibility that the FMU might have learned to “manage for the audit�; that is, make changes to satisfy the requirements of the scheme, even if these changes have no tangible impact on outcomes. Additionally, most of the changes in management practices indicated by assessment of CAR evolution are process-based and not performance-based, and therefore are of only indirect importance to the maintenance of forest values.28 5.2 Literature Reviews Literature reviews on the impacts of forest management certification have mostly centered on particular issues (for example, biodiversity, wildlife, local communities) and regions. Few specific aspects of management (for example, riparian buffer zones, tree retention) are assessed. Diverse approaches have been employed to determine changes related to forest management certification, with some studies using research results comparing before and after certification and other studies looking at certified versus noncertified operations. Other studies have looked at impacts on different aspects of natural resource management (for example, community forestry and enterprises, biodiversity).29 A recent and complete literature-based study on the general topic of certification reviewed 134 documents on timber, fish, bananas, coffee, and general agricultural practices (Blackman and Rivera 2010). Of these, only 14 employed designs that appropriately considered the confounding effects of selection bias and other critical factors. Of these 14, the only study on forest management compared a certified and a nearby uncertified community forest operation in 27 For some references that used that indirect approach to assess changes produced by certification, please see Rametsteiner and Simula (2003); Newsom and Hewitt (2005); Newsom et al. (2006); Peña-Claros et al. (2009). 28 Nussbaum and Simula (2004); Bartley (2007); Peña-Claros et al. (2009); van Kuijk et al. (2009); Newsom (2005); Newsom et al. (2006). 29 Certified vs. noncertified operations: Schulte-Herbruggen and Davies (2006) and Karmann and Smith (2009). Emphasis on community forestry: Molnar (2004) and Corso et al. (2008). On community enterprises: Butterfield et al. (2005). On biodiversity: Walrecht et al. (2012). 25 Measuring the Conservation Impact of Certification Brazil and reported slight positive environmental and socioeconomic impacts of certification. Selection bias was at least partially avoided in this study because the communities had similar land tenure arrangements and ran their own operations for the primary purpose of timber production (Barbosa de Lima et al. 2009). The complexity of objectives and assessment level (individual operation vs. system-level improvements) complicates the evaluation of certification’s impact. But research from Fairtrade International and the Forest Stewardship Council argue that the fact that these schemes have been able to increase their market share demonstrates that they have been able to transform market structures and serve a purpose for market participants (Gandenberger et al. 2011). There have been attempts to analyze the macro or system-level impacts of forest certification. One study analyzed the macroeconomic impacts of FSC forest certification. Generally, they did not find any notable correlation between the forest areas certified and national-level development outcomes. The main observation was that SFM certification tends to be more widely applied in more developed countries. They concluded that this would be due to closer linkages to sensitive markets (Marx and Cuypres 2010). Another compendium of positive experiences from certification was based on a literature survey of an extensive sample of case studies from various FSC-certified forests. Although the report provides numerous examples where there is clear correlation between certification and positive outcomes, it does not discuss the theory of change behind the transition. It also does not discuss whether the improvements were simultaneous with certification (correlation) or due to certification (causality) (Karmann and Smith 2009). 5.3 Interviews with Participants in the Certification Process The impacts of forest management certification as perceived by FMUs, the timber industry, local communities, and buyers have been compiled at the global and more regional levels. The impressions of both supporters and critics of forest certification were solicited and most contributors were positive about its impacts. These assessments have often concluded that certification has contributed more to improve tropical forestry than any other initiative, even if they also stress the need to generate on-the-ground evidence to demonstrate impacts (Frost et al. 2003; Zagt et al. 2010; Sheil et al. 2010). 5.4. Reviewing Single Management Practices Rainforest Alliance and Wright State University researchers recently tested an approach to assess the impacts of best-management practices (BMPs) on resource management by reviewing well-designed and well-executed field studies that evaluated the effects of one specific requirement of certification: the establishment of set-asides including riparian buffer zones (Newsom et al. 2012). Their reasoning is that given the lack of properly designed studies on certified areas, they could look at studies addressing the impacts of some BMPs that are also requirements of certification—in this case, establishment of set-asides—to indirectly assess the potential impact of certification on this particular requirement. They reviewed properly designed and executed studies that documented ecological impacts of the establishment of set-asides, including riparian buffer zones, on specific response variables (for example, species abundance, biodiversity, population viability, water/air/soil quality). The authors contend that unbundling some of the component activities underlying the complexity of sound resource management (that is, forest management certification) will facilitate their assessment. Other researchers also have proposed this strategy in the context of other evaluation challenges (Jagger et al. 2010). Although this method is based on sound and well-designed research approach, it does not look at the integral effect of the certification intervention as a whole. This approach will miss any indirect effects of certification. A model of change is not considered as part of this approach, and this deficiency precludes an overall understanding of the 26 Measuring the Conservation Impact of Certification impacts of certification. The positive side of this method is that unbundling the practices of the complex forest management certification intervention could help understand particularities related to contextual factors, clarify causal mechanisms, and thereby facilitate their assessment. 5.5 Additional Analytical Steps in Assessing Forest Certification Impact The literature review clearly shows that there are uncertainties on how to evaluate the impact of forest management certification and its contribution to maintaining forest values. Some issues have not been adequately included in the assessments and should be strengthened when considering alternative approaches to assessing the impacts of certification. The following three potential areas have been identified: i. Promote a better understanding of the structure of the forest sector (for example, types of FMUs and their practices), and how it evolves as a function of the effects of factors external to forest management certification. A necessary step for evaluating the impacts of certification is to compile up-to-date information about FMUs, forest managers and owners, and the contexts in which they operate in particular regions. The variables used to generate this typology are to be selected on the basis of previous efforts at characterizing the forest industry sector (see Table 3). In addition to providing a necessary understanding of which types of FMUs should be included in evaluative comparisons, this typology will provide insights into factors influencing, among other things, the characteristics of local legal regimes, forest sector evolution in each proposed study area, and factors influencing forest sector dynamics.30 Table 3. Variables Influencing the Short- and Long-Term Outcomes of Forest Management and the Impacts of Certification BIOPHYSICAL ECONOMIC SOCIAL area (ha) tenure type (public, private) area population density (#/km2) previously logged (yes, no) type of FMU (community, concession, workers: locals (#/%), nationals (#/%), state, or private owner) foreign (#/%), women (#/%) area logged/yr. (ha/yr.) type and duration of permit ethnic structure in the area volume harvested/yr. (m3/yr.) origin of forest manager (country) recognized resource use and tenure rights of local communities (yes, no) no. species marketed existing and potential conflicts origin of capital (country) between managers and local communities or other stakeholders (yes, no) Source: Developed by participants in discussions on the impacts of forest management certification in October and November 2011, supported by PROFOR, CIRAD, and CIFOR (see Acknowledgments). ii. Develop a more specific list of the expected outcomes of certification and identify the potential pathways through which these outcomes are achieved. The standards of the different certification schemes are clearly stated and provide a general idea of how the goals are to be achieved through improved forest management. To help determine if these goals have been reached, it 30 Ruiz-Pérez et al. (2005); Salazar and Gretzinger (2005); and Pereira et al. (2010). Changes in macroeconomic conditions and other factors: Cattaneo (2005); Sunderlin et al. (2008); and Banerjee and Alavalapati (2009). 27 Measuring the Conservation Impact of Certification might be useful to select an agreed-upon list of specific outcome indicators that can be measured on the ground and provide certainty about certification impacts. It is important to be aware of the scale (in both time and space) through which these outcomes can be achieved. Table 4 presents an example of this activity, based on the FSC requirements and developed through expert consultation. Table 4. Outcome Areas and Outcome Statements Expected after Certification Outcome areas Biophysical Social/Livelihoods Economic Governance/Policy Hydrological Fewer logging Economically viable. There is effective control of functions and accidents. access and restriction of services (e.g., flow illegal use of the resource. regimes and water quality) are maintained. Biodiversity is Workers’ housing Minimized waste Stakeholders (firms, maintained at the meets minimum from harvesting. government, social actors) genetic, population, national standards. understand the availability and landscape of the timber resource and levels. the conditions of management (management plan of good quality). Productive capacity Land and resource Benefits distributed There are explicit, is not impaired and rights of local fairly among all transparent, and legitimate Outcome statements future harvest communities are involved. ways to negotiate volumes are not alienated. competing goals of secured. stakeholders. Carbon emissions Workers negotiate Compensation to all Sanctions are clear and and pollution from and receive livable affected in a fair and penalties are appropriate harvesting wages, have job timely manner. and applied. operations are security, and minimized. required legal services. Ecological processes Workers receive Sanctions avoided. Enforcement of legal are not threatened training. frameworks is high. by forest management. Fires are Negotiation Provision of Government agencies appropriately processes are clear ecosystem services. receive taxes, fees, and managed or (i.e., FPIC). royalties in a timely controlled. manner. Regeneration is Workers are Improved There is proper use of maintained. provided with and allowed resources from the operational use safety gear. efficiency. part of the FMU and corresponding institutions. Source: Developed by participants in discussions on the impacts of forest management certification in October and November 2011, supported by PROFOR, CIRAD, and CIFOR (see Acknowledgments). 28 Measuring the Conservation Impact of Certification “Outcome statements� refer to expected long-term changes resulting from the intervention that maintain forest values. These outcomes are expected to be reached at the level of the FMU or at the landscape level. Figure 4. Example of Impact Pathways Labor union pressure Law enforcement Workers use safety gear Fewer logging accidents Company values and principles Workers trained Certification Source: Adapted from session with participants in discussions on the impacts of forest management certification in October and November 2011, supported by PROFOR, CIRAD, and CIFOR (see Acknowledgments). Connected to the identification of the more detailed outcomes in table 4 is the need to propose a chain of processes that could link actions related to the certification intervention with these outcomes. For example, the specific goal of “Fewer logging accidents� (shaded box in Table 4), could be achieved in different ways, including the motivation to become FSC-certified. If this were the only factor to explain that workers in a particular FMU do not suffer logging accidents, then this positive outcome was achieved because of FSC certification and the positive impact can be attributed to FSC certification. If, however, there were also clear demands from the workers union or revised regulations and enforcement of occupation safety legislation, the positive outcome could have been achieved because of the influence of other factors, not specifically FSC certification (Figure 4). It is also important to consider the indirect pathways through which forest management certification might influence expected outcomes or elements of the context. Examples of this last case include improvements in management practices of noncertified FMUs near to certified ones, and certification principles inspiring changes in national forest policy-making.31 31 Bartley (2007); certification principles used in national forest policy-making: Nittler and Nash (1999); Keskitalo et al. (2009). 29 Measuring the Conservation Impact of Certification iii. Propose an explicit model on how the change in forest management practices associated with certification will come about (“theory of change�), and how to adapt the model of change to the characteristics of each region/country. A more comprehensive development of the theory of change can be key to guiding understanding of the particular roles and interactions among contextual factors (for example, time of implementation of certification, legal frameworks, capacity and technological knowledge, timber market characteristics) and actions related with the certification intervention (for example, involvement of local stakeholders, changes in timber harvesting practices). The resulting models of change, tailored to particular regions and localities, will reflect the specific issues that need to be tracked to develop an adequate knowledge of the impacts of certification. 5.6 Dynamics along the Forest Management and Certification Continuum The position of an FMU on the forest management and certification continuum (Figure 2 and 3) can change over time in response to a variety of investments and other drivers. In particular, contextual factors that operate at local, national, and international levels can influence FMU decisions to opt for certification and, once certified, to remain so. Market dynamics, consumer preferences and acquisition power can change and influence suppliers’ decisions with regard to certification. Changes in legislation and its enforcement, technological capacities, and cost-benefit ratios can affect FMU decisions regarding certification. Enhanced understanding of these dynamics will increase knowledge on the impacts of forest management certification (see box 1). 30 Measuring the Conservation Impact of Certification Box 1. Dynamics of Forest Sector Based on Expert Knowledge This exercise was developed for the Brazilian Amazon by L. Mazzei (Empresa Brasiliera de Pesquisa Agropecuária - EMBRAPA), M. Lentini (Tropical Forest Foundation), and W. Baitz (CIKEL). Although the numbers used are not yet precise, it is a foundation for the formulation of hypotheses related to the factors influencing certification decisions. Preliminary results of this analysis suggest dynamism and interesting trends. For instance, 89 percent of the firms operating illegally (that is, no management plan and no legal permit) to supply a considerable volume of commercial timber in the region (about 40 percent; Mazzei, Lentini and Baitz, pers.com) remained illegal over the three-year monitoring period. Importantly, the remaining 11 percent of firms could become certified if proper incentives were in place, such as those needed to enhance management practices (for example, reduced impact logging training). This policy decision will be contingent on the results of the evaluation process to properly justify the merits of the intervention. It is also interesting that among the 20 percent of certified firms that dropped or lost their certification, almost all continued to employ improved practices or at least RIL. Finally, the likelihood of a firm that started the certification process achieving certification was only 50 percent. The transition probabilities were calculated for a three-year interval among forest management options of firms in the Brazilian Amazon. The boxes represent the states along the forest management and legality continua. The arrows depict the direction of the transitions among these states. The numbers indicate transition probabilities among states (for example, of the 30 percent of firms that lost certification, 28 percent still perform quality management). 31 Measuring the Conservation Impact of Certification 6. The Way Forward Several development practitioners have promoted SFM certification in their forest sector operations. These have included multilateral and bilateral donors as well as civil society organizations. Certification has also been promoted in community forestry, where it is often seen as a way to improve market access and to ensure the high quality of forest management and extraction. Larger, “industrial scale� forest operations are not widely supported by international donors. Nevertheless, even here, for example, the World Bank Operational Guidelines require that commercial logging operations be certified.32 Certification has also been promoted by national forest authorities when forest policies have been reformed. In some countries (for example, Georgia), proposals have been made that certification would be compulsory for private forest concessions (long-term leases). This may be seen as a way of ensuring good management of forests even under limited forest administration and enforcement budgets. In Australia, the 2012 approved Illegal Logging Prohibition Bill bans the importation and domestic supply of illegal timber. In the legislation, timber importers can use third-party certification to demonstrate due diligence by timber importers.33 Therefore, SFM certification, despite originally being a voluntary market-based instrument, has gained traction as a law enforcement tool. These examples demonstrate that certification is often seen as an essential tool in ensuring that forest management is both environmentally and socially sustainable. At the same time, it is evident that there has not been extensive analysis on the impacts of certification or an evaluation of certification as a policy instrument. The positive impacts of certification are taken for granted and the success of certification is measured by how widely it has been applied.34 Particularly if certification is supported by donor agencies as part of a larger intervention, the impact of certification alone is hard (or even unnecessary) to establish with scientific precision. Often, the projects have received considerable technical assistance for management, marketing, and even infrastructure development in addition to certification support. There is anecdotal evidence that communities participating in certification often have been able to increase the revenues derived from forest management and commercialization. However, it is difficult if not impossible to say how much of those benefits are due to donor community support and how much to SFM certification alone. As discussed earlier in this report, there is inadequate methodological knowledge to precisely measure the impact of forest management certification as an individual intervention. How should forest practitioners view certification? Even if there is no extensive scientific research to confirm the positive, transformational change stemming from forest management certification, it does not mean that investments in forest certification have not provided any benefits. Despite the methodological uncertainties with the evaluation of forest management certification, this report does not propose that domestic or international support for it should be discontinued. There is not adequate research to state that certification has led to transformational change in forest management at the field level or that forest outcomes 32 World Bank Operational Policy 4.36: Forests states that “… 9. To be eligible for Bank financing, industrial-scale commercial harvesting operations must also: a) be certified under an independent forest certification system acceptable to the Bank as meeting standards of responsible forest management and use…� 33 The legislation states that: “The regulations may provide for due diligence requirements for importing regulated timber produ cts to be satisfied, wholly or partly, by compliance with specified laws, rules or processes, including the following: … (b) rules or processes established or accredited by an industry or certifying body…� Parliament of Australia (2012). 34 For instance, “Rather than assessing actual on-the-ground impacts of certification, we measured ‘success’ through proxies such as certified forest management area and shares of certified timber exports� (Ebeling and Yasué 2009, p. 1146). 32 Measuring the Conservation Impact of Certification would directly change once an FMU seeks or is granted forest management certification. However, even if this causality has not been established, there is no reason to assume that certified forests would not be well managed. Forest certification can be an acceptable proxy for good forest management, as it demonstrates that key principles of good management are adhered to and standards are followed. It does not demonstrate that things drastically improved because of certification or that the same development outcomes could not have been achieved by other means. Certification can be seen as a sufficient but not necessary condition for demonstrated good forest management. This is a particularly important benefit for donors and financiers who, while not having direct control over management practices, carry notable reputational risk in forest operations. Certification is also finding new uses and audiences. For example, there has been an ongoing discussion on whether SFM and chain-of-custody certification could be a risk management tool for the financial sector when it supports investments in wood production and processing. As a tool, it could be used to mitigate reputational and business risks with increased legality and sustainability requirements in global trade and public procurement.35 Forests provide notable environmental, social, and economic benefits, and the global landscape for forests and forestry has changed in recent years. REDD+, timber trade legality initiatives, and forest user and indigenous people issues have all increased the attention paid to forests and the services they provide. It is still unclear what kind of influences these new initiatives have on forest certification. For example, increased and tighter legality requirements of the amended U.S. Lacey Act and the EU Timber Regulation may “crowd out� forest certification as a tool to demonstrate compliance with legislation. After all, all timber entering U.S. or EU markets should be legal, whether certified or not.36 However, these initiatives may also increase management costs and thus make formal certification more attractive as the marginal cost of certification declines. It is also possible that the impacts of forest management certification might have been larger early in its implementation history, when national legal frameworks and overall natural resource governance were still weak. Currently, despite all its limitations, certification is the only globally recognized system of understanding and communicating how well forests are managed. It still has severe limitations: only a small percentage of the world’s forests are currently certified, certification does not measure change over time, and even the concept of SFM is interpreted differently by various stakeholders. Comparisons are also difficult since certification may have impacts that reach beyond certified forests themselves. Competition between different certification schemes (FSC and PEFC) has led to a “race to the top� and the overall management standards in the sector have improved (Overdevest 2010). This would have led to positive spillover effects across the industry. Overdevest’s 2010 study covered only industrialized countries (Finland, Sweden, and the United States) where certification is relatively widely spread. It is less clear if such positive spillover effects could be found in countries with lower certification adoption (for example, developing countries). 35 See, for example, PEFC (2012) and PROFOR (2013). 36 However, SFM certification also may be used to demonstrate due diligence in wood procurement. 33 Measuring the Conservation Impact of Certification Box 2. Trade Pattern in Certified Bananas There is not much research or evidence on the market impact of SFM certification on small, developing country producers. However, there has been more research in other commodities. In global banana trade, organic-certified bananas are sold at a premium (20–50 percent higher than “regular� bananas, depending on the country). At the same time, trade intermediaries have higher margins on certified products. During the global food price peak in late 2000s, the prices of certified bananas were more rigid than those of regular bananas, reducing the relative price premium. This indicates that the organic banana business is less volatile and more predictable than that of ordinary bananas. Source: FAO 2008. National governments and international donors have diverse objectives when it comes to forest policies and operations. How could these organizations best engage with forest certification and expand on the experiences from the past two decades since FSC, PEFC, and other schemes came to play a role? Some key recommendations and conclusions can be made. i. It is essential to use systematic analyses when making decisions on forest management and practices. These analyses include better understanding of the complex dynamics and theory of change of forest sector interventions as well as of the ultimate objectives of forest management. Given the incomplete understanding of the impacts of forest certification highlighted in this report, it is essential that forest policy decisions be based on improved knowledge of on-the-ground impacts and possible tradeoffs. Filling this gap is of particular importance when updating national forest policies. ii. Despite not having full understanding of all the dimensions and impacts of forest management certification, it can be assumed that certified forests are, on average, likely better managed than noncertified forests. Certification also assures the public that sustainability of forest management is promoted. Therefore, it is advisable that certification is maintained as one tool within the range of public forest policies, and it is supported by national governments and donor organizations. iii. This report is an output from a wider program to research the on-the-ground impacts of forest management certification. Implementation of the research will require extensive support, collaboration, and sharing of experiences by many stakeholders and institutions. It is essential that projects that deal with forest certification collect adequate baseline information on management practices, forests uses, and social variables before project activities starts. These assessments will allow better tracking of changes over time that can be attributed to the project intervention. iv. Forests provide several benefits. Forest management certification mainly deals with the productive functions of the forests and on ensuring that these can be used as economic goods without jeopardizing social benefits and environmental sustainability. National governments and donor agencies should continue their support for well- managed sustainable production forestry. The emerging trends in REDD+ and the promotion of sustainably produced wood products and wood energy to substitute non-renewable products and fuels increase the interest in forest management. Forest certification represents one instrument to deal with this increased pressure on forests. Scarcity of quantifiable evaluation results does not mean that forest certification would not provide value added to forest stakeholders. Methodological challenges in measuring the impacts cannot be interpreted as lack of impact; 34 Measuring the Conservation Impact of Certification they only show that additional methodological innovation is needed and the underlying theory of change needs to be formulated. Also, more indirect ways of measuring impacts may need to be developed; these could also be based on using proxy indicators. As discussed in section 0, forest certification increases the information available in the marketplace for all participants and this improved information increases confidence in certified producers. Having better and scientifically verified information on the impact would improve the information base even further and could also be used to improve the design of certification schemes themselves. Since certification involves both management practices and auditing process costs, it is essential that schemes focus on issues that yield development impacts. In the meantime, certification remains a valuable confidence- and consensus-building tool that defines one way to measure the quality of forest management. 35 Measuring the Conservation Impact of Certification 7. References Agrawal, A. 2007. “Forests, Governance, and Sustainability: Common Property Theory and Its Contributions�. International Journal of the Commons 1: 51–76. Agrawal, A., A. Chhatre, and R. D. Hardin. 2008. “Changing Governance of the World’s Forests.� Science 320: 1460– 1462. Andam, K. S., P. J. Ferraro, A. Pfaff, G. A. Sanchez-Azofeifa, and J. A. Robalino. 2008. “Measuring the Effectiveness of Protected Area Networks in Reducing Deforestation.� Proceedings of the National Academy of Sciences of the United States of America 105: 16089–16094. Annie E. Casey Foundation. 2004. “Theory of Change: A Practical Tool for Action, Results and Learning.� Prepared for Annie E. Casey Foundation (www.aecf.org) by Organizational Research Services (www.organizationalresearch.com). Araujo, M., S. Kant, and L. Couto. 2009. “Why Brazilian Companies Are Certifying Their Forests?� Forest Policy and Economics 11: 579–585. Auld, G. 2010. “Assessing Certification as Governance: Effects and Broader Consequences for Coffee.� Journal of Environment and Development 19 (2): 215–241 ———, L. H. Gulbrandsen, and C. L. McDermott. 2008.� CertificationSschemes and the Impacts on Forests and Forestry.� Annual Review of Environment and Resources 33: 187–211. Bamberger, M., F. Carden, and J. Rugh. 2009. “Alternatives to the Conventional Counterfactual.� Real World Evaluation: Summary of Session 713 Think Tank. American Evaluation Association, Orlando. Retrieved January 30, 2012, from http://www.realworldevaluation.org/ Banerjee, O., and J. Alavalapati. 2009. “A Computable General Equilibrium Analysis of Forest Concessions in Brazil.� Forest Policy and Economics 11: 244-252. Barbosa de Lima, A., A. L. Novaes Keppe, F. E. Maule, G. Sparovek, M. Corréa Alves, and R. F. Maule. 2009. “Does Certification Make a Difference?� Impact Assessment Study on FSC/SAN Certification in Brazil. www.imaflora.org/arquivos/Does_certification_make_a_difference.pdf. Barrett, C. B., and M. R. Carter. 2010. “The Power and Pitfalls of Experiments in Development Economics: Some Non-Random Reflections.� Applied Economic Perspectives and Policy 4: 515–548. Bartley, T. 2007. “How Certification Matters: Examining Mechanisms of Influence.� Prepared for a working group on the assessment of sustainability standards, certification and labels, Indiana University, Bloomington, IN, September. Blackman, A., and J. Rivera. 2010. “Environmental Certification and the Global Environment Facility: STAP Advisory Document�. Prepared on behalf of the Scientific and Technical Advisory Panel (STAP) of the Global Environment Facility (GEF), Washington, DC. Bowler, D., L. Buyung-Ali, J. R. Healey, J. P. G. Jones, T. Knight, and A. S. Pullin. 2010. The Evidence Base for Community Forest Management as a Mechanism for Supplying Global Environmental Benefits and Improving Local Welfare.� Scientific and Technical Advisory Panel (STAP) of the Global Environmental Facility (GEF), Washington, DC. 36 Measuring the Conservation Impact of Certification Butterfield. R., E. Hansen, R. Fletcher, and H. Nikinmaa. 2005. “Forest Certification and Small Forest Enterprises: Key Trends and Impacts, Benefits, and Barriers.� Forest Trends, Washington, DC. Cashore, B., and G. Auld. 2012.� Forestry Review.� In Toward Sustainability: The roles and Limitations of Certification, by Steering Committee of the State-of-Knowledge Assessment of Standards and Certification, pp. A88–A125. Washington, DC: RESOLVE, Inc. ———, and D. Newsom. 2004. Governing Through Markets: Forest Certification and the Emergence of Non-State Authority. New Haven, CT: Yale University Press. Cattaneo, A. 2001. “Deforestation in the Brazilian Amazon: Comparing the Impacts of Macroeconomic Shocks, Land Tenure, and Technological Change.� Land Economics 77: 219-240. Cerutti, P. O., L. Tacconi, R. Nasi, and G. Lescuyer. 2011. “Legal vs. Certified Timber: Preliminary Impacts of Forest Certification in Cameroon.� Forest Policy and Economics 13: 184-190. Chambers, R. 2009. “So That The Poor Count More: Using Participatory Methods For Impact Evaluation.� Journal of Development Effectiveness 1: 243–246. Chen, J. M., J. L. Innes, and A. Tikina. 2010. “Private Cost-Benefits of Voluntary Forest Product Certification.� International Forestry Review 12: 1–12. Chen, J., A. Tikina, R. Kozak, J., Innes, P. Duinker, and B. Larson. 2011. “The Efficacy of Forest Certification: Perceptions of Canadian Forest Products Retailers.� The Forestry Chronicle 87:636–643. Clark, M. R., and J. S. Kozar. 2011. “Comparing Sustainable Forest Management Certifications Standards: A Meta- Analysis.� Ecology and Society. 16: 3. http://www.Ecologyandsociety.Org/Vol16/Iss1/Art3/ Coleman, E. A. 2009. “Institutional Factors Affecting Ecological Outcomes in Forest Management.� Journal of Policy Analysis and Management 28:122–46. Collier, D. 2011. “Understanding Process Tracing.� Political Science and Politics 44: 823–830. Conservation Measures Partnership. 2007. Open Standards for the Practice of Conservation. Version 2.0. Washington, DC: Conservation Measures Partnership. Cook, T. D., M. Scriven, C. L. S. Coryn, and S. D. H. Evergreen. 2010. “Contemporary Thinking about Causation in Evaluation: A Dialogue with Tom Cook and Michael Scriven.� American Journal of Evaluation 31: 105–117. Corso, E., S. Desmet, I. Fernandez, D. Harrison, A. Portinga, and L. Woelders. 2008. “Literature Study and Comparative Analysis of the Benefits of FSC Certification in Community Forestry.� Wageningen University and Research Centre, Wageningen, the Netherlands. Crow, S., and C. Danks. 2010. “Why Certify? Motivations, Outcomes and the Importance of Facilitating Organizations in Certification of Community-Based Forestry Initiatives.� Small Scale Forestry 9: 195–11. Cubbage, D. W., D. Richter, M. Thompson, S. Moore, J. Edeburn, M. Chesnutt, J. Cox, and W. Boyette. 2003. Forest Certification Costs and Benefits: The Experience of the Southern Center for Sustainable Forests. Washington, DC: Society of American Foresters. http://Scsf.Env.Duke.Edu/Files/Certncsafprocs.Pdf. Cubbage, F., D. Diaz, and P. Yapura, 2010. “Impacts of Forest Management Certification in Argentina and Chile.� Forest Policy and Economics 12: 497–504. 37 Measuring the Conservation Impact of Certification Ebeling, J., and M. Yasué. 2009. “The Effectiveness of Market-Based Conservation in the Tropics: Forest Certification in Ecuador and Bolivia.� Journal of Environmental Management 90: 1145–1153. Elliott C. 2000. “Forest Certification: A Policy Network Perspective.� CIFOR. Bogor Barat, Indonesia. FAO (Food and Agriculture Organization). 2008. “Certification in the Value Chain for Fresh Fruits – The Example of Banana Industry.� FAO Commodity Studies 4, Rome, FAO. Ferraro, P. 2009. “Counterfactual Thinking and Impact Evaluation in Environmental Policy.� New Directions for Evaluation 122:75-84. Ferraro, P. J., and S. Pattanayak. 2006. “Money for Nothing? A Call for Empirical Evaluation of Biodiversity Conservation Investments.� PLOS Biology 4: 482–488. Ferraro, P. J., M. M. Hanauer, and K. R. E. Sims. 2011. “Conditions Associated with Protected Area Success in Conservation and Poverty Reduction.� Proceedings of the National Academy of Sciences 108: 13913–13918. Fischer, C., F. Aguilar, P. Jawahar, and R. Sedjo. 2005. “Forest Certification; Towards Common Standards?� Discussion Paper 05-10. Resources for the Future, Washington, DC. Frost, B., J. Mayers, and S. Roberts. 2003. “Growing Credibility: The Impact of Forest Certification on Forests and People in South Africa.� Forestry and Land Use Program, International Institute for Environment and Development - IIED, London. Furman, R. 2005. “Lessons from Implementing a Theory of Change M&E System in the Livelihoods and Landscapes Strategy Initiative (Draft Version).� Forest Conservation Program, International Union for Conservation of Nature - IUCN, Gland, Switzerland. Gan, J. 2005. “Forest Certification Costs and Global Forest Product Markets and Trade: A General Equilibrium Analysis.� Canadian Journal of Forest Research 35: 1731–1743. Gandenberger, C, Garrelts, H., and Wehlau, D. 2011. “Assessing the Effects of Certification Networks on Sustainable Production and Consumption: The Cases of FLO and FSC.� Journal Consumer Policy 4: 107–126. Garbarino, S., and J. Holland. 2009. “Quantitative and Qualitative Methods in Impact Evaluation and Measuring Results.� Governance and Social Development Resource Centre (GSDRC), Birmingham UK. Gertler, P. J., S. Martinez, P. Premand, L. B. Rawlings, and C. M. J. Vermeersch. 2011. Impact Evaluation in Practice. Washington DC: World Bank.Http://Siteresources.Worldbank.Org/EXTHDOFFICE/Resources/5485726- 1295455628620/Impact_Evaluation_in_Practice.Pdf Greenstone, M., and T. Gayer. 2007. “Quasi Experimental and Experimental Approaches to Environmental Economics.� The Frontiers of Environmental Economics – Resources for the Future, RFF 07-22, Washington DC. Grimm, V. , E. Revilla, U. Berger, F. Jeltsch, W. M. Mooij, S.F. Railsback, H.Thulke, J. Weiner, T. Wiegand, and D. L. Deangelis. 2005. “Pattern-Oriented Modeling of Agent-Based Complex Systems: Lessons from Ecology.� Science 310: 987–991. Haufler, V. 2010. “The Kimberly Process Certification Scheme: An Innovation in Global Governance and Conflict Prevention.� Journal of Business Ethics 89: 403–416. Holling, C. S. 1978. Adaptive Environmental Assessment and Management. New York: John Wiley and Sons. 38 Measuring the Conservation Impact of Certification Holvoet, B., and M. Muys. 2003. “Sustainable Forest Management Worldwide: A Comparative Assessment of Standards.� International Forestry Review 6: 99–122. Jagger, P., E. O. Sills, K. Lawlor, and W. D. Sunderlin. 2010. “A Guide to Learning about Livelihoods Impacts of REDD+ Projects.� Occasional Paper 56, CIFOR, Bogor, Indonesia. Johansson, J., and L. Gun. 2011. “Can Voluntary Standards Regulate Forestry? Assessing the Environmental Impacts of Forest Certification in Sweden.� Forest Policy and Economics 13: 191–198. Kapòs, V., A. Balmford, E. Aveling, P. Bubb, P. Carey, A. Entwistle, J. Hoplins, T. Mulliken, A. Stattersfield, M. Walpole, and A. Manica. 2008. “Calibrating Conservation: New Tools for Measuring Success.� Conservation Letters 1: 155–164. Karmann, M., and A. Smith (eds.). 2009. FSC Reflected in Scientific and Professional Literature: Literature Study on the Outcomes and Impacts of FSC Certification. Bohn, Germany: FSC International Center. Keskitalo, E. C. H., C. Sandström, M. Tysiachniouk, and J. Johansson. 2009. “Local Consequences of Applying International Norms: Differences in the Application of Forest Certification in Northern Sweden, Northern Finland, and Northwest Russia.� Ecology and Society 14 (2): 1. http://www.Ecologyandsociety.Org/Vol14/Iss2/Art1/. Kollert, W., and P. Lagan. 2007. “Do Certified Tropical Logs Fetch a Market Premium? A Comparative Price Analysis from Sabah, Malaysia.� Forest Policy and Economics 9: 862–868. Leeuw, F., and J. Vaessen. 2009. Impact Evaluations and Development; NONIE Guidance on Impact Evaluation. Washington, DC: World Bank. http://www.Worldbank.Org/Ieg/Nonie/Guidance.Html. Marx, A., and D. Cuypres. 2010. “Forest Certification as a Global Environmental Governance Tool: What Is the Macro-Effectiveness of the Forest Stewardship Council?� Regulation and Governance 4: 408–434. Mcdermott, C. L., E. Noah, and B. Cashore. 2008. “Differences That ‘Matter’? A Framework for Comparing Environmental Certification Standards and Government Policies.� Journal of Environmental Policy & Planning 10: 47–70. Miradi. 2007. Miradi: Adaptive Management Software for Conservation Projects. Accessed January 2012 at https://Miradi.Org. Molnar, A. 2004. “Forest Certification and Communities.� International Forestry Review 6: 173–179. Montreal Process. 2009. Criteria and Indicators for the Conservation and Sustainable Management of Temperate and Boreal Forests. 4th ed. Tokyo, Japan. The Montreal Process. Nebel, G., L. Quevedo, J. B. Jacobsen, and F. Helles. 2005. “Developing and Economic Significance of Forest Certification: The Case of FSC in Bolivia.� Forest Policy and Economics 7: 175–186. Nelson, A., and K. M. Chomitz. 2011. “Effectiveness of Strict vs. Multiple Use Protected Areas in reducing Tropical Forest Fires: A Global Analysis Using Matching Methods.� Plos One 6: E22722. Nelson, V., and A. Martin. 2011. “2011 Impact Evaluation of Social and Environmental Voluntary Standard Systems (SEVSS): Using Theories of Change.� Working Paper 1, Natural Resources Institute - NRI, University of Greenwich, U.K., November. Newsom, D., and D. Hewitt. 2005. The Global Impact of Smartwood Certification. New York: RainForest Alliance. 39 Measuring the Conservation Impact of Certification Newsom, D., V. Bahm, and B. Cashore. 2006. “Does Forest Certification Matter? An Analysis of Operation Level Changes Required During the Smartwood Certification Process in the United States.� Forest Policy and Economics 9: 197–208. Newsom, D., V. Bahn, E. Kennedy, S. Adhikari, and J. Miller. 2012. Testing a BMP-Based Approach for Gaps in Certification Impacts Research. In Steering Committee Report. 2012. Steering Committee of the State-of- Knowledge Assessment of Standards and Certification Toward sustainability: The roles and limitations of certification. Washington, DC: RESOLVE, Inc. Nittler, J. B., and D. W. Nash. 1999. “The Certification Model for Forestry in Bolivia.� Journal of Forestry 97: 32–36. Nussbaum, R., and M. Simula. 2004. “Forest Certification: A Review of Impacts and Assessment Frameworks.� The Forest Dialog, Yale University, School of Forestry and Environmental Studies, New Haven, CT. Oliver, R. 2004. The Certification Matrix: Finding Your Way Through Forest Certification Schemes. Brussels, Belgium. Confederation of European Paper Industries - CEPI. http://www.Forestrycertification.Info. Overdevest, C. 2010. “Comparing Forest Certification Schemes: The Case of Ratcheting Standards in the Forest Sector.� Socio-Economic Review 8: 47–76. Pacheco, P. 2004. “What Lies Behind Decentralization? Forest, Powers and Actors in Lowlands Bolivia.� European Journal of Development Research 16: 90–109. Parliament of Australia. 2012. “Illegal Logging Prohibition Bill 2012.� http://www.Aph.Gov.Au/Parliamentary_Business/Bills_Legislation/Bills_Search_Results/Result?Bid=R4740 . Pattanayak, S. 2010. “Rough Guide to Evaluation of Environmental and Development Programs.� http://www.Sandeeonline.Com/Updloads/Documents/Publications/847_PUB_Working_Paper_40.Pdf). Pattanayak, S. K., S. Wunder, and P. J. Ferraro. 2010. “Show Me the Money: Do Payments Supply Environmental Services in Developing Countries?� Review of Environmental Economics and Policy 4: 254–274. Paull, J. 2010. “From France to the World: The International Organic Federation of Organic Agriculture Movements (IFOAM).� Journal of Social Research and Policy 2(December). PEFC (Programme for the Endorsement of Forest Certification). 2012. “The Programme for the Endorsement of Forest Certification and the Financial Sector: How Certification Can Help Financial Professionals Manage Environmental Risks and Verify Sustainability.� Available at : http://www.scribd.com/doc/111678454/PEFC-and- the-Financial-Sector22 (Accessed on March 18 2013). Peña-Claros, M., S. Blommerde, and F. Bongers. 2009. “Assessing the Progress Made: An Evaluation of Forest Management Certification in the Tropics.� Tropical Resource Management Paper 95, Wageningen University, the Netherlands. Pereira, D., D. Santos, M. Vedoveto, J. Guimaraes, and A. Verissimo. 2010. Fatos Florestais Da Amazonia 2010. Belem, Brazil: IMAZON. Pokorny, B., and M. Steinbrenner. 2005. “Collaborative Monitoring of Production and Costs of Timber Harvest Operations in the Brazilian Amazon.� Ecology and Society 10: 3. http://www.Ecologyandsociety.Org/Vol10/Iss1/Art3/. PROFOR (Program on Forests). 2013. Certification, Verification and Governance in Forestry in Southeast Asia. Washington, DC: PROFOR. 40 Measuring the Conservation Impact of Certification Pullin, A. S., T. M. Knight, and A. R. Watkinson. 2009. “Linking Reductionist Science and Holistic Policy Using Systematic Reviews: Unpacking Environmental Policy Questions to Construct an Evidence-Based Framework.� Journal of Applied Ecology 46: 970–975. Rametsteiner, E., and M. Simula. 2003. “Forest Certification – An Instrument to Promote Sustainable Forest Management?� Journal of Environmental Management 67: 87–98. Richards, M., and S. N. Panfil. 2010. Manual for Social Impact Assessment of Land-Based Carbon Projects. Version 1. Washington, DC: Forest Trends, Climate, Community & Biodiversity Alliance, RainForest Alliance, and Fauna & Flora International. Rogers, P. J. 2008. “Using Programme Theory to Evaluate Complicated and Complex Aspects of Interventions.� Evaluation 14: 29–48. ———. 2009. “Matching Impact Evaluation Design to the Nature of the Intervention and the Purpose of the Evaluation.� Journal of Development Effectiveness 1: 217–226. Rossi, P. H., M. W. Lipsey, and H. Freeman 2004. Evaluation: A Systematic Approach. Thousand Oaks, CA: Sage Publications. Rowe, A. E. (forthcoming). “Evaluation of Natural Resource Interventions.� American Journal of Evaluation. Rugh, J., M. Steinke, B. Cousins, and M. Bamberger. 2010. “Summary of Discussion of the Alternative to the Statistical Counterfactual Think Tank.� American Evaluation Association - AEA 2010 Annual Conference, San Antonio, TX. Ruiz-Pérez, M., et al. 2005. “Logging in the Congo Basin: A Multi-Country Characterization of Timber Companies.� Forest Ecology and Management 214: 221–236. Salazar, M., and S. Gretzinger. 2005. “Costos y Beneficios de la Certificación Forestal y Mecanismos para la Resolución de Obstáculos Comunes.� WWF Centroamérica and PROARCA, San José, Costa Rica. Schroeder, R. A. 2010. “Tanzanite as Conflict Gem: Certifying a Secure Commodity Chain in Tanzania.� Geoforum 41 (1): 56–65. Schulte-Herbruggen, B., and G. Davies. 2006. Wildlife Conservation and Tropical Timber Certification. London: Zoological Society of London. Sheil, D., F. E. Putz and R. J. Zagt (eds.). 2010. “Biodiversity Conservation in Certified Forests.� European Tropical Forest Research Network 51. Wageningen, the Netherlands Simula, M., S. Astana, and R. Ishmael. 2004. “Report on Financial Cost-Benefit Analysis of Forest Certification and Implementation of Phased Approaches.� International Tropical Timber Council, Rep. ITTC (XXXVII)/13. Sunderlin, W. D., J. Hatcher, and M. Liddle. 2008. From Exclusion to Ownership? Challenges and Opportunities in Advancing Forest Tenure Reform. Washington, DC: Rights and Resources Initiative. Takahashi, T., G. C. Van Kooten, and I. Vertinksi. 2003. “Why Might Forest Companies Certify? Results from a Canadian Survey.� International Forestry Review 5: 329–337. Tikina, A., and J. L. Innes. 2008. “A Framework for Assessing the Effectiveness of Forest Certification.� Canadian Journal of Forest Research 38: 1357–1365. 41 Measuring the Conservation Impact of Certification U.S. Government Accountability Office (ed.). 2009. Program Evaluation: A Variety of Rigorous Methods Can Help Identify Effective Interventions in United States. Washington, DC: Government Accountability Office. Van Kooten, G. C., H. W. Nelson, and I. Vertinsky. 2005. “Certification of Sustainable Forest Management Practices: A Global Perspective of Why Countries Certify.� Forest Policy and Economics 7: 857–867. Van Kuijk, M., F. E. Putz, and R. Zagt. 2009. “Effects of Forest Certification on Biodiversity.� Tropenbos International, Wageningen, the Netherlands. Viana, V., J. Ervin. R. Z., Donovan, C. Elliott, and H. Gholz (eds.). 1996[?]. Certification of Forest Products: Issues and Perspectives. Washington, DC: Island Press. Vidal, N., and R. Kozak. 2008. “The Recent Evolution of Corporate Responsibility Practices in the Forestry Sector.� International Forestry Review 10: 1–13. Vogt, K., B. Larson, J. C. Gordon, D. J. Vogt, and A. Fanzeres (eds.). 2000. Forest Certification: Roots, Issue Challenges and Benefits. Boca Raton, FL: CRC Press. Walrecht, A., K. Basso, and S. Hime. 2012. “Certification and Biodiversity Exploring Improvements in the Effectiveness of Certification Schemes on Biodiversity.� Climate Change and Sustainability Services, KPMG, The Netherlands. White, H. 2009. “Some Reflections on Current Debates in Impact Evaluation.� 3ie Working Paper No. 1. New Delhi, India ———. 2010. “A Contribution to Current Debates in Impact Evaluation.� Evaluation 16: 153–164. ———, and D. Phillips. 2012. Addressing Attribution of Cause and Effect in Small N Impact Evaluations: Towards an Integrated Framework. 3ie Working Paper No. 15. New Delhi, India. Wilkie, D. 2009. Measuring Our Effectiveness – A Framework for Monitoring. Technical Manual 3. Washington, DC. Wildlife Conservation Society and U.S. Agency for International Development. Wunder, S. 2005. “Macroeconomic Change, Competitiveness and Timber Production: A Five Country Comparison.� World Development 33: 65–86. World Wilflife Fund/World Bank Global Forest Alliance. Forest Certification Assessment Guide (FCAG). 2006. “A Framework for Assessing Credible Forest Certification Systems/Schemes.� World Bank Global Forest Alliance, Washington, DC. Zagt, R., D. Sheil, and F. E. Putz. 2010. “Biodiversity Conservation in Certified Forests: An Overview.� in Biodiversity Conservation in Certified Forests, ed. D. Sheil, F. E. Putz, and R. Zagt. Wageningen, the Netherlands: ETFRN and Tropenbos International. 42 Measuring the Conservation Impact of Certification